I have read, with some dismay, the draft of Ithaka S+R’s most recent report. I offer here some critical remarks that I hope will allow for revision of the work, which I believe offers an insular, digital-nationalist, exclusionary vision for the future of scholarly communications. The views herein are my personal take, not those of any organization for which I work.
I have been thinking, this week, about the observability of AWS Lambda functions in API Gateway contexts. The major challenge is that Prometheus metrics pose a problem as they are pull-only (via a scraping endpoint). Prometheus metrics are stored in a temporary disk cache and then pulled off-site by Grafana etc.
LocalStack is a great cloud emulation layer. It lets you simulate interaction with AWS, which is great for writing integration tests. However, I wanted a system that, when run locally, would spin up the LocalStack server and then destroy it when done. But when running the test on GitLab CI, it will use the “service” provision of their continuous integration system and connect to that.
In my new role at Crossref I work on a series of data pipelines for research and development projects. These are resource-intensive data processing tasks that need to be executed periodically on a schedule, with good observability, but also with parallel processing capacity. Amazon’s Managed Workflows for Apache Airflow (MWAA) seems like an ideal solution for this.
I am currently conducting a research project at Crossref that requires me to build a database using large backend files (e.g. building a relational database from a 3GB XML file). We need to rebuild this monthly, so Apache Airflow seemed a good tool to run these periodic tasks. There are, however, lots of “gotchas” in this framework that can trip up a newcomer and I thought it might be helpful to document some of these.
So, after three years of shielding, I got Covid. I contracted it at hospital (or on my way there). How do I know? Because I don’t go anywhere else. I thought, though, that it might be useful to document my experience of getting antiviral treatments for Covid as an extremely clinically vulnerable individual. I developed a sore throat on Saturday evening (17th December 2022) but tested negative.
As I posted a while ago, from January 2023 I will be working at Crossref while retaining my university Professorship. I wanted, here, to outline a few of the projects that I hope to work on once I get started there. I should say upfront: I am afraid there is no time estimate on these and we can’t guarantee to prioritise any particular project. But if there is one that stands out to you, do let me know, as this serves as a useful community gauge.
Like many years, 2022 was a year of health problems for me. The entire year has been overshadowed by the episode of kidney failure that I suffered as a result of BK virus associated nephropathy. It is fair to say that I have been quite seriously unwell. I also spent a long period of this year, in my spare time, campaigning for protection of the immunocompromised with the monoclonal antibody therapy, Evusheld.
As many of you know, I have been involved for the past few months in a campaign to get Evusheld – a protective/prophylactic drug for immunocompromised people who do not respond well to Covid vaccines – available on the NHS in the UK. We have not succeeded on that front yet and I am not giving up. However, for my own protection, since AstraZeneca made the drug privately available last week, I have been able to procure it privately.
The other day I wrote about the response that we had from the UK's Department of Health and Social Care.
