Bioinformatics, computational biology, and data science updates from the field. Occasional posts on programming.

Stephen Turner

This is Paired Ends — your bioinformatics, computational biology, and data science updates from the field.

Coming soon

This week’s recap highlights a new tool from Wei Shen and Zamin Iqbal for efficient sequence alignment against millions of prokaryotic genomes (LexicMap), a new tool from Heng Li for efficiently constructing and querying a sequence index at scale, an R/Bioconductor package for detecting and correcting DNA contamination in RNA-seq data, a method for dating gene age using synteny, how AlphaFold predictions for some types of conformations are

What I'm reading this week in alignment and indexing millions of genomes at scale, gene age and family analysis and visualization, synthetic enhancers and cell fate programming, single cell omics, ...

Weekly Recap (Sep 2024, part 3)

Here are a few papers that caught my attention recently. I summarize a few in the deep dive at the top, then link to a few other papers of note later on. Subscribe to Paired Ends to get periodic summaries like this delivered to your e-mail.

What I'm reading this week in genomics, bioinformatics, cancer biology, epigenome editing, biosecurity, synthetic biology, single-cell omics, population genetics, microbial genomics, ...

What I'm reading (Aug 2024, part 2)

This post expands on a few of the papers I posted in this Twitter thread. I highlight a few in the deep dive at the top, then link to a few other papers of note later on. Subscribe to Paired Ends to get summaries like this in your e-mail as soon as I write them.

What I’m reading this week in synthetic biology, population genetics, AI in genomics/bioinformatics, microbial genomics, and more

What I'm reading (July 2024)

This week’s recap highlights perspectives in de-extinction and patent law, systematic benchmarking of scATAC-seq methods, a 91 gigabase (!) animal genome, structural variant genotyping with long reads, cell type-specific enhancer prediction, and a perspective piece in AI in biosecurity.

What I'm reading this week in de-extinction and patent law, ATAC-seq benchmarking, large animal genomes, SV genotyping with long reads, cell type-specific enhancer prediction, AI in biosecurity, ...

Weekly Recap (Sep 2024, part 1)

What I'm reading this week in single cell omics, spatial transcriptomics, kinship and selection inference in archaeogenomics, population and medical genetics, ...

What I'm reading (Aug 2024, part 4)

This post expands on a few of the papers I posted in this Twitter thread. I highlight a few in the deep dive at the top, then link to a few other papers of note later on. Subscribe to Paired Ends to get summaries like this delivered to your e-mail as soon as I write them. Subscribe now You might remember me from Getting Genetics Done where I blogged about genetics, statistics, and bioinformatics from 2008-2017.

What I’m reading this week in synthetic biology, population genetics, AI in genomics/bioinformatics, pangenomes, and more

What I'm reading (July 2024, part 2)

It’s been a big week in the genomics+bioinformatics space. This post expands on a few of the recent papers I posted in a Twitter thread. I highlight a few in the deep dive at the top, then link to a few other papers of note later on. Subscribe (free!) to Paired Ends to get summaries like this delivered to your e-mail as soon as I write them.

What I'm reading this week in SV analysis with long reads, spatial transcriptomics, gene prediction and annotation, synthetic biology, AI in genomics &amp;

What I'm reading (Aug 2024, part 1)

It’s a short week here in the US. As I reflect on the tools that shape modern bioinformatics and data science it’s striking to see how far we’ve come in the 20 years I’ve been in this field. Today’s ecosystem is rich with tools that make our work faster, better, enjoyable, and increasingly accessible.

Data science and bioinformatics tech I'm thankful for in 2024: tidyverse, RStudio, Positron, Bluesky, blogs, Quarto, bioRxiv, LLMs for code, Ollama, Seqera Containers, StackOverflow, ...

Tech I'm thankful for (2024)

I just returned from a week in Barcelona where I attended the Nextflow Summit and nf-core hackathon, and I can hardly contain my excitement for the near term future of bioinformatics, computational biology, and open science in general.

Highlights from the 2024 Nextflow Summit and nf-core hackathon in Barcelona

Nextflow Summit Barcelona 2024

I wrote my first public blog post in 2009. I started Getting Genetics Done to share what I was learning at the end of my PhD/postdoc through my first few years as faculty. Some of the earliest posts were simple, such as how to write and run a simple Perl script, to bigger topics like why it’s usually a bad idea to categorize continuous variables in a linear model.

A short essay on writing about what you're learning

Learning in Public

<strong>
 Using LLMs in R
</strong>
Most of the developer tooling for AI/LLM training and evaluation is Python-centric, but just over the past few months we’ve seen a surge of new tooling for AI/LLM
<em>
 applications
</em>
for the R ecosystem.
<strong>
 ollamar
</strong>
and
<strong>
 rollama
</strong>
provide wrappers around the Ollama API allowing you to run LLMs locally on your machine.

The lang package overrides the ? and help() functions in your R session. If you are using RStudio or Positron, the translated help page will appear in the usual help pane.

Use an LLM to translate help documentation on-the-fly

A Google search for “R vs Python” returns thousands of hits across sites like Reddit, IBM, Datacamp, Coursera, Kaggle, and many others.

Python for R users

<em>
 <strong>
  TL;DR
 </strong>
 : For a very simple analysis (means by group on 100M rows), duckdb was 125x faster than base R, and 28x faster than readr+dplyr, without having to read data from disk into memory. The duckplyr package wraps DuckDB's analytical query processing techniques in a dplyr-compatible API. Learn more at duckdb.org/docs/api/r and duckplyr.tidyverse.org.
</em>
I wanted to see for myself what the fuss was about with DuckDB.

Using DuckDB in R to analyze 100 million rows of data in 3 seconds

DuckDB vs dplyr vs base R

Llama 3.1 405B is the first open-source LLM on par with frontier models GPT-4o and Claude 3.5 Sonnet. I’ve been running the 70B model locally for a while now using Ollama + Open WebUI, but you’re not going to run the 405B model on your MacBook.

How to create and deploy a free GPT4-class chatbot on HuggingFace Assistants for any GitHub repo, using an R package as an example, in less than 60 seconds

Create a free Llama 3.1 405B-powered chatbot on a GitHub repo in &lt;1 min

This post is about the R package development experience with Positron, the new IDE from Posit based on VS Code. This is not a tutorial on R package development in general — there are great resources for that elsewhere. Read on.Subscribe to Paired Ends to get future posts like this delivered to your e-mail.
<strong>
 RStudio, VS Code, and Positron
</strong>
Back in 2011 I wrote a blog post about a relatively new IDE for R called RStudio.

First impressions using Positron for R (and Python!) package development

R package development in Positron

If you use ChatGPT, Claude, or even some local model through Ollama or HuggingFace Assistants, you’ll know that the chat interface makes it challenging to feed in an entire repo like a Python or R package, because functions, tests, etc. can be scattered across many files throughout a repo.

The open source repo2txt web app will take a GitHub repo URL, displays the directory structure, lets you choose which files to include, and provides a single plain text file you can feed into any LLM.

Turn a GitHub repo into a single text file for LLM-friendly input

I’ve been using the llama3.1:70b model just released by Meta using Ollama running on my MacBook Pro. Ollama makes it easy to talk to a locally running LLM in the terminal (ollama run llama3.1:70b) or via a familiar GUI with the open-webui Docker container. Here I’ll demonstrate using the ollamar package on CRAN to talk to an LLM running locally on my Mac.

Using R to prompt llama3.1:70b running on my laptop with Ollama + ollamar to tell me what's interesting about a set of genes, and to summarize recent bioRxiv preprints

Use R to prompt a local LLM with ollamar

<em>
 <strong>
  TL;DR:
 </strong>
 Codeium offers a free Copilot-like experience in Positron. You can install it from the Open VSX registry directly within the extensions pane in Positron.
</em>

Positron doesn't have GitHub Copilot, but Codeium and others are good alternatives.

AI code completion in Positron

Last month I published a paper and an R package for summarizing preprints from bioRxiv using a local LLM. I wrote about it here: Llama 3.2 was just released today (announcement). The biggest news is the addition of a multimodal vision model, but I was intrigued by the reasonably good performance of the tiny 3B text model. I used this as an excuse to update the biorecap R package.

The biorecap package can now summarize bioRxiv and medRxiv preprints, and uses the llama-3.2-3b model by default.

Llama 3.2 summaries of bioRxiv and medRxiv preprints with biorecap

<em>
 <strong>
  VP (Pete) Nagraj
 </strong>
 is a long time friend, colleague, and collaborator, and is the author of this post. Pete and I have co-authored over a dozen publications, and have taught several graduate courses in data science together. Pete leads the health security analytics / infectious disease modeling and forecasting group at Signature Science, where we started this work together last year.
</em>

PLANES provides a set of methods for evaluating the plausibility of epidemiological signals and forecasts. The PLANES methods are available in the rplanes R package and Shiny app.

PLANES: Plausibility Analysis of Epidemiological Signals

OpenAI introduced the ability to create custom GPTs back in November 2023. I wanted to try to create one of these, and in the spirit of learning in public this post describes how I made it. But first, what does it do?Gene Info Custom GPT
<strong>
 Gene Info custom GPT
</strong>
The Gene Info custom GPT takes a list of human gene symbols as input.

Gene Info: a custom GPT that takes a list of gene symbols and provides summary information, gene ontology terms, and provides contextual information like pathway or disease involvement.

Gene Info Custom GPT

I'm still catching up on papers from my late 2024 backlog. This week’s recap highlights autonomous microbial sensors for detecting TNT in soil, genome size estimation from long reads, STABIX for indexing and compressing GWAS summary statistics, and Clair3-RNA for deep learning-based small variant calling on long-read RNA-seq data.

Autonomous microbial sensors for detecting TNT in soil, genome size estimation from long reads, indexing + compressing GWAS summary stats, Clair3-RNA for small variant calling on long-read RNA-seq...

Weekly Recap (Jan 2025 part 2)

I'm still catching up on papers from my late 2024 backlog. This week’s recap highlights a browser application for visualizing pathogen dispersal, a DNA language model evaluation benchmark on regulatory DNA, regularized ensemble polygenic risk prediction with GWAS summary statistics, multimodal analysis of RNA-seq data for complex trait genetics, and a deep dive on blastp’s E-value.

Visualizing pathogen dispersal, DNA language model benchmarks for regulatory DNA, polygenic risk prediction with GWAS summary stats, multimodal analysis of RNA-seq data, blastp's E-value, ...

Weekly Recap (Jan 2025 part 3)

I’ve been blogging about genetics, statistics, computational biology, data science, and science in general for over 15 years. I published my first blog post on Getting Genetics Done in 2009, and started this blog last year after taking a few years off.
<strong>
 Science blogging has significantly contributed to my personal and professional growth as a scientist.
</strong>
Writing takes time and effort — time I could have spent elsewhere.

Writing a technical blog improves your writing, forces you to learn new things, helps others and yourself, and helps your career. This post: why blog, what to blog about, how to start, and using AI.

Technical blogging for growth and learning

The majority of developers use LLMs to help write code, present company included. When I’m working in languages I know well, they're fantastic at handling the grunt work: generating boilerplate, suggesting completions, and writing tedious tests and documentation.

I used Claude to build a Chrome extension to add a post-to-Bluesky button on bioRxiv. I've never written Javascript or a Chrome extension before. It took 5 minutes.

Write code in unfamiliar territory with AI

The Baader–Meinhof phenomenon (aka the frequency illusion) is the name for that thing that happens when you buy a new car, and suddenly you notice that same model car everywhere you drive.

Something a little different this week: book recommendations! De-extinction is a common theme across a few of the fiction and nonfiction books I read this year.

What I'm reading: de-extinction edition

<em>
 I had good intentions to give NaNoWriMo a try this year but didn’t get very far. Instead I gave OpenAI’s Creative Writing Coach GPT a try for a (very) short story I had in mind, inspired by my frustration trying to access closed-access research articles for a review article I’m preparing.
</em>

A parable of scientific publishing, open access, and preprints

The Enlightenment Conservatory

This week’s recap highlights the Evo model for sequence modeling and design, biomedical discovery with AI agents, improving bioinformatics software quality through teamwork, a new tool from Brent Pedersen and Aaron Quinlan (vcfexpress) for filtering and formatting VCFs with Lua expressions, a new paper about the NHGRI-EBI GWAS Catalog, and a review paper on designing and engineering synthetic genomes.

The Evo model for sequence modeling and design, AI agents in biomedical research, improving bioinfo software with teamwork, VCF manipulation, GWAS Catalog, designing+engineering synthetic genomes...

Weekly Recap (Dec 2024, part 3)

<strong>
 Background
</strong>
<strong>
 Bluesky, atrrr, local LLMs
</strong>
I’ve written a few posts lately about Bluesky — first, Bluesky for Science, about Bluesky as a home for Science Twitter expats after the mass eXodus, another on using the atrrr package to expand your Bluesky network. I’ve also spent some time looking at R packages to provide an interface to Ollama.

Summary, translation, and sentiment analysis of Bluesky posts on a topic using atrrr, tidyverse/ellmer, and mlverse/mall with Claude and other open models, and gistr to put the summary on GitHub.

Bluesky conversation analysis with local and frontier LLMs with R/Tidyverse

A few days ago I wrote about translating R package help documentation using a local LLM (e.g. llama3.x)… …when Mick Watson commented: I was already thinking of wiring up something like this using local AI models — something to summarize podcasts, conference recordings, etc. The relatively new (as of this writing) Gemini 2.0 Flash model will do this for you for YouTube videos. But what if you wanted to do this offline using a local LLM?

Using local LLMs to extract and summarize audio transcripts from YouTube and podcasts. Examples: DESeq2 tutorial from YouTube, and the Nextflow Podcast.

Video to audio to transcript to summary using local AI: whisperfile and llama3.3

I joined Twitter1 way back in 2009. For nearly 10 years “scitwitter” was an amazing place for discussion, discovery, and engagement with the scientific community. The #Rstats and #pydata hashtags were great places to learn about something new in programming, #icanhazpdf was great for getting papers you didn’t have access to, and conference live-tweeting was common and useful for those of us with FOMO not able to make it in person.

Bluesky starter packs for genomics, bioinformatics, R, and Nextflow

Bluesky for Science

In the spirit of learning in public,1 today I learned about the .keep argument in dplyr. This doesn’t add anything you can’t do with a select or transmute, but might help simplify some of your dplyr pipelines.2 In the examples below I’m using a few rows from the built-in iris dataset to demonstrate how to use the .keep argument by creating a new ratio variable that’s the ratio of the sepal length to width.

Use the .keep argument in dplyr::mutate() to control which variables to retain after mutating to create new variables.

TIL: dplyr::mutate()'s .keep argument

This week’s recap highlights an AI agent for automated multi-omic analysis (AutoBA), rapid species-level metagenome profiling and containment (sylph), a review on genome-wide association analysis beyond SNPs, private information leakage from scRNA-seq count matrices, and a method to “unlearn” viral knowledge in protein language models as a means to develop safe PLM-based variant effect analysis (PROEDIT).  Others that caught my attention include

What I'm reading in agentic multiomic analysis, metagenome profiling, GWAS beyond SNPs, private information leakage from scRNA-seq, unlearning viral knowledge for safe protein language models, ...

Weekly Recap (Nov 2024, part 2)

This week's recap highlights a new pipeline for metagenome quality assessment and taxonomic annotation (MAGFlow &amp;

Nextflow for metagenomics QA &amp; taxonomic classification, SNP/SV benchmarking, SNV calling with nanopore data, assembly polishing with transformers, Nextflow for variant simulation &amp; evaluation, ...

Weekly Recap (Nov 2024, part 1)

This week’s recap highlights a new Nextflow workflow for calculating polygenic scores with adjustments for genetic ancestry, a paper demonstrating that whole exome plus imputation on more samples is more powerful than whole genome sequencing for finding more trait associated variants, a new deep-learning-based splice site predictor that improves spliced alignments, a new method for accurate community profiling of large metagenomic datasets, and

Nextflow workflow for polygenic scoring, WES+imputation &gt; WGS, DL method for splice site prediction, metagenomics profiling, and a new NIST/GIAB matched tumor-normal with 13 sequencing modalities

Weekly Recap (Oct 2024, part 3)

This week’s recap highlights a new way to turn Nextflow pipelines into web apps, DRAGEN for fast and accurate variant calling, machine-guided design of cell-type-targeting cis-regulatory elements, a Nextflow pipeline for identifying and classifying protein kinases, a new language model for single cell perturbations that integrates knowledge from literature, GeneCards, etc., and a new method for scalable protein design in a relaxed sequence

Nextflow to web apps, variant analysis with DRAGEN, synthetic cis-regulatory elements, Nextflow for kinase ID+characterization, language model for single cell analysis, scalable protein design, ...

Weekly Recap (Dec 2024, part 2)

Last week I posted about a web app that turns a GitHub repo into a single text file for LLM-friendly input. This is great for capturing LLM-friendly text from a GitHub repo, but what about any other arbitrary website or PDF? I was catching up on Simon Willison’s newsletter reading about an app he made with Claude artifacts that uses the Jina Reader API to generate Markdown from a website. You don’t need to use the API to do this.

Add r.jina.ai/ in front of any URL to turn the entire page into Markdown for LLM-friendly input. Works on PDFs too.

Turn any webpage into markdown for LLM-friendly input

This week’s recap highlights the WorkflowHub registry for computational workflows, building a virtual cell with AI, a review on bioinformatics methods for prioritizing causal genetic variants in candidate regions, a benchmarking study showing deep learning methods are best for variant calling in bacterial nanopore sequencing, and a new ML model from researchers at Genentech for predicting cell-type- and condition-specific gene expression across

WorkflowHub computational workflow registry, building a virtual cell with AI, bioinformatics for prioritizing causal variants, DL methods for nanopore variant calling, predicting gene expression, ...

Weekly Recap (Dec 2024, part 1)

This week’s recap highlights pangenome graph construction with nf-core/pangenome, building pangenome graphs with PGGB, benchmarking algorithms for single-cell multi-omics prediction and integration, RNA foundation models, and a Nextflow pipeline for characterizing B cell receptor repertoires from non-targeted bulk RNA-seq data.

Pangenome graph construction with nf-core/pangenome, pangenome graphs, single cell multi-omics, RNA foundation models, Nextflow for B-cell receptor sequencing from nontargeted bulk RNA-seq, ...

Weekly Recap (Nov 2024, part 3)

<em>
 This post is inspired by the Bluesky Network Analyzer made by @theo.io.
</em>
I’m encouraging everyone I know online to join the scientific community on Bluesky. In that post I link to several starter packs — lists of accounts posting about a topic that you can follow individually or all at once to start filling out your network. I started following accounts of people I knew from X and from a few starter packs I came across.

Find people followed by the people you follow, but who you don't follow, using R and the atrrr package

Expand your Bluesky network with R

In the spirit of Learning in Public, I wanted an excuse to explore (1)
<strong>
 click
</strong>
for creating command line interfaces, (2)
<strong>
 Cookiecutter
</strong>
project templates, and (3) modern tools in the Python packaging ecosystem. If you’re primarily an R developer like me, I recently wrote about resources for getting better at Python for R users.

Demonstration for how to build a Python command line utility with Click using a Cookiecutter project template, and how to publish to PyPI and Docker Hub

Build a Python CLI with Click+Cookiecutter

This week’s recap highlights protein design with RoseTTAFold, surveillance with wastewater sequencing, T2T human genomes, Vitessce for visualization of multimodal spatial single-cell data, and Taxometer for taxonomic classification of metagenomics contigs.

Protein design with RoseTTAFold, wastewater sequencing T2T human genomes, single-cell spatial transcriptomics, taxonomic classification, ...

Weekly Recap (Oct 2024, part 4)

This week’s recap highlights a new method for gene-level alignment of single-cell trajectories, an R package for integrating gene and protein identifiers across biological sequence databases, characterization of SVs across humans and apes, universal prediction of cellular phenotypes, a method to quantify cell state heritability versus plasticity and infer cell state transition with single cell data, and a new AI-driven, natural language-oriented

Gene-level alignment of scRNA-seq trajectories, integrating identifiers across databases, SVs in humans and apes, universal prediction of cellular phenotypes, and a bioinformatics analysis chatbot

Weekly Recap (Oct 2024, part 2)

I am in the middle of writing a review / perspectives paper. One that I’m confident will be exciting once we get it published. Some sections of the review cover subject matter at the outer periphery of my expertise.

Listen now | Inciteful with the Zotero plugin is the perfect combo for filling in gaps in your knowledge of the literature in a particular field. BioloGPT and Semantic Scholar are useful too.

Inciteful+Zotero to find relevant literature

This week’s recap highlights a new multispecies codon optimization method, personalized pangenome references with vg, a commentary on the wild west of spike-in normalization, a new pipeline for comprehensive and scalable polygenic scoring across ancestrally diverse populations, a paper showing deep learning / transformer-based methods don’t outperform simple linear models for predicting gene expression after genetic perturbations, and finally, a

What I'm reading this week on multispecies codon optimization, personalized pangenome references, spike-in normalization, automating polygenic scoring, transformers vs linear models, bactoneurons, ...

Weekly Recap (Oct 2024, part 1)

This week’s recap highlights a Nextflow pipeline for eQTL detection, an end-to-end pipeline for spatial transcriptomics (visium) data analysis, a method for identification of perturbed cell types in single cell RNA-seq data, a method for guide assignment in single-cell CRISPR screens, a tool for on-target/off-target analysis of gene editing outcomes, and “digital microbes” for collaborative team science on emerging microbes.

A new eQTL Nextflow workflow, spatial transcriptomics / scRNA-seq, guide assignment in CRISPR screens, on-/off-target analysis after CRISPR editing experiments, digital microbes, paleogenomics, ...

Weekly Recap (Sep 2024, part 5)

Yesterday I wrote about base R vs. dplyr vs. duckdb for a simple summary analysis. In that post I simulated 100 million rows of a dataset and wrote to disk as CSV. I then benchmarked how long it took to read in and compute a simple grouped mean. One thing I
<em>
 didn’t
</em>
do here was separate the time it took to read data into memory (for base R and dplyr) versus computing the actual summary.

Parquet is interoperable between Python and R, fast to read+write, works well with databases, and stores complex data types (e.g., tibble listcols). Use it instead of CSV. Many pros, few (no?) cons.

Use nanoparquet instead of readr/CSV

I recently stumbled across Phil Ewels’s ~18 minute nf-core/bytesize talk on Excalidraw: For years I’ve been using draw.io for making flowcharts and diagrams for documentation, papers, presentations, and for general brainstorming and communication with my team, clients, and collaborators.1 Excalidraw (excalidraw.com) looks like an attractive alternative.

Excalidraw might be my new go-to open-source tool over draw.io for creating and sharing workflow diagrams. It's simple, fast, aesthetically pleasing, and has amazing VS Code integration.

Excalidraw: create and share workflow diagrams with end-to-end encryption

This is an experiment... ignore this post

Paired Ends Podcast

This week’s recap highlights a new nf-core workflow for multi-omics trait association studies, a new tool for linking genotype to phenotype (G2P) by directly sequencing alleles from CRISPR base editing experiments, the SplitsTree app for interactive analysis and visualization using phylogenetic trees and networks, mapping cellular interactions from spatially resolved transcriptomics data, a study of marine microbial diversity and bioprospecting

Regulatory element mapping, phylogenetic tree analysis+viz, mapping cellular interactions with scRNA-seq, bioprospecting, transformers in single-cell omics, nf-core/omicsgenetraitassociation, ...

Weekly Recap (Sep 2024, part 4)

Google has a new experimental1 tool called Illuminate (
<strong>
 illuminate.google.com
</strong>
) that takes a link to a preprint2 and creates a podcast discussing the paper. When I tested this with a few preprints, the podcasts it generated are about 6-8 minutes long, featuring a male and female voice discussing the key points of the paper in a conversational style. There are some obvious shortcomings.

Listen now | Google's new Illuminate tool takes a PDF from arXiv and generates a short podcast discussion on the paper's key points

Illuminate preprints with an AI-generated podcast discussion

This week’s recap highlights Google/Deepmind’s new AlphaProteo tool for protein design, tools for protein structure alignment and analysis, biases in polygenic risk scores due to overlap and kinship, highly variable gene selection in single cell RNA-seq, and reconstruction of a 4.2 billion year old last universal common ancestor of life on Earth (spoiler alert: CRISPR-Cas is &gt;4B years old!).  Others that caught my attention include a new

What I'm reading this week in protein design with AlphaProteo, biases in polygenic risk scores, scRNA-seq HVG selection, reconstruction of a 4.2 billion year old last universal common ancestor, ...

Weekly Recap (Sep 2024, part 2)

<strong>
 TL;DR
</strong>
I wrote an R package that summarizes recent bioRxiv preprints using a locally running LLM via Ollama+ollamar, and produces a summary HTML report from a parameterized RMarkdown template.

The biorecap package summarizes bioRxiv preprints using a local LLM. I used two LLMs to help me write the package, and two other LLMs to help me write a paper about the package, published on arXiv.

biorecap: an R package for summarizing bioRxiv preprints with a local LLM

What I'm reading this week in structural variant analysis, population/medical genetics, infectious disease modeling+forecasting, ML in single cell omics, paleogenetics, pangenomes, cancer genomics ...

What I'm reading (Aug 2024, part 3)

<em>
 Something a little different for this week’s recap. I’ve been thinking a lot lately about the practice of data science education in this era of widely available (and really good!) LLMs for code. Commentary at the top based on my own data science teaching experience, with a deep dive into a few recent papers below.
</em>

Listen now | Balancing growth with guidance: teaching data science in the age of AI assistants

AI in data science education

Happy New Year! I’m still catching up on papers from my late 2024 backlog.

Directed evolution by a protein language model, AI learning to run transcript assemblers, a review/introduction to pangenomics, Alphafold2 refinement for protein design, metagenomic binning...

Weekly Recap (Jan 2025, part 1)

Paired Ends

## TL;DR

- I wrote an R package that summarizes recent bioRxiv preprints using a
  locally running LLM via Ollama+ollamar, and produces a summary HTML
  report from a parameterized RMarkdown template. The package is on
  GitHub and you can install it with devtools:
  **<https://github.com/stephenturner/biorecap>**.

- I published a paper about the package on arXiv: Turner, S. D. (2024).
  biorecap: an R package for summarizing bioRxiv preprints with a local
  LLM. *arXiv*, 2408.11707.
  **<https://doi.org/10.48550/arXiv.2408.11707>**.

- I wrote both the package and the paper with assistance from LLMs:
  GitHub copilot for documentation, llama3.1:70b for tests,
  llama3.1:405b via HuggingFace assistants for drafting, GPT-4o for
  editing.

[Subscribe now](https://blog.stephenturner.us/subscribe?){.button
.primary}

## The biorecap package

I recently started to explore prompting a local LLM (e.g. Llama3.1) from
R via Ollama. Last week I [wrote about how to do
this](https://blog.stephenturner.us/p/use-r-to-prompt-a-local-llm-with),
with a few contrived examples: (1) trying to figure out what's
interesting about a set of genes, and (2) summarizing a set of preprints
retrieved from bioRxiv's RSS feed. The gene set analysis really was
contrived --- as I mentioned in the footnote to [that
post](https://blog.stephenturner.us/p/use-r-to-prompt-a-local-llm-with),
you'd never actually want to do a gene set analysis this way when there
are plenty of well-understood first-principles approaches to gene set
analysis.

The second example wasn't so contrived. I subscribe to bioRxiv's RSS
feeds, along with many other journals and blogs in genetics,
bioinformatics, data science, synthetic biology, and others. The
fusillade of new preprints and peer-reviewed papers relevant to my work
is relentless. Late last year [bioRxiv started adding AI summaries to
newly published
preprints](https://connect.biorxiv.org/news/2023/11/08/summaries), but
this required multiple clicks to get out of my RSS feed onto the paper's
landing page, another to click into the AI summary. I wanted a way to
give me a quick TL;DR on all the recent papers published in particular
subject areas (e.g., bioinformatics, genomics, or synthetic biology).

Shortly after putting together that one-off demo, on a sultry Sunday
afternoon in Virginia too hot to do anything outside, I took the code
from that post, generalized it a bit, and created the biorecap package:
**<https://github.com/stephenturner/biorecap>**.

You can install it with devtools/remotes:

    remotes::install_github("stephenturner/biorecap", 
                            dependencies=TRUE)

Create a report with 2-sentence summaries of recent papers published in
the bioinformatics, genomics, and synthetic biology sections, using
llama3.1:8b running locally on your laptop:

    my_subjects <- c("bioinformatics", "genomics", "synthetic_biology")

    biorecap_report(output_dir=".", 
                    subject=my_subjects, 
                    model="llama3.1")

This report will look different each time you run it, depending on
what's been published that day in the sections you're interested in.

::: {.captioned-image-container}
<figure>
<a
href="https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6773a849-41ed-4bef-89bd-3c00f5188c87_984x999.png"
class="image-link image2 is-viewable-img" target="_blank"
data-component-name="Image2ToDOM"></a>
<div class="image2-inset">
<img
src="https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6773a849-41ed-4bef-89bd-3c00f5188c87_984x999.png"
class="sizing-normal"
data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/6773a849-41ed-4bef-89bd-3c00f5188c87_984x999.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:999,&quot;width&quot;:984,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:259079,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false}"
srcset="https://substackcdn.com/image/fetch/w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6773a849-41ed-4bef-89bd-3c00f5188c87_984x999.png 424w, https://substackcdn.com/image/fetch/w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6773a849-41ed-4bef-89bd-3c00f5188c87_984x999.png 848w, https://substackcdn.com/image/fetch/w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6773a849-41ed-4bef-89bd-3c00f5188c87_984x999.png 1272w, https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6773a849-41ed-4bef-89bd-3c00f5188c87_984x999.png 1456w"
sizes="100vw" loading="lazy" width="984" height="999" />
<div class="image-link-expand">
<div class="pencraft pc-display-flex pc-gap-8 pc-reset">
<div class="pencraft pc-reset icon-container restack-image">
<img
src="data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyMCIgaGVpZ2h0PSIyMCIgdmlld2JveD0iMCAwIDI0IDI0IiBmaWxsPSJub25lIiBzdHJva2U9ImN1cnJlbnRDb2xvciIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS1saW5lam9pbj0icm91bmQiIGNsYXNzPSJsdWNpZGUgbHVjaWRlLXJlZnJlc2gtY3ciPjxwYXRoIGQ9Ik0zIDEyYTkgOSAwIDAgMSA5LTkgOS43NSA5Ljc1IDAgMCAxIDYuNzQgMi43NEwyMSA4IiAvPjxwYXRoIGQ9Ik0yMSAzdjVoLTUiIC8+PHBhdGggZD0iTTIxIDEyYTkgOSAwIDAgMS05IDkgOS43NSA5Ljc1IDAgMCAxLTYuNzQtMi43NEwzIDE2IiAvPjxwYXRoIGQ9Ik04IDE2SDN2NSIgLz48L3N2Zz4="
class="lucide lucide-refresh-cw" />
</div>
<div class="pencraft pc-reset icon-container view-image">
<img
src="data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyMCIgaGVpZ2h0PSIyMCIgdmlld2JveD0iMCAwIDI0IDI0IiBmaWxsPSJub25lIiBzdHJva2U9ImN1cnJlbnRDb2xvciIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS1saW5lam9pbj0icm91bmQiIGNsYXNzPSJsdWNpZGUgbHVjaWRlLW1heGltaXplMiI+PHBvbHlsaW5lIHBvaW50cz0iMTUgMyAyMSAzIDIxIDkiPjwvcG9seWxpbmU+PHBvbHlsaW5lIHBvaW50cz0iOSAyMSAzIDIxIDMgMTUiPjwvcG9seWxpbmU+PGxpbmUgeDE9IjIxIiB4Mj0iMTQiIHkxPSIzIiB5Mj0iMTAiPjwvbGluZT48bGluZSB4MT0iMyIgeDI9IjEwIiB5MT0iMjEiIHkyPSIxNCI+PC9saW5lPjwvc3ZnPg=="
class="lucide lucide-maximize2" />
</div>
</div>
</div>
</div>
<figcaption>Figure 1 from <a
href="https://doi.org/10.48550/arXiv.2408.11707">Turner 2024 arXiv</a>:
Example biorecap report for bioinformatics, genomics, and synthetic
biology from August 6, 2024.</figcaption>
</figure>
:::

The package documentation provides further instructions on usage.

I mentioned I used LLMs to help write this package. I started out using
[Positron for package
development](https://blog.stephenturner.us/p/r-package-development-in-positron),
but quickly fell back to RStudio because I wanted GitHub copilot
integration. Among other things, Copilot is great for quickly writing
Roxygen documentation for relatively simple functions. I also used
llama3.1:70b running locally via [Open
WebUI](https://github.com/open-webui/open-webui) to help me write some
of the unit tests using testthat. Starting from the code I worked out in
the [previous
post](https://blog.stephenturner.us/p/use-r-to-prompt-a-local-llm-with),
it took about 2 hours to get the package working, and another hour or so
to write documentation and tests and set up GitHub actions for pkgdown
deployment and R CMD checks on PRs to main.

### The biorecap paper

I published a short paper about the package on arXiv: Turner, S. D.
(2024). biorecap: an R package for summarizing bioRxiv preprints with a
local LLM. *arXiv*, 2408.11707.
<https://doi.org/10.48550/arXiv.2408.11707>.

::::::::: {.file-embed-wrapper component-name="FileToDOM"}
:::::::: {.file-embed-container-reader}
:::::: {.file-embed-container-top}
::::: {.file-embed-details}
::: {.file-embed-details-h1}
biorecap-preprint
:::

::: {.file-embed-details-h2}
266KB ∙ PDF file
:::
:::::

[[Download]{.file-embed-button-text}](https://blog.stephenturner.us/api/v1/file/7b6dfcba-cc37-44dc-9ec8-9c23ad01edc0.pdf){.file-embed-button
.wide}
::::::

::: {.file-embed-description}
The biorecap preprint published on arXiv
:::

[[Download]{.file-embed-button-text}](https://blog.stephenturner.us/api/v1/file/7b6dfcba-cc37-44dc-9ec8-9c23ad01edc0.pdf){.file-embed-button
.narrow}
::::::::
:::::::::

I used two LLMs to assist me in writing the package, which itself uses
an LLM to summarize research published on bioRxiv. It only makes sense
to close the circle and use an LLM to help me write a preprint to
publish on arXiv[1](#footnote-1){#footnote-anchor-1 .footnote-anchor
component-name="FootnoteAnchorToDOM" target="_self"} describing the
software.

I'll write a post soon about how to set up a llama3.1:405b-powered
chatbot on HuggingFace connected to a GitHub repository to be able to
ask questions about the codebase in that repo. It's free and takes about
60 seconds or less. I started out doing this, asking for help crafting a
narrative outline and introduction section for a paper based on the code
in the repo. I ended up scrapping this and writing most of the first
draft text myself, then using GPT-4o to help with editing, tone, and
length reduction. It's hard to exactly put my finger on it, but what I
ended up with still had that feeling of "sounds like it was written by
ChatGPT." I did some of the final editing on my own, and used Mike
Mahoney's [arXiv template Quarto
plugin](https://github.com/mikemahoney218/quarto-arxiv) to write and
typeset the final document.

[Subscribe now](https://blog.stephenturner.us/subscribe?){.button
.primary}

:::: {.footnote component-name="FootnoteToDOM"}
[1](#footnote-anchor-1){#footnote-1 .footnote-number
contenteditable="false" target="_self"}

::: {.footnote-content}
Why not publish this on bioRxiv? [bioRxiv's
FAQ](https://www.biorxiv.org/about/FAQ) notes that they don't allow
methods, tool, or protocol papers without being placed in the context of
a full research paper with an introduction, results, and discussion
section. Enforcement of this boundary has been inconsistent. I've had
preprints rejected from bioRxiv in the past having far more new data and
results than the paper here does. In this case, arXiv was easier and
faster.
:::
::::
