TipHey, I’m writing a book about this! I’m actually writing a book about this stuff. It turns out there isn’t a lot of literature on how to do post-training at the level too big for single-GPU laptop-sized hobby projects and requiring enterprise reliability on one hand, but not quite at the scale of multi-team distributed post-training you’d get in foundation labs.
There’s an old parable about a drunk searching for his keys under a streetlamp. A passerby stops to help, and after some fruitless searching asks, “Are you sure you lost them here?” The drunk replies, “No, I lost them in the alley. But the light’s better here.” I think about this story a lot when I read AI predictions.
TipHey, I’m writing a book about this! I’m actually writing a book about this stuff. It turns out there isn’t a lot of literature on how to do post-training at the level too big for single-GPU laptop-sized hobby projects and requiring enterprise reliability on one hand, but not quite at the scale of multi-team distributed post-training you’d get in foundation labs.
TipHey, I’m writing a book about this! I’m actually writing a book about this stuff. It turns out there isn’t a lot of literature on how to do post-training at the level too big for single-GPU laptop-sized hobby projects and requiring enterprise reliability on one hand, but not quite at the scale of multi-team distributed post-training you’d get in foundation labs.
In the dying days of the Roman Republic, there existed a class of functionaries called nomenclatores . Their job was to whisper the names of approaching citizens into their patron’s ear so that the great man could greet each one as if they were intimates.
Hype, slop, craft: the evolutionary ecology of AI There’s a patch of ground near Chernobyl that botanists call Рудий ліс, the Red Forest. In the immediate aftermath of the 1986 disaster, 1 the radiation killed mostly everything, turning the pine trees a rust-red colour before they died. That’s poetic, but hardly unexpected – not even Polesian pines can withstand a firehose of low enriched uranium decay products.
The Latin verb agere means “to do, to act, to drive forward”. From it we get not only “agent” but also “action”, “actor”, and curiously enough, “agile”. The Romans understood what we seem to have forgotten: that agency is fundamentally about motion, about transformation, about the capacity to change the world.
This is about the time of year when I get the sudden and irresistible urge to make kimchi.
In 1790, the French Academy of Sciences commissioned a rather ambitious survey. The goal was to measure the distance from the North Pole to the Equator along the meridian passing through Paris, then use that measurement to define a new universal unit of length: the metre.
Roughly 541 million years ago, something extraordinary happened in Earth’s oceans. Over a geologically brief period of perhaps 20 million years, the fossil record explodes with an almost obscene diversity of body plans.
Around two years ago, almost to the day, I spent an absolutely frantic evening in Berkeley, going through enough coffee to power a mid-sized city, hammering away at my laptop on trying to figure out what comes after LLMs. What to most people was still barely on the horizon at the time has been a subject I have been working on in various capacities for the best part of the past decade, on and off.