The Continual Learning Opportunity

Every AI model launches today with its knowledge locked, but the problems the models work on keep changing. For example, cybersecurity systems must adapt to new adversarial methods quickly and effectively. Without a way to keep learning, models lose effectiveness as their environment changes. We believe solving this problem, known as continual learning, is one of the largest opportunities in AI today.

We've looked at aspects of this problem before through the lens of modular memory stores. An earlier writeup from my colleague Daniel explored how memory helps models stay current between training runs. Continual learning extends that thread from memory to learning which enables AI models to perform effectively on the axes of speed and consistency as well as peak quality.

What is continual learning?

Picture a model's weights as billions of dials. Training sets those dials as the model sees examples, makes guesses, learns how far off it is, and nudges the relevant dials toward a better guess. Repeat this across billions of examples and the final values encode what the model knows and how its concepts relate to one another. Continual learning keeps turning those dials after training ends. Instead of freezing the weights the model keeps absorbing new information rather than going stale when training stops.

The hard part is turning dials to learn something new without the model losing stability and forgetting important, previously known information. That failure is called catastrophic forgetting, and avoiding it means balancing stability, holding onto old knowledge, against plasticity, absorbing new knowledge.

Why does this matter now?

As labs and startups race to automate research, continual learning is a primary bottleneck. Some of the leading startups in the space have raised over $150M in the last three months, some at seed valuations above $1B.

Continual learning is necessary, but not sufficient, for self-improving systems. A model that cannot take in new information will never improve itself properly as the other necessary capability is judgment. Deciding what to learn and where to explore next are separate problems which have been explored by labs and independent researchers alike.

Anthropic wrote in a June 2026 piece that Claude wrote more than eighty percent of the code merged into Anthropic's codebase and the typical engineer shipped eight times as much code per day as in 2024. On a test where models pick the next step in a real experiment, the best model beat the human choice 51 percent of the time in November 2025 and 64 percent by April 2026.

Who captures the value?

The biggest beneficiaries of continual learning will be companies that own unique inference data and workflows. When your product generates a stream of real usage and outcomes that no competitor can replicate that stream can become a training signal seeding a model that keeps improving at tasks.

Capturing the full value of that data, however, is expensive. Doing it alone requires thousands of leading edge GPUs, interconnect systems, enough memory to hold weights and computations, and dozens of talented researchers. A handful of workflow owners have those resources. Given that most do not an opportunity exists for standalone continual learning companies to fill the gap. We see a major opportunity for standalone companies that help workflow owners build and update their own models on top of open weight bases, alongside the workflow owners themselves who put their proprietary data to work.

How will the problem get solved?

Two paths stay within continual learning's strict definition. Both update the weights, one through retraining, one by changing the hardware that makes updates cheap. A third, adjacent path leaves the weights alone and gives the model an external memory store instead. All three are complementary as much as rivals, and most systems may end up using more than one.

• Updating the weights. New knowledge is folded into the weights. The model stores the update everywhere it runs, with nothing to retrieve at query time. This cost shows up in training. Every update is a training run, and editing weights without disturbing old knowledge is the central research problem.

• Changing the hardware. The second approach argues the substrate itself is part of the problem. Today's chips keep compute and memory in separate places, so every weight update moves data across that gap. New chip designs co-locate memory and compute so an update becomes a local, physical change to the device instead of a trip to a data center. A voice model that mishears an unfamiliar accent could adapt on the device itself, at low cost, rather than wait for a retrain.

• Giving the model a memory. This adjacent path leaves the weights frozen and gives the model a persistent external memory that updates as new information arrives. It sits outside continual learning's strict definition, but it aims at the same goal, a model that stays current. Facts pulled from an external store can be cited and edited at low cost, which knowledge baked into the weights cannot. But standalone memory products have struggled to differentiate, and foundation model providers treat memory as a lever for switching costs. Most systems that update weights will also want a retrieval layer alongside them.

These approaches have different cost structures. Weight updates push cost into training, and that spending flows to compute providers and whoever owns the update pipeline. Memory approaches push cost into inference, where every call retrieves context and burns tokens. The spending shifts rather than disappears, which is why we expect value to accrue across the whole stack. The open research question underneath all of it is what information requires an update to the weights and what can stay retrievable at query time. According to a research scientist at a leading lab “Right now, in a fairly naive approach, we’re trying to save and structure as much as we can, because deciding what to forget is really the core of understanding what constitutes intelligence. A potential path is to develop ‘rules of construction’ for what is core to solving a problem.”

Are you building here?

If you own a workflow, some questions worth asking first are whether you are generating data a competitor could not replicate, exposed to competitors without your own learning loop, and whether you can fund the compute yourself or if it’s better to partner. 

The overarching question then is when, not if, and in our view it comes down to cost and performance. At Flybridge we’ve been investing in this space since 2023 through Arcee who started off by productizing continual pre-training and model merging to give enterprises domain-specialized models that run in their own cloud at a fraction of frontier-model cost. 

Meanwhile, Prime Intellect's infrastructure let Ramp train a small model on its own spreadsheet-search data and the result beat a leading frontier model on accuracy while running faster and cheaper than a smaller one. This represented a task narrow and high-volume enough that a bespoke model wins on both cost and quality. A foundation model paired with a memory layer is sufficient until training and maintaining a smaller model costs less than what you already pay to deploy existing models in a workflow.

If you are building continual learning infrastructure, or you own a workflow and unique data and want to turn it into a durable edge, we want to hear from you. We are also actively investing in this space and would love to compare notes.