"Across algorithm engineering, mathematical optimization, and GPU kernel engineering, Dream-RSI achieves competitive or improved discovery quality while substantially reducing discovery cost in several settings."
Even today's lighter weight models know how to write kernels and optimize them. I've had DeepSeek 4.1 Flash tune the crap out custom CUDA kernels on my own codebase and it was entirely competent at it. And cheap.
The innovation pieces will be in the harnesses to support this. Which I guess is partially what's going on here.
This is like asking why a paper called “I discovered God. This is the specific proof.” is revolutionary. Just tedious levels of incredulity.
This looks like an optimization of current training methods, and a good one, but not "RSI" in the sense of a system that can perpetually improve itself forever.
I struggle with the argument that RSI doesn't already exist like you say, it's existed since before the term LLM (hey, one that can be defined!) was common parlance. Though the biggest use for those is not superintelligence, it's to serve you ads and get your kids addicted to TikTok.
It has technically existed for a long time (for longer than the name), but only on academical applications for extremely limited intelligences that could only create something like themselves. And that is still the only form that exists today.
It was never powerful enough to optimize ads distribution, and all the claims people are pushing around today are plain bullshit.
What matters isn't 95% of humans, it's 95% of actual professionals. Benchmarking an AI accountant against people with zero accounting experience is worse than worthless.
I'd say it's much closer to the concept of continual learning, but I'm only a few pages deep and haven't groqued it fully yet.
Imagine you have a problem you want to solve (let's say, identify an OCR'd handwritten character, e.g. the MNIST Dataset). You tell 3 agents "Hey, each of you take a stab at getting really good at recognizing characters from this dataset. You can take 10 refinement steps to continue to improve ". You can't give each agent unlimited steps of course, because you have a finite amount of compute.
So each agent goes off, and by the end, Agent 1 got to 90% accuracy, Agent 2 got to 80% accuracy, and Agent 3 got to 89% accuracy. Agent 1 wins, of course.
But then you look at the refinement steps, and after 2 steps, Agent 1 was _already at_ 90% accuracy. So the agent spent the next 8 steps basically not moving at all. Agent 3 on the other hand, perhaps was continuously climbing in accuracy at every refinement step, but hit step 10 and had to stop.
Now because you recorded every step from every agent, you know what you'd do differently next time -- you'd not allocate as many steps to Agent 1, and give Agent 3 more steps, because perhaps that might result in Agent 3 coming up with a better answer.
From my understanding, that's what they built in the form of a "search" controller -- a way to evaluate automatically and reapply how you could allocate resources more effectively, when applied to a new problem.
But I guess my misunderstanding is how applicable the search controller is when applied to new problems -- just because one pathway stalled early for one problem, doesn't mean it would work for another?
Do you think this could be extrapolated to areas with no objectively verifiable results / outcomes?
(Outside of math & science)
This iterative, online optimization of an exploration policy is not recursively intelligent in any way. It simply reallocates the available computational resources to more promising (hopefully) parts of the search space as system conditions change over time.
The whole idea behind RL is that the agent improves over time by making actions in the environment and observing the next state and the reward then modifying its policy.
The exploration-exploitation dilemma still stands. "dreaming in the replay simulator" is jargon for the same replay ideas presented when Deep RL was first introduced (Q-learning with experience replay and all that).
ctrl+f 'danger'
Yup, we're all gonna die :(
RSI is the new sexy. Models are RSI-ing themselves towards the singularity, these folks' agents are RSI-ing themselves towards mastery of their training environments, and my pet cat is RSI-ing himself into the best cat that he can be.
The TalkRL podcasts on this line of work are reasonable accessible and quite interesting. https://www.talkrl.com/episodes/danijar-hafner https://www.talkrl.com/episodes/danijar-hafner-on-dreamer-v4...