Breaking News — DeepSeek AI has officially signaled the imminent arrival of its next-generation model, R2, while simultaneously unveiling a novel technique called SPCT (Self-Principled Critique Tuning) designed to dramatically improve how large language models scale during inference. The dual announcement, detailed in a newly published research paper titled “Inference-Time Scaling for Generalist Reward Modeling,” positions the company at the forefront of a paradigm shift from pre-training to post-training optimization.
The SPCT method allows general reward models (GRMs) to dynamically generate principles and critiques, optimizing reward generation without relying on static rules. According to the paper, this is achieved through a combination of rejection fine-tuning and rule-based online reinforcement learning, effectively enabling models to self-correct and refine reasoning in real time.
“This approach represents a fundamental shift in how we think about scaling,” said Wu Yi, an assistant professor at Tsinghua University’s Institute for Interdisciplinary Information Sciences (IIIS). “Instead of simply adding more compute during training, DeepSeek is pushing the boundaries of what can be achieved during the inference phase.”
Background
The development comes amid a broader industry move toward post-training scaling, following the success of models like OpenAI’s o1. These models leverage increased reinforcement learning during training and extended “thinking time” during testing—generating long internal chains of thought before responding to users.
DeepSeek’s own R1 series had already validated the potential of pure reinforcement learning without supervised fine-tuning. The new R2 model builds on that foundation, hinting at even more advanced reasoning capabilities through enhanced inference-time optimization.
The core mechanism of large language models—next token prediction—provides vast knowledge but lacks deep planning and long-term outcome prediction. Reinforcement learning acts as a critical complement, providing an internal world model that simulates potential outcomes of different reasoning paths.
What This Means
The SPCT technique and R2 model signal that the competitive landscape for advanced AI is moving beyond pre-training scale. Companies that master inference-time optimization could gain significant advantages in complex reasoning tasks, from scientific research to autonomous decision-making.
“The relationship between LLMs and reinforcement learning is multiplicative,” explained Wu Yi. “Only when a strong foundation of understanding, memory, and logical reasoning is built during pre-training can reinforcement learning fully unlock its potential to create a complete intelligent agent.”
For developers and enterprises, this means AI systems may soon exhibit more systematic long-term planning, fewer logical errors, and greater adaptability in real-world scenarios. DeepSeek’s R2, when launched, is expected to set new benchmarks in reasoning, especially in domains requiring multi-step problem solving.
Immediate impact: Competitors are racing to replicate or surpass DeepSeek’s approach. The open-source community is expected to rapidly adopt the SPCT methodology, accelerating innovation across the field.