Faster Convergence
The Reliable Update Principle
Restricting off-policy updates to the common sub-measure between current relevance and replay support.
1. Dual-Constraint Bounded Measure
Updates satisfy $q_t(s) \le \min(\mu_t(s), \nu_t(s))$, eliminating the gradient variance explosion inherent to unbounded exact density ratios.
2. Classifier Posterior Approximation
Binary classifier posterior $D^*(s)$ provides a principled, lightweight, and strictly bounded $[0, 1]$ approximation of optimal update weights.
3. Coverage Regularization Floor
Label smoothing preserves a non-zero floor $\epsilon \mu_t(s)$, maintaining a healthy Effective Sample Size (ESS) throughout training.
HumanoidBench Aggregated Benchmark
Normalized IQM Scores across 9 humanoid tasks against 4 competitive baselines.
Key Takeaway: Reliable Replay (Purple Curve) consistently achieves superior sample efficiency (Env Steps) and wall-clock training throughput across both FastTD3 and FastSAC backbones.
Generalization to Scaled Simulation Platforms
Robust sample efficiency gains across diverse high-dimensional continuous control frameworks.
Demonstrates consistent returns and faster policy convergence on high-DoF tasks (LeapCubeReorient, G1JoystickRoughTerrain).
Seamlessly accelerates policy learning in massively parallel environments including Allegro Hand and H1 Rough Terrain.