What is RLVR fine-tuning and how does it address thermal energy storage scheduling?

RLVR (Reinforcement Learning with Verifiable Rewards) converts exact dynamic programming action values into dense rewards. Using only 30 training prompts, it turns an open-source reasoning model into a high-level scheduler that outputs heat pump setpoints, achieving significant carbon emission reductions.

Why does this research matter for building energy efficiency and decarbonization?

The fine-tuned model reduced office building carbon emissions from 70.5 to 61.2 kg-CO2, approaching the DP optimum of 60.8. This demonstrates that reasoning models can provide a scalable, interpretable paradigm for city-scale energy management beyond traditional black-box optimization.

What developments should we watch for in this area going forward?

Research is advancing toward whole-building testing at higher fidelity and city-scale energy dispatchers. Notably, GPT-5 approaches DP performance without any training, signaling that reasoning capability will become a key focus for complex control tasks across the energy sector.

基於驗證器增強微調的推理模型在熱能儲存控制中的應用

本文提出了一種利用可驗證獎勵的強化學習微調（RLVR）方法，將開源推理模型適配用於建築熱能儲存（TES）調度問題。針對傳統模型預測控制（MPC）和強化學習在跨建築擴展時的困難，研究通過將精確的動態規劃（DP）動作值轉化為密集獎勵，僅用30個訓練提示對模型進行微調，使其成為輸出熱泵設定值的高級調度器。在包含已知最優解的辦公建築基準測試中，微調後的模型將碳排放從70.5 kg-CO2降至61.2 kg-CO2，接近DP最優值60.8 kg-CO2。此外，GPT-5無需特定訓練即可接近DP和MPC性能，而GPT-4o表現較差，突顯了推理能力的重要性。軌跡分析表明，RLVR主要穩定了候選比較、前瞻和可行性檢查等規劃模式。該方法為建築儲能調度和城市級能源管理提供了可擴展的解決方案。

Sources

arXiv