How does implicit feedback reshape LLM alignment?

This approach uses mouse trajectories and eye gaze instead of explicit ratings. The study introduces the IFLLM dataset with 1,336 conversations from 59 participants, capturing natural behavioral signals to quantify preferences, addressing the high cost and scarcity of traditional human feedback.

What is the impact of this method on model performance?

Implicit feedback boosted reward model accuracy from 55% to 64%. When combined with Direct Preference Optimization (DPO), it nearly tripled response quality across eight major LLMs. This proves that real-world behavioral data captures user preferences more effectively than text-only signals.

What should be considered for future adoption of this technology?

While enabling low-cost, high-fidelity alignment, privacy and ethics are critical. Future work must address how to protect user privacy while collecting implicit behavioral data. Researchers should also explore fusing more complex implicit signals for even better model understanding.

滑鼠與視線洩露偏好：基於隱式回饋的大語言模型對齊新方法

現有大模型對齊方法依賴顯式人類回饋，標註成本高且參與度有限。本文提出利用滑鼠軌跡、眼球注視等隱式訊號進行對齊。作者建構IFLLM資料集，收集59名參與者1336輪多輪對話的行為資料。實驗顯示，基於隱式回饋的獎勵模型將準確率從55%提升至64%，應用DPO後八個大模型回應品質相對提升近三倍。研究證明了野外隱式回饋的巨大價值，開源資料與程式碼為低成本高保真對齊開闢了新路徑。

Sources

arXiv