BiSCo-LLM is a codebook-free binary spherical coding framework that compresses LLM weights to near 2 bits per weight. It maps local weights onto a unit hypersphere and binarizes them into compact symbolic streams, eliminating the need for explicit codebook storage.

Why does this matter?

It eliminates the massive codebook storage overhead that has plagued vector quantization, enabling efficient LLM deployment on resource-constrained edge and mobile devices while maintaining competitive accuracy at ultra-low bitrates.

What should we watch next?

Watch for adoption by major inference frameworks, and how the hybrid precision strategy—residual encoding plus protected 8-bit channels—scales to larger models and diverse downstream tasks.

BiSCo-LLM：無需碼本的二進製球面編碼實現極低比特大模型壓縮

針對大語言模型部署中面臨的記憶體容量、權重頻寬及檢查點儲存瓶頸，本文提出BiSCo-LLM，一種無需顯式碼本的二進製球面編碼框架，旨在解決極低比特（接近2 bits/權重）下的量化表示能力受限問題。傳統標量量化在低比特下精度損失嚴重，而向量量化則依賴龐大的碼本查找與額外儲存。BiSCo-LLM透過將局部權重映射至單位超球面並二值化為緊湊的球面碼，以位打包符號流替代顯式VQ質心，大幅降低儲存開銷。結合殘差二進製球面量化編碼重建誤差，並引入類別恢復蒸餾技術，有效緩解了局部權重重建與整體模型行為之間的失配。實驗表明，該方法在極低比特下實現了優異的性能平衡。

Sources

arXiv