What is BiSCo-LLM and how does it compress large language models?

BiSCo-LLM is a codebook-free binary spherical coding framework that maps local weight blocks onto a unit hypersphere, binarizes them into compact spherical codewords, and replaces explicit VQ centroids with bit-packed sign streams, combined with residual quantization and class-recovery distillation for extreme low-bit compression.

Why does BiSCo-LLM matter for LLM deployment?

It resolves the long-standing trade-off between scalar quantization (which loses representational power below 2 bits) and vector quantization (which requires massive codebook storage). By eliminating codebook dependence, BiSCo-LLM drastically reduces memory and bandwidth needs, enabling LLM deployment on edge devices and mobile phones.

What should we watch for regarding BiSCo-LLM's future?

The framework shows potential compatibility with LoRA fine-tuning adapters, suggesting a path toward lightweight personalized LLM deployment. Its codebook-free design also lowers the barrier for open-source communities to reproduce and distribute compressed models, potentially accelerating AI adoption across diverse scenarios.

BiSCo-LLM：無需碼本的二進制球面編碼實現極低比特大模型壓縮

針對大語言模型部署中面臨的記憶體容量、權重頻寬及檢查點儲存瓶頸，現有低比特壓縮方法在標量量化與向量量化之間難以兼顧效率與表示能力。本文提出BiSCo-LLM，一種無需碼本的二進制球面編碼框架，旨在實現極低比特下的權重壓縮。該方法首先將局部權重塊映射至單位超球面並二值化為緊湊的球面碼字，以位打包的符號流替代顯式VQ質心；其次，透過殘差二進制球面量化階段編碼基礎編解碼器留下的重建誤差，提供無需儲存碼本的顯式率失真路徑；最後，引入類別恢復蒸餾技術，在替換Transformer模組後減少局部權重重建與整體模型行為之間的失配。此外，採用8位保護通道輔助穩定敏感通道。實驗表明，該框架在平衡壓縮率與模型效能方面表現優異，為極端低比特場景下的LLM部署提供了新範式。

Sources

arXiv