Publication · 2023

Scissorhands: Exploiting the persistence of importance hypothesis for LLM KV cache compression at test time

Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, Anshumali Shrivastava · NeurIPS 2023