active project
Epsilon Engine
Data-center inference engine with state-of-the-art efficiency knobs for fast, quality-controlled serving across varying workloads.
Epsilon Engine is a data-center level inference engine built around a toolkit of efficiency knobs. The goal is to keep serving fast under changing workloads while retaining explicit quality control.
Efficiency knobs
- Sparse attention
- KV cache compression
- KV offloading
- Verified algorithms
- Entropy compression
- MLP sparsification
- MLP skipping
- Speculative decoding
- Agentic long context
People
Papers
- Inference time context sparsity: Illusion or opportunity?
- SOCKET: Soft Collision Kernel Estimator for Sparse Attention
- Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity
- vAttention: Verified sparse attention via sampling
- vCache: Verified Semantic Prompt Caching
- vCache: Verified semantic prompt caching
- HashAttention: Semantic sparsity for faster inference
- Scissorhands: Exploiting the persistence of importance hypothesis for LLM KV cache compression at test time