active project
Epsilon Edge
Edge inference engine with state-of-the-art efficiency knobs for fast, quality-controlled serving across varying workloads.
Epsilon Edge is an edge inference engine with the same class of efficiency knobs as our data-center stack, tuned for constrained devices and on-prem edge deployments. It targets fast execution under varying workloads with quality control in the loop.
Efficiency knobs
- Sparse attention
- KV cache compression
- KV offloading
- Verified algorithms
- Entropy compression
- MLP sparsification
- MLP skipping
- Speculative decoding
- Agentic long context
People
Papers
- Inference time context sparsity: Illusion or opportunity?
- SOCKET: Soft Collision Kernel Estimator for Sparse Attention
- Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity
- vAttention: Verified sparse attention via sampling
- vCache: Verified Semantic Prompt Caching
- vCache: Verified semantic prompt caching
- HashAttention: Semantic sparsity for faster inference
- Scissorhands: Exploiting the persistence of importance hypothesis for LLM KV cache compression at test time