MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...
The AI infra landscape has reached a critical inflection point where dominant hyperscalers are being forced into the backseat ...