LLM Open Source Programs 2025 May

Created in May, 2025

2025 · LLM opensource · products

---

Prompt Caching / KV Cache

LMCache

• How LMCache Turbocharges Enterprise LLM Inference Frameworks
• LMCache x Mooncake: Unite to Pioneer KVCache-Centric LLM Serving System

Mooncake

Inference

llm-d

• Introducing the next generation of AI inference, powered by llm-d

Dynamo

• NVIDIA Dynamo Adds GPU Autoscaling, Kubernetes Automation, and Networking Optimizations
• Beyond the Algorithm with NVIDIA: Introducing NVIDIA Dynamo

SGLang

• Deploying DeepSeek with PD Disaggregation and Large-Scale Expert Parallelism on 96 H100 GPUs

vLLM

Agentic AI

Strands Agents (blog, GitHub)