Anonymous
2 months ago
Run any open-source LLM — DeepSeek, Llama, and more — as an OpenAI-compatible API endpoint in the cloud. By BentoML.
OpenLLM makes it easy to deploy and serve open-source large language models in production. Supports models like DeepSeek, Llama, Mistral, and more with an OpenAI-compatible API. Features include quantization, streaming, batching, and GPU optimization. Built on BentoML for production-grade serving with monitoring and scaling.
Sign in to join the discussion
No comments yet. Be the first to share your thoughts.