inference
by xorbitsai (organization)
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API. (description from the project’s GitHub page)
Repository facts
- Stars
- 9,616
- Forks
- 884
- Open issues and pull requests
- 31
- Primary language
- Python
- License
- Apache License 2.0 (Apache-2.0)
- Owner
- xorbitsai · Organization
- Created
- Jun 14, 2023
- Last push
- Oct 18, 2026
Topics
Add this badge to your README
Shows this repository’s rank in AI on GitHub and links back to this page. Ranks are by stars and change when the list is refreshed. The badge is not an endorsement or a rating. About the badges
Markdown (for a README)
[](https://unitedstatesofamericasuperintelligence.com/github/xorbitsai--inference/)
HTML
<a href="https://unitedstatesofamericasuperintelligence.com/github/xorbitsai--inference/"><img src="https://unitedstatesofamericasuperintelligence.com/badges/github/xorbitsai--inference.svg" alt="Ranked #851 in AI on GitHub" height="20"></a>
Related projects
Projects that share this one’s GitHub topics.
Rank 3,852: SemiAnalysisAI/InferenceX
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
1,874 starsPythonApache-2.0Last push Oct 18, 2026
Rank 8,611: zhongkaifu/TensorSharp
A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-comp…
592 starsC#BSD-3-ClauseLast push Oct 18, 2026
Rank 7,564: pegainfer-project/pegainfer
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
724 starsRustApache-2.0Last push Oct 17, 2026
Rank 1,389: gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
5,844 starsPythonApache-2.0Last push Oct 16, 2026
Rank 7,018: XHToken/Spark-X2.5
Spark-x2.5 open model series. Pushing the Limits of Agentic Capabilities in On-Device Models
812 starsApache-2.0Last push Sep 20, 2026
Rank 7,913: StartLuxLabs/StartLux-Decision
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
676 starsPythonApache-2.0Last push Oct 17, 2026