Rank #5,301 1,229 starsPython

MInference

by microsoft (organization)

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy. (description from the project’s GitHub page)

View on GitHub Homepage

Repository facts

Stars: 1,229
Forks: 84
Open issues and pull requests: 99
Primary language: Python
License: MIT License (MIT)
Owner: microsoft · Organization
Created: May 22, 2024
Last push: Oct 10, 2026

Rank #5,301 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Add this badge to your README

Ranked #5,301 in AI on GitHub Shows this repository’s rank in AI on GitHub and links back to this page. Ranks are by stars and change when the list is refreshed. The badge is not an endorsement or a rating. About the badges

Markdown (for a README)

[![Ranked #5,301 in AI on GitHub](https://unitedstatesofamericasuperintelligence.com/badges/github/microsoft--minference.svg)](https://unitedstatesofamericasuperintelligence.com/github/microsoft--minference/)

HTML

<a href="https://unitedstatesofamericasuperintelligence.com/github/microsoft--minference/"><img src="https://unitedstatesofamericasuperintelligence.com/badges/github/microsoft--minference.svg" alt="Ranked #5,301 in AI on GitHub" height="20"></a>

Repository facts

Add this badge to your README

Support Us