MLPerf
MLPerf is an evaluation tool maintained by MLCommons145. Its code is public4567 under the Apache License 2.0 (MLPerf Inference reference implementations)6 and the Apache License 2.0 (MLPerf Training reference implementations)7.
MLPerf is MLCommons' family of system performance benchmarks. Its suites cover training (time to train a model to a target quality), inference in datacenter, edge, mobile, and tiny settings, and also client PCs, storage, automotive, HPC training, and inference endpoints. Results are submitted in rounds, reviewed by the submitting organizations, and published together by MLCommons.1329
- Website: MLCommons benchmarks overview (external site: mlcommons.org)
- Repository: MLPerf Inference reference implementations (external site: github.com)
- Repository: MLPerf Training reference implementations (external site: github.com)
- Documentation: MLPerf Inference rules (external site: github.com)
- Documentation: MLPerf Training rules (external site: github.com)
- Paper: MLPerf Inference benchmark paper (arXiv 1911.02549) (external site: arxiv.org)
Availability and license
Overall availability
Reference implementations and rules are public on GitHub under Apache 2.0, and published results can be browsed on the MLCommons site. Submitting results requires a signed Contributor License Agreement, and use of results with the MLPerf trademark must follow MLCommons' results messaging guidelines.67819
Availability is separate from permission: read the license before using or redistributing.
Apache License 2.0 (MLPerf Inference reference implementations) (external site: raw.githubusercontent.com)6
Plain-language guide to the Apache License 2.0Apache License 2.0 (MLPerf Training reference implementations) (external site: raw.githubusercontent.com)7
Plain-language guide to the Apache License 2.0Apache License 2.0 (MLPerf policies) (external site: raw.githubusercontent.com)8
Plain-language guide to the Apache License 2.0
MLPerf is a registered trademark of MLCommons. The submission rules state that, after publication, code and results are public and free for use under the MLPerf Terms of Use. Benchmark datasets come from separate sources, and their terms were not reviewed for this record.19
Component reuse rights
- weights
- Unknown — no complete fact-level rights review
- code
- Unknown — no complete fact-level rights review
- data
- Unknown — no complete fact-level rights review
- documentation
- Unknown — no complete fact-level rights review
No complete system-rights review is recorded for this release.
Public materials checklist
| Item | Status | Notes and evidence |
|---|---|---|
| CodeIs the evaluation code published? | Public | Reference implementations for the Inference and Training suites are on GitHub under Apache 2.0; the Training README says they are starting points and not fully optimized.4567 |
| Tasks / dataAre the tasks or test data available? | Public | Each reference implementation includes instructions for downloading its dataset; the Training README points to MLCommons storage for downloads. Terms vary by dataset and were not reviewed individually.54 |
| MethodologyIs the method for scoring described? | Public | Rules for Inference and Training are published, along with general submission rules covering divisions, availability categories, peer review, and publication.239 |
| ReproducibilityAre instructions for reproducing results published? | Public | Reference implementations include Dockerfiles and run scripts, and submissions must include READMEs and software information so results can be replicated.59 |
| LimitationsAre known limitations documented? | Partial | The Training README warns that reference implementations are not intended for real performance measurement, and results pages note that published results are sometimes modified or invalidated, with a change log. No broader discussion of benchmark validity was reviewed.52 |
What it is useful for
Run and use notes
- At review, the Inference README's most recent round is MLPerf Inference v6.1 (submission deadline July 31, 2026), and the Inference: Datacenter page links to v6.1 results. The Training README lists MLPerf Training v6.1 (submission deadline October 16, 2026), while the Training results page shows v6.0 results.4253
Organization context
In the news
MLPerf Training adds an LLM post-training benchmark, starting with round v6.1
MLCommons announced on September 24, 2026 that MLPerf Training is adding an LLM post-training benchmark, beginning with the v6.1 submission round in October 2026. The benchmark reports the wall-clock time a system needs to reach a quality target when post-training Qwen3.5-397B-A17B with reinforcement learning with verifiable rewards, using Group Relative Policy Optimization, on Python software engineering problems from the R2E-Gym dataset. Agents work in sandboxed Linux containers, and the specification and reference implementation are in the mlcommons/training repository.
MLCommons publishes MLPerf Inference v6.1 results with two new tests
MLCommons published results for version 6.1 of the MLPerf Inference benchmark suite on September 16, 2026. The round added two tests: an end-to-end retrieval-augmented generation test that runs a question-answering pipeline of several separate models, and an Edge Agentic Inference test built around multi-turn workloads such as agentic coding. Speculative decoding is now allowed in the interactive scenario for two of the inference benchmarks. Results are published on MLCommons' datacenter and edge results pages.
MLCommons publishes MLPerf Storage v3.0 results with KV cache and vector database tests
MLCommons published results for version 3.0 of its MLPerf Storage benchmark suite on September 1, 2026. The suite measures the performance of storage systems for machine learning workloads, and v3.0 adds a KV Cache test of storage performance for LLM inference cache read and write operations and a Vector Database test covering vector indexing and querying. Version 3.0 also adds support for an S3 object storage access layer alongside the existing POSIX layer, supported for training, checkpointing, and some vector database tests. Results are on the MLCommons Storage benchmark results page.
MLCommons releases MLPerf Client v2.0 with image generation and agentic AI tests
MLCommons released version 2.0 of MLPerf Client, its benchmark for running AI workloads locally on PCs, on August 18, 2026. The release adds an image generation category, with Flux.2 klein 4B as an experimental test, and an agentic AI category with Software Engineering Agent and Data Analyst Agent scenarios that break end-to-end results into LLM inference and tool execution time. Phi 4 Mini Instruct replaces Phi 3.5 mini instruct in the required LLM workloads, and Qwen 3 8B is added as an experimental test. MLCommons says v2.0 is free to download for Windows, macOS, and Linux from GitHub. USASI has not tested MLPerf Client.
MLCommons releases MLPerf Endpoints v0.7, a benchmark for buyers of hosted inference
On July 28, 2026, MLCommons announced MLPerf Endpoints v0.7, a foundation release of a new MLPerf benchmark aimed at enterprise buyers procuring inference compute from neoclouds, cloud providers, and managed services. The release covers three benchmarks with initial results from CoreWeave, Google, Intel, KRAI, and NVIDIA, and supports automated submission pipelines, continuous review tooling, and dynamic result visualization. Results and tooling are at endpoints.mlcommons.org. MLCommons says it will deliver MLPerf Endpoints v1.0 later in 2026, with more buyer-centric rules and agentic workloads.
U.S. eligibility
Eligible · basis: U.S.-governed project
MLPerf is developed and published by MLCommons, which holds the MLPerf trademark and hosts the reference implementations and rules in its GitHub organization. MLCommons Association is listed by the IRS as a 501(c)(6) organization with a Dover, Delaware address.1410
Sources
This listing is not an endorsement, a safety assessment, or a federal approval.
Share or cite this page
- X (opens X)
- Bluesky (opens Bluesky)
- LinkedIn (opens LinkedIn)
- Reddit (opens Reddit)
- Hacker News (opens Hacker News)
- Email (opens your email app)
- RSS updates
Cite this page
APA style
USASI. (2026, September 29). MLPerf. United States of America Superintelligence. https://unitedstatesofamericasuperintelligence.com/open/mlperf/
BibTeX
@misc{usasi_mlperf,
author = {{USASI}},
title = {MLPerf},
year = {2026},
month = sep,
howpublished = {\url{https://unitedstatesofamericasuperintelligence.com/open/mlperf/}},
note = {United States of America Superintelligence. Last updated 2026-09-29}
}The date is when this page was last updated. Add the date you read it if your style needs one.
For maintainers: a “Listed in USASI” badge
Links to this page. It says the entry is listed, not endorsed or rated. About the badge
Markdown (for a README)
[](https://unitedstatesofamericasuperintelligence.com/open/mlperf/)
HTML
<a href="https://unitedstatesofamericasuperintelligence.com/open/mlperf/"><img src="https://unitedstatesofamericasuperintelligence.com/badges/listed.svg" alt="Listed in the USASI catalog" height="20"></a>