Megatron-LM and Megatron Core
Project record
NVIDIA's Megatron-LM repository contains two components: Megatron Core, a library of GPU-optimized building blocks for training transformer models at scale (tensor, pipeline, data, expert, and context parallelism; FP16, BF16, FP8, and FP4 mixed precision), and Megatron-LM, a reference training setup with pre-configured scripts built on Megatron Core. The README says Megatron Core development moved to GitHub in December 2025, with all development and CI now happening in the open.1
- Repository: GitHub repository (external site: github.com)
- Documentation: Megatron Core developer guide (external site: docs.nvidia.com)
- License: License (external site: raw.githubusercontent.com)
- Paper: Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism (arXiv 1909.08053) (external site: arxiv.org)
Availability and license
Overall availability
Documented as available to the general public. Access conditions and license terms may still apply.12
Availability is separate from permission: read the license before using or redistributing.
The LICENSE file applies BSD 3-Clause terms (copyright NVIDIA Corporation) to all files unless otherwise noted, and states that the repository also contains third-party code under the Apache License 2.0 and MIT License, identified by file headers. The README badge reads "Apache", and the megatron-core PyPI metadata gives "Apache 2.0" in its license field while listing a BSD license classifier; this record follows the LICENSE file.215
Public materials checklist
| Item | Status | Notes and evidence |
|---|---|---|
| Source codeIs the source code publicly readable? | Public | Public GitHub repository; the README says all development and CI now happen in the open.1 |
| DocumentationIs user documentation published? | Public | The Megatron Core developer guide covers installation, a first training run, data preparation, and parallelism.34 |
| Training codeDoes it include code for training models? | Public | Includes training scripts and examples, such as a Llama 3 8B FP8 training script, plus post-training (quantization, distillation, pruning) and reinforcement learning code.14 |
| Data informationAre the data it expects or ships with documented? | Public | The quickstart documents the expected JSONL input and the preprocessing script that tokenizes it into binary .bin/.idx files.4 |
| ReproducibilityAre instructions for reproducing reported results published? | Unknown | The README describes its scaling benchmark configurations and points to the Megatron Bridge performance summary; whether published instructions reproduce those results was not assessed.1 |
What it is useful for
The README presents Megatron-LM for research teams, learning distributed training, and experimentation, and Megatron Core for developers building custom training frameworks. NVIDIA's Megatron Bridge library builds on Megatron Core for Hugging Face checkpoint conversion and training recipes.1
Run and use notes
- The installation guide lists PyPI (megatron-core), source, and NGC PyTorch container installs; it recommends NVIDIA Turing or later GPUs, requires Hopper, Ada, or Blackwell GPUs for FP8, and lists PyTorch 2.6 or later. The guide lists Python 3.10 or later (3.12 recommended), while the README says the 0.17.0 release drops Python 3.10 support and the current PyPI package (0.19.2) requires Python 3.12 or later.315
Organization context
U.S. eligibility
Eligible · basis: U.S.-governed project
The repository is published under NVIDIA's GitHub organization, its license names NVIDIA Corporation as copyright holder for the project's own code, and the megatron-core PyPI package lists NVIDIA as author and maintainer. NVIDIA's principal executive offices are in Santa Clara, California, per its Form 10-Q for the quarter ended July 26, 2026.256
Sources
This listing is not an endorsement, a safety assessment, or a federal approval.