Independent project. Not a U.S. government website.Independent project. Not a United States government website.
Support our projectSupport usSupport our project · Buy me a coffee (opens Buy Me a Coffee)SWE-bench is a dataset that tests systems’ ability to solve GitHub issues automatically. The dataset collects 2,294 Issue-Pull Request pairs from 12 popular Python repos…
153 likes95,343 downloads in the last 30 daysNo license given
Open-SWE-Traces: Advancing Distillation for Software Engineering Agents 🚨 What's New [09/26] Release v1.2: Added new agent trajectories generated by Qwen3.8-27B for min…
152 likes24,045 downloads in the last 30 daysCC BY 4.0
Action100M: A Large-scale Video Action Dataset Paper | GitHub Action100M is a large-scale dataset constructed from 1.2M Internet instructional videos (14.6 years of dura…
151 likes805 downloads in the last 30 daysVideo classificationFAIR NONCOMMERCIAL RESEARCH LICENSE
Egocentric-100K is the largest dataset of manual labor. You can visualize the dataset here. Egocentric-100K is state-of-the-art in hand visibility and active manipulatio…
150 likes168,543 downloads in the last 30 daysApache 2.0Gated
This dataset contains 3.3K expert-level pairwise human preferences for model responses generated by 6 models in response to 80 MT-bench questions. The 6 models are GPT-4…
150 likes4,312 downloads in the last 30 daysQuestion answeringCC BY 4.0
SWE-rebench-OpenHands-Trajectories is a dataset of multi-turn agent trajectories for software engineering tasks, collected using Qwen/Qwen3-Coder-480B-A35B-Instruct with…
150 likes9,686 downloads in the last 30 daysCC BY 4.0
This pretraining dataset, for generative AI model training, preserves high-value math and code while enriching it with diverse multilingual Q&A, fueling the next generat…
150 likes52,206 downloads in the last 30 daysText generationOther licenseGated
Terminal-Corpus: Large-Scale SFT Dataset for Terminal Agents Terminal-Corpus is a large-scale Supervised Fine-Tuning (SFT) dataset designed to scale the terminal interac…
150 likes21,907 downloads in the last 30 daysQuestion answeringCC BY 4.0
The EarthView Dataset is a comprehensive collection of multispectral earth imagery. The dataset is divided into four distinct subsets sourced from Satellogic, Sentinel-1…
149 likes1,688 downloads in the last 30 daysCC BY 4.0Gated
CL-bench: A Benchmark for Context Learning Dataset Description CL-bench is a benchmark for evaluating language models' context learning abilities. Resolving tasks in CL-…
149 likes1,301 downloads in the last 30 daysText generationOther license
We released PMC-Patients-V2 (in JSON format with the same keys), which is based on 2024 PMC baseline and contains 250,294 patients. The data collection pipeline remains…
149 likes1,648 downloads in the last 30 daysCC BY-NC-SA 4.0
Common Voice is Mozilla's initiative to help teach machines how real people speak. The dataset currently consists of 7,335 validated hours of speech in 60 languages, but…
148 likes1,522 downloads in the last 30 daysSpeech recognitionCC0 1.0
Quick links: [🌐 Website] [📜 Paper] [💻 Code] ParseBench is a benchmark for evaluating document parsing systems on real-world enterprise documents, with the following c…
148 likes28,041 downloads in the last 30 daysApache 2.0
Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, Alham Fikri Aji, Dataset Description We distill the knowledge from large language models by performing sent…
148 likes387 downloads in the last 30 daysCC BY-NC 4.0
This dataset provides the entire corpus of arXiv's LaTeX source files, pre-parsed, formatted, and aligned with official metadata in ready-to-query Parquet files. Why I B…
148 likes15,752 downloads in the last 30 daysText generationDual license
TACO is a benchmark for Python code generation, it includes 25443 problems and 1000 problems for train and test splits.
147 likes27,074 downloads in the last 30 daysText generationApache 2.0
This dataset is the WizardLM dataset victor123/evol_instruct_70k, removing instances of blatant alignment. 54974 instructions remain. inspired by All credit to anon82314…
147 likes205 downloads in the last 30 daysNo license given
注意：此数据集是由ChatGPT产生的，未经过严格校验，内容可能包含错误。使用过程中请注意这一点。 instruction中包含多轮对话的上文内容，以Human:和Assistant:区分，output中包含当前助手角色的回答。 样例 { "instruction"…
146 likes1,071 downloads in the last 30 daysGPL 3.0
The GitHub Code clean dataset in a more filtered version of codeparrot/github-code dataset, it consists of 115M code files from GitHub in 32 programming languages with 6…
146 likes40,166 downloads in the last 30 daysApache 2.0
UPD: due to Hugging Face storage limitations we can't upload the metadata file here, so feel free to download it from Google drive: Dear researchers and engineers, you'r…
146 likes214 downloads in the last 30 daysCC BY-NC 4.0Gated
主に，英語で構築されたLLMモデルなどに対して，チャット(Instruction)応答タスクに関してLoRAなどでチューニングするために使用できます． ※様々な公開言語資源を利用させていただきました．関係各位にはこの場を借りて御礼申し上げます． updates 2023/5/15にAlpaca datasetがNCにライセンス変更された…
146 likes384 downloads in the last 30 daysCC BY-SA 4.0
146 likes229 downloads in the last 30 daysOpenRAIL
本数据集是参考Alpaca方法基于GPT4得到的self-instruct数据，约5万条。 Dataset from It is the chinese dataset from Usage and License Notices The data is intended and licensed for research use on…
146 likes2,737 downloads in the last 30 daysText generationCC BY 4.0
AI Village is an ongoing experiment by AI Digest in which a group of AI agents — built on frontier models from Anthropic, OpenAI, and Google — live together in a long-ru…
145 likes2,181 downloads in the last 30 daysAi village research termsGated
Training Data for Text Embedding Models [!NOTE] This repository contains raw datasets, all of which have also been formatted for easy training in the Embedding Model Dat…
145 likes2,909 downloads in the last 30 daysFeature extraction (embeddings)No license given
The Financial data from Yahoo! *** Key Points to Note *** All financial data is sourced from Yahoo!Ⓡ Finance, Nasdaq!Ⓡ, and the U.S. Department of the Treasury via publi…
144 likes149,751 downloads in the last 30 daysODC-By
The Nemotron-Pre-Training-Dataset-v2.1 extends the previously released Nemotron pretraining datasets with refreshed, higher-quality, and more diverse data across math, c…
144 likes9,433 downloads in the last 30 daysText generationOther licenseGated
✨ (New!) Access Tahoe-100M directly from your coding agents using the Tara plugin The Tara plugin is the new agent-native route for exploring Tahoe datasets including Ta…
144 likes126,553 downloads in the last 30 daysCC0 1.0
IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages Updates [23 December 2025] We now have 11,200 hours of transcribed data! 🎉 O…
143 likes22,284 downloads in the last 30 daysCC BY 4.0Gated
The dataset consists of 4 versions: bigcode/the-stack-v2: the full "The Stack v2" dataset bigcode/the-stack-v2-dedup: based on the bigcode/the-stack-v2 but further near-…
143 likes2,285 downloads in the last 30 daysText generationOther licenseGated
RAGBEnch is a large-scale RAG benchmark dataset of 100k RAG examples. It covers five unique industry-specific domains and various RAG task types. RAGBench examples are s…
143 likes3,630 downloads in the last 30 daysCC BY 4.0
Abstract from the paper introducing this dataset: High-quality conversational datasets are essential for developing AI models that can communicate with users. One way to…
143 likes3,129 downloads in the last 30 daysCC BY 4.0
An Open Robustness Benchmark for Jailbreaking Language Models NeurIPS 2024 Datasets and Benchmarks Track Paper | Leaderboard | Benchmark code What is JailbreakBench? Jai…
143 likes27,924 downloads in the last 30 daysMIT
ChatMed-Dataset is a dataset of 110,113 medical query-response pairs (in Chinese) generated by OpenAI's GPT-3.5 engine. The queries are crawled from several online medic…
143 likes198 downloads in the last 30 daysText generationCC BY 4.0
FinanceBench is a first-of-its-kind test suite for evaluating the performance of LLMs on open book financial question answering (QA). This is an open source sample of 15…
143 likes7,137 downloads in the last 30 daysCC BY-NC 4.0
Samantha has been trained in philosophy, psychology, and personal relationships. She is an Assistant - but unlike other Assistants, she also wants to be your friend and…
143 likes1,098 downloads in the last 30 daysApache 2.0
The Belebele Benchmark for Massively Multilingual NLU Evaluation Belebele is a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants…
142 likes47,040 downloads in the last 30 daysQuestion answeringCC BY-SA 4.0
MegaMath: Pushing the Limits of Open Math Copora Megamath is part of TxT360, curated by LLM360 Team. We introduce MegaMath, an open math pretraining dataset curated from…
142 likes51,038 downloads in the last 30 daysText generationODC-By
This is a dataset designed to enhance the overall truthfulness of LLMs, without sacrificing immersion when roleplaying as a human. For example, in normal AI assistant mo…
142 likes594 downloads in the last 30 daysCC BY 4.0
Everyday conversations for Smol LLMs finetunings This dataset contains 2.2k multi-turn conversations generated by Llama-3.1-70B-Instruct. We ask the LLM to generate a si…
141 likes1,707 downloads in the last 30 daysApache 2.0
Leaderboard: the panel at the top of this page has two views. Use the Task menu in its top-right corner to switch between Lhtb2 Solved (LHTB 2.0, out of 77 tasks) and Lh…
141 likes30,815 downloads in the last 30 daysText generationApache 2.0
The Mathematics Aptitude Test of Heuristics (MATH) dataset consists of problems from mathematics competitions, including the AMC 10, AMC 12, AIME, and more. Each problem…
141 likes13,662 downloads in the last 30 daysMIT
Note the ODC-BY license, indicating that different licenses apply to subsets of the data. This means that some portions of the dataset are non-commercial. We present the…
140 likes1,087 downloads in the last 30 daysQuestion answeringODC-By
Mathematics database. This dataset code generates mathematical question and answer pairs, from a range of question types at roughly school-level difficulty. This is desi…
140 likes3,062 downloads in the last 30 daysNo license given
ChartVerse-SFT-1800K is an extended large-scale chart reasoning dataset with Chain-of-Thought (CoT) annotations, developed as part of the opendatalab/ChartVerse project.…
140 likes3,187 downloads in the last 30 daysVisual question answeringApache 2.0
The data fields are the same among all splits. conversations: a List of string . head -n 1 sharegpt_gpt4.jsonl {"conversations":[ {'from': 'human', 'value': '採用優雅現代中文，用中…
139 likes1,222 downloads in the last 30 daysText classificationCC BY 4.0
(TBD) Automated ArXiv question answering via large language models Github | Homepage | Simple QA - Hugging Face Space Automated Question Answering with ArXiv Papers Late…
139 likes271 downloads in the last 30 daysQuestion answeringCC BY 4.0
A dataset of 1.7 million arXiv articles for applications like trend analysis, paper recommender engines, category prediction, co-citation networks, knowledge graph const…
138 likes890 downloads in the last 30 daysTranslationCC0 1.0
Dataset for summarization of long documents.Adapted from this repo.Note that original data are pre-tokenized so this dataset returns " ".join(text) and add "\n" for para…
138 likes6,778 downloads in the last 30 daysSummarizationNo license given
This dataset contains questions and answers from the Stack Overflow Data Dump for the purpose of preference model training. Importantly, the questions have been filtered…
138 likes10,727 downloads in the last 30 daysQuestion answeringCC BY-SA 4.0
GraphWalks: a multi hop reasoning long context benchmark In Graphwalks, the model is given a graph represented by its edge list and asked to perform an operation. Exampl…
138 likes5,580 downloads in the last 30 daysMIT
Adopted by NVIDIA's Nemotron family of models! 🤗 HuggingFace ｜ Slack | WeChat Overview OpenResearcher is a fully open agentic large language model (30B-A3B) designed fo…
138 likes3,489 downloads in the last 30 daysMIT
Trendyol Cybersecurity Defense Instruction-Tuning Dataset (v2.0) 🚀 TL;DR 53,202 meticulously curated system/user/assistant instruction-tuning examples covering 200+ spe…
138 likes3,534 downloads in the last 30 daysText generationApache 2.0
PyTorch original implementation of Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset
137 likes2,261 downloads in the last 30 daysQuestion answeringCC BY-NC 4.0
Open Yap 1K: 1,000 hours of full-duplex natural conversation, free for commercial use Today we're releasing Open Yap 1K: 1,000 hours of dual-channel English conversation…
137 likes2,537 downloads in the last 30 daysAudio to audioCC BY 4.0
In this paper, we present OMNI-EDIT, which is an omnipotent editor to handle seven different image editing tasks with any aspect ratio seamlessly. Our contribution is in…
137 likes57,729 downloads in the last 30 daysMIT
The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The dataset…
137 likes300,798 downloads in the last 30 daysImage classificationUnknown license
This dataset by AbacusAI was crafted by Eric Hartford This is a synthetic dataset, generated mainly with Mistral-Medium and dolphin-2.7-mixtral-8x7b The purpose of this…
136 likes113 downloads in the last 30 daysApache 2.0
One million executable, interpretable CAD construction sequences synthesized entirely without real-world data. Zero-to-CAD: Agentic Synthesis of Interpretable CAD Progra…
136 likes12,273 downloads in the last 30 daysText to 3DApache 2.0
The Berkeley function calling leaderboard is a live leaderboard to evaluate the ability of different LLMs to call functions (also referred to as tools). We built this da…
136 likes134,515 downloads in the last 30 daysApache 2.0
This is a dataset of Spotify tracks over a range of 125 different genres. Each track has some audio features associated with it. The data is in CSV format which is tabul…
136 likes3,939 downloads in the last 30 daysFeature extraction (embeddings)BSD
MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning Code: Project Page: MegaScience is a large-scale mixture of high-quality open-source d…
136 likes6,165 downloads in the last 30 daysText generationCC BY-NC-SA 4.0
Multi-modal Bi-lingual Instruction Dataset for Vision Language Models
136 likes21,701 downloads in the last 30 daysImage to textOther license
The Nemotron-Pre-Training-Dataset-v2.1 extends the previously released Nemotron pretraining datasets with refreshed, higher-quality, and more diverse data across math, c…
136 likes4,505 downloads in the last 30 daysText generationOther licenseGated
🚀 V2 is out! This dataset has a successor: fable-5-premium-v2 — new users should start there. A rigorously cleaned, high-quality supervised fine-tuning (SFT) dataset bu…
136 likes2,357 downloads in the last 30 daysText generationMIT
SMOL (Set for Maximal Overall Leverage) is a collection professional translations into 221 Low-Resource Languages, for the purpose of training translation models, and ot…
135 likes7,006 downloads in the last 30 daysTranslationCC BY 4.0
Post-OCR correction is a large corpus of 1 billion words containing original texts with a varying number of OCR mistakes and an experimental multilingual post-OCR correc…
135 likes957 downloads in the last 30 daysCC0 1.0
SWE-chat: Coding Agent Interactions From Real Users in the Wild 📄 COLM 2026 Paper: arxiv.org/abs/2604.20779 🌐 Website: swe-chat.com Dataset Summary SWE-chat captures r…
135 likes13,676 downloads in the last 30 daysText generationODC-ByGated
ABC-130k is the largest open-source robot teleoperation dataset. It contains bimanual manipulation trajectories collected on two-arm YAM stations. Episodes are distribut…
135 likes235,875 downloads in the last 30 daysRoboticsApache 2.0Gated
Dataset for O1 Replication Journey: A Strategic Progress Report Usage from datasets import load_dataset dataset = load_dataset("GAIR/o1-journey", split="train") Citation…
134 likes323 downloads in the last 30 daysNo license given
"A Question-Entailment Approach to Question Answering". Asma Ben Abacha and Dina Demner-Fushman. BMC Bioinformatics, 2019.
134 likes7,377 downloads in the last 30 daysQuestion answeringNo license given
134 likes15,760 downloads in the last 30 daysCC BY 4.0
The Open Super-large Crawled Aggregated coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the Ungol…
134 likes1,003 downloads in the last 30 daysFill maskCC0 1.0Gated
This repository contains the RL training datasets used in the paper Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Images
134 likes285 downloads in the last 30 daysApache 2.0
A 900-prompt factuality benchmark from Google DeepMind, designed to evaluate agents on difficult multi-step information-seeking tasks across 17 different fields. ▶ Googl…
133 likes16,252 downloads in the last 30 daysQuestion answeringApache 2.0
Recent advancements in AI, particularly in large language models (LLMs), have led to significant breakthroughs in mathematical reasoning capabilities. However, existing…
133 likes24,504 downloads in the last 30 daysApache 2.0
The Nemotron-Instruction-Following-Chat-v1 dataset is designed to broadly strengthen the model’s interactive capabilities, spanning open-ended chat, precise instruction…
133 likes2,839 downloads in the last 30 daysCC BY 4.0
Mind2Web is a dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website. Existin…
133 likes12,741 downloads in the last 30 daysCC BY 4.0
This dataset contains a diverse set of natural Japanese speech, collected from terrestrial television streams. It contains more than 35000 hours of audio. Paper: ReazonS…
133 likes906 downloads in the last 30 daysSpeech recognitionOther licenseGated
This is a subset of SmolTalk dataset adapted for smol models with less than 1B parameters. We used it to build SmolLM2-360M-Instruct and SmolLM2-135M-Instruct. We do SFT…
132 likes12,980 downloads in the last 30 daysApache 2.0
🚀 Creating the highest-quality pre-training datasets for LLMs 🌟 📄 PAPER 🤗 CLIMBLAB 🤗 CLIMBMIX 🏠 HOMEPAGE Figure 1: Continuously training a 1B model yields a 2.0% i…
132 likes10,153 downloads in the last 30 daysText generationCC BY-NC 4.0
Nemotron-Personas-Japan は、日本における人口の多様性と豊かさを捉えることを目的とし、実世界の人口統計、地理的分布、性格特性の分布に基づいて合成的に生成されたペルソナのオープンソースデータセットです。名前、性別、年齢、背景、婚姻状況、学歴、職業、居住地などの統計に基づいて生成した初のデータセットされた Nemotr…
132 likes2,979 downloads in the last 30 daysText generationCC BY 4.0
👉 Looking for the newest release? The current flagship is ai4privacy/pii-masking-openpii-1.5m. 1.6M samples, 30 languages, 19 PII classes, Asia Pacific extension.?** Th…
131 likes3,947 downloads in the last 30 daysText classificationNo license given
The NQ corpus contains questions from real users, and it requires QA systems to read and comprehend an entire Wikipedia article that may or may not contain the answer to…
131 likes17,928 downloads in the last 30 daysQuestion answeringCC BY-SA 3.0
Atlas Map: Map of Cleaned Data
131 likes108 downloads in the last 30 daysText generationApache 2.0
Please see paper & website for more information: Overview Cambrian-10M is a comprehensive dataset designed for instruction tuning, particularly in multimodal settings in…
131 likes25,234 downloads in the last 30 daysVisual question answeringApache 2.0
UltraInteract is a large-scale, high-quality alignment dataset specifically designed for complex reasoning tasks. For each instruction, it includes a preference tree con…
131 likes1,103 downloads in the last 30 daysMIT
The largest agentic-coding trace dataset to date: 112 B tokens of execution-free agentic trajectories covering 122 K pull requests, 3 K repositories, and 16 programming…
130 likes3,273 downloads in the last 30 daysText generationApache 2.0
本项目由交易猫基金会支持（交易猫基金会 CA：0x8a99b8d53eff6bc331af529af74ad267f3167777）。 精简版币安期货历史数据 - 只包含 4 个主要币种，适合快速下载和研究使用。 📊 数据概览 文件 记录数 压缩大小 时间范围 说明 candles_1m_main4_*.bin.zst 998 万 3…
130 likes490 downloads in the last 30 daysTime-series forecastingMIT
包含从12560本网文提取的约21.7M条可用于训练小说生成的中文指令数据(novel_json_tokens512.zip)。下载链接： 提取码：jlh3 以及从中提取出的包含50k条数据的子集(novel_cn_token512_50k.json)。其中输入和输出都不多于 512 tokens。 样例 在原有小说文本基础上，依据下列…
130 likes606 downloads in the last 30 daysMIT
Movie Review Dataset. This is a dataset of containing 5,331 positive and 5,331 negative processed sentences from Rotten Tomatoes movie reviews. This data was first used…
129 likes60,908 downloads in the last 30 daysText classificationUnknown license
Note: This dataset is part of the lineup CodeX by Modotte. You can get lots of datasets in this same lineup, with the main focus on providing very high-quality datasets…
129 likes765 downloads in the last 30 daysText generationApache 2.0
Paper: Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode SupervisionCode: Nemotron-MathProofs-v1 documentation Dataset Descrip…
129 likes1,083 downloads in the last 30 daysText generationCC BY-SA 4.0
This repository hosts data for Scalable 3D Captioning with Pretrained Models and View Selection for 3D Captioning via Diffusion Ranking, including descriptive captions f…
129 likes24,506 downloads in the last 30 daysText to 3DODC-By
OmniScience: A Large-scale Multi-modal Dataset for Scientific Image Understanding 🚀 2026-05-17: This work was accepted by the KDD 2026 Dataset & Benchmark Track. 🚀 202…
129 likes742 downloads in the last 30 daysImage to textCC BY-NC-SA 4.0Gated
RealWorldQA is a benchmark designed for real-world understanding. The dataset consists of anonymized images taken from vehicles, in addition to other real-world images.…
129 likes5,115 downloads in the last 30 daysCC BY ND 4.0
Qwen3.5-reasoning-700x is a high-quality distilled dataset. This dataset uses the high-quality instructions constructed by Alibaba-Superior-Reasoning-Stage2 as the seed…
128 likes276 downloads in the last 30 daysQuestion answeringApache 2.0
Fine-T2I: An Open, Large-Scale, and Diverse Dataset for High-Quality T2I Fine-Tuning [arxiv] by Xu Ma, Yitian Zhang, Qihua Dong, Yun Fu Northeastern Univeristy Please se…
128 likes22,856 downloads in the last 30 daysImage to textApache 2.0
Paper | Code | Project Page Dataset Description This dataset is specifically designed for training the agentic capabilities of Large Language Models (LLMs). The dataset…
128 likes1,406 downloads in the last 30 daysText generationODC-By
A large crowd-sourced dataset for developing natural language interfaces for relational databases
128 likes1,613 downloads in the last 30 daysUnknown license
Help keep USASI useful.
Optional. No USASI account required. Payment takes place on the linked provider’s website (Buy Me a Coffee).
About supporting this project