crawlee-python
by apify (organization)
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy… (description from the project’s GitHub page)
Repository facts
- Stars
- 9,597
- Forks
- 812
- Open issues and pull requests
- 101
- Primary language
- Python
- License
- Apache License 2.0 (Apache-2.0)
- Owner
- apify · Organization
- Created
- Jan 10, 2024
- Last push
- Oct 16, 2026
Topics
Add this badge to your README
Shows this repository’s rank in AI on GitHub and links back to this page. Ranks are by stars and change when the list is refreshed. The badge is not an endorsement or a rating. About the badges
Markdown (for a README)
[](https://unitedstatesofamericasuperintelligence.com/github/apify--crawlee-python/)
HTML
<a href="https://unitedstatesofamericasuperintelligence.com/github/apify--crawlee-python/"><img src="https://unitedstatesofamericasuperintelligence.com/badges/github/apify--crawlee-python.svg" alt="Ranked #854 in AI on GitHub" height="20"></a>
Related projects
Projects that share this one’s GitHub topics.
Rank 313: apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML,…
26,183 starsTypeScriptApache-2.0Last push Oct 16, 2026
Rank 52: unclecode/crawl4ai
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.
85,765 starsPythonApache-2.0Last push Oct 15, 2026
Rank 50: D4Vinci/Scrapling
🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily…
88,435 starsPythonBSD-3-ClauseLast push Oct 18, 2026
Rank 459: getmaxun/maxun
Turn any website into a structured API. Extract, automate, search and monitor the web.
17,888 starsTypeScriptAGPL-3.0Last push Oct 18, 2026
Rank 1,010: alirezamika/autoscraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
8,054 starsPythonMITLast push Jul 29, 2026
Rank 10: firecrawl/firecrawl
Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
192,681 starsTypeScriptAGPL-3.0Last push Oct 18, 2026