Ecosystem overview for everything related to crawling.
Crawlee is a web scraping and browser automation library developed by Apify, designed to help developers build reliable scrapers quickly. It mimics human browsing behavior to effectively bypass modern bot protections. The library provides a unified interface for both HTTP and headless browser (e.g., Playwright, Puppeteer) crawling. Key features include persistent request queues, pluggable data storage, automatic proxy rotation, and session management, with CLI support for rapid project bootstrapping and easy deployment to cloud platforms.