A free, high-scale, async Medium scraper with request abstraction and HTML-to-Markdown parser. Quickly discover and convert Medium articles to clean Markdown with our intuitive web interface.
Collect real-time data from any website with Decodo’s Web Scraping API and award-winning proxies.
– Free trials available
– 125M+ IPs in 195+ locations
– 100+ ready-made scraping templates
– Extensive documentation
– 24/7 tech support
ThorData Web Scraper provides unblockable proxy infrastructure and scraping solutions for reliable, real-time web data extraction at scale. Perfect for AI training data collection, web automation, and large-scale scraping operations that require high performance and stability.
Key Advantages of ThorData:
- Massive proxy network: Access to 60M+ ethically sourced residential, mobile, ISP, and datacenter IPs across 190+ countries.
- Enterprise-grade reliability: 99.9% uptime with ultra-low latency (<0.5s response time) for uninterrupted data collection.
- Flexible proxy types: Choose from residential, mobile (4G/5G), static ISP, or datacenter proxies based on your needs.
- Cost-effective pricing: Starting from $1.80/GB for residential proxies with no traffic expiration and pay-as-you-go model.
- Advanced targeting: City-level geolocation targeting with automatic IP rotation and unlimited bandwidth options.
- Ready-to-use APIs: 120+ scraper APIs and comprehensive datasets purpose-built for AI and data science workflows.
ThorData is SOC2, GDPR, and CCPA compliant, trusted by 4,000+ enterprises for secure web data extraction.
👉 Learn more: ThorData Web Scraper | Get Started
The easiest way to use Medium Scraper is through our web interface:
Pull our pre-built image from GitHub Container Registry:
docker run -p 8000:8000 ghcr.io/sarperavci/medium-scraper:latestThen open your browser to http://localhost:8000
# Install web dependencies
pip install medium-scraper[web]
# Run web server
cd web && python app.pyDecodo provides a powerful API for scraping Medium articles. Our web interface supports this API out of the box.
To use the Decodo API, you need to get an API key from Decodo. Sign up and get your API key from Decodo.
Once you have your API key, you can set it in the web interface by clicking the "Advanced Settings", set the Sender to decodo-webscraping-api and paste your API key in the Decodo API Key field.
You can set up custom proxies in the web interface by clicking the "Advanced Settings", set the Sender to requests and paste your proxy list in the Proxies field.
You can set up proxyless in the web interface by clicking the "Advanced Settings", set the Sender to requests and leave the Proxies field empty.
- Intuitive GUI for scraping Medium articles
- Real-time progress tracking via WebSocket
- Download results as ZIP files
- Job history and persistent storage
- Multiple request modes:
- Decodo API: Smart managed scraping (requires Decodo API key)
- Custom Proxies: Bring your own proxy list
- Proxyless: Direct requests with your IP
These features are also available when using the library programmatically. See our Library Documentation for details.
For programmatic usage of the core library, please refer to our Library Documentation which provides detailed examples.
The command-line interface offers powerful scraping capabilities. See our CLI Documentation for comprehensive usage instructions.
pip install medium-scraperThe library supports multiple request backends:
- RequestsRequestSender: Standard requests library (works with custom proxies or proxyless)
- DecodoScraperRequestSender: Advanced scraping with Decodo API (requires API key)
- CachedRequestSender: Adds caching to any sender
Choose the appropriate sender based on your needs:
from medium_scraper import RequestsRequestSender, DecodoScraperRequestSender
# For simple use cases (proxyless or with custom proxies)
sender = RequestsRequestSender()
# For advanced scraping with Decodo (requires API key from https://decodo.com)
sender = DecodoScraperRequestSender(api_key="your-decodo-api-key")The library is designed to be fully async:
import asyncio
from medium_scraper import MediumExplorer
async def main():
explorer = MediumExplorer()
articles = await explorer.get_tag_articles("python", limit=5)
for article in articles:
print(f"Title: {article.title}")
print(f"URL: {article.url}")
asyncio.run(main())For more detailed information about each component, please see the documentation in the docs folder: