X (Twitter) Data Scraper
Extract and analyze X/Twitter data programmatically.
When to Use
- Fetch tweets from a specific user
- Search X for keywords/topics
- Analyze Twitter data and sentiment
- Monitor social media trends
Quick Start
# Get user tweets
python3 scripts/get_user_tweets.py elonmusk 20
# Search for topic
python3 scripts/search_tweets.py "machine learning" 30
Setup
Credentials: Create ~/.openclaw/credentials/x_api_tokens.env:
X_BEARER_TOKEN=Bearer YOUR_TOKEN_HERE
Get token: https://developer.twitter.com/en/portal/dashboard
Scripts
| Script | Purpose |
|---|---|
get_user_tweets.py | Fetch user timeline |
search_tweets.py | Search by keyword |
fetch_x_playwright.py | Browser-based scraping |
x_api_client.py | API client module |
Advanced Search
# Exclude replies/retweets
python3 scripts/get_user_tweets.py elonmusk 20 --no-replies --no-retweets
# Complex search
python3 scripts/search_tweets.py "(AI OR ML) from:elonmusk lang:en" 20
Troubleshooting
- 401 Unauthorized: Check Bearer token format
- 403 Forbidden: Search API needs Elevated access
- 429 Rate Limited: Wait 15 minutes
License
MIT