---

name: twitter-json-workflow description: "Extract tweet text from stored Twitter/X JSONL and plan safe optional live streaming." disable-model-invocation: true metadata: disco-role: operating license: MIT

twitter-json-workflow

Use this sub-skill for portable tweet-text extraction from stored Twitter/X JSONL and for cautious, opt-in planning around live streaming.

Route here for

• JSONL tweet archives that need text, full_text, or extended_tweet.full_text extraction
• malformed-line tolerant parsing, newline-safe plain-text export, and optional JSONL output
• a dry-run first pass for Tweepy-based live capture with no hardcoded secrets
• keeping source data ready for later NLP, topic, or sentiment work

Route elsewhere

• Sentiment modeling, classifier training, or topic analysis on the extracted text: use the tutorial-resource-map sibling for learning resources or downstream tooling for actual model work
• R/jsonlite workflows: use the bundled Python extractor instead
• Long-running production ingestion or dashboarding: use a dedicated downstream service

Read first

• Workflows
• Data formats
• Troubleshooting
• Offline extractor
• Live-stream template

Quick decision map

Need	Use
Convert JSONL tweets to one clean text line per tweet	scripts/extract_tweet_text.py
Preserve one output row per input tweet with lineage	scripts/extract_tweet_text.py --format jsonl
Prefer a nested field such as extended_tweet.full_text	scripts/extract_tweet_text.py --field ...
Dry-run a planned live stream	scripts/twitter_stream_template.py
Connect to streaming only after credentials and policy are ready	scripts/twitter_stream_template.py --connect

Typical path

1. Start with the field order and output choices in references/data-formats.md.
2. Run the offline extractor on a tiny JSONL sample and confirm malformed lines are reported, not fatal.
3. If live capture is needed, use the stream template in dry-run mode first and only add --connect once credentials and network policy are confirmed.
4. Hand extracted text off to the next analysis stage; do not use this sub-skill for model training itself.

Example commands

python scripts/extract_tweet_text.py tweets.jsonl > tweets.txt
python scripts/extract_tweet_text.py tweets.jsonl --format jsonl > tweets.cleaned.jsonl
python scripts/twitter_stream_template.py --track python data
python scripts/twitter_stream_template.py --track python data --connect