Documents · SKILL.md

web-scraping

web-scraping

Web scraping for RAG ingestion. Covers Firecrawl API, Crawl4AI (open source), trafilatura (main-content extraction), BeautifulSoup + readability-lxml, Scrapy for large crawls, Playwright for JS-rendered pages, sitemap.xml discovery, robots.txt, polite crawling, anti-bot, and deduplication. USE WHEN: user mentions "web scraping", "crawl website", "Firecrawl…

Independent GitHub snapshot · Automated classification · Original documentation governs

Cclaude-dev-suite / claude-dev-suiteMIT

Skill documentation

GitHub

Loading documentation from the source…

MAKE IT PART OF YOUR WORKFLOW

Install skill

TERMINAL

npx skills add claude-dev-suite/claude-dev-suite --skill web-scraping

Run the command in your terminal. Review the code and license before installing.

ZIP · Complete skill directory with resources and licenses

SKILL.md · Instructions file only

Repository stars: Not synced
License: MIT
Version / branch: HEAD · d3511119
Repository updated: Not synced

Review scripts and permissions before installing.

Source code

web-scraping

Skill documentation

More skills to explore

canvas-design

docx

pdf