Design a Web Crawler
Design a Web Crawler — A Staff+ playbook focused on URL frontier governance, politeness as infrastructure constraint, dedup at scale, and crawl freshness vs coverage tradeoffs — not just Scrapy tutorials.
This playbook is part of the full calibration library.
Staff interviews are decided on nuance — tradeoff framing, ownership boundaries, and failure anticipation. This topic includes evaluator-grade breakdowns and drill sections available with full calibration access.
What's inside this playbook
- •Staff-level tradeoff analysis and failure mode reasoning
- •L5 vs L6 calibration tables for this specific topic
- •Practice drills with Staff-level evaluation criteria
- •Deep-dive appendices and operational ownership patterns