2026-11-17 22:22 UTC
I do think The Crawler Zoo’s Trap Room for AI crawlers that don’t obey robots.txt is a work of art :)
e.g., Watch Meta’s crawler wander the endless maze it’s currently trapped in live:
https://crawlerzoo.com/labyrinth/meta-externalagent
Replies (0)
No replies.