User-Agent: * Disallow: User-agent: bingbot User-agent: Yandex User-agent: DuckDuckBot User-agent: ClaudeBot User-agent: AhrefsBot User-agent: SemrushBot User-agent: PetalBot User-agent: criteobot User-agent: grapeshot User-agent: ImagesiftBot Disallow: Crawl-delay: 1