crawlerbase.de / trace Bot-Datenbank →
www.amazon.de 2 Hops/385 ms/200 HTTP/2
+ Neu Re-Run →

Übersicht

final URL: https://www.amazon.de/
2
Hops
385 ms
Total
60
Bots erlaubt
0
Bots blockiert

Redirect-Chain

2 Hops
  1. 00
    301 HTTP/1.1 https://amazon.de
    TTFB 96 ms Total 96 ms Size 163 B IP 3.253.177.16 Server Server
    → https://www.amazon.de/
    TLS Issuer C = US, O = DigiCert Inc, OU = www.digicert.com, CN = GeoTrust TLS RSA CA G1 Gültig noch 147 Tage
    Alle Header (7)
    serverServer
    dateWed, 24 Jun 2026 03:14:52 GMT
    content-typetext/html
    transfer-encodingchunked
    connectionkeep-alive
    locationhttps://www.amazon.de/
    x-amz-rid0BK22D3DVPYVMFJSM2JE
  2. 01
    200 HTTP/2 https://www.amazon.de/
    TTFB 95 ms Total 289 ms Size 165,4 KB IP 3.168.211.192 Server Server
    TLS Issuer C = US, O = Amazon, CN = Amazon RSA 2048 M01 Gültig noch 187 Tage
    Alle Header (24)
    content-typetext/html;charset=UTF-8
    dateWed, 24 Jun 2026 03:14:52 GMT
    varyAccept-Encoding,User-Agent
    x-frame-optionsSAMEORIGIN
    serverServer
    x-amz-ridHQK77V7E96950GVR3WVN
    content-languagede-DE
    x-content-type-optionsnosniff
    accept-chect,rtt,downlink,device-memory,sec-ch-device-memory,viewport-width,sec-ch-viewport-width,dpr,sec-ch-dpr,sec-ch-ua-full-version-list,sec-ch-ua-platform,sec-ch-ua-platform-version
    x-xss-protection1;
    cache-controlno-cache
    x-ua-compatibleIE=edge
    content-encodinggzip
    expires-1
    accept-ch-lifetime86400
    content-security-policyupgrade-insecure-requests;report-uri https://metrics.media-amazon.com/
    pragmano-cache
    content-security-policy-report-onlydefault-src 'self' blob: https: data: mediastream: 'unsafe-eval' 'unsafe-inline';report-uri https://metrics.media-amazon.com/
    strict-transport-securitymax-age=47474747; includeSubDomains; preload
    x-cacheMiss from cloudfront
    via1.1 efd15bffbe1bb9bdb658c776a58450d2.cloudfront.net (CloudFront)
    x-amz-cf-popTXL52-P1
    alt-svch3=":443"; ma=86400
    x-amz-cf-idMy6hkGo-m0F2n_5vQn5LHfe3EnBzMfL2D5CM9k70LtVS5R9ORJEp6A==

robots.txt / amazon.de

vorhanden 102 Gruppen 0 Sitemaps
robots.txt Quelltext (7,3 KB)
amazon.de/robots.txt
User-agent: * Disallow: /dp/product-availability/ Disallow: /dp/rate-this-item/ Disallow: /exec/obidos/account-access-login Disallow: /exec/obidos/change-style Disallow: /exec/obidos/di Disallow: /exec/obidos/dt Disallow: /exec/obidos/dt/assoc/handle-buy-box Disallow: /exec/obidos/flex-sign-in Disallow: /exec/obidos/handle-buy-box Disallow: /exec/obidos/refer-a-friend-login Disallow: /exec/obidos/subst/associates/join Disallow: /exec/obidos/subst/marketplace/sell-your-collection.html Disallow: /exec/obidos/subst/marketplace/sell-your-stuff.html Disallow: /exec/obidos/subst/partners/friends/access.html Disallow: /exec/obidos/tg/cm/member/ Disallow: /gp/cart Disallow: /gp/content-form Disallow: /gp/customer-images Disallow: /gp/customer-media/upload Disallow: /gp/customer-reviews/common/du Disallow: /gp/customer-reviews/write-a-review.html Disallow: /gp/flex Disallow: /gp/gfix Disallow: /gp/history Disallow: /gp/item-dispatch Disallow: /gp/legacy-handle-buy-box.html Disallow: /gp/reader Disallow: /gp/registry/wishlist/*/reserve Disallow: /gp/richpub/listmania/createpipeline Disallow: /gp/music/clipserve Disallow: /gp/music/wma-pop-up Disallow: /gp/recsradio Disallow: /gp/sign-in Disallow: /gp/slides/make-money Disallow: /gp/structured-ratings/actions/get-experience.html Disallow: /gp/twitter/ Disallow: /gp/vote Disallow: /gp/voting/ Disallow: /gp/yourstore Disallow: /ap/signin Disallow: /gp/registry/search.html Disallow: /gp/orc/rml/ Disallow: /gp/dmusic/mp3/player Disallow: /gp/entity-alert/external Disallow: /gp/customer-reviews/dynamic/sims-box Disallow: /review/dynamic/sims-box Disallow: /gp/redirect.html Disallow: /gp/customer-media/actions/delete/ Disallow: /gp/customer-media/actions/edit-caption/ Disallow: /gp/dmusic/ Allow: /gp/dmusic/promotions/PrimeMusic Allow: /gp/dmusic/promotions/AmazonMusicUnlimited Disallow: /gp/customer-media/product-gallery/B007HCCOD0 Disallow: /gp/help/customer/display.html?*nodeId=200534000 Disallow: /gp/help/customer/display.html?*nodeId=200145650 Disallow: /gp/feature.html?*docId=1000632623 Disallow: /gp/aag Disallow: /gp/socialmedia/giveaways Disallow: /gp/aw/so.html Disallow: /gp/aw/sp.html?*t=returns Disallow: /gp/pdp/profile/ Disallow: /gp/product/product-availability Disallow: /gp/offer-listing Disallow: /dp/twister-update/ Disallow: /dp/e-mail-friend/ Disallow: /gp/registry/wishlist/ Disallow: /gp/switch-language Disallow: /wishlist/ Allow: /wishlist/universal Allow: /wishlist/vendor-button Allow: /wishlist/get-button Disallow: /gp/wishlist/ Allow: /gp/wishlist/universal Allow: /gp/wishlist/vendor-button Allow: /gp/wishlist/ipad-install Disallow: /registry/wishlist/ Disallow: /local/ajax/ Disallow: /gp/rate-it Disallow: /o/di Disallow: /o/dt Disallow: /gp/help/customer/display.html?nodeId=200393390 Disallow: /forum/kindle Disallow: /gp/profile Disallow: /reviews/iframe Disallow: /gp/ask-widget/askWidget* Disallow: /ss/customer-reviews/lighthouse/ Disallow: /gp/aw/ol/ Disallow: /gp/promotion/ Disallow: /hz/leaderboard/top-reviewers/ Disallow: /hz/leaderboard/hall-of-fame/ Disallow: /review/top-reviewers/ Disallow: /review/top-reviewers Disallow: /review/hall-of-fame Disallow: /reviews/top-reviewers/ Disallow: /reviews/top-reviewers Disallow: /reviews/hall-of-fame Disallow: /hz/help/contact/*/message/$ Disallow: /-/ Allow: /-/en/ Allow: /-/da/ Allow: /-/en$ Disallow: /gp/aw/shoppingAids/ Disallow: /hp/video/ Disallow: /gp/video/api Disallow: /gp/video/settings Disallow: /gp/video/library Disallow: /gp/video/watchlist Disallow: /gp/video/mystuff Disallow: /gp/video/profiles Disallow: /gp/video/search Disallow: /gp/video/auth Disallow: /gp/video/salp/i User-agent: EtaoSpider Disallow: / User-agent: GPTBot Disallow: / User-agent: CCBot Disallow: / User-Agent: PerplexityBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: GoogleAgent-Mariner Disallow: / User-agent: GoogleAgent-Shopping Disallow: / User-agent: ClaudeBot Disallow: / User-agent: meta-externalagent Disallow: / User-agent: Bytespider Disallow: / User-agent: Scrapy Disallow: / User-agent: PetalBot Disallow: / User-agent: Devin Disallow: / User-agent: omgili Disallow: / User-agent: AI2Bot Disallow: / User-agent: Gemini-Deep-Research Disallow: / User-agent: PanguBot Disallow: / User-agent: MistralAI-User Disallow: / User-agent: Diffbot Disallow: / User-agent: Sidetrade indexer bot Disallow: / User-agent: Google-NotebookLM Disallow: / User-agent: DuckAssistBot Disallow: / User-agent: Copilot Disallow: / User-agent: Ai2Bot-Dolma Disallow: / User-agent: Google-CloudVertexBot Disallow: / User-agent: meta-externalfetcher Disallow: / User-agent: iaskspider Disallow: / User-agent: wpbot Disallow: / User-agent: aiHitBot Disallow: / User-agent: cohere-ai Disallow: / User-agent: FriendlyCrawler Disallow: / User-agent: img2dataset Disallow: / User-agent: VelenPublicWebCrawler Disallow: / User-agent: YouBot Disallow: / User-agent: Brightbot Disallow: / User-agent: Claude-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-SearchBot Disallow: / User-agent: Perplexity-User Disallow: / User-agent: ISSCyberRiskCrawler Disallow: / User-agent: * Disallow: /aaut/* User-agent: CopilotNative Disallow: / User-agent: CopilotSapphire Disallow: / User-agent: imgproxy Disallow: / User-agent: SemrushBot-SWA Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: Timpibot Disallow: / User-agent: Thinkbot Disallow: / User-agent: AddSearchBot Disallow: / User-agent: AI2Bot-DeepResearchEval Disallow: / User-agent: Andibot Disallow: / User-agent: Anomura Disallow: / User-agent: atlassian-bot Disallow: / User-agent: BuddyBot Disallow: / User-agent: Channel3Bot Disallow: / User-agent: ChatGLM-Spider Disallow: / User-agent: Claude-Web Disallow: / User-agent: Cloudflare-AutoRAG Disallow: / User-agent: CloudflareBrowserRenderingCrawler Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: Cotoyogi Disallow: / User-agent: Crawl4AI Disallow: / User-agent: Datenbank Crawler Disallow: / User-agent: DeepSeekBot Disallow: / User-agent: Echobot Bot Disallow: / User-agent: EchoboxBot Disallow: / User-agent: factset_spyderbot Disallow: / User-agent: Google-Firebase Disallow: / User-agent: GoogleOther Disallow: / User-agent: Grok-DeepSearch Disallow: / User-agent: GrokBot Disallow: / User-agent: iAskBot Disallow: / User-agent: IbouBot Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: imageSpider Disallow: / User-agent: Kangaroo Bot Disallow: / User-agent: KlaviyoAIBot Disallow: / User-agent: KunatoCrawler Disallow: / User-agent: laion-huggingface-processor Disallow: / User-agent: Linguee Bot Disallow: / User-agent: LinkupBot Disallow: / User-agent: Manus-User Disallow: / User-agent: meta-webindexer Disallow: / User-agent: netEstate Imprint Crawler Disallow: / User-agent: Panscient Disallow: / User-agent: panscient.com Disallow: / User-agent: PhindBot Disallow: / User-agent: Poggio-Citations Disallow: / User-agent: QualifiedBot Disallow: / User-agent: SBIntuitionsBot Disallow: / User-agent: ShapBot Disallow: / User-agent: TavilyBot Disallow: / User-agent: TerraCotta Disallow: / User-agent: TwinAgent Disallow: / User-agent: webzio-extended Disallow: / User-agent: WRTNBot Disallow: / User-agent: xAI-Grok Disallow: / User-agent: YaK Disallow: / User-agent: ZanistaBot Disallow: /

robots.txt / www.amazon.de

vorhanden 102 Gruppen 0 Sitemaps
robots.txt Quelltext (7,3 KB)
www.amazon.de/robots.txt
User-agent: * Disallow: /dp/product-availability/ Disallow: /dp/rate-this-item/ Disallow: /exec/obidos/account-access-login Disallow: /exec/obidos/change-style Disallow: /exec/obidos/di Disallow: /exec/obidos/dt Disallow: /exec/obidos/dt/assoc/handle-buy-box Disallow: /exec/obidos/flex-sign-in Disallow: /exec/obidos/handle-buy-box Disallow: /exec/obidos/refer-a-friend-login Disallow: /exec/obidos/subst/associates/join Disallow: /exec/obidos/subst/marketplace/sell-your-collection.html Disallow: /exec/obidos/subst/marketplace/sell-your-stuff.html Disallow: /exec/obidos/subst/partners/friends/access.html Disallow: /exec/obidos/tg/cm/member/ Disallow: /gp/cart Disallow: /gp/content-form Disallow: /gp/customer-images Disallow: /gp/customer-media/upload Disallow: /gp/customer-reviews/common/du Disallow: /gp/customer-reviews/write-a-review.html Disallow: /gp/flex Disallow: /gp/gfix Disallow: /gp/history Disallow: /gp/item-dispatch Disallow: /gp/legacy-handle-buy-box.html Disallow: /gp/reader Disallow: /gp/registry/wishlist/*/reserve Disallow: /gp/richpub/listmania/createpipeline Disallow: /gp/music/clipserve Disallow: /gp/music/wma-pop-up Disallow: /gp/recsradio Disallow: /gp/sign-in Disallow: /gp/slides/make-money Disallow: /gp/structured-ratings/actions/get-experience.html Disallow: /gp/twitter/ Disallow: /gp/vote Disallow: /gp/voting/ Disallow: /gp/yourstore Disallow: /ap/signin Disallow: /gp/registry/search.html Disallow: /gp/orc/rml/ Disallow: /gp/dmusic/mp3/player Disallow: /gp/entity-alert/external Disallow: /gp/customer-reviews/dynamic/sims-box Disallow: /review/dynamic/sims-box Disallow: /gp/redirect.html Disallow: /gp/customer-media/actions/delete/ Disallow: /gp/customer-media/actions/edit-caption/ Disallow: /gp/dmusic/ Allow: /gp/dmusic/promotions/PrimeMusic Allow: /gp/dmusic/promotions/AmazonMusicUnlimited Disallow: /gp/customer-media/product-gallery/B007HCCOD0 Disallow: /gp/help/customer/display.html?*nodeId=200534000 Disallow: /gp/help/customer/display.html?*nodeId=200145650 Disallow: /gp/feature.html?*docId=1000632623 Disallow: /gp/aag Disallow: /gp/socialmedia/giveaways Disallow: /gp/aw/so.html Disallow: /gp/aw/sp.html?*t=returns Disallow: /gp/pdp/profile/ Disallow: /gp/product/product-availability Disallow: /gp/offer-listing Disallow: /dp/twister-update/ Disallow: /dp/e-mail-friend/ Disallow: /gp/registry/wishlist/ Disallow: /gp/switch-language Disallow: /wishlist/ Allow: /wishlist/universal Allow: /wishlist/vendor-button Allow: /wishlist/get-button Disallow: /gp/wishlist/ Allow: /gp/wishlist/universal Allow: /gp/wishlist/vendor-button Allow: /gp/wishlist/ipad-install Disallow: /registry/wishlist/ Disallow: /local/ajax/ Disallow: /gp/rate-it Disallow: /o/di Disallow: /o/dt Disallow: /gp/help/customer/display.html?nodeId=200393390 Disallow: /forum/kindle Disallow: /gp/profile Disallow: /reviews/iframe Disallow: /gp/ask-widget/askWidget* Disallow: /ss/customer-reviews/lighthouse/ Disallow: /gp/aw/ol/ Disallow: /gp/promotion/ Disallow: /hz/leaderboard/top-reviewers/ Disallow: /hz/leaderboard/hall-of-fame/ Disallow: /review/top-reviewers/ Disallow: /review/top-reviewers Disallow: /review/hall-of-fame Disallow: /reviews/top-reviewers/ Disallow: /reviews/top-reviewers Disallow: /reviews/hall-of-fame Disallow: /hz/help/contact/*/message/$ Disallow: /-/ Allow: /-/en/ Allow: /-/da/ Allow: /-/en$ Disallow: /gp/aw/shoppingAids/ Disallow: /hp/video/ Disallow: /gp/video/api Disallow: /gp/video/settings Disallow: /gp/video/library Disallow: /gp/video/watchlist Disallow: /gp/video/mystuff Disallow: /gp/video/profiles Disallow: /gp/video/search Disallow: /gp/video/auth Disallow: /gp/video/salp/i User-agent: EtaoSpider Disallow: / User-agent: GPTBot Disallow: / User-agent: CCBot Disallow: / User-Agent: PerplexityBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: GoogleAgent-Mariner Disallow: / User-agent: GoogleAgent-Shopping Disallow: / User-agent: ClaudeBot Disallow: / User-agent: meta-externalagent Disallow: / User-agent: Bytespider Disallow: / User-agent: Scrapy Disallow: / User-agent: PetalBot Disallow: / User-agent: Devin Disallow: / User-agent: omgili Disallow: / User-agent: AI2Bot Disallow: / User-agent: Gemini-Deep-Research Disallow: / User-agent: PanguBot Disallow: / User-agent: MistralAI-User Disallow: / User-agent: Diffbot Disallow: / User-agent: Sidetrade indexer bot Disallow: / User-agent: Google-NotebookLM Disallow: / User-agent: DuckAssistBot Disallow: / User-agent: Copilot Disallow: / User-agent: Ai2Bot-Dolma Disallow: / User-agent: Google-CloudVertexBot Disallow: / User-agent: meta-externalfetcher Disallow: / User-agent: iaskspider Disallow: / User-agent: wpbot Disallow: / User-agent: aiHitBot Disallow: / User-agent: cohere-ai Disallow: / User-agent: FriendlyCrawler Disallow: / User-agent: img2dataset Disallow: / User-agent: VelenPublicWebCrawler Disallow: / User-agent: YouBot Disallow: / User-agent: Brightbot Disallow: / User-agent: Claude-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-SearchBot Disallow: / User-agent: Perplexity-User Disallow: / User-agent: ISSCyberRiskCrawler Disallow: / User-agent: * Disallow: /aaut/* User-agent: CopilotNative Disallow: / User-agent: CopilotSapphire Disallow: / User-agent: imgproxy Disallow: / User-agent: SemrushBot-SWA Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: Timpibot Disallow: / User-agent: Thinkbot Disallow: / User-agent: AddSearchBot Disallow: / User-agent: AI2Bot-DeepResearchEval Disallow: / User-agent: Andibot Disallow: / User-agent: Anomura Disallow: / User-agent: atlassian-bot Disallow: / User-agent: BuddyBot Disallow: / User-agent: Channel3Bot Disallow: / User-agent: ChatGLM-Spider Disallow: / User-agent: Claude-Web Disallow: / User-agent: Cloudflare-AutoRAG Disallow: / User-agent: CloudflareBrowserRenderingCrawler Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: Cotoyogi Disallow: / User-agent: Crawl4AI Disallow: / User-agent: Datenbank Crawler Disallow: / User-agent: DeepSeekBot Disallow: / User-agent: Echobot Bot Disallow: / User-agent: EchoboxBot Disallow: / User-agent: factset_spyderbot Disallow: / User-agent: Google-Firebase Disallow: / User-agent: GoogleOther Disallow: / User-agent: Grok-DeepSearch Disallow: / User-agent: GrokBot Disallow: / User-agent: iAskBot Disallow: / User-agent: IbouBot Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: imageSpider Disallow: / User-agent: Kangaroo Bot Disallow: / User-agent: KlaviyoAIBot Disallow: / User-agent: KunatoCrawler Disallow: / User-agent: laion-huggingface-processor Disallow: / User-agent: Linguee Bot Disallow: / User-agent: LinkupBot Disallow: / User-agent: Manus-User Disallow: / User-agent: meta-webindexer Disallow: / User-agent: netEstate Imprint Crawler Disallow: / User-agent: Panscient Disallow: / User-agent: panscient.com Disallow: / User-agent: PhindBot Disallow: / User-agent: Poggio-Citations Disallow: / User-agent: QualifiedBot Disallow: / User-agent: SBIntuitionsBot Disallow: / User-agent: ShapBot Disallow: / User-agent: TavilyBot Disallow: / User-agent: TerraCotta Disallow: / User-agent: TwinAgent Disallow: / User-agent: webzio-extended Disallow: / User-agent: WRTNBot Disallow: / User-agent: xAI-Grok Disallow: / User-agent: YaK Disallow: / User-agent: ZanistaBot Disallow: /

Bot-Matrix

Pfad / 60 erlaubt 0 blockiert
Suchmaschinen (13)
BotUA-TokenmatchedRegelVerdict
Googlebot
Google
 Googlebot * implizit erlaubt
Googlebot-Image
Google
 Googlebot-Image * implizit erlaubt
Googlebot-News
Google
 Googlebot-News * implizit erlaubt
AdsBot-Google
Google
 AdsBot-Google * implizit erlaubt
Mediapartners-Google
Google
 Mediapartners-Google * implizit erlaubt
Bingbot
Microsoft
 bingbot * implizit erlaubt
DuckDuckBot
DuckDuckGo
 DuckDuckBot * implizit erlaubt
YandexBot
Yandex
 YandexBot * implizit erlaubt
Baiduspider
Baidu
 Baiduspider * implizit erlaubt
Applebot
Apple
 Applebot * implizit erlaubt
Sogou web spider
Sogou
 Sogou web spider * implizit erlaubt
Yeti (Naver)
Naver
 Yeti * implizit erlaubt
SeznamBot
Seznam.cz
 SeznamBot * implizit erlaubt
AI / LLM (17)
BotUA-TokenmatchedRegelVerdict
Google-Extended
Google
 Google-Extended * implizit erlaubt
Applebot-Extended
Apple
 Applebot-Extended * implizit erlaubt
GPTBot
OpenAI
 GPTBot * implizit erlaubt
ChatGPT-User
OpenAI
 ChatGPT-User * implizit erlaubt
OAI-SearchBot
OpenAI
 OAI-SearchBot * implizit erlaubt
ClaudeBot
Anthropic
 ClaudeBot * implizit erlaubt
Claude-User
Anthropic
 Claude-User * implizit erlaubt
Claude-SearchBot
Anthropic
 Claude-SearchBot * implizit erlaubt
PerplexityBot
Perplexity
 PerplexityBot * implizit erlaubt
Perplexity-User
Perplexity
 Perplexity-User * implizit erlaubt
CCBot
Common Crawl
 CCBot * implizit erlaubt
Bytespider
ByteDance
 Bytespider * implizit erlaubt
meta-externalagent
Meta
 meta-externalagent * implizit erlaubt
Amazonbot
Amazon
 Amazonbot * implizit erlaubt
cohere-ai
Cohere
 cohere-ai * implizit erlaubt
Diffbot
Diffbot
 Diffbot * implizit erlaubt
MistralAI-User
Mistral
 MistralAI-User * implizit erlaubt
SEO-Tools (10)
BotUA-TokenmatchedRegelVerdict
AhrefsBot
Ahrefs
 AhrefsBot * implizit erlaubt
AhrefsSiteAudit
Ahrefs
 AhrefsSiteAudit * implizit erlaubt
SemrushBot
Semrush
 SemrushBot * implizit erlaubt
MJ12bot (Majestic)
Majestic
 MJ12bot * implizit erlaubt
DotBot (Moz)
Moz
 DotBot * implizit erlaubt
rogerbot (Moz)
Moz
 rogerbot * implizit erlaubt
BLEXBot (WebMeUp)
WebMeUp
 BLEXBot * implizit erlaubt
DataForSeoBot
DataForSEO
 DataForSeoBot * implizit erlaubt
Screaming Frog SEO Spider
Screaming Frog
 Screaming Frog SEO Spider * implizit erlaubt
Sitebulb
Sitebulb
 Sitebulb * implizit erlaubt
Social / Messenger (11)
BotUA-TokenmatchedRegelVerdict
facebookexternalhit
Meta
 facebookexternalhit * implizit erlaubt
meta-externalfetcher
Meta
 meta-externalfetcher * implizit erlaubt
Twitterbot
X (Twitter)
 Twitterbot * implizit erlaubt
LinkedInBot
LinkedIn
 LinkedInBot * implizit erlaubt
Slackbot-LinkExpanding
Slack
 Slackbot-LinkExpanding * implizit erlaubt
Discordbot
Discord
 Discordbot * implizit erlaubt
TelegramBot
Telegram
 TelegramBot * implizit erlaubt
WhatsApp
Meta
 WhatsApp * implizit erlaubt
Pinterestbot
Pinterest
 Pinterestbot * implizit erlaubt
Reddit Bot
Reddit
 Reddit * implizit erlaubt
TikTokSpider
ByteDance
 TikTokSpider * implizit erlaubt
Archive (1)
BotUA-TokenmatchedRegelVerdict
archive.org_bot
Internet Archive
 archive.org_bot * implizit erlaubt
Monitoring (3)
BotUA-TokenmatchedRegelVerdict
UptimeRobot
UptimeRobot
 UptimeRobot * implizit erlaubt
Pingdom
SolarWinds
 Pingdom * implizit erlaubt
StatusCake
StatusCake
 StatusCake * implizit erlaubt
Security-Scanner (5)
BotUA-TokenmatchedRegelVerdict
CensysInspect
Censys
 CensysInspect * implizit erlaubt
Shodan
Shodan
 Shodan * implizit erlaubt
Palo Alto Expanse
Palo Alto
 expanse * implizit erlaubt
l9scan / leakix
LeakIX
 l9scan * implizit erlaubt
NetcraftSurveyAgent
Netcraft
 NetcraftSurveyAgent * implizit erlaubt

Mehr Details, Verifikations-Hinweise und ein robots.txt-Generator in der Bot-Datenbank.