主 题：我的小站常见爬虫有这些
发 布 者：cgkweb
标签分类： 日常
时 间：2026-09-25 19:07:28
内容预览：爬虫名称User-AgentBaiduspiderMozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)PerplexityBotMozilla/5.0 (compatible; PerplexityBot/1.0; +https://docs.perplexity.ai/bot)Claude-SearchBotMozilla/5.0 (compatible; Claude-SearchBot/1.0; +https://support.anthropic.com/en/articles/8854369-claude-search-bot)BravebotMozilla/5.0 (compatible; Bravebot/1.0; +https://search.brave.com/help/brave-search-crawler)Kimi-SearchBotMozilla/5.0 (compatible; Kimi-SearchBot/1.0; +https://kimi.moonshot.cn)DuckAssistBotMozilla/5.0 (compatible; DuckAssistBot/1.0; +https://duckduckgo.com/duckassistbot)Meta-ExternalAgentMeta-ExternalAgent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler/)QwenbotMozilla/5.0 (compatible; Qwenbot/1.0; +https://tongyi.aliyun.com)GPTBotGPTBot
直达链接： https://www.nodeseek.com/post-948543-1
发 布 者：cgkweb
标签分类： 日常
时 间：2026-09-25 19:07:28
内容预览：爬虫名称User-AgentBaiduspiderMozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)PerplexityBotMozilla/5.0 (compatible; PerplexityBot/1.0; +https://docs.perplexity.ai/bot)Claude-SearchBotMozilla/5.0 (compatible; Claude-SearchBot/1.0; +https://support.anthropic.com/en/articles/8854369-claude-search-bot)BravebotMozilla/5.0 (compatible; Bravebot/1.0; +https://search.brave.com/help/brave-search-crawler)Kimi-SearchBotMozilla/5.0 (compatible; Kimi-SearchBot/1.0; +https://kimi.moonshot.cn)DuckAssistBotMozilla/5.0 (compatible; DuckAssistBot/1.0; +https://duckduckgo.com/duckassistbot)Meta-ExternalAgentMeta-ExternalAgent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler/)QwenbotMozilla/5.0 (compatible; Qwenbot/1.0; +https://tongyi.aliyun.com)GPTBotGPTBot
直达链接： https://www.nodeseek.com/post-948543-1
