{
"ok": true,
"summary": {
"domain": "blog.cloudflare.com",
"origin": "https://blog.cloudflare.com",
"homepageStatus": 200,
"robots": {
"url": "https://blog.cloudflare.com/robots.txt",
"status": 200,
"present": true,
"userAgent": "SteadylineBot",
"userAgentGroupsCount": 1,
"matchedGroup": "*",
"disallow": [
"/_emdash/admin",
"/preview/"
],
"allow": [
"/"
],
"crawlDelay": null,
"blocksEverything": false,
"aiBotsBlocked": []
},
"sitemaps": [
{
"url": "https://blog.cloudflare.com/sitemap.xml",
"status": 200,
"type": "index",
"entries": 8,
"error": null
},
{
"url": "https://blog.cloudflare.com/sitemap-posts-5.xml",
"status": 200,
"type": "urlset",
"entries": 78,
"error": null
}
],
"sitemapsGuessed": false,
"feeds": [
{
"url": "https://blog.cloudflare.com/rss/",
"foundVia": "link-rel-alternate",
"format": "rss",
"title": "Cloudflare Blog",
"itemCount": 20,
"latestItem": "Mon, 02 Nov 2026 14:00:00 GMT"
}
],
"llmsTxt": {
"url": "https://blog.cloudflare.com/llms.txt",
"present": false,
"bytes": null,
"excerpt": null
},
"llmsFullTxt": {
"url": "https://blog.cloudflare.com/llms-full.txt",
"present": false,
"bytes": null,
"excerpt": null
},
"urlCounts": {
"scannedInSitemaps": 21,
"matchedFilters": 20,
"filteredOut": 0,
"disallowedByRobots": 0,
"output": 20,
"truncatedAtMaxUrls": true
},
"analyzedAt": "2026-11-02T18:05:54.225Z",
"error": null
},
"urlCount": 20,
"urls": [
{
"url": "https://blog.cloudflare.com/zh-tw/cloudflare-basin",
"lastmod": "2026-10-12T03:39:57.788Z",
"changefreq": null,
"priority": null,
"sourceSitemap": "https://blog.cloudflare.com/sitemap-posts-5.xml",
"allowedByRobots": true
},
{
"url": "https://blog.cloudflare.com/zh-cn/cloudflare-basin",
"lastmod": "2026-10-12T03:48:01.854Z",
"changefreq": null,
"priority": null,
"sourceSitemap": "https://blog.cloudflare.com/sitemap-posts-5.xml",
"allowedByRobots": true
}
],
"meta": {
"requestId": "5f0c2a7e-0000-4000-8000-000000000000",
"durationMs": 2813
}
}
curl --location --request GET 'https://zylalabs.com/api/14711/website+sitemap+and+url+discovery+api/38030/discover+urls?domain=blog.cloudflare.com&maxUrls=50&includeUrls=true&includeLlmsTxt=true' --header 'Authorization: Bearer YOUR_API_KEY'
注册后,每个开发者都会被分配一个个人 API 访问密钥,这是一个唯一的字母和数字组合,用于访问我们的 API 端点。要使用 网站地图和网址发现 API 进行身份验证,只需在 Authorization 标头中包含您的 bearer token。
| 标头 | 描述 |
|---|---|
授权
|
必需
应为 Bearer access_key. 订阅后,请查看上方的"您的 API 访问密钥"。
|
从robots.txt、XML网站地图、网站地图索引、RSS提要和llms.txt中通过一次请求发现网站的所有公共URL
是的 使用includePatterns和excludePatterns(以逗号分隔的子字符串)以及lastmodSince来仅保留最近更改的页面
不 每个请求都是按需处理的 结果会在响应中返回 关于您的请求没有任何信息会被保留 超过短期操作日志
您收到一个带有明确代码的JSON错误(例如invalid_input、no_result或timeout),而不是部分数据。因无效输入而被拒绝的请求将以状态400回复