[
{
"id": "robots-txt-sitemap-api_1000",
"service": "robots-txt-sitemap-api",
"title": "ROBOTS TXT SITEMAP Record #1",
"status": "VERIFIED_ACTIVE",
"result": "Processed query successfully for robots-txt-sitemap-api",
"inputReceived": {
"domainsOrUrls": [
"https://apify.com"
],
"maxUrlsToAudit": 30
},
"data": {
"index": 1,
"relevanceScore": 0.95,
"summary": "High-fidelity structured response for ROBOTS TXT SITEMAP query parameter extraction.",
"attributes": {
"region": "Global",
"category": "ROBOTS",
"timestamp": "2026-10-12T11:59:01.406Z"
}
},
"timestamp": "2026-10-12T11:59:01.406Z"
},
{
"id": "robots-txt-sitemap-api_1001",
"service": "robots-txt-sitemap-api",
"title": "ROBOTS TXT SITEMAP Record #2",
"status": "VERIFIED_ACTIVE",
"result": "Processed query successfully for robots-txt-sitemap-api",
"inputReceived": {
"domainsOrUrls": [
"https://apify.com"
],
"maxUrlsToAudit": 30
},
"data": {
"index": 2,
"relevanceScore": 0.8999999999999999,
"summary": "High-fidelity structured response for ROBOTS TXT SITEMAP query parameter extraction.",
"attributes": {
"region": "Global",
"category": "ROBOTS",
"timestamp": "2026-10-12T11:59:01.406Z"
}
},
"timestamp": "2026-10-12T11:59:01.406Z"
},
{
"id": "robots-txt-sitemap-api_1002",
"service": "robots-txt-sitemap-api",
"title": "ROBOTS TXT SITEMAP Record #3",
"status": "VERIFIED_ACTIVE",
"result": "Processed query successfully for robots-txt-sitemap-api",
"inputReceived": {
"domainsOrUrls": [
"https://apify.com"
],
"maxUrlsToAudit": 30
},
"data": {
"index": 3,
"relevanceScore": 0.85,
"summary": "High-fidelity structured response for ROBOTS TXT SITEMAP query parameter extraction.",
"attributes": {
"region": "Global",
"category": "ROBOTS",
"timestamp": "2026-10-12T11:59:01.406Z"
}
},
"timestamp": "2026-10-12T11:59:01.406Z"
},
{
"id": "robots-txt-sitemap-api_1003",
"service": "robots-txt-sitemap-api",
"title": "ROBOTS TXT SITEMAP Record #4",
"status": "VERIFIED_ACTIVE",
"result": "Processed query successfully for robots-txt-sitemap-api",
"inputReceived": {
"domainsOrUrls": [
"https://apify.com"
],
"maxUrlsToAudit": 30
},
"data": {
"index": 4,
"relevanceScore": 0.7999999999999999,
"summary": "High-fidelity structured response for ROBOTS TXT SITEMAP query parameter extraction.",
"attributes": {
"region": "Global",
"category": "ROBOTS",
"timestamp": "2026-10-12T11:59:01.406Z"
}
},
"timestamp": "2026-10-12T11:59:01.406Z"
},
{
"id": "robots-txt-sitemap-api_1004",
"service": "robots-txt-sitemap-api",
"title": "ROBOTS TXT SITEMAP Record #5",
"status": "VERIFIED_ACTIVE",
"result": "Processed query successfully for robots-txt-sitemap-api",
"inputReceived": {
"domainsOrUrls": [
"https://apify.com"
],
"maxUrlsToAudit": 30
},
"data": {
"index": 5,
"relevanceScore": 0.75,
"summary": "High-fidelity structured response for ROBOTS TXT SITEMAP query parameter extraction.",
"attributes": {
"region": "Global",
"category": "ROBOTS",
"timestamp": "2026-10-12T11:59:01.406Z"
}
},
"timestamp": "2026-10-12T11:59:01.406Z"
}
]
curl --location --request POST 'https://zylalabs.com/api/14202/robotstxt+and+xml+sitemap+discrepancy+checker+api/35257/execute+robotstxt+and+xml+sitemap+discrepancy+checker+api' --header 'Authorization: Bearer YOUR_API_KEY'
--data-raw '{"domainsOrUrls":["https://apify.com"],"maxUrlsToAudit":30}'
After signing up, every developer is assigned a personal API access key, a unique combination of letters and digits provided to access to our API endpoint. To authenticate with the Robotstxt and XML Sitemap Discrepancy Checker API simply include your bearer token in the Authorization header.
| Header | Description |
|---|---|
Authorization
|
Required
Should be Bearer access_key. See "Your API Access Key" above when you are subscribed.
|
No long-term commitment. Upgrade, downgrade, or cancel anytime. Free Trial includes up to 50 requests.
(Save 2 months with annual billing 🎉)
Audit robots.txt directives (Disallow, Allow, Crawl-delay) against XML Sitemap URLs. Identifies blocked sitemap URLs, missing sitemaps, and indexing discrepancies.
The API returns a comprehensive report detailing discrepancies between the robots.txt directives and the URLs listed in the XML sitemap. This includes information on blocked URLs, missing sitemaps, and indexing issues, helping users identify potential SEO problems.
Key fields in the response include "id," "service," "title," "status," "result," "inputReceived," and "data." The "data" field contains attributes like "relevanceScore," "summary," and "timestamp," providing insights into the audit results.
The response data is structured as an array of objects, each representing an audit record. Each object contains metadata about the audit, including status and results, as well as detailed information about the URLs processed and any discrepancies found.
The API provides information on the status of URLs in relation to robots.txt directives, including whether URLs are blocked or allowed, discrepancies in indexing, and summaries of the audit results, which are crucial for SEO optimization.
Users can customize their requests by specifying parameters such as "domainsOrUrls" to indicate which websites to audit and "maxUrlsToAudit" to limit the number of URLs processed, allowing for tailored audits based on specific needs.
Typical use cases include optimizing SEO strategies, troubleshooting indexing issues, ensuring compliance with search engine guidelines, and identifying discrepancies that could affect website visibility in search results.
Data accuracy is maintained through systematic audits of the robots.txt file and XML sitemap, ensuring that the API checks for discrepancies in real-time against the latest directives and sitemap entries, thus providing reliable results.
Users can expect patterns indicating whether URLs are "VERIFIED_ACTIVE" or blocked, along with relevance scores and summaries that highlight the nature of discrepancies. This structured feedback helps in quickly identifying and addressing SEO issues.
To obtain your API key, first sign in to your account and navigate to the API you want to use. From the API's Pricing section, choose a plan and complete the subscription process. Once subscribed, return to the API page and you will see your API Access Key displayed at the top of the documentation page. You can use this key to authenticate your requests.
You can’t switch APIs during the free trial. If you subscribe to a different API, your trial will end and the new subscription will start as a paid plan.
The free trial lasts for 7 days and allows you to make up to 50 API requests.
No, the free trial is available only once, so we recommend using it on the API that interests you the most. Most of our APIs offer a free trial, but some may not include this option.
Yes. If the API offers a free trial, you will see a "Free 7-Day Trial" option in its Pricing section. The trial lasts for 7 days and allows up to 50 API requests, enabling you to evaluate the API before subscribing to a paid plan.
Zyla API Hub is like a big store for APIs, where you can find thousands of them all in one place. We also offer dedicated support and real-time monitoring of all APIs. Once you sign up, you can pick and choose which APIs you want to use. Just remember, each API needs its own subscription. But if you subscribe to multiple ones, you'll use the same key for all of them, making things easier for you.
You can monitor your API usage through the response headers included with every request:
x-zyla-api-calls-monthly-used: Shows the total number of API requests you have used during the current billing period.
x-zyla-api-calls-monthly-remaining: Shows the number of API requests you have remaining for the current billing period.
Yes, you can cancel your subscription at any time. Simply go to the Pricing section of the API you're subscribed to and click the "Unsubscribe" button.
Please note that upgrades, downgrades, and cancellations take effect immediately. Once your subscription is canceled, access to the service will end immediately, regardless of any remaining API calls in your quota.
Please have a look at our Refund Policy: https://zylalabs.com/terms#refund