The /contents endpoint now returns detailed status information for each URL instead of HTTP error codes, providing better visibility into individual content fetch results.
/contents
endpoint to provide more granular status information for each URL you request. Instead of returning HTTP error codes directly, the endpoint now includes a statuses
field that gives you detailed information about each content fetch operation.
/contents
endpoint will now only return an error if there’s an internal issue on our end. All other cases are handled through the new statuses
field./contents
endpoint would return HTTP error codes when content fetching failed. This approach had limitations when multiple URLs failed for different reasons, making it unclear which specific error to return.
Now, the endpoint returns a statuses
field containing individual status information for each URL, allowing you to handle different failure scenarios appropriately.
"success"
or "error"
"error"
CRAWL_NOT_FOUND
: Content not found (404)CRAWL_TIMEOUT
: Request timed out (408)SOURCE_NOT_AVAILABLE
: Access forbidden or source unavailable (403)CRAWL_UNKNOWN_ERROR
: Other errors (500+)statuses
field: