Search
Hand crawlers a map of the pages you want found
Your robots file does not point at a sitemap, or the address it points at does not serve a reachable XML file, so crawlers have no list of your pages.
Why it matters
A sitemap is a plain list of the pages you want indexed, with the date each one last changed. Crawlers use it to find pages that few other pages link to, and to notice updates sooner. Without one the site can still be crawled, but new pages wait longer to appear, and pages reachable only through your own navigation may wait a very long time.
How you would notice it
- New pages take weeks to appear in search results.
- Search Console reports that no sitemap was found, or that it could not be read.
- A page you know exists never gets indexed.
What to do
Stage 1
- Make sure a real sitemap file exists at a stable address, usually /sitemap.xml. Most site builders and plugins generate one for you.
A hand-built site can start with the format above and add one url entry per page. List the pages you want found, not every file on the server.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://www.example.com/services/</loc>
</url>
</urlset>Stage 2
- Declare the sitemap in robots.txt with a Sitemap line naming the full address.
The address must be the one that actually answers, spelled exactly. A declared address that 404s is worse than no declaration, because the check reports it as broken rather than absent.
User-agent: *
Allow: /
Sitemap: https://www.example.com/sitemap.xmlStage 3
- Open the declared address in a browser and confirm you see XML, starting with a urlset element, not a page-not-found screen and not your homepage.
Some hosts answer every unknown address with the homepage and a friendly success status. That is not a sitemap, and the check reads the first bytes of the file to tell the difference.
How to check it worked
Confirm it worked
Open the sitemap address you declared and confirm the browser shows XML with your pages inside it. Add the same address to Search Console, which reports whether it was accepted. Re-run the Siege Test and confirm the Sitemap recommendation is gone.