XML Sitemap Best Practices (and What Silently Breaks Them)

An XML sitemap is the simplest way to tell search engines which pages on a site actually matter. Done well, it helps Google find and understand your important pages faster. Done carelessly, it feeds Google a list full of redirects, dead pages, and URLs you never wanted indexed, and slowly erodes trust in the site. Here are the practices that keep a sitemap useful, the mistakes that quietly break it, and why a sitemap that was perfect last month may not be today.

Deltio watches your client sitemaps for the changes that break them silently, and alerts you on Slack when something shifts. From £20 a month.

What a sitemap is actually for

An XML sitemap is a list of the URLs you want search engines to know about, along with a little metadata about each. Its job is discovery: it helps engines find pages, understand which ones you consider canonical, and prioritise crawling, especially on large sites or ones with pages that are not well linked internally.

It helps to be clear about what a sitemap is not. It is not a ranking factor, and including a URL does not force Google to index it. A sitemap is a recommendation, not a command. That is exactly why the quality of the list matters so much: it is a signal of what you think is important, and a messy list sends a messy signal.

The best practices that matter

Most sitemap advice comes down to one principle: the sitemap should be a clean, honest list of the pages you want indexed, and nothing else.

• Only include canonical, indexable URLs. Every URL in the sitemap should return a 200 status, be canonical to itself, and not carry a noindex. If a page is not meant to be indexed, it does not belong in the sitemap.
• No redirects, no dead pages. A URL that 301-redirects or returns a 404 should not be in there. These are the most common pollutants, and they tell Google your list is out of date.
• Keep it current. The sitemap should update when pages are added or removed. A sitemap that still lists pages you deleted six months ago is actively misleading.
• Use honest lastmod dates. The lastmod field should reflect when the page's content genuinely changed. If every URL claims it was modified today, the field becomes noise and engines learn to ignore it.
• Respect the size limits. A single sitemap file should stay under 50,000 URLs and 50MB uncompressed. Past that, split into multiple sitemaps and tie them together with a sitemap index file.
• Reference it in robots.txt and submit it in Search Console. Add a Sitemap: line to robots.txt so engines can discover it, and submit it in Google Search Console so you can see how it is being processed.

None of this is complicated. The difficulty is that a sitemap is generated by software, and software changes without telling you.

The mistakes that quietly break a sitemap

The dangerous sitemap problems are not the ones that throw an error. They are the ones that leave the file valid but wrong.

• URLs that shouldn't be there. A CMS or plugin starts including noindex pages, tag archives, redirected URLs, or 404s. The file still parses fine; it just now tells Google to care about pages you don't.
• URLs silently disappearing. A regeneration bug or a bad deploy drops a chunk of real pages from the sitemap. Nothing errors. Google simply has a shorter list to discover, and over time crawling of those pages slows.
• Stale lastmod on everything. A platform stamps today's date on every URL at every rebuild. Now the field is worthless, and worse, it can waste crawl budget re-fetching pages that never changed.
• An orphaned sitemap. The sitemap URL changes or is removed from robots.txt, and engines stop finding the fresh version.

The common thread is that all of these happen after launch, triggered by an update nobody connected to the sitemap, and none of them are visible unless you go and look.

Why sitemaps break silently, and why that hurts agencies

A sitemap is almost always machine-generated, by the CMS, an SEO plugin, or a build step. That means any update to any of those can change it: a plugin version bump, a template change, a migration, a caching layer. The person making that change is rarely thinking about the sitemap, so the breakage is a side effect nobody notices.

If you manage one site, you might eventually spot it. If you manage twenty client sites you don't control, you won't, because there is no reason to open a sitemap on a Tuesday for a site that seems fine. Meanwhile a client site has quietly dropped four hundred URLs from its sitemap, or started listing every redirected page, and the first sign is a slow slide in crawling and indexing weeks later.

How to keep a sitemap honest over time

You can audit a sitemap by hand: open it, check the URL count, spot-check that entries return 200 and are indexable, and confirm it is referenced in robots.txt. That works for a site you remember to check.

Monitoring is what covers the sites you don't. Deltio watches the sitemaps on your client sites and flags the changes that matter: the URL count jumping or dropping, URLs disappearing, and the related indexing signals shifting on the pages the sitemap points to. When something changes, you get a plain alert on Slack naming the site, the same day it happens, instead of inferring it from a traffic report a month later. It watches the neighbouring signals the same way, noindex, canonicals, and robots.txt, because a sitemap rarely breaks in isolation. You can see how those alerts work in SEO change alerts, and the robots.txt side in how to check your robots.txt.

The short version

A good XML sitemap is a clean, current list of only the canonical, indexable pages you want found, kept under the size limits and referenced in robots.txt. The best practices are easy; keeping them true as the site changes is the hard part, because the file is generated by software that gets updated without anyone thinking about the sitemap. On sites you are responsible for, the practice that actually protects you is being told the day the list changes.

Add your first client site, connect Slack, and Deltio starts watching the same day. Start your 14-day trial and find out what's really in your clients' sitemaps.