Sitemaps and last modified dates

Our sitemap is generated from the site structure rather than maintained as a hand-edited XML file.

Why generation matters

Manual sitemaps drift. New pages get forgotten, old pages remain listed, and update dates become meaningless. A generated sitemap uses the same content sources that create the pages.

Route families are included deliberately

Static pages, articles, services, locations, portfolio entries, and documentation can each be mapped into sitemap entries with appropriate priorities and update frequencies.

Last modified should mean something

We use repository history to generate last-modified dates for source files. That is stronger than setting every page to today's date on every build.

Sitemap dates are hints

A changed lastmod value should reflect a real source change. Search engines can use that signal when deciding what to recrawl.

The sitemap is not the architecture

A page should still be linked from the site where appropriate. The sitemap helps discovery and recrawl behavior, but it does not replace navigation or internal linking.

The sitemap is generated from the same registries that create the pages

Our site architecture already knows which articles, services, locations, portfolio entries, and documentation pages exist. The sitemap should consume that same source of truth instead of maintaining a second inventory. On the connectrader site, static pages are listed with their source files while dynamic sections are mapped from their content libraries. A documentation item automatically contributes its /docs/slug URL. A portfolio item contributes its case-study URL. The route generator and the sitemap are describing the same application from different angles.

This reduces a common failure mode where a page exists but never enters the sitemap, or an old URL remains listed after the page has been removed.

Lastmod is generated before the build

Several of our repos run a prebuild script that asks Git for the most recent commit date affecting each relevant source file. The result is written into a small map that the application can import while generating the sitemap and structured data.

That means the date is available without running Git commands on every request. The expensive or environment-dependent work happens once during the build.

Shared templates can change dynamic pages

A content file is not the only source that affects a generated document. If the dynamic case-study template changes, every case study can change visually or semantically even when the individual MDX files do not. Our sitemap logic can combine the content date with the route-template date and use whichever is later. This is more honest than pretending only the body file matters.

Change frequency and priority are hints, not ranking controls

XML sitemaps support values such as changefreq and priority. We use them conservatively as descriptive hints. They are not levers that force a search engine to crawl or rank a page. A service page can reasonably have a different update expectation than a legal policy, but marking everything as daily and priority 1.0 does not make the site more important.

The useful property is consistency

The sitemap should point at canonical production URLs, include the pages intended for indexing, use modification dates grounded in source history, and update automatically when the content system changes. The deeper date logic is covered in Why lastmod should come from real source history. The larger goal is that the sitemap behaves like an output of the codebase rather than a manually curated artifact that can drift away from it.