sitemapgen

sitemapgen is a Python 3.11 tool that writes Sitemaps protocol files. It uses the standard library. sitemapgen build reads a text file with one absolute URL per line, or a JSONL file that can include lastmod, changefreq, priority, images, and alternates. sitemapgen crawl walks one host. The output is a UTF-8 XML urlset. When a file would pass 50,000 URLs or 50 MiB, further URLs go into more files and sitemap.xml becomes a sitemap index. The same command can write a plain-text URL list, and it can add a Sitemap line to robots.txt. It runs from the command line. The source is on GitHub.

Python

The package has no third-party dependencies. Each URL is checked against the protocol. URLs that do not fit are left out and printed on stderr.