[Bug] Sphinx docs sitemap html_baseurl is wrong and conflicts with canonical urls

[echoix]

The new version of sphinx-sitemap added a dependency on an extension to get the git timestamps. The sitemap generated with it was quite nice, here's a diff:

(Also note that these URLs are kinda wrong, just like the canonical url in the html pages, as both use "html_baseurl" config in conf.py, and is set twice with different values.)
[...]

For pages with a git source, the metadata is added nicely, and the footer has the date changed:

Originally posted by @echoix in #5925 (comment)

Describe the bug

Sphinx sitemap extension uses html_baseurl. Generating the canonical url tag by sphinx also uses html_baseurl. html_baseurl is set twice, to conflicting values. So the last value is used by both, and both are pointing to non-existing paths

Issue created when the main branch was at 1c216c2, but this conflict seems to predate this, possibly since sphinx-sitemap was added.

[echoix]

@neteler Are we getting penalized for these urls when they are hosted?