How well kept are indie websites?
95% of indie sites serve HTTP/2, 87% have a robots.txt and 84% a sitemap.xml — but only 56% send an HSTS header, so 45% have all four basics. Google Trust Services and Let's Encrypt issue 93% of the certificates those sites present.
- have a sitemap.xml
- 84%
- have a sitemap.xml
- n = 7,273
- have a robots.txt
- 87%
- have a robots.txt
- send an HSTS header
- 56%
- send an HSTS header
- serve HTTP/2
- 95%
- serve HTTP/2
Key findings
- 45% of indie sites have all four basics this chapter checks — HTTP/2, HSTS, a robots.txt and a sitemap.xml. HSTS is the one most often missing: 56% send it.
- 87% publish a robots.txt and 84% a sitemap.xml; 78% have both, the pair search engines read first.
- Free, automated certificates have won: Google Trust Services (48%) and Let's Encrypt (45%) issue nearly every certificate, and 95% had 90 days or less left when checked — the mark of certificates that renew themselves.
- 24% of sites settle on a www address; the rest live on the bare domain. 87% answer at the address their maker submitted without a single redirect.
- HSTS is decided by the host more than the maker: 100% of sites on Vercel send it, against 35% of sites behind Cloudflare, where the owner has to switch it on.