Skip to content

Documentation / Domains

Search Engine Crawling

Keep staging systems and platform domains out of Google and other search engines.

Not every Odoo instance should show up in Google. Test systems, staging copies, preview deployments and the automatic platform domain of a production system are usually reachable from the internet, but you rarely want them indexed — duplicate content, confused customers landing on a staging shop, or an internal URL appearing in search results.

calmo.cloud lets you block search engine crawling per hostname, so the same Odoo service can be indexable on shop.company.com and hidden on erp.production.acme.calmo.cloud at the same time.

How it works #

When crawling is blocked for a hostname, calmo.cloud configures Traefik to add the header

X-Robots-Tag: noindex, nofollow, noarchive

to every response served on that hostname. Search engines like Google and Bing honour this header on all content types (HTML pages, PDFs, images) and remove the affected URLs from their index — even pages that Odoo's own robots.txt would allow. Requests on hostnames without the flag are served exactly as before.

Because the header is added by the reverse proxy in front of Odoo, this works for every Odoo version and does not require any change inside Odoo. It is enforced per domain, so a hostname that shares an Odoo database with a public shop stays hidden while the shop stays visible.

The header is a signal to well-behaved crawlers, not an access restriction. Anyone with the URL can still open the page. Use it to keep systems out of search results, not to protect confidential data.

Only live services can be indexed #

Test, demo, copy, preview and sandbox services are always blocked on all of their hostnames — custom domains and the platform domain alike. Nothing needs to be configured for them, and the toggles described below are shown as locked. Only services of type Live decide per hostname whether search engines may index them.

Service type Search engine crawling
Live Allowed by default on custom domains; configurable per hostname and for the platform domain
Test, Demo Always blocked on all hostnames
Copy, Preview, Sandbox Always blocked on all hostnames

Custom domains (live services) #

Blocking is a setting on each hostname:

  1. Navigate to your Odoo service
  2. Click the Hostnames tab
  3. Click Edit on the hostname
  4. Enable Prevent search engine crawling
  5. Save and redeploy the service

The No index column in the hostnames table shows which hostnames are currently blocked.

Redirect hostnames (see Redirects) never route to Odoo, so the setting is not shown for them — the redirect target decides whether the destination is indexable.

Platform domain (system domain, live services) #

The platform domain {service}.{server}.{team}.calmo.cloud is not a regular hostname entry, so it has its own toggle directly on the service:

  1. Open the Odoo service and click Edit
  2. Enable Prevent system domain crawling
  3. Save and redeploy the service

New services created in the UI have this enabled by default. Copies inherit the setting from their source service (it only matters once a copy is promoted to a live service).

Default for provisioned services #

When you provision servers from a provisioning template, the template's Prevent system domain crawling toggle (in the Services Configuration section, enabled by default) decides whether services created from that template block crawling on their platform domain. Turn it off if the platform domain of provisioned systems is meant to be public.

Applying the change #

Like hostname and redirect changes, crawler blocking is part of the service's routing configuration and takes effect on the next redeploy. The service shows the usual Redeploy required notice, and the View changes panel lists the affected hostnames under Crawling blocked.

After the redeploy, curl -I https://your-hostname/ shows the X-Robots-Tag: noindex, nofollow, noarchive header on a blocked hostname and no such header on an indexable one.

Troubleshooting #

The page is still listed in Google
  • Search engines remove pages the next time they crawl them; this can take from a few days to several weeks. You can speed it up with the URL removal tool in Google Search Console.
  • Make sure the service was redeployed after enabling the toggle.
  • Google can only see the header if it can fetch the page. If you additionally block crawlers via robots.txt in Odoo, Google keeps the old entry without ever seeing the noindex.
My public shop disappeared from search results
  • Check that the flag is only set on the hostnames you want hidden. Blocking is per hostname; the platform domain toggle only affects the *.calmo.cloud domain, not your custom domains.
Redirects Uptime monitoring