Crawler
About the Declarebase crawler
DeclarebaseBot looks for the official documents of building products on manufacturers' websites. This way contractors find the right datasheet and see what is missing or expired.
What it does
It looks for documents
DeclarebaseBot reads public product pages and documents: technical datasheets (TDS), declarations of performance (DoP), CE markings and EPDs. It notes where a document lives and when we last checked it.
Recognise it
The user agent
Every request from the crawler carries this user agent:
DeclarebaseBot/1.0 (+https://declarebase.com/bot)The link points to this page.
Rules
robots.txt and TDM reservations
DeclarebaseBot follows the rules in your robots.txt, for DeclarebaseBot and for all bots (*). Did you reserve text and data mining in a machine-readable way (Article 4 of EU Directive 2019/790)? It respects that too.
Pace
How often it visits
At most one request per second per website. Most visits happen at night, Belgian time. It comes back to check that a document is still current.
Opt out
Opt out or ask for removal
Don't want DeclarebaseBot on your site? Add this to your robots.txt:
User-agent: DeclarebaseBot
Disallow: /To have links or data from your site removed, mail hallo@declarebase.com with your domain or the pages. We reply by email.