SupercarMenuBot

SupercarMenuBot is the price crawler of SupercarMenu, a neutral comparison of luxury car rental prices in Dubai. Once a day it reads the public car pages of rental companies and takes the daily price, the model year, the car name, the page address and a short piece of text around the price. It never logs in, never fills in forms and never books.

How it behaves

• User agent: SupercarMenuBot/1.0 (+https://supercarmenu.com/about-our-bot)
• Obeys robots.txt (RFC 9309) and any Crawl-delay; at most one request every 2 seconds per website.
• Stops for the day at the first bot check, HTTP 403 or 429. It never tries to get past a challenge.
• Signs its requests with Web Bot Auth (HTTP Message Signatures, RFC 9421). Public key: https://supercarmenu.com/.well-known/http-message-signatures-directory

Stop it, or send your prices directly

To stop the crawler, disallow SupercarMenuBot in your robots.txt, or write to [email protected] and we switch your website off within 2 working days. Rental companies can also send their prices to us directly instead of being crawled.