Version: 1.0
Obeys Robots.txt: Yes
User-Agent String: Mozilla/5.0 (compatible; VelenPublicWebCrawler/1.0; +https://velen.io)
What is Hunter robot?
Every month, Hunter robot crawls billions of web pages. It works in a way similar to Google but with a focus on business data. We categorize the information in our tools to allow hundreds of thousands of professionals to connect with other businesses.
What is Hunter robot doing on your website?
The robot only analyzes public web pages and navigates at a slow pace. It never visits more than one page every two seconds for a website to make sure other visitors aren’t slowed down by the crawling.
Does it respect robots.txt file?
Yes. The robot strictly respect robots.txt, both disallow and allow rules (we do not, however, support nonstandard extensions). If for whatever reason you wish to disable it on your website, add the two following lines into to your robots.txt file:
User-agent: VelenPublicWebCrawler
Disallow: /
If you think that Hunter robot is someway misbehaving on your website or if you have any questions about it, please don’t hesitate to contact us at contact@hunter.io.
