Now a startup called 80legs says it’s selling web crawling as a service. Web crawling means browsing and indexing online content in an automated fashion.

Our web crawling and competitor analysis solutions integrate advanced analytics and web scraping techniques to help businesses benchmark their performance against that of their competitors.

Quantzig announces the completion of its recent data harvesting and web crawling analysis engagement.

On August 19, 2025, Firecrawl announced the closing of a $14.5 million Series A funding round led by Nexus Venture Partners, ...

Yahoo today announced that it has released the source code for its Anthelion web crawler designed for parsing structured data from HTML pages under an open source license.