AI tools for website crawling

3,708 live tools · ranked by monthly traffic and community votes · dead links removed automatically

Has API Verified
1

Apify is a web scraping and automation platform built around a marketplace of ready-to-run programs called Actors.

FREEMIUM Development & Code 20 days ago
38 0
2
Exa

Search built for AI

FREEMIUM Development & Code 9 months ago
383 0
3

Olostep is a web data API for AI agents and developers.

FREEMIUM Development & Code 7 months ago
169 0
4

Firecrawl is a web data API that gives AI agents and applications clean content from the live web.

FREEMIUM Development & Code 20 days ago
39 0
5

Tavily is a real-time search and web access API for AI agents.

FREEMIUM Development & Code 20 days ago
35 0
6

Hyperbrowser delivers cloud-based browser infrastructure designed for AI agents and automated applications

FREEMIUM Development & Code 7 months ago
227 0
7

BrowserAct is a no-code web scraping platform that builds a reusable scraper from one prompt.

FREEMIUM Development & Code 7 months ago
132 0
8

SearchCans is a search and content-extraction API service for AI workflows.

FREEMIUM Development & Code 7 months ago
117 0
9

Parallel Web Systems builds web infrastructure for AI agents.

FREE Development & Code 20 days ago
50 0