AI tools for website crawling

3,708 live tools · ranked by monthly traffic and community votes · dead links removed automatically

Has API Verified
1

Apify is a web scraping and automation platform built around a marketplace of ready-to-run programs called Actors.

FREEMIUM Development & Code 20 days ago
38 0
2
Exa

Search built for AI

FREEMIUM Development & Code 9 months ago
383 0
3

Olostep is a web data API for AI agents and developers.

FREEMIUM Development & Code 7 months ago
169 0
4

Brand.dev is a web scraping API for AI agents and LLM products.

FREEMIUM Development & Code 6 months ago
122 0
5

Firecrawl is a web data API that gives AI agents and applications clean content from the live web.

FREEMIUM Development & Code 20 days ago
39 0
6

Tavily is a real-time search and web access API for AI agents.

FREEMIUM Development & Code 20 days ago
35 0
7

Web scraping without writing code

FREEMIUM Development & Code 9 months ago
180 0
8

Hyperbrowser delivers cloud-based browser infrastructure designed for AI agents and automated applications

FREEMIUM Development & Code 7 months ago
227 0
9

BrowserAct is a no-code web scraping platform that builds a reusable scraper from one prompt.

FREEMIUM Development & Code 7 months ago
132 0
10

SearchCans is a search and content-extraction API service for AI workflows.

FREEMIUM Development & Code 7 months ago
117 0