web-scraping

11 tools and resources

NEW

Fetches known URLs from various sources for a given domain

urlgrab Logo

urlgrab

0 (0)

A golang utility to spider through a website searching for additional links.

Automate mass scanning of technologies used on websites

Katana Logo

Katana

0 (0)

A next-generation crawling and spidering framework for extracting data from websites

httpx Logo

httpx

0 (0)

A fast and multi-purpose HTTP toolkit for sending HTTP requests and parsing responses

URO Logo

URO

0 (0)

A tool to declutter URL lists for crawling and pentesting

A tool for mining URLs from dark corners of Web Archives for bug hunting/fuzzing/further probing

Online Telegram bot for collecting information on individuals from various websites.