getallurls (gau)

Fetches known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan for any given domain. It can also filter URLs by status code, MIME type, and more. You can also specify a configuration file to use for every subsequent run of gau. You can install gau from source, from a pre-built binary, or via Docker. Note that piping a command (echo "example.com" | gau) will not work with the Docker container. Also, be aware that ohmyzsh's git plugin has an alias which maps gau to the git add --update command, which can cause a binary conflict. You can donate to CommonCrawl and the Internet Archive.