crawley
Crawley is a web crawler written in Go that follows the unix-way philosophy for web resource discovery and collection. The tool supports multiple network protocols including HTTP, HTTPS, FTP, and SFTP for comprehensive web crawling capabilities. It provides various authentication methods such as basic authentication, digest authentication, and Kerberos to access protected resources. Key features include the ability to crawl nested resources recursively, filter crawled content based on specified criteria, and save discovered resources directly to disk for offline analysis. The crawler can be programmatically integrated into Go applications through its API interface. The tool is designed for security professionals who need to systematically discover and collect web resources for reconnaissance, asset discovery, or security assessment purposes.
FEATURES
EXPLORE BY TAGS
SIMILAR TOOLS
Open-source security automation platform for automating security alerts and building AI-assisted workflows.
A community repository of workflow templates for the Ayehu NG platform that enables automated IT and business process execution.
SOARCA is an open-source SOAR platform that automates security incident response workflows using standardized CACAOv2 playbooks and multiple integration interfaces.
Fast Intercept is a security automation platform that empowers users to maximize their existing security products and automate routine tasks.
Incident response and case management solution for efficient incident response and management.
Shuffle is a platform for automating security workflows with confidence, offering templates, collaboration tools, and a large app library.
An open-source, drag-and-drop security workflow builder with integrated case management for automating security workflows and tackling alert fatigue.
Shuffle Automation provides an open-source platform for security orchestration, automation, and response.
RedELK is a SIEM tool designed for red teams to monitor and receive alerts about blue team detection activities during penetration testing engagements.
PINNED
A cybersecurity platform that offers vulnerability scanning, Windows Defender and 3rd party AV management, and MFA compliance reporting, among other features.
Proton Pass is a cross-platform password manager that provides encrypted storage, password generation, and security monitoring features with integrated 2FA and dark web monitoring capabilities.
NordVPN is a commercial VPN service that encrypts internet connections and hides IP addresses through a global network of servers, featuring integrated threat protection and multi-device support.
Fractional CISO service that helps B2B companies implement security leadership to win enterprise deals, achieve compliance, and develop strategic security programs.