crawley Logo

crawley

0
Free
Visit Website

The unix-way web crawler Crawley is a simple web crawler written in Go. Features: * Supports multiple protocols (http, https, ftp, sftp) * Supports multiple authentication methods (basic auth, digest auth, kerberos) * Supports crawling of nested resources * Supports filtering of crawled resources * Supports saving of crawled resources to disk Example usage: ```go import "github.com/s0rg/crawley" func main() { crawler := crawley.NewCrawler() crawler.AddURL("https://example.com") crawler.Start() } ```

FEATURES

ALTERNATIVES

A simple tool to take screenshots of HTTPS websites

A python tool for discovering endpoints, parameters, and wordlists in a given target

A tool for enumerating X-Forwarded-For headers in HTTP requests

An open source, self-hosted implementation of the Tailscale control server.

A free DNS recursive service that blocks malicious host names and protects user privacy.

TCPFLOW is a tool for capturing data transmitted over TCP connections.

Independent software vendor specializing in network security tools and network forensics.

A Linux command-line tool that allows you to kill in-progress TCP connections based on a filter expression, useful for libnids-based applications that require a full TCP 3-way handshake for TCB creation.

PINNED

ImmuniWeb® Discovery Logo

ImmuniWeb® Discovery

ImmuniWeb Discovery is an attack surface management platform that continuously monitors an organization's external digital assets for security vulnerabilities, misconfigurations, and threats across domains, applications, cloud resources, and the dark web.

Attack Surface Management
InfoSecHired Logo

InfoSecHired

An AI-powered career platform that automates the creation of cybersecurity job application materials and provides company-specific insights for job seekers.

Resources
Mandos Brief Newsletter Logo

Mandos Brief Newsletter

A weekly newsletter providing cybersecurity leadership insights, industry updates, and strategic guidance for security professionals advancing to management positions.

Resources
Checkmarx SCA Logo

Checkmarx SCA

A software composition analysis tool that identifies vulnerabilities, malicious code, and license risks in open source dependencies throughout the software development lifecycle.

Application Security
Check Point CloudGuard WAF Logo

Check Point CloudGuard WAF

A cloud-native web application and API security solution that uses contextual AI to protect against known and zero-day threats without signature-based detection.

Application Security
Orca Security Logo

Orca Security

A cloud-native application protection platform that provides agentless security monitoring, vulnerability management, and compliance capabilities across multi-cloud environments.

Cloud Security
DryRun Logo

DryRun

A GitHub application that performs automated security code reviews by analyzing contextual security aspects of code changes during pull requests.

Application Security
Wiz Logo

Wiz

Wiz Cloud Security Platform is a cloud-native security platform that enables security, dev, and devops to work together in a self-service model, detecting and preventing cloud security threats in real-time.

Cloud Security