Skip to content
/ lychee Public
forked from lycheeverse/lychee

🔗 A link checker. Crazy, I know.

Notifications You must be signed in to change notification settings

slohse/lychee

 
 

Repository files navigation

lychee

Rust

...because who says I can't write yet another link checker?

What?

This thing was created from Hello Rust Episode 10. It's a link checker that treats Github links specially by using a GITHUB_TOKEN to avoid getting blocked by the rate limiter.

Lychee demo

Why?

The existing link checkers were not flexible enough for my use-case. lychee runs all requests fully asynchronously and has a low memory/CPU footprint.

Features

This comparison is made on a best-effort basis. Please create a PR to fix outdated information.

lychee awesome_bot muffet broken-link-checker linkinator linkchecker markdown-link-check fink
Language Rust Ruby Go JS TypeScript Python JS PHP
Async/Parallel ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
Static binary ✔️ ✖️ ✔️ ✖️ ✖️ ️ ✖️ ✖️ ✖️
Markdown files ✔️ ✔️ ✖️ ✖️ ✖️ ✖️ ️ ✔️ ✖️
HTML files ✔️ ✖️ ✖️ ✔️ ✔️ ✖️ ✖️ ✖️
Txt files ✔️ ✖️ ✖️ ✖️ ✖️ ✖️ ✖️ ✖️
Website support ✔️ ✖️ ✔️ ✔️ ✔️ ✔️ ✖️ ✔️
Chunked encodings ✔️ ? ? ? ? ✖️ ✔️ ✔️
GZIP compression ✔️ ? ? ✔️ ? ✔️ ? ✖️
Basic Auth ✖️ ✖️ ✖️ ✔️ ✖️ ✔️ ✖️ ✖️
Custom user agent ✔️ ✖️ ✖️ ✔️ ✖️ ✔️ ✖️ ✖️
Relative URLs ✔️ ✔️ ✖️ ✔️ ✔️ ✔️ ✔️ ✔️
Skip relative URLs ✔️ ✖️ ✖️ ? ✖️ ✖️ ✖️ ✖️
Include patterns ✖️ ✔️ ✖️ ✔️ ✖️ ✖️ ✖️ ✖️
Exclude patterns ✔️ ✖️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
Handle redirects ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️ ✔️
Ignore insecure SSL ✔️ ✔️ ✔️ ✖️ ✖️ ✔️ ✖️ ✔️
File globbing ✔️ ✔️ ✖️ ✖️ ✔️ ✖️ ✔️ ✖️
Limit scheme ✔️ ✖️ ✖️ ✔️ ✖️ ✔️ ✖️ ✖️
[Custom headers] ✔️ ✖️ ✔️ ✖️ ✖️ ✖️ ✔️ ✔️
Summary ✔️ ✔️ ✔️ ? ✔️ ✔️ ✖️ ✔️
HEAD requests ✔️ ✔️ ✖️ ✔️ ✔️ ✔️ ✖️ ✖️
Colored output ✔️ ? ✔️ ? ✔️ ✔️ ✖️ ✔️
[Filter status code] ✔️ ✔️ ✖️ ✖️ ✖️ ✖️ ✔️ ✖️
Custom timeout ✔️ ✔️ ✔️ ✖️ ✔️ ✔️ ✖️ ✔️
E-mail links ✔️ ✖️ ✖️ ✖️ ✖️ ✔️ ✖️ ✖️
Progress bar ✔️ ✔️ ✖️ ✖️ ✖️ ✔️ ✔️ ✔️
Retry and backoff ✔️ ✖️ ✖️ ✖️ ✔️ ✖️ ✔️ ✖️
Skip private domains ✔️ ✖️ ✖️ ✖️ ✖️ ✖️ ✖️ ✖️
[Use as lib] ✖️ ✔️ ✖️ ✔️ ✔️ ✖️ ✔️ ✖️
Quiet mode ✔️ ✖️ ✖️ ✖️ ✔️ ✔️ ✔️ ✔️
Amazing lychee logo ✔️ ✖️ ✖️ ✖️ ✖️ ✖️ ✖️ ✖️
Config file ✔️ ✖️ ✖️ ✖️ ✔️ ✔️ ✔️ ✖️

Planned features:

  • report output in HTML, SQL, CSV, XML, JSON, YAML... format
  • report extended statistics: request latency
  • recursion
  • use colored output (https://crates.io/crates/colored)
  • skip duplicate urls
  • request throttling

Users

How?

cargo install lychee

Set an environment variable with your token like so GITHUB_TOKEN=xxxx.

Run it inside a repository with a README.md or specify a file with

lychee <yourfile>

Comparison

Collecting other link checkers here to crush them in comparison. :P

Thanks

...to my Github sponsors and Patreon sponsors for supporting these projects. If you want to help out as well, go here.

[custom headers]: rust-lang/crates.io#788) [filter status code]: tcort/markdown-link-check#94 [skip private domains]: https://github.com/appscodelabs/liche/blob/a5102b0bf90203b467a4f3b4597d22cd83d94f99/url_checker.go [use as lib]: raviqqe/liche#13

About

🔗 A link checker. Crazy, I know.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Rust 99.2%
  • HTML 0.8%