Archiving tools

jenniferplusplus · 2025-02-03T02:29:28.208Z

Some tools that are likely to be useful:

grab-site, the archival web crawler used by archive-team

github.com

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

heritrix, the archival web crawler used by the Internet Archive

github.com

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

schoeneh · 2025-02-03T14:58:16.977Z

A post was merged into an existing topic: Video-Download

schoeneh · 2025-02-03T14:57:21.031Z

A post was merged into an existing topic: Webscraping

schoeneh · 2025-02-03T15:38:23.910Z