Releases: fredwu/crawler
Releases · fredwu/crawler
v1.5.0
v1.4.0
- [Added] Allow multiple instances of Crawler sharing the same queue
- [Improved] Logger will now log entries as
debugorwarn
Full Changelog: v1.3.0...v1.4.0
v1.3.0
- [Added]
:storeoption, defaults tonilto save memory usage - [Added]
:max_pagesoption - [Added]
Crawler.running?/1to check whether Crawler is running - [Improved] The queue is being supervised now
Full Changelog: v1.2.0...v1.3.0
v1.2.0
v1.2.0 [2023-09-29]
- [Added]
Crawler.Store.all_urls/0to find all scraped URLs - [Improved] Memory usage optimisations
Full Changelog: v1.1.2...v1.2.0