Releases: adbar/courlan
Releases · adbar/courlan
Release list
courlan-0.9.2
- add blogspot archives to type filter
- maintenance: upgrade urllib3 and review code
courlan-0.9.1
- network tests: larger throughput
- UrlStore: optional compression of rules (#21), added
reset()(#22) andget_all_counts()methods - UrlStore fixes:
signalin #18,total_url_number - updated Readme
Full Changelog: v0.9.0...v0.9.1
courlan-0.9.0
- hardening of filters and URL parses (#14)
- normalize punicode to unicode
- methods added to
UrlStore:get_crawl_delay(),print_unvisited_urls() UrlStorenow triggers exit code 1 when interrupted- argument added to
extract_links():no_filter - code refactoring: simplifications
Full Changelog: v0.8.3...v0.9.0
courlan-0.8.3
- fixed bug in domain name extraction
- uniform logging parameters
Full Changelog: v0.8.2...v0.8.3
courlan-0.8.2
- full type hinting
- maintenance: code linted
Full Changelog: v0.8.1...v0.8.2
courlan-0.8.1
- add type annotations and check with
mypy url_filter()function moved from Trafilatura- code style: use
black
courlan-0.8.0
- performance optimizations
- fast track for domain extraction (
extract_domain(url, fast=True)), now taking subdomains into account
Full Changelog: v0.7.2...v0.8.0
courlan-0.7.2
UrlStore: threading lock and convenience functions added
courlan-0.7.1
- bug in sampling fixed
UrlStore: validation by default
Full Changelog: v0.7.0...v0.7.1
courlan-0.7.0
UrlStoreclass added: data store containing URLs with relevant information- code cleaning and maintenance (bugs, simplification)
Full Changelog: v0.6.0...v0.7.0