Crawler made as a side project in Python 3.6.6
These URL crawling classes can handle some, not all (and definitely not very many), URL's. This is a side project for learning about web scraping, crawling, and whatnot.
It should be noted that I try to maintain a high level of PEP codestyle and linting guidelines. I used the PIP package manager with the pylint, pycodestyle and flake8 packages.