-
Notifications
You must be signed in to change notification settings - Fork 29
[WIP] Victor Mikhan #55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
viktarmikhan
wants to merge
26
commits into
introduction-to-python-bsuir-2019:master
Choose a base branch
from
viktarmikhan:master
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 7 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
88ee332
Add .idea files to gitignore
7838b27
init component module. add parser load class. add few arguments for p…
0787966
add all required optional and positional argument to parser
6a3c62c
add helpers classes. add feed classes for parsing rss data. fill setu…
6f19af3
complete default parsing without json. add feed formater for provisi…
150cec4
add json feature output to rss reader
998c60b
add cli entry_point to conf
ffb9b3a
add logging module to rss_reader. add logging message to rss_reader
6d5c011
fix setup module from package. add caching mechanism without loading …
dca3b86
add helper "map" for working with dicts. modifing cache mechanism for…
b261db6
add colorize libs to requirements. add colorize output to rss-reader …
e362f8c
add initialization validation to no bool rss-reader param source, limit
05e5b43
change to-html validator exceptions. add part of convertation module …
02e1a57
modifying path in project and setup file. change db cache structurtur…
9d92b67
complete converter pdf and improve converter by jinja templates. add …
ebf3f5f
fixing json output parser. Add docstrings to several modules
97aeefd
docstring parser component
f0d5996
docstring helper and converter modules
7267b1b
Remove the ~/test directory
ed06aa8
add docstring to cache module
bc21b97
modify Redme. add Manifest. change json structure
32b3b26
just change some small tip in setup.py
5d49282
another tip in setup.py
viktarmikhan 9dd95d0
fix install bug
aa1789a
add yaml and lxml libraries to requirements to avoiding errors after …
e91eaa7
add feature to read cache rss data without internet connection. set …
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -102,3 +102,6 @@ venv.bak/ | |
|
|
||
| # mypy | ||
| .mypy_cache/ | ||
|
|
||
| #PyCharm | ||
| .idea | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| import rss_reader | ||
|
|
||
|
|
||
| def main(): | ||
| rss_reader.main() | ||
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| main() |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| import feedparser | ||
| import sys | ||
| from components.feed.feed_entry import FeedEntry | ||
| from components.feed.feed_formatter import FeedFormatter | ||
|
|
||
|
|
||
| class Feed: | ||
|
|
||
| def __init__(self, args): | ||
| self._is_json = args.json | ||
| self._limit = args.limit | ||
| self._url = args.source | ||
| self._entities_list = [] | ||
| self._feeds_title = '' | ||
|
|
||
| if args.verbose : | ||
| sys.exit('[--verbose] did\'t implement yet') | ||
|
|
||
| self._parse_feeds() | ||
|
|
||
| def show_feeds(self): | ||
| FeedFormatter.is_json = self._is_json | ||
| output = FeedFormatter.generate_output( | ||
| self._entities_list, self._limit, self._feeds_title | ||
| ) | ||
|
|
||
| print(output) | ||
|
|
||
| def _parse_feeds(self): | ||
| feeds = feedparser.parse(self._url) | ||
|
|
||
| self._set_global_feed_data(feeds.feed) | ||
|
|
||
| for feed in feeds.entries: | ||
| self._append_feed_entry(feed) | ||
|
|
||
| def _set_global_feed_data(self, feed): | ||
| self._feeds_title = feed.title | ||
|
|
||
| def _append_feed_entry(self, feed): | ||
| self._entities_list.append(FeedEntry(feed)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import html | ||
| from bs4 import BeautifulSoup | ||
|
|
||
|
|
||
| class FeedEntry: | ||
|
|
||
| def __init__(self, feed): | ||
| self.title = html.unescape(feed.title) | ||
| self.date = feed.published | ||
| self.link = feed.link | ||
| self.description = self._process_description(feed.description) | ||
| self.links = self._process_links(feed.links) | ||
|
|
||
| def _process_links(self, links): | ||
| def format_links(link, count): | ||
| return f'[{count}]: {link["href"]} ({link["type"]})\n' | ||
|
|
||
| return ''.join( | ||
| format_links(link, count) for count, link in enumerate(links, start=1) | ||
| ) | ||
|
|
||
| def _process_description(self, description): | ||
| return html.unescape( | ||
| BeautifulSoup(description, 'html.parser').get_text() | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| import json | ||
|
|
||
|
|
||
| class FeedFormatter: | ||
|
|
||
| is_json = False | ||
|
|
||
| @classmethod | ||
| def generate_output(cls, feeds, limit, title): | ||
| if not cls.is_json: | ||
| return cls._default_output(feeds, limit, title) | ||
|
|
||
| return cls._json_output(feeds, limit, title) | ||
|
|
||
| @classmethod | ||
| def _default_output(cls, feeds, limit, title): | ||
| formatted_feeds = ''.join(cls._single_feed_format_default(feed) for feed in feeds[:limit]) | ||
|
|
||
| return 'Feed: {0}\n\n{1}'.format(title, formatted_feeds) | ||
|
|
||
| @classmethod | ||
| def _json_output(cls, feeds, limit, title): | ||
| formatted_feeds = ',\n'.join(cls._single_feed_format_json(feed) for feed in feeds[:limit]) | ||
|
|
||
| #tmp | ||
| output = json.dumps({ | ||
| "title" : title, | ||
| "items" : formatted_feeds | ||
| }, indent=4, sort_keys=True) | ||
|
|
||
| return formatted_feeds | ||
|
|
||
| @classmethod | ||
| def _single_feed_format_default(self,feed): | ||
| return f'\ | ||
| \rTitle: {feed.title}\n\ | ||
| \rDate: {feed.date}\n\ | ||
| \rLink: {feed.link}\n\n\ | ||
| \r{feed.description}\n\n\ | ||
| \rLinks:\n\r{feed.links}\n\n' | ||
|
|
||
| @classmethod | ||
| def _single_feed_format_json(cls, feed): | ||
| return json.dumps({ | ||
| "item": { | ||
| "link": feed.link, | ||
| "body": { | ||
| "title": feed.title, | ||
| "date": feed.date, | ||
| "links": feed.links, | ||
| "description": feed.description | ||
| } | ||
| } | ||
| }, indent=4) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| class Singleton(object): | ||
|
|
||
| _instance = None | ||
|
|
||
| def __new__(class_, *args, **kwargs): | ||
| if not isinstance(class_._instance, class_): | ||
| class_._instance = object.__new__(class_, *args, **kwargs) | ||
|
|
||
| return class_._instance |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| from .optional.version import * | ||
| from .optional.json import * | ||
| from .optional.limit import * | ||
| from .optional.verbose import * | ||
| from .positional.source import * |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| from abc import ABC, abstractmethod | ||
|
|
||
|
|
||
| class ArgumentsAbstract(ABC): | ||
|
|
||
| # @property | ||
| # def _parser(self): | ||
| # return self._parser | ||
|
|
||
| def __init__(self, parser): | ||
| self._parser = parser | ||
|
|
||
| @abstractmethod | ||
| def add_argument(self): | ||
| pass |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| from components.parser.arguments.arguments_abstract import ArgumentsAbstract | ||
|
|
||
|
|
||
| class Json(ArgumentsAbstract): | ||
|
|
||
| def __init__(self, parser): | ||
| super().__init__(parser) | ||
|
|
||
| def add_argument(self): | ||
| self._parser.add_argument( | ||
| '--json', action='store_true', help='Print result as JSON in stdout' | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| from components.parser.arguments.arguments_abstract import ArgumentsAbstract | ||
|
|
||
|
|
||
| class Limit(ArgumentsAbstract): | ||
|
|
||
| def __init__(self, parser): | ||
| super().__init__(parser) | ||
|
|
||
| def add_argument(self): | ||
| self._parser.add_argument( | ||
| '--limit', type=int, default=3, help='Limit news topics if this parameter provided' | ||
| ) | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| from components.parser.arguments.arguments_abstract import ArgumentsAbstract | ||
|
|
||
|
|
||
| class Verbose(ArgumentsAbstract): | ||
|
|
||
| def __init__(self, parser): | ||
| super().__init__(parser) | ||
|
|
||
| def add_argument(self): | ||
| self._parser.add_argument( | ||
| '--verbose', default=False, action='store_true', help='Outputs verbose status messages' | ||
| ) | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| from components.parser.arguments.arguments_abstract import ArgumentsAbstract | ||
| import conf | ||
|
|
||
|
|
||
| class Version(ArgumentsAbstract): | ||
|
|
||
| def __init__(self, parser): | ||
| super().__init__(parser) | ||
|
|
||
| def add_argument(self): | ||
| self._parser.add_argument( | ||
| '-v', '--version', action='version', version=conf.__version__, help='Print version info' | ||
| ) |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| from components.parser.arguments.arguments_abstract import ArgumentsAbstract | ||
|
|
||
|
|
||
| class Source(ArgumentsAbstract): | ||
|
|
||
| def __init__(self, parser): | ||
| super().__init__(parser) | ||
|
|
||
| def add_argument(self): | ||
| self._parser.add_argument( | ||
| 'source', type=str, help='RSS URL' | ||
| ) | ||
|
|
||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| import argparse | ||
| import importlib | ||
|
|
||
|
|
||
| class Parser: | ||
|
|
||
| _arguments_list = ( | ||
| 'source', | ||
| 'version', | ||
| 'json', | ||
| 'verbose', | ||
| 'limit', | ||
| ) | ||
|
|
||
| @property | ||
| def parser(self): | ||
| return self._parser | ||
|
|
||
| @parser.setter | ||
| def parser(self, description): | ||
| self._parser = argparse.ArgumentParser(description) | ||
|
|
||
| def __init__(self, description, usage): | ||
| self._parser = argparse.ArgumentParser(description=description, usage=usage) | ||
| self._init_arguments() | ||
|
|
||
| def get_args(self): | ||
| return self._parser.parse_args() | ||
|
|
||
| def _init_arguments(self): | ||
| for argument in self._arguments_list: | ||
| module = importlib.import_module('components.parser.arguments') | ||
| argument_class = getattr(module, argument[0].upper() + argument[1:]) | ||
| argument_class(self._parser).add_argument() | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| __author__ = 'Mikhan Victor' | ||
| __email__ = 'victormikhan@gmail.com' | ||
| __package__ = 'rss_reader' | ||
| __version__ = '1.0.0' | ||
| __status__ = 'Prototype' | ||
| __description__ = 'RSS Reader' | ||
| __url__ = 'https://github.com/victormikhan/PythonHomework' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| from components.helper.singleton import Singleton | ||
| from components.parser.parser import Parser | ||
| from components.feed.feed import Feed | ||
|
|
||
|
|
||
| class App(Singleton): | ||
|
|
||
| def __init__(self): | ||
| console = Parser( | ||
| 'Pure Python command-line RSS reader.', | ||
| 'rss_reader.py [-h] [--version] [--json] [--verbose] [--limit LIMIT] source' | ||
| ) | ||
|
|
||
| self._console_args = console.get_args() | ||
| self._feed = Feed(self._console_args) | ||
|
|
||
| @classmethod | ||
| def start(cls): | ||
| return cls()._feed.show_feeds() | ||
|
|
||
|
|
||
| def main(): | ||
| try: | ||
| App.start() | ||
| except KeyboardInterrupt: | ||
| print(f'\nStop reader') | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| import setuptools | ||
| import conf | ||
|
|
||
| setuptools.setup( | ||
| name=conf.__package__, | ||
| version=conf.__version__, | ||
| author=conf.__author__, | ||
| author_email=conf.__email__, | ||
| description=conf.__description__, | ||
| url=conf.__url__, | ||
| packages=[conf.__package__], | ||
| python_requires='>=3.8', | ||
| entry_points={ | ||
| 'console_scripts': | ||
| ['rss-reader = cmd:main'] | ||
| } | ||
| ) | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
не получается установить пакет