Add GitHub OSV Live V2 Importer Pipeline #1977
Conversation
* Add GitHub OSV Live V2 Importer * Add tests for the GitHub OSV Live V2 Importer * Tested functionally using the Live Evaluation API in #1969 Signed-off-by: Michael Ehab Mikhail <[email protected]>
Signed-off-by: Michael Ehab Mikhail <[email protected]>
michaelehab
changed the title
There was a problem hiding this comment.
@michaelehab Nice work! Just a few nits for your consideration.
| if resp.status_code != 200: | ||
| return [] |
There was a problem hiding this comment.
| if resp.status_code != 200: | |
| return [] | |
| response.raise_for_status() |
| date_str = adv.get("published") or adv.get("modified") | ||
|
|
||
| if date_str: | ||
| from datetime import datetime |
There was a problem hiding this comment.
Keep all imports at the top, unless it’s a heavy library that is only used once. In that case, a local import is acceptable.
| from datetime import datetime | |
| except Exception: | ||
| pass |
There was a problem hiding this comment.
Please use specific exceptions for clarity and safety
| return purl.name | ||
|
|
||
|
|
||
| def fetch_github_osv_advisories_for_purl(purl: PackageURL): |
There was a problem hiding this comment.
Simple and concise
| def fetch_github_osv_advisories_for_purl(purl: PackageURL): | |
| def fetch_osv_advisories(purl: PackageURL): |
| pkg = {"ecosystem": ecosystem, "name": _osv_package_name(purl)} | ||
| # Query by package to get all advisories for that package; we filter GHSA below. | ||
| body = {"package": pkg} |
There was a problem hiding this comment.
| pkg = {"ecosystem": ecosystem, "name": _osv_package_name(purl)} | |
| # Query by package to get all advisories for that package; we filter GHSA below. | |
| body = {"package": pkg} | |
| body = {"package": {"ecosystem": ecosystem, "name": _osv_package_name(purl)}} |
|
|
||
| for adv in self.advisories: | ||
| adv_id = adv.get("id") | ||
| advisory_url = build_github_repo_advisory_url(adv, adv_id) |
There was a problem hiding this comment.
Use consistent naming: choose either full names or abbreviations, but stay consistent
| advisory_url = build_github_repo_advisory_url(adv, adv_id) | |
| adv_url = build_github_repo_advisory_url(adv, adv_id) |
| } | ||
|
|
||
|
|
||
| def build_github_repo_advisory_url(adv: dict, adv_id: Optional[str]) -> str: |
There was a problem hiding this comment.
Please add some unit tests for the function build_github_repo_advisory_url.
| from datetime import datetime | ||
|
|
||
| try: | ||
| dt = datetime.fromisoformat(date_str.replace("Z", "+00:00")) |
There was a problem hiding this comment.
Did you try using dateparser? I think it can simplify the logic there.
Add GitHub OSV Live V2 Importer
Add tests for the GitHub OSV Live V2 Importer
Tested functionally using the Live Evaluation API in Add Live Evaluation API endpoint and PyPa live pipeline importer #1969