Integration[Harbor]: initial integration #2283

50-Course · 2025-10-12T10:22:31Z

Description

What

Ocean's integration for GoHarbor. Allows data synchronization with Harbor's resources, currently, four major sources:

Projects
Users
Repositories
Artifacts

Why -

How

Performs data resync via connection polling, leveraging the framework's internals and the integration to work these resource kinds - effectively keeping information in-sync across both platforms.

Type of change

New Integration (non-breaking change which adds a new integration)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

Integration able to create all default resources from scratch
Resync finishes successfully
Resync able to create entities
Resync able to update entities
Resync able to detect and delete entities
Scheduled resync able to abort existing resync and start a new one
Tested with at least 2 integrations from scratch
Tested with Kafka and Polling event listeners
Tested deletion of entities that don't pass the selector

Integration testing checklist

Preflight checklist

Handled rate limiting
Handled pagination
Implemented the code in async
Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.

…a ingestion

…t.py`

This change does two things: - It wraps up the TDD approach to help us define our expected behaviors for pagination - ...and sets the setup the groundwork for our GoHarbor client

qodo-merge-pro · 2025-10-12T14:05:16Z

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
⚪	Hardcoded local path Description: Makefile contains an absolute path to a developer's local filesystem which can leak environment structure and break builds; replace with a relative path to avoid exposure. Makefile [1-1] Referred Code /home/erioluwa/projects/ocean/port_ocean/cli/cookiecutter/../../../integrations/_infra/Makefile
	SSL verification bypass Description: SSL verification flag is accepted but not enforced in the HTTP client, potentially causing requests to skip certificate verification unintentionally. client.py [80-86] Referred Code self.base_url = f"{base_url.rstrip('/')}/api/{API_VERSION}" self.username = username self.password = password self.verify_ssl = verify_ssl # do we really need SSL, let's just have it self.client, self.client.timeout = http_async_client, Timeout(DEFAULT_TIMEOUT)
	Weak auth handling Description: Basic Auth header construction is manual without using secure httpx auth mechanisms; ensure transmission occurs only over HTTPS and secrets are not logged. auth.py [4-11] Referred Code def generate_basic_auth_header(username: str, password: str) -> tuple[str, str]: """ Returns the Basic Auth header for given username and password """ credentials = f"{username}:{password}" encoded = base64.b64encode(credentials.encode('utf-8')).decode('utf-8') return 'Authorization', f'Basic {encoded}'
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
⚪	No custom compliance provided Follow the guide to enable custom compliance check.
Update

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

Copilot

Pull Request Overview

This PR introduces a new integration for GoHarbor container registry, enabling synchronization of Harbor resources into Port. The integration supports fetching projects, users, repositories, and artifacts through Harbor's v2.11.0 API.

Implements a comprehensive Harbor API client with pagination, authentication, and error handling
Provides resource synchronization for projects, users, repositories, and artifacts
Includes complete test coverage for core functionality including pagination and client operations

Reviewed Changes

Copilot reviewed 25 out of 34 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
integrations/goharbor_ocean/tests/test_pagination.py	Comprehensive test suite for pagination functionality across different Harbor resource types
integrations/goharbor_ocean/tests/test_client.py	Unit tests for Harbor client initialization and API request handling
integrations/goharbor_ocean/tests/fixtures.py	Test fixtures and mock objects for Harbor API responses
integrations/goharbor_ocean/tests/conftest.py	Test configuration setup
integrations/goharbor_ocean/harbor/client.py	Core Harbor API client implementation with async pagination and error handling
integrations/goharbor_ocean/harbor/utils/constants.py	Harbor API constants and configuration values
integrations/goharbor_ocean/harbor/utils/auth.py	Authentication utilities for Basic Auth
integrations/goharbor_ocean/harbor/schemas.py	TypedDict schemas for Harbor API response structures
integrations/goharbor_ocean/harbor/exceptions.py	Custom exception classes for Harbor integration errors
integrations/goharbor_ocean/main.py	Integration entry point with resource synchronization logic
integrations/goharbor_ocean/.port/spec.yaml	Integration specification and configuration schema
integrations/goharbor_ocean/.port/resources/blueprints.json	Port blueprints for Harbor entities
integrations/goharbor_ocean/.port/resources/port-app-config.yml	Entity mapping configuration
integrations/goharbor_ocean/pyproject.toml	Python project configuration and dependencies

Comments suppressed due to low confidence (2)

integrations/goharbor_ocean/tests/test_pagination.py:1

The pagination logic assigns 'page' to 'page_size' instead of 'page'. This will cause incorrect API requests with the page number as the page size.

import pytest

integrations/goharbor_ocean/harbor/client.py:1

Incorrect access to call_args.args - should be call_args[1] since call_args is already the args tuple.

"""

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-12T14:05:15Z

integrations/goharbor_ocean/tests/test_pagination.py

+        mock_http_response,
+        mocked_artifact_response,
+    ):
+        # should fetch artifats when both project_name and repository_name are provided


Corrected spelling of 'artifats' to 'artifacts'.

Suggested change

# should fetch artifats when both project_name and repository_name are provided

# should fetch artifacts when both project_name and repository_name are provided

Copilot · 2025-10-12T14:05:15Z

integrations/goharbor_ocean/tests/test_client.py

+    async def test_handles_server_error(
+        self, harbor_client_mocked, mock_async_client, mock_http_error
+    ):
+        # should gracefuly handle server errors


Corrected spelling of 'gracefuly' to 'gracefully'.

Suggested change

# should gracefuly handle server errors

# should gracefully handle server errors

Copilot · 2025-10-12T14:05:16Z

integrations/goharbor_ocean/harbor/client.py

+        verify_ssl: bool = False,
+    ):
+        """
+        Initalizes Harbor API client


Corrected spelling of 'Initalizes' to 'Initializes'.

Suggested change

Initalizes Harbor API client

Initializes Harbor API client

integrations/goharbor_ocean/harbor/client.py

qodo-merge-pro · 2025-10-12T14:06:29Z

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Fix incorrect pagination query parameter Fix the pagination loop by setting the `page` query parameter instead of incorrectly overwriting `page_size`. integrations/goharbor_ocean/harbor/client.py [278-286] while True: - query_params['page_size'] = page + query_params["page"] = page try: response_data = await self._send_api_request( - "GET", - endpoint, - params=query_params + "GET", endpoint, params=query_params ) Apply / Chat Suggestion importance[1-10]: 10 __ Why: This suggestion identifies a critical bug in the pagination logic where `page_size` is incorrectly used instead of `page`, which would break data synchronization.	High
	Avoid modifying shared client headers Store authentication headers in an instance variable instead of modifying the shared global `http_async_client` to prevent authentication conflicts. integrations/goharbor_ocean/harbor/client.py [85-92] -self.client, self.client.timeout = http_async_client, Timeout(DEFAULT_TIMEOUT) +self.client = http_async_client +self.client.timeout = Timeout(DEFAULT_TIMEOUT) auth_header_name, auth_header_value = generate_basic_auth_header( username, password ) -if not hasattr(self.client, "headers"): - self.client.headers = {} -self.client.headers[auth_header_name] = auth_header_value +self.auth_headers = {auth_header_name: auth_header_value} Apply / Chat Suggestion importance[1-10]: 9 __ Why: The suggestion correctly identifies a critical bug where modifying the shared `http_async_client` can cause authentication conflicts between integrations, and it proposes the correct fix.	High
	Pass authentication headers with requests Pass the instance-specific authentication headers with each API request to ensure proper authentication. integrations/goharbor_ocean/harbor/client.py [139-144] +# Get a copy of the headers to avoid modifying the original +request_headers = self.auth_headers.copy() + response = await self.client.request( method=method, url=url, params=params, json=json_data, + headers=request_headers, ) Apply / Chat Suggestion importance[1-10]: 9 __ Why: This suggestion is the necessary second part of the fix for the critical authentication bug, ensuring that the instance-specific headers are actually used in API requests.	High
	✅ ~~Handle null values in JQ~~ Suggestion Impact: The commit updated the JQ expression to use (.tags // []) \| map(.name), exactly applying the suggested null-safe handling. code diff: - tags: .tags \| map(.name) + tags: (.tags // []) \| map(.name) In the JQ mapping for `tags`, handle cases where `.tags` is `null` by providing a default empty array `[]` to prevent the `map` filter from failing. integrations/goharbor_ocean/.port/resources/port-app-config.yml [69] -tags: .tags \| map(.name) +tags: (.tags // []) \| map(.name) `[Suggestion processed]` Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies a potential mapping failure for artifacts without tags and provides a robust fix using a default empty array, preventing runtime errors during data ingestion.	Medium
Update

Co-authored-by: Copilot <[email protected]>

…ellation

github-actions bot added the size/XXL label Oct 12, 2025

50-Course added 11 commits October 12, 2025 14:58

WIP: Setup project structure, schemas and port-configurations for dat…

3ee25d8

…a ingestion

Ignore .env.example in version control

a67914a

refactor(tests): setup fixtures module and integrate with `test_clien…

ee2b4ac

…t.py`

tests: implement client tests

4682b1c

tests: implement tests for core pagination router

47f2d40

feat: implement auth credentials generator

306d67a

WIP: Implement API client for Harbor and wrap up pagination tests suites

b0f1025

This change does two things: - It wraps up the TDD approach to help us define our expected behaviors for pagination - ...and sets the setup the groundwork for our GoHarbor client

feat(harbor::client): complete pagination implementation

a3cbb21

chore: fix import errors of port_ocean http client

bd4ea79

feat(harbor::client): implement resync methods for data sync

0f1c37a

chore: update import references

7173c6b

50-Course force-pushed the feat_goharbor_port_ocean_integration branch from d7c4475 to 7173c6b Compare October 12, 2025 14:02

50-Course marked this pull request as ready for review October 12, 2025 14:04

Copilot AI review requested due to automatic review settings October 12, 2025 14:04

qodo-merge-pro bot added the Review effort 4/5 label Oct 12, 2025

Copilot AI reviewed Oct 12, 2025

View reviewed changes

50-Course added 2 commits October 12, 2025 15:17

hotfix: implement review suggestions

59b9ea4

hotfix: handle null values in JSON

a4eda10

50-Course force-pushed the feat_goharbor_port_ocean_integration branch from ddc9dee to a4eda10 Compare October 12, 2025 14:20

50-Course and others added 6 commits October 12, 2025 15:22

Merge branch 'main' into feat_goharbor_port_ocean_integration

e529883

Update integrations/goharbor_ocean/harbor/client.py

0aaec02

Co-authored-by: Copilot <[email protected]>

hotfix: implement client initialization fail-safe

fd3c738

hotfix: implement fail-safe for client init caused my subprocess canc…

6108890

…ellation

chore: remove debugger

820a3f0

hotfix: fix namecase error influencing client intialization

e32432b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Integration[Harbor]: initial integration #2283

Integration[Harbor]: initial integration #2283

Uh oh!

50-Course commented Oct 12, 2025 •

edited

Loading

Uh oh!

qodo-merge-pro bot commented Oct 12, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

Copilot AI Oct 12, 2025

Uh oh!

Uh oh!

Uh oh!

qodo-merge-pro bot commented Oct 12, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	# should fetch artifats when both project_name and repository_name are provided
	# should fetch artifacts when both project_name and repository_name are provided

	# should gracefuly handle server errors
	# should gracefully handle server errors

Integration[Harbor]: initial integration #2283

Are you sure you want to change the base?

Integration[Harbor]: initial integration #2283

Uh oh!

Conversation

50-Course commented Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

All tests should be run against the port production environment(using a testing org).

Core testing checklist

Integration testing checklist

Preflight checklist

Screenshots

API Documentation

Uh oh!

qodo-merge-pro bot commented Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Compliance Guide 🔍

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

qodo-merge-pro bot commented Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Code Suggestions ✨

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

50-Course commented Oct 12, 2025 •

edited

Loading

qodo-merge-pro bot commented Oct 12, 2025 •

edited

Loading

qodo-merge-pro bot commented Oct 12, 2025 •

edited

Loading