Adding DatasetEntity

Populate custom Dataset entity instead of generic built-in Component

Adding custom Catalog Dataset Page

Add Schema Card

Add Lineage and Quality Card

Add Support and Versioning Cards

Add Related Resources Card

Updates to mapping documentation

Update plugin versions to 0.2.0

Updates for README documentation

Author: Paul Howells

BC Data Catalog Gaps.md

@@ -1,10 +1,226 @@
-# [Backstage](https://backstage.io)
+# DevHub Backstage Integration for BC Data Catalogue Plugins
 
-This is your newly scaffolded Backstage App, Good Luck!
+This guide explains how to update the DevHub Backstage project to use the three BC Data Catalogue packages together:
 
-To start the app, run:
+- `@bcgov/plugin-catalog-backend-module-bc-data-catalogue`
+- `@bcgov/plugin-catalog-common-bc-data-catalogue`
+- `@bcgov/catalog-dataset`
+
+These packages work together to provide:
+
+- backend ingestion of BC Data Catalogue content into the Backstage catalog
+- shared dataset types and utilities
+- a frontend dataset page for rendering `Dataset` entities
+
+---
+
+## What Each Package Does
+
+### `@bcgov/plugin-catalog-backend-module-bc-data-catalogue`
+
+Provides the backend catalog module that connects to the BC Data Catalogue and creates catalog entities such as:
+
+- `Dataset`
+- `API`
+- `System`
+- `Group`
+- `User`
+
+### `@bcgov/plugin-catalog-common-bc-data-catalogue`
+
+Provides shared types, helpers, and utilities used by the backend and frontend integration.
+
+### `@bcgov/catalog-dataset`
+
+Provides the frontend dataset page and related UI components for rendering `Dataset` entities in DevHub.
+
+---
+
+## Prerequisites
+
+These packages are published to GitHub Packages and require authentication to install, even if the packages are public.
+
+### 1. Create a GitHub Personal Access Token
+
+1. Go to GitHub Settings → Developer settings → Personal access tokens → Tokens (classic)
+2. Click **Generate new token (classic)**
+3. Give it a descriptive name such as `Backstage Package Access`
+4. Select the `read:packages` scope
+5. Set a lifespan (90 days or less recommended)
+6. Authorize the token for SSO with the `bcgov` organization
+7. Copy the token
+
+### 2. Export the Token
+
+```bash
+export GITHUB_TOKEN=your_token_here
+```
+
+---
+
+## Configure Package Registry Access
+
+### `.npmrc`
 
-```sh
-yarn install
-yarn start
 ```
+@bcgov:registry=https://npm.pkg.github.com
+//npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}
+```
+
+### `.yarnrc.yaml` (if required)
+
+```yaml
+npmScopes:
+  bcgov:
+    npmRegistryServer: "https://npm.pkg.github.com"
+
+npmRegistries:
+  "https://npm.pkg.github.com":
+    npmAuthToken: "${GITHUB_TOKEN}"
+```
+
+---
+
+## Install the Packages
+
+```bash
+yarn add @bcgov/plugin-catalog-backend-module-bc-data-catalogue
+yarn add @bcgov/plugin-catalog-common-bc-data-catalogue
+yarn add @bcgov/catalog-dataset
+```
+
+---
+
+## Backend Changes
+
+### 1. Register the Backend Module
+
+Update:
+
+`packages/backend/src/index.ts`
+
+```diff
+import { createBackend } from '@backstage/backend-defaults';
+
+const backend = createBackend();
+
+// ... other backend.add() calls ...
+
++ backend.add(import('@bcgov/plugin-catalog-backend-module-bc-data-catalogue'));
+
+backend.start();
+```
+
+---
+
+### 2. Allow BC Data Catalogue API Access
+
+Update `app-config.yaml`:
+
+```yaml
+backend:
+  reading:
+    allow:
+      - host: catalogue.data.gov.bc.ca
+        scheme: https
+```
+
+---
+
+### 3. Configure Provider Schedule
+
+```yaml
+catalog:
+  providers:
+    bc-data-catalogue:
+      env: dev
+      schedule:
+        frequency: 10m
+```
+
+---
+
+## Frontend Changes
+
+The dataset plugin requires **two changes** in the DevHub frontend.
+
+---
+
+### 1. Add Dataset Route
+
+Update:
+
+`packages/app/src/App.tsx`
+
+```diff
++ import { CatalogDatasetPage } from '@bcgov/catalog-dataset';
+
+const routes = (
+  <FlatRoutes>
+    ...
++   <Route path="/catalog-dataset" element={<CatalogDatasetPage />} />
+  </FlatRoutes>
+);
+```
+
+---
+
+### 2. Register Dataset Page in Entity Switch
+
+Update:
+
+`packages/app/src/components/catalog/EntityPage.tsx`
+
+```diff
++ import { CatalogDatasetPage } from '@bcgov/catalog-dataset';
+
+<EntitySwitch>
+  <EntitySwitch.Case if={isKind('user')} children={userPage} />
+  <EntitySwitch.Case if={isKind('system')} children={systemPage} />
+  <EntitySwitch.Case if={isKind('domain')} children={domainPage} />
+
++ <EntitySwitch.Case if={isKind('dataset')} children={<CatalogDatasetPage />} />
+
+  <EntitySwitch.Case>{defaultEntityPage}</EntitySwitch.Case>
+</EntitySwitch>
+```
+
+---
+
+## Expected Result
+
+After completing all steps:
+
+- BC Data Catalogue datasets are ingested into Backstage
+- `Dataset` entities appear in the catalog
+- Selecting a dataset opens the custom dataset page
+- Dataset-specific UI (schema, lineage, support, etc.) is rendered
+
+---
+
+## Summary
+
+### Install
+
+- `@bcgov/plugin-catalog-backend-module-bc-data-catalogue`
+- `@bcgov/plugin-catalog-common-bc-data-catalogue`
+- `@bcgov/catalog-dataset`
+
+### Backend
+
+- Register backend module
+- Allow API host
+- Configure provider schedule
+
+### Frontend
+
+- Add route in `App.tsx`
+- Add EntitySwitch case in `EntityPage.tsx`
+
+---
+
+## Notes
+
+- Keep `GITHUB_TOKEN` out of source control
+- Additional hosts may need to be added to `backend.reading.allow`
+- The dataset plugin relies on all three packages working together

README.md

@@ -9,6 +9,10 @@ backend:
   reading:
     allow:
       # Allow BC Data Catalogue host access from backend
+      - host: api.gov.bc.ca
+        scheme: https
+      - host: jh-etk.api.gov.bc.ca
+        scheme: https
       - host: catalogue.data.gov.bc.ca
         scheme: https
       - host: raw.githubusercontent.com

app-config.yaml

@@ -1,13 +1,15 @@
 {
   "name": "root",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "private": true,
   "engines": {
     "node": "20 || 22"
   },
   "backstage": {
     "pluginPackages": [
-      "@bcgov/plugin-catalog-backend-module-bc-data-catalogue"
+      "@bcgov/plugin-catalog-backend-module-bc-data-catalogue",
+      "@bcgov/plugin-catalog-common-bc-data-catalogue",
+      "@bcgov/plugin-catalog-dataset"
     ]
   },
   "scripts": {
@@ -55,5 +57,8 @@
       "prettier --write"
     ]
   },
-  "packageManager": "yarn@4.4.1"
+  "packageManager": "yarn@4.4.1",
+  "dependencies": {
+    "@apidevtools/swagger-parser": "^12.1.0"
+  }
 }

design/APIsDialog.png

@@ -39,6 +39,7 @@
     "@backstage/plugin-techdocs-react": "^1.3.0",
     "@backstage/plugin-user-settings": "^0.8.23",
     "@backstage/theme": "^0.6.6",
+    "@bcgov/plugin-catalog-dataset": "workspace:^",
     "@material-ui/core": "^4.12.2",
     "@material-ui/icons": "^4.9.1",
     "react": "^18.0.2",