Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .github/workflows/validate_descriptions.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Validate Extension Descriptions

on:
pull_request:
paths:
- 'extensions/*/description.yml'
- 'schema/description.cue'
- 'scripts/validate_descriptions.sh'
- '.github/workflows/validate_descriptions.yml'
push:
branches:
- main
paths:
- 'extensions/*/description.yml'
- 'schema/description.cue'
- 'scripts/validate_descriptions.sh'
- '.github/workflows/validate_descriptions.yml'
workflow_dispatch:

jobs:
validate:
name: Validate description.yml files
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Install CUE
uses: cue-lang/setup-cue@v1.0.1
with:
version: 'latest'

- name: Validate description.yml files
run: |
chmod +x scripts/validate_descriptions.sh
./scripts/validate_descriptions.sh

- name: Summary
if: success()
run: |
echo "✅ All description.yml files validated successfully!" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Total files validated:** $(find extensions -name 'description.yml' | wc -l)" >> $GITHUB_STEP_SUMMARY
50 changes: 50 additions & 0 deletions VALIDATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Contributing Extension Validation Guide

When adding or modifying a DuckDB community extension, your `description.yml` file must pass validation.

## Quick Start

Before submitting a PR:

```bash
# Install CUE (one-time setup)
# macOS:
brew install cue

# Linux:
curl -fsSL https://cuelang.org/install.sh | sh

# Validate your changes
./scripts/validate_descriptions.sh
```

## What Gets Validated?

All `extensions/*/description.yml` files are checked for:
- Required fields (name, description, language, build, maintainers)
- Correct data types and formats
- Valid GitHub repository references
- Proper structure and syntax

## Common Issues

| Error | Solution |
|-------|----------|
| Missing required field | Add the field (e.g., `license: MIT`) |
| Invalid GitHub format | Use `owner/repo` format in `repo.github` |
| Wrong build system | Use `cmake`, `CMake`, or `cargo` |
| Invalid version type | Quote string versions: `version: "1.0.0"` |

## Full Documentation

See [schema/README.md](schema/README.md) for:
- Complete schema reference
- Field-by-field documentation
- Example files
- Troubleshooting guide

## CI Validation

Your PR will automatically run validation. Fix any errors before the PR can be merged.

**Need help?** Check the [validation documentation](schema/README.md) or ask in your PR.
2 changes: 1 addition & 1 deletion extensions/capi_quack/description.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ extension:
description: Provides a hello world example demo from the C/C++ C API template
version: 0.0.1
language: C/C++
build: CMake
build: cmake
license: MIT
requires_toolchains: "python3"
maintainers:
Expand Down
2 changes: 1 addition & 1 deletion extensions/duckgl/description.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ extension:
- nkwork9999

repo:
github: nkwork9999/duckgl # あなたのリポジトリ名に合わせて
github: nkwork9999/duckgl
ref: 0de60a34e7bc04141fbc6b2efd6dcad6a80ac1e8

docs:
Expand Down
185 changes: 185 additions & 0 deletions schema/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
# Extension Description Validation

This repository uses [CUE](https://cuelang.org/) to validate all `description.yml` files for community extensions. This ensures consistency, catches errors early, and maintains high quality across all extension definitions.

## What is Validated?

The validation checks:
- **Required fields**: name, description, language, build, maintainers, repo information
- **Field types**: strings, numbers, lists are properly formatted
- **GitHub repository format**: valid owner/repo structure
- **Git references**: commit hashes or valid tags
- **Build system**: valid build types (cmake, CMake, cargo)
- **Consistent structure**: all extensions follow the same schema

## Schema Definition

The schema is defined in [`schema/description.cue`](../schema/description.cue) using CUE language. It specifies:

### Required Top-Level Sections
- `extension`: Extension metadata
- `repo`: Repository information
- `docs` (optional): Documentation

### Extension Fields

**Required:**
- `name`: Extension name (non-empty string)
- `description`: Brief description (non-empty string)
- `language`: Programming language (e.g., "C++", "Rust")
- `build`: Build system ("cmake", "CMake", or "cargo")
- `maintainers`: List of maintainer GitHub usernames

**Optional:**
- `version`: Version string or number
- `license`: License identifier (e.g., "MIT", "Apache-2.0")
- `excluded_platforms`: Platforms to exclude (string or list)
- `requires_toolchains`: Required toolchains (string or list)
- `vcpkg_commit`: Specific vcpkg commit hash
- And more (see schema file for complete list)

### Repository Fields

**Required:**
- `github`: Repository in "owner/repo" format
- `ref`: Git commit hash or tag

**Optional:**
- `ref_next`: Next reference for testing
- `canonical_name`: Override canonical name

### Documentation Fields

**Optional:**
- `hello_world`: Quick start example
- `extended_description`: Detailed documentation
- `docs_url`: External documentation URL

## Validating Locally

### Prerequisites

Install CUE on your system:

**macOS:**
```bash
brew install cue
```

**Linux:**
```bash
curl -fsSL https://cuelang.org/install.sh | sh
```

**From source:**
```bash
go install cuelang.org/go/cmd/cue@latest
```

### Run Validation

From the repository root:

```bash
./scripts/validate_descriptions.sh
```

This will validate all 153 `description.yml` files and report any errors.

### Validate a Single File

```bash
cue vet schema/description.cue extensions/YOUR_EXTENSION/description.yml -d "#Description"
```

## CI/CD Integration

Validation runs automatically on:
- **Pull Requests** that modify any `description.yml` file
- **Pushes** to the main branch
- **Manual trigger** via workflow_dispatch

The workflow is defined in [`.github/workflows/validate_descriptions.yml`](../.github/workflows/validate_descriptions.yml).

### Status Checks

Pull requests must pass validation before merging. If validation fails:
1. Review the error messages in the CI logs
2. Fix the reported issues in your `description.yml`
3. Push the changes to re-run validation

## Common Validation Errors

### Missing Required Field
```
extension.license: incomplete value string
```
**Fix:** Add the missing field to your `description.yml`

### Invalid GitHub Format
```
repo.github: invalid value "invalid-format" (does not match ^[a-zA-Z0-9_-]+/[a-zA-Z0-9_.-]+$)
```
**Fix:** Use format "owner/repository-name"

### Wrong Type
```
extension.version: conflicting values "1.0.0" and number
```
**Fix:** Version can be either string or number, but YAML interpretation matters. Quote strings: `version: "1.0.0"`

### Invalid Build System
```
extension.build: conflicting values "cmake" and "make"
```
**Fix:** Use one of the allowed values: "cmake", "CMake", or "cargo"

## Example Valid description.yml

```yaml
extension:
name: example
description: An example DuckDB extension
version: "1.0.0"
language: C++
build: cmake
license: MIT
maintainers:
- your-github-username

repo:
github: your-org/your-extension
ref: abc123def456... # 40-character commit hash

docs:
hello_world: |
SELECT example_function();
extended_description: |
Detailed documentation about your extension.
Can be multiple lines.
```

## Contributing

When adding or modifying extensions:

1. Ensure your `description.yml` follows the schema
2. Run validation locally before pushing
3. Address any validation errors before submitting PR
4. CI will automatically validate your changes

## Schema Updates

If you need to add new fields to the schema:

1. Update `schema/description.cue` with the new field definition
2. Update this documentation
3. Test against all existing extensions
4. Submit PR with schema changes and documentation

## Resources

- [CUE Language Documentation](https://cuelang.org/docs/)
- [CUE Tutorials](https://cuelang.org/docs/tutorials/)
- [Schema Definition](../schema/description.cue)
- [Validation Script](../scripts/validate_descriptions.sh)
Loading
Loading