Skip to content

Commit 5e3baf1

Browse files
mjumbewumjumbewu
committed
feat: Add Django admin support for managing AnonymousValues
1 parent 95e04c0 commit 5e3baf1

13 files changed

Lines changed: 554 additions & 4 deletions

File tree

‎openspec/changes/archive/2026-08-21-add-anonymous-values-admin/.openspec.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
schema: spec-driven
2+
created: 2026-08-20

‎openspec/changes/archive/2026-08-21-add-anonymous-values-admin/design.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
## Context
2+
3+
`AnonymousValues` records store privacy-sensitive demographic/survey data separated from `Place` and `Submission` records. While API endpoints allow protected querying, administrators currently have no interface in Django admin to view or manage these records.
4+
5+
See `proposal.md` for motivation and `specs/` for normative requirements.
6+
7+
## Goals / Non-Goals
8+
9+
**Goals:**
10+
- Provide a full-featured `AnonymousValuesAdmin` with changelist filtering, searching, owner scoping, and JSON editing.
11+
- Provide direct navigation from `DataSetAdmin` to the dataset's anonymous values.
12+
- Support dataset cloning for `AnonymousValues` via `CloneableModelMixin` and `DataSet.clone_related`.
13+
- Ensure non-superusers can only access anonymous values for datasets they own.
14+
15+
**Non-Goals:**
16+
- Inline editing of anonymous values directly inside `DataSetAdmin` (high volume of records makes changelist filtering more performant and scalable).
17+
- Any link or association from `PlaceAdmin` or `SubmissionAdmin` to `AnonymousValues` (violates data separation principles).
18+
19+
## Decisions
20+
21+
### Decision 1: Changelist Link on DataSetAdmin vs Tabular Inline
22+
- **Chosen**: Add an `anonymous_values` readonly link field on `DataSetAdmin` that points to `/admin/sa_api_v2/anonymousvalues/?dataset={slug}`.
23+
- **Rationale**: Datasets may have thousands of anonymous records. Inlining them inside the dataset change form would significantly degrade page load performance. A filtered changelist view gives admins pagination, search, and bulk operations.
24+
- **Alternatives Considered**: Tabular inline on `DataSetAdmin` (rejected due to performance and clutter).
25+
26+
### Decision 2: Custom owner and dataset columns on AnonymousValuesAdmin
27+
- **Chosen**: Implement `owner(self, obj)` returning `obj.dataset.owner.username` and `dataset(self, obj)` returning `obj.dataset.slug` on `AnonymousValuesAdmin.list_display`.
28+
- **Rationale**: Gives administrators clear context of which user owns the dataset and which dataset slug the record belongs to at a glance.
29+
30+
### Decision 3: JSON Editing with PrettyAceWidget
31+
- **Chosen**: Use `PrettyAceWidget` with form-level JSON validation in `AnonymousValuesAdmin.get_form`.
32+
- **Rationale**: Keeps consistency with `SubmittedThingAdmin`, providing syntax highlighting, auto-formatting, and error checking for JSON blobs.
33+
34+
### Decision 4: Dataset Cloning via CloneableModelMixin
35+
- **Chosen**: Have `AnonymousValues` inherit from `CloneableModelMixin`, call `anon_val.clone(overrides={'dataset': onto})` in `DataSet.clone_related`, and add `'anonymous_values'` to `clone_related_dataset_data` prefetch list in `tasks.py`.
36+
- **Rationale**: Ensures cloned datasets created via Django admin object actions (`clone_dataset`) duplicate all related data including anonymous records with fresh UUID primary keys.
37+
38+
## Risks / Trade-offs
39+
40+
- **[Search Performance on large JSONB datasets]** → PostgreSQL handles JSONB search efficiently. The admin search is scoped by owner/dataset filters.
41+
- **[Accidental data association]** → Admin views maintain complete relational separation (no links to places or submissions).

‎openspec/changes/archive/2026-08-21-add-anonymous-values-admin/proposal.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
## Why
2+
3+
While anonymous demographic and survey values can now be submitted and retrieved via the API, administrators and dataset owners cannot currently inspect, manage, search, or clone anonymous data records within the Django admin interface. Adding Django admin support and dataset cloning integration allows administrators to manage anonymous values alongside other dataset resources without compromising the structural separation and privacy of the underlying data.
4+
5+
## What Changes
6+
7+
- **AnonymousValues Admin Registration**: Register `AnonymousValues` in Django admin with a changelist showing ID, dataset owner, dataset slug, set name, and formatted JSON data.
8+
- **Filtering & Search**: Support filtering by dataset slug (`DataSetFilter`) and `set_name`, and searching across `set_name` and `data`.
9+
- **Owner-Scoped Querysets**: Restrict non-superusers to viewing and managing anonymous values for datasets they own.
10+
- **Dataset Admin Integration**: Add an `anonymous_values` changelist link to `DataSetAdmin`'s readonly fields, linking directly to the dataset's filtered anonymous records.
11+
- **JSON Editing Widget**: Use `PrettyAceWidget` with form validation for editing anonymous JSON data blobs in the admin change form.
12+
- **Dataset Cloning Support**: Inherit `CloneableModelMixin` on `AnonymousValues`, integrate with `DataSet.clone_related`, and update background celery cloning tasks to copy anonymous records when cloning a dataset.
13+
- **Model Verbose Names**: Set `verbose_name` and `verbose_name_plural` to `"Anonymous values"` on `AnonymousValues.Meta`.
14+
15+
## Capabilities
16+
17+
### New Capabilities
18+
- `anonymous-data-admin`: Exposes `AnonymousValues` in the Django admin with changelist filtering, searching, owner-scoped access, JSON editing, and navigation links from `DataSetAdmin`.
19+
20+
### Modified Capabilities
21+
- `anonymous-data-model`: Updates `AnonymousValues` to be cloneable (`CloneableModelMixin`) and integrated into `DataSet.clone_related` when cloning datasets.
22+
23+
## Impact
24+
25+
- **Models**: `AnonymousValues` and `DataSet.clone_related` in `sa_api_v2/models/core.py`.
26+
- **Admin**: `sa_api_v2/admin.py` adding `AnonymousValuesAdmin` and updating `DataSetAdmin`.
27+
- **Celery Tasks**: `sa_api_v2/tasks.py` adding `anonymous_values` to `clone_related_dataset_data` prefetch list.
28+
- **Dependencies & DB**: No new database migrations or package dependencies required.

‎openspec/changes/archive/2026-08-21-add-anonymous-values-admin/specs/anonymous-data-admin/spec.md‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
## Purpose
2+
3+
Provides Django admin interface integration for AnonymousValues, allowing administrators and dataset owners to inspect, filter, search, view, edit, and navigate to anonymous records.
4+
5+
## ADDED Requirements
6+
7+
### Requirement: AnonymousValues model admin registration and changelist display
8+
The system SHALL register `AnonymousValues` in the Django admin site via `AnonymousValuesAdmin`. The changelist view SHALL display the following columns:
9+
- `id`: UUID of the record
10+
- `owner`: username of the dataset's owner
11+
- `dataset`: slug of the associated dataset
12+
- `set_name`: logical submission set name (e.g. `"places"`, `"comments"`)
13+
- `data`: formatted JSON content of the anonymous data blob
14+
15+
#### Scenario: Admin changelist shows anonymous record columns
16+
- **WHEN** an admin navigates to the `AnonymousValues` changelist in Django admin
17+
- **THEN** each row SHALL display the record's UUID, the dataset owner's username, the dataset slug, the set name, and the data blob
18+
19+
### Requirement: Changelist filtering and search
20+
The `AnonymousValuesAdmin` SHALL support filtering and full-text searching:
21+
- Filtering by dataset slug via `DataSetFilter`
22+
- Filtering by `set_name`
23+
- Searching by `set_name` and `data` content
24+
25+
#### Scenario: Filter anonymous values by dataset slug
26+
- **WHEN** an admin filters by a dataset slug in the `AnonymousValues` changelist
27+
- **THEN** only `AnonymousValues` belonging to that dataset SHALL be displayed
28+
29+
#### Scenario: Filter anonymous values by set_name
30+
- **WHEN** an admin filters by a `set_name` (e.g. `"comments"`) in the `AnonymousValues` changelist
31+
- **THEN** only `AnonymousValues` with that `set_name` SHALL be displayed
32+
33+
#### Scenario: Search anonymous values by data content
34+
- **WHEN** an admin searches for a keyword that appears in `data` (e.g. `"Asian"` or `"25-34"`)
35+
- **THEN** matching `AnonymousValues` records SHALL be returned in search results
36+
37+
### Requirement: Owner-scoped access for non-superusers
38+
For users who are not superusers, `AnonymousValuesAdmin` SHALL restrict the queryset to records where `dataset__owner=request.user`.
39+
40+
#### Scenario: Superuser sees all anonymous records
41+
- **WHEN** a superuser accesses the `AnonymousValues` admin changelist
42+
- **THEN** all `AnonymousValues` across all datasets SHALL be visible
43+
44+
#### Scenario: Non-superuser sees only owned dataset anonymous records
45+
- **WHEN** an authenticated non-superuser accesses the `AnonymousValues` admin changelist
46+
- **THEN** only `AnonymousValues` belonging to datasets owned by that user SHALL be visible
47+
48+
### Requirement: Change form with JSON editing
49+
The `AnonymousValuesAdmin` change and add form SHALL allow viewing and editing `AnonymousValues` fields:
50+
- `id` SHALL be a read-only field
51+
- `dataset` SHALL use a raw ID lookup field (`raw_id_fields`)
52+
- `data` SHALL use a JSON editing widget (`PrettyAceWidget`) with JSON syntax validation on save
53+
54+
#### Scenario: Editing anonymous values in admin form
55+
- **WHEN** an admin edits an `AnonymousValues` record with valid JSON in the `data` field
56+
- **THEN** the changes SHALL be validated and saved to the database
57+
58+
#### Scenario: Invalid JSON in data field is rejected
59+
- **WHEN** an admin submits invalid JSON in the `data` field
60+
- **THEN** form validation SHALL raise an error and prevent saving
61+
62+
### Requirement: Dataset admin navigation link
63+
`DataSetAdmin` SHALL include an `anonymous_values` read-only field that renders an HTML link pointing to the `AnonymousValues` changelist filtered by that dataset's slug (`/admin/sa_api_v2/anonymousvalues/?dataset={slug}`).
64+
65+
#### Scenario: Dataset change form renders anonymous values link
66+
- **WHEN** an admin views a `DataSet` change form in Django admin
67+
- **THEN** the `anonymous_values` read-only field SHALL render a link to `/admin/sa_api_v2/anonymousvalues/?dataset={slug}`

‎openspec/changes/archive/2026-08-21-add-anonymous-values-admin/specs/anonymous-data-model/spec.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
## ADDED Requirements
2+
3+
### Requirement: AnonymousValues is cloneable with datasets
4+
`AnonymousValues` SHALL implement model cloning via `CloneableModelMixin`. When a `DataSet` is cloned, all `AnonymousValues` associated with the source dataset SHALL be cloned onto the new dataset with a new UUID primary key.
5+
6+
#### Scenario: Dataset clone copies anonymous values
7+
- **WHEN** a dataset with associated `AnonymousValues` records is cloned
8+
- **THEN** matching `AnonymousValues` records SHALL be created for the new dataset with identical `set_name` and `data` blobs, each assigned a distinct auto-generated UUID
9+
10+
### Requirement: Model verbose naming
11+
The `AnonymousValues` model Meta SHALL define `verbose_name` and `verbose_name_plural` both as `"Anonymous values"`.
12+
13+
#### Scenario: Model Meta verbose names
14+
- **WHEN** the model Meta options for `AnonymousValues` are evaluated
15+
- **THEN** `verbose_name` SHALL be `"Anonymous values"`
16+
- **AND** `verbose_name_plural` SHALL be `"Anonymous values"`

‎openspec/changes/archive/2026-08-21-add-anonymous-values-admin/tasks.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
## 1. Model & Cloning Integration
2+
3+
- [x] 1.1 Inherit `CloneableModelMixin` on `AnonymousValues` and set `verbose_name = 'Anonymous values'` and `verbose_name_plural = 'Anonymous values'` in `AnonymousValues.Meta`.
4+
- [x] 1.2 Update `DataSet.clone_related` in `sa_api_v2/models/core.py` to clone all associated `anonymous_values` onto the target dataset.
5+
- [x] 1.3 Update `clone_related_dataset_data` in `sa_api_v2/tasks.py` to include `'anonymous_values'` in the queryset `prefetch_related` list.
6+
- [x] 1.4 Write unit tests for `AnonymousValues` cloning and `DataSet.clone_related` integration.
7+
8+
## 2. AnonymousValues Admin
9+
10+
- [x] 2.1 Create `AnonymousValuesAdmin` class in `sa_api_v2/admin.py` with `list_display = ('id', 'owner', 'dataset', 'set_name', 'data')` and custom column helper methods.
11+
- [x] 2.2 Configure `list_filter = ('set_name', DataSetFilter)` and `search_fields = ('set_name', 'data')` on `AnonymousValuesAdmin`.
12+
- [x] 2.3 Configure `raw_id_fields = ('dataset',)` and `readonly_fields = ('id',)` on `AnonymousValuesAdmin`.
13+
- [x] 2.4 Implement `get_queryset` on `AnonymousValuesAdmin` to restrict non-superusers to `dataset__owner=request.user`.
14+
- [x] 2.5 Implement `get_form` on `AnonymousValuesAdmin` with `PrettyAceWidget(mode='json', ...)` and JSON validation on the `data` field.
15+
- [x] 2.6 Register `AnonymousValues` with `admin.site.register(models.AnonymousValues, AnonymousValuesAdmin)`.
16+
17+
## 3. DataSet Admin Navigation
18+
19+
- [x] 3.1 Add `anonymous_values(self, instance)` helper method on `DataSetAdmin` to render an HTML link to `/admin/sa_api_v2/anonymousvalues/?dataset={slug}`.
20+
- [x] 3.2 Add `'anonymous_values'` to `DataSetAdmin.readonly_fields`.
21+
22+
## 4. Automated Tests
23+
24+
- [x] 4.1 Write admin tests covering:
25+
- `AnonymousValuesAdmin` changelist rendering, custom columns (`owner`, `dataset`), filters, and search
26+
- `AnonymousValuesAdmin.get_queryset` scoping for superusers vs dataset owners
27+
- `AnonymousValuesAdmin` form validation rejecting invalid JSON
28+
- `DataSetAdmin` change form rendering the `anonymous_values` link
29+
- Dataset cloning duplicating all associated `AnonymousValues`

‎openspec/specs/anonymous-data-admin/spec.md‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# anonymous-data-admin Specification
2+
3+
## Purpose
4+
5+
Provides Django admin interface integration for AnonymousValues, allowing administrators and dataset owners to inspect, filter, search, view, edit, and navigate to anonymous records.
6+
7+
## Requirements
8+
9+
### Requirement: AnonymousValues model admin registration and changelist display
10+
The system SHALL register `AnonymousValues` in the Django admin site via `AnonymousValuesAdmin`. The changelist view SHALL display the following columns:
11+
- `id`: UUID of the record
12+
- `owner`: username of the dataset's owner
13+
- `dataset`: slug of the associated dataset
14+
- `set_name`: logical submission set name (e.g. `"places"`, `"comments"`)
15+
- `data`: formatted JSON content of the anonymous data blob
16+
17+
#### Scenario: Admin changelist shows anonymous record columns
18+
- **WHEN** an admin navigates to the `AnonymousValues` changelist in Django admin
19+
- **THEN** each row SHALL display the record's UUID, the dataset owner's username, the dataset slug, the set name, and the data blob
20+
21+
### Requirement: Changelist filtering and search
22+
The `AnonymousValuesAdmin` SHALL support filtering and full-text searching:
23+
- Filtering by dataset slug via `DataSetFilter`
24+
- Filtering by `set_name`
25+
- Searching by `set_name` and `data` content
26+
27+
#### Scenario: Filter anonymous values by dataset slug
28+
- **WHEN** an admin filters by a dataset slug in the `AnonymousValues` changelist
29+
- **THEN** only `AnonymousValues` belonging to that dataset SHALL be displayed
30+
31+
#### Scenario: Filter anonymous values by set_name
32+
- **WHEN** an admin filters by a `set_name` (e.g. `"comments"`) in the `AnonymousValues` changelist
33+
- **THEN** only `AnonymousValues` with that `set_name` SHALL be displayed
34+
35+
#### Scenario: Search anonymous values by data content
36+
- **WHEN** an admin searches for a keyword that appears in `data` (e.g. `"Asian"` or `"25-34"`)
37+
- **THEN** matching `AnonymousValues` records SHALL be returned in search results
38+
39+
### Requirement: Owner-scoped access for non-superusers
40+
For users who are not superusers, `AnonymousValuesAdmin` SHALL restrict the queryset to records where `dataset__owner=request.user`.
41+
42+
#### Scenario: Superuser sees all anonymous records
43+
- **WHEN** a superuser accesses the `AnonymousValues` admin changelist
44+
- **THEN** all `AnonymousValues` across all datasets SHALL be visible
45+
46+
#### Scenario: Non-superuser sees only owned dataset anonymous records
47+
- **WHEN** an authenticated non-superuser accesses the `AnonymousValues` admin changelist
48+
- **THEN** only `AnonymousValues` belonging to datasets owned by that user SHALL be visible
49+
50+
### Requirement: Change form with JSON editing
51+
The `AnonymousValuesAdmin` change and add form SHALL allow viewing and editing `AnonymousValues` fields:
52+
- `id` SHALL be a read-only field
53+
- `dataset` SHALL use a raw ID lookup field (`raw_id_fields`)
54+
- `data` SHALL use a JSON editing widget (`PrettyAceWidget`) with JSON syntax validation on save
55+
56+
#### Scenario: Editing anonymous values in admin form
57+
- **WHEN** an admin edits an `AnonymousValues` record with valid JSON in the `data` field
58+
- **THEN** the changes SHALL be validated and saved to the database
59+
60+
#### Scenario: Invalid JSON in data field is rejected
61+
- **WHEN** an admin submits invalid JSON in the `data` field
62+
- **THEN** form validation SHALL raise an error and prevent saving
63+
64+
### Requirement: Dataset admin navigation link
65+
`DataSetAdmin` SHALL include an `anonymous_values` read-only field that renders an HTML link pointing to the `AnonymousValues` changelist filtered by that dataset's slug (`/admin/sa_api_v2/anonymousvalues/?dataset={slug}`).
66+
67+
#### Scenario: Dataset change form renders anonymous values link
68+
- **WHEN** an admin views a `DataSet` change form in Django admin
69+
- **THEN** the `anonymous_values` read-only field SHALL render a link to `/admin/sa_api_v2/anonymousvalues/?dataset={slug}`

‎openspec/specs/anonymous-data-model/spec.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,3 +60,18 @@ The `set_name` field SHALL use the same values as `Submission.set_name` for subm
6060
#### Scenario: Anonymous data from submission creation uses submission's set_name
6161
- **WHEN** anonymous data is created as part of a submission to the `"comments"` submission set
6262
- **THEN** the `AnonymousValues.set_name` SHALL be `"comments"`
63+
64+
### Requirement: AnonymousValues is cloneable with datasets
65+
`AnonymousValues` SHALL implement model cloning via `CloneableModelMixin`. When a `DataSet` is cloned, all `AnonymousValues` associated with the source dataset SHALL be cloned onto the new dataset with a new UUID primary key.
66+
67+
#### Scenario: Dataset clone copies anonymous values
68+
- **WHEN** a dataset with associated `AnonymousValues` records is cloned
69+
- **THEN** matching `AnonymousValues` records SHALL be created for the new dataset with identical `set_name` and `data` blobs, each assigned a distinct auto-generated UUID
70+
71+
### Requirement: Model verbose naming
72+
The `AnonymousValues` model Meta SHALL define `verbose_name` and `verbose_name_plural` both as `"Anonymous values"`.
73+
74+
#### Scenario: Model Meta verbose names
75+
- **WHEN** the model Meta options for `AnonymousValues` are evaluated
76+
- **THEN** `verbose_name` SHALL be `"Anonymous values"`
77+
- **AND** `verbose_name_plural` SHALL be `"Anonymous values"`

0 commit comments

Comments
 (0)