update docs

Author: eepMoody

.cursor

@@ -46,29 +46,60 @@ python manage.py migrate                # Apply migrations
 python manage.py collectstatic         # Static files
 ```
 
+### Search Index Management
+The search system uses three indexes: SQLite FTS5, Whoosh (Haystack), and spaCy vector embeddings.
+
+```bash
+# Rebuild all search indexes (2-3 minutes, use for CI/deployment)
+pipenv run python manage.py quickindex
+
+# Lower-level commands:
+pipenv run python manage.py buildindex --v1 --v2  # Build indexes from scratch
+pipenv run python manage.py indexctl pack         # Pack indexes for distribution
+pipenv run python manage.py indexctl unpack       # Unpack indexes after clone
+```
+
+- **quickindex**: Convenience command that runs `buildindex --v1 --v2` then `indexctl pack`
+- Packed indexes stored in `search/indexes/` (committed to repo)
+- After running `quickindex`, commit `search/indexes/` and `search/index_manifest.json`
+- GitHub Action `rebuild_search_index.yml` can trigger index rebuilds
+
 ### Fixture Data Loading
 - Fixtures are automatically loaded by quicksetup
 - Data format: JSON files with Django fixture format
 - v2 fixtures in `data/v2/publisher/document/` structure
+- Each document folder contains model-specific JSON files (e.g., `Spell.json`, `SpellCastingOption.json`)
 
-## Testing System (APPROVAL TESTS)
+### Spell Data Structure
+- `Spell.json` - Core spell definitions with `higher_level` text field
+- `SpellCastingOption.json` - Structured casting options (slot levels, damage scaling, etc.)
+- **Rule**: Every spell with `higher_level` text must have corresponding entries in `SpellCastingOption.json`
+- Casting option types: `default`, `slot_level_1` through `slot_level_9`, `player_level_5/11/17` for cantrips
 
-### How Approval Tests Work
-The project uses **approval testing** - tests compare actual API responses against pre-approved JSON files.
+## Testing System
 
-#### Test Structure
+### Test Types
+1. **Approval Tests** - Compare API responses against pre-approved JSON files
+2. **Data Integrity Tests** - Validate relationships and completeness of fixture data
+
+### Test Structure
 - Tests in: `api_v2/tests/test_objects.py`
 - Approved responses: `api_v2/tests/responses/*.approved.json`
 - Failed test responses: `api_v2/tests/responses/*.received.json`
 
-#### Running Tests
+### Running Tests
 ```bash
 # Requires running server for integration tests
-python manage.py runserver 8000 &
-python -m pytest api_v2/tests/test_objects.py -v
+pipenv run python manage.py runserver 8000 &
+pipenv run pytest api_v2/tests/ -v
 pkill -f "python manage.py runserver"  # Stop server
 ```
 
+### Data Integrity Tests (`TestSpellCastingOptions`)
+- `test_all_spells_with_higher_level_have_casting_options` - Validates every spell with `higher_level` text has casting options
+- `test_no_duplicate_casting_option_types` - Ensures no spell has duplicate casting option types
+- **When adding new spells**: If the spell has `higher_level` text, you must also add entries to `SpellCastingOption.json`
+
 #### Updating Test Expectations (IMPORTANT PATTERN)
 When API responses change (like adding new fields):
 
@@ -160,10 +191,13 @@ pkill -f "python manage.py runserver"
 
 ```bash
 # Full development reset
-python manage.py quicksetup --clean --noindex
+pipenv run python manage.py quicksetup --clean --noindex
+
+# Run all tests
+pipenv run pytest api_v2/tests/ -v
 
 # Run specific test
-python -m pytest api_v2/tests/test_objects.py::TestObjects::test_document_example -v
+pipenv run pytest api_v2/tests/test_objects.py::TestObjects::test_document_example -v
 
 # Find fixture files
 find data/v2/ -name "*.json" | grep -i document
@@ -173,6 +207,7 @@ curl -s "http://localhost:8000/v2/documents/srd-2024/" | python -m json.tool
 
 # Update all failing tests at once
 cd api_v2/tests/responses/ && for file in *.received.json; do mv "$file" "${file%.received.json}.approved.json"; done
+
 ```
 
 ## Project Conventions

README.md

@@ -124,19 +124,21 @@ pipenv run python manage.py spectacular --color --file openapi-schema.yml` to bu
 ```
 
 # Contributing
-See [contribution guide](CONTRIBUTING.md).
+
+We welcome contributions! Please join our [Discord](https://discord.gg/9RNE2rY) to coordinate with the team, or check out the [issue board](https://github.com/open5e/open5e-api/issues) to see what's being worked on.
 # Tests
 
-Tests are located in the `api/tests` directory. These should be run before pushing new changes to the main repository. These tests require that the api is [running](##run) at `http://localhost:8000`.
+Tests are located in `api/tests` and `api_v2/tests`. Run them before pushing new changes. Tests require the API to be [running](##run) at `http://localhost:8000`.
 
 ```bash
 pipenv run pytest
 ```
 
 ## Approval tests
-Approval tests are run against the approved files in `api/tests/approved_files` as `*.approved.*` . If a test fails then the recieved input will be stored in a `*.recieved.*` file. If you wish to approve the changes, replace the old approved file with the recieved file.
+Approval tests compare API responses against pre-approved JSON files in `api_v2/tests/responses/*.approved.json`. If a test fails, the received response is saved as `*.received.json`. To approve changes, rename the received file to replace the approved file.
+
+Received files should not be committed to git.
 
-Recieved files shall not be included in the git repo.
 
 # Deployment