Skip to content

Fix steward review flow and submission search #744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JoaquinBN merged 5 commits into from
Jun 8, 2026
Merged

Fix steward review flow and submission search #744

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4988a60
Fix steward review flow and search
JoaquinBN Jun 8, 2026
6ee4536
Address steward review PR feedback
JoaquinBN Jun 8, 2026
8b290b4
Save direct builder rubric reviews
JoaquinBN Jun 8, 2026
375e280
Require direct builder rubric reviews
JoaquinBN Jun 8, 2026
557ad83
Clear stale builder rubric reviews
JoaquinBN Jun 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions backend/contributions/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -727,9 +727,10 @@ def __str__(self):

class ProjectMilestoneReview(BaseModel):
"""
Latest structured Builder Project rubric proposal for a submitted
contribution. The existing SubmittedContribution proposed_* fields remain
the queue summary; this record is the detailed rubric source of truth.
Latest structured Builder Project rubric review for a submitted
contribution. This can come from a proposal or a final steward decision.
The existing SubmittedContribution proposed_* fields remain the queue
summary; this record is the detailed rubric source of truth.
"""
Comment on lines 728 to 734

@coderabbitai coderabbitai Bot Jun 8, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify whether backend/CLAUDE.md changed in this PR and mentions ProjectMilestoneReview/rubric flow context.
git diff --name-only origin/dev...HEAD | sed -n '/^backend\/CLAUDE\.md$/p'
git diff origin/dev...HEAD -- backend/CLAUDE.md | rg -n "ProjectMilestoneReview|rubric|builder_project|steward"

Repository: genlayer-foundation/points

Length of output: 150

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Remotes =="
git remote -v || true
echo
echo "== Branches =="
git branch -a || true
echo

echo "== Candidate base refs merge-base with HEAD =="
candidates=(origin/dev origin/main origin/master dev main master HEAD~1)
for base in "${candidates[@]}"; do
  if git rev-parse --verify "${base}" >/dev/null 2>&1; then
    if mb=$(git merge-base "${base}" HEAD 2>/dev/null); then
      echo "OK: ${base} merge-base=${mb}"
      # only check diff for backend/CLAUDE.md to keep output small
      names=$(git diff --name-only "${base}"...HEAD -- backend/CLAUDE.md || true)
      if [[ -n "${names}" ]]; then
        echo "CHANGED FILES: ${names}"
        echo
        echo "== Diff hits for rubric-flow terms in backend/CLAUDE.md =="
        git diff "${base}"...HEAD -- backend/CLAUDE.md | rg -n "ProjectMilestoneReview|rubric|builder_project|steward" || true
      else
        echo "backend/CLAUDE.md not changed vs ${base}"
      fi
      echo
    else
      echo "FAIL: ${base} no merge-base"
    fi
  else
    echo "SKIP: ${base} not present"
  fi
done

echo "== Current backend/CLAUDE.md contains terms? =="
rg -n "ProjectMilestoneReview|rubric|builder_project|steward" backend/CLAUDE.md || true

Repository: genlayer-foundation/points

Length of output: 1063

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Term search in backend/CLAUDE.md =="
rg -n "ProjectMilestoneReview|rubric|milestone|builder_project|milestone review|steward review" backend/CLAUDE.md || true

echo
echo "== Was backend/CLAUDE.md changed in the latest commit (HEAD^..HEAD)? =="
git diff --name-only HEAD^..HEAD -- backend/CLAUDE.md || true

echo
echo "== If multi-commit, changed in last 5 commits? =="
# show changed file names for each commit (relative to its parent)
for c in $(git rev-list --max-count=5 --reverse HEAD); do
  echo "-- $c --"
  git diff --name-only "${c}^..${c}" -- backend/CLAUDE.md | sed 's/^/  /'
done

Repository: genlayer-foundation/points

Length of output: 636

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== git status =="
git status -sb || true
echo

echo "== HEAD info =="
git rev-parse HEAD || true
echo

echo "== Does HEAD^ exist? =="
if git rev-parse HEAD^ >/dev/null 2>&1; then
  echo "HEAD^ exists"
else
  echo "HEAD^ missing (shallow/single commit)"
fi
echo

echo "== Files changed in current HEAD commit (if any) =="
git show --name-only --pretty='' HEAD -- backend/CLAUDE.md || true
echo

echo "== backend/CLAUDE.md: context around line ~60-120 =="
sed -n '50,120p' backend/CLAUDE.md || true
echo

echo "== backend/CLAUDE.md: context around line ~240-310 =="
sed -n '230,310p' backend/CLAUDE.md || true
echo

echo "== Confirm term presence/absence in backend/CLAUDE.md =="
rg -n "ProjectMilestoneReview|builder_project|rubric|milestone review|milestone" backend/CLAUDE.md || true

Repository: genlayer-foundation/points

Length of output: 8295

Ensure backend/CLAUDE.md reflects the ProjectMilestoneReview rubric-flow semantics
backend/CLAUDE.md was updated in this change, but it only documents the AI Review Agent and “Steward Submissions” endpoints; it doesn’t mention ProjectMilestoneReview (or the rubric/builder-to-steward flow clarification). Update backend/CLAUDE.md to explicitly describe how ProjectMilestoneReview fits into the submitted contribution/rubric source-of-truth workflow.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@backend/contributions/models.py` around lines 728 - 734, Update
backend/CLAUDE.md to document the new ProjectMilestoneReview rubric-flow by
describing that ProjectMilestoneReview is the canonical structured rubric record
for a submitted contribution (superseding the SubmittedContribution's proposed_*
queue summary), explain how it can be produced by the AI Review Agent or by a
steward decision, and outline the flow from builder rubric ->
ProjectMilestoneReview creation -> usage by steward submissions endpoints
(include references to the ProjectMilestoneReview model name and
SubmittedContribution proposed_* fields and mention where consumers should read
the record as the source-of-truth); keep the wording concise and add one example
request/response or sequence bullet showing the transition from proposal to
ProjectMilestoneReview to final steward decision.

Source: Coding guidelines

ACTION_CHOICES = [
('accept', 'Accept'),
Expand Down
13 changes: 9 additions & 4 deletions backend/contributions/rubric_review.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,10 +102,15 @@ def _normalize_score(value, section_key):
return score


def normalize_rubric_review_payload(payload, proposed_action):
def normalize_rubric_review_payload(
payload,
proposed_action,
require_overall_reason=True,
action_field='proposed_action',
):
if payload is None:
raise serializers.ValidationError({
'rubric_review': 'Rubric review is required for Builder Project proposals.'
'rubric_review': 'Rubric review is required for Builder Project reviews.'
})
if not isinstance(payload, dict):
raise serializers.ValidationError({
Expand All @@ -123,15 +128,15 @@ def normalize_rubric_review_payload(payload, proposed_action):
'extras',
)
overall_reason = str(payload.get('overall_reason') or '').strip()
if not overall_reason:
if require_overall_reason and not overall_reason:
raise serializers.ValidationError({
'overall_reason': 'Overall reason is required.'
})

if gate_failures:
if proposed_action != 'reject':
raise serializers.ValidationError({
'proposed_action': 'Gate failures must be submitted as reject proposals.'
action_field: 'Gate failures must be submitted as reject reviews.'
})
return {
'gate_failures': gate_failures,
Expand Down
24 changes: 24 additions & 0 deletions backend/contributions/serializers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -895,12 +895,36 @@ class StewardSubmissionReviewSerializer(serializers.Serializer):
template_id = serializers.PrimaryKeyRelatedField(
queryset=ReviewTemplate.objects.all(), required=False, allow_null=True,
)
rubric_review = serializers.JSONField(required=False)

def validate(self, data):
"""Validate the review action and required fields."""
action = data.get('action')
submission = self.context.get('submission')
request = self.context.get('request')
current_contribution_type = submission.contribution_type if submission else None
effective_contribution_type = (
data.get('contribution_type')
if action == 'accept'
else current_contribution_type
) or current_contribution_type
current_requires_rubric = uses_project_rubric(current_contribution_type)
requires_rubric = uses_project_rubric(effective_contribution_type)
requires_direct_rubric = requires_rubric and action in ['accept', 'reject']
if requires_direct_rubric or 'rubric_review' in data:
if requires_rubric:
data['rubric_review'] = normalize_rubric_review_payload(
data.get('rubric_review'),
action,
require_overall_reason=False,
action_field='action',
)
elif current_requires_rubric:
data.pop('rubric_review', None)
else:
raise serializers.ValidationError({
'rubric_review': 'Rubric review is only accepted for Builder Project reviews.'
})

if action == 'accept':
if 'points' not in data or data.get('points') is None:
Expand Down
207 changes: 207 additions & 0 deletions backend/contributions/tests/test_project_milestone_rubric.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,213 @@ def test_project_accept_proposal_allows_no_points(self):
self.assertIsNotNone(note)
self.assertNotIn('points**', note.message)

def test_direct_project_accept_and_reject_require_rubric_review(self):
for action in ['accept', 'reject']:
submission = self.create_submission()
data = {'action': action}
if action == 'accept':
data.update({
'points': 75,
'contribution_type': self.project_type.id,
'user': self.submitter.id,
})
else:
data['staff_reply'] = 'Not enough project evidence.'

with self.subTest(action=action):
response = self.client.post(
f'/api/v1/steward-submissions/{submission.id}/review/',
data=data,
content_type='application/json',
)

self.assertEqual(response.status_code, 400)
self.assertIn('rubric_review', response.data)
self.assertFalse(ProjectMilestoneReview.objects.filter(submitted_contribution=submission).exists())

def test_direct_standard_accept_does_not_require_rubric_review(self):
submission = self.create_submission(self.standard_type)

response = self.client.post(
f'/api/v1/steward-submissions/{submission.id}/review/',
data={
'action': 'accept',
'points': 5,
'contribution_type': self.standard_type.id,
'user': self.submitter.id,
},
content_type='application/json',
)

self.assertEqual(response.status_code, 200)
submission.refresh_from_db()
self.assertEqual(submission.state, 'accepted')
self.assertFalse(ProjectMilestoneReview.objects.filter(submitted_contribution=submission).exists())

def test_direct_standard_submission_reclassified_to_project_requires_and_stores_rubric(self):
submission = self.create_submission(self.standard_type)

missing_rubric_response = self.client.post(
f'/api/v1/steward-submissions/{submission.id}/review/',
data={
'action': 'accept',
'points': 75,
'contribution_type': self.project_type.id,
'user': self.submitter.id,
},
content_type='application/json',
)

self.assertEqual(missing_rubric_response.status_code, 400)
self.assertIn('rubric_review', missing_rubric_response.data)

response = self.client.post(
f'/api/v1/steward-submissions/{submission.id}/review/',
data={
'action': 'accept',
'points': 75,
'contribution_type': self.project_type.id,
'user': self.submitter.id,
'rubric_review': rubric_payload(overall_reason=''),
},
content_type='application/json',
)

self.assertEqual(response.status_code, 200)
submission.refresh_from_db()
self.assertEqual(submission.state, 'accepted')
self.assertEqual(submission.contribution_type, self.project_type)
review = ProjectMilestoneReview.objects.get(submitted_contribution=submission)
self.assertEqual(review.action, 'accept')
self.assertEqual(review.review_flow, ContributionType.REVIEW_FLOW_BUILDER_PROJECT)
self.assertEqual(response.data['rubric_review']['sections'], review.sections)

def test_direct_project_more_info_clears_stale_proposal_rubric_review(self):
submission = self.create_submission()
self.client.post(
f'/api/v1/steward-submissions/{submission.id}/propose/',
data={
'proposed_action': 'accept',
'proposed_contribution_type': self.project_type.id,
'proposed_user': self.submitter.id,
'rubric_review': rubric_payload(),
},
content_type='application/json',
)
self.assertTrue(ProjectMilestoneReview.objects.filter(submitted_contribution=submission).exists())

response = self.client.post(
f'/api/v1/steward-submissions/{submission.id}/review/',
data={
'action': 'more_info',
'staff_reply': 'Please add clearer milestone evidence.',
},
content_type='application/json',
)

self.assertEqual(response.status_code, 200)
submission.refresh_from_db()
self.assertEqual(submission.state, 'more_info_needed')
self.assertIsNone(submission.proposed_action)
self.assertFalse(ProjectMilestoneReview.objects.filter(submitted_contribution=submission).exists())
self.assertIsNone(response.data['rubric_review'])

def test_direct_project_accept_stores_rubric_review(self):
submission = self.create_submission()
payload = rubric_payload(
sections={
'genlayer_fit': {'score': 4},
'contract_quality': {'score': 3, 'reason': ''},
'engineering': {'score': 3, 'reason': ' '},
'frontend_ux': {'score': 2},
},
extras=['live_deployment'],
overall_reason='',
)

response = self.client.post(
f'/api/v1/steward-submissions/{submission.id}/review/',
data={
'action': 'accept',
'points': 75,
'contribution_type': self.project_type.id,
'user': self.submitter.id,
'rubric_review': payload,
},
content_type='application/json',
)

self.assertEqual(response.status_code, 200)
submission.refresh_from_db()
self.assertEqual(submission.state, 'accepted')
review = ProjectMilestoneReview.objects.get(submitted_contribution=submission)
self.assertEqual(review.action, 'accept')
self.assertIsNone(review.confidence)
self.assertEqual(review.gate_failures, [])
self.assertEqual(review.extras, ['live_deployment'])
self.assertEqual(review.overall_reason, '')
self.assertEqual(review.sections['genlayer_fit']['score'], 4)
self.assertEqual(review.sections['genlayer_fit']['reason'], '')
self.assertEqual(review.sections['engineering']['reason'], '')
self.assertEqual(response.data['rubric_review']['sections'], review.sections)

note = SubmissionNote.objects.filter(
submitted_contribution=submission,
is_proposal=False,
).first()
self.assertIsNotNone(note)
self.assertEqual(note.data['rubric_review_id'], review.id)
self.assertIn('Rubric scores', note.message)

def test_direct_project_reject_stores_gate_failure_rubric_review(self):
submission = self.create_submission()

response = self.client.post(
f'/api/v1/steward-submissions/{submission.id}/review/',
data={
'action': 'reject',
'staff_reply': 'The repository does not build.',
'rubric_review': gate_payload(),
},
content_type='application/json',
)

self.assertEqual(response.status_code, 200)
submission.refresh_from_db()
self.assertEqual(submission.state, 'rejected')
review = ProjectMilestoneReview.objects.get(submitted_contribution=submission)
self.assertEqual(review.action, 'reject')
self.assertEqual(review.gate_failures, ['repo_does_not_build'])
self.assertEqual(review.sections, {})
self.assertEqual(response.data['rubric_review']['gate_failures'], ['repo_does_not_build'])

note = SubmissionNote.objects.filter(
submitted_contribution=submission,
is_proposal=False,
).first()
self.assertIsNotNone(note)
self.assertEqual(note.data['rubric_review_id'], review.id)
self.assertIn('Gate failed', note.message)

def test_direct_project_accept_rejects_gate_failure_rubric_review(self):
submission = self.create_submission()

response = self.client.post(
f'/api/v1/steward-submissions/{submission.id}/review/',
data={
'action': 'accept',
'points': 75,
'contribution_type': self.project_type.id,
'user': self.submitter.id,
'rubric_review': gate_payload(),
},
content_type='application/json',
)

self.assertEqual(response.status_code, 400)
self.assertIn('action', response.data)
self.assertFalse(ProjectMilestoneReview.objects.filter(submitted_contribution=submission).exists())

def test_project_reject_proposal_stores_scores_reasons_and_extras_when_gate_passes(self):
submission = self.create_submission()
payload = rubric_payload(
Expand Down
72 changes: 72 additions & 0 deletions backend/contributions/tests/test_steward_permissions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,52 @@ def test_steward_can_review_submissions(self):
self.assertIsNotNone(self.submission.converted_contribution)
self.assertEqual(self.submission.converted_contribution.points, 50)

def test_second_steward_cannot_accept_already_accepted_submission(self):
"""A stale second accept must not create another contribution."""
second_steward_user = User.objects.create_user(
email='second-steward@test.com',
address='0x3333333333333333333333333333333333333333',
password='testpass123',
)
second_steward = Steward.objects.create(user=second_steward_user)
StewardPermission.objects.create(
steward=second_steward,
contribution_type=self.contribution_type,
action='accept',
)

self.client.force_authenticate(user=self.steward_user)
first_response = self.client.post(
f'/api/v1/steward-submissions/{self.submission.id}/review/',
{
'action': 'accept',
'points': 50,
'contribution_type': self.contribution_type.id
},
format='json'
)
self.assertEqual(first_response.status_code, status.HTTP_200_OK)
self.submission.refresh_from_db()
first_contribution_id = self.submission.converted_contribution_id

second_client = APIClient()
second_client.force_authenticate(user=second_steward_user)
second_response = second_client.post(
f'/api/v1/steward-submissions/{self.submission.id}/review/',
{
'action': 'accept',
'points': 75,
'contribution_type': self.contribution_type.id
},
format='json'
)

self.assertEqual(second_response.status_code, status.HTTP_400_BAD_REQUEST)
self.submission.refresh_from_db()
self.assertEqual(self.submission.reviewed_by, self.steward_user)
self.assertEqual(self.submission.converted_contribution_id, first_contribution_id)
self.assertEqual(Contribution.objects.count(), 1)

def test_accept_checks_permission_against_final_contribution_type(self):
self.client.force_authenticate(user=self.steward_user)

Expand Down Expand Up @@ -329,6 +375,32 @@ def test_steward_can_request_more_info(self):
self.submission.refresh_from_db()
self.assertEqual(self.submission.state, 'more_info_needed')
self.assertEqual(self.submission.staff_reply, 'Please provide URL evidence')

def test_steward_can_accept_more_info_needed_submission(self):
"""A submission waiting for more info remains reviewable after follow-up."""
self.submission.state = 'more_info_needed'
self.submission.reviewed_by = self.steward_user
self.submission.reviewed_at = timezone.now()
self.submission.staff_reply = 'Please provide URL evidence'
self.submission.last_edited_at = timezone.now() + timezone.timedelta(minutes=1)
self.submission.save()

self.client.force_authenticate(user=self.steward_user)
response = self.client.post(
f'/api/v1/steward-submissions/{self.submission.id}/review/',
{
'action': 'accept',
'points': 50,
'contribution_type': self.contribution_type.id
},
format='json'
)

self.assertEqual(response.status_code, status.HTTP_200_OK)
self.submission.refresh_from_db()
self.assertEqual(self.submission.state, 'accepted')
self.assertIsNotNone(self.submission.converted_contribution)
self.assertEqual(self.submission.converted_contribution.points, 50)

def test_steward_can_create_highlight(self):
"""Test that stewards can create highlights when accepting."""
Expand Down
Loading