Skip to content

Add deployment options #5958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
## Fixes

What bugs does this fix? Use the syntax to auto-close the issue:

https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword

E.g.: "Fixes: #XYZ"


## Summary

What does this fix, how did you fix it, what approach did you take, what gotchas are there in your code or compromises did you make?


## Deployment

1. The following labels control the deployment of this PR if they’re applied. Please choose which should be applied, then apply them to this PR:

- [ ] <kbd>skip-deploy</kbd> — The entire deployment can be skipped.

This might be the case for a small fix, a tweak to documentation or something like that.

- [ ] <kbd>skip-web-deploy</kbd> — The web tier can be skipped.

This is the case if you're working on code that doesn't affect the front end, like management commands, tasks, or documentation.

- [ ] <kbd>skip-celery-deploy</kbd> — Deployment to celery can be skipped.

This is the case if you make no changes to tasks.py or the code that tasks rely on.

- [ ] <kbd>skip-cronjob-deploy</kbd> — Deployment to cron jobs can be skipped.

This is the case if no changes are made that affect cronjobs.

1. What extra steps are needed to deploy this beyond the standard deploy?

Do scripts need to be run or things like that?

If this is more than a quick thing, a new issue should be created in our infra repo:

https://github.com/freelawproject/infrastructure/issues/new

If you don’t have access to it, just put the steps here.


## Screenshots

If this changes the front end, please include desktop and mobile screenshots or videos showing the new feature.

<details>
<summary>Desktop</summary>

YOUR IMAGE(S) HERE

</details>


<details>
<summary>Mobile</summary>

YOUR IMAGE(S) HERE

</details>
57 changes: 55 additions & 2 deletions .github/workflows/docker-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,50 @@ env:
EKS_NAMESPACE: court-listener

jobs:
get-pr-labels:
runs-on: ubuntu-latest
outputs:
labels: ${{ steps.find_pr.outputs.labels }}
steps:
- uses: actions/checkout@v3
- name: Find merged PR by merge_commit_sha
id: find_pr
uses: actions/github-script@v6
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const commitSha = context.sha;
const { data: prs } = await github.rest.pulls.list({
owner: context.repo.owner,
repo: context.repo.repo,
state: "closed",
sort: "updated",
direction: "desc",
per_page: 1
});

const pr = prs.find(p => p.merge_commit_sha === commitSha);
if (!pr) {
core.setOutput("labels", "");
core.warning("No merged PR found for this commit.");
return;
}

const labels = pr.labels.map(l => l.name);
core.setOutput("labels", labels.join(","));
core.setOutput("pr_number", pr.number);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we using this output for anything?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Removed.

console.log(`PR #${pr.number} labels: ${labels.join(", ")}`);

build:
needs: get-pr-labels
runs-on: ubuntu-latest
if: >
${{
!contains(needs.get-pr-labels.outputs.labels, 'skip-deploy') &&
!contains(needs.get-pr-labels.outputs.labels, 'skip-web-deploy') &&
!contains(needs.get-pr-labels.outputs.labels, 'skip-celery-deploy') &&
!contains(needs.get-pr-labels.outputs.labels, 'skip-cronjob-deploy')
}}
Comment on lines +51 to +56
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this skip the image build and push if any of the skip labels is applied?

Is that correct? For instance, if the author adds the skip-web-deploy label, we’d still need the image to be built and pushed so that the Celery and cronjob deployments receive the changes, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if any of these labels are applied, it won't run this step, and I'm pretty sure that means the deploy won't run since it needs the build step. I'm not positive, but I think that's how it will work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that block of code means that if any skip- label is present, the build process is skipped, which is my concern.

For example, if the user only passes skip-celery-deploy, it should still deploy to the web service and the cron job, right? But because there’s no build, the deploy action is completely omitted?

Shouldn’t we only skip the build step when the entire deploy is skipped with skip-deploy?

steps:
- uses: actions/checkout@v4
- name: Login to Docker Hub
Expand All @@ -25,8 +67,9 @@ jobs:
make push-image --file docker/django/Makefile -e VERSION=$(git rev-parse --short HEAD)

deploy:
needs: build
needs: [get-pr-labels, build]
runs-on: ubuntu-latest
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-deploy') }}
concurrency: production
steps:
- uses: actions/checkout@v4
Expand Down Expand Up @@ -91,24 +134,34 @@ jobs:
exit 1
- name: Delete Temporary Pod
run: kubectl delete pod -n ${{ env.EKS_NAMESPACE }} temp-pod-${{ steps.vars.outputs.sha_short }}

# Rollout new versions one by one (watch "deployments" in k9s)
- name: Rollout cl-python
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-web-deploy') }}
run: kubectl set image -n ${{ env.EKS_NAMESPACE }} deployment/cl-python web=freelawproject/courtlistener:${{ steps.vars.outputs.sha_short }}-prod
- name: Watch cl-python rollout status
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-web-deploy') }}
run: kubectl rollout status -n ${{ env.EKS_NAMESPACE }} deployment/cl-python

- name: Rollout cl-celery-prefork
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-celery-deploy') }}
run: kubectl set image -n ${{ env.EKS_NAMESPACE }} deployment/cl-celery-prefork cl-celery-prefork=freelawproject/courtlistener:${{ steps.vars.outputs.sha_short }}-prod
- name: Watch cl-celery-prefork rollout status
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-celery-deploy') }}
run: kubectl rollout status -n ${{ env.EKS_NAMESPACE }} deployment/cl-celery-prefork

- name: Rollout cl-celery-prefork-bulk
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-celery-deploy') }}
run: kubectl set image -n ${{ env.EKS_NAMESPACE }} deployment/cl-celery-prefork-bulk cl-celery-prefork-bulk=freelawproject/courtlistener:${{ steps.vars.outputs.sha_short }}-prod
- name: Watch cl-celery-prefork-bulk rollout status
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-celery-deploy') }}
run: kubectl rollout status -n ${{ env.EKS_NAMESPACE }} deployment/cl-celery-prefork-bulk

- name: Rollout cl-celery-prefork-es-sweep
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-celery-deploy') }}
run: kubectl set image -n ${{ env.EKS_NAMESPACE }} deployment/cl-celery-prefork-es-sweep cl-celery-prefork-es-sweep=freelawproject/courtlistener:${{ steps.vars.outputs.sha_short }}-prod
- name: Watch cl-celery-prefork-es-sweep rollout status
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-celery-deploy') }}
run: kubectl rollout status -n ${{ env.EKS_NAMESPACE }} deployment/cl-celery-prefork-es-sweep

- name: Rollout cl-scrape-rss
Expand Down Expand Up @@ -136,9 +189,9 @@ jobs:
- name: Watch cl-iquery-probe rollout status
run: kubectl rollout status -n ${{ env.EKS_NAMESPACE }} deployment/cl-iquery-probe


# Watch "cronjobs" in k9s
- name: Update cronjobs
if: ${{ !contains(needs.get-pr-labels.outputs.labels, 'skip-cronjob-deploy') }}
run: |
CRONJOB_NAMES=$(kubectl get cronjobs -n court-listener -o jsonpath='{.items.*.metadata.name}' -l image_type=web-prod);
for name in $CRONJOB_NAMES; do
Expand Down