Clarify development process, re: branches

[patcon]

Re: https://github.com/CivicTechTO/polis/wiki/Development

I don't believe our branching workflow will allow us to easily work with upstream unless we change something 🙏

Specifically, we need to be clear which branches we create new feature branches from (upstream edge, or our downstream analog that's sync'd exactly to upstream, aka our edge), and which branches we expect to be able to submit PRs into (either edge-civictechto or [upstream] edge is what needs to be safe/easy to do). PRs to our repo's edge (the synchronized branch that mirrors upstream) should never be done, and ideally be impossible.

Hopefully this helps show why.

Currently: (non-ideal)

upstream:
               D---E---F edge (default)

civictechto:

                           G---H 123-feature
                          /
                     A---B---C edge (default)
                    /
               D---E---F remote/upstream/edge (headless)

You can see that making our feature branches from our edge (different history that upstream edge) will mean commits A+B from our repo get sent along and pollute upstream PRs. With this setup, we can't make PRs from a singular feature branch that work with either our own default branch or upstreams -- we'd need to create two feature branches for each feature -- one off of commit "A" and another off of commit "E"

Proposed: (i think improvement?)

upstream:
               D---E---F edge (default)

civictechto:

                      G---H 123-feature
                     /
                    /A---B---C edge-civictechto (default)
                   //
               D---E---F edge (sync'd to upstream, protected as reference, never pushed to)

Now, so long as we're always making PRs off of our repo's edge (which will always be sync'ed to upstream edge, and should be protected to never accept anything else), then we can keep submitting our own PRs from a single feature branch: both (a) 123-feature to edge-civictechto, and (b) 123-feature to upstream repo's edge

[patcon]

Relavant @NewJerseyStyle comment: #32 (comment) (will respond soon, and more concisely than last night)

[patcon]

@NewJerseyStyle Can you confirm that you've read the main body of this issue, before I respond here to the message from the other thread? I know I confused things between the two issues, but my above explanation is the clearest. Before I try to clarify again, I just want to make sure we're working from the same information, and then I'll try to fix whatever miscommunication I've left :)

[NewJerseyStyle]

Thanks for the pause.

I have just read all related comments across the repo.

Here is what are we trying to do in case you want to know what it looks like in our eye

1. We try to minimize the number of branch so it is easier to understand
2. We do not pull from upstream, we merge
3. We use edge to track the latest development of all sources (from compdemocracy/polis and from PRs of our member) #32 (comment)

So based on the proposed branching strategy, I would propose:

Upstream:

%%{init: { 'theme': 'base', 'gitGraph': {'mainBranchName': 'edge'}} }%%
      gitGraph
        commit id:"D"
        commit id:"E"
        commit id:"F"

Loading

Civictechto

%%{init: { 'theme': 'base', 'gitGraph': {'mainBranchName': 'edge (upstream)'}} }%%
gitGraph
    checkout 'edge (upstream)'
    commit id:"D"
    branch "main (default)"
    checkout 'edge (upstream)'
    commit id:"E"
    commit id:"F"
    checkout "main (default)"
    branch dev
    branch A-feature
    commit id:"A"
    checkout dev
    merge A-feature
    branch B-feature
    commit id:"B"
    checkout dev
    branch C-feature
    checkout dev
    merge B-feature
    branch 123-feature
    checkout C-feature
    commit id:"C"
    checkout dev
    merge C-feature
    checkout 123-feature
    commit id:"G"
    commit id:"H"
    checkout dev
    merge  'edge (upstream)'
    checkout "main (default)"
    merge dev
    checkout  'edge (upstream)'
    commit
    commit
    checkout dev
    merge 123-feature

Loading

edge only accept update from upstream, and only merge into dev

dev do not commit, only PR from feature/issue/doc/edge other branches

main only accept update from dev

Works are done on other branch and PR to dev

[patcon]

Response to #32 (comment)

I'm sorry, I took for granted some patterns, and so didn't think much about the questions being asked in the moment. I flippantly but strongly advocated for maintaining the "edge"branch namespace, instead of creating a new branch/namespace called "main" or "dev", which would relate our branches to upstream counterparts in unclear ways (only knowable by reading docs, instead of just looking at branch name). That was poorly thought-out communication on my part.

There was an unspoken and failed assumption of shared knowledge that our "edge" namespace still needs to have a diff name if we're pushing things to it in our repo. I didn't realize the knowledge gap until this convo, and didn't realize that I had strongly recommended otherwise in a more abstract conversation (that we push to one called edge, when I was only trying to advocate "let's not reimagine our name[space]s as main or dev, but just go with the flow of the edge choice upstream"). I wasn't thinking hard enough when answering, bc I didn't see the value of figuring out the branching strategy before we were working, and that was my failure.

I've been working with open source repos for so long that I forget what's common knowledge and what requires prior conversation, for which I apologize.

[NewJerseyStyle]

Thanks patcon... Although I had no feeling about the issue and see no reason to ask you for apologize. Anyway your apologize has been accepted. Let's move on.

Now I assume we have synced on all view and perspective in the past about the issue and identified some key factors triggered the issue and causes problem.

May I ask for your opinion on my proposal of adding branch dev to merge all proposed change in the repository, and adding branch main to make release of our polis version? Feel free to comment and alter the idea to make it better. 🤗

[patcon]

Ugh, my brain hurts from all this.

edge only accept update from upstream, and only merge into dev ✅ if update means sync (in UI)

dev do not commit, only PR from feature/issue/doc/edge other branches ✅

main only accept update from dev ✅ (though seems premature to add this branch before playing with a lesser process, when we can just use what you call dev to start)

1. this seems a big step in right direction. thanks!
2. [high priority] edge should be protected in our repo (on sync'ing from upstream, like in screenshot)
3. [high priority] your dev should be default. having a slower branch as default seems non-ideal. If dev is active development, it should be default, so PRs get created from it automatically by GitHub defaults, and repo activity shows as activity in GitHub UI, not hidden away in a non-default branch)
4. [optional/mid priority] the value of renaming our equivalent branch of the upstream edge todev seems marginal. If we keep it as edge-whatever, edge/whatever or even whatever/edge it preserves relationship via namespacing for anyone just skimming the branch list. edge-whatever is my highest recommendation, as it shows up right beside edge when someone is trying to understand how this project differs from what they understand from knowing upstream. Just like "the best code is the code you don't right", "the best docs are the ones you don't need to write". Aligning with upstream namespacing feels marginally beneficial imho 🙏
5. [optional/low priority] main need not exist yet, and may be premature workflow overhead? Lots of projects just use mainline development without the dev/main distinction that was recommended by old styles like git-flow.
6. having said all that, even if we don't wish to integrate any of the recommendations I've made here (which is fine, it will likely be more your process to manage and support), even exactly as depicted, this is certainly an improvement 🎉

Ok, i think I'm caught up! 😓

[NewJerseyStyle]

I guess I was too old school, let me read some branching strategy articles before edit my proposal 😓

[NewJerseyStyle]

1. ✅
2. ✅ locked (may need to revert one merge I made yesterday)
3. GitHub flow to use dev as default ✅
   The reason I try to maintain main branch is to allow failure on dev branch in case of low quality PR.
   I assume with E2E test it should be fine.
   In case of new feature E2E test not covered, we can do hotfix. And we can do CI/CD triggers only on event when label is added.
4. use edge-civictechto as dev as default ✅
5. with GitHub flow, main is no longer needed ✅
6. update on branch and in slack 📝

civictechto branches

%%{init: { 'theme': 'base', 'gitGraph': {'mainBranchName': 'edge (upstream)'}} }%%
gitGraph
    commit id:"D"
    branch edge-civictechto
    checkout "edge (upstream)"
    commit id:"E"
    commit id:"F"
    checkout edge-civictechto
    branch A-feature
    commit id:"A"
    checkout edge-civictechto
    merge A-feature
    branch B-feature
    commit id:"B"
    checkout edge-civictechto
    branch C-feature
    checkout edge-civictechto
    merge B-feature
    branch 123-feature
    checkout C-feature
    commit id:"C"
    checkout edge-civictechto
    merge C-feature
    checkout 123-feature
    commit id:"G"
    commit id:"H"
    checkout edge-civictechto
    merge  'edge (upstream)'
    merge 123-feature

Loading

[NewJerseyStyle]

Fix #37 to finish adoption of the new arrangement