How to deal with git conflicts when trying to automate Quarto blog postings on multiple machines?

[smach]

Description

I have a Quarto blog that I publish via GitHub and Netlify. A version of the repo on my Linux cloud server runs a cron job that updates events listings daily, pushes to GitHub, and publishes. I use another version of the repo on my local machine to write and publish my blog posts, pulling from the repo first to make sure all my _site files like the site map and search are up to date.

The problem comes if I want to work on a post locally that I want to render/preview but not publish for a few days, such as my "Election Results" with fake data I don't want to publish until next week with real data. Even if I set draft: true, if I render, files in _site seem to get changed, and then I end up with merge conflicts if I try to keep up to date with the repo. So I end up just doing hard resets which blows away anything else I might want to keep.

Any suggestions on how to handle this? Is there something else I can do to keep my rendering of a draft post from changing files in my _site directory? Thank you.

[mcanouil]

This is more of a Git question than Quarto. Regarding draft, the behaviour is as expected/documented (https://quarto.org/docs/websites/website-drafts.html).

There are many ways you can "solve" your issue git restore ..., git push --force ..., git pull ..., git checkout ... -- ....

git fetch
git pull
git checkout --ours _site/
git add _site/
git commit

or set a .gitattributes with a merge strategy (https://git-scm.com/docs/merge-config)

# 1. Create .gitattributes file
echo "_site/* merge=ours" >> .gitattributes

# 2. Define the merge strategy
git config merge.ours.driver true

# 3. Commit the .gitattributes file
git add .gitattributes
git commit -m "Set merge strategy for _site directory"

I must admit I did not understand your workflow, especially why you would want to commit/push _site before you actually update your branch (git fetch / git pull).
If your CI/CD is doing rendering, hopefully it's done only on push or on some more rare events, thus updating first before doing anything should be your workflow as it ensure you'll have no conflicts at all in the first place.

[smach]

Thanks for the response. Do you know what happens if I want to work on my Election Results post over several weeks while also adding additional posts? This is a very common workflow for a website that publishes both features and timely news. What if it's Monday and I work on my Election Results draft, tweak it, render locally, and _site files change? Then it's Tuesday and I post a short item by working on it locally, rendering, _site files change again, and I pushing to git remote? Will quarto know not to include the draft _site content in ANY of the _site files and only the live content for things like search results, RSS, sitemap?

[mcanouil]

I see, then you would need some automatic update which starts to get a bit tricky.

Note that the conflict resolution approach I mentioned can target whatever you want, in the example I showed the top-level _site, but you can target any sub-directory or files.

Another approach would be to use render-target to fully exclude rendering of some of your posts or use _ prefix for your draft.

[smach]

Thank you.

[cderv]

The problem comes if I want to work on a post locally that I want to render/preview but not publish for a few days, such as my "Election Results" with fake data I don't want to publish until next week with real data. Even if I set draft: true, if I render, files in _site seem to get changed, and then I end up with merge conflicts if I try to keep up to date with the repo. So I end up just doing hard resets which blows away anything else I might want to keep.

@smach regarding your comment on draft and rendered doc, I suggest looking at the different way draft are designed to be rendered: https://quarto.org/docs/websites/website-drafts.html#appearance-of-drafts

The default is to create the .html file in the output dir but make it empty, and don't list the file in the navigation. For the default blog, here is the example of content

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"></html>

that is all; And the post is not in the listing

So when you render your site, your draft post will not be publish on your website.

So I understand that if you commit your _site folder, this would be a conflict when you get it back on your branch. As it would be empty file coming from your git, and locally a non-empty html if your last state is a previewed version as quarto preview will create a non empty html in output dir for previewing.

So in addition to the git workflow previously mention, you could consider adapting workflows.

As a reminder, quarto render and quarto preview on a project will both remove output dir and recreate one. So I don't know where you publish, but you could also don't commit your _site folder and consider only publish source and having quarto render called on CI. This is how we work with quarto.org leveraging Quarto freeze feature to avoid re-executing old computation (and so commiting the freezer _freezer and not the _site dir).

If you need to commit your _site, you could also leverage this behavior by considering you locally need to run quarto render before doing a git pull, or indeed configuring git as mentioned to configure that any _site content locally can be replaced by repo one (and running quarto render locally would replace everything anyway).

Another idea for workflow adaptation is considering Project profile (https://quarto.org/docs/projects/profiles.html) where you could have one with your published output directory that would commit output-dir: _site and another profile for draft work locally (output-dir: _site_draft ?) for your local work, and you would not commit this one. You could then have some kind of draft workflow for local work which would not conflict with your published one.

Those are just ideas with features Quarto offers. There is no best solution, only the one that fit your personal workflow.

Hope this helps with yours.

[smach]

Thank you very much @cderv for the detailed and helpful reply. I think my underlying issue is that I was expecting my Quarto content management system to act the way my CMS behaved at work, and it doesn't, so I need to come up with alternatives. For now I think git is a good way for me to handle this. I can have my remote pull overwrite everything in my _site directory only before rendering a post I want to publish, IF I had earlier rendered a post I just wanted to look at but still keep working on. I publish on Netlify, so I do need to commit my _site directory as far as I know. This is probably somewhat of an edge case for Quarto users in general, but not that unusual for a journalism site posting both news and longer term features. That seemed to work out for my election week posts.

[mcanouil]

You could configure Netlify to read from a different branch/directory which is only rendered/committed via GitHub Action.

[cderv]

I publish on Netlify, so I do need to commit my _site directory as far as I know.

For quarto.org, we do leverage quarto publish netlify through the action:
https://github.com/quarto-dev/quarto-web/blob/d6a67f4664dfbeb43f384d33622a485aa01bb344/.github/workflows/publish.yml#L52-L64

This way we render on CI, and publish the _site folder.

Sharing in case that helps.

[smach]

Oh interesting, I had no idea that was possible. Thank you!

[cderv]

We have several integration with publishing providers and netlify is one of them: https://quarto.org/docs/publishing/netlify.html