qmd files will not open in JLab if Quarto installed with both `quarto.deb` and `conda install quarto`

[eeholmes]

updated title to better reflect the real problem. Note this does not have to do with jupyterlab-quarto.
vrs 1.6.40 or vrs 1.5.57

opening of qmd from JupyterLab shows this

[cderv]

opening of qmd from JupyterLab now shows this

Is this happening for all qmd ? or is it for a specific one ?

[eeholmes]

All qmd. Including with no yaml. Changing the ending to .md allows it to open. Quarto version 1.6.40 and JLab 4.3.0. Newest jupyterlab-quarto extension.

This JupyterHub container image is where it doesn't work. Has newest Quarto
https://github.com/nmfs-opensci/container-images/pkgs/container/container-images%2Fpy-rocket-geospatial-2/350325948?tag=2025.02.05

The JupyterHub container image works. I made one change to use Quarto 1.5.57 when installing R and it then worked.
https://github.com/nmfs-opensci/container-images/pkgs/container/container-images%2Fpy-rocket-geospatial-2/354664472?tag=2025.02.12

[cscheid]

We've tried reproducing this outside of the docker image and could not (with only pip install quarto-cli jupyter jupytext jupyterlab-quarto). But we can see the bug in the docker image, so it must be something specific to that configuration.

[cderv]

@eeholmes can you point me to the dockerfile associated with the images ?

When I try them, it seems conda installed quarto, the binary quarto --version is not working.

jovyan@44590c2674cb:~$ quarto --version
cat: /srv/conda/envs/notebook/share/version: No such file or directory

As Jupytext is calling quarto --version, it may not like the bad output and don't handle the problem. So maybe install of 1.6.42 is broken

[eeholmes-uw]

@cderv @cscheid Let me do some more debugging to test when this is happening before you poke into it.

I think it is something about having quarto installed via Rocker scripts and installed via conda. My guess is something about that is causing these odd behaviors with open qmd's in JLab in these images.

[cderv]

Sure - We'll late you confirm this issue. The other one was definitely a problem in Quarto that will be solved and backported to 1.6. Thank you for reporting !!

[eeholmes]

Update: This seems to have something to do with having quarto installed in two places and doesn't have anything to do with jupyterlab-quarto and it appears in 1.5.57 so is not related to 1.6. I don't know enough about how qmd's are opened in Jupyter Lab to debug. jupytext is involved I gather.

Quarto installed with conda install quarto (actually via environment.yml`) and with https://github.com/rocker-org/rocker-versioned2/blob/master/scripts/install_quarto.sh (which is doing a wget on quarto.deb and dpkg-ing that). Somehow this messes up jupytext.

• /srv/conda/envs/notebook/bin/quarto
• /usr/local/bin/quarto

Path: /srv/conda/envs/notebook/bin:/srv/conda/condabin:/srv/conda/envs/notebook/bin:/srv/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/texlive/bin/linux

Image that shows problem. This has version Quarto 1.5.57 and no jupyterlab-quarto.
ghcr.io/nmfs-opensci/py-rocket-base/test:6ea6ca2

Note with this image, inside a notebook I get this. Works fine in terminal in Jupyter Lab. This is only when run in a notebook.

!quarto --version
cat: /srv/conda/envs/notebook/share/version: No such file or directory

!which quarto
/srv/conda/envs/notebook/bin/quarto

Why is it looking for /srv/conda/envs/notebook/share/version and not /srv/conda/envs/notebook/share/quarto/version??

Solution: Since the bug with qmd's opened with Jupyter Lab is a quarto-cli bug (and not related to the qmd not opening issue), I will only install quarto to /usr/local/bin and not install to the conda environment. Fingers crossed that jupytext is ok with that.

[eeholmes]

Ok, I have come to the conclusion that installing quarto via

QUARTO_DL_URL="https://github.com/quarto-dev/quarto-cli/releases/download/v${QUARTO_VERSION}/quarto-${QUARTO_VERSION}-linux-${ARCH}.deb"
wget "$QUARTO_DL_URL" -O quarto.deb
dpkg -i quarto.deb

is catastrophically incompatible installing quarto with conda install quarto in Jupyter Lab (where kernel is using the conda env). It doesn't matter if I install quarto via conda before or after installing with quarto.deb. Version of quarto doesn't matter.

Things like this that work fine if I only install quarto into the conda environment, show this if I have installed quarto via deb file:

!quarto check
/srv/conda/envs/notebook/bin/quarto: line 191: /srv/conda/envs/notebook/bin/tools/x86_64/deno: No such file or directory

Quarto seems to be very confused as if there is some prefix that is being set wrong on the file paths.

@cscheid My best guess is that when Quarto is installed with .deb it is setting some assumptions about how to construct the file paths to things like deno and version. For some reason, inside a Jupyter notebook, when one installs quarto with conda (before or after installing with quarto.deb), these assumptions don't work. In a terminal in Jupyter Lab, things seem ok but running the !quarto --version etc in the Jupyter notebook is going south.

If I only, install quarto via conda, things work fine. I still need to test if I only install with deb. I was having a hard time testing that since JLab was destroying my qmd files.

[cscheid]

Thanks for the additional information! Unfortunately we're not the ones who maintain the conda installation, so we really can't help with things related to /src/conda/..., etc.

[eeholmes]

@cscheid Got it! I will see if I can find the appropriate place to post.

[cderv]

Unfortunately we're not the ones who maintain the conda installation, so we really can't help with things related to /src/conda/..., etc.

@eeholmes This is maintained at conda-forge/quarto-feedstock so you could post there. It is community maintained with our help for Quarto expertise questions.

Why install quarto twice by the way ? Wouldn't the conda install be enough ?

[eeholmes]

@cderv This is coming up in work I am doing on a Docker image that has Pangeo Python environment (conda based) and rocker-geospatial R environment. https://nmfs-opensci.github.io/py-rocket-base/

Keeping conda off the path is essential for the R environment to work given the nature of these R geospatial packages and how they are working with GDAL (at least for what we do).

I was experimenting with having the Python environment be entirely conda so was trying having quarto installed via conda in addition regular install. But I can definitely have quarto only installed via .deb (and no conda install).

I will post to conda-forge/quarto-feedstock once I have time to prep a minimal example.

The conda design choice is not up to me, since the purpose of this work is to mimic the Pangeo image design choices. Hard part is getting the R/RStudio environment to work right side-by-side with the conda environments. This emulating the rocker-geospatial approach for the R/RStudio side is one of a few that we are trying (so, the answer to "have you tried...." is probably yes.)