Converting Dockerfile into a Singularity definition file

[andersy005]

Currrently, repo2singularity relies on repo2docker to create a docker image that repo2singularity converts into a singularity image file (SIF). I was thinking that we could use the output of repo2docker --no-build to generate a singularity definition file. This would allows us to bypass the intermediary step of creating a docker image.

• step 1: Generate Dockerfile

$ jupyter-repo2docker --no-build --debug --ref 9ced85dd9a84859d0767369e58f33912a214a3cf https://github.com/norvig/pytudes 2> dockerfile.tmp
$ sed -n -e '/FROM/,$p' dockerfile.tmp > Dockerfile

• Step 2: Convert Dockerfile into SIF using spython tool

$ spython recipe Dockerfile >> Singularity

• Step 3: Build the singularity container locally (if the user has sudo permissions)

$ sudo singularity build myimage.sif Singularity

Or build the singularity container on a remote builder (https://cloud.sylabs.io/builder) (does not require sudo)

singularity --remote build myimage.sif Singularity

Unfortunately, as it is pointed out in the documentation, this output is present for debugging purposes only :(. There are things that are hard coded because they are being copied from the local environment. So, this option of bypassing docker images seems to be out of scope for repo2singularity. I am just opening this issue for future reference.

Xref:

• Repo2Docker: make it easy to start from arbitrary docker image

• the Dockerfile is not enough to generate the image jupyterhub/repo2docker#202

[kmpaul]

@andersy005 This is actually an interesting idea, and I'd like to pursue it. I also just heard about someone who is using gradle + jib to build the layers of a Docker image without using Docker. Thus, getting around the need to be able to run Docker on the HPC system.

What I don't know is if there is a Python equivalent to jib or similar.