Merge pull request #6 from gedankenstuecke/binderrize

Expand on docker/binder descriptions

Author: amandasaurus

Dockerfile

@@ -5,6 +5,7 @@ RUN apt-get update \
  ca-certificates gnupg lsb-release locales \
  wget curl \
  git-core unzip \
+ netcat \
 && locale-gen $LANG && update-locale LANG=$LANG 
 
 
@@ -47,19 +48,22 @@ RUN mv fonts/ /usr/share/fonts/type1/gsfonts
 
 # Install python libraries
 
-RUN pip install pyyaml nik4 requests
+RUN pip install pyyaml nik4 requests notebook jupyterlab ipywidgets
 
 # Install carto for stylesheet
 RUN npm install -g [email protected]
 
+ENV HOME=/home/postgres
 
-COPY . /workdir
-
-RUN mkdir -p /workdir/openstreetmap-carto/data 
-RUN mkdir -p /workdir/output
-
-WORKDIR /workdir
+# Make sure the contents of our repo are in ${HOME}
+COPY . ${HOME}
+RUN usermod -u 1000 postgres
+RUN chown -R 1000 ${HOME}
+USER postgres
 
-RUN chown postgres:postgres -R /workdir  
+RUN mkdir -p ${HOME}/openstreetmap-carto/data 
+RUN mkdir -p ${HOME}/output
+RUN mkdir -p ${HOME}/pgdata
+WORKDIR ${HOME}
 
-USER postgres
+ENTRYPOINT ["./entrypoint-new.sh"]

README.md

@@ -2,7 +2,11 @@
 
 Make a image of OSM data of an area from 2 dates, showing what was changed.
 
-## Installation
+You can either install the necessary dependencies on your computer or run the script via Docker (both via MyBinder or in a local container).
+
+[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/gedankenstuecke/osm-mapping-party-before-after/binderrize?labpath=make-images.ipynb)
+
+## Making a local Installation
 
 1. *On macOS:* [install Homebrew](https://brew.sh/#:~:text=Install%20Homebrew)
 1. [Install `pipx`](https://pipx.pypa.io/stable/installation/#installing-pipx)
@@ -26,10 +30,19 @@ Make a image of OSM data of an area from 2 dates, showing what was changed.
    cd osm-mapping-party-before-after
    ```
 
-### Install & use via Docker image
+## Use via Docker or MyBinder
+
+If you do not want to install the whole pipeline yourself, you can run this setup in a Docker container that is ready-to-use and that can also run in the free-to-use open source cloud infrastructure of _MyBinder_. 
+
+If you want to use the _MyBinder_ version, [click here and wait a bit](https://mybinder.org/v2/gh/gedankenstuecke/osm-mapping-party-before-after/binderrize?labpath=make-images.ipynb). This will launch the version online in a virtual machine and lets you interact with the code through a small Python notebook that will launch automatically and contains all necessary instructions.
+
+The MyBinder version has two drawbacks: 1. You will have to upload the OSM history file (_\*.osh.pbf_) into the container. Depending on the region of interest these can be quite large. 2. Creating the maps will take longer, as other external downloads will have to be downloaded on the fly. See [this blog post for more details](https://tzovar.as/map-comparisons/).
+
+If you want to run the Docker image on your own computer, you can find the necessary [image is available on Docker Hub under `gedankenstuecke/osm-mapping-party-before-after`](https://hub.docker.com/r/gedankenstuecke/osm-mapping-party-before-after). 
 
-Alternatively, if you do not want to install the whole pipeline yourself, you can run this setup in a Docker container that is ready to use.
-We start, by cloning this repository:
+### Building the Docker container from scratch
+
+Alternatively, you can also build the container locally if you want to make changes to it/improve it: We start by cloning this repository:
 
 ```bash
 git clone --recurse-submodules https://github.com/amandasaurus/osm-mapping-party-before-after
@@ -45,6 +58,9 @@ docker build -t before_after_builder .
 ```
 
 This step needs to be run only once, and it can take a few minutes as it will download and build all the dependencies needed. 
+
+#### Running the container locally
+
 Once the step is finished, you can launch the container like so:
 
 ```bash
@@ -53,26 +69,26 @@ Once the step is finished, you can launch the container like so:
 
 The command takes two parameters:
 
-1. the name of the container (as specified above)
-2. a full path to a directory you want to use for accessing and writing files to from the container (i.e. Input and output files). Depending on how you installed/run Docker, this folder might need full read/write permissions for other users (e.g. run `chmod 777`). 
+1. the name of the container (e.g. as specified during the `docker build` step)
+2. a full path to a directory you want to use for accessing and writing files to from the container (i.e. input and output files). Depending on how you installed/run Docker, this folder might need full read/write permissions for other users (e.g. run `chmod 777`). 
 
-The `docker_run.sh` command will launch the container itself, including the necessary postgres database etc.
-It will also create two virtual docker volumes (named `pgdata` and `osm_data`). 
-These volumes are used to store external data, i.e. the database contents and the `openstreetmap-carto` external files.
+The `docker_run.sh` command will then launch the container itself, including the necessary postgres database and the notebook interface. It will also create two virtual docker volumes (named `pgdata` and `osm_data`). These volumes are used to store external data, i.e. the database contents and the `openstreetmap-carto` external files, so that subsequent launches are faster than the initial one.
 
-Once the docker container is running, you can connect into it to get a bash inside to run the commands as outlined below using:
+After running the `./docker_run.sh` command you will see a lot of text running by while the container sets itself up. At the end of this, you should see this: 
 
-```bash
-docker exec -ti map-before-after bash
+```
+To access the server, open this file in a browser:
+        file:///home/postgres/.local/share/jupyter/runtime/jpserver-1-open.html
+    Or copy and paste one of these URLs:
+        http://1ce38bef580f:8888/tree?token=asecrettokenwithlotsofcharacters
+        http://127.0.0.1:8888/tree?token=asecrettokenwithlotsofcharacters
 ```
 
-The resulting shell puts you into the equivalent of the root of this repository, in a folder called `/workdir`. From there you can use this repository as outlined below. 
-
-All output files are saved in the `/workdir` by default, you can from there move them into `/workdir/output` inside the container to access files on your host operating system.
+Clicking on the `http://127.0.0.1:8888/tree?…` link will open the notebook interface in your web-browser, where you can open the `make-images.ipynb` notebook, which will have all the necessary instructions to get started.
 
-A [ready-built image is available on Docker Hub](https://hub.docker.com/r/gedankenstuecke/osm-mapping-party-before-after).
+## Usage of `make.sh` without the container
 
-## Usage
+If you have tile-building setup on your computer, you can run `make.sh` without Docker, following these steps:
 
 1. Download an OSM history file (`.osh.pbf`) e.g. from [Geofabrik's internal download server](https://osm-internal.download.geofabrik.de/?landing_page=true). You will need to log in with an OSM account.
 1. Calculate the `BBOX` with [BBoxFinder.com](http://bboxfinder.com/).

docker_run.sh

@@ -13,4 +13,4 @@ IMAGE_NAME=$1
 OUTPUT_DIR=$2
 
 
-docker run --rm -e POSTGRES_PASSWORD="unused" -v pgdata:/var/lib/postgresql/data -v osm_data:/workdir/openstreetmap-carto/data -v $OUTPUT_DIR:/workdir/output --name map-before-after $IMAGE_NAME
+docker run --rm -e POSTGRES_PASSWORD="unused" -v pgdata:/home/postgres/pgdata -v osm_data:/home/postgres/openstreetmap-carto/data -v $OUTPUT_DIR:/home/postgres/external --name map-before-after -p 8888:8888 $IMAGE_NAME jupyter notebook --NotebookApp.default_url=/lab/ --ip=0.0.0.0 --port=8888

entrypoint-new.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+
+export POSTGRES_PASSWORD='unused'
+export PGDATA="/home/postgres/pgdata"
+
+nohup docker-entrypoint.sh postgres &
+
+while ! nc -z localhost 5432; do
+	echo "waiting for postgres"
+	sleep 1;
+done;
+
+exec "$@"