From 87e2efe7baec89c858aa8cc1a1c7fb20068d6e32 Mon Sep 17 00:00:00 2001
From: Lars Bilke <lars.bilke@ufz.de>
Date: Mon, 11 Feb 2019 12:53:42 +0100
Subject: [PATCH] [web] Container maker documentation.
---
.../docs/devguide/advanced/docker.pandoc | 24 ++++---
.../docs/devguide/advanced/singularity.pandoc | 54 ++++++++++++++++
.../branching-model.pandoc | 3 +-
.../docs/userguide/basics/container.pandoc | 64 +++++++++++++++++++
.../basics/ogs-container-maker-params.png | 3 +
5 files changed, 137 insertions(+), 11 deletions(-)
create mode 100644 web/content/docs/devguide/advanced/singularity.pandoc
create mode 100644 web/content/docs/userguide/basics/container.pandoc
create mode 100644 web/content/docs/userguide/basics/ogs-container-maker-params.png
diff --git a/web/content/docs/devguide/advanced/docker.pandoc b/web/content/docs/devguide/advanced/docker.pandoc
index 22f4fd31dfc..c3f8234b5a4 100644
--- a/web/content/docs/devguide/advanced/docker.pandoc
+++ b/web/content/docs/devguide/advanced/docker.pandoc
@@ -11,19 +11,20 @@ weight = 1034
## Introduction
-> [Docker](https://www.docker.com/whatisdocker) containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.
+> Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.
+>
+> <cite>[www.docker.com/whatisdocker](https://www.docker.com/whatisdocker)</cite>
See the [docs](https://docs.docker.com) for installation instructions.
## Images
-Docker images can be created by [Dockerfiles](https://docs.docker.com/reference/builder/) which can be stacked and thus form a directed graph. OGS-6 image definitions can be found at the [ufz/dockerfiles](https://github.com/ufz/dockerfiles)-repo. Built images can be found at [Docker Hub](https://hub.docker.com/u/ogs6/).
+Docker images can be created by [Dockerfiles](https://docs.docker.com/reference/builder/) which can be stacked and thus form a directed graph. OGS-6 image definitions are created with [ufz/ogs-container-maker](https://github.com/ufz/ogs-container-maker). Built images can be found at [Docker Hub](https://hub.docker.com/u/ogs6/).
To build an image by yourself create a Dockerfile:
```
-FROM ubuntu:14.04
-MAINTAINER Jon Doe <jon.doe@foo.com>
+FROM ubuntu:17.10
RUN ...
```
@@ -31,9 +32,10 @@ RUN ...
Run the `build` command:
```bash
-$ docker build -t repo/image_name path/to/directory
+$ docker build --rm -t repo/image_name path/to/directory
```
+- `--rm` Cleans up after exiting the container
- `-t` specifies a name for the image, can be arbitrary chosen (but should match the corresponding image on Docker Hub if there is one)
- The path should specify the directory where the Dockerfile is located
@@ -44,37 +46,38 @@ Now you can see your build image with `$ docker images`.
To run commands inside a container:
```bash
-$ docker run -t image_name command_to_run
+$ docker run --rm -t image_name command_to_run
```
To run an interactive shell add the `-i`-switch:
```bash
-$ docker run -i -t image_name /bin/bash
+$ docker run --rm -i -t image_name
```
It is useful to mount folders from the host operating system in the Docker container, e.g. to edit source code on your host with your favorite editor:
```bash
-$ docker run -i -t -v /host/directory:/container/directory image_name /bin/bash
+$ docker run --rm -i -t -v /host/directory:/container/directory image_name
```
## Prebuilt OGS-6 Docker images
There are docker images provided on [Docker Hub](https://hub.docker.com/u/ogs6/) which include everything necessary to build OGS-6.
-There are images for `gcc`, `clang` and `mingw`.
+There are images for `gcc` and `clang`.
E.g. to start developing OGS:
```bash
docker pull ogs6/gcc
-docker run -t -i -v /path/to/ogs/source/on/host:/usr/src/ogs ogs6/gcc
+docker run --rm -t -i -v /path/to/ogs/source/on/host:/tmp/ogs ogs6/gcc
```
You are now in the Docker container the ogs source code from your host computer mounted at `/usr/src/ogs`. Now you can build and run OGS:
```bash
+cd /tmp
mkdir build; cd build
cmake ../ogs
make ogs
@@ -86,4 +89,5 @@ make ogs
The following images are used by Jenkins:
- `ogs6/gcc`, defined in [scripts/docker/Dockerfile.gcc.full](https://github.com/ufz/ogs/blob/master/scripts/docker/Dockerfile.gcc.full)
+- `ogs6/gcc:gui`, defined in [scripts/docker/Dockerfile.gcc.gui](https://github.com/ufz/ogs/blob/master/scripts/docker/Dockerfile.gcc.gui)
- `ogs6/clang`, defined in [scripts/docker/Dockerfile.clang.full](https://github.com/ufz/ogs/blob/master/scripts/docker/Dockerfile.clang.full)
diff --git a/web/content/docs/devguide/advanced/singularity.pandoc b/web/content/docs/devguide/advanced/singularity.pandoc
new file mode 100644
index 00000000000..4ac4110fc57
--- /dev/null
+++ b/web/content/docs/devguide/advanced/singularity.pandoc
@@ -0,0 +1,54 @@
++++
+date = "2018-09-21T11:00:13+01:00"
+title = "Singularity"
+author = "Lars Bilke"
+weight = 1034
+
+[menu]
+ [menu.devguide]
+ parent = "advanced"
++++
+
+## Introduction
+
+[Singularity](https://www.sylabs.io) is a Linux container runtime similar to Docker. Key advantages over Docker are
+
+- Container don't run with root privileges
+- You are the same user with the same privileges inside the container as on the host
+- Container can run on HPC systems and seamlessly integrate with resource managers and MPI
+- Container can leverage NVidia GPUs
+
+Singularity per default mounts your home directory and also passes your current working directory when starting a container. Therefore it is easy to use it for development.
+
+### Prerequisites
+
+- Linux only!
+- [Install Git with LFS]({{< ref "prerequisites" >}}/#step-install-git)
+- [Install Singularity](https://www.sylabs.io/guides/2.6/user-guide/quick_start.html#quick-installation-steps)
+
+### Build OGS inside a container
+
+```bash
+[git clone ogs]
+singularity pull docker://ogs6/gcc # Downloads the image to gcc.simg
+singularity shell gcc.simg
+[Now inside the container]
+mkdir build; cd build
+cmake ../ogs -DCMAKE_BUILD_TYPE=Release
+make
+./bin/ogs
+```
+
+### Run OGS inside a Container (called from outside)
+
+Once ogs executable is built it can be called from **outside** the container:
+
+```bash
+singularity exec gcc.simg build/bin/ogs some/path/project.prj
+```
+
+This starts the container, mounts your home directory inside the container, passes the current working directory and runs the ogs executable (which is in your home directory which is mounted inside the container) with the passed project file. Everything works as expected and is transparent to the user. When ogs finishes the container stops and you returns to the host system.
+
+## Container generator
+
+You can download a prebuilt container from Docker Hub as shown above (e.g. `singularity pull docker://ogs6/gcc`). But we also provide a [container generator]({{< ref "container.pandoc" >}}) to create a specific container for your needs.
diff --git a/web/content/docs/devguide/development-workflows/branching-model.pandoc b/web/content/docs/devguide/development-workflows/branching-model.pandoc
index 109c2cd045a..ecc90b51626 100644
--- a/web/content/docs/devguide/development-workflows/branching-model.pandoc
+++ b/web/content/docs/devguide/development-workflows/branching-model.pandoc
@@ -20,7 +20,8 @@ The following explanation is taken from an [in-depth article](https://www.atlass
> The main advantage of the Forking Workflow is that contributions can be integrated without the need for everybody to push to a single central repository. Developers push to their own server-side repositories, and only the project maintainer can push to the official repository. This allows the maintainer to accept commits from any developer without giving them write access to the official codebase.
>
> The result is a distributed workflow that provides a flexible way for large, organic teams (including untrusted third-parties) to collaborate securely. This also makes it an ideal workflow for open source projects.
-> <cite> https://www.atlassian.com/git/tutorials/comparing-workflows#!workflow-forking </cite>
+>
+> <cite><a href="https://www.atlassian.com/git/tutorials/comparing-workflows#!workflow-forking">www.atlassian.com/git/tutorials/comparing-workflows#!workflow-forking</a> </cite>
>
The workflow is summarized in the following image from the [GitHub blog](https://github.com/blog/2042-git-2-5-including-multiple-worktrees-and-triangular-workflows):
diff --git a/web/content/docs/userguide/basics/container.pandoc b/web/content/docs/userguide/basics/container.pandoc
new file mode 100644
index 00000000000..51e879e9aec
--- /dev/null
+++ b/web/content/docs/userguide/basics/container.pandoc
@@ -0,0 +1,64 @@
++++
+date = "2018-11-14T15:00:13+01`:00"
+title = "Running OGS in a container"
+author = "Lars Bilke"
+weight = 10
+
+[menu]
+ [menu.userguide]
+ parent = "basics"
++++
+
+## With Singularity
+
+### Prerequisites
+
+* Linux (you may run Singularity with [Vagrant](https://app.vagrantup.com/sylabs) on macOS or Windows as well)
+* A running [installation](https://www.sylabs.io/guides/2.6/user-guide/quick_start.html#quick-installation-steps) of Singularity 2.6
+
+### Get a container image
+
+TODO: Download prebuilt images
+
+Build images with [ogs-container-maker](https://github.com/ufz/ogs-container-maker) on [Jenkins](https://jenkins.opengeosys.org/job/ufz/job/ogs-container-maker/job/master/build) (login required, ask Lars Bilke for access):
+
+{width=450px}
+
+It will build all combinations of `format`, `openmpi_versions` and `pm`. When the build finishes you can download the container (`*.simg` files) at the jobs page under `Build Artifacts`.
+
+### Run OGS inside a Container (called from outside)
+
+```bash
+singularity exec ogs-openmpi-2.1.1-conan.simg ogs some/path/project.prj
+```
+
+This starts the container, mounts your home directory inside the container, passes the current working directory and runs the ogs executable (which is in your home directory which is mounted inside the container) with the passed project file. Everything works as expected and is transparent to the user. When ogs finishes the container stops and you returns to the host system.
+
+```bash
+# Create output directories
+mkdir -p _out _out_mpi
+# Run serial benchmark
+singularity exec ogs-openmpi-2.1.1-conan.simg ogs -o _out [ogs-sources]/Tests/Data/Mechanics/Linear/disc_with_hole.prj
+# Run serial benchmark with output validation (via vtkdiff)
+singularity exec ogs-openmpi-2.1.1-conan.simg -o _out -r [ogs-sources]/Tests/Data/Mechanics/Linear [ogs-sources]/Tests/Data/Mechanics/Linear/disc_with_hole.prj
+# Run parallel benchmark with MPI
+mpirun -np 4 singularity exec ogs-openmpi-2.1.1-conan.simg -o _out_mpi [ogs-sources]/Tests/Data/Mechanics/Linear/disc_with_hole.prj
+```
+
+You can run other contained executables, e.g. `vtkdiff`:
+
+```bash
+singularity exec ogs-openmpi-2.1.1-conan.simg vtkdiff --help
+```
+
+You can interactively explore the container with `singularity shell` (you can see that you are **in** the container because of the `Singularity [container image file]:...>` prefix of the shell):
+
+```bash
+# Shell into container
+singularity shell ogs-openmpi-2.1.1-conan.simg
+# List files in the container
+Singularity ogs-openmpi-2.1.1-conan.simg:...> ls /usr/local/ogs/bin
+... ogs tetgen vtkdiff
+# Exit the container and get back to your hosts shell
+Singularity ogs-openmpi-2.1.1-conan.simg:...> exit
+```
diff --git a/web/content/docs/userguide/basics/ogs-container-maker-params.png b/web/content/docs/userguide/basics/ogs-container-maker-params.png
new file mode 100644
index 00000000000..592011235ef
--- /dev/null
+++ b/web/content/docs/userguide/basics/ogs-container-maker-params.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c2f5970d485d937ff42ed5020d74214646f123fb23496dc6c819e1b24f73a36
+size 87146
--
GitLab