Skip to content
Snippets Groups Projects
Commit a50886a6 authored by Lars Bilke's avatar Lars Bilke
Browse files

[web] Finished advanced section in devguide.

parent 95f0f591
No related branches found
No related tags found
No related merge requests found
Showing with 458 additions and 0 deletions
+++
date = "2018-02-26T11:00:13+01:00"
title = "Build with ninja"
author = "Lars Bilke"
weight = 1031
[menu]
[menu.devguide]
parent = "advanced"
+++
## Install ninja
::: {.win}
Download *ninja.zip* from the [latest GitHub release](https://github.com/ninja-build/ninja/releases/latest). Unzip it and make sure that the directory containing the `ninja.exe` is the `PATH`-environment variable!
:::
::: {.linux}
Install with your package manager:
```bash
$ sudo apt-get install ninja-build
```
:::
::: {.mac}
Install via Homebrew:
```bash
$ brew install ninja
```
:::
## Configure for ninja and build
Run CMake with the ninja-generator:
```bash
$ cmake ../source/dir -GNinja
$ ninja
```
::: {.note}
### <i class="fas fa-exclamation-triangle"></i> Visual Studio remarks
When you configure with the Ninja generator you have to run CMake from the appropriate Visual Studio Command Prompt! From there you can both use `cmake` as well as `cmake-gui` which starts the GUI. In the GUI select the `Ninja` generator and leave the toggle `Use default native compilers` on.
:::
+++
date = "2018-02-26T11:00:13+01:00"
title = "Conan package manager"
author = "Lars Bilke"
weight = 1032
[menu]
[menu.devguide]
parent = "advanced"
+++
::: {.note}
### <i class="fas fa-exclamation-triangle"></i> Conan 1.0.0 required
As of OpenGeoSys 6.1.0 Conan version 1.0.0 is required! Please update Conan by running `pip install --upgrade conan` or by downloading the Windows installer.
:::
The [Conan package manager](https://www.conan.io) helps to install all required libraries in a convenient way on every platform. See [Setup pre-requisites](../../getting-started/prerequisites) for installation instructions. Set the CMake option `OGS_USE_CONAN=ON` to enable Conan.
## Advanced usage
Per default when Conan is enabled (`OGS_USE_CONAN=ON`) it will try to fetch prebuilt binaries from the [OGS Conan repository](https://ogs.jfrog.io/ogs/conan/) at https://ogs.jfrog.io/ogs/api/conan/conan. With the CMake option `OGS_CONAN_BUILD` you define what gets build locally. This option can be set to:
- `missing` - Default, only builds packages which are not available as a prebuilt binary for the current configuration
- `all` - Builds all packages locally
- `never` - Builds no package locally
- `[a list of libraries to build]`, e.g. `"Eigen3;Boost"`. For names see [ConanSetup.cmake](https://github.com/ufz/ogs/blob/master/scripts/cmake/ConanSetup.cmake). Make sure to set this back to 'never' after the libs have been built. Otherwise it would rebuild the libs on the next CMake run.
+++
date = "2018-02-26T11:00:13+01:00"
title = "Configuration options"
author = "Lars Bilke"
weight = 1033
[menu]
[menu.devguide]
parent = "advanced"
+++
## CMake options
Some of these options are enabled by default ("*Defaults* to *ON*") otherwise they must be expicitly set to *ON*.
#### General
CMake switches to enable / disable parts of OGS.
- `OGS_BUILD_CLI` - Builds the simulator. *Defaults* to *ON*.
- `OGS_BUILD_GUI` - Builds the Data Explorer. *Defaults* to *OFF*.
- `OGS_BUILD_TESTS` - Builds the test executables. *Defaults* to *ON*.
- `OGS_BUILD_UTILS` - Builds several utilities.
- `OGS_NO_EXTERNAL_LIBS` - Disables all external dependencies.
#### Debugging
- `CMAKE_BUILD_TYPE` - Defaults to `Debug` which builds with debugging infos, set to `Release` for an optimized build.
- `OGS_PROFILE` - Builds with profiling flags (`-pg`).
- `OGS_CMAKE_DEBUG` - Prints out the values of all defined CMake variables at CMake configuration time.
#### Optimization
- `CMAKE_BUILD_TYPE` - Set to `Release` to build with optimization flags, set to `Debug` for debugging.
#### Logging
- `OGS_DISABLE_LOGGING` - Disables all logog output messages. This really strips out all logog-code (useful for performance testing).
#### Testing
- `OGS_COVERAGE` - Enables code coverage measurements with gcov/lcov. TODO
#### Advanced options
- `Boost_USE_STATIC_LIBS` - Set this to ON to search for statically linked Boost libraries. Defaults to ON on Windows, to OFF otherwise.
- `OGS_CXX_FLAGS` - Appends user-given compiler flags. Note that existing (CMake-given) flags are not replaced.
- `OGS_ADDITIONAL_SUBMODULES_TO_CHECKOUT` - Specifies optional submodules which are checked out at CMake-time. This is intended to be used by Jenkins.
- `OGS_PACKAGE_ADDITIONAL_BINARIES` - Package additional binaries (given as a `;`-separated list with relative paths to `CMAKE_BINARY_DIR`) into redistributables. Is used for bundling the OGS File Converter with the Data Explorer.
- `OGS_USE_OPTIONAL_SUBMODULES` - Includes optional submodules into the build. The submodules have to be checked out before. Defaults to ON.
- `OGS_CPU_ARCHITECTURE` - Optimizes for the given CPU architecture see [-march](https://gcc.gnu.org/onlinedocs/gcc-4.5.3/gcc/i386-and-x86_002d64-Options.html)-flag. Defaults to `native`. For redistributable binaries set to `generic` on Linux and `core2` on Mac OS.
- `CMAKE_LIBRARY_SEARCH_PATH` - Additional library installation path, e.g. `/opt/local` or `C:/libs`
+++
date = "2018-02-26T11:00:13+01:00"
title = "Docker"
author = "Lars Bilke"
weight = 1034
[menu]
[menu.devguide]
parent = "advanced"
+++
## Introduction
> [Docker](https://www.docker.com/whatisdocker) containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.
See the [docs](https://docs.docker.com) for installation instructions.
## Images
Docker images can be created by [Dockerfiles](https://docs.docker.com/reference/builder/) which can be stacked and thus form a directed graph. OGS-6 image definitions can be found at the [ufz/dockerfiles](https://github.com/ufz/dockerfiles)-repo. Built images can be found at [Docker Hub](https://registry.hub.docker.com/repos/ogs6/).
To build an image by yourself create a Dockerfile:
```
FROM ubuntu:14.04
MAINTAINER Jon Doe <jon.doe@foo.com>
RUN ...
```
Run the `build` command:
```bash
$ docker build -t repo/image_name path/to/directory
```
- `-t` specifies a name for the image, can be arbitrary chosen (but should match the corresponding image on Docker Hub if there is one)
- The path should specify the directory where the Dockerfile is located
Now you can see your build image with `$ docker images`.
## Run a container
To run commands inside a container:
```bash
$ docker run -t image_name command_to_run
```
To run an interactive shell add the `-i`-switch:
```bash
$ docker run -i -t image_name /bin/bash
```
It is useful to mount folders from the host operating system in the Docker container, e.g. to edit source code on your host with your favorite editor:
```bash
$ docker run -i -t -v /host/directory:/container/directory image_name /bin/bash
```
## Prebuilt OGS-6 Docker images
There are docker images provided on [Docker Hub](https://hub.docker.com/u/ogs6/) which include everything necessary to build OGS-6.
There are images for `gcc`, `clang` and `mingw`. For every compiler there is one basic image with the compiler ready (`-dev-essentials`), one image with all dependencies for the OGS simulator (`-base`) and maybe an image for Data Explorer development (`-gui`).
E.g. to start developing OGS:
```bash
docker pull ogs6/gcc-base
docker run -t -i -v /path/to/ogs/source/on/host:/usr/src/ogs ogs6/gcc-base /bin/bash
```
You are now in the Docker container the ogs source code from your host computer mounted at `/usr/src/ogs`. Now you can build and run OGS:
```bash
mkdir build; cd build
cmake ../ogs
make ogs
./bin/ogs
```
web/content/docs/devguide/advanced/gitlab-login.png

130 B

+++
date = "2018-02-26T11:00:13+01:00"
title = "GitLab"
author = "Lars Bilke"
weight = 1038
[menu]
[menu.devguide]
parent = "advanced"
+++
## Introduction
[GitLab](https://gitlab.com) is a web-based Git repository manager similar to [GitHub](https://github.com). GitLab can be self-hosted and we do just that with <https://gitlab.opengeosys.org>. For the moment all OGS repositories are still hosted at [github.com/ufz](https://github.com/ufz) but we store benchmark data files on our own GitLab because of storage limits on GitHub.
To submit a PR containing also new benchmark files (or in general: containing files tracked by `git-lfs`) you need an account on our GitLab server.
## Setup an account
- Creating a GitLab account can be done by simply using your existing GitHub account: click the GitHub logo (octocat) on the [Gitlab sign-in page](https://gitlab.opengeosys.org/users/sign_in)
![](../gitlab-login.png)
- You will be redirected to GitHub (please login there) and asked for authorization.
- Your new user account will be blocked at first, please let us know we will unblock it
- Once unblocked, on the [ogs group page](https://gitlab.opengeosys.org/ogs) click the button `Request access`. We will then give access to the repo. If the button is not there we already gave you access.
- Create a [GitLab personal access token](https://gitlab.opengeosys.org/profile/personal_access_tokens), enable the following scopes: `api`, `read_user`. This is the password you have to use when pushing a PR which also contains `git-lfs`-tracked files. Your user name is the same as your GitHub user name.
- For caching credentials you may want to use a [git credential helper](https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html#credentials-are-always-required-when-pushing-an-object).
+++
date = "2018-02-26T11:00:13+01:00"
title = "Log and Debug Output"
author = "Lars Bilke"
weight = 1035
[menu]
[menu.devguide]
parent = "advanced"
+++
## Introduction
For application output we use [Logog](http://johnwbyrd.github.com/logog) which is already integrated in OGS. Logog provides several verbosity levels which can be used with simple macro calls:
```cpp
ERR("An error message!")
WARN("A warning message.")
INFO("An information message...")
```
As arguments you can use the same functionality as in [sprintf](http://www.cplusplus.com/reference/cstdio/sprintf/):
```cpp
int foo = 9001;
int maxfoo = 9000;
if (foo > maxfoo)
WARN("Foo is over %d! Current value is %d.", maxfoo, foo );
```
For more information see the [Logog documentation](http://johnwbyrd.github.com/logog/quickstart.html).
On release builds the log level is `LOGOG_LEVEL_INFO`, on debug it is `LOGOG_LEVEL_DEBUG` and it can be overriden with the CMake option `OGS_LOG_LEVEL`. You can completely disable logging with the CMake-option `OGS_DISABLE_LOGGING` set to `ON`.
+++
date = "2018-02-26T11:00:13+01:00"
title = "Redistributable builds"
author = "Lars Bilke"
weight = 1037
[menu]
[menu.devguide]
parent = "advanced"
+++
## Introduction
If a binary runs on a different machine depends on a lot of factors. The following sections provide some tips to maximize portability. In general Windows binaries are more portable than *nix binaries.
## CPU Architecture Optimization
::: {.win}
### Default compiler flags
- `/favor:blend`: Optimizes for AMD **and** Intel architectures, see [MSDN docs](https://msdn.microsoft.com/en-us/library/ms173505.aspx)
- `/arch:sse2` on [32-bit](https://msdn.microsoft.com/en-us//library/7t5yh4fd.aspx), `/arch` undefined on [64-bit](https://msdn.microsoft.com/en-us/library/jj620901.aspx)
### Optimization CMake options
- `OGS_CPU_ARCHITECTURE`: possible values `AMD64` or `INTEL64` (sets `/favor:{AMD64 | INTEL64}` flag)
- `OGS_ENABLE_AVX2`: boolean (sets `/arch:AVX2` when set to true), requires at least [Intel Haswell or AMD Excavator-based processors](https://msdn.microsoft.com/en-us//library/7t5yh4fd.aspx)
:::
::: {.linux}
### Default compiler flags
- `-march=native`: Optimizes for current cpu
### CMake options
#### For redistribution
- `OGS_CPU_ARCHITECTURE`: set to `generic` for good balance between optimization and portability; set to `core2` for maximum portability, more info on [gcc docs](https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html#index-mtune-15)
#### For optimization
- `OGS_CPU_ARCHITECTURE`: tot to `native` for best optimization for your current cpu, possible values are [listed here](http:/bilityoverflow.com/a/25095818/80480), more info on [gcc docs](https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html#index-mtune-15)
- `OGS_ENABLE_AVX2`: boolean (sets `-march=core-avx2` when set to true), requires at least Haswell processors
:::
::: {.mac}
See Linux-tab!
:::
## Shared vs. static libraries
Try to use static libraries as much as possible. For OGS itself use `BUILD_SHARED_LIBS=OFF`, which already defaults to `OFF`.
Third-party libraries installed via the systems package package manager are often provided as shared libraries. You may want to use [Conan]({entry:674:url}) which provides lots of libraries (e.g. VTK) as static libs.
## Package the build
Use the `package`-target which tries to gather all dependencies and fixes up shared library paths:
```bash
make package
```
This creates a zip- or tar-archive which should be redistributable.
+++
date = "2018-02-26T11:00:13+01:00"
title = "Using ccache"
author = "Lars Bilke"
weight = 1036
[menu]
[menu.devguide]
parent = "advanced"
+++
::: {.note}
### <i class="fas fa-exclamation-triangle"></i> GCC-like compilers only
Tested on GCC and Clang.
:::
## Introduction
[ccache](https://ccache.samba.org) speeds up compilation times by caching object files and reusing them on subsequent builds. This even works for complete rebuilds (i.e. deleting the full build-directory). If ccache is found on the system it is automatically used.
## Configuration
Add the following line to your [ccache config file](https://ccache.samba.org/manual.html#_configuration) which is required for pre-compiled headers:
```
sloppiness = pch_defines,time_macros
```
See the [ccache docs](https://ccache.samba.org/manual.html#_configuration_settings) for other available options.
### ccache and Clang
Set the option `run_second_cpp = true` or `export CCACHE_CPP2=true` to suppress lots of [false positive compiler warnings](http://peter.eisentraut.org/blog/2014/12/01/ccache-and-clang-part-3/).
## Usage on envinf1
Just load the module:
```
module load /global/apps/modulefiles/ccache/3.3.3
```
+++
date = "2018-02-26T11:00:13+01:00"
title = "Working on envinf1"
author = "Lars Bilke"
weight = 1036
[menu]
[menu.devguide]
parent = "advanced"
+++
## Introduction
Members of the Department Environmental Informatics of the Helmholtz Centre for Environmental Research - UFZ can use the `envinf1`-machine which is tightly connected to the Eve cluster system.
## Build OGS-6
Load required modules by sourcing the environment script:
```bash
$ source scripts/env/envinf1/cli.sh
```
Then do the [build configuration]({{< relref "build-configuration.pandoc" >}}) and [build]({{< relref "build.pandoc" >}}) the project.
## Build the Data Explorer
Load required modules by sourcing the environment script:
```bash
$ source scripts/env/envinf1/gui.sh
```
Then do the [build configuration]({{< relref "build-configuration.pandoc" >}}) and [build]({{< relref "build.pandoc" >}}) the project.
## Install and use Conan
Follow the instructions on using Python's `virtualenv` [on the Eve-wiki](https://wiki.ufz.de/eve/index.php/Python#virtualenv) for setting up a local Python environment. Then you can `pip install conan` and use Conan.
## Additional Features
[Doxygen]({entry:devguide/generating-documentation:url}) documentation:
```bash
$ module load doxygen/1.8.7-1_gcc_4.8.1_CentOS6
```
You can {entry:devguide/build-with-ninja:link}:
```bash
$ module load ninja/1.4.0-1_gcc_4.8.1_CentOS6
```
## Build OGS-5
```bash
$ module load cmake/2.8.12.2-1_gcc_4.8.1_CentOS6
$ module load gcc/4.8.1-3_CentOS6
```
## Use ParaView
On `envinf1` start ParaView-server with e.g. 4 cores:
```bash
module load paraview/5.0.0-egl
mpirun -np 4 pvserver
```
This starts the server on port 11111. If another user has already started the server you need to start your server on another port with e.g.:
```bash
mpirun -np 4 pvserver --server-port=11112
```
On your local PC start ParaView (you have to use the same version, 5.0.0 in this case). And connect to the server on `envinf1.eve.ufz.de:11111` (or specify another port here).
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment