Skip to content

[CI] Introduce maintenance tests to prevent regular developers from encountering pipeline errors unrelated to their merge requests

Separate regression and maintenance testing. Introduce maintenance tests. Reworked regression tests now prevent regular developers from encountering pipeline errors unrelated to their merge requests Maintenance test signal as early as possible to maintainers if OGS is "future-proof".

From:

Regression test
OGS latest
Environment latest
Trigger Merge-request

to:

Regression test Maintenance test
OGS latest stable (tip of master)
Environment stable (pinned) latest
Trigger Merge-request scheduled
Role Developer Maintainer

latest indicates the "object under test"

Environment may be:

  • pip (from PyPI)
  • CPM (FindPackage, GIT_TAG / VERSION) - not in this MR
  • Compiler version - not in this MR
  • ...

When to update a "stable" an environment:

  • MUST: Change of underlying stack (e.g. Relevant update of envinf)
  • MUST: An MR with a change in the request environment (e.g. a new lib or a new version of an already used lib is needed)
  • OPTIONAL: On any other occasion (e.g. OGS release)

General idea was tested for several months in OGSTools successfully:

image

After a pyvista update to 0.46 the maintenance tests signaled succesfully a problem. But regulare developers are not disturbed (2 later pipelines are running)


Implemented with uv. New pyproject.toml in Tests/Data specifying notebook test dependencies (formerly defined in requirements*.txt-files, see Tests/Data/README.md for usage.

Typical Workflow:

  • Trigger: Maintenance test fails
    1. Check latest updates of 3rd party packages and their influence
    1. MR: Quick fix: Restrict versions in (open) dependencies, add/open ToDo
    1. Maintenance test succeeds (this allows other maintenance issues to be found)
    1. Investigation and fix of own software / report to 3rd party
    1. MR: Fix and reopen dependency
Edited by Tobias Meisel

Merge request reports

Loading