Job Failed #293581 (benchmarks hang when petsc solver is not available)
Benchmarks from !4495 (merged) hang on frontend petsc jobs and time out in the end. Exemplary log output:
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: See https://petsc.org/release/overview/linear_solve_table/ for possible LU and Cholesky solvers
[0]PETSC ERROR: Could not locate a solver type for factorization type LU and matrix type mpiaij.
[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.16.5, unknown
[0]PETSC ERROR: /tmp/sonjenk/builds/k1CXQDSZ/0/ogs/build/release-petsc/bin/ogs on a named frontend2 by sonjenk Thu Mar 2 10:19:41 2023
[0]PETSC ERROR: Configure options --download-f2cblaslapack=1 --download-hypre --with-debugging=0 --with-fc=0 --with-cc=/software/easybuild-broadwell/software/OpenMPI/4.0.5-GCC-10.2.0/bin/mpicc --with-cxx=/software/easybuild-broadwell/software/OpenMPI/4.0.5-GCC-10.2.0/bin/mpic++ --prefix=/global/apps/petsc-bilke/3.16.5.foss_2020b-hypre
[0]PETSC ERROR: #1 MatGetFactor() at /gpfs0/home/bilke/code/petsc/src/mat/interface/matrix.c:4776
[0]PETSC ERROR: #2 PCSetUp_LU() at /gpfs0/home/bilke/code/petsc/src/ksp/pc/impls/factor/lu/lu.c:82
[0]PETSC ERROR: #3 PCSetUp() at /gpfs0/home/bilke/code/petsc/src/ksp/pc/interface/precon.c:1017
[0]PETSC ERROR: #4 KSPSetUp() at /gpfs0/home/bilke/code/petsc/src/ksp/ksp/interface/itfunc.c:408
[0]PETSC ERROR: #5 KSPSolve_Private() at /gpfs0/home/bilke/code/petsc/src/ksp/ksp/interface/itfunc.c:852
[0]PETSC ERROR: #6 KSPSolve() at /gpfs0/home/bilke/code/petsc/src/ksp/ksp/interface/itfunc.c:1086
See also job #293581 failed for 9932e50a.
Two issues:
-
Petsc on frontends is not compiled with LU solver -
Benchmarks should fail and not hang if solver is not available