PETSc direct solver and nonlinear convergence
I am playing a little around with the PETSc direct solvers. For a 2D ThermoRichardsMechanics model, I was able to do a little comparison: https://github.com/joergbuchwald/OGS_Solver_Comparison/blob/master/notebooks/ogs_direct_solvers.ipynb I was a little bit surprised that MUMPS and SuperLU are comparable in speed and perform worse than PardisoLU. This might be related to the fact that SuperLU is not able to get the best out of it, if only shared memory parallelization is used, even for such a relatively small model. However, if I change the process from TRM to TRF, SuperLU and MUMPS lead to divergences of the non-linear solver, while the solvers of the eigen library result in convergence. E.g., for Eigen/SparseLU the convergence in the first time step looks like:
info: === Time stepping at step #1 and time 86400 with step size 86400
info: Calculate non-equilibrium initial residuum.
info: [time] Assembly took 0.0255593 s.
info: [time] Applying Dirichlet BCs took 0.00141902 s.
info: ------------------------------------------------------------------
info: *** Eigen solver computation
info: -> scale
info: -> solve with SparseLU
info: ------------------------------------------------------------------
info: [time] Linear solver took 0.128466 s.
info: Convergence criterion, component 0: |dx|=2.7040e+02, |x|=1.6977e+04, |dx|/|x|=1.5928e-02
info: Convergence criterion, component 1: |dx|=1.8349e+07, |x|=2.8672e+09, |dx|/|x|=6.3998e-03
info: [time] Iteration #1 took 0.156709 s.
info: [time] Assembly took 0.0251955 s.
info: [time] Applying Dirichlet BCs took 0.000827585 s.
info: ------------------------------------------------------------------
info: *** Eigen solver computation
info: -> scale
info: -> solve with SparseLU
info: ------------------------------------------------------------------
info: [time] Linear solver took 0.112777 s.
info: Convergence criterion, component 0: |dx|=1.2801e+00, |x|=1.6977e+04, |dx|/|x|=7.5402e-05
info: Convergence criterion, component 1: |dx|=1.6601e+07, |x|=2.8701e+09, |dx|/|x|=5.7842e-03
info: [time] Iteration #2 took 0.14005 s.
info: [time] Assembly took 0.0252076 s.
info: [time] Applying Dirichlet BCs took 0.000828188 s.
info: ------------------------------------------------------------------
info: *** Eigen solver computation
info: -> scale
info: -> solve with SparseLU
info: ------------------------------------------------------------------
info: [time] Linear solver took 0.114727 s.
info: Convergence criterion, component 0: |dx|=8.3958e-02, |x|=1.6977e+04, |dx|/|x|=4.9455e-06
info: Convergence criterion, component 1: |dx|=1.0971e+05, |x|=2.8701e+09, |dx|/|x|=3.8224e-05
info: [time] Iteration #3 took 0.142027 s.
info: [time] Assembly took 0.025058 s.
info: [time] Applying Dirichlet BCs took 0.000829301 s.
info: ------------------------------------------------------------------
info: *** Eigen solver computation
info: -> scale
info: -> solve with SparseLU
info: ------------------------------------------------------------------
info: [time] Linear solver took 0.112178 s.
info: Convergence criterion, component 0: |dx|=2.3529e-03, |x|=1.6977e+04, |dx|/|x|=1.3860e-07
info: Convergence criterion, component 1: |dx|=6.4345e+03, |x|=2.8701e+09, |dx|/|x|=2.2419e-06
info: [time] Iteration #4 took 0.139362 s.
info: [time] Assembly took 0.0249046 s.
info: [time] Applying Dirichlet BCs took 0.000832059 s.
info: ------------------------------------------------------------------
info: *** Eigen solver computation
info: -> scale
info: -> solve with SparseLU
while the same iteration for PETSc/SuperLU look like:
[0] info: Calculate non-equilibrium initial residuum.
[0] info: [time] Assembly took 0.0280363 s.
[0] info: [time] Applying Dirichlet BCs took 0.000159599 s.
================================================
Linear solver gmres with lu preconditioner
converged in 1 iterations (relative convergence criterion fulfilled).
================================================
[0] info: [time] Linear solver took 0.0704457 s.
[0] info: Convergence criterion, component 0: |dx|=2.7047e+02, |x|=1.6977e+04, |dx|/|x|=1.5932e-02
[0] info: Convergence criterion, component 1: |dx|=1.8351e+07, |x|=2.8672e+09, |dx|/|x|=6.4004e-03
[0] info: [time] Iteration #1 took 0.100025 s.
[0] info: [time] Assembly took 0.0314264 s.
[0] info: [time] Applying Dirichlet BCs took 4.282e-05 s.
================================================
Linear solver gmres with lu preconditioner
converged in 1 iterations (relative convergence criterion fulfilled).
================================================
[0] info: [time] Linear solver took 0.061531 s.
[0] info: Convergence criterion, component 0: |dx|=1.9299e+01, |x|=1.6971e+04, |dx|/|x|=1.1372e-03
[0] info: Convergence criterion, component 1: |dx|=8.9796e+09, |x|=6.2842e+09, |dx|/|x|=1.4289e+00
[0] info: [time] Iteration #2 took 0.0943907 s.
[0] info: [time] Assembly took 0.029758 s.
[0] info: [time] Applying Dirichlet BCs took 5.0979e-05 s.
================================================
Linear solver gmres with lu preconditioner
converged in 1 iterations (relative convergence criterion fulfilled).
================================================
[0] info: [time] Linear solver took 0.0443403 s.
[0] info: Convergence criterion, component 0: |dx|=9.9866e+01, |x|=1.6943e+04, |dx|/|x|=5.8943e-03
[0] info: Convergence criterion, component 1: |dx|=8.9881e+09, |x|=2.8760e+09, |dx|/|x|=3.1252e+00
[0] info: [time] Iteration #3 took 0.0755801 s.
[0] info: [time] Assembly took 0.0328867 s.
[0] info: [time] Applying Dirichlet BCs took 5.8344e-05 s.
================================================
Linear solver gmres with lu preconditioner
converged in 1 iterations (relative convergence criterion fulfilled).
================================================
[0] info: [time] Linear solver took 0.0453074 s.
[0] info: Convergence criterion, component 0: |dx|=1.1677e+02, |x|=1.6976e+04, |dx|/|x|=6.8784e-03
[0] info: Convergence criterion, component 1: |dx|=1.2890e+10, |x|=1.1174e+10, |dx|/|x|=1.1535e+00
[0] info: [time] Iteration #4 took 0.0796937 s.
[0] info: [time] Assembly took 0.0314263 s.
[0] info: [time] Applying Dirichlet BCs took 5.0755e-05 s.
Divergence detected, use command option -ksp_monitor or -log_summary to check the details.
[0] info: [time] Linear solver took 0.0464852 s.
[0] error: Newton: The linear solver failed.
[0] info: [time] Solving process #0 took 0.427737 s in time step #1
[0] error: The nonlinear solver failed in time step #1 at t = 86400 s for process #0.
[0] info: [time] Output of timestep 1 took 0.0167153 s.
[0] critical: /home/buchwalj/github/ogs/ProcessLib/TimeLoop.cpp:733 solveUncoupledEquationSystems()
[0] error: Time stepper cannot reduce the time step size further.
[0] info: OGS terminated on 2021-09-01 21:49:01+0200.
[0] error: OGS terminated with error.
From my understanding, there should be no differences in the solution between different direct solvers. Does anyone have an idea what could have caused these differences?