Could this be related to this Merge Request: !4132 (merged)
It seems OGS now links to libgomp
and libiomp5
again:
Container Build:
Apptainer> ldd /usr/local/ogs/build/bin/ogs | grep omp
libiomp5.so => /opt/intel/oneapi/compiler/2021.4.0/linux/compiler/lib/intel64_lin/libiomp5.so (0x00007fe935200000)
libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fe935633000)
Native Build:
ldd ogs | grep omp
libiomp5.so => /opt/intel/oneapi/compiler/2023.1.0/linux/compiler/lib/intel64_lin/libiomp5.so (0x00007f90c2800000)
libgomp.so.1 => /home/ogs_auto_jenkins/temporary_versions/native/master/ogs6_2024-03-17/bin/../lib64/libgomp.so.1 (0x00007f90c2d45000)
FYI @ThieJan
The new MKL version does not automatically provide blas and lapack. On our systems we are compiling them, so that they can be used by PETSc.
I have been trying to create an MR for ogscm, but I am missing something.
I am trying to add these commands to be executed before PETSc configure
make --directory=/opt/intel/oneapi/mkl/2021.4.0/lib --makefile=/opt/intel/oneapi/mkl/2021.4.0/tools/builder/makefile libintel64 interface=lp64 export=/opt/intel/oneapi/mkl/2021.4.0/tools/builder/blas_example_list name=/opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libbl>
make --directory=/opt/intel/oneapi/mkl/2021.4.0/lib --makefile=/opt/intel/oneapi/mkl/2021.4.0/tools/builder/makefile libintel64 interface=lp64 export=/opt/intel/oneapi/mkl/2021.4.0/tools/builder/lapack_example_list name=/opt/intel/oneapi/mkl/2021.4.0/lib/intel64/lib>
ln -s /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libblas.so /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libblas.so.3 && \
ln -s /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/liblapack.so /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/liblapack.so.3 && \
The result should look like this:
# http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-lite-3.18.6.tar.gz
RUN mkdir -p /var/tmp && wget -q -nc --no-check-certificate -P /var/tmp http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-lite-3.18.6.tar.gz && \
mkdir -p /var/tmp && tar -x -f /var/tmp/petsc-lite-3.18.6.tar.gz -C /var/tmp -z && \
make --directory=/opt/intel/oneapi/mkl/2021.4.0/lib --makefile=/opt/intel/oneapi/mkl/2021.4.0/tools/builder/makefile libintel64 interface=lp64 export=/opt/intel/oneapi/mkl/2021.4.0/tools/builder/blas_example_list name=/opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libbl>
make --directory=/opt/intel/oneapi/mkl/2021.4.0/lib --makefile=/opt/intel/oneapi/mkl/2021.4.0/tools/builder/makefile libintel64 interface=lp64 export=/opt/intel/oneapi/mkl/2021.4.0/tools/builder/lapack_example_list name=/opt/intel/oneapi/mkl/2021.4.0/lib/intel64/lib>
ln -s /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libblas.so /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libblas.so.3 && \
ln -s /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/liblapack.so /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/liblapack.so.3 && \
cd /var/tmp/petsc-3.18.6 && CC=mpicc CXX=mpicxx F77=mpif77 F90=mpif90 FC=mpifort ./configure --prefix=/usr/local/petsc --CFLAGS='-O3' --CXXFLAGS='-O3' --FFLAGS='-O3' --download-f2cblaslapack=0 --download-f2cblaslapack=1 --download-fc --download-hypre --download-mum>
make -j$(nproc) && \
make -j$(nproc) install && \
echo "/usr/local/petsc/lib" >> /etc/ld.so.conf.d/hpccm.conf && ldconfig && \
rm -rf /var/tmp/petsc-3.18.6 /var/tmp/petsc-lite-3.18.6.tar.gz
ENV CMAKE_PREFIX_PATH=/usr/local/petsc
My changes to ogs.py so far: kristofkessler/container-maker@main...mkl_blaslapack
What am I missing? Thanks.
Thanks. The MR fixed this issue.
Kristof Kessler (9dd5d0e1) at 08 Mar 07:53
Not sure why the pipeline fails. It seems to be an issue with code style/formatting.
Kristof Kessler (2cdc2dbf) at 06 Mar 15:21
changed code styling
Fix for #15
In the end, it was not necessary to build the libraries. The problem was that --download-f2cblaslapack=1
is always set, if --download-ptscotch
is a PETSc argument. If blaslapack is downloaded then it seems that the libraries in the MKL folder are ignored.
Kristof Kessler (4994795c) at 06 Mar 10:24
added check if --download-f2cblaslapack is already specified as pet...
Kristof Kessler (7e5997d6) at 06 Mar 10:22
Kristof Kessler (37e959af) at 06 Mar 08:52
Update ogs.py
Kristof Kessler (05173790) at 06 Mar 08:44
fixed typo
Kristof Kessler (7ff3e3ba) at 06 Mar 08:42
added check if --download-f2cblaslapack is already specified as pet...
Kristof Kessler (991507fe) at 06 Mar 08:37
removed build of blas lapack libraries
Kristof Kessler (34473088) at 01 Mar 16:06
Update ogs.py
Kristof Kessler (80798623) at 01 Mar 13:45
Update ogs.py
Kristof Kessler (8191c0c1) at 01 Mar 08:26
Update ogs.py
Kristof Kessler (347784ea) at 29 Feb 15:35
Update ogs.py
Kristof Kessler (2fc89629) at 29 Feb 15:35
Update ogs.py
Kristof Kessler (7b202255) at 29 Feb 15:33
Update ogs.py
Kristof Kessler (371cf77f) at 29 Feb 15:29
Update ogs.py