Compiling fails at TACC because `cmake` passes the `-fallow-argument-mismatch` Fortran argument erroneously to the c compiler

[grantcurell]

When you run cmake you see in the output -- (EC) Our code has not yet been updated to work with GNU compilers 10.x; adding extra options to be more permissive: -fallow-argument-mismatch.. Later when you try to run the build you get cc1: error: command-line option ‘-fallow-argument-mismatch’ is valid for Fortran but not for C [-Werror]. It appears cmake is incorrectly passing a Fortran-only argument to the c compiler. Because warnings are set to cause failure, this leads to the build failing.

login2.ls6(189)$ cado cmake
Using build directory '/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0'
/work/10644/grantcurell/gem/cado: running 'cmake' command:
cmake -S . -B /work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0
-- (EC) EC_CMAKE_RPN_DIR /work/10644/grantcurell/gem/cmake_rpn/modules
-- (EC) COMPILER_SUITE=gnu
-- (EC) Git version: 5.3.0-a11-8-gfa4c1db
-- (EC) GEM VERSION = 5.3.0-a11
-- (EC) Adding build_info target
-- (EC) Configuring for RelWithDebInfo build type
-- (EC) Using default presets: default/Linux-x86_64/gnu
-- (EC) Target architecture: native
-- (EC) Our code has not yet been updated to work with GNU compilers 10.x; adding extra options to be more permissive: -fallow-argument-mismatch.
-- (EC) Building WITH OpenMP
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_Fortran: -fopenmp (found version "4.5")
-- (EC) Building WITHOUT OpenACC
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - found
-- Found BLAS: /scratch/projects/compilers/intel24.1/oneapi/mkl/2024.1/lib/libmkl_gf_lp64.so;/scratch/projects/compilers/intel24.1/oneapi/mkl/2024.1/lib/libmkl_gnu_thread.so;/scratch/projects/compilers/intel24.1/oneapi/mkl/2024.1/lib/libmkl_core.so;/scratch/tacc/apps/gcc/13.2.0/lib64/libgomp.so;-lpthread;-lm;-ldl
-- Looking for Fortran cheev
-- Looking for Fortran cheev - found
-- Found LAPACK: /scratch/projects/compilers/intel24.1/oneapi/mkl/2024.1/lib/libmkl_gf_lp64.so;/scratch/projects/compilers/intel24.1/oneapi/mkl/2024.1/lib/libmkl_gnu_thread.so;/scratch/projects/compilers/intel24.1/oneapi/mkl/2024.1/lib/libmkl_core.so;/scratch/tacc/apps/gcc/13.2.0/lib64/libgomp.so;-lpthread;-lm;-ldl;-lpthread;-lm;-ldl
-- (EC) CMAKE_C_FLAGS=-ftree-vectorize -march=native -Wall -Wpedantic
-- (EC) CMAKE_Fortran_FLAGS=-fconvert=big-endian -fcray-pointer -frecord-marker=4 -fno-second-underscore -ffree-line-length-none -finit-real=nan -ftree-vectorize -march=native -fstack-arrays -Wall -fallow-argument-mismatch
-- (EC) Sources directory is: /work/10644/grantcurell/gem
-- (EC) Build directory is: /work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0
-- (EC) Operating system is: RockyLinux-8.10-x86_64
-- (EC) Compiler version: 13.2.0
-- (EC) Setting working directory prefix to /work/10644/grantcurell/gem/work-RockyLinux-8.10-x86_64-gnu-13.2.0
-- (EC) Using RPN-SI installed libraries = FALSE
-- (EC) Generating librmn Makefiles
-- (EC) Adding /work/10644/grantcurell/gem/cmake_rpn/modules to CMAKE_MODULE_PATH
-- (EC) EC Initialisation already done
-- (EC) rmn version = 20.1.0-a8
-- (EC) Adding build_info target
-- (EC) Generating external dependencies
-- (EC) Generating json-c dependency
-- Wrote /work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/rmn/dep/json-c/config.h
-- Wrote /work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/rmn/dep/json-c/json_config.h
-- Wrote /work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/rmn/dep/json-c/doc/Doxyfile
-- (EC) Generating udunits2 dependency
DEFAULT_UDUNITS2_XML_PATH = "/work/10644/grantcurell/gem/work-RockyLinux-8.10-x86_64-gnu-13.2.0/share/udunits/udunits2.xml"
-- Unable to find CUnit library. Unit-testing not enabled.
-- (EC) EC Initialisation already done
-- (EC) Adding build_info target
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_Fortran: -fopenmp (found version "4.5")
-- Generating App library
-- Generating App ompi library
-- Build architecture: Linux-x86_64
CMake Warning (dev) at src/rpn-si/rmn/CMakeLists.txt:110 (add_custom_command):
  The following keywords are not supported when using
  add_custom_command(OUTPUT): PRE_BUILD.

  Policy CMP0175 is not set: add_custom_command() rejects invalid arguments.
  Run "cmake --help-policy CMP0175" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) at src/rpn-si/rmn/CMakeLists.txt:121 (add_custom_command):
  The following keywords are not supported when using
  add_custom_command(OUTPUT): PRE_BUILD.

  Policy CMP0175 is not set: add_custom_command() rejects invalid arguments.
  Run "cmake --help-policy CMP0175" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- (EC) Python interface will NOT be built
-- (EC) Generating tdpack Makefile
-- (EC) EC Initialisation already done
-- (EC) tdpack version = 1.6.3.6
-- (EC) Adding build_info target
-- (EC) Generating vgrid Makefile
-- (EC) EC Initialisation already done
-- (EC) vgrid version = 6.9.6
-- (EC) Adding build_info target
-- (EC) Building WITH OpenMP
-- (EC) Generating vgrid librairie
-- (EC) Generating targets for vgrid utilities
-- (EC) Generating rpncomm Makefile
-- (EC) EC Initialisation already done
-- (EC) rpncomm version = 4.6.4
-- (EC) Generating modelutils Makefile
-- (EC) EC Initialisation already done
-- (EC) modelutils version = 1.8.0-a10
-- (EC) Adding build_info target
-- (EC) Building WITH OpenMP
-- (EC) Generating modelutils main Makefile
-- (EC) Generating modelutils librairies Makefile
-- (EC) Generating rpnphy Makefile
-- (EC) EC Initialisation already done
-- (EC) rpnphy version = 6.3.0-a10
-- (EC) Adding build_info target
-- (EC) Building WITH OpenMP
-- (EC) Generating rpnphy binaries Makefile
-- (EC) Generating rpnphy libraries Makefile
-- (EC) Generating gemdyn Makefile
-- (EC) EC Initialisation already done
-- (EC) gemdyn version = 5.3.0-a10
-- (EC) Adding build_info target
-- (EC) Building WITH OpenMP
-- (EC) Generating gem_monitor* and toc2nml makefiles
-- (EC) Generating gemdyn libraries Makefile
-- (EC) Generating gemdyn binaries Makefile
-- (EC) Generating maingem* Makefile
-- (EC) Generating cclargs_lite Makefile
-- (EC) Generating fst-tools Makefile
-- (EC) EC Initialisation already done
-- (EC) fst-tools version = 20.1.0-a5
-- (EC) Adding build_info target
-- (EC) Generating filetype Makefile
-- Configuring done (13.4s)
-- Generating done (3.9s)
-- Build files have been written to: /work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0
/work/10644/grantcurell/gem/cado: Command 'cmake' completed successfully:
cmake -S . -B /work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0
login2.ls6(190)$ cado work -j
Using build directory '/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0'
/work/10644/grantcurell/gem/cado: running 'work' command:
cmake --build /work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0 --target work -- -j
[  1%] Built target RPN_MPI_interfaces
[  1%] Generating rmn_build_info.h
[  1%] Generating rpnphy_build_info.h
[  1%] Generating fst-tools_build_info.h
[  1%] Generating modelutils_build_info.h
[  1%] Generating vgrid_build_info.h
[  1%] Built target cclargs_lite
[  1%] Built target udunits2_doc
[  1%] Generating tdpack_build_info.h
[  1%] Built target ArrayGetter
[  2%] Built target WhiteBoardFortranHeader
[  2%] Built target gem_monitor_end
[  2%] Built target RPN_COMM_interfaces
[  2%] Generating App_build_info.h
[  3%] Generating gemdyn_build_info.h
[  3%] Generating gem_build_info.h
[  3%] Built target c_binding_extras
[  3%] Built target json-c
[  3%] Built target libudunits2-static
[  3%] Built target gem_monitor_output
[  4%] Built target json-c-static
-- (EC) BUILD_INFO_OUTPUT_DIR=/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/fst-tools
-- (EC) BUILD_INFO_OUTPUT_DIR=/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/tdpack
[  4%] Built target libudunits2-shared
[  4%] Built target c_binding_extras_include
-- (EC) BUILD_INFO_OUTPUT_DIR=/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/vgrid
-- (EC) BUILD_INFO_OUTPUT_DIR=/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/rmn/App
[  4%] Built target tdpack_build_info
[  4%] Built target fst-tools_build_info
[  4%] Built target vgrid_build_info
[  4%] Built target App_build_info
-- (EC) BUILD_INFO_OUTPUT_DIR=/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/rmn
[  4%] Built target rmn_build_info
[  4%] Built target udunits2
[  6%] Built target tdpack-static
[  6%] Building C object src/rmn/App/src/CMakeFiles/App-ompi-static.dir/App_MPMD.c.o
[  6%] Building C object src/rmn/App/src/CMakeFiles/App-ompi-static.dir/App_Timer.c.o
[  6%] Building C object src/rmn/App/src/CMakeFiles/App-static.dir/App.c.o
[  6%] Building C object src/rmn/App/src/CMakeFiles/App-ompi-static.dir/str.c.o
[  6%] Building C object src/rmn/App/src/CMakeFiles/App-ompi-static.dir/App.c.o
[  6%] Built target tdpack-shared
cc1: error: command-line option ‘-fallow-argument-mismatch’ is valid for Fortran but not for C [-Werror]
cc1: error: command-line option ‘-fallow-argument-mismatch’ is valid for Fortran but not for C [-Werror]
cc1: error: command-line option ‘-fallow-argument-mismatch’ is valid for Fortran but not for C [-Werror]
cc1: error: command-line option ‘-fallow-argument-mismatch’ is valid for Fortran but not for C [-Werror]
-- (EC) BUILD_INFO_OUTPUT_DIR=/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/gemdyn
-- (EC) BUILD_INFO_OUTPUT_DIR=/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/rpnphy
-- (EC) BUILD_INFO_OUTPUT_DIR=/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0/src/modelutils
-- (EC) BUILD_INFO_OUTPUT_DIR=/work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0
[  6%] Built target gem_build_info
[  6%] Built target gemdyn_build_info
[  6%] Building Fortran object src/rpncomm/src/CMakeFiles/rpncomm.dir/RPN_MPI_get_mpi_definitions.F90.o
[  6%] Built target rpnphy_build_info
[  6%] Built target modelutils_build_info
In file included from /work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App_Timer.c:1:
/work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App_Timer.h:22:38: warning: initializer element is not constant [-Wpedantic]
   22 | static const TApp_Timer NULL_TIMER = (const TApp_Timer) {
      |                                      ^
In file included from /work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App.h:6,
                 from /work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App_MPMD.h:6,
                 from /work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App_MPMD.c:1:
/work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App_Timer.h:22:38: warning: initializer element is not constant [-Wpedantic]
   22 | static const TApp_Timer NULL_TIMER = (const TApp_Timer) {
      |                                      ^
/work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App_MPMD.c:24:45: warning: initializer element is not constant [-Wpedantic]
   24 | static const TComponent default_component = (TComponent){
      |                                             ^
/work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App_MPMD.c:33:42: warning: initializer element is not constant [-Wpedantic]
   33 | static const TComponentSet default_set = (TComponentSet) {
      |                                          ^
/work/10644/grantcurell/gem/src/rpn-si/rpncomm/src/RPN_MPI_get_mpi_definitions.F90:103:14:

  103 |     ml%host = get_host_id()              ! get linux host id
      |              1
Warning: Possible change of value in conversion from INTEGER(8) to INTEGER(4) at (1) [-Wconversion]
/work/10644/grantcurell/gem/src/rpn-si/rpncomm/src/RPN_MPI_get_mpi_definitions.F90:109:16:

  109 |       lr%host = get_host_id()            ! get linux host id
      |                1
Warning: Possible change of value in conversion from INTEGER(8) to INTEGER(4) at (1) [-Wconversion]
In file included from /work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App.h:6,
                 from /work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App.c:23:
/work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App_Timer.h:22:38: warning: initializer element is not constant [-Wpedantic]
   22 | static const TApp_Timer NULL_TIMER = (const TApp_Timer) {
      |                                      ^
cc1: all warnings being treated as errors
make[6]: *** [src/rmn/App/src/CMakeFiles/App-ompi-static.dir/build.make:93: src/rmn/App/src/CMakeFiles/App-ompi-static.dir/App_Timer.c.o] Error 1
make[6]: *** Waiting for unfinished jobs....
cc1: all warnings being treated as errors
make[6]: *** [src/rmn/App/src/CMakeFiles/App-ompi-static.dir/build.make:107: src/rmn/App/src/CMakeFiles/App-ompi-static.dir/str.c.o] Error 1
In file included from /work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App.h:6,
                 from /work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App.c:23:
/work/10644/grantcurell/gem/src/rpn-si/rmn/App/src/App_Timer.h:22:38: warning: initializer element is not constant [-Wpedantic]
   22 | static const TApp_Timer NULL_TIMER = (const TApp_Timer) {
      |                                      ^
cc1: all warnings being treated as errors
make[6]: *** [src/rmn/App/src/CMakeFiles/App-ompi-static.dir/build.make:121: src/rmn/App/src/CMakeFiles/App-ompi-static.dir/App_MPMD.c.o] Error 1
[  6%] Linking Fortran static library librpncomm_4.6.4_2021.12.a
'rpn_comm.inc' -> 'RPN_COMM.inc'
[  6%] Linking Fortran static library libApp.a
cc1: all warnings being treated as errors
make[6]: *** [src/rmn/App/src/CMakeFiles/App-ompi-static.dir/build.make:79: src/rmn/App/src/CMakeFiles/App-ompi-static.dir/App.c.o] Error 1
make[5]: *** [CMakeFiles/Makefile2:2950: src/rmn/App/src/CMakeFiles/App-ompi-static.dir/all] Error 2
make[5]: *** Waiting for unfinished jobs....
[  6%] Built target App-static
'librpncomm_2021.12.a' -> 'librpncomm_4.6.4_2021.12.a'
'librpncomm.a' -> 'librpncomm_4.6.4_2021.12.a'
'librpn_comm.a' -> 'librpncomm_4.6.4_2021.12.a'
'rpn_comm.inc' -> 'RPN_COMM.inc'
[ 11%] Built target rpncomm
make[4]: *** [Makefile:156: all] Error 2
gmake[3]: *** [CMakeFiles/work.dir/build.make:73: CMakeFiles/work] Error 2
gmake[2]: *** [CMakeFiles/Makefile2:1314: CMakeFiles/work.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:1321: CMakeFiles/work.dir/rule] Error 2
gmake: *** [Makefile:202: work] Error 2
/work/10644/grantcurell/gem/cado: Command 'work' completed failed with code 2:
cmake --build /work/10644/grantcurell/gem/build-RockyLinux-8.10-x86_64-gnu-13.2.0 --target work -- -j
login2.ls6(191)$

[grantcurell]

Back at it today. The build system does not seem to honor compiler flags. For example cado cmake -DCMAKE_COMPILE_WARNING_AS_ERROR=OFF does not work. The build system still passes this flag as ON even though the Fortran flags for the c compiler are harmless.

[grantcurell]

If anyone else encounters this, I wasn't able to get cado to work. I ultimately did this:

module load autotools/1.4   pmix/3.2.3   xalt/3.1   TACC   gcc/13.2.0  impi/21.12    fftw3/3.3.10   cmake/3.29.5   mkl/24.1
git clone --recursive https://github.com/ECCC-ASTD-MRD/gem.git
ln -s /scratch/tacc/apps/gcc13_2/impi21/fftw3/3.3.10/include/* src/gemdyn/gemdyn/CMakeFiles/gemdyn.dir/base/
cd gem
git checkout benchmark-5.3
git submodule update --init --recursive
./download-dbase.sh .
./download-dbase-benchmarks.sh .
. ./.common_setup gnu
mkdir build
cd build
cmake ..
make -j$(nproc)

and that built correctly. I'm not sure why, but I couldn't get make to pick up fftw so I just did a symbolic link to get the compilation over the hump.

I did try cleaning up the fallow flags and that will get you further down the compilation pipeline, but it would still error out in various different places:

fix_gem_flags() {
  build_dir="build-${GEM_ARCH}"
  echo "⟹ Scrubbing -fallow-argument-mismatch and -Werror from $build_dir …"
  find "$build_dir" -name 'flags.make' -print0 |
    xargs -0 sed -i.bak -e 's/-fallow-argument-mismatch//g' \
                        -e 's/[[:space:]]-Werror[[:space:]]/ /g'
  echo "   done."
}