mpi-based SOILWAT2 keeps input and output files open requiring a large number of available file descriptors. The number depends on the number of input files and variables, the number of output groups and variables, and the number of temporal strides across which the output is distributed.
On some platforms with some compiler and settings, an insufficient number of file descriptors leads to a segfault, e.g.,
*** Process received signal ***
Signal: Segmentation fault: 11 (11)
Signal code: Invalid permissions (2)
Failing at address: 0x10
[ 0] 0 libsystem_platform.dylib 0x000000019e650624 _sigtramp + 56
[ 1] 0 libmpi.40.dylib 0x0000000100b291cc mca_common_ompio_file_close + 764
[ 2] 0 libmpi.40.dylib 0x0000000100b291cc mca_common_ompio_file_close + 764
[ 3] 0 libmpi.40.dylib 0x0000000100bc6c7c mca_io_ompio_file_close + 28
[ 4] 0 libmpi.40.dylib 0x0000000100a97048 file_destructor + 44
[ 5] 0 libmpi.40.dylib 0x0000000100a97814 ompi_file_open + 408
[ 6] 0 libmpi.40.dylib 0x0000000100af8b88 MPI_File_open + 420
[ 7] 0 libnetcdf.22.dylib 0x000000010061fc50 openmagic + 372
[ 8] 0 libnetcdf.22.dylib 0x000000010061e828 check_file_type + 132
[ 9] 0 libnetcdf.22.dylib 0x000000010061d3d8 NC_infermodel + 1272
[10] 0 libnetcdf.22.dylib 0x00000001006012f8 NC_open + 520
[11] 0 libnetcdf.22.dylib 0x000000010062d4c0 nc_open_par + 80
[12] 0 SOILWAT2 0x0000000100242fa0 SW_NC_open_par + 48
[13] 0 SOILWAT2 0x00000001002561a4 SW_MPI_open_files + 2348
[14] 0 SOILWAT2 0x0000000100214b10 main + 1080
[15] 0 dyld 0x000000019e276b98 start + 6076
*** End of error message ***
This should not result in a segfault but captured as error, and the program should properly terminate early.
Instead of a segfault, other platforms and/or with other compiler settings, an insufficient number of file descriptors leads to an error message, e.g.,
mca_sharedfp_sm_file_open: Error, unable to open file for mmap: /path/to/temporaryFile.nc_xxxx.sm
and
Mem_Malloc() by open_output_files() called with existing error.
However, SOILWAT2 gets stuck indefinitely after this error if there is more than one I/O process.
mpi-based SOILWAT2 keeps input and output files open requiring a large number of available file descriptors. The number depends on the number of input files and variables, the number of output groups and variables, and the number of temporal strides across which the output is distributed.
On some platforms with some compiler and settings, an insufficient number of file descriptors leads to a segfault, e.g.,
This should not result in a segfault but captured as error, and the program should properly terminate early.
Instead of a segfault, other platforms and/or with other compiler settings, an insufficient number of file descriptors leads to an error message, e.g.,
and
However, SOILWAT2 gets stuck indefinitely after this error if there is more than one I/O process.