configure fails to find cfitsio

[majashdown]

The configure script fails to work with recent versions of cfitsio, even if the location of the library explicitly passed to it:

...
checking fitsio.h usability... yes
checking fitsio.h presence... yes
checking for fitsio.h... yes
checking for ffopen in user specified location... no
checking for ffopen in default location... no
configure: error: Could not find the CFITSIO library!

I think this is because cfitsio needs to be linked with -lcurl.

[keskitalo]

Thank you for the report, Mark. I just tried running the configure script on my OSX system. I have cfitsio installed through macports. Like you said, configure first failed to find cfitsio. When I explicitly used --with-cfitsio=/opt/local it worked fine.

Does it solve the problem for you as well? If not, can you provide more details about the system you are running on?

[majashdown]

I am doing this on Cori, using the GNU compilers (7.1.0). I have compiled the most recent version of cfitsio, 3.42. I am explicitly passing the location of cfitsio to configure:

CC=cc MPICC=cc CXX=CC MPICXX=CC ./configure --prefix=/global/homes/m/maja1/programs/hpc4cmb/install/cori --use-cfitsio=/global/homes/m/maja1/programs/hpc4cmb/install/cori

It is finding fitsio.h, but it fails to link the test program that calls ffopen. The config.log file shows the failure is caused by some undefined references.

configure:17832: checking for fitsio.h
configure:17832: result: yes
configure:17843: checking for ffopen in user specified location
configure:17863: cc -o conftest -O3 -fopenmp  -I/global/homes/m/maja1/programs/hpc4cmb/install/cori/include  -fopenmp conftest.c -L/global/homes/m/maja1/programs/hpc4cmb/install/cori/lib -lcfitsio  -lm >&5
/global/homes/m/maja1/programs/hpc4cmb/install/cori/lib/libcfitsio.a(drvrfile.o): In function `file_openfile':
/global/homes/m/maja1/programs/hpc4cmb/cfitsio/drvrfile.c:232: warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/global/homes/m/maja1/programs/hpc4cmb/install/cori/lib/libcfitsio.a(cfileio.o): In function `fits_init_cfitsio':
/global/homes/m/maja1/programs/hpc4cmb/cfitsio/cfileio.c:4782: undefined reference to `https_open'
/global/homes/m/maja1/programs/hpc4cmb/cfitsio/cfileio.c:4782: undefined reference to `https_checkfile'
/global/homes/m/maja1/programs/hpc4cmb/cfitsio/cfileio.c:4809: undefined reference to `https_file_open'
/global/homes/m/maja1/programs/hpc4cmb/cfitsio/cfileio.c:4840: undefined reference to `https_file_open'
/global/homes/m/maja1/programs/hpc4cmb/cfitsio/cfileio.c:4840: undefined reference to `https_checkfile'
/global/homes/m/maja1/programs/hpc4cmb/install/cori/lib/libcfitsio.a(cfileio.o): In function `ffihtps':
/global/homes/m/maja1/programs/hpc4cmb/cfitsio/cfileio.c:7431: undefined reference to `curl_global_init'
/global/homes/m/maja1/programs/hpc4cmb/install/cori/lib/libcfitsio.a(cfileio.o): In function `ffchtps':
/global/homes/m/maja1/programs/hpc4cmb/cfitsio/cfileio.c:7444: undefined reference to `curl_global_cleanup'
/global/homes/m/maja1/programs/hpc4cmb/install/cori/lib/libcfitsio.a(cfileio.o): In function `ffvhtps':
/global/homes/m/maja1/programs/hpc4cmb/cfitsio/cfileio.c:7454: undefined reference to `https_set_verbose'
/usr/bin/ld: link errors found, deleting executable `conftest'

[keskitalo]

Thank you for the details. It seems that the Cray-provided MPI-compiler wrapper, cc is refusing to link to libcurl. This is a new cfitsio dependency, added in 3.42: [https://heasarc.gsfc.nasa.gov/FTP/software/fitsio/c/docs/changes.txt](cfitsio change log). It does not look like there is an explicit option to disable linking to it but you can trick the cfitsio build system into thinking curl is being provided as a part of anaconda:
CURLCONFIG="echo anaconda" ./configure --prefix=$PREFIX. This will cause the build system to reject curl and omit the parts of cfitsio (namely https support) that rely on it.

[majashdown]

Thanks for the work-around. I have run configure successfully now.

I have come across another error when compiling. The library is built successfully, but there is a error when trying to link the test_libconviqt executable:

/bin/sh ../libtool  --tag=CXX   --mode=link CC  -O3 -fopenmp  -fopenmp -o test_libconviqt test_libconviqt.o libconviqt.la -L/global/homes/m/maja1/programs/hpc4cmb/install/cori/lib -lcfitsio -lm 
libtool: link: CC -O3 -fopenmp -fopenmp -o .libs/test_libconviqt test_libconviqt.o  ./.libs/libconviqt.so -L/global/homes/m/maja1/programs/hpc4cmb/install/cori/lib /opt/gcc/7.1.0/snos/lib/../lib64/libgfortran.so /opt/gcc/7.1.0/snos/lib/../lib64/libquadmath.so /opt/gcc/7.1.0/snos/lib/../lib64/libstdc++.so -lcfitsio -lm -fopenmp -Wl,-rpath -Wl,/global/homes/m/maja1/programs/hpc4cmb/install/cori/lib -Wl,-rpath -Wl,/opt/gcc/7.1.0/snos/lib/../lib64
/usr/bin/ld: attempted static link of dynamic object `./.libs/libconviqt.so'
collect2: error: ld returned 1 exit status
Makefile:690: recipe for target 'test_libconviqt' failed

[tskisner]

Hello @majashdown , if you are installing from a git checkout at NERSC, then you'll need autotools in order to run the autogen.sh script. Are you just using the default versions of autotools on the system? If so, they are probably old.

Can you try installing this release tarball (just run configure, not autogen.sh), to see if that works?

https://github.com/hpc4cmb/libconviqt/releases/download/1.0.2/libconviqt-1.0.2.tar.bz2

If it does work, then likely that was the problem. The toast-deps module includes newer versions of autotools, but I realize that software stack is also set up to use the Intel compiler, which it seems you don't want.

Also a big warning- if you use MKL for Lapack/Blas and also use gcc, then do not try to mix that compiled code with python. See:

hpc4cmb/toast#110

Basically python/numpy dlopens MKL with Intel thread support. Later, if the python process dlopen's another library linked to MKL with gnu thread support, MKL (with the wrong thread support) will be seen as already loaded. This causes silent race conditions and memory corruption in threaded code.