Issues with linking on Big Sur

[mserylak]

Hi,

I am trying to build sigproc on Big Sur with gcc-10 (Homebrew GCC 10.2.0_1). After cloning the repository (git clone https://github.com/SixByNine/sigproc.git) I run bootstrap.sh and configure script:

./configure --prefix=/Users/m.serylak/Software/sigproc/install LDFLAGS="-L/usr/local/lib" CPPFLAGS="-I/usr/local/include" --x-libraries=/usr/local/lib --with-x --with-psrxml-dir=/Users/m.serylak/Software/psrxml/install --enable-shared

The compilation runs well with some warnings until I run into problem with duplicate symbols while linking. Here is the output where trouble starts:

rm -f libsigproc.a

ar cru libsigproc.a add_channels.o add_samples.o aliases.o angle_split.o aoftm.o bandfactors.o bit.o cel2gal.o chan_freqs.o close_log.o cprofc.o decimate_data.o decimate_header.o deg2dms.o dmdelay.o dmshift.o eraseDP.o error_message.o fetch_hdrval.o ffreq.o file_exists.o filterbank_header.o fold_data.o fold_header.o fshift_prof.o gmrt2fb.o head_parse.o help.o ignored_channels.o indexx.o inputclip.o inv_cerf.o lookup.o mjd.o mkheader.o norm_prof.o np2.o nrselect.o nrsort.o nrutil.o nsamples.o ooty2fb.o open_file.o open_files.o open_log.o pack_unpack.o phcalc.o print_version.o process.o prof_adds.o prof_cent.o prof_ddis.o prof_sbas.o prof_sumc.o prof_sumifs.o pspm2fb.o pspm_decode.o pspm_prof.o pspm_tstart.o pulsar2k2fb.o pulse.o put.o random.o read_aoscan.o read_block.o read_blocksubzero.o read_header.o read_polyco.o readtim.o recipes.o reverse_baseline_normalise.o scaledata.o scale_prof.o scamp2fb.o select.o send_stuff.o shift_prof.o single_ch.o sizeof_file.o slalib.o smooth.o ssm.o strings_equal.o submean.o submedian.o subzero.o swap_bytes.o thresh_1d.o typeof_inputdata.o update_log.o uttime.o vanvleck.o vmax.o vmin.o wapp2fb.o wappcorrect.o wapp_prof.o whiten.o write_epn.o write_profiles.o write_pulses.o ffft.o dedisperse_data.o dedisperse_header.o machine2prf.o mkheaderlex.o alfa_position.o bpp2fb.o mjk_random.o mjk_cmd.o newgmrt2fb.o psrfits2fb.o readpsrfits_hd.o

ranlib libsigproc.a

/bin/sh ../libtool --tag=F77 --mode=link gfortran -fno-second-underscore -g -O2 -lfftw3f_threads -L/lib -lfftw3f -lm -L/usr/local/lib -o seek seek.o dosearch.o ffft.o fftdata.o find_norm.o formspec.o freq.o glun.o sumhrm.o seekin.o baseline.o readspec.o writespec.o singlepulse.o getrmea.o getmeanrms.o zapit.o timer.o indexxf77.o slfit.o meanrms.o getrrms.o readdat.o resample.o rotate_time.o zap_birdies.o mmzap.o recon_prof.o length.o rdhead.o rdfbtab.o getddis.o getmjd.o readtim.o pulse.o read_header.o angle_split.o clock.o open_file.o strings_equal.o aliases.o open_files.o single_ch.o write_pulses.o thresh_1d.o smooth.o fftwdata.o libsigproc.a -lm -lcfitsio -lm libtool: link: gfortran -fno-second-underscore -g -O2 -o seek seek.o dosearch.o ffft.o fftdata.o find_norm.o formspec.o freq.o glun.o sumhrm.o seekin.o baseline.o readspec.o writespec.o singlepulse.o getrmea.o getmeanrms.o zapit.o timer.o indexxf77.o slfit.o meanrms.o getrrms.o readdat.o resample.o rotate_time.o zap_birdies.o mmzap.o recon_prof.o length.o rdhead.o rdfbtab.o getddis.o getmjd.o readtim.o pulse.o read_header.o angle_split.o clock.o open_file.o strings_equal.o aliases.o open_files.o single_ch.o write_pulses.o thresh_1d.o smooth.o fftwdata.o -lfftw3f_threads -L/lib -lfftw3f -L/usr/local/lib libsigproc.a -lcfitsio -lm

ld: warning: directory not found for option '-L/lib'
duplicate symbol '_tstart' in:
readtim.o
read_header.o
duplicate symbol '_az_start' in:
readtim.o
read_header.o
duplicate symbol '_za_start' in:
readtim.o
read_header.o
duplicate symbol '_project' in:
readtim.o
read_header.o
duplicate symbol '_culprits' in:
readtim.o
read_header.o
duplicate symbol '_nbits' in:
readtim.o
read_header.o
duplicate symbol '_nchans' in:
readtim.o
read_header.o
duplicate symbol '_nbeams' in:
readtim.o
read_header.o
duplicate symbol '_npuls' in:
readtim.o
read_header.o
duplicate symbol '_nifs' in:
readtim.o
read_header.o
duplicate symbol '_header_tobs' in:
readtim.o
read_header.o
duplicate symbol '_mjdobs' in:
readtim.o
read_header.o
duplicate symbol '_analog_power' in:
readtim.o
read_header.o
duplicate symbol '_wapp_scan_number' in:
readtim.o
read_header.o
duplicate symbol '_scan_number' in:
readtim.o
read_header.o
duplicate symbol '_tsamp' in:
readtim.o
read_header.o
duplicate symbol '_isign' in:
readtim.o
read_header.o
duplicate symbol '_refdm' in:
readtim.o
read_header.o
duplicate symbol '_ibeam' in:
readtim.o
read_header.o
duplicate symbol '_srcl' in:
readtim.o
read_header.o
duplicate symbol '_gal_l' in:
readtim.o
read_header.o
duplicate symbol '_src_dej' in:
readtim.o
read_header.o
duplicate symbol '_src_raj' in:
readtim.o
read_header.o
duplicate symbol '_raw_foff' in:
readtim.o
read_header.o
duplicate symbol '_foff' in:
readtim.o
read_header.o
duplicate symbol '_data_type' in:
readtim.o
read_header.o
duplicate symbol '_source_name' in:
readtim.o
read_header.o
duplicate symbol '_rawdatafile' in:
readtim.o
read_header.o
duplicate symbol '_frequency_table' in:
readtim.o
read_header.o
duplicate symbol '_telescope_id' in:
readtim.o
read_header.o
duplicate symbol '_machine_id' in:
readtim.o
read_header.o
duplicate symbol '_barycentric' in:
readtim.o
read_header.o
duplicate symbol '_pulsarcentric' in:
readtim.o
read_header.o
duplicate symbol '_srcb' in:
readtim.o
read_header.o
duplicate symbol '_gal_b' in:
readtim.o
read_header.o
duplicate symbol '_raw_fch1' in:
readtim.o
read_header.o
duplicate symbol '_fch1' in:
readtim.o
read_header.o
duplicate symbol '_lst0' in:
readtim.o
read_header.o
duplicate symbol '_ast0' in:
readtim.o
read_header.o
duplicate symbol '_iindx' in:
pulse.o
thresh_1d.o
duplicate symbol '_nsmax' in:
pulse.o
thresh_1d.o
duplicate symbol '_opterr' in:
pulse.o
thresh_1d.o
duplicate symbol '_ndm' in:
pulse.o
thresh_1d.o
duplicate symbol '_thresh' in:
pulse.o
thresh_1d.o
duplicate symbol '_optarg' in:
pulse.o
thresh_1d.o
duplicate symbol '_iterate' in:
pulse.o
thresh_1d.o
duplicate symbol '_file_name' in:
pulse.o
thresh_1d.o
duplicate symbol '_input_file' in:
pulse.o
thresh_1d.o
duplicate symbol '_optind' in:
pulse.o
thresh_1d.o
duplicate symbol '_nsubband' in:
pulse.o
thresh_1d.o
duplicate symbol '_iindx' in:
pulse.o
single_ch.o
duplicate symbol '_nsmax' in:
pulse.o
single_ch.o
duplicate symbol '_opterr' in:
pulse.o
single_ch.o
duplicate symbol '_ndm' in:
pulse.o
single_ch.o
duplicate symbol '_thresh' in:
pulse.o
single_ch.o
duplicate symbol '_optarg' in:
pulse.o
single_ch.o
duplicate symbol '_iterate' in:
pulse.o
single_ch.o
duplicate symbol '_file_name' in:
pulse.o
single_ch.o
duplicate symbol '_input_file' in:
pulse.o
single_ch.o
duplicate symbol '_optind' in:
pulse.o
single_ch.o
duplicate symbol '_nsubband' in:
pulse.o
single_ch.o
duplicate symbol '_iindx' in:
pulse.o
open_files.o
duplicate symbol '_nsmax' in:
pulse.o
open_files.o
duplicate symbol '_opterr' in:
pulse.o
open_files.o
duplicate symbol '_ndm' in:
pulse.o
open_files.o
duplicate symbol '_thresh' in:
pulse.o
open_files.o
duplicate symbol '_optarg' in:
pulse.o
open_files.o
duplicate symbol '_iterate' in:
pulse.o
open_files.o
duplicate symbol '_file_name' in:
pulse.o
open_files.o
duplicate symbol '_input_file' in:
pulse.o
open_files.o
duplicate symbol '_optind' in:
pulse.o
open_files.o
duplicate symbol '_nsubband' in:
pulse.o
open_files.o
duplicate symbol '_iindx' in:
pulse.o
write_pulses.o
duplicate symbol '_nsmax' in:
pulse.o
write_pulses.o
duplicate symbol '_opterr' in:
pulse.o
write_pulses.o
duplicate symbol '_ndm' in:
pulse.o
write_pulses.o
duplicate symbol '_thresh' in:
pulse.o
write_pulses.o
duplicate symbol '_optarg' in:
pulse.o
write_pulses.o
duplicate symbol '_iterate' in:
pulse.o
write_pulses.o
duplicate symbol '_file_name' in:
pulse.o
write_pulses.o
duplicate symbol '_input_file' in:
pulse.o
write_pulses.o
duplicate symbol '_optind' in:
pulse.o
write_pulses.o
duplicate symbol '_nsubband' in:
pulse.o
write_pulses.o
ld: 83 duplicate symbols for architecture x86_64
collect2: error: ld returned 1 exit status
make[3]: *** [seek] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Looks like there are some problems with multiple definitions of functions in the code - at least that is what I found after some rummaging for answers online. I suspect that this might be the linker used. How should go about solving this one? Thanks in advance!

Best,
Maciej

[SixByNine]

This is because sigproc uses dodgy global variables for header parameters. Any code that includes header.h cannot be linked with any other code that includes header.h. I'm not really 100% sure why older versions of gcc allowed this, but I guess it somehow worked. I think the fix is to do something like tempo2 and make a globals object file that actually has the global variables and change header.h to be "extern" i.e. assume that the actual variable is stored elsewhere. Or fix sigproc not to use globals. Or just use python code (there are some pure python codes for working with sigproc now that can be just as fast as sigproc C code - see e.g. https://filtools.readthedocs.io/en/latest/).

I think at the moment I don't have time to fix this, and really I'm not sure how many people actually use sigproc itself anymore. The data format is widely used, but that actual software I think not so much.

[tomaszmrugalski]

The same variables are defined in multiple places. This is in general a programming error. It seems gcc versions 9 and earlier turned a blind eye on this (and possibly linked with just one copy of each duplicated variable). That is no longer the case with gcc 10. I tried to fix this and made some progress, but the changes are substantial, so I gave up.

A workaround for this particular problem for me was to go with gcc-9:

export CC=gcc-9
export CXX=g++-9

[evanocathain]

So, as Mike says, sigproc does some dodgy stuff which for gcc<10 was fine. You can do, as Tomasz suggests, and use gcc-9 or earlier if you want. That will make it 'just work'. However that is only true on x86_64 architectures!

If you are on an arm64 architecture (e.g. Apple M1) you will not be able to do this. For instance if you try "brew install gcc@9" you will get "gcc@9: The x86_64 architecture is required for this software." and you are basically stuck, i.e. you should probably fix the sigproc headers/globals or do something else.