Skip to content

Trouble Exporting to netcdf #98

@JonathonMSmith

Description

@JonathonMSmith

Describe the bug
After creating an sgp4 instrument with missions I have tried to save to netcdf.
When I use the to_netcdf function I get the error

AttributeError: NetCDF: Not a valid data type or _FillValue type mismatch

I tried saving a cnofs ivm instrument with to_netcdf and got a different error, but I was still unable to save it.

I'm wondering if this is because I'm not setting the kwargs correctly.

To Reproduce

Python 3.9.13 (main, Aug 25 2022, 18:29:29) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pysat

In [2]: sat1 = pysat.Instrument('missions', 'sgp4')

In [3]: sat1.load(2018,1)
<ipython-input-3-46da04f1f5b8>:1: DeprecationWarning: Meta now contains a class for global metadata (MetaHeader). Default attachment of global attributes to Instrument will be Deprecated in pysat 3.2.0+. Set `use_header=True` in this load call or on Instrument instantiation to remove this warning.
  sat1.load(2018,1)

In [4]: sat1.to_netcdf4('/Users/jmsmit37/sat1_30_degrees.nc')
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In [4], line 1
----> 1 sat1.to_netcdf4('/Users/jmsmit37/sat1_30_degrees.nc')

File ~/anaconda3/envs/missions/lib/python3.9/site-packages/pysat/_instrument.py:3618, in Instrument.to_netcdf4(self, fname, base_instrument, epoch_name, zlib, complevel, shuffle, preserve_meta_case, export_nan, unlimited_time, modify)
   3615 inst = self if modify else self.copy()
   3617 # Write the output file
-> 3618 pysat.utils.io.inst_to_netcdf(inst, fname=fname,
   3619                               base_instrument=base_instrument,
   3620                               epoch_name=epoch_name, zlib=zlib,
   3621                               complevel=complevel, shuffle=shuffle,
   3622                               preserve_meta_case=preserve_meta_case,
   3623                               export_nan=export_nan,
   3624                               unlimited_time=unlimited_time)
   3626 return

File ~/anaconda3/envs/missions/lib/python3.9/site-packages/pysat/utils/io.py:1816, in inst_to_netcdf(inst, fname, base_instrument, epoch_name, mode, zlib, complevel, shuffle, preserve_meta_case, check_type, export_nan, unlimited_time, meta_translation, meta_processor)
   1809 cdfkey = out_data.createVariable(epoch_name, 'i8',
   1810                                  dimensions=(epoch_name),
   1811                                  zlib=zlib,
   1812                                  complevel=complevel,
   1813                                  shuffle=shuffle)
   1815 # Attach epoch metadata
-> 1816 cdfkey.setncatts(export_meta[epoch_name])
   1818 # Attach the time index to the data
   1819 cdfkey[:] = (inst.index.values.astype(np.int64)
   1820              * 1.0E-6).astype(np.int64)

File src/netCDF4/_netCDF4.pyx:4408, in netCDF4._netCDF4.Variable.setncatts()

File src/netCDF4/_netCDF4.pyx:1694, in netCDF4._netCDF4._set_att()

File src/netCDF4/_netCDF4.pyx:1965, in netCDF4._netCDF4._ensure_nc_success()

AttributeError: NetCDF: Not a valid data type or _FillValue type mismatch

Expected behavior
netcdf file is produced from the instrument

Desktop (please complete the following information):

  • OS: Mac OS Monterey
  • Version 12.5
  • python version 3.9.13
  • pysat version 3.0.4
  • pysatMissions version 0.3.3
  • netCDF4 version 1.6.0

Additional context
All the dependencies for pysatMissions were installed from the git repo using
pip install -r requirements.txt

apexpy and aacgmv2 were installed with pip

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions