File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -506,6 +506,25 @@ a convenience and are not required to read these formats. In many cases using
506506the :func: `xarray.open_dataset ` function in a custom file handler is a much
507507better idea.
508508
509+ .. note ::
510+ Be careful about the data types of the datasets your reader is returning.
511+ It is easy to let the data be coerced into double precision floats (`np.float64 `). At the
512+ moment, satellite instruments are rarely measuring in a resolution greater
513+ than what can be encoded in 16 bits. As such, to preserve processing power,
514+ please consider carefully what data type you should scale or calibrate your
515+ data to.
516+
517+ Single precision floats (`np.float32 `) is a good compromise, as it has 23
518+ significant bits (mantissa) and can thus represent 16 bit integers exactly,
519+ as well as keeping the memory footprint half of a double precision float.
520+
521+ One commonly used method in readers is :meth: `xarray.DataArray.where ` (to
522+ mask invalid data) which can be coercing the data to `np.float64 `. To ensure
523+ for example that integer data is coerced to `np.float32 ` when
524+ :meth: `xarray.DataArray.where ` is used, you can do::
525+
526+ my_float_dataarray = my_int_dataarray.where(some_condition, np.float32(np.nan))
527+
509528One way of implementing a file handler is shown below:
510529
511530.. code :: python
You can’t perform that action at this time.
0 commit comments