Update phytoplankton dataset (chl_pp)

[BBeltz1]

• Refactor get function to accommodate new datafile
• Update indicator and build package
• Re-knit comparison document

[atyrell3]

It looks like data-raw/raw_input_chl_pp.csv isn't actually in the repo -- is it possible to add that file?

[BBeltz1]

No, it exceeds Github's file size limit.

[jcaracappa1]

This looks good. Will need to update for the final bit of 2025 before IR

[jcaracappa1]

It looks like data-raw/raw_input_chl_pp.csv isn't actually in the repo -- is it possible to add that file?

If we subset just the columns that are used in ecodata its only 11MB vs 168MB
dplyr::select(PERIOD, VARIABLE, VALUE, SUBAREA, UNITS)

[BBeltz1]

We could but then the raw input would essentially be a duplicate of the dataset. We would have to either manipulate the input file manually before uploading to Github or write new code elsewhere in the package that subsets the input file. This is what the get function is doing already. The only difference would be that the "input file" would be wide and ecodata::chl_pp would be long.

[BBeltz1]

We could but then the raw input would essentially be a duplicate of the dataset. We would have to either manipulate the input file manually before uploading to Github or write new code elsewhere in the package that subsets the input file. This is what the get function is doing already. The only difference would be that the "input file" would be wide and ecodata::chl_pp would be long.

Actually, they would be identical except the manipulated input file would have the original column names rather than the standardized column names. Not wide vs long. Otherwise, they would be identical files.

[jcaracappa1]

Ok then it's probably fine to keep it out in that case

[khyde]

I am running the remainder 2025 data now and will let you know when it is complete. It will still be considered preliminary because the final data may not be available until February. I don't think the final data will change the narrative, but there will be another update.
Next year I will have this built into the workflow and we should discuss what information should be included in the final dataset to reduce the total size. The current version has additional products/time periods that we aren't currently using plus quite a bit of metadata, both of which can probably be slimmed down a bit.

[khyde]

@BBeltz1
The latest preliminary chl_pp data through 2025 is now available.
/EDAB_Archive/nadata/PROJECTS/SOE_PHYTOPLANKTON/V2026/DATA_EXTRACTS/SOE_FORMAT

[BBeltz1]

@khyde just noting that i've seen your message. i will process the update as soon as i can. thanks kim!