Pandas Python Code Generator #562
Description
Story
As a analyzer I want to be able to work with data in CSV files so I can standardize my analysis
Description
We want to be able to read data from weird CSV and TSV formats from an experiment. We will use the Pandas library as the basis for this transformer.
We will use the python code generator with the following assumptions:
- The function will be called with an open file handle
- The function will return a Pandas dataframe
There will be a new python code generator based on the uproot python code generator. Its transform_single_file.py script will write the data frame to parquet using the dataframe to_parquet method.
It looks like it is possible to stream directly from the Dataframe to a parquet object in minio https://stackoverflow.com/a/57838851