Unzip Transformer #341
Description
Story
As an Astrophysicist, I want to extract compressed data from the Spitzer Photometry and Accurate Rotation Curves website so I can get accepted rotation curves for my analysis
Description
There are zip files available to download via http. We would like to perform analysis on the data contained within them so we need a way to inject them into the ServiceX object store.
This is the first case of a transformer that doesn't require a code-gen. We will use this opportunity to make the code-gen property of the transform request optional. If the code-gen property is not included, the transform workflow will skip the code generation step. The transformer manager will handle the case that there is no generated code configmap to mount in the transformers.
The user will provide the transformer "science container" in the transform request. It will be sslhep/unzip-transformer:latest
The transformer will download the zip file via HTTP and unzip the results into the usual output minio bucket.
See if we can use a streaming unzip to make this scalable by avoiding keeping the entire zip file and unzipped contents local to the transformer.
Test Case
We specifically want to see how this works with http://astroweb.cwru.edu/SPARC/Rotmod_LTG.zip