When serialising to Parquet the https://github.com/PxTools/PCAxis.Serializers will on some tables fail with ArgumentException("Invalid TimeScaleType or value."). The reason is that both PX file format and Paxiom only support a given set of timescales. When custom timescales are defined in CNMM they will be classified in Paxiom as TimeScaleType=NotSet or they will have a valid TimeScaleType but with invalid codes.
Examples
curl -s "https://data.qa.ssb.no/api/pxwebapi/v2/tables/07314?lang=en" | jq | grep timeUnit
"timeUnit": "Other",
curl -is "https://data.qa.ssb.no/api/pxwebapi/v2/tables/07314/data?lang=en&outputFormat=parquet" | grep HTTP
HTTP/2 500
curl -s "https://data.qa.ssb.no/api/pxwebapi/v2/tables/04555?lang=en" | jq | grep timeUnit
"timeUnit": "Annual",
curl -is "https://data.qa.ssb.no/api/pxwebapi/v2/tables/04555/data?lang=no&outputFormat=parquet&valuecodes%5BContentsCode%5D=Personer&valuecodes%5BTid%5D=2006,2006%20Gml&valuecodes%5BKjonn%5D=*&valuecodes%5BArbledUndeSyss%5D=*" | grep HTTP
HTTP/2 500
curl -s "https://data.qa.ssb.no/api/pxwebapi/v2/tables/04555/data?lang=no&outputFormat=px&valuecodes%5BContentsCode%5D=Personer&valuecodes%5BTid%5D=2006,2006%20Gml&valuecodes%5BKjonn%5D=*&valuecodes%5BArbledUndeSyss%5D=*" | grep TIMEVAL
TIMEVAL("?r")=TLIST(A1),"2006","2006 Gml";
Possible solution
The reason we look at TimeScaleType is that we wan't to make a timestamp column i parquet with at valid DataTime. One solution would be to make the timestamp column optional and only populate the column when we can produce a DataTime
When serialising to Parquet the https://github.com/PxTools/PCAxis.Serializers will on some tables fail with
ArgumentException("Invalid TimeScaleType or value."). The reason is that both PX file format and Paxiom only support a given set of timescales. When custom timescales are defined in CNMM they will be classified in Paxiom asTimeScaleType=NotSetor they will have a valid TimeScaleType but with invalid codes.Examples
Possible solution
The reason we look at TimeScaleType is that we wan't to make a timestamp column i parquet with at valid DataTime. One solution would be to make the timestamp column optional and only populate the column when we can produce a DataTime