QDataSet

Introduction to QDataSet

QDataSet is the name of the data representation within Autoplot. Every system ends up having to represent its data in some format. For example, suppose you are building an analysis that needs to handle length measurements as a function of time. You could accept two arrays, one for the times and one for the lengths. What units is the length in? Your documentation would state that the lengths must be expressed in meters. How is time represented? The documentation could then state that the times need to be integer milliseconds since the midnight of the day they were collected? You can see that with simple arrays, documentation (which is only understood by a human) is needed. Most systems end up developing a standard data representation model to handle data. The complexity of this model is often unnecessary, incrementatally developed as new types of data were introduced into the system. QDataSet was developed as a response to many failed data models, learning from them in hopes to provide a general-purpose system for modeling data. All data in Autoplot is handled with this model, demonstrating its flexibility and extensibility.

Nominal, Interval, and Ratiometric Data

There should be a section talking about these three data types handled. Stevens identifies four types of data, of which we use three. Nominal data can be thought of as an array of strings, like ['Chicago', 'Paris', 'Beijing'] or [ 'Time', 'Density', 'B-x', 'B-y', 'B-z' ]. Comparisons for equality can be made, but no other operators are allowed. Interval data is defined with a basis. The Celcuis temperature scale is an example, where the number of degrees above freezing is recorded. Comparisons can be made and differences are meaningful. Time locations like CDF_TT2000 and us2000 (the number of microseconds since 2000-01-01T00:00Z) are another example. Last, we have ratiometric data which have a physical zero, such as the heat energy or density. These can also be divided.

• A "simple bundle" is a rank 2 dataset having nominal data for DEPEND_1. Note there is "bundle data set" which allows for bundling complex data, with metadata attached to each bundled data set.

Schemes

QDataSet uses "Duck Typing", which is to say if it looks like a duck, it's a duck. There's no strong typing which asserts that a QDataSet can only be interpreted in some way. The page QDataSet.Schemes attempts to enumerate different types and the properties they have.