Visualize case + ensemble members

[mgrover1]

It would be neat to have something similar to a dask task graph, similar to this

for the catalog building (helping to visualize experiments and their branches) - example would be the CESM-LE, where we could have the experiment, number of ensemble members, components within each, and streams. This be helpful when visualizing what is all in the catalog.

[andersy005]

I concur! This would be neat. My question is "how do we accomplish this" :) ?

[andersy005]

My question is "how do we accomplish this" :) ?

The visualization part is simple (via py-graphviz) once we have the graph. So, how do we construct the graph and what level of granularity should we aim for?

[mgrover1]

I think to start, it would be helpful to have a "default" of case --> member_id --> stream --> list of variables maybe?

What is required for py-graphviz to construct the "graph"? A dictionary?

[andersy005]

What is required for py-graphviz to construct the "graph"? A dictionary?

A dict of dicts is the simplest interface but you could use the built-in Graph object, too. Here are some examples: https://pygraphviz.github.io/documentation/latest/auto_examples/index.html

[mgrover1]

Perhaps something like this

[mgrover1]

I am considering adding the following as part of a blog post on ESDS - would this block make sense? As well as the visual?

# Create Digraph object
dot = Digraph(graph_attr={'rankdir':'LR'})

num_node = 1

# Loop through the different experiments
for experiment in df.experiment.unique():
    exp_i = num_node
    dot.node(str(exp_i), label=experiment)
    num_node+=1
    for component in df.loc[df.experiment == experiment].component.unique():
        comp_i = num_node
        dot.node(str(comp_i), label=component)
        dot.edge(str(exp_i), str(comp_i))
        num_node+=1
        for frequency in df.loc[(df.experiment == experiment) & 
                                (df.component == component)].frequency.unique():
            freq_i = num_node
            dot.node(str(freq_i), label=frequency)
            dot.edge(str(comp_i), str(freq_i))
            num_node+=1
        comp_i+=1
    exp_i+=1

Here's an example of part of the output (output is a svg file)