Refactor Classification Trends Visualization with Alternative Encodings & Smoothing

[veratek]

Overview

The current multi-line plots of classification trends over time can become cluttered and hard to interpret, especially as the number of classifications increases.

This issue proposes exploring alternative visual encodings, smoothing techniques, and normalization strategies to:

• Improve clarity
• Enhance interpretability
• Highlight relative dominance, shifts, and anomalies

The goal is to make these visualizations more useful for exploring historical patterns in museum acquisitions and curatorial trends.

---

Objectives

• Replace or supplement lineplots with more expressive visual encodings
• Apply smoothing to reduce visual noise and emphasize trends
• Normalize values to support fair cross-classification comparisons

---

Suggested Enhancements

1. Alternative Visual Encodings

a. Stacked Area Charts

Purpose: Show how classifications contribute proportionally to the whole over time.

• Normalize counts per year (row-wise proportions)
• Use top N classifications; group the rest as "Other"
• Use matplotlib.stackplot() or plotly.express.area()

b. Heatmaps

Purpose: Reveal intensity and burst patterns across time.

• Create pivot table (years × classifications)
• Apply optional row-wise normalization or log-scaling
• Use seaborn.heatmap() or plotly.express.imshow()

c. Rank Plots

Purpose: Highlight relative positioning and competition among classifications.

• Rank classifications by count per year
• Plot rank over time using seaborn.lineplot() or similar
• Invert y-axis (rank 1 at top)

---

2. Smoothing Techniques

a. LOESS Smoothing

Purpose: Smooth local trends with flexibility.

• Use statsmodels.nonparametric.lowess()
• Apply per classification
• Optionally include confidence intervals

b. EWMA (Exponentially Weighted Moving Average)

Purpose: Emphasize recent trends while smoothing noise.

• Use pandas.Series.ewm(span=n).mean()
• Compare output to rolling mean (e.g., rolling(window=3).mean())

---

3. Normalization Strategies

a. Proportional Normalization

Purpose: Express each classification as a share of the total per year.

• Normalize row-wise to sum to 1.0 (percentages)
• Use in stacked area or heatmaps

b. Z-score Normalization

Purpose: Highlight spikes or dips relative to each classification’s historical trend.

• Standardize columns: (x - mean) / std
• Clip extreme values if needed
• Use for anomaly-focused heatmaps

---

Deliverables

• At least one working visualization from each category:
  • Alternative encoding (e.g., stacked area)
  • Smoothing method (e.g., LOESS or EWMA)
  • Normalization approach (e.g., proportional or z-score)
• Side-by-side comparison with baseline lineplots
• Summary of what each method reveals
• Discussion of which approach supports curatorial insight best

---

Dependencies / Prerequisites

• Preprocessed classification-by-year matrix
• Cleaned year columns (e.g., clean_years())
• Familiarity with seaborn, matplotlib, or Plotly

---

Priority

High – this will significantly improve the visual quality and interpretability of classification trend analysis.