Fix some bugs (#244)

Datseris · kahaaga · web-flow · commit 76bf7cafe8b6 · 2023-01-12T16:59:17.000+02:00
* fix diversity * spatial dispersion tests fail * test for correct error exception * up version * add a docs comment that entropies are c omplexity measures * Add test case for pre-symbolised version This would have caught the error for `entropy_normalized` at the correct level as described in ##244 * Simplify tests. No need to round. * Correctly propagate user-specified `L` to `outcome_space` * mass rename `ComplexityEstimator` from `ComplexityMeasure` * document better Complkextiy Estimator * fix and test symbolicperm with views Co-authored-by: Kristian Haaga <kahaaga@gmail.com>
diff --git a/Project.toml b/Project.toml
@@ -2,7 +2,7 @@ name = "ComplexityMeasures"
 uuid = "ab4b797d-85ee-42ba-b621-05d793b346a2"
 authors = "Kristian Agasøster Haaga <kahaaga@gmail.com>, George Datseries <datseris.george@gmail.com>"
 repo = "https://github.com/juliadynamics/ComplexityMeasures.jl.git"
-version = "2.3.0"
+version = "2.3.1"
 
 [deps]
 Combinatorics = "861a8166-3701-5b0c-9a16-15d98fcdc6aa"
diff --git a/docs/src/complexity.md b/docs/src/complexity.md
@@ -1,9 +1,17 @@
 # Complexity measures
 
+In this page we document estimators for complexity measures that are not entropies in the strict mathematical sense. The API is almost identical to [`entropy`](@ref) and is defined by:
+
+- [`complexity`](@ref)
+- [`complexity_normalized`](@ref)
+- [`ComplexityEstimator`](@ref)
+
 ## Complexiy measures API
+
 ```@docs
 complexity
 complexity_normalized
+ComplexityEstimator
 ```
 
 ## Approximate entropy
diff --git a/docs/src/index.md b/docs/src/index.md
@@ -16,6 +16,8 @@ Before exploring the features of ComplexityMeasures.jl, it is useful to read thr
 In these scientific literature, words like *probabilities*, *entropies*, and other *complexity measures* are used (and abused) in multiple contexts, and are often used interchangeably to describe similar concepts. The API and documentation of ComplexityMeasures.jl aim to clarify the meaning and usage of these words, and to provide simple ways to obtain probabilities, entropies, or other complexity measures
 from input data.
 
+For ComplexityMeasures.jl _entropies are also complexity measures_, while sometimes a distinction is made so that "complexity measures" means anything beyond entropy. However we believe the general nonlinear dynamics community agrees with our take, as most papers that introduce different entropy flavors, call them complexity measures. Example: _"Permutation Entropy: A Natural Complexity Measure for Time Series"_ from Brandt and Pompe 2002.
+
 ### Probabilities
 
 Entropies and other complexity measures are typically computed based on _probability
diff --git a/src/complexity.jl b/src/complexity.jl
@@ -1,38 +1,41 @@
-export ComplexityMeasure
+export ComplexityEstimator
 export complexity
 export complexity_normalized
 
 """
-    ComplexityMeasure
+    ComplexityEstimator
 
-Supertype for (entropy-like) complexity measures.
+Supertype for estimators for various complexity measures that are not entropies
+in the strict mathematical sense.
+See [`complexity`](@ref) for all available estimators.
 """
-abstract type ComplexityMeasure end
+abstract type ComplexityEstimator end
 
 """
-    complexity(c::ComplexityMeasure, x)
+    complexity(c::ComplexityEstimator, x)
 
-Estimate the complexity measure `c` for [input data](@ref input_data) `x`, where `c` can
-be any of the following measures:
+Estimate a complexity measure according to `c`
+for [input data](@ref input_data) `x`, where `c` can
+be any of the following estimators:
 
 - [`ReverseDispersion`](@ref).
 - [`ApproximateEntropy`](@ref).
 - [`SampleEntropy`](@ref).
 - [`MissingDispersionPatterns`](@ref).
 """
-function complexity(c::C, x) where C <: ComplexityMeasure
+function complexity(c::C, x) where C <: ComplexityEstimator
     T = typeof(x)
     msg = "`complexity` not implemented for $C and input data of type $T."
     throw(ArgumentError(msg))
 end
 
 """
-    complexity_normalized(c::ComplexityMeasure, x) → m ∈ [a, b]
+    complexity_normalized(c::ComplexityEstimator, x) → m ∈ [a, b]
 
 The same as [`complexity`](@ref), but the result is normalized to the interval `[a, b]`,
 where `[a, b]` depends on `c`.
 """
-function complexity_normalized(c::C, x) where {C <: ComplexityMeasure}
+function complexity_normalized(c::C, x) where {C <: ComplexityEstimator}
     T = typeof(x)
     msg = "`complexity_normalized` not implemented for $C and input data of type $T."
     throw(ArgumentError(msg))
diff --git a/src/complexity_measures/approximate_entropy.jl b/src/complexity_measures/approximate_entropy.jl
@@ -6,7 +6,7 @@ using Statistics
 export ApproximateEntropy
 
 """
-    ApproximateEntropy <: ComplexityMeasure
+    ApproximateEntropy <: ComplexityEstimator
     ApproximateEntropy([x]; r = 0.2std(x), kwargs...)
 
 An estimator for the approximate entropy (ApEn; Pincus, 1991)[^Pincus1991] complexity
@@ -68,7 +68,7 @@ constructed from the input timeseries ``x(t)`` as
 [^Pincus1991]: Pincus, S. M. (1991). Approximate entropy as a measure of system complexity.
     Proceedings of the National Academy of Sciences, 88(6), 2297-2301.
 """
-Base.@kwdef struct ApproximateEntropy{I, B, R} <: ComplexityMeasure
+Base.@kwdef struct ApproximateEntropy{I, B, R} <: ComplexityEstimator
     m::I = 2
     τ::I = 1
     base::B = MathConstants.e
diff --git a/src/complexity_measures/missing_dispersion.jl b/src/complexity_measures/missing_dispersion.jl
@@ -5,7 +5,7 @@ using ComplexityMeasures: total_outcomes, missing_outcomes
 export MissingDispersionPatterns
 
 """
-    MissingDispersionPatterns <: ComplexityMeasure
+    MissingDispersionPatterns <: ComplexityEstimator
     MissingDispersionPatterns(est = Dispersion())
 
 An estimator for the number of missing dispersion patterns (``N_{MDP}``), a complexity
@@ -51,7 +51,7 @@ See also: [`Dispersion`](@ref), [`ReverseDispersion`](@ref), [`total_outcomes`](
 [^Zhou2022]: Zhou, Q., Shang, P., & Zhang, B. (2022). Using missing dispersion patterns
     to detect determinism and nonlinearity in time series data. Nonlinear Dynamics, 1-20.
 """
-Base.@kwdef struct MissingDispersionPatterns{D} <: ComplexityMeasure
+Base.@kwdef struct MissingDispersionPatterns{D} <: ComplexityEstimator
     est::D = Dispersion()
 end
 
diff --git a/src/complexity_measures/reverse_dispersion_entropy.jl b/src/complexity_measures/reverse_dispersion_entropy.jl
@@ -4,7 +4,7 @@ import ComplexityMeasures: total_outcomes
 export ReverseDispersion
 
 """
-    ReverseDispersion <: ComplexityMeasure
+    ReverseDispersion <: ComplexityEstimator
     ReverseDispersion(; c = 3, m = 2, τ = 1, check_unique = true)
 
 Estimator for the reverse dispersion entropy complexity measure (Li et al., 2019)[^Li2019].
@@ -45,7 +45,7 @@ unique element, then a `InexactError` is thrown when trying to compute probabili
 [^Li2019]: Li, Y., Gao, X., & Wang, L. (2019). Reverse dispersion entropy: a new
     complexity measure for sensor signal. Sensors, 19(23), 5203.
 """
-Base.@kwdef struct ReverseDispersion{S <: Encoding} <: ComplexityMeasure
+Base.@kwdef struct ReverseDispersion{S <: Encoding} <: ComplexityEstimator
     encoding::Type{S} = GaussianCDFEncoding # any encoding at accepts keyword `c`
     c::Int = 3 # The number of categories to map encoded values to.
     m::Int = 2
diff --git a/src/complexity_measures/sample_entropy.jl b/src/complexity_measures/sample_entropy.jl
@@ -7,7 +7,7 @@ export SampleEntropy
 export entropy_sample
 
 """
-    SampleEntropy([x]; r = 0.2std(x), kwargs...)
+    SampleEntropy([x]; r = 0.2std(x), kwargs...) <: ComplexityEstimator
 
 An estimator for the sample entropy complexity measure (Richman & Moorman,
 2000)[^Richman2000], used with [`complexity`](@ref) and [`complexity_normalized`](@ref).
@@ -20,7 +20,7 @@ The keyword argument `r` is mandatory if an input timeseries `x` is not provided
     should be determined from the input data, for example as some proportion of the
     standard deviation of the data.
 - `m::Int = 1`: The embedding dimension.
-- `τ::Int = `: The embedding lag.
+- `τ::Int = 1`: The embedding lag.
 
 ## Description
 
@@ -67,7 +67,7 @@ See also: [`entropy_sample`](@ref).
     analysis using approximate entropy and sample entropy. American Journal of
     Physiology-Heart and Circulatory Physiology, 278(6), H2039-H2049.
 """
-Base.@kwdef struct SampleEntropy{R} <: ComplexityMeasure
+Base.@kwdef struct SampleEntropy{R} <: ComplexityEstimator
     m::Int = 2
     τ::Int = 1
     r::R
diff --git a/src/deprecations.jl b/src/deprecations.jl
@@ -7,6 +7,8 @@ function entropy(e::EntropyDefinition, est::DiffEntropyEst, x)
     end
 end
 
+@deprecate ComplexityMeasure ComplexityEstimator
+
 # From before 2.0:
 @deprecate TimeScaleMODWT WaveletOverlap
 function probabilities(x::Vector_or_Dataset, ε::Union{Real, Vector{<:Real}})
diff --git a/src/multiscale.jl b/src/multiscale.jl
@@ -36,7 +36,7 @@ function multiscale_normalized end
 """
     multiscale(alg::MultiScaleAlgorithm, e::EntropyDefinition, est::DiffEntropyEst, x; kwargs...)
     multiscale(alg::MultiScaleAlgorithm, e::EntropyDefinition, est::ProbabilitiesEstimator, x; kwargs...)
-    multiscale(alg::MultiScaleAlgorithm, c::ComplexityMeasure, x::AbstractVector; kwargs...)
+    multiscale(alg::MultiScaleAlgorithm, c::ComplexityEstimator, x::AbstractVector; kwargs...)
 
 Compute the multi-scale entropy `e` with estimator `est`, or the complexity measure `c`,
 for timeseries `x`.
@@ -60,7 +60,7 @@ factor `1`, the original time series is considered.
 ## Arguments
 
 - `e::EntropyDefinition`. A valid [entropy type](@ref entropies), i.e. `Shannon()` or `Renyi()`.
-- `c::ComplexityMeasure`. A valid complexity measure, i.e. [`SampleEntropy`](@ref),
+- `c::ComplexityEstimator`. A valid complexity measure, i.e. [`SampleEntropy`](@ref),
     or [`ApproximateEntropy`](@ref).
 - `alg::MultiScaleAlgorithm`. A valid [multiscale algorithm](@ref multiscale_algorithms),
     i.e. `Regular()` or `Composite()`, which determines how down-sampling/coarse-graining
diff --git a/src/multiscale/multiscale.jl b/src/multiscale/multiscale.jl
@@ -1,17 +1,17 @@
-function multiscale(alg::MultiScaleAlgorithm, e::ComplexityMeasure, x::AbstractVector;
+function multiscale(alg::MultiScaleAlgorithm, e::ComplexityEstimator, x::AbstractVector;
         maxscale::Int = 8)
 
     downscaled_timeseries = [downsample(alg, s, x) for s in 1:maxscale]
     return complexity.(Ref(e), downscaled_timeseries)
 end
 
-function multiscale_normalized(alg::Regular, e::ComplexityMeasure, x::AbstractVector;
+function multiscale_normalized(alg::Regular, e::ComplexityEstimator, x::AbstractVector;
         maxscale::Int = 8)
     downscaled_timeseries = [downsample(alg, s, x) for s in 1:maxscale]
     return complexity_normalized.(Ref(e), downscaled_timeseries)
 end
 
-function multiscale(alg::Composite, e::ComplexityMeasure, x::AbstractVector;
+function multiscale(alg::Composite, e::ComplexityEstimator, x::AbstractVector;
         maxscale::Int = 8)
 
     downscaled_timeseries = [downsample(alg, s, x) for s in 1:maxscale]
@@ -22,7 +22,7 @@ function multiscale(alg::Composite, e::ComplexityMeasure, x::AbstractVector;
     return complexities
 end
 
-function multiscale_normalized(alg::Composite, e::ComplexityMeasure, x::AbstractVector;
+function multiscale_normalized(alg::Composite, e::ComplexityEstimator, x::AbstractVector;
         maxscale::Int = 8)
     downscaled_timeseries = [downsample(alg, s, x) for s in 1:maxscale]
     complexities = zeros(Float64, maxscale)
diff --git a/src/probabilities_estimators/diversity.jl b/src/probabilities_estimators/diversity.jl
@@ -51,7 +51,7 @@ function probabilities_and_outcomes(est::Diversity, x::AbstractVector{T}) where
     return probabilities_and_outcomes(ValueHistogram(rbc.binning), ds)
 end
 
-outcome_space(est::Diversity) = outcome_space(binning_for_diversity(est))
+outcome_space(est::Diversity) = outcome_space(encoding_for_diversity(est.nbins))
 total_outcomes(est::Diversity) = est.nbins
 
 function similarities_and_binning(est::Diversity, x::AbstractVector{T}) where T <: Real
@@ -62,12 +62,13 @@ function similarities_and_binning(est::Diversity, x::AbstractVector{T}) where T
     @inbounds for i in 1:(length(Y)-1)
         ds[i] = cosine_similarity(Y[i], Y[i+1])
     end
-    # Cosine similarities are all on [-1.0, 1.0], so just discretize this interval.
-    binning = binning_for_diversity(est)
-    rbc = RectangularBinEncoding(binning)
+    # Cosine similarities are all on [-1.0, 1.0], so just discretize this interval
+    rbc = encoding_for_diversity(est.nbins)
     return ds, rbc
 end
 
 cosine_similarity(xᵢ, xⱼ) = sum(xᵢ .* xⱼ) / (sqrt(sum(xᵢ .^ 2)) * sqrt(sum(xⱼ .^ 2)))
 
-binning_for_diversity(est::Diversity) = FixedRectangularBinning(-1.0, 1.0, est.nbins)
+function encoding_for_diversity(nbins::Int)
+    RectangularBinEncoding(FixedRectangularBinning(-1.0, 1.0, nbins))
+end
diff --git a/src/probabilities_estimators/spatial/spatial_dispersion/SpatialDispersion.jl b/src/probabilities_estimators/spatial/spatial_dispersion/SpatialDispersion.jl
@@ -209,4 +209,11 @@ function total_outcomes(est::SpatialDispersion)::Int
     end
 end
 
-outcome_space(est::SpatialDispersion) = outcome_space(Dispersion(; c = est.c, m = est.m))
+
+function outcome_space(est::SpatialDispersion)
+    if est.skip_encoding
+        return outcome_space(Dispersion(; c = est.L, m = est.m))
+    else
+        return outcome_space(Dispersion(; c = est.c, m = est.m))
+    end
+end
diff --git a/src/probabilities_estimators/symbolic_permutation.jl b/src/probabilities_estimators/symbolic_permutation.jl
@@ -26,7 +26,7 @@ A probabilities estimator based on ordinal permutation patterns.
 
 When passed to [`probabilities`](@ref) the output depends on the input data type:
 
-- **Univariate data**. If applied to a univariate timeseries (`Vector`), then the timeseries
+- **Univariate data**. If applied to a univariate timeseries (`AbstractVector`), then the timeseries
     is first embedded using embedding delay `τ` and dimension `m`, resulting in embedding
     vectors ``\\{ \\bf{x}_i \\}_{i=1}^{N-(m-1)\\tau}``. Then, for each ``\\bf{x}_i``,
     we find its permutation pattern ``\\pi_{i}``. Probabilities are then
@@ -197,7 +197,7 @@ end
 # types is whether they compute some additional weights that are affecting
 # how the probabilities are counted.
 
-function probabilities(est::PermProbEst{m}, x::Vector{T}) where {m, T<:Real}
+function probabilities(est::PermProbEst{m}, x::AbstractVector{T}) where {m, T<:Real}
     dataset::Dataset{m,T} = embed(x, m, est.τ)
     return probabilities(est, dataset)
 end
@@ -234,7 +234,7 @@ function probabilities_and_outcomes(est::PermProbEst{m}, x::Vector_or_Dataset) w
     # function that both maps to integers with `decode` but also keeps track of
     # the permutation pattern vectors. Anyways, I don't think `outcomes` is a function
     # that will be called often, so we can live with this as is.
-    if x isa Vector
+    if x isa AbstractVector
         dataset = embed(x, m, est.τ)
     else
         dataset = x
diff --git a/test/entropies/interface.jl b/test/entropies/interface.jl
@@ -5,7 +5,7 @@ using Random: Xoshiro
     x = rand(1000)
     @test_throws MethodError entropy(x, 0.1)
     est = AlizadehArghami() # the AlizadehArghami estimator only works for Shannon entropy
-    @test_throws MethodError entropy(Tsallis(), AlizadehArghami(), x)
+    @test_throws ErrorException entropy(Tsallis(), AlizadehArghami(), x)
 end
 
 @testset "normalization" begin
diff --git a/test/probabilities/estimators/permutation.jl b/test/probabilities/estimators/permutation.jl
@@ -23,8 +23,8 @@ end
     @testset "direct" begin
         p1 = probabilities(est, x)
         p2 = probabilities(est, y)
-        @test sum(p1) ≈ 1.0
-        @test sum(p2) ≈ 1.0
+        @test sum(vec(p1)) ≈ 1.0
+        @test sum(vec(p2)) ≈ 1.0
         @test length(p1) == 1 # analytic
         @test length(p2) == 6 # analytic
     end
@@ -35,9 +35,17 @@ end
         @test sum(p2) ≈ 1.0
     end
 
+    @testset "vector" begin
+        z = y[:, 1]
+        w = view(z, 1:length(z)-1)
+        p1 = probabilities(est, z)
+        p2 = probabilities(est, w)
+        @test sum(vec(p1)) ≈ sum(vec(p2)) ≈ 1
+    end
+
 end
 
-# TODO: would be nioce to have analytic tests for amplitude aware
+# TODO: would be nice to have analytic tests for amplitude aware
 @testset "Uniform distr." begin
     m = 4
     τ = 1
diff --git a/test/probabilities/estimators/spatial/spatial_dispersion.jl b/test/probabilities/estimators/spatial/spatial_dispersion.jl
@@ -1,5 +1,4 @@
 using ComplexityMeasures
-using StaticArrays
 
 # Create an analytical test from scratch.
 x = [
@@ -47,11 +46,13 @@ est = SpatialDispersion(stencil, x, c = 3, periodic = true)
 # The following data `y` has 2 classes. c = 2 ensures that this remains the case after
 # symbolization. We can also skip symbolization all together, but then we must specify
 # `L` as the total possible number of symbols the input data can take.
-y = rand(0:1, 100, 100);
+using Random: Xoshiro
+y = rand(Xoshiro(1234), 0:1, 100, 100);
 est_y = SpatialDispersion(stencil, y, c = 2)
 est_y_presymb = SpatialDispersion(stencil, y; skip_encoding =  true, L = 2)
 
-@test 0.99 <= round(entropy_normalized(est_y, y), digits = 2) <= 1.0
-@test 0.99 <= round(entropy_normalized(est_y_presymb, y), digits = 2) <= 1.0
+@test 0.99 <= entropy_normalized(est_y, y) <= 1.0
+@test 0.99 <= entropy_normalized(est_y_presymb, y) <= 1.0
 
 @test outcome_space(est_y) == outcome_space(Dispersion(c = est_y.c, m = est_y.m))
+@test outcome_space(est_y_presymb) == outcome_space(Dispersion(c = 2, m = est_y.m))
