Skip to content

Commit 6f74846

Browse files
sethaxendevmotionmschauer
authored
Add Censored distribution (#1470)
* Implement Censored * Export Censored and censored * Apply suggestions from code review Co-authored-by: David Widmann <[email protected]> * Reorganize code * Move check to constructor * Make newline optional * Reformat * Improve show method * Add checks for finiteness * Implement expectations * Satisfy required params behavior * Overload truncated for DiscreteUniform * Don't re-censor * Handle case where censored to single point * Reformat * Add censored docs page * Update censored.md * Document censored * Add truncated DiscreteUniform tests * Add un-called truncated_uniform test suite * Skip check for DiscreteUniform * Add loglikelihood Useful to avoid redundant computation of bound probabilities when performing censored regression * Include bounds in support * Update src/truncated/discrete_uniform.jl Co-authored-by: David Widmann <[email protected]> * Remove unnecessary checks for finiteness * Swap order of checks * Update censored.jl * Support using missing to specify unbounded * Apply suggestions from code review Co-authored-by: David Widmann <[email protected]> * Improve docstrings * Reformat * Rewrap signature * Use ismissing everywhere * Reformat * Fix maximum * Fix support check * Support fields that are equivalent This allows equality checking for distributions with fields for which `==` or `isapprox` is not defined but `===` is * Add basic censored tests * Apply suggestions from code review Co-authored-by: Moritz Schauer <[email protected]> * Resolve ambiguity * Resolve ambiguity * Use equivalence check for missing everywhere * Fix ccdf and logccdf * Better construct mixture equivalent * Implement mixture model for continuous censored dists * Add uniform tests * Enhance discrete uniform tests * Add Poisson tests * Fix implementation of entropy * Test mean/std of censored Poisson * Add normal tests * Remove unused methods * Test bounds check * Add log-likelihood test * Add test for censorship to empty interval * Support censorship to empty intervals * Add comments about expectations * Add more checks for inferrability * Test explicit expectations for DiscreteUniform * Rewrite censored docstring * Indicate interval censoring * Update docs/src/index.md Co-authored-by: Moritz Schauer <[email protected]> * Apply suggestions from code review Co-authored-by: David Widmann <[email protected]> * Don't use now-deleted function * Fix tes for new params definition * Make loglikelihood type-stable * Add comment about equivalence check * Unabbreviate name * Remove inlines * Add plaintext mimetype * Add more truncated DiscreteUniform tests * Add xexpx * Compute probabilities on log scale * Clarify variable names * Eliminate deprecation warnings * Fix inferrability error * Reformat * Add check_args keyword * Promote each xi * Fix checking of args * Use bounds also to set types * Use check_args macro * Check for ArgumentError * Use eps to detect empty truncation * Use more accurate variable name * Use oneunit * Improve type-stability everywhere * Exhaustively test for type-inferribility * Remove now-unneeded test * Revert "Improve type-stability everywhere" This reverts commit f024f45. * Add convenient types * Use convenient types * Add missing check for in support of uncensored * Avoid promoting inputs based on bounds * Avoid promoting inputs based on bounds * Separate expectations to own methods * Check in interval first * Don't promote input types * Apply suggestions from code review Co-authored-by: David Widmann <[email protected]> * Remove lingering reference to udnefined type * Docstring fixes * Consistently use d0 for uncensored * Use nothing instead of missing * Reorganize * Add keyword constructor * Document and promote keyword constructors * Use keyword arguments in show * Don't use isnothing in tests * Don't print nothing Fails on old Julia versions * Export Censored * Once sentence per line * Docstring improvements * Fix variable rendering * Add missing comma * Revert "Export Censored" This reverts commit ccef571. * Prepend with module so docs build * Document with module prepended * Use function ... end syntax * Apply suggestions from code review Co-authored-by: David Widmann <[email protected]> * LeftCensored ⇌ RightCensored * Apply suggestions from code review Co-authored-by: David Widmann <[email protected]> * Introduce xexpy * Remove test for no-longer-supported case * Simplify implementations * Rearrange left before right * Increment version number * Use information theory notation for entropy * Use d0 notation * Add header * Test infinite bounds just in case * Apply suggestions from code review Co-authored-by: Moritz Schauer <[email protected]> * Add and test partially truncated overloads for discrete uniform * Update src/censored.jl Co-authored-by: David Widmann <[email protected]> Co-authored-by: David Widmann <[email protected]> Co-authored-by: Moritz Schauer <[email protected]>
1 parent cbbc1a5 commit 6f74846

File tree

14 files changed

+951
-7
lines changed

14 files changed

+951
-7
lines changed

‎Project.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name = "Distributions"
22
uuid = "31c24e10-a181-5473-b8eb-7969acd0382f"
33
authors = ["JuliaStats"]
4-
version = "0.25.44"
4+
version = "0.25.45"
55

66
[deps]
77
ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"

‎docs/make.jl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ makedocs(
1111
"types.md",
1212
"univariate.md",
1313
"truncate.md",
14+
"censored.md",
1415
"multivariate.md",
1516
"matrix.md",
1617
"reshape.md",

‎docs/src/censored.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Censored Distributions
2+
3+
In *censoring* of data, values exceeding an upper limit (right censoring) or falling below a lower limit (left censoring), or both (interval censoring) are replaced by the corresponding limit itself.
4+
The package provides the `censored` function, which creates the most appropriate distribution to represent a censored version of a given distribution.
5+
6+
A censored distribution can be constructed using the following signature:
7+
8+
```@docs
9+
censored
10+
```
11+
12+
In the general case, this will create a `Distributions.Censored{typeof(d0)}` structure, defined as follows:
13+
14+
```@docs
15+
Distributions.Censored
16+
```
17+
18+
In general, `censored` should be called instead of the constructor of `Censored`, which is not exported.
19+
20+
Many functions, including those for the evaluation of pdf and sampling, are defined for all censored univariate distributions:
21+
22+
- [`maximum(::UnivariateDistribution)`](@ref)
23+
- [`minimum(::UnivariateDistribution)`](@ref)
24+
- [`insupport(::UnivariateDistribution, x::Any)`](@ref)
25+
- [`pdf(::UnivariateDistribution, ::Real)`](@ref)
26+
- [`logpdf(::UnivariateDistribution, ::Real)`](@ref)
27+
- [`cdf(::UnivariateDistribution, ::Real)`](@ref)
28+
- [`logcdf(::UnivariateDistribution, ::Real)`](@ref)
29+
- [`logdiffcdf(::UnivariateDistribution, ::T, ::T) where {T <: Real}`](@ref)
30+
- [`ccdf(::UnivariateDistribution, ::Real)`](@ref)
31+
- [`logccdf(::UnivariateDistribution, ::Real)`](@ref)
32+
- [`quantile(::UnivariateDistribution, ::Real)`](@ref)
33+
- [`cquantile(::UnivariateDistribution, ::Real)`](@ref)
34+
- [`invlogcdf(::UnivariateDistribution, ::Real)`](@ref)
35+
- [`invlogccdf(::UnivariateDistribution, ::Real)`](@ref)
36+
- [`median(::UnivariateDistribution)`](@ref)
37+
- [`rand(::UnivariateDistribution)`](@ref)
38+
- [`rand!(::UnivariateDistribution, ::AbstractArray)`](@ref)
39+
40+
Some functions to compute statistics are available for the censored distribution if they are also available for its truncation:
41+
- [`mean(::UnivariateDistribution)`](@ref)
42+
- [`var(::UnivariateDistribution)`](@ref)
43+
- [`std(::UnivariateDistribution)`](@ref)
44+
- [`entropy(::UnivariateDistribution)`](@ref)
45+
46+
For example, these functions are available for the following uncensored distributions:
47+
- `DiscreteUniform`
48+
- `Exponential`
49+
- `LogUniform`
50+
- `Normal`
51+
- `Uniform`
52+
53+
[`mode`](@ref) is not implemented for censored distributions.

‎docs/src/index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,4 @@ The [*Distributions*](https://github.com/JuliaStats/Distributions.jl) package pr
77
* Probability density/mass functions (pdf) and their logarithm (logpdf)
88
* Moment-generating functions and characteristic functions
99
* Maximum likelihood estimation
10-
* Distribution composition (Cartesian product of distributions, truncated distributions)
10+
* Distribution composition and derived distributions (Cartesian product of distributions, truncated distributions, censored distributions)

‎src/Distributions.jl‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,7 @@ export
182182
canonform, # get canonical form of a distribution
183183
ccdf, # complementary cdf, i.e. 1 - cdf
184184
cdf, # cumulative distribution function
185+
censored, # censor a distribution with a lower and upper bound
185186
cf, # characteristic function
186187
cquantile, # complementary quantile (i.e. using prob in right hand tail)
187188
component, # get the k-th component of a mixture model
@@ -295,6 +296,7 @@ include("samplers.jl")
295296
# others
296297
include("reshaped.jl")
297298
include("truncate.jl")
299+
include("censored.jl")
298300
include("conversion.jl")
299301
include("convolution.jl")
300302
include("qq.jl")
@@ -334,7 +336,7 @@ information.
334336
Supported distributions:
335337
336338
Arcsine, Bernoulli, Beta, BetaBinomial, BetaPrime, Binomial, Biweight,
337-
Categorical, Cauchy, Chi, Chisq, Cosine, DiagNormal, DiagNormalCanon,
339+
Categorical, Cauchy, Censored, Chi, Chisq, Cosine, DiagNormal, DiagNormalCanon,
338340
Dirichlet, DiscreteUniform, DoubleExponential, EdgeworthMean,
339341
EdgeworthSum, EdgeworthZ, Erlang,
340342
Epanechnikov, Exponential, FDist, FisherNoncentralHypergeometric,

0 commit comments

Comments
 (0)