org.das2.qds.util

Class AutoHistogram

java.lang.Object

org.das2.qds.util.AutoHistogram

public final class AutoHistogram
extends java.lang.Object

Self-configuring histogram dynamically adjusts range and bin size as data is added. Also it tries to identify outlier points, which are available as a Map<Double,Integer> going from value to number observed. Also for each bin, we keep track of a running mean and variance, which are useful for identifying continuous bins and total moments. Introduced to support automatic cadence algorithm, should be generally useful in data discovery.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	USER_PROP_BIN_START
static java.lang.String	USER_PROP_BIN_WIDTH
static java.lang.String	USER_PROP_INVALID_COUNT
static java.lang.String	USER_PROP_MIN_GT_ZERO
static java.lang.String	USER_PROP_OUTLIERS
static java.lang.String	USER_PROP_TOTAL Long, total number of valid points.

Constructor Summary

Constructors

Constructor and Description
AutoHistogram()

Method Summary

Modifier and Type	Method and Description
void	addPropertyChangeListener(java.beans.PropertyChangeListener listener)
static int	binOf(QDataSet hist, double d) convenient method for getting the bin location of a value from a completed histogram's metadata.
QDataSet	doit(QDataSet ds)
QDataSet	doit(QDataSet ds, QDataSet wds) do a histogram, dynamically shifting the bins and changing the bin size.
DDataSet	getHistogram() get the histogram of the data accumulated thus far.
static RankZeroDataSet	mean(QDataSet hist) returns the mean of the dataset that has been histogrammed.
static RankZeroDataSet	moments(QDataSet hist) returns the mean of the dataset that has been histogrammed.
QDataSet	monoExtent(QDataSet dep0) fast extent only works when monotonic.
static QDataSet	peakIds(QDataSet hist) return a list of all the peaks in the histogram.
static QDataSet	peaks(QDataSet hist) return a list of all the peaks in the histogram.
void	removePropertyChangeListener(java.beans.PropertyChangeListener listener)
void	reset()
static QDataSet	simpleRange(QDataSet hist2) returns the simple range, the min and the max containing the data.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

USER_PROP_BIN_START

public static final java.lang.String USER_PROP_BIN_START

See Also:

Constant Field Values

USER_PROP_BIN_WIDTH

public static final java.lang.String USER_PROP_BIN_WIDTH

See Also:

Constant Field Values

USER_PROP_INVALID_COUNT

public static final java.lang.String USER_PROP_INVALID_COUNT

See Also:

Constant Field Values

USER_PROP_OUTLIERS

public static final java.lang.String USER_PROP_OUTLIERS

See Also:

Constant Field Values

USER_PROP_MIN_GT_ZERO

public static final java.lang.String USER_PROP_MIN_GT_ZERO

See Also:

Constant Field Values

USER_PROP_TOTAL

public static final java.lang.String USER_PROP_TOTAL

Long, total number of valid points.

See Also:

Constant Field Values

Constructor Detail

AutoHistogram

public AutoHistogram()

Method Detail

reset

public void reset()

getHistogram

public DDataSet getHistogram()

get the histogram of the data accumulated thus far.

Returns:

binOf

public static int binOf(QDataSet hist,
                        double d)

convenient method for getting the bin location of a value from a completed histogram's metadata. Note this is inefficient since it must do HashMap lookups to get the bin width and bin start, so use this carefully.

Parameters:

hist -

d -

Returns:

the index of the bin for the point.

doit

public QDataSet doit(QDataSet ds)

monoExtent

public QDataSet monoExtent(QDataSet dep0)

fast extent only works when monotonic. Returns null if there is no valid data.

Parameters:

dep0 -

Returns:

rank 1 bins dataset or null

doit

public QDataSet doit(QDataSet ds,
                     QDataSet wds)

do a histogram, dynamically shifting the bins and changing the bin size. returns a QDataSet with the planes: each bin has the number of points in each bin. total total of values in the bin depend_0 is the bin names. The property QDataSet.USER_PROPERTIES contains a map with the following keys: binStart, Double, the left side of the first bin. binWidth, Double, the bin width. total, Integer, the number of points in the distribution. outliers, Map<Double,Integer>, outliers and the count of each outlier.

Parameters:

ds -

wds - WeightsDataSet or null.

Returns:

addPropertyChangeListener

public void addPropertyChangeListener(java.beans.PropertyChangeListener listener)

removePropertyChangeListener

public void removePropertyChangeListener(java.beans.PropertyChangeListener listener)

mean

public static RankZeroDataSet mean(QDataSet hist)

returns the mean of the dataset that has been histogrammed.

Parameters:

hist -

Returns:

moments

public static RankZeroDataSet moments(QDataSet hist)

returns the mean of the dataset that has been histogrammed.

Parameters:

hist - a rank 1 dataset with each bin containing the count in each bin. DEPEND_0 are the labels for each bin. The property "means" returns a rank 1 dataset containing the means for each bin. The property "stddevs" contains the standard deviation within each bin.

Returns:

rank 0 dataset (a Datum) whose value is the mean, and the property("stddev") contains the standard deviation

simpleRange

public static QDataSet simpleRange(QDataSet hist2)

returns the simple range, the min and the max containing the data.

Parameters:

hist2 - the result of autoHistogram.

Returns:

rank 1 bins dataset showing the min and max. value(0) is the min, value(1) is the max.

peakIds

public static QDataSet peakIds(QDataSet hist)

return a list of all the peaks in the histogram. A peak is defined as a local maximum, then including the adjacent bins consistent with the peak population, and not belonging to another peak.

Parameters:

hist - the output of autohistogram, which has "means" and "stddevs" properties.

Returns:

QDataSet which varies with hist.

peaks

public static QDataSet peaks(QDataSet hist)

return a list of all the peaks in the histogram. See peakIds to see how peaks are identified. Once the bins of a peak have been identified, then the mean and stddev of each peak is returned. Note the stddev typically reads low, probably because the tails have been removed.

Parameters:

hist - the result of AutoHistogram

Returns:

QDataSet rank 1 dataset with length equal to the number of identified peaks