org.das2.qds

Class DataSetOps

java.lang.Object

org.das2.qds.DataSetOps

public class DataSetOps
extends java.lang.Object

Useful operations for QDataSets, such as slice2, leafTrim. TODO: identify which functions appear here instead of Ops.java.

Field Summary

Fields

Modifier and Type	Field and Description
static int	DS_LENGTH_LIMIT absolute length limit for plots.

Constructor Summary

Constructors

Constructor and Description
DataSetOps()

Method Summary

Modifier and Type	Method and Description
static int[]	addElement(int[] array, int value) adds an element to the array
static int[]	addElement(int value, int[] array) adds an element to the beginning of an array
static WritableDataSet	applyIndex(QDataSet ds, int idim, QDataSet sort, boolean deps) Applies the sort index to the idim-th dimension of the qube dataset ds.
static QDataSet	applyIndex(QDataSet ds, QDataSet indices) return the dataset with records rearranged according to indices.
static ArrayDataSet	applyIndexAllLists(QDataSet rods, QDataSet[] lists) handle special case where rank 1 datasets are used to index a rank N array.
static void	applyIndexInSitu(WritableDataSet ds, QDataSet sort) apply the sort to the data on the zeroth dimension.
static boolean	boundsContains(QDataSet bounds, Datum xValue, Datum yValue) return true of the bounds overlaps with the x and y values.
static java.lang.String[]	bundleNames(QDataSet bundleDs) return the names of the dataset that can be unbundled.
static boolean	changesDimensions(java.lang.String p) indicate if this one operator changes the dimensions.
static boolean	changesDimensions(java.lang.String c0, java.lang.String c1) indicate if the operators change dimensions of the dataset.
static boolean	changesIndependentDimensions(java.lang.String p) indicate if this one operator changes the independent dimensions.
static boolean	changesIndependentDimensions(java.lang.String c0, java.lang.String c1) indicate if the operators change dimensions of the dataset.
static QDataSet	dbAboveBackgroundDim0(QDataSet ds, double level) normalize the level-th percentile from: rank 1: each element (same as removeBackground1) rank 2: each column of the dataset rank 3: each column of each rank 2 dataset slice.
static QDataSet	dbAboveBackgroundDim1(QDataSet ds, double level) normalize the nth-level percentile from: rank 1: each element rank 2: each row of the dataset rank 3: each row of each rank 2 dataset slice.
static QDataSet	dbAboveBackgroundDim1(QDataSet ds, double level, boolean power) normalize the nth-level percentile from: rank 1: each element rank 2: each row of the dataset rank 3: each row of each rank 2 dataset slice.
static QDataSet	dependBounds(QDataSet ds) return a bounding qube of the independent dimensions containing the dataset.
static QDataSet	dependBoundsSimple(QDataSet ds) return a bounding qube of the independent dimensions containing the dataset.
static QDataSet	flattenBundleDescriptor(QDataSet bundle1) returns a bundle descriptor roughly equivalent to the BundleDescriptor passed in, but will describe each dataset as if it were rank 1.
static QDataSet	flattenRank2(QDataSet ds) flatten a rank 2 dataset.
static QDataSet	flattenRank3(QDataSet ds) flatten a rank 3 dataset.
static QDataSet	flattenRank4(QDataSet ds) flatten a rank 4 dataset.
static QDataSet	flattenWaveform(QDataSet ds) flatten a rank 2 dataset where the y depend variable is just an offset from the xtag.
static QDataSet	getBackgroundLevel(QDataSet ds, double level) Get the background level by sorting the data.
static java.lang.Class	getComponentType(QDataSet ds) return the class type that can accurately store data in this dataset.
protected static QDataSet	getContextForUnbundle(QDataSet bundle1, int index) given the bundle descriptor bds, return the dataset to be put in the context property.
static QDataSet	getNthPercentileSort(QDataSet ds, double n) returns the value from within a distribution that is the nth percentile division.
static QDataSet	grid(QDataSet ds) takes rank 2 link (x,y,z) and makes a table from it z(x,y)
static QDataSet	histogram(QDataSet ds, double min, double max, double binsize) returns a rank 1 dataset that is a histogram of the data.
static int	indexOfBundledDataSet(QDataSet bundleDs, java.lang.String name) return the index of the named bundled dataset.
static boolean	isProcessAsync(java.lang.String c) return true if the process described in c is probably a slow process that should be done asynchronously.
static MutablePropertyDataSet	leafTrim(QDataSet ds, int start, int end) pull out a subset of the dataset by reducing the number of columns in the last dimension.
static java.lang.String	makeProcessStringCanonical(java.lang.String s) replace any component reference C, to explicit "|unbundle(C)"
static MutablePropertyDataSet	makePropertiesMutable(QDataSet dataset) return a dataset that has mutable properties.
static WritableDataSet	makeWritable(QDataSet dataset) return a dataset that is writable.
static RankZeroDataSet	moment(QDataSet ds) performs the moment (mean,variance,etc) on the dataset.
static QDataSet	processDataSet(java.lang.String c, QDataSet fillDs, ProgressMonitor mon) apply process to the data.
static int[]	removeElement(int[] array, int index) removes the index-th element from the array.
static MutablePropertyDataSet	slice(QDataSet ds, int dimension, int index) slice on the dimension.
static MutablePropertyDataSet	slice0(QDataSet ds, int index) slice on the first dimension.
static MutablePropertyDataSet	slice1(QDataSet ds, int index) slice dataset operator assumes a qube dataset by picking the index-th element of dataset's second dimension, without regard to tags.
static MutablePropertyDataSet	slice2(QDataSet ds, int index) slice dataset operator assumes a qube dataset by picking the index-th element of dataset's second dimension, without regard to tags.
static MutablePropertyDataSet	slice3(QDataSet ds, int index) slice dataset operator assumes a qube dataset by picking the index-th element of dataset's second dimension, without regard to tags.
static java.util.Map<java.lang.String,java.lang.Object>	sliceProperties(java.util.Map<java.lang.String,java.lang.Object> properties, int sliceDimension) we've sliced a dataset, removing an index.
static java.util.Map<java.lang.String,java.lang.Object>	sliceProperties0(int index, java.util.Map<java.lang.String,java.lang.Object> props) method to help dataset implementations implement slice.
static QDataSet	sort(QDataSet ds) returns a list of indeces that sort the dataset.
static QDataSet	sprocess(java.lang.String c, QDataSet fillDs, ProgressMonitor mon) sprocess implements the poorly-named filters string / process string of Autoplot, allowing clients to "pipe" data through a chain of operations.
static double	suggestFillForComponentType(java.lang.Class c) return a fill value that is representable by the type.
static QDataSet	transpose2(QDataSet ds) transpose the rank 2 qube dataset so the rows are columns and the columns are rows.
static MutablePropertyDataSet	trim(QDataSet ds, int offset, int len) reduce the number of elements in the dataset to the dim 0 indeces specified.
static MutablePropertyDataSet	trim(QDataSet dep, int start, int stop, int stride) reduce the number of elements in the dataset to the dim 0 indeces specified.
static QDataSet	unbundle(QDataSet bundleDs, int ib) Extract a bundled dataset from a bundle of datasets.
static QDataSet	unbundle(QDataSet bundleDs, int ib, boolean highRank) Extract a bundled dataset from a bundle of datasets.
static QDataSet	unbundle(QDataSet bundleDs, java.lang.String name) Extract the named bundled dataset.
static QDataSet	unbundleDefaultDataSet(QDataSet bundleDs) extract the dataset that is dependent on others, or the last one.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DS_LENGTH_LIMIT

public static final int DS_LENGTH_LIMIT

absolute length limit for plots. This is used to limit the elements used in autoranging, etc.

See Also:

Constant Field Values

Constructor Detail

DataSetOps

public DataSetOps()

Method Detail

makePropertiesMutable

public static MutablePropertyDataSet makePropertiesMutable(QDataSet dataset)

return a dataset that has mutable properties. If the dataset parameter already has, then the dataset is returned. If the dataset is a MutablePropertyDataSet but the immutable flag is set, then the dataset is wrapped to make the properties mutable.

Parameters:

dataset - dataset

Returns:

a MutablePropertyDataSet that is has a wrapper around the dataset, or the dataset.

See Also:

DataSetWrapper

makeWritable

public static WritableDataSet makeWritable(QDataSet dataset)

return a dataset that is writable. If the dataset parameter of this idempotent function is already writable, then the dataset is returned. If the dataset is a WritableDataSet but the immutable flag is set, then the a copy is returned.

Parameters:

dataset -

Returns:

a WritableDataSet that is either a copy of the read-only dataset provided, or the parameter writable dataset provided.

slice

public static MutablePropertyDataSet slice(QDataSet ds,
                                           int dimension,
                                           int index)

slice on the dimension. This saves from the pain of having this branch all over the code.

Parameters:

ds - the rank N data to slice.

dimension - the dimension to slice, 0 is the first.

index - the index to slice at.

Returns:

the rank N-1 result.

slice0

public static MutablePropertyDataSet slice0(QDataSet ds,
                                            int index)

slice on the first dimension. Note the function ds.slice(index) was added later and will typically be more efficient. This will create a new Slice0DataSet. DO NOT try to optimize this by calling native trim, some native slice implementations call this. TODO: This actually needs a bit more study, because there are codes that talk about not using the native slice because it copies data and they just want metadata. This probably is because Slice0DataSet doesn't check for immutability, and really should be copying. This needs to be fixed, making sure the result of this call is immutable, and the native slice really should be more efficient, always.

Parameters:

ds - rank 1 or more dataset

index - the index to slice at

Returns:

rank 0 or more dataset.

See Also:

QDataSet.slice(int)

slice1

public static MutablePropertyDataSet slice1(QDataSet ds,
                                            int index)

slice dataset operator assumes a qube dataset by picking the index-th element of dataset's second dimension, without regard to tags.

Parameters:

ds - rank 2 or more dataset

index - the index to slice at

Returns:

rank 1 or more dataset.

slice2

public static MutablePropertyDataSet slice2(QDataSet ds,
                                            int index)

slice dataset operator assumes a qube dataset by picking the index-th element of dataset's second dimension, without regard to tags.

Parameters:

ds - rank 3 or more dataset

index - the index to slice at.

Returns:

rank 2 or more dataset.

slice3

public static MutablePropertyDataSet slice3(QDataSet ds,
                                            int index)

slice dataset operator assumes a qube dataset by picking the index-th element of dataset's second dimension, without regard to tags.

Parameters:

ds - rank 4 or more dataset.

index - index to slice at

Returns:

rank 3 or more dataset.

trim

public static MutablePropertyDataSet trim(QDataSet ds,
                                          int offset,
                                          int len)

reduce the number of elements in the dataset to the dim 0 indeces specified. This does not change the rank of the dataset. DO NOT try to optimize this by calling native trim, some native trim implementations call this.

Parameters:

ds - the dataset

offset - the offset

len - the length, (not the stop index!)

Returns:

trimmed dataset

trim

public static MutablePropertyDataSet trim(QDataSet dep,
                                          int start,
                                          int stop,
                                          int stride)

reduce the number of elements in the dataset to the dim 0 indeces specified. This does not change the rank of the dataset.

Parameters:

dep - the dataset.

start - first index to include

stop - last index, exclusive

stride - the step size, e.g. 2 is every other element.

Returns:

trimmed dataset

flattenRank2

public static QDataSet flattenRank2(QDataSet ds)

flatten a rank 2 dataset. The result is a n,3 dataset of [x,y,f]. History:

• modified for use in PW group.
• missing DEPEND_1 resulted in NullPointerException, so just use 0,1,2,..,n instead and always have rank 2 result.

Parameters:

ds - rank 2 table dataset

Returns:

rank 2 dataset that is that is array of (x,y,f).

flattenRank3

public static QDataSet flattenRank3(QDataSet ds)

flatten a rank 3 dataset. The result is a n,4 dataset of [x,y,z,f], or if there are no tags just rank 1 f. For a rank 3 join (array of tables), the result will be ds[n,3].

Parameters:

ds - rank 3 table dataset

Returns:

rank 2 dataset that is array of (x,y,z,f) or rank 1 f.

flattenRank4

public static QDataSet flattenRank4(QDataSet ds)

flatten a rank 4 dataset. The result is an n,5 dataset of [x1,x2,x3,x4,f], or if there are no tags just rank 1 f.

Parameters:

ds - rank 3 table dataset

Returns:

rank 2 dataset that is array of (x1,x2,x3,x4,f) or rank 1 f.

flattenWaveform

public static QDataSet flattenWaveform(QDataSet ds)

flatten a rank 2 dataset where the y depend variable is just an offset from the xtag. This is a nice example of the advantage of using a class to represent the data: this requires no additional storage to handle the huge waveform. Note the new DEPEND_0 may have different units from ds.property(DEPEND_0).

Parameters:

ds - rank 2 waveform with tags for DEPEND_0 and offsets for DEPEND_1

Returns:

rank 1 waveform

grid

public static QDataSet grid(QDataSet ds)

takes rank 2 link (x,y,z) and makes a table from it z(x,y)

Parameters:

ds - rank 2 link (x,y,z)

Returns:

a table from it z(x,y)

removeElement

public static int[] removeElement(int[] array,
                                  int index)

removes the index-th element from the array.

Parameters:

array - length N array

index - the index to remove

Returns:

array without the element, length N-1.

addElement

public static int[] addElement(int[] array,
                               int value)

adds an element to the array

Parameters:

array - length N array

value - the value to append

Returns:

array with the element, length N+1.

addElement

public static int[] addElement(int value,
                               int[] array)

adds an element to the beginning of an array

Parameters:

array - length N array

value - the value to append

Returns:

array with the element, length N+1.

leafTrim

public static MutablePropertyDataSet leafTrim(QDataSet ds,
                                              int start,
                                              int end)

pull out a subset of the dataset by reducing the number of columns in the last dimension. This does not reduce rank. This assumes the dataset has no row with length>end. This is extended to support rank 4 datasets. TODO: This probably doesn't handle bundles property. TODO: slice and trim should probably be implemented here for efficiently.

Parameters:

ds - rank 1 or more dataset

start - first index to include.

end - last index, exclusive

Returns:

dataset of the same rank.

sort

public static QDataSet sort(QDataSet ds)

returns a list of indeces that sort the dataset. I don't like this implementation, because it requires that an array of Integers (not int[]) be created. Invalid measurements are not indexed in the returned dataset. If the sort is monotonic, then the property MONOTONIC will be Boolean.TRUE.

Parameters:

ds - rank 1 dataset, possibly containing fill.

Returns:

indeces that sort the data.

applyIndexInSitu

public static void applyIndexInSitu(WritableDataSet ds,
                                    QDataSet sort)

apply the sort to the data on the zeroth dimension. The dataset must be mutable, and the dataset itself is modified. This was introduced to support AggregatingDataSource but should be generally useful.

Parameters:

ds - a writable dataset that is still mutable.

sort - the new sort indeces.

getComponentType

public static java.lang.Class getComponentType(QDataSet ds)

return the class type that can accurately store data in this dataset. This was motivated by DDataSets and FDataSets, but also IndexGenDataSets.

Parameters:

ds - the dataset.

Returns:

the class that can store this type. double.class is returned when the class cannot be identified.

See Also:

ArrayDataSet.create(java.lang.Class, int[])

suggestFillForComponentType

public static double suggestFillForComponentType(java.lang.Class c)

return a fill value that is representable by the type.

Parameters:

c - the class type, including double.class, float.class, etc.

Returns:

a fill value that is representable by the type.

applyIndex

public static QDataSet applyIndex(QDataSet ds,
                                  QDataSet indices)

return the dataset with records rearranged according to indices.

Parameters:

ds - rank N dataset, where N>0

indices - rank 1 dataset, length m.

Returns:

length m rank N dataset.

See Also:

applyIndex(org.das2.qds.QDataSet, int, org.das2.qds.QDataSet, boolean)

applyIndex

public static WritableDataSet applyIndex(QDataSet ds,
                                         int idim,
                                         QDataSet sort,
                                         boolean deps)

Applies the sort index to the idim-th dimension of the qube dataset ds. TODO: consider sorting multiple dimensions at once, to reduce excessive copying. TODO: this should probably (and would easily) be redone by using dataset implementation that applies the sort on the ith index when read. See SubsetDataSet which would do this nicely. TODO: note the Jython stuff does this to, using a different implementation. Reconcile these...

Parameters:

ds - rank 1,2, or 3 qube dataset

idim - the dimension being sorted.

sort - rank 1 dataset of new indeces, needn't be same size as index.

deps - do dependencies as well. Note this does not rearrange planes!

Returns:

new dataset that is a copy of the first, resorted.

See Also:

for similar functionality, Ops.decimate(org.das2.qds.QDataSet, int, int), applyIndexAllLists(org.das2.qds.QDataSet, org.das2.qds.QDataSet[])

applyIndexAllLists

public static ArrayDataSet applyIndexAllLists(QDataSet rods,
                                              QDataSet[] lists)

handle special case where rank 1 datasets are used to index a rank N array. Supports negative indices. This was extracted from PyQDataSet because it should be useful in Java codes as well.

Parameters:

rods - the dataset

lists - datasets of rank 0 or rank 1

Returns:

the array extracted.

See Also:

which is similar

histogram

public static QDataSet histogram(QDataSet ds,
                                 double min,
                                 double max,
                                 double binsize)

returns a rank 1 dataset that is a histogram of the data. Note there will also be in the properties: count, the total number of valid values. nonZeroMin, the smallest non-zero, positive number

Parameters:

ds - rank N dataset

min - the min of the first bin. If min=-1 and max=-1, then automatically set the min and max.

max - the max of the last bin.

binsize - the size of each bin.

Returns:

a rank 1 dataset with each bin's count. DEPEND_0 indicates the bin locations.

moment

public static RankZeroDataSet moment(QDataSet ds)

performs the moment (mean,variance,etc) on the dataset.

Parameters:

ds - rank N QDataSet.

Returns:

rank 0 dataset of the mean. Properties contain other stats: stddev, RankZeroDataSet validCount, Integer, the number valid measurements invalidCount, Integer, the number of invalid measurements

transpose2

public static QDataSet transpose2(QDataSet ds)

transpose the rank 2 qube dataset so the rows are columns and the columns are rows.

Parameters:

ds - rank 2 Qube DataSet.

Returns:

rank 2 Qube DataSet

sliceProperties0

public static java.util.Map<java.lang.String,java.lang.Object> sliceProperties0(int index,
                                                                                java.util.Map<java.lang.String,java.lang.Object> props)

method to help dataset implementations implement slice. 2010-09-23: support rank 2 DEPEND_2 and DEPEND_3 2010-09-23: add BINS_1 and BUNDLE_1, Slice0DataSet calls this. 2010-02-24: BUNDLE_0 handled. 2011-03-25: add WEIGHTS_PLANE

Parameters:

index - the index to slice at in the zeroth index.

props - the properties to slice.

Returns:

the properties after the slice.

sliceProperties

public static java.util.Map<java.lang.String,java.lang.Object> sliceProperties(java.util.Map<java.lang.String,java.lang.Object> properties,
                                                                               int sliceDimension)

we've sliced a dataset, removing an index. move the properties. This was Ops.sliceProperties For example, after slicing the zeroth dimension (time), what was DEPEND_1 is becomes DEPEND_0.

Parameters:

properties - the properties to slice.

sliceDimension - the dimension to slice at (0,1,2...QDataSet.MAX_HIGH_RANK)

Returns:

the properties after the slice.

flattenBundleDescriptor

public static QDataSet flattenBundleDescriptor(QDataSet bundle1)

returns a bundle descriptor roughly equivalent to the BundleDescriptor passed in, but will describe each dataset as if it were rank 1. This is useful for when the client can't work with mixed rank bundles anyway (like display data).

Parameters:

bundle1 -

Returns:

bundleNames

public static java.lang.String[] bundleNames(QDataSet bundleDs)

return the names of the dataset that can be unbundled.

Parameters:

bundleDs -

Returns:

and array of the bundle names.

Throws:

java.lang.IllegalArgumentException - when bundleDs is not a bundle.

See Also:

unbundle(org.das2.qds.QDataSet, java.lang.String)

indexOfBundledDataSet

public static int indexOfBundledDataSet(QDataSet bundleDs,
                                        java.lang.String name)

return the index of the named bundled dataset. This cleans up the name so that is contains just a Java-style identifier. Also, ch_1 is always implicitly index 1. Last, if safe names created from labels match that this is used. For example,

bds=ripplesVectorTimeSeries(100)
2==indexOfBundledDataSet( bds, "Z" ) 

demonstrates its use. Last, extraneous spaces and underscores are removed to see if this will result in a match.

Parameters:

bundleDs - a bundle dataset with the property BUNDLE_1 or DEPEND_1 having EnumerationUnits, (or BUNDLE_0 for a rank 1 dataset).

name - the named dataset.

Returns:

the index or -1 if the name is not found.

unbundle

public static QDataSet unbundle(QDataSet bundleDs,
                                java.lang.String name)

Extract the named bundled dataset. For example, extract B_x from bundle of components.

Parameters:

bundleDs - a bundle of datasets

name - the name of the bundled dataset, or "ch_<i>" where i is the dataset number

Returns:

the named dataset

Throws:

java.lang.IllegalArgumentException - if no named dataset is found.

See Also:

unbundle(org.das2.qds.QDataSet, int)

unbundleDefaultDataSet

public static QDataSet unbundleDefaultDataSet(QDataSet bundleDs)

extract the dataset that is dependent on others, or the last one. For example, the dataset ds[:,"x,y"] → y[:]

Parameters:

bundleDs - a bundle of datasets

Returns:

the default dataset

See Also:

Schemes.bundleDataSet()

unbundle

public static QDataSet unbundle(QDataSet bundleDs,
                                int ib)

Extract a bundled dataset from a bundle of datasets. The input should be a rank 2 dataset with the property BUNDLE_1 set to a bundle descriptor dataset. See BundleDataSet for more semantics. Note we support the case where DEPEND_1 has EnumerationUnits, and this is the same as slice1.

Parameters:

bundleDs - the bundle dataset.

ib - index of the dataset to extract. If the index is within a dataset, then the entire dataset is returned.

Returns:

the ib-th dataset from the bundle.

Throws:

java.lang.IndexOutOfBoundsException - if the index is invalid.

java.lang.IllegalArgumentException - if the dataset is not a bundle dataset, with either BUNDLE_1 or DEPEND_1 set.

unbundle

public static QDataSet unbundle(QDataSet bundleDs,
                                int ib,
                                boolean highRank)

Extract a bundled dataset from a bundle of datasets. The input should be a rank 2 dataset with the property BUNDLE_1 set to a bundle descriptor dataset. See BundleDataSet for more semantics. Note we support the case where DEPEND_1 has EnumerationUnits, and this is the same as slice1.

Parameters:

bundleDs - the bundle dataset.

ib - index of the dataset to extract. If the index is within a dataset, then the entire dataset is returned.

highRank - if true, then if the dataset at ib is rank 2 or greater, then then the entire dataset is returned. If false, only the slice of the dataset is returned.

Returns:

the ib-th dataset from the bundle.

Throws:

java.lang.IndexOutOfBoundsException - if the index is invalid.

java.lang.IllegalArgumentException - if the dataset is not a bundle dataset, with either BUNDLE_1 or DEPEND_1 set.

See Also:

Ops.bundle(org.das2.qds.QDataSet, org.das2.qds.QDataSet)

getContextForUnbundle

protected static QDataSet getContextForUnbundle(QDataSet bundle1,
                                                int index)

given the bundle descriptor bds, return the dataset to be put in the context property.

Parameters:

bundle1 - rank 2 bundle descriptor of length n with indexed properties. This was introduced when sloppy slice code was using the NAME and dropping the LABEL.

index - 0<=index<n index of the unbundle

Returns:

rank 0 QDataSet

getNthPercentileSort

public static QDataSet getNthPercentileSort(QDataSet ds,
                                            double n)

returns the value from within a distribution that is the nth percentile division. This returns a fill dataset (Units.dimensionless.getFillDouble()) when the data is all fill.

Parameters:

ds - the dataset

n - percent between 0 and 100.

Returns:

getBackgroundLevel

public static QDataSet getBackgroundLevel(QDataSet ds,
                                          double level)

Get the background level by sorting the data. The result is rank one less than the input rank.

Parameters:

ds - rank 1, 2, or rank 3 join.

level - the level between 0 and 100.

Returns:

dbAboveBackgroundDim1

public static QDataSet dbAboveBackgroundDim1(QDataSet ds,
                                             double level)

normalize the nth-level percentile from:

• rank 1: each element
• rank 2: each row of the dataset
• rank 3: each row of each rank 2 dataset slice.

If the data is already in dB, then the result is a difference. This is assuming the units are similar to voltage, not a power, we think, containing code like 20 * Math.log10( ds / background ).

Parameters:

ds -

level - the percentile level, e.g. 10= 10%

Returns:

the result dataset, in dB above background.

dbAboveBackgroundDim1

public static QDataSet dbAboveBackgroundDim1(QDataSet ds,
                                             double level,
                                             boolean power)

normalize the nth-level percentile from:

• rank 1: each element
• rank 2: each row of the dataset
• rank 3: each row of each rank 2 dataset slice.

If the data is already in dB, then the result is a difference. This is assuming the units are similar to voltage, not a power, containing code like 20 * Math.log10( ds / background ).

Parameters:

ds -

level - the percentile level, e.g. 10= 10%

power - if true, return 10*Math.log10(ds / background ).

Returns:

the result dataset, in dB above background.

dbAboveBackgroundDim0

public static QDataSet dbAboveBackgroundDim0(QDataSet ds,
                                             double level)

normalize the level-th percentile from: rank 1: each element (same as removeBackground1) rank 2: each column of the dataset rank 3: each column of each rank 2 dataset slice. There must be at least 10 elements. If the data is already in dB, then the result is a difference. This is assuming the units are similar to voltage, not a power, we think, containing code like 20 * Math.log10( ds / background ).

Parameters:

ds -

level - the percentile level, e.g. 10= 10%

Returns:

the result dataset, in dB above background.

isProcessAsync

public static boolean isProcessAsync(java.lang.String c)

return true if the process described in c is probably a slow process that should be done asynchronously. For example, do a long fft on a different thread and use a progress monitor. Processes that take a trivial, constant amount of time should return false, and may be completed on the event thread,etc.

Parameters:

c - process string, as in sprocess.

Returns:

true if the process described in c is probably a slow process

processDataSet

public static QDataSet processDataSet(java.lang.String c,
                                      QDataSet fillDs,
                                      ProgressMonitor mon)
                               throws java.lang.RuntimeException,
                                      java.lang.Exception

apply process to the data. This is like sprocess, except that the component can be extracted as the first step. In general these can be done on the same thread (like slice1), but some are slow (like fftPower). This is a copy of PlotElementController.processDataSet.

Parameters:

c - the process string, like "bgsmx|slice0(9)|histogram()"

fillDs - the input dataset.

mon - a monitor for the processing

Returns:

dataset resulting form filters.

Throws:

java.lang.Exception - when the processStr cannot be processed.

java.lang.RuntimeException - when the component (e.g. bgsmx) is not found.

sprocess

public static QDataSet sprocess(java.lang.String c,
                                QDataSet fillDs,
                                ProgressMonitor mon)
                         throws java.lang.Exception

sprocess implements the poorly-named filters string / process string of Autoplot, allowing clients to "pipe" data through a chain of operations. For example, the filters string "|slice0(9)|histogram()" will slice on the ninth index and then take a histogram of that result. See http://www.papco.org/wiki/index.php/DataReductionSpecs (TODO: wiki page was lost, which could probably be recovered.) There's a big problem here: if the command is not recognized, then it is ignored. We should probably change this, but the change should be at a major version change in case it breaks things.

Parameters:

c - process string like "slice0(9)|histogram()"

fillDs - The dataset loaded from the data source controller, with initial filters (like fill) applied.

mon - monitor for the processing.

Returns:

the dataset after the process string is applied.

Throws:

java.text.ParseException - when the string cannot be parsed

java.lang.Exception - when a function cannot be processed (e.g. index out of bounds)

See Also:

http://autoplot.org/developer.dataset.filters, http://autoplot.org/developer.panel_rank_reduction

changesDimensions

public static boolean changesDimensions(java.lang.String p)

indicate if this one operator changes the dimensions. For example, |smooth doesn't change the dimensions, but fftPower and slice do.

Parameters:

p - the filter, e.g. "|smooth"

Returns:

true if the dimensions change.

changesIndependentDimensions

public static boolean changesIndependentDimensions(java.lang.String p)

indicate if this one operator changes the independent dimensions. For example, |smooth doesn't change the dimensions, but |multiply also doesn't change the independent dimension.

Parameters:

p - the filter, e.g. "|smooth"

Returns:

true if the dimensions change.

makeProcessStringCanonical

public static java.lang.String makeProcessStringCanonical(java.lang.String s)

replace any component reference C, to explicit "|unbundle(C)"

Parameters:

s - the process string, like "X|fftPower(512,2)"

Returns:

canonical version, like "|unbundle(X)|fftPower(512,2)"

changesIndependentDimensions

public static boolean changesIndependentDimensions(java.lang.String c0,
                                                   java.lang.String c1)

indicate if the operators change dimensions of the dataset. Often this will result in true when the dimensions do not change, this is the better way to err.

Parameters:

c0 - old value for the process string, e.g. "|slice0(0)"

c1 - new value for the process string, e.g. "|slice0(0)|slice1(0)"

Returns:

true if the dimensions would be different.

changesDimensions

public static boolean changesDimensions(java.lang.String c0,
                                        java.lang.String c1)

indicate if the operators change dimensions of the dataset. Often this will result in true when the dimensions do not change, this is the better way to err.

Parameters:

c0 - old value for the process string, e.g. "|slice0(0)"

c1 - new value for the process string, e.g. "|slice0(0)|slice1(0)"

Returns:

true if the dimensions would be different.

dependBoundsSimple

public static QDataSet dependBoundsSimple(QDataSet ds)

return a bounding qube of the independent dimensions containing the dataset. If r is the result of the function, then for

• rank 1: r.slice(0) x bounds, r.slice(1) y bounds
• rank 2 waveform: r.slice(0) x bounds, r.slice(1) y bounds
• rank 2 table:r.slice(0) x bounds r.slice(1) DEPEND_0 bounds.
• rank 3 table:r.slice(0) x bounds r.slice(1) DEPEND_0 bounds.

This does not take DELTA_PLUS and DELTA_MINUS into account. When all the data is fill, ds[0,0] will be positive infinity.

Parameters:

ds - a rank 1,2, or 3 dataset.

Returns:

a bounding qube of the independent dimensions

dependBounds

public static QDataSet dependBounds(QDataSet ds)

return a bounding qube of the independent dimensions containing the dataset. If r is the result of the function, then for

• rank 1: r.slice(0) x bounds, r.slice(1) y bounds
• rank 2 waveform: r.slice(0) x bounds, r.slice(1) y bounds
• rank 2 table:r.slice(0) x bounds r.slice(1) DEPEND_0 bounds.
• rank 3 table:r.slice(0) x bounds r.slice(1) DEPEND_0 bounds.

Parameters:

ds - rank 1,2, or 3 dataset.

Returns:

a bounding qube of the independent dimensions

boundsContains

public static boolean boundsContains(QDataSet bounds,
                                     Datum xValue,
                                     Datum yValue)

return true of the bounds overlaps with the x and y values.

Parameters:

bounds - bounding box

xValue - the x range

yValue - the y range

Returns:

true of the bounds overlap

Throws:

java.lang.IllegalArgumentException - if the units cannot be converted

java.lang.IllegalArgumentException - if the bounds does not have BINS_0 and BINS_1.