org.das2.qds

Class ArrayDataSet

java.lang.Object

org.das2.qds.AbstractDataSet

org.das2.qds.ArrayDataSet

All Implemented Interfaces:

MutablePropertyDataSet, QDataSet, WritableDataSet

Direct Known Subclasses:

BDataSet, DDataSet, FDataSet, IDataSet, LDataSet, SDataSet

public abstract class ArrayDataSet
extends AbstractDataSet
implements WritableDataSet

ArrayDataSet is the abstract base class for QDataSets which are backed by Java arrays For example, DDataSet is a QDataSet which uses a double array to store its data. Data is stored in 1-D Java arrays for performance, inspired by https://www.cs.cmu.edu/~artigas/papers/cacm01.pdf. Note for modern versions of Java, Arrays of Arrays are implemented with no performance cost. A number of static methods were initially defined in DDataSet, then copied into FDataSet and others when they were made. This super implementation will parent all such datasets and provide common methods.

Field Summary

Fields

Modifier and Type	Field and Description
protected static java.util.logging.Logger	logger

Fields inherited from class org.das2.qds.AbstractDataSet

properties

Fields inherited from interface org.das2.qds.QDataSet

AVERAGE_TYPE, BIN_MAX, BIN_MAX_NAME, BIN_MIN, BIN_MIN_NAME, BIN_MINUS, BIN_MINUS_NAME, BIN_PLUS, BIN_PLUS_NAME, BINS_0, BINS_1, BUNDLE_0, BUNDLE_1, BUNDLE_2, BUNDLE_3, CACHE_TAG, CADENCE, CONTEXT_0, CONTEXT_1, COORDINATE_FRAME, DEFAULT_FILL_VALUE, DELTA_MINUS, DELTA_MINUS_NAME, DELTA_PLUS, DELTA_PLUS_NAME, DEPEND_0, DEPEND_1, DEPEND_2, DEPEND_3, DEPENDNAME_0, DEPENDNAME_1, DESCRIPTION, ELEMENT_DIMENSIONS, ELEMENT_LABEL, ELEMENT_NAME, FILL_VALUE, FORMAT, JOIN_0, LABEL, LIMIT_HUGE_DATASET, MAX_HIGH_RANK, MAX_PLANE_COUNT, MAX_RANK, MAX_UNIT_BUNDLE_COUNT, METADATA, METADATA_MODEL, MIN_WAVEFORM_LENGTH, MONOTONIC, NAME, NOTES, PLANE_0, QUBE, RENDER_TYPE, SCALE_TYPE, SOURCE, START_INDEX, TITLE, TYPICAL_MAX, TYPICAL_MIN, UNITS, USER_PROPERTIES, VALID_MAX, VALID_MIN, VALUE_AVERAGE_TYPE_GEOMETRIC, VALUE_AVERAGE_TYPE_LINEAR, VALUE_AVERAGE_TYPE_MOD24, VALUE_AVERAGE_TYPE_MOD360, VALUE_AVERAGE_TYPE_MODPI, VALUE_AVERAGE_TYPE_MODTAU, VALUE_AVERAGE_TYPE_NONE, VALUE_BINS_MIN_MAX, VALUE_BINS_MIN_MAX_INCLUSIVE, VALUE_COORDINATE_FRAME_COMPLEX_NUMBER, VALUE_METADATA_MODEL_ISTP, VALUE_METADATA_MODEL_SPASE, VALUE_RENDER_TYPE_COMPOSITE_IMAGE, VALUE_RENDER_TYPE_DIGITAL, VALUE_RENDER_TYPE_EVENTS_BAR, VALUE_RENDER_TYPE_NNSPECTROGRAM, VALUE_RENDER_TYPE_SERIES, VALUE_RENDER_TYPE_TRIANGLE_MESH, VALUE_SCALE_TYPE_LINEAR, VALUE_SCALE_TYPE_LOG, VERSION, WEIGHTS

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ArrayDataSet(java.lang.Class componentType)

Method Summary

Modifier and Type	Method and Description
void	about() print some info about this ArrayDataSet.
void	append(ArrayDataSet ds) append the dataset with the same geometry but different number of records (zeroth dim) to this.
static ArrayDataSet	append(ArrayDataSet ds1, ArrayDataSet ds2) append the second dataset onto this dataset.
boolean	canAppend(ArrayDataSet ds) return true if the dataset can be appended.
protected void	checkFill() check for fill property and set local variable.
static ArrayDataSet	copy(java.lang.Class c, QDataSet ds) Copy to array of specific type.
static ArrayDataSet	copy(QDataSet ds) copies the dataset into a writable ArrayDataSet, and all of its depend datasets as well.
static ArrayDataSet	copy(QDataSet ds, LongReadAccess lra)
static ArrayDataSet	create(java.lang.Class c, int[] qube) create a dataset with the backing type and the dimensions given.
protected static ArrayDataSet	create(int rank, int len0, int len1, int len2, int len3, java.lang.Object back)
static ArrayDataSet	createRank0(java.lang.Class c) create a rank 0 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class
static ArrayDataSet	createRank1(java.lang.Class c, int len0) create a rank 1 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class
static ArrayDataSet	createRank2(java.lang.Class c, int len0, int len1) create a rank 2 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class
static ArrayDataSet	createRank3(java.lang.Class c, int len0, int len1, int len2) create a rank 3 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class
static ArrayDataSet	createRank4(java.lang.Class c, int len0, int len1, int len2, int len3) create a rank 4 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class
static java.lang.Object	flattenArray(java.lang.Object array, int[] qube)
protected abstract java.lang.Object	getBack() provide access to the backing array.
protected abstract java.lang.Object	getBackCopy() return a copy of the backing array, to support ArrayDataSet.copy.
protected abstract int	getBackJvmMemory() return the size of the backing in JvmMemory.
protected abstract java.lang.Object	getBackReadOnly() return the backing array to the client, who promises not to modify the memory.
java.lang.Class	getComponentType() return the component type of the backing array, such as double.class.
void	grow(int newRecCount) grow the internal store so that append may be used to resize the dataset.
static java.lang.Class	guessBackingStore(QDataSet ds) guess the type of the backing store, returning double.class if it cannot be determined.
protected static java.util.Map	joinProperties(ArrayDataSet ds1, ArrayDataSet ds2) join the properties of the two datasets.
int	jvmMemory() returns the size of the dataset in bytes.
int	length() return the length of the first dimension
int	length(int i) return the length of the second dimension, for the ith element of the first dimension.
int	length(int i0, int i1) return the length of the third dimension, for the ith element of the first dimension and jth element of the second dimension.
int	length(int i0, int i1, int i2) return the length of the fourth dimension for the ith, jth and kth elements of the first three dimensions.
static ArrayDataSet	maybeCopy(java.lang.Class c, QDataSet ds) Copy the dataset to a ArrayDataSet only if the dataset is not already a particular instance of the given type of ArrayDataSet.
static ArrayDataSet	maybeCopy(QDataSet ds) Copy the dataset to an ArrayDataSet only if the dataset is not already an ArrayDataSet that is mutable.
static ArrayDataSet	monotonicSubset(ArrayDataSet ds) ensure that there are no non-monotonic or repeat records, by removing the first N-1 records of N repeated records.
static ArrayDataSet	monotonicSubset2(ArrayDataSet ds) ensure that there are no non-monotonic or repeat records, by starting at the middle of the dataset and finding the monotonic subsection.
void	putLength(int len) Shorten the dataset by changing it's dim 0 length parameter.
int	rank() returns the rank of the dataset, which is the number of indeces used to access data.
protected abstract void	setBack(java.lang.Object back) reset the back to this new backing array.
QDataSet	setUnits(Units units) set the units for this dataset.
java.lang.String	toString() return a human-readable string representation of this QDataSet.
static ArrayDataSet	wrap(java.lang.Object array, int[] qube, boolean copy) return the array as ArrayDataSet The array must be a 1-D array and the dimensions of the result are provided in qube.

Methods inherited from class org.das2.qds.AbstractDataSet

capability, checkImmutable, isImmutable, makeImmutable, property, property, putProperty, putProperty, slice, svalue, trim, value, value, value, value, value

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.das2.qds.WritableDataSet

putValue, putValue, putValue, putValue, putValue

Methods inherited from interface org.das2.qds.MutablePropertyDataSet

isImmutable, makeImmutable, putProperty, putProperty

Methods inherited from interface org.das2.qds.QDataSet

capability, property, property, slice, svalue, trim, value, value, value, value, value

Field Detail

logger

protected static final java.util.logging.Logger logger

Constructor Detail

ArrayDataSet

protected ArrayDataSet(java.lang.Class componentType)

Method Detail

create

protected static ArrayDataSet create(int rank,
                                     int len0,
                                     int len1,
                                     int len2,
                                     int len3,
                                     java.lang.Object back)

create

public static ArrayDataSet create(java.lang.Class c,
                                  int[] qube)

create a dataset with the backing type and the dimensions given. The following component types are supported: double.class, float.class, long.class, int.class, short.class, and byte.class.

Parameters:

c - the backing type, such as double.class, float.class, etc.

qube - dimensions of the dataset

Returns:

the dataset

See Also:

getComponentType(), which also supports unsigned types.

createRank0

public static ArrayDataSet createRank0(java.lang.Class c)

create a rank 0 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class

Parameters:

c - the primitive class of each element.

Returns:

the ArrayDataSet

createRank1

public static ArrayDataSet createRank1(java.lang.Class c,
                                       int len0)

create a rank 1 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class

Parameters:

c - the primitive class of each element.

len0 - the length of the dimension

Returns:

the ArrayDataSet

createRank2

public static ArrayDataSet createRank2(java.lang.Class c,
                                       int len0,
                                       int len1)

create a rank 2 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class

Parameters:

c - the primitive class of each element.

len0 - the length of the dimension

len1 - the length of the dimension

Returns:

the ArrayDataSet

createRank3

public static ArrayDataSet createRank3(java.lang.Class c,
                                       int len0,
                                       int len1,
                                       int len2)

create a rank 3 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class

Parameters:

c - the primitive class of each element.

len0 - the length of the dimension

len1 - the length of the dimension

len2 - the length of the dimension

Returns:

the ArrayDataSet

createRank4

public static ArrayDataSet createRank4(java.lang.Class c,
                                       int len0,
                                       int len1,
                                       int len2,
                                       int len3)

create a rank 4 dataset of the class, which can be double.class, float.class, long.class, int.class, short.class and byte.class

Parameters:

c - the primitive class of each element.

len0 - the length of the dimension

len1 - the length of the dimension

len2 - the length of the dimension

len3 - the length of the dimension

Returns:

the ArrayDataSet

flattenArray

public static java.lang.Object flattenArray(java.lang.Object array,
                                            int[] qube)

wrap

public static ArrayDataSet wrap(java.lang.Object array,
                                int[] qube,
                                boolean copy)

return the array as ArrayDataSet The array must be a 1-D array and the dimensions of the result are provided in qube.

Parameters:

array - 1-D array

qube - dimensions of the dataset

copy - copy the data so that original data is not modified with putValue

Returns:

ArrayDataSet

See Also:

create(java.lang.Class, int[])

monotonicSubset2

public static ArrayDataSet monotonicSubset2(ArrayDataSet ds)

ensure that there are no non-monotonic or repeat records, by starting at the middle of the dataset and finding the monotonic subsection. The input need not be writable.

Parameters:

ds - ArrayDataSet, which must be writable.

Returns:

dataset, possibly with records removed.

monotonicSubset

public static ArrayDataSet monotonicSubset(ArrayDataSet ds)

ensure that there are no non-monotonic or repeat records, by removing the first N-1 records of N repeated records.

Parameters:

ds - ArrayDataSet, which must be writable.

Returns:

dataset, possibly with records removed.

rank

public int rank()

Description copied from interface: QDataSet

returns the rank of the dataset, which is the number of indeces used to access data. Only rank 0, 1, 2, 3, and 4 datasets are supported in the interface. When a dataset's rank is 5 or greater, it should implement the HighRankDataSet interface which affords a slice operation to reduce rank.

Specified by:

rank in interface QDataSet

Specified by:

rank in class AbstractDataSet

Returns:

the rank, or number of indeces used to access data.

length

public final int length()

Description copied from interface: QDataSet

return the length of the first dimension

Specified by:

length in interface QDataSet

Overrides:

length in class AbstractDataSet

Returns:

the length of the first dimension

length

public final int length(int i)

Description copied from interface: QDataSet

return the length of the second dimension, for the ith element of the first dimension. Note if there are zero elements in the first dimension, but this is a QUBE, then this should not throw an IndexOutOfBoundsException.

Specified by:

length in interface QDataSet

Overrides:

length in class AbstractDataSet

Parameters:

i - the index

Returns:

the length of the second dimension at index i.

length

public final int length(int i0,
                        int i1)

Description copied from interface: QDataSet

return the length of the third dimension, for the ith element of the first dimension and jth element of the second dimension.

Specified by:

length in interface QDataSet

Overrides:

length in class AbstractDataSet

Parameters:

i0 - the index

i1 - the index

Returns:

the length of the third dimension at index i.

length

public final int length(int i0,
                        int i1,
                        int i2)

Description copied from interface: QDataSet

return the length of the fourth dimension for the ith, jth and kth elements of the first three dimensions.

Specified by:

length in interface QDataSet

Overrides:

length in class AbstractDataSet

Parameters:

i0 - the index

i1 - the index

i2 - the index

Returns:

the length of the fourth dimension for the ith, jth and kth elements of the first three dimensions.

getComponentType

public java.lang.Class getComponentType()

return the component type of the backing array, such as double.class.

Returns:

the component type of the backing array.

See Also:

create(java.lang.Class, int[])

putLength

public void putLength(int len)

Shorten the dataset by changing it's dim 0 length parameter. The same backing array is used, so the element that remain will be the same. This can only be used to shorten datasets, see grow to lengthen.

Parameters:

len - new number of records

See Also:

grow(int)

grow

public void grow(int newRecCount)

grow the internal store so that append may be used to resize the dataset.

Parameters:

newRecCount - new number of records, which includes the old records.

See Also:

putLength(int)

append

public void append(ArrayDataSet ds)

append the dataset with the same geometry but different number of records (zeroth dim) to this. An IllegalArgumentException is thrown when there is not enough room. See grow(newRecCount). Not thread safe--we need to go through and make it so...

Parameters:

ds -

canAppend

public boolean canAppend(ArrayDataSet ds)

return true if the dataset can be appended. Note this assumes that the same length, etc. This just checks that we have the number of spare records in the backing store.

Parameters:

ds - the dataset to test

Returns:

true if the dataset can be appended.

maybeCopy

public static ArrayDataSet maybeCopy(QDataSet ds)

Copy the dataset to an ArrayDataSet only if the dataset is not already an ArrayDataSet that is mutable.

Parameters:

ds - the dataset to copy, which may be an ArrayDataSet

Returns:

an ArrayDataSet.

maybeCopy

public static ArrayDataSet maybeCopy(java.lang.Class c,
                                     QDataSet ds)

Copy the dataset to a ArrayDataSet only if the dataset is not already a particular instance of the given type of ArrayDataSet.

Parameters:

c - component type, e.g. float.class double.class int.class, etc.

ds - the dataset.

Returns:

ArrayDayaSet of component type c, e.g. double.class, float.class, etc.

getBack

protected abstract java.lang.Object getBack()

provide access to the backing array.

Returns:

the backing array for the data.

getBackJvmMemory

protected abstract int getBackJvmMemory()

return the size of the backing in JvmMemory.

Returns:

the backing array for the data.

getBackCopy

protected abstract java.lang.Object getBackCopy()

return a copy of the backing array, to support ArrayDataSet.copy.

Returns:

a copy of the array for the data.

getBackReadOnly

protected abstract java.lang.Object getBackReadOnly()

return the backing array to the client, who promises not to modify the memory.

Returns:

the backing array for the data, which must not be modified.

setBack

protected abstract void setBack(java.lang.Object back)

reset the back to this new backing array.

Parameters:

back - the new backing array.

copy

public static ArrayDataSet copy(QDataSet ds,
                                LongReadAccess lra)

copy

public static ArrayDataSet copy(java.lang.Class c,
                                QDataSet ds)

Copy to array of specific type. For example, copy( double.class, ds ) would return a copy in a DDataSet.

Parameters:

c - the primitive type to use (e.g. double.class, float.class, int.class).

ds - the data to copy.

Returns:

ArrayDataSet of specific type.

copy

public static ArrayDataSet copy(QDataSet ds)

copies the dataset into a writable ArrayDataSet, and all of its depend datasets as well. Note this does not verify that the data is a qube!!!

Parameters:

ds - the data to be copied.

Returns:

a copy of the data.

See Also:

guessBackingStore(org.das2.qds.QDataSet)

guessBackingStore

public static java.lang.Class guessBackingStore(QDataSet ds)

guess the type of the backing store, returning double.class if it cannot be determined.

Parameters:

ds - the dataset

Returns:

the backing store class, one of double.class, float.class, long.class, int.class, short.class, or byte.class.

See Also:

copy

checkFill

protected void checkFill()

check for fill property and set local variable.

append

public static ArrayDataSet append(ArrayDataSet ds1,
                                  ArrayDataSet ds2)

append the second dataset onto this dataset. The two datasets need only have convertible units, so for example two time arrays may be appended even if their units don't have the same base, and the times will be converted. Only properties of the two datasets that do not change are preserved. When two rank 0 datasets are appended, the result will be rank 1.

Parameters:

ds1 - rank N dataset

ds2 - rank N dataset of the same type and compatible geometry as ds1.

Returns:

dataset with the data appended.

joinProperties

protected static java.util.Map joinProperties(ArrayDataSet ds1,
                                              ArrayDataSet ds2)

join the properties of the two datasets. This was introduced to support append, but may be useful in other contexts. Note there is special handling of properties, like CACHE_TAG and TYPICAL_MIN, TYPICAL_MAX. Note MONOTONIC assumes the ds2 will be added after ds1.

Parameters:

ds1 - dataset, typically a time series.

ds2 - dataset, typically a time series.

Returns:

the two sets combined.

setUnits

public QDataSet setUnits(Units units)

set the units for this dataset. This is a convenience method intended to encourage setting the data units. For example in Jython:

ds= linspace(0.,10.,100).setUnits( Units.seconds )

Parameters:

units - units object, like Units.seconds

Returns:

this dataset.

toString

public java.lang.String toString()

Description copied from class: AbstractDataSet

return a human-readable string representation of this QDataSet.

Overrides:

toString in class AbstractDataSet

Returns:

a human-readable string representation of this QDataSet.

See Also:

DataSetUtil.toString(org.das2.qds.QDataSet)

jvmMemory

public int jvmMemory()

returns the size of the dataset in bytes.

Returns:

the size of the dataset in bytes.

See Also:

which stores data outside of the JVM.

about

public void about()

print some info about this ArrayDataSet.