org.das2.fsm.FileStorageModel

Represents a method for storing data sets in a set of files by time. The client provides a regex for the files and how each group of the regex is interpreted as a time digit. The model can then be used to provide the set of files that cover a time range, etc. This new implementation uses a TimeParser object to more quickly process file names.

cacheCleanup

remove files that have been identified as old versions.

Returns:

containsFile

return true if the file came (or could come) from this FileStorageModel.

Parameters

Returns:

containsName

return true if the name came (or could come) from this FileStorageModel.

Parameters

Returns:

create

creates a FileStorageModel for the given template, which uses:

%Y-%m-%dT%H:%M:%S.%{milli}Z";
    %Y  4-digit year
    %m  2-digit month
    %d  2-digit day of month
    %j  3-digit day of year
    %H  2-digit Hour
    %M  2-digit Minute
    %S  2-digit second
    %v  best version by number  Also %(v,sep) for 4.3.2  or %(v,alpha)
    %{milli}  3-digit milliseconds 

Parameters

Returns:

generateNamesFor

generate the names of the files that would cover this range. This was taken from Autoplot's org.virbo.jythonsupport.Util. TODO: versioning, etc.

Parameters

Returns:

getBestFilesFor

return the best files found for the range, without progress feedback.

Parameters

Returns:

getBestNamesFor

return the names in the range, minding version numbers, or all available names if the range is null. This will list directories.

Parameters

Returns:

getCacheTagFor

Parameters

Returns:

getChildFileSystem

return a child filesystem, with special code for LocalFileSystems to support Windows. TODO: look into .zip child.

Parameters

Returns:

See Also:

getField

return the field value for the given name. For example, if the spec is $Y/$m/$d/$Y$m$d_v$(v,sep).dat and the name matched is 2014/04/04/20140404_v2.3.dat then calling this for the field "v" would result in "2.3" This should not be used to retrieve fields that are components of the time range, such as $Y or $m.

Parameters

Returns:

getFileFor

download the file for the given name within the filesystem.

Parameters

Returns:

getFileSystem

return the filesystem used to implement this.

Returns:

getFilenameFor

return a filename that would intersect the range. Note this file may not actually exist. This may be used to quantize the range. The template may not contain versions.

Parameters

Returns:

getFilesFor

Parameters

Returns:

getNameFor

Provides a way to recover the model name of a file. The returned File from getFilesFor can be anywhere, so it would be good to provide a way to get it back into a FSM name. For example, a filesystem might download the remote file to a cache directory, which is the File that is provided to the client, sometimes the client will need to recover the name of the corresponding FileObject, so this maps the File back to the name.

Parameters

Returns:

getNamesFor

return the names in the range, or all names if the range is null.

Parameters

Returns:

getParent

returns the parent or null if none exists. None will exist when there are no more wildcards. (/home/foo/$Y/$m-$d.dat has parent /home/foo/$Y which has parent null.)

Returns:

getRangeFor

return the time range represented by this name.

Parameters

Returns:

getRepresentativeFile

return a random file from the FSM, which can be used to represent a typical file. For example, we need to look at metadata to see what is available.

Parameters

Returns:

getRoot

return the root of the filesystem as a string.

Returns:

hasField

returns true if the parser has the field.

Parameters

Returns:

quantize

return the timerange that contains the given timerange and exactly contains a set of granules. This needs to be synchronized because the timeParser.

Parameters

Returns:

setContext

set the datum range giving context to the files. For example, filenames are just $H$M$S.dat, and the context is "Jan 17th, 2015"

Parameters

Returns:

splitIndex

This returns the index splitting the static part of the filename from the templated part. For example, http://autoplot.org/data/C1_CP_EDI_EGD__$Y$m$d_V$v.cef is split into:

http://autoplot.org/data/
C1_CP_EDI_EGD__$Y$m$d_V$v.cef

http://emfisis.physics.uiowa.edu/Flight/RBSP-A/Quick-Look/
$Y/$m/$d/rbsp-a_magnetometer_4sec-gsm_emfisis-Quick-Look_$Y$m$d_v$(v,sep).cdf

This new version uses regexs and is more complete than versions found in Autoplot, and they should eventually use this instead. Note the Autoplot one returns the index of the last /, whereas this returns that index plus one.

Taken from Autoplot's AggregatingDataSourceFactory, where Autoplot just has a URI and needs to get a file list. See also org/autoplot/pngwalk/WalkUtil.java splitIndex, which also allows wildcards like *.

Parameters

Returns:

toString

Returns: