## org.das2.fsm.FileStorageModel

Represents a method for storing data sets in a set of files by time. The client provides a regex for the files and how each group of the regex is interpreted as a time digit. The model can then be used to provide the set of files that cover a time range, etc. This new implementation uses a TimeParser object to more quickly process file names.

## cacheCleanup

cacheCleanup( ) → void

remove files that have been identified as old versions.

### Returns:

void (returns nothing)
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## containsFile

containsFile( java.io.File file ) → boolean

return true if the file came (or could come) from this FileStorageModel.

### Parameters

file - the file

### Returns:

true if the file came (or could come) from this FileStorageModel.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## containsName

containsName( String name ) → boolean

return true if the name came (or could come) from this FileStorageModel.

### Parameters

name - the name within the filesystem.

### Returns:

true if the name came (or could come) from this FileStorageModel.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## create

create( org.das2.util.filesystem.FileSystem root, String template ) → org.das2.fsm.FileStorageModel

creates a FileStorageModel for the given template, which uses:

    %Y-%m-%dT%H:%M:%S.%{milli}Z";
        %Y  4-digit year
        %m  2-digit month
        %d  2-digit day of month
        %j  3-digit day of year
        %H  2-digit Hour
        %M  2-digit Minute
        %S  2-digit second
        %v  best version by number  Also %(v,sep) for 4.3.2  or %(v,alpha)
        %{milli}  3-digit milliseconds

product_$(o,id=ftp://stevens.lanl.gov/pub/projects/rbsp/autoplot/orbits/rbspa_pp).png

### Parameters

root - FileSystem source of the files.
template - describes how filenames are constructed. This is converted to a regular expression and may contain regex elements without groups. The string may contain $ instead of percents as long as there are no percents in the string.

### Returns:

a newly-created FileStorageModel.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## create

create( org.das2.util.filesystem.FileSystem root, String template, String fieldName, org.das2.datum.TimeParser.FieldHandler fieldHandler ) → org.das2.fsm.FileStorageModel

creates a FileStorageModel for the given template, but with a custom FieldHandler and field.

### Parameters

root - FileSystem source of the files.
template - describes how filenames are constructed. This is converted to a regular expression and may contain regex elements without groups. The string may contain $ instead of percents as long as there are no percents in the string.
fieldName - custom field name
fieldHandler - TimeParser.FieldHandler to call with the field contents.

### Returns:

a newly-created FileStorageModel.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## generateNamesFor

generateNamesFor( DatumRange range ) → String[]

generate the names of the files that would cover this range. This was taken from Autoplot's org.virbo.jythonsupport.Util. TODO: versioning, etc.

### Parameters

range - the time range to cover.

### Returns:

the string names, each in the context of the filesystem.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getBestFilesFor

getBestFilesFor( DatumRange targetRange ) → java.io.File[]

return the best files found for the range, without progress feedback.

### Parameters

targetRange - a DatumRange

### Returns:

a java.io.File[]
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getBestFilesFor

getBestFilesFor( DatumRange targetRange, ProgressMonitor monitor ) → java.io.File[]

Get the files for the range, using versioning info ($v,etc).

### Parameters

targetRange - range limit, or null if no constraint used here.
monitor - the monitor

### Returns:

a list of files that can be opened.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getBestNamesFor

getBestNamesFor( DatumRange targetRange, ProgressMonitor monitor ) → String[]

return the names in the range, minding version numbers, or all available names if the range is null. This will list directories.

### Parameters

targetRange - range limit, or null.
monitor - a ProgressMonitor

### Returns:

array of names within the system.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getBestNamesFor

getBestNamesFor( DatumRange targetRange ) → String[]

return the names in the range, minding version numbers, or all available names if the range is null. This will list directories.

### Parameters

targetRange - range limit, or null.

### Returns:

array of names within the system.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getCacheTagFor

getCacheTagFor( org.das2.fsm.FileStorageModel fsm, DatumRange range, String[] names ) → org.das2.datum.CacheTag

### Parameters

fsm - a FileStorageModel
range - a DatumRange
names - a java.lang.String[]

### Returns:

org.das2.datum.CacheTag
[search for examples] [view on GitHub] [view on old javadoc] [view source]
getCacheTagFor( org.das2.fsm.FileStorageModel fsm, DatumRange range, java.io.File[] files ) → org.das2.datum.CacheTag [view source]

## getChildFileSystem

getChildFileSystem( org.das2.util.filesystem.FileSystem root, String child, ProgressMonitor monitor ) → org.das2.util.filesystem.FileSystem

return a child filesystem, with special code for LocalFileSystems to support Windows. TODO: look into .zip child.

### Parameters

root - a FileSystem
child - a String
monitor - a ProgressMonitor

### Returns:

the FileSystem

### See Also:

https://sourceforge.net/p/autoplot/bugs/2132
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getField

getField( String field, String name ) → String

return the field value for the given name. For example, if the spec is $Y/$m/$d/$Y$m$d_v$(v,sep).dat and the name matched is 2014/04/04/20140404_v2.3.dat then calling this for the field "v" would result in "2.3" This should not be used to retrieve fields that are components of the time range, such as $Y or $m.

### Parameters

field - field, for example "v"
name - name, for example 2014/04/04/20140404_v2.3.dat

### Returns:

the field value, for example, "2.3" when the spec is $Y/$m/$d/$Y$m$d_v$v.dat
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getFileFor

getFileFor( String name ) → java.io.File

download the file for the given name within the filesystem.

### Parameters

name - the name within the filesystem.

### Returns:

null or a local file which can be opened.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getFileFor

getFileFor( String name, ProgressMonitor monitor ) → java.io.File

download the file for the given name within the filesystem.

### Parameters

name - the name within the filesystem.
monitor - monitor for the download.

### Returns:

null or a local file which can be opened.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getFileSystem

getFileSystem( ) → org.das2.util.filesystem.FileSystem

return the filesystem used to implement this.

### Returns:

filesystem
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getFilenameFor

getFilenameFor( Datum start, Datum end ) → String

return a filename that would intersect the range. Note this file may not actually exist. This may be used to quantize the range. The template may not contain versions.

### Parameters

start - a Datum
end - a Datum

### Returns:

a String
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getFilesFor

getFilesFor( DatumRange targetRange ) → java.io.File[]

### Parameters

targetRange - a DatumRange

### Returns:

java.io.File[]
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getFilesFor

getFilesFor( String[] names, ProgressMonitor monitor ) → java.io.File[]

download the files for each of the given names within the filesystem.

### Parameters

names - array of names within the filesystem
monitor - monitor for the downloads (or null).

### Returns:

local files that can be opened.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getFilesFor

getFilesFor( DatumRange targetRange, ProgressMonitor monitor ) → java.io.File[]

Download the files within the range. This might catch a bad link where getNamesFor does not.

### Parameters

targetRange - range limit, or null if no constraint used here.
monitor - the monitor

### Returns:

a list of files that can be opened.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getNameFor

getNameFor( java.io.File file ) → String

Provides a way to recover the model name of a file. The returned File from getFilesFor can be anywhere, so it would be good to provide a way to get it back into a FSM name. For example, a filesystem might download the remote file to a cache directory, which is the File that is provided to the client, sometimes the client will need to recover the name of the corresponding FileObject, so this maps the File back to the name.

### Parameters

file - the file

### Returns:

the canonical name of the file.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getNamesFor

getNamesFor( DatumRange targetRange ) → String[]

return the names in the range, or all names if the range is null.

### Parameters

targetRange - range limit, or null.

### Returns:

array of names within the system.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getNamesFor

getNamesFor( DatumRange targetRange, ProgressMonitor monitor ) → String[]

return the names in the range, or all available names if the range is null. This will list directories.

### Parameters

targetRange - range limit, or null.
monitor - a ProgressMonitor

### Returns:

array of names within the system.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getParent

getParent( ) → org.das2.fsm.FileStorageModel

returns the parent or null if none exists. None will exist when there are no more wildcards. (/home/foo/$Y/$m-$d.dat has parent /home/foo/$Y which has parent null.)

### Returns:

parent or null.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getRangeFor

getRangeFor( String name ) → DatumRange

return the time range represented by this name.

### Parameters

name - like 2013-10-31

### Returns:

the timerange representing the day 2013-10-31
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getRepresentativeFile

getRepresentativeFile( ProgressMonitor monitor ) → String

return a random file from the FSM, which can be used to represent a typical file. For example, we need to look at metadata to see what is available.

### Parameters

monitor - progress monitor in case a file must be downloaded.

### Returns:

a reference to the file within the FileSystem, or null if one is not found.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getRepresentativeFile

getRepresentativeFile( ProgressMonitor monitor, String childRegex ) → String

return a random file from the FSM, which can be used to represent a typical file. For example, we need to look at metadata to see what is available.

### Parameters

monitor - progress monitor in case a file must be downloaded.
childRegex - the parent must contain a file/folder matching childRegex

### Returns:

a reference to the file within the FileSystem, or null if one is not found.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getRepresentativeFile

getRepresentativeFile( ProgressMonitor monitor, String childRegex, DatumRange range ) → String

Return a random file from the FSM, which can be used to represent a typical file. For example, we need to look at metadata to see what is available. This is introduced to support discovery, where we just need one file to get started. Before, there was code that would list all files, then use just the first one. This may return a skeleton file, but getFileFor() must return a result. This implementation does the same as getNames(), but stops after finding a file.

### Parameters

monitor - progress monitor in case a file must be downloaded.
childRegex - the parent must contain a file/folder matching childRegex, or null if there is no child.
range - hint at the range where we are looking, or null if there is not time range constraint.

### Returns:

null if no file is found
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getRepresentativeFile

getRepresentativeFile( org.das2.fsm.FileStorageModel ths, ProgressMonitor monitor, String childRegex, DatumRange range, int depth ) → String

Return a random file from the FSM, which can be used to represent a typical file. For example, we need to look at metadata of a given file to see what is available inside. This is introduced to support discovery, where we just need one file to get started. Before, there was code that would list all files, then use just the first one. This may return a skeleton file, but getFileFor() must return a result. This implementation does the same as getNames(), but stops after finding a file.

### Parameters

ths - a FileStorageModel
monitor - progress monitor in case a file must be downloaded.
childRegex - the parent must contain a file/folder matching childRegex, or null if there is no child.
range - hint at the range where we are looking.
depth - the recursion depth, useful for debugging.

### Returns:

null if no file is found
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## getRoot

getRoot( ) → String

return the root of the filesystem as a string.

### Returns:

the root of the filesystem as a string.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## hasField

hasField( String field ) → boolean

returns true if the parser has the field.

### Parameters

field - e.g. "x"

### Returns:

true if the parser has the field.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## quantize

quantize( DatumRange timeRange ) → DatumRange

return the timerange that contains the given timerange and exactly contains a set of granules. This needs to be synchronized because the timeParser.

### Parameters

timeRange - arbitrary time range

### Returns:

list of file timeranges covering file input timeRange.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## setContext

setContext( DatumRange trdr ) → void

set the datum range giving context to the files. For example, filenames are just $H$M$S.dat, and the context is "Jan 17th, 2015"

### Parameters

trdr - the context

### Returns:

void (returns nothing)
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## splitIndex

splitIndex( String surl ) → int

This returns the index splitting the static part of the filename from the templated part. For example, http://autoplot.org/data/C1_CP_EDI_EGD__$Y$m$d_V$v.cef is split into:

    http://autoplot.org/data/
    C1_CP_EDI_EGD__$Y$m$d_V$v.cef

and http://emfisis.physics.uiowa.edu/Flight/RBSP-A/Quick-Look/$Y/$m/$d/rbsp-a_magnetometer_4sec-gsm_emfisis-Quick-Look_$Y$m$d_v$(v,sep).cdf:

    http://emfisis.physics.uiowa.edu/Flight/RBSP-A/Quick-Look/
    $Y/$m/$d/rbsp-a_magnetometer_4sec-gsm_emfisis-Quick-Look_$Y$m$d_v$(v,sep).cdf

``

This new version uses regexs and is more complete than versions found in Autoplot, and they should eventually use this instead. Note the Autoplot one returns the index of the last /, whereas this returns that index plus one.

Taken from Autoplot's AggregatingDataSourceFactory, where Autoplot just has a URI and needs to get a file list. See also org/autoplot/pngwalk/WalkUtil.java splitIndex, which also allows wildcards like *.

Note, ages ago %Y was valid as well as $Y. This is no longer supported, since it is useful to have Python templates working alongside URI_Templates.

### Parameters

surl - a string like http://autoplot.org/data/C1_CP_EDI_EGD__$Y$m$d_V$v.cef

### Returns:

an integer indicating the split index, so that surl.substring(0,i) includes the slash.
[search for examples] [view on GitHub] [view on old javadoc] [view source]

## toString

toString( ) → String

### Returns:

java.lang.String
[search for examples] [view on GitHub] [view on old javadoc] [view source]