\documentstyle[11pt]{article}

\def\mus{$\mu$s }

\def\today{\ifcase\month\or
  January\or February \or March\or April\or May\or June \or July\or
  August\or September\or October\or November\or December\fi
 \space\number\day\space\number\year}


\setlength{\oddsidemargin}{0in}		%  1 inch (0 + 1) from left.
\setlength{\topmargin}{-0.5in}		%  1 inch (1.5 - 0.5) from top.
\setlength{\textwidth}{6.5in}		%  1 inch (8.5 - 1 - 6.5) from right.
\setlength{\textheight}{9in}		%  1 inch (11 - 1 - 9) from bottom.

\addtolength{\parskip}{0.5\baselineskip}

%  The environment centerpage centers text vertically on the page.  This is
%  useful within a titlepage environment.

\newenvironment{centerpage}{\mbox{} \protect\vspace*{\fill}}{
	\protect\vspace*{\fill} \mbox{} \protect\\ \mbox{}}

\begin{document}		
\bibliographystyle{plain}		% Define bibliography style.

\begin{titlepage}
\hrule
{\noindent\bf CEPPAD experiment}\\
\vspace{-0.7\baselineskip}\\
{\noindent\Huge\bf POLAR}
\vspace{2mm}
\hrule
\vspace{3mm}
\centerline{\bf RAL CEPPAD SOFTWARE NOTE No. 1}
\vspace{3mm}
\hrule
\noindent Version 2.0 \hfill Date 4 November 1998
\vspace{2mm}
\hrule

\begin{centerpage}

\begin{center}
{\Large\bf RAL CEPPAD display software}\\
\mbox{}\\
Martin Carter\\
RAL\\
\mbox{}\\
carter@cds4.bnsc.rl.ac.uk
\end{center}

\end{centerpage}

\end{titlepage}

\section{General}

\subsection{Revisions}

\begin{verbatim}

Version 1.00: Removed interpolation option within SW. Changed resample algorithm for CLEAN
15/5/98       IES data so that preserves counts in top energy channel.
              Added spin average tp DETECTOR vs SECTOR format.
              Added instrument type to get file to eliminate problem with 
              PICKFILE not dealing properly with directories other than 
              initial directory.

Version 2.00: Added Pitch Angle calculation to SURVEY plot tools and to analysis tool.
04/11/98      Added SUN and EARTH mask generation for IPS.
              Added sinusoidal map projection for ANGLE/ANGLE plot with NORTH aligned
              with magnetic field direction.
              Abandoned use of intermediate ephemeris files. Now uses the OR CDF files directly.
              Also requires AT CDF files.
              Also requires magnetic field file. Generates a MF file from the 10 minute magel, az
              data in HK file. Also checks for MFE CDF file.
     
\end{verbatim}

\subsection {Summary and Purpose}

This document describes the current implementation of 
the RAL CEPPAD display software and serves as a programmer's guide. 
It details what tasks are performed by the software and
how the tasks are implemented.

\section{Introduction}

The RAL CEPPAD display software is intended for display and plot generation of CEPPAD normal mode data. The software  allows display and SURVEY plot generation of the IES, IPS, HIST instruments and some HK data.
RATES mode data are not displayed.

Level 0 files containing raw CEPPAD telemetry are produced on a UT daily basis.
IDL software for the extraction of the data from a level 0 file and production of level 1 files has been
produced by Aerospace : make\_level\_1 (Jim Roedder/Mike Redding). The level 1 product contains uncalibrated, compressed data organized into
a time tagged, spin ordered format. 
IDL software to read the level 1 files and store the data in IDL COMMON blocks has also been produced by 
Aerospace : get\_cep\_data. This COMMON block data is subsequently used by RAL CEPPAD software to create an intermediate format data file.
The RAL display software runs off the intermediate format data files.

The intermediate data format reduces the contents of the COMMON
block data to single manageable units spanning a complete or part of a UT day.  
The intermediate format contains decompressed raw data together with time information, calibration data and data quality 
information.

The display software supports concatenation of data from different days and display of calibrated or uncalibrated data.
The present software supports :
\begin{itemize}

\item
2-D image display of energy against time for different sectors and detectors i.e spectrograms; 

\item
lineplot display of energy against time for different sectors and detectors; 

\item
azimuth counts against time for different detectors i.e roll plots; 

\item
polar/azimuth plots for a given time for each energy band (either polar/azimuth projection or sinusoidal projection);

\item
polar/azimuth plots for a number of times for a single, integral energy band (either polar/azimuth projection or sinusoidal projection).

\item
pitch angle against time plots for each energy band.

\end{itemize}


\section{CEPPAD IES data}

The IES experiment has 9 detectors arranged in a 180 degree polar fan
with 15 differential 
energy channels and 1 integral energy channel and up to 32 azimuthal sectors 
per 6 second spin. The data for a given detector/energy channel combination may be 
summed over a binary number of sectors within a spin and over a binary number of spins up to 
16 spins. The data within a 16 spin block is complete. The organization of the summation over 
sectors and spins is controlled by an Accumulation/Telemetry (TM) table.

The particular ADC bins chosen for each energy channel are defined by lookup tables
for each detector. A number of different sets of lookup tables are defined. The lookup tables are specific to a particular mode of operation of the IES.
The IES may use 4 different integration times : 2 us, 10 us, 50 us or 100 us, and has 2 different pixel sizes
that may be used : small and large. The small pixel gives a geometric factor approximately 19.13 times smaller than the 
large pixel. The different modes result in different calibration factors to the data to convert to flux.

These tables have already been modified. The software takes into account the times at which modifications to the telemetry tables
take place.


\section { CEPPAD Level 0 to Level 1 data conversion. }

The software for the level 0 to level 1 conversion is as supplied by AEROSPACE (vs 2.04) with some minor alterations and bug fixes.


\subsection{CEPPAD Level 0 product.}

The CEPPAD level 0 data contains data from the four experiments : IPS, IES, HITSp and HISTe. 
It is blocked into UT daily periods and typically contains about 45 MBytes of data.

\subsection{CEPPAD Level 1 product.}

The level 1 product contains uncalibrated, compressed data organized into
time tagged, spin ordered format. The various level 1 data are separated into 
up to nine types of level 1 data files. 

The normal mode data are split into a new file whenever the TM table or mode changes.
The new file is indicated by the incrementing the value of \_vv in the file name.

The level 1 files relevant to the CEPPAD IES data are the following :

\begin{itemize}

\item CEPPAD housekeeping level 1 file;

\item CEPPAD spin data header level 1 file;

\item CEPPAD normal mode IES data level 1 file;

\item CEPPAD IES rates-histogram level 1 file;

\end{itemize}

The normal mode IES level 1 files contain a number of logical records.

The first record is a header record (HR). 
This record has all the values ( Accumulation/Telemetry table ) used 
for calculating the mapping parameters. 

Each subsequent record contains data accumulated over a unique set of 16 spins.
Within each record the data is ordered first by spin time, that is, 
all data whose acqusition started on a particular spin are assigned 
the time for that spin.  The second level of ordering is energy.  
The third  level of  ordering is channel number. Finally, the last 
level of ordering  is by sector.  Each pixel ( channel/energy range 
combination) has a defined  number of sectors per spin.  Definitions 
of sectors per spin for each pixel will be found in Accumulation/Telemetry
format tables.  

\subsection{ CEPPAD level 1 normal mode IDL COMMON block data format}

The IDL routine 'get\_cep\_data' reads the specified level 1 file and returns 
the data contained through IDL COMMON blocks.


\section { RAL CEPPAD intermediate data format }

The CEPPAD normal mode data stored in the IDL COMMON blocks is obtained from a single 
normal mode file for each instrument. A new normal mode file is created every time the TM table changes or the 
mode changes. Thus the data for a single UT day may be split into multiple normal mode files 
and require multiple calls to the IDL routine 'get\_cep\_data'.

The size of the data for a single UT day depends on the TM tables used and the mode but 
typically corresponds to a data rate of 1 kbyte/spin or 14 MByte / day. When the data is 
decompressed into 32 bit words this will be increased by a factor 4.

The raw data for different detector/energy combinations are accumulated over different number of sectors and
spins giving different azimuthal angular resolution and time resolution and a complicated data format whose 
size depends on the TM tables.
Typically the integral energy channel will have high time and azimuthal resolution.

In order to simplify and reduce the data to more manageable sizes several intermediate data formats are defined for the
RAL display software which meet the needs of the various analyses required. These average or duplicate the raw data to 
give a fixed sampling for all detectors and energies. The different intermediate data formats allow averaging over 
different numbers of sectors and different numbers of spins.
The data is calibrated at this stage.

Data correction may also be applied to the data at this stage. Formats containing the string 'CLEAN'
have data correction applied. The processing performed is described below.

These formats are :

\begin{enumerate}

\item
'INTEGRAL\_ENERGY' : integral energy counts per spin per sector : \hfill\break
This is a fixed size array of counts for each detector/sector combination for the integral energy channel for each spin.
Where the raw data is accumulated over several sectors or several spins the data is interpolated.
For the IES this has $ 9 \times 1 \times 32 \times 14400= 4$ MSamples per day.

\item
'FULL\_RESOLUTION' : energy counts per sector : \hfill\break
This is a fixed size array of counts for each detector/energy combination for each sector.
The times for the sectors are obtained by interpolating spin times. 
Where the raw data is accumulated over several sectors or several spins the data is interpolated and 
results in some interleaving of the data.
For the IES this has $ 9 \times16 \times 32 \times 14400=64$ MSamples per day.

\item
'SURVEY' : all detectors, all energies, sectors averaged to give 16 sector values, averaged over 16 spins. 
Contains 24 hours of data. Data are calibrated to gives flux/nsectors so that summing over sectors gives flux directly.
For the IES this has $ 9 \times16 \times 16 \times 900=2$ MSamples per day.

\item
'CLEAN\_SURVEY' : corrected version of SURVEY (suitable only for IES data). All detectors, 10 resampled energies, 
sectors averaged to give 16 sector values, averaged over 16 spins. 
Contains 24 hours of data. Data are calibrated to gives flux/nsectors so that summing over sectors gives flux directly.
This gives $ 9 \times10 \times 16 \times 900=1.3$ MSamples per day.

\item
'SPIN\_n' : all detectors, all energies, sectors averaged to give n sector values for each spin.
For the IES this has 2n MSamples per day.

\item
'CLEAN\_n' : calibrated and corrected version of the above (suitable only for IES data). 
All detectors, 10 resampled energies, 
sectors averaged to give n sector values for each spin. 
This gives 1.3n MSamples per day.

\item
'SPIN\_nZm' : all detectors, all energies, sectors averaged to give n sector values, averaged over m spins :
For the IES this has $\rm 2n\over m$ MSamples per day.

\item
'CLEAN\_nZm' : calibrated and corrected version of the above (suitable only for IES data). 

\end{enumerate}

Note that the  MAJOR\_SPIN\_AVERAGED and INTEGRAL\_ENERGY products retain information about the
azimuthal angle ( FULL\_RESOLUTION retains the counts per sector but not the sector number). The values n and m in the 
above must
 be powers of two.
The size of output for the other CEPPAD instruments depends upon the number of detectors and the number of energies for
these instruments.

The above formats are ignored for HK intermediate format files.

The intermediate format data is stored in separate files for each UT day and for each CEPPAD instrument.
The intermediate format consists of a header structure and a data structure array. The two structures are
stored in IDL XDR files using the IDL SAVE routine and are accessed using the IDL RESTORE routine.
The structure descriptions are given in Appendix A. The variables stored in the SAVEd files are input\_header and
input\_data.


\section{Orbit and attitude data}

CDF files containing orbit and attitude data are distributed with the level zero data product on CDROM. The intermediate format file creation software
copies these files for later use at the same time as performing the level zero to level one conversion. The display software retrieves these files as required. 

Orbit files obtained from CDROM are stored in :
GETENV('PO\_CEP\_OR\_DIR') +'/pre' and '/def', and have names yymmddnn.cdf .

Orbit files obtained from the SPOF are stored in : GETENV('PO\_CEP\_OR\_DIR') +'/istp', and have names like
 po\_or\_pre\_yyyymmdd\_vnn.cdf .
 

\section{HK data}

HK level 1 files may be converted to intermediate format HK data files using cep\_l1toif. Only the IES temperature data is stored at present.
The data may be displayed using ies\_plot (LINEPLOT display with LOG button off is suggested).

A separate MF file containing magnetic field data from the HK file may also be created.
\section {Setting up the software environment}

\section {Setting up the software environment}

The RAL CEPPAD format conversion and display routines require various environment variables to be set 
up which indicate the location of level 0, level 1 and RAL intermediate format  files {\it etc}. 
An examples of how to set these up is contained within the file env.pro. This IDL routine will need modification 
to point to the correct directories on the host machine. The environment variables may equally well be set up
outside of IDL.

Note that the full directory path is required in the environment variable definitions.

Note that on VMS machines the setenv routine should be replaced by the setlog routine.

To set up the environment variables needed from an IDL session the following command should be entered :

\begin{verbatim}

IDL> @env ; set up environment variables in IDL session and compile software

\end{verbatim}


 
\section { RAL CEPPAD intermediate data format conversion software.}

The IDL routines for conversion of CEPPAD level 0 to level 1 to RAL intermediate formats and generation of survey plots
are packaged in a single routine
cep\_survey. 

The routine cep\_survey may be called directly for example :

\begin{verbatim}

IDL> PRINT, cep_survey()

\end{verbatim}

may be used to generate the survey plot in a window from the level 0 file.


\section { RAL display software }

The RAL display software uses the RAL intermediate data formats described above.
Various display routines may be called within a widget environment which supports 
extraction and calibration of the data. 

Invoke the display software using :

\begin{verbatim}

IDL> ies_plot         

\end{verbatim}

The software prompts for the RAL intermediate format file name. Note the RAL intermediate format filename
uses day of year as label. The SELECT DATA TOOL widget will appear.


\subsection{RAL SELECT DATA TOOL}
 
 This allows selection of the RAL intermediate data format files to be displayed; the display routine; 
 the energy bands to be used; the detectors to be used and period of interest to be displayed.

The display routines consists of the following  :

\begin{itemize}

\item
SPECTRAL : 2-D images of selected energy bands vs time for selected detectors or sectors for single energy band (last band)
 vs time for selected detectors. 

\item
LINEPLOT : Line plot of selected energy bands vs time for selected detectors or sectors for single energy band (last band)
 vs time for selected detectors.

\item
ANGLE/ANGLE : Detector vs sector plots for given time for selected energy bands.

\item
TIMELINE : Detector vs sector plots for selected times for single energy band (last band).

\end{itemize}

The format of the data displayed by SPECTRAL or LINEPLOT may be selected from :

\begin{itemize}

\item
ENERGY vs DETECTOR : Displays selected energy bands for each detector summed over sectors against time. 

\item
ENERGY vs SECTOR : Displays selected energy bands for each sector (up to a maximum of eight) for a single detector (specified by FULL DETECTOR) against time. 

\item
PITCH ANGLE vs ENERGY: Displays pitch angle against time for each energy band.

\item
SECTOR vs DETECTOR : Displays sectors for integral energy band for selected detectors against time. 

\item
DETECTOR vs SECTOR : Displays selected detectors for integral energy band for 5 sectors starting with FULL SECTOR against time. 

\end{itemize}

The integral energy band is the last energy band defined in the select DATA tool.

A description of each element of the SELECT DATA TOOL widget follows :

\begin{itemize}

\item
CCLRC logo : The text widget is used for information display and error messages. 
The logo gives the program version number.

\item
DONE : Data selection completed. Go plot data.

\item
ADD FILE : Combine another data file with data already loaded. This may be used to produce plots 
spanning more than one UT day. Note the new data file must be of the same type as the data already loaded.
Note data saved from within the DISPLAY TOOL cannot be added to as an anonymous structure is used for 
processed data.

\item
NEW FILE : Select an intermediate format data file for loading. Previously loaded data is lost.

\item
QUIT : Quit to IDL.

\item
DATA FILES : Data files currently loaded. For information only.

\item
DATA TYPE : Data type field from the header structure. For information only.

\item
DISPLAY PROCEDURE : The display procedure and output format currently selected. 

\item 
CHANGE DISPLAY : A pull down menu allowing selection of the display procedure and output format.

\item 
NO. OF POINTS etc : Fields from the header structure. For information only.

\item 
SELECT ENERGY BANDS : Allows selection of energy bands to be used.
Display also gives, for information, either the NUMBER OF ENERGY BANDS SELECTED or the TOP and BOTTOM channels 
depending on whether 0 bands selected.
From the SELECT BANDS TOOL : entering values in the NO. OF ENERGY BANDS box and hitting RETURN changes 
the no. of different energy bands for display. If equal to zero
then the energy bands default to the energy channels else the program  prompts the user to select the channels for each energy band.
For SPECTRAL the number of bands defaults to zero.
Note that choosing too many bands for selection may confuse the widget display.
If the number of bands is zero then the user is offered a
choice of the bottom and top channels for display. The default top channel excludes the integral
channel unless the data has been CLEANed.

\item
TIME INTERVAL : Allows selection of the time interval for data processing and display.
The fields within this are self explanatory.
Note that the time can be altered directly in the text widget.

\item
FULL WINDOW DETECTOR : In the DISPLAY TOOL one detector is singled out for the full range display.
This allows selection of the detector used. This also determines the detector used in the ENERGY vs SECTOR display.
If the FULL WINDOW ONLY option is off then the FULL WINDOW DETECTOR must be one of the selected detectors.

\item
FULL WINDOW SECTOR : This determines the sector used for the full window in the ENERGY vs SECTOR and the DETECTOR vs SECTOR displays. This also
determines the start sector used in the DETECTOR vs SECTOR display. 

\item
FULL WINDOW ONLY : Only produces the full range display. This is useful for setting up the display
initially without having to wait for all the detectors to be displayed each time something is changed.

\item
XTRA : Selects from : XTRA to display the extra L shell, M Lat, Alt and MLT axes; LUTS to display look up table in use; 
or NONE for no extra axes.

\item 
DETECTORS : Selects which detectors to use for display. Note the performance and widget display suffer
if too many detectors are used.

\end{itemize}



\subsection{RAL DISPLAY TOOL}

This displays the data from the selected detectors and energy bands and allows various display options
and various output files to be produced. 

A description of each element of the DISPLAY TOOL widget follows :

\begin{itemize}

\item
CCLRC logo : The text widget is used for information display and error messages. The logo may also be used to cancel
processing or plotting of the current data set. It may be used to skip plotting of the full window axes.

\item
DISMISS : Quit to IDL.

\item
SAVE : Save the currently selected data an intermediate format file. The data may subsequently be restored from the 
SELECT DATA TOOL.

\item
RETURN : Exits to the SELECT DATA TOOL.

\item 
COLOUR TABLE : Allows selection of the colour table used for display.

\item
PRINT : Allows a number of different outputs to be produced. Note that the current colour table will be used for output unless
BWPS selected.

\begin{itemize}

\item 
BWPS : Black and white postscript file output. Line type and colour table optimized for black and white output.

\item
PS : Colour postscript file output.

\item
GIF : Colour GIF file output.

\item
WIN : Window output. This is useful for comparing results from different plots.

\item
TXT : Text file output.

\end{itemize}

\item
LOG : Choice of linear or logarithmic display of the data.

\item
TRACK : Choice of whether to track in full window or not.

\item
CAL : Choice of whether to apply data calibration or not. Converts to /cm2/sr. This will be reversed if data already calibrated.

\item
RAT : Choice of whether to apply rate calibration or not. Converts to /sec. This will be reversed if data already given as rates.

\item
DIV : Choice of whether to divide data by energy band size or not. Converts to /keV. This will be reversed if data already divided.

\item
SUB : Choice of whether to apply pedestal subtraction or not. This will be insensitive if already applied previously or relevant data
does not exist.

\item
SHIFT : Choice of whether to apply pedestal shift and data resampling. This will be insensitive if already applied previously or relevant data
does not exist. 

\item
MIN : Minimum value for display. If negative will adjust to minimum value in all processed data (excluding dodgy data). 

\item
MAX : Maximum value for display. If negative will adjust to maximum value in all processed data (excluding dodgy data). 

\end{itemize}

Some diagnostic information is output above the axis in the widget plots. The solid line indicates the look up table used.
The points indicate end of level 1 data file ($\diamond$); bad times ($+$); bad counts ($\ast$) and bad look up table number ($\triangle$). 
The major frames positions are indicated by coloured
points along the upper plot axis.

Note that SHIFT AND SUB are applied to the current data set. In particular if the data has already been spin averaged then 
they cannot be applied to each sector before spin averaging. This will result in some differences between correcting the
data in the DISPLAY TOOL and data in a CLEAN intermediate format file under some circumstances.

\subsection{SPECTRAL display procedure}
 
Produces a data vs time 2-D plot where count rates are colour coded and 
either selected energy bands (ENERGY vs DETECTOR or ENERGY vs SECTOR), or sectors for the last energy band 
(SECTOR vs DETECTOR), or detectors for the last energy band 
(DETECTOR vs SECTOR) are displayed
along the y axis. Some additional DISPLAY TOOL buttons are present :

\begin{itemize}

\item
BINS : Show energy band sizes rather than energy band numbers. Re-plots data.

\item
NO BAD : Does not plot bad quality data.

\item
ELOG  : Gives logarithmic energy axis display.
 
\end{itemize}

\subsection{LINEPLOT display procedure}
 
Produces a line plot of data vs time. Counts are displayed along the y axis. 
The selected energy bands (ENERGY vs DETECTOR or ENERGY vs SECTOR), or sectors for the last energy band 
(SECTOR vs DETECTOR), or detectors for the last energy band 
(DETECTOR vs SECTOR) 
are individually plotted. Some additional DISPLAY TOOL buttons are present :

\begin{itemize}

\item
LINES : Show lines in plot.

\item
POINTS : Show points in plot.
 
\item
NO BAD : Does not plot bad quality data .
 
\end{itemize}

\subsection{PROFILES display procedure}
 
Produces line profile plots of spectra for the time selected by the cursor from a SPECTRAL display.
Some additional DISPLAY TOOL buttons are present :

\begin{itemize}

\item
BINS : Show energy band sizes rather than energy band numbers along x-axis (when x-axis corresponds to energy).

\item
RNG  : Average over data from current time-RNG to current time+RNG. RNG is in seconds. Only data with the same LUT number
as the current time are used.

\item
ELOG  : Gives logarithmic energy axis display. 

\end{itemize}

\subsection{ANGLE/ANGLE display procedure}
 
Displays 2-D images of detector vs sector for selected energy bands for particular time. 
This gives a polar angle vs azimuth plot when all relevant detectors are selected.
Earth (+) and sun(*) positions are plotted and B-field (large-small diamond).

Some additional DISPLAY TOOL buttons are present :

\begin{itemize}

\item
MASK : Draws EARTH and SUN mask on polar/azimuth plot.

\item
PROJ  : Draws sinusoidal map projection with NORTH aligned along B field. NB higher resolution used for
print output.

\end{itemize}


\subsection{TIMELINE display procedure}
 
Displays 2-D images of detector vs sector for last energy band for range of times.
This gives a polar angle vs azimuth plot when all relevant detectors are selected.
Earth (+) and sun(*) positions are plotted and B-field (large-small diamond).
Some additional DISPLAY TOOL buttons are present :

\begin{itemize}

\item
/2 : Displays every 2nd time sample.

\item
/4 : Displays every 4th time sample.

\item
/8 : Displays every 8th time sample.

\item
/16 : Displays every 16th time sample.

\item
MASK : Draws EARTH and SUN mask on polar/azimuth plot.

\item
PROJ  : Draws sinusoidal map projection with NORTH aligned along B field. NB higher resolution used for
print output.

\end{itemize}

Note, if more than one button is pressed then the largest value will be taken.


\section{Portability}

The software requires IDL version 5 or above.

It has been tested on DIGITAL ALPHAStations and SUN SPARCStations.

The level 0, level 1, intermediate format files and look up table data are portable between different machines.

Note that the level 0, level 1, RAL intermediate  format and look up table data files are all unformatted files.

Using the PICKFILE file select widget under unix on ALPHAstations have found a limit of about 1024 files after which 
no files are apparently found. 


\end{document}      
\end{article}
