Interactive Dark Current Subtraction
This tool can be used indifferently with all LWS AOTs. Its use is critical
to assess and improve the quality of the data. Fresh users must ask
advise from instrument experts at RAL or IPAC.
Invoking the Interactive Tool
The command to be given to start this tool and a description of optional
keywords are given below:
; Example ia_dark,dir='/user/iso/data/',tdt='44700574',/update
; Arguments Name I/O
string the tdt number associated with the
observation. For example if the tdt
number is 44700574 then the routine will look for data files l*44700574.fits
If not specified the routine assumes that files to be read are called
l*.fit (viz the SOC processed files from the CD_ROM).
I string the directory from which
the input files are to be read (and where output
files will be written). It can be a relative or absolute path. e.g. dir='data/'.
If not specified then the data are read from the user's curent working
keyword If set, the routine reads the MC-FITS format data files
ARC_DAT directory. Only expected to be used by the instrument team.
/update I keyword
If set, it is assumed that the user will update a set of previously
determined dark currents, and resubtract them from the original SPD data.
no longer mandatory (starting from version 2.1 of this routine, delivered
7.3) to re-analyse all 10 detectors.
The following Widget will pop-up (the plot area will be empty; button
'LW3' has been hit in the example below):
A Description of the User Interface
The black area is the main plot window,
with its control buttons (Replot, Unzoom, Hardcopy) on its right side.
In the example above the 'SW3' button has been hit (see below) and the
data (not yet calibrated either in flux or wavelength) collected in the
AOT for detector LW3 are shown as a function of time (ITK)
in Log-Log scale; the user can zoom in by clicking and dragging using the
left button of the mouse. The data are plotted as a function of time and
the Y-axis is linear, and they refer to both internal calibration
and scientific data; in particular:
A smaller plot area is located on the top-right of the main window. This
area is not interactive, and presents the full (calibration + scientific)
data set plotted with Y-axis in logarithmic scale and as a function
of time in seconds (so you can have an idea of the duration of your observation).
While the appearance of the main plot area will change as you go along
with the processing, this smaller plot will always stay there. In addition
to the data types shown in the main plot area (and described above), the
smaller plot also show as green points the
data taken while the Illuminators were switched-on.
two sets of red points: this is the dark
current measurement done at the beginning of the AOT; remember
that the second set of red points IS NOT GOING TO BE USED in the
processing as the detectors at this stage are still recovering (check that
the signal is decreasing with time) due to the strong signal received from
another set of red points may appear if the
Flashes have been done in the OLD
scheme. This set is not used in the processing (see previous point).
a set of white points: these are the actual
scientific data. In the case shown above they come from an L01 AOT ; the
time series of scans accumulated is clearly discernible. Remember that
at this stage the data is not yet calibrated so each scan resembles the
instrument transmission profile (referred to as Relative
Spectral Response Function - RSRF) for that detector.
another two sets of red points: this is the
measurement done at the end of the AOT (see point 1).
a final set of red points (see point 2).
Below the plot area the following groups of buttons are visible from
left to right:
At the bottom of the Widget there is a text area where messages
are prompted to the user about actions to be performed and displaying information.
Information about the properties of each data point is also reported in
this message window; just click on the main plot area with central button
of the mouse, and the characteristics of the nearest data point will be
reported: time, photocurrent, raster point ID, detector number, line number
and scan number.
The 10 LWS detectors buttons
buttons: there is 1 button for each dark measurement performed in the AOT.
Aside each of these buttons there is a text field which will show the estimated
DC value (see below), and where numbers cna be inserted directly by hand
(remember to hit Return); further on the right there are other text fields
(again, 1 for each DC measurement) showing the DC values estimated automatically
by the OLP.
Masking buttons: pressing one of them make
the right button of the mouse active (click and drag) to select portions
of the data to be masked/unmasked. In the first case data within the selected
area will turn to blue and will be ignored in further processing; in the
second case data previously masked are brought back to white colour and
become again available for processing.
Operations buttons: a series of buttons performing actions which
will be explained in detaile in the following and providing options for
the temporal interpolation of the DC and its subtraction from the data.
The DC estimate & subtraction Cookbook
The following is the list of the steps that should be followed for the
DC estimate & subtraction.
Select one detector by clicking on one of the detector buttons; a plot
similar to the one shown in the example above should appear.
Select the 'Dark # n' button (likely starting from the first): the
button is a 'pull down' menu and two possibilities are available and should
be selected in sequence:
'Zoom Flash' will cause to enlarge the region of the DC
measurement selected; the plot is shown in linear scales to be able
to include also the negative points. The points that will be averaged to
produce the DC estimate are the red ones at the extreme left of the plot,
so the user will want to zoom more on that region using the left button
of the mouse. It is time now to check that bad points (such as 0s or
clear outlier points like glitch remnants) are not included in the DC estimate;
to get rid of those points use the button in the 'Masking
Options' section. Some thing like this should be seen:
'Estimate Dark' will perform a n-sigma clipped median averaging
of the individual measurements (red points), print,the resulting DC photocurrent
in the area left of the 'Dark # n' button, in the text area at the
bottom of the widget ('greyed' at this stage) and on the terminal window.
A white dashed line is overplotted representing the averaged value and
a small widget pops-up asking how to proceed further:
'I am happy with that' accept that value: click it if you are happy
with the result.
'I want to change clip level': you are not satisfied with the estimate
and want to try again the n-sigma median averaging but with a different
Just input a new n value in the small text field and click the button:
the average is soon recomputed and the new value overplotted.
'Fix it by hand': there is no way of getting what you think is the
correct value (use a grain of salt and a lot of common sense) via averaging
so you want to set a particular value by hand. Click that button and then
click again on the plot with the right button of the mouse just
at the Y position you think is correct.
'Do not accept' if you are not sure how to proceed and you want
to think a little bit about it.
The user can also insert his own estimate of dark current by hand directly
in the relevant text field under 'Dark Measures' ; remember to
hit 'return' button
Everything done till now should be repeated for each DC
measurement (each 'Dark #n' button) and for each detector. In
some cases the user may want, for a particular detector and DC measurement,
to adopt the DC value that OLP has automatically
estimated; this can be done by clicking on the 'left arrow' button in the
'Dark Measures' section (see below)
Note that apart from the 'Quit' button, all other buttons in
the 'Operations' area will stay inactive until DC values have been estimated
for each dark measure
and for each detector. When 'Operations' buttons are active, it is time
to do the temporal interpolation, for each detector, of the previously
determined DC values to determine the DC for each 'science' data point.
Click on detector button to select a particular detector (you have to do
it for all detectors): the complete data time series is shown along with
adopted DC values in the proper text fields.
There are two ways the user can interpolate between estimated DC values:
linear interpolation or polynomial fit.
'Linear interpolation' button is a pull-down menu where the user
can select among the four options described below
'Use all darks' will use all DC values estimated;
'Use first dark only' will assume that dark current is costant throughout
the whole AOT and equal to the first estimated DC value;
'Use last dark only' will assume that dark current is costant throughout
the whole AOT and equal to the last estimated DC value;
'Use average of darks' will assume that dark current is costant
throughout the whole AOT and equal to the arithmetic average of all estimated
DC values (which is what is automatically done by OLP);
the user is strongly encouraged to use this option unless there are good
reasons to think the DC is varying throughout the AOT.
'Polynomial Fit' button is a pull-down menu where the user
can select 1st, 2nd or 3rd order polynomial
fit to all estimated DC values.
An option that has to be used with lot of caution is the
possibility of 'adding points' to help fancy DC interpolation if
the user has serious reasons to believe that the dark current is doing
strange things during the AOT. Clicking on 'Add Points' enables
the user to set an additional point on the plot by clicking with the right
button of the mouse (a red cross appears); if the user is not satisfied
with the points added he can get rid of them by clicking on 'Clear extra
points' . Various attempts can be done and they all will be overplotted:
When the user is happy with the interpolated dark current hit 'Subtract!'
and the DC-subtracted data will be overplotted in yellow
(multiple subtract actions will have no effect); note that the last interpolation
done is subtracted, although many attempts may still be visible on the
plot. Hitting 'Subtract' makes no harm as you can always make another interpolation
and subtract again: the 'Subtract' button always acts on the original data
(but it does not overwrites them).
Note: in case of a faint source or of a
Fabry-Perot observation, the DC values will be comparable to the source
signal and, as a result, the DC-subtracted spectrum (yellow
points) will be very faint and possibly off-scale: use the 'Zoom'
button to increase the dynamical range of the Y axis so that the yellow
points can be seen.
When the DC is subtracted, a label will appear on the detector button
pertinent to the data under analysis ('LW3' in the present case) to remind
the user that DC has been subtracted for that detector (something which
does not prevent the user to do it again, see above); the label is different,
according to the type of dark current that has been interpolated, according
to the following table:
|Type of Interpolation Performed
|Linear, using all Darks (including extra points, if any)
|Linear, using first dark only (i.e. DC constant and equal to
the first measured value)
|Linear, using last dark only (i.e. DC constant and equal to
the last measured value)
|Linear, using the average of first and last darks (i.e. DC constant
and equal to the arithmetic average of first and last measured dark currents)
|1st order Polynomial interpolation (using all dark current values,
including extra points, if any)
|2nd order Polynomial interpolation (using all dark current values,
including extra points, if any)
|3rd order Polynomial interpolation (using all dark current values,
including extra points, if any)
When dark current has been interpolated and subtracted for all 10 detectors
the user click 'Write new LSPD and Exit'. A new LSPD file is written
on the disk (same location of the input files as specified by the 'dir'
variable in the calling command) with a '_newdarks' appended. The user
can go on for further processing.
Document by: Sergio Molinari, IPAC/Caltech
Last Update: Jan 19, 1999