Data Reduction Recipe for an LWS02 AOT.

The case of a faint source.

Steve Lord & Sergio Molinari


0. About this recipe...

What this recipe is: What this recipe is not:

1. Definitions & Requirements

2. Inspecting your Data

A preliminary inspection of your data is useful to get a feeling of what you have in your hands, and to spot potential problems you will have to take care of during the data reduction. Let's start by entering ISAP:

3. Reprocessing your Data with LIA

Let's start this Section with a recommendation: it is always advisable to have a go with LIA, even if the preliminary analysis with ISAP did not show any particular problem. It is good to have a look at the dark current and at the raw data to check that everything is all right; a trend of decreasing dark current, e.g., may reflect in an increased scatter of your grating scans (i.e. an increased noise of your averaged spectrum). Besides, the comparison of your data with the Dark Currents will tell you if you are detecting signal or you just reached the sensitivity limit of the instrument.

There are 4 steps in the LIA reprocessing of an LWS02 AOT:

3.1 Dark Currents

The tool to be used is IA_DARK. A full tutorial describing its functionality is accessible at; in the following, it is assumed that you are familiar with that document (or at least have it at hand).

While at the ISAP> prompt, type: IA_DARK, tdt='TDT', where TDT is the eight digit number attached to the filename of all you data files. If you are running ISAP in a directory which is different from the one where your data files are stored, an additional parameter has to be given; the call would then be: IA_DARK,tdt='TDT',dir='your directory'. Once the widget is up, click on detector  such as LW1. You will see something like this:

Fig. 5

The red crosses are the DC measurements, while the white points are your (uncalibrated) data; everything is plotted as a function of time. If your observations was taken at a revolution number earlier than about 400 you may see DC measurements (red crosses) taken in between the observation; however only the first and the last (before and after your observation) DC measurements can actually be used.

Here the various scans of the grating can be recognized as an oscillating pattern on your data.  The discontinuities in the oscillation segments above represent the different "lines" scanned in the execution of the LWS02 AOT. Remember that this data is still uncalibrated at this stage and the transmission profiles (called RSRF, 1 with the appropriate RSRF segment or each line/detector combination) of the instrument are still to be divided out. To check where the different lines and cans start and end, go with the mouse on a point and click the middle button of the mouse: the text area at the bottom of the IA_DARK widget will give you all the information regarding the nearest point to the mouse actual position, e.g.:

Raster ID is 1 1 - Line # 0 - Scan # 0 - ITK = 228327855 - Phc = 8.64105e-16 Amps

Again, it is important to remember that at this stage the instrumental transmission is not yet divided out and, if there is sufficient flux from your source, it will manifest itself as a periodic pattern throughout your dataset.  It is important to notice that that glitches are present also during the dark current measurements; if you bring up the second  DC measurement you will see this:

Fig. 6

You will note the  gap between the first and second point here - a glitch occurred  and was removed  possibly leaving a trailing ramp.   The median-clipped averaged performed on the whole set of points above gives the DC estimated made by the OLP, which is visible in the box 'OLP Used...' in Fig. 6; it is clear however that this may be an overestimate because at least the point after  the gap is artificially high.  (This is a fairly mild example of this effect. )   It seems that the best estimate should be based on all but the second point, which we mask out and go ahead and make the DC estimate without it.

When you come to the subtraction of your estimated DCs from the data, a lot of common sense is needed. Take the example in Fig. 7.

Fig. 7

The problem here is that the source flux  (white)  is weaker than the first dark current measurement (first set of red points, with the mean shown as a red line). Yet the source flux includes dark current. This indicates that the first dark current measurement is measuring more dark current than is present during the observation and is not reliable. One possible choice is to rely solely on the second dark, which is weaker than the source flux (red points at the bottom) or else give up on making the correct dark subtraction and instead  remove all the dark current and continuum by  interpolating a "dark" function following the lower envelope of your data.  However, note here how the glitches  (SW1 has the worst) are altering the detector gain in time and causing steep falloffs. Compounding the problems here is the fact that their are  discontinuities caused by switching to  the different line wavelengths. For multi-line LWS02 observations, when dark measurements are unreliable,  systematic changes in dark current and gain are difficult to untangle from intrinsic line and continuum fluxes associated with the successively observed lines.

Likewise, with an LWS02  Raster Map ,  signal variation  is bound to occur during the observation due to the source's spatial structure, and between the starting and ending dark observations, it is practically impossible to isolate a systematic trend in the instrument behaviour .

Consider the ia_dark result for the LW1 detector for the same observation...

Fig. 7a

You should by now be convinced that the oscillations you see in the signal (white points) in Fig. 7a is due to spectral scanning back and forth over the line region interval and is dominated by the detector's relative spectral response  (which has not yet been divided-out) - and the oscillation is seen here in 5 different  segments, one for each line observed. The oscillations will disappear when we process our spectrum to the AAR stage. The best (and most conservative) thing you want to do at this stage is to assume that the DC is constant through out the observation. In general, if the DC measurements done before and after the observation are comparable (as with the example above), within the noise of the single measurements, we recommend you choose the linear interpolation option which uses the average of the two measurements. This is the option applied by the OLP, and unless the two measurements are significantly different or there is a clear DC trend in your data, you should stick with that option.

 In general, the advanced options of ia_dark; the option to add points and to fit with polynomial functions, are not advised for LWS02.... typically, there is insufficient data to be able to decide to take such actions. The exceptional case would be where a line was observed by scanning so many times that an instrumental trend was visible.  In such a case, the user is advised to distinguish between
dark variations, which elevate or lower the mean signal strength, leaving peak to peak variations constant, and responsively trends, which change the span of peak to peak variations range and cause line fluxes obtained in individual scans to change. If your data are of sufficient quality (i.e., you can see the effect described below) then here are the corrective actions:

3.2 Responsivity Drifts

While there may be responsively drifts within the period of time that an LWS02 observation is taken, at this time there is no way to isolate and correct these drifts. However, absolute responsively changes  (a constant detector gain change from the nominal) may be measured and corrected as described below.

3.3 Absolute Responsivity Correction Factors
The absolute responsively correction factors can revised using the routine IA_ABSCORR, whose tutorial is available at Since the estimate of these factors is based on the Illuminator Flash data, the characteristics of the source observed are not important; the tutorial has all the information you need to proceed.

3.4 Recalibration

The recalibration of your data is independent on the type of source observed. Instructions to recalibrate your data are found in

4. Data Analysis with ISAP

At this time, you should have in your directory a file called LSANxxxxxxxx_SHORT.FITS produced by SHORT_AAL; load this file into ISAP. You can do a straight average and compare the result with the averaged spectrum for the original LSAN file produced by the OLP. Once you are done, we can start with the real work to analyze this data.