The appearance of the anti-persistence effect and the refined baseline hardware plan make it reasonable to expand the raw data usage for flattening computations, and the plan has evolved to near complete usage. Part of the puzzle involves the calibration scans, which are currently expected to be one degree long and composed of 42 frames. It is expected that flattening solutions for calibration scans may have to be borrowed for survey scans through extremely dense regions such as the galactic plane, and the use of two 21-frame stacks does not appear to be an advisable approach for flattening calibration scans. Since only one stack seems feasible for these scans, it appears desirable to be able to use a single 42-frame stack. This would simplify the borrowing for survey scans as well as apply maximum information for the calibration scans themselves.
DFLAT uses a PARAMETER statement to dimension the stack arrays, so it should be easy to expand the maximum stack size from 25 to 42. This will make the program larger; it has been a 16 MB program and would become a 25 MB program. This seems acceptable, however, given the advantages expected and the large amount of memory anticipated for the production machines.
Examination of some cases of anti-persistence has revealed that it is a more subtle effect than had been appreciated. The eye's high efficiency at spotting artifacts in the coadded images may not be shared by a method based on deviations from linearity without a lot of false triggerings. This method will still be implemented by changing the DFLAT code to threshold in units of the trimmed-average standard deviation and apply the corrective action to the sky offset correction frames rather than the mask, but in addition it appears that the persistence contamination may be attacked more effectively by using the 42-frame stacks on the survey scans as well as the calibration scans (also thereby eliminating the temptation to prepare two versions of DFLAT).
A 260-frame scan can be covered by six 42-frame stacks with only eight frames being left out. The 42-frame stacks can use trimming of the outer 12 pixels on each end to eliminate any persistence contamination, leaving 18 frames for the trimmed averaging (more than in the protopipeline, hence lower flattening noise). The linearity-deviation filter can probably be thresholded at a higher level to catch any problems that get past the highly trimmed averaging.
Some final discussion of this subject involved the possibility of identifying persistence contamination in the stack by its appearance in the sorted pixel array. In order to trim-average, the 42 values for an unmasked pixel in the stack will be sorted, then the high and low values will be dis carded before the rest are averaged. With persistence contamination present, a recognizable signature is almost certain to be present in the sorted array. Some use could possibly be made of this information. Meanwhile, the approach will be to expect the larger stack size and more extensive trimming to solve the problem. This will be the baseline approach in the next version of the PIXCAL/DFLAT SDS, and an integrated processor will be delivered in the near term for testing with scans known to contain persistence problems.