IPAC 2MASS Working Group Meeting #69 Minutes

IPAC 2MASS Working Group Meeting #69 Minutes, 8/22/95

Attendees: T. Chester, R. Cutri, T. Evans, J. Fowler, L. Fullmer, T. Jarrett, G. Kopan, B. Light, C. Lonsdale, H. McCallon, M. Moshir, S. Wheelock, J. White


  1. New Subsystem: MAPCOR
  2. Flowchart Software: da Vinci
  3. SDS Signatories
  4. Protocam Data Processing
  5. CSFlat Results
  6. New FRD and FDD
  7. POSMAN/COFF Changes


  1. New Subsystem: MAPCOR -- A splinter group meeting was held Monday 8/21/95 to determine how best to incorporate the Read1 merge, persistence and ghost removal, and aperture corrections to the PSF-fit photometry into the 2MAPPS design. The attendees were R. Cutri, J. Fowler, G. Kopan, and J. White. The design already included Read1 point source extraction in the STATS subsystem and Read1 merging in the PROPHOT subsystem. Persistence and ghost removal processing was not included in the pipeline, probably because these have traditionally been postprocessing cleanup activities or part of catalog preparation. To apply aperture corrections to the PSF-fit photometry is a recent design change and had not yet been inserted into the pipeline. The purpose of the meeting was to resolve these issues.

    The desire to remove persistence and ghosts as early in the processing as possible was tempered with the decision not actually to remove these things but to flag them. "Confused" objects, i.e., objects too close to relatively bright objects, would also be flagged. The process of flagging all three types of artifacts or damaged goods was dubbed "purging", although the actual deletions would occur downstream from the production pipeline. Flagged objects would presumably not be processed as good point sources, however, i.e., they would not be calibrated, aperture-corrected, band-merged, used for pointing reconstruction, associated with existing catalogs, etc.

    The Read1 merge function had been appended to PROPHOT simply because that was the logical point in the pipeline to do that task. Since the other tasks are also logically done at the same point, but since none of these tasks are especially functionally similar to PROPHOT's tasks, it was decided to remove the Read1 merge from PROPHOT and create a new subsystem to do these additional tasks.

    The new subsystem was named MAPCOR (Merge And Purge & APerture CORrections [where the "AP" does double duty in the acronym]). Its input is the Read1 source list from STATS (now known as FREXAS; see item 7 below) and the Read2-Read1 source list from PROPHOT. It will be invoked by the PCP subsystem to process these lists in all three bands after PIXPHOT has finished processing the scan. Its output will be source lists in three bands, where the Read1 photometry has been merged, ghosts, persistence, and "confused" objects have been "purged", and aperture corrections have been computed and applied to the PSF-fit photometry. This will be the input to the BANDMERGE subsystem.

    After this splinter meeting, another was held with T. Chester, R. Cutri, and J. Fowler, to determine a cognizant engineer candidate for MAPCOR. It became obvious that T. Evans was the logical first choice if she could accept the additional workload, because she is the cognizant engineer of the DBMAN subsystem, which depends on data structure definitions which can be made at this time by the MAPCOR cognizant engineer. In fact, DBMAN has the most drawnout schedule specifically because its design is so dependent on other subsystems' interface definitions. It is hoped that the DBMAN schedule can be compressed downstream enough to make room for MAPCOR design in the shorter term; in order to allow this, a September 1 milestone for the initial DBMAN SDS has been pushed back to May 1, 1996, in order to insert a MAPCOR SDS milestone on October 15. T. Evans accepted these challenges, and all involved wish to express our thanks to her and our confidence in her ability to accomplish these tasks.

  2. Flowchart Software: da Vinci -- The freeware flowcharting program "da Vinci" found by M. Moshir has been installed under Solaris by T. Evans, who distributed copies of the manual and sent email instructions to the working group regarding how to get started with it. This program produces postscript output which can be included in TeX and other document formats. It is our first approach to implementing the requirement for all SDS's to be delivered in postscript format with flowcharts included, as discussed in last week's minutes.

  3. SDS Signatories -- With the possible exceptions of EXEC, TAPELOAD, and PCP, all 2MAPPS subsystems' SDS's will have science-team signatories. A few subsystems have obvious associations with the science team, but most do not. R. Cutri will provide information to M. Skrutskie to aid in assignments of science team members to 2MAPPS subsystems. J. Fowler will provide R. Cutri with a list of the current subsystem names and one-line functional descriptions.

  4. Protocam Data Processing -- R. Cutri reported that the April data processing has reached the postprocessing stage, i.e., all proto-pipeline runs are completed, the photometric corrections are done, and about 60 scans have been recovered from position reconstruction problems. The Read1 merge, persistence and ghost removal, and new improved galaxy processing are now underway.

    T. Evans reported that all of the older protocam data (prior to April '95) have been completely processed through band-merging, APM and GSC association, and insertion into SyBase, and soon everything will be available via XCATSCAN.

  5. CSFlat Results -- R. Cutri reported that 10 M67 scans in the April 95 data have been processed by CSFlat, the responsivity-image/sky-offset flattening method. This method computes responsivity scale-factor images from twilight flats (or other types of measurements which may be used in the survey, e.g., dome flats) and offsets from the sky frames. The resulting flattened frames were processed by Roc to obtain ROUND-type results but with higher spatial resolution (6x10 subregions in the focal plane). The relative flux variations, which were about 20% peak-to-peak with the previous sky-flat method, dropped to about 5%. The dispersion in the repeatibility for KAMPhot and aperture photometry was un- changed, indicating that the multi-aperture averaging was generally reducing the in-scan variations rather effectively. The superior uniformity of the new flattening method makes it the obvious default for 2MAPPS implementation, however. Two effects were also observed that are not well understood, though: (a.) the offset between KAMPhot and aperture photometry became smaller in some (but not all) scans, while the dispersion remained constant; (b.) the in-scan relative-flux response in Ks appears actually to have reversed slightly from the previous shape, forming a slightly S-shaped pattern which is at least smooth and has a much lower amplitude than before.

  6. New FRD and FDD -- R. Cutri reported that the October 15 milestone for subsystem SDS's in support of the November 8 external review (see last week's minutes) also applies to new versions of the 2MAPPS Functional Requirements Document (FRD) and Functional Design Document (FDD).

  7. POSMAN/COFF Changes -- H. McCallon reported that he has been considering a design change that would involve POSMAN computing or refining the frame-to-frame offsets with the use of astrometric catalog sources. In the current design, STATS extracts single-frame point sources in all three bands for COFF (a STATS subsystem module) to use in computing frame offsets. Howard proposed moving POSMAN to where COFF is in the pipeline, leaving POSMAN's old spot occupied by a cleanup-and-tweak module that refines the position reconstruction given the larger set of point-source extractions from PROPHOT. The goal is to eliminate the random-walk error accumulation by applying an astrometric catalog at the point where the frame offsets are computed. This requires the astrometric catalog to be applied before coadding and PSF-fitting, although it can (and probably will) also be applied downstream for final adjustments.

    Two improvements to the position reconstruction can be expected from this approach: (a.) frame offset corrections can be implemented at a stage where the frames themselves can be treated as "stiff", i.e., individual frames need not be effectively compressed or stretched as in the current scheme which adjusts the in-scan positions of sources without regard to frame relationships; (b.) the use of an in-scan cubic spline correction can be abandoned. The improvements for ordinary well-behaved scans might not be significant, but a worthwhile gain in robustness should result from replacing the cubic spline, which is somewhat vulnerable to astrometric catalog errors. Such errors should be more obvious in their discrepancies relative to the frame offsets. The scheme is also appealing in that it uses more of the information simultaneously rather than sequentially.

    Nevertheless, this represents a serious system design change, and so a splinter session was held after the regular meeting. Most of the discussion above is based on the splinter session, which was attended by J. Fowler, G. Kopan, and H. McCallon. Howard estimated the total complexity to be about the same as before, however, and with the improved robustness and greater theoretical appeal, it was decided to adopt the modification.

    This eliminates the COFF module from the STATS subsystem and the rest of the system design. Its position will be taken by POSMAN, whose old position will be taken by a POSMAN module to be named by H. McCallon. G. Kopan reported that the functions performed by STATS have gone beyond what the name implies, and that subsystem will be renamed FREXAS (FRame EXtraction And Statistics And Seeing [the "AS" does double duty in the acronym]).