From mazz@ipac.caltech.eduWed Feb 26 07:22:22 1997
Date: Tue, 25 Feb 1997 12:58:22 -0800 (PST)
From: Joe Mazzarella 
To: ISIS -- Nian-Ming Chiu ,
    Roc Cutri , Tracey Evans ,
    John Good , Carol Lonsdale ,
    Joe Mazzarella ,
    Sherry Wheelock 
Cc: Tom Chester 
Subject: 12Feb97 2MASS DBWG Meeting Minutes. Meeting today 1:15PM, SCR

12-Feb-97 2MASS DBWG (ISISWG) Meeting Minutes

ATTENDED: Nian-Ming, Tracy, John, Joe, Roc


Tracy: Loading/indexing test results report:
Method 1: Load the table with indexing on.
          ==> Requires a re-build of internal pointers/indices.
Method 2: Drop indices; load data; re-create indices.
Comparison: Even with non-clustered indices, Methid 1 takes ~10 times
longer than method 2.
Left to do: Unclear how timing is effected by the number of indices.
Note: Indices add ~20% to the initial space requirements.
      (To be clarified later.)

Roc: There will be an end-to-end operational run-through using either
'faked' 3-channel data or a real night of telescope data in mid/end March.
This should include DBMAN to test population of the DB, and hopefully
the ability to run some basic queries. (V1.0 of the pipeline is due ~17 Mar.)

1) Is enough disk space configured to handle the DBMAN output? 
Approximate needs:

(<#objects/scan>~5000)*(~75 scans/night) ~ 4e5 objects/night
==> Assume average of ~0.5e6 objects/night.
           peak    of ~1.0e6 objects/night.

(~1e6 objects/night)*(~600 bytes/object) ~ 6e8 = 0.6 GB/night
==> Assume 1 GB/night to be safe for run-through.

We will have 8*9GB~70 GB (/k*) shoebox disks on karloff available 
for Informix as soon as team mambers clear the space.
==> Enough for first ~2 months of the mission.

See actions below.

2) Graphics output for Web pages:  To simplify usage of Web HTML for
2MASS analysis docs (e.g., basic in-lined graphics) and ISIS, we need to
produce results in GIF (or JPEG) format in addition to PostScript. We
need to learn how to produce GIF/JPEG directory from our plotting
packages, and/or how to convert PostScript to GIF/JPEG in an
automated way without loss of resolution. (Display of PS with
Ghostview and doing a screen-dump with xv does not cut it!).  Mazz
agreed to look into solutions (NETPBM, etc.)


      o Email to ipac-staff about 18-20 Informix classes.
      o Draft agenda and viewgraphs for IPAC ISIS meeting (w/ Joe)
      o Discussions with WIRE indicate they will purchase their own
        UltraSPARC 2 to be configured as their own informix server.
      o Progress on SIS's and timing tests. See discusion above.
      o Met w/ Sherry regarding 2MASS postage-stamp server.
        Link to current server in place. We will make it more
        modular and integrated into ISIS later.
      o Draft agenda and viewgraphs for IPAC ISIS meeting (w/ John)
      o Distributed copies of CDS/ESA "Astronomical Server URL" memo.
      o Summarized disk space available for 'tmass' db for first few 
        months of mission.
        Answer: 8*9GB~70 GB (/k*) shoebox disks on karloff

(Long-term ongoing actions omitted here. See task schedule on our Web.)

      o Loading/Index timing experiments:
        w/ and w/o various indexes, clustered indexes, etc.
      o Refinements on SIS's for DBMAN & WSDB field definitions: 
        extended sources, mag & color uncertainties and upper limits.
      o Treat galaxy data. (See previous memos).
      o Calibration scans need SIS's and tables in the DBMS.
        ~40 fields expected; define indexing scheme.
      o Infrastructure (& User System w/ Joe) design docs.
      o Design/prototype infrastructure (& User System w/ Joe) code.
      o Call Cassini Project and get their story on the raid disk.
      o Find out more (w/ Nian) about HP/Informix business.
        Does this hardware/software bundle offer significant benefits?
        Summarize in email to team. Send email.
      o Discussions (w/ Joe) with IPAC mission managers in preparation 
        for general ISIS req/design 'kick-off' meeting.
      o Working on high-priority Survey Operations DB (non-Informix) project.
      o Refine mag/color uncertainties and upper limits (w/ Sci Team) 
        for DNMAN SIS (to Tracy).
      o Firm up agenda and viewgraphs for IPAC ISIS meeting 
        (w/ Joe&John&Carol)
      o Discussions (w/ John) with IPAC mission managers in preparation 
        for general ISIS req/design 'kick-off' meeting.
        ISO & SIRTF (George & Rick): Friday, 07-Feb, 11:00
        WIRE (Tim & Perry): TBD
        MSX (Mehrdad): TBD
      o Reschedule ISIS meeting and include (at least draft) agenda.
      o User System (& Infrastructure w/ John) design docs.
      o Design/prototype User System (& Infrastructure w/ John) code.
      o Make an "RTB" for Sybase->Informix db migration (FSC to start)
      o Firm up agenda and viewgraphs for IPAC ISIS meeting 
        (w/ Joe&John&Roc)
NEW ACTIONS (See also ONGOING above and in Web Task List)

      o Develope initial "CatScan" interface for end-March run-through.
      o Please design and document the data "containers" (partitions) we 
        will use for the source DB. How will table fields be partitioned
        vertically (split into relations), and how will objects=rows be
        partitioned in data-space to maximize query efficiency, etc.?
      o Plan for incorporating /k* disks into Informix space as the
        disks are cleared and become available. It would be nice to
        have one 9 GB disk totally available for extension of tmass
        by ~7 March in preparation for DBMAN run-through by mid/end
      o Develope and test initial DBMAN for end-March run-through.

NOTE: Joe, John, Tracy, Sherry, Nian will be at Informix ESQL/C class
      next week, so there will be no DBWG meeting.