From watts@physics.rutgers.edu Fri Mar 16 23:57:50 2001
Date: Thu, 14 Dec 2000 16:49:36 -0500
From: Terry Watts <watts@physics.rutgers.edu>
To: Stefano Belforte <stefano.belforte@ts.infn.it>,
     Marjorie Shapiro <Mdshapiro@lbl.gov>, Mark Lancaster <markl@hep.ucl.ac.uk>,
     Jim Amundson <amundson@fnal.gov>, Stephen Wolbers <wolbers@fnal.gov>,
     Terry Watts <watts@physics.rutgers.edu>, rharris@fnal.gov, ksmcf@fnal.gov,
     sexton@fnal.gov, Rob Snihur <snihur@fnal.gov>,
     Semeria Franco <Franco.Semeria@bo.infn.it>
Cc: watts@physics.rutgers.edu
Subject: DFCatalog and remote sites (fwd) FYI

Hi - Some part of the DFC needs to be exported.  -  Terry

------- Forwarded Message
To: lmark@cdfsga.fnal.gov
Cc: watts, shapiro@kfesg.lbl.gov, litvinse@fnal.gov, stdenis@fnal.gov,
        lammel@cdfsga.fnal.gov
Subject: DFCatalog and remote sites
Date: Tue, 12 Dec 2000 16:44:46 -0500
From: Terry Watts <watts@physics.rutgers.edu>

Hi Mark - Marjorie remarked in the Export Review that we should talk over 
differences between the DFCatalog and Calibration tables.  Some thoughts:

1. At Fermilab, the Catalog is divided between production space and user/group 
space, both in Oracle. Production space has:
 a. Tables connected with runsections and the dataset registry.
 b. Tables to describe datasets, filesets, files, all created by either the 
online pipeline for raw data, or the production farm for pad datasets.

User/group space has only (b) but for user and group datasets created in FCC.

We would need to export (a) from production space in a way similar to the 
one-way export of calibration.  (b) would be part of the transfer of meta data 
between sites, see below.

2. Some remote sites will create datasets and filesets, e.g. Monte Carlo, so 
will need a local Catalog to contain class (b) of tables.

3. Remote sites will use copies of datasets created elsewhere, at Fermilab or 
a another remote site.  The local Catalog would contain only datasets and 
filesets that are available on local disk (this is a proposal).

4. For both 2 & 3, a transfer mechanism is needed for the actual data (tapes 
probably) and class (b) tables in the DFCatalog.  I wonder if your export 
mechanism can be the basis of the transfer mechanism for subsets of the class 
(b) tables, i.e. for a subset of only one particular dataset?

I assume that import at FCC of DFCatalog class (b) tables would go only into 
user/group space and not into production space.

Note that part of the UK/CDF GRID application planned at Rutherford-Appleton 
Lab would use some transfer mechanism to move small datasets (both actual data 
and Catalog meta data) between sites by network.  Presumably your export 
mechanism could be this transfer mechanism for the Catalog tables?

  Cheers,  Terry



------- End of Forwarded Message