From - Tue Aug 20 16:05:25 2002 Return-Path: Received: from fnal.gov (heffalump.fnal.gov [131.225.9.20]) by castor.ts.infn.it (8.11.6/8.11.6) with ESMTP id g7EIPRS32545 for ; Wed, 14 Aug 2002 20:25:28 +0200 (MET DST) Received: from fnal.gov ([131.225.233.18]) by smtp.fnal.gov (PMDF V6.0-24 #37519) with ESMTP id <0H0U00BL1J6FH3@smtp.fnal.gov> for stefano.belforte@ts.infn.it; Wed, 14 Aug 2002 13:25:27 -0500 (CDT) Date: Wed, 14 Aug 2002 13:25:27 -0500 From: Fedor Ratnikov Subject: Re: why so slow ? Sender: ratnikov@fnal.gov To: stefano.belforte@ts.infn.it, Liz Sexton-Kennedy Cc: David Waters Message-id: <3D5AA097.26C9FE13@fnal.gov> MIME-version: 1.0 X-Mailer: Mozilla 4.77 [en] (X11; U; Linux 2.4.18 i686) Content-type: text/plain; charset=koi8-r Content-transfer-encoding: 7bit X-Accept-Language: en References: <3D57B921.DCFCDC0@ts.infn.it> <000901c2423c$a9d3dba0$0b00a8c0@fnal.gov> Hi, the reason is that this file (gb01defd.0001exo0) contains many EmptyRunsectionRanges records in the very beginning. These records are not accompanied by BOR records that is reasonable because there are no physics events belonging to those runs in the file. But FileInput is triggered by ERS record and generates artificial BOR record. The later is processed by framework and beginRun methods of contributing modules are called. This obviously cause a very time consuming access to the calibration DB. Original DHInput ignores all BOR for runs containing no physics event in the input file. Liz, could you update FileInput to avoid BOR triggering by ERS records? Could FileInput also skip BOR if there are no corresponding physics events available? Stefano, the file itself looks suspicious: it contains 20 empty runs in the very beginning (all events were reported as filtered out) and many runs with ~5 events per runsections afterwards. I suspect ERS records in the beginning should not be there. What is the history of this file? Cheers, Fedor. Fedor Ratnikov wrote: > Hello, > In this case they are quite different underlying interfaces: DHInput uses > old single branch > ROOT interface and FileInput/SamInput/DHInput_2 use new multi branch > ROOT interface. > Most likely new FileInput (that is base for other multi root fashioned > modules) needs some polishing to gain performance. > Thanks for notation, Stefano! > Cheers, Fedor. > > ----- Original Message ----- > From: "Stefano Belforte" > To: "David Waters" > Cc: > Sent: Monday, August 12, 2002 8:33 AM > Subject: Re: why so slow ? > > > Interesting... FileInput behaves like SamInput indeed > > (jsut runnign it now, it has all the irritating long pauses). > > It looks like in the end simply DHInput does a better job > > at skipping all those begin run. It may have something to > > do with the empy run sections that Alan was pointing at... > > > > David Waters wrote: > > > > > > On Mon, 12 Aug 2002, Stefano Belforte wrote: > > > > > > Hi Stefano, > > > > > > Interesting. Not sure I know the answer without looking in more detail. > > > SamInput module should behave identically to FileInput module once it's > > > actually retrieved a file name from SAM. The guts of SamInput are > > > identical to FileInput apart from the details of how it gets the file > > > name. I'm not actually very familiar with DHInput, so cannot immediately > > > tell you if this should be the same as FileInput. > > > > > > If you have time, can you compare SamInput with FileInput ? This would > > > narrow down the problem to something clever that DHInput does that > simpler > > > FileInput does not do. > > > > > > Cheers, Dave. > > > > > > > Why is SamInput so slow ? > > > > I mean, I run on a file already on local cache (always the > > > > same), but once the AC++ executable (SamInputTest) has > > > > started it takes miknutes to run EventDump on the first > > > > 10 events. If I start SamInputTest by hand and use DHInput > > > > on the same file, it takes less then a second (of course after > > > > a long pause at the beginning, to swallow some initial DB > > > > I guess). But DHInput, goes fast from one event to the > > > > next, SamInput seems like stopping for a minute or so > > > > at each event. > > > > I tried editing getFilesFromSam.tcl > > > > to remove SamInput debug statements and even RCP access, > > > > but makes no difference. > > > > > > > > The file I am accessing is the one in the test-sam-1g > > > > dataset, that I already have in local cache since the > > > > inital sam test installation: gb01defd.0001exo0 > > > > > > > > Notice also that using SAM I get a lot of message slike: > > > > %ERLOG-e ConfigManager::newRunSection: Cannot find beginRun record > > > > and > > > > %ERLOG-w Missing Begin Run.: The last begin run had a run number of > > > > 122623 > > > > that are absent if I use DHInput > > > > > > > > Note also that adding AC++ "show" command at the end of the > > > > tcl files says that DHINput was called 22 times (2 per > > > > event is "the usual", do not ask me why). While SamInput > > > > was called (for the same begin -nev 10) 214 times ! > > > > > > > > Maybe the begin run is the problem, and things will be > > > > better with a file with more events per run, but... > > > > once we are inside a file, what does SAM require at each > > > > begin run that DHInput does not ? > > > > > > > > I attach the log files (stdout + stderr). > > > > > > > > Stefano > > > > > > > > -- > > > > Stefano Belforte - I.N.F.N. tel : +39 040 375-6261 (fax: > 375-6258) > > > > Area di Ricerca - Padriciano 99 e-mail: Stefano.Belforte@ts.infn.it > > > > 34012 TRIESTE TS - Italy Web : > http://www.ts.infn.it/~belforte > > > > > > > > > > -- > > > *************************************** > > > * Dr. David Waters * > > > * Department of Physics and Astronomy * > > > * University College London * > > > * Gower Street * > > > * London WC1E 6BT * > > > * * > > > * Tel : (UK)-20-7679-7141 * > > > * (UK)-7812-748843 (mobile) * > > > * Fax : (UK)-20-7679-7145 * > > > *************************************** > > -- MS318(CDF/Rutgers) Fermilab, Batavia, IL 60510 USA Tel.:+1(630)840-8435 Fax:+1(630)840-6315