Re: deletion in MSS pools

Thank you Jon, that's what I wanted to know. My flush script takes care
of the existence of the file and simply skip it if the pnfs reference
does not exist anymore. I am not sure yet how I will handle this issue
in the future. The undelete feature is interesting but there are some
tools over there that copy files and delete them immediately (for
monitoring for example). As our MSS endpoint is publicly published, it
can be used in an incorrect manner.

Lionel

bakken wrote:
> It depends on the script that flushes files to tape, the one listed in
> your pool setup file.
>
> Here at FNAL things get flushed to tape quickly, so precious files get
> queued in dCache for tape writing within minutes of the user storing
> them. Usually there is a long tape queue, so there is a period in which
> the user can delete the file before it reaches tape - just as you say.
>
> We allow undeletes, so we go ahead and store the deleted file to tape,
> but since the filename isn't available anymore due to path deletion, we
> store it using its pnfsid as the filename. Then if the user tells us he
> made a mistake, we can recover it.
>
> So the actions are independent, and I would be against a change where
> files deleted from pnfs are removed from the tape writing queue. I
> think the right thing to do would be to modify the tape flushing script
> to detect this condition and have that script decide on what site action
> should be performed.
>
> -- Jon There's no place like 127.0.0.1
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> Jon A. Bakken bakken@fnal.gov (630) 840-4790
>
> On Thu, 22 Dec 2005, Greig A Cowan wrote:
>
>>
>> Hi Lionel,
>>
>> I thought the cleaner in the pnfsDomain did this for you. Do you have it
>> enabled in your pnfs.batch file?
>>
>> Cheers,
>> Greig
>>
>> On Thu, 22 Dec 2005, Lionel Schwarz wrote:
>>
>>> Hi all,
>>>
>>> I noticed that deleting a file in pnfs that is not yet flushed into MSS
>>> leads to inconsistency between pnfs and the pool. Indeed, the file is
>>> not known anymore by pnfs but is still in the queue for migration:
>>>
>>> > queue ls queue
>>> Class@Hsm : hpss-sc3:dteam@osm SUSPENDED
>>> 0001000000000000010D9778 osm hpss-sc3:dteam -
>>>
>>>
>>> > pf 0001000000000000010D9778
>>> (3) java.lang.NullPointerException from ac_pf_$_1
>>> java.lang.NullPointerException : null
>>>
>>>
>>> Do I need to manually clean the queues (considering the queues are
>>> suspended) or does the pool clean its queue by itself after a while?
>>>
>>> regards,
>>>
>>
>> --
>> ========================================================================
>> Dr Greig A Cowan http://www.ph.ed.ac.uk/~gcowan1
>> School of Physics, University of Edinburgh, James Clerk Maxwell Building
>>
>> TIER-2 STORAGE SUPPORT PAGES: http://wiki.gridpp.ac.uk/wiki/Grid_Storage
>> ========================================================================
>>

-- 
Lionel Schwarz
CCIN2P3/CNRS Villeurbanne, France
lionel.schwarz@in2p3.fr