home | documentation | downloads | feedback | search | imprint

The 1.8.0-release

	dCache 1.8.0 production.
Please make yourself familiar on how to configure Space Reservation, part of the SRM chapter of the the dCache book.

dCache Releases of 1.8.0 (dCache Core Server, SRM + SRM 2.2)

OS, Platform independent

dCache Releases of 1.8.0 Client SRM

dCache Releases of 1.8.0 Client DCAP

The change log summary

ChangeLog

15 => 15p3

14 => 15

13 => 14

1. various fixes in the new startup scripts
2. cleaning up install.sh: added logging, moved from sh to bash
3. fixed socked adapter
4. new release of Chimera
5. fixed FTP protocol violation in ftp door, which causes remote gsiftp transfers to fail.
6. added option to steer number of concurrent removes on SRM remove.
7. ls in stace manager includes more options
8. More changes.

12 => 13

1. All changes of the ccrc08 branch have been applied to the release as well.
2. More changes.

10 => 12

1. An improved SrmSpaceManager has been introduced, which requires a schema migration.
2. Space manager schema migration script added.
3. Improved addCacheLocation in PnfsManager.
4. Improved ftp door
5. Fixed file descriptor leak in ActiveAdapter
6. Inter-cell communication uses 'keep alive' sockets, which should fix domain reconnect problems.
7. make srm copy version 1 to version 2 in push mode specify TargetStorageType=PERMANENT, make srmCopy and srmPrepareToPut reject non-PERMANENT storage types, when copies are into dCache
8. Added pnfs2chimera migration scripts.
9. Fixed sticky bit leaking in pin manager.
10. Added user's DN to remoteGsiTransfers billing record
11. With this release, the (new) improved PinManager is becoming the default.
12. Fixed the 'pnfs register' command such that it does not register files that are not marked either precious or cached yet.

9 => 10

1. Fixed several space management related bugs in pools (although their likely are more bugs in this area - don't assume all problems are fixed!).
2. Restructured pool inventory generation. If pool startup fails during the inventory phase, a pool is now disabled in such a way that it cannot be enabled with 'pool enable' (as that would lead to further errors).
3. Removed the -modify flag to pools (it was not fully implemented and was dangerous to use).
4. Improved exception handling in pools.
5. Added periodic consistency checks for space accounting inconsistencies. If inconsistencies are detected, a pool is automatically marked as read-only to protect against further degradation of the pool.
6. Fixed bugs in handling of URI based HSM scripts when used with Berkeley DB based pools.
7. Fixed race condition and error handling in p2p client.
8. Fixed bug in pool repair code used during pool startup, which would previously propagate a bad file size to PNFS.
9. Added command for listing the named sticky bits in the repository file in the pool, and for creating/modifying sticky bits and their lifetimes.
10. added printouts to trace space allocations
11. added error report in case of entry.getSize() not equal to entry.getDataFile.length() on cache repository entry removeal
12. added start/stop/restart to chimera-based nfs interface
13. added namespace migration utility
14. set gap command in the pool supports unit (like 5G )
15. fixed leading zero in adler32 checksum: 675af7 == 00675af7
16. gsidcap sends VOinfo to space manager if enabled
17. mover queues can reuse working thread ( thread pool instead of new Thread)
18. added execution bit for shell scripts in dcache-server-X.tgz
19. no hidden on-the-fly checksum computing when switched off on the target pool of a P2P-transfer
20. newer xrootd clients are working against dCache (the mover can now handle asyncronous requests)

8 => 9

1. Fixed dead lock in the pool
2. Fixed pool space miss calculation
3. added jar files used by replica manager
4. permission hadler replaced with a new implementation
5. intenal improvements

6 => 8

1. Added some safety checks to the GFtp/2 mover to protect against potential problems in the data channel implementations. It does so by adjusting the space reservation after the transfer.
2. Older releases freed all reserved space when something goes wrong with the space manager. This is not correct when a file is left on the pool, and can lead to pool overbooking.
3. All pools participating in replication MUST be declared in PoolManager.conf in ResilientPools pool group or other group presented to Resilient Manager.
4. DB schema has changed and must be re-initialized by provided template:

   $ dropdb replicas
   $ createdb replicas
   $ psql -U enstore replicas < /opt/d-cache/etc/psql_install_replicas.sql
   

5. Pool code changed back so PnfsAddCacheLocationMessage is sent out when file added to the pool. This resolves the issue when files been in receiving.transfer state not replicated.
6. Files not found in pnfs now correctly excluded from replications.
7. One more pnfs error report signature was added to exclude files removed from pnfs.
8. Flags "all pools are resilient" ('*') are removed from templates replica.batch and dCacheSetup.template. This is not supported for sometime as it proved to prone to configuration errors, we removed it now from default configuration.
9. replica manager : auto release of excluded files after 12 hour timeout. Configurable via option ExcludeExpirationTO.
10. replica manager : All ongoing replications when pool goes down are stopped.
11. replica manager : connection to DB is split using DB connection pooling
12. replica manager : resilient manager monitoring now reports reason for the file exclusion.

5 => 6

1. space token for a file is retrieved independent on the state of the file
2. possible bottlneck having to do with sequence generation
3. proper error message on bad pnfs config
4. clenaup of "Unexpected reply" in pnfsDomain.log
5. srmLs showes ONLINE/NEARLINE
6. pnfsManager do not ties to read level2 on directories
7. srmLs showes space tocken of a file

4 => 5

1. fixed broken in -4 directroy creation

3 => 4

1. fixed unpinning of ONLINE files after transfer
2. fixed 'binary' compatibility with 1.8.0-2
3. fixed billingDB error in flush/restore logging
4. fixed retry blacklist in PoolManager
5. added restore suspend if hsmcp exit code is 71
6. added chimera-based nfs server ( Evaluation only )

2 => 3

1. improved SrmSpaceManager performance
2. added logs for pool up messages
3. added pool repository tests
4. fixed flushing of files which left prior migration from 1.7
5. fixed active flush counter
6. added history capabilities into admin interface

1 => 2

1. detect that the ProtocolInfo is GridProtocolInfo in SelectPool msg, take VO Group and role from it, take Access Latency and RetentionPolicy from storage info for implicit space reservation for non-srrm transfers
2. added LinkGroup into match page

0 => 1

1. Get rid of orphaned files on pools after a gridftp/dcap fails.
2. Correcting statistics in PnfsManager. Now all messages should be counted.
3. Removed the batchfile-parameter "calculate-transfer-crc'. Checksumming for DCap now entirely steered by the checksum module policy.
4. /usr/bin/java now default path in dCacheSetup. (We need 1.5 for dCache 1.8 anyway)
5. All the technical details.

dCache beta Releases of 1.8.0

ChangeLog

beta25 => beta26

1. You need to change your configuration in order to run this release
2. Caching of authorizations in srm and caching of gums (saml-vo-mapping) callouts in gPlazma have been added to version 1.8.
3. Typo fixed in configuration "linkGroupAuthiorizationFileName" to "SpaceManagerLinkGroupAuthorizationFileName".

beta23 => beta25

1. added release id into SrmPing
2. added fast shutdown procedure
3. improved cog-libraries
4. Updated FTP client ( used bu pools in srmCopy) to use GFD.47 GETPUT if available.
5. show AccessLatensy and RetentionPolicy in psu ls -l linkGroup command
6. better behavior of ftp doors in cafe of client disconnects

beta22 => beta23

1. FIX OF : srmStatusOfGetRequest returns an inconsistent return code at request level

beta21 => beta22

1. Fixed PoolManager.conf file corruptions on save in beta21.
2. Added 'disable ' and 'enabled ' commands in the LoginBroker. This allows to block a door from appearing in 'ls -binary' and thus to drain a door prior to shutting it down (since the srm door will stop generating TURLs for it).
3. Moved 'rep ls' and 'rep rmclass' processing to background threads.
4. Modify getFileMetadata so that it does not throw an exception if a user doesn't have proper permission to read its own file. This led to a situation where a user could remove permissions from a file and coudn't modify it any more at all.

beta20 => beta21

beta19 => beta20

1. added berkeleyDB based pool metadata repository support
2. added initial version of chimera based cleaner
3. SRM: Space reservation with retentionpolicy=REPLICA and unspecified accesslatency resolves in reservation of CUSTODIAL-NEARLINE type of space when the access latency optional parameter is not specified.
4. SRM: An srmPrepareToPut or an srmStatusOfPutRequest returns SRM_NO_FREE_SPACE at file and request level if the space specified is expired instead of returning SRM_FAILURE at file level and SRM_SPACE_LIFETIME_EXPIRED at request level, or (if the space token is no longer valid) SRM_INVALID_REQUEST at request level and SRM_FAILURE at file level.
5. Remove FTP mover: added custom checksum support
6. PoolManager: added commands to manipulate Access Latency and retention Policy of a LinkGroup
7. FtpDoor: Fixed a file descriptor leak in how the socket adapter is used when passive pools are enabled.
8. FtpDoor: Fixed problem with hanging FTP doors when the transfer times out.

beta18 => beta19

1. srmGetSpaceTokens returns SRM_INVALID_REQUEST if the space token description provided does not refer to any existing space token
2. Added support for Space Manager LinkGroup Authorization file

   File syntax
   LinkGroup Name followed by the list of the FQAN, one FQAN on separate line, followed by an empty line, which is used as a record separator, or by the end of file.
   Any line that starts with # is a comment and may appear anywhere.
   File location is specified by defining

   SpaceManagerLinkGroupAuthiorizationFileName = <FILENAME>
   

   in the dCacheSetup

3. Modified SpaceManager database schema to support mode descriptive properties of the LinkGroups. The LinkGroup definition in the pool manager will change in the next release, which will be available end of this week. These changes should be transparent to the users.
4. Users can now use FQAN like syntax when specifying roles in the PoolManager.conf file.
   

   <VO>/<Role=ROLE> instead of 
   <VO>Role=<Role=ROLE> 
   

   The old syntax is still supported.
5. SRM V1 advisory delete will work with higher reliability.
6. If the AccessLatency and RetentionPolicy tags are specified in the pnfs directories, the default AL and RP will be based on these tag values.
7. Space occupied by a file is returned to a Space Reservation to which this file belongs, if the file is deleted.(T0D1)
8. Option for enabling implicit space reservation for SRM V1.1 and SRM V2.2 is renamed to srmImplicitSpaceManagerEnabled, and is enabled by default (used to be misleadingly called "srmSpaceManager")
9. These are changes to fix SRM implementation problems compiled by Flavia :
   • RM_ABORTED at requets level, if aborted.
   • SRM_NOT_SUPPORTED if no requestToken is specified.
   • catch SRMAuthorisation exeption in getTurl call and set request status to SRM_AUTHORIZATION_FAILURE.
   • throw SRMAuthorisation exeption in getTurl if path is not part of user's root path.
   • SRM_NOT_SUPPORTED at request level if wrong protocol is specified and SRM_FAILURE at file level.
   • SRM_AUTHENTICATION_FAILURE instead of 5BSRM_AUTHORIZATION_FAILURE in handleRequest method if gPlama authentication fails.

beta15 => beta18

• SRM: advisory delete only remove the pnfs entry without changing the files to cached
• SRM: return SRM_INTERNAL_ERROR if pnfs is not mounted
• SRM: return SRM_NOT_SUPPORTED at request level if wrong protocol is specified and SRM_FAILURE on file request level
• TransferObserver: fixed null pointer exception
• Ftpv2: added a sanity check for requested transfer parameters
• ChimeraNameSpaceProvider: added checksum support
• PnfsManagerV3: added checksum manipulation commands
• dcap: supports up to 1024 entries in readv call
• ReplicaManager: synchronized with 1.7
• pool: internal code refactoring
• pool: fixed storageInfo conflicts with 1.7

beta14 => beta15

• SRM Space Manager: return space back to the reservation upon file deletion
• SRM Space Manager: Default access latency and retention policy is now based on directory AccessLatency and RetentionPolicy tags if these are present
• SRM: return SRM_AUTHENTICATION_FAILURE instead of SRM_AUTHORIZATION_FAILURE in handleRequest method if gPlama authentication fails
• cells: use keep alive on cells intercommunication TCP connections
• SocketAdapter: Supress error messages when the adapter is being closed.
• SocketAdapter: Suppress error message for expected behaviour (transfer finished message may be received after the transfer was cleaned up already)
• GFtp-2: org.globus.tcp.port.range is used for defining the port range for passive transfers
• BasicNameSpaceProvider: throw FileNotFoundCacheException instead of IllegalArgumentException when file is not found
• PnfsHandler: less flood in log files
• pool: fixed file descriptor leak in HSM connectivity
• gsiftp door/movers/: added support for user defined checksum

beta13 => beta14

• Chimera: fixed java.lang.ArrayIndexOutOfBoundsException in getTag if tag does not exist

beta11 => beta13

• srmMkDir and SRMRmDir return SRM_AUTHORIZATION_FAILURE, SRM_DUPLICATION_ERROR and SRM_INVALID_PATH when appropriate
• space manager understands lifetime=-1 as infinite
• get-space-tokens does not check ownership
• reserve space admin command takes lifetime in seconds, or -1 for infinite
• SRMServerV2.java modified so that srm 2.2 interface returns a simpler error message if a particular method is not implemented
• srmrmdir client is updated: new boolean option "recursive " is added, previously used numeric option "recusion_depth" was not appropriate.
• PnfsCompanion uses database connectionpool
• New tags to define default AccessLatency and RetentionPolicy, .(tag)(AccessLatency) and .(tag)(RetentionPolicy)
• fixed a bug in link group vo based selection

beta10 => beta11

• srmRm returns SRM_INVALID_PATH if file does not exist
• removed NullPointerExeption in case of file hs no CRC
• added axis, cells and sql tracing into log4j config of SRM
• tomcat respects memory option from srm_setup.env
• more thread allowed to process space manager requests
• pin manager provides lifetime while pinning
• web pages use css
• A new administrative function was added to dCache space manager, that allows an administrator to reserve space in a given LinkGroup.

beta9 => beta10

• improved database connection pooling
• fixed ArrayOutOfBounds error in job storage
• SpaceManager uses thread pool to process requests

beta8 => beta9

• added traceing of cell message exchange

beta7 => beta8

• Pool Manager
  • new thread pool
• Pool
  • Multi HSM instance support.
  • flush script does not need mounted pnfs
  • URI like syntax for flush script output
  • fixed flush notification message
• PnfsManager
  • store and retrieve HSM location in URI like format
• Dcap
  • added vector read
• Chimera integration
  • fixed chimera.batch files
  • fixed null pointer exception messages in chimeraDomain log file in case of running pathfinder for missing files

beta6 => beta7

• Pool Manager
  • Faster PoolSelection by improved thread concurrency in PoolSelectionUnitV2
  • Improved Thread Pool.
  • BUG FIX : null pointer exception fixed (if pool does't send its hostname).
• SRM
  • copy request uses space token.
  • fixed batch file.
• Pool
  • Repository does no longer check existance of control file (Avoids 'stat' operation).
  • Fixed missing '\' in pool.batch file.

beta3 => beta6

• pool: repository supports olf format control files ( some exceptions still possible)
  
• pnfs manager: less stack traces on regular error conditions (file not found and so on)
  
• pool: fixed in ChecksumModule
  
• pool: fixed memmory usage of ftp mover e
  
• srm: database schema fix
  
• srm: valid reply CheckGarbageSpaceCollector use case
  
• pool manager: do not use pool from linkGroups if linkGroup is not specified in request
  
• added Chimera support (enabling instructions comming soon)
  

Copyright dCache.org © 2003 - 2008