1.9.12 Release Notes

dCache 1.9.12 focuses on:

Table of contents

Upgrade Instructions

Incompatibilities

Please consider the following changes when upgrading from a version before 1.9.12-1:

Compatibility

It is safe to mix pools of releases 1.9.5, 1.9.6, 1.9.7, 1.9.8, 1.9.9, 1.9.10, 1.9.11 and 1.9.12. Head nodes and doors must be upgraded to 1.9.12 together and cannot be mixed with head nodes or doors of releases before 1.9.12. Components of different 1.9.12 releases can be mixed freely.

Compatibility Matrix

We distinguish between pool components and head nodes. Any component which is not a pool is considered a head node, including doors. The following table shows compatibility between different releases.

1.9.12-1 Head1.9.12-1 Pool
Head 1.9.5nono
1.9.7nono
1.9.8nono
1.9.9nono
1.9.10nono
1.9.11nono
1.9.12yesyes
Pool 1.9.5yesyes
1.9.7yesyes
1.9.8yesyes
1.9.9yesyes
1.9.10yesyes
1.9.11yesyes
1.9.12yesyes

1.9.12-1

Pools

The /opt/d-cache/bin/dcache pool commands have been updated to provide an ls subcommand to list configured pools. The create subcommand has been extended to add the new pool directly to the layout file. The command no longer creates the pool setup file and instead adds the pool specific configuration to the layout file. The command thus provides the functionallity of both the create and add commands found in some earlier releases.

The migration module was extended to reduce interference between multiple migration jobs running on the same pool. The migration module now respects the pool mode on the target pool.

SRM

The SRM used to have the ability to periodically submit vacuum commands to the database. This command is PostgreSQL specific and thus tied the SRM to PostgreSQL. PostgreSQL has for quite a while supported auto vacuuming, and manually executing the vacuum command is no longer necessary. We have therfore removed the submission of vacuum commands from dCache. The configuration parameters srmVacuum and srmVacuumPeriod are obsolete. We recommend using at least PostgreSQL 8.3 and preferably 8.4, as earlier versions of PostgreSQL are known to have issues with auto vacuuming.

Some SRM operations may be processed synchronously or asynchronously, at the server's discretion. Since 1.9.10 dCache can start to process such requests synchronously and, if this is taking too long, reply asynchronously and continue to work on the operation in the background. In dCache 1.9.12 this behaviour is enabled by default for get, put, bring online, and list operations. The default switch over time is 1 second. The old behaviour can be restored by setting the srm*SwitchToAsynchronousModeDelay parameters.

The RemoteHttpTransferManager cell has been removed. The functionality has been merged into the RemoteGsiftpTransferManager cell, which has in turn been renamed to RemoteTransferManager. The SRM was extended to support third party srmCopy transfers from HTTP resources. Third party upload to HTTP resources and support for HTTPS is not available yet in the SRM.

DCAP

DCAP supports prestaging from tape. The DCAP door used to rely on a stager service to perform the staging. In dCache 1.9.12 the DCAP door instead uses the PinManager to trigger staging. The dummy-prestager service is obsolete and can be deleted from the layout.

PNFS manager

The PNFS manager has been restructured. One consquence is that the output of the info command in the cell admin interface has changed slightly: It is now grouped into sections in line with how it is done in pool manager and pools. Third party scripts may need to be adjusted.

WebDAV

WebDAV doors now submit remove entries to billing when files are deleted.

The HTML rendering of a directory listing now includes a column containing the file size. The HTML is customizable through a template file. As a consequence the properties webdav.images.logo, webdav.images.directory, webdav.images.file, webdav.style.css, webdavLogoPath, webdavDirIconPath, webdavFileIconPath, webdavCssPath, dirIconPath, fileIconPath, and cssPath are obsolete. Please read the document embedded in the default template file /opt/d-cache/share/webdav/templates/list.stg for details on customizing the look and feel.

The property webdavAnonymousListing was introduced to prevent unauthorized users from listing directories.

The property webdavProtocol was extended with a third option, https-jglobus, to enable HTTPS support based on the JGlobus library. The advantages compared to the https option are that host and CA certificates are read and refreshed directly from /etc/grid-security/ and that proxy certificates (including VOMS certificates) can be used for client authentication.

Info Provider

The info provider is an XSLT script that transforms the output of the info service into GLUE compliant LDIF. The XSLT script used to be stored in /opt/d-cache/etc/glue-1.3.xml and used to contain both the transformation logic and local configuration parameters.

In dCache 1.9.12 the info provider has been extended to support GLUE 2.0 in addition to GLUE 1.3, and it has been restructured to separate configuration parameters from the transformation logic.

The old info provider file, /opt/d-cache/etc/glue-1.3.xml, is obsolete. All site specific configuration is done within the file /opt/d-cache/etc/info-provider.xml. The configuration parameters httpHost, xsltProcessor, xylophoneConfigurationDir, xylophoneConfigurationFile, xylophoneXSLTDir, and saxonDir are deprecated and have been replaced by other parameters. The configuration parameter httpPort is obsolete. The parameter info-provider.publish specifies whether to publish GLUE 1.3, 2.0, or both. See /opt/d-cache/share/defaults/info-provider.properties for details.

Pin Manager

The pin manager is responsible for staging and pinning files on behalf of SRM and DCAP doors. In this release the pin manager has been reimplemented from scratch. The design has been significantly simplified to increase performance and robustness of this critical component.

The database schema is separate from the old implementation's schema. The new schema is created the first time the pin manager is started. Existing pins are automatically imported if the same database is used. We strongly recommend using the same database. The tables of the old pin manager are not deleted, however downgrading is not possible without loosing the pins that were created with the new pin manager.

The command line interface of the pin manager has changed slightly and third party scripts may have to be updated.

Pool Manager

Pool manager has seen a fair amount of cleaning in this release, however user visible changes should be small. The most significant change is that pool manager no longer verifies the correctness of location information stored in the name space, that is, on which pools a file is stored. Instead pool manager expects doors to verify the information and deal with any stale locations. This change eliminates a round trip to the pool and thus reduces the latency of opening files.

Pool mode changes are now logged at warning level by pool manager.

Doors and DNS lookups

Most doors and movers have been updated to avoid DNS lookups and reverse lookups that slow down transfers. Instead IP addresses are passed around. This may break existing deployments on multi-homed hosts. When this happens doors may need to be bound explicitly to particular network interfaces.

DCAP, FTP, SSH and admin doors now support the listen property to bind the door to a particular IP address.

gPlazma

The get mapping cell command was added. The command was present in earlier releases, but was dropped when the gPlazma cell was reimplemented in 1.9.9.

Database Support

The /opt/d-cache/bin/dcache script has been updated with several new commands related to databases used by various services.

The subcommand database ls lists the databases used by services on this host and is supported by all services that use an external database.

Other database subcommands only apply to services using the LiquiBase schema management library. Currently pin manager is the only service that uses this library. Please consult the man page for details about these commands.

Scripts

Besides changes to the pool and database commands, /opt/d-cache/bin/dcache now supports the services command. The command is similar to the services command found in dCache 1.9.5: It lists all configured services and the domains they belong to. The status command was extended to provide additional information. The new check-config command provides diagnostic information about obsolete and deprecated properties used.

The migration script was moved and renamed to /opt/d-cache/libexec/migrate-from-1.9.5.sh. It was updated to follow best practice by using the host name as the name of the layout. The new version does its best to remove lines copied verbatime from the original template into dCacheSetup. Many deprecated properties are automatically renamed to the new names. At the end of the migration the script invokes the check-config command to report about any deprecated or obsolete properties being used.

The /opt/d-cache/install/install.sh script is no longer needed and has been removed. The remaining two scripts in /opt/d-cache/install/ have been moved to /opt/d-cache/libexec/.

Configuration Properties

Lots of minor issues have been fixed in the configuration system. Many properties that were already deprecated or obsoleted in earlier releases are now marked as such and generate errors or warnings. Other properties that were removed during the transition to the new configiration system in 1.9.7 have been reintroduced as deprecated properties. Default declarations of most services have been moved from batch files and into properties files in /opt/d-cache/share/defaults/.

Changelog 1.9.11-1 to 1.9.12-1