Executive summary

Highlights from this release:

  • Significant performance and storage space improvements in Chimera.

  • Almost all certificiate handling code has been ported to the EMI CANL library rather than JGlobus. Provides OCSP support and EUGRIDPMA name space support.

  • Concurrent request processing in gPlazma.

  • Pool group patterns in admin shell.

  • Improved UberFTP support.

  • WebDAV door can report file locality.

  • Fair share scheduling for SRM transfers.

  • SRM database managed by liquibase.

  • Configurable kXR_Qconfig support for xrootd.

Incompatibilities

  • Strict RFC 2818 host name checking in SRM/HTTP clients - this in particular affects server side srmCopy and WebDAV COPY. It is expected that this will break with server side copy to/from some WLCG sites. Globus Toolkit enables the same strict mode by default by the end of the year, so we expect the affected sites to get new host certificates soon. Additionally, server side copy is not used much, so it is likely not be a big issue for dCache.

  • Significant schema changes in Chimera. Upon upgrade the schema will be updated, which may take a while for large sites. To downgrade, the schema changes have to be rolled back with the appropriate command in dCache before downgrading dCache.

  • PnfsManager now uses a single request queue. This could possibly affected third-party monitoring scripts that scrape the info output.

  • All forbidden and obsolete properties in 2.13 have been removed and all deprecated properties in 2.13 have been marked obsolete.

  • RC4 ciphers are banned by default and the option to ban Diffie Hellman key exchange if broken has been removed.

  • gPlazma now instantiates plugins separately for every use in gplazma.conf. This means that the configuration has to be specified for all lines, not just the first. This provides the flexibility to use the same plugin several times with different configurations.

  • The WebDAV door enables BASIC authentication by default over HTTPS.

  • Pools always compute checksums on the fly and the ontransfer checksum policy has been dropped.

  • The SRM by default uses a fair share scheduler for scheduling transfers. Thus the observed behaviour under load may have changed.

  • SRM 1 support is deprecated and disabled by default.

  • SRM scheduler states have been simplified. Third party monitoring scripts that track request states may have to be updated.

  • SRM no longer retries SRM requests internally. Failures are propagated to the client to provide fail-fast behaviour.

  • SRM no longer resolves host names in SURLs when determining whether they are local to this dCache instance or not. In particular, this may affect sites that use DNS aliases.

  • The SRM database schema is now managed by liquibase and the schema is updated upon upgrade. To downgrade, the schema changes have to be rolled back with the appropriate command in dCache before downgrading dCache. As a consequence, user information for existing SRM requests will be lost from the database. Third party scripts that access the SRM database directly may have to be updated.

Release 2.14.51

admin

While migration move tasks on pools were working correctly, for migration info command an error occurred, that the current user (root) wasn’t allowed to execute anything (due to missing ACLs). This is now fixed.

Changelog 2.14.50..2.14.51

68975a9
[maven-release-plugin] prepare release 2.14.51
ee228bb
admin: Fix Inconsistent ACL enforcement, RT 9207
1421637
[maven-release-plugin] prepare for next development iteration

Release 2.14.50

httpd

The “Disk Space Usage” webpage (/usageInfo) contains a table showing information about each pool in the dCache cluster. The “Layout” column showed the capacity usage graphically, with different colours showing how much of that pool’s capacity is being used for different tasks. This release fixes the Layout heading to describe a previously undocumented colour.

Changelog 2.14.49..2.14.50

ad1ba42
[maven-release-plugin] prepare release 2.14.50
36e6e64
[maven-release-plugin] prepare for next development iteration
37a50d3
httpd: Fixed table headers in usageInfo

Release 2.14.49

pool

If the communication between a pool and PnfsManager times out, the error message is not well suited to diagnosing the problem: Failed to instantiate mover due to unsupported checksum type: Request to [>PnfsManager@local] timed out. The checksum type is not playing an important role here. Hence, this patch updates the error message.

Changelog 2.14.48..2.14.49

b30310c225
[maven-release-plugin] prepare release 2.14.49
1f225db503
pool: fix error message on timeout
4df7bf679d
[maven-release-plugin] prepare for next development iteration

Release 2.14.48

webdav

A recent update, commit 5abc0e1c, improved the behaviour of the Milton WebDAV libraries if an IOException occurs during an upload. That patch, unfortunately, did not address all issues, and when non-spec-conformant clients are used against dCache, stacktraces can be triggered.

This patch corrects that behaviour. Also, in case of errors, the error code returned in case of any problems was changed from 400 to 500, which should signal cliens that they are free to retry the transfer after a timeout.

Changelog 2.14.47..2.14.48

f35052b139
[maven-release-plugin] prepare release 2.14.48
4e71787f3e
webdav: make Milton work-around more robust
8207aefbff
[maven-release-plugin] prepare for next development iteration

Release 2.14.47

Changes affecting multiple services

The version of the PostgreSQL driver used by dCache internally was brought up to 9.4.1212. This fixes the issue described in liquibase bug 2939.

system-test

The system-test module, used for demonstration or testing purposes, comes with a built-in X.509 infrastructure. With this release, expired certificates are replaced by new ones.

Changelog 2.14.46..2.14.47

07e8ec485b
[maven-release-plugin] prepare release 2.14.47
77d71f072f
system-test: update disposable-CA generated credentials
a523fb322f
postgresql driver: update version to 9.4.1212
26e6beceb1
[maven-release-plugin] prepare for next development iteration

Release 2.14.46

chimera

There was an issue with a symbolic link to a directory where destination where destination contained trailing slash. This is now fixed.

Changelog 2.14.45..2.14.46

9fbbeb2
[maven-release-plugin] prepare release 2.14.46
0890408
[maven-release-plugin] prepare for next development iteration
c478efd
chimera : handle empty paths elements in path2inode stored procedure

Release 2.14.45

chimera

The current release fixed database query for storing multiple checksums for a file.

ftp

The Socket read method may return zero to indicate that no bytes were read. Although this is not an error, such occurances will result in a transfer failing.

This is now fixed.

Changelog 2.14.44..2.14.45

3e285f6
[maven-release-plugin] prepare release 2.14.45
7431759
ftp: prevent execution of most commands when unwrapped
5799780
chimera: fixed database query for storing multiple checksums for a file.
5116a7c
ftp: do not fail proxy transfer if read returns zero bytes
76250c3
[maven-release-plugin] prepare for next development iteration

Release 2.14.44

ftp

The current release added implementation of MLSC. As a result Globus is able to query the contents of dCache directories using FTP and without creating additional TCP connections.

Changelog 2.14.43..2.14.44

6e65fa8
[maven-release-plugin] prepare release 2.14.44
bc0bc02
ftp: implement the MLSC command
1943889
[maven-release-plugin] prepare for next development iteration

Release 2.14.43

ftp

The current release improves compatibility between dCache FTP client and Globus GridFTP server.

srm

During an ATLAS stress-test of tape recalls, it was discovered that various sites had relatively short request lifetimes. However, the SRM spec provides the opportunity for the server to inform the client (FTS, in this case) of what lifetime a request actually has. The current release includes the requests remaining lifetime in the response from the server.

The current release improves the documentation to help Admins to have a better understanding how to configure dCache correctly.

Changelog 2.14.42..2.14.43

810b42c
[maven-release-plugin] prepare release 2.14.43
f7cd009
ftp: add support for paths relative to home directory
8249c4c
ftp: Add support for SITE WHOAMI command
e6f1e5f
ftp: update parsing of CLIENTINFO command
d73e230
srm: include remaining request lifetime in various responses
de71502
srm: update srm request.*.lifetime configuration properties documentation
db2c488
ftp: modify facts describing namespace ownership
344c0b5
ftp: add support for SITE TASKID command
1d6fabb
ftp: add initial support for checksum performance markers
90880cb
ftp: show SIZE facts for directories
ca29a64
ftp: add support in OPTS RETR for specifying performance marker frequency
9f51f16
[maven-release-plugin] prepare for next development iteration

Release 2.14.42

xrootd

In https://github.com/xrootd/xrootd/issues/459, it became apparent that dCache could improve xrdcp compatibility by sending checksum information in lower case. This release contains this change, which should improve xrootd operations.

Changelog 2.14.41..2.14.42

5c18fda
[maven-release-plugin] prepare release 2.14.42
55f8cbd
xrootd : use lower case for checksum algorithm names when replying to checksum queries.
6e7c89c
[maven-release-plugin] prepare for next development iteration

Release 2.14.41

srm

The SRM code has been made more robust against races between file deletions and copies.

systemtest

The ‘system-test’ script was updated to ensure anonymous dcap tests succeed.

Changelog 2.14.40..2.14.41

4158477
[maven-release-plugin] prepare release 2.14.41
6c40d2f
systemtest: allow anonymous dcap activity
5a1279f
srm: fix recovery procedure in internal copy if source is deleted
02430b6
[maven-release-plugin] prepare for next development iteration

Release 2.14.40

cleaner

Users reported that they wanted to see space freed up by cleaner processes to be reported as free as soon as possible. This patch sends notifications about freed up space more often, resulting in quicker status updates.

pool

A problem was fixed that could cause the csm check to fail on pools containing broken files.

Changelog 2.14.39..2.14.40

4146bf6
[maven-release-plugin] prepare release 2.14.40
60764e5
cleaner: Send notification more often
02e9d86
pool: Fix csm check command in the pressence of broken files
124fb56
[maven-release-plugin] prepare for next development iteration

Release 2.14.39

ftpclient

The current release improves compatibility between dCache FTP client and Globus GridFTP server.

Changelog 2.14.38..2.14.39

f04f1e7
[maven-release-plugin] prepare release 2.14.39
be1932c
ftpclient: fix multiline ADAT reponses
fa93188
ftpclient: encrypt SITE CLIENTINFO command
6b2cf9b
[maven-release-plugin] prepare for next development iteration

Release 2.14.38

dcap

Connections of non-DCAP clients to a dCache no longer result in stack-traces in the logs.

poolmanager

PoolManager was updated to properly handle the dcache.authz.staging.pep and dcache.authz.staging parameters. This allows to enable stage protection properly.

Changelog 2.14.37..2.14.38

d1c87a9
[maven-release-plugin] prepare release 2.14.38
3aeb652
dcap: don’t create stack-trace if tunnel fails due to bad client
051f57f
PoolManager : stage protection, fix error in stage.fragment
fa39df6
[maven-release-plugin] prepare for next development iteration

Release 2.14.37

Changes affecting multiple services

Internal notification processing between cleaners and Pin Manager, Replica Manager and Space Manager was improved and runs quicker now.

admin

A bug in the Admin service that caused it to not detect certain error replies from other components was fixed.

billing

If a dCache instance was shut down while the billing service was in the middle of a refresh, an exception was logged and shutdown was delayed. A change in exception handling fixes this rare scenario, ensuring a quick shutdown and no unnecessary log entries.

dcap

This release makes it possible for admins to ban outdated, problematic versions of dcap clients. Some old client versions contain a bug that causes the client to make unsatisfiable requests to a pool with no way for dCache to reject the request: the client will simply retry.

The client version limits are exposed using the new configuration property dcap.limits.client-version. The default is to allow all dcap client versions, unchanged from the previous behaviour.

This release fixes a regression through which Kerberos dcap would not work for host principals containing a ‘-’ character. GSI dcap was not affected.

doors

Fixed a bug in the lb set tags command in doors that prevented setting an empty list of tags.

pool

A bug that caused non-critical stack traces to be logged on the pool after stage or deletion failure from nearline storage has been fixed.

Changelog 2.14.36..2.14.37

48ab090
[maven-release-plugin] prepare release 2.14.37
bbed67b
dcap: expose dcap client version limit
10e2317
dcap: fix Kerberos dcap if principal contains a ‘-’
3799546
dcap: fix regression in handling old version
1c2d2c5
pool: Suppress two stack traces in nearline storage handling
5dc425d
cleaner: Send notifications concurrently
fe023c9
billing: fix stacktrace and slow shutdown if in refresh
bd28892
doors: Allow setting an empty list of login broker tags
77cdddf
[maven-release-plugin] prepare for next development iteration
307e556
dcache: Fix detection of message errors in admin

Release 2.14.36

doors

Fixed a bug in which information on stage pool and number of attempts were lost when retrying pool selection requests.

Changelog 2.14.35..2.14.36

aa669de
[maven-release-plugin] prepare release 2.14.36
9a52fdc
doors: Ensure that pool selection context survives between retries
cfe45ad
[maven-release-plugin] prepare for next development iteration

Release 2.14.35

billing

Fixed a problem in the output of the dcache billing command using JSON or YAML when the billing format includes a custom date format.

dcap

Add support for the dcap client supplying additional version information.

gplazma

Fix explain login and test login commands so they are able to test logging in with username and password.

Add examples to explain login command help. Our thanks to Onno Zweers for this change.

httpd

Fix regression in the transfers.txt output format.

Update the transfers.html page so it no longer includes <unknown> for default/unknown protocols. With this version of dCache, these are represented by a ? character. The webadmin page is updated to give consistent output.

scripts

Fix the billing indexer to ignore the format string, if present.

Changelog 2.14.34..2.14.35

b513ad0
[maven-release-plugin] prepare release 2.14.35
96b772a
Active Transfers: substitute ? for <unknown> on html pages
ec8a1ee
common: add support for UserNamePrincipal as user:<name>
0a4af22
Added ‘explain login’ examples to help text in Gplazma2LoginStrategy.java
a869f1c
billing: Strip format string from attribute name
895b33e
transferObserverV1: replace Args with Joiner to construct transfers.txt linesMotivation:
a0928a3
billing: Make billing indexer work with custom format strings
9a3c997
[maven-release-plugin] prepare for next development iteration
f69b830
dcap: add support for clients presenting more version metadata

Release 2.14.34

commons

If dCache’s internal ShellApplication framework detects a critical behaviour that might indicate a bug in the application, error messages now include an explicit request to send a mail to the developers and more relevant information for assessing and reproducing the situation.

gplazma2

This release fixes a small bug in GPlazma which inappropriately tried to handle non-DN subjects in x509 certificates. These will usually fail in gPlazma anyway, but the reported error was confusing.

srm

Handling of DNS names without trailing dots in certificates has been made more robust and universal.

Changelog 2.14.33..2.14.34

fc15abe
[maven-release-plugin] prepare release 2.14.34
a70b77c
commons: log bugs with stack-trace and instructions
6f5294f
gplazma2-xacml: remove erroneous creation of placeholder extensions
014ce27
[maven-release-plugin] prepare for next development iteration
849c1b8
srm: remove trailing dot from reverse lookup result

Release 2.14.33

cells

In rare cases, an interrupt needed to cleanly shut down the location manager connector would not arrive. This issue was corrected, ensuring more reliable behaviour on cell shutdown.

Fixed a problem in which threads could inappropriately be created as daemon threads, causing problems in killing those threads when the cell shuts down.

Changelog 2.14.32..2.14.33

1d86940
[maven-release-plugin] prepare release 2.14.33
14d0735
cells: Fix lost interrupt exception
c045712
cells: Ensure that newly created threads are non-daemon normal priority threads
8513e45
[maven-release-plugin] prepare for next development iteration

Release 2.14.32

commons

The \s admin command uses the toString method to serialise the requested so that the remote cell may correctly parse it. This did not always work: ‘=’ characters in arguments were escaped but not unescaped; arguments that start with a ‘-’ character were not escaped; empty words were lost. In the current release this is fixed and \s command works as expected.

dcache

The current release fixes a regression in which the exit code of check-config would always be zero even when errors were detected.

Changelog 2.14.31..2.14.32

3f9b3d3
[maven-release-plugin] prepare release 2.14.32
b74342f
dcache: Generate proper exit code for check-config command
9336cda
commons: fix Args string parsing and toString method
f0a6386
[maven-release-plugin] prepare for next development iteration

Release 2.14.31

alarms

Alarm email notifications are now sent only on the first occurrence of given alarm (i.e., for that alarm instance’s unique ID).

If an alarm has been closed and not deleted, but then occurs again, the counter for receiving that alarm is now reset to 1, in order to treat this as a new (set of) occurrences, and to guarantee a new notification will be sent.

webadmin

Due to an implementation detail in a library used for the webadmin pages, filtering tables was a bit unintuitive until now: A filter that was set in a certain table column would reappear on tables in other browser windows if they had similar columns.

This behaviour was corrected, and tables on different pages exposed simultaneously in different browser tabs are now filtered independently. However, the fix also has the side effect that now with page reloads and form submissions the filters are cleared. Any commands, however, will always be issued correctly.

Filtering rows in webadmin tables could occasionally lead to unintuitive behaviour with regard to selections: Filtering a table and hiding rows may be included in a “select all” or “deselect all” operation. This was fixed, and selection/Deselection of hidden rows is now prevented.

Issues with filter boxes disappearing or filters resetting have been solved by disabling AJAX auto refresh for the affected pages.

Changelog 2.14.30..2.14.31

65df3fc
[maven-release-plugin] prepare release 2.14.31
1c40ccd
dcache-webadmin: synchronize client-side filtering with server-side selection of rows on pages using picnet table filters
e243006
alarms: reset count history on reopened alarm
bd728db
dcache-webadmin: disable saving table filter settings to browser cookies
3075b49
dcache-webadmin: disable AJAX autorefresh on pages using picnet table filter library
ba3523e
alarms: only send email on first alarm occurrence
37f6a13
[maven-release-plugin] prepare for next development iteration

Release 2.14.30

chimera

Chimera occasionally suffered from (operationally irrelevant) IllegalStateExceptions. Those are now avoided.

doors

Doors could get stuck temporarily if a file was deleted during pool selection. This has been fixed, and in such cases, transfers are now aborted properly.

Changelog 2.14.29..2.14.30

1aa2fcb
[maven-release-plugin] prepare release 2.14.30
15daf6d
doors: Abort transfer if file is deleted during pool selection
726b7a8
chimera: Fix IllegalStateException in inode cache
08823f2
[maven-release-plugin] prepare for next development iteration

Release 2.14.29

billing

When using the dcache billing command with a non-default date format in the billing file, an unneccessary stack trace was printed. This has been corrected.

srm

The SRM should periodically (by default every 10 minutes) delete obsolete historic data (older than 10 days by default) from the database. For cases where there are problems with that process, error logging and robustness against temporary database problems have been improved.

webadmin

The Wicket library used by dCache internally issued warnings about upcoming naming changes. Those cluttered the log files, and are silenced from the current version on.

Changelog 2.14.28..2.14.29

acc657b
[maven-release-plugin] prepare release 2.14.29
aafe0bd
srm: make out-of-date historic data deletion more robust
189dbd0
webadmin: silence warning about future change in wicket
745d1bc
[maven-release-plugin] prepare for next development iteration
fd1d0a8
billing: Removing erroneous stack trace output

Release 2.14.28

ftp

The Apache Commons FtpClient can issue the LIST command with the non-standard -a option. Which was causing dCache to switch output format from the long (ls -l-like) to the short (ls-like) response. This is fixed now and dCache is more compatible with Apache Commons FtpClient.

Changelog 2.14.27..2.14.28

dfd09d0
[maven-release-plugin] prepare release 2.14.28
9af35c6
ftp: improve compatibility with Apache Commons FtpClient
4c17cd6
[maven-release-plugin] prepare for next development iteration

Release 2.14.27

cells

If the create command in CellShell fails because of unreadable setup files, it throws an IOException. This was incorrectly reported as a bug. Reporting has been corrected now.

In rare cases, active transfers would show up with an incorrect state in the active transfers page of the admin backend. This was fixed, so that ransfers which are staging from non-DCAP doors are correctly indicated (in yellow) on the active transfers page, instead of showing up as “No Mover found” (in red).

common

If there is an IOException when trying to read a setup file, the corresponding file name is now listed in the error message.

poolmanager

An issue with PoolManager prevented it from delivering correct cost estimates. This was fixed, resulting in improved estimations of pool load.

Changelog 2.14.26..2.14.27

23263d1
[maven-release-plugin] prepare release 2.14.27
a61373e
cells: IOException is not a bug in create command
8bf50db
common: include filename in error message
f0d2757
poolmanager: Fix incorrect correction of pool cost
3e656bf
cells: handle empty string pool value on staging in TransferObserver
13bd117
[maven-release-plugin] prepare for next development iteration

Release 2.14.26

alarms

Log entries that were promoted to alarm status and that show up in the webadmin table can now contain more detailed information.

Changelog 2.14.25..2.14.26

05895da
[maven-release-plugin] prepare release 2.14.26
9cbee43
alarms: add ndc info to alarm info
9f2b5f0
[maven-release-plugin] prepare for next development iteration

Release 2.14.25

gplazma2-argus

Fixed a problem with the gPlazma argus plugin that caused it to fail with a ClassNotFoundException.

Changelog 2.14.24..2.14.25

bafb5a4
[maven-release-plugin] prepare release 2.14.25
03ec581
gplazma2-argus: Update to Argus client 2.2.0 to fix dependency on VOMS library
2478dd6
[maven-release-plugin] prepare for next development iteration

Release 2.14.24

alarms

A change to the alarms system improves handling of alarms with unset types.

nfs

A race condition in the NFS door that could result in the creation of multiple inconsistent copies of a file being uploaded has been fixed.

pool

A regression was fixed that caused the jtm go command to occasionally not work.

When a transfer’s status is queried before the transfer is initiated, which can occasionally happen for queued requests, Exceptions were logged. This behaviour has now been corrected, providing more robust operation.

Several race conditions in the pool’s migration module are fixed now.

A regression prevented queues to be set to not handle any jobs at all. The fix allows to pass a limit of 0 to mover set max active.

A regression was fixed that caused pools to “leak” movers if those were cancelled while still being queued.

spacemanager

Fixes a compatibility problem with NFS in which space manager would fail with a duplicate key error.

srm

A hint to describe the necessity to include escaping has been added.

The current release fixes issues in which the use of SRM third party copy operations could cause the SRM cell to become unresponsive, possibly even run out of memory.

Changelog 2.14.23..2.14.24

912669a
[maven-release-plugin] prepare release 2.14.24
a7c5866
srm: add hint to escape IDs
45c4db9
nfs: Fix race condition in transfer startup
17e0259
info: fix broken unit-test
15aa4b1
srm: Resolve message thead blocking issues with SRM third party copy
7d3d568
spacemanager: Work around for doors resubmitting PoolAcceptFileMessage
f67c8a0
pool: Fix several race conditions in migration module
5874750
pool: Fix regression in mover set max active command
79cfa23
pool: Fix mover leak
9042ba0
pool: Fix synchronization regression in jtm
e48472c
pool: avoid NPE when querying status of a 3rd-party HTTP transfer
2d97cdf
alarms: fix NPE in type setter
b7ebe35
[maven-release-plugin] prepare for next development iteration

Release 2.14.23

admin

The admin door now generates SSH keys to ensure compatibility with OpenSSH 7. Additionally, a new property admin.paths.host-keys was introduced in the admin.properties file, allowing to specify the location of keys.

script

When a pool’s metadata conversion operations would fail, an error caused a script to report successful conversions. This error has been fixed now.

Changelog 2.14.22..2.14.23

f308728
[maven-release-plugin] prepare release 2.14.23
9c37845
admin: Fix compatibility with OpenSSH 7
a422596
script: Do not claim success if meta data conversion failed
2008471
[maven-release-plugin] prepare for next development iteration

Release 2.14.22

admin

First observed on Ubuntu Xenial, dCache fails to install on modern Linux distributions due to the short key length of the SSH 1 keys generated in the post install script. This patch removes those keys and their generation code. dCache has been supporting only modern key formats for quite some time now, so this change should not have any impact on users.

pool

When creating movers, some error conditions are expected to occur and dCache is designed to transparently recover from these. Consequently, this patch lowers the log level for the related error messages to reflect that their causes are harmless.

Fixed a staging problem that would lead to failures in nearline COPY operations.

srm

Some race conditions during SRM startup were fixed. Those race conditions could potentially have lead to failures to expire jobs and to wrong job counts in the SRM schedulers.

webdav

Until now, trying to access a file for which the client was not authorized would generate a reply with a status code 200 OK, but an empty body, rather than an error page. This patch corrects that behaviour and also improves exception handling for that case.

Changelog 2.14.21..2.14.22

1472172
[maven-release-plugin] prepare release 2.14.22
ae318fc
admin: Drop old ssh 1 keys
4831b01
pool: Lower log level of certain failures to create mover
9eb8dd0
pool: fix staging for CopyNearlineStorage
a890967
webdav: Fix error reporting when client is unauthorized
e7135f5
srm: Fix job expiration during service startup
ab3a2dd
[maven-release-plugin] prepare for next development iteration

Release 2.14.21

billing

The data used to create the 24 hour billing overviews is aggregated in hourly intervals before creating the plots. However, if there is very high activity on the system during an entire 24 hour period, there have occasionally been timeouts when querying the database for this aggregate data. This patch makes the data aggregation more robust against such situations, resulting in lower latency for histogram generation and no more timeouts.

core

The Online Certificate Status Protocol (OCSP) is used to query status information about certificates during authentication. dCache supports this protocol, but relies on a functioning OCSP server for it to work properly. This patch changes the default OCSP mode for dCache to IGNORE, effectively disabling it, which is helpful for sites without a working OCSP server in place.

gplazma

Previously, attempts to authenticate users against an htpasswd entry that was malformed resulted in a stack trace. This patch modifies the error handling so that only a detailed error message (“Bad entry in file: hash does not start ‘\(1\)’ or ’\(apr1\)”) is logged.

Changelog 2.14.20..2.14.21

340638c
[maven-release-plugin] prepare release 2.14.21
907a59e
(2.14) billing: use in-memory buffer for hourly aggregate data
f25df85
Disable OCSP by default
53c3804
[maven-release-plugin] prepare for next development iteration
ea3120a
gplazma: don’t generate a stack-trace if htaccess is malformed

Release 2.14.20

Changes affecting multiple services

Fixed an issue with the dcache heap dump command when called with a simple file name as the output path. In this case the dump could in some cases be written to a different directory while the script claimed the dump had failed. The dcache dump heap command has a --force option for cases in which the JVM is unresponsive. This option was ignored for processes not running as root. This is fixed now.

cells

Fixed a problem causing FTP and DCAP per connection instances to subscribe to topics they should not subscribe to. This reduces overhead caused by routing updates.

Fixes a problem during shutdown in which communication tunnels between domains were shut down too early.

Fixed an issue that would cause log messages in which placeholders had not been replaced with actual values.

A bouncing message bug in System cell is fixed.

pnfsmanager

Setting atime-gap to –1 (default value) should disable file’s last access time updates. Nevertheless, this was not the case and atime update was always enabled. This is fixed now and file’s last access time can be disabled as described in the documentation.

pool

Fixes an issue with pools becoming unresponsive in case of slow DNS reverse lookups.

Fix race condition in request scheduler.

If FTP clients disconnect mid-transfer, pools log a DoorTransferFinished delivery failure as the door is gone. This is fixed now and log messages like Failed to deliver DoorTransferFinishedMessage message are suppressed in pools. When cancelling a job in a state that doesn’t allow cancellation, an illegal state exception is thrown. This was logged as a bug. This is now fixed.

Pool to pool transfers are supposed to be cancellable, but was not working as the HttpURLConnection does not appear to react to thread interrupt. This could lead to migration job and rebalance job cancellation appearing to hang. This is now fixed.

Fixes several performance regressions in pools that reduced mover creation rate and could cause pools to become unresponsive due to lock contention.

srm

Fixed a bug that caused delivery failures of credential service announcements to be logged. The ls -completed=n command has been observed to fail with SRMInvalidRequestException. This is fixed now and the output format of listing list requests has been changed to match that of other requests.

Changelog 2.14.19..2.14.20

84328f0
[maven-release-plugin] prepare release 2.14.20
f92cdcc
srm: Fix listing of completed list requests
2aeec7f
dcache: fix heap dump to simple file names
786ebc6
script: Make dump heap –force work for non-root processes
0f82952
script: Add missing she-bang
c22210d
pool: Fix p2p cancellation
dc08cb7
srm: Do not expose TURL before request is ready
0cebf53
pool: Don’t log illegal state exception on migration job cancellation as a bug
d37fde7
pool: Reduce lock contention on mover creation
435cc50
pool: Fix race condition in request scheduler
0d50cd3
pool: Avoid reverse DNS lookup in HTTP mover
159eea7
billing: additional fixes to insert triggers
ec3ac5d
pool: Suppress logging of delivery failure of DoorTransferFinished
e782584
system-test: update disposable-CA generated credentials
a34cfee
pnfsmanager: fix atime update regression
b7aef46
cells: Fix logging formatting string
91fc2e0
cells: Avoid bouncing message on no-route errors in System cell
ec75096
srm: Suppress message delivery failures for credential service announcements
3559ff9
cells: Fix tunnel shutdown order
07350e6
cells: Do not subscribe to topics in per-session door instances
b403160
[maven-release-plugin] prepare for next development iteration

Release 2.14.19

chimera

An internal database trigger was updated to insert data in the correct table, fixing a problem with the Enstore client.

When accessing a file for reading, the atime value must be updated. Previously, due to an error, the ctime (intended to reflect the time of changes to file attributes) was also changed. This update corrects that problem.

http

In order to increase the performance of the Billing system, reverse DNS lookups were removed from the code. While this will result in IP addresses representing hosts in the billing file, DNS performance no longer impacts overall system performance.

many

When representing checksums in the admin interface and configuration files, checksums are now presented in an improved format.

pool

The nearline storage subsystem uses thread pools to manage its workload. Since some tasks are blocking, very high activity can cause these thread pools to grow beyond effective sizes. This may even lead to the pool becoming unresponsive.

This change introduces a new configuration property, “pool.limits.nearline-threads”, which limits the thread pool size. The default value, 30, is chosen to be sufficient for almost all imaginable use cases while at the same time avoiding potential problems with resource exhaustion.

srm

Due to a timing issue, an initial service announcement in the SRM was sent before any listeners could register for those announcements. Thus, upon startup, a delivery error would be logged. With this patch, sending of the initial message is delayed until after the registration of listeners, and the irrelevant error messages are avoided.

Changelog 2.14.18..2.14.19

f27d953
[maven-release-plugin] prepare release 2.14.19
a10ce6c
chimera : fix trigger that populates data in t_locationinfo and t_inodes on insert or update of t_level_4
cc07d28
common: fix ChecksumType.toString()
c80ef7e
http: avoid dns reverse lookup on HttpProtocolInfo#toString()
3343fde
chimera: do not update ctime on atime only attribute update
e411aae
[maven-release-plugin] prepare for next development iteration
d101c0d
srm: Delay announcing credential service after cell start
d76de7f
pool: Improve scalability of nearline storage subsystem

Release 2.14.18

doors

Fixes a bug in which a host name set in *.net.listen properties was not preserved when publishing a door or generating SURLs.

Changes affecting multiple services

When building rpm files a package which is now explicitly required as a dependency.

cells

This change fixes a bug in routing manager that would leave orphaned topic routes in dCache domain.

pool

The nearline storage subsystem has a thread pool for various tasks. Some of these tasks are blocking. Since the thread pool is unlimited, a high inflow of new requests can cause the thread pool to grow rapidly and even exceed thread limitations. In that case the pool dies. In the current release a new pool.limits.nearline-threads property is introduced which allow to limit the number of threads used by the nearline storage subsystem. Note that, the default is 30 threads.

Changelog 2.14.17..2.14.18

e03110e
[maven-release-plugin] prepare release 2.14.18
00ce67f
pool: Fix regression breaking hopping mananger
b43a335
cells: Fix route removal in routing manager
488d51f
rpm: explicitly require which package
9cd14bc
doors: Preserve name when publishing the address of doors
0d73d58
[maven-release-plugin] prepare for next development iteration

Release 2.14.17

cells

LoginManager would occasionally generate error messages similar to “Discarding listening on $LOCATION 53684’ because its age of 18721640 ms exceeds its time to live of 4500 ms.”. This was due to erroneous reuse of old message envelopes in the internal messaging. This change fixes that problem.

This change addresses a potential problem in which messages sent between cells in the same domain could appear older than they are and thus would risk being discarded due to the time-to-live being expired.

Contains corrections to cells logging. The routing manager pinboard now shows information previously logged to various other cells.

nfs

This change avoids ERR_PERM errors on NFS writes in situations where NFS doors do not receive redirect messages within the allowed timeframe.

pool

This patch avoids errors occuring when mover doors would not receive a start mover request within the allowed timeframe, providing more robustness on heavily loaded instances.

srm

A Tier–1 site reported problems with a major WLCG VO’s read requests. Investigating the source of the problems showed that the srm_ifce library, used by the (outdated) GFAL v1 and the (supported) GFAL v2 SRM libraries, drastically limits the permitted lifetime of requests without providing admins any way to configure this.

For sites seeing errors related to desiredTotalRequestTime being exceeded, this change provides the new configuration option srm.request.maximum-client-assumed-bandwidth in srm.properties as a work-around.

Sites not observing such errors do not need to change anything with regard to this value.

Changelog 2.14.16..2.14.17

441214c
[maven-release-plugin] prepare release 2.14.17
b13fa36
srm: add short request lifetime work-around
99bb9c0
cells: Set correct logging context in cell callbacks
317c4af
cells: Improve robustness of message time to live
1284b71
cells: Fix erroneous reuse of message envelope in location manager registration
1827932
nfs: try to re-use transfer class on client retry
0f6d8a6
pool: handle duplicated start mover requests
a8d4ca0
[maven-release-plugin] prepare for next development iteration

Release 2.14.16

chimera

Fix a regression causing directories to inherit ACLs as if they were files rather than directories. Soon we will provide a procedure to clean-up already existing wrongly inherited directories.

Changelog 2.14.15..2.14.16

7b3dc90
[maven-release-plugin] prepare release 2.14.16
189fd27
chimera: Fix regression in inheriting ACLs on directory creation
75a76a4
[maven-release-plugin] prepare for next development iteration

Release 2.14.15

ftp

On very short transfers, two internal messages could occasionally arrive in the wrong order. This would cause clients to see the “226 Transfer complete.” message without the “150 Opening BINARY data connection for” immediate reply. This rare issue is now fixed.

pnfsmanager

Billing entries for SRM uploads recently lost the storage class part of the entry. This update fixes that issue. We observed an error caused by the parallel execution of two uploads, both trying to create the same (previously non-existing) directory). This modification fixes the underlying race condition, allowing such transfers to succeed.

pool

Fixed a regression introduced in 2.13 in which internal options like -c:puts were erroneously included to the call out to the HSM script.

poolmanager

This modification fixes a potential race condition in pool manager.

This modification fixes a race condition in pool manager that could theoretically have provided erroneous data to pin manager, space manager, srm, xrootd and webdav. This modification fixes a race condition in pool manager that could theoretically have provided erroneous data to pin manager, space manager, srm, xrootd and webdav.

scripts

The ctlcluster utility is now fully supported under Linux.

srm

When writing to a path /a/b/c, if b exists and is a file, SRM currently returns SRM_INTERNAL_ERROR. This modification changes that behaviour so that the more appropriate status SRM_INVALID_PATH is returned. During SRM-based operations, some sites reported problems with the delegation of user credentials. This modification remedies those problems, while at the same time reducing the CPU usage of establishing 3rd-party SRM connections.

webdav

This modification corrects the error reporting under WebDAV. When attempting to delete a non-existing file, unauthenticated users receive a 401 Unauthorized response, while authenticated users receive a 404 Not Found response.

Changelog 2.14.14..2.14.15

0e8915d
[maven-release-plugin] prepare release 2.14.15
3026bfd
pnfsmanager: Fix regression in SRM billing entries
c2166d9
pool: Fix filtering of options in script HSM driver
3919bb1
pnfsmanager: Fix race leading to transaction failures in Chimera
7340b3e
srm: Disable delegation on srmCopy to or from other SRMs
4eb54e9
srm: Return SRM_INVALID_PATH when target directory is a file
7e848e5
ftp: Fix race on short transfers
c4c7f53
[maven-release-plugin] prepare for next development iteration
ae8a0ea
Fix compatibility issues with ctlcluster
cc6aa1f
poolmanager: Acquire read lock when serializing cost module and partition manager
63f8fa5
poolmanager: Acquire read lock when serializing pool selection unit
e46f168
poolmanager: Fix race condition in pool selection unit
c90dcc1
webdav: fix 404 error if attempting to delete a nonexistent file

Release 2.14.14

doors

Fixes a race condition and a responsiveness issue in doors.

spacemanager

Fixes a race condition during space manager startup that could lead to log messages about failed link group updates and failed transfers.

Changelog 2.14.13..2.14.14

d32ea47
[maven-release-plugin] prepare release 2.14.14
ed94389
spacemanager: Fix pool monitor fetch race during startup
84fac07
doors: Fix contention point and race in login broker publishing
6537ac6
[maven-release-plugin] prepare for next development iteration

Release 2.14.13

pnfsmanager

With this release, PnfsManager adds safety checks rejecting invalid upload paths that SRM might erroneously supply. This release hardens an installation against possible bugs triggering data loss.

This release adds a check that detects failed or incomplete SRM uploads and prevents the file from being committed to its final path. Common symptoms of this bug were zero sized files that experiment catalogues registered as successfully uploaded.

pool

This patch improves support for HTTP Keep-Alive connections. On closing a connection, in accordance with RFC 2616, an HTTP header indicating closure of the connection is sent out.

On highly loaded systems, DNS slowdowns could lead to pools and/or domains becoming unresponsive. This fix increases resilience against DNS problems.

Increased performance and resilience in situations where NFS clients send many requests at once.

srm

This release adds a check to detect broken uploads using SRM during the final stage of file transmission. While it causes transfers to take a little more time, resilience against upload failures is increased.

webdav

dCache has supported 3rd-party HTTP push requests for WebDAV for some time. This release adds support for 3rd-party HTTP pull requests.

Changelog 2.14.12..2.14.13

c0a3ec5
[maven-release-plugin] prepare release 2.14.13
8762d5e
pool: send connection header when closing a connection.
410ef0f
pool: Avoid lock contention on DNS lookup in p2p component
7186416
srm: Check for broken files during srmPutDone
7a6f5a6
pnfsmanager: Check file size and upload completion when committing temporary upload paths
6084a86
pnfsmanager: Protect against erroneous upload paths
ab69eac
pool: use “same thread strategy” for nfs movers
d43f045
webdav: add support for 3rd-party HTTP pull
ec8ce92
[maven-release-plugin] prepare for next development iteration

Release 2.14.12

Changes affecting multiple services

Several cases of slow performance were reported while deleting directory in Chimera. This is now fixed.

pool

dCache pool yaml command dumps the meta data of a pool using the human an machine readable YAML format. A regression causing a null pointer exception has been fixed.

When command execution to migrate files between pools (e.g. migration concurrency or migration copy) is interrupted due to the failure to find migration job the returned error message is considered as a bug. This is now fixed so that a new message is returned indicating that the job being requested does not exist.

srm

When file upload is cancelled the value of temporary upload path tracked by SRM could be a value different from a regular path, either because it was changed outside of dCache, or it contains entries from a very old version of dCache. This could result in data loss while canceling upload. The current release fixed a potential data loss scenario.

statistics

The statistics service creates static HTML pages that describe dCache usage over time as simple files that the webadmin service can serve. This includes information about pools and store-units. The problem is that the statistics webpages do not show information about any pool or store-unit that contains a / in the name. This is now fixed. A side-effect is that the history of any pool or store-unit containing a ^ in the name is lost.

xrootd

In XRootd Protocol Specification kXR_dirlist request is used to list the contents of a directory. XRootd version 3.0.0 introduced the kXR_dstat option to the kXR_dirlist command which returns stat information for each entry. The new release adds support for kXR_dstat in dCache. Note that for VOs that rely on token authorization, this is a requirement for functional verbose listing.

Changelog 2.14.11..2.14.12

a5fc47c
[maven-release-plugin] prepare release 2.14.12
9a67073
xrootd: Add kXR_dstat support
c2e7620
chimera: Alter statistics target for t_tags(itagid)
f41c5c2
srm: Add safe-guard against invalid file ID in put requests
0a10e60
pool: Don’t consider failure to find migration job a bug
fe3bc12
statistics: encode ‘/’ in filenames
53d31a4
[maven-release-plugin] prepare for next development iteration
07d5b22
pool: Fix NPE in pool yaml tool

Release 2.14.11

alarms

On heavily loaded systems creating alarms at very high frequencies and with comparatively slow databases for alarm logging, components ran out of memory and blocked. This release introduces a mechanism that will drop alarm messages that would cause the system to fail, ensuring continued operation. Note that no messages that are critical to system operation will ever be dropped by this change.

nfs

Pinning files is now a non-blocking operation. For files stored on tape, this should result in a more responsive system behaviour, avoiding NFS blocking in situations with many concurrent pin requests.

Changelog 2.14.10..2.14.11

1e2eb2c
[maven-release-plugin] prepare release 2.14.11
cdee225
(2.14) alarms: change executor to have bounded queue and discard events on overrun
755f85c
nfs: use noitify instead of blocking sendAndWait when sending pin/unpin messages via touch “.(get)(<file_name>)(pin)” command
7868f81
[maven-release-plugin] prepare for next development iteration

Release 2.14.10

Changes affecting multiple services

Sometimes when a cell start up was interrupted an error message was logged as a bug. This is now fixed.

info-provider

The GLUE infomation provider supplies information about the dCache instance, which is important for the clients in WLCG area. Because in dCache different doors can have different roots, clients may need to adjust their path when accessing dCache through different doors. The info-provider is updated so that a new path root property is provided. This allows clients to modify paths, as necessary. Note that the SRM door already supports this translation when redirecting clients for transfers.

pool

Transfers into or out of dCache that do not go through the client (“3rd-party transfers”) may be initiated through srm and webdav doors. This release fixes allows https transfers without any X.509 credential.

Changelog 2.14.9..2.14.10

174e0bd
[maven-release-plugin] prepare release 2.14.10
d67f620
info-provider: publish door root path
84acc3c
cells: Suppress illegal state exception during initialization
eda4a70
pool: fix HTTPS third-party transfers without X.509 credential
34703bf
[maven-release-plugin] prepare for next development iteration

Release 2.14.9

nfs

This release includes the PNFS-ID in the door’s logging output when proxying data movement.

Fix infinite loop for a specific failure mode when writing data into dCache: with this release, dCache will indicate a permanent error.

spacemanager

Spacemanager backs off when it encounters a problem writing to the database. Previously, if the problem was due to deadlocks then the two tasks involved are delayed by the same amount, which means it is possible that subsequent attempt will also deadlock. This release randomises the delay to reduce the likelihood of this problem occuring.

Changelog 2.14.8..2.14.9

0fd5707
[maven-release-plugin] prepare release 2.14.9
1c93e26
spacemanager: Randomize backoff in case of transient errors
bb84805
nfs-proxy: include file’s pnfsid into debug context
70403d9
nfs: convert FILE_IN_CACHE into NFS_EIO
27d3f09
[maven-release-plugin] prepare for next development iteration

Release 2.14.8

pool

Fix buffer leak in the HTTP mover.

Improve scalability of pools when opening files or changing sticky flags.

Resolve problem where starting a migration job on a pool with many files can result in an unresponsive pool.

This release fixes a bug that caused the output of the rep ls -s admin command to contain negative values.

Improve pool scalability of pool, allowing it to remain responsive even when undertaking many pool-to-pool transfers.

xrootd

The prevelance of old versions (3.x) of xrootd client along with that version’s unfortunate behaviour when replying asynchronously has forced this release to revert the asychronous response to open requests.

Fix dCache handling of open requests where uploads were considered downloads.

Changelog 2.14.7..2.14.8

e7f2ff1
[maven-release-plugin] prepare release 2.14.8
d7abddf
pool: Fix lock contention during heavy p2p activity
aa3cad2
Revert “pool: Expose Berkeley DB configuration as dCache properties”
2e068a5
pool: Fix buffer leak in HTTP mover
c0f8ae6
xrootd: Fix classification of uploads
4f777e5
pool: Expose Berkeley DB configuration as dCache properties
966ca5a
xrootd: Roll back asynchronous reply on open
935649a
pool: Fix lock starvation in migration module
6ba57be
pool: Fix accounting error in repository statistics
0a48e12
pool: Avoid lock contention when opening files and setting sticky flags
063afcc
pool: Simplify synchronization during repository setup
51a05f6
[maven-release-plugin] prepare for next development iteration

Release 2.14.7

Changes affecting multiple services

Don’t log Error while reading from tunnel: java.nio.channels.AsynchronousCloseException when a domain shuts down.

cleaner

Fix cleaner so it no longer sends cleaning requests to pools it knows cannot enact these requests. This prevents the cleaner from spamming a disabled pool.

pool

No longer spam log file with 666:"Client disconnected without closing file." when an xrootd client closes a file it was reading. This is still logged if the client fails to close a file opened for writing.

Improve pool startup time.

This release provides more robust monitoring of a pool’s health by periodically testing the data directory in addition to testing the metadata.

With this release, the pool provides a more robust behaviour when the client requests a file without being redirected from a door. The pool will redirect the client back to its previously used door, if known. Additionally, the pool no longer logs when this happens in the pool log file, but only in the pinboard.

This release fixes buffer leaks in the xrootd mover. It also removes endsess spam from pool log files.

The pool’s info output now includes a progress report during initialisation.

Changelog 2.14.6..2.14.7

a3466a5
[maven-release-plugin] prepare release 2.14.7
ab3dab0
Fix broken pull-request PR:2096
9aedb72
pool: Upgrade xrootd4j to 3.0.2
aebd82c
pool: Redirect xrootd client to door on failure to open file
6039b7d
pool: Do not fail read if xrootd client doesn’t close file
849f59f
cleaner: Fix black listing in case of disabled pools
77ee337
cells: Don’t log AsynchronousCloseException when tunnel closes
923c13d
pool: Show progress during repository initialization
e7b1a00
pool: Fix health check of file store
8981994
pool: Use disk ordered cursor for listing meta data on pool startup
92d2531
[maven-release-plugin] prepare for next development iteration

Release 2.14.6

Changes affecting multiple services

Update the Spring, Milton, AspectJ, Jetty and DataNucleus-core libraries to latest version. All dCache services are affected.

gplazma

Fix debug logging of the voms gPlazma plugin so it shows a clear and complete description of which FQAN is mapped; previously the log would contain confusing entries if the first FQAN is not mapped exactly.

pool

If a 3rd-party transfer fails then the pool may log and report incomplete information on why this happened. This release fixes this problem.

On startup, a pool will scan its stored files and associated metadata to ensure consistent and accurate information. As this can take some time, with this release the pool will log how long these scans take to provide objective information when optimising this process.

srm

This release fixes the interpretation of srm.persistence.enable.store-transient-state when srm.persistence.enable.history is disabled. Sites that have history disabled may put additional load on the SRM database if store-transient-state is true; the old behaviour can be restored by setting srm.persistence.enable.store-transient-state to false.

Fix the report sent to the srm client so that the failure code and error message always match. Earlier releases of this branch sometimes returned an inconsistent combination of failure code and error message.

Fix potential for inconsistent information being logged in the SRM database, where the request state was different than the last logged transition.

The SRM provides legacy support for SRM clients that require the TCP connection to be closed without the orderly process that SSL/TLS clients normally expect. Unfortunately, this work-around contained a bug where a client interaction can trigger an thread being caught in a tight-loop, constantly consuming CPU. This release fixes that problem.

Changelog 2.14.5..2.14.6

bda53c5
[maven-release-plugin] prepare release 2.14.6
fef8106
pool: Log information on runtime for listing the repository
5d248cd
common-security,ftp-client: Allow compilation with Java 7
4397121
common: Move Checksums to fix compilation with Java 7
e445f6b
common: Move NetLoggerBuilder out of common
7283ebe
srm: Lock job while saving to create consistent persistent state
7b7ec53
srm: Fix saving of transient states to database
dbaa0ba
srm: Fix legacy close (again)
8dd8768
2.14: upgrade third party dependencies
e80adbe
http–3rd-party: ensure IOException logged with toString
e9757a7
srm-client: Fix GSI delegation for old servers
744d25b
srm: Fix race in state reporting
81a0f65
gplazma: Improve logging in voms plugin
93e6e8e
info: fix test to be less critical on timing
93f193e
[maven-release-plugin] prepare for next development iteration

Release 2.14.5

Changes affecting multiple services

Enable SRM to discover FTP and dcap doors faster after restart. May also resolve messages like Failed to deliver LoginBrokerInfoRequest message <1449649391771:176> to [>LoginBrokerRequestTopic@local]: Route for >*@srm-bombayDomain< not found at >dCacheDomain<. All ftp and dcap doors should be updated.

Fix tab completion for non-default cells (messaging) topologies; all domains must be upgraded to deploy the fix. Sites with default topologies are unaffected.

The doors that use TLS (SRM, gsi-dcap, gsi-ftp, gsi-xrootd) must verify the user’s certificate. Previously this was both slow and could only take advantage of a single core. This release caches the result for 5 minutes, ameliorating both problems.

For non-default star topologies, there are intermediate domains that pass messages between dCacheDomain and end domains. This release fixes propagation of routing information by these intermediate domains. Sites with default topologies are unaffected; those with non-default topologies should upgrade all intermediate domains.

admin

When using the admin interface, there are three possible behaviours: interactive mode (new interface), no-terminal mode (new interface targeted for scripts) and legacy mode (old interface). The latter two could become stuck in an endless loop if the client disconnects before all data was sent. This is fixed with this release.

gplazma

Fix how X.509 Distinguish Names (DNs) are handled. This could cause the x509 plugin to fail, logged as a NoSuchMethodError message.

pool

Add new options to the rep set sticky command to allow filtering by access-latency, retention-policy, storage class and cache class.

Add an option to migration module to support filtering by cache class.

Add the -meta-only option to migration module. This limits the affected replicas to those where the file’s data is not transferred; i.e., the target is some existing replica of the file. Local replicas that do not exist on any other pool are skipped.

Extend the migration module’s -sticky option to allow negated selection. This is marked by prefixing the sticky owner with a -. For example, -sticky=-system selects replicas that do not have the system sticky.

Fix persistency of the sticky bits. Should a replica’s list of sticky bits be modified by either the rep set sticky command or the migration module modifying some existing replica then the sticky flags are held in memory but not written to the Berkeley DB. The next restart of the pool will loose that information, potentially resulting in data loss. Pools that store metadata as files are not affected.

poolmanager

Previous releases of dCache contained a bug where replicas generated by pool-to-pool copies failed to include the access latency and retention policy. While not directly affecting dCache operations, the result is that this information is no longer reliable.

spacemanager

Fix listing by PNFS-ID. Glob support is removed as it was non-functional.

srm

Allow SRM to start after it was shut down with IN_PROGRESS jobs. Solves Failed to restore job: Illegal state transition from InProgress to Queued.

Update migration procedure to remove obsolete states from the history table.

Changelog 2.14.4..2.14.5

2adfacd
[maven-release-plugin] prepare release 2.14.5
9be3fcc
pool: Do not output expired sticky flags
f7f7e0b
pool: Only save sticky bits if not already set
c74b297
poolmanager: Fix missing access latency and retention policy on pool to pool copy
01859f7
admin: Fix endless loop in non-interactive mode
fb94bc7
pool: Make bulk sticky bit operation robust against repository changes
5506ae8
pool: Throw IllegalArgumentException on rep set sticky errors
d7fce30
spacemanager: Fix listing by pnfs id
127c9fa
Add caching layer for TLS validation
227337e
pool: Extend migration module with -meta-only option
f23b270
pool: Add option to migration module to filter by cache class
5971062
pool: Add bulk mode for rep set sticky command
9268623
pool: Add migration option to filter by absense of sticky flags
3bd9ab2
pool: Fix persistence of sticky bits
a1cfc59
cells: Tab complete on distant downstream domains too
4f79b86
cells: Fix route propagation trigger on non-default topologies
f4cfd03
srm: Allow transition from InProgress to Queued
ad96e6c
srm: Map obsolete states in request history tables
07705d6
gplazma: Make GlobusPrincipal compatibly with JGlobus
756489e
[maven-release-plugin] prepare for next development iteration
6780e53
ftp,dcap: Fix LoginBrokerRequestTopic subscription

Release 2.14.4

nfs

A bug-fix introduced with 2.14.3 also introduced a regression that prevented directory listing over NFS protocol. This is fixed with this release.

Changelog 2.14.3..2.14.4

7c13aba
[maven-release-plugin] prepare release 2.14.4
bf173c2
chimera: fix regressin introduced by fa9a749c4
1af6698
[maven-release-plugin] prepare for next development iteration

Release 2.14.3

Changes affecting multiple services

The update to Chimera that came with 2.14 introduced a possible deadlock. These are clearly logged with ERROR: deadlock detected in the domain log file, and occur when a file is both deleted and its attributes are updated at nearly the same time. This problem is fixed with this release. Domains hosting either the nfs door or the pnfsmanager service should be upgraded.

Fix a problem in how cached values are updated within Chimera. Domains hosting either an nfs door or the pnfsmanager service should be upgraded.

A request to read a file stored only on tape when the user does not have permission to stage files will fail. Additionally, a user attempting to upload a file that would trigger creating a new space-reservation but the user is not authorised to create such a reservation will also fail. With earlier versions of dCache, in both these cases, the door would retry the request until the times out. This version fixes this issue. All doors are affected.

This release fixes a caching issue where changes to inode metadata (e.g., ownership or permissions) for / (the root directory of Chimera) are not visible until the service is restarted. This affects NFS doors and pnfsmanager service.

Update Chimera so it does not trigger DuplicateKeyException, so avoiding errors like current transaction is aborted, commands ignored until end of transaction block; nested exception is org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block. Both NFS doors and pnfsmanager are affected. Sites that use PostgreSQL v9.5 or later are encouraged to use the newer dCache driver: chimera.db.dialect = PgSQL95.

nfs

Fix the NFS door so it no longer logs Post-processing failed: No such file or directory if a file is deleted while still open.

pnfsmanager

Fix support for enstore.

Fix the Access Latency and Retention Policy of files when listing a directory.

Changelog 2.14.2..2.14.3

d62b652
[maven-release-plugin] prepare release 2.14.3
0289f60
transfers: fail request if pool/space manager reject request
ff64d88
chimera: Prevent filling of stat cache of root inode
c16c259
chimera: When updating stat cache, take the file type into account
b577b92
chimera: Read AL and RP on directory listing
566f6a5
Motivation: enstore based installations need a special trigger to keep t_inodes table in sync.
fc13ac4
nfs: ingore file_not_found on close
45dddd1
chimera: Make FsSqlDriver#addInodeLocation PostgreSQL compatible
49526f2
chimera: Do not ignore DuplicateKeyExceptions
4d89937
chimera: Fix deadlock
0184615
[maven-release-plugin] prepare for next development iteration

Release 2.14.2

pnfsmanager

Fix the error message (logged by the domain hosting pnfsmanager) if an attempt to finalise an SRM upload fails within pnfsmanager, or if an attempt to cancel an SRM upload fails within pnfsmanager.

pool

Fix the NFS mover so that, when the client is reading a file and reaches the end of file, the EOF flag is set correctly. This bugfix applies both to files read through pNFS and those read through the door.

When a pool is initialising FTP-based third-party transfers, load all trusted certificate authorities once rather than loading them on demand. This results in better performance over the lifetime of the pool when transferring many files to remote servers with certificates issued by different certificate authorities.

poolmanager

When reporting an alarm that a pool is declared DOWN, the message mistakenly omitted the name of the pool. This is now fixed.

webdav

Update to the latest version of milton.

xrootd

Update the alice-token plugin to allow the host name check to succeed on dual-stack (IPv4 and IPv6) machines.

Log initial response (usually that an asynchronous reply is forthcoming) when client attempts to open a file.

Fix door so that it no longer logs: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

Changelog 2.14.1..2.14.2

802c243
[maven-release-plugin] prepare release 2.14.2
197b757
xrootd: Update alice token plugin to fix IPv6 compatibility
8aabecb
xrootd: Update to xrootd4j 3.0.1
a049d16
system-test: Update ctlcluster command to configure CRL settings
af58f03
xrootd: Fix logging of Netty exceptions
85edc0a
webdav: update to latest milton
b3a9355
PnfsManager: remove copy-n-paste error in error message
595d885
(2.14) pool manager: add pool name to alarm
21b8272
pool: Make context factory non-lazy for remote gsiftp transfers
a022aa0
system-test: Fix grid-security settings
1f2c6b7
nfs: set EOF flag when channel indicated EOF
1040300
rpm: enforce SL5 compatibility when building RPM packages
afcd1b2
[maven-release-plugin] prepare for next development iteration

Release 2.14.1

admin

Fix potential IndexOutOfBoundsException should the response from the acm cell be malformed.

alarms

Update the text format of logged and emailed alarms so that messages include the type of alarm rather than simply the phrase ALARM [ ALARM_TYPE ]. Note that the alarms.email.encoding-pattern and alarms.history.encoding-pattern properties have been updated; sites that have modified either of these properties must review their configuration to take advantage of this update.

pool

Fix an intended pool-to-pool transfer optimisation: the receiving pool failed to reuse a delayed mover, should the pool-to-pool request timeout and be retried.

Changelog 2.14.0..2.14.1

9121417
[maven-release-plugin] prepare release 2.14.1
4edc896
srm-client, dcache: fixed passing incompatible arguments to functions
975e5ba
rpm: build SNAPSHOT RPM with filenames using only commit id
4d830fa
dcache-nfs, dcache: removed unecessary use of non-short-circuit logic
3a7b359
(2.14) alarms: modify logback encoding to show actual alarm type
70048cd
[maven-release-plugin] prepare for next development iteration

Release 2.14.0

Improved execution time statistics

Several cells provide information on request execution times. The code has been improved to reduce the effect of numerical instabilities. This improves the accuracy of the collected statistics. Rather than the population standard deviation we now report the sample standard deviation.

Use CANL for certificate verification

CANL is a library provided by EMI that provides support for OpenSSL style trust stores, PEM encoded certificates, proxy certificates, certificate path verification, CRL and OCSP validation, etc. In the past, dCache has relied upon the JGlobus library for such operations, but JGlobus has not seen much maintenance lately. We have switched to using CANL instead as we feel it is more up to date (an important aspect for security critical libraries), provides a cleaner API, has more features, and doesn’t lock us into old versions of other third party libraries. There may also be performance benefits - we know for certain that the startup time for client tools is improved, and time will tell if the server side is faster too.

The amount of changes required in dCache to implement this was quite large and touched most components, although the externally visible number of changes should be low. Still, with this amount of changes there is bound to be a regression or two.

One feature of JGlobus missing in CANL is an implementation of the GSI protocol. GSI is essentially TLS with an additional credential delegation step after the initial TLS handshake. In dCache 2.13 we introduced our own GSI implementation in dCache to provide support for non-blocking IO in the SRM door. In 2.14 this has been adopted by all doors providing GSI support, i.e. SRM, FTP and DCAP. Furthermore, the client side of GSI has been implemented and is used by our FTP and SRM clients, as well as our GridSite delegation shell.

As part of porting DCAP and FTP to our own GSI implementation, the SSL context is now cached between sessions, greatly reducing the per-connection overhead of these doors. Consequently these doors now expose settings for key pair cache lifetime:

#  ---- Delegation key pair reuse lifetime
#
#  X.509 clients may delegate credentials to dCache using GSI or the dedicated GridSite
#  delegation service. The delegation works by the server generating a certificate
#  signing request which the client signs. Since generating a key pair as part of the
#  signing request is expensive, dCache caches and reuses the key pair.
#
#  This setting controls for how long a key pair is reused. A compromised key could be used
#  for new signing requests for this amount of time. If the reuse time is too short, the
#  overhead of generating new key pairs increases.
#
dcache.authn.gsi.delegation.cache.lifetime = 30000
dcache.authn.gsi.delegation.cache.lifetime.unit = MILLISECONDS

The default host and CA certificate refresh periods have been significantly reduced to 60 seconds:

# ---- Host certificate refresh period
#
# This option influences in which intervals the host certificate will be
# reloaded on a running door.
#
dcache.authn.hostcert.refresh = 60
dcache.authn.hostcert.refresh.unit = SECONDS

# ---- CA certificates refresh period
#
# Grid-based authentication usually requires to load a set of
# certificates that are accepted as certificate authorities. This
# option influences in which interval these trust anchors are
# reloaded.
#
dcache.authn.capath.refresh = 60
dcache.authn.capath.refresh.unit = SECONDS

Since CANL provides configurable name space verification, CRL validation, and OCSP validation, all doors and pools relying on TLS now respect the following new configuration properties:

# ---- Certificate Authority Namespace usage mode
#
# A CA namespace restricts the certificates dCache accepts as issued by a given CA. The namespace
# is defined in .namespaces (EURGIDPMA) or .signing_policy (GLOBUS) files alongside the CA certificate.
#
# This setting controls the rules governing the verification of these namespaces. The following
# documentation is copied from the EMI CANL library used by dCache.
#
#       GLOBUS_EUGRIDPMA
#
#          A Globus EACL is checked first. If found for the issuing CA then it is used and enforced.
#             If not found then EuGridPMA namespaces definition is searched. If found for the issuing CA
#                then it is enforced.
#                   If no definition is present then namespaces check is considered to be passed.
#
#       EUGRIDPMA_GLOBUS
#
#          An EuGridPMA namespaces definition is checked first. If found for the issuing CA then it is enforced.
#             If not found then Globus EACL definition is searched. If found for the issuing CA
#                then it is enforced.
#                   If no definition is present then namespaces check is considered to be passed.
#
#       GLOBUS
#
#          A Globus EACL is checked only. If found for the issuing CA then it is used and enforced.
#             If no definition is present then namespaces check is considered to be passed.
#
#       EUGRIDPMA
#
#          An EuGridPMA namespaces definition is checked only. If found for the issuing CA then it is enforced.
#             If no definition is present then namespaces check is considered to be passed.
#
#       GLOBUS_EUGRIDPMA_REQUIRE
#
#          A Globus EACL is checked first. If found for the issuing CA then it is used and enforced.
#             If not found then EuGridPMA namespaces definition is searched. If found for the issuing CA
#                then it is enforced.
#                   If no definition is present then namespaces check is considered to be failed.
#
#       EUGRIDPMA_GLOBUS_REQUIRE
#
#          An EuGridPMA namespaces definition is checked first. If found for the issuing CA then it is enforced.
#             If not found then Globus EACL definition is searched. If found for the issuing CA
#                then it is enforced.
#                   If no definition is present then namespaces check is considered to be failed.
#
#       GLOBUS_REQUIRE
#
#          A Globus EACL is checked only. If found for the issuing CA then it is used and enforced.
#             If no definition is present then namespaces check is considered to be failed.
#
#       EUGRIDPMA_REQUIRE
#
#          An EuGridPMA namespaces definition is checked only. If found for the issuing CA then it is enforced.
#             If no definition is present then namespaces check is considered to be failed.
#
#       EUGRIDPMA_AND_GLOBUS
#
#          Both EuGridPMA namespaces definition and Globus EACL are enforced for the issuer.
#             If no definition is present then namespaces check is considered to be passed.
#
#       EUGRIDPMA_AND_GLOBUS_REQUIRE
#
#          Both EuGridPMA namespaces definition and Globus EACL are enforced for the issuer.
#             If no definition is present then namespaces check is considered to be failed.
#
#       IGNORE
#
#          CA namespaces are fully ignored, even if present.
#
dcache.authn.namespace-mode=EUGRIDPMA_AND_GLOBUS_REQUIRE

# ---- Certificate Revocation List usage mode
#
# CAs regularly publish certificate revocation lists (CRLs) containing the serial id of
# certificates that have been revoked. Such certificates should not be accepted by dCache.
#
# Such CRLs are stored in .r? files alongside the CA certificate. It is outside the scope
# of dCache to refresh the CRLs.
#
# This setting controls how dCache makes use of such CRLs. The following documentation is copied
# from the EMI CANL library used by dCache.
#
#       REQUIRE
#
#          A CRL for CA which issued a certificate being validated
#             must be present and valid and the certificate must not be on the list.
#
#       IF_VALID
#
#          If a CRL for CA which issued a certificate being validated
#             is present and valid then the certificate must not be listed on the CRL.
#                If the CRL is present but it is outdated (or anyhow else corrupted) then the validation fails.
#                   If CRL is missing then validation is successful.
#
#       IGNORE
#
#          CRL is not checked even if it exists.
#
dcache.authn.crl-mode=REQUIRE

# ---- On-line Certificate Status Protocol usage mode
#
# On-line Certificate Status Protocol (OCSP) is an alternative to CRLs in which
# dCache consults an external service to check the validity of a particular certificate.
#
# This setting controls how dCache makes use of this protocol. The following documentation is copied
# from the EMI CANL library used by dCache.
#
#       REQUIRE
#
#          Require, for each checked certificate, that at least one valid OCSP responder is defined and
#             that at least one responder of those defined returns a correct certificate status.
#                If all OCSP responders return error or unknown status, the last one received is treated as a
#                   critical validation error.
#                      Not suggested, unless it is guaranteed that well configured responder(s) is(are) defined
#                         and can handle all queries without timeouts.
#
#       IF_AVAILABLE
#
#          Use OCSP for each certificate if a responder is available. OCSP 'unknown' status and
#             query errors (as timeout) do not cause the validation to fail.
#                Also a lack of defined responder doesn't cause the validation to fail.
#
#       IGNORE
#
#          Do not use OCSP.
#
dcache.authn.ocsp-mode=IF_AVAILABLE

Finally, logging from CANL is different. Our initial impression is also that it is significantly better.

FTP Client

Another area in which dCache relied on the JGlobus client was for the FTP client. An FTP client is needed as part of our SRM client as well as for SRM and WebDAV third party copy. In the quest to drop the JGlobus dependency, we have forked the JGlobus FTP client and ported it to CANL. There should be minimal user visible changes from this. The most interesting may be that our client now identifies itself to the server as being dCache and using the dCache version number.

SRM client

As mentioned above, the SRM client has been updated to use our own GSI implementation on top of CANL for certificate handling. On a host with a large number of CA certificates, the startup time is significantly reduced.

The SRM protocol is implemented using SOAP over HTTP over GSI. Due to the old SOAP version used we are bound to the Axis 1 library. In the past we relied upon the minimalistic HTTP client integrated into Axis 1. In dCache 2.14 we added an Axis handler to use the Apache HTTP components client. The primary benefits are:

  • Proper keep-alive support, meaning that the client no longer needs to reestablish the connection to the server for every low level SRM request. This both reduces the latency experienced by the client and reduces load on the server.

  • Strict RFC 2818 host name verification.

In partiular the latter point has the potential to break compatibility with some sites. This is the case if the host certificate used by the site does not comply with RFC 2818. Without strict RFC 2818 support, TLS/GSI connections are susceptible to man in the middle attacks. The Globus Toolkit (the C library, not the JGlobus Java version) will switch to strict RFC 2818 support by default at the end of 2015 and we expect that most sites will quickly request certificates with correct subject alternative names.

If compatibility with non-compliant sites is required, we recommend using a version of the SRM client prior to 2.14. On the server side, strict RFC 2818 host name verification is only used for server side srmCopy or WEBDAV copy. If these are needed with non-compilant sites, we suggest using dCache 2.13.

Updated help pages for many admin shell commands

dCache has for several years been in a transition between two infrastructures for providing admin shell commands. The newer of these provides much better help pages. In dCache 2.14 many commands have been ported from old to the new scheme.

Admin shell gains pool group globs

The new admin shell introduced in dCache 2.13 provides bulk commands with cell name globs. In dCache 2.14 pool group globs have been added to these commands. A pool group glob follows the pattern pool/poolgroup, with either side accepting ? and * wildcards. It expands to all pools that match the left side of the pattern and which are within a pool group matching the right side. Either side may be empty and is equivalent to *.

Thus / will match any pool that appears in any pool group, /atlas_disk matches any pool in atlas_disk, and *_rack1/atlas_buffer matches any pool with a name ending in _rack1 in the atlas_buffer pool group. These could for instance be used to perform bulk commands over a set of pool:

\s /atlas_tape flush set interval 600
\s /atlas_tape save

Significant schema changes in Chimera

The Chimera database schema has been updated to improve throughput, reduce latency and reduce disk space. It will be updated automatically the first time the PnfsManager is started, but is is advisable to apply the schema changes ahead of time by using the dcache database update command. It is also advisable to make a backup of the database prior to upgrading as well as run a test migration on a clone of the database to know how long it will take as well discover any incompatibilities with local schema modification.

Once upgraded to dCache 2.14, one cannot downgrade to 2.13 without rolling back the schema changes. This must be done before downgrading using the dcach database rollbackToDate command.

Several schema changes are applied durin upgrade:

  • The . and .. directory entries are no longer stored in the database. Since Chimera is stored in a relational database, it can find parrent directories without these backpointers. Where applicable, these directories entries are created on the fly in dCache.

  • Directory tags are no longer created by a trigger. In the past the trigger reacted upon the insertion of the .. entry, but since we no longer insert this entry we cannot rely on this trigger. Furthermore, for the temporary directories created for every SRM upload the tags should not be copied and the trigger needlessly slowed down creating such directories.

  • Inodes are now created using the new f_create_inode stored procedure. This eliminates several round trips between the client and PostgreSQL.

  • The type of the itagid column of the t_tags_inodes table has been changed to a 64 bit auto sequence field. This reduces storage space as well as CPU load for these tables.

  • The t_access_latency and t_retention_policy tables have been merged into the t_inodes table. The former two tables are dropped. This reduces storage space as well as latency in creating and reading inodes.

  • The address mask is removed from the t_acl. Chimera ACLs no longer support access control by client IP.

Chimera PostgreSQL 9.5 driver

Chimera has pluggable RDBMS drivers. dCache 2.14 adds a driver specific for PostgreSQL 9.5 and newer. This driver avoids the excessive logging caused by primary key uniqueness violations that are common with Chimera.

To activate, set chimera.db.dialect to PgSQL95.

Chimera race conditions and performance improvements

Many rare races and sublte bugs have been fixed, and lots of small performance improvements have been made.

PnfsManager uses single request queue

Due to the extensive changes in Chimera, PnfsManager can now use a single request queue shared between all threads. Thus the info output no longer shows a queue per thread and it is easier to keep all threads busy.

Technically, there is a queue per thread-group, but since thread-groups are not currently used by Chimera, for all intends and purposes there is only one general request queue.

Forbidden and obsolete properties have been dropped

Properties marked as forbidden or obsolete in 2.13 have been removed from dCache 2.14. Properties marked deprecated in 2.13 have been marked obsolete. Please verify and fix any warnings produced by dcache check-config before upgrading.

Cryptographic cipher selection

dCache now bans RC4 ciphers by default. The ability to disable Diffie Hellman key exchange on JVMs in which this is broken has been removed since Diffie Hellman now works on all supported JVMs.

FTP gains new SITE commands

The FTP door now supports the SITE CHGRP, SITE SYMLINKFROM and SITE SYMLINKTO non-standard commands to change the file group and create symbolic links. These commands are supported by the UberFTP client.

FTP proxy limits internal network interface

FTP doors may act as proxies for the data connection. In this case the pool establishes a TCP connection to the door. Previously the door would listen to the wildcard address (i.e. all interfaces) even though only one specific address was sent to the pool to connect to. In dCache 2.14 the door now limits the server socket to this one address.

gPlazma drops support for plugin caching

gPlazma used to only instantiate a plugin once even if it was mentioned several times in gplazma.conf. This meant that only a single configuration could be applied, thus making several usecases impossible to handle.

dCache now instantiates every plugin every time it is referenced in gplazma.conf and each instance can have its own configuration. The options for forcing enabling the old behaviour have been removed.

gPlazma supports concurrent request processing

In the past gPlazma was single threaded, which in particular was a problem with plugins that called out to external services. gPlazma is now multi-threaded and will happily call several plugins concurrently. Even the same plugin may be called concurrently, which means that all plugins - including those by third-parties - must be thread safe.

gPlazma XACML plugin update for improved spec compliance

The XACML plugin has been ported to CANL too and in the process it was updated to improved compliance with the specification. The client library used by the plugin unfortunately depends on JGlobus and thus the current version of dCache still ships with JGlobus.

DCAP delegates X.509 and VOMS proxy processing to gPlazma

Previously DCAP would validate the VOMS signatures of a proxy in the door, thus requiring a vomsdir to be set up on the door. gPlazma now delegates this to gPlazma and the gPlazma voms plugin. Thus it is no longer necessary to keep a vomsdir setup on GSIDCAP doors.

NFS door publish exported paths login broker information

The NFS door now include the exported paths as part of the login broker information. This information is available to the info service and the srm door.

NFS door sees large number of fixes and improvements

As always, the NFS door has received a fair share of fixes and improvements. If you rely heavily on the NFS door, using the latest version of dCache is recommended.

Pool manager generates sorted configuration files

Previously entries where output in hash order, which meant that minor changes in configuration could cause drastic changes to the order.

WebDAV door links to https version of dCache home page

Thanks to a contribution by Christoph Anton Mitterer, the footer on WebDAV door directory listings now point to the https version of the dCache home page.

WebDAV door can report file locality

SRM has the concept of file locality, i.e. whether a file is on disk, on tape, both, offline or lost. The WebDAV door now exposes this information too:

  • The HTML rendering of directory listings uses the icon to indicate the file locality.

  • For programatic access, the WebDAV PROPFIND method.

An example of the latter follows:

$ curl -X PROPFIND -H Depth:0 http://localhost:2880/disk/test-1447231167-1 \
  --data '<?xml version="1.0" encoding="utf-8"?>
          <D:propfind xmlns:D="DAV:">
              <D:prop xmlns:R="http://www.dcache.org/2013/webdav"
                      xmlns:S="http://srm.lbl.gov/StorageResourceManager">
                  <R:Checksums/>
                  <S:AccessLatency/>
                  <S:RetentionPolicy/><S:FileLocality/>
              </D:prop>
          </D:propfind>'

<?xml version="1.0" encoding="utf-8" ?>
<d:multistatus xmlns:cal="urn:ietf:params:xml:ns:caldav"
               xmlns:cs="http://calendarserver.org/ns/"
               xmlns:card="urn:ietf:params:xml:ns:carddav"
               xmlns:ns2="http://www.dcache.org/2013/webdav"
               xmlns:ns1="http://srm.lbl.gov/StorageResourceManager"
               xmlns:d="DAV:">
    <d:response>
        <d:href>/disk/test-1447231167-1</d:href>
        <d:propstat>
            <d:prop>
                <ns1:FileLocality>ONLINE</ns1:FileLocality>
                <ns1:RetentionPolicy>REPLICA</ns1:RetentionPolicy>
                <ns2:Checksums>adler32=3b8711d6</ns2:Checksums>
                <ns1:AccessLatency>ONLINE</ns1:AccessLatency>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
</d:multistatus>

WebDAV door provides human-friendly file sizes

An abbreviated file size is shown as a hint in the WebDAV door HTML rendering. To see, however the mouse over the file entry.

WebDAV door enables BASIC authentication by default over https connections

This provides a means for clients without a certificate to authenticate with the server. Note that BASIC authentication should not be used over unencrypted connections.

Pools now always compute checksums on the fly

Recently we changes the default checksum policy for new pools from onwrite to ontransfer. The latter requires less resources as it computes the checksum during the upload rather than reading the file back from disk after the transfer.

In dCache 2.14 the ontransfer checksum calculation is no longer optional - it is always computed for streaming uploads. The onwrite policy can still be enabled, but it would calculate the checksum a second time. A future update of dCache will remove the onwrite policy.

SRM database schema changes

The SRM database is now managed by the liquibase schema management library. An existing schema from a previous installation is automatically detected, but we recommend doing a test upgrade on a clone of the database to check compatibility with any local schema modifications.

Several changes are made to the schema of the SRM database:

  • Missing indexes are added.

  • Unused indexes are removed.

  • The count column of the lsrequests table is renamed to cnt to avoid conflicts with the like named SQL keyword.

  • The encoding of user information is redone entirely and user information is now stored in a binary format.

In particular the latter change is worth paying attention to. Due to this change, existing requsts will fail during upgrade as the existing user information is deleted upon upgrade. Already finished requests will be kept in the database until they are garbage collected, but they too loose all information about who created it. Furthermore, the binary encoding in the new user record means that it is no longer easy to access this information in local queries. This change was necessary to work around limitations in the previous serialization - in short, we cannot serialize X.500 names in string form without loosing information.

SRM gains a pluggable transfer strategy mechanism

Plugins may decide in which order to serve transfer requests by clients. Two strategies are shipped with dCache with the default implementing a simple fair share strategy:

# ---- Request transfer strategy
#
# srmPrepareToPut and srmPrepareToGet requests enter the READY state when the TURL is handed
# out to the client. If the request is processed asynchronously (client is polling for the result),
# the request stays in the RQUEUED state until the client queries the result.
#
# A transfer strategy plugin has the ability to prevent the transition from the RQUEUED to READY
# state. A plugin may allow the number of concurrent transfers to be limited, or may provide
# some fair-share between parties.
#
# Two plugins ship with dCache:
#
#     first-come-first-served
#
#         The maximum number of requests in the READY state is limited by
#         srm.request.*.max-transfers, but otherwise TURLs are handed out to
#         whomever comes first.
#
#     fair-share
#
#         The maximum number of requests in the READY state is limited by
#         srm.request.*.max-transfers, but slots are kept free such that each party with
#         requests in RQUEUED will receive its fair share.
#
#           A configurable discriminator defines the grouping between which to provide a
#           fair share.
#
srm.plugins.transfer = fair-share

# Discriminator for the fair share transfer strategy.
srm.transfer.fair-share!discriminator = ${srm.plugins.discriminator}

Deprecation of SRM 1 support

The SRM 1 protocol has long been superseeded by the SRM 2.2 protocol. dCache 2.14 adds options to disable SRM 1 support and disables it by default:

#
# SRM versions to support.
#
# Comma separated list of SRM versions to support.
#
(any-of?1|2)srm.version=2

You can reenable SRM 1 support by adding it to the above configuration option. Please contact us if you do so we can ascertain when to drop SRM 1 entirely.

This change also removes support for third party srmCopy to/from SRM 1 servers, no matter whether SRM 1 support in the server is enabled or not.

SRM request scheduler state simplification

SRM scheduler states have been simplified by collapsing several states into the new InProgress state. Besides changing the output of the info command, this will change the state values observed in the database. Sites that monitor these states will have to update their monitoring scripts. The job state mappings can be viewed here: https://github.com/dCache/dcache/blob/master/modules/srm-server/src/main/resources/org/dcache/srm/request/sql/srmjobstate–214.csv

SRM no longer retries requests

SRM used to have support to retry requests internally upon errors. The implementation of this functionality was problematic since

  • many errors were not retried even when they should, and worse

  • many errors that should not be retried were.

In the interest of fail fast behaviour and simpler code, support for internally retrying requests has been dropped. Failures are propagated to the client and it is up to the client to determine whether and how to retry. This also allows client to more quickly fall back to other sites in case of problems.

SRM changes algorithm to detect a local SURL

The SRM protocol has a concept of site URLs and classifies these into those local to the storage system and those belonging to other storage systems. This classification is important when the SRM is to determine which file is local and which is remote in a third party SRM copy operation, or just to determine if the SURL is indeed local in put or get operations.

The most important changes are:

  • srmCopy is now limited to having a local source or local destination SURL. It no longer allows a transfer between a local non-SRM door and a remote system.

  • no DNS lookup is performed in determining whether the SURL is local. Previously the host name would be resolved to an IP addressed and the IP addressed would be compared to local addresses. Now the list of local host names is determined during startup and the host name in the SURL is compared to the local names directly. An administrator still has control over which names to consider local by setting the srm.net.local-hosts property.

Several minor SRM fixes

Several corner cases and race conditions have been fixed, as well as better failure handling to prevent leaking pins or upload directories.

Configurable kXR_Qconfig replies for xrootd

The xrootd protocol has a mechanism to query the server configuration using a kXR_Qconfig request. dCache provided a few hard-coded properties, but the xrootd protocol specifies an open list of properties.

In dCache 2.14 the admin may inject additional properties to be returned to the client. These are configured through the dCache configuration system:

#  ----- Custom kXR_Qconfig responses
#
#   xrootd clients may query the server configuration using a kXR_Qconfig request.
#   These key value pairs can be queried. Additional key value pairs may be added as
#   needed, see the xrootd protocol specification at http://xrootd.org/ for details.
#
(prefix)xrootd.query-config = kXR_Qconfig responses
xrootd.query-config!version = dCache ${dcache.version}
xrootd.query-config!sitename = ${dcache.description}
xrootd.query-config!role = none

#  ----- Custom kXR_Qconfig responses
#
#   xrootd clients may query the server configuration using a kXR_Qconfig request.
#   These key value pairs can be queried. Additional key value pairs may be added as
#   needed, see the xrootd protocol specification at http://xrootd.org/ for details.
#
(prefix)pool.mover.xrootd.query-config = kXR_Qconfig responses
pool.mover.xrootd.query-config!version = dCache ${dcache.version}
pool.mover.xrootd.query-config!sitename = ${dcache.description}
pool.mover.xrootd.query-config!role = none

Many third party libraries have been updated

Most notably the new version of the PostgreSQL JDBC driver has some nice performance improvements.

Changelog from 2.13.0 to 2.14.0

388ee5d
chimera: Propagate failures to read or write in-db data
7f38e2a
srm: Rename count column
f6aa2a3
common-security: Do not log stack traces for CANL notifications
4d2ce2d
srm: Use correct logging context when saving jobs
984c385
common-security: Lower log messages for CANL notifications
dc54153
srm-client: Add initialization of Axis handler in SRM 1 client
88be0f1
srm: Fix NPE when saving requests with an unknown user
90689dd
srm: Fix race in user persistence
c1a2040
gplazma: fix erroneous logging in x509 plugin
ab6c401
pom: generate rpm friendly dirty versions number
9c851b0
gplazma: remove support for plugin caching
1c53eea
gplazma: make x509 generation of LoA principals optional
bac28ce
[maven-release-plugin] prepare branch 2.14
886fe81
webdav: enable BASIC authn for https by default
99e5ec1
Logging CANL notifications to correct cells context
9a2689e
gplazma2-voms: Fix error message
dba3da8
pool: Lazily initialize trust manager for remote https and gsiftp movers
d58e8a0
pom: Downgrade liquibase to 3.3.2
6d6676e
cells: Fix NPE during shutdown
c58fc42
srm: Reimplement legacy close
ed89d6b
dcap,ftp: Add remote address to logging context while creating child cell
06cc7d1
gsidcap: Avoid including payload data in the log
4ffef54
gplazma2: Call plugins concurrently
50581b4
Downgrade PostgreSQL jdbc driver
3586071
liquibase: Downgrade to 3.3.3
e604095
srm: Do not attempt to drop sequence objects on upgrade
9060d2d
srm: Add missing rollback
55b603e
srm: Fix quoting for deleting old authorization tables
cd565a6
srm: Fix renaming of count column
3196ffe
gplazma2-xacml: Add JGlobus dependency
a6674ba
Update third party dependencies
a3b2467
srm: Add missing index on bringonlinerequests_protocols foreign key
0baeb6a
srm: Drop unused indexes on transitiontime columns
242ed7b
srm: Drop indexes on nextjobid column
90cd0b2
srm: Rename count column of lsrequest table
480c8f5
srm: Replace AuthorizationRecord with new user manager
1f86515
x509: add Principals that describe LoA and Entity-Definition
c3984dc
pool: Do not announce cell until the cell is running
d08db11
pool: Refine Berkeley DB failure handling
f95ef8f
srm: Report correct estimated wait time for SRM 2.2 copy requests
b8359f0
common-security: Renamed javatunnel package
9b77b04
Revert “Update third party dependencies”
e60ea77
Update third party dependencies
da7693d
voms: Fix resource leak regression
69eac64
srm: Fix regression in database schema for ls requests
9f7311c
srm: Fix regression in extracting paths from local SURLs
977ddd8
srm: Port request id table schema management to liquibase
82183bf
ScriptNearlineStorage: Fix missing -uri parameter for fetch
26f7ca1
srm: Port schema management to liquibase
ee59461
doors: Add CANL listeners for logging errors and warnings
a7556e5
Reduce duplication between context factories
e84a24d
system-test: apply IF_VALID crl policy to all services
b1767a2
pool: java8 into IoQueueManager
9e7687c
pool: simplify pool queues
f10ee1f
build: Fix build performance regression caused by git describe
9d512b0
Revert “build: Fix build performance regression caused by git describe”
59c54ef
build: Fix build performance regression caused by git describe
1b1793f
srm: Fix regression in detection of local SURLs
47b11bf
Drop JGlobus
a84b439
xrootd: Upgrade to xrootd4j
85b5949
webdav: update file icon(s) in directory HTML page so they show file locality
d4ec10f
ftp-client: Resolving merge conflict
4fe019b
webdav: Respect webdav.static-content.uri property
49fcd3e
doors: Do not log failure to delete absent files on upload failures:
79a3ce7
ftp-client: Port to our own GSI implementation
d1aaa05
javatunnel: Add client side GSI implementation
173e7df
ftp-client: Rip out DCAU support
bdb4e83
javatunnel: Decouple from Jetty
bfd3f90
ftp-client: Push X509Credential to GSSCredential conversion into the client
d3b2477
ftp-client: Refactor control channel implementation
f262180
ftp-client: Port TCP port range and buffer size handling to dCache
b5d649b
ftp-client: Use Unix style newline encoding
90df88f
ftp-client: Automatic IntelliJ fixes
4ec8439
ftp-client: Add missing @Override annotations
c1d39dc
ftp-client: Use our own networking and version information
aa28fbd
common: Refactor PortRange and move to common module
7b9d121
ftp-client: Automatic code reformatting
a0b56d6
ftp-client: Delete dead code
b4a31b2
ftp-client: Move code to org.dcache.ftp.client
330792d
ftp-client: Import JGlobus gridftp client code
22ab9db
srm-client: Port SRM client to Apache HTTP components and CANL
3780943
pool: Turn remote HTTP movers into dedicates transfer service and port to CANL
cecc4c1
nfs4: fix race in request processing
bc168bf
libs: update to nfs4j–0.11.2
bd5c32a
webdav: Add robots.txt
501ed18
srm-client: Get rid of GlobusURL
28d23b7
Stop using JGlobus for delegation CSR
68b18ba
srm: Port most credential handling code from JGlobus to CANL
cbe6985
pool: implementing annotated command syntax for pools (3)
099bc30
pool: implementing annotated command syntax for admin interface commands (4)
76d2ac5
pool: implementing annotated command syntax for admin interface commands (SchedulerEntry)
d4613a5
hsmcp: update to match new HSM interface.
0d6b1c8
cells: clean up
1af9745
webdav: Port COPY credentials to CANL
eaf91db
gplazma: Port to CANL
9be9dfb
common: Copy GlobusPrincipal to dCache code base
5778614
ftp,dcap,srm,httpd,webdav: Switch to CANL for certificate handling
f311c37
ftp: Refactor FTP door to separate settings from the interpreter
5d4fcda
ftp: Reuse DssContxtFactory
66845ec
webadmin: improve user-friendliness of alarms page on load
9165b28
old replica manager: prevent pool being listed as offline when there are files with corrupt metadata
bad630a
webdav: add support for querying file locality.
99e4a00
Avoid deprecated sameThreadExecutor
fc1fc7c
webdav: Kill abandoned movers
57ee102
xrootd: Kill mover on aborted write
719a1c7
pool: Fix transfer prioritization
5c3800d
pool: Add nearline storage default timeouts
b2372cb
ftp: Refactor interpreter instantiation
5329756
xrootd: Fix race condition in request cancelation
0644d66
pool: Let script nearline storage provider scale down when lowering limits
223f02c
pom: remove dependency on grizzly
10a20e4
srm-client: Delete dead code
d465784
Delete dead test code
85c4086
srm: Delete dead test code
251b67b
dcap: Fix broken argument parsing (for krb5)
634c712
cells: Refactor option parsing
92dd85f
system-test: Restrict port range to avoid conflicts with services
506f4ab
dcap: Fix broken argument parsing
708fdd4
dcap: Fix broken argument parsing
f0eb468
xacml: fix voms attribute validation
576848f
ftp: Limit proxy to the interface that the pool connects to
55c5c5d
Fix parsing of single port port range
6b57edd
cells,javatunnel: Revert dependencies
e0ddf83
dcap: Fix socket factory argument parsing
1428da0
system-test: Restrict more sockets to loopback
ec6d153
Fix build and startup regression
1323016
srm: Disable SRM 1 by default
1a8b2a1
srm: Drop dead configuration code
a94ce34
srm: Remove standalone server
adbf507
srm: Drop generic support for retrying requests
e9542f7
srm: Delete upload directory if SRM request gets garbage collected
4fc93f0
srm: Move final state check out of Scheduler
78c2996
srm,ftp,dcap: Respect banned ciphers
431e801
pool: Minor refactoring of remote HTTP mover
0168ba9
pool: Fix error reporting in remote HTTP mover
b746c36
pool: Update HTTP client and avoid deprecated calls
1472ae2
libs: update to nfs4j–0.11.1
59a67f4
ftp: Don’t hold locking during proxy shut down
9a424a8
ftp: Fix deadlock triggered by failures
9a5c117
HSM Script: pass arguments in array
1d3bf97
srm: Unpin files when aborting prepareToGet
99bdf4b
srm: Refactor Scheduler#schedule method
5783a1f
srm: Add in-memory history logging
8c35457
srm: Simplify scheduler states
06377ce
srm: Refactor exception handling in Scheduler
13daf0e
srm: Refactor CopyRequest to make more fields final
7edc287
srm: Don’t use SRM 1 for srmCopy
fa95576
srm: Remove third party copy from SRM front-end code
ea23dc8
srm: Avoid URL copying in copy requests
e0c4af0
srm: Make list of file requests immutable
5cf98e4
srm: Drop dead code
e87fa4a
common: update NetLoggerBuilder to use Logger during building
4f69954
chimera: Another attempt at fixing auto-generated key extraction
95be135
ftp: Fix reporting of CRL expiration
e709eac
chimera: Fix Postgresql compatibility
b43de19
admin: Update help strings
d01c45c
admin: JavaDoc and some renaming
9abbbc7
admin: Add pool group patterns
71d2e00
libs: Update to Wicket 7
e124392
libs: Upgrade Jetty
ccfe23d
libs: Update to Spring Framework 4.2
ce37d36
libs: Update jline
b5fbb05
libs: Update dependencies
4848a7f
chimera: Use auto-increment field as itagid
83e2d8b
Port more Guava code to Java 8
30b1b44
chimera: Refactor transaction template use
d298cbb
pnfsmanager: Fix regression in cancelling uploads
3a94450
chimera: fix object type check on create
718212c
nfs: fix behavior on parentOf
5ad1fd5
pools: documentation and implementation of command annotation for pool sweeper commands
3c995b9
topology: Improve documentation of topo cell commands
b1ac329
docs: Point README at BUILDING
849a960
cells: xgetcellinfo and info commands documentation
e92b91c
cells: admin interface commands conversion with implementation of the command annotation for better documentation
54ee545
pool: removal of crash command in PoolV4
72c2765
chimera: Fix incomplete stat bug in listing
87254c4
nfs: fix lookup of ‘..’ after 2677f7ac
9699094
nfs: do not check for proxyIO factory
ec3d568
javatunnel: Fix build failure
81d4076
srm: return invalid request when the result of ls request is unknown
49428fb
Relocate GSI related classes to javatunnel
7f314ff
dcap,ftp: Wrap GsiEngine to provide GSI support
b6c66a4
chimera: Resolve performance regression in directory deletion
25e5832
chimera: Optimize inode creation with stored procedure
0d19395
chimera: Populate stat cache on inode creation
66e47ae
pnfsmanager: Improve caching in Chimera inode wrapper
2677f7a
chimera: Do not store ‘.’ and ‘..’ entries in the database
2228bb8
chimera: Fix hard link creation in the pressence of ACLs
7074fe1
chimera: Drop trigger to copy tags
2c4042a
pnfsmanager: Fix races in file deletion
8ca7b9b
pnfsmanager: Fix races and hard link issues in move
f9a2473
chimera: Fix races in move/rename logic
04b1887
chimera: Fix two race conditions in file deletion
6aaf08b
chimera: Fix remove by id
c299bb8
pnfsmanager: Use a single shared request queue for threads
ed7b90e
chimera: Port Chimera to Spring JDBC
6f6d48a
nfs: cleanup exception handling/conversion
787724b
chimera: Close datasource on shutdown
afc5e02
libs: switch to nfs4j–0.11
254a0ce
nfs: integrate ChimeraVFS into dCache code
10cfa65
chimera: merge access_latency and retention_policy into t_inodes
9525a3d
chimera: Fix performance issue in directory deletion for HSQLDB
10d7343
Improve execution time statistics
6f7adf0
ftp: Refactor GSS FTP door on top of DssApi
6f8d7a2
gsidcap: Let GSI DCAP use gPlazma for certificate chain processing
666b447
dcap: Refactor javatunnel
14f23b2
chimera: Refactor and fix performance test tool
73b2d94
doors: Fix timeout handling during retries
1ec6400
Fix timeout math to avoid overflow
604a0ba
srm: Fix credential delegation
517055b
doors: Fix string parsing for door tags and paths
b4fd156
chimera: Invalidate stat cache
5a1399b
pnfsmanager: Refactor inode wrapper
1cc90bd
dcap: Respect dcap.authz.anonymous-operations when publishing login information
56edf34
srm: Cache the vomsdir and ca stores
0f77a7a
srm,dcap: Respect cadir and vomsdir settings
143652b
cells: conversion and improve documentation of get hostname command Motivation:
8a0915d
cells: conversion and improve documentation of get hostname command
ecbb816
CellGlue: Bug fix in CellGlue
3711b70
rpm: remove “commented out” macros lines from spec file
decafd2
cells: Fix dumpster and default route removal
c6eeea9
cellshell: admin interface commands (method -> class conversion)
6e2f642
edited
2622c8e
edited
d00b041
chimera: fix nameof and pathof for paths containing unicode
a20f6d1
pool: include client’s InetAddress when discovering corresponding mover
749e39e
poolmanager: Make psu output deterministic
9ba6599
httpd: Fix transfer collection for duplicate movers
9dae274
pnfsmanager: Remove uid,gid and mode from upload path creation
40a9516
chimera: Let SRM respect setgid on upload
e9355db
srm: Add authorization to put done and abort requests
bac6cea
pnfsmanager: Minor refactoring of access mask check for delete
e2c25b1
pnfsmanager: Move PnfsID verification on delete into name space provider
848a10f
pnfsmanager: Push file type check for file deletion to name space provider
08bbd57
pnfsmanager: Deprecate path field in PnfsDeleteEntryMessage
8dab192
module: cells
2784404
nfs4: more java8 cleanups
c67b553
nfs: publish exports paths in login broker
c280f98
nfs: add an utility class to convert CacheExceptions into NFS Exceptions
a71eac1
dcache-webadmin: add TimeoutCacheException to catch clause in billing service
b270a63
Fix broken commit 3bfcc2da
3bfcc2d
pool: fix ftp mover to provide better logging when failing to connect
108e86b
crypto: remove banning of broken DH ciphers
7205931
crypto: ban RC4 by default
7de9b7b
PoolManager: batch file typo
9806a32
PoolManager: batch file typo
09db1ed
properties: update description of dcache.net.listen
3890809
ftp: consolidate transaction-log creation
e8aa7d1
crypto: refine handling of broken ciphers
26182b7
libs: update to nfs4j–0.10.6
21cc1d3
improve to the webdav.templates.config!footer URL
2643c63
ftp: minor refactor
ad7214f
chimera: make discovery of directory’s parent more efficient
571f65d
pool: always use on-transfer checksum
abc4583
doors: Include root path in login broker info output
4e1f316
admin: Fix cell name tab completion
e23a630
cell: Don’t quote string command in event logger
24b08f9
srm: Fix read path check for srmPrepareToGet
0235ddb
pool: Avoid rereading cache entry when enqueing file for flush
9f5871d
pool: Explicitly enable compressed oops to calculate correct cache size
4ec8d0c
pool: Fix locking bug causing high memory usage during pool initialization
3e3800c
chimera: fix FsInode_CONST#stat()
a1bc2be
chimera: add PostgreSQL 9.5 specific driver
b37e0eb
chimera: Fix path resolution on delete
8337f1f
gplazma: Remove synchronization making gPlazma single threaded
80b613a
xrootd: Refactor xrootd redirect handler concurrency
ce113f3
xrootd: Fix ‘xrootd logs stack-trace on malformed request’
ca0b0a5
move execution of the superclass method before any concrete class initializations
8b4d397
gplazma: Fix VOMS plugin regressions
5e19ef8
pool: do not list a repository during initialization
73441f8
xrootd: Add session to access log
cecd533
xrootd: Add support for custom kXR_Qconfig replies
7e25b6b
pool: update abstractMover to use ChecksumChannel on write
cd7637d
pool: Fix race condition in migration module
095d689
xrootd: Fix GSI authentication
14ee077
pool: Disable pool on meta data failures
1d7f0b4
srm: fix race condition in ls response
3c70fa2
chimera: fix acl triggers and changeset rollback procedure
a90d59b
srm: Add transfer strategy plugin mechanism
a34fe50
srm: Fix wrong Spring factory bean type
a16d63a
srm: Fix illegal state transition from Failed to RetryWait
f72fa50
acl: remove origin evaluation
e6847b4
PoolStatisticsV0: refactoring
df69b4a
PoolStatisticsV0: more whitespace changes
831a988
PoolStatisticsV0: fix indentation
91e1084
gplazma-xacml: add GlobusPrincipal (DN) to identified principals
c345ed2
systemtest: fix install command in credentials command
0dc4063
chimera: throw FileExistChimeraException if tag already exists
75749f3
srm: Fix merge failure in ‘Make supported SRM protocol versions configurable’
ad21ac8
srm: Make supported SRM protocol versions configurable
36fc0b5
webadmin: do not display numerical value for max restores or stores
e1d998f
p2p: include source pool into thread name
0263743
src: add README.md to make Github page more friendly
f22243f
Revert “webadmin: do not display numerical value for max restores or stores”
fcd2542
webadmin: do not display numerical value for max restores or stores
44b3ed9
srm: Fix illegal state transition from Canceled to Failed
10e2f67
all: Clean up FQAN extraction from Subject
0762be1
ftp: add support for SITE SYMLINKFROM / SYMLINKTO commands
7b3fbd5
boot: Don’t fail on missing layout file
5279408
all: Replace direct Predicate instantiation with lambdas
a1f50f6
all: Replace direct Function instantiation with lambdas
63ad6e8
srm: Use Java 8 constructs for SchedulingStrategy implementations
edfa418
srm: Refactor scheduler to avoid unused queues and per user counters
214963f
srm: Reimplement scheduling policy to resolve scalability issues
6302038
webdav: provide a human-friendly size as file-size hint
3a67610
ftp: add support for SITE CHGRP command
38ec2c7
httpd: Do now show maximum for restore and flush queues
53f637e
deb: Downgrade recommended dependencies and add to description
ec1c6bb
poolmanager: Reduce log level of p2p denial
0da7ac6
pool: remove redundant code in AbstractMover
1823449
pool: do not use RandomAccessFile in FileRepositoryChannel
99b1d03
libs: update to nfs4j–0.10.5
f1a0f51
infoprovider: remove single SRM instance limitation
edffb45
pool: simplify duplicate request handling
57f5e2c
pool: do not create new stateid in NfsMover#getStateId()
890fd80
pool: reduce load on back-end file system
2cda82d
pool: introduce unique port number for nfs mover
0b8d50d
pool: Store nearline storage timeouts to pool setup file
fae7341
pnfsmanager: Create base upload directories without tags and acls
9feb7c3
pnfsmanager: Inherit ACLs on upload with SRM
d38b70a
chimera: Add ACL insert triggers for HSQLDB
73fc966
Fix limited class path generation
f39aa93
pool: Let random pool selection select pools with removable files
8b72498
crypto: allow banning of RC4 cipher suites
dfafdf6
spacemanager: Allow unowned files in reservations
7090629
srm: Don’t log erroneous stack trace
c7581cf
Delete deprecated classes
bc29c80
configuration: Mark deprecated properties as obsolete
d8244d4
configuration: Delete obsolete properties
5400ea8
configuration: Delete forbidden properties
20cce56
nfs: include read/write information to transfers
59bf7e4
replicamanager: use Collections.emptyIterator() when needed
6a2d877
releases: update dCache version to v2.14