Highlights

In preparation of upcoming changes, dCache 4.0 contains many small architectural improvements, bug fixes and structural changes that will be needed for new features that will be incorporated into later releases. Apart from greater stability and improved error handling in many circumstances, there are few user-visible changes in this release.

For systems where local users also tend to be members of VOs, dCache 4.0 brings a helpful change in the way group membership is assigned. gPlazma can now be configured so that several authentication plugins cooperate in selecting and setting the primary GID of a user account, so that a user can work in a mixed environment where he or she is recognized through both a local account and a VOMS identity.

In order to facilitate moving clients to using different checksums, the pools can now be configured to calculate several checksums for any uploaded file at once. A file may thus get both an ADLER32 and an MD5 checksum, for example.

Incompatibilities

The change to gPlazma mentioned above may change the behaviour of some more exotic gPlazma configurations. In case a configuration exploits the fact that previously, attempts to assign multiple primary GIDs led to failures, any behaviour built on that assumption will now change because the authentication plugins will cooperate to assign one of the GIDs as primary and the other as secondary.

Acknowledgments

Once again, we have received important code contributions from our users.

We would like to thank Christoph Anton Mitterer for his work on systemd integration, Onno Zweers for spotting an https issue with WebDAV, and Robin Wenzel for his contributions during the HTW course.

Differences from dCache v3.2

The notes for release 4.0.0 detail the differences from dCache 3.2. Please read this section very carefully when upgrading from this version.

Release 4.0.41

ftp

Now clients can request the checksum value of a file not owned by that user and where dCache does not already know the checksum value.

pool

The current release fixed some logging on the pool where messages were recorded against an arbitrary context (i.e., the bit in square brackets), resulting in misleading information.

Changelog 4.0.40..4.0.41

63f22d4
[maven-release-plugin] prepare release 4.0.41
adb82a6
pool: fix CDC for repository listener notification
ab22edd
ftp: store calculated checksum using root privileges
7a1241a
[maven-release-plugin] prepare for next development iteration

Release 4.0.40

webdav

When users request a macaroon via an HTTP POST request targeting a specific path, a caveat is created that restricts the macaroon to that path (requests to / result in a non-limited macaroon).

Commit 99c726e3 resulted in users getting back a non-limited macaroon for every request. This issue was fixed with this release.

Changelog 4.0.39..4.0.40

a7d821186e
[maven-release-plugin] prepare release 4.0.40
eae47330f3
systemtest: fix OpenSSL DN format change
6022036556
webdav: fix path-to-caveat for macaroon minting endpoint
e1f5f5ea04
[maven-release-plugin] prepare for next development iteration

Release 4.0.39

alarms

Pool errors involving a fatal repository fault, for instance, can be sent now as an email alarm without having to send all pool disabled alarms.

gplazma

The JAAS gplazma plugin no longer logs a stacktrace on bad configuration.

pool

The current release improved error messages making them clearer by avoid using the same error message in multiple places.

srm

A stack-traces fixed for concurrent updates in pin-manager and similar (expected) failures.

transfermanager

Now Third-party transfers fail if the client is requesting to copy a file from dCache that has not fully been uploaded.

webdav

Disabling basic authn should not now disable macaroons. The current release fixed webdav.authn.basic and frontend.authn.basic so that setting these configuration properties to false no longer blocks macaroons from being accepted in the HTTP Authorization header.

The current release impoved error messaging for unauthenticated request.

IllegalArgumentException exception is fixed now and attempts by a client to copy a file that has not fully been uploaded results in a clear error response.

Changelog 4.0.38..4.0.39

6130d2b
[maven-release-plugin] prepare release 4.0.39
47a0806
webdav/frontend: disabling basic authn should not disable macaroons
796ae0b
srm: do not log a stack-trace on expected Exception errors
cf4dada
transfermanager: fail third-party copy if the file is still being uploaded
cfae242
webdav: fail COPY early if file is currently being uploaded
65791b3
transfermanager: abort transfer if there is a bug
7e91526
gplazma: JAAS plugin logs a stack-trace on misconfiguration
5969503
pool: avoid using the same error message in multiple places
7627e34
alarms: add pool dead alarm
9bfef0b
[maven-release-plugin] prepare for next development iteration
d3864b2
pool: don’t update atime on flush
8ee187f
scripts: fix ‘dcache pool yaml’ command
fc2b228
webdav: 401 for unauthenticated requests; message in status line

Release 4.0.38

ftp

The current release fixed MLSC command for non-small directories and Globus is now able to list directories with > 100 directories.

Changelog 4.0.37..4.0.38

95552ec
[maven-release-plugin] prepare release 4.0.38
dd51676
ftp: fix MLSC command for non-small directories
06df72a
[maven-release-plugin] prepare for next development iteration

Release 4.0.37

billing

Database connection loss now is reported for billing.

webdav

The current release fixed the problem where all but one requests fail, if multiple concurrent PUT requests have directories in the path that do not already exist.

Changelog 4.0.36..4.0.37

78f3f48
[maven-release-plugin] prepare release 4.0.37
dd446d5
dcache: wrap billing data source with AlarmEnabledDataSource
6f1e59c
common: fix random data generation in TimeseriesHistogram unit test
2cc139b
webdav: work-around Milton racy API for creating collections
93ae769
webdav: fix name of root
d1931d6
[maven-release-plugin] prepare for next development iteration

Release 4.0.36

gplazma

Since update to newer BC and voms-java-api libraries sites report VOMS certificate validation errors like This is now fixed.

srm

The dcache ports command now includes the srm’s TLS/SSL interface.

Changelog 4.0.35..4.0.36

38c75f5
[maven-release-plugin] prepare release 4.0.36
e2928ad
gplazma voms plugin: add trust anchor refresh paramater
1001dd6
srm: include TLS/SSL port in ‘dcache ports’ command
790ca89
[maven-release-plugin] prepare for next development iteration

Release 4.0.35

Changes affecting multiple services

The current release corrected the properties for access-log.

Changelog 4.0.34..4.0.35

173b6a1
[maven-release-plugin] prepare release 4.0.35
b83c282
correct the properties for access-log
4c8281b
[maven-release-plugin] prepare for next development iteration

Release 4.0.34

Changes affecting multiple services

If a client specifies a checksum value with either a WebDAV or FTP upload, a Restriction check by-passed due to missing path warning was logged occasionally. This was fixed now, ensuring that restrictions are always applied.

pool

Space reservations on pools that are connected to tape showed a problem with failing restore requests: If a restore failed, the space that was reserved to hold the file that was supposed to come in from tape was not freed again but kept in the ‘sticky’ state. This resulted in lots of unusable space on pools that could only be reclaimed through a restart.

With the current release, this issue is fixed and space is freed as soon as possible after a failed restore request.

resilience

A very rare race-condition is fixed where a failed upload results in resilience recording a stack-trace.

webdav

An issue with the Milton WebDAV library prevented Partial (or vector-read) GET requests from succeeding. This was fixed now through both an update of the dependency and a local patch while we wait for the proposed fix to be included upstream.

Changelog 4.0.33..4.0.34

0488d2dc37
[maven-release-plugin] prepare release 4.0.34
164b539921
webdav: fix proxied partial (vector-read) GET requests
e7dc702f29
pool: fix pool space accounting on failed restores
ccbfedab7d
resilience: fix NPE if file unlinked when resilience processes a broken file
307d443951
ftp/webdav: fix bypass of restrictions
e426e1667b
[maven-release-plugin] prepare for next development iteration

Release 4.0.33

alarms

An internal issue with the alarms configuration was fixed, which should prevent a rare NullPointerException from occuring.

dcap

Creating a file or directory using the DCAP protocol with a URL as parameter, the file permissions were not set correctly.

With the current release, this was corrected, and such files use the client-supplied file permissions. If none are provided, the default modes 0700 (for directories) and 0600 (for files) are used.

xrootd

An uncaught exception in xrootd doors was fixed.

Changelog 4.0.32..4.0.33

62657c3f71
[maven-release-plugin] prepare release 4.0.33
cb1020343e
alarms: fix persistence.xml configuration
6423f196ea
dcap: fix permission propagation with DCAP
8c2e8da647
dcache-xrootd: handle possible race condition in directory listing
8b772286f2
[maven-release-plugin] prepare for next development iteration

Release 4.0.32

statistics

Metadata merge was using max when it should had used min, this is now fixed.

Changelog 4.0.31..4.0.32

53c9888
[maven-release-plugin] prepare release 4.0.32
cf06159
common: fix histogram metadata merge
6574f09
[maven-release-plugin] prepare for next development iteration

Release 4.0.31

ftp

The current release provides better protection against leaking proxy/data TCP sockets if client aborts a proxied transfer.

srm

Clients that use the gridsite protocol, such as davix, can now delegate their credential.

Changelog 4.0.30..4.0.31

4172208
[maven-release-plugin] prepare release 4.0.31
3b060dd
ftp: make shutdown more robust
30ae5dd
common: fix bug in CountingHistogram index computation
ccef077
[maven-release-plugin] prepare for next development iteration
a87fa21
srm: gridsite fix querying validity of delegated credential

Release 4.0.30

ftp

The performance markers that dCache sends back to the client in FTP transfers are now more robust against bugs.

nfs

When transient errors in pools cause NFS transfers to have to wait and retry, the system’s behaviour is now more robust and no StackOverflowErrors should be logged any more.

scripts

Maven’s findbugs plugin is now granted more working memory in order to make builds, especially on our continuous integration system, more robust.

srm

Certificate lifetime considerations for VOMS proxy certificates are improved in this release: if a client delegates a credential where the VOMS AC expires before the X.509 proxies, dCache now will not use the credential beyond the AC expiry time. This avoids unnecessary authentication errors.

webdav

When the WebDAV door is considering an HTTP third-party-copy request that uses grid-site delegation, there is a minimum 20 minute validity that any existing delegated credential must satisfy. If this is not satisfied then dCache will request a fresh delegated credential.

Until now, if the client failed to delegate a fresh certificate then the subsequent COPY request was rejected. This release changes that behaviour and enables such transfers.

Changelog 4.0.29..4.0.30

7cd371066d
[maven-release-plugin] prepare release 4.0.30
37a7212a0d
scripts: Avoid findbugs memory errors
a619fa8c93
nfs: increase request retry delay when selecting/starting pool or mover
b5371e9fae
webdav: adjust minimum validity after requesting delegation
7ff0b933bc
srmmanager/webdav: consider VOMS AC validity of delegated credential
f226a8503b
ftp: make performance marker task robust.
0f14230923
[maven-release-plugin] prepare for next development iteration

Release 4.0.29

pool

Diagnostic logging for failed HTTP third-party transfers was improved.

Billing records for failed transfers now show more detailed information.

The handling of cancelled flush requests for nearline media was rewritten to be more efficient. This resolves issues where pools report “Flush of 0000… failed with: CacheException” followed by “Pool restart required: Internal repository error”.

Compatibility with DPM was improved by increasing HTTP GET requests’ timeouts. This should allow more transfers to succeed.

poolmanager

Supplying poolmanager with an unresolvable hostname as the target will now result in an UnknownHostException instead of the previous behaviour where an (unnecessary) NullPointerException was thrown.

srm

Logging of errors in the SRM credential store was improved.

webdav

If a non-resolvable host name is given as the source or destination of a third-party copy request, WebDAV will now fail the transfer immediately instead of waiting for a Poolmanager timeout.

Diagnostic logging for failed HTTP third-party transfers was improved.

xrootd

dCache allows xrootd clients to specify a query/opaque string in a kXR_mv request’s source path.

Changelog 4.0.28..4.0.29

14ee345617
[maven-release-plugin] prepare release 4.0.29
e7f1b255c1
pool: HTTP TPC rework exception logging
90e0a66e0b
pool: increase TPC socket timeout for GET requests
a5d3b52cf8
srm: fix credential store logging
46650bca9d
pool: update log status using exception class name if no message
fe91d5e0ea
xrootd: strip off query part from kXR_mv source
53f015e9a1
webdav: fail TPC request early on unknown hostname
6bbd2be1e0
nearline-provider: do not propagate thread interrupt flag
e5ba64481a
poolmanager: fix NPE on unknown host
5268bd3785
webdav: improve logging of TPC requests
1c22d720e2
[maven-release-plugin] prepare for next development iteration

Release 4.0.28

Changes affecting multiple services

In order to more easily identify a rejected macaroon in the logs, its ID is now included in the log message.

An irrelevant stacktrace was logged on unexpected CacheExceptions. This was removed, leading to less clutter in the logs.

Different macaroons that were issued against the same secret are now discernible in the logs.

Users now get more information about the reasons why an invalid macaroon was rejected: HTTP requests that are made with an invalid macaroon have a 401 HTTP response with the status-line explanation phrase that describes why the macaroon is invalid.

The access log file also logs why a macaroon was rejected.

core

A library dependency was updated to avoid CVE–2018–11771. This patch introduces no user-visible changes.

gplazma

Invalid macaroon logins no longer “spam” gPlazma.

pnfsmanager

When creating a macaroon to allow uploading of data, the desired path may not already exist. Without restrictions, WebDAV will auto-create parent directory items that are missing, or the client can create these directory elements explicitly with MKCOL.

With restrictions (such as from a macaroon) such directory creation currently requires the MANAGE activity, which allows other actions beyond the scope of this scenario. With this release, the behaviour was changed so that a user with a macaroon that authorises them to upload data into a particular directory will be able to create parent directories to achieve uploading the data.

pool

A regression caused pools that had their size only specified in a layout file to report a size of 8 Exabytes. This issue was fixed.

dCache now supports a DPM-specific HTTP extension that indicates the checksum calculation is not yet complete, avoiding potential data corruption with third-party copies: If DPM is calculating a checksum, then any RFC 3230 (i.e., with a ‘Want-Digest’ header) GET or HEAD request returns ‘202 Accepted’ respond status line and an HTML page as the response entity. Since dCache considers any 2xx response as success, the HTML page was previously accepted as the file’s contents, resulting in data corruption.

dCache pools no longer log a stack-trace for non-bug P2P failures.

srm

The domain ‘.access’ log file now contains log information for grid-site delegation activity, which facilitates debugging of http third-party-copying issues.

transfermanagers

The “restriction check by-passed” warning for each WebDAV-initiated third-party transfer is fixed.

webdav

A user may request a macaroon by making an HTTP POST request to the WebDAV door. This log entry was augmented by the ID and type of macaroon used.

A previous patch needed a bit of an update to ensure that X.509-with-FQAN authenticated third-party transfers with macaroons work under all circumstances. This is now ensured.

xrootd

The --zip option of xrootd clients is now supported.

Changelog 4.0.27..4.0.28

3bf021ba1e
[maven-release-plugin] prepare release 4.0.28
d46ba16630
xrootd: add support for kXR_stat on open files
27c9a340ac
pool: P2P failures trigger stack-trace
545d5b6fa4
webdav: obtain FQAN from X.509 credential for gridsite
a586818c11
core: avoid sending bad macaroons to gplazma
5e929b214e
webdav: update access log to record macaroon request details
a9341e53a5
transfermanager: fix missing path
c0399321e5
libs: update to commons-compress–1.18
155e4a994f
macaroons: include macaroon id in error message
a32b8e4615
pool: fix pool’s runtime configured size regression (b70b0d9)
218baaf895
core: provide better feedback and logging if a macaroon is rejected
53f5d50733
pool: update HTTP TPC to support retrying GET and HEAD requests for DPM
780729397a
srm: add gridsite delegation interface access-log
cccc2840cb
macaroons: fix logged id
f489eabbca
core: avoid stacktrace on arbitrary CacheException
c78c707209
[maven-release-plugin] prepare for next development iteration
f1dae40568
pnfsmanager: allow restricted user with UPLOAD to create parent directories

Release 4.0.27

poolmanager

This release increases responsiveness for users that are not allowed to stage files, and for NFS users who access offline files. In cases where such a user issued a read request at the same time that Pool Manager handled a staging request, the first request would block for the duration of the staging – potentially quite a while. From now on, users that are not allowed to stage receive appropriate error messages as soon as possible, without having to wait for anyone else.

xrootd

Support for xrootd mkdir was improved.

Changelog 4.0.26..4.0.27

1663c9b03f
[maven-release-plugin] prepare release 4.0.27
c462f71ec9
xrootd: update to xrootd4j dependency to 3.2.3
185a3309e4
poolmanager: do not squash request if state is not allowed
a06303a09b
[maven-release-plugin] prepare for next development iteration

Release 4.0.26

sysytemd

Systemd did not inherite the system-wide limits and was completely ignoring /etc/security/limits.d/92-dcache.conf. This is now fixed and the limits successfully loaded and enabled as expected.

vehicles

The current release has fixed serialization regression in FileCorruptedCacheException.

Changelog 4.0.25..4.0.26

1ac0254
[maven-release-plugin] prepare release 4.0.26
e586e55
vehicles: fix serialization regression in FileCorruptedCacheException
f9bb407
[maven-release-plugin] prepare for next development iteration
22b1b4b
systemd: Add /etc/security/limits.d/92-dcache.conf in the dcache systemd unit and generator.

Release 4.0.25

Changes affecting multiple services

This rlease fixes an issue with WebDAV 3rd-party-copy requests that are authorized using a macaroon that is only valid for writing a specific file.

NOTE: both the webdav door and transfermanagers must be updated before the fix is effective.

pool

In order to help with debugging issues with partial FTP transfers, dCache pools now are able to log considerable information about failed FTP transfers.

This is controlled by the new property pool.mover.ftp.enable.log-aborted-transfers.

webdav

dCache can now transfer data with a remote site, authenticating with that remote site using a delegated X.509 credential, but authenticating locally with a macaroon.

xrootd

This release updates xrootd4j, which should help fix occasional “pad block corrupted” issues with older clients.

Changelog 4.0.24..4.0.25

48c07f240b
[maven-release-plugin] prepare release 4.0.25
290f74b701
pom.xml: update xrootd4j dependency to 3.2.3
cafc1ed327
webdav: use TLS credential directly for gridsite
abe9a39d34
pool: instrument ftp mover to show partial transfers
d797026a5f
webdav+transfermanagers: support TPC pull with targeted macaroons
e2b1af7319
[maven-release-plugin] prepare for next development iteration

Release 4.0.24

frontend

The current release fixed broken directory QoS reporting and now frontend now more accurately describes the QoS of directories; i.e., the QoS that newly written files will receive when written into this directory, assuming none of the targeted pools are volatile.

webdav

the macaroon creation with multiple path restrictions failed with a http error 500 and the error message. This is now fixed and the macaroon creation succeeds when multiple path restrictions are defined.

The current release improved error handling for PROPFIND request.

Changelog 4.0.23..4.0.24

1c28c0f
[maven-release-plugin] prepare release 4.0.24
e98ada9
frontend: fix broken directory qos reporting
49e4bf9
webdav: avoid throwing any exception when listing a directory for PROPFIND
ca73622
webdav/macaroon: Fix macaroon creation with multiple path restrictions.
a84d53f
[maven-release-plugin] prepare for next development iteration

Release 4.0.23

ftp

dCache now has the ability to log the current status of a transfer at the point the client decided to abort an FTP transfer. This should support a post mortem investigation on why a transfer was cancelled.

nfs

With the current release the timeout of pnfshandler is configurable and nfs door quicker recovers from situations, when a PnfsManager is not available.

Changelog 4.0.22..4.0.23

f0a1bc1
[maven-release-plugin] prepare release 4.0.23
2a6a8af
ftp: add ability to log client-aborted transfers
c319b03
nfs: make timeout of pnfshandler configurable
4501e58
dcache: release dcache-view version 1.3.3
03f1f5c
[maven-release-plugin] prepare for next development iteration

Release 4.0.22

NFS

When two clients A and B operate on a file in quick succession, A opening the file and B deleting it before LAYOUTGET is called, dCache puts the transfer into the list of active transfers and returned NFS4ERR_NOENT. If a client tries to optimize the corresponding CLOSE call away, as some do, the entries are never removed from the list, effectively creating a leak.

This problem was fixed. Clients now receive an NFS4ERR_STALE message in those cases.

core

Certain transfer failures, such as attempting to use a space-reservation that has insufficient capacity, resulted in the door eventually reporting a time-out problem to the client.

A typical error message would resemble

Request to [>SpaceManager@local ... ] timed out.

This problem was traced to an internal misconfiguration of a messaging component and is fixed from this release onwards.

frontend

The reporting of a file’s QoS status in frontend was improved. Files that are being scheduled for moving to tape are now reported as ‘tape’ instead of ‘disk’.

pool

A bug was fixed that occasionally caused problems with the pools’ Berkeley DB. This could, for example, be triggered by removing files which were in a flush queue.

A typical error message was, e.g.

27 Aug 2018 12:09:33 (cat2_lhcbtape) [Frontend-dcacheview PoolDataRequest] Fault occurred in repository: Internal repository error. Pool restart required: : CacheExcept
ion(rc=204;msg=Meta data lookup failed and a pool restart is required: (JE 7.3.7) Environment must be closed, caused by: com.sleepycat.je.ThreadInterruptedException: En
vironment invalid because of previous exception: (JE 7.3.7) /space/lhcb/tape/pool/meta java.lang.InterruptedException THREAD_INTERRUPTED: InterruptedException may cause
incorrect internal state, unable to continue. Environment is invalid and must be closed.)
27 Aug 2018 12:09:33 (cat2_lhcbtape) [Frontend-dcacheview PoolDataRequest] Pool mode changed to disabled(fetch,store,stage,p2p-client,p2p-server,dead): Pool restart req
uired: Internal repository error

webdav

Web clients (such as web-browsers) make OPTIONS pre-flight requests to discover what they are allowed to do, according to the CORS standard.

Unfortunately, some web-browsers make the OPTIONS request without presenting any credentials. If the resource is within a protected directory then dCache currently fails the OPTIONS request.

This release introduces a new behaviour where such requests will always succeed, so that browser pre-flight requests are not hampered.

Changelog 4.0.21..4.0.22

70845920c2
[maven-release-plugin] prepare release 4.0.22
744b85aecc
nearline-provides: do not interrupt processing thread on cancel
6114916a07
nfs41: invalidate open-state on layoutget if file is removed
8b9706a3f1
webdav: always respond to OPTIONS request
1020c907e5
core: ensure pool/poolmanager communication receives errors
014507c39d
frontend: add targetQoS for not-yet-flushed tape files
3c0761343d
[maven-release-plugin] prepare for next development iteration
97f716c151
dcache: release dcache-view version 1.3.2

Release 4.0.21

gplazma

The OidcAuthPlugin plugin was updated so that users whos op does not claim name, and does not claim given_name nor family_name can use dCache.

pool

This release fixed the log stack-trace for queue admin commands and now bad admin input for the following admin commands no longer results in a stack-trace being logged:

  • queue activate
  • queue activate class
  • queue remove class
  • queue suspend class
  • queue resume class
  • queue remove pnfsid

poolmanager

NPE is fixed when staging files back from tape and poolmanager.enable.cache-hit-message is true.

webdav

The current release updated default credential delegation for third-party copy so that now requesting a third-party copy using a macaroon does not trigger a failed attempt to OpenID-Connect delegation.

Changelog 4.0.20..4.0.21

107d922
[maven-release-plugin] prepare release 4.0.21
e04ff3b
poolmanager: fix NullPointerException when staging files and reporting hits
f088229
gplazma: oidc fix FullNamePrincipal creation
de926dd
libs: update jetty to version 9.4.11
df6040f
pool: ‘queue’ admin commands not the log stack-trace on bad arguments
3725c41
webdav: update default credential delegation for third-party copy
bd1e000
[maven-release-plugin] prepare for next development iteration

Release 4.0.20

history

This release fixes a bug that could cause startup errors in the history service in the face of network errors.

many

Remote pool monitor would occasionally log stack traces from exceptions
when a domain shut down due to an interrupt. This has been fixed, reducing the number of irrelevant log entries in such situations.

Changelog 4.0.19..4.0.20

7ad65f8310
[maven-release-plugin] prepare release 4.0.20
e74c5b7484
dcache-history: handle Gson syntax errors explicitly
eb7c3fc2fd
cells: add handling of RemoteProxyFailureException nested InterruptedException to UncaughtException handler
3b908b94aa
[maven-release-plugin] prepare for next development iteration

Release 4.0.19

nfs

dCache 4.0 and 3.2 now use nfs4j version 0.15.4, which includes bugfixes for rarely observed deadlocks and incomplete directory listing over nfs.

Changelog 4.0.18..4.0.19

14dd7236a5
[maven-release-plugin] prepare release 4.0.19
649f080b2a
pom: update nfs4j–0.15.4 bugfix version
7a7482e0b6
[maven-release-plugin] prepare for next development iteration

Release 4.0.18

PNFS

pool

HTTP responses now contain more meaningful messages along with the HTTP response codes, instead of only just showing stock messages like “400 Bad request”.

webdav

When the WebDAV door proxies a transfer and a transfer failure occurs, the door previously always just reported “500 Internal Error”. This reporting is now improved, with any more detailed error messages from (possibly) other services taking precedence. For example, if a pool returns a 400 error code, thus complaining about the client’s request, this code is reported instead, which should help with diagnosing the error’s cause.

Changelog 4.0.17..4.0.18

c0bbdf1212
[maven-release-plugin] prepare release 4.0.18
a60f4c4448
webdav: pass on status message phrase to client
81f0c6f7e4
pool: update HTTP mover to report errors as HTTP status message phrase
05aee17272
pnfsmanager: fix digest name handling in get file checksum command
ba593fd70a
[maven-release-plugin] prepare for next development iteration

Release 4.0.17

resilience

Resilience suffered from a bug that would lead to a NoSuchElementException when a pool name no longer mapped to a location known to the Resilience service. This issue has been fixed.

When multiple pools go offline it is possible that all replicas for a given resilient file become unreadable. If the file is not CUSTODIAL, and thus cannot be restored from tape, the discovery of such a file during scanning will generate an error in the ‘history errors’ listing, in the resilience domain .resilience log, and will also raise a general alarm concerning the pool.

There currently exists a command, ‘inaccessible’, which generates a listing of the pnfsids on a given pool which in the current state of dCache have no readable replicas. However, this command takes a while to complete (asynchronously), and the output is written to a file which must be viewed by logging in.

This release introduces ‘refering pool’ information to the error output so that grepping the resilience log for a given pool becomes easier, and adds options to the command to check further details.

Changelog 4.0.16..4.0.17

5e3fca2473
[maven-release-plugin] prepare release 4.0.17
437ac98f5a
dcache-resilience: improve inaccessible file accounting
f060954745
dcache-resilience: skip invalid cancel filters
d36c17d6c9
[maven-release-plugin] prepare for next development iteration

Release 4.0.16

scripts

A regression in the dcache pool convert command was fixed; the command works again.

scripts

The instructions that are printed out once dcache pool convert completes successfully now correctly point to the property that needs to be updated, namely pool.plugins.meta.

Changelog 4.0.15..4.0.16

b602706c09
[maven-release-plugin] prepare release 4.0.16
c34830a7b3
pool: fix ‘dcache pool convert’ command
105b9df7b7
scripts: update reference to configuration property
dbf3dc2f25
pool: fix metadata migration tool to use Path
1b82ee43de
[maven-release-plugin] prepare for next development iteration

Release 4.0.15

pool

This release improves dCache’s robustness against network errors: In case registering a file with PNFS manager fails due to a timeout, the request is retried transparently.

Changelog 4.0.14..4.0.15

d80652cb16
[maven-release-plugin] prepare release 4.0.15
40818ef57e
vehicles: fail-fast on invalid path
c821b17e26
pool: retry request to pnfs manager if timed out
fd3ac055bf
[maven-release-plugin] prepare for next development iteration

Release 4.0.14

door

The current release added support for a door advertising multiple hostnames or IP addresses. dCache doors can now advertise multiple interfaces, including DNS aliases.

webdav

Milton’s OPTIONS handler was returning a 404 error if an OPTIONS request targets an entity that did not exist. This behavior deviated from Apache httpd server and was resulting in failed uploads for dcache-view. The current release fixed.

Changelog 4.0.13..4.0.14

5561a75
[maven-release-plugin] prepare release 4.0.14
f9a8084
gplazma.properties: hint to enable roles
eec385b
doors: support advertising multiple addresses in LoginBroker
07a1332
webdav: do not return 404 for OPTIONS request targeting absent entity
e4c543d
[maven-release-plugin] prepare for next development iteration

Release 4.0.13

dcache-resilience

There was a small regression in the way resilience computes the number of operations necessary to adjust copies when a storage unit definition changes.

The current rellease fixed computation of operation count when storage requirements change.

ftp

In order to aid diagnosing problems when FTP response being lost, now dcache logs failures to wrap/encrypt responses.

webdav

The current release improved error handling for client authentication with OpenID-Connect. A more complete set of information is now logged if OIDC delegation fails, supporting the ability to discover why the delegation failed.

Changelog 4.0.12..4.0.13

4a0a41b
[maven-release-plugin] prepare release 4.0.13
5566df3
scripts: add support for parsing ZooKeeper transaction logs
cd53a25
ftp: log failures to wrap/encrypt responses
57a2b50
dcache-resilience: fix computation of operation count when storage requirements change
aaebfe4
webdav: log errors if OIDC delegation fails
5e18652
[maven-release-plugin] prepare for next development iteration

Release 4.0.12

ftp

Error reporting in the FTP service has been improved: in some mixed IPv4 / IPv6 scenarios, only unclear error messages were reported.

Changelog 4.0.11..4.0.12

921fcf22cc
[maven-release-plugin] prepare release 4.0.12
8d43f6b3b2
ftp: returned error is too vague for meaningful investigation
2c70ac8c40
[maven-release-plugin] prepare for next development iteration

Release 4.0.11

nfs

Situations, where selection process was incomplete could not be manually recovered When selection process incomplete, due to PoolManager restart, there was no way to trigger a new selection. The current release added two new commands to nfs door: transfer retry and transfer forget.

The first command manually re-activates existing transfer by re-trying selection process. The second one should be used to completely ‘forget’ the stale transfer and let client to trigger a fresh selection process.

Changelog 4.0.10..4.0.11

1fb57bc0
[maven-release-plugin] prepare release 4.0.11
2c55714
nfs: add commands to reactivate stale transfers
f7b7fa2
[maven-release-plugin] prepare for next development iteration

Release 4.0.10

gplazma

gplazma now supports a Fermilab-specific authorization data source in JSON format.

pool

During active ftp transfers, connection problems would lead to the rather unhelpful error message “451 General problem”. This error reporting was refactored, so that diagnosis of the cause is now greatly facilitated.

poolmanager

A potential NullPointerException (that was not observed in real-world usage until now) was fixed in Pool Manager.

resilience

A correction to resilience’s error handling results in no more reports on non-resilient (but corrupted) files.

spacemanager

In order to facilitate debugging, Space Manager now logs link-group related content in greater detail.

Changelog 4.0.9..4.0.10

f6a357ef9a
[maven-release-plugin] prepare release 4.0.10
bd37b4e2ef
poolmanager: fix migration command if named pool is removed
c72a4f8912
dcache-resilience: repair over-aggressive handling of broken file messages
64f90b4c0e
pool: fix error message for failed active FTP transfers
00edd07b93
fix the project version in pom.xml
ba6ecef630
gplazma-fermi: fix last modified check in junit test
2e9f266807
spacemanager: add remote pool monitor debug logging
04836c16d7
gplazma-fermi: add mapping plugin to support VO group and username from file
347aedb537
[maven-release-plugin] prepare for next development iteration

Release 4.0.9

Changes affecting multiple services

This version removes the (by now unused) directory /var/lib/cell-info from a default installation. The directory was previously used to store cell info data. With the introduction of the history service in dCache 3.2.1, this became obsolete.

A small bug-fix addresses wrong directory permissions on tar or Debian packages, where the directory /var/lib/dcache/pool-history had the wrong permissions.

The deprecated properties for configuring alarms have now finally been made unavailable.

chimera

An internal update enables chimera to use PostgreSQL 10.

nfs

A modification in IP address handling greatly increases the speed of NFS client access for Linux clients in mixed IPv4/IPv6 environments.

Prior to Linux 4.12, Linux clients with only an IPv4 address would wait for (timeout * retry) seconds when connecting to pools with both a v4 and v6 address. This was fixed upstream in Linux 4.12, but that fix was not backported to e.g. RHEL 7 yet.

Changelog 4.0.8..4.0.9

2f76e7a4a9
[maven-release-plugin] prepare release 4.0.9
8a0338b78c
nfs: filter out IPv6 DS addresses if client connected with v4
b7c2e58c40
chimera: adjust postgres driver provider to new version schema
67ff8d620d
[maven-release-plugin] prepare for next development iteration
210c68ce85
skel: remove extraneous cell-info dir
0039f4f881
packaging: add missing chown and chmod on pool-history
3ed5e40382
skel: make deprecated alarms properties forbidden

Release 4.0.8

dcache-frontend

When the history service was not running or there were other issues with pool connectivity, this was leading to incomplete or missing JSON data from the history service. This is now fixed.

webdav

dCache now supports RFC 3230 clients that request digests using multiple HTTP headers.

Changelog 4.0.7..4.0.8

9ccc4f2
[maven-release-plugin] prepare release 4.0.8
6bc0ec2
webdav: support multiple RFC 3230 ‘Want-Digest’ headers
5dcc727
dcache-frontend: avoid null dereferencing for incomplete pool history data
9f095f9
dcache: avoid NPE from initialization race in RestoreRequestsReceiver in HttpPooMgrEngineV3
099b96a
[maven-release-plugin] prepare for next development iteration

Release 4.0.7

resilience

Logging for cases where file replication was fatally aborted was improved. Previously, alarms messages pertained to the PNFSID of the affected files. In rare cases, like when facing network congestion, many hundreds of alarms could be created. With this change, alarms messages refer to the storage unit, and a suffix based on an hourly timestamp is added to the alarm message. The alarm will thus be incremented during the hour but a new alarm will be created (only) hourly; in this way, those receiving email alerts will receive them once an hour.

Changelog 4.0.6..4.0.7

f2aeaa9cc0
[maven-release-plugin] prepare release 4.0.7
af00a51946
alarms: fix broken path
215206dd06
[maven-release-plugin] prepare for next development iteration
c987846933
substituted Calendar for Instance which was failing.
d2721111b9
dcache-resilience: avoid spamming alarms with abort messages

Release 4.0.6

ftp

A bug (that was not observed in real-world settings yet) that might have caused NullPointerExceptions was fixed in the ftp service.

httpd

dCache will no longer log a stack-trace if HTTP requests are made asking for information from the info service when the info service is not running.

poolmanager

An earlier change in PoolManager introduced regular broadcasting of the stage request queue to various internal listeners. In some cases, this could lead to erroneous NoRouteToHost error messages being logged. These error messages are now being suppressed.

dCache 3.0 introduced a regression where a dCache domain does not start up if it hosts a poolmanager with poolmanager.conf containing either the “rc set sameHostCopy” or the “rc set sameHostRetry” command. This regression was fixed.

resilience

Error handling within the Resilience service was improved.

Changelog 4.0.5..4.0.6

44480946d6
[maven-release-plugin] prepare release 4.0.6
483a90aeda
dcache-resilience: handle properly RuntimeExceptions from tasks
1bc74ba32b
ftp: ensure adapter is closed
ea690bd082
ftp: remove rare NullPointerException when proxying data
775cd7a7f4
httpd: do not log an exception if info cell not running
26771a24b2
poolmanager: silence NoRouteToCell for stage queue topic
51c0a3f5c3
poolmanager: fix poolmanager startup with certain poolmanager.conf content
5a7e6775d3
[maven-release-plugin] prepare for next development iteration

Release 4.0.5

dcache-resilience

When a checksum or broken file message/error is generated, Resilience makes a best effort to (a) remove the broken copy and (b) make another replica. This, of course, is not always possible, particularly if the broken file is the only accessible copy. This resulted in faulty behavior particularly the thrashing noted in the case of a restaging operation which results in a checksum error. This is now fixed.

The current release improved error handling for resilience. It fixed unnecessary Migration Task exceptions resulting from source pools with no replica in the repository.

Now it should be possible for Resilience to use pools blocked only for writes from doors.

packaging

Upgrading to dCache v3.2 (or newer) was resulting in a broken dCache installation due to a missing services.sh file. This is now fixed and upgrade to dCache v3.2 (or newer) from dCache v3.1 (or older) no longer breaks dCache by removing /usr/share/dcache/lib/services.sh.

Changelog 4.0.4..4.0.5

929693c
[maven-release-plugin] prepare release 4.0.5
0468c8b
bad commit put DOWN twice
b6f229b
dcache-resilience: define non-writable pool to mean p2p-client is disabled
f0000cd
dcache: fix remote pool monitor wait bug
dcec6c5
dcache-resilience: repair handling of broken files*
3d0bff3
packaging: check ‘services.sh’ after old rpm removed
1b6734b
[maven-release-plugin] prepare for next development iteration
4f991cf
dcache-resilience: fix bug in source handling with Clear Cache Location messages

Release 4.0.4

cells

The current release added explicit ZooKeeper/Curator monitoring. Events generated by ZooKeeper and Curator are now logged in a new, which may help diagnose problems that are suspected to come from bad ZooKeeper interaction.

frontend

The current release improved the error handling to work with Jackson exceptions.

Changelog 4.0.3..4.0.4

721c637
[maven-release-plugin] prepare release 4.0.4
4f05a13
dcache-resilience: fix wrong assumption about error type in Message
a667e11
cells: add explicit ZooKeeper/Curator monitoring
6e3f58d
frontend: Map requests with bad JSON to HTTP 400 Bad Request status code
8091927
[maven-release-plugin] prepare for next development iteration

Release 4.0.3

nfs

NFS door has been updated to return NFS4ERR_LAYOUTUNAVAILABLE for DOT files.

star

The current release improved documentation to help dCache admins to have a better understanding of how to generate StAR record.

The current release fixed fix printing exception error message for dcache-star script if there’s a problem when run with newer versions of Python.

Changelog 4.0.2..4.0.3

c0d95dc
[maven-release-plugin] prepare release 4.0.3
acf2137
nfs: return LAYOUTUNAVAILABLE for DOT files
feb0d3c
star: fix printing exception error message
fd66ac6
star: update documentation to provide better description of script
00e7464
[maven-release-plugin] prepare for next development iteration

Release 4.0.2

gplazma

A previous release introduced the possibility of several authenticatio plugins cooperating in setting primary group IDs. This patch updates the multimap plugin so that it can also be used in such a scenario.

info

The info service collects information about who is allowed to reserve space. Since some of this information, like VOs, usernames and gids, may be considered sensitive information, this update allows admins to control whether or not to publish them. The default behaviour is unchanged from the previous behaviour, i.e. info publishes everything. If a site admin wants to change this, the info.limits.show-only-vo-authz property can be set to true.

nfs

Accessing a nonexisting file on recent NFS implementations could cause a FileNotFoundChimeraFsException, which is now caught and properly handled.

pool

In rare circumstances, running info on a pool could cause a NullPointerException. This issue has been fixed.

scripts

The dcache script and manpage still refered explicitely to Java 6. This patch changes the phrasing of the respective text.

Changelog 4.0.1..4.0.2

cd1ebaae54
[maven-release-plugin] prepare release 4.0.2
39127754f4
nfs: fix ServerFault on FileNotFoundHimeraFsException
c867d0cad6
[maven-release-plugin] prepare for next development iteration
6df84ff96b
scripts: update reference to JDK to avoid mentioning specific java version
7724b0693a
info: allow admin to control whether non-VO / non-FQAN identities are shown
743093ffcb
authentication: update multimap to cooperate in generating primary gids
29c0144cd1
pools: fix NPE from info command at startup

Release 4.0.1

chimera

A database deadlock was observed in some rare situations with the latest 3.2 releases. This patch resolves the issue, ensuring trouble-free chimera operation.

Changelog 4.0.0..4.0.1

bc32f2c4be
[maven-release-plugin] prepare release 4.0.1
596a316dfb
chimera: fix deadlock in Postgres driver
96a77e3a9f
[maven-release-plugin] prepare for next development iteration

Release 4.0.0

Authentication

gPlazma may be configured so that plugins cooperate in building a list of gids for an authenticated user. Somtimes it is desirable for “group membership” (i.e. gid principals) to come from multiple sources; for example, a user who known to dCache personally and is also asserting their VO membership may wish to access resources based on her identity or based on her VO membership. The new release changes the plugins’ behaviour so that a plugin only adds the first gid as a primary gid if there is no existing primary gid for this user. If there is an existing primary gid then the gid is still added, but as a secondary gid.

Cells

The system cell’s ‘route’ command has been improved so it provides a nicer layout of information and allows filtering out available routes.

FTP

As a workaround for a unusual behaviour in the Globus FTP client, dCache no longer logs java.io.IOException: Connection reset by peer exceptions.

httpd

It was necessary to update the status gathering methods to allow for correct reporting in high availability setups. With the current release, all http and frontend services correctly report all restore requests anywhere in the system.

NFS

dCache’s NFS implementation now supports COMMIT operations.

Poolmanager

Timeouts in contacting PoolManager no longer result in a stack-trace being logged.

RESTful API

With this release, an extensive set of RESTful information services is now available. The APIs will soon be published separately with documentation. All of the services are reachable via the https port 3880. For the most part, these services implement GET to return JSON objects describing dCache state.

The basic paths are:

https://host:3880/api/v1/alarms https://host:3880/api/v1/billing https://host:3880/api/v1/cells https://host:3880/api/v1/namespace https://host:3880/api/v1/id https://host:3880/api/v1/links https://host:3880/api/v1/partitions https://host:3880/api/v1/poolgroups https://host:3880/api/v1/pools https://host:3880/api/v1/pool-preferences https://host:3880/api/v1/restores https://host:3880/api/v1/units https://host:3880/api/v1/transfers

From them it is possible to obtain such things as lists of billing records for a given file, full information on a given pool, including current queue activity, a list of transfers or restores for a given file, histogram data for pool queues, etc.

Most of these services require the user to have admin authorization and to express the ‘admin’ role. For instance:

curl -k -u $USERNAME#admin:$PASSWD ‘https://host:3880/api/v1/billing/reads/$PNFSID’

Fuller information on the subpaths, parameters and descriptions of the JSON objects returned is forthcoming.

Scripts

dCache-StAR now supports PostgreSQL servers on non-standard ports.

The installation on RPM-based systems will now ensure that the user dcache is a member of the group dcache, ensuring that admin ssh keys are usable.

Spacemanager

Error reporting was improved for cases where a user tries to upload into dCache using a space-token where there is no selectable link for this operation. The error message now includes more details.

SRM

dCache now allows an SRM client to specify from which linkgroup a reservation should be made:

dCache, in general, supports space reservation through linkgroups: each space reservation is bound to a particular linkgroup. Uploads involving a reservation automatically select that linkgroup.

Each linkgroup has an independent set of links. The set of links within a single linkgroup may be configured to accept only a subset of all upload requests; for example, only uploads into a particular path (i.e., a specific storage class).

If dCache is configured with more than one linkgroup then it is possible that an upload is allowed by one linkgroup, but is not acceptable to another. Attempts to upload with a reservation made from the “wrong” linkgroup will fail. Therefore, the user may care from which linkgroup a reservation is made.

SRM Manager

If an old configuration still contains references to either of the deprecated properties srmmanager.net.port or srmmanager.net.local-hosts, the move away from these old properties is now documented better.

Systemtest

Since OpenSSL changed the format of DN strings from a slash-separated to a comma-separated one, systemtest needed to be updated to work with the new format.

WebDAV

dCache now supports RFC 3230-style Want-Digest headers. These allow clients to request specific checksumming algorithms.

WebDAV now issues a warning when https redirects are enabled as, in that situation, authentication will be done over https but files will be transfered over plain http.

The WebDAV protocol includes a special return code to indicate that a PUT request was denied because the filesystem is full: 507 Insufficient Storage. This code is now returned in case the underlying dCache system has no available space left.

xrootd

The xrootd implementation now supports the flag kxr_posc, available for the kXR_open operation. In cases where files may be created but (perhaps due to errors) not properly closed, files could end up abandoned with a 0-byte file size. This flag may be used to switch between two behaviours: having the flag set will persist the file, having it unset will ensure that such files are cleaned up.

Changelog from 3.2.0 to 4.0.0

0593745d10
[maven-release-plugin] prepare branch 4.0
d1aaa0cd0f
authentication: allow plugins to cooperate in generating gids
0576e8a571
webdav.properties: warn about redirects to http.
55375da263
systemtest: work with new OpenSSL DN format
c7960f1df9
spacemanager: allow SRM clients to specify linkgroup in reserve requests
2e10b3df7e
srmclient: add support for specifying linkgroup when reserving space
cbb7868d7b
pom: update postgres jdbc driver and HikariCP
50f7d3b22d
dcache-frontend: set killed mover status to CANCELLED on transfer shapshot
1e2d6a1bb4
spacemanager: provide space-specific error message on bad upload
c86eed2b89
webdav: return 507 Insufficient Storage when dCache is full
d905594c23
pnfsmanager: update slow logging admin command help
38c76ed832
dcache-frontend, pools: implement filtering, sorting and paging for pool activity requests
9a7c8af747
dcache-frontend, billing: implement sorting, filtering and paging for record requests
4a2c45b0b3
dcache-frontend: pool info API changes to support sorting and filtering
b8f1a7f8e0
dcache-frontend: billing API changes to support filtering and sorting
3b31bcc56a
dcache-frontend: refactor selection resource
7bedc8c956
nfs: fix transfer leak, if the door failed to start a mover
f06bbb7479
httpd, dcache-frontend: support requests for restore listing when there are multiple pool managers
7d0435cd2b
pool: fix double logging on remote FTP transfer error
311718f650
srmclient: avoid stack-trace and repeated logging
b905d49c43
srmmanager: provide better hints on obsolete properties
7b08a3a660
pool: Fix how certain bugs are logged
08d93a5817
star: support PostgreSQL running on non-standard TCP ports
46b4598c30
rpm: don’t assume existing dcache user is member of dcache group
334488d193
cells: don’t log stack-trace on starting cell with same name as running cell
6c6fdc07b7
statistics: avoid stack-trace on internal timeout
832bd860b4
nfs: fix documentation of nfs.enable.pnfsmanager-query-on-move
8172371192
admin: do not report attempts to connect to missing cell as a bug
ee416ed05e
cleaner: shutdown cleanly
62ca394df9
maven: include dcache.service in Debian packages
8e93126fcb
pool: fix stack-trace when closing dcap mover connection
2dfc71f598
admin: upgrade sshd library to 1.6.0
81e9d17a2c
httpd: return 404 status code on an unknown page
3bc9c497f1
systemd: adding a global dcache.service which pulls in dcache@*.service
304374339f
packaging: include empty var directory: ‘cell-info’
83ef9ffa71
dcache-frontend: eliminate stray spring resource
13a5760baa
dcache-frontend: modify pool and pool group API to be more consistent
bde210e528
webdav: add support for RFC 3230 Want-Digest on PUT requests
c5650e729f
chimera: update schema migration when creating ‘lost+found’ directory.
dea4beb746
caretaker: switch Optional from Guava to Java–8
2c75e5a111
pool: fix stack-trace on bad command input
b14f37759a
pool: fix stacktrace on FaultEvent logging
bb14aa3203
system: Don’t log stack-trace on fatal JVM error
af0a7cd9d8
webdav: restrict RFC 3230 Digest to requested algorithm
aa7975668b
switch to rsyslog v8 configuration language
2ece2fbb95
move logfiles back to /var/log/dcache
3ae071ebe8
dcache-frontend: fix ConcurrentModificationException in ReadWriteData
5d612dc12d
srmclient: refactor ‘srm’ helper script, enforcing environment variables
ccd7d5aaae
history: do not block on startup if PoolManager is not running
e1f4711c9a
dcache-chimera: use Spring to inject resource dependencies for ChimeraCleaner
c6645eaa09
systemd: do not explicitly set StandardOutput=
f91039318a
dcache-frontend: add backend filtering and sorting for snapshot services*
3df441db5e
dcache-frontend: add backend filtering and sorting for alarms service*
5f0968069f
dcache-frontend: adjust level of timeout logging
cdf6aaafb4
dcache-frontend: adjust REST API for Pool Info Resources
df6493a7b2
dcache-frontend: change alarms service API to include proper PATCH and DELETE calls*
8135b51ab9
pool: move space allocation into repository channel
8dbe29df5b
chimera: fix typo in method name
c6f8f7c917
statistics: fix race in “create stat” admin command
6620983d30
webadmin: remove broken css included only for IE
1e198b0942
webadmin: fix CSS in poolgroup view
dd8c44d4aa
ftpclient: use lambdas
7adf2a0332
core: use lambdas
0a79fd1350
srm-common: use lambdas
bfe8293ddf
srmclient: use lambdas
1828ab5bc2
resilience: use lambdas
9e2b73bb3b
ftp: use lambdas
66d1bac7fa
dache-frontend: fix regression in pool info api
d74096f019
common: use lambdas
f84ac19091
common-security: use lambdas
e5b62e12ed
common-cli: use lambdas
7a28588374
chimera: use lambdas
99b9ad4b86
srm: fix stacktrace on database failure
6964af1f37
dcache-chimera: rename ChimeraCleaner and RequestTracker
b6777469d9
dcache-chimera: move RequestTracker to org.dcache.chimera.namespace
1a531efd0b
dcache-frontend: fix shutdown not to cause stack trace in collection services
d1187b74ad
frontend: use Spring to inject resource dependencies
5b20a52bb3
ftp: add work-around for broken Globus FTP client
d1bdf6fd4e
alarms: revert LogEntry.compareTo() to throw NPE on null object
f2f9e624a6
nfs: change the way how directory cookies are generated
a5155ffd0a
resilience: force tag partition checking on scans from admin command and periodic checks
77204df4fe
alarms: fix natural order comparator to use timestamp first
4adcade245
frontend: improve consistency in ObjectMapper
84f61ced67
nfs: fix COMMIT operation sent to pools
5f08bd71f4
frontend: consistent naming and placement
27607d17dd
frontend: use Spring to create resource objects
fa177dad34
pool: refactor ChecksumMover interface
c236c4df31
pool: light refactoring of ChecksumModule
8a53321c87
frontend: serialise Java List objects as JSON List
63d8375a70
pool: remove call to removed command
4038a2ed9f
alarms: fix shutdown timeout
617b74e55f
nfs: use verifier4#valueOf() to create verifier from a log value
9c4e1c6841
dcache-restful-api: batch POST and DELETE on alarms
bcde997e17
checksums: minor refactor and drop some dead code
6f1eab5337
pool: bypass checksum validation if namespace does have a current value
201635ed64
frontend: upgrade to jersey v2.26
6d93ee6f52
frontend: refactor requestor’s identity handling
8ba8a40d02
pool: avoid ‘null’ and other nondescript error messages
a482edd29a
core: don’t use Jersey’s repackaged Guava
fefa20ec11
pool: refactor String.format log messages
377abd9651
cells: update route commands
3bb13fb12a
dcache-restful-api: change the way in which transfers and restores support paging
f689473727
nfs: fix server error when processing access log
def4b6e7c7
nearline-storage: pass allocator to NearlineStorageHandler
e3ad2a513b
dcap: add TTL information to dcap messages
4c66ad3bee
alarms: remove default value for LogEntry received
00ecfe01af
dcache-restful-api: add offset and limit to fetch of alarms
02daf8d4f7
many: add diagnostic information about remote pool monitor
f599f12a61
pool: dont disable pool if mover cancelled before open
cae812b42f
systemtest: authorise developer to login as ‘admin’ via ssh
8f7bc92668
nfs: let nfs door to use existing file location on write
36f471d65e
nfs: fix invalid condition test in access log processing
5515fe429e
dcache: add elapsed time to NearlineData JSON
c45da5d55e
dcache-restful-api: fix configuration bug in billing collection utils
6f4d59338d
dcache-history: fix bug which forces overwrite of data when pool unavailable
e4d4a6077e
dcache-restful-api: fix aggregation issues for pool info and move aggregation to history service
ee58ddf08a
src: fix broken commit 8e223c53e7e3e80
8e223c53e7
nfs: fix javadoc and remove unused imports
16c5abf9e8
dcache: fix NPE bug in TransferInfo.toFormattedString
31fb29e6a3
vehicles: use telescopic constructors for PnfsClearCacheLocationMessage
00b52a359a
dcache-restful-api: avoid NPE in PoolDataRequestProcessor
3825c16a74
dcache-restful-api: lower collector timeout defaults
644b693b07
dcache: fix bug in PoolSelectionUnitV2 match()
a2f148bbd8
pom: update scala dependencies to version 2.12.3
57b9646466
dcache-xrootd: Fix login handshake to support xrootd clients (> 4.7.0)
68050b340e
dcache-restful-api: Add POST for enable/disable or kill mover to pool info service
d5c39daec3
dcache-restful-api: add selection resource and providers
4fcb617771
httpd: restore millis to transfer time for transfers.txt
f87436d3f2
dcache-xrootd: (WIP) Add support to the xrootd (kxr)posc flag in (kXR)open.
fbdef471bd
pool: introduce ForwardingRepositoryChannel - a RepositoryChannel decorator
7cdd8d1820
pool: fix log and alarms duplication when rebuilding broken entry
0893a6828e
pool: remove deprecated command
182e1d2c08
checksums: remove unnecessary Checksum#forType static method
85128586b0
checksums: remove unnecessary parseChecksum call
6f5da5592e
checksum: drop ChecksumFactory
7b8702c012
cells: better error reporting to ensure bugs are understood
66eb380a4e
webdav: refactor MiltonHandler’s support of OPTIONS
7dc058e7e6
webdav: fix regression in OPTIONS response
7c7c510b70
resilience: use scheduled executor to provide optional task launch delay
4bed3af3ac
commons: use Guava to generate hexadecimal strings from byte arrays
87abe73747
dcache (collection service): handle execution exceptions correctly
647a82b4b4
libs: update to emergency fix nfs4j–0.15.3
9d763ab983
libs: update nfs4j to version 0.15.2
b2041cc257
pool: support calculating multiple checksums on uploaded data
cc656a36b3
pool: remove more checksum dead code
d6a55f6bf3
pool: refactor checksum handling
dab89fd89d
pool: remove dead checksum code
8ab445bb00
pool: fix data integrity regression for 3rd-party GridFTP pull transfers
47157c37fc
pool: fix regression in GridFTP OPTS CKSM command
68c1f7c011
common: fix time computation in TimeseriesHistogramTest
9a43bdfe13
Revert “pool: fix data integrity regression for 3rd-party GridFTP pull transfers”
0697f4f8ab
pool: fix data integrity regression for 3rd-party GridFTP pull transfers
9fe1d035c6
resilience: handle file deletion during scan correctly
eff4c847a1
pool: fix logging in mongoDB metadata repository
3d265c0333
pool: ceph: ignore file-not-found on remove
068d9758ba
alarms: set severity from priority in request handler
c65e057bf7
dcache-restful-api: return incomplete info instead of throwing NoSuchElementException
7f092c7c16
resilience: add pool operation logging
36975ead39
resilience: handle storage unit NoSuchElement failure
5d1e8a7566
pool: update dcap mover to use ChecksumChannel
a424af0f61
srmclient: parameterise shell path of srmclient utilities
8893a4f8fc
nfs: shutdown callback ScheduledExecutorService on shutdown
91d32bf275
pnfsmanager: slight simplication
2b10acfefd
systemtest: update robot to use system-test’s trust-store
07718e0988
pool: small tweaks to IO statistics
a518121975
libs: update to nfs4j–0.15.1
4cfe75ce56
access-log: update nfs door to provide access log
31fc3a2dcb
pool: update ftp mover to use ChecksumChannel
8d6f6bc2a4
webdav: adjust header parsing to be case insensitive
6cbac61c2e
ftp-client: refactoring slf4j logging messages
c18f0882ab
dcache-srm: refactoring slf4j logging messages (#3446)
51871cb790
src: replace Guava Methods with native Java alternatives
19e5cd3b0d
cells,dcap,ftp: Support for accepting connections from an allowed list of subnets and IP addresses
a2a51d1c9f
resilience: handle all cases where no locations for file may be discovered
0e60fe2373
resilience: distinguish correctly between file not in repository and file not found
b15896cfe6
resilience: fix bug in formatting and handling of cache exception types
3e3c9fc7f2
dcache-restful-api: extract the locations from storage info and add to JSON attributes
c77b773c5b
common: fix bug in histogram max index computation
fa977c66dc
srmclient: remove non-functioning script with BASH dependencies
24cbab02ea
[maven-release-plugin] prepare for next development iteration