Highlights

  • dCache now supports Java 11 as its platform
  • Documentation, especially the dCache Book, received a major revision and will remain in focus
  • HTTP 3rd-party-copying has matured to a feature-rich, well tested state
  • Pinboard includes timestamps
  • updated external dependencies

Incompatibilities

This release breaks compatibility with pools running dcache version 3.2 or earlier.

Acknowledgments

We thank HTW Berlin students Marcel Munce, Ferdinand Wolff and MaKrHTW (???) as well as Onno Zweers from surfSARA for their contributions.

Release 5.0.4

admin

he current support added direct command execution capability and it works with support for semicolon (;) separated list of commands like so: ssh -p <port> user@example.com "command1; command2; command3".

alarms

Pool errors involving a fatal repository fault, for instance, can be sent now as an email alarm without having to send all pool disabled alarms.

billing

Better formating for storageInfo key, when events send with kafka producer.

dcache-view

Troubles when using firefox and/or safari to browse dcache-view were reported. This is now fixed.

gplazma

The JAAS gplazma plugin no longer logs a stacktrace on bad configuration.

pool

The current release improved error messages making them clearer by avoid using the same error message in multiple places.

The current release boosts performance for ceph pool.

The current release fixed lookup for canonical hostname for IPv6 addresses and now secure HTTP transfers work over IPv6 and problems are easier to diagnose.

The current release improved error message (previously “Could not create mover”) to provide more information about why the mover could not be created.

srm

A stack-traces fixed for concurrent updates in pin-manager and similar (expected) failures.

transfermanager

When transfermanager returns an error to the caller (e.g., WebDAV) if there was a problem starting the mover. This message did not include any details describing on which pool this failure occured. This is now fixed and HTTP TPC failures in which the pool does not start the mover now include the pool’s address in the error message. This allows admins to investigate further.

Now Third-party transfers fail if the client is requesting to copy a file from dCache that has not fully been uploaded.

webdav

Disabling basic authn should not now disable macaroons. The current release fixed webdav.authn.basic and frontend.authn.basic so that setting these configuration properties to false no longer blocks macaroons from being accepted in the HTTP Authorization header.

The current release impoved error messaging for unauthenticated request.

IllegalArgumentException exception is fixed now and attempts by a client to copy a file that has not fully been uploaded results in a clear error response.

The current release added switch to reject macacroons sent unencrypted and now following security recommendations sites may configure dCache to reject any macaroons send over an unencrypted channel. The default behaviour is to continue accepting macaroons sent over an unencrypted channel to avoid breaking existing deployment.

xrootd

The current release fixed access logging when xrootd door is configured with HAproxy.

Changelog 5.0.3..5.0.4

4a0c3d0
[maven-release-plugin] prepare release 5.0.4
e033d6f
webdav/frontend: add switch to reject macaroons sent unencrypted
35e0d9a
xrootd: fix access logging when xrootd door is configured with HAproxy
ac195b1
webdav/frontend: disabling basic authn should not disable macaroons
7ca1116
srm: do not log a stack-trace on expected Exception errors
ba8079f
transfermanager: fail third-party copy if the file is still being uploaded
325caee
webdav: fail COPY early if file is currently being uploaded
0b580ee
transfermanager: abort transfer if there is a bug
086a416
gplazma: JAAS plugin logs a stack-trace on misconfiguration
869ca7e
transfermanager: include pool address in the mover start failure message
ecdad0d
pool: update error messages to make them distinct
6557695
pool: avoid using the same error message in multiple places
3a91693
alarms: add pool dead alarm
49af685
pool: fix lookup for canonical hostname for IPv6 addresses
dc6fd90
admin : add direct command execution capability
cefff6c
pool: grow file prior FTP upload
3ffc516
[maven-release-plugin] prepare for next development iteration
78146ac
pool: don’t update atime on flush
ee4ac8a
pnfsmanager: do not emit IN_ATTRIB event on atime updates
edb882d
scripts: fix ‘dcache pool yaml’ command
48b54c8
webdav: 401 for unauthenticated requests; message in status line
345d322
dcache, frontend: release dcache-view version 1.5.3
4200c68
door: fix issue 4551 (wring storage)
57515f1
dcache: update kafka-client lib version to 2.1.0

Release 5.0.3

dcache-frontend

The current release added documentation concerning restores.

ftp

The current release fixed MLSC command for non-small directories and Globus is now able to list directories with > 100 directories.

xrootd4j

The current release updated the xrootd4j including the following fixes and improvements: add ERROR status to tpc info, change the protocol version to int, prevent NPE when constructing error response, fix path handling in move request, correctly handle multiple authn protocols as indicated by server, handle correctly IO/Security exceptions on credential loading and distinguish correctly between kXR_wait and kXR_waitresp.

Changelog 5.0.2..5.0.3

359b183
[maven-release-plugin] prepare release 5.0.3
ca48ef5
ftp: fix MLSC command for non-small directories
d113cb5
dcache-xrootd: remove mv request hack
76e846c
dcache-frontend: add documentation concerning restores
ce48a8c
pom.xml: update xrootd4j dependency to 3.3.4
b3c20ee
dcache-frontend: undefined suid parameter on transfers should be NULL not “null”
72338e6
[maven-release-plugin] prepare for next development iteration

Release 5.0.2

billing

Database connection loss now is reported for billing.

docs

Improve documentation by clarifying requirement gplazma.ldap.try-uid-mapping option in cookbook-transport-security.md.

nfs

The NFS door assumed that routable IP address, like 130.199.49.35, in general can’t access private subnet, like 10.1.1.1. This assumption was not always true for all sites and ended up with non functional pNFS deployment. This is now fixed.

pool

The current release improved performance for CEPH based pools and fixed regression affected chunked uploads.

The IO chunk size can be configured now for read as well as for write.

webdav

The current release fixed the problem where all but one requests fail, if multiple concurrent PUT requests have directories in the path that do not already exist.

Changelog 5.0.1..5.0.2

2d1f0e7
[maven-release-plugin] prepare release 5.0.2
bd9ac6d
nfs: do not filter device’s IP addresses based on site locality
b3102cd
dcache: wrap billing data source with AlarmEnabledDataSource
8499f47
common: fix random data generation in TimeseriesHistogram unit test
0ebb667
docs: clarify requirement gplazma.ldap.try-uid-mapping option
09ac758
webdav: work-around Milton racy API for creating collections
829a0d6
webdav: fix name of root
c3a548d
pom: use rados4j–0.0.3 with ARM64 support
b730cf4
[maven-release-plugin] prepare for next development iteration
e14cef6
pool: let http mover respect pool.mover.http.chunk-size
0f7763e
pool: fix HTTP chunked upload
5940b45
pool: grow file prior HTTP upload

Release 5.0.1

dcache-view

The following new functionalities have been added for dCache View: Now it is possible to use macaroon for file sharing. Files can be shared by sending the generated link, QR code or macaroon for the files to the person you want to give access to your files. Gravatar request is now make optional and how the images are stored are now more efficient to reduce the number of request made.

gplazma

Since update to newer BC and voms-java-api libraries sites report VOMS certificate validation errors like This is now fixed.

nfs

The current release fixed the bug introduced by ByteBuffer#limit, which is used instead of Buffer#limit.

srm

The dcache ports command now includes the srm’s TLS/SSL interface.

Changelog 5.0.0..5.0.1

f6d2455
[maven-release-plugin] prepare release 5.0.1
b870a14
pom: use nfs4j–0.17.10
a3c86b1
gplazma voms plugin: add trust anchor refresh paramater
cb15ef8
srm: include TLS/SSL port in ‘dcache ports’ command
5eb104c
[maven-release-plugin] prepare for next development iteration
b770e51
dcache, frontend: release dcache-view version 1.5.1

Release 5.0.0

Admin

A new property in the frontend frontend.authz.unlimited-operation-visibility now controls visibility of operations exposing file metadata. The default is false, meaning non-admin users can only see file operations for files which they own or which are anonymous. Setting it to true allows everyone access. (267d937c79).

Monitoring information exposed through the HTTP GET method is now available to all users and not only admin role users. (32597dc77a).

The admin data fields like the lists of pools, groups, units, etc., are now sorted by default for the admin REST API. (4928eff71d).

The dCache admin ssh server now supports kerberos as an authentication mechanism (along with password and publickey).

cbab40a841 added the following property to configure admin ssh server authentication:

(any-of?kerberos|password|publickey)admin.ssh.authn.enabled = password,publickey

The keytab’s location can be set under

admin.ssh.authn.kerberos.keytab-file = /etc/krb5.keytab

Alarms

A bug impeding reception of email alarms when the XML database is used has been fixed.

DCAP

Improved features: when using dcap URL to create a file or a directory, they are created with dcap get desired file permissions.

Frontend

dCache now supports more scientific file formats: HDF4 and 5 files as well as ROOT files are now identified and treated as such.

The new configuration property (one-of?true|false)dcache.enable.authn.anonymous-fallback-on-failed-login = true allows modifying the behaviour of the frontend in case of failed logins: dCache has a hard-coded “feature” where a user providing bad authentication (e.g., wrong password, expired OIDC access-token or macaroon) is treated as the anonymous user. This has proved counter-intuitive, as wrong/expired credentials often appear to succeed for some operations (e.g., directory listing), while failing others (upload/download). Providing the new property allows to set a fail-fast behaviour in those cases, providing a quicker response to users.

To support inotify events, a new plugin for SSE is introduced. Clients can discover changes in dCache namespace using an interface modelled after the inotify(7) API (See dCache book for detail).

dCache View is updated to a new version (v1.5), see dcache-view repository for new feature details.

FTP

Bug which have been fixed:

The leaking server sockets issue , when a client aborts a proxied transfers with kafka ebnabled is now fixed. No further server sockets leaked when a proxy is being used, Kafka notification is enabled, and the client aborts the transfer.

Improved features: Improve date value formatting when sending billing events via Kafka.

gplazma

The credential information (e.g., distinguished name) is now logged for x509 certification chain validation and FQAN extraction failures. (9c39e149e0).

Large numerical value gids may be used to define roles fro groupid (gid). (11b34011ae).

Wildcard match of FQANS is possible for the VO group (vo-group.json) gplazma plugin. 173dca3a96).

A new role, “observer”, is defined and available for according read-only access to system or file information. (4aa440ab2a).

The Storage AuthzDB file format is updated to accept an optional ‘max-upload=<value>’ element after the ‘read-write’ or ‘read-only’ value. The label is optional. If present, the value describes the maximum file size the user can upload. (e3dce67083)

As some newer authentication mechanisms embed usage limitations; i.e., a user may authenticate in a way that limits what that user can do (E.g. SciTokens) New authentication plugins have the possibility to specify a Restriction as part of the authentication process. Existing authentication plugins are supported as before. (204024b9e8).

A new configuration option has been introduced to capture all information about an OpenID-Connect provider, which is some external service that dCache users can authenticate against.

This configuration property is a map. Each entry of the map associates a nickname with information about that provider. The nickname is used when logging problems with the provider. The information is the URI of the issuer endpoint. This must be a valid URL that starts ‘https://’.

The following example associates the nickname ‘google’ with Google’s issuer endpoint.

gplazma.oidc.provider!google = https://accounts.google.com/

(bab4e635ac).

History

Error handling in the history service was improved.

Info

The info service now publishes the time that information was collected along with the actual data. The timestamp is available via the last-updated attribute.

Info clients (such as info-provider and storage-report) are now informed of the number of files stored in a space reservation.

NFS

When pNFS client uses flex_file layout IO errors with pool (data server) are reported to NFS door. The erros can be interpreted as:

NFS4ERR_NXIO: The client was unable to establish any communication with the storage device.

NFS4ERR_*: The client was able to establish communication with the storage device and is returning one of the allowed error codes.

PNFS Manager

A user with a macaroon that authorises them to upload data into a particular directory will now also be able to create parent directories to achieve uploading the data.

A bug that prevented get file checksum from working in some cases was fixed.

Pool

Fixed pool repository space accounting leak on failed restores from tape (815ce3eb6a).

Added Cross-Origin Resource Sharing (CORS) support for HTTP requests (049c87a814) required by dCacheView.

Fixed HTTPS redirected transfers by returning pool canonical hostname in the redirected URLs. (7f81b8e79d).

Fixed stopwath error to ensure that IO-statistics collecting is more robust, avoiding stack-traces with the message ‘This stopwatch is already stopped’ (86ede8a240).

Better handling of HTTP 3-rd party transfers - improved logging of exceptions (a98d667c16), increased socket timeout for GET requests (845cfe0bda). Improved error logging in billing by using exception calss name if exception has null message (24de520285).

Removed stack-trace logging of checked exceptions on P2P failures (7a570355fa). Fixed pool runtime configured size regression (f5ba0103ea).

Updated HTTP 3-rd party copy to support retrying GET and HEAD requests for better ineroperability with DPM (d0a621c775).

Updated FTP mover to log additional information if it detects partial transfers (e725f7b9e7). Dropped subject from StorageInfoMessage (0e60cdcaaa). Fixed regression when restoring files from tape (7cdcf4e0a7). Fixed NullPointerException on flush when using Kafka to collect billing records (4e396b9234). Fixed protocol movers to handle out of disk/out of capacity errors.

Eliminated stack trace generated by bad input for the following admin commands:

queue activate
queue activate class
queue remove class
queue suspend class
queue resume class
queue remove pnfsid

(0faa607806).

Added support for Content-MD5 request header (4d954e6b5f).

Updated HTTP mover to report errors as HTTP status message phrase so that clients that log the status line now provide their users with more detailed information about what caused a transfer to fail (6fcaeca34c).

Fixed regression that broke dcache pool convert command (80461b2f9a and 80461b2f9a).

Introduced a retry loop to retry file attributes update in timeout to pnfs manager (8c60877527.

Pool Manager

Select Read Pool requests for which the user does not have enough permissions now do not affect other requests any more.

Several smaller bugfixes for Pool Manager also went into this release.

Resilience

Bugs which have been fixed: (1) an error is no longer reported when trying to handle a broken file which has already been unlinked; (2) the entire pool scan no longer fails when one file in the list is not resilient or has no locations; (3) filters referencing invalid pool names no longer cause scan cancel to fail.

Improved features: (1) command retry errors immediately reprocesses the most recent failed file operations; (2) the command pool ls now displays the number of file operation errors encountered during a given scan; (3) the list of pools is now sorted by STATE (RUNNING, WAITING, IDLE) and then by pool name in ascending lexicographic order; (4) the inaccessible command now has options to check the status of the job, to display the current contents of the ‘inaccessible list’ file for that pool, and to clean up/delete that file; (5) ‘referring pool’ has been added to the inaccessible alarm to enable grep’ing the resilience log for a given scanned pool.

SRM / SRM Manager

Fixes in gridsite delegation storage handling - fixed querying validity of delegated credential stored on the gridsite end-point allowing clients like davix-cp to work (839604e45f) with dCache; fixed handling of delegated credential with VOMS AC that expires before the X.509 (54658383d1); imporved error reporting (41976be12d); added add gridsite delegation interface access-log (5392271fcf).

SRM client has been updated to support X509_CERT_DIR environmental variable (ed8b86e604).

Fixed handling of duplicate SURLs by SRM client (36b9e0c7d6).

WebDAV

A lot of work has gone into making 3rd party copying functionality more robust and scalable.

XRootD

Third-party copy was introduced in 4.2, and continues to be improved. For further information on configuration, please refer to the documentation in The Book (5.0).

Bug fixes and improvements: (1) the correct error (kXR_NoSpace) is now returned to the client when there is no more disk space; (2) xrootd now fails fast if the MaxUploadSize is supplied, and the file is too large; (3) the xrootd door spring configuration no longer fails to load when kafka is not activated; (4) the ‘stat’ request now supports both open file handles as well as paths, enabling use of the --zip option; (5) dCache no longer logs a stack trace when a client requests a file be created, the parent directory does not exist, and the make parent option is omitted; (6) a source path containing a query part on a mv request no longer causes the request to fail; (7) a potential race condition preventing directory listing now is correctly handled; (8) support for the ‘tpc’ query on the pools has been added in order to comply with the newer (4.9) XrootD clients; (9) it is now no longer necessary nor correct to list ‘access-log’ among the xrootd plugins; this log handler is added automatically as it is for other doors; (10) file handles and query strings are now included in the access log information; (11) logging of failed authentication is improved to include more useful information, like the DN; (12) it is now possible to identify all entries in the access-log from the same TCP connection via a session identifier.

Zookeeper

The internal Zookeeper version was updated to 3.4.13.

Changelog from 4.2.0 to 5.0.0

4631292f51
Revert “docker: Add a way to create docker image”
d411179bb7
[maven-release-plugin] prepare branch 5.0
48868f886e
docs: include CSS for profiled output
c65c5a39d6
dCap -> DCap
5ecd79a82a
pool: add session to xrootd activity
3046573a15
xrootd: fix logging of failed authentication
b13dd2b29c
xrootd/pool: add extra information to xrootd access log file
12e2cdd75d
pool: stop using deprecated API from netty
1d454088cb
utils: remove historic code from diskCacheV111.util.Adler32
cfaefaaba5
docs: TheBook. Also modify child selector pre>code to have auto overflow (scroll).
a9eadd51a6
docs: Minor improvements to install.md
ebaf8076f4
change pre code overflow to auto so that extra long lines are horizontally scrolled.
2c6c4a15f9
dcache-xrootd: hard-code order of access log handler in pipeline
c388d550e1
webdav: fix proxied partial (vector-read) GET requests
0ba973a103
nfs: overcome the 16 group limit of AUTH_SYS
4a9c7206e7
docs: restructure README.md to promote ‘The Book’
63a037ed2e
docs: removed outdated maven archetype option
815ce3eb6a
pool: fix pool space accounting on failed restores
abeaa757e5
docs: Merged OpenID Connect info into gPlazma page
ce15eeaca0
docs: fix archive filenames when generating without profile
65107ac647
docs: move assembly directory to correct location
8975179b97
resilience: fix NPE if file unlinked when resilience processes a broken file
0c8a33e3e4
dcap: fix another potential restriction by-pass
9d4354a9a0
dcap: use path when requesting changes in with PnfsManager
1873fa41bf
docs: add basic navigation
196c5af6ce
ldap: search user by uidNumber attribute if only UidPrincipal is provided
e8d8dbac46
docs: added link to admin service
8cb9eb8fd3
docs: fix text in introduction. Added srm. srmmanager ans spacemanager all linked to config-SRM.md
a02c60a809
docs: stage protection Issue: https://github.com/dCache/dcache/issues/4477
0375c56c15
ftp/webdav: fix bypass of restrictions
e43ad0ebc0
docs: fix classic SE migration guide
0f253b2491
docs: fix reference to admin ‘cd’ command
f61890e942
docs: unify reference to config-message-passing.md
2e69719b3f
docs: merge config-cellpackage.md into config-message-passing.md
af46b787af
Update config-hsm.md
9841279a6e
docs: remove obsolete references to dcache.broker.domain
fa51c31fcd
common: fix email principal string representation
f6d3d95bb3
docs: fix typo in postgres service example
7a795e2c79
dcache-xrootd: add missing query support for tpc on pools
725a30a944
alarms: fix persistence.xml configuration
0ee97b3185
pom.xml : revert back to 3.3.3 version of xrootd4j
553056da1e
dcap: fix permission propagation with DCAP
1440a469fd
docs: drop “The ….”, copy admin to config-admin.md
9ba80a6ce3
docs: Imported HSM plugin page from Github wiki
308edcaf58
dcache-xrootd: handle possible race condition in directory listing
d74ea0c250
docs: remove the HARDWRAPS generation option
b56133e9db
docs: handle ‘detached head’ checkouts when building github link
1674cb743b
docs: fix reference to hoppingmanager
427e74a661
docs: added missing-files service description
8ce1bf0edc
docs: replace cd with \c
0aedbeae8b
docs: replace cd with \c
7068c0c639
docs: Solved Github markdown rendering issue on readme.md
d96bbcb4e1
docs: edited roles section of gplazma
6f441c95a1
docs: Updated documentation of standard ports
a40054c52c
nfs41: update reset pool command to issue cb device notify
08a706ca95
documentation (TheBook): edit frontend, history, gplazma
67f398dbdd
docs: add an ‘Edit Me’ ribbon to dCache pages
72ef6933f6
libs: use nfs4j–0.17.8
83ff307b29
docs: describe dCache pools with CEPH as a backend
c1f651827a
test: add test to validate pool’s runtime configure limit
09d4b376ab
docs: TheBook remove all trailing white-space
8ab972ff0f
docs: Add assembly stage to produce output archive
3b97acc1b6
libs: use postgres-jdbc 42.2.5
c9d4b927ae
docs: add dCache.org profile
9b800c317c
docs: Removed outdated central flushing page
6d38fa2a1f
documentation : fix markdown
10848b4fe6
documentation: some more work on admin section
52cdd9aff8
docs: Link fixes, renamed config-resilience to lowercase
249554a345
docs: add custom header/footer and CSS to TheBook
d8fed7ecbc
docs: add build task to create HTML version of The Book
0008d573bc
docs: Some refinements in the Preface
cc092b14be
the book : add to admin and SRM sections
e7c138fffe
spellcheck
d2c56741f7
the book: remove references to ssh1, rename Securiting to Securing and change “Getting in Touch” to “Getting to know” :)
f4e648b7fc
nfs: use server builder when door is constructed
ab9be7546d
The Book: fix typo
c5185b6f6d
made minor changes to intro and added description of kerberos authentication mechanism in admin ssh server
d6f58b4075
pom: use official release of jacoco
1623ebebdc
pom: use jacoco with java11 support
8f045ec447
dcache-frontend: optimize delivery of aggregate sweeper histograms
a8f838941c
common: fix histogram metadata merge
c246cab1c5
dcache-frontend,dcache-history: revisit NPE fix
851cc59dc9
nfs4: only block pool selection on the first attempt
eb662a97df
restful: Return not Found/404 for non existing pool.
dd2bf55dd2
dcache: release dcache-view version 1.5.1
a941746bbc
nfs: reset pool selection task if pool disabled before redirect
9c39e149e0
gplazma2: Log credential information on x509 cert. chain validation and FQAN extraction failures.
df931cde9a
nfs: ignore JdbcFs errors when constructing acceess log entries
8c9f0ad857
frontend: filter out any IP addresses in door information
bdd96052fb
libs: update nfs4j to version 0.17.7
d1baecfe29
nfs: handle chimera exception on remove of a missing file
049c87a814
pool: add CORS support for HTTP requests
7f81b8e79d
pool: use hostname in HTTPS redirection URL
c63b52bf76
libs: use java–11 compatible aspectj–1.9.2
640796b795
dcache: release dcache-view version 1.5.0
64fbd38cef
dcap: fix NullPointerException:
b319876458
restful: Rename operations to have unique operationId.
9071d4d9f5
ftp: avoid kafka bug, make shutdown more robust
80c7260883
dcache-history,dcache-frontend: check for serialized error when handling pool data request messages
86ede8a240
pool: fix stopwatch error
d9447453ce
common: fix bug in CountingHistogram index computation
c04714455b
libs: update to nfs4j–0.17.6
d67f348e48
cookbook transport security: fix markup and language
e9a1dfc5f6
transport security: CAA records
839604e45f
srm: gridsite fix querying validity of delegated credential
91244c8c74
chimera: support origin tag discovery
1316bfd5a3
Revert “chimera: fix postgres 95 optimization regression”
a9abaa7863
chimera: fix postgres 95 optimization regression
61f765d41f
Documentation: TheBook/config-xrootd.md
6c37e7a294
nfs: increase request retry delay when selecting/starting pool or mover
267d937c79
dcache-frontend: provide switch to control visibility of file operations for non-admin users
bb7649fd73
webdav: adjust minimum validity after requesting delegation
54658383d1
srmmanager/webdav: consider VOMS AC validity of delegated credential
4ac0b589d0
ftp: make performance marker task robust.
333d09a28f
ftp: avoid NullPointerException if adapter is not connected
fca9f993d1
webdav/frontend: add mime types for more scientific file formats
dc624af2ce
cookbook-transport-security: fix typo
a98d667c16
pool: HTTP TPC rework exception logging
845cfe0bda
pool: increase TPC socket timeout for GET requests
f0ecdacd75
scripts: Avoid findbugs memory errors
41976be12d
srm: fix credential store logging
24de520285
pool: update log status using exception class name if no message
25c2c80c24
docs: How to use kafka with dCache
9085ad3ef0
storagedescriptor: update information based on WLCG feedback
a2c4f93487
xrootd: strip off query part from kXR_mv source
895e88c792
webdav: fail TPC request early on unknown hostname
97579c0697
nearline-provider: do not propagate thread interrupt flag
b169df497a
poolmanager: fix NPE on unknown host
1e94e9bb4f
webdav: improve logging of TPC requests
54369c76ea
docs: describe gplazma mutator plugin
65f692867a
docs: describe gplazma-jass plugin
02aa1b424a
libs: upgrade to bouncycastle 1.54, CANL to 2.5.0 and voms-api-java to 3.3.0
75b9a4de20
libs: update aspectj to java11 friendly version 1.9.1
c839a3f381
zookeeper: remove ZooKeeperConnectionExceptionAspect
52be70e5a0
libs: use zookeeper–3.4.13
d48a8b6950
xrootd: fix broken configuration property
3a4573d3e5
Revert “packaging: use private BC 1.50 release that provides JSSE compatible handling of key agreement secret generation.”
2ddfabeb18
src: move away from ListenableFuture
4422bf0466
dcache: use getAddress for uniform client IPs in Transfer info
3ec201ee41
docs: clean up ‘cd’ (and adding ‘\c’)
684f4f37e4
docs: clean up ‘cd’ (and adding ‘\c’)
16f8b1014a
docs: clean up ‘cd’ (and adding ‘\c’)
e755639780
libs: update to commons-compress–1.18
7826c26528
docs: minimal postgres version is 9.2
65505cc8d8
docs: remove reference to Solaris packages
d5cdb1720e
Markdown & typo fix
8fe4e1aea2
packages: remove solaris packages
cf896feae7
Create cookbook-transport-security.md
042ec5e59d
book: fronted, history, xrootd TPC, dot commands, and various indexing changes
bf0d2c623c
book: cleanup alarms, billing, resilience
ea100ecc7f
doc:clean up command line
11b34011ae
gplazma: support large gid values for roles
7a570355fa
pool: P2P failures trigger stack-trace
e655d926a5
webdav: obtain FQAN from X.509 credential for gridsite
0ee7f96d57
transfermanager: fix missing path
5b90514c15
doc:corrections
f5ba0103ea
pool: fix pool’s runtime configured size regression (b70b0d9)
aea18ac9f3
httpd: add path to context/transfers.json
0fcb1926a1
ftp: java.lang.IllegalStateException: Cannot send after the producer is closed.
5b65b41923
core: avoid sending bad macaroons to gplazma
60cb2ab08b
webdav: update access log to record macaroon request details
50ab46b5c2
macaroons: include macaroon id in error message
f797e2f7bc
core: provide better feedback and logging if a macaroon is rejected
e77c317092
Added individual entries for each service in index.
290b91a4e9
docs: Remove info docs about pre–1.9 config
7c72a06a3f
Added individual entries for each service in index.
002c55d569
docs: document all_root nfs export option
07d23e3e74
updating intro and intouch chapters
1230bea81f
dcache-frontend: fix array out of bounds exception in cell info service
a9cf7c6a13
Update intro.md
7a3efaf13d
Update intro.md
4d77e6ec17
clean up links
62332877a2
Update intro.md
803d66c451
Update intro.md
f3702019d9
Update intro.md
45efd38270
changes for book
f97b47179d
pool: fix numerical out-of-range error
5392271fcf
srm: add gridsite delegation interface access-log
052e7a1d67
macaroons: fix logged id
cc2079aa1e
core: avoid stacktrace on arbitrary CacheException
d0a621c775
pool: update HTTP TPC to support retrying GET and HEAD requests for DPM
ececd7871d
pnfsmanager: allow restricted user with UPLOAD to create parent directories
32a166949b
xrootd: remove spurious stack-trace
f0c2ed02fb
xrootd: add support for kXR_stat on open files
192ceaee1c
xrootd: update to xrootd4j dependency to 3.3.3
b6660d2af9
packaging: use private BC 1.50 release that provides JSSE compatible handling of key agreement secret generation.
3d56d28b59
dcache-frontend: fix error message for IdResource
b473eaf6ff
ftp: better address selection for cross-family passive proxied transfers
00fa700c83
webdav/frontend: make anonymous fallback on bad login optional
9a5efb73d2
info/space-manager: monitor number of files in reservation
2dbac4d573
info: display the timestamps when metrics were collected
9488cf474b
poolmanager: do not squash request if state is not allowed
1d08b62dba
libs: use nfs4j–0.17.5
6e90136c12
dcache-xrootd: add missing kafka property
1f1113fa5d
dcache-frontend: add path filter to transfers
d6337e2f12
dcache: add path to transfer information
6c67943842
vehicles: fix serialization regression in FileCorruptedCacheException
869f682936
dcache: Adjust Date formating of Timestamp value for date key for kafka producer
978d9f645b
libs: update to use spring–5.1 release
173dca3a96
gplazma2-fermi plugin (vogroup plugin): allow for wildcard match of fqans
5119c2d283
Update README.md
bfe6f03460
Update preface.md
a41b71a788
Update and rename readme.md to README.md
439b0fe75a
reformat readme.md to book
ff790ee07d
Update readme.md
1fb1e12be3
add readme.md to book
32597dc77a
dcache-frontend: remove admin restrictions on GET and filter transfers on uid if not admin
05d2ecc493
pom.xml: update to xrootd4j dependency to 3.3.1
d07e79e7ca
Revert “dcache-dcap: add uid/gid to transfer info for plain/anonymous dcap”
e725f7b9e7
pool: instrument ftp mover to show partial transfers
0e60cdcaaa
pool: do not include subject with StorageInfoMessage
932e53bdfa
ftp: fix regression in unit-tests
88022876e9
libs: update to nfs4j–0.17.4
9ef883a988
dcache: Creating multiple KafkaProducer instances results in ‘Too many open files’
5b0a1e05fb
webdav: use TLS credential directly for gridsite
3ce0c39393
dcache: add configuration for the Kafka producer timeout
e149c0e305
webdav+transfermanagers: support TPC pull with targeted macaroons
e44fef2e06
Revert “dcache: Creating multiple KafkaProducer instances results in ‘Too many open files’”
93186ef6af
dcache: Creating multiple KafkaProducer instances results in ‘Too many open files’
e07ed8bac9
TransferManager: remove state history class and corresponding table responsible for storing request state changes. It is not used, but may grow rapidly in database.
4aa440ab2a
gplazma-role: add observer role
270b7827c0
dcache-dcap: add uid/gid to transfer info for plain/anonymous dcap
17b5f57e25
systemd: Add /etc/security/limits.d/92-dcache.conf in the dcache systemd unit and generator.
eab3977d89
frontend: add events.html an events test client for webbrowsers
ce1ba5c64d
frontend: add SSE plugin to support inotify events
dae3cf0f09
frontend: fix broken directory qos reporting
e3dce67083
gplazma: support max-upload in storage-authzdb files.
f08a82720f
webdav: avoid throwing any exception when listing a directory for PROPFIND
da1b6259f2
nfs: Instrument NFS door to support inotify events
705a69fc5f
pnfsmanager: add instrumentation to support sending events
b3e803c421
core: add an EventReceiver that sends inotify events
c0ec01c269
frontend: update SSE selection support
6b05aafc30
xrootd: Add support for MaxUploadSize LoginAttribute
884b98910e
dcap: add support for MaxUploadSize LoginAttribute
2d0c003d25
ftp: add support for MaxUploadSize LoginAttribute
594948859c
webdav: add support for MaxUploadSize LoginAttribute
0f8e7ad13f
macaroons: add max-upload caveat
e18020821b
common: add standard byte-size parser
99c726e32c
webdav/macaroon: Fix macaroon creation with multiple path restrictions.
204024b9e8
gplazma: add support for authentication plugins registering restrictions
6ba3c3366d
webdav: add cdmi and dcache-view tags to default properties
1f32e50971
frontend: expose list of available doors
6ed6883e8b
ftp: add ability to log client-aborted transfers
2e81036724
nfs: make timeout of pnfshandler configurable
7cdcf4e0a7
poolmanager: fix staging files from tape
779b458844
dcache: release dcache-view version 1.4.5
6e9d76e22c
dcache-xrootd: add necessary gsi properties for tpc credentials
5069488e79
nearline-provides: do not interrupt processing thread on cancel
5d4515d87a
nfs41: invalidate open-state on layoutget if file is removed
fba83d52ae
docs: markdownify glossary
439834e244
docs: update acl chapter
4e396b9234
pool: fix NPE on flush
a2e18c0d0f
webdav: always respond to OPTIONS request
b7d0836d48
dcache: release dcache-view version 1.4.4
f964703ab9
docs: fix format of index.md
95478a3056
core: ensure pool/poolmanager communication receives errors
af9238bff2
frontend: add targetQoS for not-yet-flushed tape files
8c579ab237
pool: update xrootd handler to support OutOfDisk exception
c8afca7be0
pool: update dcap mover to handle OutOfDiskException
299b435d35
pool: ftp mover suppresses including the IP address when out-of-storage
0a87fa7850
pool: update http mover to return 507 status code on out-of-capacity
5eb12297d2
authn: add MaxUploadSize LoginAttribute
c51663e73a
vehicles: allow doors to specify a maximum file size
706da8f5be
pool: support maximum size when creating a replica
4857f8c813
dcache: release dcache-view version 1.4.3
36b9e0c7d6
srm-client : fix handling of duplicate SURLs
21cdebdb98
docs: add export section to nfs chapter
88c2ebdfb2
pool: fix NullPointerException
81249c9ba7
docs: describe DNS TXT records for nfs id mapping
845700d0c5
docs: revert zookeeper config chapther update
0221e455a8
poolmanager: fix NullPointerException when staging files and reporting hits
b0170e5746
libs: update hazelcast to version 3.10.4
bd49fc1976
gplazma: oidc fix FullNamePrincipal creation
04c4c230fe
docs: Imported HA / replicable services from wiki
aa56d42000
docs: Imported message passing from wiki
75adff92f3
docs: Imported Zookeeper page from wiki
32bd7ac998
docs: Imported Resilience page from wiki
6359c9d951
docs: Recreated main index
ee3eaf4331
libs: update jetty to version 9.4.11
0faa607806
pool: ‘queue’ admin commands not the log stack-trace on bad arguments
c3868cc593
dcache-xrootd: fix third-party billing records
926dfa34b0
docs: remove obsolete chapter about x86_64 deployments
4d8ea3ef60
docs: chimera: clarify PnfsManager, Chimera and db relationship
e9d1615c69
docs: markdonify cookbook ToC
69ff42a865
docs: chimera: remove redundant warning on update of default files
be5308cbeb
docs: remove obsolete multiple-pnfsmanagers chapter
020fc3f17a
docs: config-nfs: describe exports.d directory
5dc9dd3007
docs: update and markdonify config-nfs chapter
00ce43ab30
docs: markdownify config-chimera.md
e287245f87
docs: remove docbook build instruction
977bbc8744
docs: markdonify config chapter’s ToC
2d2e0625e4
docs: fix zookeeper configuration link
bd93447ac8
docs: first pass-through introduction and install guide.
7ec4483204
docs: add zookeeper configuration chapter
90825470f0
docs: remove obsolete lagacy pnfs configuration
6693faf417
docs: move dcahce book into dcache source tree
91d81b79a2
ftp: fix NullPointerException
8e02472d63
ftp: fix scope of used pool stub
bab4e635ac
gplazma: oidc support OIDC providers with non-root issuer URLs.
20de92c346
chimera: do not treat CockroachDB as PostgreSQL.
c93949d506
pool: retry allocation on OutOfDiskException from Allocator
41dcc64d85
pool: refactor Allocator interface and implementations
13fa2bef38
webdav: try fetching HTTP status code when pool disconnects an upload
476c23038f
webdav: report pool response when relaying data
48fbb4be1d
webdav: update default credential delegation for third-party copy
ed8b86e604
srmclient: update delegation client to support X509_CERT_DIR en.var.
7d28721696
srm-server: unified logger Code Convention
7e1324958d
Revert “utils: ensure that pool and poolmanager subs set in Transfer.class”
4aea7281f1
dcache-xrootd: repair handling of delayed sync errors to client
cb3410af35
Revert “util: update Transfer should use poolStub when communicating with a pool”
f237447353
vehicles: remove unused field in infoMessage class
3a46e66a55
dcache-history: handle Gson syntax errors explicitly
c6c850f7d1
vehicles: remove backward compatibility with 2.1 and 2.16 in InfoMessage
5b8235fab4
cells: add handling of RemoteProxyFailureException nested InterruptedException to UncaughtException handler
a79e6b18ab
scripts: allow new java versions as 9, 10 and 11
9904302e9f
dcap: clean code changes
97191a6d42
ftp: add kafka to push messages
8c03fb8c54
dcache: remove unused Spring JDO class
fa8e7dc982
dcache: remove unused jndi initializers from httpd
1fcf3a53b9
core: Removed NotFilter.
791159188f
acl-vehicles: unified logger dcache codestyle
000531916d
core: replaced collection of Predicates by predicate-chain. Removed NotStickyOwnerFilter.
11a9aeb384
core: replaced CacheEntryFilter by native java.util.function.Predicate
a358b50418
docs: fix markdown formatting in README.md
48dd00d072
dcache-resilience: report file check errors instead of fail fast
db2bed33cb
dcache-resilience: add command to retry failed files
66ea9aca4d
dcap: enable possibility to push transfer events to Kafka
590cff1d7c
dcache-xrootd: add support for redirect handling during third-party-copy
c357d030e9
dcache-xrootd: add third-party support to pool (dcache as destination)
ccff945673
dcache-xrootd: add support for third-party copy to door
c147358c5f
poolmanager: fix broke commit 6df58e4d
6de13c8fc7
poolmanager: fix broke commit 6df58e4d
6df58e4dfe
src: remove historical quota manager code
2c13454f3b
nfs: add inotify monitoring wrapper
e6b3e60f31
core: add inotify monitoring wrapper NameSpaceProvider
09a45c33be
pom: use nfs4j–0.17.3 bugfix version
9171a6a7a6
core: split ConfigurationProperties into separate classes
4f63f15433
core: add wrapping namespace
50b8be9ad8
webdav: pass on status message phrase to client
52e13b9a10
ftp: update SocketAdapter to use data transfer direction enum
857c2ce5a0
ftp: describe data transfer mode consistently within the door
2b6113a1cd
webdav: note usage of response element
4d954e6b5f
pool: add support for Content-MD5 request header
73e86d9e9e
webdav: add partial support for Content-MD5 header in uploads
6fcaeca34c
pool: update HTTP mover to report errors as HTTP status message phrase
14fb9ebf59
admin: fix regression in startup
654611041f
dcache-resilience: propagate file op error to pool op and display
9914d17687
pnfsmanager: fix digest name handling in get file checksum command
c3f1b224a6
pom: update package version to 5.0.0-SNAPSHOT
fe96ba2b8f
dcache-resilience: display pool operation list in sorted order
cbaf8faba8
dcache-resilience: improve inaccessible file accounting
0028308a63
dcache-resilience: skip invalid cancel filters
303f5e70a4
nfs: log IO errors report by clients on layout return
d843d6f491
nfs: move layout driver initialization into NFSv41Door#init method
72d6279134
nfs: include pool name into PoolDS object
6e20c58284
utils: ensure that pool and poolmanager subs set in Transfer.class
bae20c30d8
nfs: propagate layoutreturn data to the layout driver
a1b09acf79
frontend.authn.protocol by default https
80461b2f9a
pool: fix ‘dcache pool convert’ command
8f76fe401b
scripts: update reference to configuration property
a2122bff9c
ftp: fix scope of used pool stub
011b3b2439
pool: fix metadata migration tool to use Path
9aed07097f
BUILDING.md: fix typos
7e17e4841d
BUILDING.md - installing a new Maven
64875efd9b
cells: remove historical MemoryWatch
3be6c54171
cells: remove historical dead code
46552da5f6
cells: do not enforce timeformat in pinboard
44fa96e66d
util: update Transfer should use poolStub when communicating with a pool
8167c60f35
ftp: always close proxied data connection if client closes their half
66d304f439
vehicles: fail-fast on invalid path
c117cdc951
util: fix invalid number of args for logging in RunSystem
8c60877527
pool: retry request to pnfs manager if timed out
81a0a97ee1
dcache-frontend: invalidate transfer when killed mover not found
78787fe8ea
dcache-frontend: add “Requires admin role” to alarms methods (Swagger)
429e8ab220
gplazma.properties: hint to enable roles
60c2c3c69a
README.md: add link to packages
cbab40a841
admin: add kerberos authentication support to admin ssh server
4928eff71d
dcache-frontend: provide default sorting on RESTful admin data fields
2b7af29035
doors: support advertising multiple addresses in LoginBroker
05335bdaf1
[maven-release-plugin] prepare for next development iteration