What's new in dCache 12.1
Release notes
Highlights
- Beta support for S3 interface
- Capacity reporting in NFS and xrootd doors
- Mutual TLS for domain tunnels
- Multiple OIDC providers in dCache View
Incompatibilities
- Java 21 is now required; Java 21 to 25 are supported.
- Removed file hopping. Sites must migrate to permanent migration jobs.
- Removed the remaining SRM services and code.
- Kafka: starting 11.2 billing cell cans send billing records to katka. Since 12.1 this is the only option.
- Files' last access time is enabled by default.
- The
systemdservices are now started with even more restrictions. - The
frontend.authn.oidc.token-urlproperty has no default value.
Acknowledgments
We want to thank Jerome Pansanel and Geonmo Ryu for their contributions.
Release 12.1.0
Admin shell
The \s command takes a #<zone> suffix, e.g. \s pool*#<zone> rep ls, to target only cells in that zone.
Domains register their zone in ZooKeeper at startup when dcache.zone is set.
Cells
Core domains can now require a client certificate for tunnel connections (dcache.broker.channel.require-client-auth=true).
A remote domain or pool that doesn’t present a valid cert can’t connect. Optionally, the certificate CNs that are
allowed to connect can be restricted with dcache.broker.channel.allowed-cn.file (default /etc/dcache/tunnel-allowed-cn.conf).
They are stored in ZooKeeper and checked after the TLS handshake. Connections from other CNs are rejected and never
added to the shard tunnel list.
Chimera
New pnfsids are generated as time-ordered UUIDv7, which improves insert performance of the namespace database. Existing pnfsids are unchanged.
Httpd
Since dCache version 11.0, we have changed the look and feel of the admin pages.
Not all admins like that change. Starting with version 12.1, we allow the creation
of custom styles with the option to default back to the old style.
The new config property httpd.look-and-feel is introduced with values classic and new.
Kafka
Since dCache 11.2, Kafka events are sent by the billing service only. With this release, the unused
Kafka producers are removed from doors, pools and transfermanagers, and their *.enable.kafka and
*.kafka.* properties are obsolete. Use billing.enable.kafka and billing.kafka.* instead.
NFS and xrootd
Both doors can now report real capacity to clients, e.g. for df or xrdfs spaceinfo. The new property
dcache.capacity.source accepts namespace (fixed 1 EiB, the default), pools (space of the pools the
path is written to) or reservation (space of the path’s WriteToken reservation).
Packaging
The systemd services are now started with even more restrictions:
PrivateTmp=yes
ProtectClock=yes
ProtectControlGroups=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=invisible
RemoveIPC=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX AF_NETLINK
RestrictNamespaces=yes
RestrictRealtime=yes
SystemCallArchitectures=native
UMask=0027
KeyringMode=private
LockPersonality=yes
MemoryDenyWriteExecute=no
Note that dCache now has a private /tmp, and files it creates are no longer world-readable.
PnfsManager
Site admins often need to find files that haven’t been accessed for some time. Usually such information requires a merge of namespace and
billing information. However, dCache can maintain the last access time of files, which was disabled by default. With this version,
pnfsmanager.atime-gap defaults to 604800 (7x24x3600) seconds, so a file’s atime is updated if its last access was more than a week ago.
To restore the previous behaviour, set pnfsmanager.atime-gap=-1.
Pool
File hopping is removed: ReplicationHandler, the hoppingmanager service, PoolMgrReplicateFileMsg and the
related commands (set replication) are gone. pool.destination.replicate and pool.destination.replicate.ip
are now obsolete. The Book has a new guide, “Replacing file hopping”, on using permanent migration jobs instead.
Layouts that still define hoppingmanager must be updated.
Stopping a pool through systemd no longer causes a backtrace or a slow shutdown. PoolManager now correctly
shows the pool as disabled(...,dead).
PoolManager
Zone-aware P2P now copies files to a pool in the client’s zone, worked out from the client’s IP address through zone-aware NetUnits. Before, it used the door’s zone.
REST API
GET responses now carry a Last-Modified header showing when the data was last refreshed. It was added to
the monitoring info services first, then to all REST resources, and is documented in the Book. This is useful
for Prometheus (Issue 8151).
Mover listings now include the client IP, UID, DN and GIDs. This replaces parsing mover ls -u over SSH (Issue 8181).
Fixed a 500 error on /api/v1/cells, caused by a stray @Autowired field left over from the Last-Modified work.
S3
dCache now provides an S3 door (dcache-s3), letting S3 clients use dCache
as an S3-compatible object store. By default, the s3-door listens on port 9000.
File access granted based on Unix permissions. The objects with forward slash (‘/’)
in the names are mapped as files with subdirectories. On object removal, the created
subdirectories are left behind. Multipart uploads and RestoreObject for tape files are supported.
Integration with gPlazma:
File /etc/dcache/s3-keys.conf:
AKIASYSTEMTEST system-test-secret
File /etc/dcache/multi-mapfile
s3:AKIASYSTEMTEST username:tigran uid:1 gid:2,true
The new “The S3 Service” chapter in The Book covers setup and limitations.
SRM
Following the removal of SRM in 12.0, the remaining SRM code and the srm and srmmanager services
are removed from the dCache code base.
WebDAV
The retry loop for the checksum race condition is back, but it only runs for requests that ask for a digest, so listings are fast again.
Xrootd
Since version 11.2, the xrootd door rejected prepare requests with Unsupported. Now prepare stages
and pins custodial files, and evict releases the pin. The pin lifetime is set by xrootd.pin.lifetime
(default 12 hours).
dcache-view
OIDC code flow login (dcache-view-2)
dcache-view supports login with an OIDC provider using the authorization code flow. Auto-discovery of OIDC authorization/token endpoints from the issuers well-known document is enabled now. Admins only need to configure the OIDC issuer URL (plus client id and secret) for a provider; the authorization and token endpoints are resolved automatically and stay correct even if the provider changes.
Users can now log out of dCache View. If they logged in through an OIDC provider, they are logged out there as well and then return to dCache View. For this to work, the dCache View address must be allowed as a return address after logout in the provider’s settings. Otherwise, users are only logged out of dCache.
1. Frontend
frontend.authn.oidc.client-id = <client-id>
frontend.authn.oidc.client-secret = <client-secret>
frontend.authn.oidc.issuer = <issuer>
frontend.static!dcache-view.oidc-provider-name-list = <provider-name>
frontend.static!dcache-view.oidc-client-id-list = <client-id>
The frontend finds the authorization and token endpoints from the issuer
(OIDC discovery, <issuer>/.well-known/openid-configuration).
Register this redirect URI with the OIDC provider:
<dcache-view-origin>/api/v1/auth/callback
For several providers, give space-separated values in the same order, and set
frontend.static!dcache-view.oidc-issuer-list to the list of issuers.
2. gPlazma provider
gplazma.oidc.provider!<op-name> = <issuer> -suppress=audience
3. gplazma.conf
auth optional oidc
map sufficient multimap
session sufficient authzdb
The multimap plugin reads etc/multi-mapfile by default. Another file can be set with
gplazma.multimap.file=<path> on the map line.
4. Mapping (multi-mapfile)
oidc:<subject-id>@<op-name> username:<username> uid:<uid> group:<group> gid:<gid>,true
The subject ID appears in the dCache log after a first login attempt:
OidcSubjectPrincipal[<subject-id>@<op-name>].
5. Authorization (storage-authzdb)
authorize <username> read-write <uid> <gid> / / /
Changelog from 12.0.0 to 12.1.0
- d8b34cc8b1
- [maven-release-plugin] prepare release 12.1.0
- 3e515c9214
- [maven-release-plugin] prepare branch @{releaseLabel}
- 9bd02ecdcd
- s3: enforce S3 as protocol name
- 3f678c43ed
- [skip-ci] docs: describe frontend.srr.paths SRR path mappings
- 26c911bda9
- cells: suppress error when killing non-existing cell
- 3e6f0edd09
- [skip-ci] docs: document streaming verification of signed S3 bodies
- 4ca12a5aa3
- pool: remove file hopping / replication
- e6efa763c4
- s3proxy: verify signed request bodies while streaming them to the pool
- 2398041c00
- xrootd: reduce number of DNS requests
- 537de7590c
- docs: xrootd: clarify how the door selects pools for capacity reporting
- b4dddf5fd9
- [skip-ci] docs: document capacity reporting in the NFS and xrootd doors (#8277)
- 03a34ea8b2
- nfs, xrootd: report the capacity of the pools or a space reservation
- 7d00903f4b
- s3: fix handling of IPv6 addresses
- 3d33e59c84
- dcache-vehicles: remove orphan srm messages
- 15682a772c
- [skip-ci] docs: correct wildcard description for allowed tunnel CNs (#8274)
- 4945b39f20
- LocationManager: check for allowed CN before opening tunnel
- 32e8630e14
- pom: use xrootd4j-4.6.7
- 3dc032b423
- [skip-ci] docs: document SciTag propagation to remote party in HTTP-TPC
- 1a5f361e21
- pool: propagate SciTag to remote party in HTTP-TPC transfers
- b742bfaa70
- cells: fix broken junit text
- 3c3ff60890
- cells: announce correct domain role on outbound core tunnels
- 70b0684d02
- ci: allow duplicates for jackson-core and jackson-databind
- c485d1b111
- [skip-ci] docs: add S3 door chapter to the dCache Book
- d9a34ee72c
- [skip-ci] docs: add S3 door chapter to the dCache Book
- 09f87606d2
- s3: fix issues found with the ceph s3-tests
- a12453d91c
- s3proxy: answer ListObjectVersions for stores without versioning
- 30c1bec7dc
- s3: support multipart uploads
- e5256da1ea
- s3: support writes in the S3 door
- 9671df5ca0
- s3proxy: accept signatures over a Content-Type charset changed by Jetty 9
- 292513ee8a
- s3: verify the door with the AWS SDK and the system test
- 64e792cc8d
- s3: add S3 door
- 9f4a1f25a5
- s3: support RestoreObject for tape-only objects
- 76dd5863e6
- s3: add proxied read transfers
- 373c7c7805
- s3: add read-only dCache blob store
- 09a44d853d
- s3: add blob store locator with lazy gPlazma login
- 1c00bec742
- s3: add file based S3 key store
- 0ce4ad3bbb
- gplazma: add S3 access key principal
- 0ce4f82c97
- s3: vendor s3proxy module
- 2062c440d2
- xrootd: improve handling of prepare options
- e344de3d8a
- pool: expose dcache version in NFS mover
- 1a362ebe48
- pool: make nearline request lifecycle transitions atomic
- 38d8a956fd
- libs: update to slf4j-2.0.20 and logback-1.6.5
- e893c66e01
- [skip-ci] docs: document pool startup failure when no IP addresses are found
- 97b72f274a
- pool: fail pool startup if we can’t identify pool’s IPs
- 9684cedf14
- [skip-ci] docs: document multiple OIDC providers in dCache View
- d013535cc9
- dcache-view: support multiple OIDC providers end-to-end
- 982259da49
- [skip-ci] docs: document httpd.look-and-feel property
- 00810bb092
- https: add optional new look-and-feel
- d5e80fe89c
- [skip-ci] docs: document exclusive lock on files open for write over NFS
- 5bd6b33394
- nfs: deny any concurrent access to a file opened for write
- ecdd82e074
- nfs: remove transfer from the door list if pool selection failed
- 94833441ac
- [skip-ci] docs: clarify zone-scoped admin-shell command examples
- c6c7999c1e
- admin-shell: bulk send commands to zone
- b06732627c
- dcache-view: fix OIDC logout not ending the Keycloak session
- b03065e7a0
- fix: restore comment and kafka clean up
- 7fd50715c1
- transfer: fix missing mover info before sending billing notification
- 62e3713583
- build(deps): bump com.fasterxml.jackson.core:jackson-databind
- 3e4e2eb55b
- billing:upgrading kafka version two 3.9.2
- 8ed4742510
- kafka:deleting kafka cleannig up
- dc5244c193
- build(deps): bump com.fasterxml.jackson.core:jackson-databind
- 46a26ceed2
- add mutual TLS to domain tunnels
- e035243c80
- retry mover start if redirect arrives before moverId
- 1e186bfe18
- pom: bump xtootd4j version to 2.6.5
- 86e23539da
- ci: add robot emoji to AI-driven jobs
- a5592a6c71
- xrootd: implement “ping” call
- a436b23e15
- [skip-ci] ci: improve instruction to docubot on how to push to github
- d7646623a7
- jetty: drop per-client and global rate limiter
- ddf3cfeb0b
- transfermanagers: deleting kafka producer
- 02b012753f
- pool: deleting kafka producer
- b3d32d7950
- [skip-ci] docs: note removal of Kafka support from NFS door
- edfe9a5ed5
- nfs: deleting kafk producer
- 082a687d1a
- xrootd: deleting kafk producer
- ecee1b75cf
- webdav: deleting the kafka producer for webdav
- 7ceae99207
- [skip-ci] docs: note removal of Kafka producer from FTP door
- 961276bbe9
- ftp: deleting kafka producer
- 6555df8e99
- xroot: enforce SHA1 for certificate hash calculation
- 793723d5f8
- build(deps): bump org.bouncycastle:bcprov-jdk18on from 1.79 to 1.85
- 2d7470f275
- ci: add [fast_pass] and [only: <suite>] options for selective pipeline execution
- 17b2a90a52
- [skip-ci] docs: document VOMS root FQAN matching in grid-vorolemap
- 8dca32a9bd
- voms: fix FQAN parsing for root /vo group
- 67c068d33c
- ci: add OIDC authorization code flow test
- 0651d3ae85
- ci: fix the PR checks failing
- c1620a5659
- poolmanager: fix zone selection for IPv6 clients
- d687e15c34
- [skip-ci] docs: remove obsolete DCAP Kafka properties from Kafka documentation
- c3f770a106
- dcap:deleting kafka producer
- fe195ff85a
- ci: add AI-based job to auto update documentation
- 501e1ba74c
- dcahce-view: fixing issue 8165, dCache View UI after upgrade
- 932dc9d508
- docs: remove SRM from documentation
- e0c0ee426c
- xrootd: implement prepare function
- 5d13b535d6
- utils: fix buildGenericWantDigest for RFC9530 (issue8191)
- 2002409302
- src: remove remaining SRM code
- fbaec62d74
- gplazma2-voms: enforce openssl SHA1 mode for certificate hash calculation
- 34bb7b71ca
- srm: remove SRM releated code
- 6410753db1
- libs: update CaNL to 2.9.0
- 7d0495d9d6
- REST: fix 500 for GET on api/v1/cells
- 1988d67456
- remove unused @Autowired field
- 8baef5db60
- REST: add Last-Modified Header to all rest Resource responses to GET
- f8d9a84838
- dcache-view: adapt OIDC login UI to discovery-based config
- 2d6a740110
- libs: removed unused jdom library
- 9da3e4d2e8
- door: p2p to clients zone derived by ip address
- 35e66fe7d3
- pnfsmanager: set default atime gap to one week
- 82b83ae3c8
- pom: use nfs4j-0.28.0 with client reboot recovery fixes
- 9d78a559a6
- use OIDC Provider Discovery endpoint
- 7adec58914
- common-security: add workaround bouncycastle EC algorithm name mismatch
- 3f5eb52f19
- REST: return user info for mover
- 05a0b726f0
- frontend: let OIDC callback complete when anonymous-operations=NONE
- 9e3525c4b5
- pool: remove unused class members in dcap mover
- 7922a28729
- pool: fix treadsafety when displaying DCAP mover last access time
- 19b5372682
- libs: drop custom milton2 version
- 504bd0ace3
- libs: use xrootd4j-4.6.2
- e76a959cca
- dcache-nearline-plugin-archetype: use latest plugin release for testing
- 249d639346
- gplazma2-ldap: drop dependency on ldap4testing
- de34cf9eb2
- pom: explicitly specify maven central to enforce the search order
- 1a14b1c7c0
- libs: switch to nfs4j-0.28.4
- f2a18942eb
- libs: use Sleepycat from maven central
- 7a19a22e61
- add jwdp agent to dockerfile
- d243c7108e
- pool: fix pool shutdown message order
- 72a7c1e13a
- systemd: harden systemd-based deployments
- 55b369feb9
- webdav: reintroduce workaround for checksum race condition
- 2b9e9a4a8f
- build(deps): bump io.netty:netty-codec-haproxy
- 8774837587
- build(deps): bump com.fasterxml.jackson.core:jackson-core
- a283b80298
- build(deps): bump org.postgresql:postgresql from 42.7.11 to 42.7.12
- 77340d6671
- Book: add section about Last-Modified Header for REST monitoring resources
- c11937e162
- REST: add Last-Modified Header to GET response
- 5b085d2a51
- domain: generate certificate files for zookeeper TLS in tests
- 4ead0b29fe
- Transfer: remove TransferTest that is wrong
- bd48e47a28
- rep ls: fix crash when access latency or retention policy is undefined
- 42802f236a
- transfer: restore ability to define alternative/additional checksum
- 4f4384c39d
- pool: Address slow initialization of hot file migration jobs
- 7823bae321
- dcache-view: vendoring npm and bower
- 6288b6c75f
- nfs: restart pool selection process on timeout for reads
- 644a889442
- pom: enable java21 to build dCache
- 0db178dc91
- ci: use java25 for dCache runtime
- 903d0f2e2f
- chimera: uuidv7 for new pnfsid generation
- 3d55bb1e99
- smc: remove extra semicolons
- 466c692a29
- scripts: accept java25 as runtime environment
- 5c9d83c40f
- ci: fix upload_container job
- 13c63246c3
- chimera: update shell command to set a proper error code on exit
- 4077ddd9cd
- [maven-release-plugin] prepare for next development iteration
- 306c412a0c
- docs: update TheBook SciTags docs for PR 8044
- c995f7888e
- frontend: Add an option to set SRR path mappings