release notes | Book: 3.2, 4.0, 4.1, 4.2, 5.0, 5.1, 5.2, 6.0 (unreleased) | Wiki | Q&A black_bg
Web: Multi-page, Single page | PDF: A4-size, Letter-size | eBook: epub black_bg

Installing a dCache instance

In the following the installation of a dCache instance will be described. The Chimera name space provider, some management components, and the SRM need a PostgreSQL server installed. We recommend running this PostgreSQL on the local node. The first section describes the configuration of a PostgreSQL server. After that the installation of Chimera and of the dCache components will follow. During the whole installation process root access is required.

[return to top]

Prerequisites

In order to install dCache the following requirements must be met:

  • An RPM-based Linux distribution is required for the following procedure. For Debian derived systems we provide Debian packages and for Solaris the Solaris packages or the tarball.

  • dCache requires Java 8 JRE. Please use the latest patch-level and check for upgrades frequently. It is recommended to use JDK as dCache scripts can make use of some extra features that JDK provides to gather more diagnostic information (heap-dump, etc). This helps when tracking down bugs.

  • PostgreSQL must be installed and running. We recommend the use of PostgreSQL version 9.2 (at least PostgreSQL version 8.3 is required).

    Important

    For good performance it is necessary to maintain and tune your PostgreSQL server. There are several good books on this topic, one of which is PostgreSQL 9.0 High Performance.

[return to top]

Installation of the dCache Software

The RPM packages may be installed right away, for example using the command:

[root] # rpm -ivh dcache-3.2.0-1.noarch.rpm

The actual sources lie at http://www.dcache.org/downloads/IAgree.shtml. To install for example Version 3.2.0-1 you would use this:

[root] # rpm -ivh http://www.dcache.org/downloads/1.9/repo/3.2/dcache-3.2.0-1.noarch.rpm

The client can be found in the download-section of the above url, too.

[return to top]

Readying the PostgreSQL server for the use with dCache

Using a PostgreSQL server with dCache places a number of requirements on the database. You must configure PostgreSQL for use by dCache and create the necessary PostgreSQL user accounts and database structure. This section describes how to do this.

[return to top]

Starting PostgreSQL

Install the PostgreSQL server with the tools of the operating system.

Initialize the database directory (for PostgreSQL version 9.2 this is /var/lib/pgsql/9.2/data/) , start the database server, and make sure that it is started at system start-up.

[root] # service postgresql-9.2 initdb
Initializing database:                                     [  OK  ]
[root] # service postgresql-9.2 start
Starting postgresql-9.2 service:                           [  OK  ]
[root] # chkconfig postgresql-9.2 on

[return to top]

Enabling local trust

Perhaps the simplest configuration is to allow password-less access to the database and the following documentation assumes this is so.

To allow local users to access PostgreSQL without requiring a password, ensure the file pg_hba.conf, which (for PostgreSQL version 9.2) is located in /var/lib/pgsql/9.2/data, contains the following lines.

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
# IPv6 local connections:
host    all             all             ::1/128                 trust

Note

Please note it is also possible to run dCache with all PostgreSQL accounts requiring passwords. See the section called “Configuring Access to PostgreSQL for more advice on the configuration of PostgreSQL.

Restarting PostgreSQL

If you have edited PostgreSQL configuration files, you must restart PostgreSQL for those changes to take effect. On many systems, this can be done with the following command:

[root] # service postgresql-9.2 restart
Stopping postgresql-9.2 service:                           [  OK  ]
Starting postgresql-9.2 service:                           [  OK  ]

[return to top]

Configuring Chimera

Chimera is a library providing a hierarchical name space with associated meta data. Where pools in dCache store the content of files, Chimera stores the names and meta data of those files. Chimera itself stores the data in a relational database. We will use PostgreSQL in this tutorial. The properties of Chimera are defined in /usr/share/dcache/defaults/chimera.properties. See Chapter 4, Chimera for more information.

[return to top]

Creating users and databases for dCache

Create the Chimera database and user.

[root] # createdb -U postgres chimera
CREATE DATABASE
[root] # createuser -U postgres --no-superuser --no-createrole --createdb --pwprompt chimera
Enter password for new role:
Enter it again:
You do not need to enter a password.

The dCache components will access the database server with the user srmdcache.

[root] # createuser -U postgres --no-superuser --no-createrole --createdb --pwprompt srmdcache
Enter password for new role:
Enter it again:
You do not need to enter a password.

Several management components running on the head node as well as the SRM will use the database dcache for storing their state information:

[root] # createdb -U srmdcache dcache

There might be several of these on several hosts. Each is used by the dCache components running on the respective host.

Create the database used for the billing plots.

[root] # createdb -O srmdcache -U postgres billing

And run the command dcache database update.

[root] # dcache database update
PnfsManager@dCacheDomain:
INFO  - Successfully acquired change log lock
INFO  - Creating database history table with name: databasechangelog
INFO  - Reading from databasechangelog
many more like this...
      

Now the configuration of Chimera is done.

Before the first start of dCache replace the file /etc/dcache/gplazma.conf with an empty file.

[root] # mv /etc/dcache/gplazma.conf /etc/dcache/gplazma.conf.bak
[root] # touch /etc/dcache/gplazma.conf

dCache can be started now.

[root] # dcache start
Starting dCacheDomain done

So far, no configuration of dCache is done, so only the predefined domain is started.

[return to top]

Configuring dCache

[return to top]

Terminology

dCache consists of one or more domains. A domain in dCache is a Java Virtual Machine hosting one or more dCache cells. Each domain must have a name which is unique throughout the dCache instance and a cell must have a unique name within the domain hosting the cell.

A service is an abstraction used in the dCache configuration to describe atomic units to add to a domain. It is typically implemented through one or more cells. dCache keeps lists of the domains and the services that are to be run within these domains in the layout files. The layout file may contain domain- and service- specific configuration values. A pool is a cell providing physical data storage services.

[return to top]

Configuration files

In the setup of dCache, there are three main places for configuration files:

  • /usr/share/dcache/defaults
  • /etc/dcache/dcache.conf
  • /etc/dcache/layouts

The folder /usr/share/dcache/defaults contains the default settings of the dCache. If one of the default configuration values needs to be changed, copy the default setting of this value from one of the files in /usr/share/dcache/defaults to the file /etc/dcache/dcache.conf, which initially is empty and update the value.

Note

In this first installation of dCache your dCache will not be connected to a tape sytem. Therefore please change the values for pnfsmanager.default-retention-policy and pnfsmanager.default-access-latency in the file /etc/dcache/dcache.conf.

pnfsmanager.default-retention-policy=REPLICA
pnfsmanager.default-access-latency=ONLINE

Layouts describe which domains to run on a host and which services to run in each domain. For the customized configuration of your dCache you will have to create a layout file in /etc/dcache/layouts. In this tutorial we will call it the mylayout.conf file.

Important

Do not update configuration values in the files in the defaults folder, since changes to these files will be overwritten by updates.

As the files in /usr/share/dcache/defaults/ do serve as succinct documentation for all available configuration parameters and their default values it is quite useful to have a look at them.

[return to top]

Defining domains and services

Domains and services are defined in the layout files. Depending on your site, you may have requirements upon the doors that you want to configure and domains within which you want to organise them.

A domain must be defined if services are to run in that domain. Services will be started in the order in which they are defined.

Every domain is a Java Virtual Machine that can be started and stopped separately. You might want to define several domains for the different services depending on the necessity of restarting the services separately.

The layout files define which domains to start and which services to put in which domain. Configuration can be done per domain and per service.

A name in square brackets, without a forward-slash (/) defines a domain. A name in square brackets with a forward slash defines a service that is to run in a domain. Lines starting with a hash-symbol (#) are comments and will be ignored by dCache.

There may be several layout files in the layout directory, but only one of them is read by dCache when starting up. By default it is the single.conf. If the dCache should be started with another layout file you will have to make this configuration in /etc/dcache/dcache.conf.

Example:

dcache.layout=mylayout
This entry in /etc/dcache/dcache.conf will instruct dCache to read the layout file /etc/dcache/layouts/mylayout.conf when starting up.

Example:

These are the first lines of /etc/dcache/layouts/single.conf:

dcache.broker.scheme=none

[dCacheDomain]
[dCacheDomain/admin]
[dCacheDomain/poolmanager]

[dCacheDomain] defines a domain called dCacheDomain. In this example only one domain is defined. All the services are running in that domain. Therefore no messagebroker is needed, which is the meaning of the entry messageBroker=none.

[dCacheDomain/admin] declares that the admin service is to be run in the dCacheDomain domain.

Example:

This is an example for the mylayout.conf file of a single node dCache with several domains.

[dCacheDomain]
[dCacheDomain/topo]
[dCacheDomain/info]

[namespaceDomain]
[namespaceDomain/pnfsmanager]
[namespaceDomain/cleaner]
[namespaceDomain/dir]

[poolmanagerDomain]
[poolmanagerDomain/poolmanager]

[adminDoorDomain]
[adminDoorDomain/admin]

[httpdDomain]
[httpdDomain/httpd]
[httpdDomain/billing]

[gPlazmaDomain]
[gPlazmaDomain/gplazma]

Note

If you defined more than one domain, a messagebroker is needed, because the defined domains need to be able to communicate with each other. This means that if you use the file single.conf as a template for a dCache with more than one domain you need to delete the line messageBroker=none. Then the default value will be used which is messageBroker=cells, as defined in the defaults /usr/share/dcache/defaults/dcache.properties.

[return to top]

Creating and configuring pools

dCache will need to write the files it keeps in pools. These pools are defined as services within dCache. Hence, they are added to the layout file of your dCache instance, like all other services.

The best way to create a pool, is to use the dcache script and restart the domain the pool runs in. The pool will be added to your layout file.

[<domainname>/pool]
name=<poolname>
path=/path/to/pool
pool.wait-for-files=${path}/data

The property pool.wait-for-files instructs the pool not to start up until the specified file or directory is available. This prevents problems should the underlying storage be unavailable (e.g., if a RAID device is offline).

Note

Please restart dCache if your pool is created in a domain that did not exist before.

Example:

[root] # dcache pool create /srv/dcache/p1 pool1 poolDomain
Created a pool in /srv/dcache/p1. The pool was added to poolDomain in
file:/etc/dcache/layouts/mylayout.conf.

In this example we create a pool called pool1 in the directory /srv/dcache/p1. The created pool will be running in the domain poolDomain.

Mind the Gap!

The default gap for poolsizes is 4GiB. This means you should make a bigger pool than 4GiB otherwise you would have to change this gap in the dCache admin tool. See the example below. See also the section called “The Admin Interface”.

(local) admin > cd <poolname>
(<poolname>) admin > set gap 2G
(<poolname>) admin > save

Adding a pool to a configuration does not modify the pool or the data in it and can thus safely be undone or repeated.

[return to top]

Starting dCache

Restart dCache to start the newly configured components dcache restart and check the status of dCache with dcache status.

Example:

[root] # dcache restart
Stopping dCacheDomain 0 1 done
Starting dCacheDomain done
Starting namespaceDomain done
Starting poolmanagerDomain done
Starting adminDoorDomain done
Starting httpdDomain done
Starting gPlazmaDomain done
Starting poolDomain done
[root] # dcache status
DOMAIN            STATUS  PID   USER
dCacheDomain      running 17466 dcache
namespaceDomain   running 17522 dcache
poolmanagerDomain running 17575 dcache
adminDoorDomain   running 17625 dcache
httpdDomain       running 17682 dcache
gPlazmaDomain     running 17744 dcache
poolDomain        running 17798 dcache

Now you can have a look at your dCache via The Web Interface, see the section called “The Web Interface for Monitoring dCache: http://<httpd.example.org>:2288/, where <httpd.example.org> is the node on which your httpd service is running. For a single node dCache this is the machine on which your dCache is running.

[return to top]

Java heap size

By default the Java heap size and the maximum direct buffer size are defined as

dcache.java.memory.heap=512m
dcache.java.memory.direct=512m

Again, these values can be changed in /etc/dcache/dcache.conf.

For optimization of your dCache you can define the Java heap size in the layout file separately for every domain.

Example:

[dCacheDomain]
dcache.java.memory.heap=2048m
dcache.java.memory.direct=0m
...
[utilityDomain]
dcache.java.memory.heap=384m
dcache.java.memory.direct=16m

Note

dCache uses Java to parse the configuration files and will search for Java on the system path first; if it is found there, no further action is needed. If Java is not on the system path, the environment variable JAVA_HOME defines the location of the Java installation directory. Alternatively, the environment variable JAVA can be used to point to the Java executable directly.

If JAVA_HOME or JAVA cannot be defined as global environment variables in the operating system, then they can be defined in either /etc/default/dcache or /etc/dcache.env. These two files are sourced by the init script and allow JAVA_HOME, JAVA and DCACHE_HOME to be defined.

[return to top]

Installing dCache on several nodes

Installing dCache on several nodes is not much more complicated than installing it on a single node. Think about how dCache should be organised regarding services and domains. Then adapt the layout files, as described in the section called “Defining domains and services”, to the layout that you have in mind. The files /etc/dcache/layouts/head.conf and /etc/dcache/layouts/pool.conf contain examples for a dCache head-node and a dCache pool respectively.

Important

You must configure a domain called dCacheDomain but the other domain names can be chosen freely.

Please make sure that the domain names that you choose are unique. Having the same domain names in different layout files on different nodes may result in an error.

On any other nodes than the head node, the property dcache.broker.host has to be added to the file /etc/dcache/dcache.conf. This property should point to the host containing the special domain dCacheDomain, because that domain acts implicitly as a broker.

Tip

On dCache nodes running only pool services you do not need to install PostgreSQL. If your current node hosts only these services, the installation of PostgreSQL can be skipped.