data:Dcache/192/server

From Dgiref

Please open a NGI-DE ticket if you experience any Installation or Configuration problem.

Dcache/192/server

Prepare

Operating system: Scientific Linux v.5.1 64 bit

Optimizing the configuration: 

Use minimal operating system installation without firewall.
To verify installed packages use the command 
rpm -qa | grep package_name
Install the following additional packages:
yum -y install wget yum rpm make gcc gcc-c++ tar sed zlib openssl
After the installation is complete, turn off any unnecessary services (like gpm, sendmail, cups, haldaemon, messagebus, pcmcia, anacron, atd) with the following command: 
chkconfig <SERVICE> off

Configure the following settings for the server:

java 1.6 SDK.
PostgreSQL version 8 or higher must be installed and running.

Firewall configuration

The dCache frontend runs SE services (GRIS, GridFTP, SRM). Which ports are used for dCache is essentially not important. But it's advisable when all sites that support dCache are using the same ports, so compatibility is automatically achived in an easy manner. When you are setting up your dCache system (by means of dCacheConfigure.sh) you can configure the ports to be opend in site-info.def with the varibales...

DCACHE_PORT_RANGE_PROTOCOLS_SERVER_GSIFTP: Sets the portrange for dcache as a GSIFTP server in "passive" mode. Default value is from 50000 till 52000 ("50000,52000").
DCACHE_PORT_RANGE_PROTOCOLS_CLIENT_GSIFTP: Sets the portrange for dcache as a GSIFTP client in "active" mode. Default value is from 33115 till 33125 ("33115,33125").
DCACHE_PORT_RANGE_PROTOCOLS_SERVER_MISC: Sets the portrange for dcache as a (GSI)DCAP and xrootd server in "passive" mode. Default value is from 60000 till 62000 ("60000,62000").

And this is what dCache developers suggest to configure the firewall. (table taken from dCache book, chapter 22)

Protocol	Port(s)	Direction	Nodes
dCap	22125	incoming	doorDomain (admin node)
dCap	any	outgoing	pools
GSIdCap	22128	incoming	gsidcapDomain (where GSIDCAP=yes in node_config)
GSIdCap	any	outgoing	pools
GridFTP	2811	incoming	gridftpDomain (where GRIDFTP=yes in node_config)
	20000-25000	outgoing (active FTP)	pools
	20000-25000	incoming (passive FTP)	gridftpDomain
SRM v1	8443	incoming	srmDomain
SRM v2	8444	incoming	srmDomain

To change these configurations after setting up dCache, go to /opt/d-cache/config/dCacheSetup and modify the values for (if you want to adopt the settings proposed by the developers)...

in the "Java Configuration" section the parameters
- Dorg.globus.tcp.port.range to "20000,25000"
- Dorg.dcache.net.tcp.portrange to "33115,33215"
in the section "Network Configuration"
- dCapPort to "22125"
- dCapGsiPort to "22128"
- gsiFtpPortNumber to "2811"
- srmPort to "8443"
- clientDataPortRange to "20000,25000"

administrator's script: prepare.sh

```
 
```
```
#!/bin/bash
```
```
# prepare to install dcache server
```
```
 
```

# Declare the variables section ------------

# Please insert your actual configuration

# The two users '''edguser''' and '''edginfo''' must be added on information provider nodes

# They are not needed on other nodes but, since their presence will do no harm, they may be

```
# added on all nodes.
```
```
# EDGUSER=edguser
```
```
# EDGINFO=edginfo
```

# from here ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

```
EDGUSER=edguser
```
```
EDGINFO=edginfo
```

# till here ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

```
 
```
```
#-> start routine
```

adduser -r -s /bin/false -d /home/edg $EDGUSER

adduser -r -s /bin/false -d /home/edg $EDGINFO

```
 
```

# clean all for yum to remove unnecessary configurations

```
yum clean all
```
```
 
```
```
# install prerequisites:
```

yum -y install jdk postgresql postgreql-libs postgresql-server

```
yum update --exclude *kernel*
```
```
 
```
```
export JAVA_HOME=/usr/java/jdk1.6.0_13
```
```
export PATH=$JAVA_HOME/bin/:$PATH
```
```
 
```
```
# run dbms server and initialize db
```
```
echo `/etc/init.d/postgresql start`
```
```
echo `chkconfig postgresql on`
```
```
#<- end routine
```

Install

WARNING: 
Two users edguser and edginfo must be added on information provider nodes (in general this is considered to be the "admin node"). They are not needed on other nodes but, since their presence will do no harm, they may be added on all nodes.

Getting the dCache sources: Now to install finally dCache install the following bundles (yum install should install all given arguments, but it's more reliable to install the bundles separately)

administrator's script: install.sh

```
 
```
```
#!/bin/bash
```
```
# install dcache script
```
```
 
```
```
# install dCache server and clients
```

rpm -Uhv http://www.dcache.org/downloads/1.9/dcache-server-1.9.2-7.noarch.rpm

rpm -Uhv http://www.dcache.org/downloads/1.8.0/dcache-dcap-1.8.0-15p8.sl4.i586.rpm

rpm -Uhv http://www.dcache.org/downloads/1.9/dcache-srmclient-1.9.2-4.noarch.rpm

Configure

Get dCache working: When all sources are installed, some configurations have to be done. For this purpose dCache brought some configuration script with it: dCacheConfigure.sh. This script needs two other files for further statements: site-info.def and users.conf. Both of which might be know to people that already have worked with gLite. In fact, it are the same files extended with some dCache specific statements. You can find (and should make a copy to start with) templates located at /opt/d-cache/share/doc/dCacheConfigure/examples.

mkdir /root/nodeconfig
cp /opt/d-cache/share/doc/dCacheConfigure/examples/* /root/nodeconfig

The users.conf: Maybe your site already has a ready-made users.conf, just get it then. Otherwise the default one shipped with dCache is a good starting point. For documentation on the content please look at CERN TWiki.

The site-info.def file

Same for the site-info.def, if your site already has one, use it.

Locating Java: As dCache uses Java, you will need to know the path to your installation of Java Development Kit (JDK). Please note this is not the path to the runtime environment (JRE), which is often a subdirectory within the JDK.

          [root@dcache-node] rpm -qa|grep jdk
          jdk-1.5.0_16-fcs
          [root@dcache-node] rpm -ql jdk-1.5.0_16-fcs | grep bin/java
          /usr/java/jdk1.5.0_16/bin/java
          /usr/java/jdk1.5.0_16/jre/bin/java

In the above example, the JDK path is /usr/java/jdk1.5.0_16.

You must update your /root/nodeconfig/site-info.def file, altering the values of the following variables:

MY_DOMAIN
JAVA_LOCATION
USERS_CONF
DCACHE_ADMIN
DCACHE_POOLS
DCACHE_DOOR_SRM
DCACHE_DOOR_GSIFTP
DCACHE_DOOR_XROOTD
DCACHE_DOOR_LDAP
VOS

WARNING: 
Please be aware that there are two DCACHE_DOOR_XROOTD variables declared in the site-info.def file. This will be fixed with the next release, but in the mean-time, simply delete the extra instance.

When running dCacheConfigure.sh, you must explicitly allow the script to reset various configuration options. This means you must either uncomment the existing lines or add extra lines.

    RESET_DCACHE_CONFIGURATION=yes
    RESET_DCACHE_PNFS=yes
    RESET_DCACHE_RDBMS=yes

With these lines in place, you must run dCacheConfigure.sh to configure dCache. Be patient, it may take several minutes to run.

WARNING: 
After running dCacheConfigure.sh you should remove or comment-out the three lines that start RESET_. Failure to do this will reset vital data when you next run dCacheConfigure.sh, almost certainly resulting in data loss.</p>

Suitable values are given below.

administrator's script: configure.sh

```
 
```
```
#!/bin/bash
```
```
# configure dcache server
```
```
 
```

# configuration of dCache can be done automatically, if provided with configuration files

# these configuration files are site-info.def, users.conf and groups.conf

# these are offered ready-made for the reference installation

```
 
```

# Declare the variables section ------------

```
#
```

# Please insert your actual configuration

```
#
```

# VO_NAMES="whitespace-separated list of your supported VOs"

```
#
```

# SITE_INFO_DEF=path to the site-info.def

```
#
```

# SITE_INFO_DEF_TMP=temporary site-info.def path

```
#
```
```
# MY_DOMAIN="domain name"
```
```
#
```
```
# JAVA_PATH="path to jdk instance"
```
```
#
```
```
# USERS_CONF="path to the users.conf"
```
```
#
```
```
# GROUPS_CONF="path to the groups.conf"
```
```
#
```
```
# DCACHE_ADMIN="admin node.\$MY_DOMAIN"
```
```
#
```

# DCACHE_POOLS="pool node.\$MY_DOMAIN:7:/pools/1"

```
#
```

# DCACHE_DOOR_SRM="\"door srm node.\$MY_DOMAIN\""

```
#
```

# DCACHE_DOOR_GSIFTP="\"door gsiftp node.\$MY_DOMAIN\""

```
#
```

# DCACHE_DOOR_GSIDCAP="\"door gsidcap node.\$MY_DOMAIN\""

```
#
```

# DCACHE_DOOR_DCAP="\"door dcap node.\$MY_DOMAIN\""

```
#
```
```
#
```

# from here ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

VO_NAMES="\"astrogrid aerogrid bisgrid bwgrid c3grid dgcms dgtest education fingrid gdigrid hepcg ingrid kerndgrid lifescience medigrid partgrid progrid textgrid wisent\""

SITE_INFO_DEF=/opt/d-cache/config/site-info.def

```
SITE_INFO_DEF_TMP=/tmp/site-info.def
```
```
#MY_DOMAIN="\$(hostname -d)"
```
```
JAVA_PATH=${JAVA_HOME}
```
```
# USERS_CONF will be set automatically
```
```
# GROUPS_CONF will be set automatically
```

DCACHE_ADMIN="dgiref-dcache.\$MY_DOMAIN"

```
# DCACHE_POOLS="\"\\\ \\
```

# dgiref-dcache.\$MY_DOMAIN:7:/pools/1 \\\ \\

# dgiref-dcache.\$MY_DOMAIN:7:/pools/2 \\\ \\ "

DCACHE_DOOR_SRM="\"dgiref-dcache.\$MY_DOMAIN\""

DCACHE_DOOR_GSIFTP="\"dgiref-dcache.\$MY_DOMAIN\""

DCACHE_DOOR_GSIDCAP="\"dgiref-dcache.\$MY_DOMAIN\""

DCACHE_DOOR_DCAP="\"dgiref-dcache.\$MY_DOMAIN\""

# till here ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

```
 
```
```
#
```

# check if the value $1 exists and fill it with $2

```
#
```
```
function checkAndFill {    
```
```
    if grep -q $1 $SITE_INFO_DEF_TMP;
```
```
    then        
```

        echo "will be updated: ${1} with ${2} to $SITE_INFO_DEF_TMP"

```
        sed -i "/$1/ c\
```
```
$1$2" $SITE_INFO_DEF_TMP
```
```
 
```
```
    else
```

        echo "will be inserted: ${1} with ${2} to $SITE_INFO_DEF_TMP"

```
        echo $1$2 >> $SITE_INFO_DEF_TMP
```
```
    fi
```
```
}
```
```
 
```

echo `wget -O  ${SITE_INFO_DEF_TMP} http://dgiref.d-grid.de/svn/dgiref/PROD/repl/configs/site-info.def`

cp -f $SITE_INFO_DEF ${SITE_INFO_DEF}.save

```
 
```

# if there is already a configuration file for glite, than use it

if [ -r /opt/glite/yaim/etc/users.conf ];

```
then
```

	USERS_CONF=/opt/glite/yaim/etc/users.conf

```
else
```

    if [ -r /opt/d-cache/config/users.conf ];

```
    then
```

        USERS_CONF=/opt/d-cache/config/users.conf

```
    else
```

        wget -O /opt/d-cache/config/users.conf http://dgiref.d-grid.de/svn/dgiref/PROD/repl/configs/users.conf

        USERS_CONF=/opt/d-cache/config/users.conf

```
    fi
```
```
fi
```
```
 
```

# if there is already a configuration file for glite, than use it

if [ -r /opt/glite/yaim/etc/groups.conf ];

```
then
```

	GROUPS_CONF=/opt/glite/yaim/etc/groups.conf

```
else
```

    if [ -r /opt/d-cache/config/groups.conf ];

```
	then
```

        GROUPS_CONF=/opt/d-cache/config/groups.conf

```
    else
```

        wget -O /opt/d-cache/config/groups.conf http://dgiref.d-grid.de/svn/dgiref/PROD/repl/configs/groups.conf

        GROUPS_CONF=/opt/d-cache/config/groups.conf

```
    fi
```
```
fi
```
```
 
```

# fill in the variables in site-info.def

# checkAndFill MY_DOMAIN= "${MY_DOMAIN}"

checkAndFill USERS_CONF= "${USERS_CONF}"

checkAndFill GROUPS_CONF= "${GROUPS_CONF}"

checkAndFill JAVA_LOCATION= "${JAVA_PATH}"

```
 
```

# Remember to replace dcache-node.fzk.de with the FQDN of this host

checkAndFill DCACHE_ADMIN= "${DCACHE_ADMIN}"

```
 
```

# Here on the poolnode two pools "1" and "2" are created at /pools with size of 7 GB

# checkAndFill DCACHE_POOLS= "${DCACHE_POOLS}"

```
 
```

# it's adviced to set the SRM service on a seperate node

checkAndFill DCACHE_DOOR_SRM= "${DCACHE_DOOR_SRM}"

```
 
```

# it's also adviced for bigger sites to set GridFTP services on separate nodes

checkAndFill DCACHE_DOOR_GSIFTP= "${DCACHE_DOOR_GSIFTP}"

checkAndFill DCACHE_DOOR_GSIDCAP= "${DCACHE_DOOR_GSIDCAP}"

```
 
```
```
# should be only (!) on the admin node
```

checkAndFill DCACHE_DOOR_DCAP= "${DCACHE_DOOR_DCAP}"

```
 
```

# if you don't want to offer a door to a protocol then don't set the respective value...

#DCACHE_DOOR_XROOTD="dcache-doornode.\$MY_DOMAIN"

#DCACHE_DOOR_LDAP="dcache-doornode.\$MY_DOMAIN"

```
 
```
```
# configurations for the firewall ports
```

checkAndFill DCACHE_PORT_RANGE_PROTOCOLS_SERVER_GSIFTP= "20000,25000"

checkAndFill DCACHE_PORT_RANGE_PROTOCOLS_CLIENT_GSIFTP= "33115,33215"

checkAndFill DCACHE_PORT_RANGE_PROTOCOLS_SERVER_MISC= "20000,25000"

```
 
```

# for installation (and _only_ for installation and reconfiguration!)

# these varialbes have to be set to "yes"

checkAndFill RESET_DCACHE_CONFIGURATION= "yes"

```
#
```
```
checkAndFill RESET_DCACHE_PNFS= "yes"
```
```
checkAndFill RESET_DCACHE_RDBMS= "yes"
```
```
 
```
```
checkAndFill VOS= "${VO_NAMES}"
```
```
 
```
```
mv -f $SITE_INFO_DEF_TMP $SITE_INFO_DEF
```
```
 
```
```
# run dCacheConfigure.sh
```

/opt/d-cache/bin/dCacheConfigure.sh -c config_sedcache -s $SITE_INFO_DEF

```
 
```
```
# for saftey reset these variables
```

checkAndFill RESET_DCACHE_CONFIGURATION= "no"

```
checkAndFill RESET_DCACHE_PNFS= "no"
```
```
checkAndFill RESET_DCACHE_RDBMS= "no"
```

Initial test

check dCache web interface on the dcache server port 2288
accessing file system with standard commands
copying data using dCache protocols
deleting files in dCache

Additional information can be used from the previous dcache instance.

administrator's script: test.sh

```
 
```
```
#!/bin/bash
```
```
# test
```

# accessing file system with standard commands

```
ls /pnfs/fzk.de/data/dgtest/
```

touch /pnfs/fzk.de/data/dgtest/test.blub

```
rm /pnfs/fzk.de/data/dgtest/test.blub
```
```
# copying data using dCache protocols 
```

dccp /bin/bash gsidcap://dgiref-dcache.fzk.de:22128/pnfs/fzk.de/data/dgtest/testbin-1

```
# delete data with srmrm 
```

srmrm srm://dgiref-dcache.fzk.de:8443/pnfs/fzk.de/data/dgtest/testbin-1

Update

To update/remove the dCache server state, use:

administrator's script: update.sh

```
 
```
```
#!/bin/bash
```
```
# update dcache server
```

yum update glite-version desy-SE_dcache_admin_postgres lcg-CA lcg-vomscerts desy-SE_dcache_info desy-SE_dcache_pool

data:Dcache/192/server

Contents

Dcache/192/server

Prepare

Install

Configure

Initial test

Update

Views

Personal tools

general

guidelines

reference installation

Search

internal

Toolbox