middleware:Unicore/62/server

From Dgiref

See also troubleshooting for this page.

Please open a NGI-DE ticket if you experience any Installation or Configuration problem.

UNICORE server v.6.2

Prepare

Software

Scientific Linux version 5.4 64 bit
Sun Java Runtime Environment version >= 1.6.0
Perl version >= 5.4
Torque Client

Optimizing the configuration: 

Use minimal operating system installation without firewall.
To verify installed packages use the command 
rpm -qa | grep package_name
Install the following additional packages:
yum -y install wget yum rpm make gcc gcc-c++ tar sed zlib openssl
After the installation is complete, turn off any unnecessary services (like gpm, sendmail, cups, haldaemon, messagebus, pcmcia, anacron, atd) with the following command: 
chkconfig <SERVICE> off

Configure the following settings for the server:

Server Certificates for the Gateway service, the XUUDB service, the unicorex service

Users

A userid to start and administer the above services (not root!)
A userid to execute status queries to the local resource management system (qstat) (not root!)

Others

The worker nodes must NFS-Export a directory /opt/unicore6/data/FILESPACE. The directory has to be mounted under the same path as on the worker node which exports it and has to have access rwx for all.
NFS-Export of users home directories
Register your Resource in GRRS: Registration Form.

Firewall configuration

The UNICORE Gateway is the entry to a site, it authenticates the users and forwards all client requests to the corresponding local service. The Gateway's port (8080 by default) needs to be accessible from the Internet. Therefore is must be open in the firewall for incoming https connections.

The unicorex service registers with the central UNICORE Registry which is running on https://dgrid-unic.fz-juelich.de with Port 9110. unicorex must be able to make a connection to this system on the given port, so that the firewall should allow that outgoing connection (how to open port in firewall).

administrator's script: prepare.sh

```
 
```
```
#!/bin/bash
```
```
# prepare
```
```
 
```

# Declare the variables section ------------

# Please insert your actual configuration

```
# USER_UNICORE=user to install unicore
```

# from here ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

```
USER_UNICORE=unicore
```

# till here ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

```
 
```
```
# install prerequisites
```
```
yum -y install perl python
```
```
yum -y install jdk
```

# assign the JAVA_HOME environment variable

```
JAVA_HOME=/usr/java/jdk1.6.0_17
```

echo "export JAVA_HOME=/usr/java/jdk1.6.0_17" >> /etc/profile.d/jdk.sh

```
 
```

# create non-root user for unicore installation

adduser $USER_UNICORE -d /localhome/unicore

```
# stop local firewall
```
```
echo `service iptables stop`
```
```
 
```

# create directory for the grid host certificates

```
mkdir /etc/grid-security/
```

# after copy the host certificate and host key into /etc/grid-security/

Install

the components Gateway, UNICORE/X, XUUDB, TSI will be used. The component Registry will not be used.

Step 0. download the unicore*.tgz archive
Step 1. untar into /opt/unicore6
Step 2. edit /opt/unicore6/configure.properties:
- and replace the Vsite Name and Component ID each with your site name (e.g. FZJ-JUGGLE). Make sure that this Name is exactly the same as you are using for registering your resource with the VOMS service.
- replace all hostnames with the hostname of your machine.
- edit "Use external registry" section and enter as Registry URL: https://dgrid-unic.fz-juelich.de:9110/D-Grid/services/Registry?res=default_registry
- setup xuudbType as dn
Step 3. run python ./configure.py $USER_UNICORE
Step 4. Install the TSI
- goto /opt/unicore6/tsi and execute ./Install.sh
- enter 6 to choose the TSI for the Torque batch system. This copies all necessary Torque related scripts and configuration files to their proper location.
- Leave the install directory at the default value (tsi_linux_torque)
- Confirm the installation.
- run ./Install_permissions.sh to correctly set file access permissions for the TSI.
Step 5. Extract the file dgrid_extensions.tgz in the installation directory /opt/unicore6.

Privileges/Access control

Make sure that the UNICORE-Administrator is the only user with write and execute privilege to all gateway, unicorex, and xuudb directories and data sets. data sets belonging to the TSI should have write and execute privilege for root only and no one else. There is one exception to this: the script tsi/tsi_ls must be executable by all UNICORE users ( this is done by the install_permissions script).

The /opt/unicore6/data/FILESPACE directory has to be rwx for all UNICORE users. This directory is going to host a subdirectory for each job that belongs to the user who submitted the job. Thereby it is made sure that each user can only access his own data.

administrator's script: install.sh

```
 
```
```
#!/bin/bash
```
```
# install unicore 6.2
```

# Declare the variables section ------------

# Please insert your actual configuration

```
# for installation
```

# BASE_URL=http://mirror.scc.kit.edu/downloads/src/unicore

```
# PACKAGE=unicore-servers-6.2.0-p1
```
```
# WORKDIR=/opt
```
```
# INSTALL_PATH=${WORKDIR}/unicore6
```
```
# USER_UNICORE=user to install unicore
```
```
# for configuration
```

# HOST=name of host for xuudb, gateway, unicore/x, tsi, registry

```
# DOMAIN=domain name
```
```
 
```

# from here ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

```
# for installation
```

BASE_URL=http://mirror.scc.kit.edu/downloads/src/unicore/2010.1

```
PACKAGE=unicore-servers-6.2.2
```
```
WORKDIR=/opt
```
```
INSTALL_PATH=${WORKDIR}/unicore6
```
```
USER_UNICORE=unicore
```
```
# for configuration
```
```
HOST=`hostname -f`
```
```
DOMAIN=fzk.de
```

# till here ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

```
 
```
```
#-> start routine
```
```
# load parameters from prepare section
```
```
cd `dirname $0`
```
```
source prepare.sh
```
```
 
```

# it is recommended to install Unicore in /opt directory.

```
wget ${BASE_URL}/${PACKAGE}.tgz
```
```
tar xzvf ${PACKAGE}.tgz 
```
```
mv -f ${PACKAGE} $INSTALL_PATH
```

# Optionaly, remove the unicore tarball.

```
rm -f ${PACKAGE}.tgz
```
```
 
```
```
chown $USER_UNICORE $INSTALL_PATH
```
```
cd $INSTALL_PATH
```
```
 
```

sed -i "s/DEMO-SITE/DGIREF_SITE/g" ./configure.properties

sed -i "s/useExternalRegistry=false/useExternalRegistry=true/g" ./configure.properties

sed -i "s/urlExternalRegistry=https:\/\/localhost:8080\/REGISTRY/urlExternalRegistry=https:\/\/dgrid-unic.fz-juelich.de:9110\/D-Grid/g" \

```
./configure.properties
```

sed -i "s/hostname/$HOST/g" ./configure.properties

sed -i "s/xuudbType=normal/xuudbType=dn/g" ./configure.properties

```
 
```
```
python ./configure.py $USER_UNICORE
```
```
 
```
```
# install tsi
```
```
cd tsi/
```
```
chmod +x Install.sh
```
```
chmod +x Install_permissions.sh
```
```
./Install.sh
```
```
# choosed 6: tsi/linux_torque
```

# Finish installation by editing tsi_linux_torque/tsi.

# Execute "Install_permissions.sh tsi_linux_torque" after update.

./Install_permissions.sh tsi_linux_torque

# Check that all parent directories of /opt/unicore6/tsi are world executable.

# Otherwise the tsi_ls script cannot be executed.

```
chmod -R +x /opt/unicore6/tsi
```
```
 
```
```
# install d-grid tools
```
```
cd $INSTALL_PATH
```
```
wget ${BASE_URL}/dgrid_extensions.tgz
```
```
tar xzvf dgrid_extensions.tgz
```
```
rm -f dgrid_extensions.tgz
```
```
#<- end routine
```

Configure

For the integration into UNICORE monitoring (Common Information Service - CIS) you have to configure the information provider (CIP) part of unicorex by defining your site's static values in /etc/unicore-unicorex/site-info.glue. CIP is automatically started together with unicorex.

After starting the XUUDB (as described below) add the D-Grid users to it. This should be done using dgridmap as described in Anbindung an Ressource- und User-Management, especially check the unicore6 example at the bottom of the page.

It is recommended to run dgridmap together with the update of the XUUDB regularly (e.g. using cron to run it daily) to make sure that the XUUDB is updated.

Configure the Gateway

Make sure that your Gateway Port is open in your site's firewall for incoming https connections.

use create_gw_truststore.py from dgrid extensions tools to create a truststore with all certificates from the EU GridPMA. Remark: Run this tool regualry to keep the truststore up to date.

security.properties
- define the keystore containing the private key of the gateway and the truststore created above, each with its corresponding passphrase:

        keystore=/path/to/your/gateway/keystore.p12
        keystorepassword=******
        truststore=/path/to/your/gateway/truststore.jks
        truststorepassword=******

Configure the Unicorex and XNJS

wsrflite.xml. Set keystore and truststore:

        <!-- UNICORE/X server identity -->
        <property name="unicore.wsrflite.ssl.keystore" value="/path/to/your/unicorex/keystore.p12"/>
        <property name="unicore.wsrflite.ssl.keypass" value="*******"/>
        <property name="unicore.wsrflite.ssl.keytype" value="PKCS12"/>
        <!-- UNICORE/X truststore -->
        <property name="unicore.wsrflite.ssl.truststore" value="/path/to/your/unicorex/truststore.jks"/>
        <property name="unicore.wsrflite.ssl.truststorepass" value="*******"/>
        <property name="unicore.wsrflite.ssl.truststoretype" value="JKS"/>

simpleidb. It contains all settings to convert the abstract job descriptions received from the Clients into executable jobs for the target system
- In sections "Applications" and "Scripts" you have to adapt the paths to applications and script interpreters your site is offering. You can also add your own applications to the list or remove e.g. Perl, Python, CSH or KSH if you don't want to offer them to the users.
- Section "Resources" the base characteristics of the target system are defined, e.g. number of CPUs, amount of memory, architecture, etc. They are used for scheduling purposes and for providing the user with the information through the client. Adapt the settings to define your system.
xnjs_legacy.xml to describe the system where your TSI is running
Edit site-info.glue to provide static information for the common information provider (CIP) component.

Configure the TSI
- the tsi/conf/tsi.properties is already adapted to Torque, but check once again
- The location of Torque commands (qsub, qstat, ...) is defined as /usr/bin. If you need to change this path adapt in tsi/tsi the following line:
```
my $pbs_bin_dir = "/usr/bin";
```

Configure the XUUDB

Do not forget to add the DN of the certificate configured in xuudb_client.conf to xuudb/conf/xuudb.acl to enable administrative access.

keytool -import -file /etc/grid-security/hostcert.pem -keystore /path/yo/your/truststore.jks

XUUDB distinguishes between Client and Server configuration.

Edit xuudb_server.conf to define keystore and truststore:

        xuudb_keystore_file=/path/yo/your/xuudb/keystore.p12
        xuudb_keystore_type=PKCS12
        xuudb_keystore_password=******
        xuudb_truststore_file=/path/to/your/xuudb/truststore.jks
        xuudb_truststore_type=JKS
        xuudb_truststore_password=******

Edit xuudb_client.conf to define keystore and truststore to use with unicore-xuudb-admin (administrator interface)

        xuudb_keystore_file=/path/yo/your/xuudb/keystore.p12
        xuudb_keystore_type=PKCS12
        xuudb_keystore_password=******
        xuudb_truststore_file=/path/to/your/xuudb/truststore.jks
        xuudb_truststore_type=JKS
        xuudb_truststore_password=******

Example: add users to XUUDB:

manually:

/usr/bin/unicore-xuudb-admin adddn <gcID> <DN> <xlogin> <role>

where <gcID> is the gcID as defined in unicorex, <DN> the user's Distinguished Name (formated according to RFC 2253), <XLOGIN> his/her userid on the target system and <role> his/her role, usually it is "User".

from file:

        cd /path/yo/your/xuudb
        # backup current configuration into the ''NEW'' xuudbBackup.csv
        /usr/bin/unicore-xuudb-admin export xuudbBackup.csv
        # generate mapfile for UNICORE 6:
        dgridmap -cert-path /root/certificates -output-xu /path/yo/your/xuudb/xuudb_in.csv
        # Import the user records at the same time delete all the old records:
        /usr/bin/unicore-xuudb-admin import xuudb_in.csv clearDB

Using the Dgrid Tools

Independently from the rpm Unicore packages, the dgrid_extensions.tgz tarball includes some tools. Actually, only two scripts are in use: create_gw_truststore.py together with x509.py to create the trustore repository.

administrator's script: configure.sh

```
 
```
```
#!/bin/bash
```
```
# configure unicore 6
```
```
 
```
```
INSTALL_PATH=/opt/unicore6
```
```
password=unicore
```
```
hostName=https://dgiref-unicore.fzk.de
```

hostcertPath=/opt/unicore6/certs/dgiref-unicore.fzk.de.p12

truststorePath=/opt/unicore6/certs/truststore.jks

```
 
```

# truststore configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

echo `$JAVA_HOME/bin/keytool -import -file /etc/grid-security/hostcert.pem -keystore $INSTALL_PATH/certs/truststore.jks`

```
 
```

# Gateway configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

```
echo "\
```

keystore=/opt/unicore6/gateway/conf/gateway_keystore.jks

```
keystorepassword=$password
```

truststore=/opt/unicore6/gateway/conf/gateway_keystore.jks

```
truststorepassword=$password
```

" > $INSTALL_PATH/gateway/conf/security.properties

```
 
```

# Unicorex/XNJS configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

cat /opt/unicore6/unicorex/conf/wsrflite.xml

<!-- UNICORE/X server identity (keystore definition) -->

<property name="unicore.wsrflite.ssl.keystore"

value="/opt/unicore6/unicorex/conf/keystore.jks"/>

<property name="unicore.wsrflite.ssl.keypass" value="$password"/>     <!-- $password -->

<property name="unicore.wsrflite.ssl.keytype" value="PKCS12"/>

<property name="unicore.wsrflite.ssl.keyalias" value="njs test certificate"/>

```
 
```

<!-- UNICORE/X truststore (can be the same as the keystore) -->

<property name="unicore.wsrflite.ssl.truststore"

value="/opt/unicore6/unicorex/conf/keystore.jks"/>

<property name="unicore.wsrflite.ssl.truststorepass" value="$password"/>  <!-- $password -->

<property name="unicore.wsrflite.ssl.truststoretype" value="JKS"/>

```
 
```

cat /opt/unicore6/unicorex/conf/xnjs_legacy.xml

```
# edit simpleidb
```

# vi /opt/unicore6/unicorex/conf/simpleidb

```
# edit xnjs_legacy.xml
```

# vi /opt/unicore6/unicorex/conf/xnjs_legacy.xml

```
vi unicorex/conf/uas.config
```
```
# host/port of XUUDB
```
```
xuudb_http_host=$hostName
```
```
xuudb_http_port=34463
```

# Grid component ID used when querying XUUDB xuudb_gcid=fzk-dgiref

```
 
```

# tsi configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# setup path to the qstat into /opt/unicore6/tsi/tsi_linux_torque/tsi

```
# my $pbs_bin_dir = "/usr/local/bin";
```
```
 
```

# xuudb configuration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

```
# configure xuudb_server
```

cat /opt/unicore6/xuudb/conf/xuudb_server.conf

```
xuudb_keystore_file=$hostcertPath
```
```
xuudb_keystore_password=$password
```
```
xuudb_keystore_type=PKCS12
```
```
 
```
```
xuudb_truststore_file=$truststorePath
```
```
xuudb_truststore_type=JKS
```
```
xuudb_truststore_password=$password
```
```
# configure xuudb_client
```

cat /opt/unicore6/xuudb/conf/xuudb_client.conf

```
xuudb_keystore_file=$hostcertPath
```
```
xuudb_keystore_password=$password
```
```
xuudb_keystore_type=PKCS12
```
```
 
```
```
xuudb_truststore_file=$truststorePath
```
```
xuudb_truststore_type=JKS
```
```
xuudb_truststore_password=$password
```
```
 
```
```
# generate mapfile for UNICORE 6:
```

dgridmap -cert-path /root/certificates -output-xu /opt/unicore6/xuudb/xuudb_in.csv

# Import the user records at the same time delete all the old records:

/opt/unicore6/xuudb/bin/admin.sh import xuudb_in.csv clearDB

```
# check
```
```
/opt/unicore6/xuudb/bin/admin.sh list
```
```
 
```
```
# start XUUDB
```
```
$INSTALL_PATH/xuudb/bin/start.sh
```

# After starting the XUUDB add the D-Grid users

# See in configuration description section

```
 
```
```
# D-Grid tools configuration
```

cp $INSTALL_PATH/tools/unicore6.sh /etc/init.d/

```
# create the cron for unicore 6
```

echo "0 1 * * * /opt/unicore6/tools/alteDatenL_unix.pl /opt/unicore6/ 30" >> /etc/cron.d/unicore6.sh

```
chmod +x /etc/cron.d/unicore6.sh
```

Proceed

The directory /opt/unicore6/ contains scripts for startup and shutdown of the installed UNICORE services. Please make sure the gateway, unicorex, and xuudb components are not started as root! Please make sure that the TSI is started under userid root, as it has to execute the jobs on behalf of the user!

For starting gateway, unicorex, and xuudb the UNICORE administrator userid executes

        su $user
        ./start.sh

For starting the TSI the user root executes

su root
        tsi/bin/start.sh

All services can be stopped using

        ./opt/unicore6/stop.sh         # stop gateway, unicorex, xuudb
        ./opt/unicore6/tsi/bin/stop.sh #stops the TSI

administrator's script: proceed.sh

```
 
```
```
#!/bin/bash
```
```
# start unicore 6
```
```
cd /opt/unicore6
```
```
./start.sh
```
```
./tsi/bin/start_tsi
```

Initial test

For each service you can check /opt/unicore6/<service>/logs/startup.log to make sure the component has started properly. You can also use

        # to check whether the corresponding services are in the process table.
        ps -ef | grep <unicore-admin>  # you should see:
           java … eu.unicore.gateway.Gateway
           java … de.fzj.unicore.xuudb.server.XUUDBServer -start
           java … de.fzj.unicore.uas.UAS conf/uas.config VsiteName
 
        ps -ef | grep tsi # shows whether the TSI is up and running.

administrator's script: test.sh

```
 
```
```
#!/bin/bash
```
```
# test unicore 6 installation
```
```
ps -ef | grep tsi
```
```
ps -ef | grep unicore
```

Update

To remove Unicore 6 from your system just use:

administrator's script: update.sh

```
 
```
```
#!/bin/bash
```
```
# update procedures for unicore 6
```

middleware:Unicore/62/server

Contents

UNICORE server v.6.2

Prepare

Install

Configure

Proceed

Initial test

Update

Views

Personal tools

general

guidelines

reference installation

Search

internal

Toolbox