middleware:Glite/32/server

From Dgiref

See also troubleshooting for this page.

Please open a NGI-DE ticket if you experience any Installation or Configuration problem.

gLite CREAM CE

Prepare

Software

Scientific Linux version 5.4 64 bit
Java JDK >= 1.6.0
perl
Torque Client

Optimizing the configuration: 

Use minimal operating system installation without firewall.
To verify installed packages use the command 
rpm -qa | grep package_name
Install the following additional packages:
yum -y install wget yum rpm make gcc gcc-c++ tar sed zlib openssl
After the installation is complete, turn off any unnecessary services (like gpm, sendmail, cups, haldaemon, messagebus, pcmcia, anacron, atd) with the following command: 
chkconfig <SERVICE> off

Configure the following settings for the server:

Server Certificates for gLite CE

The supported installation method for SL5 is the yum tool, and you have to configure yum repositories yourself and install the meta packages using your preferred way.

Please note that YAIM IS NOT SUPPORTING INSTALLATION

gLite 3.2 packages for WN
- glite-GLEXEC_wn
- glite-MPI_utils
- glite-TORQUE_client
- glite-TORQUE_utils

administrator's script: prepare.sh

```
 
```
```
#!/bin/bash
```
```
# prepare gLite to install
```

REPO_URL="http://svn.rz.uni-karlsruhe.de/svn/dgiref/PROD/cf3/repl/repos/external/"

```
 
```
```
# Missing packages installation 
```
```
yum -y install perl openssl
```
```
yum -y remove jpackage-utils
```
```
rm /etc/yum.repos.d/sl-dgiref.repo
```
```
yum clean all
```
```
 
```

wget -O /etc/yum.repos.d/jpackage17.repo ${REPO_URL}/jpackage17.repo

```
# disable sl-base repo
```
```
# cat /etc/yum.repos.d/sl.repo
```
```
#[sl-base]
```
```
# ...
```
```
#enabled=0
```
```
#...
```
```
 
```
```
yum -y remove xml-commons*
```

yum -y install xml-commons* jpackage-utils-1.7.3 java jdk.x86_64

```
# enable sl-base repo
```
```
# cat /etc/yum.repos.d/sl.repo
```
```
#[sl-base]
```
```
# ...
```
```
#enabled=1
```
```
#...
```
```
 
```

wget -O /etc/yum.repos.d/lcg-CA.repo ${REPO_URL}/glite/32/lcg-CA.repo

wget -O /etc/yum.repos.d/glite-TORQUE_client.repo ${REPO_URL}/glite/32/glite-TORQUE_client.repo

wget -O /etc/yum.repos.d/glite-TORQUE_utils.repo ${REPO_URL}/glite/32/glite-TORQUE_utils.repo

wget -O /etc/yum.repos.d/glite-BDII.repo ${REPO_URL}/glite/32/glite-BDII.repo

wget -O /etc/yum.repos.d/glite-CREAM.repo ${REPO_URL}/glite/32/glite-CREAM.repo

```
 
```

# create directory for the grid host certificates

```
mkdir /etc/grid-security/
```

# after copy the host certificate and host key into /etc/grid-security/

```
 
```
```
# Host certificates
```

# The host certificate and the associated key are copied in the directory /etc/grid-security:

cp hostcert.pem hostkey.pem /etc/grid-security

chmod 400 /etc/grid-security/hostkey.pem

chmod 644 /etc/grid-security/hostcert.pem

```
 
```
```
# VOMS server certificate
```

# Copy the d-grid VOMS server certificate into /etc/grid-security/vomsdir

wget -O /etc/grid-security/vomsdir/dgrid-voms.fzk.de http://mirror.scc.kit.edu/downloads/src/glite/2009.1/dgrid-voms.fzk.de

```
 
```

# GSI configuration for the GridKA CA (needed for grid-cert-request, etc.):

# either download and install the GSI configuration rpm

rpm -ihv http://mirror.scc.kit.edu/downloads/src/glite/2009.1/ca_FZK-local-1.0-1.noarch.rpm

```
 
```
```
# prepare JAVA_HOME environment
```

echo "export JAVA_HOME=/usr/java/latest" > /etc/profile.d/java.sh

```
source /etc/profile.d/java.sh
```
```
 
```
```
# disable selinux for BDII
```
```
echo 0 > /selinux/enforce
```

Install

The D-Grid reference installation uses the CREAM CE variant for the gLite computing resources. Hence the following three main gLite components must be installed on the CE (Computing Element):

Computing Element: glite-CREAM package
Information system: glite-BDII package
Batch system components:
- glite-TORQUE_utils package
- glite-TORQUE_client

All nodes except UI, WN and BDII require the host certificate/key files to be installed. The /etc/grid-security should contain: hostcert.pem - containing the machine public key; hostkey.pem - containing the machine private key

See also: https://twiki.cern.ch/twiki/bin/view/LCG/GenericInstallGuide320

administrator's script: install.sh

```
 
```
```
#!/bin/bash
```
```
# install glite
```
```
# install CAs
```
```
yum -y install lcg-CA
```
```
# install CE packages
```
```
yum -y install glite-BDII 
```
```
yum -y install glite-CREAM
```

# install torque client packages (elective)

yum -y install glite-TORQUE_client glite-TORQUE_utils

Configure

To install the gLite Monitoring services (BDII and RGMA), please refer to gLite services page.

Generally speaking the gLite configuration done by the YAIM packages (for the YAIM description check YAIM guide). There are three important site-specific configuration files:

site-info.def has site-specific configuration, (check also: /opt/glite/etc/gip/ldif/glite-info-site.ldif)
users.conf to set up users,
groups.conf for access rules.

you can use the script to generate the users.conf file from list of VOs

The files structure description can be found: into the /opt/glite/yaim/examples/ (for example users.conf.README). The file users.conf must be created or adapted for all VOs users. During the configuration, the YAIM configuration tool creates these users if they are not exist yet. If the user accounts already exist YAIM do not change the UIDs/GIDs. The entries are controlled in the directory /etc/grid-security/gridmapdir.

Certificates

The certificate installation procedure can be done by the two ways:

Use the apt savannah.fzk.de repository. Examples:

install the fzk-vomscert package from the apt repository:

 
        rpm savannah.fzk.de repository/fzk security
        cat << EOF > /etc/apt/sources.list.d/fzk.list 
              ###
              ### FZK apt repository containing some packages needed for DGrid
              ###   Currently these are the VOMS server certificate, and the GridKa-CA
              ###   configuration rpms. Do not remove this repository.
              ###
              rpm http://savannah.fzk.de repository/fzk security
 
        EOF
 
        apt-get update
        apt-get install fzk-vomscert

GSI configuration. Install the ca_FZK-local package from the following apt repository:
```
 rpm savannah.fzk.de repository/fzk security
```

Use the d-grid download area (see the following script)

site-info.def

Find a description of the different general variables in the site info configuration variables wiki:

System administrators are free to choose the configuration structure they prefer. It's possible to keep all the configuration variables in big site-info.def or maintain a smaller site-info.def together with services/, nodes/ and/or vo.d/ directories.

Configuration flow in YAIM. This is the order in which the different configuration files are sourced:

/opt/glite/yaim/defaults/site-info.pre
/root/siteinfo/site-info.def

/opt/glite/yaim/defaults/site-info.post

Mandatory service specific variables from /opt/glite/yaim/examples/siteinfo/services/glite-creamce

Variable Name	Description	Value type	Version
BATCH_CONF_DIR	Path where lsf.conf is located. Only when configuring LSF as a batch system.	String	4.0.4-12
BLPARSER_HOST	Fully qualified name of machine hosting the BLAH blparser	String	4.0.4-12
CEMON_HOST	Fully qualified name of CEMon host (do not use localhost !)	String	4.0.4-12
CREAM_DB_USER	Cream DB user name	String	4.0.4-12
CREAM_DB_PASSWORD	CREAM DB password	String	4.0.9-2

configure cream-CE with /opt/glite/yaim/bin/yaim -c -s "/opt/glite/yaim/site-info.def" -n creamCE -n TORQUE_utils
after the cream CE is configured, configure the Blparser (in our case on the same machine as creamCE, but it not only the case: http://igrelease.forge.cnaf.infn.it/doku.php?id=doc:guides:devel:install-cream32#blparser_configuration) with /opt/glite/yaim/bin/yaim -r -s <site-info.def> -n creamCE -f config_cream_blparser then service tomcat5 restart

Cream CE post-configuration
Enable DN authorization (optional)
CREAM supports two types of authorization (AuthZ) mechanisms: one AuthZ is VOMS based while the other AuthZ is specified by Grid User DNs (via the gridmapPDP). So if you want to also enable specific user DNs, list them in the /etc/grid-security/grid-mapfile, e.g.:
"/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=X Y/Email=x.y@pd.infn.it" .egee

"/C=IT/O=INFN/OU=Personal Certificate/L=Padova/CN=W Z/Email=w.z@pd.infn.it" .egee

administrator's script: configure.sh

```
 
```
```
#!/bin/bash
```
```
# create users
```

echo `useradd edguser -d /localhome/edguser`

echo `useradd edginfo -d /localhome/edginfo`

```
echo `useradd rgma -d /localhome/rgma`
```
```
 
```
```
# configure gLite
```

cp /opt/glite/yaim/examples/siteinfo/site-info.def /opt/glite/yaim/site-info.def

cp /opt/glite/yaim/examples/groups.conf /opt/glite/yaim/etc/groups.conf

cp /opt/glite/yaim/examples/users.conf /opt/glite/yaim/etc/users.conf

cp /opt/glite/yaim/examples/wn-list.conf /opt/glite/yaim/etc/wn-list.conf

```
 
```

# Since the site-info.def file contains passwords, it should NOT be readable for users!

```
chmod 600 /opt/glite/yaim/site-info.def
```
```
chmod 700 /opt/glite/yaim
```
```
 
```

# --------------------------------------------------------------infrastructure part

# fill in /opt/glite/yaim/etc/groups.conf

```
vi /opt/glite/yaim/etc/groups.conf
```
```
 
```

# fill in /opt/glite/yaim/etc/users.conf

```
vi /opt/glite/yaim/etc/users.conf
```
```
 
```
```
cat /opt/glite/yaim/etc/wn-list.conf
```
```
# list of WNs
```
```
# wn01.fzk.de ...
```
```
 
```
```
#Mandatory service specific variables
```

cat /opt/glite/yaim/examples/siteinfo/services/glite-creamce

```
#
```
```
# YAIM creamCE specific variables
```
```
#
```
```
 
```

# LSF settings: path where lsf.conf is located

```
# BATCH_CONF_DIR=lsf_install_path/conf
```
```
#
```

# CE-monitor host (by default CE-monitor is installed on the same machine as

```
# cream-CE)
```
```
CEMON_HOST=dgiref-glite32.fzk.de
```
```
#
```
```
# CREAM database user
```
```
# CREAM_DB_USER=creamdbuser
```
```
#
```
```
# Machine hosting the BLAH blparser.
```

# In this machine batch system logs must be accessible.

#BLPARSER_HOST=set_to_fully_qualified_host_name_of_machine_hosting_blparser_server

```
 
```
```
# -------------------------
```

# The following variables already have a default value defined in

# defaults/glite-creamce.pre, but if needed you can overwrite their values

```
#
```
```
# CREAM databases settings
```

# By default the cream db is on localhost and accessible from localhost.

# Setting ACCESS_BY_DOMAIN to yes, you allow the cream db access from all

```
# computers in your domain.
```
```
#ACCESS_BY_DOMAIN=no
```
```
#
```

# To refer to the the port where Blah Log Parser is running

```
#BLP_PORT=33333
```
```
#
```

# To refer to the parser listening cream port

```
#CREAM_PORT=56565
```
```
 
```

# Value to be published as GlueCEStateStatus instead of Production

```
#CREAM_CE_STATE=Special
```
```
#
```

# The following parameter sets the BLAH jobId prefix (it MUST be 6 chars

# long, begin with cr and terminate by '_')

# It is important in case of more than one ce connecting to the same blparser.

# In this case, it is better that each CREAM_CE has its own prefix

```
#BLAH_JOBID_PREFIX=cream_
```
```
 
```
```
 
```

# --------------------------------------------------------------site-info.* part

cat /opt/glite/yaim/defaults/site-info.pre

```
# ...
```
```
MY_DOMAIN=$(hostname -d)
```
```
INSTALL_ROOT=/opt
```
```
# REG_HOST=iwrrgma.fzk.de
```
```
EDG_HOME_DIR=/localhome/edguser
```
```
EDGINFO_HOME_DIR=/localhome/edginfo
```
```
GLITE_HOME_DIR=/localhome/glite
```
```
BDII_HOME_DIR=/localhome/edguser
```
```
BDII_PASSWORD="input password here"
```
```
# ...
```
```
 
```

wget -O /tmp/site-info.patch http://dgiref.d-grid.de/svn/dgiref/PROD/cf3/repl/scripts/site-info.patch

```
# apply patch
```
```
patch -p0 < /tmp/site-info.patch
```
```
 
```
```
echo 0 > /selinux/enforce
```
```
source /opt/glite/yaim/site-info.def
```
```
# configure TORQUE_utils
```

/opt/glite/yaim/bin/yaim -c -s "/opt/glite/yaim/site-info.def" -n TORQUE_utils

```
# configure creamCE
```

/opt/glite/yaim/bin/yaim -c -s "/opt/glite/yaim/site-info.def" -n creamCE

```
# configure blparser
```

/opt/glite/yaim/bin/yaim -c -s "/opt/glite/yaim/site-info.def" -n creamCE -f config_cream_blparser

```
service tomcat5 restart
```
```
chmod 777 /opt/glite/var/
```

Proceed

administrator's script: proceed.sh

```
 
```
```
#!/bin/bash
```
```
# proceed
```

# --------------------------------------------------------------Check syntax & configure system

```
service gLite restart
```

Initial test

To check the creamCE installation (you can also use: http://gkswiki.fzk.de/index.php5/Testing_CREAM_CE)

Run the CheckCreamConf script
Open your browser (where a valid certificate must be installed) to https://<hostname-of-cream-ce>:8443/ce-cream/services A page with link to the CREAM WSDL should be shown. (Because a jdk bug in elliptic crypto implementation, this could not work with certain browsers. Disabling "Use TLS 1.0" with Firefox can help)
Check in the CREAM log file (/opt/glite/var/log/glite-ce-cream.log) for the following strings:

org.glite.ce.cream.ws.StartUpManager - CREAM started!
org.glite.ce.cream.jobmanagement.cmdexecutor.blah.BLParserClient - Connection with BLParser (xxx) correctly established

Test glexec on the CREAM CE:
- Log on the CREAM CE: su tomcat -
- Consider a user proxy (e.g. /tmp/user.proxy) for a user authorized to use that CREAM CE. This proxy file must belong to tomcat.tomcat
- Do the following (This should return the id of the local user mapped to that Grid user):

export GLEXEC_MODE="lcmaps_get_account"
export GLEXEC_CLIENT_CERT=user.proxy
/opt/glite/sbin/glexec /usr/bin/id

Try a gsiftp (e.g. using globus-url-copy or uberftp) towards that CREAM CE. E.g.:

globus-url-copy gsiftp://<hostname-of-cream-ce>/etc/fstab -

Try the following command from a UI:

glite-ce-allowed-submission <<hostname-of-cream-ce>>:8443
It should report:
Job Submission to this CREAM CE is enabled

Try a submission to that CE using the glite-ce-job-submit command, e.g.:

$ /bin/cat test.jdl
[
executable="/bin/sleep";
arguments="1";
]
$ glite-ce-job-submit -a -r <hostname-of-cream-ce>:8443/cream-pbs-dteam test.jdl 
https://<hostname-of-cream-ce>:8443/CREAM336256203

Check the status of that job, which eventually should be DONE-OK:

$ glite-ce-job-status https://<hostname-of-cream-ce>:8443/CREAM336256203
******  JobID=<hostname-of-cream-ce>:8443/CREAM336256203 
       Status        = [DONE-OK]
       ExitCode      = [0]

Try a submission to that CE using the glite-ce-job-submit command, and then tries to cancel it (using the glite-ce-job-cancel command).

$ /bin/cat test.jdl
[
executable="/bin/sleep";
arguments="1000";
]

$ glite-ce-job-submit -a -r <hostname-of-cream-ce>:8443/cream-pbs-dteam test.jdl 
https://<hostname-of-cream-ce>:8443/CREAM510970530
$ glite-ce-job-cancel https://<hostname-of-cream-ce>:8443/CREAM510970530

- Check the status of that job, which eventually should be CANCELLED:

$ glite-ce-job-status https://<hostname-of-cream-ce>:8443/CREAM510970530

******  JobID=[https://<hostname-of-cream-ce>:8443/CREAM510970530
       Status        = [CANCELLED]
       ExitCode      = []

In case of errors, please see the pages available here

administrator's script: test.sh

```
 
```
```
#!/bin/bash
```
```
# test the CREAM-CE configuration
```

wget http://svn.rz.uni-karlsruhe.de/svn/dgiref/PROD/cf3/repl/scripts/perl/CheckCreamConf.pl

```
chmod +x CheckCreamConf.pl
```
```
./CheckCreamConf.pl
```
```
 
```
```
# initial tests for gLite installation 
```
```
### Create a voms proxy 
```

[grid user] $ voms-proxy-init --voms dgtest

```
 
```
```
### Show proxy info
```
```
[grid user] $ voms-proxy-info --al
```

#subject   : /C=DE/O=GermanGrid/OU=FZK/CN=Grid User/CN=proxy

#issuer    : /C=DE/O=GermanGrid/OU=FZK/CN=Grid User

#identity  : /C=DE/O=GermanGrid/OU=FZK/CN=Grid User

```
#type      : proxy
```
```
#strength  : 512 bits
```
```
#path      : /tmp/x509up_u7632
```
```
#timeleft  : 7:46:28
```

#=== VO dgtest extension information ===

```
#VO        : dgtest
```

#subject   : /C=DE/O=GermanGrid/OU=FZK/CN=Grid Jrad

#issuer    : /O=GermanGrid/OU=FZK/CN=host/dgrid-voms.fzk.de

#attribute : /dgtest/Role=NULL/Capability=NULL

```
 
```
```
### Create a sample job
```
```
[user]$ vi hostname.jdl
```
```
Executable = "hostname.sh";
```
```
stdOutput = "stdout";
```
```
stdError = "stderr";
```
```
InputSandbox = {"hostname.sh"};
```
```
OutputSandbox = {"stdout", "stderr"}
```
```
 
```
```
### Create input file for the job
```
```
[user]$ vi hostname.sh 
```
```
 hostname 
```
```
 /usr/bin/id sleep 10
```
```
 
```
```
### Submit a sample job.
```

[grid user]$ glite-wms-job-submit -a hostname.jdl

#Connecting to the service https://iwrrb.fzk.de:7443/glite_wms_wmproxy_server

#====================== glite-wms-job-submit Success ======================

```
#
```

#The job has been successfully submitted to the WMProxy Your job identifier is:

```
#
```

#https://iwrrb.fzk.de:9000/TsAUEzstiFMmbupVY37KWg

```
#
```

#==========================================================================

```
 
```
```
### Show job status
```

[user] $ glite-wms-job-status https://iwrrb.fzk.de:9000/TsAUEzstiFMmbupVY37KWg

```
 
```

### If status is '''done''' get the job output and store it locally

[user] $ glite-wms-job-output --dir . https://iwrrb.fzk.de:9000/TsAUEzstiFMmbupVY37KWg

middleware:Glite/32/server

Contents

gLite CREAM CE

Prepare

Install

Configure

Proceed

Initial test

Views

Personal tools

general

guidelines

reference installation

Search

internal

Toolbox