cluster:Torque/257/server

From Dgiref

Jump to: navigation, search

Please open a NGI-DE ticket if you experience any Installation or Configuration problem.

TORQUE v.2.5.7 server

Prepare

Operating system: Scientific Linux version 5.6 64 bit

Configure the following settings for the host:

UMD repo
NFS-server
Create Host Based Authentication for Torque clients. See ssh auth

If you don't want to use the maui scheduler instead of the torque scheduler, exclude the maui packets installation and maui configuration from the scripts

administrator's script: prepare.sh

```
 
```
```
#!/bin/bash
```

inside_network="10.0.171.0/255.255.255.0"

```
# install umd
```
```
#clean old
```

rm /etc/yum.repos.d/UMD* /etc/yum.repos.d/epel*

wget http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

wget http://repository.egi.eu/sw/production/umd/1/sl5/x86_64/updates/umd-release-1.0.2-1.el5.noarch.rpm

rm -f epel-release-5-4.noarch.rpm umd-release-1.0.2-1.el5.noarch.rpm

yum -y install epel-release-5-4.noarch.rpm

```
yum -y install yum-priorities
```

yum -y install umd-release-1.0.2-1.el5.noarch.rpm

sed -i -e "s/priority=.*/priority=5/g" /etc/yum.repos.d/UMD-1-base.repo

sed -i -e "s/priority=.*/priority=4/g" /etc/yum.repos.d/UMD-1-updates.repo

```
 
```
```
#nfs-server
```
```
yum -y instal nfs-utils-lib nfs-utils
```

echo "/var/torque/server_logs $inside_network (rw,async,no_root_squash)" >> /etc/exports

echo "/var/torque/server_priv/accounting $inside_network (rw,async,no_root_squash)" >> /etc/exports

```
/etc/rc.d/init.d/nfslock start
```
```
/etc/rc.d/init.d/nfs start
```

Install

Install Torque-server with yum from umd repo
Install MAUI scheduler

administrator's script: install.sh

```
 
```

yum -y install torque.x86_64 torque-server

yum -y install maui maui-server maui-client

Configure

Create default configuration for torque-server
Configure server name in /var/torque/server_name
Configure allowed WNs with number of CPUs into /var/torque/server_priv/nodes
Customization of Torque qmgr
- Create queue
- Create a server parameters (auth users, timouts, log level, torque managers, etc)
Create munge key for torque communication
Create maui scheduler configuration file

administrator's script: configure.sh

```
 
```
```
#!/bin/bash
```
```
/usr/sbin/pbs_server -t create
```
```
 
```

echo `hostname -f` > /var/torque/server_name

```
 
```

echo "wn01 np=2" > /var/torque/server_priv/nodes

echo "wn02 np=2" >> /var/torque/server_priv/nodes

echo "wn03 np=2" >> /var/torque/server_priv/nodes

echo "wn04 np=2" >> /var/torque/server_priv/nodes

echo "wn05 np=2" >> /var/torque/server_priv/nodes

echo "wn06 np=2" >> /var/torque/server_priv/nodes

echo "wn07 np=2" >> /var/torque/server_priv/nodes

echo "wn08 np=2" >> /var/torque/server_priv/nodes

echo "wn09 np=2" >> /var/torque/server_priv/nodes

echo "wn10 np=2" >> /var/torque/server_priv/nodes

```
 
```
```
 
```
```
# Create queues
```
```
echo `qmgr << 
```
```
'create queue dgipar
```
```
set queue dgipar queue_type = Execution
```
```
set queue dgipar max_queuable = 10
```
```
set queue dgipar max_running = 5
```

set queue dgipar acl_host_enable = false

set queue dgipar resources_max.walltime = 72:00:00

set queue dgipar resources_default.walltime = 72:00:00

```
set queue dgipar enabled = True
```
```
set queue dgipar started = True
```
```
#
```
```
# Create and define queue dgiseq
```
```
#
```
```
create queue dgiseq
```
```
set queue dgiseq queue_type = Execution
```
```
set queue dgiseq Priority = 100
```
```
set queue dgiseq max_queuable = 150
```
```
set queue dgiseq max_running = 130
```

set queue dgiseq acl_host_enable = False

set queue dgiseq resources_max.nodect = 6

set queue dgiseq resources_max.walltime = 288:00:00

set queue dgiseq resources_default.nodes = 5:ppn=2

```
set queue dgiseq enabled = True
```
```
set queue dgiseq started = True
```
```
#
```
```
# Set server attributes.
```
```
#
```

set server acl_hosts = dgiref-batch.fzk.de

```
set server managers = root@*
```

set server managers += tomcat@dgiref-glite.fzk.de

```
set server operators = root@*
```
```
set server default_queue = dgiseq
```
```
set server log_events = 511
```
```
set server mail_from = admin
```
```
set server query_other_jobs = True
```
```
set server scheduler_iteration = 600
```
```
set server node_check_rate = 150
```
```
set server tcp_timeout = 12
```
```
set server poll_jobs = False
```
```
set server log_level = 3
```
```
set server log_file_roll_depth = 7
```
```
set server next_job_number = 2387
```

set server authorized_users += *@dgiref-globus.fzk.de

set server authorized_users += *@dgiref-login.fzk.de

set server authorized_users += *@dgiref-ogsadai.fzk.de

set server authorized_users += *@dgiref-batch.fzk.de

set server authorized_users += *@dgiref-glite.fzk.de

set server authorized_users += *@dgiref-unicore.fzk.de

set server authorized_users += *@dgiref-dcache.fzk.de

set server authorized_users += *@dgiref-glite32.fzk.de

set server authorized_users += *@dgiref-globus50.fzk.de

set server authorized_users += *@dgiref-globus40.fzk.de

```
'`
```
```
 
```
```
# configure munge
```
```
/usr/sbin/create-munge-key
```

# Generating a pseudo-random key using /dev/urandom completed.

#chown munge:munge /var/log/munge/munged.log*

```
 
```
```
#configure for maui
```
```
echo"
```
```
UI configuration example
```
```
# @(#)maui.cfg David Groep 20031015.1
```
```
# for MAUI version 3.2.5
```
```
#
```

SERVERHOST              dgiref-batch.fzk.de

```
ADMIN1                  root
```
```
ADMINHOST               sn05
```
```
RMTYPE[0]           PBS
```
```
RMHOST[0]           sn05
```
```
RMSERVER[0]         sn05
```
```
 
```
```
SERVERPORT            40559
```
```
SERVERMODE            NORMAL
```
```
 
```

# Set PBS server polling interval. Since we have many short jobs

# and want fast turn-around, set this to 10 seconds (default: 2 minutes)

```
RMPOLLINTERVAL        00:00:10
```
```
 
```

# a max. 10 MByte log file in a logical location

```
LOGFILE               /var/log/maui.log
```
```
LOGFILEMAXSIZE        10000000
```
```
LOGLEVEL              3
```
```
">/var/spool/maui/maui.cfg
```

Proceed

Make pbs_server, munge, maui a services for OS and add it to autoboot system. So it will be started during boot.

administrator's script: proceed.sh

```
 
```
```
service munge restart
```
```
service maui restart
```
```
service pbs_server restart
```
```
 
```
```
chkconfig maui on
```
```
chkconfig munge on
```
```
chkconfig pbs_server on
```
```
chkconfig pbs_mom off
```

Initial test

From a user account, it should be possible to use a 'Hello World' job submitting, as well as an interactive shell on a WN
The job results are as files STDIN.o<JOBID> (std-output) and STDIN.e<JOBID> (std-error).
Test MAUI
The test on the gLite-CE should work as edginfo user configuration of gLite-packages.
To check the status of the job query, the qstat command is used within the lifetime of submitted jobs.

administrator's script: test.sh

```
 
```
```
#!/bin/bash
```
```
#-> start routine
```

# verify all queues are properly configured

```
qstat -q
```
```
# view additional server configuration
```
```
qmgr -c 'p s'
```

# verify all nodes are correctly reporting

# If everything Ok, you can see the all WNs details with the WN status "free".

```
pbsnodes -a 
```
```
# submit a basic job
```
```
echo "sleep 30" | qsub
```
```
# verify jobs display
```
```
qstat
```
```
# check queues:
```
```
qstat -Q
```
```
 
```
```
# shutdown server
```
```
qterm -t quick
```
```
#<- end routine
```

cluster:Torque/257/server

Contents

TORQUE v.2.5.7 server

Prepare

Install

Configure

Proceed

Initial test

Views

Personal tools

general

guidelines

reference installation

Search

internal

Toolbox