cluster:Accounting

From Dgiref

Introduction

D-Grid accounting is based on the Distributed Grid Accounting System (DGAS), implemented by the DataGrid. The primary task is to record every user data about the grid resources usage.

Accounting structure in D-Grid can be separated into two types:

VO internal accounting. The Virtual Organization (VO) need to have own DGAS server and installed DGAS clients.
Central accounting. Only the DGAS clients will be installed.

This two types can be combined and get the opportunity for VO to have data into their own server and publish it on the central accounting server. The accounting information can be visualized with the Web-based interface - HLRmon. HLRmon uses data from the DGAS database with so-called Home Location Register (HLR) and stores the accumulated information of each site. The use of individual user data refers HLRmon directly from the DGAS database.

The access to the data can be done with Command Line Interface (CLI) to the DGAS or with HLRmon. There are some user roles which can manipulate with the accounting data: VO user, VO manager, Site manager, ROC manager.

the DGAS client installation depends on the batch system in use (Torque/PBS or other batch system).
the D-Grid reference installation uses the Torque batch system, hence there are two options for DGAS installation:

for site with gLite CE in use
for site without gLite CE in use.

Package: 1.0
os: Scientific Linux version 5.6 64 bit

manuals: DGAS client with / without gLite CE

Information links

Please open a NGI-DE ticket if you experience any Installation or Configuration problem.

Accounting with glite server

Install

The DGAS client is located into the gLite 3.1 server and based on the LCG-CE packages installation:

glite-dgas-common
glite-dgas-hlr-clients
lcg-dgas-tools

administrator's script: install.sh

```
 
```
```
#!/bin/sh
```

# install the accounting client with glite CE

```
# on the glite CE
```

yum install glite-dgas-common.i386 glite-dgas-hlr-clients.i386 lcg-dgas-tools.i386

Configure

The following steps should be done to configure the accounting client:

The torque /var/spool/torque/server_priv/accounting directory should be exported and mounted by the DGAS client.
The following file should be configured:
1. /glite/etc/dgas_atmClient.conf
2. /glite/etc/dgas_ce_getAcctLogd.conf (standard configuration, nothing to be changing)
3. /glite/etc/dgas_ce_pushd.conf (siteName abbriviation without spaces!)
4. /glite/etc/dgas_gianduia.conf (siteName abbriviation without spaces!)
5. /glite/etc/dgas_VOUserHLR.conf (standard configuration, nothing to be changing)
6. /opt/glite/sbin/glite-urcollector.pl (includes changes into the writeGianduiottoFile function)
The /etc/grid-security directory store the following important files:
1. /etc/grid-mapfile.dgas_special
2. /etc/groupmapfile.dgas
SPEC Benchmark
1. /opt/glite/etc/dgas_gianduia.conf to be registered for its own resources in the Home Location Register
Register the user into the DGAS database - send the email with DN into the accounting@rvs.uni-hannover.de
copy the initial DGAS scripts into the /etc/init.d and configure the activation with chkconfig

administrator's script: configure.sh

```
 
```
```
#!/bin/sh
```

# configure the accounting client with glite CE

# mount /var/spool/torque/server_priv/accounting from batch server

# Note: the batch server should export the directory with NFS

```
TORQUE_SERVER=dgiref-batch.fzk.de
```
```
 
```

mkdir -p /var/spool/torque/server_priv/accounting

```
mount -a
```
```
 
```
```
# 
```
```
cat /opt/glite/etc/dgas_ce_pushd.conf 
```
```
#
```
```
cat /opt/glite/etc/dgas_gianduia.conf
```
```
#
```

cat /opt/glite/sbin/glite-urcollector.pl

```
 
```
```
#
```
```
cat /opt/glite/etc/dgas_gianduia.conf
```
```
 
```
```
# copy to the /etc/init.d
```

cp /opt/glite/sbin/glite-dgas-urcollector /etc/init.d

cp /opt/glite/sbin/glite-dgas-pushd /etc/init.d

```
 
```
```
# configure chkconfig
```
```
chkconfig glite-dgas-urcollector on
```
```
chkconfig glite-dgas-pushd on
```

Please open a NGI-DE ticket if you experience any Installation or Configuration problem.

Accounting without glite server

Install

The standalone DGAS client with additional packages can be installed.

the client location is the batch server to generate users records from the batch logs.
additional package includes
- the gianduia components to automatically transform the Accounting logs into the DGAS records
- standalone DGAS client to send data to the DGAS server

The necessary grid mapfiles can be obtained automatically with configuration for the gianduia package.

administrator's script: install.sh

```
 
```
```
#!/bin/sh
```

# install the accounting client without glite CE

```
# on the batch server
```

rpm -ihv http://mirror.scc.kit.edu/downloads/rpms/accounting/2010.1/atmClient-standalone-1.0.1-1.i386.rpm

rpm -ihv http://mirror.scc.kit.edu/downloads/rpms/accounting/2010.1/gianduia-1.0.3-1.i386.rpm

Configure

The following steps should be done to configure the accounting client:

The torque /var/spool/torque/server_priv/accounting directory should be exported and mounted by the DGAS client.
The following file should be configured:
1. /glite/etc/dgas_atmClient.conf
2. /glite/etc/dgas_ce_getAcctLogd.conf (standard configuration, nothing to be changing)
3. /glite/etc/dgas_ce_pushd.conf (siteName abbriviation without spaces!)
4. /glite/etc/dgas_gianduia.conf (siteName abbriviation without spaces!)
5. /glite/etc/dgas_VOUserHLR.conf (standard configuration, nothing to be changing)
6. /opt/glite/sbin/glite-urcollector.pl (includes changes into the writeGianduiottoFile function)
The /etc/grid-security directory store the following important files:
1. /etc/grid-mapfile.dgas_special
2. /etc/groupmapfile.dgas
SPEC Benchmark
1. /opt/glite/etc/dgas_gianduia.conf to be registered for its own resources in the Home Location Register
Register the user into the DGAS database - send the email with DN into the accounting@rvs.uni-hannover.de
copy the initial DGAS scripts into the /etc/init.d and configure the activation with chkconfig

administrator's script: configure.sh

```
 
```
```
#!/bin/sh
```

# configure the accounting client without glite CE

# mount /var/spool/torque/server_priv/accounting from batch server

# Note: the batch server should export the directory with NFS

```
TORQUE_SERVER=dgiref-batch.fzk.de
```
```
 
```

mkdir -p /var/spool/torque/server_priv/accounting

```
mount -a
```
```
 
```
```
# 
```
```
cat /opt/glite/etc/dgas_ce_pushd.conf 
```
```
#
```
```
cat /opt/glite/etc/dgas_gianduia.conf
```
```
#
```

cat /opt/glite/sbin/glite-urcollector.pl

```
 
```
```
#
```
```
cat /opt/glite/etc/dgas_gianduia.conf
```
```
 
```
```
# copy to the /etc/init.d
```

cp /opt/glite/sbin/glite-dgas-urcollector /etc/init.d

cp /opt/glite/sbin/glite-dgas-pushd /etc/init.d

```
 
```
```
# configure chkconfig
```
```
chkconfig glite-dgas-urcollector on
```
```
chkconfig glite-dgas-pushd on
```

cluster:Accounting

Contents

Introduction

Accounting with glite server

Install

Configure

Accounting without glite server

Install

Configure

Views

Personal tools

general

guidelines

reference installation

Search

internal

Toolbox