Client setup

Client protocol extension

Install a suitable kernel module

% rpm -q kernel-module-openafs-`uname -r`

Restart of the AFS service

Upon startup of the afs service, the following should show up:

# service afs restart
# dmesg | grep patch
libafs with   'rxosd support' 'vicep-access' patches

(vicep-access is not necessary for RxOSD operation per se, but is required for making proper use of a Lustre or GPFS backend.)

Because afs restart doesn't work often, is is better to reboot the client.

Proper afs commands

Install a suitable openafs package

# rpm -q openafs
# fs protocol
Enabled protocols are  RXOSD (1 parallel streams on connections with rtt > 10 ms).
# osd help
osd: Commands are:

Memory cache

When using a cluster filesystem backend, memcache has proven to be the fastest alternative (at least for Lustre via Infiniband).

In /etc/sysconfig/afs:

OPTIONS="-memcache -stat 8192 -chunksize 20 -daemons 8 -volumes 64"

This selects memcache and boosts the chunk size to 1MB. To avoid a low overall number of cache chunks:


256 MB seems a reasonable size.

Accessing a cluster filesystem backend

<!> In Zeuthen, the Lustre symlinks is handled by the SLZ_vicep_lustre_links RPM.

Generally (e.g. Lustre):

Fileserver setup

Adding an rxosd service

It is sensible to have fileserver and rxosd processes share the same machines. This way, access to files even in a single volume can be spread to all fileservers (given that clients don't access the same (set of) file(s) all the time).

Using a Lustre storage backend

Basically, this is the same process as adding a regular rxosd service. The vice partition resides in a Lustre filesystem, however. The rxosd process must be launched with a special parameter:

# bos create <server-machine> rxosd simple "/usr/afs/bin/rxosd -lustrehack"

(!) A larger maximum file size for your Lustre-OSD is not required, the fileserver will never get to see a size that surpasses any client's cache size.

Lustre mountpoint


# mount -t lustre /vicept
# touch /vicept/OnlyRxosd

The rest is identical to the above.

If Lustre is mounted anywhere else:

# ln -s /lustre/vpxl /vicepal
# touch /vicepal/OnlyRxosd
# touch /vicepal/AlwaysAttach

The last command enables the use of a regular directory as a vice partition. The rest is identical to the above.

Verifying the correct setup of the OSD

Assuming both service instance and vice partition were available at the time of updating the database, osd l should contain something along the lines of

  2 zyklop20-al    10694 gb   0.0 % up         64  64     zyklop20 37 (1mb-2tb)

The "up" signifies that the OSD is ready.

A "down" entry does not have to be a problem. It can (among other things) mean

Database servers


If appropriate openafs-server packages are in use, there is nothing to do.

# rpm -q openafs-server
# rpm -qf /usr/afs/bin/osdserver

For a minimum intrusion on existing DB servers, use the dedicated openafs-osdserver package:

% rpm -q openafs-server openafs-osdserver
% rpm -qf /usr/afs/bin/osdserver

It has no dependencies and will install regardless of the rest of your local AFS installation.

Launching the OSDDB service

With admin privileges:

bos create <db-machine> osddb simple /usr/afs/bin/osdserver

Once the osdservers are up and running, clients should be able to see this:

% osd l
 id name(loc)     ---total space---      flag  prior. own. server lun size range
  1 local_disk                                 wr  rd                 (0kb-1mb)

The local_disk entry is a default.

Enabling OSD for Volumes ...

 # vos setfields osdtestvol2 -osd 1

Update Client and Server

AfsOsd/SetupInstructions (last edited 2010-03-08 10:11:58 by ReneStandke)