Installation of MUSR DAQ software on a linux system (PPC/VxWorks version)

This document describes how to set up the frontend software necessary for running the MuSR Data Acquisition system on a linux computer.

Hardware

The relevent VME hardware consists of

• Highland/BNC V680 TDC
• MVME2306 PowerPC processor (i.e. 32Mbytes or larger)
• SIS3803 Scaler
• TRIUMF IO Board
• MVME162 cpu for CAMP

The host is a linux pc.

The TDC has the following front panel connections:

        IN 0-7          standard signal inputs
        IN 8            reference input
        GATE IN         gate input
        AUX1            sample/reference input signal
        AUX2            not used
        CLOCK I/O       not used

A microengine in the TDC running custom-built code (by A.Daviel, TRIUMF) filters the data and determines when a valid event has occurred.

Currently the IO board uses output 15 for sample/ref. level.
 

Software

The standard  Midas DAQ system is used.  Communication between the host (linux pc) and frontend (ppc) is mostly via RPC over Ethernet.
Note: we are currently using VxWorks (for Motorola MVME2300 version 5.3.1) as the operating system for ppc and linux 7.2 on the host.

In addition to the standard Midas code and various header files, the frontend software (running on the ppc under VxWorks)  contains the following components:
 

Module	Description
fev680.c	Midas frontend polls for valid data from the TDC, which are histogrammed in a reserved area of memory (i.e. histogram memory HM). Periodically the scaler is readout, and the scaler and histogram data  is sent to the host (linux pc) as Midas events, and saved onto disk.
sis3803.c	driver for scaler module
musrvmeio.c	driver for IO module

The CAMP software is a separate package consisting of a server and client. The server and related software is downloaded to the RAM disk of the MVME162. It is NOT part of the standard Midas DAQ system.
 

Ethernet address

Contact the Triumf DAS staff to obtain an Ethernet address if using a new ppc board. An Ethernet address is also needed for the MVME162  board for CAMP.

Note that the target nodenames for the MVME HM/TDC node are of the form "m15hmvw" to distinguish them from the MVME CAMAC nodes, whose nodenames are of the form "m15vw". Both the MVME nodes "m15vw" and "m15hmvw" are associated with the beamline M15. New modules should also follow this naming scheme.

NOTE:  For MUSR, a range of  run numbers is allocated to each beamline. If a new beamline is designated, the software that allocates the next run number will have to be modified.
 

Create the directories needed

On the linux pc, make an account "musrdaq" and a beamline account  "m20" ("m20" is used throughout as an example).
The account "musrdaq" will not be accessible to the users, and contains code for the DAQ system. Users will log in as the beamline account, and run the DAQ software from the musrdaq account. This means users cannot accidentally delete vital files.

Log on as musrdaq. Copy .login and .cshrc files for both accounts from another DAQ machine.

Be sure to fix all references to the beamline in these files, i.e. if copying from m15, change all occurrences of m15 to m20.

Currently, there are only two beamline-specific references in .cshrc or .login :

1. setenv BEAMLINE m20      (musrdaq account)
2. setenv CAMP_HOST m20vw (musrdaq and m20 accounts)

Create  needed subdirectories:
In beamline account, create the directory for saving the rigs,  e.g.
mkdir ~m20/online/musr/rigs

and in the musrdaq account, i.e.
mkdir ~musrdaq/musr

As su:
mkdir /var/log/midas
and set permissions so everyone can write (needed for perlscript message files).

mkdir /data/m20/2002
and set the owner and group to  "m20". On some machines a group containing users m20 and murdaq is set up.

VxWorks system boot files and startup file for ppc

Login as "midas" and create the directory from which ppc will boot:
/home1/midas/vw-ppc/config/m20hmvw
Copy across (e.g. from midm15)  the VxWorks system files vxWorks and vxWorks.sym to this directory.

Also copy  the musr startup file (musrppc.cmd) from midm15,
scp midas@midm15:/home1/midas/vw-ppc/config/m15hmvw/musrppc.cmd .
and
edit it to change  beamline "m15" to "m20".
 
 

Install the code from  DAQ's CVS

mkdir musr
cd musr

Checkout from cvs the relevent directories, i.e.

• camp
• mud
• mdarc
• musr_midas
• libc_tw
• sis
• vmeio
• vw

These should appear as ~musrdaq/musr/camp  etc.  Use cvs commands such as:
cvs -d "user@midas:/usr/local/cvsroot" checkout -d camp musr/camp
to check out the above directories. Substitute your username for "user".

Note:
CVSROOT and CVS_RSH must be defined to access cvs, ie.
CVSROOT=musrdaq@midas:/usr/local/cvsroot
CVS_RSH=ssh
 

Create the Midas experiment

All MUSR DAQ systems use the Midas experiment name "musr".
The experiment must be added to /etc/exptab, so (as su) add a line:

musr   /home1/m20/online/musr m20
 

Create experim.h

Save the odb contents of another DAQ experiment (e.g. m15)  using odb save command, and copy across to your system ,
e.g.
cd ~m20/online/musr
scp m15@midm15:/home/m15/online/musr/musr.odb .

Edit this file (musr.odb) , and remove all /system, /alarm, /programs stuff, and change any beamline values to your beamline (e.g. "m15" to "m20).  Examples are the camp hostname, the rig path, the saved data area.

Load this file into your odb, and save an experim.h, e.g.
odb
[midm20:musr:S]/>load musr.odb
[midm20:musr:S]/>make

Rig and Mode Files

Copy examples from m15 to
~m20/online/musr/rigs
keeping the same directory structure (i.e. if a rig named "test_rig" exists with modes "try1" and "try2", a subdirectory "test_rig" must be created, containing the rig file test_rig.rig and mode files try1.mode and try2.mode. Rigs and Modes can easily be edited, rename and created later by the MUSR Run Control program.

Build the programs

After creating experim.h in ~m20/online/musr, the programs can be built.
Unfortunately there is currently no Makefile that builds everything.
 

• As musrdaq, build the standard libraries for camp:

cd ~musrdaq/libc_tw/linux
make
cd ~musrdaq/mud
make
cd ~musrdaq/camp/linux
make

• Build  MUSR code running on the linux pc:

cd ../mdarc
make
cd  ../musr_midas/musr_config
make
cd ../midas_tcl
make

• Build the MUSR VxWorks frontend running on the ppc :

cd ../frontend

mkdir m20        (this is NOT a cvs directory so does not get updated by a cvs update command)
Get copies of the latest code from the cvs directories:
cp Makefile m20/
cp *.* m20/  V680.c,V680.h,fev680.c,show_commands.c should be copied
cp ../../sis/sis3803.* m20/
cp ../../vmeio/musrvmeio.* m20/
cp /home1/midas/Midas/drivers/bus/camacnul.c m20/
make

The ppc code is built in ~musrdaq/musr/musr_midas/frontent/ppcobj
This is where the ppc loads its files (except system files).
 
 
 
 

PPC Startup File

For MUSR experiments, the startup file for the ppc is musrppc.cmd. Copy the startup file from another DAQ system e.g. from m15 into /home1/midas/vw-ppc/config/m20hmvw
Edit  musrppc.cmd for m20, changing all occurences of m15 to m20. Supply the correct names ip addresses for the ppc and the host.
 

Allow the MVME board to nfs mount the host's filesystem

The ppc needs permission to nfs-mount the host's filesystem.  Follow the instructions  to add permissions for nfs and for rshell. The ppc uses rshell to access the startup command file.

Attaching a debug terminal and ethernet to the MVME2305

The front panel of the MVME2305 board has two connectors of type RJ45. Standard cables are used to connect to Ethernet via the 10/100baseT connector.

Connect the RJ45 "debug" connector to  the serial port on the linux box. It is best to modify one of the old blue switch boxes (e.g. in M15) so that two MVME boards may both share a single serial port.  Cables can be made up  copied from the existing cables in M15 .

Access ppc consol with seyon

As su, make a symbolic link between the device seyon uses (/dev/modem) and the serial port:
ln -s /dev/ttyS1 /dev/modem
Bring up a seyon window:
seyon &
 

Setting the boot parameters of the ppc

Determine the boot parameters of the ppc by using those for and existing DAQ machine as a template. The bootup parameters for m20hmvw are shown below:
 

M20HMVW (for VME HM/TDC) booted from midm20  (linux)

boot device                : dc
processor number    : 2
host name                  : midm20
file name                    : /home1/midas/vw-ppc/config/m20hmvw/vxWorks
inet on ethernet (e) : 142.90.101.46
host inet (h)              : 142.90.101.74
user (u)                       : midas
ftp password (pw)     :
flags (f)                       : 0x8
target name (tn)       : m20hmvw
startup script (s)       : /home1/midas/vw-ppc/config/m20hmvw/musrppc.cmd

Follow the instructions to change the boot parameters after bringing up the ppc consol (using seyon).

NOTE:  It is a good idea the first time the ppc is booted with an untested startup script,  to boot with a blank "startup script" parameter.  After bootup, enter the startup script  by hand, i.e.
-> </home1/midas/vw-ppc/config/m20hmvw/musrppc.cmd

Reboot the ppc and make sure it is able to successfully nfs-mount the host's filesystem.
-> nfsDevShow

If all works it should go through the startup script without errors.

Testing the new system

Log on as the beamline (m20).
Execute kill-all and start-all and make sure they are working properly.
Start and stop a run, save data etc.
Make sure that the run number checker produces the correct real run number for your system.
 
 

VxWorks system boot files and startup file for CAMP MVME162

To setup the CAMP MVME162 board to boot from linux  follow these instructions.
 

suz@triumf.ca, TRIUMF Data Acquisition Group