hmrpc.html
Installing MVME TDC and HM Support via the MVME2305 PowerPC
This document describes how to set up the software necessary for running the MuSR Data Acquisition using a Highland/BNC V680 TDC in VME via an MVME2305 PowerPC processor, plus the associated software histogram memory (HM).
The hardware setup consists of a VAX, a MVME2305 board with a MPC604 PowerPC microprocessor and an 8-channel Highland V680 or BNL B980 TDC in a VME crate.
The TDC has the following front panel connections:
IN 0-7 standard signal inputs IN 8 reference input GATE IN gate input AUX1 sample/reference input signal AUX2 not used CLOCK I/O not used
A microengine in the TDC running custom-built code (by A.Daviel, TRIUMF) filters the data and determines when a valid event has occurred.
The software consists of a VxWorks program (by P-A Amaudruz,TRIUMF) running on the MVME2305 which polls for valid data from the TDC. Valid data are then histogrammed in a reserved area of memory known as the Histogram Memory (HM). The HM is read out at intervals by the VAX data acquisition program, which stores the histogram data onto disk. The VAX and MVME2305 communicate through the ethernet (mostly RPC).
Update local DNS tables
Contact the Triumf DAS staff to obtain an Ethernet address for the new MVME board. If the DAQ computer is using local name serving (this is true for all MUSR DAQ nodes) then you must update the DNS tables, or the VAX will not recognise the MVME board.
Allow the MVME board to nfs mount the VAX's filesystem
Add permission for the MVME board to NSF-mount the VAX's filesystem.
Booting the MVME2305
We are currently using VxWorks (for Motorola MVME2300 version 5.3.1) as the operating system. Development (e.g. cross-compilation) is done on "dasdevpc" (a pc running the WindRiver Tornado software package under Linux). The "midas" account is used for development on dasdevpc, where the original versions of the files are kept. The directory "dasdev::dsk1:[musr.vw.hm]" contains the latest versions of these files suitable for loading from a VAX.
First, create the directory "dsk1:[musr.vw.hm]" on the target VAX.
Copy the following files from DASDEV::dsk1:[musr.vw.hm] into the equivalent directory on the target VAX.
vxworks. (VxWorks kernel)
vxworks.sym (VxWorks symbol table)
startup_*.cmd (startup script modified for VAX)
hm_camconvert.o (HM driver using pseudo-CAMAC calls)
hmrpcsrv.o (HM RPC Server)
libmidas.o (midas library)
v680.o (main program for TDC and HM)
vmeio.o (debug - may be removed )
vmeio.txt (debug - may be removed )
memsiz.txt (size of HM available for this hardware)
vpar.txt (script to reload object modules)
.rhosts (file to permit listed users to run rshell on the host)
Startup scripts (startup_*.cmd) are needed for each target host node. You will specifiy the particular file for your host in the boot parameters of the MVME board.
Edit the .rhosts file to allow the MVME board to run rshell on the VAX. Once you have the proper files setup on the VAX, you must configure the boot parameters on the MVME, which are stored in RAM.
Attaching a debug terminal and ethernet to the MVME2305
The front panel of the MVME2305 board has two connectors of type RJ45. Cables can be made up copied from the existing cables on M15 or DASDEV. Connect the connector labelled "10/100baseT" to M01DAC MODA MVME DASDEV MODA MVMEthe ethernet (a repeater/converter to thin-wire ethernet is necessary in the counting rooms). Connect the RJ45 "debug" connector to a VT100 or the serial port of the NCD X-windows terminal. The CAMAC MVME board may already have such a connection (from device "/tyCo/0"). It is best to modify one of the old blue switch boxes (e.g. in M15) so that two MVME boards may both share the single serial port.
To configure the boot parameters
Display a serial port window on the NCD X-windows terminal.If the MVME2305 board is brand new, you must first flash the Boot Rom. using Motorola PPC1-Bug (see sundown/usr1/local/tornado_ppc_1.0.1/target/config/mv2304_orig/target.txt).Consult the VxWorks Programmer's Guide for more details on how to go about setting the parameters. Note that the boot-up parameters require a username and password are necessary in order to download the kernel and symbol table using FTP. This user must have read access the kernel and symbol table.
Note that the target nodenames for the MVME HM/TDC node are of the form "m15hmvw" to distinguish them from the MVME CAMAC nodes, whose nodenames are of the form "m15vw". Both the MVME nodes "m15vw" and "m15hmvw" are associated with the beamline M15.
You may now boot the MVME. It will take a few minutes to boot, and a display ( known as v680_display) should eventually appear on the MVME debug display. This shows various parameters for the MVME HM and (by default) is continuously updated.
Modifying the VAX Software
Edit the file dasdev::dsk1:[musr.manager]modas_startup.com, and add the target VAX to the definition of which nodes have a VME HM and TDC installed. Define the logical hm_host as the target VME HM nodename. Similarly edit the file install_mvme_hm.com in that account. Copy these files across to the target node to dsk1:[musr.manager]. This ensures that when the system is rebooted, and the MVME CAMAC interface is selected, the MuSR software will be configured for VME HM and TDC automatically.
NOTE: It is NOT necessary to reboot to change a VAX running with a MVME CAMAC system to a MVME HM/TDC system. A few software changes suffice.
Note also that changing to MVME HM/TDC does not affect I-Musr software in any way. It is only relevent to TD-Musr, therefore I-Musr will not be further mentioned here.
Location of MODAS files
The MODAS system expects TD-MuSR files in the following directories:
ed:[com] - command files ed:[exe] - executables ed:[rcdb] - run control database filesIn order for the system to work with multiple CAMAC interfaces, it was decided that the executables for the interfaces would be stored in different directories, and upon bootup, the executable directory ed:[exe] is aliased to the appropriate permanent directory. So, complete sets of TD-MuSR executables are now stored in:
ed:[exe_camif] - Q-Bus executables ed:[exe_camrpc] - RPC (MVME) executables ed:[exe_camscsi] - direct SCSI to Jorway 73A executablesIf the RPC (MVME) CAMAC interface is chosen, for example, then the following commands are performed at system startup to make the required aliases:
$ set file/enter=dsk1:[musr.exp.td_musr]exe.dir -
dsk1:[musr.exp.td_musr]exe_camrpc.dir
This is done in dsk1:[musr.manager]modas_startup.com.
Setting up new directories for MVME HM/TDC support
To setup the VAX for a VME TDC & HM via the MVME2305, three new directories have to be created, i.e.
ed:[exe_vmerpc] - RPC (MVME) executables with VME HM/TDC support
ed:[camcom] - command files for CAMAC only
ed:[vmecom] - command files with VME HM/TDC support
In addition to ed:[exe], ed:[com] will now be assigned to
point to the appropriate directory.
If the RPC (MVME) CAMAC interface is chosen, for example,
and the MVME VME/HM is available on the system,
then the following two commands will be performed at system startup
to make the required aliases:
$ set file/enter=dsk1:[musr.exp.td_musr]exe.dir -
dsk1:[musr.exp.td_musr]exe_vmerpc.dir
$ set file/enter=dsk1:[musr.exp.td_musr]com.dir -
dsk1:[musr.exp.td_musr]vmecom.dir
This is done in dsk1:[musr.manager]modas_startup.com.
Create the above three directories on the target VAX. Copy all the existing files from ed:[com] to the new directories. The directory ed:[com] must be emptied, i.e.
$create/dir ed:[camcom] $create/dir ed:[vmecom] $create/dir ed:[exe_vmerpc] $copy ed:[com]*.*;* ed:[camcom] $copy ed:[com]*.* ed:[vmecom] $delete ed:[com]*.*;* $copy ed:[exe_camrpc]*.* ed:[exe_vmerpc]Now copy the new versions of files from DASDEV to the new vme accounts, overwriting the old ones as follows:
$copy dasdev::dsk1:[musr.exp.td_DEV.SOURCES.VME_HM]*.exe ed:[exe_vmerpc]/log
$purge ed:[exe_vmerpc]*.exe
$copy dasdev::dsk1:[musr.exp.td_DEV.vmecom]*.com ed:[vmecom]/log
$purge ed:[vmecom]
New version of MUD
A new version of mud_util is needed to provide support for the VME HM resolution code values.
$copy dasdev::dsk1:[musr.exp.td_DEV.sources.vme_hm.mud]mud_util.exe mud:
Changes to Run Control Data Base (RCDB)
Some new mode parameters must be added to the RCDB (Run Control Data Base) for the VME HM case. These new parameters are listed in dasdev::dsk1:[musr.exp.td_dev.sources.vme_hm.rcdb]readme.txt.
Currently, there are 5 new parameters required, shown below with their default values:
VME_TDC_AUX1 DISABLE HIS_SUFFIX _s _r HIS_FIRST_BKG 0 HIS_LAST_BKG 0 VME_MODE_TITLE sample referenceCopy across the updated rcdb logical tables, i.e.
$copy dasdev::dsk1:[musr.exp.td_DEV.sources.vme_hm.rcdb]rcdb*.tab ed:[rcdb]*.*/log
NOTE - It is prudent to rename (and purge) the existing *.RCDB files in ed:[rcdb] to another directory, since the old *.RCDB files are not compatible with the VME HM/TDC system, and should not be used. Keep one rcdb file in ed:[rcdb] to act as a template (see below).
e.g. $ cr/dir ed:[cam_rcdb] $ rename ed:[rcdb]*.rcdb;* ed:[cam_rcdb] $ rename ed:[rcdb]*.tab;* ed:[cam_rcdb] $ rename ed:[cam_rcdb]arig.rcdb ed:[rcdb] ! get one file back
Then use the COPY command in ed:[rcdb]rcdb.com to copy a rig, and rename the original rig back to ed:[cam_rcdb], e.g.
$ @ed:[rcdb]rcdb.com COPY
.......
Specify name of rig to be copied: ARIG
Specify name of new rig: VME_HM
.......
$ rename ed:[rcdb]arig.rcdb ed:[cam_rcdb]
Then add the new mode parameters
listed above to the new rig VME_HM using the ADD command in ed:[rcdb]rcdb.com. You must be
logged in as a Musr user, e.g. as M15 on M15DAC. The ADD
command will ask to which rigs you wish to add the new parameter.
Note that you must type the rig name in
UPPER CASE for the COPY and ADD commands.
$ @ed:[rcdb]rcdb.com ADD
.......
Specify name of variable to be added: HIS_SUFFIX
........
Specify initial setting for VME_MODE_TITLE: _s _r
........
HIS_SUFFIX will be initialised to its default value of _s _r.
The following Rig RCDB's exist in ED:[RCDB]
VME_HM
Add new variable to all rigs ["Y"]? y
Variable VME_MODE_TITLE will be added to rigs:
VME_HM
Processing Rig VME_HM.
.......
You must repeat the procedure for each new VME mode parameter in turn.
Increase system resources available to the MODAS user
Since the VME HM allows for greatly increased histogram lengths, it is necessary to increase the system resources available to two accounts: MUSR_NTJ and the MUSR user on the system (e.g. M15 on M15DAC).
From a privileged account,
$ set def sys$system $ mcr authorize UAF> show musr_ntj UAF> show m15 ( supply appropriate username)Compare the values for pagefile quota (Pgflquo), byte limit (Bytlm), working set quota (WSquo) with those on M15, and increase the values appropriately for both the accounts, e.g.
UAF> modify musr_ntj / pgflquo=100000
Check the system page and swap file sizes
$ show memThe page file size has had to be substantially increased on M15DAC and DASDEV to enable long histograms to be processed both by the data archiver (DARC) and processed by Physica. Increase the page file accordingly (modify sys$system:modparams.dat) and run autogen. It is as well to repeat the autogen after large histograms have actually been used on the system. If the pagefile quota etc, are insufficient, the data archiver and physica will both fail due to insufficient resources.
Testing the new software
Execute the command
$ @dsk1:[musr.manager]install_mvme_hm.com
This should set up the file entries ed:[com] and ed:[exe], and define the system logical
HM_RPC_HOST. Check that these point to the correct directories, and that HM_RPC_HOST is
correctly set.
Try the data acquisition program
Run the data acquisition program (TD). Select the new rig you made (VME_HM) and set up the rig parameters for VME TDC/HM, followed by the mode parameters. Check the new RCDB parameters have appeared, i.e. check group logical names such as VME_TDC_AUX1 and HIS_SUFFIX now exist, and that DATA_FORMAT is correct ("TRIUMF MSR"). Check the host names in logicals CAMP_HOST and HM_RPC_HOST are pointing to the correct MVME modules. Then attempt a test run.If necessary, @dsk1:[musr.manager]install_mvme_hm.com can be run again to remove the VME HM/TDC support. It should not be necessary to revert to the old version of mud_util.
Then check that dsk1:[musr.manager]modas_startup.com is working (execute it), then try rebooting the target VAX. Check that the data acquisition system has come up correctly and is still working properly. Backup the system.
Special note for M01DAC
M01DAC is the spare for M15DAC, M20DAC and M9BDAC. It has a special version of ed:[com]get_rno.com to select one of M15, M20, M9B (or DEV) beamlines.suz@triumf.ca, TRIUMF Data Acquisition Group
