mctest

mctest (mxtest in McXtrace) is a tool to compile and run a set of instruments, e.g. all instruments in the examples folder.

usage: mctest.py [-h] [--ncount NCOUNT] [-n N] [--mpi MPI] [--openacc] [--config [CONFIG]] [--instr [INSTR]] [--comp COMP] [--mccoderoot [MCCODEROOT]] [--testroot [TESTROOT]]
                 [--testdir [TESTDIR]] [--limit LIMIT] [--verbose] [--skipnontest] [--suffix SUFFIX] [--nexus] [--lint] [--permissive] [--local LOCAL]

options:
  -h, --help            show this help message and exit
  --ncount NCOUNT       ncount sent to mcrun
  -n N                  ncount sent to mcrun
  --mpi MPI             mpi nodecount sent to mcrun
  --openacc             openacc flag sent to mcrun
  --config [CONFIG]     test this specific config only - label name or absolute path
  --instr [INSTR]       test only intruments matching this filter (py regex). Comma-separated list allowed for multiple filters.
  --comp COMP           test only intruments utilising COMP. Useful for testing the instrument suite after component changes.
  --mccoderoot [MCCODEROOT]
                        manually select root search folder for mccode installations
  --testroot [TESTROOT]
                        output test results in a datetime folder in this root
  --testdir [TESTDIR]   output test results directly in this dir (overrides testroot)
  --limit LIMIT         test only the first [LIMIT] instrs
  --verbose             output a test/notest instrument status header before each test
  --skipnontest         Skip compilation of instruments without a test
  --suffix SUFFIX       Add suffix to test directory name, e.g. 3.x-dev_suffix
  --nexus               Compile for / use NeXus output format everywhere
  --lint                Just run the c-linter
  --permissive          Use zero return-value even if some tests fail. Useful for full test con systems that are only partially functional.
  --local LOCAL         Instruments to test are NOT picked up from MCCODE installation, instead from --local=DIR

Via the mcviewtest tool, the test output can be visualised in the form of a html table McStas nightlies page. The input for such a table is a directory containing folders produced by mctest, e.g.:

mcstas-3.99.99_lint_1e6_Linux
mcstas-3.99.99_openacc_1e7_Linux
mcstas-3.99.99_openacc_5e7_Linux

A set of useful example one-liners for mctest are listed below. Output folders will be named according to the mccode version, statistics and options selected for mctest

• mctest --testdir=. will test all instruments from your current installation, placing output in a folder below the current folder.
• mctest --testdir=. -n1e7 does the same but with a higher-than-default statistic of 1e7
• mctest --testdir=. -n1e7 --mpi=2 does the same but runs with 2 mpi tasks
• mctest --testdir=. -n1e7 --mpi=2 --suffix=run2 does the same but adds a suffix to the output foldername
• mctest --testdir=. -n1e7 --mpi=2 --instr=PSI runs only instruments matching PSI in the filename
• mctest --testdir=. -n1e7 --mpi=2 --local=folder runs the instruments located within the directory folder
• mctest --testdir=. -n1e7 --mpi=2 --local=folder --lint runs the cppcheck linter on the instruments located within the directory folder

At the end of a completed test, a summary will be printed with overall status - all good:

Running tests / getting status...
BNL_H8         :   Display OK (4s)
BNL_H8         :   5.12    [val: 9.54331e-10 / 9.84026e-10 = 97 %]
BNL_H8_simple  :   Display OK (2s)
BNL_H8_simple  :   2.73    [val: 9.6339e-10 / 9.84026e-10 = 98 %]
======================================
Overall test result:
SUCCESS

In the event of errors this is indicated in the summary in this style:

Running tests / getting status...
BNL_H8         :   NO COMPILE
BNL_H8_CRASH   :   Display FAILED (2s)
BNL_H8_CRASH   :   3.41    [val: -1 / 9.84026e-10 = -101623331091 %] + !! RUNTIME FAILURE - see run_stdout_1.txt !!  <--- BIG DISCREPANCY??
BNL_H8_simple  :   Display OK (2s)
BNL_H8_simple  :   1.50    [val: 0.0 / 9.84026e-10 = 0 %] <--- BIG DISCREPANCY??
======================================
Overall test result:
FAILED! One or more tests errored (1 compile errs / 1 runtime errs / 2 values off)

(BNL_H8 had a coding error introduced, BNL_H8_CRASH had an exit(-1) in initialise introduced and BNL_H8_simple runs but produces data that are numerically off i.e.=0)