Architecture

Cesari is the bigger in-house cluster of the G-CSC.
It consists of 26 nodes with 2 10-core processors each, resulting in 20 cores/node and 520 cores total.
Each Compute node has 32 GB RAM, which means when running one mpi threads per core (=20 per node), you get 1.6 GB RAM per thread.
You can also use hyperthreading (=2 mpi threads per core), resulting in 40 threads/node an 0.8 GB RAM per thread.
The log-in node is configured like a normal node (2x 10 cores), but has 64 GB of RAM.
Job scheduling is done via ugsubmit - Job Scheduling on Clusters (auto-detect)

Module System

module avail: displays all modules available.
module list: displays all modules currently loaded.
module load MODULENAME: Loads the module MODULENAME
module unload MODULENAME: Unloads the module MODULENAME

Note: Please note that you have to use that module load each time you log in, so it makes sense to put them into your .bashrc.

Compiling ug4

GCC

First you have to select a MPI version to run jobs in parallel:

    module load openmpi-gcc

then you can use the normal cmake ...

LLVM

LLVM/clang (recommended):

    module load llvm
    module load openmpi-clang

To compile with the clang-compiler, use

  cmake -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang ..

Running parallel Jobs

To start Jobs on Cesari, use ugsubmit / uginfo / ugcancel .
ugsubmit - Job Scheduling on Clusters .
ugsubmit supports cesari without any configuration (auto-detect). By default it uses 20 threads/node (no hyperthreading). You can override that behaviour with the parameter -nppn.

Debugging

Debugging : DDT can be used via module load ddt