Submitting with Ganga

Instructions from Veronika (Feb, 2016):

1. Setup

  • Update the Erasmus package in Santiago to get the latest version of the files

  • You will probably also need Bender to make sure you are always using the correct version when testing

  • First thing you need is a GRID certificate. Once it is imported and everything, initiate a proxy with lhcb-proxy-init

  • Next setup the environment. You need Ganga and Erasmus to run the jobs. Do: SetupErasmus and then SetupGanga.

OBSOLETE (for Ganga 6.0) Do: SetupErasmus v10r3 and then SetupGanga v600r44 -c x86_64-slc6-gcc48-opt. Use the commands exactly as they are here, as there are some compatibility issues with the versions and at the moment you need these settings.

2. Running the jobs

  • The jobs can be submitted with a script with a very creative name called submit_to_ganga_v601r13.py, which is in the latest version of the Ks2Pi0MuMuTuples Erasmus package. Make sure that the datacards are also there (four files in /Erasmus_HEAD/Phys/Ks2Pi0MuMuTuples/datacards starting with "LHCb_Collision")

  • The script at the moment can run data from 2011 and 2012, magnet up and magnet down. So in total 4 possible jobs. Best is to run them separately. All you need to change is the line send_job(SIMULATION,"YEAR","MAGNET_CONFIGURATION").

  • Once you have chosen which job you want to run and modified the script, open a new screen with screen.

  • In the new screen, open Ganga with the command ganga. Then you will see the Ganga interface. You can use it as an interactive python.

  • Write ganga submit_to_ganga.py and then it will start submitting the jobs. This is a slow procedure, so you might have to go home, while it is still submitting. For the process to continue, detach the screen with Ctrl+Alt+A, D.

  • You can later reopen it with screen -r.

  • Don't run two sessions of Ganga in parallel because it will cause problems.

  • Later you can check how the jobs are progressing with the command in ganga jobs.

  • The files will be saved in ~/gangadir/workspace/USERNAME/LocalXML/NUMBER_OF_JOB. There you also have an output of the jobs.

  • With jobs(NUMBER_OF_JOB) you can see all the subjobs and their status - new, submitted, running, completing, completed or failed.

  • Sometimes jobs will be failing. If it is just a couple, it is probably an issue of the grid. If it is many, probably it is your script, so you will need to check what is going on in the file of the subjobs in the file ~/gangadir/workspace/USERNAME/LocalXML/NUMBER_OF_JOB/NUMBER_OF_SUBJOB/output/stdout

  • To resubmit just jobs that are failing, use what is written in the script ganga_resubmit_failed.py.

  • Once they are all completed, you can add all of the output root files, which are in the output dir, into one file with hadd. Best is to name it after the year and the magnet and put data somewhere, so that we know.

3.Issues

  • Ask Carlos. Carlos will know best.