Matlab
Table Of Contents:
MATLAB
MATLAB (MATrix LABoratary)[1] is a commercial software package for numerical computing that allows matrix manipulation, function and data plotting, implementation of algorithm and much more. MATLAB is installed for both serial (single-processor) and distributed (multi-processors) use.
Important Notes
MATLAB may have slight changes in different versions. If you have issues running the sample examples, please refer to MATLAB site [1]. GPU support by release is available here (e.g. R2020b is not supported on Ampere).
Refer to Memory Management and License Management sections in this document regarding memory and license information. Your job may prematurely exit if the requested memory is not enough. You will encounter license error if there are not enough licences to run your job.
If a MATLAB script involves multiple functions and a function compiled as a mex file in non HPC environment (e.g your PC), you need to compile the mex file in the cluster first.
Create separate directories for different versions of MATLAB e.g. R2014b, R2015b
If possible, use MATLAB C Compiler (MCC) capability as described in MCC section below so as to run MATLAB job without checking out MATLAB licenses.
Please use minimum workers to run your job considering the available MDCS licenses. Start with 4 workers. Each MDCS worker is going to checkout 1 MDCS license.
In the slurm profile (e.g. SLURMProfile2015b.settings), please include the appropriate slurm arguments (e.g. -p gpufermi --gres=gpu:2) inside the quotes for both Independent and Communicating functions. It has been left blank for the default resources.
If you need to use commands in startup, create a startup.m file in the userpath folder, which is on the MATLAB search path.
Installed Versions
All the available versions of MATLAB for use can be viewed by issuing the following command. This applies for other applications as well.
module avail matlab
output:
---------------------- /usr/local/share/modulefiles -------------------------
matlab/R2016b (D) matlab/R2017b matlab/R2018b matlab/R2019b matlab/R2020b
The default version is identified by "(default)" behind the module name and can be loaded as:
module load matlab
The other versions of MATLAB can be loaded as:
module load matlab/<version>
Interactive Job Submission
Request a compute node (--x11 for GUI).
srun --pty --mem=4gb /bin/bash
(Note: For better GUI experience, find the details at HPC Visual Access)
Load matlab module
module load matlab
Run MATLAB. Add -nodisplay flag if you do not want GUI. Use -singleCompThread flag if you are running a serial/distributed job.
matlab -singleCompThread -nodisplay
Check the version info and toolboxes availalble for that version.
>> ver
output:
----------------------------------------------------------------------------------------------------
MATLAB Version: 8.6.0.267246 (R2015b)
...
Operating System: Linux 2.6.32-642.3.1.el6.x86_64 #1 SMP Sun Jun 26 18:16:44 EDT 2016 x86_64
Java Version: Java 1.7.0_60-b19 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
---------------------------------------------------------------------------------------------------
MATLAB Version 8.6 (R2015b)
Simulink Version 8.6 (R2015b)
MATLAB Coder Version 3.0 (R2015b)
MATLAB Compiler Version 6.1 (R2015b)
MATLAB Distributed Computing Server Version 6.7 (R2015b)
Parallel Computing Toolbox Version 6.7 (R2015b)
...
Debugging
It is the process of isolating and fixing run time errors:
Algorithmic in nature
Apparent when getting unexpected output
Difficult to track down
Debugger required
Basic Debugging Steps:
Click in the breakpoint alley at an executable line where you want to set the breakpoint
command line: dstop in file.m at #n
Run the matlab script; the control stops at the first breakpoint
Prompt in the command line changes to K>>
Use Debug Menu or command line for running the following debug commands:
dbstep: execute the current line of the file
dbcont: resume execution of file until the next breakpoint
dbquit: Exit Debug mode
Debug called function?
dbstep in dbstep out
Debugging can be implemented through graphical user interface, as well as by using debugging functions from the MATLAB Command Window. Visit MATLAB website for details.
Batch Job Submission
Serial Jobs
Copy the sample MATLAB script "fixedmontecarlo.m" and SLURM script "calPi.slurm (included below)" to your home directory. The sample and auxiliary files for all applications are located at /usr/local/doc/<APPLICATION>; for MATLAB, it is /usr/local/doc/MATLAB
cp /usr/local/doc/MATLAB/fixedmontecarlo.m ./
MATLAB script "fixedmontecarlo.m" calculates the value of PI using Monte Carlo method. The SLURM script "calPi.slurm" includes a request for one processor (-n 1) and 2 hours of wall time. Note that the script includes the command to load the matlab module and specifies the "-nodisplay" option. The script specifies that the actual MATLAB program (or "M-file") to be run (-r) is contained in the file "fixedmontecarlo.m".
calPi.slurm:
#!/bin/bash
#SBATCH -o test.o%j
#SBATCH --time=02:00:00
#SBATCH -N 1 -n 1 --mem=5gb
#Copy to the scratch directory and change directory
cp -r fixedmontecarlo.m $PFSDIR
cd $PFSDIR
#Load MATLAB module
module load matlab
#RUN MATLAB script
matlab -singleCompThread -nodisplay -r fixedmontecarlo
quit
echo Master process running on $HOSTNAME
echo Directory is `pwd`
NPROCS=$(( $SLURM_NPROCS*$SLURM_NNODES))
echo This job is allocated $NPROCS nodes
echo `date`
#Copy Back all the output files to the present working directory
cp -ru * $SLURM_SUBMIT_DIR
Submit the Job:
sbatch calPi.slurm
You should find the output file test.o<jobid> with the value of PI. Please view the file for output and possible errors.
Input Parameters or Arguments:
If you have the matlab function with input parameters, you can use the following template where the lower and upper bound are defined as function arguments to find the prime numbers in that range.
matlab -singleCompThread -nodisplay -r 'primeNumbers(1,131072)'
Memory Management
Memory should also be used as a resource. Though you are running a serial job, depending on your memory requirement, you need to request the processors appropriately. Memory per proc can be calculated as Available Memory / Processors for type of nodes indicated at Server & Storage Estimate your memory using valgrid utility described at Valgrind Utility or MATLAB Meory Profiling (MATLAB Website). For memory test, run your job reserving the whole node so that it won't affect other's jobs. Check the memory usage using top command.
Test: Copy MATLAB script file solveEq.m from /usr/local/doc/MATLAB and run it with with two different values of dim (dim = 3*10^5 and dim=5*10^5). If you submit the job and check the memory using "top" command, you can find the surge of memory from ~14gb to ~40gb by just replacing 3 by 5.
License Management
Check MATLAB licenses by issuing the commands below:
/usr/local/matlab/R2019b/etc/glnxa64/lmutil lmstat -a -c 27000@swc-licenses-1.case.edu
Looking for Distributed (Parallel) Computing Licenses
/usr/local/matlab/R2019b/etc/glnxa64/lmutil lmstat -a -c 27000@swc-licenses-1.case.edu | grep Distr
output:
Users of Distrib_Computing_Toolbox: (Total of 10000 licenses issued; Total of 13 licenses in use)
You can check the number of licenses you have checked out using "| grep <caseID>" at the end of the above commands.
/usr/local/matlab/R2019b/etc/glnxa64/lmutil lmstat -a -c 27000@swc-licenses-1.case.edu | grep <caseID>
output:
<caseID> comp142 …
<caseID> comp137 …
You can use MATLAB C Compiler (MCC) feature not to checkout other licenses as described in MCC Section below.
MATLAB C Compiler (MCC) - R2015b onward)
Using MCC, you can make an executable of your MATLAB script and run as many instances of it as you want. Regarding MDCS jobs, though you will be checking out MDCS licenses, the PCT and other campus-wide licenses are not checked out. So, it is applicable to MDCS jobs as well.
Compiling:
module load matlab
mcc -m <matlab_script>
(Note: For non-GUI jobs, use -R -nodisplay and for serial job, use -R -singleCompThread)
The compiler produces two outputs <matlab_script> executable and run_<matlab_script>.sh shell script. You can include the executable in the SLURM script or run it interactively.
There are two licenses for the matlab compiler (mcc), and managing the license is important to allow everyone who needs the license to use it. Please think carefully before using mcc within a matlab session -- that will check-out the license to your use for the duration of the matlab session. The alternative is to compile with mcc from the command line -- either linux or in a dos shell. Read for more details.
If you are using number as an input to the function, in the MATLAB script, you need to include the str2num function as showed:
<variable_name>=str2num(<input>);
Serial Sample Job: Copy MATLAB script "plot_function.m" and job script "mcc_plotFunction.slurm" from /usr/local/doc/MATLAB and submit the MCC job. Compilation is performed through the script as well.
sbatch mcc_plotFunction.slurm
MDCS jobs: Copy MATLAB script "central_theorem.m" and job script "mcc_CenTh.slurm" from /usr/local/doc/MATLAB and submit MCC MDCS job.
sbatch mcc_CenTh.slurm
Open the output file:
evince-previewer plot_central.ps
Matlab Parallel Jobs
Please refer to this link for running Matlab Parallel jobs.
Running MATLAB remotely from PC
MATLAB configuration allows distributed jobs to be submitted from a remote machine that has MATLAB installed. Distributed jobs should find their way to the cluster and send the results back to the remote machine. Please see step-Step-by-step how-to on running Matlab jobs on the cluster from a remote machine.
MATLAB GPU
MATLAB has GPU support for NVIDIA-CUDA enabled GPU devices which let you accelerate your application. You can look for GPU enabled MATLAB functions in MATLAB website.
GPU Device Info
Request the GPU node
srun --x11 -p gpu -C gpuk40 --gres=gpu:1 -N 1 -c 2 --time=1:00:00 --mem=5gb --pty /bin/bash
Load MATLAB module
module load matlab
Open matlab and issue gpuDevice command
matlab -nodisplay
>>gpuDevice
output:
parallel.gpu.CUDADevice handle
Package: parallel.gpu
Properties:
Name: 'Tesla M2090'
Index: 1
ComputeCapability: '2.0'
SupportsDouble: 1
DriverVersion: 5
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [65535 65535]
SIMDWidth: 32
TotalMemory: 5.6366e+09
FreeMemory: 5.5386e+09
MultiprocessorCount: 16
ClockRateKHz: 1301000
ComputeMode: 'Exclusive process'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 0
CanMapHostMemory: 1
DeviceSupported: 1
DeviceSelected: 1
GPU Batch Job
Example: Copy MATLAB script file "myCal.m" and the job script file "cuda_mat.slurm" from /usr/local/doc/MATLAB and submit the job:
sbatch cuda_mat.slurm
Note the function Array with the keyword "gpu" in the front; Array function replaced by gpuArray. You may also want to refer to Jacket application installed in HPC for GPU MATLAB jobs.
For more on GPU Benchmarking, visit GPU Benchmark @ HPC.
MATLAB Profiling and Benchmarking
Profiling
Profiling is a way to measure where a program spends time. MATLAB parallel profiling enables to see how much time each worker spends on evaluating each function and how much time communicating and waiting for other workers.
Performance Improvement:
Identify the functions that are consuming the most time and try to optimize them
Decide whether the number of times the code calls a particular function is reasonable
Identify the functions within your code that might be calling other time-consuming functions that can be several layers down in the code
Serial (GUI):
profile on
<matlab_script> % Compare array_random.m & vector_gammarand.m
profile off
profile viewer
See three optimization effort in vector_gammarand.m file. Compare its performance with array_random.m. Copy both the files from /usr/local/doc/MATLAB.
nValues = 1e7;
% 1. Preallocation of 1 by 10^7 contiguous block of memory and initialization
firstArray = ones(1,nValues);
secondArray = 2*ones(1,nValues);
resultArray = zeros(1, nValues);
% 2. Using Vector Instead of Array
tic()
resultArray = firstArray .* exp(secondArray);
toc()
% 3. Using the function having better execution time
nValues = 1e4;
tic()
for i=1:nValues
gamrnd(2, 2); % instead of random('gamma', 2, 2);
end
toc()
Serial Profiling output for both array_rand.m and vector_gamrand.m.
Parallel (GUI):
pmode open
P>>mpiprofile on
P>> <matlab-par-script>
P>>mpiprofile off
P>>mpiprofile viewer
Parallel Profiling output:
Visit MATLAB website for details.
Benchmarking
Benchmark helps in testing the performance of MATLAB code. For basic benchmarking, in the MATLAB prompt, type:
bench
Refer to HPC Guide to GPU Benchmark and also look for MATLAB website [1] for other benchmarking tools including MDCS and GPU.
Result of Matlabpool (parpool) benchmark results for Black Jack (card game):
MATLAB Coder
Matlab Coder is now available for all CWRU user. Matlab coder is a very useful tool to convert your Matlab code to a C/C++ codes, either to convert it completely or to use part of the function to merge it with a different code snippet.
In the Matlab prompt, type coder and follow the instructions.
>>coder
More information about Matlab Coder on the Mathworks. The manual (preparation, acceleration, code generation) quick start guides for Matlab 2013b can be downloaded from the /usr/local/doc/MATLAB directory.
Install MATLAB Engine API for Python in your home directory
Reference at [4].
Setup environment variables
BUILDDIR=/home/<caseID>/<path>
MATLABROOT=/usr/local/matlab/<matlab-version>
export INSTALLDIR=`python -m site --user-site`
Build the package
cd $MATLABROOT/extern/engines/python
python setup.py build --build-base=$BUILDDIR install --user
Add the user directory to the python search path
python
>>> import sys, os
>>> sys.path.append(os.getenv("INSTALLDIR"))
Test: Import the matlab engine
>>> import matlab.engine
Troubleshooting MATLAB Issues
1. Validation using SLURM profile did not go through
Solution: move your matlab directory at /home/<CaseID> to something else like matlab-test
mv ~/matlab ~/matlab-test
Troubleshooting
Issue: Encounter the following error when trying to execute the compiled file using MCC:
./tempt: symbol lookup error: /usr/local/matlab/R2020b/sys/os/glnxa64/libstdc++.so.6: undefined symbol: __cxa_thread_atexit_impl
Solution:
HPC system libraries can be older than what matlab expects, and so they bundle newer libraries with their releases to be preloaded. Set the environment variable:
export LD_PRELOAD="<matlabroot>/bin/glnxa64/glibc-2.17_shim.so"
There is a good (brief) discussion in this thread.
https://itectec.com/matlab/matlab-error-running-matlab-parallel-server-toolbox-admincenter/
Issue: quit/exit throwing - "Error using exit Unrecognized function or variable 'Settings'".
Solution: To workaround this issue, you can ensure the preferences are properly initialized by calling "settings" in your MATLAB code.
>> settings
References
[1] MATLAB Home: http://www.mathworks.com/
[2] Users Guide Site
[3] Coder - http://www.mathworks.com/products/matlab-coder/
[4] MATLAB Engine API for Python: https://www.mathworks.com/help/matlab/matlab_external/install-matlab-engine-api-for-python-in-nondefault-locations.html