IT Tips

Code of Conduct

Please contact Chris Johnson or Chris Page with any questions or suggestions 

Maths Compute Servers

• To make access to the machines as simple as possible, no job scheduling system is used. However, this does mean that it is incumbent on users to make sure they are not using the facility in a way that causes problems for others.

• Most importantly, be careful that you do NOT run CPU-intensive jobs on the login server vummath (130.88.16.53). You must always log into one of the compute nodes csN (see table above) before running programs.

• Before running jobs on a compute node, use the $ top, uptime or pcinfo commands to check how many other jobs are already running. The number(s) labelled "load average" give an indication of how many CPUs are being used. If possible, avoid running jobs on machines where this number is already greater or equal to the number of CPUs (see table above).

• If you are running long jobs (taking more than a few hours in total), you must use nice command to lower the priority of your jobs. Instead of running $ yourcode, run $ nice yourcode. If the machine is not already full of jobs this makes no difference to the speed of your code, but it allows others to use the machine effectively for short jobs or interactive programs while your code is running.

• Machines cs1 and cs11 are high memory nodes (768GiB and 1280GiB RAM respectively), and cs7-cs11 are the only ones with the commercial MAGMA software installed. If you don't need these features, and if there is space available elsewhere, you should prioritise running jobs on the other machines: cs2-cs6 and cs12.

• Particularly if you are a heavy user of this system (using multiple CPUs for several days), please consider if your programs are appropriately optimised. However, many computing jobs are simply too big for the departmental servers, and the university provides larger-scale computing facilities for just such tasks. These include Condor (currently free to access, for multiple small jobs) and the CSF (some free access, for larger or highly parallel jobs), each of which is 50-100 times the size of the Departmental computing cluster. Please see UoM High Performance Computing Facilities for more details or email a question to its-ri-team@manchester.ac.uk.

Connecting to the UoM & Maths Network Drives

Important: before you start, please make sure you have installed and are connected to GlobalProtect (see general guidelines).

Mounting remote file systems from Linux / MacOS 

If not already available,  install sshfs (e.g.,  $ apt install sshfs  on Ubuntu; see the end note on how to install this on Mac OS)

• Maths Linux Home

$ sshfs USERNAME@vummath.ma.man.ac.uk:/home/USERNAME/ ~/Shared/MathsLinuxHome/

$ fusermount -u ~/Shared/MathsLinuxHome/  # unmount

• RDS Research IT Storage (an extra 2FA step is needed on-campus without active GlobalProtect VPN)

$ sshfs USERNAME@rds-ssh.itservices.manchester.ac.uk:/mnt/eps01-rds/YOUR_RDS_SPACE/ ~/Shared/RDS/

$ fusermount -u ~/Shared/RDS/

• P Drive  (see also P-drive web-access and notes below for connecting from a file browser)

$ sudo mount -t cifs -o user=USERNAME,domain=ds.man.ac.uk,sec=ntlmsspi,uid=`id -u`,gid=`id -g` //nask.man.ac.uk/home$ ~/Shared/PDrive/

$ sudo umount ~/Shared/PDrive/

• MS OneDrive via rclone  (first, run  $ rclone config  - see setup tips)

$ rclone --vfs-cache-mode writes mount onedrive: ~/Shared/OneDrive  # assuming remote connection name 'onedrive'

$ sudo umount ~/Shared/OneDrive/

Alternatively, download and use onedrive synchronisation tool (see Blackboard discussion space): 

$ ./onedrive --monitor --resync --sync-shared-folders

• Google Drive via gdfuse

$ google-drive-ocamlfuse ~/Shared/GDrive/

$ fusermount -u ~/Shared/GDrive/RDS-SSH Research IT storage

• Dropbox via dbxfs  (alternatively, use the vendor-supplied synchronisation tool)

$ dbxfs ~/Shared/Dropbox/

$ fusermount -u ~/Shared/Dropbox/

For uploading a large file (> 10 GB) or multiple files, use dropbox_uploader script [somewhat obsolete: rclone is recommended instead; see below]

$ ./dropbox_uploader -s -p upload /LOCAL_FOLDER /REMOTE_FOLDER

Copy/synchronise a secure copy of local files to Dropbox with rclone

If you do not have UoM Business Dropbox account, please first complete the request form (see also the guidelines).

1. Download and install rclone: direct links for Windows, Linux & MacOS (see installation guide).

2. Setup Dropbox access by running the following in a terminal/command shell, following the instructions (select defaults) and then approving rclone access to the UoM Dropbox (note: all your configuration details are stored locally; see more details on rclone.org/dropbox)
[Linux/MacOS] $ rclone config
[Win] > rclone.exe config

3. Set up Dropbox encryption (assuming your UoM Dropbox storage is called remote:)
(i) Select/create a dedicated folder on your Dropbox, say, remote:/BACKUP
(ii) run rclone config  and select crypt (encrypt other remotes), following the instructions and pointing it to remote:/BACKUP
  (iii) test the secure remote (assuming you called it secure:)
[Linux/MacOS] $ rclone ls secure:
[Win] > rclone.exe ls secure:

4. Copy local files to your secure Dropbox space (remove --dry-run for actual transactions)
[Linux/MacOS] $ rclone copy --dry-run --tpslimit 12 -PL /local/path/ secure:
[Win] > rclone.exe copy --dry-run --tpslimit 12 -P C:\...\local\path\ secure:

NOTE 1: There is a slightly less well-tested GUI mode that may be more intuitive (please use with caution):
$ rclone rcd --rc-web-gui
NOTE 2: If you want to keep the remote copy identical to your local copy, you could use the sync option, but please always use --dry-run first!
[Linux/MacOS] $ rclone sync --dry-run --tpslimit 12 -PL /local/path/ secure:
[Win] > rclone.exe sync --dry-run --tpslimit 12 -P C:\...\local\path\ secure:

5. * Mounting (connecting) the secure Dropbox space as a local drive/folder, without using extra storage space on your local machine.
(i) Unless you are running Linux, you first need to install WinFsp for Windows and FUSE-T for MacOS.
(ii) To mount (connect) the remote secure storage, use the following in the terminal/command shell:
[Linux/MacOS] $ rclone mount secure: /local/path-to-mount --vfs-cache-mode writes
[Win] > rclone.exe mount secure: X: --network-mode --vfs-cache-mode writes
(iii) To unmount (disconnect), just press [Ctrl]+[C] (Win/Linux) or [Cmd]+[.] (MacOS).
Please see rclone mount for more details and caveats, in particular, for Windows and MacOS users.

Accessing Network Drives from a File Browser 

(Based on the how-to instructions prepared by Aaron Russell)

1.   [Linux]  Open "Nautilus" file manager and click "Connect to server".

[Mac OS]  Open "Finder" and click "Go" -> "Connect to Server".

[Windows]  Launch file manager and right-click on "This PC" -> "Map Network Drive".

2. In the "Server address" box enter the following and click "Connect":

•  P drive

[Linux/Mac]  smb://nask.man.ac.uk/home$  or  smb://10.2.81.12/home$/

[Windows]  \\nask.man.ac.uk\home$

• Maths Linux home

[Linux]  sftp://vummath.ma.man.ac.uk/home/<Username>   (replace <Username> with your UoM user name)

• RDS storage space

[Linux/Mac]  smb://nasr.man.ac.uk/epsrss$/snapped/replicated/<Folder>   or  smb://10.2.82.6/epsrss$/snapped/replicated/<Folder>

[Windows]  \\nasr.man.ac.uk\epsrss$\snapped\replicated\<Folder>   (where <Folder> is your group's research share on the RDS)

Alternative method: access Maths Linux Home via sshfs from Mac OS

1.  Install osxfuse and sshfs, e.g. via a homebrew package-manager on Mac OS

$ brew cask install osxfuse

$ brew install sshfs

2.  Mount & unmount the remote drive

$ sshfs -o reconnect -o defer_permissions -o noappledouble -o volname=MathsLinuxHome <Username>@vummath.ma.man.ac.uk:/home/<Username> ~/Desktop/MathsLinuxHome/

$ fusermount -u ~/Desktop/MathsLinuxHome/  # unmount

Complied by Igor Chernyavsky with contributions by Chris Johnson, Paul Johnson and Aaron Russell