The current Skytree official release version is 11.6
1. Register for an account at the Canadian Astronomy Data Centre: http://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/auth/register.html .
2. Request a CANFAR account via request to CanfarHelp at nrc-cnrc.gc.ca . Please include a few sentences describing what you wish to utilize the system for.
3. Await manual setting up of CANFAR account by CANFAR sysadmin using CADC username.
4. Request SkytreeServer<version>.tgz and skytree-client.lic: nick.ball at nrc-cnrc.gc.ca .
5. Copy files to CANFAR:
scp -p SkytreeServer<version>.tgz skytree-client.lic <username>@canfar.dao.nrc.ca:~
6. ssh to CANFAR:
ssh <username>@canfar.dao.nrc.ca
7. Run CANFAR setup, answering yes to creating a .netrc. This creates a public ssh key required for creating a virtual machine:
canfarsetup
8. Create virtual machine (VM) for Skytree (supply desired name for VM):
vmcreate <vmname>
9. Await email confirming VM creation.
10. Copy files to VM (VM IP may be obtained from VM creation email, or vmlist):
scp -p SkytreeServer<version>.tgz skytree-client.lic <vm IP address>:~
11. ssh to the VM as root:
ssh root@<vm IP address>
12. Untar the Skytree Server package to /opt/skytree, and move the license into it, and copy over all contents to /opt/skytree (recommended location for install):
cd /home/<username>
mkdir /opt/skytree
tar -xzf SkytreeServer<version>.tgz --strip 1 -C /opt/skytree
mv skytree-client.lic /opt/skytree
13. Create a group for Skytree Server users:
groupadd skytree-user
14. Setup the appropriate privileges for the installed files:
chown root /opt/skytree
chgrp skytree-user /opt/skytree
chmod 750 /opt/skytree
cd /opt/skytree
chown -R root *
chgrp -R skytree-user *
chmod -R 740 *
chmod 750 skytree-server
chmod 750 dataprep
chmod 750 dataprep/*.sh
chmod 750 datasets
chmod 750 docs
15. Create symlinks to the installed executable files for direct access from shell for all users:
ln -s /opt/skytree/dataprep/* /usr/local/bin
ln -s /opt/skytree/skytree-server /usr/local/bin
16. Users need to be added to the skytree-user group:
usermod -g skytree-user <username>
17. Exit the VM:
exit
18. ssh as oneself to the VM:
vmssh <vmname>
19. Setup user environment variables.
For BASH users, add the following lines to /home/<username>/.bashrc:
export SKYTREE_LICENSE_PATH=/opt/skytree/skytree-client.lic
export CLASSPATH=$CLASSPATH:/opt/skytree/dataprep/skytree.jar
For TCSH users, add the following lines to /home/<username>/.cshrc:
setenv SKYTREE_LICENSE_PATH /opt/skytree/skytree-client.lic
setenv CLASSPATH $CLASSPATH:/opt/skytree/dataprep/skytree.jar
20. From any directory, test the installation by running the following commands:
skytree-server
generate-header.sh
Both the above commands should give options available to the user.
To see what algorithms are available, e.g., nn, type skytree-server --help. For help on each algorithm, type skytree-server <algorithm_name> --help.
VOSpace Storage System
To access CANFAR's data storage, VOSpace, either go to the CANFAR dashboard, for interactive access, or see the VOSpace page on the wiki.
The following steps (as on the wiki) work for setting up the Python client on the VM, which allows VOSpace to be viewed as a mounted filesystem:
sudo yum install fuse
sudo yum install fuse-devel
sudo /usr/sbin/usermod -a -G fuse `whoami`
Log out, log in again
sudo yum install python26
sudo yum install python26-distribute
sudo easy_install-2.6 vos
getCert
mountvofs
Now commands like
vls vos:<username>
work, and can be used in scripts (e.g., vcp).
Batch Mode
Follow instructions for processing as with any other software on CANFAR: https://wiki.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/canfar/index.php/Introduction .
Skytree can be called from scripts in the same way as it is run interactively.
Assuming you have VOSpace set up as described above, to run an example batch job, do the following:
1. Download these 4 files: run_skytree_example.sh, skytree_example.condor, skytree_example.sh, sdss100kx4.txt .
2. Place run_skytree_example.sh and skytree_example.condor on the CANFAR head node, and skytree_example.sh on your VM, each one in your home directory.
3. Place sdss100kx4.txt in your home directory on VOSpace via "Manage Storage" on the CANFAR dashboard (left-hand navigation bar -> add files).
4. Edit the two .shs and the .condor to change <username>, <vmname>, and <user email> to your own as appropriate.
5. Make sure each .sh is executable (chmod 755, or similar).
6. Shut down your VM (vmstop <vmname>, from the head node, and wait for the email).
7. Make sure your VOSpace and VM are public:
Go to your VOSpace home directory on the dashboard as in step 3
Click the two dots at the top
Click the icon next to your username
Edit permissions to public
Do the same for the .img.gz of your VM in the top level of your VOSpace
8. From the head node, submit the job:
condor_submit skytree_example.condor
It should output the files distances.out and indices.out to your VOSpace, and a logfile, stdout and stderr to the files indicated in the .condor. You can monitor the job with condor_q <username>.