Post date: Dec 14, 2014 5:5:35 PM
#12Dec14 jobs were done by morning of 14Dec14. In /stygoparnus/ there's one abc folder for each of the 50 batches of 1000 simulations. In each folder is the file with the 1000 simulations and a copy of ms. Will have to extract these files out of their folders before concatenating simulations.
#I can now connect to the USU cluster with the VPN. Had to download Cisco AnyConnect Secure Mobility Client onto my computer (for instructions, see https://usu.service-now.com/kb_view.do?sysparm_article=KB0011480).
#In wrap_qsub_rc_abc.pl, changed:
my $nsim = 2000
my $nrep = 100
#Could do 150 or 200 next.
#Runs for 96 hours, so can run as many sims as will fit in 96 hours.
#14Dec14: job #s 168593-168692
#All but two of the 14Dec14 jobs finished the morning of 17Dec14. I started another 100 jobs (2000 sims each) on 17Dec14: #172329-172428.
#17Dec14. Stygoparnus ABC diagnostics one more time now that I have more simulations to make sure correlation patterns are staying the same:
#To pull out abc sim file from each abc directory (one directory gets made per run) and concatenate them and add one header to the top of the file, no perl script is necessary. Instead:
tail -q -n +2 abcdir*/abc* > abccat.txt
wc -l abccat.txt
#246000
#head -n 1 abcdir55qPx1aeVeiJ/abcout_sp_z5jhlCzB84 > hdr.txt #now have a file with just the header
cat hdr.txt abccat.txt > abccathead.txt
less abccathead.txt #check it to make sure it looks right!
#Results: The patterns were just about the same as with 500 sims. We lost some correlation between parameter migrate and summary stats, but looks similar to the amount of correlation seen in the Eurycea dataset (both Eurycea and Stygoparnus datasets have few populations -- 3 vs. 4). Proceed with the rest of the simulations!
#19Dec14. All but two of the 100 runs I started 17Dec14 finished. So, I started another 100 runs (2000 sims each).
#21Dec14. A little over half of the 100 runs I started 19Dec14 finished (it's been about 54 hours). I started another 100 runs today (2000 sims each). Job #s: 175013-175112
#23Dec14. About half of the 21Dec14 runs finished (it's been 46+ hours). Started another 150 runs, 2000 sims each (300,000 sims total). Once these are finished, I'll have 1,150,000 sims total and I'll be done! Job #s 175570 - 175719
#26Dec14. 43 of the 21Dec14 jobs stalled and didn't finish within the 96 hours allotted, so I started 50 last jobs (2000) sims each. If they finish successfully, I'll be done. Job #s: 176294-176343.