Post date: Nov 17, 2014 3:30:39 PM
A bit more than half of the jobs with the modal genotypes finished, but none of those with the mean genotype finished. This was caused by I/O bottlenecks. I am now re-running all of the mean genotype runs but writing to /local/scratch/. Here is an example:
cd /local/scratch/
sleep 140
pimass-lin -g /home/A01963476/projects/timema_wgexperiment/gemma/fix_geno_timemaC.txt -p /home/A01963476/projects/timema_wgexperiment/gemma/pheno_samplesC.txt -w 5000000 -s 10000000 -num 1000 -o pimass_timemaCrep28 -cc
rsync -avz output/pimass_timemaCrep28.[a-z]* /home/A01963476/projects/timema_wgexperiment/gemma//output_pimass/
Regarding the jobs that finished, there are still mixing problems and even 30 chains probably isn't enough. For examples, correlations between PIPs for different loci are relatively low for individual chains (0.001 to 0.01, 0.01-0.02 for betas), but higher for groups of chains (average across sets of 8) = 0.35 (beta = . So, I do think there is information here, I want to run another 50 chains of each. I need to avoid having these clobber each other in terms of I/O. So, I am running 50 additional chains for each data set, now with a thinning interval of 5000:
cd /local/scratch/
sleep 245
pimass-lin -g /home/A01963476/projects/timema_wgexperiment/gemma/fix_geno_timemaC.txt -p /home/A01963476/projects/timema_wgexperiment/gemma/pheno_samplesC.txt -w 5000000 -s 10000000 -num 5000 -o pimass_timemaCrep49 -cc
rsync -avz output/pimass_timemaCrep49.[a-z]* /home/A01963476/projects/timema_wgexperiment/gemma//output_pimassX/