created by Geraldine_VdAuwera
on 2012-08-09
Objective
Test that Queue is correctly installed, and that the supporting tools like Java are in your path.
Prerequisites
Steps
The command we're going to run is a very simple command that asks Queue to print out a list of available command-line arguments and options. It is so simple that it will ALWAYS work if your Queue package is installed correctly.
Note that this command is also helpful when you're trying to remember something like the right spelling or short name for an argument and for whatever reason you don't have access to the web-based documentation.
Action
Type the following command:
java -jar <path to Queue.jar> --help
replacing the <path to Queue.jar> bit with the path you have set up in your command-line environment.
Expected Result
You should see usage output similar to the following:
usage: java -jar Queue.jar -S <script> [-jobPrefix <job_name_prefix>] [-jobQueue <job_queue>] [-jobProject <job_project>] [-jobSGDir <job_scatter_gather_directory>] [-memLimit <default_memory_limit>] [-runDir <run_directory>] [-tempDir <temp_directory>] [-emailHost <emailSmtpHost>] [-emailPort <emailSmtpPort>] [-emailTLS] [-emailSSL] [-emailUser <emailUsername>] [-emailPass <emailPassword>] [-emailPassFile <emailPasswordFile>] [-bsub] [-run] [-dot <dot_graph>] [-expandedDot <expanded_dot_graph>] [-startFromScratch] [-status] [-statusFrom <status_email_from>] [-statusTo <status_email_to>] [-keepIntermediates] [-retry <retry_failed>] [-l <logging_level>] [-log <log_to_file>] [-quiet] [-debug] [-h] -S,--script <script> QScript scala file -jobPrefix,--job_name_prefix <job_name_prefix> Default name prefix for compute farm jobs. -jobQueue,--job_queue <job_queue> Default queue for compute farm jobs. -jobProject,--job_project <job_project> Default project for compute farm jobs. -jobSGDir,--job_scatter_gather_directory <job_scatter_gather_directory> Default directory to place scatter gather output for compute farm jobs. -memLimit,--default_memory_limit <default_memory_limit> Default memory limit for jobs, in gigabytes. -runDir,--run_directory <run_directory> Root directory to run functions from. -tempDir,--temp_directory <temp_directory> Temp directory to pass to functions. -emailHost,--emailSmtpHost <emailSmtpHost> Email SMTP host. Defaults to localhost. -emailPort,--emailSmtpPort <emailSmtpPort> Email SMTP port. Defaults to 465 for ssl, otherwise 25. -emailTLS,--emailUseTLS Email should use TLS. Defaults to false. -emailSSL,--emailUseSSL Email should use SSL. Defaults to false. -emailUser,--emailUsername <emailUsername> Email SMTP username. Defaults to none. -emailPass,--emailPassword <emailPassword> Email SMTP password. Defaults to none. Not secure! See emailPassFile. -emailPassFile,--emailPasswordFile <emailPasswordFile> Email SMTP password file. Defaults to none. -bsub,--bsub_all_jobs Use bsub to submit jobs -run,--run_scripts Run QScripts. Without this flag set only performs a dry run. -dot,--dot_graph <dot_graph> Outputs the queue graph to a .dot file. See: http://en.wikipedia.org/wiki/DOT_language -expandedDot,--expanded_dot_graph <expanded_dot_graph> Outputs the queue graph of scatter gather to a .dot file. Otherwise overwrites the dot_graph -startFromScratch,--start_from_scratch Runs all command line functions even if the outputs were previously output successfully. -status,--status Get status of jobs for the qscript -statusFrom,--status_email_from <status_email_from> Email address to send emails from upon completion or on error. -statusTo,--status_email_to <status_email_to> Email address to send emails to upon completion or on error. -keepIntermediates,--keep_intermediate_outputs After a successful run keep the outputs of any Function marked as intermediate. -retry,--retry_failed <retry_failed> Retry the specified number of times after a command fails. Defaults to no retries. -l,--logging_level <logging_level> Set the minimum level of logging, i.e. setting INFO get's you INFO up to FATAL, setting ERROR gets you ERROR and FATAL level logging. -log,--log_to_file <log_to_file> Set the logging location -quiet,--quiet_output_mode Set the logging to quiet mode, no output to stdout -debug,--debug_mode Set the logging file string to include a lot of debugging information (SLOW!) -h,--help Generate this help message
If you see this message, your Queue installation is ok. You're good to go! If you don't see this message, and instead get an error message, proceed to the next section on troubleshooting.
Let's try to figure out what's not working.
Action
First, make sure that your Java version is at least 1.6, by typing the following command:
java -version
Expected Result
You should see something similar to the following text:
java version "1.6.0_12" Java(TM) SE Runtime Environment (build 1.6.0_12-b04) Java HotSpot(TM) 64-Bit Server VM (build 11.2-b01, mixed mode)
Remedial actions
If the version is less then 1.6, install the newest version of Java onto the system. If you instead see something like
java: Command not found
make sure that java is installed on your machine, and that your PATH variable contains the path to the java executables.
On a Mac running OS X 10.5+, you may need to run /Applications/Utilities/Java Preferences.app and drag Java SE 6 to the top to make your machine run version 1.6, even if it has been installed.
official, queue, basic, intro, developer, analyst, test, install
Updated on 2013-06-17
From jphekman on 2014-02-06
I got the error message:
Caused by: java.net.UnknownHostException: ansci-5ln0zv1.ansci.illinois.edu: ansci-5ln0zv1.ansci.illinois.edu: Name or service not known
It appears that Queue is attempting to phone home, and being flummoxed by the fact that name service for this machine doesn’t work outside of our firewall? I would be happy to give it the machine’s IP to use instead, but am not sure how.
I also tried adding “-et NO_ET”, but that did not make a difference.
I’d love to know how to get this working — I want to parallelize some SNP calls so I don’t have to wait two weeks for the job to finish running!
Thanks very much,
Jessica
From Geraldine_VdAuwera on 2014-02-06
Hi Jessica,
If you are working behind a firewall, you’ll be better off just deactivating the Phone Home feature. You’ll need to request a deactivation key for the NO_ET setting to work. See http://www.broadinstitute.org/gatk/request-key for details.
From jphekman on 2014-02-07
Thanks! I have submitted my request. I appreciate the quick response!
From jphekman on 2014-02-11
I got the key, but am unable to get Queue to work. The command line is:
java -jar /usr/local/Queue/Queue.jar -et NO_ET -K hekman2_illinois.edu.key -h
And I’m getting the same error message as before, so I conclude that phone home is still enabled. The key file is correctly named and in the current working directory.
Thanks!
From Geraldine_VdAuwera on 2014-02-11
The problem here is that the key is a GATK argument, not a Queue argument. You can’t actually pass GATK arguments directly to Queue from the command line; you need to write a scala script, and within that, you either define arguments to pass in the key at command line, or you hard-code it into the script for convenience.
I’m thinking your original issue wasn’t a phone home problem since Queue itself doesn’t phone home. I had assumed you ran a script that called GATK, but if not, then something else is wrong. Can you please post the full error message you got?
From jphekman on 2014-02-11
Here ‘tis:
[hekman2@ansci-5ln0zv1 ~]$ java -jar /usr/local/Queue/Queue.jar -et NO_ET -K hekman2_illinois.edu.key -h
Exception in thread “main” java.lang.ExceptionInInitializerError at org.broadinstitute.sting.queue.engine.QGraphSettings.(QGraphSettings.scala:65) at org.broadinstitute.sting.queue.QCommandLine.(QCommandLine.scala:86) at org.broadinstitute.sting.queue.QCommandLine$.main(QCommandLine.scala:50) at org.broadinstitute.sting.queue.QCommandLine.main(QCommandLine.scala)
Caused by: java.net.UnknownHostException: ansci-5ln0zv1.ansci.illinois.edu: ansci-5ln0zv1.ansci.illinois.edu: Name or service not known at java.net.InetAddress.getLocalHost(InetAddress.java:1473) at org.broadinstitute.sting.queue.util.SystemUtils$.(SystemUtils.scala:36) at org.broadinstitute.sting.queue.util.SystemUtils$.(SystemUtils.scala) … 4 more
Caused by: java.net.UnknownHostException: ansci-5ln0zv1.ansci.illinois.edu: Name or service not known at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293) at java.net.InetAddress.getLocalHost(InetAddress.java:1469) … 6 more
From Geraldine_VdAuwera on 2014-02-11
Oh yeah that’s something else — but it’s not anything I recognize, sorry. I would recommend contacting your IT department to ask for help with your local environment.
From pdexheimer on 2014-02-12
@jphekman, you might point your IT guys at this page: http://stackoverflow.com/questions/1881546/inetaddress-getlocalhost-throws-unknownhostexception
From jphekman on 2014-03-04
Phew! So as it turns out, the problems with Queue were the first sign that our lab computer was dying. We ended up getting an entirely new machine. I just tried Queue out on it, and it works perfectly (well, at least so far as giving the help message without complaint). So the problem does seem to have been on our end. Thanks for all the help!
From Geraldine_VdAuwera on 2014-03-04
Sorry to hear about your lab computer, @jphekman; may it rest in piece(s). But glad to hear that Queue is behaving properly on the new machine !
From sorrywm on 2014-04-22
I am getting an ExceptionInInitializerError when I try to run Queue (v. 3.1.1), as follows:
Exception in thread “main” java.lang.ExceptionInInitializerError at org.broadinstitute.sting.queue.engine.QGraph.(QGraph.scala:71) at org.broadinstitute.sting.queue.QCommandLine.(QCommandLine.scala:89) at org.broadinstitute.sting.queue.QCommandLine$.main(QCommandLine.scala:50) at org.broadinstitute.sting.queue.QCommandLine.main(QCommandLine.scala)
Caused by: java.lang.NullPointerException at org.reflections.Reflections.scan(Reflections.java:220) at org.reflections.Reflections.scan(Reflections.java:166) at org.reflections.Reflections.(Reflections.java:94) at org.broadinstitute.sting.utils.classloader.PluginManager.(PluginManager.java:79)
I am running java version 1.8, on a Mac. Please let me know if I can provide any other information for troubleshooting. Thank you!
From Geraldine_VdAuwera on 2014-04-22
@sorrywm That might be a java version issue. We have not tested the software with java 1.8. I can only help you if this also happens when you run on java 1.7.
From RachaBella on 2016-03-09
Hey Geraldine
I have followed the steps one by one, i run mvn clear verify, it worked fine, but then i cant run java -jar dist/Queue.jar — help , it says enable to find jarfile queue.jar , i really dont understand why. thanks for the help.
From Geraldine_VdAuwera on 2016-03-11
You may need to adapt the file path to the jar file. Can you find the jar file in the dist directory?
From zhangjing123 on 2016-07-07
Hi Geraldine,@Geraldine_VdAuwera
I run the command java -jar Queue.har —help , but I got the error:
Exception in thread “main” java.lang.UnsupportedClassVersionError: org/broadinstitute/gatk/utils/commandline/CommandLineProgram : Unsupported major.minor version 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:803) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:803) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
my java verion is “1.7.0_101.
I don’t know what’s wrong, thanks for your help!
From Geraldine_VdAuwera on 2016-07-07
If you’re running GATK/ Queue 3.6 you need Java 1.8.
From zhangjing123 on 2016-07-08
Yes! I change the java version 1.8, the problem was solved. thank you , Geraldine.
From fchen6 on 2016-10-28
> @Geraldine_VdAuwera said:
> You may need to adapt the file path to the jar file. Can you find the jar file in the dist directory?
Hi I don’t know where to find dist directory. Will it be at the same directory /gatk or elsewhere? Thank you!
From Geraldine_VdAuwera on 2016-10-28
@fchen6 Sorry, I forgot this has changed a little, now the directory is called `target`. It should be at the top level of the repo directory.
From fchen6 on 2016-11-03
>
Geraldine_VdAuwera said: >
fchen6 Sorry, I forgot this has changed a little, now the directory is called `target`. It should be at the top level of the repo directory.
@Geraldine_VdAuwera , so is Queue.jar actually at /gatk/public/gatk-queue/target/gatk-queue-3.6.jar ? Is it the same as Queue.jar? Because it is very small (less than 1 Mb) and when I type:
java -jar ./gatk/public/gatk-queue/target/gatk-queue-3.6.jar —help
It returns that: “no main manifest attribute, in ./gatk/public/gatk-queue/target/gatk-queue-3.6.jar”
Thank you.
From Geraldine_VdAuwera on 2016-11-03
No, the target directory should be a top-level directory in the source code package, and it would contain Queue.jar itself. The jar you are looking at is not meant for running the program directly.
From fchen6 on 2016-11-03
> @Geraldine_VdAuwera said:
> No, the target directory should be a top-level directory in the source code package, and it would contain Queue.jar itself. The jar you are looking at is not meant for running the program directly.
After I run the command “mvn clean verify”, I think it goes successfully,
there is no “target” at the root directory of gatk. (I clone from “https://github.com/broadgsa/gatk.git”)
All I have is:
$ ll -h
total 66K
-rwxrwxr-x 6.5K Nov 3 10:42 ant-bridge.sh*
-rw-rw-r— 7.4K Nov 3 10:42 intellij_example.tar.bz2
drwxrwxr-x 2.0K Nov 3 10:42 licensing/
-rw-rw-r— 44K Nov 3 10:42 pom.xml
drwxrwxr-x 2.0K Nov 3 10:42 public/
-rw-rw-r— 81 Nov 3 10:42 README.md
drwxrwxr-x 2.0K Nov 3 10:42 settings/
So, could you please tell me where can I find the target directory? Thank you.
From Geraldine_VdAuwera on 2016-11-04
Oh, I see the problem — you need to do this with broadgsa/gatk-protected.
From fchen6 on 2016-11-04
> @Geraldine_VdAuwera said:
> Oh, I see the problem — you need to do this with broadgsa/gatk-protected.
Thank you @Geraldine_VdAuwera! It is very helpful. I made it.
From ndelloca on 2017-10-23
Sorry I’m started to work with linux and with Gatk also. When I test the Queue insatallation I got the following message
Error: Unable to access jarfile Queue.jar
How I can resolve this problem?
From Sheila on 2017-10-24
@ndelloca
Hi,
What is the command you are trying to run? Have a look at [this thread](https://gatkforums.broadinstitute.org/gatk/discussion/6436/how-to-fix-the-error-unable-to-access-jarfile-genomeanalysistk-jar) which may help. You may also be interested in [WDL](https://software.broadinstitute.org/wdl/) which is replacing Queue :smile:
-Sheila