created by Geraldine_VdAuwera
on 2017-10-16
A long time ago in a galaxy far, far away, we started work on a brand new version of GATK in which the engine framework was to be completely revamped, streamlined and accelerated, with support for cloud technologies and an impressively expanded scope of analysis (copy number! structural variation! somatic and germline versions of everything!). Oh, and it would be [fully open-source](https://software.broadinstitute.org/gatk/blog?id=9645).
Today that new beginning is tantalizingly close to fruition. We’ve had a series of beta versions out for preview for about three months, and we’ve actually had several segments of our genome production pipeline running a subset of fully-vetted GATK4 tools for over a year. Aside from a few remaining technical issues that are actively being addressed, the work left to be done before general release mainly involves clean-up and streamlining of user-facing functionality: what gets logged and how, argument names and syntax, documentation and so on.
So it’s time to set a date and put a ring on it! I’m thrilled to announce the happy event will take place on Jan 9, 2018.
——
Wait, why January, you ask? Earlier this summer I announced that we hoped to push out the general 4.0 release by the end of September. Obviously it’s now mid-October and it’s not out, so what’s up with that?
To be quite frank, what’s up is mainly that my initial assessment was not sufficiently realistic. I underestimated how much time it would take to identify and resolve technical issues during the beta testing phase (things I learned: humans like to take vacations during summer months) and how much work we still needed to do from the support side to make the migration to GATK4 as painless as it could be.
So when we realized we were going to blow straight through my September estimate, we undertook a much more thorough status review. We formulated the Jan 9 release date based on a generous budgeting of time that assumes the work will be complete by late November, early December at the latest, giving us a few weeks’ worth of padding to cope with any last-minute surprises. And this avoids the end of year holiday period, which is a bad time to release anything anyway — except perhaps a final beta for any early adopters out there looking for an excuse to get away from their in-laws.
As we get closer to the big day we’ll post some additional details of all the goodies that will accompany the release, so stay tuned for further announcements on this blog.
From SkyWarrior on 2017-10-17
Great news! I hope that the latest implementation of pairHMM (AVX-512) is also added to the latest version.
Before that I am hoping a beta release that includes AVX-512 support for pairHMM :)
Also a question for legacy tools. Will the latest GATK4 include the UG from 3.7 or 3.8 for just fallback purposes ?
From Geraldine_VdAuwera on 2017-10-17
A new beta ([4.beta.6](https://github.com/broadinstitute/gatk/releases)) actually went out yesterday with the latest AVX-512 support :)
I’ll have to check on plans wrt UG — right now I don’t think we plan to include it.
From SkyWarrior on 2017-10-17
Great! And it is working!!
2018 will be an awesome year!
From jhpark on 2017-10-22
Great News !!
From Pepetideo on 2017-11-14
Fantastic to see a great team back from the dark side of commercialization and closed source development. It took a few years but things seem on the right path now.
Happy for you guys. Thank you for your hard work
From Geraldine_VdAuwera on 2017-11-14
Ah, @Pepetideo, so dramatic :-D
It’s nice to be back. Have you tried out the beta version yet?
From Pepetideo on 2017-11-17
Dramatic is my middle name ;-)
Go back to close source and you will see me throw a Hamlet ! hehehehe
No… I have not yet tested it … but I read with great interest your paper on CNV detection.
Great stuff :)
From Geraldine_VdAuwera on 2017-11-17
I’ll pass that on to the CNV devs, thanks! You may be interested to hear that we finally wrote a paper on HaplotypeCaller… It’s in biorxiv here: https://www.biorxiv.org/content/early/2017/11/14/201178.1