Aviation Neurologist Dr. John Hastings MD https://airandspace.si.edu/support/wall-of-honor/john-d-hastings-md Who is a Commercial Pilot with 6,100 hours and is a former Senior Aviation Medical Examiner has been a critic of the FAA's Cogscreen AE stating that the test is overly sensitive and labels many pilots "false positive" to ADHD and other cognitive short comings. Dr. Hastings uses the analogy of "dolphins" as to false positive pilots when catching tuna. https://lpba.org/Uploads/ConventionsEvents/Winter2019/190309%20-%20945am%20-%20John%20Hastings%20Powerpoint.pdf mirrored https://drive.google.com/file/d/1LLqx2kPDXED2Lp6XbR5j3fWo00RIPsPi/view
Dr. Hastings in a previous presentation has stated that Cognitive Testing is not an accurate way to gauge pilot performance and outside of real world conditions, flight simulators are a better tool of determining pilots cognitive ability than proprietary psychological tests that are left up to a lot of interpretation. http://civilavmed.org/wp-content/uploads/2016/03/The-Real-World-Jack-Hastings.pdf mirrored https://drive.google.com/file/d/1x2Kamji50asfu-CKcoLCFHZBkJnTrrlD/view
This 3rd party report referencing both the work of Dr. Hastings and Cogscreen AE creator Dr. Gary Kay PhD confirms this by rejecting the correlation between Cogscreen and flight simulator performance, if anything it shows there is no correlation and that the Cogscreen should be rejected as a priamary tool to determining a pilot's cognative ability if used at all. Page 39 https://www.icao.int/safety/aviation-medicine/Documents/Potocko%20ICAO%20AsMA%202019%20Public.pdf mirrored https://drive.google.com/file/d/1WAA3VJIvyeJMgKal8D7AU1kgjnblI1y3/view
Dr. Hastings even stated other ICAO do not use cognitive assessments such as the UK test pilots using simulators. https://www.lpba.org/Uploads/ConventionsEvents/wINTER2019/190309%20-%20945am%20-%20John%20Hastings%20Handout.pdf mirrored https://drive.google.com/file/d/1J4JF6NIijnbzbMm68MFqLJYKEAqQlsXf/view
The Cogscreen AE is so biased against pilots that 70% of the pilot candidates that applied to FedEX were rejected due to failing the Cogscreen AE leading to a lawsuit that FedEx lost. https://www.airlinepilotforums.com/archive/index.php/t-60252.html
The Cogscreen itself is being kept proprietary by design to prevent peer reviews from picking the test apart. From what little information that has been published states that the sample size was as low as 704 pilots, and that was up from the initial sample size of 584 pilots which only had 28 women bringing up the question of the lack of diversity on top of such a small sample size to begin with. https://scholarworks.wmich.edu/cgi/viewcontent.cgi?article=2299&context=dissertations (page 93) mirrored https://drive.google.com/file/d/1i4bRkcv9qWY8-aABIlbEYDWn1RmvzEgH/view
It was only in 2013 the sample size was increased up to 704 when the FAA had 120 pilots that attended the annual EAA Airventure event in Oshkosh take the Cogscreen for free but their individual results would remain anonymous. If this test was to be used as the end-all-be-all for keeping your medical certificate then why did pilots that bombed the test at Airventure (AND FedEX) not have their medical certificates revoked? https://www.flyingmag.com/pilots-places/pilots-adventures-more/pilots-sought-cognitive-tests-airventure/ and https://www.aopa.org/news-and-media/all-news/2013/july/25/take-cogscreen-test-at-airventure to put this into perspective with total published sample size as low as 704 pilots out of as of 2019 an estimated 664,565 active pilots would result in a total sample size of 0.1% of active Pilots. It doesn't appear that most of the people who take the Cogscreen and added to the overall average, meaning the percentiles are artificially created.
This United States Air Force white paper was one of the few publications that gave insight on how the Cogscreen was scored, specifically with the math portion if you don't answer Layered mental math questions and 35 seconds you're in the bottom 10 percentile and fail page 19 https://apps.dtic.mil/dtic/tr/fulltext/u2/a554594.pdf mirrored https://drive.google.com/file/d/13tKfITjODx2bgI2Nx4yWGjiASueDsgFU/view Cogscreen graders would seem to sujetively fail pilots that don't finish in 25 seconds either.
Then there is this publication from Canada which said that experienced pilot trade speed for accuracy, Page 61
https://curve.carleton.ca/system/files/etd/e40f9124-679e-48af-9d1a-f91e65029547/etd_pdf/30c60c68d17834245fb1ca02037ec4aa/cass-cansituationawarenessbepredictedinvestigating.pdf mirrored https://drive.google.com/file/d/10BTax1aQkbrsNi-36h36efnYFHmNqU9t/view
What is also confirmed by Delta Airlines own numbers when they required their pilots to take the Cogscreen "https://corescholar.libraries.wright.edu/cgi/viewcontent.cgi?article=1050&context=isap_2011 page 6 of 7 "their CogScreen profiles indicated that they appeared to focus more on accuracy at the expense of throughput, a combination of speed and accuracy"
This is inline with the Canadian study, experienced pilots favor accuracy over speed. mirrored https://drive.google.com/file/d/10BTax1aQkbrsNi-36h36efnYFHmNqU9t/view
However the Cogscreen's scoring algorithm from what little information that has been published seems to favor speed over accuracy putting experienced pilots at a disadvantage to how this test is scored. However the Cogscreen's scoring algorithm from what little information that has been published seems to favor speed over accuracy putting experienced pilots at a disadvantage to how this test is scored.
The FAA claims in their own white papers that the Cogscreen is 100% accurate and those that fail the Cogscreen have a 50/50 percent chance of passing other cognitive tests. Nothing is 100% accurate and it shows that the Cogscreen is a biased subjective test at best that labels the cognitive ability of the test wrong so many times. Page 43 http://civilavmed.org/wp-content/uploads/2016/03/Neurocognitive-Assessment-The-FAA-Perspective_CAMA-Sunday-at-AsMA-2017.pdf mirrored https://drive.google.com/file/d/1IKLQwNG5EZYFQQ4sG6JTw2lqSssp9jpk/view
This report contradicts the FAA saying there is very little at best correlation between performance on the Cogscreen and performance in the cockpit https://www.icao.int/safety/aviation-medicine/Documents/Potocko%20ICAO%20AsMA%202019%20Public.pdf see pages 39-41
mirroredhttps://drive.google.com/file/d/1WAA3VJIvyeJMgKal8D7AU1kgjnblI1y3/view
As for other criticisms of the Cogscreen, here's more case law that states the Cogscreen doesn't really say anything about one's cognitive ability section B https://caselaw.findlaw.com/us-7th-circuit/1040781.html stating The FAA, when it reviewed the most advanced cognitive testing
technology, known as CogScreen-AE, concluded that that test ``cannot sufficiently identify age-related cognitive function deficits that would impact pilot performance and aircraft safety.'' On appeal of this decision in Yetman v. Garvey 261 F.3d at 675 (7th Cir. 2001), the Court of Appeals affirmed the FAA's decision, concluding: ``Ultimately, we find that substantial evidence supports the FAA's finding that CogScreen-AE is not, at this point, an adequate cognitive tool for determining whether an exemption to the Age 60 Rule is warranted.''
https://www.govinfo.gov/content/pkg/CHRG-109shrg63516/html/CHRG-109shrg63516.htm
The FAA has a history of grounding pilots under false mental health diagnosises, often for political reasons like that of Delta Airlines whistleblower Karlene Petitt https://en.wikipedia.org/w/index.php?title=Stephen_Dickson_(executive)&oldid=1002912228 See last paragraph
Page 399 acknowledges that "mood stabilizing" drugs have been so over prescribed that it may become increasingly difficult to find potential pilots who do not use these medications. https://scholar.smu.edu/cgi/viewcontent.cgi?article=4088&context=jalc and page 408 ADHD diagnosis's are becoming common to an epidemic level. page 419 states that "Any chance that the FAA might relax its mental health requirements likely went out the window after Germanwings." This is a self fulfilling prophesy. These draconian measures where any issue would lead to a grounding encourcages pilots to hide their mental health issues which is what lead to Germanwings. That pilot did not seek treatment for his depression which would have grounded him that lead to the incident. This stone age philosophy to mental health will cause more Germanwings incidents not prevent them.