The FAA paid one person, Dr. Gary kay PhD to not only develop the Cogscreen AE test but the entire cognitive battery. Here is the deposition of Dr Gary K admitting to as much we're basically admits that not only are conflicts of interest involved in creating the test but the actual test results AR entirely subjective when it comes to scoring.
https://www.benglasslaw.com/library/Direct-and-Cross-Examination-of-Neurologist-Gary-Kay.pdf mirrored https://drive.google.com/file/d/1jYQzCziLxJe7LLIyAWu30iFACaypZNR0/view
Here are some key points from Kay's testimony using the page number printed in the depo and not the total page count of the PDF
Page 68 gary kay admits he works for the FAA
page 69 says he created the cogscreen
page 71 admits he's still very hands on with the cogscreen development and the FAA pays him $100 an hour
page 74 mostly evaluates airline pilots
page 79 says he doesn't want info to get out on the internet as it will ruin the validity of the test
page 80 admits though neuropsychologists can exchange info with each other, just not the public
page 97 talks about adhd
page 102 says it's illegal for technicians or students to administer Cogscreen in Washington DC due the chances of creating false positives due to the test being overly sensitive.
page 119 says you do well on backwards counting you would do well with math
page 121 doesn't take much difference to drop huge percentiles
page 124-125 admits the PASAT is an evil test and "makes everyone cry"
page 155 admits cog screen is a conflict of interest
page 167 Kay admits that this line of questioning diminished his aviation practice
This is also inline with Dr. Bruce Chen claiming the Cogscreen "self validates" as the test detects if someone prepped for it and self disqualifies https://www.pilotsofamerica.com/community/threads/websites-to-study-for-cogscreen-related-add-tests.111688/
The Cogscreen apparently had not been subject to impartial peer reviews which would have torn this thing apart. The few "peer reviews" that were allowed were under the caveat of being biased towards supporting the Cogscreen. Senior HIMS AME Dr. Brent Blue pointed out that 80+ of these publications had Dr. Kay as either the lead or contributing author, hardly impartial and a major conflict of interest. https://sites.google.com/site/no2cog/supporting-evidence/senior-hims-ame-dr-brent-blue-says-cogscreen-never-stopped-an-accident?authuser=0
Pretty much the psychologist scores the test to benefit their client which is against the test taker. Conflict of interest is the test taker actually has to pay for the test. He also admits the tests are painful and the last test in the battery makes people cry. The Plaintiff was awarded nearly $451k and the defendants did not appeal.
Gary Kay was on the loosing side of another lawsuit in 2014 where a molecular biologist lost his career to an automotive accident where the loosing side made the claims, well the now former biologist could find employment by "more networking" the Molecular biologist was awarded a judgment of $1.5 million. Page 6 http://juryverdicts.net/Virginia6-14.pdf
mirrored https://drive.google.com/file/d/1lmGEGHRLHFCrJ_gPucoQHPliZBPMeVTZ/view
As for other criticisms of the Cogscreen, here's more case law that states the Cogscreen doesn't really say anything about one's cognative ability section B https://caselaw.findlaw.com/us-7th-circuit/1040781.html and here's a link that says that 70% of the Fedex pilot applicants are rejected because they failed the Cogscreen. https://www.airlinepilotforums.com/archive/index.php/t-60252.html updated https://www.airlinepilotforums.com/hiring-news/60252-fedex-changing-interview-process.html These Pilots are very experienced with thousands of hours of jet time alone let alone probably at least five thousand hours of Total time , they are the equivalent experience of mainline airline pilots. Gary Kay was quotes as saying retaking the Cogscreen won't change the results, https://www.airlinepilotforums.com/hiring-news/60252-fedex-changing-interview-process-2.html but if that's the case why is Kay so paranoid to letting information about the test get out. If people's attempts to study and prepare for the test won't affect the outcome, then why is the test proprietary? It's either nothing can be done to prepare for the test OR Kay doesn't want people to prepare for the test because they would do better on it, exposing the Cogscreen as "junk science" That 70% failure rate was also confirmed here https://www.pilotsofamerica.com/community/threads/cogscreen-if-all-pilots-had-to-take-it-how-many-would-pass.141322/
Kay also admitted the results of the test are up to a lot of interpretation and are skewed towards the client, and against the test taker. So Kay skewed the results of the test he was deposed of to benefit state farm to avoid an insurance payout, likewise the results of many pilot's test were skewed to yank their medical certificate.
This review on vitals.com, a promenent health professionals review site says it all https://www.vitals.com/doctors/Dr_Gary_G_Kay.html "Unethical, a corporate hack, acting the interests of employers. He designed a test battery used for termination of employees. Complete pseudoscience"
Even how Sample size for the "baselines" of the Cogscreen was calculated isn't that scientific. The sample size is roughly 704 individuals, and that's after adding 120-150 pilots at EAA's Airventure 2013 https://www.aopa.org/news-and-media/all-news/2013/july/25/take-cogscreen-test-at-airventure and https://www.flyingmag.com/pilots-places/pilots-adventures-more/pilots-sought-cognitive-tests-airventure/ The original baseline only consisted of 584 pilots. https://books.google.com/books?id=8_tmbkk8OQ8C&pg=PA88&lpg=PA88&dq=cogscreen+584&source=bl&ots=6CF94idP5Q&sig=ACfU3U0UiooqmrzT_cu89HIEh_m4E7Km3Q&hl=en&sa=X&ved=2ahUKEwjw2PDkl6b0AhXJk2oFHTn3DjsQ6AF6BAgNEAM#v=onepage&q=cogscreen%20584&f=false and https://books.google.com/books?id=8_tmbkk8OQ8C&pg=PA88&lpg=PA88&dq=cogscreen+584&source=bl&ots=6CF94idP5Q&sig=ACfU3U0UiooqmrzT_cu89HIEh_m4E7Km3Q&hl=en&sa=X&ved=2ahUKEwjw2PDkl6b0AhXJk2oFHTn3DjsQ6AF6BAgNEAM#v=onepage&q=cogscreen%20584&f=false
That's only 0.1% of the total active pilot pool of 664,565 from 2019. Hardly a large enough pool to have an accurate sample cognitive abilities of pilots.
Notice the scores are pooled and anonymized. So if someone at Airventure bombs (as in gets lower than 15% on a section) they don't lose their medical, nor does it give an accurate average since it covers up individual failures of the testing sample. This breakdown is for simplicity sake, in that there are over 10 sections of the test and more than 4 people took it.
X ➞ Test Takers 00 95 90 85 | 67.5 Y
60 05 80 70 | 53.75 ⬇
95 95 95 00 | 71 Test Sections
80 30 20 80 | 52.5
56 56 71 59
X could be the test takers and Y could the the sections. Notice how with the way the average was created second from the bottom a tester could get a 0 on on section but still averaged 71 but the bottom score is 52 and that tester technically didn't bomb any sections because each score is over 15. In fact that tester would have gotten the lowest overall score. This hides the fact that more than likely 80% of pilots will bomb at least on section of the test as per conversations on Reddit.
Apparently the FAA is aware of the issues regarding the Cogscreen from at least 2014 with regards to issues stemming for calibration of the equipment due to the software being created in 1987 and being administered by technicians instead of the psychologist license holders as well as the creation of a more realistic baseline database, One that shows the "bottom 15%" is really 80% which however has not been implemented. http://www.avcanada.ca/forums2/viewtopic.php?t=9459
The Cogscreen is also overly sensitive to hypoxic conditions due to higher altitudes lowering ambient air pressure, lowered oxygen concentrations, as well as higher carbon dioxide concentrations in the testing environment. https://www.researchgate.net/publication/235208953_Subtle_Cognitive_Effects_of_Moderate_Hypoxia
In fact the FAA seemingly admits to keep using obsolete psychological and cognitive assessment tests because it flunks more pilots, even though the reason why those tests are now considered obsolete is due to the high amount of those flunks being false positives and wrongly label people as mentally defective when they were perfectly normal.
https://www.faa.gov/ame_guide/media/MMPI-2_vs_MMPI-3.pdf and https://www.nationalpsychologist.com/kahler-v-kansas
In this thread in the Pilots of America Forum it questions how many pilots would actually have medicals if every pilot was forced to take the Cogscreen. https://www.pilotsofamerica.com/community/threads/cogscreen-if-all-pilots-had-to-take-it-how-many-would-pass.141322/
One of the posters jbarrass posted this in the thread
Well, well, well. Looks like someone at the FAA has been following this thread
The FAA is recruiting pilots attending EAA AirVenture Oshkosh 2023 to participate in a PAID FAA research study. For this study, the FAA needs pilots (18-plus years of age) who have flown at least once in the last 6 months (including training simulator time), and who hold a current medical certification Class I, II, or III (not BasicMed). The study session takes about 4 hours and pays $300, $400, or $500 (dependent on medical certification class), and payment is dispersed immediately following participation!
Participation involves completing two computerized cognitive tests related to tasks such as working memory, attention, mental rotation, and multitasking performance. This study has received FAA IRB approval. The purpose of this study is to obtain pilot normative data for these computerized tests. The FAA uses these tests to help recertify pilots for flying following a medical event (e.g., stroke, head injury, certain medication). The results of this study will help ensure that aeromedical decision-making is based on the most current scientific data, and will contribute to the safety of the national airspace system. FAA Cognitive Test PAID Study (Cogstudy) FAQs.
https://www.eaa.org/airventure/eaa-...cruiting-pilots-for-airventure-research-study
This is interesting and I wonder what the real point is. The FAQ says:
"This FAA research study about the use of computerized cognitive tests as a non-invasive way to measure cognitive function in pilot populations. The purposes of this research are to: 1) evaluate computerized cognitive tests to determine if they are acceptable for use as a cognitive screening tool; and 2) obtain pilot normative datasets for each cognitive test. The FAA uses computerized cognitive tests only for those pilots who may have a medical condition associated with aeromedically significant cognitive impairment (e.g., stroke, head injury, certain medications) and who wish to return to flight or duty status. Normative datasets are a collection of test scores that represent what is usual or expected in a representative sample of pilots. The outcome of this research will ensure that FAA processes for aeromedical decision-making are consistent with best clinical practices for aerospace medicine and current scientific knowledge."
By excluding people without medicals (i.e. BasicMed/SP/balloon/Gliders) they are showing a strong selection bias and skewing the results. By paying different rates for class of medical they are skewing it even more (towards the C1 folks).
Peer reviewed normative data sets for cognitive tests are available and in wide use already.
And the statement
"FAA uses computerized cognitive tests only for those pilots who may have a medical condition associated with aeromedically significant cognitive impairment (e.g., stroke, head injury, certain medications) and who wish to return to flight or duty status."
Is a bold face lie. They are commonly used in HIMS cases.
This would fail any high school statistics class.
as well as
I'd love to cite the actual statistics. But the FAA treats them like state secrets for some reason. If they are available somewhere, please point me to them.
All they really need to do for credibility is to publish the results in a peer reviewed professional journal. Anybody think that's likely? Not me.
That is a very serious question because the it's thrown out that "only the bottom 15% of the Cogscreen takers are not granted medicals" yet what is the bottom 15%. It sure seems that the FAA is making up the numbers and are subjectively grounding pilots and not giving the reasons as to why. A major Sixth Amendment violation.
This shows the subjectively graded Cogscreen and other FAA cognative tests by Gary Kay have nothing to do with safety and are really about creating a cottage industry that holds pilots hostage for as much money can be squeezed out of them by dangling the hope with no guarantee of getting their medicals back. Far from it, these trapped pilots are subjectively graded as fails in order to keep them in hostage indefinitely. It's time to let the safest pilots have their unrestricted medicals back without any strings attached and to put these mental health quacks cottage industry of holding pilots hostage out of business.