Conversation On Reddit About The Cogscreen

plaid_rabbit posted research on the cogscreen.  It gives a ton of false positives

https://www.reddit.com/r/flying/comments/jetcs5/im_quitting_with_900_hours/g9hn8j5/?context=3

https://www.reddit.com/r/flying/comments/jetcs5/im_quitting_with_900_hours/g9gz250/?utm_source=reddit&utm_medium=web2x&context=3

Just an examinee. Nothing else to make me qualified. I'm good at finding stuff on the internet and researching. So you're much more qualified then me here. Please teach me!

have the right control conditions to get rid of the noise such as measurement errors from touchscreens.

So my statement about the touchscreens comes from the earlier papers. The original cogscreen norms were done with lightpens. It's now moved to "elo Accutouch LCD Touchscreen", which are 5 wire resisitive displays. I've done a fair bit of research about the research on cogscreen, and haven't found anyone that's published papers saying that the new format and the old format produce similar results. Kay is working on Cogscreen2, so I expect that'll fix these problems. There's a fair bit of anecdotal evidence that they aren't very accurate, and my experience, and the experience of others shows that the cogscreen equivalent of the trail marking test isn't properly sensitive. I did poorly on the cogscreen trail marking test equivalent (which is about input accuracy), but passed the classic paper trail marking tests. But that's just based on anecdotal evidence. I wish I had access to the score results to see what they looked like to see if it was or wasn't a common problem.

For example, there is a well-known...

Yep, cogscreen includes variation of the WCS test. I didn't do well my first time on it, but with some test prep and aced it the second. So I'm curious what that says about the test and about my neuropsych. What do you say?

I'm not sure what to make of your statement about 5%...

Digging back for citations, it's actually higher then that. I remember finding a PPT done by Kay (I can't find it though). He stated that he reduced complex tests for (some neurocog defect, SSRIs maybe?) that the current test format avoided 50% of people from having to take additional tests, and of those, about 50% went on to getting their medical. Cogscreen was designed as a screening test, so it has a high false-positive rate. Looking at the partial norm data I have, 45% of pilots have one score below the 5th percentile in the speed category, and that's doesn't include the accuracy or throughput category. So based on the norms for regional jet pilots, at least 45% of the pilots will fail at least one item of the cogscreen. It was designed to be a screen with a low false negative rate, but it has a high false positive rate.

Third, keep in mind...

I totally get it, but I passed, and think it was kind of dumb. How does the fact that I experienced mild depression and took an SSRI indicate I have a neurocog issue? I do know there's some concern, but the results aren't conclusive either way.

I accept that the FAA has been über-conservative to a point where it is counter-productive...

This is basically my problem. Yes, anxiety is an issue. The problem is that there are likely many pilots out there, with much worse anxiety then OP, that aren't getting treated. I'm more concerned about the untreated anxiety out there, then the treated anxiety.

I can get you papers on this if you'd like. A good starting point would be https://apps.dtic.mil/dtic/tr/fulltext/u2/a314586.pdf It's got a brief description of the test, (See page 5, paragraph 2), along with norms for military aviators. You probably have a better feel for what the standard deviations tell you about the test. Mind sharing your thoughts?

Another conversation at this address had this to say.  https://www.reddit.com/r/flying/comments/oro9tx/cogscreen_fail/

level 1

TL;DR: No advice, just saying I wish you the best on taking care of this. FAA sucks. Good luck.

The cogscreen is a frustrating thing. My understanding is that scoring in the bottom 15% of pilots on any aspect of the test constitutes failing. But that's strange, because that means you might very well be better on that part of the test than 15% of the folks flying around out there... but because you took an SSRI, it's not good enough, even though it's good enough for the folks who didn't take an SSRI. (Remember, for anyone who doesn't know, the Cogscreen-AE is normalized for PILOTS, not the general population.)

And remember, below the fifteenth percentile on ANY part of the test constitutes a failure. More than 15% of pilots score below the fifteenth percentile on at least one part of the test, since they're not perfectly correlated. Since there's a whole bunch of parts of the test (10+ I believe), my guess would be as many as half of pilots don't cut it on at least one of them. (If they were perfectly uncorrelated with 10 parts, you'd expect 80% of pilots to fail at least one part.)

Just part of the craziness of the FAA. I was reading a website for medical advice for Transport Canada the other day, and according to the site, their approach is more along the lines of, if you have the wherewithal to get through training, you're cognitively there enough to meet our standards. Seems to make much more sense to me.

I wonder if now the FAA is gonna demand these cogscreens for everyone who ever had COVID, in light of recent news of the long-lasting cognitive impacts of the disease. Wouldn't be too crazy; I think they already do cogscreens for all ATC. Might finally give pause to all the great many pilots who brush this sort of stuff aside as the kind of thing only the weirdos who can't manage their own emotions have to deal with (not my belief, but how I think some of the "macho" pilots types think about it; though fortunately that attitude is rare on this sub--I would guess due to demographics, particularly age).

Edit: one other note on the biggest problem with the Cogscreen-AE. There is absolutely no empirical evidence that it actually increases the safety of flight. There is evidence that higher performance on the Cogscreen-AE correlates with higher performance on some observed flight maneuvers. However, nothing has explored the relation between Cogscreen scores and safety; it's all armchair analysis that results in aviators cumulatively paying hunderds of thousands, perhaps millions, of dollars a year, with no clear safety payoff.