### A' (A-prime) as sensitivity in signal detection (and why to use A instead)

If you believe Signal Detection theory, a reasonable measure of the distance between two distributions is d', which is essentially the z-score difference between the signal and noise distribution. Theoretically, d' as a measure of sensitivity requires satisfaction of assumptions that are never tested, and often may not be testable.  For example, in the figure to the left, what if the signal and noise distribution have different variances.  In this case, d' is misleading or could be very wrong.

A bigger practical problems is as follows:
d' = Z(Hit Rate) - Z(False Alarm Rate), where Z is the inverse cumulative of the normal distribution.  But what happens when HR or FAR is 0 or 1?  The value is either -inf or +inf!  And d' is undefined.

This wouldn't be  a problem if these values never happened, but a false alarm rate of 0 is probably the most frequently obtained false alarm rate, and it is very common to have hit rates of 0 or 1.  What is a person to
do?

Pollack and Norman (1964) proposed a 'non-parametric' measure based on the following logic.  First, let's you had direct access to the underlying distributions.  For equal-variance distributions, these are are monotonically non-decreasing functions.
Assume that the Receiver operating characteristic (ROC) function, formed by plotting the cumulative hit rate versus false alarm rate.  Below are two examples of ROC functions from Mueller and Weidemann (2008) and Weidemann and Mueller (2008), using either (simulated) confidence ratings or (real) response time distributions to examine the shape of the underlying distributions.   Pollack and Norman (1965) based their measure on the assumption that a 'proper' ROC function should be monotonically non-decreasing. Consequently, a single point specifies a family of possible ROC functions through that point.  They discussed several possible ROC measures, as shown in the Figure below.  For example, the area covered by polyogon I is the smallest proper ROC curve.  They also argued that the  largest proper ROC is either I+A1 or I+A2.  Thus, based on this logic, they proposed a measure called A', which is the average between I+A1 and I+A2 (I+A1 + I + A2)/2.  These are the proper ROC curves with steepest and shallowest slopes.

T
here are a number of problems with this.  First, it, like d', ignores the possibility of differenc
es in variance between the two distributions (which can lead to 'improper' ROC curves).  But more importantly, the meaning of A', if it is considered at all, is typically stated as something like "An average between the smallest and largest proper ROC curves that can pass through a point".  Given the above figure, the smallest is I, but the largest is either I+A1, or I+A2, or maybe something else. Thus, the stated interpretation of A' is not really what A' measures at all.

Smith (1995) pointed this out and gave the formula people should use if they mean what they really mean by A'.  This was either I+A1 or I+A2, depending on where the point lay.  But it turned out he made an error--the largest-area ROC curve through a point in the upper left quadrant actually looks like the figure below.  When p is in the upper right quadrant, the largest area is I+A2.  When it is in the lower left quadrant, it is I+A1. But when it is in the upper left quadrant, it is neither, rather it is a polygon where the top slanted line intersects neither (0,1) or (1,1). This correction was published by Zhang and Mueller (2005), and called A. If you want to use A', you probably should use this formula instead (and cite Zhang & Mueller while you are at it). The proof is in the paper, but the formula to measure this is:  Also in the paper, there is a bias measure associated with A, which is related to the slope of the ROC function at the point of the observation.  The formula is as follows, with b=1 being unbiased.  You can use log(b) as a symmetric bias measure.

Computing A
The R code for computing these values is as follows:

A <-function(h,f)
{
if(f<=.5 & h>=.5)
{
a <- .75 + (h-f)/4 - f*(1-h)
} else if(f<=h & h<=.5)
{
a <- .75 + (h-f)/4 - f/(4*h)
} else {
a <- .75 + (h-f)/4 - (1-h)/(4 * (1-f))
}
return(a)
}

b <- function(h,f)
{
if(f<=.5 & h>=.5)
{
b <-(5-4*h)/(1+4*f)
} else if(f<=h & h<=.5)
{
b <-(h^2+h)/(h^2+f)
} else {
b <- ((1-f)^2 + (1-h))/((1-f)^2 + (1-f))
}
return(b)
}