AUROC2R - Metacognitive Sensitivity
The following is a R porting of the nice matlab function coming from Fleming and Lau ( Fleming, Stephen M., and Hakwan C. Lau. "How to measure metacognition." Frontiers in human neuroscience 8 (2014): 443 ).
As they state in their nice paper: "The area under the type 2 ROC curve (AUROC2) can then be used as a measure of metacognitive sensitivity. This method is more advantageous than the gamma and phi correlations because it is bias-free (i.e., it is theoretically uninfluenced by the overall propensity of the subject to say high confidence) and in contrast to type 2 d′ does not make parametric assumptions that are known to be false."
Adapted R code below for the AUROC2 function:
i <- Nratings+1H2 <- c()FA2 <- c()for (c in 1:Nratings){H2[i-1] <- length(which(conf == c & correct == 1)) + 0.5FA2[i-1] <- length(which(conf == c & correct == 0)) + 0.5i <- i-1}
H2 <- H2/sum(H2);FA2 <- FA2/sum(FA2)cum_H2 <- c(0, cumsum(H2))cum_FA2 <- c(0, cumsum(FA2))
i <- 1k <- c()for(c in 1:Nratings){k[i] <- (cum_H2[c+1] - cum_FA2[c])^2 - (cum_H2[c] - cum_FA2[c+1])^2i <- i+1}auroc2 <- 0.5 + 0.25*sum(k)
return(auroc2)
}