Mag. Dr. Alexandra Graf

Alexandra Graf and Peter Bauer

Center for Medical Statistics, Informatics and Intelligent Systems

Medical University of Vienna

In gene expression or proteomic studies large numbers of variables are investigated. We generally can not assume that a few of the investigated variables show large effects. Instead we often hope that there is at least a combination of several variables, which, e.g. allow prediction of the response of an individual patient to a particular therapy. The task of selecting useful variables with rather moderate effects from a very large number of candidates and estimating suitable scores to be used for the prediction of a clinical outcome in future patients is a hard exercise.

We evaluate variable selection by multiple tests controlling the false discovery rate (FDR) to build a linear score for prediction of a clinical outcome in high-dimensional data. Quality of prediction is assessed by the receiver operating characteristic curve (ROC) for prediction in independent patients. Thus we try to combine both goals: prediction and controlled structure estimation. We show that the FDR-threshold which provides the ROC-curve with the largest area under the curve (AUC) varies largely over the different parameter constellations not known in advance.

Hence, we investigated a cross validation procedure based on the maximum rank correlation estimator to determine the optimal selection threshold. This procedure (i) allows to choose an appropriate selection criterion, (ii) provides an estimate of the FDR close to the true FDR and (iii) is simple and computationally feasible also for rather moderate to small sample sizes. Low estimates of the cross validated AUC (the estimates generally being positively biased) and large estimates of the cross validated FDR may indicate a lack of sufficiently prognostic variables and/or too small sample sizes. The method is applied to an example dataset.