Comparative analysis of three risk assessment tools in Australian patients with prostate cancer

In many countries, prognostic tools, which draw on the experience of thousands of patients with cancer, are used to predict cancer outcomes, but accuracy varies. This paper compares the accuracy of three widely used tools predicting prostate cancer recurrence after surgery in Australian patients. The results show that all tools were good at predicting which patients were most likely to experience recurrence and which were least. However, prediction of absolute risk varied and the oldest tool was the most accurate. OBJECTIVE To compare performance of the CAPRA score and two commonly used risk assessment nomograms, the 1998 Kattan and the 2006 Stephenson, in an untested Australian cohort. PATIENTS AND METHODS We present data on 635 men from the South Australian Prostate Cancer Clinical Outcomes Database who underwent radical prostatectomy between January 1996 and May 2009 and had all required variables for predicting biochemical recurrence (BCR). BCR was defined as prostate-specific antigen ≥0.2 ng/mL or secondary treatment for a rising prostate-specific antigen. Accuracy was evaluated using Harrell's concordance index, plotting calibration curves, and constructing decision analysis curves. RESULTS Concordance indices were high for all three tools: 0.791, 0.787 and 0.744 for the 2006 Stephenson nomogram, CAPRA score and 1998 Kattan nomogram respectively. At 3 years, calibration of the tools (agreement between predicted and observed BCR-free probability) was close to ideal for the 1998 Kattan nomogram, whereas the 2006 Stephenson model underestimated and the CAPRA model overestimated BCR-free probability. The 1998 Kattan and 2005 CAPRA tools performed better than the 2006 Stephenson nomogram across a wide range of threshold probabilities using decision curve analysis. CONCLUSION All three tools discriminate between patients' risk effectively. Absolute estimates of risk are likely to vary widely between tools, however, suggesting that models should be validated and, if necessary, recalibrated in the population to which they will be applied. Recent development does not mean a nomogram is more accurate for use in a particular population.