This function takes the correct labels and predictions for all samples and evaluates the results using the
Area Under the Receiver Operating Characteristic (ROC) Curve (AU-ROC)
and the Precision-Recall Curve (PR)
as metric. Predictions can be supplied either for a single case or as matrix after resampling of the dataset.
Prediction results are usually produced with the function make.predictions.
evaluate.predictions(siamcat, verbose = 1)
siamcat | object of class siamcat-class |
---|---|
verbose | control output: |
object of class siamcat-class with the
slot eval_data
filled
This functions calculates for the predictions in the
pred_matrix
-slot of the siamcat-class-object several
metrices. The Area Under the Receiver Operating Characteristic (ROC)
Curve (AU-ROC) and the Precision-Recall Curve will be evaluated and
the results will be saved in the eval_data
-slot of the
supplied siamcat-class-object. The eval_data
-slot
contains a list with several entries:
$roc.average
average ROC-curve across repeats or a
single ROC-curve on complete dataset;
$auc.average
AUC value for the average ROC-curve;
$ev.list
list of length(num.folds)
, containing
for different decision thresholds the number of false
positives, false negatives, true negatives, and
true positives;
$pr.list
list of length(num.folds)
, containing
the positive predictive value (precision) and true positive
rate (recall) values used to plot the PR curves.
For the case of repeated cross-validation, the function will additonally return
$roc.all
list of roc objects (see roc)
for every repeat;
$aucspr
vector of AUC values for the PR curves
for every repeat;
$auc.all
vector of AUC values for the ROC curves
for every repeat.
data(siamcat_example) # simple working example siamcat_evaluated <- evaluate.predictions(siamcat_example)#> Error in label$label: $ operator not defined for this S4 class