getResults-methods {Rmagpie} | R Documentation |
This method provides an easy interface to access the results of one-layer and two-layers of cross-validation directly from an object assessment.
object |
Object of class assessment . Object assessment of interest |
layer |
numeric . Indice that states which layer of cross-validation must be
accessed.
Set to 1 to acces the one-layer cross-validation,
Set to c(1,i) to acces the ith repeat of the one-layer cross-validation,
Set to 2 to acces to the two-layers cross-validation,
Set to c(2,i) to access the ith repeat of the two-layers cross-validation,
Set to c(2,i,j) to access the jth inner layer of ith repeat of the two-layers cross-validation,
Set to c(2,i,j,k) to access the kth repeat of the jth inner layer of ith repeat
of the two-layers cross-validation |
topic |
character. Argument that specifies which kind of
result is requested, the possible values are
"errorRate" : Access to cross-validation error rate, standard error on cross-validated
error rate, error rate per fold, number of samples per fold and error rate per class,
"selectedGenes" : Access to the genes selected for each fold or their frequency of
selection among the folds and the repeats,
"bestOptionValue" : For one-layer of cross-validation, access to the best
option value (size of gene subset for SVM-RFE or thresholds for NSC) corresponding
to the best value of the cross-validated error rate. For the two-layers of
cross-validation, access the average best option value (over the repeats and folds).
"executionTime" : Time used to run the selected layer in seconds. |
errorType |
character. Optional, ignored if topic is not "errorRate" .
Specify the type of error rate requested, the possible values are:
missing or "all" to access all the following error rates
"cv" to access the cross-validated error rate,
"se" to access the standard error on the cross validated error rate,
"fold" to access the error rate per fold (not available in certain cases see
section value for more details),
"noSamplesPerFold" to access the number of samples in each fols (not available in
certain cases see section value for more details),
"class" to acces the error rate per class |
genesType |
character. Optional, ignored if topic is not "selectedGenes" .
Specify the type of display of genes selected, the possible values are:
missing
"fold" to access the genes selected for each fold (not available in certain case see
section value for more details),
"frequ" to access the genes order by their frequency among the folds(not available in certain case see
section value for more details) |
if there is no error, the value returned by the method depends on the arguments namely,
layer
, topic
, errorType
and genesType
.
If layer
is 1
General |
Get the results of the repeated one-layer cross-validation corresponding to
the object of class assessment. If the one-layer cross-validation has not been performed and the user try to access it
then the function return an error indicating that he must call runOneLayerExtCV first. |
if topic is |
|
If errorType= |
All the following error rates |
If errorType= |
numeric . Cross-validated error-rate for each value of option tried
obtained by one-layer of cross-validation (1 value per value of option). |
If errorType= |
numeric . Standard error on cross-validated error-rate for each
value of option tried obtained by one-layer of cross-validation (1 value
per value of option). |
If errorType= |
numeric. Class cross-validated error rate error for each value of option tried obtained by one-layer of cross-validation (1 value per class and value of option). |
Else |
Error signaling that the topic is not appropriate. |
if topic is |
|
If genesType= |
list . Each elelement of the list corresponds to the genes selected for each model
ordered by frequency. |
Else |
Error signaling that the topic is not appropriate. |
if topic is |
Size of subset (for RFE-SVM) or threshold (for NSC) corresponding to the minimum cross-validated error rate. |
if topic is |
Time in second to perform this one-layer cross-validation. |
General |
Get the results of the ith repeat of the one-layer cross-validation corresponding to
the object of class assessment. If the one-layer cross-validation has not been performed and the user try to access it
then the function return an error indicating that he must call runOneLayerExtCV first. |
if topic is |
|
If errorType= |
All the following error rates |
If errorType= |
numeric. Cross-validated error-rate for each value of option tried obtained by one-layer of cross-validation on the ith repeat(1 value per subset). |
If errorType= |
numeric. Standard error on cross-validated error-rate for each value of option tried obtained by one-layer of cross-validation on the ith repeat (1 value per value of option). |
If errorType= |
numeric. Class cross-validated error rate error for each value of option tried obtained by one-layer of cross-validation on the ith repeat (1 value per class and value of option). |
If errorType= |
numeric. Class cross-validated error rate error for each fold and each value of option tried obtained by one-layer of cross-validation on the ith repeat (1 value per class and value of option). |
Else |
Error signaling that the topic is not appropriate. |
if topic is |
|
If genesType= |
list. Each elelement of the list corresponds to the genes selected for each model ordered by frequency. |
If genesType= |
list. Each elelement of the list corresponds to a model and contains a list of which one element correspond to the genes selected in a particular fold. |
Else |
Error signaling that the topic is not appropriate. |
if topic is |
numeric. Size of subset (for RFE) or threshold (for NSC) corresponding to the minimum cross-validated error rate in the ith repeat of the one-layer cross-validation. |
if topic is |
Time in second to perform this repeat of one-layer cross-validation. |
General |
Get the results of the repeated two-layers cross-validation corresponding to
the object of class assessment. If the two-layer cross-validation has not been performed and the user try to access it
then the function return an error indicating that he must call runTwoLayerExtCV first. |
if topic is 'errorRate' |
|
If errorType= |
All the following error rates |
If errorType= |
numeric. Cross-validated error-rate obtained by two-layers of cross-validation (1 value). |
If errorType= |
numeric. Standard error on cross-validated error-rate obtained by two-layers of cross-validation (1 value). |
If errorType= |
numeric. Class cross-validated error rate obtained by two-layers (1 value per class) |
Else |
Error signaling that the topic is not appropriate. |
if topic is |
numeric. Average best number of genes for SVM-RFE of threshold for NSc obtained among the folds. |
if topic is |
Time in second to perform this two-layers cross-validation. |
General |
Get the results of the ith repeated of the two-layers cross-validation corresponding to
the object of class assessment. If the two-layer cross-validation has not been performed and the user try to access it
then the function return an error indicating that he must call runTwoLayerExtCV first. |
if topic is 'errorRate' |
|
If errorType= |
All the following error rates |
If errorType= |
numeric. Cross-validated error-rate obtained by two-layers of cross-validation in this repeat. (1 value). |
If errorType= |
numeric. Standard error on cross-validated error-rate obtained by two-layers of cross-validation in this repeat (1 value). |
If errorType= |
numeric. Class cross-validated error rate obtained by two-layers in this repeat |
If errorType= |
numeric. Error rate obtained on each of the folds in the second layer in this repeat(1 value per fold). of cross-validation (value per class). |
Else |
Error signaling that the topic is not appropriate. |
if topic is |
|
If genesType= |
list. Each elelement of the list corresponds to a fold and contains a list of the genes selected in this particular fold. |
Else |
Error signaling that the topic is not appropriate. |
if topic is |
numeric. Average best number of genes obtained among the folds in this repeat. |
if topic is |
Time in second to perform this repeat of two-layers cross-validation. |
If |
This layer corresponds to the jth inner layer of one-layer cross-validation performed inside the ith repeat of the two-layers cross-validation. The returned values are similar to the one returned by a repeated one-layer cross-validation. |
If |
This layer corresponds to the kth repeat of the jth inner layer of one-layer cross-validation performed inside the ith repeat. The returned values are similar to the one returned by a repeat of one-layer cross-validation. |
Camille Maumet
#dataPath <- file.path("C:", "Documents and Settings", "c.maumet", "My Documents", "Programmation", "Sources", "SVN", "R package", "data") #aDataset <- new("dataset", dataId="vantVeer_70", dataPath=dataPath) #aDataset <- loadData(aDataset) data('vV70genesDataset') mySubsets <- new("geneSubsets", optionValues=c(1,2,4,8,16,32,64,70)) myassessment <- new("assessment", dataset=vV70genes, noFolds1stLayer=5, noFolds2ndLayer=4, classifierName="svm", typeFoldCreation="original", svmKernel="linear", noOfRepeat=2, featureSelectionOptions=mySubsets) myassessment <- runOneLayerExtCV(myassessment) myassessment <- runTwoLayerExtCV(myassessment) # --- Access to one-layer CV --- # errorRate # 1-layer CV: error Rates getResults(myassessment, 1, 'errorRate') # 1-layer CV: error Rates - all") getResults(myassessment, 1, 'errorRate', errorType='all') # 1-layer CV: error Rates - cv getResults(myassessment, 1, 'errorRate', errorType='cv') # 1-layer CV: error Rates - se getResults(myassessment, 1, 'errorRate', errorType='se') # 1-layer CV: error Rates - class getResults(myassessment, 1, 'errorRate', errorType='class') # genesSelected # 1-layer CV: genes Selected getResults(myassessment, 1, 'genesSelected') # 1-layer CV: genes Selected - frequ getResults(myassessment, 1, 'genesSelected', genesType='frequ') # 1-layer CV: genes Selected - model 7 getResults(myassessment, 1, 'genesSelected', genesType='frequ')[[7]] getResults(myassessment, 1, 'genesSelected')[[7]] # bestOptionValue # 1-layer CV: best number of genes getResults(myassessment, 1, 'bestOptionValue') # executionTime # 1-layer CV: execution time getResults(myassessment, 1, 'executionTime') # --- Access to 2nd repeat of one-layer CV --- # Error rates # 1-layer CV repeat 2: error Rates getResults(myassessment, c(1,2), 'errorRate') # 1-layer CV repeat 2: error Rates - all getResults(myassessment, c(1,2), 'errorRate', errorType='all') # 1-layer CV repeat 2: error Rates - cv getResults(myassessment, c(1,2), 'errorRate', errorType='cv') # 1-layer CV repeat 2: error Rates - se getResults(myassessment, c(1,2), 'errorRate', errorType='se') # 1-layer CV repeat 2: error Rates - fold getResults(myassessment, c(1,2), 'errorRate', errorType='fold') # 1-layer CV repeat 2: error Rates - noSamplesPerFold getResults(myassessment, c(1,2), 'errorRate', errorType='noSamplesPerFold') # 1-layer CV repeat 2: error Rates - class getResults(myassessment, c(1,2), 'errorRate', errorType='class') # genesSelected # 1-layer CV repeat 2: genes Selected getResults(myassessment, c(1,2), 'genesSelected') # 1-layer CV repeat 2: genes Selected - frequ getResults(myassessment, c(1,2), 'genesSelected', genesType='frequ') # 1-layer CV repeat 2: genes Selected - model 7 (twice) getResults(myassessment, c(1,2), 'genesSelected', genesType='frequ')[[7]] getResults(myassessment, c(1,2), 'genesSelected')[[7]] # 1-layer CV repeat 2: genes Selected - fold getResults(myassessment, c(1,2), 'genesSelected', genesType='fold') # 1-layer CV repeat 2: best number of genes getResults(myassessment, c(1,2), 'bestOptionValue') # 1-layer CV repeat 2: execution time getResults(myassessment, c(1,2), 'executionTime') # --- Access to two-layers CV --- # Error rates # 2-layer CV: error Rates getResults(myassessment, 2, 'errorRate') # 2-layer CV: error Rates - all getResults(myassessment, 2, 'errorRate', errorType='all') # 2-layer CV: error Rates - cv getResults(myassessment, 2, 'errorRate', errorType='cv') # 2-layer CV: error Rates - se getResults(myassessment, 2, 'errorRate', errorType='se') # 2-layer CV: error Rates - class getResults(myassessment, 2, 'errorRate', errorType='class') # bestOptionValue # 2-layer CV: best number of genes (avg) getResults(myassessment, 2, 'bestOptionValue') # executionTime # 2-layer CV: execution time getResults(myassessment, 2, 'executionTime') # --- Access to two-layers CV access to repeats --- # Error rates # 2-layer CV repeat 1: error Rates getResults(myassessment, c(2,1), 'errorRate') # 2-layer CV repeat 1: error Rates - all getResults(myassessment, c(2,1), 'errorRate', errorType='all') # 2-layer CV repeat 1: error Rates - cv getResults(myassessment, c(2,1), 'errorRate', errorType='cv') # 2-layer CV repeat 1: error Rates - se getResults(myassessment, c(2,1), 'errorRate', errorType='se') # 2-layer CV repeat 1: error Rates - fold getResults(myassessment, c(2,1), 'errorRate', errorType='fold') # 2-layer CV repeat 1: error Rates - noSamplesPerFold getResults(myassessment, c(2,1), 'errorRate', errorType='noSamplesPerFold') # 2-layer CV repeat 1: error Rates - class getResults(myassessment, c(2,1), 'errorRate', errorType='class') # genesSelected # 2-layer CV repeat 1: genes Selected getResults(myassessment, c(2,1), 'genesSelected') # 2-layer CV repeat 1: genes Selected - fold getResults(myassessment, c(2,1), 'genesSelected', genesType='fold') # 2-layer CV repeat 1: best number of genes getResults(myassessment, c(2,1), 'bestOptionValue') # 2-layer CV repeat 1: execution time getResults(myassessment, c(2,1), 'executionTime') # --- Access to one-layer CV inside two-layers CV --- # errorRate # 2-layer CV repeat 1 inner layer 3: error Rates getResults(myassessment, c(2,1,3), 'errorRate') # 2-layer CV repeat 1 inner layer 3: error Rates - all getResults(myassessment, c(2,1,3), 'errorRate', errorType='all') # 2-layer CV repeat 1 inner layer 3: error Rates - cv getResults(myassessment, c(2,1,3), 'errorRate', errorType='cv') # 2-layer CV repeat 1 inner layer 3: error Rates - se getResults(myassessment, c(2,1,3), 'errorRate', errorType='se') # 2-layer CV repeat 1 inner layer 3: error Rates - class getResults(myassessment, c(2,1,3), 'errorRate', errorType='class') # genesSelected # 2-layer CV repeat 1 inner layer 3: genes Selected getResults(myassessment, c(2,1,3), 'genesSelected') # 2-layer CV repeat 1 inner layer 3: genes Selected - frequ getResults(myassessment, c(2,1,3), 'genesSelected', genesType='frequ') # 2-layer CV repeat 1 inner layer 3: genes Selected - model 7 getResults(myassessment, c(2,1,3), 'genesSelected', genesType='frequ')[[7]] getResults(myassessment, c(2,1,3), 'genesSelected')[[7]] # bestOptionValue # 2-layer CV repeat 1 inner layer 3: best number of genes getResults(myassessment, c(2,1,3), 'bestOptionValue') # executionTime # 2-layer CV repeat 1 inner layer 3: execution time getResults(myassessment, c(2,1,3), 'executionTime') # --- two-layers CV access to repeat 1, inner layer 2 repeat 2 --- # Error rates # 2-layer CV inner layer 3 repeat 2: error Rates getResults(myassessment, c(2,1,3,1), 'errorRate') # 2-layer CV repeat 1 inner layer 3 repeat 1: error Rates - all getResults(myassessment, c(2,1,3,1), 'errorRate', errorType='all') # 2-layer CV repeat 1 inner layer 3 repeat 1: error Rates - cv getResults(myassessment, c(2,1,3,1), 'errorRate', errorType='cv') # 2-layer CV repeat 1 inner layer 3 repeat 1: error Rates - se getResults(myassessment, c(2,1,3,1), 'errorRate', errorType='se') # 2-layer CV repeat 1 inner layer 3 repeat 1: error Rates - class getResults(myassessment, c(2,1,3,1), 'errorRate', errorType='class') # 2-layer CV repeat 1 inner layer 3 repeat 1: error Rates - fold getResults(myassessment, c(2,1,3,1), 'errorRate', errorType='fold') # 2-layer CV repeat 1 inner layer 3 repeat 1: error Rates - noSamplesPerFold getResults(myassessment, c(2,1,3,1), 'errorRate', errorType='noSamplesPerFold') # genesSelected # 2-layer CV repeat 1 inner layer 3 repeat 1: genes Selected getResults(myassessment, c(2,1,3,1), 'genesSelected') # 2-layer CV repeat 1 inner layer 3 repeat 1: genes Selected - fold getResults(myassessment, c(2,1,3,1), 'genesSelected', genesType='fold') # 2-layer CV repeat 1 inner layer 3 repeat 1: genes Selected - model 3 fold 1(twice) getResults(myassessment, c(2,1,3,1), 'genesSelected', genesType='fold')[[3]][[1]] # 2-layer CV repeat 1 inner layer 3 repeat 1: genes Selected frequ - model 3 getResults(myassessment, c(2,1,3,1), 'genesSelected')[[3]] # 2-layer CV repeat 1 inner layer 3 repeat 1: best number of genes getResults(myassessment, c(2,1,3,1), 'bestOptionValue') # 2-layer CV repeat 1 inner layer 3 repeat 1: execution time getResults(myassessment, c(2,1,3,1), 'executionTime')