|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES All Classes | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectde.jstacs.classifiers.assessment.ClassifierAssessment
de.jstacs.classifiers.assessment.RepeatedHoldOutExperiment
public class RepeatedHoldOutExperiment
This class implements a repeated hold-out experiment for assessing classifiers. The methodology used by a repeated hold-out experiment is as follows. The user supplies a dataset for each class the classifiers are capable to predict. In each step the given datasets are randomly, mutually exclusive partitioned into a test and a train dataset of user specified size. Afterwards the train datasets are used to train the classifiers and the test datasets are used to assess the performance of the classifiers to predict the elements therein using user specified assessment measures. Additional the user defines how often this procedure is repeated.
Field Summary |
---|
Fields inherited from class de.jstacs.classifiers.assessment.ClassifierAssessment |
---|
myAbstractClassifier, myModel, myTempMeanResultSets, skipLastClassifiersDuringClassifierTraining |
Constructor Summary | |
---|---|
|
RepeatedHoldOutExperiment(AbstractClassifier... aCs)
Creates a new RepeatedHoldOutExperiment from a set of
AbstractClassifier s. |
|
RepeatedHoldOutExperiment(AbstractClassifier[] aCs,
boolean buildClassifiersByCrossProduct,
TrainableStatisticalModel[]... aMs)
This constructor allows to assess a collection of given AbstractClassifier s and those constructed using the given
TrainableStatisticalModel s by a
RepeatedHoldOutExperiment . |
protected |
RepeatedHoldOutExperiment(AbstractClassifier[] aCs,
TrainableStatisticalModel[][] aMs,
boolean buildClassifiersByCrossProduct,
boolean checkAlphabetConsistencyAndLength)
Creates a new RepeatedHoldOutExperiment from an array of
AbstractClassifier s and a two-dimensional array of TrainableStatisticalModel
s, which are combined to additional classifiers. |
|
RepeatedHoldOutExperiment(boolean buildClassifiersByCrossProduct,
TrainableStatisticalModel[]... aMs)
Creates a new RepeatedHoldOutExperiment from a set of
TrainableStatisticalModel s. |
Method Summary | |
---|---|
protected void |
evaluateClassifier(NumericalPerformanceMeasureParameterSet mp,
ClassifierAssessmentAssessParameterSet assessPS,
DataSet[] s,
ProgressUpdater pU)
Evaluates the classifier. |
Methods inherited from class de.jstacs.classifiers.assessment.ClassifierAssessment |
---|
assess, assess, assess, getClassifier, getNameOfAssessment, prepareAssessment, test, train |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
protected RepeatedHoldOutExperiment(AbstractClassifier[] aCs, TrainableStatisticalModel[][] aMs, boolean buildClassifiersByCrossProduct, boolean checkAlphabetConsistencyAndLength) throws IllegalArgumentException, WrongAlphabetException, CloneNotSupportedException, ClassDimensionException
RepeatedHoldOutExperiment
from an array of
AbstractClassifier
s and a two-dimensional array of TrainableStatisticalModel
s, which are combined to additional classifiers. If
buildClassifiersByCrossProduct
is true
, the
cross-product of all TrainableStatisticalModel
s in aMs
is built to
obtain these classifiers.
aCs
- the predefined classifiersaMs
- the TrainableStatisticalModel
s that are used to build additional
classifiersbuildClassifiersByCrossProduct
- Determines how classifiers are constructed using the given
models. Suppose a k-class problem. In this case, each
classifier is supposed to consist of k models, one responsible
for each class. S_i
be the set of all models in
aMs[i]
. Let S
be the set
S_1 x S_2 x ... x S_k
(cross-product).true
: all possible classifiers consisting of a
subset (set of k models) of S
are constructed false
: one classifier consisting of the models
aMs[0][i]
,aMs[1][i]
,...,
aMs[k][i]
for a fixed i
is
constructed. In this case, all second dimensions of
aMs
have to be equal, say m
. In
total m
classifiers are constructed.checkAlphabetConsistencyAndLength
- indicates if alphabets and lengths shall be checked for
consistency
IllegalArgumentException
- if the classifiers have different lengths
WrongAlphabetException
- if the classifiers use different alphabets
CloneNotSupportedException
- if something went wrong while cloning
ClassDimensionException
- if there is something wrong with the class dimension of the
classifierClassifierAssessment.ClassifierAssessment(AbstractClassifier[],
TrainableStatisticalModel[][], boolean, boolean)
public RepeatedHoldOutExperiment(AbstractClassifier... aCs) throws IllegalArgumentException, WrongAlphabetException, CloneNotSupportedException, ClassDimensionException
RepeatedHoldOutExperiment
from a set of
AbstractClassifier
s.
aCs
- contains the classifiers to be assessed.assess( ... )
.s
in order (s[0]
contains foreground data, s[1]
contains
background data)
IllegalArgumentException
- if the classifiers have different lengths
WrongAlphabetException
- if not all given classifiers are defined on the same
AlphabetContainer
CloneNotSupportedException
- if something went wrong while cloning
ClassDimensionException
- if there is something wrong with the class dimension of the
classifierClassifierAssessment.ClassifierAssessment(AbstractClassifier...)
public RepeatedHoldOutExperiment(boolean buildClassifiersByCrossProduct, TrainableStatisticalModel[]... aMs) throws IllegalArgumentException, WrongAlphabetException, CloneNotSupportedException, ClassDimensionException
RepeatedHoldOutExperiment
from a set of
TrainableStatisticalModel
s. The argument buildClassifiersByCrossProduct
determines how these TrainableStatisticalModel
s are combined to classifiers.
buildClassifiersByCrossProduct
- S_i
be the set of all models in
aMs[i]
. Let S
be the set
S_1 x S_2 x ... x S_k
(cross-product).true
: all possible classifiers consisting of a
subset (set of k models) of S
are constructed false
: one classifier consisting of the models
aMs[0][i]
,aMs[1][i]
,...,
aMs[k][i]
for a fixed i
is
constructed. In this case, all second dimensions of
aMs
have to be equal, say m
. In
total m
classifiers are constructed.aMs
- aMs[i]
) contains the
models according to class i
.s
... . s
in order (s[0]
contains foreground data, s[1]
contains
background data)
IllegalArgumentException
- if the classifiers have different lengths
WrongAlphabetException
- if not all given classifiers are defined on the same
AlphabetContainer
CloneNotSupportedException
- if something went wrong while cloning
ClassDimensionException
- if there is something wrong with the class dimension of the
classifierClassifierAssessment.ClassifierAssessment(boolean, TrainableStatisticalModel[][])
public RepeatedHoldOutExperiment(AbstractClassifier[] aCs, boolean buildClassifiersByCrossProduct, TrainableStatisticalModel[]... aMs) throws IllegalArgumentException, WrongAlphabetException, CloneNotSupportedException, ClassDimensionException
AbstractClassifier
s and those constructed using the given
TrainableStatisticalModel
s by a
RepeatedHoldOutExperiment
.
aCs
- contains some AbstractClassifier
that should be
assessed in addition to the AbstractClassifier
s
constructed using the given
TrainableStatisticalModel
sbuildClassifiersByCrossProduct
- S_i
be the set of all models in
aMs[i]
. Let S
be the set
S_1 x S_2 x ... x S_k
(cross-product).true
: all possible classifiers consisting of a
subset (set of k models) of S
are constructed false
: one classifier consisting of the models
aMs[0][i]
,aMs[1][i]
,...,
aMs[k][i]
for a fixed i
is
constructed. In this case, all second dimensions of
aMs
have to be equal, say m
. In
total m
classifiers are constructed.aMs
- aMs[i]
) contains the
models according to class i
.s
... . s
in order (s[0]
contains foreground data, s[1]
contains
background data)
IllegalArgumentException
- if the classifiers have different lengths
WrongAlphabetException
- if not all given classifiers are defined on the same
AlphabetContainer
CloneNotSupportedException
- if something went wrong while cloning
ClassDimensionException
- if there is something wrong with the class dimension of the
classifierClassifierAssessment.ClassifierAssessment(AbstractClassifier[],
boolean, TrainableStatisticalModel[][])
Method Detail |
---|
protected void evaluateClassifier(NumericalPerformanceMeasureParameterSet mp, ClassifierAssessmentAssessParameterSet assessPS, DataSet[] s, ProgressUpdater pU) throws IllegalArgumentException, Exception
evaluateClassifier
in class ClassifierAssessment
mp
- defines which performance measures are used to assess
classifierspU
- A KFoldCrossValidation
is not allowed to be aborted.
The given ProgressUpdater
is never used in this
method.s
- contains the data to be used for assessment. The order of the
samples is important. s
. If
the models are trained directly, the order of given models
during initiation of this assessment object determines, which
sample will be used for training which model. In general the
first model will be trained using the first sample in
s
... . s
in order (s[0]
contains foreground data, s[1]
contains
background data)
assessPS
- contains parameters for a run of this
RepeatedHoldOutExperiment
. Must be of type
RepeatedHoldOutAssessParameterSet
.
IllegalArgumentException
- if the given assessPS
is not of type
RepeatedHoldOutAssessParameterSet
Exception
- if something went wrong
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES All Classes | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |