Generation and evaluation of semi-artificial data

Description

The package semiArtificial contains methods to generate and evaluate semi-artificial data sets. Different data generators take a data set as an input, learn its properties using machine learning algorithms and generates new data with the same properties.

Details

The package currently includes the following data generators:

• a RBF network based generator using rbfDDA model from RSNNS package.

• generator using density tree forest for unsupervised data,

• generator using random forest for classification and regression.

Data evaluation support tools include:

• statistical evaluation: mean, median,standard deviation, skewness, kurtosis, medcouple, L/RMC,

• evaluation based on clustering using Adjusted Rand Index (ARI) and Fowlkes-Mallows index (FM),

• evaluation based on prediction with a model, e.g., random forests.

Further software and development versions are available at http://lkm.fri.uni-lj.si/rmarko/software/.

Author(s)

Marko Robnik-Sikonja

References

Marko Robnik-Sikonja: Not enough data? Generate it!. Technical Report, University of Ljubljana, Faculty of Computer and Information Science, 2014

Other references are available from http://lkm.fri.uni-lj.si/rmarko/papers/

See Also

rbfDataGen, treeEnsemble, newdata, dataSimilarity, dsClustCompare, performanceCompare.

Rejection of new instances based on their distance to existing instances

Description

The function contains three data cleaning methods, the first two reject instances whose distance to their nearest neighbors in the existing data are too small or too large. The first checks distance between instances disregarding class, the second checks distances between instances taking only instances from the same class into account. The third method reassigns response variable using the prediction model stored in the generator teObject.

Usage

cleanData(teObject, newdat, similarDropP=NA, dissimilarDropP=NA, 
          similarDropPclass=NA, dissimilarDropPclass=NA, 
		  nearestInstK=1, reassignResponse=FALSE, cleaningObject=NULL) 

Arguments

Details

The function uses the training instances stored in the generator teObject to compute distribution of distances from instances to their nearestInstK nearest instances. For classification problems the distributions can also be computed only for instances from the same class. Using these near distance distributions the function rejects all instances too close or too far away from existing instances.

The default value of similarDropP, dissimilarDropP, similarDropPclass, and dissimilarDropPclass is NA and means that the near/far values are not rejected. The same effect has value 0 for similarDropP and similarDropPclass, and value 1 for dissimilarDropP and dissimilarDropPclass.

Value

The method returns a list object with two components:

Author(s)

Marko Robnik-Sikonja

See Also

treeEnsemble, newdata.TreeEnsemble.

Examples

# inspect properties of the iris data set
plot(iris, col=iris$Species)
summary(iris)

irisEnsemble<- treeEnsemble(Species~.,iris,noTrees=10)

# use the generator to create new data with the generator
irisNewEns <- newdata(irisEnsemble, size=150)

#inspect properties of the new data
plot(irisNewEns, col = irisNewEns$Species) #plot generated data
summary(irisNewEns)

clObj <- cleanData(irisEnsemble, irisNewEns, similarDropP=0.05, dissimilarDropP=0.95, 
                   similarDropPclass=0.05, dissimilarDropPclass=0.95, 
		           nearestInstK=1, reassignResponse=FALSE, cleaningObject=NULL) 
head(clObj$cleanData)

Evaluate statistical similarity of two data sets

Description

Use mean, standard deviation, skewness, kurtosis, Hellinger distance and KS test to compare similarity of two data sets.

Usage

dataSimilarity(data1, data2, dropDiscrete=NA)

Arguments

Details

The function compares data stored in data1 with data2 on per attribute basis by computing several statistics: mean, standard deviation, skewness, kurtosis, Hellinger distance and KS test.

Value

The method returns a list of statistics computed on both data sets:

Author(s)

Marko Robnik-Sikonja

See Also

newdata.RBFgenerator.

Examples

# use iris data set, split into training and testing data
set.seed(12345)
train <- sample(1:nrow(iris),size=nrow(iris)*0.5)
irisTrain <- iris[train,]
irisTest <- iris[-train,]

# create RBF generator
irisGenerator<- rbfDataGen(Species~.,irisTrain)

# use the generator to create new data
irisNew <- newdata(irisGenerator, size=100)

# compare statistics of original and new data
dataSimilarity(irisTest, irisNew)

Evaluate clustering similarity of two data sets

Description

Similarity of two data sets is compared with a method using any of clustering comparison metrics: Adjusted Rand Index (ARI), Fowlkes-Mallows index(FM), Jaccard Index (J), or Variation of Information index (VI).

Usage

dsClustCompare(data1, data2) 

Arguments

Details

The function compares data stored in data1 with data2 by first performing partitioning around medoids (PAM) clustering on data1. Instances from data2 are than assigned to the cluster with the closest medoid. In second step PAM clustering is performed on data2 and instances from data1 are assigned to the clusters with closest medoids. The procedure gives us two clusterings on the same instances which we can compare using any of ARI, FM, J, or VI. The higher the value of ARI/FM/J the more similar are the two data sets, and reverse is true for VI, where two perfectly matching partitions produce 0 score. For random clustering ARI returns a value around zero (negative values are possible) and for perfectly matching clustering ARI is 1. FM and J values are strictly in [0, 1].

Value

The method returns a value of a list containing ARI and/or FM, depending on the parameters.

Author(s)

Marko Robnik-Sikonja

See Also

newdata.RBFgenerator.

Examples

# use iris data set

# create RBF generator
irisGenerator<- rbfDataGen(Species~.,iris)

# use the generator to create new data
irisNew <- newdata(irisGenerator, size=200)

# compare ARI computed on clustering with original and new data
dsClustCompare(iris, irisNew)

Generate semi-artificial data using a generator

Description

Using a generator build with rbfDataGen or treeEnsemble the method generates size new instances.

Usage

## S3 method for class 'RBFgenerator'
newdata(object, size, var=c("estimated","Silverman"), 
                               classProb=NULL, defaultSpread=0.05, ... )
## S3 method for class 'TreeEnsemble'
newdata(object, fillData=NULL, 
                               size=ifelse(is.null(fillData),1,nrow(fillData)), 
                               onlyPath=FALSE, classProb=NULL, 
                               predictClass=FALSE, ...) 

Arguments

Details

The function uses the object structure as returned by rbfDataGen or treeEnsemble. In case of RBFgenerator the object contains descriptions of the Gaussian kernels, which model the original data. The kernels are used to generate a required number of new instances. The kernel width of provided kernels can be set in two ways. By setting var="estimated" the estimated spread of the training instances that have the maximal activation value for the particular kernel is used. Using var="Silverman" width is set by the generalization of Silverman's rule of thumb to multivariate case (unreliable for larger dimensions).

In case of TreeEnsemble generator no additional parameters are needed, except for the number of generated instances.

Value

The method returns a data.frame object with required number of instances.

Author(s)

Marko Robnik-Sikonja

See Also

rbfDataGen, treeEnsemble.

Examples

# inspect properties of the iris data set
plot(iris, col=iris$Species)
summary(iris)

# create RBF generator
irisRBF<- rbfDataGen(Species~.,iris)
# create treesemble  generator
irisEnsemble<- treeEnsemble(Species~.,iris,noTrees=10)


# use the generator to create new data with both generators
irisNewRBF <- newdata(irisRBF, size=150)
irisNewEns <- newdata(irisEnsemble, size=150)

#inspect properties of the new data
plot(irisNewRBF, col = irisNewRBF$Species) #plot generated data
summary(irisNewRBF)
plot(irisNewEns, col = irisNewEns$Species) #plot generated data
summary(irisNewEns)

Evaluate similarity of two data sets based on predictive performance

Description

Depending on the type of problem (classification or regression), a classification performance (accuracy, AUC, brierScore, etc) or regression performance (RMSE, MSE, MAE, RMAE, etc) on two data sets is used to compare the similarity of two data sets.

Usage

performanceCompare(data1, data2, formula, model="rf", stat=NULL, ...) 

Arguments

Details

The function compares data stored in data1 with data2 by comparing models constructed on data1 and evaluated on both data1 and data2 with models built on data2 and evaluated on both data1 and data2. The difference between these performances are indicative on similarity of the data sets if used in machine learning and data mining. The performance indicator used is determined by parameter stat.

Value

The method returns a list of performance indicators computed on both data sets:

Author(s)

Marko Robnik-Sikonja

See Also

newdata.RBFgenerator.

Examples

# use iris data set

# create RBF generator
irisGenerator<- rbfDataGen(Species~.,iris)

# use the generator to create new data
irisNew <- newdata(irisGenerator, size=200)

# compare statistics of original and new data
performanceCompare(iris, irisNew, Species~.)

A data generator based on RBF network

Description

Using given formula and data the method builds a RBF network and extracts its properties thereby preparing a data generator which can be used with newdata.RBFgenerator method to generate semi-artificial data.

Usage

 rbfDataGen(formula, data, eps=1e-4, minSupport=1, 
            nominal=c("encodeBinary","asInteger")) 

Arguments

Details

Parameter formula is used as a mechanism to select features (attributes) and the prediction variable (response, class). Only simple terms can be used and interaction terms are not supported. The simplest way is to specify just the response variable using e.g. class ~ .. See examples below.

A RBF network is build using rbfDDA from RSNNS package. The learned Gaussian kernels are extracted and used in data generation with newdata.RBFgenerator method.

Value

The created model is returned as a structure of class RBFgenerator, containing the following items:

Author(s)

Marko Robnik-Sikonja

References

Marko Robnik-Sikonja: Not enough data? Generate it!. Technical Report, University of Ljubljana, Faculty of Computer and Information Science, 2014

Other references are available from http://lkm.fri.uni-lj.si/rmarko/papers/

See Also

newdata.RBFgenerator.

Examples

# use iris data set, split into training and testing, inspect the data
set.seed(12345)
train <- sample(1:nrow(iris),size=nrow(iris)*0.5)
irisTrain <- iris[train,]
irisTest <- iris[-train,]

# inspect properties of the original data
plot(irisTrain, col=irisTrain$Species)
summary(irisTrain)

# create rbf generator
irisGenerator<- rbfDataGen(Species~.,irisTrain)

# use the generator to create new data
irisNew <- newdata(irisGenerator, size=200)

#inspect properties of the new data
plot(irisNew, col = irisNew$Species) #plot generated data
summary(irisNew)

A data generator based on forest

Description

Using given formula and data the method treeEnsemble builds a tree ensemble and turns it into a data generator, which can be used with newdata method to generate semi-artificial data. The methods supports classification, regression, and unsupervised data, depending on the input and parameters. The method indAttrGen generates data from the same distribution as the input data, but assuming conditionally independent attributes.

Usage

 treeEnsemble(formula, dataset, noTrees = 100, minNodeWeight=2, noSelectedAttr=0, 
  	problemType=c("byResponse","classification", "regression","density"),
    densityData=c("leaf", "topDown", "bottomUp","no"),
    cdfEstimation = c("ecdf","logspline","kde"), 
	densitySplitMethod=c("balancedSplit","randomSplit","maxVariance"),
    estimator=NULL,...) 

indAttrGen(formula, dataset, cdfEstimation = c("ecdf","logspline","kde"), 
           problemType="byResponse") 

Arguments

Details

Parameter formula is used as a mechanism to select features (attributes) and the prediction variable (response) from the data. Only simple terms can be used and interaction terms are not supported. The simplest way is to specify just the response variable using e.g. class ~ .. For unsupervised problems all variables can be selected using formula ~ .. See examples below.

A forest of trees is build using R code. The base models of the ensemble are classification, regression or density trees with additional information stored at the appropriate nodes. New data can be generated using newdata method.

The method indAttrGen generates data from the same distribution as the input data (provided in dataset), but assumes conditional independence of attributes. This assumption makes the generated data a simple baseline generator. Internally, the method calls treeEnsemble with parameters noTrees=1, minNodeWeight=nrow(dataset), densityData="leaf".

Value

The created model is returned with additional data stored as a list and also in the trees. The model can be used with function newdata to generate new values.

Author(s)

Marko Robnik-Sikonja

References

Marko Robnik-Sikonja: Not enough data? Generate it!. Technical Report, University of Ljubljana, Faculty of Computer and Information Science, 2014

Other references are available from http://lkm.fri.uni-lj.si/rmarko/papers/

See Also

newdata.

Examples

# use iris data set, split into training and testing, inspect the data
set.seed(12345)
train <- sample(1:nrow(iris),size=nrow(iris)*0.5)
irisTrain <- iris[train,]
irisTest <- iris[-train,]

# inspect properties of the original data
plot(iris[,-5], col=iris$Species)
summary(iris)

# create tree ensemble generator for classification problem
irisGenerator<- treeEnsemble(Species~., irisTrain, noTrees=10)

# use the generator to create new data
irisNew <- newdata(irisGenerator, size=200)

#inspect properties of the new data
plot(irisNew[,-5], col = irisNew$Species) # plot generated data
summary(irisNew)

## Not run: 
# create tree ensemble generator for unsupervised problem
irisUnsupervised<- treeEnsemble(~.,irisTrain[,-5], noTrees=10)
irisNewUn <- newdata(irisUnsupervised, size=200)
plot(irisNewUn) # plot generated data
summary(irisNewUn)

# create tree ensemble generator for regression problem
CO2gen<- treeEnsemble(uptake~.,CO2, noTrees=10)
CO2New <- newdata(CO2gen, size=200)
plot(CO2) # plot original data
plot(CO2New) # plot generated data
summary(CO2)
summary(CO2New)

## End(Not run)