class LogisticRegression extends ClassifierReal
The LogisticRegression class supports (binomial) logistic regression. In this
case, 'x' may be multi-dimensional '[1, x_1, ... x_k]'. Fit the parameter
vector 'b' in the logistic regression equation
logit (p_y) = b dot x + e = b_0 + b_1 * x_1 + ... b_k * x_k + e
where 'e' represents the residuals (the part not explained by the model) and 'y' is now binary.
- See also
see.stanford.edu/materials/lsoeldsee263/05-ls.pdf
- Alphabetic
- By Inheritance
- LogisticRegression
- ClassifierReal
- Error
- Classifier
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
LogisticRegression(x: MatriD, y: VectoI, fn: Array[String] = null, cn: Array[String] = Array ("no", "yes"))
- x
the input/design matrix augmented with a first column of ones
- y
the binary response vector, y_i in {0, 1}
- fn
the names for all features/variables
- cn
the names for both classes
Value Members
-
def
backwardElim(cols: Set[Int]): (Int, VectoD, VectoD)
Perform backward elimination to remove the least predictive variable from the existing model, returning the variable to eliminate, the new parameter vector and the new quality of fit.
Perform backward elimination to remove the least predictive variable from the existing model, returning the variable to eliminate, the new parameter vector and the new quality of fit. May be called repeatedly. FIX - use cols parameter
- cols
the columns of matrix x included in the existing model
-
def
calcCorrelation: MatriD
Calculate the correlation matrix for the feature vectors 'fea'.
Calculate the correlation matrix for the feature vectors 'fea'. If the correlations are too high, the independence assumption may be dubious.
- Definition Classes
- ClassifierReal
-
def
calcCorrelation2(zrg: Range, xrg: Range): MatriD
Calculate the correlation matrix for the feature vectors of Z (Level 3) and those of X (level 2).
Calculate the correlation matrix for the feature vectors of Z (Level 3) and those of X (level 2). If the correlations are too high, the independence assumption may be dubious.
- zrg
the range of Z-columns
- xrg
the range of X-columns
- Definition Classes
- ClassifierReal
-
def
classify(z: VectoD): (Int, String, Double)
Classify the value of 'y = f(z)' by evaluating the formula 'y = sigmoid (b dot z)'.
Classify the value of 'y = f(z)' by evaluating the formula 'y = sigmoid (b dot z)'. Return the best class, its name and quality metric
- z
the new vector to classify
- Definition Classes
- LogisticRegression → Classifier
-
def
classify(xx: MatriD): VectoI
Classify all of the row vectors in matrix 'xx'.
Classify all of the row vectors in matrix 'xx'.
- xx
the row vectors to classify
- Definition Classes
- ClassifierReal
-
def
classify(z: VectoI): (Int, String, Double)
Given a new discrete (integer-valued) data vector 'z', determine which class it belongs to, by first converting it to a vector of doubles.
Given a new discrete (integer-valued) data vector 'z', determine which class it belongs to, by first converting it to a vector of doubles. Return the best class, its name and its relative probability
- z
the vector to classify
- Definition Classes
- ClassifierReal → Classifier
-
def
coefficient: VectoD
Return the vector of coefficient/parameter values.
-
def
crossValidate(nx: Int = 10, show: Boolean = false): Double
Test the accuracy of the classified results by cross-validation, returning the accuracy.
Test the accuracy of the classified results by cross-validation, returning the accuracy. The "test data" starts at 'testStart' and ends at 'testEnd', the rest of the data is "training data'. FIX - should return a StatVector
- nx
the number of crosses and cross-validations (defaults to 10x).
- show
the show flag (show result from each iteration)
- Definition Classes
- Classifier
-
def
crossValidateRand(nx: Int = 10, show: Boolean = false): Double
Test the accuracy of the classified results by cross-validation, returning the accuracy.
Test the accuracy of the classified results by cross-validation, returning the accuracy. This version of cross-validation relies on "subtracting" frequencies from the previously stored global data to achieve efficiency. FIX - are the comments correct? FIX - should return a StatVector
- nx
number of crosses and cross-validations (defaults to 10x).
- show
the show flag (show result from each iteration)
- Definition Classes
- Classifier
-
def
featureSelection(TOL: Double = 0.01): Unit
Perform feature selection on the classifier.
Perform feature selection on the classifier. Use backward elimination technique, that is, remove the least significant feature, in terms of cross- validation accuracy, in each round.
- TOL
tolerance indicating negligible accuracy loss when removing features
- Definition Classes
- ClassifierReal
-
def
fit(y: VectoI, yp: VectoI, k: Int = 2): VectoD
Return the quality of fit.
Return the quality of fit. Assumes both 'train_null' and 'train' have already been called.
- y
the actual class labels
- yp
the predicted class labels
- k
the number of class labels
- Definition Classes
- LogisticRegression → Classifier
-
def
fitLabel: Seq[String]
Return the labels for the fit.
Return the labels for the fit. Override when necessary.
- Definition Classes
- LogisticRegression → Classifier
-
final
def
flaw(method: String, message: String): Unit
- Definition Classes
- Error
-
def
forwardSel(cols: Set[Int]): (Int, VectoD, VectoD)
Perform forward selection to add the most predictive variable to the existing model, returning the variable to add, the new parameter vector and the new quality of fit.
Perform forward selection to add the most predictive variable to the existing model, returning the variable to add, the new parameter vector and the new quality of fit. May be called repeatedly. FIX - implement method
- cols
the columns of matrix x included in the existing model
- val index_prSq: Int
-
def
ll(b: VectoD): Double
For a given parameter vector 'b', compute '-2 * Log-Likelihood (-2l)'.
For a given parameter vector 'b', compute '-2 * Log-Likelihood (-2l)'. '-2l' is the standard measure that follows a Chi-Square distribution.
- b
the parameters to fit
- See also
www.statisticalhorizons.com/wp-content/uploads/Allison.StatComp.pdf
www.stat.cmu.edu/~cshalizi/350/lectures/26/lecture-26.pdf
-
def
ll_null(b: VectoD): Double
For a given parameter vector 'b = [b(0)]', compute '-2 * Log-Likelihood (-2l)'.
For a given parameter vector 'b = [b(0)]', compute '-2 * Log-Likelihood (-2l)'. '-2l' is the standard measure that follows a Chi-Square distribution.
- b
the parameters to fit
- See also
www.statisticalhorizons.com/wp-content/uploads/Allison.StatComp.pdf
www.stat.cmu.edu/~cshalizi/350/lectures/26/lecture-26.pdf
-
def
reset(): Unit
Reset or re-initialize the frequency tables and the probability tables.
Reset or re-initialize the frequency tables and the probability tables.
- Definition Classes
- LogisticRegression → Classifier
-
def
size: Int
Return the number of data vectors in training/test-set (# rows).
Return the number of data vectors in training/test-set (# rows).
- Definition Classes
- ClassifierReal → Classifier
-
def
test(xx: MatriD, yy: VectoI): Double
Test the quality of the training with a test-set and return the fraction of correct classifications.
Test the quality of the training with a test-set and return the fraction of correct classifications.
- xx
the real-valued test vectors stored as rows of a matrix
- yy
the test classification vector, where 'yy_i = class for row i of xx'
- Definition Classes
- ClassifierReal
-
def
test(itest: IndexedSeq[Int]): Double
Test the quality of the training with a test-set and return the fraction of correct classifications.
Test the quality of the training with a test-set and return the fraction of correct classifications.
- itest
indices of the instances considered test data
- Definition Classes
- ClassifierReal → Classifier
-
def
test(testStart: Int, testEnd: Int): Double
Test the quality of the training with a test dataset and return the fraction of correct classifications.
Test the quality of the training with a test dataset and return the fraction of correct classifications. Can be used when the dataset is randomized so that the testing/training part of a dataset corresponds to simple slices of vectors and matrices.
- testStart
the beginning of test region (inclusive).
- testEnd
the end of test region (exclusive).
- Definition Classes
- Classifier
-
def
train(itest: IndexedSeq[Int]): LogisticRegression
For the full model, train the classifier by fitting the parameter vector (b-vector) in the logistic regression equation using maximum likelihood.
For the full model, train the classifier by fitting the parameter vector (b-vector) in the logistic regression equation using maximum likelihood. Do this by minimizing '-2l'. FIX: Use improved BFGS implementation or IRWLS
- itest
the indices of the instances considered as testing data@param itestStart the indices of test test data
- Definition Classes
- LogisticRegression → Classifier
- See also
en.wikipedia.org/wiki/Iteratively_reweighted_least_squares
stats.stackexchange.com/questions/81000/calculate-coefficients-in-a-logistic-regression-with-r
-
def
train(): Classifier
Train the classifier by computing the probabilities from a training dataset of data vectors and their classifications.
Train the classifier by computing the probabilities from a training dataset of data vectors and their classifications. Must be implemented in any extending class. Can be used when the whole dataset is used for training.
- Definition Classes
- Classifier
-
def
train(testStart: Int, testEnd: Int): Classifier
Train the classifier by computing the probabilities from a training dataset of data vectors and their classifications.
Train the classifier by computing the probabilities from a training dataset of data vectors and their classifications. Must be implemented in any extending class. Can be used when the dataset is randomized so that the training part of a dataset corresponds to simple slices of vectors and matrices.
- testStart
starting index of test region (inclusive) used in cross-validation
- testEnd
ending index of test region (exclusive) used in cross-validation
- Definition Classes
- Classifier
-
def
train_null(): Unit
For the null model, train the classifier by fitting the parameter vector (b-vector) in the logistic regression equation using maximum likelihood.
For the null model, train the classifier by fitting the parameter vector (b-vector) in the logistic regression equation using maximum likelihood. Do this by minimizing -2l.
-
def
vc_default: Array[Int]
Return default values for binary input data (value count 'vc' set to 2).
Return default values for binary input data (value count 'vc' set to 2). Also may be used for binning into two categories.
- Definition Classes
- ClassifierReal
-
def
vif: VectoD
Compute the Variance Inflation Factor (VIF) for each variable to test for multi-collinearity by regressing 'xj' against the rest of the variables.
Compute the Variance Inflation Factor (VIF) for each variable to test for multi-collinearity by regressing 'xj' against the rest of the variables. A VIF over 10 indicates that over 90% of the variance of 'xj' can be predicted from the other variables, so 'xj' is a candidate for removal from the model. FIX or remove