class RidgeRegression[MatT <: MatriD, VecT <: VectoD] extends Predictor with Error
The RidgeRegression
class supports multiple linear ridge regression.
In this case, 'x' is multi-dimensional [x_1, ... x_k]. Ridge regression puts
a penalty on the L2 norm of the parameters b to reduce the chance of them taking
on large values that may lead to less robust models. Both the input matrix 'x'
and the response vector 'y' are centered (zero mean). Fit the parameter vector
'b' in the regression equation
y = b dot x + e = b_1 * x_1 + ... b_k * x_k + e
where 'e' represents the residuals (the part not explained by the model). Use Least-Squares (minimizing the residuals) to solve for the parameter vector 'b' using the regularized Normal Equations:
b = fac.solve (.) with regularization x.t * x + λ * I
Five factorization techniques are provided:
'QR' // QR Factorization: slower, more stable (default) 'Cholesky' // Cholesky Factorization: faster, less stable (reasonable choice) 'SVD' // Singular Value Decomposition: slowest, most robust 'LU' // LU Factorization: similar, but better than inverse 'Inverse' // Inverse/Gaussian Elimination, classical textbook technique
- See also
statweb.stanford.edu/~tibs/ElemStatLearn/
- Alphabetic
- By Inheritance
- RidgeRegression
- Error
- Predictor
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
RidgeRegression(x: MatT, y: VecT, lambda_: Double = 0.1, technique: RegTechnique = Cholesky)
- x
the centered input/design m-by-n matrix NOT augmented with a first column of ones
- y
the centered response m-vector
- lambda_
the shrinkage parameter (0 => OLS) in the penalty term 'lambda * b dot b'
- technique
the technique used to solve for b in (x.t*x + lambda*I)*b = x.t*y
Type Members
- type Fac_QR = Fac_QR_H[MatT]
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
val
b: VectoD
- Attributes
- protected
- Definition Classes
- Predictor
-
def
backwardElim(cols: Set[Int]): (Int, VectoD, VectoD)
Perform backward elimination to remove the least predictive variable from the existing model, returning the variable to eliminate, the new parameter vector, the new quality of fit.
Perform backward elimination to remove the least predictive variable from the existing model, returning the variable to eliminate, the new parameter vector, the new quality of fit. May be called repeatedly. FIX - update implementation
- cols
the columns of matrix x to be included in the existing model
-
def
build(x: MatriD, y: VectoD): Predictor
- Definition Classes
- Predictor
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @native() @throws( ... )
-
def
coefficient: VectoD
Return the vector of coefficient/parameter values.
Return the vector of coefficient/parameter values.
- Definition Classes
- Predictor
-
def
diagnose(yy: VectoD): Unit
Compute diagostics for the regression model.
Compute diagostics for the regression model.
- yy
the response vector
- Attributes
- protected
- Definition Classes
- RidgeRegression → Predictor
-
val
e: VectoD
- Attributes
- protected
- Definition Classes
- Predictor
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
eval(yy: VectoD = y): Unit
Compute the error and useful diagnostics.
Compute the error and useful diagnostics.
- yy
the response vector
- Definition Classes
- RidgeRegression → Predictor
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
def
fit: VectoD
Return the quality of fit.
Return the quality of fit.
- Definition Classes
- RidgeRegression → Predictor
-
def
fitLabels: Seq[String]
Return the labels for the fit.
Return the labels for the fit.
- Definition Classes
- RidgeRegression → Predictor
-
final
def
flaw(method: String, message: String): Unit
- Definition Classes
- Error
-
def
forwardSel(cols: Set[Int]): (Int, VectoD, VectoD)
Perform forward selection to add the most predictive variable to the existing model, returning the variable to add, the new parameter vector and the new quality of fit.
Perform forward selection to add the most predictive variable to the existing model, returning the variable to add, the new parameter vector and the new quality of fit. May be called repeatedly.
- cols
the columns of matrix x included in the existing model
-
def
gcv(yy: VectoD): Double
Find an optimal value for the shrinkage parameter 'λ' using Generalized Cross Validation (GCV).
Find an optimal value for the shrinkage parameter 'λ' using Generalized Cross Validation (GCV).
- yy
the response vector
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
val
index_rSq: Int
- Definition Classes
- Predictor
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
val
mae: Double
- Attributes
- protected
- Definition Classes
- Predictor
-
def
metrics: Map[String, Any]
Build a map of selected quality of fit measures/metrics.
Build a map of selected quality of fit measures/metrics.
- Definition Classes
- Predictor
-
val
mse: Double
- Attributes
- protected
- Definition Classes
- Predictor
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
predict(z: VectoD): Double
Predict the value of y = f(z) by evaluating the formula below.
Predict the value of y = f(z) by evaluating the formula below.
- z
the new vector to predict
- Definition Classes
- RidgeRegression → Predictor
-
def
predict(z: VectoI): Double
Given a new discrete data vector z, predict the y-value of f(z).
Given a new discrete data vector z, predict the y-value of f(z).
- z
the vector to use for prediction
- Definition Classes
- Predictor
-
val
rSq: Double
- Attributes
- protected
- Definition Classes
- Predictor
-
def
residual: VectoD
Return the vector of residuals/errors.
Return the vector of residuals/errors.
- Definition Classes
- Predictor
-
val
rmse: Double
- Attributes
- protected
- Definition Classes
- Predictor
-
val
sse: Double
- Attributes
- protected
- Definition Classes
- Predictor
-
val
ssr: Double
- Attributes
- protected
- Definition Classes
- Predictor
-
val
sst: Double
- Attributes
- protected
- Definition Classes
- Predictor
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
train(yy: VectoD = y): RidgeRegression[MatT, VecT]
Train the predictor by fitting the parameter vector (b-vector) in the multiple regression equation
Train the predictor by fitting the parameter vector (b-vector) in the multiple regression equation
yy = b dot x + e = [b_1, ... b_k] dot [x_1, ... x_k] + e
using the least squares method.
- yy
the response vector
- Definition Classes
- RidgeRegression → Predictor
-
def
vif: VectoD
Compute the Variance Inflation Factor 'VIF' for each variable to test for multi-collinearity by regressing 'xj' against the rest of the variables.
Compute the Variance Inflation Factor 'VIF' for each variable to test for multi-collinearity by regressing 'xj' against the rest of the variables. A VIF over 10 indicates that over 90% of the variance of 'xj' can be predicted from the other variables, so 'xj' is a candidate for removal from the model.
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @throws( ... )
-
def
xtx_λI(λ: Double): Unit
Compute x.t * x + λI.
Compute x.t * x + λI.
- λ
the shrinkage parameter