Packages

c

scalation.analytics

RidgeRegression

class RidgeRegression[MatT <: MatriD, VecT <: VectoD] extends Predictor with Error

The RidgeRegression class supports multiple linear ridge regression. In this case, 'x' is multi-dimensional [x_1, ... x_k]. Ridge regression puts a penalty on the L2 norm of the parameters b to reduce the chance of them taking on large values that may lead to less robust models. Both the input matrix 'x' and the response vector 'y' are centered (zero mean). Fit the parameter vector 'b' in the regression equation

y = b dot x + e = b_1 * x_1 + ... b_k * x_k + e

where 'e' represents the residuals (the part not explained by the model). Use Least-Squares (minimizing the residuals) to solve for the parameter vector 'b' using the regularized Normal Equations:

b = fac.solve (.) with regularization x.t * x + λ * I

Five factorization techniques are provided:

'QR' // QR Factorization: slower, more stable (default) 'Cholesky' // Cholesky Factorization: faster, less stable (reasonable choice) 'SVD' // Singular Value Decomposition: slowest, most robust 'LU' // LU Factorization: similar, but better than inverse 'Inverse' // Inverse/Gaussian Elimination, classical textbook technique

See also

statweb.stanford.edu/~tibs/ElemStatLearn/

Linear Supertypes
Error, Predictor, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. RidgeRegression
  2. Error
  3. Predictor
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new RidgeRegression(x: MatT, y: VecT, lambda_: Double = 0.1, technique: RegTechnique = Cholesky)

    x

    the centered input/design m-by-n matrix NOT augmented with a first column of ones

    y

    the centered response m-vector

    lambda_

    the shrinkage parameter (0 => OLS) in the penalty term 'lambda * b dot b'

    technique

    the technique used to solve for b in (x.t*x + lambda*I)*b = x.t*y

Type Members

  1. type Fac_QR = Fac_QR_H[MatT]

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. val b: VectoD
    Attributes
    protected
    Definition Classes
    Predictor
  6. def backwardElim(cols: Set[Int]): (Int, VectoD, VectoD)

    Perform backward elimination to remove the least predictive variable from the existing model, returning the variable to eliminate, the new parameter vector, the new quality of fit.

    Perform backward elimination to remove the least predictive variable from the existing model, returning the variable to eliminate, the new parameter vector, the new quality of fit. May be called repeatedly. FIX - update implementation

    cols

    the columns of matrix x to be included in the existing model

  7. def build(x: MatriD, y: VectoD): Predictor
    Definition Classes
    Predictor
  8. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  9. def coefficient: VectoD

    Return the vector of coefficient/parameter values.

    Return the vector of coefficient/parameter values.

    Definition Classes
    Predictor
  10. def diagnose(yy: VectoD): Unit

    Compute diagostics for the regression model.

    Compute diagostics for the regression model.

    yy

    the response vector

    Attributes
    protected
    Definition Classes
    RidgeRegressionPredictor
  11. val e: VectoD
    Attributes
    protected
    Definition Classes
    Predictor
  12. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  14. def eval(yy: VectoD = y): Unit

    Compute the error and useful diagnostics.

    Compute the error and useful diagnostics.

    yy

    the response vector

    Definition Classes
    RidgeRegressionPredictor
  15. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  16. def fit: VectoD

    Return the quality of fit.

    Return the quality of fit.

    Definition Classes
    RidgeRegressionPredictor
  17. def fitLabels: Seq[String]

    Return the labels for the fit.

    Return the labels for the fit.

    Definition Classes
    RidgeRegressionPredictor
  18. final def flaw(method: String, message: String): Unit
    Definition Classes
    Error
  19. def forwardSel(cols: Set[Int]): (Int, VectoD, VectoD)

    Perform forward selection to add the most predictive variable to the existing model, returning the variable to add, the new parameter vector and the new quality of fit.

    Perform forward selection to add the most predictive variable to the existing model, returning the variable to add, the new parameter vector and the new quality of fit. May be called repeatedly.

    cols

    the columns of matrix x included in the existing model

  20. def gcv(yy: VectoD): Double

    Find an optimal value for the shrinkage parameter 'λ' using Generalized Cross Validation (GCV).

    Find an optimal value for the shrinkage parameter 'λ' using Generalized Cross Validation (GCV).

    yy

    the response vector

  21. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  22. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  23. val index_rSq: Int
    Definition Classes
    Predictor
  24. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  25. val mae: Double
    Attributes
    protected
    Definition Classes
    Predictor
  26. def metrics: Map[String, Any]

    Build a map of selected quality of fit measures/metrics.

    Build a map of selected quality of fit measures/metrics.

    Definition Classes
    Predictor
  27. val mse: Double
    Attributes
    protected
    Definition Classes
    Predictor
  28. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  29. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  30. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  31. def predict(z: VectoD): Double

    Predict the value of y = f(z) by evaluating the formula below.

    Predict the value of y = f(z) by evaluating the formula below.

    z

    the new vector to predict

    Definition Classes
    RidgeRegressionPredictor
  32. def predict(z: VectoI): Double

    Given a new discrete data vector z, predict the y-value of f(z).

    Given a new discrete data vector z, predict the y-value of f(z).

    z

    the vector to use for prediction

    Definition Classes
    Predictor
  33. val rSq: Double
    Attributes
    protected
    Definition Classes
    Predictor
  34. def residual: VectoD

    Return the vector of residuals/errors.

    Return the vector of residuals/errors.

    Definition Classes
    Predictor
  35. val rmse: Double
    Attributes
    protected
    Definition Classes
    Predictor
  36. val sse: Double
    Attributes
    protected
    Definition Classes
    Predictor
  37. val ssr: Double
    Attributes
    protected
    Definition Classes
    Predictor
  38. val sst: Double
    Attributes
    protected
    Definition Classes
    Predictor
  39. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  40. def toString(): String
    Definition Classes
    AnyRef → Any
  41. def train(yy: VectoD = y): RidgeRegression[MatT, VecT]

    Train the predictor by fitting the parameter vector (b-vector) in the multiple regression equation

    Train the predictor by fitting the parameter vector (b-vector) in the multiple regression equation

    yy = b dot x + e = [b_1, ... b_k] dot [x_1, ... x_k] + e

    using the least squares method.

    yy

    the response vector

    Definition Classes
    RidgeRegressionPredictor
  42. def vif: VectoD

    Compute the Variance Inflation Factor 'VIF' for each variable to test for multi-collinearity by regressing 'xj' against the rest of the variables.

    Compute the Variance Inflation Factor 'VIF' for each variable to test for multi-collinearity by regressing 'xj' against the rest of the variables. A VIF over 10 indicates that over 90% of the variance of 'xj' can be predicted from the other variables, so 'xj' is a candidate for removal from the model.

  43. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  44. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  45. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  46. def xtx_λI(λ: Double): Unit

    Compute x.t * x + λI.

    Compute x.t * x + λI.

    λ

    the shrinkage parameter

Inherited from Error

Inherited from Predictor

Inherited from AnyRef

Inherited from Any

Ungrouped