c

scalation.minima

StochasticGradient

class StochasticGradient extends Minimizer with Error

The StochasticGradient class solves unconstrained Non-Linear Programming (NLP) problems using the Stochastic Gradient Descent algorithm. Given a function 'f' and a starting point 'x0', the algorithm computes the gradient and takes steps in the opposite direction. The algorithm iterates until it converges. The algorithm is stochastic in sense that only a single batch is used in each step of the optimimation. Examples (a number of rows) are are chosen for each batch. FIX - provide option to randomly select samples in batch

See also

leon.bottou.org/publications/pdf/compstat-2010.pdf dir_k = -gradient (x) minimize f(x)

Linear Supertypes
Error, Minimizer, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. StochasticGradient
  2. Error
  3. Minimizer
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new StochasticGradient(fxy: (MatrixD, VectorD, VectorD) ⇒ Double, dx: MatrixD, dy: VectorD, batch: Int = 10, exactLS: Boolean = true)

    dx

    the data matrix

    dy

    the response vector

    batch

    the batch size

    exactLS

    whether to use exact (e.g., GoldenLS) or inexact (e.g., WolfeLS) Line Search

Type Members

  1. type Pair = (VectorD, VectorD)
    Definition Classes
    Minimizer

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. val EPSILON: Double
    Attributes
    protected
    Definition Classes
    Minimizer
  5. val MAX_ITER: Int
    Attributes
    protected
    Definition Classes
    Minimizer
  6. val STEP: Double
    Attributes
    protected
    Definition Classes
    Minimizer
  7. val TOL: Double
    Attributes
    protected
    Definition Classes
    Minimizer
  8. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  9. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  10. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  11. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  12. def f(x: VectorD): Double

    The objective function for the ith batch.

    The objective function for the ith batch.

    x

    the vector to optimize (e.g., model parameters)

  13. def fg(x: VectorD): Double

    The objective function 'f' plus a weighted penalty based on the constraint function 'g'.

    The objective function 'f' plus a weighted penalty based on the constraint function 'g'. Override for constrained optimization and ignore for unconstrained optimization.

    x

    the coordinate values of the current point

    Definition Classes
    Minimizer
  14. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  15. final def flaw(method: String, message: String): Unit
    Definition Classes
    Error
  16. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  17. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  18. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  19. def lineSearch(x: VectorD, dir: VectorD, step: Double = STEP): Double

    Perform an exact 'GoldenSectionLS' or inexact 'WolfeLS' line search.

    Perform an exact 'GoldenSectionLS' or inexact 'WolfeLS' line search. Search in direction 'dir', returning the distance 'z' to move in that direction.

    x

    the current point

    dir

    the direction to move in

    step

    the initial step size

    Definition Classes
    StochasticGradient → Minimizer
  20. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  21. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  23. def solve(x0: VectorD, step: Double = STEP, toler: Double = EPSILON): VectorD

    Solve the Non-Linear Programming (NLP) problem using the Stochastic Gradient Descent algorithm.

    Solve the Non-Linear Programming (NLP) problem using the Stochastic Gradient Descent algorithm.

    x0

    the starting point

    step

    the initial step size

    toler

    the tolerance

    Definition Classes
    StochasticGradient → Minimizer
  24. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  25. def toString(): String
    Definition Classes
    AnyRef → Any
  26. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )

Inherited from Error

Inherited from Minimizer

Inherited from AnyRef

Inherited from Any

Ungrouped