class StochasticGradient extends Minimizer with Error
The StochasticGradient class solves unconstrained Non-Linear Programming (NLP)
problems using the Stochastic Gradient Descent algorithm. Given a function 'f'
and a starting point 'x0', the algorithm computes the gradient and takes steps
in the opposite direction. The algorithm iterates until it converges. The
algorithm is stochastic in sense that only a single batch is used in each step
of the optimimation. Examples (a number of rows) are are chosen for each batch.
FIX - provide option to randomly select samples in batch
- See also
leon.bottou.org/publications/pdf/compstat-2010.pdf dir_k = -gradient (x) minimize f(x)
- Alphabetic
- By Inheritance
- StochasticGradient
- Error
- Minimizer
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new StochasticGradient(fxy: (MatrixD, VectorD, VectorD) => Double, dx: MatrixD, dy: VectorD, batch: Int = 10, exactLS: Boolean = true)
- dx
the data matrix
- dy
the response vector
- batch
the batch size
- exactLS
whether to use exact (e.g.,
GoldenLS) or inexact (e.g.,WolfeLS) Line Search
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- val EPSILON: Double
- Attributes
- protected
- Definition Classes
- Minimizer
- val MAX_ITER: Int
- Attributes
- protected
- Definition Classes
- Minimizer
- val STEP: Double
- Attributes
- protected
- Definition Classes
- Minimizer
- val TOL: Double
- Attributes
- protected
- Definition Classes
- Minimizer
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native() @HotSpotIntrinsicCandidate()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def f(x: VectorD): Double
The objective function for the ith batch.
The objective function for the ith batch.
- x
the vector to optimize (e.g., model parameters)
- def fg(x: VectorD): Double
The objective function 'f' plus a weighted penalty based on the constraint function 'g'.
The objective function 'f' plus a weighted penalty based on the constraint function 'g'. Override for constrained optimization and ignore for unconstrained optimization.
- x
the coordinate values of the current point
- Definition Classes
- Minimizer
- final def flaw(method: String, message: String): Unit
- Definition Classes
- Error
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def lineSearch(x: VectorD, dir: VectorD, step: Double = STEP): Double
Perform an exact 'GoldenSectionLS' or inexact 'WolfeLS' line search.
Perform an exact 'GoldenSectionLS' or inexact 'WolfeLS' line search. Search in direction 'dir', returning the distance 'z' to move in that direction.
- x
the current point
- dir
the direction to move in
- step
the initial step size
- Definition Classes
- StochasticGradient → Minimizer
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- def solve(x0: VectorD, step: Double = STEP, toler: Double = EPSILON): VectorD
Solve the Non-Linear Programming (NLP) problem using the Stochastic Gradient Descent algorithm.
Solve the Non-Linear Programming (NLP) problem using the Stochastic Gradient Descent algorithm.
- x0
the starting point
- step
the initial step size
- toler
the tolerance
- Definition Classes
- StochasticGradient → Minimizer
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated