class StochasticGradient extends Minimizer with Error
The StochasticGradient class solves unconstrained Non-Linear Programming (NLP)
problems using the Stochastic Gradient Descent algorithm. Given a function 'f'
and a starting point 'x0', the algorithm computes the gradient and takes steps
in the opposite direction. The algorithm iterates until it converges. The
algorithm is stochastic in sense that only a single batch is used in each step
of the optimimation. Examples (a number of rows) are are chosen for each batch.
FIX - provide option to randomly select samples in batch
- See also
leon.bottou.org/publications/pdf/compstat-2010.pdf dir_k = -gradient (x) minimize f(x)
- Alphabetic
- By Inheritance
- StochasticGradient
- Error
- Minimizer
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
StochasticGradient(fxy: (MatrixD, VectorD, VectorD) ⇒ Double, dx: MatrixD, dy: VectorD, batch: Int = 10, exactLS: Boolean = true)
- dx
the data matrix
- dy
the response vector
- batch
the batch size
- exactLS
whether to use exact (e.g.,
GoldenLS) or inexact (e.g.,WolfeLS) Line Search
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
val
EPSILON: Double
- Attributes
- protected
- Definition Classes
- Minimizer
-
val
MAX_ITER: Int
- Attributes
- protected
- Definition Classes
- Minimizer
-
val
STEP: Double
- Attributes
- protected
- Definition Classes
- Minimizer
-
val
TOL: Double
- Attributes
- protected
- Definition Classes
- Minimizer
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
f(x: VectorD): Double
The objective function for the ith batch.
The objective function for the ith batch.
- x
the vector to optimize (e.g., model parameters)
-
def
fg(x: VectorD): Double
The objective function 'f' plus a weighted penalty based on the constraint function 'g'.
The objective function 'f' plus a weighted penalty based on the constraint function 'g'. Override for constrained optimization and ignore for unconstrained optimization.
- x
the coordinate values of the current point
- Definition Classes
- Minimizer
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
flaw(method: String, message: String): Unit
- Definition Classes
- Error
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
lineSearch(x: VectorD, dir: VectorD, step: Double = STEP): Double
Perform an exact 'GoldenSectionLS' or inexact 'WolfeLS' line search.
Perform an exact 'GoldenSectionLS' or inexact 'WolfeLS' line search. Search in direction 'dir', returning the distance 'z' to move in that direction.
- x
the current point
- dir
the direction to move in
- step
the initial step size
- Definition Classes
- StochasticGradient → Minimizer
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
-
def
solve(x0: VectorD, step: Double = STEP, toler: Double = EPSILON): VectorD
Solve the Non-Linear Programming (NLP) problem using the Stochastic Gradient Descent algorithm.
Solve the Non-Linear Programming (NLP) problem using the Stochastic Gradient Descent algorithm.
- x0
the starting point
- step
the initial step size
- toler
the tolerance
- Definition Classes
- StochasticGradient → Minimizer
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )