The Optimizer trait provides methods to optimize and auto_optimize parameters. Given training data x and y for a Neural Network, fit the parameters b.
Attributes
- Companion
- object
- Graph
-
- Supertypes
- Known subtypes
Members list
- Not inherited
- MonitorLoss
- StoppingRule
- public
- protected
Value members
Abstract methods
Given training data x and y for a Neural Network, fit the parameters b, returning the value of the lose function and the number of epochs.
Given training data x and y for a Neural Network, fit the parameters b, returning the value of the lose function and the number of epochs.
Value parameters
- b
-
the array of parameters (weights & biases) between every two adjacent layers
- etaI
-
the lower and upper bounds of learning/convergence rate
- f
-
the array of activation function family for every two adjacent layers
- x
-
the m-by-n input matrix (training data consisting of m input vectors)
- y
-
the m-by-ny output matrix (training data consisting of m output vectors)
Attributes
Concrete methods
Given training data x and y for a Neural Network, fit the parameters b, returning the value of the lose function and the number of epochs. Find the best learning rate within the interval etaI.
Given training data x and y for a Neural Network, fit the parameters b, returning the value of the lose function and the number of epochs. Find the best learning rate within the interval etaI.
Value parameters
- b
-
the array of parameters (weights & biases) between every two adjacent layers
- etaI
-
the lower and upper bounds of learning/convergence rate
- f
-
the array of activation function family for every two adjacent layers
- opti
-
the array of activation function family for every two adjacent layers
- x
-
the m-by-n input matrix (training data consisting of m input vectors)
- y
-
the m-by-ny output matrix (training data consisting of m output vectors)
Attributes
Freeze layer flayer during back-propogation (should only impact the optimize method in the classes extending this trait). FIX: make abstract (remove ???) and implement in extending classes
Freeze layer flayer during back-propogation (should only impact the optimize method in the classes extending this trait). FIX: make abstract (remove ???) and implement in extending classes
Value parameters
- flayer
-
the layer to freeze, e.g., 1 => first hidden layer
Attributes
Return a permutation vector generator that will provide a random permutation of index positions for each call permGen.igen (e.g., used to select random batches).
Return a permutation vector generator that will provide a random permutation of index positions for each call permGen.igen (e.g., used to select random batches).
Value parameters
- m
-
the number of data instances
- rando
-
whether to use a random or fixed random number stream
Attributes
Inherited methods
Collect the next value for the loss function.
Collect the next value for the loss function.
Value parameters
- loss
-
the value of the loss function
Attributes
- Inherited from:
- MonitorLoss
Plot the loss function versus the epoch/major iterations.
Plot the loss function versus the epoch/major iterations.
Value parameters
- optName
-
the name of optimization algorithm (alt. name of network)
Attributes
- Inherited from:
- MonitorLoss
Stop when too many steps have the cost measure (e.g., sse) increasing. Signal a stopping condition by returning the best parameter vector, else null.
Stop when too many steps have the cost measure (e.g., sse) increasing. Signal a stopping condition by returning the best parameter vector, else null.
Value parameters
- b
-
the current parameter value (weights and biases)
- sse
-
the current value of cost measure (e.g., sum of squared errors)
Attributes
- Inherited from:
- StoppingRule
Stop when too many steps have the cost measure (e.g., sse) increasing. Signal a stopping condition by returning the best parameter vector, else null.
Stop when too many steps have the cost measure (e.g., sse) increasing. Signal a stopping condition by returning the best parameter vector, else null.
Value parameters
- b
-
the current value of the parameter vector
- sse
-
the current value of cost measure (e.g., sum of squared errors)
Attributes
- Inherited from:
- StoppingRule
Inherited fields
Attributes
- Inherited from:
- StoppingRule