class NeuralNet_XL extends PredictorMat2
The NeuralNet_XL class supports multi-output, multi-layer (input, multiple hidden and output)
Neural-Networks. It can be used for both classification and prediction,
depending on the activation functions used. Given several input vectors and output
vectors (training data), fit the weight and bias parameters connecting the layers,
so that for a new input vector 'v', the net can predict the output value.
Defaults to two hidden layers.
This implementation is partially adapted from Michael Nielsen's Python implementation found in
- See also
github.com/mnielsen/neural-networks-and-deep-learning/blob/master/src/network2.py
github.com/MichalDanielDobrzanski/DeepLearningPython35/blob/master/network2.py ------------------------------------------------------------------------------
- Alphabetic
- By Inheritance
- NeuralNet_XL
- PredictorMat2
- Predictor
- Model
- Error
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
NeuralNet_XL(x: MatriD, y: MatriD, nz: Array[Int] = null, fname_: Strings = null, hparam: HyperParameter = Optimizer.hp, f: Array[AFF] = Array (f_tanh, f_tanh, f_id), itran: FunctionV_2V = null)
- x
the m-by-nx data/input matrix (training data having m input vectors)
- y
the m-by-ny response/output matrix (training data having m output vectors)
- nz
the number of nodes in each hidden layer, e.g., Array (9, 8) => 2 hidden of sizes 9 and 8
- fname_
the feature/variable names (if null, use x_j's)
- hparam
the hyper-parameters for the model/network
- f
the array of activation function families between every pair of layers
- itran
the inverse transformation function returns responses to original scale
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
val
_1: VectorD
- Attributes
- protected
- Definition Classes
- PredictorMat2
-
def
analyze(x_: MatriD = x, y_: VectoD = y(0), x_e: MatriD = x, y_e: VectoD = y(0)): PredictorMat2
Analyze a dataset using this model using ordinary training with the 'train' method.
Analyze a dataset using this model using ordinary training with the 'train' method. Only uses the first output variable's value.
- x_
the data/input matrix (training/full)
- y_
the response/output vector (training/full)
- x_e
the data/input matrix (testing/full)
- y_e
the response/output vector (testing/full)
- Definition Classes
- PredictorMat2 → Predictor
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
var
b: IndexedSeq[NetParam]
- Attributes
- protected
-
def
backwardElim(cols: Set[Int], index_q: Int = index_rSqBar, first: Int = 1): (Int, PredictorMat2)
Perform backward elimination to find the least predictive variable to remove from the existing model, returning the variable to eliminate, the new parameter vector and the new Quality of Fit (QoF).
Perform backward elimination to find the least predictive variable to remove from the existing model, returning the variable to eliminate, the new parameter vector and the new Quality of Fit (QoF). May be called repeatedly.
- cols
the columns of matrix x currently included in the existing model
- index_q
index of Quality of Fit (QoF) to use for comparing quality
- first
first variable to consider for elimination (default (1) assume intercept x_0 will be in any model)
- Definition Classes
- PredictorMat2
- See also
Fitfor index of QoF measures.
-
def
backwardElimAll(index_q: Int = index_rSqBar, first: Int = 1, cross: Boolean = true): (Set[Int], MatriD)
Perform forward selection to find the most predictive variables to have in the model, returning the variables added and the new Quality of Fit (QoF) measures for all steps.
Perform forward selection to find the most predictive variables to have in the model, returning the variables added and the new Quality of Fit (QoF) measures for all steps.
- index_q
index of Quality of Fit (QoF) to use for comparing quality
- first
first variable to consider for elimination
- cross
whether to include the cross-validation QoF measure
- Definition Classes
- PredictorMat2
- See also
Fitfor index of QoF measures.
-
def
buildModel(x_cols: MatriD): NeuralNet_XL
Build a sub-model that is restricted to the given columns of the data matrix.
Build a sub-model that is restricted to the given columns of the data matrix.
- x_cols
the columns that the new model is restricted to
- Definition Classes
- NeuralNet_XL → PredictorMat2
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @HotSpotIntrinsicCandidate()
-
def
compute_df_m(n: Array[Int]): Int
Compute the degrees of freedom for the model (based on nx, n's, ny = 1).
Compute the degrees of freedom for the model (based on nx, n's, ny = 1). Rough extimate based on total number of parameters - 1. FIX: use better estimate
- n
the number of nodes in each hidden layer
-
def
compute_nz(nx: Int, ny: Int): Array[Int]
Compute default values for the number nodes in each hidden layer, based on the number of nodes in the input and output layers using average of prior layer and output layer rule.
Compute default values for the number nodes in each hidden layer, based on the number of nodes in the input and output layers using average of prior layer and output layer rule. Rule [2] (nx + ny) / 2, (nx + 3ny) / 4, ...
- nx
the number of nodes in the input layer
- ny
the number of nodes in the output layer
-
def
compute_nz(nx: Int): Array[Int]
Compute default values for the number nodes in each hidden layer, based on the number of nodes in the input layer using the drop one/two rule.
Compute default values for the number nodes in each hidden layer, based on the number of nodes in the input layer using the drop one/two rule. Rule [1] nx, nx - 2, ...
- nx
the number of nodes in the input layer
-
def
corrMatrix(xx: MatriD): MatriD
Return the correlation matrix for the columns in data matrix 'xx'.
Return the correlation matrix for the columns in data matrix 'xx'.
- xx
the data matrix shose correlation matrix is sought
- Definition Classes
- Predictor
-
def
crossValidate(k: Int = 10, rando: Boolean = true): Array[Statistic]
- Definition Classes
- PredictorMat2
- val df_m: Int
-
var
ee: MatriD
- Attributes
- protected
- Definition Classes
- PredictorMat2
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
var
eta: Double
- Attributes
- protected
- Definition Classes
- PredictorMat2
-
def
eval(ym: Double, y_e: VectoD, yp: VectoD): PredictorMat2
Compute the error (difference between actual and predicted) and useful diagnostics for the test dataset.
Compute the error (difference between actual and predicted) and useful diagnostics for the test dataset. Requires predicted responses to be passed in.
- ym
the training/full mean actual response/output vector
- y_e
the test/full actual response/output vector
- yp
the test/full predicted response/output vector
- Definition Classes
- PredictorMat2
-
def
eval(x_e: MatriD, y_e: MatriD): PredictorMat2
Evaluate the quality of the fit for the parameter/weight matrices on the entire dataset or the test dataset.
Evaluate the quality of the fit for the parameter/weight matrices on the entire dataset or the test dataset. Considers all the response/output variables/columns.
- x_e
the test/full data/input data matrix
- y_e
the test/full response/output response matrix
- Definition Classes
- PredictorMat2
-
def
eval(x_e: MatriD = x, y_e: VectoD = y.col(0)): PredictorMat2
Evaluate the quality of the fit for the parameter/weight matrices on the entire dataset or the test dataset.
Evaluate the quality of the fit for the parameter/weight matrices on the entire dataset or the test dataset. Only considers the first response/output variable/column.
- x_e
the test/full data/input matrix
- y_e
the test/full response/output vector (first column only)
- Definition Classes
- PredictorMat2 → Model
-
val
fitA: Array[Fit]
- Definition Classes
- PredictorMat2
-
def
fitLabel: Seq[String]
Return the labels for the quality of fit measures.
Return the labels for the quality of fit measures.
- Definition Classes
- PredictorMat2
-
def
fitMap: IndexedSeq[Map[String, String]]
Return 'fitMap' results for each y-column and print the overall 'rSq' average over all y-columns.
Return 'fitMap' results for each y-column and print the overall 'rSq' average over all y-columns.
- Definition Classes
- PredictorMat2
-
final
def
flaw(method: String, message: String): Unit
- Definition Classes
- Error
-
var
fname: Strings
- Attributes
- protected
- Definition Classes
- PredictorMat2
-
def
forwardSel(cols: Set[Int], index_q: Int = index_rSqBar): (Int, PredictorMat2)
Perform forward selection to find the most predictive variable to add the existing model, returning the variable to add and the new model.
Perform forward selection to find the most predictive variable to add the existing model, returning the variable to add and the new model. May be called repeatedly.
- cols
the columns of matrix x currently included in the existing model
- index_q
index of Quality of Fit (QoF) to use for comparing quality
- Definition Classes
- PredictorMat2 → Predictor
- See also
Fitfor index of QoF measures.
-
def
forwardSelAll(index_q: Int = index_rSqBar, cross: Boolean = true): (Set[Int], MatriD)
Perform forward selection to find the most predictive variables to have in the model, returning the variables added and the new Quality of Fit (QoF) measures for all steps.
Perform forward selection to find the most predictive variables to have in the model, returning the variables added and the new Quality of Fit (QoF) measures for all steps.
- index_q
index of Quality of Fit (QoF) to use for comparing quality
- cross
whether to include the cross-validation QoF measure
- Definition Classes
- PredictorMat2
- See also
Fitfor index of QoF measures.
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
def
getNetParam(layer: Int = 1): NetParam
Return the network parameters (weights and biases) for the given 'layer'.
Return the network parameters (weights and biases) for the given 'layer'.
- layer
the layer to get the parameters from
-
def
getX: MatriD
Return the data matrix 'x'.
Return the data matrix 'x'. Mainly for derived classes where 'x' is expanded from the given columns in 'x_', e.g.,
QuadRegressionadd squared columns.- Definition Classes
- PredictorMat2 → Predictor
-
def
getY: VectoD
Return the first response vector 'y.col(0)'.
Return the first response vector 'y.col(0)'. Mainly for derived classes where 'y' is transformed.
- Definition Classes
- PredictorMat2 → Predictor
-
def
getYY: MatriD
Return the response matrix 'y'.
Return the response matrix 'y'. Mainly for derived classes where 'y' is transformed.
- Definition Classes
- PredictorMat2
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
def
hparameter: HyperParameter
Return the hyper-parameters.
Return the hyper-parameters.
- Definition Classes
- PredictorMat2 → Model
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- val itran: FunctionV_2V
-
val
layers: Range
- Attributes
- protected
-
val
m: Int
- Attributes
- protected
- Definition Classes
- PredictorMat2
-
val
modelConcept: URI
An optional reference to an ontological concept
An optional reference to an ontological concept
- Definition Classes
- Model
-
def
modelName: String
An optional name for the model (or modeling technique)
An optional name for the model (or modeling technique)
- Definition Classes
- Model
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
val
nl: Int
- Attributes
- protected
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
val
nx: Int
- Attributes
- protected
- Definition Classes
- PredictorMat2
-
val
ny: Int
- Attributes
- protected
- Definition Classes
- PredictorMat2
-
def
parameter: VectoD
Return the parameter/weight vector (first layer, first output).
Return the parameter/weight vector (first layer, first output).
- Definition Classes
- PredictorMat2 → Model
-
def
parameters: NetParams
Return the parameters (weight matrices and bias vectors).
Return the parameters (weight matrices and bias vectors).
- Definition Classes
- NeuralNet_XL → PredictorMat2
-
def
predict(z: MatriD = x): VectoD
Given a new input matrix 'z', predict the output/response matrix 'f(z)'.
Given a new input matrix 'z', predict the output/response matrix 'f(z)'. Return only the first output variable's value.
- z
the new input matrix
- Definition Classes
- PredictorMat2 → Predictor
-
def
predict(z: VectoD): Double
Given a new input vector 'z', predict the output/response value 'f(z)'.
Given a new input vector 'z', predict the output/response value 'f(z)'. Return only the first output variable's value.
- z
the new input vector
- Definition Classes
- PredictorMat2 → Predictor
-
def
predict(z: VectoI): Double
Given a new discrete data/input vector 'z', predict the 'y'-value of 'f(z)'.
Given a new discrete data/input vector 'z', predict the 'y'-value of 'f(z)'.
- z
the vector to use for prediction
- Definition Classes
- Predictor
-
def
predictV(v: MatriD = x): MatriD
Given an input matrix 'x', predict the output/response matrix 'f(x)'.
Given an input matrix 'x', predict the output/response matrix 'f(x)'.
- v
the input matrix
- Definition Classes
- NeuralNet_XL → PredictorMat2
-
def
predictV(v: VectoD): VectoD
Given a new input vector 'v', predict the output/response vector 'f(v)'.
Given a new input vector 'v', predict the output/response vector 'f(v)'.
- v
the new input vector
- Definition Classes
- NeuralNet_XL → PredictorMat2
-
def
report: String
Return a basic report on the trained model.
Return a basic report on the trained model.
- Definition Classes
- PredictorMat2 → Model
- See also
'summary' method for more details
-
def
reset(eta_: Double): Unit
Reset the learning rate 'eta'.
Reset the learning rate 'eta'. Since this hyper-parameter needs frequent tuning, this method is provided to facilitate that.
- eta_
the learning rate
- Definition Classes
- PredictorMat2
-
def
resetDF(df_update: PairD): Unit
Reset the degrees of freedom to the new updated values.
Reset the degrees of freedom to the new updated values. For some models, the degrees of freedom is not known until after the model is built. Caveat: only applies to the first response/output variable.
- df_update
the updated degrees of freedom (model, error)
- Definition Classes
- PredictorMat2
-
def
residual: VectoD
Return the vector of residuals/errors for first response/output variable/column.
Return the vector of residuals/errors for first response/output variable/column.
- Definition Classes
- PredictorMat2 → Predictor
-
def
residuals: MatriD
Return the matrix of residuals/errors.
Return the matrix of residuals/errors.
- Definition Classes
- PredictorMat2
-
val
sizes: Array[Int]
- Attributes
- protected
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
test(modelName: String, doPlot: Boolean = true): Unit
Test the model on the full dataset (i.e., train and evaluate on full dataset).
Test the model on the full dataset (i.e., train and evaluate on full dataset).
- modelName
the name of the model being tested
- doPlot
whether to plot the actual vs. predicted response
- Definition Classes
- Predictor
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
train(x_: MatriD = x, y_: MatriD = y): NeuralNet_XL
Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors).
Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors). Iterate over several epochs, where each epoch divides the training set into 'nbat' batches. Each batch is used to update the weights.
- x_
the training/full data/input matrix
- y_
the training/full response/output matrix
- Definition Classes
- NeuralNet_XL → PredictorMat2
-
def
train(x_: MatriD, y_: VectoD): PredictorMat2
Given data matrix 'x_' and response vector 'y_', fit the parameter 'b' (weights and biases).
Given data matrix 'x_' and response vector 'y_', fit the parameter 'b' (weights and biases).
- x_
the training/full data/input matrix
- y_
the training/full response/output vector, e.g., for the first variable/column
- Definition Classes
- PredictorMat2 → Model
-
def
train0(x_: MatriD = x, y_: MatriD = y): NeuralNet_XL
Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors).
Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors). Iterate over several epochs (no batching). b.w(l) *= 1.0 - eta * (lambda / m) // regularization factor, weight decay
- x_
the training/full data/input matrix
- y_
the training/full response/output matrix
- Definition Classes
- NeuralNet_XL → PredictorMat2
-
def
train2(x_: MatriD = x, y_: MatriD = y): NeuralNet_XL
Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors).
Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors). Iterate over several epochs, where each epoch divides the training set into 'nbat' batches. Each batch is used to update the weights. This version preforms an interval search for the best 'eta' value.
- x_
the training/full data/input matrix
- y_
the training/full response/output matrix
- Definition Classes
- NeuralNet_XL → PredictorMat2
-
def
trainSwitch(which: Int, x_: MatriD = x, y_: MatriD = y): PredictorMat2
Switch between 'train' methods: simple (0), regular (1) and hyper-parameter optimizing (2).
Switch between 'train' methods: simple (0), regular (1) and hyper-parameter optimizing (2).
- which
the kind of 'train' method to use
- x_
the training/full data/input matrix
- y_
the training/full response/output matrix
- Definition Classes
- PredictorMat2
-
def
vif(skip: Int = 1): VectoD
Compute the Variance Inflation Factor 'VIF' for each variable to test for multi-collinearity by regressing 'x_j' against the rest of the variables.
Compute the Variance Inflation Factor 'VIF' for each variable to test for multi-collinearity by regressing 'x_j' against the rest of the variables. A VIF over 10 indicates that over 90% of the variance of 'x_j' can be predicted from the other variables, so 'x_j' may be a candidate for removal from the model. Note: override this method to use a superior regression technique.
- skip
the number of columns of x at the beginning to skip in computing VIF
- Definition Classes
- PredictorMat2
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
Deprecated Value Members
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] ) @Deprecated
- Deprecated