Packages

class NeuralNet_XL extends PredictorMat2

The NeuralNet_XL class supports multi-output, multi-layer (input, multiple hidden and output) Neural-Networks. It can be used for both classification and prediction, depending on the activation functions used. Given several input vectors and output vectors (training data), fit the weight and bias parameters connecting the layers, so that for a new input vector 'v', the net can predict the output value. Defaults to two hidden layers. This implementation is partially adapted from Michael Nielsen's Python implementation found in

See also

github.com/mnielsen/neural-networks-and-deep-learning/blob/master/src/network2.py

github.com/MichalDanielDobrzanski/DeepLearningPython35/blob/master/network2.py ------------------------------------------------------------------------------

Linear Supertypes
PredictorMat2, Predictor, Model, Error, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. NeuralNet_XL
  2. PredictorMat2
  3. Predictor
  4. Model
  5. Error
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new NeuralNet_XL(x: MatriD, y: MatriD, nz: Array[Int] = null, fname_: Strings = null, hparam: HyperParameter = Optimizer.hp, f: Array[AFF] = Array (f_tanh, f_tanh, f_id), itran: FunctionV_2V = null)

    x

    the m-by-nx data/input matrix (training data having m input vectors)

    y

    the m-by-ny response/output matrix (training data having m output vectors)

    nz

    the number of nodes in each hidden layer, e.g., Array (9, 8) => 2 hidden of sizes 9 and 8

    fname_

    the feature/variable names (if null, use x_j's)

    hparam

    the hyper-parameters for the model/network

    f

    the array of activation function families between every pair of layers

    itran

    the inverse transformation function returns responses to original scale

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. val _1: VectorD
    Attributes
    protected
    Definition Classes
    PredictorMat2
  5. def analyze(x_: MatriD = x, y_: VectoD = y(0), x_e: MatriD = x, y_e: VectoD = y(0)): PredictorMat2

    Analyze a dataset using this model using ordinary training with the 'train' method.

    Analyze a dataset using this model using ordinary training with the 'train' method. Only uses the first output variable's value.

    x_

    the data/input matrix (training/full)

    y_

    the response/output vector (training/full)

    x_e

    the data/input matrix (testing/full)

    y_e

    the response/output vector (testing/full)

    Definition Classes
    PredictorMat2Predictor
  6. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  7. var b: IndexedSeq[NetParam]
    Attributes
    protected
  8. def backwardElim(cols: Set[Int], index_q: Int = index_rSqBar, first: Int = 1): (Int, PredictorMat2)

    Perform backward elimination to find the least predictive variable to remove from the existing model, returning the variable to eliminate, the new parameter vector and the new Quality of Fit (QoF).

    Perform backward elimination to find the least predictive variable to remove from the existing model, returning the variable to eliminate, the new parameter vector and the new Quality of Fit (QoF). May be called repeatedly.

    cols

    the columns of matrix x currently included in the existing model

    index_q

    index of Quality of Fit (QoF) to use for comparing quality

    first

    first variable to consider for elimination (default (1) assume intercept x_0 will be in any model)

    Definition Classes
    PredictorMat2
    See also

    Fit for index of QoF measures.

  9. def backwardElimAll(index_q: Int = index_rSqBar, first: Int = 1, cross: Boolean = true): (Set[Int], MatriD)

    Perform forward selection to find the most predictive variables to have in the model, returning the variables added and the new Quality of Fit (QoF) measures for all steps.

    Perform forward selection to find the most predictive variables to have in the model, returning the variables added and the new Quality of Fit (QoF) measures for all steps.

    index_q

    index of Quality of Fit (QoF) to use for comparing quality

    first

    first variable to consider for elimination

    cross

    whether to include the cross-validation QoF measure

    Definition Classes
    PredictorMat2
    See also

    Fit for index of QoF measures.

  10. def buildModel(x_cols: MatriD): NeuralNet_XL

    Build a sub-model that is restricted to the given columns of the data matrix.

    Build a sub-model that is restricted to the given columns of the data matrix.

    x_cols

    the columns that the new model is restricted to

    Definition Classes
    NeuralNet_XLPredictorMat2
  11. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  12. def compute_df_m(n: Array[Int]): Int

    Compute the degrees of freedom for the model (based on nx, n's, ny = 1).

    Compute the degrees of freedom for the model (based on nx, n's, ny = 1). Rough extimate based on total number of parameters - 1. FIX: use better estimate

    n

    the number of nodes in each hidden layer

  13. def compute_nz(nx: Int, ny: Int): Array[Int]

    Compute default values for the number nodes in each hidden layer, based on the number of nodes in the input and output layers using average of prior layer and output layer rule.

    Compute default values for the number nodes in each hidden layer, based on the number of nodes in the input and output layers using average of prior layer and output layer rule. Rule [2] (nx + ny) / 2, (nx + 3ny) / 4, ...

    nx

    the number of nodes in the input layer

    ny

    the number of nodes in the output layer

  14. def compute_nz(nx: Int): Array[Int]

    Compute default values for the number nodes in each hidden layer, based on the number of nodes in the input layer using the drop one/two rule.

    Compute default values for the number nodes in each hidden layer, based on the number of nodes in the input layer using the drop one/two rule. Rule [1] nx, nx - 2, ...

    nx

    the number of nodes in the input layer

  15. def corrMatrix(xx: MatriD): MatriD

    Return the correlation matrix for the columns in data matrix 'xx'.

    Return the correlation matrix for the columns in data matrix 'xx'.

    xx

    the data matrix shose correlation matrix is sought

    Definition Classes
    Predictor
  16. def crossValidate(k: Int = 10, rando: Boolean = true): Array[Statistic]
    Definition Classes
    PredictorMat2
  17. val df_m: Int
  18. var ee: MatriD
    Attributes
    protected
    Definition Classes
    PredictorMat2
  19. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  20. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  21. var eta: Double
    Attributes
    protected
    Definition Classes
    PredictorMat2
  22. def eval(ym: Double, y_e: VectoD, yp: VectoD): PredictorMat2

    Compute the error (difference between actual and predicted) and useful diagnostics for the test dataset.

    Compute the error (difference between actual and predicted) and useful diagnostics for the test dataset. Requires predicted responses to be passed in.

    ym

    the training/full mean actual response/output vector

    y_e

    the test/full actual response/output vector

    yp

    the test/full predicted response/output vector

    Definition Classes
    PredictorMat2
  23. def eval(x_e: MatriD, y_e: MatriD): PredictorMat2

    Evaluate the quality of the fit for the parameter/weight matrices on the entire dataset or the test dataset.

    Evaluate the quality of the fit for the parameter/weight matrices on the entire dataset or the test dataset. Considers all the response/output variables/columns.

    x_e

    the test/full data/input data matrix

    y_e

    the test/full response/output response matrix

    Definition Classes
    PredictorMat2
  24. def eval(x_e: MatriD = x, y_e: VectoD = y.col(0)): PredictorMat2

    Evaluate the quality of the fit for the parameter/weight matrices on the entire dataset or the test dataset.

    Evaluate the quality of the fit for the parameter/weight matrices on the entire dataset or the test dataset. Only considers the first response/output variable/column.

    x_e

    the test/full data/input matrix

    y_e

    the test/full response/output vector (first column only)

    Definition Classes
    PredictorMat2Model
  25. val fitA: Array[Fit]
    Definition Classes
    PredictorMat2
  26. def fitLabel: Seq[String]

    Return the labels for the quality of fit measures.

    Return the labels for the quality of fit measures.

    Definition Classes
    PredictorMat2
  27. def fitMap: IndexedSeq[Map[String, String]]

    Return 'fitMap' results for each y-column and print the overall 'rSq' average over all y-columns.

    Return 'fitMap' results for each y-column and print the overall 'rSq' average over all y-columns.

    Definition Classes
    PredictorMat2
  28. final def flaw(method: String, message: String): Unit
    Definition Classes
    Error
  29. var fname: Strings
    Attributes
    protected
    Definition Classes
    PredictorMat2
  30. def forwardSel(cols: Set[Int], index_q: Int = index_rSqBar): (Int, PredictorMat2)

    Perform forward selection to find the most predictive variable to add the existing model, returning the variable to add and the new model.

    Perform forward selection to find the most predictive variable to add the existing model, returning the variable to add and the new model. May be called repeatedly.

    cols

    the columns of matrix x currently included in the existing model

    index_q

    index of Quality of Fit (QoF) to use for comparing quality

    Definition Classes
    PredictorMat2Predictor
    See also

    Fit for index of QoF measures.

  31. def forwardSelAll(index_q: Int = index_rSqBar, cross: Boolean = true): (Set[Int], MatriD)

    Perform forward selection to find the most predictive variables to have in the model, returning the variables added and the new Quality of Fit (QoF) measures for all steps.

    Perform forward selection to find the most predictive variables to have in the model, returning the variables added and the new Quality of Fit (QoF) measures for all steps.

    index_q

    index of Quality of Fit (QoF) to use for comparing quality

    cross

    whether to include the cross-validation QoF measure

    Definition Classes
    PredictorMat2
    See also

    Fit for index of QoF measures.

  32. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  33. def getNetParam(layer: Int = 1): NetParam

    Return the network parameters (weights and biases) for the given 'layer'.

    Return the network parameters (weights and biases) for the given 'layer'.

    layer

    the layer to get the parameters from

  34. def getX: MatriD

    Return the data matrix 'x'.

    Return the data matrix 'x'. Mainly for derived classes where 'x' is expanded from the given columns in 'x_', e.g., QuadRegression add squared columns.

    Definition Classes
    PredictorMat2Predictor
  35. def getY: VectoD

    Return the first response vector 'y.col(0)'.

    Return the first response vector 'y.col(0)'. Mainly for derived classes where 'y' is transformed.

    Definition Classes
    PredictorMat2Predictor
  36. def getYY: MatriD

    Return the response matrix 'y'.

    Return the response matrix 'y'. Mainly for derived classes where 'y' is transformed.

    Definition Classes
    PredictorMat2
  37. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  38. def hparameter: HyperParameter

    Return the hyper-parameters.

    Return the hyper-parameters.

    Definition Classes
    PredictorMat2Model
  39. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  40. val itran: FunctionV_2V
  41. val layers: Range
    Attributes
    protected
  42. val m: Int
    Attributes
    protected
    Definition Classes
    PredictorMat2
  43. val modelConcept: URI

    An optional reference to an ontological concept

    An optional reference to an ontological concept

    Definition Classes
    Model
  44. def modelName: String

    An optional name for the model (or modeling technique)

    An optional name for the model (or modeling technique)

    Definition Classes
    Model
  45. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  46. val nl: Int
    Attributes
    protected
  47. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  48. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  49. val nx: Int
    Attributes
    protected
    Definition Classes
    PredictorMat2
  50. val ny: Int
    Attributes
    protected
    Definition Classes
    PredictorMat2
  51. def parameter: VectoD

    Return the parameter/weight vector (first layer, first output).

    Return the parameter/weight vector (first layer, first output).

    Definition Classes
    PredictorMat2Model
  52. def parameters: NetParams

    Return the parameters (weight matrices and bias vectors).

    Return the parameters (weight matrices and bias vectors).

    Definition Classes
    NeuralNet_XLPredictorMat2
  53. def predict(z: MatriD = x): VectoD

    Given a new input matrix 'z', predict the output/response matrix 'f(z)'.

    Given a new input matrix 'z', predict the output/response matrix 'f(z)'. Return only the first output variable's value.

    z

    the new input matrix

    Definition Classes
    PredictorMat2Predictor
  54. def predict(z: VectoD): Double

    Given a new input vector 'z', predict the output/response value 'f(z)'.

    Given a new input vector 'z', predict the output/response value 'f(z)'. Return only the first output variable's value.

    z

    the new input vector

    Definition Classes
    PredictorMat2Predictor
  55. def predict(z: VectoI): Double

    Given a new discrete data/input vector 'z', predict the 'y'-value of 'f(z)'.

    Given a new discrete data/input vector 'z', predict the 'y'-value of 'f(z)'.

    z

    the vector to use for prediction

    Definition Classes
    Predictor
  56. def predictV(v: MatriD = x): MatriD

    Given an input matrix 'x', predict the output/response matrix 'f(x)'.

    Given an input matrix 'x', predict the output/response matrix 'f(x)'.

    v

    the input matrix

    Definition Classes
    NeuralNet_XLPredictorMat2
  57. def predictV(v: VectoD): VectoD

    Given a new input vector 'v', predict the output/response vector 'f(v)'.

    Given a new input vector 'v', predict the output/response vector 'f(v)'.

    v

    the new input vector

    Definition Classes
    NeuralNet_XLPredictorMat2
  58. def report: String

    Return a basic report on the trained model.

    Return a basic report on the trained model.

    Definition Classes
    PredictorMat2Model
    See also

    'summary' method for more details

  59. def reset(eta_: Double): Unit

    Reset the learning rate 'eta'.

    Reset the learning rate 'eta'. Since this hyper-parameter needs frequent tuning, this method is provided to facilitate that.

    eta_

    the learning rate

    Definition Classes
    PredictorMat2
  60. def resetDF(df_update: PairD): Unit

    Reset the degrees of freedom to the new updated values.

    Reset the degrees of freedom to the new updated values. For some models, the degrees of freedom is not known until after the model is built. Caveat: only applies to the first response/output variable.

    df_update

    the updated degrees of freedom (model, error)

    Definition Classes
    PredictorMat2
  61. def residual: VectoD

    Return the vector of residuals/errors for first response/output variable/column.

    Return the vector of residuals/errors for first response/output variable/column.

    Definition Classes
    PredictorMat2Predictor
  62. def residuals: MatriD

    Return the matrix of residuals/errors.

    Return the matrix of residuals/errors.

    Definition Classes
    PredictorMat2
  63. val sizes: Array[Int]
    Attributes
    protected
  64. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  65. def test(modelName: String, doPlot: Boolean = true): Unit

    Test the model on the full dataset (i.e., train and evaluate on full dataset).

    Test the model on the full dataset (i.e., train and evaluate on full dataset).

    modelName

    the name of the model being tested

    doPlot

    whether to plot the actual vs. predicted response

    Definition Classes
    Predictor
  66. def toString(): String
    Definition Classes
    AnyRef → Any
  67. def train(x_: MatriD = x, y_: MatriD = y): NeuralNet_XL

    Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors).

    Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors). Iterate over several epochs, where each epoch divides the training set into 'nbat' batches. Each batch is used to update the weights.

    x_

    the training/full data/input matrix

    y_

    the training/full response/output matrix

    Definition Classes
    NeuralNet_XLPredictorMat2
  68. def train(x_: MatriD, y_: VectoD): PredictorMat2

    Given data matrix 'x_' and response vector 'y_', fit the parameter 'b' (weights and biases).

    Given data matrix 'x_' and response vector 'y_', fit the parameter 'b' (weights and biases).

    x_

    the training/full data/input matrix

    y_

    the training/full response/output vector, e.g., for the first variable/column

    Definition Classes
    PredictorMat2Model
  69. def train0(x_: MatriD = x, y_: MatriD = y): NeuralNet_XL

    Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors).

    Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors). Iterate over several epochs (no batching). b.w(l) *= 1.0 - eta * (lambda / m) // regularization factor, weight decay

    x_

    the training/full data/input matrix

    y_

    the training/full response/output matrix

    Definition Classes
    NeuralNet_XLPredictorMat2
  70. def train2(x_: MatriD = x, y_: MatriD = y): NeuralNet_XL

    Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors).

    Given training data 'x_' and 'y_', fit the parameters 'b' (weight matrices and bias vectors). Iterate over several epochs, where each epoch divides the training set into 'nbat' batches. Each batch is used to update the weights. This version preforms an interval search for the best 'eta' value.

    x_

    the training/full data/input matrix

    y_

    the training/full response/output matrix

    Definition Classes
    NeuralNet_XLPredictorMat2
  71. def trainSwitch(which: Int, x_: MatriD = x, y_: MatriD = y): PredictorMat2

    Switch between 'train' methods: simple (0), regular (1) and hyper-parameter optimizing (2).

    Switch between 'train' methods: simple (0), regular (1) and hyper-parameter optimizing (2).

    which

    the kind of 'train' method to use

    x_

    the training/full data/input matrix

    y_

    the training/full response/output matrix

    Definition Classes
    PredictorMat2
  72. def vif(skip: Int = 1): VectoD

    Compute the Variance Inflation Factor 'VIF' for each variable to test for multi-collinearity by regressing 'x_j' against the rest of the variables.

    Compute the Variance Inflation Factor 'VIF' for each variable to test for multi-collinearity by regressing 'x_j' against the rest of the variables. A VIF over 10 indicates that over 90% of the variance of 'x_j' can be predicted from the other variables, so 'x_j' may be a candidate for removal from the model. Note: override this method to use a superior regression technique.

    skip

    the number of columns of x at the beginning to skip in computing VIF

    Definition Classes
    PredictorMat2
  73. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  74. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  75. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated

Inherited from PredictorMat2

Inherited from Predictor

Inherited from Model

Inherited from Error

Inherited from AnyRef

Inherited from Any

Ungrouped