Neural network with fully connected layers, built from scratch.
-
Optimizers: Stochastic Gradient Descent (SGD), L-BFGS
-
Minibatch learning (as generalization for both online learning and batch learning), Momentum and moving average
-
Loss function: mean squared error (MSE)
-
Activation functions: linear, sigmoid, softplus
-
Regularization: L2 (Tichonov), early stopping
-
Metrics: accuracy, mean euclidean error (MEE)
-
Validation techniques: grid search, k-fold cross validation (with parallelization)
-
Utilities: rescaling, 1-of-K encoding, plot routines, read/write routines for dataset where the project has been tested on
This project has been made as part of Machine Learning and Computational Mathematics courses at the University of Pisa.
MainCup.py refers to a private contest whose dataset has not been disclosed.