Introduction

Introduction

In a standard neural network, each layer takes an input vector, every element of which is connected to each neuron in the layer with a specific “weight”. Moreover, each neuron in a layer has a specific “bias” associated with it, designed so that the neuron will only produce a meaningful output if the linear combination of weighted inputs (the neuron’s excitation) is greater than that bias. This output is determined by passing the neuron’s net excitation to an activation function. With the inputs propagating through each layer of the network in this fashion, the neural network produces an output corresponding to both an input and the parameters (weights and biases) of the network. The network then learns through a Stochastic Gradient Descent algorithm, which updates all of the network’s parameters in attempts to minimize a cost function that defines the relationship between a network’s produced output and a desired output. This entire training process is repeated for a specified number of epochs to improve accuracy.

A convolutional neural network (CNN) is similar to a standard neural network except it adds convolutional layers at the beginning. Convolutional layers arrange neurons into grids, and convolve those grids with input images. The parameters of the convolutional layers are the weights of each neuron in each filter’s kernel and the bias applied to each filter. The output of the convolutional layers is then passed to the fully connected layers of a standard neural network.

Our project focused on applying convolutional neural networks to handwritten digits, allowing us to build a system that was able to recognize the digits 0-9 with a great deal of accuracy. Because of the simplicity of this problem, we were better able to understand the structure of the tool used to solve the problem. With this deeper understanding of convolutional neural networks, we are better equipped to solve more complex problems.

Project motivation

Previous work