The Development Cycle of Neural Networks

The development (testing) cycle

To see where the testing hurdle is, we’ll tune our neural network with an iterative process. We‘ll perform the following steps:

1. Tune the network’s hyperparameters.
2. Train the network on the training set.
3. Test the network on the test set.
4. Repeat until we are happy with the network’s accuracy.

This process is similar to the ML equivalent of software development, so we can simply call it the development cycle.

We already went through a few iterations of development in the previous chapter, when we measured the network’s performance with different batch sizes. However, we overlooked a ...