I'm trying to train an ANN. Over several hundred iterations (or thousands, depending on parameters), the training error decreases with each training iteration. But then, it starts to rise again and continues to do so until the network has completely unlearned everything it learned and is doing as bad (or in some cases worse) than the completely untrained network.

I can understand that a network might get to the stage where it can't learn any more, but why might my network be unlearning?

Let me give you some additional detail.

Details

Task details

Learn position evaluation function for tic-tac-toe given all valid positions and the correct evaluation (0, 0.5, 1). Board state represented as...

• 9 x "is this cell blank"
• 9 x "is there an X on this cell"
• 9 x "is there an O on this cell"
• 1 x "is in X's turn"
• 1 x "is in O's turn"

Network details

MLP network. Full connectivity between all layers.

• 1 input layer, 29 neurons
• 1 hidden layer, 29 rectified linear neurons
• 1 output layer, 1 neuron
• Random weight initialization between -0.5 and +0.5
• Learning rate 0.005

Training details

Back-propagation with the L2 loss function.

Training using the full dataset (i.e. I'm not holding any back as a test set). Each iteration trains the full set of positions as a batch and then applies the weight updates.

Reported error after each iteration is the average of the difference between the network's output and the correct output for each position.

Implementation details

Using the Neuroph library, the network is created like this...

if (USE_RELU)
{
  lTransferFunction = TransferFunctionType.RECTIFIED_LINEAR;
  INITIAL_LEARNING_RATE = 0.005;
  lInitialWeightMin = -0.5;
  lInitialWeightMax = 0.5;
}
else
{
  lTransferFunction = TransferFunctionType.SIGMOID;
  INITIAL_LEARNING_RATE = 0.05;
  lInitialWeightMax = 1 / Math.sqrt(mInputSize);
  lInitialWeightMin = -lInitialWeightMax;
}

mNetwork = new MultiLayerPerceptron(lTransferFunction,
                                    mInputSize,     // Input layer, 1 neuron per base proposition
                                    mInputSize,     // Hidden layer(s)
                                    mOutputSize);   // Output layer, 1 neuron per role (except for fixed sum,
                                                    // where we only need 1).
mNetwork.randomizeWeights(lInitialWeightMin, lInitialWeightMax);

// Create a learning rule.
mLearningRule = createLearningRule();

<...>
}

private BackPropagation createLearningRule()
{
  BackPropagation lLearningRule = new BackPropagation();
  lLearningRule.setMaxIterations(1);
  lLearningRule.setLearningRate(INITIAL_LEARNING_RATE);
  lLearningRule.setNeuralNetwork(mNetwork);
  return lLearningRule;
}

...and trained like this...

double lTotalErr = 0;
int lNumIterations = 0;
for (int lii = 0; lii < lNumIterations; lii++)
{
  if (lii % 10 == 0)
  {
    LOGGER.info("Iteration " + lii + ", err " + ((lTotalErr * 100.0) / 10.0));
    lTotalErr = 0;
  }
  mLearningRule.doOneLearningIteration(mTrainingSet);
  lTotalErr += mLearningRule.getErrorFunction().getTotalError();
}

Alternatives

In an attempt to debug, I've tried modifying various parameters (without success).

Learning rate

Higher learning rates give quicker initial convergence, but the lowest reported error rate is higher (as expected). The "unlearning" phase is reached more quickly. With very high learning rates, it immediately diverges.

Lower learning rates cause slower learning to a better value, but the unlearning still happens - and it still unlearns all the way to being completely untrained.

Hidden layer size

Varying the hidden layer between 1/3 of the input size and 3x the input size doesn't help. With smaller hidden layers each training cycle is quick but more cycles are required and the best result achieved is worse. Opposite for larger hidden layers. In all cases, learning is followed by unlearning.

Number of hidden layers

I've tried using a couple of hidden layers. It didn't help.

Neuron type

I've tried using sigmoid neurons instead of rectified linear neurons (with appropriately smaller initial weights and larger learning rate). This tends to remain stable for a bit longer, but eventually suffers the same divergence.