Reduction axis 1 is empty in shape [x,0]

I'm trying to train the SVHN (street view house numbers) dataset for object-detection in tensorflow (to do some basic OCR on numbers). So far I have successfully followed the pet-training example from the object-detection-tensorflow guides. When I train the network based on the sample faster_rcnn_resnet101.config, after a few dozen steps I get:

    INFO:tensorflow:Error reported to Coordinator: 
<class 'tensorflow.python.framework.errors_impl.InvalidArgumentError'>, 
Reduction axis 1 is empty in shape [3,0]
[[Node: Loss/RPNLoss/Match/cond/ArgMax_1 = ArgMax[T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/cpu:0"]
(Loss/RPNLoss/Match/cond/ArgMax/Switch:1, 
Loss/RPNLoss/Match/cond/ArgMax_1/dimension)]]

I have no clue what to change or improve. Has someone seen this before? What is going wrong here? Is it simply a wrong config-setting? The only parameter I changed (besides path-info) is num_classes: 10 (10 digits)

Thanks for any hints.

My label-map looks like this:

item {
  id: 0
  name: 'none_of_the_above'
}

item {
  id: 1
  name: '1'
}

item {
  id: 2
  name: '2'
}
... with id: 10 being '0'

As supposed here https://github.com/tensorflow/models/blob/master/object_detection/g3doc/running_pets.md I used the pretrained COCO-model faster_rcnn_resnet101 and also the config-file from that: https://github.com/tensorflow/models/blob/master/object_detection/samples/configs/faster_rcnn_resnet101_pets.config The only things I adapted are the paths and:

faster_rcnn {
    num_classes: 11
    image_resizer {
      keep_aspect_ratio_resizer {
        min_dimension: 64
        max_dimension: 900
      }
    }

Due to the fact that the images from SVHN are rather small, I adapted the dimensions here and removed all images that are smaller than 64px in height or width. I didn't pay attention to clear the training-directory until now. But I tried now and the same error occurs. I'm currently trying with the pretrained inception-model, maybe this works out.