I am using Deep Q Learning to try to create a simple 2D self driving car simulation in Python. The state is the distance to the edge of the road at a few locations, and the actions are left, right, accelerate, brake. When simply controlling steering, it can navigate any map, but introduced to speed, it can't learn to brake around corners, causing it to crash.
I have tried alot of different hyperparameters, and the below graph is the best I can get it.
Here are the settings I used.
"LEARNING_RATE": 1e-10,
"GD_MOMENTUM": 0.9,
"DISCOUNT_RATE": 0.999,
"EPSILON_DECAY": 0.00002,
"EPSILON_MIN": 0.1,
"TARGET_NET_COPY_STEPS": 17000,
"TRAIN_AMOUNT": 0.8,
My guess is that it can't take into account rewards that far in the future, so I increased the movement per frame but it didn't help.
For the neural networks, I am using my own library (which I have verified works), with 12 layers, increasing up to a max of 256 nodes, using relu. I have tried different configurations, which were either worse or the same.
You can find the code here, but there is alot of code for other features: enter link description here