i am work on a project.i use the dqn to maximize return. this picture are some env states.
i found that dqn did learn a bit, but after a while it stopped improving and even started to decline.
this my current Hyperparameters:lr 0.01,lr_decay 9e-7,mineps 0.1, maxeps 1,epsdecay 1/25000,memory buffer 200000,discount 0.999,Weight_update_frequency 20step,batch_size 16.
i try a lot.i use the lr:0.001,0.01,0.0001,used eps:0.05,0,02. the lr_decay:1e-7,1.5e-7,1.2e-7,eps_decay:1/10000,1/20000,1/25000,1/40000,1/50000.Weight_update_frequency:4epoch,20step.batch_size:16,32,64.the model size:64,64,128,32,32,64.
Its trend is almost the same, and it has learned something, but then the shaking is very serious, or even declines.
this is current training figure, anyway,it is better than most of my other training figure.