Running queue with load balancer Laravel

I am using a load balancer and 2 servers use the database to listen for jobs. The problem is that when I dispatch a job and it get picked by a server but it is not completed before the retry time runs out of a queue job, it will start running on the second server and fail with a message saying that the 'Job has attempted to run to many times or run for too long'. But the first server continues to complete the job and it finishes successfully. I tried everything, starting from using Queue::before, writing functions to handle the logic for a job that is in progress but I had no luck, maybe someone can help. Thank you in advance.