Why tasks take longer when more workers are used

I am currently struggling to understand why single tasks are slower when more workers are used. For example, with one worker, each task takes ~0.5s and with six workers, each task takes ~1s. The computer I am working on has 8 cores so I was not expecting this change.

Can you tell me what I am doing wrong, or why there is this difference? Below is the code I am using:

from dask.distributed import Client, LocalCluster

def compute():
    for _ in range(30000000):
        3*4

def gathered(list_):
    pass

if __name__ == "__main__":
    cluster = LocalCluster(n_workers=6, threads_per_worker=1)
    client = Client(cluster)

    graph = {}
    node_names = []

    for i in range(24):
        node_name = ('final_model', i+1)
        node_names.append(node_name)

        graph.update({
            node_name: (compute,)
        })

    graph.update({
        "gathered": (
            gathered,
            node_names
        )
    })

    client.get(graph, 'gathered')
    client.close()

Thank you for your time/help!