I am using ThreadPoolExecutor module in Python but I have a problem when executing my script.
Here what I want to do:
I have this json file, called result:
{
"hostname1": {
"leaf_ip": "10.234.12.4",
"hosts": [
{
"ip": "10.42.78.59",
"mac": "ab:cd:ef:gh:ij:01",
"vlan": "107"
},
{
"ip": "10.42.78.60",
"mac": "ab:cd:ef:gh:ij:02",
"vlan": "161"
},
{
"ip": "10.42.78.61",
"mac": "ab:cd:ef:gh:ij:03",
"vlan": "161"
}
]
},
"hostname2": {
"leaf_ip": "10.234.12.5",
"hosts": [
{
"ip": "10.42.78.62",
"mac": "ab:cd:ef:gh:ij:04",
"vlan": "107"
},
{
"ip": "10.42.78.64",
"mac": "ab:cd:ef:gh:ij:06",
"vlan": "161"
}
]
},
"hostname3": {
"leaf_ip": "10.234.12.6",
"hosts": [
{
"ip": "10.42.78.65",
"mac": "ab:cd:ef:gh:ij:07",
"vlan": "107"
},
{
"ip": "10.42.78.67",
"mac": "ab:cd:ef:gh:ij:09",
"vlan": "161"
}
]
},
"hostname4": {
"leaf_ip": "10.234.12.6",
"hosts": []
}
}
And I want to fill it by adding the "port" in each host like that:
"ip": "10.42.78.59",
"mac": "ab:cd:ef:gh:ij:01",
"vlan": "107",
"port": "port1"
In order to get the port, I am contacting the leaf (leaf_ip in the json file) and I'm launching a command on the switch, then I get the result that I am filtering.
Here is the whole code:
def main_loop(result):
with ThreadPoolExecutor(max_workers=os.cpu_count() + 4) as executor:
for hostname in result.keys():
if result[hostname]["hosts"] != []:
leaf_ip = result[hostname]["leaf_ip"]
gateway_session = SSHSession('10.11.121.12','admin', password='******').open()
remote_session = gateway_session.get_remote_session(leaf_ip, 'admin', password='*********')
for host in result[hostname]["hosts"]:
i = result[hostname]["hosts"].index(host)
executor.submit(get_port, hostname, i, remote_session, result)
return result
def get_port(hostname, i, remote_session, result):
mac = result[hostname]["hosts"][i]["mac"]
leaf_command = 'sh mac address-table address ' + mac + ' | json'
process = remote_session.get_cmd_output(leaf_command)
if process != b'':
if type(json.loads(process)["TABLE_mac_address"]["ROW_mac_address"]) is dict:
result[hostname]["hosts"][i]["port"] = json.loads(process)["TABLE_mac_address"]["ROW_mac_address"]["disp_port"]
elif type(json.loads(process)["TABLE_mac_address"]["ROW_mac_address"]) is list:
result[hostname]["hosts"][i]["port"] = json.loads(process)["TABLE_mac_address"]["ROW_mac_address"][0]["disp_port"]
So to sum up, for each hostname in the json, then for each host in the list of hosts if it is not empty, I connect with the leaf_ip corresponding to the hostname, and I will retrieve data on the switch that I add to my json after filtering.
After having launched my script, I notice that the elements of the beginning do not receive the port whereas those of the end yes. There seems to be a problem with multithreading because it works fine without it. I specify that the json is actually much longer with hundreds of elements, hence the need for multithreading.
I feel like I used multithreading wrong and would like to know if anyone has any idea what my mistake was.
Thank you