Function for creating a graph of levels using networkx library

user_14569599 0
Viewed 128

I want to create a function where in it will loop twice or thrice depending on the levels mentioned by me for creating a graph using networkx library.

If a person knows skill Python then he knows AWS and Hadoop too [level 1]

Then I want to search skills related to AWS and Hadoop [level 2].

Please help me to create a function with a exception handling wherein it is possible that the related skills do not have their related skills in the data.

this is how pairs look

This is the standard function I created for level 1.

def network(pair, skill):
  skil=[]
  related_skil=[]
  for line in pair[:306444]:
    s, r = line[0], line[1]
    #print(s)
    if (s == skill):
      skil.append(s)
      related_skil.append(r)
      
  skill=pd.DataFrame(columns=["skills"],data=skil)
  relat_skil=pd.DataFrame(columns=["related_skills"],data=related_skil)

  df=pd.concat([skill,relat_skil],axis=1)
  
  G=nx.Graph()
  G=nx.from_pandas_edgelist(df,'skills','related_skills')

  figure(figsize=(12,10))
  a = nx.draw(G,node_color='red',with_labels=True)

  return a

network(pair=pairs, skill='Python')

where pair is a list zip file of skills and related skills pairs=list(zip(skill,related skill))

networkx graph python
1 Answers

The networkx library contains probably all you need to solve the question

Given a skill A, is skill B a related skill?

We'll assume that related skills are directional (knowing Python might lead to knowing AWS, but not vice versa). For this, we'll use the directed graph model.

We can also use shortest path length to model relatedness level and use that to color and layer the output graph.

In the program below, unrelated skills are removed to simplify the graph: Prolog is not related to Python (arrows are wrong direction), C and Embedded systems are not connected to the Python skill graph.

import itertools
import matplotlib.pyplot as plt
import networkx as nx

def skill_rel(skill, rel_skill):
    if skill not in G:
        print(f"{skill} skill no longer in graph")
        return
    if rel_skill not in G:
        print(f"{rel_skill} skill no longer in graph")
        return
    related = nx.has_path(G, skill, rel_skill)
    print(f"{skill} skill and relation to {rel_skill}",
          related, end=' ')
    if related:
        print('level',
              nx.shortest_path_length(G, skill, rel_skill))

def level_rel(skill, rel_skill):
    related = nx.has_path(G, skill, rel_skill)
    if related:
        return nx.shortest_path_length(G, skill, rel_skill)
    else:
        return 999

pairs = [('Python', 'AWS'),
         ('Python', 'Machine Learning'),
         ('Python', 'Big Data'),
         ('AWS', 'Hadoop'),
         ('C', 'Embedded Systems'),
         ('Prolog', 'Artificial Intelligence'),
         ('Machine Learning', 'Deep Learning'),
         ('Machine Learning', 'Artificial Intelligence')]
skill = 'Python'
G = nx.DiGraph()
G.add_edges_from(pairs)
# Remove unrelated skills (>= level 5)
unrelatedness = 5
unrelated = [node for node in G if level_rel(skill, node) >= unrelatedness]
G.remove_nodes_from(unrelated)
# Group/color nodes based on relatedness to use shell graph drawing
nlevel = [(node, level_rel(skill, node)) for node in G]
g1 = itertools.groupby(sorted(nlevel, key=lambda x: x[1]), key=lambda x: x[1])
shells = tuple(tuple(x[0] for x in g) for _,g in g1)
pos = nx.shell_layout(G, nlist=shells)
colors = [lvl[1] for lvl in nlevel]
nx.draw_networkx(G, pos, node_color=colors, with_labels=True)
plt.show()
skill_rel(skill, 'Hadoop')
skill_rel('Prolog', 'Hadoop')
print('Level 2 related skills from', skill, shells[2])

Output graph with levels colored:

Output graph with levels colored

Output:

# Python skill and relation to Hadoop True level 2
# Prolog skill no longer in graph
# Level 2 related skills from Python
# ('Hadoop', 'Artificial Intelligence', 'Deep Learning')
user_56603150
Related