Importing product tree in excel to python nested dictionary

Viewed 43

I have a catalog for parts in a project I am working on, arranged in a hierarchy, with references. As you can see in excel tree, the references adds a new digit the lower you go in the tree. The parts may or not have a code associated to them, and are called by their actual names, not "Part 1", "Part 2" (can't show because of confidentiality).

Excel sheet with product tree

I want to create a python nested dictionary/json that follows the same structure as shown in the excel. Final result should look like this:

1-Satellite:{ 11-Plateform:
                         { 111-Subsystem 11:
                                       {Part 1: {System code: ...
                                                 Other data 1: ...
                                                 Other data 2: ...
                                        Part 2: {System code: ...
                                                 Other data 1: ...
                                                 Other data 2: ...
                           112-Subsystem 12:
                                       {Part 17: ...
                                        ...
                                        ...
               12-Payload:
                          ...
                          ...
}}}

Initially, I imported the data with the panda library This allows me to select the column according to its actual name, for instance "sheet.Segment" "sheet.Part" etc. I used the references to make sure that the dictionary would contain the right parts in the right subsystem, system, etc. Whoever, like I stated before, not every part has a system code, so they don't appear in the dictionary with this technique.

For loop cycling through the xlsx file:

import os
import pandas as pd

ScriptPath = os.path.realpath(__file__)
##
BasePath = ScriptPath.replace("/scripts/AIT_catalogJSON.py", "")

xlsx_file = BasePath + ("/0_Organisation/Creme_Catalog_EM.xlsx")
csv_file = BasePath + ("/0_Organisation/Creme_Catalog_EM.csv")

# wb_obj = openpyxl.load_workbook(xlsx_file) 

# # Read the active sheet:
# sheet = wb_obj.active

#excel2json.convert_from_file(xlsx_file)

sheet_xlsx = pd.read_excel(xlsx_file)

Catalog = {}

for segment in range(0,len(sheet_xlsx.Segment)):
    if type(sheet_xlsx.Segment[segment]) is str:
        
        
        for system in range(0,len(sheet_xlsx.System)):
            if type(sheet_xlsx.System[system]) is str:
                
                Catalog[sheet_xlsx.System[system]] = {}
                
                for subsystem in range(0,len(sheet_xlsx.Subsystem)):
                    if type(sheet_xlsx.Subsystem[subsystem]) is str:
                        
                        if sheet_xlsx.Subsystem[subsystem][0:2]==sheet_xlsx.System[system][0:2]:
                            Catalog[sheet_xlsx.System[system]][sheet_xlsx.Subsystem[subsystem]]={}
                        
                            for part in range(0,len(sheet_xlsx.Part)):
                                if type(sheet_xlsx.System_code[part]) is str:
                                
                                    if sheet_xlsx.System_code[part][0:3]==sheet_xlsx.Subsystem[subsystem][0:3]:
                                        Catalog[sheet_xlsx.System[system]][sheet_xlsx.Subsystem[subsystem]][sheet_xlsx.Part[part]]={}

I am thinking about using the line indexes instead to read whatever parts or subsystems or systems are contained within their parent group. Any input or tipp on a better way to approach this issue would be really helpful!

1 Answers

you can use ffill() to copy your indented groups across rather than looping over rows and columns in a pandas Dataframe. Loops in pandas are an anti-pattern.

sheet_xlsx = pandas.read_excel(xlsx_file)

sheet_xlsx['Segment'] = sheet_xlsx['Segment'].ffill(axis=0)
sheet_xlsx['System'] = sheet_xlsx['System'].ffill(axis=0)
sheet_xlsx['Subsystem'] = sheet_xlsx['Subsystem'].ffill(axis=0)
sheet_xlsx = sheet_xlsx.dropna(subset='Part')
sheet_xlsx = sheet_xlsx.set_index(['Segment', 'System', 'Subsystem', 'Part'])
sheet.xlsx.to_json()
Related