Parsing JSON with nested Dict & Lists

Question

I'm pretty new to Python and using Libraries effectively. I'm currently stuck parsing a JSON to a Dataframe.

This is my dict :

   data =  {
 "event": {
  "id": 323725,
  "code": 981,
  "sport": "Football",
  "tournament": "Bresil D1",
  "name": "Sao Paulo - Fortaleza Ce",
  "homeTeam": "Sao Paulo",
  "awayTeam": "Fortaleza Ce",
  "markets": [
   {
    "marketName": "R\u00e9sultat Final",
    "status": "active",
    "antepost": false,
    "marketID": "0",
    "marketFixedID": "285",
    "columnType": 0,
    "marketGroupID": [
     "g_s-441_11111111",
     "g_s-441_261",
     "g_s-441_11111111",
     "g_s-441_11111111"
    ],
    "marketOrder": 1,
    "odds": [
     {
      "id": "323725_0_400",
      "name": "SAO PAULO",
      "short": "SAO PAULO",
      "clean": "1",
      "status": "active",
      "odd": 1.85,
      "handicap": 0
     },
     {
      "id": "323725_0_401",
      "name": "X",
      "short": "X",
      "clean": "X",
      "status": "active",
      "odd": 2.65,
      "handicap": 0
     },
     {
      "id": "323725_0_402",
      "name": "FORTALEZA CE",
      "short": "FORTALEZA CE",
      "clean": "2",
      "status": "active",
      "odd": 3.5,
      "handicap": 0
     }
    ],
    "minimumRestriction": 1
   },
   {
    "marketName": "R\u00e9sultat Final avec Handicap (0:1)",
    "status": "active",
    "antepost": false,
    "marketID": "10",
    "marketFixedID": "295",
    "columnType": 0,
    "marketGroupID": [
     "g_s-441_11111111",
     "g_s-441_265",
     "g_s-441_11111111",
     "g_s-441_11111111"
    ],
    "marketOrder": 2,
    "isHandicap": true,
    "var0": -1,
    "odds": [
     {
      "id": "323725_10_445",
      "name": "1",
      "short": "1",
      "clean": "1",
      "status": "active",
      "odd": 2.6,
      "handicap": 0
     },
     {
      "id": "323725_10_446",
      "name": "X",
      "short": "X",
      "clean": "X",
      "status": "active",
      "odd": 2.9,
      "handicap": 0
     },
     {
      "id": "323725_10_447",
      "name": "2",
      "short": "2",
      "clean": "2",
      "status": "active",
      "odd": 1.75,
      "handicap": 0
     }
    ],
    "minimumRestriction": 1
   },
   {
    "marketName": "Moins\/Plus 1,5",
    "status": "active",
    "antepost": false,
    "marketID": "19",
    "marketFixedID": "304",
    "columnType": 2,
    "marketGroupID": [
     "g_s-441_11111111",
     "g_s-441_11111111",
     "g_s-441_11111111",
     "g_s-441_11111111"
    ],
    "marketOrder": 4,
    "isHandicap": false,
    "var0": 1.5,
    "odds": [
     {
      "id": "323725_19_469",
      "name": "Moins ",
      "short": "Moins ",
      "clean": "Moins ",
      "status": "active",
      "odd": 2.65,
      "handicap": 0
     },
     {
      "id": "323725_19_470",
      "name": "Plus",
      "short": "Plus",
      "clean": "Plus",
      "status": "active",
      "odd": 1.15,
      "handicap": 0
     }
    ],
    "minimumRestriction": 1
   },
}

My goal is to parse this dict into a Dataframe containing "marketName" and each individual "odd" & it's "name", inside it.

First I tried to pull wanted data like this :

markets = data['event']['markets']
for m in markets:

 marketName= m['marketName']
 odds = m['odds']

From here I don't know how to deal with this data and put it correctly inside a coerent Dataframe

user_8708364 · Answer

Try using a dictionary comprehension and pd.DataFrame.join:

markets = pd.DataFrame({k: v for i in data['event']['markets'] for k, v in i.items() if k in ['marketName', 'odds']})
markets = markets[['marketName']].join(pd.DataFrame(markets['odds'].tolist()).add_prefix('odds_'))

And now:

print(markets)

Outputs:

        marketName odds_clean  odds_handicap        odds_id odds_name  \
0  Moins\/Plus 1,5     Moins               0  323725_19_469    Moins    
1  Moins\/Plus 1,5       Plus              0  323725_19_470      Plus   

   odds_odd odds_short odds_status  
0      2.65     Moins       active  
1      1.15       Plus      active

Parsing JSON with nested Dict & Lists

1 Answers