TAGS :Viewed: 16 - Published at: a few seconds ago

[ How to concat a dataframe in pandas? ]

I am fetching the data from mongoDB to python through pymongo and then converting it into pandas dataframe

df = pd.DataFrame(list(db.dataset2.find()))

This is how data looks like in mongoDB.

"dish" : [
      {
        "dish_id"          : "005" ,
        "dish_name"        : "Sandwitch",
        "dish_price"       : 50,
        "coupon_applied"   : "Yes",              
        "coupon_type"      : "Rs 20 off"
      },
      {
        "dish_id"          : "006" ,
        "dish_name"        : "Chicken Hundi",
        "dish_price"       : 125,
        "coupon_applied"   : "No",
        "coupon_type"      : "Null"

      }
   ],

I want to seperate dish attributes into two rows in pandas dataframe. Here is the code which does that. (There are 3 dish documents) so, I am iterating it through for loop.

for i in range(0,len(df.dish)):
data_dish = json_normalize(df['dish'][i])
print data_dish

But it gives me below output..

 coupon_applied   coupon_type   dish_id     dish_name       dish_price  
0            Yes   Rs 20 off     001     Chicken Biryani         120   
1             No        Null     001      Paneer Biryani         100   

coupon_applied  coupon_type     dish_id   dish_name        dish_price  
0        Yes       Rs 40 off     002     Mutton Biryani      130   
1        No          Null        004      Aaloo tikki         95   


coupon_applied   coupon_type    dish_id   dish_name        dish_price 
0     Yes         Rs 20 off      005      Sandwitch           50   
1     No             Null        006     Chicken Hundi        125   

And I want output in following format..

  coupon_applied   coupon_type   dish_id     dish_name       dish_price  
0     Yes          Rs 20 off      001     Chicken Biryani      120   
1     No             Null         001      Paneer Biryani      100   
2     Yes          Rs 40 off      002     Mutton Biryani       130   
3     No             Null         004      Aaloo tikki         95   
4     Yes         Rs 20 off       005      Sandwitch           50   
5     No             Null         006     Chicken Hundi        125   

Can you help me with this? thanks in advance :)

Answer 1


There is

dishes = [json_normalize(d) for d in df['dish']]
df = pd.concat(dishes, ignore_index=True)

Answer 2


You should be able to get a list of dataframes in a list and then concat them.

Inizialize a new Dataframe:

df = pd.DataFrame()

Create an empty list of Dataframes:

dflist = []

Loop and append dataframes

for i in range(0,len(df.dish)):
    data_dish = json_normalize(df['dish'][i])
    dflist.append(data_dish)

Then concat the list into the full dataframe:

df = pd.concat(dflist, ignore_index=True)