使用嵌套字典和列表创建Panda DataFrame：dict:{dict:{dict:[list]}}

huangapple
go评论102阅读模式
英文:

Create Panda DataFrame using nested dictionaries and a list: dict:{dict:{dict:[list]}}

问题

  1. data = {
  2.    "etherA": {
  3.       "vlanY": {
  4.          "local": ['mac01', 'mac02'],
  5.          "external": ['mac03', 'mac02']
  6.       }
  7.    },
  8.    "etherB": {
  9.       "vlanZ": {
  10.          "local": ['mac06', 'mac09'],
  11.          "external": ['mac01', 'mac02', 'mac03']
  12.       }
  13.    }
  14. }
  16. import pandas as pd
  18. # Create an empty DataFrame with the desired column names
  19. df = pd.DataFrame(columns=['interface', 'vlan', 'dyn', 'mac-address'])
  21. # Loop through the nested dictionary and flatten the data
  22. for interface, nested_dict in data.items():
  23.     for vlan, dyn_dict in nested_dict.items():
  24.         for dyn, mac_list in dyn_dict.items():
  25.             for mac in mac_list:
  26.                 df = df.append({'interface': interface, 'vlan': vlan, 'dyn': dyn, 'mac-address': mac}, ignore_index=True)
  28. # Print the resulting DataFrame
  29. print(df)

This code will create the DataFrame you desire from the nested dictionary without using multiple for loops.

英文:

I have a series of nested dicts with a list as the deepest value.

  1. data = {
  2.    "etherA": {
  3.       "vlanY": {
  4.          "local": ['mac01', 'mac02'],
  5.          "external": ['mac03', 'mac02']
  6.       }
  7.    },
  8.    "etherB": {
  9.       "vlanZ": {
  10.          "local": ['mac06', 'mac09'],
  11.          "external": ['mac01', 'mac02', 'mac03']
  12.       }
  13.    }
  14. }

To load the dict into a dataframe, I create the column headers and then loop through the dict and add a list to the end of the dataframe.

  1. df = pd.DataFrame.from_dict({
  2.    'interface': [],
  3.    'vlan': [],
  4.    'dyn': [],
  5.    'mac-address': []
  6. })
  8. for a in data:
  9.    for b in data[a]:
  10.       for c in data[a][b]:
  11.          for d in data[a][b][c]:
  12.             df.loc[len(df)] = [a, b, c, d]

Final output:

  2. print(df)
  4.   interface   vlan       dyn mac-address
  5. 0    etherA  vlanY     local       mac01
  6. 1    etherA  vlanY     local       mac02
  7. 2    etherA  vlanY  external       mac03
  8. 3    etherA  vlanY  external       mac02
  9. 4    etherB  vlanZ     local       mac06
  10. 5    etherB  vlanZ     local       mac09
  11. 6    etherB  vlanZ  external       mac01
  12. 7    etherB  vlanZ  external       mac02
  13. 8    etherB  vlanZ  external       mac03

The "for loops" ultimately do what I need it to, but is there a panda method for getting the data from the dict into the dataframe?

I've read through numerous other posts and have tried their answers and suggestions. Most are dealing with a single nested dictionary and none have dealt with a nested, nested, nested list. A few of the suggested questions are what I was trying to achieve and the answer was to loop through to essentially flatten the data before appending it to the dataframe,so that may be the best course.

答案1

得分: 1

以下是翻译好的代码部分：

  1. import pandas as pd
  3. data = {
  4.    "etherA": {
  5.       "vlanY": {
  6.          "local": ['mac01', 'mac02'],
  7.          "external": ['mac03', 'mac02']
  8.       }
  9.    },
  10.    "etherB": {
  11.       "vlanZ": {
  12.          "local": ['mac06', 'mac09'],
  13.          "external": ['mac01', 'mac02', 'mac03']
  14.       }
  15.    }
  16. }
  18. df = pd.DataFrame([
  19.     {'interface': interface, 'vlan': vlan, 'dyn': dyn, 'mac-address': mac}
  20.     for interface, vlan_dict in data.items()
  21.     for vlan, dyn_dict in vlan_dict.items()
  22.     for dyn, mac_list in dyn_dict.items()
  23.     for mac in mac_list
  24. ])

这段代码生成的DataFrame如下：

  1.  interface   vlan       dyn mac-address
  2. 0    etherA  vlanY     local       mac01
  3. 1    etherA  vlanY     local       mac02
  4. 2    etherA  vlanY  external       mac03
  5. 3    etherA  vlanY  external       mac02
  6. 4    etherB  vlanZ     local       mac06
  7. 5    etherB  vlanZ     local       mac09
  8. 6    etherB  vlanZ  external       mac01
  9. 7    etherB  vlanZ  external       mac02
  10. 8    etherB  vlanZ  external       mac03
英文:

Another way to do this is:

  1. import pandas as pd
  3. data = {
  4.    "etherA": {
  5.       "vlanY": {
  6.          "local": ['mac01', 'mac02'],
  7.          "external": ['mac03', 'mac02']
  8.       }
  9.    },
  10.    "etherB": {
  11.       "vlanZ": {
  12.          "local": ['mac06', 'mac09'],
  13.          "external": ['mac01', 'mac02', 'mac03']
  14.       }
  15.    }
  16. }
  18. df = pd.DataFrame([
  19.     {'interface': interface, 'vlan': vlan, 'dyn': dyn, 'mac-address': mac}
  20.     for interface, vlan_dict in data.items()
  21.     for vlan, dyn_dict in vlan_dict.items()
  22.     for dyn, mac_list in dyn_dict.items()
  23.     for mac in mac_list
  24. ])

which gives

  1.  interface   vlan       dyn mac-address
  2. 0    etherA  vlanY     local       mac01
  3. 1    etherA  vlanY     local       mac02
  4. 2    etherA  vlanY  external       mac03
  5. 3    etherA  vlanY  external       mac02
  6. 4    etherB  vlanZ     local       mac06
  7. 5    etherB  vlanZ     local       mac09
  8. 6    etherB  vlanZ  external       mac01
  9. 7    etherB  vlanZ  external       mac02
  10. 8    etherB  vlanZ  external       mac03
  11. ​

答案2

得分: 0

以下是代码部分的翻译：

  1. 我建议首先创建元组列表：
  3. L = [(a, b, c, d) for a in data
  4.                for b in data[a]
  5.                for c in data[a][b]
  6.                for d in data[a][b][c]]
  7. df = pd.DataFrame(L, columns=['interface', 'vlan', 'dyn', 'mac-address'])
  9. 或者：
  11. L = [(a, b, c, d) for a, d in data.items()
  12.                for b, d1 in d.items()
  13.                for c, d2 in d1.items()
  14.                for d in d2]
  15. df = pd.DataFrame(L, columns=['interface', 'vlan', 'dyn', 'mac-address'])
  17. print(df)
  19.   interface   vlan       dyn mac-address
  20. 0    etherA  vlanY     local       mac01
  21. 1    etherA  vlanY     local       mac02
  22. 2    etherA  vlanY  external       mac03
  23. 3    etherA  vlanY  external       mac02
  24. 4    etherB  vlanZ     local       mac06
  25. 5    etherB  vlanZ     local       mac09
  26. 6    etherB  vlanZ  external       mac01
  27. 7    etherB  vlanZ  external       mac02
  28. 8    etherB  vlanZ  external       mac03
英文:

I suggest create list of tuples first:

  1. L = [(a,b,c,d) for a in data 
  2.                for b in data[a] 
  3.                for c in data[a][b] 
  4.                for d in data[a][b][c]]
  5. df = pd.DataFrame(L, columns=['interface','vlan','dyn','mac-address'])

Or:

  1. L = [(a,b,c,d) for a, d in data.items()
  2.                for b, d1 in d.items()
  3.                for c, d2 in d1.items()
  4.                for d in d2]
  5. df = pd.DataFrame(L, columns=['interface','vlan','dyn','mac-address'])
  7. print (df)
  9.   interface   vlan       dyn mac-address
  10. 0    etherA  vlanY     local       mac01
  11. 1    etherA  vlanY     local       mac02
  12. 2    etherA  vlanY  external       mac03
  13. 3    etherA  vlanY  external       mac02
  14. 4    etherB  vlanZ     local       mac06
  15. 5    etherB  vlanZ     local       mac09
  16. 6    etherB  vlanZ  external       mac01
  17. 7    etherB  vlanZ  external       mac02
  18. 8    etherB  vlanZ  external       mac03

答案3

得分: 0

  1. import pandas as pd
  3. data = {
  4.     "etherA": {
  5.         "vlanY": {
  6.             "local": ['mac01', 'mac02'],
  7.             "external": ['mac03', 'mac02']
  8.         }
  9.     },
  10.     "etherB": {
  11.         "vlanZ": {
  12.             "local": ['mac06', 'mac09'],
  13.             "external": ['mac01', 'mac02', 'mac03']
  14.         }
  15.     }
  16. }
  18. df = pd.json_normalize(data, sep='_')
  19. flatten_dict = df.to_dict(orient='records')[0]
  20. res = []
  21. for k, v in flatten_dict.items():
  22.     for i in v:
  23.         res.append(k.split("_")+[i])
  24. res_df = pd.DataFrame(res, columns=["interface", "vlan", "dyn", "mac-address"])
  25. print(res_df)
英文:

Firstly, you can flatten the nested dictionary using pd.json_normalize, then, you can build a list of lists and turn it into a DataFrame.

  1. import pandas as pd
  3. data = {
  4.     "etherA": {
  5.         "vlanY": {
  6.             "local": ['mac01', 'mac02'],
  7.             "external": ['mac03', 'mac02']
  8.         }
  9.     },
  10.     "etherB": {
  11.         "vlanZ": {
  12.             "local": ['mac06', 'mac09'],
  13.             "external": ['mac01', 'mac02', 'mac03']
  14.         }
  15.     }
  16. }
  18. df = pd.json_normalize(data, sep='_')
  19. flatten_dict = df.to_dict(orient='records')[0]
  20. res = []
  21. for k, v in flatten_dict.items():
  22.     for i in v:
  23.         res.append(k.split("_")+[i])
  24. res_df = pd.DataFrame(res, columns=["interface", "vlan", "dyn", "mac-address"])
  25. print(res_df)

huangapple
  • 本文由 发表于 2023年2月27日 13:51:04
  • 转载请务必保留本文链接：https://go.coder-hub.com/75577131.html
  • dataframe
  • pandas
  • python
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定