表情符号计数和分析使用Python pandas

huangapple
go评论107阅读模式
英文:

Emoji count and analysis using python pandas

问题

以下是您要翻译的代码部分：

进行笑脸计数的代码

  1. import pandas as pd
  2. import regex as re
  3. import emoji
  5. # 假设您的 DataFrame 名为 'df'，包含评论的列名为 'Document'
  6. comments = df['Document']
  8. # 初始化一个空字典以存储笑脸计数和类型
  9. smiley_data = {'Smiley': [], 'Count': [], 'Type': []}
  11. # 定义一个正则表达式模式以匹配笑脸
  12. pattern = r'([\U0001F600-\U0001F64F\U0001F300-\U0001F5FF\U0001F680-\U0001F6FF\U0001F1E0-\U0001F1FF])'
  14. # 遍历评论
  15. for comment in comments:
  16.     # 从评论中提取笑脸及其类型
  17.     smileys = re.findall(pattern, comment)
  18.     
  19.     # 增加计数并存储笑脸及其类型
  20.     for smiley in smileys:
  21.         if smiley in smiley_data['Smiley']:
  22.             index = smiley_data['Smiley'].index(smiley)
  23.             smiley_data['Count'][index] += 1
  24.         else:
  25.             smiley_data['Smiley'].append(smiley)
  26.             smiley_data['Count'].append(1)
  27.             smiley_data['Type'].append(emoji.demojize(smiley))
  28.             
  29. # 从笑脸数据创建一个 DataFrame
  30. smiley_df = pd.DataFrame(smiley_data)
  32. # 按计数降序排序 DataFrame
  33. smiley_df = smiley_df.sort_values(by='Count', ascending=False)
  35. # 打印笑脸数据
  36. smiley_df

我已经完成了代码的翻译。如果您需要任何其他帮助，请随时告诉我。

英文:

I am working on a sentiment analysis topic and there are a lot of comments with emojis.

I would like to know if my code is correct or is there a way to optimize it as well?

Code to do smiley count

  1. import pandas as pd
  2. import regex as re
  3. import emoji
  5. # Assuming your DataFrame is called 'df' and the column with comments is 'Document'
  6. comments = df['Document']
  8. # Initialize an empty dictionary to store smiley counts and types
  9. smiley_data = {'Smiley': [], 'Count': [], 'Type': []}
  11. # Define a regular expression pattern to match smileys
  12. pattern = r'([\U0001F600-\U0001F64F\U0001F300-\U0001F5FF\U0001F680-\U0001F6FF\U0001F1E0-\U0001F1FF])'
  14. # Iterate over the comments
  15. for comment in comments:
  16.     # Extract smileys and their types from the comment
  17.     smileys = re.findall(pattern, comment)
  18.     
  19.     # Increment the count and store the smileys and their types
  20.     for smiley in smileys:
  21.         if smiley in smiley_data['Smiley']:
  22.             index = smiley_data['Smiley'].index(smiley)
  23.             smiley_data['Count'][index] += 1
  24.         else:
  25.             smiley_data['Smiley'].append(smiley)
  26.             smiley_data['Count'].append(1)
  27.             smiley_data['Type'].append(emoji.demojize(smiley))
  28.             
  29. # Create a DataFrame from the smiley data
  30. smiley_df = pd.DataFrame(smiley_data)
  32. # Sort the DataFrame by count in descending order
  33. smiley_df = smiley_df.sort_values(by='Count', ascending=False)
  35. # Print the smiley data
  36. smiley_df

I am majorly not sure if my below code block is getting all the smileys

  1. # Define a regular expression pattern to match smileys
  2. pattern = r'([\U0001F600-\U0001F64F\U0001F300-\U0001F5FF\U0001F680-\U0001F6FF\U0001F1E0-\U0001F1FF])'

would like to know what can I do with this analysis. something else on top of it - some charts maybe?

I am also sharing a test dataset that will generate similar smiley counts as those available in my real data. Please note that the test dataset only has known smileys if there is something else. it won't be there like in a real dataset.

Test Dataset

  1. import random
  2. import pandas as pd
  4. smileys = ['👍', '👌', '😍', '🏻', '😊', '🙂', '👎', '😃', '🏼', '💩']
  6. # Additional smileys to complete the required count
  7. additional_smileys = ['😄', '😎', '🤩', '😘', '🤗', '😆', '😉', '😋', '😇', '🥳', '🙌', '🎉', '🔥', '🥰', '🤪', '😜', '🤓',
  8.                       '😚', '🤭', '🤫', '😌', '🥱', '🥶', '🤮', '🤡', '😑', '😴', '🙄', '😮', '🤥', '😢', '🤐', '🙈', '🙊',
  9.                       '👽', '🤖', '🦄', '🐼', '🐵', '🦁', '🐸', '🦉']
  11. # Combine the required smileys and additional smileys
  12. all_smileys = smileys + additional_smileys
  14. # Set a random seed for reproducibility
  15. random.seed(42)
  17. # Generate a single review
  18. def generate_review(with_smiley=False):
  19.     review = "This movie"
  20.     if with_smiley:
  21.         review += " " + random.choice(all_smileys)
  22.     review += " is "
  23.     review += random.choice(["amazing", "excellent", "fantastic", "brilliant", "great", "good", "okay", "average",
  24.                              "mediocre", "disappointing", "terrible", "awful", "horrible"])
  25.     review += random.choice(["!", "!!", "!!!", ".", "..", "..."]) + " "
  26.     review += random.choice(["Highly recommended", "Definitely worth watching", "A must-see", "I loved it",
  27.                              "Not worth your time", "Skip it"]) + random.choice(["!", "!!", "!!!"])
  28.     return review
  30. # Generate the random dataset
  31. def generate_dataset():
  32.     dataset = []
  33.     review_count = 5000
  35.     # Generate reviews with top smileys
  36.     for smiley, count, _ in top_smileys:
  37.         while count > 0:
  38.             review = generate_review(with_smiley=True)
  39.             if smiley in review:
  40.                 dataset.append(review)
  41.                 count -= 1
  43.     # Generate reviews with additional smileys
  44.     additional_smileys_count = len(additional_smileys)
  45.     additional_smileys_per_review = review_count - len(dataset)
  46.     additional_smileys_per_review = min(additional_smileys_per_review, additional_smileys_count)
  48.     for _ in range(additional_smileys_per_review):
  49.         review = generate_review(with_smiley=True)
  50.         dataset.append(review)
  52.     # Generate reviews without smileys
  53.     while len(dataset) < review_count:
  54.         review = generate_review()
  55.         dataset.append(review)
  57.     # Shuffle the dataset
  58.     random.shuffle(dataset)
  59.     return dataset
  61. # List of top smileys and their counts
  62. top_smileys = [
  63.     ('👍', 331, ':thumbs_up:'),
  64.     ('👌', 50, ':OK_hand:'),
  65.     ('😍', 41, ':smiling_face_with_heart-eyes:'),
  66.     ('🏻', 38, ':light_skin_tone:'),
  67.     ('😊', 35, ':smiling_face_with_smiling_eyes:'),
  68.     ('🙂', 14, ':slightly_smiling_face:'),
  69.     ('👎', 12, ':thumbs_down:'),
  70.     ('😃', 12, ':grinning_face_with_big_eyes:'),
  71.     ('🏼', 10, ':medium-light_skin_tone:'),
  72.     ('💩', 10, ':pile_of_poo:')
  73. ]
  75. # Generate the dataset
  76. dataset = generate_dataset()
  78. # Create a data frame with 'Document' column
  79. df = pd.DataFrame({'Document': dataset})
  81. # Display the DataFrame
  82. df

Thank you in advance!

答案1

得分: 3

更新

如果您更喜欢使用emoji包，您可以这样做：

  1. import emoji
  3. text = df['Document'].str.cat(sep='\n')
  4. out = (pd.DataFrame(emoji.emoji_list(text)).value_counts('emoji')
  5.          .rename_axis('Smiley').rename('Count').reset_index()
  6.          .assign(Type=lambda x: x['Smiley'].apply(emoji.demojize)))

输出：

  1. >>> out
  2.    Smiley  Count                              Type
  3. 0   👍      331   :thumbs_up:
  4. 1   👌      50     :OK_hand:
  5. 2   👳      41     :light_skin_tone:
  6. 3   😍      41     :smiling_face_with_heart-eyes:
  7. 4   😊      35     :smiling_face_with_smiling_eyes:
  8. 5   🙂      15     :slightly_smiling_face:
  9. 6   👎      14     :thumbs_down:
  10. 7   😇      13     :grinning_face_with_big_eyes:
  11. 8   💩      10     :pile_of_poo:
  12. 9   👶🏼     10     :medium-light_skin_tone:
  13. 10  😜      3      :winking_face_with_tongue:
  14. 11  🦉      3      :owl:
  15. 12  🤖      2      :robot:
  16. 13  😵      2      :expressionless_face:
  17. 14  👽      2      :alien:
  18. 15  🤫      2      :shushing_face:
  19. 16  😂      2      :crying_face:
  20. 17  🤪      2      :zany_face:
  21. 18  🙈      2      :see-no-evil_monkey:
  22. 19  🙉      2      :speak-no-evil_monkey:
  23. 20  😇      1      :smiling_face_with_halo:
  24. 21  🤮      1      :face_vomiting:
  25. 22  🤥      1      :face_with_hand_over_mouth:
  26. 23  🤡      1      :clown_face:
  27. 24  🙏      1      :smiling_face_with_open_hands:
  28. 25  🙄      1      :face_with_rolling_eyes:
  29. 26  😲      1      :grinning_squinting_face:
  30. 27  🐸      1      :frog:
  31. 28  😞      1      :face_with_open_mouth:
  32. 29  🐼      1      :panda:
  33. 30  😘      1      :kissing_face_with_closed_eyes:
  34. 31  😎      1      :smiling_face_with_sunglasses:
  35. 32  😘      1      :face_blowing_a_kiss:

您可以使用str.extractall来避免循环，然后使用value_counts来计算出现次数。最后，对每个表情进行"demojize"（这是最慢的部分）：

  1. out = (df['Document'].str.extractall(pattern).value_counts()
  2.                      .rename_axis('Smiley').rename('Count').reset_index()
  3.                      .assign(Type=lambda x: x['Smiley'].apply(emoji.demojize)))

输出：

  1. >>> out
  2.    Smiley  Count                              Type
  3. 0   👍      331   :thumbs_up:
  4. 1   👌      50     :OK_hand:
  5. 2   👳      41     :light_skin_tone:
  6. 3   😍      41     :smiling_face_with_heart-eyes:
  7. 4   😊      35     :smiling_face_with_smiling_eyes:
  8. 5   🙂      15     :slightly_smiling_face:
  9. 6   👎      14     :thumbs_down:
  10. 7   😇      13     :grinning_face_with_big_eyes:
  11. 8   💩      10     :pile_of_poo:
  12. 9   👶🏼     10     :medium-light_skin_tone:
  13. 10  😜      3      :winking_face_with_tongue:
  14. 11  😵      2      :expressionless_face:
  15. 12  🙉      2      :see-no-evil_monkey:
  16. 13  😭      2      :crying_face:
  17. 14  🙊      2      :speak-no-evil_monkey:
  18. 15  👽      2      :alien:
  19. 16  😷      1      :face_with_hand_over_mouth:
  20. 17  🤢      1      :face_vomiting:
  21. 18  🤡      1      :clown_face:
  22. 19  😇      1      :smiling_face_with_open_hands:
  23. 20  🙄      1      :face_with_rolling_eyes:
  24. 21  😳      1      :grinning_squinting_face:
  25. 22  🐸      1      :frog:
  26. 23  😚      1      :face_with_open_mouth:
  27. 24  🐼      1      :panda:
  28. 25  😘      1      :kissing_face_with_closed_eyes:
  29. 26  😎      1      :smiling_face_with_sunglasses:
  30. 27  😘      1      :face_blowing_a_kiss:

模式部分正确吗？我没有漏掉任何表情吗？

您的模式是不正确的。我不知道您想要提取的完整列表，但下面是一个用于调试的代码：

  1. # 添加拉丁1代码 --v
  2. pattern2 = '([\\U00000000-\\U000000FF\\U0001F600-\\U0001F64F\\U0001F300-\\U0001F5FF\\U0001F680-\\U0001F6FF\\U0001F1E0-\\U0001F1FF])'
  4. other = df['Document'].str.replace(pattern2, '', regex=True)
  5. print(other[other != ''])
  7. # 输出/错过的表情
  8. 1149    🤗
  9. 1238    🦉
  10. 1305    🤫
  11. 1424    🤫
  12. 1978    🤭
  13. 2611    🤮
  14. 2623    🦉
  15. 2959    🤡
  16. 3717    🤪
  17. 4045    🦉
  18. 4067    🤖
  19. 4699    🤖
  20. 4975    🤪
  21. Name: Document, dtype: object
英文:

Update

If you prefer to use emoji package, you can do:

  1. import emoji
  3. text = df['Document'].str.cat(sep='\n')
  4. out = (pd.DataFrame(emoji.emoji_list(text)).value_counts('emoji')
  5.          .rename_axis('Smiley').rename('Count').reset_index()
  6.          .assign(Type=lambda x: x['Smiley'].apply(emoji.demojize)))

Output:

  1. >>> out
  2.    Smiley  Count                              Type
  3. 0       👍    331                       :thumbs_up:
  4. 1       👌     50                         :OK_hand:
  5. 2       🏻     41                 :light_skin_tone:
  6. 3       😍     41    :smiling_face_with_heart-eyes:
  7. 4       😊     35  :smiling_face_with_smiling_eyes:
  8. 5       🙂     15           :slightly_smiling_face:
  9. 6       👎     14                     :thumbs_down:
  10. 7       😃     13     :grinning_face_with_big_eyes:
  11. 8       🏼     10          :medium-light_skin_tone:
  12. 9       💩     10                     :pile_of_poo:
  13. 10      😜      3        :winking_face_with_tongue:
  14. 11      🦉      3                             :owl:
  15. 12      🤖      2                           :robot:
  16. 13      😑      2             :expressionless_face:
  17. 14      👽      2                           :alien:
  18. 15      🤫      2                   :shushing_face:
  19. 16      😢      2                     :crying_face:
  20. 17      🤪      2                       :zany_face:
  21. 18      🙈      2              :see-no-evil_monkey:
  22. 19      🙊      2            :speak-no-evil_monkey:
  23. 20      😇      1          :smiling_face_with_halo:
  24. 21      🤮      1                   :face_vomiting:
  25. 22      🤭      1       :face_with_hand_over_mouth:
  26. 23      🤡      1                      :clown_face:
  27. 24      🤗      1    :smiling_face_with_open_hands:
  28. 25      🙄      1          :face_with_rolling_eyes:
  29. 26      😆      1         :grinning_squinting_face:
  30. 27      🐸      1                            :frog:
  31. 28      😮      1            :face_with_open_mouth:
  32. 29      🐼      1                           :panda:
  33. 30      😚      1   :kissing_face_with_closed_eyes:
  34. 31      😎      1    :smiling_face_with_sunglasses:
  35. 32      😘      1             :face_blowing_a_kiss:

You can use str.extractall to avoid a loop then use value_counts to count the number of occurences. Finally, "demojize" each smiley (the slowest part):

  1. out = (df['Document'].str.extractall(pattern).value_counts()
  2.                      .rename_axis('Smiley').rename('Count').reset_index()
  3.                      .assign(Type=lambda x: x['Smiley'].apply(emoji.demojize)))

Output:

  1. >>> out
  2.    Smiley  Count                              Type
  3. 0       👍    331                       :thumbs_up:
  4. 1       👌     50                         :OK_hand:
  5. 2       🏻     41                 :light_skin_tone:
  6. 3       😍     41    :smiling_face_with_heart-eyes:
  7. 4       😊     35  :smiling_face_with_smiling_eyes:
  8. 5       🙂     15           :slightly_smiling_face:
  9. 6       👎     14                     :thumbs_down:
  10. 7       😃     13     :grinning_face_with_big_eyes:
  11. 8       💩     10                     :pile_of_poo:
  12. 9       🏼     10          :medium-light_skin_tone:
  13. 10      😜      3        :winking_face_with_tongue:
  14. 11      😑      2             :expressionless_face:
  15. 12      🙈      2              :see-no-evil_monkey:
  16. 13      😢      2                     :crying_face:
  17. 14      🙊      2            :speak-no-evil_monkey:
  18. 15      👽      2                           :alien:
  19. 16      😎      1    :smiling_face_with_sunglasses:
  20. 17      😘      1             :face_blowing_a_kiss:
  21. 18      😚      1   :kissing_face_with_closed_eyes:
  22. 19      🐸      1                            :frog:
  23. 20      😇      1          :smiling_face_with_halo:
  24. 21      😮      1            :face_with_open_mouth:
  25. 22      😆      1         :grinning_squinting_face:
  26. 23      🙄      1          :face_with_rolling_eyes:
  27. 24      🐼      1                           :panda:

> The pattern part is correct? I am not missing out on any emoticons?

Your pattern is not right. I don't know the full list you want to extract but below you have a code to debug it:

  1. #     add latin1 codes --v
  2. pattern2 = '([\\U00000000-\\U000000FF\\U0001F600-\\U0001F64F\\U0001F300-\\U0001F5FF\\U0001F680-\\U0001F6FF\\U0001F1E0-\\U0001F1FF])'
  4. other = df['Document'].str.replace(pattern2, '', regex=True)
  5. print(other[other != ''])
  7. # Output / Missed emojis
  8. 1149    🤗
  9. 1238    🦉
  10. 1305    🤫
  11. 1424    🤫
  12. 1978    🤭
  13. 2611    🤮
  14. 2623    🦉
  15. 2959    🤡
  16. 3717    🤪
  17. 4045    🦉
  18. 4067    🤖
  19. 4699    🤖
  20. 4975    🤪
  21. Name: Document, dtype: object

答案2

得分: 2

感谢 @corralien 和 @cuzi，我能够使用下面的代码获得最终结果。它不使用模式，而是使用emoji.analyze(text, join_emoji=True)函数：

  1. import emoji
  3. out = (df['Document'].apply(lambda text: [token.chars for token in emoji.analyze(text, join_emoji=True) 
  4.                        if isinstance(token.value, emoji.EmojiMatch)]).explode().value_counts()
  5.                       .rename_axis('Smiley').rename('Count').reset_index())
  7. out

> 输出

  1. index	Smiley	Count
  2. 0	👍	331
  3. 1	👌	50
  4. 2	😍	41
  5. 3	🏻	41
  6. 4	😊	35
  7. 5	🙂	15
  8. 6	👎	14
  9. 7	😃	13
  10. 8	🏼	10
  11. 9	💩	10
  12. 10	😜	3
  13. 11	🦉	3
  14. 12	🤖	2
  15. 13	🤪	2
  16. 14	😢	2
  17. 15	🙈	2
  18. 16	😑	2
  19. 17	🤫	2
  20. 18	🙊	2
  21. 19	👽	2
  22. 20	🤭	1
  23. 21	🤗	1
  24. 22	😇	1
  25. 23	🐸	1
  26. 24	🤮	1
  27. 25	🤡	1
  28. 26	😚	1
  29. 27	😎	1
  30. 28	😘	1
  31. 29	🐼	1
  32. 30	😆	1
  33. 31	😮	1
  34. 32	🙄	1
英文:

Thanks to @corralien and @cuzi, I was able to get my final result using the below code. It doesn't use patterns but uses emoji.analyze(text, join_emoji=True) function: -

  1. import emoji
  3. out = (df['Document'].apply(lambda text: [token.chars for token in emoji.analyze(text, join_emoji=True) 
  4.                        if isinstance(token.value, emoji.EmojiMatch)]).explode().value_counts()
  5.                       .rename_axis('Smiley').rename('Count').reset_index())
  7. out

> Output

  1. index	Smiley	Count
  2. 0	👍	331
  3. 1	👌	50
  4. 2	😍	41
  5. 3	🏻	41
  6. 4	😊	35
  7. 5	🙂	15
  8. 6	👎	14
  9. 7	😃	13
  10. 8	🏼	10
  11. 9	💩	10
  12. 10	😜	3
  13. 11	🦉	3
  14. 12	🤖	2
  15. 13	🤪	2
  16. 14	😢	2
  17. 15	🙈	2
  18. 16	😑	2
  19. 17	🤫	2
  20. 18	🙊	2
  21. 19	👽	2
  22. 20	🤭	1
  23. 21	🤗	1
  24. 22	😇	1
  25. 23	🐸	1
  26. 24	🤮	1
  27. 25	🤡	1
  28. 26	😚	1
  29. 27	😎	1
  30. 28	😘	1
  31. 29	🐼	1
  32. 30	😆	1
  33. 31	😮	1
  34. 32	🙄	1

huangapple
  • 本文由 发表于 2023年6月27日 17:30:34
  • 转载请务必保留本文链接：https://go.coder-hub.com/76563480.html
  • emoji
  • emoticons
  • pandas
  • python
  • sentiment-analysis
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定