Perform multiple regex operations on each line of text file and store extracted data in respective column

huangapple go评论65阅读模式
英文:

Perform multiple regex operations on each line of text file and store extracted data in respective column

问题

以下是您要的翻译部分:

import pandas as pd
import re
columns = ['Request Type', 'Channel', 'AG']
exp = re.compile(r'<(.*)\s+xmlns'
                 r'<Channel>(.*?)</Channel>'
                 r'<Param Name="AG">.*?<Value>(.*?)</Value>')
final = []
with open("test.txt") as f:
    for line in f:
        result = re.search(exp, line)
        final.append(result)

df = pd.DataFrame(final, columns)
print(df)
import pandas as pd
import re
columns = ['Request Type', 'Channel', 'AG']
exp = re.compile(r'<(.*)\s+xmlns'
                 r'<Channel>(.*?)</Channel>'
                 r'<Param Name="AG">.*?<Value>(.*?)</Value>')
final = []
with open("test.txt") as f:
    for line in f:
        result = re.search(exp, line)
        final.append(result)

df = pd.DataFrame(final, columns)
print(df)

您的代码用于从文本文件中提取数据,并创建一个带有请求类型、通道和AG的DataFrame。希望这有助于您实现预期的输出。

英文:

Data in test.txt

&lt;ServiceRQ xmlns:xsi=&quot;http://&quot;&gt;&lt;SaleInfo&gt;&lt;CityCode&gt;DXB&lt;/CityCode&gt;&lt;CountryCode&gt;EG&lt;/CountryCode&gt;&lt;Currency&gt;USD&lt;/Currency&gt;&lt;Channel&gt;TA&lt;/Channel&gt;&lt;/SaleInfo&gt;&lt;Pricing&gt;&lt;CustomParams&gt;&lt;Param Name=&quot;AG&quot;&gt;&lt;Value&gt;95HAJSTI&lt;/Value&gt;&lt;/Param&gt;&lt;/CustomParams&gt;&lt;/Pricing&gt;&lt;/ServiceRQ&gt;

&lt;SearchRQ xmlns:xsi=&quot;http://&quot;&gt;&lt;SaleInfo&gt;&lt;CityCode&gt;CPT&lt;/CityCode&gt;&lt;CountryCode&gt;US&lt;/CountryCode&gt;&lt;Currency&gt;USD&lt;/Currency&gt;&lt;Channel&gt;AY&lt;/Channel&gt;&lt;/SaleInfo&gt;&lt;Pricing&gt;&lt;CustomParams&gt;&lt;Param Name=&quot;AG&quot;&gt;&lt;Value&gt;56ASJSTS&lt;/Value&gt;&lt;/Param&gt;&lt;/CustomParams&gt;&lt;/Pricing&gt;&lt;/SearchRQ&gt;

&lt;ServiceRQ xmlns:xsi=&quot;http://&quot;&gt;&lt;SaleInfo&gt;&lt;CityCode&gt;BOM&lt;/CityCode&gt;&lt;CountryCode&gt;AU&lt;/CountryCode&gt;&lt;Currency&gt;USD&lt;/Currency&gt;&lt;Channel&gt;QA&lt;/Channel&gt;&lt;/SaleInfo&gt;&lt;Pricing&gt;&lt;CustomParams&gt;&lt;Param Name=&quot;AG&quot;&gt;&lt;Value&gt;85ATAKSQ&lt;/Value&gt;&lt;/Param&gt;&lt;/CustomParams&gt;&lt;/Pricing&gt;&lt;/ServiceRQ&gt;

&lt;ServiceRQ ......

&lt;SearchRQ ........

My code:

import pandas as pd
import re
columns = [&#39;Request Type&#39;,&#39;Channel&#39;,&#39;AG&#39;]
# data = pd.DataFrame
exp = re.compile(r&#39;&lt;(.*)\s+xmlns&#39;
                 r&#39;&lt;Channel&gt;(.*)&lt;/Channel&gt;&#39;
                 r&#39;&lt;Param Name=&quot;AG&quot;&gt;.*?&lt;Value&gt;(.*?)&lt;/Value&gt;&#39;)
final = []
with open(r&quot;test.txt&quot;) as f:
    for line in f:
        result = re.search(exp,line)
        final.append(result)

    df = pd.DataFrame(final, columns)
    print(df)

My expected output is
I want to iterate through each line and to perform the 3 regex operation and extract data from each line in text file

1. r&#39;&lt;(.*)\s+xmlns&#39;
2. r&#39;&lt;Channel&gt;(.*)&lt;/Channel&gt;&#39;
3. r&#39;&lt;Param Name=&quot;AG&quot;&gt;.*?&lt;Value&gt;(.*?)&lt;/Value&gt;&#39;)

Each regex extract respective data from single line
like

  1. extract the type of request
  2. extract the name of channel
  3. extract the value present for AG

My expected output ExcelSheet

Request Type    Channel       AG
ServiceRQ         TA        95HAJSTI  
SearchRQ          AY        56ASJSTS
ServiceRQ         QA        85ATAKSQ
 ...              ...         .....
 ...              ....        .....
and so on..

How can I achieve expected output.

答案1

得分: 1

尝试使用这个re,实际上我不知道你的文本内容的其余部分是什么样的,但根据我目前所见,这将适用。result.groups() 将提取所有组的匹配元素,然后在附加之前返回一个元组。

exp = re.compile(r'<(\w+)\s+xmlns.*?>.*?'
                 r'<Channel>(.*?)</Channel>.*?'
                 r'<Param Name="AG"><Value>(.*?)</Value></Param>')
final = []
with open("test.txt") as f:
    for line in f:
        result = re.search(exp, line)
        if result:
            final.append(result.groups())
            
df = pd.DataFrame(final, columns=columns)
print(df)

测试代码:

import pandas as pd
import re

columns = ['Request Type', 'Channel', 'AG']

file_data = """
<ServiceRQ xmlns:xsi="http://"><SaleInfo><CityCode>DXB</CityCode><CountryCode>EG</CountryCode><Currency>USD</Currency><Channel>TA</Channel></SaleInfo><Pricing><CustomParams><Param Name="AG"><Value>95HAJSTI</Value></Param></CustomParams></Pricing></ServiceRQ>
<SearchRQ xmlns:xsi="http://"><SaleInfo><CityCode>CPT</CityCode><CountryCode>US</CountryCode><Currency>USD</Currency><Channel>AY</Channel></SaleInfo><Pricing><CustomParams><Param Name="AG"><Value>56ASJSTS</Value></Param></CustomParams></Pricing></SearchRQ>
<ServiceRQ xmlns:xsi="http://"><SaleInfo><CityCode>BOM</CityCode><CountryCode>AU</CountryCode><Currency>USD</Currency><Channel>QA</Channel></SaleInfo><Pricing><CustomParams><Param Name="AG"><Value>85ATAKSQ</Value></Param></CustomParams></Pricing></ServiceRQ>
"""

exp = re.compile(r'<(\w+)\s+xmlns.*?>.*?'
                 r'<Channel>(.*?)</Channel>.*?'
                 r'<Param Name="AG"><Value>(.*?)</Value></Param>')
final = []
for line in file_data.splitlines():
    result = re.search(exp, line)
    if result:
        final.append(result.groups())
        
df = pd.DataFrame(final, columns=columns)
print(df)

  Request Type Channel        AG
0    ServiceRQ      TA  95HAJSTI
1     SearchRQ      AY  56ASJSTS
2    ServiceRQ      QA  85ATAKSQ
英文:

Try this re, actually I don't Know how the rest of your text content looks like, but this will work with what I have seen so far.<br>
result.groups() will extract matching elements of all groups then return a tuple before appending.

exp = re.compile(r&#39;&lt;(\w+)\s+xmlns.*?&gt;.*?&#39;
                 r&#39;&lt;Channel&gt;(.*?)&lt;/Channel&gt;.*?&#39;
                 r&#39;&lt;Param Name=&quot;AG&quot;&gt;&lt;Value&gt;(.*?)&lt;/Value&gt;&#39;)
final = []
with open(r&quot;test.txt&quot;) as f:
    for line in f:
        result = re.search(exp,line)
        if result:
            final.append(result.groups())
            
df = pd.DataFrame(final, columns=columns)
print(df)

Test code:

import pandas as pd
import re

columns = [&#39;Request Type&#39;,&#39;Channel&#39;,&#39;AG&#39;]

file_data = &quot;&quot;&quot;
&lt;ServiceRQ xmlns:xsi=&quot;http://&quot;&gt;&lt;SaleInfo&gt;&lt;CityCode&gt;DXB&lt;/CityCode&gt;&lt;CountryCode&gt;EG&lt;/CountryCode&gt;&lt;Currency&gt;USD&lt;/Currency&gt;&lt;Channel&gt;TA&lt;/Channel&gt;&lt;/SaleInfo&gt;&lt;Pricing&gt;&lt;CustomParams&gt;&lt;Param Name=&quot;AG&quot;&gt;&lt;Value&gt;95HAJSTI&lt;/Value&gt;&lt;/Param&gt;&lt;/CustomParams&gt;&lt;/Pricing&gt;&lt;/ServiceRQ&gt;
&lt;SearchRQ xmlns:xsi=&quot;http://&quot;&gt;&lt;SaleInfo&gt;&lt;CityCode&gt;CPT&lt;/CityCode&gt;&lt;CountryCode&gt;US&lt;/CountryCode&gt;&lt;Currency&gt;USD&lt;/Currency&gt;&lt;Channel&gt;AY&lt;/Channel&gt;&lt;/SaleInfo&gt;&lt;Pricing&gt;&lt;CustomParams&gt;&lt;Param Name=&quot;AG&quot;&gt;&lt;Value&gt;56ASJSTS&lt;/Value&gt;&lt;/Param&gt;&lt;/CustomParams&gt;&lt;/Pricing&gt;&lt;/SearchRQ&gt;
&lt;ServiceRQ xmlns:xsi=&quot;http://&quot;&gt;&lt;SaleInfo&gt;&lt;CityCode&gt;BOM&lt;/CityCode&gt;&lt;CountryCode&gt;AU&lt;/CountryCode&gt;&lt;Currency&gt;USD&lt;/Currency&gt;&lt;Channel&gt;QA&lt;/Channel&gt;&lt;/SaleInfo&gt;&lt;Pricing&gt;&lt;CustomParams&gt;&lt;Param Name=&quot;AG&quot;&gt;&lt;Value&gt;85ATAKSQ&lt;/Value&gt;&lt;/Param&gt;&lt;/CustomParams&gt;&lt;/Pricing&gt;&lt;/ServiceRQ&gt;
&quot;&quot;&quot;

exp = re.compile(r&#39;&lt;(\w+)\s+xmlns.*?&gt;.*?&#39;
                 r&#39;&lt;Channel&gt;(.*?)&lt;/Channel&gt;.*?&#39;
                 r&#39;&lt;Param Name=&quot;AG&quot;&gt;&lt;Value&gt;(.*?)&lt;/Value&gt;&#39;)
final = []
for line in file_data.splitlines():
    result = re.search(exp,line)
    if result:
        final.append(result.groups())
        
df = pd.DataFrame(final, columns=columns)
print(df)


Request Type Channel        AG
0    ServiceRQ      TA  95HAJSTI
1     SearchRQ      AY  56ASJSTS
2    ServiceRQ      QA  85ATAKSQ

huangapple
  • 本文由 发表于 2023年2月8日 19:21:45
  • 转载请务必保留本文链接:https://go.coder-hub.com/75385051.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定