Searching for hidden API to scrape data with Python

huangapple
go评论65阅读模式
英文:

Searching for hidden API to scrape data with Python

问题

I've successfully scraped data from this site:
https://football.goaloo2.com/subleague/2022-2023/40/261/?selectround=23

Now after some hints and some basic tutorials, I'm trying to scrape it using a hidden API... but on this site, I got something different from others I was able to scrape. This is my process: with Chrome, I do inspection/network and then Fetch/XHR. Reloading the page, I got what I thought would be the call (even if the preview is different from other scraped sites, there's no tree to navigate). Then I copy it as a cURL command, and after pasting it into a Python conversion site, I obtained this:

import requests

params = {
    'sclassId': '40',
    'subSclassId': '261',
    'matchSeason': '2022-2023',
    'round': '23',
    'flesh': '0.6807624444063407',
}

response = requests.get('https://football.goaloo2.com/ajax/LeagueOddsAjax', params=params)

odds = response.json()

I got this error: JSONDecodeError: Expecting value

I suspect the data is not embedded in a JSON object. Any idea on how to proceed if it is doable in Python?

英文:

i've succesfully scraped data from this site:
https://football.goaloo2.com/subleague/2022-2023/40/261/?selectround=23

Now after some hints and some basic tutorials i'm trying to scrape it using hidden API...but on this site i got something different from others i was able to scrape. This is my process: with chrome i do inspection/network and then Fetch/XHR, reloadong the page i got what i thought it would be the call (even if the preview is different from other scraped sites, there's no three to navigate); than i copy as curl(cmd) and after pasting in converting site to python i obtained this:

import requests

params = {
    'sclassId': '40',
    'subSclassId': '261',
    'matchSeason': '2022-2023',
    'round': '23',
    'flesh': '0.6807624444063407',
}
response = requests.get('https://football.goaloo2.com/ajax/LeagueOddsAjax', params=params)


odds = response.json()

I got this error: JSONDecodeError: Expecting value

I suspect data is not embedded in a json object, any idea on how to proceede if it is doable in python?

答案1

得分: 1

返回的对象不是可解码的JSON。您可以提取原始文本并将其转换为JSON格式，然后手动使用json.loads()函数加载它。请注意，网页抓取不总是标准化的，您应该检查操作的稳定性。一个非常快速而不太干净的转换如下：

d = "{\"OddsData\": {" + response.text.replace('=', ':').replace(';', ',').replace('":]', '":"').replace('oddsData[', '').replace(',,\n', '}}')
json.loads(d)

它返回如下：

{'OddsData': {'O_2247562': [[4, 2.9, 2.95, 2.6],
[16, 3, 2.95, 2.2],
[18, 2.78, 3.1, 2.53],
[60, 2.9, 2.9, 2.6],
[70, 3.1, 3, 2.4],
[81, 2.9, 3, 2.63],
[82, 2.7, 2.9, 2.45],
[88, 2.8, 3, 2.55],
[90, 2.9, 3.3, 2.33],
[104, 2.85, 2.9, 2.6],
[110, 3.05, 3.15, 2.45],
[115, 2.9, 2.9, 2.6],
[158, 2.9, 3.1, 2.45],
[173, 2.8, 2.8, 2.55],
[177, 3.02, 3.25, 2.57],
[255, 2.7, 3.25, 2.5],
[281, 2.9, 3.1, 2.38],
[370, 2.55, 2.6, 2.35],
[422, 2.75, 3.2, 2.5],
[450, 2.85, 2.95, 2.6],
[474, 2.8, 3.1, 2.49],
[499, 2.74, 3.25, 2.47],
[517, 2.78, 3.1, 2.53],
[545, 2.74, 3.25, 2.47],
[659, 2.7, 3.07, 2.46],
[976, 2.8, 3.15, 2.5]],
'L_2247562': [[3, 1.04, 0, 0.84],
[8, 1.2, 0, 0.7],
[12, 1.15, 0, 0.77],
[14, 0.95, 0, 0.8],
[17, 1.05, 0, 0.85],
[19, 1.1, 0, 0.65],
[22, 0.78, -0.25, 0.91],
[23, 1.05, 0, 0.87],
[24, 1.05, 0, 0.85],
[31, 0.8, -0.25, 1.13],
[35, 1.05, 0, 0.85],
[42, 0.99, 0, 0.8]],
'T_2247562': [[3, 1.05, 2.25, 0.81],
[4, 1.3, 2.5, 0.57],
[8, 1, 2.25, 0.85],
[9, 1.3, 2.5, 0.57],
[12, 1.05, 2.25, 0.83],
[14, 1, 2.25, 0.78],
[17, 1.08, 2.25, 0.8],
[19, 1.3, 2.5, 0.55],
[22, 0.87, 2.25, 0.78],
[23, 1.06, 2.25, 0.84],
[24, 1.
<details>
<summary>英文:</summary>
The returned object is not decodable JSON. You can take out the raw text and manipulate it into the JSON format. Then manually load it using json.loads() function. Note that, the web scrapping is not always standartized and you should check the stability of the manipulation. A very quick and dirty transformation as follows;
d = &quot;{\&quot;OddsData\&quot;: {&quot; + response.text.replace(&#39;=&#39;, &#39;:&#39;).replace(&#39;;&#39;, &#39;,&#39;).replace(&#39;&quot;]:&#39;, &#39;&quot;:&#39;).replace(&#39;oddsData[&#39;, &#39;&#39;).replace(&#39;,,\n&#39;, &#39;}}&#39;)
json.loads(d)
It returns as follows;
{&#39;OddsData&#39;: {&#39;O_2247562&#39;: [[4, 2.9, 2.95, 2.6],
[16, 3, 2.95, 2.2],
[18, 2.78, 3.1, 2.53],
[60, 2.9, 2.9, 2.6],
[70, 3.1, 3, 2.4],
[81, 2.9, 3, 2.63],
[82, 2.7, 2.9, 2.45],
[88, 2.8, 3, 2.55],
[90, 2.9, 3.3, 2.33],
[104, 2.85, 2.9, 2.6],
[110, 3.05, 3.15, 2.45],
[115, 2.9, 2.9, 2.6],
[158, 2.9, 3.1, 2.45],
[173, 2.8, 2.8, 2.55],
[177, 3.02, 3.25, 2.57],
[255, 2.7, 3.25, 2.5],
[281, 2.9, 3.1, 2.38],
[370, 2.55, 2.6, 2.35],
[422, 2.75, 3.2, 2.5],
[450, 2.85, 2.95, 2.6],
[474, 2.8, 3.1, 2.49],
[499, 2.74, 3.25, 2.47],
[517, 2.78, 3.1, 2.53],
[545, 2.74, 3.25, 2.47],
[659, 2.7, 3.07, 2.46],
[976, 2.8, 3.15, 2.5]],
&#39;L_2247562&#39;: [[3, 1.04, 0, 0.84],
[8, 1.2, 0, 0.7],
[12, 1.15, 0, 0.77],
[14, 0.95, 0, 0.8],
[17, 1.05, 0, 0.85],
[19, 1.1, 0, 0.65],
[22, 0.78, -0.25, 0.91],
[23, 1.05, 0, 0.87],
[24, 1.05, 0, 0.85],
[31, 0.8, -0.25, 1.13],
[35, 1.05, 0, 0.85],
[42, 0.99, 0, 0.8]],
&#39;T_2247562&#39;: [[3, 1.05, 2.25, 0.81],
[4, 1.3, 2.5, 0.57],
[8, 1, 2.25, 0.85],
[9, 1.3, 2.5, 0.57],
[12, 1.05, 2.25, 0.83],
[14, 1, 2.25, 0.78],
[17, 1.08, 2.25, 0.8],
[19, 1.3, 2.5, 0.55],
[22, 0.87, 2.25, 0.78],
[23, 1.06, 2.25, 0.84],
[24, 1.08, 2.25, 0.8],
[31, 1.05, 2.25, 0.85],
[35, 1.09, 2.25, 0.81],
[42, 1.04, 2.25, 0.76],
[49, 1.25, 2.5, 0.57]],
&#39;O_2247563&#39;: [[4, 3, 3.1, 2.4],
[16, 2.8, 2.7, 2.45],
[18, 2.98, 3.05, 2.39],
[60, 3, 3.05, 2.44],
[70, 2.95, 2.85, 2.6],
[81, 3.13, 3, 2.5],
[82, 2.87, 2.9, 2.35],
[88, 3, 3, 2.4],
[90, 2.8, 3.1, 2.43],
[104, 2.95, 3.05, 2.4],
[110, 3, 3.05, 2.5],
[115, 3, 3, 2.45],
[158, 2.87, 3.1, 2.45],
[173, 2.9, 2.9, 2.36],
[177, 3.06, 2.98, 2.66],
[255, 2.9, 3.2, 2.37],
[281, 3, 3.1, 2.4],
[370, 2.65, 2.75, 2.2],
[422, 2.87, 3.1, 2.45],
[450, 2.95, 3.05, 2.4],
[474, 2.86, 2.95, 2.56],
[499, 2.86, 3.2, 2.38],
[517, 2.98, 3.05, 2.39],
[545, 2.86, 3.2, 2.38],
[659, 2.84, 3.08, 2.35],
[976, 3, 3.15, 2.35]],
&#39;L_2247563&#39;: [[3, 0.79, -0.25, 1.09],
[8, 0.73, -0.25, 1.15],
[12, 0.75, -0.25, 1.18],
[14, 0.78, -0.25, 1.08],
[17, 0.8, -0.25, 1.11],
[19, 1.05, 0, 0.7],
[22, 0.64, -0.25, 1.11],
[23, 0.8, -0.25, 1.13],
[24, 0.8, -0.25, 1.11],
[31, 1.07, 0, 0.85],
[35, 0.8, -0.25, 1.11],
[42, 0.75, -0.25, 1.05]],
&#39;T_2247563&#39;: [[3, 0.79, 2, 1.07],
[4, 1.3, 2.5, 0.57],
[8, 0.73, 2, 1.15],
[9, 1.3, 2.5, 0.57],
[12, 0.75, 2, 1.15],
[14, 0.75, 2, 1.05],
[17, 1.08, 2.25, 0.8],
[19, 1.3, 2.5, 0.55],
[22, 0.63, 2, 1.06],
[23, 0.8, 2, 1.11],
[24, 1.08, 2.25, 0.8],
[31, 1.11, 2.25, 0.8],
[35, 1.09, 2.25, 0.81],
[42, 1.04, 2.25, 0.76],
[49, 1.25, 2.5, 0.55]],
&#39;O_2247564&#39;: [[4, 2.4, 3, 3.1],
[16, 2.1, 2.9, 3.3],
[18, 2.17, 3.15, 3.25],
[60, 2.45, 3, 3.05],
[70, 2.05, 3.15, 3.65],
[81, 2.3, 3.1, 3.4],
[82, 2.2, 3, 3.1],
[88, 2.2, 3.1, 3.25],
[90, 2.03, 3.3, 3.5],
[104, 2.45, 3, 3],
[110, 2.15, 3.2, 3.6],
[115, 2.4, 3, 3.1],
[158, 2.15, 3.2, 3.4],
[173, 2.37, 2.85, 2.95],
[177, 2.18, 3.28, 3.64],
[255, 2.25, 3.3, 3.1],
[281, 2.1, 3.2, 3.4],
[370, 2.2, 2.7, 2.7],
[422, 2.2, 3.2, 3.25],
[450, 2.41, 3, 3],
[474, 2.17, 3.15, 3.3],
[499, 2.23, 3.3, 3.05],
[517, 2.25, 3.15, 3.15],
[545, 2.23, 3.3, 3.05],
[659, 2.28, 3.15, 2.9],
[976, 2.3, 3.2, 3.05]],
&#39;L_2247564&#39;: [[3, 0.97, 0.25, 0.91],
[8, 0.83, 0.25, 1.03],
[12, 0.75, 0.25, 1.18],
[14, 0.88, 0.25, 0.95],
[17, 1.05, 0.25, 0.85],
[19, 1.05, 0.5, 0.7],
[22, 0.79, 0.25, 0.9],
[23, 0.98, 0.25, 0.94],
[24, 1.05, 0.25, 0.85],
[31, 0.87, 0.25, 1.05],
[35, 1.05, 0.25, 0.85],
[42, 0.99, 0.25, 0.8]],
&#39;T_2247564&#39;: [[3, 0.98, 2.25, 0.88],
[4, 1.2, 2.5, 0.6],
[8, 1.03, 2.25, 0.83],
[9, 1.2, 2.5, 0.62],
[12, 1.05, 2.25, 0.83],
[14, 0.95, 2.25, 0.88],
[17, 0.98, 2.25, 0.9],
[19, 1.2, 2.5, 0.6],
[22, 0.91, 2.25, 0.74],
[23, 0.99, 2.25, 0.91],
[24, 0.98, 2.25, 0.9],
[31, 1.05, 2.25, 0.85],
[35, 0.99, 2.25, 0.91],
[42, 0.93, 2.25, 0.85],
[49, 1.15, 2.5, 0.62]],
&#39;O_2247565&#39;: [[4, 2.3, 3.1, 3.2],
[16, 2.2, 2.9, 3],
[18, 2.16, 3.2, 3.3],
[60, 2.35, 3.05, 3.15],
[70, 2.05, 3.15, 3.65],
[81, 2.3, 3.1, 3.4],
[82, 2.15, 3, 3.1],
[88, 2.2, 3.1, 3.25],
[90, 2.27, 3.2, 3.1],
[104, 2.3, 3.1, 3.15],
[110, 2.3, 3.15, 3.25],
[115, 2.3, 3.1, 3.2],
[158, 2.25, 3.3, 3.1],
[173, 2.27, 2.9, 3.05],
[177, 2.36, 3.07, 3.46],
[255, 2.25, 3.3, 3.1],
[281, 2.25, 3.2, 3.2],
[370, 2.1, 2.75, 2.8],
[422, 2.25, 3.3, 3.1],
[450, 2.31, 3.05, 3.15],
[474, 2.35, 3.15, 2.98],
[499, 2.21, 3.3, 3.1],
[517, 2.25, 3.2, 3.1],
[545, 2.21, 3.3, 3.1],
[659, 2.22, 3.15, 2.98],
[976, 2.25, 3.2, 3.15]],
&#39;L_2247565&#39;: [[3, 0.95, 0.25, 0.93],
[8, 0.98, 0.25, 0.88],
[12, 1.02, 0.25, 0.88],
[14, 0.91, 0.25, 0.91],
[17, 1, 0.25, 0.9],
[19, 0.6, 0, 1.2],
[22, 0.91, 0.25, 0.78],
[23, 0.96, 0.25, 0.96],
[24, 1, 0.25, 0.9],
[31, 1.02, 0.25, 0.9],
[35, 1, 0.25, 0.9],
[42, 0.94, 0.25, 0.84]],
&#39;T_2247565&#39;: [[3, 0.98, 2.25, 0.88],
[4, 1.2, 2.5, 0.6],
[8, 0.95, 2.25, 0.9],
[9, 1.2, 2.5, 0.62],
[12, 1.05, 2.25, 0.83],
[14, 0.95, 2.25, 0.88],
[17, 0.98, 2.25, 0.9],
[19, 1.2, 2.5, 0.6],
[22, 0.83, 2.25, 0.82],
[23, 0.99, 2.25, 0.91],
[24, 0.98, 2.25, 0.9],
[31, 1, 2.25, 0.9],
[35, 0.99, 2.25, 0.91],
[42, 0.93, 2.25, 0.85],
[49, 1.15, 2.5, 0.62]],
&#39;O_2247566&#39;: [[4, 1.9, 3.25, 4.2],
[16, 1.8, 3, 4.3],
[18, 1.95, 3.25, 3.85],
[60, 1.95, 3.2, 4.1],
[70, 1.87, 3.1, 4.5],
[81, 1.95, 3.2, 4.2],
[82, 1.87, 3.1, 3.9],
[88, 1.91, 3.2, 4],
[90, 1.92, 3.4, 3.7],
[104, 1.95, 3.2, 4],
[110, 1.9, 3.3, 4.25],
[115, 1.91, 3.2, 4.2],
[158, 1.93, 3.3, 3.9],
[173, 1.89, 3.05, 3.95],
[177, 1.88, 3.38, 4.96],
[255, 1.93, 3.3, 3.9],
[281, 1.91, 3.4, 4],
[370, 1.75, 2.85, 3.5],
[422, 1.93, 3.3, 3.9],
[450, 1.92, 3.2, 4],
[474, 1.96, 3.25, 3.8],
[499, 1.94, 3.35, 3.75],
[517, 1.95, 3.25, 3.85],
[545, 1.94, 3.35, 3.75],
[659, 1.89, 3.26, 3.75],
[976, 1.9, 3.35, 4]],
&#39;L_2247566&#39;: [[3, 0.94, 0.5, 0.94],
[8, 0.9, 0.5, 0.95],
[12, 0.94, 0.5, 0.96],
[14, 0.88, 0.5, 0.89],
[17, 0.95, 0.5, 0.95],
[19, 0.9, 0.5, 0.8],
[22, 0.78, 0.5, 0.91],
[23, 0.95, 0.5, 0.97],
[24, 0.95, 0.5, 0.95],
[31, 0.97, 0.5, 0.95],
[35, 0.95, 0.5, 0.95],
[42, 0.89, 0.5, 0.89]],
&#39;T_2247566&#39;: [[3, 0.98, 2.25, 0.88],
[4, 1.2, 2.5, 0.6],
[8, 0.98, 2.25, 0.88],
[9, 1.2, 2.5, 0.62],
[12, 1, 2.25, 0.88],
[14, 0.93, 2.25, 0.85],
[17, 0.98, 2.25, 0.9],
[19, 1.1, 2.5, 0.65],
[22, 1.09, 2.5, 0.62],
[23, 0.99, 2.25, 0.91],
[24, 0.98, 2.25, 0.9],
[31, 1, 2.25, 0.9],
[35, 0.99, 2.25, 0.91],
[42, 0.93, 2.25, 0.85],
[49, 1.15, 2.5, 0.62]],
&#39;O_2247567&#39;: [[4, 2.75, 2.95, 2.75],
[16, 2.45, 3, 2.55],
[18, 2.6, 3.2, 2.6],
[60, 2.75, 2.9, 2.75],
[70, 2.65, 3.05, 2.7],
[81, 2.8, 3, 2.75],
[82, 2.62, 2.9, 2.55],
[88, 2.65, 3, 2.6],
[90, 2.41, 3.4, 2.7],
[104, 2.75, 2.9, 2.75],
[110, 2.55, 3.15, 2.85],
[115, 2.75, 2.9, 2.75],
[158, 2.6, 3.2, 2.6],
[173, 2.65, 2.8, 2.7],
[177, 2.67, 3.3, 2.78],
[255, 2.7, 2.9, 2.75],
[281, 2.75, 3.1, 2.7],
[370, 2.4, 2.65, 2.45],
[422, 2.6, 3.2, 2.6],
[450, 2.65, 3, 2.7],
[474, 2.62, 3.1, 2.67],
[499, 2.65, 3.25, 2.55],
[517, 2.6, 3.2, 2.6],
[545, 2.65, 3.25, 2.55],
[659, 2.58, 3.07, 2.58],
[976, 2.65, 3.1, 2.65]],
&#39;L_2247567&#39;: [[3, 0.98, 0, 0.9],
[8, 0.9, 0, 0.95],
[12, 0.84, 0, 1.06],
[14, 0.91, 0, 0.85],
[17, 0.95, 0, 0.95],
[19, 0.8, 0, 0.9],
[22, 0.81, 0, 0.88],
[23, 0.99, 0, 0.93],
[24, 0.95, 0, 0.95],
[31, 0.94, 0, 0.98],
[35, 0.95, 0, 0.95],
[42, 0.89, 0, 0.89]],
&#39;T_2247567&#39;: [[3, 1.05, 2.25, 0.81],
[4, 1.3, 2.5, 0.57],
[8, 0.9, 2.25, 0.95],
[9, 1.3, 2.5, 0.57],
[12, 0.87, 2.25, 1.01],
[14, 1, 2.25, 0.78],
[17, 1.08, 2.25, 0.8],
[19, 1.3, 2.5, 0.55],
[22, 0.77, 2.25, 0.87],
[23, 1.06, 2.25, 0.84],
[24, 1.08, 2.25, 0.8],
[31, 0.95, 2.25, 0.95],
[35, 1.09, 2.25, 0.81],
[42, 1.04, 2.25, 0.76],
[49, 1.25, 2.5, 0.57]],
&#39;O_2247568&#39;: [[4, 3.1, 3.25, 2.3],
[16, 2.85, 2.9, 2.3],
[18, 3.25, 3.15, 2.2],
[60, 3.1, 3.15, 2.32],
[70, 3, 3.15, 2.35],
[81, 3.3, 3, 2.38],
[82, 3.1, 2.9, 2.2],
[88, 3.2, 3, 2.25],
[90, 2.9, 3.3, 2.32],
[104, 3.1, 3.25, 2.25],
[110, 3, 3.2, 2.4],
[115, 3.4, 3, 2.25],
[158, 2.95, 3.2, 2.35],
[173, 3, 3, 2.25],
[177, 3.08, 3.2, 2.49],
[255, 3.2, 3.2, 2.25],
[281, 2.9, 3.2, 2.38],
[370, 2.75, 2.85, 2.05],
[422, 3.2, 3.2, 2.25],
[450, 3.05, 3.15, 2.28],
[474, 2.82, 3.1, 2.48],
[499, 3.1, 3.25, 2.23],
[517, 3.25, 3.15, 2.2],
[545, 3.1, 3.25, 2.23],
[659, 3.12, 3.1, 2.18],
[976, 3.3, 3.15, 2.2]],
&#39;L_2247568&#39;: [[3, 0.92, -0.25, 0.96],
[8, 0.75, -0.25, 1.13],
[12, 0.81, -0.25, 1.1],
[14, 0.85, -0.25, 0.91],
[17, 0.95, -0.25, 0.95],
[19, 1.05, 0, 0.7],
[22, 0.71, -0.25, 1],
[23, 0.93, -0.25, 0.99],
[24, 0.95, -0.25, 0.95],
[31, 0.8, -0.25, 1.13],
[35, 0.95, -0.25, 0.95],
[42, 0.89, -0.25, 0.89]],
&#39;T_2247568&#39;: [[3, 1.05, 2.25, 0.81],
[4, 1.3, 2.5, 0.57],
[8, 0.98, 2.25, 0.88],
[9, 1.3, 2.5, 0.57],
[12, 1.01, 2.25, 0.87],
[14, 1, 2.25, 0.78],
[17, 1.08, 2.25, 0.8],
[19, 1.2, 2.5, 0.6],
[22, 0.85, 2.25, 0.8],
[23, 1.06, 2.25, 0.84],
[24, 1.08, 2.25, 0.8],
[31, 1, 2.25, 0.9],
[35, 1.09, 2.25, 0.81],
[42, 1.04, 2.25, 0.76],
[49, 1.25, 2.5, 0.57]],
&#39;O_2247569&#39;: [[4, 2.8, 2.95, 2.7],
[16, 2.4, 2.8, 2.85],
[18, 2.53, 3, 2.84],
[60, 2.8, 2.9, 2.75],
[70, 2.5, 2.95, 3],
[81, 2.63, 3, 2.9],
[82, 2.5, 2.87, 2.7],
[88, 2.45, 2.9, 2.9],
[90, 2.36, 3.1, 3],
[104, 2.7, 3.05, 2.65],
[110, 2.5, 3, 3.05],
[115, 2.8, 2.9, 2.7],
[158, 2.45, 3.1, 2.95],
[173, 2.7, 2.75, 2.65],
[177, 2.58, 3.12, 3.13],
[255, 2.6, 3.2, 2.65],
[281, 2.5, 3, 2.9],
[370, 2.5, 2.6, 2.4],
[422, 2.5, 3.1, 2.8],
[450, 2.75, 2.9, 2.7],
[474, 2.49, 2.94, 2.95],
[499, 2.6, 3.15, 2.64],
[517, 2.53, 3, 2.84],
[545, 2.6, 3.15, 2.64],
[659, 2.66, 3, 2.54],
[976, 2.75, 3.05, 2.6]],
&#39;L_2247569&#39;: [[3, 0.93, 0, 0.95],
[8, 0.73, 0, 1.15],
[12, 0.74, 0, 1.19],
[14, 0.78, 0, 1.08],
[17, 1, 0, 0.9],
[19, 0.7, 0, 1.05],
[22, 0.69, 0, 1.03],
[23, 0.94, 0, 0.98],
[24, 1, 0, 0.9],
[31, 0.8, 0, 1.13],
[35, 1, 0, 0.9],
[42, 0.94, 0, 0.84]],
&#39;T_2247569&#39;: [[3, 0.87, 2, 0.99],
[4, 1.37, 2.5, 0.5],
[8, 0.9, 2, 0.95],
[9, 1.5, 2.5, 0.5],
[12, 0.98, 2, 0.9],
[14, 0.85, 2, 1],
[17, 0.88, 2, 1],
[19, 1.4, 2.5, 0.5],
[22, 1.06, 2.25, 0.63],
[23, 0.88, 2, 1.02],
[24, 0.88, 2, 1],
[31, 0.9, 2, 1],
[35, 0.89, 2, 1.01],
[42, 0.83, 2, 0.95],
[49, 1.37, 2.5, 0.51]],
&#39;O_2247570&#39;: [[4, 3.4, 3.1, 2.2],
[16, 3, 2.75, 2.35],
[18, 3.3, 3.15, 2.18],
[60, 3.4, 3.1, 2.2],
[70, 3, 3, 2.45],
[81, 3.4, 3.1, 2.3],
[82, 3.1, 2.9, 2.2],
[88, 3.2, 3, 2.25],
[90, 2.8, 3.3, 2.34],
[104, 3.35, 3.15, 2.2],
[110, 3, 3.15, 2.45],
[115, 3.4, 3.1, 2.2],
[158, 2.9, 3.2, 2.37],
[173, 3.3, 2.95, 2.13],
[177, 3.23, 3, 2.53],
[255, 3.2, 3.25, 2.2],
[281, 2.9, 3.2, 2.38],
[370, 3, 2.8, 1.95],
[422, 3.2, 3.25, 2.2],
[450, 3.38, 3.15, 2.16],
[474, 2.82, 3.1, 2.48],
[499, 3.15, 3.25, 2.19],
[517, 3.3, 3.15, 2.18],
[545, 3.15, 3.25, 2.19],
[659, 3.2, 3.14, 2.12],
[976, 3.35, 3.2, 2.15]],
&#39;L_2247570&#39;: [[3, 0.97, -0.25, 0.91],
[8, 0.75, -0.25, 1.13],
[12, 0.76, -0.25, 1.16],
[14, 0.9, -0.25, 0.95],
[17, 1, -0.25, 0.9],
[19, 1.05, 0, 0.7],
[22, 0.72, -0.25, 0.98],
[23, 0.98, -0.25, 0.94],
[24, 1, -0.25, 0.9],
[31, 0.8, -0.25, 1.13],
[35, 1, -0.25, 0.9],
[42, 0.94, -0.25, 0.84]],
&#39;T_2247570&#39;: [[3, 1, 2.25, 0.86],
[4, 1.2, 2.5, 0.6],
[8, 0.95, 2.25, 0.9],
[9, 1.25, 2.5, 0.6],
[12, 0.95, 2.25, 0.93],
[14, 0.98, 2.25, 0.85],
[17, 1.03, 2.25, 0.85],
[19, 1.2, 2.5, 0.6],
[22, 0.81, 2.25, 0.83],
[23, 1.01, 2.25, 0.89],
[24, 1.03, 2.25, 0.85],
[31, 1, 2.25, 0.9],
[35, 1.04, 2.25, 0.86],
[42, 0.98, 2.25, 0.81],
[49, 1.2, 2.5, 0.6]],
&#39;O_2247571&#39;: [[4, 2.7, 3.1, 2.7],
[16, 2.5, 2.75, 2.7],
[18, 2.61, 3.15, 2.63],
[60, 2.65, 3.15, 2.65],
[70, 2.85, 2.88, 2.7],
[81, 2.75, 3.1, 2.75],
[82, 2.55, 3, 2.55],
[88, 2.62, 3, 2.65],
[90, 2.6, 3.2, 2.6],
[104, 2.6, 3.3, 2.6],
[110, 2.7, 3.1, 2.7],
[115, 2.63, 3.2, 2.63],
[158, 2.6, 3.1, 2.7],
[173, 2.55, 3, 2.6],
[177, 2.72, 2.99, 2.98],
[255, 2.7, 2.95, 2.7],
[281, 2.55, 3.2, 2.75],
[370, 2.35, 2.85, 2.35],
[422, 2.7, 2.95, 2.7],
[450, 2.6, 3.15, 2.6],
[474, 2.62, 3.1, 2.67],
[499, 2.56, 3.35, 2.56],
[517, 2.61, 3.15, 2.63],
[545, 2.56, 3.35, 2.56],
[659, 2.55, 3.16, 2.55],
[976, 2.6, 3.2, 2.6]],
&#39;L_2247571&#39;: [[3, 0.94, 0, 0.94],
[8, 0.85, 0, 1],
[12, 0.93, 0, 0.97],
[14, 0.91, 0, 0.93],
[17, 0.95, 0, 0.95],
[19, 0.8, 0, 0.9],
[22, 1.15, 0.25, 0.61],
[23, 0.95, 0, 0.97],
[24, 0.95, 0, 0.95],
[31, 0.94, 0, 0.98],
[35, 0.95, 0, 0.95],
[42, 0.89, 0, 0.89]],
&#39;T_2247571&#39;: [[3, 0.92, 2.25, 0.94],
[4, 1.15, 2.5, 0.61],
[8, 0.93, 2.25, 0.93],
[9, 1.1, 2.5, 0.67],
[12, 0.97, 2.25, 0.91],
[14, 0.91, 2.25, 0.93],
[17, 0.93, 2.25, 0.95],
[19, 1.1, 2.5, 0.65],
[22, 1.05, 2.5, 0.64],
[23, 0.93, 2.25, 0.97],
[24, 0.93, 2.25, 0.95],
[31, 0.95, 2.25, 0.95],
[35, 0.94, 2.25, 0.96],
[42, 0.88, 2.25, 0.9],
[49, 1.15, 2.5, 0.62]]}}
</details>
# 答案2
**得分**: 1
响应不是`json`格式，这就是为什么`response.json()`失败
我们可以使用正则表达式来提取数据，然后使用`eval`将字符串转换为列表
```python3
import re
import requests
params = {
'sclassId': '40',
'subSclassId': '261',
'matchSeason': '2022-2023',
'round': '23',
'flesh': '0.6807624444063407',
}
response = requests.get('https://football.goaloo2.com/ajax/LeagueOddsAjax', params=params)
data = {}
for id_, lst in re.findall('oddsData\["(.+?)"\]=(.+?);', response.text):
data[id_] = eval(lst)
英文:

The response isn't in json format that's why the response.json() fails

we can use regular expressions to extract the data
and eval to convert string to list

import re
import requests

params = {
    &#39;sclassId&#39;: &#39;40&#39;,
    &#39;subSclassId&#39;: &#39;261&#39;,
    &#39;matchSeason&#39;: &#39;2022-2023&#39;,
    &#39;round&#39;: &#39;23&#39;,
    &#39;flesh&#39;: &#39;0.6807624444063407&#39;,
}

response = requests.get(&#39;https://football.goaloo2.com/ajax/LeagueOddsAjax&#39;, params=params)

data = {}
for id_, lst in re.findall(&#39;oddsData\[&quot;(.+?)&quot;\]=(.+?);&#39;, response.text):
    data[id_] = eval(lst)

huangapple
  • 本文由 发表于 2023年2月8日 17:23:46
  • 转载请务必保留本文链接：https://go.coder-hub.com/75383609.html
  • api
  • json
  • python
  • web-scraping
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定