Golang xml unmarshal html table(Golang解析XML中的HTML表格)

huangapple go评论94阅读模式
英文:

Golang xml unmarshal html table

问题

我有一个简单的HTML表格,并且希望获取所有单元格的值,即使其中包含HTML代码。

我尝试使用xml unmarshal,但是没有得到正确的结构标签、值或属性。

import (
    "fmt"
    "encoding/xml"
)

type XMLTable struct {
    XMLName xml.Name `xml:"TABLE"`
    Row []struct{
        Cell string `xml:"TD"`
    }`xml:"TR"`
}

func main() {
    raw_html_table := `
    <TABLE><TR>
    <TD>lalalal</TD>
    <TD>papapap</TD>
    <TD>fafafa</TD>
    <TD>
    <form action=\"/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf\" method=POST>
    <input type=hidden name=acT value=\"Dev\">
    <input type=hidden name=acA value=\"Anyval\">
    <input type=submit name=submit value=Stop>
    </form>
    </TD>
    </TR>
    </TABLE>`

    table := XMLTable{}
    fmt.Printf("%q\n", []byte(raw_html_table)[:15])
    err := xml.Unmarshal([]byte(raw_html_table), &table)
    if err != nil {
        fmt.Printf("error: %v", err)
    }
}

额外的信息是,如果单元格内容是HTML代码,我不关心它(只获取[]byte / string值)。因此,在解析之前,我可以删除单元格内容,但这种方式也不太容易。

欢迎使用标准的Go语言库提出任何建议。

英文:

I have a simple HTML table, and want to get all cell values even if it's HTML code inside.

Trying to use xml unmarshal, but didn't get the right struct tags, values or attributes.

import (
    &quot;fmt&quot;
    &quot;encoding/xml&quot;
)

type XMLTable struct {
XMLName xml.Name `xml:&quot;TABLE&quot;`
    Row []struct{
	    Cell string `xml:&quot;TD&quot;`
    }`xml:&quot;TR&quot;`
}

func main() {
    raw_html_table := `
    &lt;TABLE&gt;&lt;TR&gt;
    &lt;TD&gt;lalalal&lt;/TD&gt;
    &lt;TD&gt;papapap&lt;/TD&gt;
    &lt;TD&gt;fafafa&lt;/TD&gt;
    &lt;TD&gt;
    &lt;form action=\&quot;/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf&quot; method=POST&gt;
    &lt;input type=hidden name=acT value=\&quot;Dev\&quot;&gt;
    &lt;input type=hidden name=acA value=\&quot;Anyval\&quot;&gt;
    &lt;input type=submit name=submit value=Stop&gt;
    &lt;/form&gt;
    &lt;/TD&gt;
    &lt;/TR&gt;
    &lt;/TABLE&gt;`

    table := XMLTable{}
    fmt.Printf(&quot;%q\n&quot;, []byte(raw_html_table)[:15])
    err := xml.Unmarshal([]byte(raw_html_table), &amp;table)
    if err != nil {
	    fmt.Printf(&quot;error: %v&quot;, err)
    }
}

As an additional info, I don't care about cell content if it's HTML code (take only []byte / string values). So I may delete cell content before unmarshaling, but this way is also not so easy.

Any suggestions with standard golang libs would be welcome.

答案1

得分: 4

坚持使用标准库

您的输入不是有效的XML,因此即使您正确建模,也无法解析它。

首先,您使用原始的字符串文字将输入HTML定义为string,而原始字符串文字不能包含转义字符。例如:

&lt;form action=\&quot;/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf&quot; method=POST&gt;

您不能在原始字符串文字中使用\&quot;(您可以使用,但它将仅表示这两个字符),您不必这样做,可以使用简单的引号:&quot;

其次,在XML中,您不能没有将属性值放在引号中。

第三,每个元素必须有一个匹配的闭合元素,您的&lt;input&gt;元素没有关闭。

因此,例如,这一行:

&lt;input type=hidden name=acT value=\&quot;Dev\&quot;&gt;

必须更改为:

&lt;input type=&quot;hidden&quot; name=&quot;acT&quot; value=&quot;Dev&quot; /&gt;

好的,经过这些更改,输入现在是有效的XML。

如何建模呢?就像这样简单:

type XMLTable struct {
	Rows []struct {
		Cell string `xml:&quot;,innerxml&quot;`
	} `xml:&quot;TR&gt;TD&quot;`
}

解析和打印&lt;TD&gt;元素内容的完整代码如下:

raw_html_table := `
&lt;TABLE&gt;&lt;TR&gt;
&lt;TD&gt;lalalal&lt;/TD&gt;
&lt;TD&gt;papapap&lt;/TD&gt;
&lt;TD&gt;fafafa&lt;/TD&gt;
&lt;TD&gt;
&lt;form action=&quot;/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf&quot; method=&quot;POST&quot;&gt;
&lt;input type=&quot;hidden&quot; name=&quot;acT&quot; value=&quot;Dev&quot; /&gt;
&lt;input type=&quot;hidden&quot; name=&quot;acA&quot; value=&quot;Anyval&quot; /&gt;
&lt;input type=&quot;submit&quot; name=&quot;submit&quot; value=&quot;Stop&quot; /&gt;
&lt;/form&gt;
&lt;/TD&gt;
&lt;/TR&gt;
&lt;/TABLE&gt;`

table := XMLTable{}
err := xml.Unmarshal([]byte(raw_html_table), &amp;table)
if err != nil {
	fmt.Printf("error: %v\n", err)
}

fmt.Println("count:", len(table.Rows))
for _, row := range table.Rows {
	fmt.Println("TD content:", row.Cell)
}

输出结果(在Go Playground上尝试):

count: 4
TD content: lalalal
TD content: papapap
TD content: fafafa
TD content: 
    &lt;form action=&quot;/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf&quot; method=&quot;POST&quot;&gt;
    &lt;input type=&quot;hidden&quot; name=&quot;acT&quot; value=&quot;Dev&quot; /&gt;
    &lt;input type=&quot;hidden&quot; name=&quot;acA&quot; value=&quot;Anyval&quot; /&gt;
    &lt;input type=&quot;submit&quot; name=&quot;submit&quot; value=&quot;Stop&quot; /&gt;
    &lt;/form&gt;

使用正确的HTML解析器

如果您不能或不想更改输入的HTML,或者您想处理所有HTML输入而不仅仅是有效的XML,您应该使用正确的HTML解析器,而不是将输入视为XML。

请查看https://godoc.org/golang.org/x/net/html,了解符合HTML5标准的标记化器和解析器。

英文:

Sticking to the standard lib

Your input is not valid XML, so even if you model it right, you won't be able to parse it.

First, you're using a raw string literal to define your input HTML as a string, and raw string literals cannot contain escapes. For example this:

&lt;form action=\&quot;/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf&quot; method=POST&gt;

You can't use \&quot; in a raw string literal (you can, but it will mean exactly those 2 characters), and you don't have to, use a simple quotation mark: &quot;.

Next, in XML you cannot have attributes without putting their values in quotes.

Third, each element must have a matching closing element, your &lt;input&gt; elements are not closed.

So for example this line:

&lt;input type=hidden name=acT value=\&quot;Dev\&quot;&gt;

Must be changed to:

&lt;input type=&quot;hidden&quot; name=&quot;acT&quot; value=&quot;Dev&quot; /&gt;

Ok, after these the input is a valid XML now.

How to model it? Simple as this:

type XMLTable struct {
	Rows []struct {
		Cell string `xml:&quot;,innerxml&quot;`
	} `xml:&quot;TR&gt;TD&quot;`
}

And the full code to parse and print contents of &lt;TD&gt; elements:

raw_html_table := `
&lt;TABLE&gt;&lt;TR&gt;
&lt;TD&gt;lalalal&lt;/TD&gt;
&lt;TD&gt;papapap&lt;/TD&gt;
&lt;TD&gt;fafafa&lt;/TD&gt;
&lt;TD&gt;
&lt;form action=&quot;/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf&quot; method=&quot;POST&quot;&gt;
&lt;input type=&quot;hidden&quot; name=&quot;acT&quot; value=&quot;Dev&quot; /&gt;
&lt;input type=&quot;hidden&quot; name=&quot;acA&quot; value=&quot;Anyval&quot; /&gt;
&lt;input type=&quot;submit&quot; name=&quot;submit&quot; value=&quot;Stop&quot; /&gt;
&lt;/form&gt;
&lt;/TD&gt;
&lt;/TR&gt;
&lt;/TABLE&gt;`

table := XMLTable{}
err := xml.Unmarshal([]byte(raw_html_table), &amp;table)
if err != nil {
	fmt.Printf(&quot;error: %v\n&quot;, err)
}

fmt.Println(&quot;count:&quot;, len(table.Rows))
for _, row := range table.Rows {
	fmt.Println(&quot;TD content:&quot;, row.Cell)
}

Output (try it on the Go Playground):

count: 4
TD content: lalalal
TD content: papapap
TD content: fafafa
TD content: 
    &lt;form action=&quot;/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf&quot; method=&quot;POST&quot;&gt;
    &lt;input type=&quot;hidden&quot; name=&quot;acT&quot; value=&quot;Dev&quot; /&gt;
    &lt;input type=&quot;hidden&quot; name=&quot;acA&quot; value=&quot;Anyval&quot; /&gt;
    &lt;input type=&quot;submit&quot; name=&quot;submit&quot; value=&quot;Stop&quot; /&gt;
    &lt;/form&gt;

Using a proper HTML parser

If you can't or don't want to change the input HTML, or you want to handle all HTML input not just valid XMLs, you should use a proper HTML parser instead of treating the input as XML.

Check out https://godoc.org/golang.org/x/net/html for an HTML5-compliant tokenizer and parser.

答案2

得分: 1

一旦您的输入是有效的HTML(您的代码片段在属性中缺少引号),您可以使用实体和自动关闭映射来配置xml.Decoder(并使其非严格),这将起作用:

在此处运行我的修改版本

package main

import (
	"encoding/xml"
	"fmt"
	"strings"
)

type XMLTable struct {
	Rows []struct {
		Cell string `xml:",innerxml"`
	} `xml:"TR>TD"`
}

func main() {
	raw_html_table := `
	<TABLE><TR>
	<TD>lalalal</TD>
	<TD>papapap</TD>
	<TD>fafafa</TD>
	<TD>
	<form action="/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf" method="POST">
	<input type="hidden" name="acT" value="Dev">
	<input type="hidden" name="acA" value="Anyval">
	<input type="submit" name="submit" value="Stop">
	</form>
	</TD>
	</TR>
	</TABLE>`

	table := XMLTable{}
	decoder := xml.NewDecoder(strings.NewReader(raw_html_table))
	decoder.Entity = xml.HTMLEntity
	decoder.AutoClose = xml.HTMLAutoClose
	decoder.Strict = false

	err := decoder.Decode(&table)
	if err != nil {
		fmt.Printf("error: %v", err)
	}
	fmt.Printf("%#v\n", table)
}

以上是您要翻译的内容。

英文:

Once your input is valid HTML (your snippet is missing quotes in attributes), you can configure a xml.Decoder with entities and autoclose maps (and make it non-strict), which will end up working :

Run my modified version here.

package main

import (
	&quot;encoding/xml&quot;
	&quot;fmt&quot;
	&quot;strings&quot;
)

type XMLTable struct {
    Rows []struct {
        Cell string `xml:&quot;,innerxml&quot;`
    } `xml:&quot;TR&gt;TD&quot;`
}

func main() {
	raw_html_table := `
    &lt;TABLE&gt;&lt;TR&gt;
    &lt;TD&gt;lalalal&lt;/TD&gt;
    &lt;TD&gt;papapap&lt;/TD&gt;
    &lt;TD&gt;fafafa&lt;/TD&gt;
    &lt;TD&gt;
    &lt;form action=&quot;/addedUrl/;jsessionid=KJHSDFKJLSDF293847odhf&quot; method=&quot;POST&quot;&gt;
    &lt;input type=&quot;hidden&quot; name=&quot;acT&quot; value=&quot;Dev&quot;&gt;
    &lt;input type=&quot;hidden&quot; name=&quot;acA&quot; value=&quot;Anyval&quot;&gt;
    &lt;input type=&quot;submit&quot; name=&quot;submit&quot; value=&quot;Stop&quot;&gt;
    &lt;/form&gt;
    &lt;/TD&gt;
    &lt;/TR&gt;
    &lt;/TABLE&gt;`

	table := XMLTable{}
	decoder := xml.NewDecoder(strings.NewReader(raw_html_table))
	decoder.Entity = xml.HTMLEntity
	decoder.AutoClose = xml.HTMLAutoClose	
    decoder.Strict = false

	err := decoder.Decode(&amp;table)
	if err != nil {
		fmt.Printf(&quot;error: %v&quot;, err)
	}
	fmt.Printf(&quot;%#v\n&quot;, table)
}

huangapple
  • 本文由 发表于 2017年5月17日 14:36:14
  • 转载请务必保留本文链接:https://go.coder-hub.com/44017285.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定