MS SQL 解析 XML 文件到表格

huangapple go评论64阅读模式
英文:

MS SQL Parsing xml files to tables

问题

您能帮助我将XML文件解析成表格吗?我在我的SQL数据库中有成千上万个像这样的XML文件:

这些XML文件具有以下结构:

<eolresult>
  <eolpresetting>
    <datetime>08.12.2020 5:24:48</datetime>
    <order id="2049457704XSK316_FRX101xxxxxxxxxxxxxx000SK35X3PH9P40006C24A4NORL0L7P7XYA0000006T24D0XXK0006A00000000" />
  </eolpresetting>
  <testresult>NIO</testresult>
  <results>
    <!--PRN="2049457704XSK316_FRX101xxxxxxxxxxxxxx000SK35X3PH9P40006C24A4NORL0L7P7XYA0000006T24D0XXK0006A00000000"-->
    <test name="Time">
      <value name="Time of start ">08.12.2020 5:23:03</value>
      <value name="Time of save ">08.12.2020 5:24:48</value>
      <value name="Time ">105 s</value>
    </test>
    <test name="Manual test">
      <value name="Manual test was used: ">False</value>
    </test>
    <!-- 其他测试的XML数据 -->
  &lt;/results&gt;
&lt;/eolresult&gt;

我需要以某种方式解析所有这些XML文件并转换成表格,例如按照“测试名称”,因为我想通过Power BI从这些数据创建报告。或者如果有其他处理这些文件的方法,我将感激任何想法。

我想象的表格可能如下所示。每个表格包含ID、eolpresetting中的日期以及测试中的值。
示例:

MS SQL 解析 XML 文件到表格

英文:

can you please help me with parsing xml files into tables?
I have in my SQL database thousands of xml files like this:
MS SQL 解析 XML 文件到表格

These xml files got this structure:

`&lt;eolresult&gt;
&lt;eolpresetting&gt;
&lt;datetime&gt;08.12.2020 5:24:48&lt;/datetime&gt;
&lt;order id=&quot;2049457704XSK316_FRX101xxxxxxxxxxxxxx000SK35X3PH9P40006C24A4NORL0L7P7XYA0000006T24D0XXK0006A00000000&quot; /&gt;
&lt;/eolpresetting&gt;
&lt;testresult&gt;NIO&lt;/testresult&gt;
&lt;results&gt;
&lt;!--PRN=&quot;2049457704XSK316_FRX101xxxxxxxxxxxxxx000SK35X3PH9P40006C24A4NORL0L7P7XYA0000006T24D0XXK0006A00000000&quot;--&gt;
&lt;test name=&quot;Time&quot;&gt;
&lt;value name=&quot;Time of start &quot;&gt;08.12.2020 5:23:03&lt;/value&gt;
&lt;value name=&quot;Time of save &quot;&gt;08.12.2020 5:24:48&lt;/value&gt;
&lt;value name=&quot;Time &quot;&gt;105 s&lt;/value&gt;
&lt;/test&gt;
&lt;test name=&quot;Manual test&quot;&gt;
&lt;value name=&quot;Manual test was used: &quot;&gt;False&lt;/value&gt;
&lt;/test&gt;
&lt;test name=&quot;Airbag SAB&quot;&gt;
&lt;value name=&quot;Measured value - Generator&quot;&gt;2,111209&lt;/value&gt;
&lt;value name=&quot;Low limit - Generator&quot;&gt;1,74&lt;/value&gt;
&lt;value name=&quot;High limit - Generator&quot;&gt;2,26&lt;/value&gt;
&lt;value name=&quot;Measured value - Break contact 1&quot;&gt;5,069321&lt;/value&gt;
&lt;value name=&quot;Minumum value - Break contact 1&quot;&gt;1,73&lt;/value&gt;
&lt;value name=&quot;Measured value - Break contact 2&quot;&gt;5,092026&lt;/value&gt;
&lt;value name=&quot;Minumum value - Break contact 2&quot;&gt;1,73&lt;/value&gt;
&lt;value name=&quot;Test OK&quot;&gt;True&lt;/value&gt;
&lt;/test&gt;
&lt;test name=&quot;Heating&quot;&gt;
&lt;value name=&quot;Measured value&quot;&gt;1,294227&lt;/value&gt;
&lt;value name=&quot;Low limit&quot;&gt;1,22&lt;/value&gt;
&lt;value name=&quot;High limit&quot;&gt;1,49&lt;/value&gt;
&lt;value name=&quot;Test OK&quot;&gt;True&lt;/value&gt;
&lt;/test&gt;
&lt;test name=&quot;Thermistor&quot;&gt;
&lt;value name=&quot;Measured value&quot;&gt;8671,408&lt;/value&gt;
&lt;value name=&quot;Low limit&quot;&gt;4000&lt;/value&gt;
&lt;value name=&quot;High limit&quot;&gt;14000&lt;/value&gt;
&lt;value name=&quot;Test OK&quot;&gt;True&lt;/value&gt;
&lt;/test&gt;
&lt;test name=&quot;Buckle&quot;&gt;
&lt;value name=&quot;Measured value 1 - unbuckled&quot;&gt;9,730747E-02&lt;/value&gt;
&lt;value name=&quot;Measured value 2 - buckled&quot;&gt;25022,52&lt;/value&gt;
&lt;value name=&quot;Measured value 3 - unbuckled&quot;&gt;8,596408E-02&lt;/value&gt;
&lt;value name=&quot;Bucked limit&quot;&gt;3,92&lt;/value&gt;
&lt;value name=&quot;Unbuckled limit&quot;&gt;999&lt;/value&gt;
&lt;value name=&quot;Test OK&quot;&gt;True&lt;/value&gt;
&lt;/test&gt;
&lt;test name=&quot;SBR&quot;&gt;
&lt;value name=&quot;Measured value - Low force&quot;&gt;99999,9&lt;/value&gt;
&lt;value name=&quot;Low limit - Low force&quot;&gt;332&lt;/value&gt;
&lt;value name=&quot;High limit - Low force&quot;&gt;468&lt;/value&gt;
&lt;value name=&quot;Measured value - Low force - Force [kg]&quot;&gt;5,242241&lt;/value&gt;
&lt;value name=&quot;Low limit - Low force - Force [kg]&quot;&gt;5,1&lt;/value&gt;
&lt;value name=&quot;High limit - Low force - Force [kg]&quot;&gt;6,4&lt;/value&gt;
&lt;value name=&quot;Regulator Active - Low force&quot;&gt;False&lt;/value&gt;
&lt;value name=&quot;Measured value - High force&quot;&gt;99999,9&lt;/value&gt;
&lt;value name=&quot;Low limit - High force&quot;&gt;83&lt;/value&gt;
&lt;value name=&quot;High limit - High force&quot;&gt;117&lt;/value&gt;
&lt;value name=&quot;Measured value - High force - Force [kg]&quot;&gt;33,80444&lt;/value&gt;
&lt;value name=&quot;Low limit - High force - Force [kg]&quot;&gt;33,6&lt;/value&gt;
&lt;value name=&quot;High limit - High force - Force [kg]&quot;&gt;34,3&lt;/value&gt;
&lt;value name=&quot;Regulator Active - High force&quot;&gt;True&lt;/value&gt;
&lt;value name=&quot;Strain Gauge - Tare [kg]&quot;&gt;-10,77971&lt;/value&gt;
&lt;value name=&quot;Test OK&quot;&gt;False&lt;/value&gt;
&lt;/test&gt;
&lt;test name=&quot;Light&quot;&gt;
&lt;value name=&quot;Measured value&quot;&gt;21,33521&lt;/value&gt;
&lt;value name=&quot;Low limit&quot;&gt;10&lt;/value&gt;
&lt;value name=&quot;High limit&quot;&gt;40&lt;/value&gt;
&lt;value name=&quot;Test OK&quot;&gt;True&lt;/value&gt;
&lt;/test&gt;
&lt;test name=&quot;Headrests&quot;&gt;
&lt;value name=&quot;Force - Measured&quot;&gt;16,04999&lt;/value&gt;
&lt;value name=&quot;Force - Low Limit&quot;&gt;15,81&lt;/value&gt;
&lt;value name=&quot;Force - High Limit&quot;&gt;17,64&lt;/value&gt;
&lt;value name=&quot;Regulator Active&quot;&gt;True&lt;/value&gt;
&lt;value name=&quot;Strain Gauge - Tare [kg]&quot;&gt;-8,725739E-02&lt;/value&gt;
&lt;value name=&quot;Test OK&quot;&gt;True&lt;/value&gt;
&lt;/test&gt;
&lt;test name=&quot;Delivery&quot;&gt;
&lt;value name=&quot;Measured value - Backrest&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;Low limit - Backrest&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;High limit - Backrest&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;Measured value - Forwards&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;Low limit - Forwards&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;High limit - Forwards&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;Measured value - Height&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;Low limit - Height&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;High limit - Height&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;Measured value - Thigh&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;Low limit - Thigh&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;High limit - Thigh&quot;&gt;0&lt;/value&gt;
&lt;value name=&quot;Measured value - Headrest&quot;&gt;384,2014&lt;/value&gt;
&lt;value name=&quot;Low limit - Headrest&quot;&gt;320&lt;/value&gt;
&lt;value name=&quot;High limit - Headrest&quot;&gt;450&lt;/value&gt;
&lt;value name=&quot;Delivery OK&quot;&gt;False&lt;/value&gt;
&lt;/test&gt;
&lt;/results&gt;
&lt;/eolresult&gt;`

And I need to somehowe parse all of these xml files to tables for. example by the "Test name" because I would like to create a report from these data via Power BI. Or if there is some another way how to process these files, I will be glad for every idea.

What I imagine is that the tables would look something like this. Each table contain the ID, the date from the eolpresenting and the values from the tests.
Example:

MS SQL 解析 XML 文件到表格

答案1

得分: -1

要解析XML文件并将数据存储在MS SQL中的表中,您可以使用SQL Server内置的XML功能。

编写一个脚本或存储过程来解析XML文件并将数据插入表中。

  • EolResultData表位于开头

  • XML数据被正确声明为XML变量,@XMLData。

  • 使用游标来迭代XML数据并提取所需信息。

    嵌套的游标被设计成处理XML数据中的测试和值节点。

  • 解析的数据被插入到EolResultData表中。

在此模板中,您可以根据实际存储XML文件的表名和列名替换游标声明中的*YourTableName*。

此代码应解析XML文件,提取所需数据,并将其存储在EolResultData表中,该表具有ID、DateTime、TestName、ValueName和Value列。

更新:

假设多个XML记录位于<eolresults>元素下方,它可以正确处理它们,而无需使用游标,正如某人建议的那样。

修改后的模板在此处使用nodes()方法提取单独的<eolpresetting>节点,然后执行INSERT INTO SELECT操作将数据插入到EolResultData表中。它还具有额外的orderId列。

英文:

To parse XML files and store the data in tables in MS SQL, you can use the SQL Server's built-in XML functionality

Write a script or stored procedure to parse the XML files and insert the data into the tables.

  • EolResultData table is included at the beginning

  • The XML data is properly declared as an XML variable, @XMLData.

  • Cursors are used to iterate through the XML data and extract the required information.

    The nested cursors are structured to process the test and value nodes
    within the XML data.

  • The parsed data is inserted into the EolResultData table.

Here is the template you can adapt

In the above replace YourTableName in the cursor declaration with the actual table name and column name where your XML files are stored.

This code should parse the XML files, extract the required data, and store it in the EolResultData table, which has columns for ID, DateTime, TestName, ValueName, and Value.

Update:

Assuming multiple XML records are present under the &lt;eolresults&gt; element, it can handle them correctly without using cursors as someone suggested.

The modified template here uses the nodes() method to extract individual &lt;eolpresetting&gt; nodes and then performs an INSERT INTO SELECT operation to insert the data into the EolResultData table. It has the extra orderId col

答案2

得分: -1

这是您要翻译的内容:

不需要在这里使用游标,您可以直接在INSERT语句中使用.nodes

甚至可以将一个.nodes的集合与另一个集合进行交叉连接,还可以将单个节点传递给.nodes以解析第二级。

如果您正在使用包含数据的实际表格,可以将FROM更改为类似于以下内容:

这是您需要的全部代码,无需使用游标。请注意使用text(),它比从.进行隐式转换更高效。

您可能希望进一步将其转换为单独的表格,可以使用XQuery谓词,例如:

db<>fiddle

db<>fiddle

英文:

You don't need a cursor here, you can just use .nodes directly in an INSERT.

You can even cross-join one set of .nodes with another, and you can also pass a single node to .nodes to shred a second level.

INSERT INTO EolResultData
  (DateTime, OrderId, TestName, ValueName, Value)
SELECT
  DateTime = x1.presetting.value(&#39;(datetime/text())[1]&#39;, &#39;datetime&#39;),
  OrderId = x1.presetting.value(&#39;(order/@id)[1]&#39;, &#39;nvarchar(100)&#39;),
  Testname = x2.test.value(&#39;@name&#39;, &#39;nvarchar(100)&#39;),
  ValueName = x3.value.value(&#39;@name&#39;, &#39;nvarchar(100)&#39;),
  Value = x3.value.value(&#39;text()[1]&#39;, &#39;nvarchar(100)&#39;)
FROM @XMLData.nodes(&#39;eolresult/eolpresetting&#39;) x1(presetting)
CROSS APPLY @XMLData.nodes(&#39;eolresult/results/test&#39;) x2(test)
CROSS APPLY x2.test.nodes(&#39;value&#39;) x3(value);

db<>fiddle

If you are using an actual table with the data, you can change the FROM to something like this:

FROM YourTable t
CROSS APPLY (VALUES( CAST(t.SomeColumn AS xml) )) v(XMLData) -- only necessary if conversion is needed
CROSS APPLY v.XMLData.nodes(&#39;eolresult/eolpresetting&#39;) x1(presetting)
CROSS APPLY v.XMLData.nodes(&#39;eolresult/results/test&#39;) x2(test)
CROSS APPLY x2.test.nodes(&#39;value&#39;) x3(value);

This is all the code you need, no cursors necessary. Note the use of text() which is more performant than an implicit conversion from .

You may want to further transform it into separate tables, for that you can use XQuery predicates, such as this

SELECT
  DateTime = x1.presetting.value(&#39;(datetime/text())[1]&#39;, &#39;datetime&#39;),
  OrderId = x1.presetting.value(&#39;(order/@id)[1]&#39;, &#39;nvarchar(100)&#39;),
  Testname = x2.test.value(&#39;@name&#39;, &#39;nvarchar(100)&#39;),
  [Measured value] = x2.test.value(&#39;(value[@name=&quot;Measured value&quot;]/text())[1]&#39;, &#39;nvarchar(100)&#39;),
  [Low limit]      = x2.test.value(&#39;(value[@name=&quot;Low limit&quot;     ]/text())[1]&#39;, &#39;nvarchar(100)&#39;),
  [High limit]     = x2.test.value(&#39;(value[@name=&quot;High limit&quot;    ]/text())[1]&#39;, &#39;nvarchar(100)&#39;),
  [Test OK]        = x2.test.value(&#39;(value[@name=&quot;Test OK&quot;       ]/text())[1]&#39;, &#39;nvarchar(100)&#39;)
FROM @XMLData.nodes(&#39;eolresult/eolpresetting&#39;) x1(presetting)
CROSS APPLY @XMLData.nodes(&#39;eolresult/results/test[@name = &quot;Light&quot;]&#39;) x2(test);

db<>fiddle

huangapple
  • 本文由 发表于 2023年6月12日 15:25:32
  • 转载请务必保留本文链接:https://go.coder-hub.com/76454398.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定