Retrieving the "outerxml" of an element (like innerxml, but including the element itself)

I've got a Go program that's working with an input stream, i.e. os.Stdin: a very large XML file, so I can't process it all at once.

I'm wanting to extract all XML elements of a certain nature for post-processing.

I've got no trouble identifying the elements for extraction, and getting the related start and end element. However I'm not sure how to dump the whole element as a string, as opposed to only the inner XML.

For instance, imagine I have the following XML:

<a>
  <b somethingUseful="1">
    <c>Hello</c>
    <d>world</d>
  </b>
  <e>
    <foo/>
  </e>
  <!-- Imagine there were 1 billion lines in between -
       I need to stream this! -->
  <b somethingUseful="321">
    <c>Hello again</c>
  </b>
</a>

In this example, I want to output each of the <b> elements, from start to finish.

Using innerxml with DecodeElement, I'm able to get this far, in a streaming fashion:

Here comes a B:

    <c>Hello</c>
    <d>world</d>
  
Here comes a B:

    <c>Hello again</c>

So close, but it's missing the <b> tag (and attributes) itself. I haven't been able to figure out how to make that last step without sacrificing the streaming nature of the decoding.

To be clear, the output that I desire is something like:

Here comes a B:
  <b somethingUseful="1">
    <c>Hello</c>
    <d>world</d>
  </b>
Here comes a B:
  <b somethingUseful="321">
    <c>Hello again</c>
  </b>

Here's a playground that enunciates this example and what I've done in getting this far:

https://play.golang.org/p/XqJY_1pa9j

Inspired by @nothingmuch's usage of decoder.InputOffset, I use a TeeReader to split the input Reader into two: the standard which gets parsed through the decoder, and a buffer that we'll use to output the exact element (which lies between the decoder.InputOffset before and after the element is encountered).

To minimise memory usage, the buffer is continuously cleared only up to the point we know is not potentially matching. We maintain offsets to keep track of this. This added complexity is necessary because the decoder can grab bytes from the reader further ahead of the token at hand, so we need to be careful not to clear something we actually need.

So the additional memory usage is only as much as:

1. The largest two tokens that may be simultaneously stored in the buffer before it's cleared back to one.
2. The size of the actual element being output.

Here's an updated playground with the solution:

https://play.golang.org/p/H8WVDWI57r

A rather crude approach is to save the offsets and just read those bytes, by asking the decoder for the offset before the start element and after the end element.

See this playground example, which fans out the reader into two pipes, one of which goes to the XML decoder while the other pipeline is buffered and then used for extracting byte ranges corresponding to XML elements.

The XML decoding routine then writes pairs of offsets on a channel, which another thread uses to skip or output regions of interest from the copy of the reader stream. This should probably be done more seriously than the hack job I did, i.e. by using a stack and matching filter criteria.

This solution assumes Seek/ReadAt are not viable, in retrospect I probably overdid it there, this would be much simpler if you just opened the file twice, assuming it is a file.