Large PUT requests from Go to CouchDB

I've been having a vexing problem with CouchDB and Golang. When sending POST/PUT requests to CouchDB with relatively large Body sizes (the threshold seems to be ~8000 bytes or so), the connection times out and I get a "tcp: use of closed network connection" error from Go.

Eventually (a second or two later), CouchDB sends a 500 Response along with a:

{"error":"unknown_error", "reason": "noproc"}

in the body. The couchdb log also has a stack trace written to it:

Stacktrace: [{couch_db,collect_results,3,
                [{file,"couch_db.erl"},{line,833}]},
             {couch_db,write_and_commit,4,
                [{file,"couch_db.erl"},{line,845}]},
             {couch_db,update_docs,4,
                [{file,"couch_db.erl"},{line,782}]},
             {couch_db,update_doc,4,
                [{file,"couch_db.erl"},{line,426}]},
             {couch_httpd_db,update_doc,6,
                [{file,"couch_httpd_db.erl"},{line,753}]},
             {couch_httpd_db,do_db_req,2,
                [{file,"couch_httpd_db.erl"},{line,234}]},
             {couch_httpd,handle_request_int,5,
                [{file,"couch_httpd.erl"},{line,318}]},
             {mochiweb_http,headers,5,
                [{file,"mochiweb_http.erl"},{line,94}]}]

So I wrote up a quick unit test to replicate the issue (and to make sure it's not my wrapper causing the problem). I'm making the request like so:

client := &http.Client{}
req, err := http.NewRequest(
	"PUT",
	"http://localhost:5984/unittestdb/testdocid1",
	bytes.NewReader(testBody1), //testBody1 is 10000 bytes of json object
)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := client.Do(req)

This replicated the issue... so I tried curling a large json document to couchdb. That worked.

So, I fired up wireshark and examined the request made by my code to CouchDB and compared it to a request sent from curl. I noticed one additional header in the Curl Request:

Expect: 100-continue

I had to look that one up, as I couldn't recall having seen (or maybe I just hadn't had to deal with) that header before. So then I did a google search to see if the Golang http client supported the Expect/Continue functionality, and found this: https://github.com/golang/go/issues/3665

So Go doesn't support it and won't at least until 1.6. So I assumed it must be an obscure thing and wasn't the source of my problem. I spent a couple of hours randomly trying other stuff with the http.Client, Transport, etc.

Eventually, I manually set the "Expect: 100-continue" header in my http request in Go (if the body size is over a certain size), and ... it worked. No timeout, no error, no couchdb barfing stack traces into the logs.

Now I am confused, if Go doesn't support this, how is it working? Am I just masking the problem by doing this? Or can I just shrug my shoulders and move on?

I suspect there is an issue on the CouchDB side, perhaps there's a configuration setting I'm missing?

I don't know the answer, but I think that the critical spec for you is RFC 2616 section 8.2.3. The whole section is interesting, but particularly this:

> "There is an exception to this rule: for compatibility with RFC 2068,
> a server MAY send a 100 (Continue) status in response to an HTTP/1.1
> PUT or POST request that does not include an Expect request-header
> field with the "100-continue" expectation. This exception, the purpose
> of which is to minimize any client processing delays associated with
> an undeclared wait for 100 (Continue) status, applies only to HTTP/1.1
> requests, and not to requests with any other HTTP-version value."

It sounds like you're getting caught between Go's missing 100-continue implementation, and some kind of corner case on the CouchDB side. (Whatever the behavior of the client, I don't think 500 Internal error is an appropriate response from the server.)

I'd try HTTP/1.0 as a workaround, if the Go client will let you set that.