Unit-testing http request retry with httptest

I'm struggling to write a unit test for a component called HttpRequest that wraps HTTP requests and handles response unmarshalling. Recently, I added a feature to this component that allows it to retry an HTTP request if it encounters a "connection refused" error on the first attempt.

To use the HttpRequest component, I call it once like this: user, err := HttpRequest[User](config). The config parameter contains all the necessary information to execute the request, such as the URL, method, timeout, retry count, and request body. It also unmarshals the response body to an instance of the specified type (User in this case)

The problem arises when I try to test the scenario where the initial request fails with a "connection refused" error, but the second attempt is successful. The retries happen internally within the component, so I only make a single call to the component.

I've found it challenging to create a unit test for this scenario because, in order for a request to fail with "connection refused," there needs to be no listener on the port being called. The issue is that when using httptest, it always listens on a port when an instance is created, even with httptest.NewUnstartedServer. Consequently, after the httptest instance is created, I will never encounter a "connection refused" error in my client code.

However, before creating the httptest instance, I don't know which port it will be listening on. httptest always picks a random port, and there is no way to specify one programmatically. This means I can't make the HttpRequest call before creating the httptest instance.

Does anyone have any ideas on how to effectively unit test such a scenario?

NewUnstartedServer is as simple as:

func NewUnstartedServer(handler http.Handler) *Server {
	return &Server{
		Listener: newLocalListener(),
		Config:   &http.Server{Handler: handler},
	}
}

If picking a port by yourself works for you, you can do it like this:

func MyNewUnstartedServer(port int, handler http.Handler) *httptest.Server {
	addr := fmt.Sprintf("127.0.0.1:%d", port)
	l, err := net.Listen("tcp", addr)
	if err != nil {
		addr = fmt.Sprintf("[::1]::%d", port)
		if l, err = net.Listen("tcp6", addr); err != nil {
			panic(fmt.Sprintf("httptest: failed to listen on a port: %v", err))
		}
	}
	return &httptest.Server{
		Listener: l,
		Config:   &http.Server{Handler: handler},
	}
}

The code to create a listener is modified from httptest.newLocalListener.

Another option is to implement the http.RoundTripper interface and create an http.Client with this RoundTripper. Here is an example copied from net/http/client_test.go:

type recordingTransport struct {
	req *Request
}

func (t *recordingTransport) RoundTrip(req *Request) (resp *Response, err error) {
	t.req = req
	return nil, errors.New("dummy impl")
}

func TestGetRequestFormat(t *testing.T) {
	setParallel(t)
	defer afterTest(t)
	tr := &recordingTransport{}
	client := &Client{Transport: tr}
	url := "http://dummy.faketld/"
	client.Get(url) // Note: doesn't hit network
	if tr.req.Method != "GET" {
		t.Errorf("expected method %q; got %q", "GET", tr.req.Method)
	}
	if tr.req.URL.String() != url {
		t.Errorf("expected URL %q; got %q", url, tr.req.URL.String())
	}
	if tr.req.Header == nil {
		t.Errorf("expected non-nil request Header")
	}
}