How to recover from an asynchronous panic in external package

I'm learning Go and I'm trying to understand how to properly deal with panics from external packages.

Here is a runnable example, say a package defines the doFoo method. (It's located in the same package here for the sake of the example )

package main

import (
	"log"
	"net/http"

	"sync"
	"time"

	"github.com/gorilla/handlers"
	"github.com/gorilla/mux"
)
// Method from External package
func doFoo() {
	var wg sync.WaitGroup
	wg.Add(1)
	// Do some cool async stuff
	go func() {
		time.Sleep(500)
		wg.Done()
		panic("Oops !")
	}()
}

func router() *mux.Router {
	var router = mux.NewRouter().StrictSlash(true)
	router.HandleFunc("/doFoo", index).Methods("GET")
	return router
}

func main() {
	log.Fatal(http.ListenAndServe(":8080", handlers.RecoveryHandler()(router())))
}

func index(w http.ResponseWriter, r *http.Request) {
	defer func() {
		recover()
		w.WriteHeader(http.StatusInternalServerError)
	}()
	doFoo()
	w.WriteHeader(http.StatusOK)
}

Invoking the doFoo method will crash the server, I appreciate that is correct behavior, since the application is now in an undermined state. And it's best to crash and have subsequent requests forwarded to a different processes trough some load balancer.
But, my api server might still be serving other clients, it might be maintaining websockets, and I Might would also want to return a 500 error here.

Coming from nodejs, I am used to the concept of uncaughtException, for handeling uncaptured synchronous exceptions and unhandledRejection to handle uncaptured asynchronous exceptions. These two process constructs give the developer the choice to either crash the program right away ( if it makes sense ), or log the error, return a proper http code, and then maybe shutdown gracefully if needed.

In my online research I find a lot of resources saying, panic's are not like exceptions, they are unusual, you don't need to worry about them. But it seems like it's actually very easy to cause a panic when writing code. It's completely up to the developer to ensure his library does not panic, the human factor is 100% involved here.

This leads me to wonder, do I need to audit the entire code base of every single package I'm going to use, including all the package dependencies as well ? just because I have no means of safeguarding against a missed recover in some external package that will take down my whole server and ruin my user's experience ?

Or is there some strategy I am not aware of that I can fail gracefully when an asynchronous panic occurs in library code ?

I noticed there is graceful shutdown since 1.8, but I can't use this because my program has already crashed.
https://golang.org/pkg/net/http/#Server.Shutdown

There is the gorilla recovery handler, but again, this only protects against synchronous panics.
http://www.gorillatoolkit.org/pkg/handlers#RecoveryHandler

Update:
I am aware that panics are not exceptions. Restating that does not answer the question, panics and exceptions is not what this question is about. This question is about understanding what tools the language may provide to enforce boundaries without bestowing the need to read every single line in the entire package tree onto the developer. If it's not possible in the language then stating that is a valid answer. I just don't know if it is or not.

Panics are not exceptions. Do not treat them like exceptions and you will be fine.

First things first: Package APIs should never panic, they should always return an error except in certain very rare cases, and then they must be clearly documented as to when and why they can panic (regexp.MustCompile is a good example of something that may panic). Any package that panics if it hits an error (and doesn't have a very good reason to do so) is bad, don't use it.

If you do bounds checking, make sure not to acess nil pointers, etc you should never have to worry about panics.

As for recovering panics in a goroutine, unless the goroutine has its own recovery handler you can't.

If the goroutine in from a third party library, don't use that library! If they are lax enough not to check edge cases and/or are lazy enough to just panic on error, why are you using their code? Who knows what other mines it holds?

If the goroutine is your own code, try to eliminate things that can panic, then add a recovery handler to catch the ones you can't prevent if needed.