What are the some use cases for OpenTelemetry Collectors?

We are trying out multiple observability tools (like Jaeger, Prometheus, etc).

While building a demo application that sends the distributed tracing data to Jaeger. We used 'go.opentelemetry.io/otel/exporters/jaeger' to export the data to tracing data to Jaeger.

Which works fine and seem to fulfil our purpose. But while going through the Otel Documentation, we found out about "OpenTelemetry Collector".

Although we have a very high level understanding, but we do not seem to fully understand the correct use case for Otel Collector over the exporter that we are using.

TLDR; We are trying to understand the use cases and advantages of Otel Collector over the method were we directly export data to backed (In our case Jaeger).

Additional Information:

Following is the code snippet (written in Go) used to send the tracing data to Jaeger.

func tracerProvider(url string) (*tracesdk.TracerProvider, error) {
	exp, err := jaeger.New(jaeger.WithCollectorEndpoint(jaeger.WithEndpoint(url)))
	if err != nil {
		return nil, err
	}
	tp := tracesdk.NewTracerProvider(
		tracesdk.WithBatcher(exp),
		tracesdk.WithResource(resource.NewWithAttributes(
			semconv.SchemaURL,
			semconv.ServiceNameKey.String(service),
			attribute.String("environment", environment),
		)),
	)
	return tp, nil
} 



func main() {
	tp, err := tracerProvider("http://localhost:14268/api/traces")
	if err != nil {
		log.Fatal(err)
	}

	otel.SetTracerProvider(tp)
	propagator := propagation.NewCompositeTextMapPropagator(propagation.Baggage{}, propagation.TraceContext{})
	otel.SetTextMapPropagator(propagator)

// Business code

}


func serviceTwoCall(ctx context.Context, throwerror bool) *http.Response {
	url := fmt.Sprintf("http://localhost:8080", throwerror)
	req, _ := http.NewRequest(http.MethodGet, url, nil)
	otelhttptrace.Inject(ctx, req)

	client := &http.Client{}
	res, err := client.Do(req)
	if err != nil {
		fmt.Printf("Service Two Error %v", err.Error())
	}
	return res
}

Thanks!

Essentially, the advantage of the Open Telemetry Collector is that your code can remain entirely agnostic to what vendor you use for storing/querying metrics and tracing. It means that whatever that vendor is, is no longer a detail that is relevant to your application and that if in an environment you wanted to change what that was (e.g local development), you can simply configure the collector to behave differently.

The collector also means that more exporters may be supported across multiple languages. Whilst the Go implementation currently supports exporting directly to Jaeger, this may not be an option in other languages, but all languages should support the collector, which can then export off to whatever your preference is.

In my personal experience, for smaller installations it can make sense to export directly to Jaeger. Introducing another component to host within your clusters can complicate matters where your application is relatively simple. The threshold at which it makes sense to use the collector depends on many factors such as the size of your teams, who is responsible for telemetry, if other languages/runtimes are used etc.

The greatest advantages are less code for you to maintain and more exporter options out of the box. This answer also mentions other advantages.

This post (relevant excerpt below) explains the difference between the sdks and collector. My understanding is that using the sdk+collector would be the same as doing everything yourself, like you said you did.

> When it comes to collecting and ingesting the data, there are two components in play:
> - The SDK: in charge of collecting the data.
> - The Collector: the component responsible for how to receive, process, and export
> telemetry data.