cody-gateway: treat incoming traces as links instead of parents (#54235)

Lately I've noticed many (most?) of our Sentry events come with
`TraceId` that 404's on Cloud Trace, which kind of defeats the whole
purpose of our tracing. I've spot-checked and see that all our traces in
Cloud Trace have corresponding `Request` logs, but when attempting the
inverse we often get a 404 in Cloud Trace.

I think the problem is that we are treating incoming traces as parents
instead of linked traces. Treating the incoming request as a parent is
cool because you get to see the whole thing end-to-end across projects,
but it has the issue where half our traces are missing parents, and I
think this is what causes logs to include traces that 404 when you query
it on Cloud Trace

Was thinking of performing the parent/linking based on whether the
request emits a ShouldTrace policy, but we have the same issue with
on-prem customers etc if they enable tracing we'll end up with traces
missing parents on our end that causes misbehaviour in Cloud Trace. I
think I'm going to revert our usage of having remote traces be treated
as parents, and just preserve them as links instead. This still gives us
some bidirectional linking:

- In Sourcegraph/App/etc, we keep the return trace/span in trace
attributes and also send it back to clients.
- In Cody Gateway, the span link will point us to the Sourcegraph trace

## Test plan

CI

Author: bobheadxi

enterprise/cmd/cody-gateway/internal/httpapi/BUILD.bazel

@@ -9,6 +9,7 @@ import (
 	"github.com/sourcegraph/log"
 	"github.com/sourcegraph/log/hook"
 	"github.com/sourcegraph/log/output"
+	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
 
 	"github.com/sourcegraph/sourcegraph/enterprise/cmd/cody-gateway/internal/actor"
 	"github.com/sourcegraph/sourcegraph/enterprise/cmd/cody-gateway/internal/auth"
@@ -96,8 +97,11 @@ func NewDiagnosticsHandler(baseLogger log.Logger, next http.Handler, secret stri
 
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		if strings.HasPrefix(r.URL.Path, "/-/") {
-			instrumentation.HTTPMiddleware("diagnostics", requestlogger.Middleware(baseLogger, handler)).
-				ServeHTTP(w, r)
+			instrumentation.HTTPMiddleware(
+				"diagnostics",
+				requestlogger.Middleware(baseLogger, handler),
+				otelhttp.WithPublicEndpoint(),
+			).ServeHTTP(w, r)
 			return
 		}
 

enterprise/cmd/cody-gateway/internal/httpapi/diagnostics.go

@@ -5,6 +5,7 @@ import (
 
 	"github.com/gorilla/mux"
 	"github.com/sourcegraph/log"
+	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
 
 	"github.com/sourcegraph/sourcegraph/enterprise/cmd/cody-gateway/internal/auth"
 	"github.com/sourcegraph/sourcegraph/enterprise/cmd/cody-gateway/internal/events"
@@ -51,6 +52,7 @@ func NewHandler(logger log.Logger, eventLogger events.Logger, rs limiter.RedisSt
 						),
 					),
 				),
+				otelhttp.WithPublicEndpoint(),
 			),
 		)
 	}
@@ -71,6 +73,7 @@ func NewHandler(logger log.Logger, eventLogger events.Logger, rs limiter.RedisSt
 						),
 					),
 				),
+				otelhttp.WithPublicEndpoint(),
 			),
 		)
 
@@ -82,6 +85,7 @@ func NewHandler(logger log.Logger, eventLogger events.Logger, rs limiter.RedisSt
 						embeddings.NewListHandler(),
 					),
 				),
+				otelhttp.WithPublicEndpoint(),
 			),
 		)
 
@@ -102,6 +106,7 @@ func NewHandler(logger log.Logger, eventLogger events.Logger, rs limiter.RedisSt
 						),
 					),
 				),
+				otelhttp.WithPublicEndpoint(),
 			),
 		)
 	}