TraceKitTraceKit Docs
Backend SDKs

Go Integration Guide

Add TraceKit to your Go applications for automatic distributed tracing. Debug production Golang services with live breakpoints and complete request visibility.

Learn how to instrument your Go applications to send distributed traces to TraceKit.

AI Setup Available

Let AI assistants guide you through the entire setup process. Try AI Setup

One SDK, Complete Observability. Replace 80+ lines of boilerplate with 3 lines of code. Get distributed tracing + code monitoring with automatic instrumentation for all major frameworks.

  • 3-Line Setup -- vs 80+ lines manual
  • One-Line Wrappers -- Gin, Echo, GORM, Redis, gRPC
  • Code Monitoring -- Built-in, production-safe

Installation

go get github.com/Tracekit-Dev/go-sdk

Quick Start (3 Lines)

Initialize once in your main function:

package main

import (
    "context"
    "os"

    "github.com/Tracekit-Dev/go-sdk/tracekit"
)

func main() {
    // Initialize TraceKit SDK - that's it!
    sdk, _ := tracekit.NewSDK(&tracekit.Config{
        APIKey:               os.Getenv("TRACEKIT_API_KEY"),
        ServiceName:          "my-service",
        EnableCodeMonitoring: true,  // Enable live code debugging (optional)
    })
    defer sdk.Shutdown(context.Background())

    // Your application code...
}

Set EnableCodeMonitoring: true to automatically capture snapshots when errors occur. Perfect for debugging production issues! Learn more

Add Framework Middleware (One Line)

Gin Framework

r := gin.Default()
r.Use(sdk.GinMiddleware()) // All routes automatically traced!

Echo Framework

e := echo.New()
e.Use(sdk.EchoMiddleware()) // All routes automatically traced!

net/http (Standard Library)

mux.Handle("/", sdk.HTTPHandler(handler, "root")) // Automatically traced!

Instrument Data Stores (One Line Each)

Redis

client := redis.NewClient(&redis.Options{Addr: "localhost:6379"})
sdk.WrapRedis(client) // All Redis ops automatically traced!

MongoDB

opts := sdk.MongoClientOptions().ApplyURI("mongodb://localhost:27017")
client, _ := mongo.Connect(ctx, opts) // All queries automatically traced!

GORM (ORM)

db, _ := gorm.Open(postgres.Open(dsn), &gorm.Config{})
sdk.TraceGormDB(db) // All queries automatically traced!

db.Find(&users)
db.Create(&User{Name: "John"})

database/sql (PostgreSQL, MySQL, SQLite)

sqlDB, _ := sql.Open("postgres", dsn)
db := sdk.WrapDB(sqlDB, "postgresql") // All queries automatically traced!

rows, _ := db.QueryContext(ctx, "SELECT * FROM users")

HTTP Client

client := sdk.HTTPClient(&http.Client{Timeout: 10 * time.Second})
resp, _ := client.Get("https://api.example.com") // Automatically traced!

gRPC

// Server
server := grpc.NewServer(sdk.GRPCServerInterceptors()...)

// Client
conn, _ := grpc.Dial(addr, sdk.GRPCClientInterceptors()...)

Complete Example

Here's a complete working application with Gin + Redis:

package main

import (
    "context"
    "os"

    "github.com/gin-gonic/gin"
    "github.com/redis/go-redis/v9"
    "github.com/Tracekit-Dev/go-sdk/tracekit"
)

func main() {
    // 1. Initialize SDK
    sdk, _ := tracekit.NewSDK(&tracekit.Config{
        APIKey:      os.Getenv("TRACEKIT_API_KEY"),
        ServiceName: "my-api",
    })
    defer sdk.Shutdown(context.Background())

    // 2. Setup Redis with tracing
    redisClient := redis.NewClient(&redis.Options{Addr: "localhost:6379"})
    sdk.WrapRedis(redisClient)

    // 3. Setup HTTP server with tracing
    r := gin.Default()
    r.Use(sdk.GinMiddleware())

    r.GET("/api/users", func(c *gin.Context) {
        // Both endpoint and Redis calls are automatically traced!
        val, _ := redisClient.Get(c, "users").Result()
        c.JSON(200, gin.H{"data": val})
    })

    r.Run(":8080")
}

That's it! You now have complete distributed tracing with less than 20 lines of code. All HTTP endpoints and Redis operations are automatically traced with full context propagation.

Verify It Works

Start your application and make a few requests. Then open the Traces page in your TraceKit dashboard. You should see:

  • SERVER spans for each incoming HTTP request with route and status
  • CLIENT spans for outgoing HTTP calls (if instrumented)
  • DB spans for database queries (if instrumented)

Traces typically appear within 30 seconds. If you don't see them, check the Troubleshooting section.

Advanced: Manual OpenTelemetry Setup

The sections below show manual OpenTelemetry instrumentation for users who need fine-grained control. For most use cases, the TraceKit SDK above is recommended.

What Gets Traced Automatically?

With proper setup, these operations are traced automatically with zero manual instrumentation:

ComponentSpan TypeCaptured AttributesAuto-Traced?
HTTP EndpointsSERVERmethod, route, status_code, duration, client_ipYes
Database QueriesDBdb.system, db.statement, db.name, durationYes
HTTP Client CallsCLIENTmethod, url, status_code, duration, peer.serviceYes
Redis OperationsDBdb.system (redis), db.statement, durationYes
gRPC CallsSERVER/CLIENTrpc.system, rpc.service, rpc.method, status_codeYes
MongoDB QueriesDBdb.system (mongodb), db.operation, db.name, durationYes
LLM (OpenAI/Anthropic)CLIENTgen_ai.system, model, tokens, cost, finish_reason, latencyYes
Custom Business LogicCustomuser-defined attributesManual

Prerequisites

Installation

Install the required OpenTelemetry packages:

go get go.opentelemetry.io/otel
go get go.opentelemetry.io/otel/sdk
go get go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp
go get go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
go get go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin

Basic Setup

Create a tracing initialization function in your application:

1. Create tracing.go

package tracing

import (
    "context"
    "crypto/tls"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
    "go.opentelemetry.io/otel/propagation"
    "go.opentelemetry.io/otel/sdk/resource"
    "go.opentelemetry.io/otel/sdk/trace"
    semconv "go.opentelemetry.io/otel/semconv/v1.4.0"
)

func InitTracer(serviceName, endpoint, apiKey string, useSSL bool) (func(), error) {
    ctx := context.Background()

    // Configure OTLP exporter options
    var opts []otlptracehttp.Option
    opts = append(opts,
        otlptracehttp.WithEndpoint(endpoint),
        otlptracehttp.WithURLPath("/v1/traces"),
        otlptracehttp.WithHeaders(map[string]string{
            "X-API-Key": apiKey,
        }),
    )

    // Configure TLS
    if useSSL {
        opts = append(opts, otlptracehttp.WithTLSClientConfig(&tls.Config{}))
    } else {
        opts = append(opts, otlptracehttp.WithInsecure())
    }

    // Create OTLP exporter
    exporter, err := otlptracehttp.New(ctx, opts...)
    if err != nil {
        return nil, err
    }

    // Create trace provider
    tp := trace.NewTracerProvider(
        trace.WithBatcher(exporter),
        trace.WithResource(resource.NewWithAttributes(
            semconv.SchemaURL,
            semconv.ServiceNameKey.String(serviceName),
            semconv.ServiceVersionKey.String("1.0.0"),
        )),
    )

    // Set global trace provider
    otel.SetTracerProvider(tp)
    otel.SetTextMapPropagator(propagation.TraceContext{})

    // Return cleanup function
    return func() {
        _ = tp.Shutdown(ctx)
    }, nil
}

2. Initialize in main.go

package main

import (
    "log"
    "os"

    "yourapp/tracing"
)

func main() {
    // Initialize OpenTelemetry tracing
    cleanup, err := tracing.InitTracer(
        "my-service",                      // Service name
        "app.tracekit.dev",                // TraceKit endpoint
        os.Getenv("TRACEKIT_API_KEY"),     // API key from environment
        true,                              // Use SSL
    )
    if err != nil {
        log.Fatalf("Failed to initialize tracer: %v", err)
    }
    defer cleanup()

    // Your application code here...
}

Framework Integration

Gin Framework

package main

import (
    "github.com/gin-gonic/gin"
    "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"
)

func main() {
    cleanup, _ := tracing.InitTracer(...)
    defer cleanup()

    r := gin.Default()
    r.Use(otelgin.Middleware("my-service"))

    r.GET("/api/users", func(c *gin.Context) {
        c.JSON(200, gin.H{"users": []string{"alice", "bob"}})
    })

    r.Run(":8080")
}

net/http (Standard Library)

package main

import (
    "net/http"
    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

func main() {
    cleanup, _ := tracing.InitTracer(...)
    defer cleanup()

    mux := http.NewServeMux()
    handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("Hello, World!"))
    })
    mux.Handle("/", otelhttp.NewHandler(handler, "root"))

    http.ListenAndServe(":8080", mux)
}

Echo Framework

package main

import (
    "github.com/labstack/echo/v4"
    "go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho"
)

func main() {
    cleanup, _ := tracing.InitTracer(...)
    defer cleanup()

    e := echo.New()
    e.Use(otelecho.Middleware("my-service"))

    e.GET("/", func(c echo.Context) error {
        return c.String(200, "Hello, World!")
    })

    e.Start(":8080")
}

Auto-Instrumentation Libraries

These libraries automatically create child spans for common operations.

Database Queries

GORM (PostgreSQL, MySQL, SQLite):

import (
    "gorm.io/driver/postgres"
    "gorm.io/gorm"
    "go.opentelemetry.io/contrib/instrumentation/gorm.io/otelgorm"
)

func initDB() (*gorm.DB, error) {
    db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
    if err != nil {
        return nil, err
    }
    if err := db.Use(otelgorm.NewPlugin()); err != nil {
        return nil, err
    }
    return db, nil
}

database/sql with pgx:

import (
    "database/sql"
    "github.com/jackc/pgx/v5/stdlib"
    "github.com/XSAM/otelsql"
    semconv "go.opentelemetry.io/otel/semconv/v1.4.0"
)

func initDB() (*sql.DB, error) {
    driverName, err := otelsql.Register("pgx",
        otelsql.WithAttributes(semconv.DBSystemPostgreSQL),
    )
    if err != nil {
        return nil, err
    }
    db, err := sql.Open(driverName, dsn)
    if err != nil {
        return nil, err
    }
    return db, nil
}

HTTP Client Calls

import (
    "net/http"
    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

var httpClient = &http.Client{
    Transport: otelhttp.NewTransport(http.DefaultTransport),
}

func callExternalAPI(ctx context.Context) (*Response, error) {
    req, _ := http.NewRequestWithContext(ctx, "GET", "https://api.example.com/users", nil)
    resp, err := httpClient.Do(req)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()
    return parseResponse(resp)
}

Redis Operations

import (
    "github.com/redis/go-redis/v9"
    "github.com/redis/go-redis/extra/redisotel/v9"
)

func initRedis() *redis.Client {
    rdb := redis.NewClient(&redis.Options{
        Addr: "localhost:6379",
    })
    if err := redisotel.InstrumentTracing(rdb); err != nil {
        panic(err)
    }
    return rdb
}

gRPC Calls

Server:

import (
    "google.golang.org/grpc"
    "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
)

s := grpc.NewServer(
    grpc.UnaryInterceptor(otelgrpc.UnaryServerInterceptor()),
    grpc.StreamInterceptor(otelgrpc.StreamServerInterceptor()),
)

Client:

conn, err := grpc.Dial(
    "localhost:50051",
    grpc.WithUnaryInterceptor(otelgrpc.UnaryClientInterceptor()),
    grpc.WithStreamInterceptor(otelgrpc.StreamClientInterceptor()),
)

MongoDB Queries

import (
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
    "go.opentelemetry.io/contrib/instrumentation/go.mongodb.org/mongo-driver/mongo/otelmongo"
)

func initMongo(ctx context.Context) (*mongo.Client, error) {
    opts := options.Client().
        ApplyURI("mongodb://localhost:27017").
        SetMonitor(otelmongo.NewMonitor())
    client, err := mongo.Connect(ctx, opts)
    if err != nil {
        return nil, err
    }
    return client, nil
}

Message Queues

Kafka (Sarama):

import (
    "github.com/IBM/sarama"
    "go.opentelemetry.io/contrib/instrumentation/github.com/IBM/sarama/otelsarama"
)

producer, err := sarama.NewSyncProducer([]string{"localhost:9092"}, config)
producer = otelsarama.WrapSyncProducer(config, producer)

RabbitMQ (amqp091-go):

import (
    amqp "github.com/rabbitmq/amqp091-go"
    "go.opentelemetry.io/contrib/instrumentation/github.com/rabbitmq/amqp091-go/otelmqp"
)

otlChannel := otelmqp.NewChannel(ch)
err := otlChannel.PublishWithContext(ctx, "", "my-queue", false, false, amqp.Publishing{
    ContentType: "text/plain",
    Body:        []byte(msg),
})

Code Monitoring

Code monitoring lets you capture variable snapshots from running production code without stopping your application. Set non-breaking breakpoints from the dashboard and inspect variable state when they trigger.

v6.0 adds production safety features including automatic PII scrubbing, crash isolation, circuit breakers, and a remote kill switch -- all enabled by default.

Production Safety

All production safety features are enabled by default -- no configuration required.

FeatureDescription
PII Scrubbing13 built-in patterns detect sensitive data (passwords, tokens, SSNs, credit cards). Enabled by default with typed [REDACTED:type] markers.
Crash IsolationAll entry points use defer/recover to catch panics. Snapshot failures never crash your application.
Circuit BreakerAfter 3 failures within 60 seconds, the circuit opens for a 5-minute cooldown. Automatically recovers when the backend is healthy again.
Remote Kill SwitchToggle code monitoring on or off from the dashboard. Changes propagate via SSE in real-time -- no restart required.

The Go SDK uses SSE (Server-Sent Events) for auto-discovery of breakpoint changes and kill switch state. This replaces the previous 30-second polling approach, giving you instant control over production instrumentation.

Code monitoring requires EnableCodeMonitoring: true in the SDK config. Breakpoints are managed from the Code Monitoring dashboard.

LLM Instrumentation

TraceKit provides an HTTP transport wrapper that automatically instruments OpenAI and Anthropic API calls. LLM calls appear as spans with OTel GenAI semantic convention attributes.

Captured Attributes

AttributeDescription
gen_ai.systemProvider name (openai, anthropic)
gen_ai.request.modelModel name (gpt-4o, claude-sonnet-4-20250514, etc.)
gen_ai.usage.input_tokensPrompt token count
gen_ai.usage.output_tokensCompletion token count
gen_ai.response.finish_reasonsFinish reason (stop, end_turn, tool_calls)

Environment Variable Override

Use TRACEKIT_LLM_CAPTURE_CONTENT=true to enable prompt/completion capture without code changes. Useful for enabling in staging but not production.

Streaming Support

Streaming responses produce a single span covering the entire stream. The transport wraps the response body to parse SSE events and accumulate token counts. The span finalizes when the stream is closed.

View LLM cost, token usage, and latency metrics on the dedicated LLM Observability dashboard at /ai/llm in your TraceKit instance.

Local UI (Development Mode)

Debug your Go application locally without creating an account. TraceKit Local UI runs on your machine at http://localhost:9999 and automatically receives traces when you run your app in development mode.

Automatic Detection

The Go SDK automatically detects when Local UI is running on port 9999 and sends traces to both Local UI and cloud (if you have an API key configured).

Quick Start

1. Install the Local UI:

npm install -g @tracekit/local-ui

2. Start the Local UI:

tracekit-local

3. Run your app in development mode:

ENV=development go run main.go

4. Open your browser:

http://localhost:9999

Features

  • Auto-Detection -- SDK checks for Local UI at localhost:9999 on startup
  • Real-Time Updates -- See traces instantly with WebSocket live updates
  • Development Only -- Only activates when ENV=development
  • Works Offline -- No internet connection required, everything runs locally

Service Discovery

TraceKit automatically instruments outgoing HTTP calls to create service dependency graphs. When your service makes an HTTP request to another service, TraceKit creates CLIENT spans and injects trace context headers.

Custom Service Name Mappings

For local development or when service names can't be inferred from hostnames, configure service name mappings:

sdk, _ := tracekit.NewSDK(&tracekit.Config{
    APIKey:      os.Getenv("TRACEKIT_API_KEY"),
    ServiceName: "my-service",
    ServiceNameMappings: map[string]string{
        "localhost:8082": "payment-service",
        "localhost:8083": "user-service",
        "localhost:8084": "inventory-service",
    },
})

client := sdk.HTTPClient(&http.Client{})
resp, _ := client.Get("http://localhost:8082/charge")
// Creates CLIENT span with peer.service = "payment-service"

Service Name Detection

TraceKit intelligently extracts service names from URLs:

URLExtracted Service Name
http://payment-service:3000payment-service
http://payment.internalpayment
http://payment.svc.cluster.localpayment
https://api.example.comapi.example.com

Visit your TraceKit dashboard to see the Service Map -- a visual graph showing which services call which, with health metrics and latency data.

Manual Instrumentation

The Go SDK provides convenience wrappers around OpenTelemetry for common tracing operations.

package main

import (
    "context"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/codes"
)

func processOrder(ctx context.Context, orderID string) error {
    tracer := otel.Tracer("my-service")

    ctx, span := tracer.Start(ctx, "processOrder")
    defer span.End()

    span.SetAttributes(
        attribute.String("order.id", orderID),
        attribute.String("order.status", "processing"),
    )

    if err := validateOrder(ctx, orderID); err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        return err
    }

    if err := chargePayment(ctx, orderID); err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        return err
    }

    span.SetStatus(codes.Ok, "Order processed successfully")
    return nil
}

Environment Variables

Store sensitive configuration in environment variables:

# .env
TRACEKIT_API_KEY=tk_your_generated_api_key_here
TRACEKIT_ENDPOINT=app.tracekit.dev
TRACEKIT_SERVICE_NAME=my-backend-service

Load in your application:

import "os"

sdk, err := tracekit.NewSDK(&tracekit.Config{
    APIKey:      os.Getenv("TRACEKIT_API_KEY"),
    ServiceName: os.Getenv("TRACEKIT_SERVICE_NAME"),
    Endpoint:    os.Getenv("TRACEKIT_ENDPOINT"),
})

Production Configuration

Production Checklist:

  • Use HTTPS/TLS for the OTLP endpoint
  • Store API keys in a secrets manager (AWS Secrets Manager, HashiCorp Vault)
  • Set appropriate service names and versions
  • Configure resource attributes (deployment.environment, host.name, etc.)
  • Adjust sampling rates if needed for high-traffic services

Custom Metrics

The TraceKit Go SDK includes a lightweight metrics API for tracking counters, gauges, and histograms.

Counter

Track monotonically increasing values (requests, events):

counter := sdk.Counter("http.requests.total", map[string]string{
    "service": "api",
    "method":  "GET",
})
counter.Inc()
counter.Add(5)

Gauge

Track values that can go up or down (queue size, connections):

gauge := sdk.Gauge("http.connections.active", nil)
gauge.Set(42)
gauge.Inc()
gauge.Dec()

Histogram

Track value distributions (latencies, sizes):

histogram := sdk.Histogram("http.request.duration", map[string]string{
    "unit": "ms",
})
histogram.Record(45.2)
histogram.Record(123.5)

Metrics are automatically buffered and exported via OTLP. View them in the Metrics Explorer. View Custom Metrics Guide

Troubleshooting

Traces not appearing?

Cause: The SDK is not sending traces to TraceKit, usually due to misconfiguration.

Fix:

  • Verify API key is correct and not revoked
  • Check endpoint URL is app.tracekit.dev (no http:// prefix when using WithEndpoint)
  • Look for OpenTelemetry export errors in application logs
  • Confirm SamplingRate is not set to 0

Connection refused errors?

Cause: The TraceKit endpoint is unreachable from your application.

Fix:

  • Test connectivity with curl -X POST https://app.tracekit.dev/v1/traces (should return 401, not connection refused)
  • Check firewall/DNS rules

Next Steps

  • Add auto-instrumentation libraries for components you use (Redis, gRPC, MongoDB, etc.)
  • Explore your traces on the Traces page to identify performance bottlenecks
  • Optionally add custom spans for specific business logic you want to measure
  • Configure sampling for high-traffic services to reduce overhead
  • Set up alert rules to get notified when issues occur

Pro Tip

Start with framework middleware + database instrumentation. This usually gives you 80% coverage with just 2-3 lines of setup code. Add more auto-instrumentation libraries as needed.

SDKs

TraceKit SDKs for 8 backend languages and 5 frontend frameworks. Auto-instrumented tracing, error capture, code monitoring, and LLM observability.

Node.js / TypeScript Integration Guide

Learn how to integrate TraceKit with Node.js, Express, and NestJS. Automatic distributed tracing with live breakpoints for debugging production Node.js applications.

On this page

Recommended: TraceKit Go SDKInstallationQuick Start (3 Lines)Add Framework Middleware (One Line)Gin FrameworkEcho Frameworknet/http (Standard Library)Instrument Data Stores (One Line Each)RedisMongoDBGORM (ORM)database/sql (PostgreSQL, MySQL, SQLite)HTTP ClientgRPCComplete ExampleVerify It WorksAdvanced: Manual OpenTelemetry SetupWhat Gets Traced Automatically?PrerequisitesInstallationBasic Setup1. Create tracing.go2. Initialize in main.goFramework IntegrationGin Frameworknet/http (Standard Library)Echo FrameworkAuto-Instrumentation LibrariesDatabase QueriesHTTP Client CallsRedis OperationsgRPC CallsMongoDB QueriesMessage QueuesCode MonitoringProduction SafetyLLM InstrumentationCaptured AttributesEnvironment Variable OverrideStreaming SupportLocal UI (Development Mode)Quick StartFeaturesService DiscoveryCustom Service Name MappingsService Name DetectionManual InstrumentationEnvironment VariablesProduction ConfigurationCustom MetricsCounterGaugeHistogramTroubleshootingTraces not appearing?Connection refused errors?Next Steps