Fetch YouTube Transcripts in Go: A Hands-On Tutorial

Go is one of the best choices for API-heavy backend work. Its goroutines start with roughly 2KB of stack space compared to 1MB or more for OS threads (Go Developer Survey 2024 H2), which means you can run tens of thousands of concurrent HTTP calls without breaking a sweat. Pair that with Go's fast JSON decoder and a clean net/http package, and you have a solid foundation for building a high-throughput transcript pipeline.

This tutorial walks through everything you need to fetch YouTube transcripts from Go: a properly configured HTTP client, typed structs for JSON decoding, a goroutine worker pool, a token-bucket rate limiter, and idiomatic retry logic. You'll use the YouTube Transcriber API, which handles YouTube IP blocks, proxy rotation, and anti-bot challenges so your Go code only has to deal with standard HTTP responses.

If you're coming from Python, check out Get YouTube Transcripts in Python: The Complete Guide for a direct comparison of the two approaches.

---

Prerequisites

• Go 1.21 or later
• A YouTube Transcriber API key — sign up at getyoutubetranscriber.com for 100 free credits, no credit card required
• Basic familiarity with Go's net/http package

---

1. Setting Up an HTTP Client With Proper Timeouts

Never use http.DefaultClient for outbound API calls. The default client has no timeout, which means a slow or hanging connection will block a goroutine indefinitely. Define your own client at package level and reuse it:

package main

import (
    "net/http"
    "time"
)

var apiClient = &http.Client{
    Timeout: 30 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        100,
        MaxIdleConnsPerHost: 20,
        IdleConnTimeout:     90 * time.Second,
    },
}

Why these values?

• Timeout: 30s covers the full request lifecycle (dial + TLS + response body). Transcripts for long videos can take a moment to return; 30 seconds gives headroom without letting a goroutine hang forever.
• MaxIdleConnsPerHost: 20 lets you maintain a pool of keep-alive connections to the same host, which cuts TCP handshake overhead when you're fetching many transcripts.
• IdleConnTimeout: 90s matches common server-side keep-alive windows.

Go's official net/http documentation at pkg.go.dev/net/http covers every field on Transport if you need to tune further for your environment.

---

2. Structs for Decoding the YouTube Transcript API JSON

The YouTube Transcriber /api/v2/transcript endpoint returns JSON with a segments array. Define typed structs so encoding/json does the heavy lifting:

package main

import "encoding/json"

// Segment represents a single timed caption segment.
type Segment struct {
    Text     string  `json:"text"`
    Start    float64 `json:"start"`
    Duration float64 `json:"duration"`
}

// TranscriptResponse is the top-level API response.
type TranscriptResponse struct {
    VideoID  string    `json:"video_id"`
    Language string    `json:"language"`
    Segments []Segment `json:"segments"`
}

// APIError captures structured error responses from the API.
type APIError struct {
    Error   string `json:"error"`
    Message string `json:"message"`
}

Now write a small fetch function that calls the API and decodes into these types:

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
    "net/url"
)

const (
    baseURL = "https://getyoutubetranscriber.com/api/v2/transcript"
    apiKey  = "YOUR_API_KEY" // replace with your Bearer token
)

func fetchTranscript(videoID string) (*TranscriptResponse, error) {
    params := url.Values{}
    params.Set("video_url", videoID)

    req, err := http.NewRequest(http.MethodGet, baseURL+"?"+params.Encode(), nil)
    if err != nil {
        return nil, fmt.Errorf("build request: %w", err)
    }
    req.Header.Set("Authorization", "Bearer "+apiKey)
    req.Header.Set("Accept", "application/json")

    resp, err := apiClient.Do(req)
    if err != nil {
        return nil, fmt.Errorf("http do: %w", err)
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        var apiErr APIError
        _ = json.NewDecoder(resp.Body).Decode(&apiErr)
        return nil, fmt.Errorf("api error %d: %s", resp.StatusCode, apiErr.Message)
    }

    var result TranscriptResponse
    if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
        return nil, fmt.Errorf("decode: %w", err)
    }
    return &result, nil
}

Gotcha: always check resp.StatusCode before decoding. If the API returns a 4xx or 5xx, the body is an error object, not a transcript. Decoding an error body into TranscriptResponse silently produces empty structs—exactly the kind of silent failure that wastes debugging time.

---

3. Concurrent Fetching With Goroutines and a Worker Pool

If you have a list of video IDs, you want to fetch them in parallel. The naive approach—spawn one goroutine per video—works for small lists but falls apart at scale. Instead, use a fixed-size worker pool:

package main

import (
    "fmt"
    "sync"
)

// Result bundles a transcript with its video ID and any error.
type Result struct {
    VideoID    string
    Transcript *TranscriptResponse
    Err        error
}

// fetchConcurrent fetches transcripts for all videoIDs using a pool of workers.
func fetchConcurrent(videoIDs []string, workers int) []Result {
    jobs := make(chan string, len(videoIDs))
    results := make(chan Result, len(videoIDs))

    // Start worker goroutines.
    var wg sync.WaitGroup
    for i := 0; i < workers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for id := range jobs {
                tr, err := fetchWithRetry(id, 3)
                results <- Result{VideoID: id, Transcript: tr, Err: err}
            }
        }()
    }

    // Send all jobs.
    for _, id := range videoIDs {
        jobs <- id
    }
    close(jobs)

    // Wait for all workers, then close results.
    go func() {
        wg.Wait()
        close(results)
    }()

    // Collect results.
    out := make([]Result, 0, len(videoIDs))
    for r := range results {
        out = append(out, r)
    }
    return out
}

How it works:

• jobs is a buffered channel pre-filled with all video IDs.
• Each worker goroutine reads from jobs until it's closed.
• Workers write into results. The wg.Wait() in a separate goroutine closes results once all workers finish, which unblocks the final range loop.
• The pool size (workers) controls maximum parallelism. Start with 5 and tune up.

This pattern is idiomatic Go. The sync.WaitGroup ensures no result is missed, and closing channels signals completion cleanly without sentinel values.

---

4. Rate Limiting With a Token Bucket

Even with a worker pool, hammering an API without rate limiting will get you throttled. Go's time.Ticker gives you a simple token-bucket limiter without any external library:

package main

import "time"

// RateLimiter wraps a ticker to enforce a per-second call limit.
type RateLimiter struct {
    ticker *time.Ticker
}

// NewRateLimiter creates a limiter for rps requests per second.
func NewRateLimiter(rps int) *RateLimiter {
    return &RateLimiter{
        ticker: time.NewTicker(time.Second / time.Duration(rps)),
    }
}

// Wait blocks until the next token is available.
func (r *RateLimiter) Wait() {
    <-r.ticker.C
}

// Stop cleans up the underlying ticker.
func (r *RateLimiter) Stop() {
    r.ticker.Stop()
}

Then wire it into your worker goroutines:

limiter := NewRateLimiter(5) // 5 requests per second
defer limiter.Stop()

go func() {
    defer wg.Done()
    for id := range jobs {
        limiter.Wait()   // block until our slot opens
        tr, err := fetchWithRetry(id, 3)
        results <- Result{VideoID: id, Transcript: tr, Err: err}
    }
}()

Gotcha: if you need burst capacity (e.g., allow 10 requests instantly but average 5/s), reach for the golang.org/x/time/rate package instead. It implements a proper token-bucket algorithm with configurable burst size. The ticker approach above is strict: exactly one request per interval, no bursting.

---

5. Error Handling and Retries Idiomatic to Go

Go doesn't have exceptions. Every error is a value you check explicitly. For transient HTTP failures (rate limits, network blips, 5xx errors), you want exponential backoff:

package main

import (
    "errors"
    "fmt"
    "net/http"
    "time"
)

// retryableStatus returns true for errors worth retrying.
func retryableStatus(code int) bool {
    return code == http.StatusTooManyRequests ||
        code == http.StatusServiceUnavailable ||
        code == http.StatusGatewayTimeout ||
        code == http.StatusBadGateway
}

// fetchWithRetry calls fetchTranscript up to maxAttempts times with exponential backoff.
func fetchWithRetry(videoID string, maxAttempts int) (*TranscriptResponse, error) {
    var lastErr error
    backoff := 1 * time.Second

    for attempt := 1; attempt <= maxAttempts; attempt++ {
        tr, err := fetchTranscript(videoID)
        if err == nil {
            return tr, nil
        }

        // Only retry on explicitly retryable errors.
        // For this demo we inspect the error string; in production,
        // use a custom error type to carry the status code.
        lastErr = err
        fmt.Printf("attempt %d for %s failed: %v — retrying in %s\n",
            attempt, videoID, err, backoff)

        if attempt < maxAttempts {
            time.Sleep(backoff)
            backoff *= 2 // double the wait each time
        }
    }

    return nil, fmt.Errorf("all %d attempts failed for %s: %w", maxAttempts, videoID, lastErr)
}

Production tip: carry the HTTP status code inside a custom error type so retryableStatus can make precise decisions without string parsing. A simple wrapper looks like:

type HTTPError struct {
    StatusCode int
    Message    string
}

func (e *HTTPError) Error() string {
    return fmt.Sprintf("HTTP %d: %s", e.StatusCode, e.Message)
}

// Then in fetchTranscript:
return nil, &HTTPError{StatusCode: resp.StatusCode, Message: apiErr.Message}

// And in fetchWithRetry:
var httpErr *HTTPError
if errors.As(err, &httpErr) && retryableStatus(httpErr.StatusCode) {
    // retry
}

Using errors.As is the idiomatic way to inspect wrapped error types in Go 1.13+. Avoid string matching on error messages—it breaks silently when message formats change.

---

Putting It All Together

Here's a minimal main that fetches transcripts for a list of videos and prints a word count for each:

package main

import (
    "fmt"
    "strings"
)

func main() {
    videoIDs := []string{
        "dQw4w9WgXcQ",
        "9bZkp7q19f0",
        "JGwWNGJdvx8",
    }

    limiter := NewRateLimiter(5)
    defer limiter.Stop()

    results := fetchConcurrent(videoIDs, 3) // 3 workers, rate-limited to 5 rps

    for _, r := range results {
        if r.Err != nil {
            fmt.Printf("[ERROR] %s: %v\n", r.VideoID, r.Err)
            continue
        }

        words := 0
        for _, seg := range r.Transcript.Segments {
            words += len(strings.Fields(seg.Text))
        }
        fmt.Printf("[OK] %s — %d segments, ~%d words, lang=%s\n",
            r.VideoID,
            len(r.Transcript.Segments),
            words,
            r.Transcript.Language,
        )
    }
}

For a 10-minute video, you might see output like:

[OK] dQw4w9WgXcQ — 147 segments, ~1840 words, lang=en
[OK] 9bZkp7q19f0 — 312 segments, ~3900 words, lang=en
[OK] JGwWNGJdvx8 — 189 segments, ~2300 words, lang=en

---

Going Further

Once your basic pipeline is running, a few natural next steps:

Bulk endpoint for throughput: instead of individual GET requests per video, use POST /api/v2/transcripts-bulk with up to 50 video IDs per call. This slashes round-trips dramatically when processing a full YouTube channel.

Channel and playlist data: the API exposes endpoints to resolve channel handles, list channel uploads, and retrieve playlist videos. You can build a full ingestion pipeline—discover videos, then feed their IDs into your transcript worker pool—without touching the YouTube Data API v3 at all. If you're curious how the two compare, YouTube Data API v3 vs a Dedicated Transcript API covers the trade-offs.

Feeding transcripts into an LLM: once you have clean JSON segments, concatenate the text fields and pass them to an embedding model or summarization prompt. The segments already come timestamped, which is useful for building chapter markers or citation-linked summaries. For dataset-building at scale, Building Transcript Datasets for LLM Fine-Tuning walks through the full pipeline.

Context propagation: for production services, thread a context.Context through every call so you can cancel in-flight requests on shutdown or deadline. Replace http.NewRequest with http.NewRequestWithContext(ctx, ...) and cancel the context when your service receives a SIGTERM.

---

Wrapping Up

You now have a production-ready foundation for fetching YouTube transcripts in Go:

• A configured http.Client with sane timeouts and connection pooling
• Typed structs that give you compile-time safety on the JSON response
• A goroutine worker pool that keeps concurrency bounded
• A token-bucket rate limiter built from a plain time.Ticker
• Exponential-backoff retries using idiomatic errors.As unwrapping

The YouTube Transcriber API does the heavy lifting on YouTube's side—IP rotation, proxy management, anti-bot challenges—so your Go code stays focused on what Go is good at: clean concurrency and fast JSON processing.

Start with 100 free credits at getyoutubetranscriber.com and check the API docs for the full endpoint reference, including bulk transcripts, channel uploads, and playlist support.