Turn Any YouTube Channel into a RAG Index (with Timestamped Citations)
Video is one of the richest knowledge sources there is, and one of the hardest to search. Lectures, conference talks, product reviews and podcasts all sit behind a play button. In this post we'll turn a YouTube video, p
Video is one of the richest knowledge sources there is, and one of the hardest to search. Lectures, conference talks, product reviews and podcasts all sit behind a play button.
In this post we'll turn a YouTube video, playlist or whole channel into a vector index you can query, where every answer links back to the exact second in the video it came from.
Here's the result we're building. I asked "how does self-attention work?" against Andrej Karpathy's 2-hour Let's build GPT lecture:
https://www.youtube.com/watch?v=kCc8FmEb1nY&t=4299s -> ...attention number one attention is a communication mechanism...
https://www.youtube.com/watch?v=kCc8FmEb1nY&t=4490s -> ...you will use an encoder block of self attention...
https://www.youtube.com/watch?v=kCc8FmEb1nY&t=4561s -> ...the same Source X produces Keys queries and values so these nodes are self attending...
Each link opens the video at the moment the answer starts.
Step 1: get the transcripts
You have two good options.
Option A: youtube-transcript-api (free, open source)
For a handful of videos on your own machine, the youtube-transcript-api package is great:
# pip install youtube-transcript-api (tested with 1.2.4)
from youtube_transcript_api import YouTubeTranscriptApi
transcript = YouTubeTranscriptApi().fetch("kCc8FmEb1nY")
for s in transcript.snippets[:3]:
print(s.start, s.text)
It gives you raw caption lines. Two things to know before you build a pipeline on it:
- Cloud IPs get blocked. YouTube often refuses requests from AWS, GCP and similar servers, so code that works on your laptop can fail in production unless you add proxies.
- You do the rest yourself: expanding channels and playlists into video IDs, picking a language, and chunking with timestamps.
Option B: a hosted transcript API that returns RAG-ready chunks
Disclosure: I built this one. YouTube Transcripts is an Apify Actor that takes videos, playlists or channels and returns, per video, the transcript plus overlapping, timestamped chunks with metadata. Proxies and retries are handled for you, and videos without captions cost nothing.
# pip install apify-client (tested with 3.3.0)
from apify_client import ApifyClient
client = ApifyClient("YOUR_APIFY_TOKEN") # https://console.apify.com/settings/integrations
run = client.actor("t3data/youtube-transcripts").call(run_input={
"urls": ["https://www.youtube.com/watch?v=kCc8FmEb1nY"], # or a playlist, or "https://www.youtube.com/@channel"
"includeChunks": True,
"chunkMaxWords": 250, # ~330 tokens per chunk
"chunkOverlapWords": 30, # keeps sentences from being cut in half
"includeSegments": False,
})
videos = list(client.dataset(run.default_dataset_id).iterate_items())
Each chunk looks like this (real output):
{
"chunkId": "kCc8FmEb1nY-3",
"text": "very serious production grade system it is trained on uh a good chunk of internet and then there's a lot of uh pre-train...",
"startSeconds": 201.36,
"url": "https://www.youtube.com/watch?v=kCc8FmEb1nY&t=201s",
"videoId": "kCc8FmEb1nY",
"title": "Let's build GPT: from scratch, in code, spelled out.",
"channelName": "Andrej Karpathy"
}
The 2-hour lecture becomes 99 chunks. A channel works the same way: pass its URL and set maxVideosPerPlaylist.
Step 2: embed the chunks
We'll use Chroma because it runs locally and ships with a default embedding model, so there's no API key and nothing to configure:
# pip install chromadb (tested with 1.5.9)
import chromadb
db = chromadb.PersistentClient(path="./yt_index")
col = db.get_or_create_collection("youtube")
for video in videos:
if video["status"] != "ok": # e.g. a video without captions
continue
for c in video["chunks"]:
col.upsert(
ids=[c["chunkId"]],
documents=[c["text"]],
metadatas=[{"title": c["title"], "channel": c["channelName"],
"url": c["url"], "start": c["startSeconds"]}],
)
Using chunkId as the ID makes re-runs idempotent, so you can re-index a channel every week and only new videos are added.
Step 3: ask questions, get timestamped sources
res = col.query(query_texts=["how does self-attention work?"], n_results=3)
for doc, meta in zip(res["documents"][0], res["metadatas"][0]):
print(f'{meta["url"]} -> {doc[:90]}...')
That's the output at the top of this post. To get a written answer, pass the retrieved chunks to any LLM and ask it to cite the url of each chunk it used. Your users can then click straight to the moment in the video.
Tips from building this
- Chunk size: 200–400 words works well for most embedding models. Use a smaller size with more overlap for dense lectures and a larger one for chatty podcasts.
-
Auto-generated captions are good enough for retrieval. They have no punctuation, but embeddings don't mind. Check
isAutoGeneratedif you want to prefer human captions. -
Languages:
"languages": ["es", "en"]is a preference order.availableLanguagestells you what each video has. -
Keep it fresh: schedule the run weekly on a channel URL with a small
maxVideosPerPlaylist, and new uploads flow into the same index. - Cost: the hosted version is $5 per 1,000 transcripts, and videos without captions are free. It's paid per run on Apify, which has a free plan with monthly credit. For a few videos, Option A is free.
Full script
from apify_client import ApifyClient
import chromadb
client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("t3data/youtube-transcripts").call(run_input={
"urls": ["https://www.youtube.com/@AndrejKarpathy"],
"maxVideosPerPlaylist": 10,
"includeChunks": True, "chunkMaxWords": 250, "chunkOverlapWords": 30, "includeSegments": False,
})
col = chromadb.PersistentClient(path="./yt_index").get_or_create_collection("youtube")
for video in client.dataset(run.default_dataset_id).iterate_items():
for c in video.get("chunks", []):
col.upsert(ids=[c["chunkId"]], documents=[c["text"]],
metadatas=[{"title": c["title"], "url": c["url"], "start": c["startSeconds"]}])
res = col.query(query_texts=["how does self-attention work?"], n_results=3)
for doc, meta in zip(res["documents"][0], res["metadatas"][0]):
print(meta["url"], "->", doc[:90])
If you build something with this, I'd love to hear what you indexed. And if you hit a video that doesn't work, open an issue on the Actor page and I'll fix it.
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.