The AI Concepts Podcast

This episode maps out the full RAG pipeline end to end using one concrete scenario, a defense contractor building an AI assistant for fighter jet maintenance crews. It walks through both phases of the architecture, offline and online, following a real question all the way from a raw document to a grounded answer. It also covers why the architecture is modular and closes with the four failure modes that quietly break RAG systems in production.

This episode kicks off Module 6 with RAG (Retrieval Augmented Generation), the #1 architecture every serious enterprise actually uses. Discover why regular LLMs hallucinate on your private data and high-stakes queries, and how RAG fixes it by forcing the model to retrieve real documents first.
 

This episode covers reasoning models, the shift from manually guiding a model's thinking to letting the model reason through complex problems on its own before responding. It explains the concept of test-time compute, why reasoning models take longer but perform dramatically better on hard tasks, and how they change the way you should prompt. It walks through when to reach for a reasoning model versus a standard one, and closes by framing the full prompt engineering toolkit in context, from few-shot examples through reasoning models.

This episode covers structured output, how you get a model to respond in predictable, machine-readable formats like JSON instead of natural language paragraphs. It walks through three approaches, from simply asking in the prompt, to JSON mode, to schema-based constraints, and explains why each level adds more reliability. It uses real-world examples to show how structured output turns AI from a conversation partner into a software component that can feed databases, trigger workflows, and drive automation. It closes with practical tips for writing schemas and validating output in production.

This episode covers system prompts, the invisible instruction layer that shapes every model interaction before the user says a word. It explains the three-role message format, why the model is trained to treat system instructions as higher authority than user messages, and how persona prompting works by shifting which region of the training distribution the model samples from. It walks through the anatomy of a good system prompt and closes with what happens when system and user instructions conflict, including a preview of the prompt injection problem.

This episode covers chain of thought prompting, how asking a model to show its reasoning makes it measurably better at complex tasks, and why that works at a mechanical level. It walks through manual and zero-shot chain of thought, then three advanced extensions: self-consistency, Tree of Thought, and step-back prompting. It closes with when chain of thought actually helps versus when it just adds overhead.

This episode explores in-context learning, the idea that you can dramatically change how a model behaves just by showing it examples inside the prompt, without changing a single weight. It walks through zero-shot, one-shot, and few-shot prompting, when each one tends to work best, and why examples shape not just the answer but also the format, tone, and structure of the response. It also gets into some of the more surprising research around this, including how models can still perform well even when example labels are wrong, why example order can materially affect accuracy, and why one strong example can sometimes outperform several mediocre ones. The episode closes by framing few-shot prompting as one of the most practical and powerful skills in prompt engineering, while also pointing to the limits of prompting when a task becomes too complex.

This episode explores the hidden layer between your prompt and the model’s response: decoding and sampling. We look at how the model moves from a field of possible next tokens to the one it actually chooses, why the same prompt can produce different outputs, and how that variation is shaped rather than random. We walk through the core strategies you will hear over and over in prompt engineering, from greedy decoding to temperature, top-k, and top-p, and the tradeoff each one creates between precision, consistency, creativity, and control. We also touch on why these settings matter differently depending on the task, and why newer reasoning models do not always play by the same rules.

In this episode, I explore a question I heard recently that sounds simple, but matters more than it seems.

This episode tackles the lever that turns powerful LLMs into something you can actually run: quantization. We explore what it means to store model weights with fewer bits, why that can cut memory in half at 8-bit and down to roughly a quarter at 4-bit, and the real tradeoff between compression and capability as rounding error accumulates across billions of parameters. We break down why large models survive this better than small ones, why 8-bit is often near lossless, why 4-bit can still be shockingly strong, and why going below that can make models fall apart. We compare the three practical paths you will see in the wild: GPTQ (layer-wise compression with error compensation), AWQ (protecting the most important weights), and GGUF (the local-friendly format that makes CPU and GPU splitting possible).