Planograms—those deceptively simple, blocky diagrams that describe how the grocery store stocker should shelve cans of soup and buckets of detergent—are built on huge stacks of broad market research, product and sales data, category insights, visual rules, sizing and weight constraints, and contractual requirements. Generating them requires weeks of multiple marketing pros juggling constraints, regional or even store-specific needs or sales, and the art of eye-catching product placement. But what if we could automate the knowledge interpretation step and create a path toward fully multimodal planogram generation, leaving only the fun, artistic part to the marketing pros?
In this talk, we’ll walk through how I built a proof-of-concept planogram generator using AWS services, open-source models and tooling, and agentic LLM workflows. I'll demonstrate how I used a knowledge base of merchandising research and category rules, planogram best practices, and AWS services to generate a structured, human-readable planogram description—complete with constraints, product groupings, spacing logic, and rationale. We’ll explore a working architecture and how it ingests and indexes data, how prompt engineering ensures accurate interpretation of marketing guidance, and how agents orchestrate multi-step tasks like rule validation and layout adjustments.
We'll also dive into the open-source layer: the FOSS libraries that make the pipeline portable and how an agentic workflow can be built around OSS components to avoid vendor lock-in. And we'll look at the next step in the project: extending the system with multimodal output, an auto-generated diagram using agnostic tools that keep options open for later iterations.
Along the way, we’ll discuss what LLMs do well and where they fall short, with concrete techniques for prompt engineering, automated reasoning, and guardrails that make the outputs more trustworthy. Attendees will leave with architectural patterns and access to a working demo they can use to inform their own agent-driven automation projects—whether they’re in retail tech, logistics, documentation, or operations.
Whether you’re curious about real-world LLM workflows, want to see what the cloud can offer in FOSS AI/ML workflows, or just want ideas for building smarter automations in the cloud, this talk will show you how a single idea—“create a planogram for a bay of energy drinks and programmer kibble”—became a system that blends open-source flexibility, cloud resilience, and the power of modern agentic AI.
