Two Ways to Use LLMs in Your Product

Everyone is asking the same question right now: where do I add LLMs? The answers you find are usually too vague to act on (“look for repetitive tasks!”) or too specific to someone else’s product to transfer. What’s missing is a diagnostic — a way to look at your backlog and recognize the types of problems that LLMs actually solve.

There are two patterns: I’ll call them the translation layer and the personal assistant at scale, and I’ll use the code smell metaphor to describe how to recognize them. If you don’t see either smell in your product, you probably don’t have a problem LLMs solve — and that’s a legitimate conclusion, not a failure. The absence of a diagnosis is the answer. Don’t shoehorn!

The Translation Layer

This is the defensive case. You are taking fuzzy, unpredictable human input and sending it downstream to a system that has deterministic requirements. The gap between them is where things break.

The smell: your engineers are writing increasingly complicated validation logic, and it’s never enough. Every new partner, channel, or edge case adds another band-aid. The escape valve is pushing the problem back to the user — “please rewrite this to meet the requirements” — which is friction nobody wants.

An LLM doesn’t eliminate the constraints. It bridges the gap between human input and system requirements, reliably and at scale, in a way that enumerated rules never can.

Two examples from real estate tech: adapting verbose listing descriptions to channel-specific ad copy without going back to the agent to rewrite; and mapping each partner’s listing data schema to an internal standard without writing a custom ETL per partner.

That second one is worth pausing on. The LLM generates the schema mapping once. Code interprets the mapping to process every row. The LLM is not in the hot path — it runs at configuration time, not at row-processing time. This matters because a common misconception about LLM integration is that it means an API call per user action. It doesn’t have to.

The Personal Assistant at Scale

This is the offensive case. There’s a feature you’ve wanted to build for years, but it required a human in the loop for every user, and the unit economics were never there. It’s a great feature, and you know it would delight users, but it’s always been out of reach. You mentally filed it under “nice to have if we ever had the headcount.”

That’s the smell: not a system that’s breaking, but a capability that was always out of reach.

Animating listing photos into short video ads is a clear example. Previously, every listing needed a human video editor. Now the model handles it at scale — with the added constraint that it must not hallucinate features that don’t exist in the source images (adding a window, inventing a garden). The personal assistant analogy holds precisely here: a good assistant uses judgment and stays in bounds; a bad one goes off-script.

The same pattern appears outside real estate. Code review assistants fall squarely in this bucket — every engineering team has wished, at some point, that a senior engineer could review every PR. They can’t staff that. An LLM can approximate it at scale. Personalized onboarding works the same way: you always wanted a customer success rep to walk every new user through setup; you never had the headcount to do it.

Why Most Teams Get This Wrong

Most teams skip the diagnostic entirely. There’s pressure to ship AI features — from leadership, from competitors, from the general ambient noise of the moment. So they reach for LLMs before they’ve identified the smell, and the result is integrations that feel decorative: a chatbot that adds no value, a “smart” feature that users ignore, an AI wrapper around something that didn’t need one.

The framework isn’t complicated. But it requires the discipline to ask “which smell do I have?” before asking “how do I add AI?” — and the willingness to accept that the answer might be neither.