The 5 Types of LLM Apps

LLM apps can be sorted into five practical categories—ranging from chat-style assistants to fully autonomous agents—so builders can more clearly match product design, data strategy, and defensibility to the kind of outcome they’re targeting. The biggest shift behind all five is that modern large language models can handle open-ended conversation and reasoning, making capabilities that were previously limited to narrow, rules-based systems suddenly usable at scale.

The first category is conversational chatbots and agents, which turn back-and-forth dialogue into a product feature. These range from virtual friends and dating partners to customer support, appointment setting, and sales outreach. Earlier attempts often worked only in closed domains and relied on rigid heuristics; the change came with widely capable models such as ChatGPT and Google’s Lambda (described as a Google internal model), which made coherent, general-domain conversation feel interactive. For builders, the key design questions are what personality the bot should have, what it should and shouldn’t discuss, what “success” looks like for each conversation, and whether it needs memory—especially for long-running relationships where follow-ups depend on storing prior user context.

The second category is Copilots and “duets,” inspired by Microsoft and Google’s framing of assistants that help users accomplish goals faster. Two subtypes dominate. One embeds into existing software and SAS tools, using company-held customer data (analytics, knowledge, usage patterns) to guide users through specific, bounded tasks. This model tends to favor large incumbents because they already possess the data needed to personalize guidance and build interactive discovery flows. The other subtype focuses on education and learning: curating exercises and guidance as users progress through a topic. Khan Academy’s Khanmigo is cited as an example that adapts to a learner’s level, and the transcript suggests startups have room here because learning journeys can be guided without requiring the same depth of proprietary customer behavior data.

The third category is “chat with data” apps built on retrieval augmented generation (RAG). These let users ask questions in natural language while a retrieval layer pulls relevant information from sources like PDFs or a SQL database. The promise is quick, conversational access to large internal or document-based datasets. But defensibility is hard: simple RAG wrappers can be replicated quickly, and higher-quality retrieval augmentation requires careful, topic-specific customization. The transcript points to a market lesson—chat-with-PDF apps lost momentum after GPTs made similar functionality broadly accessible.

The fourth category moves beyond user-facing chat into traditional NLP tasks performed in the background. Named entity recognition, sentiment analysis, coreference resolution (tracking what “it/they/he” refers to), and data extraction are increasingly handled well by large language models, often outperforming specialized models—especially when fine-tuned.

The fifth and most ambitious category is autonomous agents. Instead of merely answering, these systems autonomously execute multi-step tasks using reasoning and decision-making, often built from modular “mini agents” rather than one all-powerful unit. A notable technique is self-critique and recursive improvement: another agent (using the same model) reviews work and catches weaknesses, producing better results than a single pass. Production remains challenging because strong reasoning/decision-making often depends on state-of-the-art models (the transcript cites latest Gemini and GPT-4-class models) or heavy fine-tuning. Still, it’s framed as the area likely to see the fastest growth over the next year or two.

Overall, the five categories—conversational agents, copilots/duets, chat-with-data RAG, NLP task automation, and autonomous agents—provide a roadmap for aligning product goals, data inputs, and competitive moats with the right LLM application pattern.