We developed a sophisticated AI agent designed to autonomously manage and resolve customer support tickets. Our primary goal was to create a system that not only provides instant, accurate support but also intelligently identifies when human intervention is necessary, thereby avoiding common AI-driven customer frustrations and improving the overall support experience.
The Problem: Scaling Support Without Sacrificing Quality
For many established B2C companies, maintaining rapid, high-quality customer support is a significant challenge, especially in competitive markets with tight margins. The situation becomes more complex when supporting a multilingual customer base, as was the case with our client who needed to provide support in both Estonian and English. The core issue was a classic business bottleneck: as the customer base grew, the support team couldn’t keep up with the volume of tickets, leading to slower response times.
However, standard AI support solutions often create more problems than they solve. They are notorious for trapping customers in frustrating loops, failing to understand the actual issue, and making it difficult to ever reach a human agent. We needed to build something better.
The Solution: A Multi-Layered, Autonomous AI Agent
Our solution is an AI agent that integrates directly with the existing ticketing system. It’s an autonomous system that constantly monitors its own ability to resolve an issue. The agent uses a multi-layered process to understand, troubleshoot, and compose solutions. If at any point the agent itself, or its internal Validator, determines it cannot fully resolve the ticket, it instantly escalates it to the appropriate human team. This escalation includes a clear, automated note that summarizes its findings and explains why the ticket is being escalated to that specific team, ensuring a seamless handover.
How It Works: System Architecture
The agent’s architecture is built around a clear, logical flow designed for accuracy and efficiency.
1. Ticketing System Integration & Initial Filtering: When a New Message arrives, it first passes through a series of pre-processing filters before ever reaching the core AI. This prevents the agent from wasting resources on irrelevant tickets.
- Client Recognition & Data Scoping: The system first checks if the ticket is from a known, authenticated client. This initial authentication is a critical security step. It’s required so we can ground the AI agent, strictly limiting its data access to only this particular client’s information (e.g., their support history, profile, and invoices). This design eliminates any possibility of accidental data leaks between different clients and ensures the agent operates with the correct context.
- Spam Detection: It then runs a spam check. If a ticket is identified as spam, it’s marked and archived accordingly.
- Response Needed? Finally, it determines if the ticket actually requires a response. For example, a simple “thank you” from a customer doesn’t need a substantive reply and can be archived.
2. The AI Agent Core: Once a ticket passes the initial filters, it enters the core of the system, where a multi-stage process begins to resolve the ticket. First, the main AI Agent analyses the ticket to understand the user’s problem. To gather context, it can leverage a collection of specialized Tools and Subagents.
The agent can use a variety of tools to access information, such as:
- Support Tickets History: To review past interactions and context.
- Client Profile: To understand the client’s status and account details.
- Client Payments: To review the client’s payment history.
- Client Invoices: To retrieve specific billing documents.
- Knowledge Base: To find standard procedures and answers.
- Navigation Instructions: To map out correct paths to different places in the client’s systems, which helps avoid AI-hallucinated navigational instructions.
- System Status: To check for ongoing outages or service disruptions.
- Company Website: For public-facing information.
- Internet Search: For general queries that aren’t client-specific.
Importantly, any tool that handles personal or account-specific data is strictly scoped to the authenticated client, ensuring privacy. When the AI agent detects that there is a technical issue that requires a deeper investigation, it delegates the specific technical parts of the investigation to a dedicated Debugging Subagent. The main agent can call upon this subagent and its specialized tools as many times as needed to gather the necessary information to resolve the ticket. This subagent has its own advanced tools, allowing it to read error logs, check system configurations, and perform network lookups to diagnose the root cause of a problem. This entire framework is designed to be extendable, allowing us to easily add new tools and subagents as needed.
After the agent has used these resources to formulate a solution, the response is not sent immediately. It is first passed to a Validator, another AI model that acts as a critical “second opinion.” The Validator’s role is multifaceted: it compares the proposed solution against the original ticket to ensure the AI is addressing the root problem the client actually had, preventing cases where the agent loses track and solves a problem it raised itself. It also double-checks any facts presented in the response—such as invoice numbers, payment details, service pricing, or other specific information—to verify that they are true. If it finds the solution lacking, it’s sent back to the main agent for revision. This validation loop is key to ensuring quality. Only after a solution is approved does it move to the final stage: the Composer. This module formats the validated information into a clear and helpful reply, following the company’s tone of voice and writing in the original language of the customer’s ticket. The final message is then sent to the customer.
3. Intelligent Escalation Detection: Throughout the process, the system is constantly watching for signs that a ticket is too complex or requires a human touch. The Escalation Detection module analyses the ticket’s content and the AI’s own confidence level.
- If an escalation is needed, the system doesn’t just flag the ticket; it intelligently routes it. It determines who the right person to handle the issue is.
- For example, if the problem relates to billing, it will Assign & Message the Billing Team. If it’s a deep technical failure the agent can’t solve, it will Assign & Message the Technical Team. This ensures that when a human does get involved, it’s the right human, and they have all the context from the AI’s initial investigation.
Backend Systems and Data Integration
To keep the AI agent supplied with accurate, up-to-date information, we implemented several automated backend processes:
- Knowledge Base Integration: The Knowledge Base tool connects directly to the company’s existing knowledge base system. Every night, a process runs that reads all new or changed entries, converts the text into vector embeddings, and stores them in our database with a link back to the original source article. This ensures the agent always has the latest procedural information.
- Website Crawling: Similarly, we crawl the client’s public website every night to detect changes. This new and updated information is also converted to embeddings and stored, allowing the agent to answer questions about public-facing content accurately.
- Navigation Mapping: Once a week, a dedicated crawler maps out the client’s websites and private portals. By using a dedicated, authenticated account, it can access areas that require a login, building a comprehensive navigation graph. This data is what powers the Navigation Instructions tool, ensuring the agent provides accurate guidance to users.
Technology Stack
This system is built using a combination of best-in-class models and technologies to ensure robustness and performance. In total, the agent orchestration uses four different AI models from Anthropic, OpenAI, and Google, each chosen for its strength in a specific task like validation, composition, or core reasoning.
Our technology stack includes: Python, PHP, Node.js, PostgreSQL, Supabase, LangChain, LangSmith, Pydantic AI, and Tavily.
Key Results & Business Impact
This system moved beyond a simple chatbot to become a genuine AI team member. It successfully filtered out noise, autonomously handled a significant percentage of common support requests, and performed initial diagnostics on complex issues before escalating them. This freed up the human support team to focus on the high-value, complex problems that truly required their expertise, dramatically reducing response times and improving customer satisfaction.
