Conversational AI
AI Analyst delivers fast key figures and reliable insights
Conversational AI makes data more accessible than ever before. Instead of waiting for reports or searching through tables, a simple question in the chat is enough today. The AI Analyst shows how this results in faster knowledge and better decisions.
Imagine: It is Monday morning and your boss is asking about current key figures, such as about a location, a product line or a customer group and its development. What sounds like a routine question often involves a complex process. First a request to the data team, then the merging of data from different sources. It can take hours or even days to get the answer.
.jpg)
With Conversational AI, everyday life is already noticeably changing for many employees today. In Microsoft Teams, they can write to the AI Analyst, who gives a precise answer within a few seconds. No waiting, no Excel spreadsheets, no detours from other departments.
The digital colleague in the chat
Behind the AI Analyst is a chatbot that makes use of the concept of Retrieval Augmented Generation (RAG). User requests are translated into an SQL query and sent to the AGK data warehouse. The data resulting from the query is then returned to the user in Microsoft Teams in natural language. For users, this creates an intuitive system that works without much explanation. Behind the scenes, a well-thought-out technical architecture ensures that this process runs seamlessly and reliably.
Retrieval Augmented Generation
Retrieval Augmented Generation is an approach in which an AI model is not only based on its pre-trained knowledge, but also retrieves relevant information from external data sources before generating an answer. This allows AI to incorporate current, contextual knowledge that goes beyond its original training knowledge. This combination of information retrieval and text generation increases both the accuracy and traceability of the results. RAG is primarily used where precise, context-rich answers are required, such as in chatbots, knowledge databases or analysis tools.
How the AI Analyst differs from classic chatbots
In contrast to many chatbots that we know from everyday life, the AI Analyst not only provides answers, but also works in a much more structured, consistent and reliable way. One important difference is his deterministic approach: When several people ask the same question, they also get the same answer, unlike what is the case with ChatGPT, for example. This creates trust and makes the results comprehensible — a key requirement for professional use.
To ensure that the analysis process remains verifiable at all times, the AI Analyst works with so-called “structured outputs.” This means that each processing step follows a fixed format and can be individually analyzed and improved as needed. This results in comprehensible and stable results instead of chatbot answers that are difficult to classify.
This creates trust and makes the results comprehensible.
When creating SQL queries, the AI Analyst also goes beyond classic chatbots. With the help of so-called “few shots” — saved sample queries including suitable SQL queries — the model always has three relevant examples as a guide. It also takes company-specific vocabulary and synonyms into account so that it correctly understands and implements technical terms precisely.
The AI Analyst is therefore not just an interlocutor, but a reliable analysis tool that delivers consistent results and can be specifically adapted to the requirements of a company.
Core components of the system architecture
The AI Analyst architecture is based on three central components. Snowflake serves as a structured and validated database that stores all relevant company data and feedback on the quality of answers. In addition, the sample few-shot tables are materialized in Snowflake and are made available to the bot via Snowflake's Cortex Search to expand the context. Cortex Search uses hybrid search to identify semantically and lexically similar questions. The selection of Snowflake follows the existing infrastructure and is transferable to other platforms.
Microsoft Azure forms the technical backbone, offers scalability, GDPR compliance and access to the latest OpenAI models. By using Azure OpenAI EU Data Zones, all data is guaranteed to remain within the EU. In addition, integration with Microsoft Entra ID enables granular access control that controls exactly which user can access which data.
As a third component, FastAPI adopts the interface logic and ensures almost real-time communication between the Microsoft Teams frontend and the backend of the bot.
From question to answer
The processing process of a user question is divided into a total of nine steps. First, the question is entered in the Microsoft Teams chat. An intent classification is then carried out using an LLM to determine whether a database query is required, additional information is required, or the answer can be derived directly from the business context.
This is followed by a use case classification, which identifies relevant data areas. Based on the data marts structured in Snowflake, it is determined in which tables to search for the answer.
The next step is to use Snowflake Cortex Search to identify similar questions and suitable SQL examples (few-shot query), which serve as a guide for the AI. This is followed by context enrichment, during which all relevant information is compiled at runtime and made available to the LLM. This includes the business context, table and column descriptions of the identified use case, sample values, and previously found few-shots.
The entire process takes just a few seconds and replaces waiting times of days or weeks.
On this basis, the LLM generates the SQL query, which is then validated. It checks whether it is syntactically correct and whether all referenced tables and columns actually exist. Up to five automatic correction loops ensure that errors are identified and corrected.
After a successful check, the query is executed in Snowflake and the raw data obtained is processed accordingly. Finally, the LLM provides answers by summarizing the data in understandable language and outputting it directly in the teams chat, optionally supplemented by an Excel export.
The AI analyst in live operation
The rollout of the AI Analyst is typically carried out in staggered phases. First, the system is tested with a focused user group before it is gradually extended to all employees. This iterative introduction enables controlled scaling and early identification of optimization potential.
The modular architecture of the system ensures continuous development in parallel with company changes. New use cases can be seamlessly integrated by simply connecting additional data marts, which allows the bot to react flexibly to changing business requirements.
The modular architecture of the system ensures continuous development.
Systematic feedback management is a key success factor. During use, users can provide feedback on the quality of the answer directly in the chat using buttons and free text. User feedback is analyzed via a dashboard, which enables both bug fixes and the extension of the few-shot examples.
This continuous optimization results in the successive coverage of edge cases and a continuous improvement in the system's response quality.
The future of data-driven collaboration
The introduction of AI Analyst is representative of a development that goes far beyond a single project. The market for conversational AI is growing rapidly worldwide. IDC predicts a volume of over 31 billion US dollars by 2028, and generative AI can create annual economic value of up to four trillion US dollars, according to McKinsey.
But beyond such figures, it is primarily about a fundamental change in the handling of data: away from centralized teams of experts towards a way of working in which every employee receives reliable answers to business-relevant questions directly in the work context.
Decisions are no longer made solely on the basis of intuition or experience, but well-founded and data-based — without detours via specialized teams. A question about current figures, followed by a time-consuming process, becomes a natural, everyday interaction.
Companies that introduce such solutions today are not only creating more efficient processes, but are also changing their culture. They make data an integral part of every decision and give their teams new scope for action — anew every Monday morning.
Would you like to discover the opportunities of an AI Analyst in your company?
