Can an AI agent actually answer questions about your business?

Can an AI agent actually answer questions about your business? Yes, reliably, if three conditions are true: it has clean access to the data and documents, it sits on top of a proper semantic layer, and it grounds every answer in cited sources. If any of the three is missing, what you'll ship is a demo that fools a board meeting and confuses everyone who tries to use it for real work the following Monday.

The "talk to your data" pitch is everywhere in 2026. Every SaaS vendor has a chat panel. Every consultancy has an AI demo. The hard part isn't getting a chat box to produce an answer. The hard part is getting it to produce the right answer 95% of the time on your actual, messy, inconsistent, hundred-SKU, seven-system business data. That's an order of magnitude harder than the demos suggest, and it's where the real engineering work lives.

What people mean when they say "Ask Your Business"

Three different things, all marketed under the same name.

1. Chat on top of a wiki. The AI reads your Notion or Confluence or SharePoint and answers questions about policies, processes, and team knowledge. Useful. Cheap. Not the same product as analytics chat, and should not be sold as if it is.

2. Chat on top of a data warehouse. The AI reads your tables and answers questions like "what were last month's sales by region" or "which customers are trending down on usage." Much harder. Requires a semantic layer, strong retrieval, and careful grounding.

3. Chat on top of everything. Combined: wiki, warehouse, email threads, CRM, ticketing system. A unified agent that can answer "why did sales dip in QLD in March" by pulling from both the warehouse (the sales number) and the CRM (the two lost deals that explain it). Expensive. Powerful. The honest goal for most businesses.

Most of the AI demos being shown in 2026 are version 1 with a data dashboard attached, sold as version 3. Know which one you're buying.

The three conditions that make it work

Condition 1: clean access. The agent needs to be able to reach the data. Proper SSO. Row-level security so that an executive sees executive data and a regional manager sees their region. Service accounts with correctly scoped permissions. Vector storage for the document corpus. This sounds like table stakes and it's where most projects spend the first four weeks.

Condition 2: a semantic layer. This is what separates "AI that reads a schema" from "AI that understands the business." When a rep asks "how is the eastern territory doing" the agent needs to know that "eastern territory" is a specific set of postcodes, that "doing" means pipeline, closed deals, and quota attainment, and that the measure of "how" should be compared to the same metric for the last period. Without a semantic layer, the agent guesses, and it guesses differently each time.

Condition 3: grounding and citation. Every answer needs to cite its source. "Sales in QLD in March were $1.2m, down 8% from February. Source: warehouse.revenue_monthly view. Confidence: high." Users trust cited answers and catch wrong ones; they don't catch confident hallucinations. Grounding is the difference between a tool that gets used daily and a tool that gets abandoned after the third wrong answer.

An ungrounded AI agent will produce one confident wrong answer in the first week of use and lose the trust of every user who hears about it. Citation and confidence scoring aren't nice-to-haves. They're the single design decision that determines whether the project ships value in year one.

What the failure looks like

The most common shipping failure we see looks like this. The project runs for six months. It produces a chat interface that works well on a curated set of demo questions. Business users get access. They ask their actual questions. They get plausible-sounding wrong answers. They report it to IT. IT escalates to the vendor. The vendor says "it's working as designed, you need to rephrase the question." Users stop using the system within a quarter.

The underlying cause is almost always one of three things:

• The retrieval layer is too broad. The agent pulls irrelevant context and synthesises noise into confident-sounding output.
• The semantic layer is incomplete or missing. Business terms map to guesses rather than defined queries.
• The evaluation loop doesn't exist. Nobody's grading the agent's answers against a test set of real questions with known right answers, so the accuracy drift goes unnoticed until users revolt.

The evaluation test set

The single most underweighted part of these projects is the evaluation test set. Before you ship, before you involve users, you need 200 to 500 real questions with known correct answers, covering the range of intents users will actually ask. Every model change, every prompt change, every retrieval change gets tested against the set. The accuracy number is the KPI, not "demo feedback" or "user satisfaction."

Businesses that skip this step end up in a loop where every change feels like an improvement until three weeks later it's clear the changes weren't improvements and accuracy has drifted in a new direction. With a test set, you see the drift the day it happens.

What it takes to do properly

For a mid-market Australian business with reasonable data infrastructure:

• Basic chat-on-wiki implementation (version 1 above): 6-10 weeks.
• Chat-on-warehouse with a real semantic layer (version 2): 10-16 weeks. Plus the semantic layer work itself, which is typically the majority of the engagement and often existed in a half-built form already.
• Full Ask Your Business combining multiple sources (version 3): 16-26 weeks. The bulk of the work is integration, governance, and evaluation, not the LLM piece.

If a vendor offers version 3 in a four-figure quote, they're quoting version 1 and hoping you won't notice. If a vendor offers a POC in two weeks, the POC will work on demo questions and fail on yours. Ask to see the test set. If there isn't one, push back.

The two buying rules

1. Ask to see accuracy on the vendor's own evaluation test set. Not demo questions. A real, held-out set of business questions with known answers. Accuracy under 90% at this stage is a warning sign. Under 80% and you're buying a demo.
2. Ask who owns the semantic layer after go-live. If the answer is "the vendor" or "nobody," the project will ossify. Your team needs to own the business-term definitions and be able to extend them without a vendor ticket.

The businesses that get this right treat Ask Your Business as a semantic-layer project with a chat interface on top. The businesses that get it wrong treat it as a chatbot project that happens to use some data. The order matters. Get it backwards and you ship a demo that impresses everyone until the third real question.