How We Built An Ask-Me-Anything Agent with KNIME

Agentic AI is emerging as a new way for organizations to access and use their data. Instead of teams manually searching through documents, databases, or internal tools, agentic applications can answer questions directly by retrieving information, processing questions, and generating useful responses — all while adapting to company-specific needs.

An Ask-Me-Anything (AMA) agent is an application that responds to internal business questions by selecting the right tools and data sources behind the scenes. In this article, we show how we built one in KNIME. You can type questions into a chat interface, and the agent retrieves and structures answers by pulling data from multiple internal systems.

For example, you might ask: Do we have customers in Prague? What’s our history with Acme Corporation? Which customers have attended more than three webinars?

Our Ask Me Anything agent already handles a wide range of internal questions about customers, the community, KNIME users, and employees. As it gains access to more data and its logic improves, it becomes increasingly effective at retrieving and structuring relevant information.

This article breaks down how KNIME makes this possible and how you can build an agent like this for your own internal data.

We’ll cover:

• The building blocks of an AI Agent
• Building an Ask Me Anything agent
• How this works in practice

Unlike static search engines, agentic AI applications analyze data, choose relevant sources, and generate clear responses. What makes it more than just automation is how the agent decides what to use and when. 

It achieves this by using:

• Tools – Perform specific tasks like retrieving CRM data, summarizing text, or translating content. Some use GenAI for advanced capabilities.
• Workflows – Chain multiple tools together to complete more complex tasks. For example, a workflow might: retrieve customer data using a data aggregation tool, pass it to a summarization tool, translate the summary using a language tool, and finally send it via email.
• Agents – Choose tools and workflows dynamically, adapting to each request rather than following a fixed sequence. Agents can be interacted with directly by users (in agentic applications) or can operate in the background, being called REST services (agentic services).

This structured approach is central to how agentic AI operates within KNIME. For a broader overview, check out this article, Agentic AI and KNIME. 

The agent is deployed as a KNIME Data App, a convenient, browser-based interface where users can type questions and get answers immediately. Here’s what it looks like:

Behind the interface is a KNIME workflow that pulls everything together. Here’s what the agent workflow looks like:

The tools the agent uses are stored on the KNIME Hub, which acts as a central repository of reusable workflows. Each workflow represents a specific capability — for example, retrieving information on: customer data, KNIME employees, top KNIME customers, or sending emails. These are the building blocks the agent can use to answer a user’s question.

Below you can see a space on the KNIME Hub containing the tools used in our application.

To access these tools, the workflow includes a Fetch Tools metanode. This metanode connects to the KNIME Hub and exposes the latest set of available tools to the agent. Keeping tools in the Hub makes the system easy to maintain and extendable. You can add new tools, or update existing ones, without needing to modify the agent logic itself.

Once the tools are fetched, the process moves to the Chat Agent. This is the component that interprets the user’s question and decides how to answer. It doesn't rely on fixed rules or a static script. Instead, it chooses dynamically by selecting the most appropriate tools and determining how to use them to generate a useful response. 

The agent doesn’t generate answers on its own — it selects the best tool for the question and executes it to retrieve relevant information.

How you describe the agent’s world matters because that shapes how it responds. The prompt controls everything. That’s where the agent gets its instructions — a kind of briefing before it answers the question. The prompt tells it: These are the tools you have. This is what each one does. Here’s how to use them. This is who you are. And here’s the question you need to answer. With all of that context, the agent figures out the best way to respond. This full prompt is assembled dynamically inside the Chat Agent component, pulling in tool descriptions from the Hub, system instructions, and the user’s question, all in real time.

There are two common approaches:

1. Technology-centric: listing specific systems like “Salesforce” or “Zendesk” and explaining what each one does. That might look something like:

But that approach can quickly become fragile. If the underlying technology changes, say, your company switches from Salesforce to another CRM, then you’ll need to update every prompt that references it.

2. Information-centric: A more robust approach is to make the prompt focus on what kind of information the tool provides, not how it’s implemented. That same part of the prompt might then say:

This version is clearer, easier to maintain, and more aligned with how the agent navigates tools: not in systems, but in answers. Let’s take a closer look at what one of these tools actually looks like.

This tool is built as a KNIME workflow that connects to Salesforce to fetch account data, pulls metrics from the KNIME Forum (currently running on Amazon), reaches into Zendesk for support tickets, and to HubSpot for marketing information. If we move the community platform to Azure, we just update the connector, and the prompt and logic stays exactly the same.

Most organizations already have dozens of technologies, APIs, and data silos. KNIME helps teams integrate those systems and create a unified view of the data people actually need. That same integration layer becomes the foundation for building data-aware tools 

Once you have these tools, you can pair them with large language models to build intelligent agents. The KNIME Hub plays a key role here too. Not only does it help manage and organize tools, but it also adds safeguards — like limiting which AI models a tool is allowed to use. You can define which LLMs are allowed based on data sensitivity or trust level and enforce it through Hub-level policies. That control matters when you're dealing with real enterprise data.

Now that we’ve outlined the building blocks, tools, and the chat agent used in this application, let’s see how the workflow operates in a real scenario.

Let’s say a team member needs a quick overview of KNIME’s relationship with a customer. This kind of information is usually scattered across Salesforce, shared drives, and account notes. Instead of manually searching through them, they submit a query:

The agentic application identifies the type of request and determines which tools are needed. In this case, it uses the Company Domain Retriever tool to find the correct company domain and the Customer Activity Insights tool to pull relevant details. 

By leveraging these tools, the system retrieves key opportunities, past interactions, and contact history, providing a structured summary of KNIME’s relationship with this customer. Instead of spending hours digging, the team gets a fast, verifiable summary.

With KNIME, you can adapt and scale this approach to fit your unique needs. Whether it’s retrieving customer insights, internal policies, technical documentation, or other enterprise knowledge, the same agentic AI principles apply. By adding new tools and refining the workflow, you can create a more capable and adaptable internal data retrieval system.

The example in this article is just one starting point. With the right tools and logic, AI agents can go far beyond internal search and automate decisions, adapt to complex tasks, and scale across different domains. Similar principles can support a wide range of use cases, like content compliance. With every new tool, the agent gets closer to answering what matters, without anyone needing to dig for it.