Valstar.dev

I was recently working on a chat app / tool workflow system against an internal system, this system had been worked on for years and had dozens if not hundreds of data objects in its setup. I did not have time to fully build out the hundreds of functions/tools to “ask anything” to the AI, but users expect the AI to either do everything you can do in the app, or its not very useful and people abandon it immediately. So… I decided to take a shortcut.

Tool Proliferation

The typical problem with building an AI agent is that users expect it to do everything, if the data exists in the system / business it’s connected to they want to be sure it can answer anything; and the first time they ask a question that the AI says it doesn’t know (but it should) they decide it’ll just be easier to use the app manually.

However, if the AI can do what they want, but also add something more to the equation like insights and/or a personal touch… or even better answer a question that would have required they look at multiple documents/pages/screens in order to answer the same question, then the user is now invested in using the tool.

To get this kind of ability generally requires writing quite a few tools, and making sure those tools can complement everything already within your application(s) or tools. This can be a very time consuming process, and easily fall out of date with the actual functionality available. Additionally within these tools or attached skills for the AI you need to describe the linkages between your data.

We Already Had a Graph API

Our internal application that I was attempting to add the AI to had the coincidence that it was a graph system. This graph had all of our complex data structures and linkages of data mapped out and even comments attached.

So me and the team hatched a quick prototype… what if we just gave the AI the schema?

It worked! but the token usage was extremely high, since the schema was essentially sent for each subsequent tool call, those tokens would add up VERY quickly.

Three Tools, Two Sub-Agents

After exploring a bit further we found that there was a very quick setup we could make that would essentially give the LLM everything it needed to answer 90% of questions asked of it… we gave it a couple sub agent types.

1. Schema Explorer (Sub-Agent)

The first agent we gave it was one where we fed in the entire schema, and instructed it to help answer questions about the graph schema with some additional information about how the business was run, and a little color to the schema.

This worked out great in answering questions about what data was available and how it was linked which followed into the next agent:

2. Query Builder (Sub-Agent)

After multiple questions could be answered with the first agent a 2nd one could be spawned with (in addition to the schema) some color around the fields and best query practices for our API was able to generate graph queries quite well and accurately.

3. Query Runner

And lastly we had a tool to simply run the query, making sure it could return the apollo errors directly, as those often contain some useful information about any bad queries.

How This Works

This leverages a similar structure you often see in the AI coding tools, where it will throw sub agents out to do research but giving it a focused set of items it can do.

This is however wildly expensive, usually in the 5-20 cent range for medium complexity problems, but was surprisingly accurate and able to generate both accurate answers and solve actual daily problems for many users.

Why Not Just Use SQL?

The SQL route is tempting, and would allow the LLM to explore the data in a similar fashion to the graph but you lose access controls and you lose all the business rules you generally assign your data.

I’ve seen lots of teams and companies try this approach, it never turns out well. You need highly technical people to be very explicit in their prompts about what data to query to get useful data.

I have seen lots of companies working towards giant data lakes for their data with the plan to add “AI” to it later… for some things this definitely makes sense e.g. market research, analytics, other things that you want deep targeted analysis of… but unstructured requests against this data (at least at the time of writing) still often fail due to lack of context about what the data means, how it connects and anything about the business processes at the company it’s being built for.

The Trade-Off: Cost vs. Speed of Development

Attaching a graph of your company’s data is certainly the quick and dirty way of giving it superpowers, but it definitely comes at a cost.

Our calls with this system tended to cost 5-10x than a similar task we could get working with dedicated tools, but this gave us the ability to get an everything AI up in days instead of months. Since this simple implementation was done we started tracking the ultimate queries being generated/run by the AI and finding common data patterns it was looking for and implementing those as direct tools; this has allowed us to drop the additional costs to only about 2x what we think we could get with direct tools, but we only have to build 5-10% (or possibly less) of the tools required to cover our entire set of business data. And honestly I don’t know if we would ever remove these functions as a backup for the LLM to explore if it can’t find the data in the existing tools.

This pattern isn’t limited to custom-built agent systems either. You could package the same three tools as an MCP server and plug it into something like Claude Code, Cursor, or any MCP-compatible client. If your team already has a GraphQL API and wants to experiment with AI-assisted data exploration without committing to a full agent build, spinning up an MCP server against your schema is a low-effort way to test the waters.