# How Dynamiq built a cost-aware legal research workflow with IBM watsonx URL: https://www.getdynamiq.ai/post/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx > How a three-agent legal research workflow on IBM watsonx routes every question by cost: Granite triages, deep research runs only when needed, all traced. Agents How a three-agent legal research workflow on IBM watsonx routes every question by cost: Granite triages, deep research runs only when needed, all traced. Vitalii Duk · Founder and CEO · April 15, 2026 · 7 min read · Updated September 29, 2026 In short For a European insurer, Dynamiq built a legal research workflow with three agents: an orchestrator, a low-cost IBM Granite classifier that triages every question, and a research agent that searches internal contracts and external legal sources only when the classifier says so. Every step is traced, and legal teams across the company reach it through IBM watsonx Orchestrate. Dynamiq built a legal research workflow for a European insurance company that answers contract, policy and compliance questions at a controlled cost. A small IBM Granite model classifies every request first, and an expensive research agent runs only when a question needs it. Every reasoning and tool step is traced, and the whole workflow is available across the company as an agent in IBM watsonx Orchestrate. This post walks through the requirements, the three-agent design, how models were chosen, and the patterns other legal and compliance teams can reuse. ## What problem did the legal team need to solve? Legal teams are expected to give fast, defensible answers across contracts, policies and compliance questions, while the volume of documents keeps growing. At this insurer, the work was spread across disconnected tools, handoffs and manual review: summarizing several contracts at once, answering policy questions against internal agreements, comparing internal language with external sources, and checking compliance across jurisdictions. Speed matters here because one overlooked clause can change liability, payment terms or compliance obligations, and lawyers often compare dozens of clauses across agreements under tight deadlines. The goal was faster, better-informed answers for the business without raising costs. ## What were the requirements? The constraints are what turned a demo into a production project: - **Tight cost control.** Expensive reasoning could not be applied to every request. - **Partly deterministic routing.** Every query had to be classified before any agent answered it. - **Full traceability.** Each reasoning step, tool call and model response had to be logged. - **Existing infrastructure.** The workflow had to use the insurer's search and document systems. - **Response time.** A query had to complete within a few minutes. - **Internal and external sources.** Proprietary legal document stores, plus external legal research through [Exa](https://exa.ai/docs/reference/search-api-guide). - **Access control.** Permissions aligned with the insurer's SOC 2 requirements. ## How does the three-agent architecture work? The design has three agents with distinct roles and cost profiles: an orchestrator, a low-cost classification agent and an advanced research agent. The orchestrator makes sure the cheapest adequate path is chosen, the classifier triages and answers simple queries, and the research agent does deep work only when the classifier recommends it. ![Architecture diagram: a legal query passes from IBM watsonx Orchestrate to a Dynamiq orchestrator agent on Grok 4 Fast, which asks a Granite 4 Small classifier how to route it and, when recommended, calls a Grok 4 Fast research agent that searches internal contracts in Milvus on watsonx.data and external legal research in Exa](https://www.getdynamiq.ai/blog/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx/1-dynamiq-runtime-1.webp) Every query goes through a low-cost classifier before any deep research runs ### The orchestrator: routing, synthesis and budget control The orchestrator follows a route, decide, synthesize pattern. It sends each request to the classifier before answering, then decides whether the low-cost path is enough or the request should escalate. It also watches token consumption across the run and applies two budget controls: - When the classifier reports low confidence on a request that would be expensive to research, the orchestrator asks for the assumptions to be confirmed first. - When costs spike, it caps the research agent's tokens per step, returns a constrained summary and offers a "continue?" control. It also treats phrases such as "our terms", "my contracts" and "internal policies" as internal queries by default. For the orchestrator model, the team found Claude Sonnet 4 and GPT-5 capable but slower and more expensive for this role. Grok 4 Fast gave the needed balance of speed, cost and quality. ### The classifier: IBM Granite for low-cost triage Every request stops first at the classification agent, which runs on [IBM Granite 4 Small](https://www.ibm.com/granite/docs/models/granite). It returns structured output with six fields: complexity, data sources, task type, reasoning, recommendation and confidence. That output decides whether the workflow answers locally, escalates to the research agent or takes a hybrid path, and whether external calls are allowed at all. When the classifier marks a request INTERNAL\_ONLY, access to Exa is turned off. - **Simple queries**, such as legal definitions, straightforward policy questions and status inquiries, are answered without escalation. - **Moderate queries**, such as contract term explanations, single-document analysis and basic compliance checks, may use the research agent. - **Complex queries**, such as comparisons across documents, clause-level compliance scoring and multi-source research, always escalate. The classifier is both a routing layer and a cost-control layer. For this triage task, Granite 4 Small delivered better price-to-performance than the Grok 4 Fast model used elsewhere in the system. ### The research agent: internal retrieval plus external legal sources The research agent runs on Grok 4 Fast and activates only when the classifier recommends an EXTERNAL\_ONLY or HYBRID path. It searches the insurer's proprietary legal documents through a retrieval-augmented generation (RAG) pipeline on Milvus in IBM watsonx.data, and uses Exa for public legal sources and case law. It produces four kinds of output: 1. **Semantic contract search** for related clauses, obligations and rights. 2. **Comparative analysis** across documents, with differences in language and business impact. 3. **Clause-level compliance scoring** on a 1 to 10 risk scale, with rationale and mitigation advice. 4. **A cited summary** that blends internal and external results and labels which is which. Each answer includes an executive summary, detailed analysis, recommendations, citations and a risk assessment that flags ambiguities, conflicts, outdated references and coverage gaps. When sources contradict each other, the agent shows the conflicting passages side by side. When retrieval comes back weak, it broadens the search and records a coverage\_gap flag instead of guessing. ## How is every step traced? To meet the traceability requirement, the agents run in an XML-structured [ReAct](https://www.ibm.com/think/topics/react-agent) inference mode, which keeps every reason, act and observe step in the same shape and easy to debug. The runtime produces timestamped logs of every reasoning decision, tool call and model response in a run. That makes postmortems straightforward across the orchestrator, agents, tools and models, including the prompts and responses exchanged with watsonx Orchestrate when the workflow is called through its API. Model and embedding versions are stored with each trace, so an answer can be tied to the exact configuration that produced it. See how [observability](https://www.getdynamiq.ai/product/observability) works in Dynamiq. ## Why run the workflow through watsonx Orchestrate? Once the multi-agent workflow was built in Dynamiq, the team added it to IBM watsonx Orchestrate as an external agent through the API, using a [bearer token](https://www.ibm.com/docs/en/watsonx/w-and-w/2.1.0?topic=keys-generating-bearer-token) and a service instance URL generated by the Dynamiq platform. That step turned a stand-alone workflow into a company-wide capability. Every authorized user can call the legal research agent from chat, alongside agents connected to systems such as SAP, Salesforce and ServiceNow. The agent appears in a governed catalog, so IT and compliance teams can see what is running, who uses it and how it connects to the rest of the stack, without Dynamiq rebuilding any integration. ## What did the insurer gain? Contract review, clause identification and answers to business questions all became substantially faster; IBM's [case study of the project](https://www.ibm.com/case-studies/dynamiq) reports the measured reductions. Cost stayed predictable because routine triage runs on Granite and never touches the more expensive reasoning model. Legal teams across the company use the agent through the interface they already work in. The answers are easier to trust: citations separate internal and external sources, reasoning steps are logged for audit, and uncertainty is flagged rather than hidden. The architecture can be deployed in the cloud or on-premises as it moves further into production. ## What can other legal and compliance teams reuse from this design? - **Route before you reason.** A small model that classifies requests keeps most traffic away from expensive reasoning. - **Make routing output structured and enforceable.** Fields such as complexity, sources and confidence let code, not the model, decide what happens next. - **Turn off external access for internal questions.** An INTERNAL\_ONLY route removes a whole class of data-leak risk. - **Cap spend per step.** A token budget with a "continue?" control prevents runaway costs on long research tasks. - **Store versions with traces.** Auditors and lawyers can tie an answer to the model and index that produced it. - **Surface conflicts.** Showing contradictory sources side by side builds more trust than a confident single answer. The same pattern applies to [document review](https://www.getdynamiq.ai/use-cases/document-review) in underwriting, claims and contracts. Read more about our [partnership with IBM](https://www.getdynamiq.ai/partners/ibm), including watsonx models and data and deployment on IBM Cloud. ## FAQ ### What is a cost-aware agent workflow? It is a workflow that matches the cost of the model to the difficulty of the task. A cheap model classifies and handles simple requests, and expensive reasoning runs only for requests that need it, with token budgets that cap spend on long tasks. ### Why use a small model like IBM Granite to classify queries? Classification is a narrow, structured task that a small model handles well. Running it first on every request costs little and keeps the larger model for work that needs deep reasoning, which lowers the average cost per question. ### How does the workflow keep internal documents private? Internal contracts are retrieved from Milvus in IBM watsonx.data, and when the classifier marks a request as internal only, access to the external research service is turned off. Citations label internal and external sources separately. ### How does a Dynamiq agent run inside IBM watsonx Orchestrate? The workflow is added to watsonx Orchestrate as an external agent through the API, using a bearer token and a service instance URL generated by the Dynamiq platform. Authorized users then call it from Orchestrate's chat like any other agent. ### Can this workflow run on-premises? Yes. The architecture can run in the cloud or on-premises, and Dynamiq itself can be self-hosted on IBM Cloud, Red Hat OpenShift, other major clouds, any Kubernetes cluster or on-prem. Vitalii Duk Founder and CEO Founder and CEO of Dynamiq, writing on AI infrastructure and enterprise agent deployments. On this page 1. [What problem did the legal team need to solve?](https://www.getdynamiq.ai/post/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx#what-problem-did-the-legal-team-need-to-solve) 2. [What were the requirements?](https://www.getdynamiq.ai/post/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx#what-were-the-requirements) 3. [How does the three-agent architecture work?](https://www.getdynamiq.ai/post/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx#how-does-the-three-agent-architecture-work) 4. [How is every step traced?](https://www.getdynamiq.ai/post/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx#how-is-every-step-traced) 5. [Why run the workflow through watsonx Orchestrate?](https://www.getdynamiq.ai/post/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx#why-run-the-workflow-through-watsonx-orchestrate) 6. [What did the insurer gain?](https://www.getdynamiq.ai/post/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx#what-did-the-insurer-gain) 7. [What can other legal and compliance teams reuse from this design?](https://www.getdynamiq.ai/post/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx#what-can-other-legal-and-compliance-teams-reuse-from-this-design) 8. [FAQ](https://www.getdynamiq.ai/post/how-dynamiq-built-a-cost-aware-legal-research-workflow-with-ibm-watsonx#faq) ## Keep reading - ### [How to create an AI agent from scratch: a step-by-step guide](https://www.getdynamiq.ai/post/how-to-create-an-ai-agent-from-scratch-step-by-step-guide) Build an AI agent step by step: define the job, choose a model, add tools, knowledge, memory and guardrails, then test and deploy. With working Python code. July 17, 2025 · 8 min read - ### [Enterprise AI agents: benefits, use cases, implementation guide](https://www.getdynamiq.ai/post/enterprise-ai-agents-benefits-use-cases-implementation-guide) What enterprise AI agents are, the main types, what makes an agent enterprise-ready, where agents deliver value, and how to implement them without stalling. July 9, 2025 · 9 min read - ### [Multi-agent AI systems: definition, benefits, limitations and how to build one](https://www.getdynamiq.ai/post/multi-agent-ai-systems-definition-benefits-limitations-how-to-build) What a multi-agent AI system is, the main coordination patterns, benefits and limits versus a single agent, regulated-industry examples, and how to build one. June 30, 2025 · 8 min read ## Put this into practice. See an agent built for your workflow, running in your environment, with our engineers. [Start free](https://app.getdynamiq.ai/signup) · [Talk to the team](https://www.getdynamiq.ai/book-a-demo)