Financial services, government, telecom, technology, manufacturing
Moving agent pilots to production on your own GPUs
How do we move agent pilots built on open-source frameworks into production on our own GPUs?
Katonic moves agents that developers have already built on open-source frameworks onto the company's own GPUs, and adds testing, guardrails and monitoring without a rewrite. Pilots become governed production services, and the dependency on a public model API ends. Framework support comes through the NVIDIA Agent Toolkit.
At a glance
| Industry | Financial services, government, telecom, technology, manufacturing |
|---|---|
| Who it is for | Head of Engineering, AI centre-of-excellence lead, enterprise architects |
| Data that stays in place | Agent code, prompts, traces and evaluation results |
| Runs air-gapped | Yes |
| Frameworks | Open-source agent frameworks, through the NVIDIA Agent Toolkit |
| Infrastructure | HPE Private Cloud AI, or HPE GPU servers in an HPE AI Factory design |
The problem
Most enterprises have agent pilots built by developers on open-source frameworks that never reach production.
The pilots have no governance, no accuracy testing and no monitoring, and the model behind them is a public API.
How it works
- The existing agent is brought into Katonic Studio as it is.
- Its model calls are pointed at models served on the company's own GPUs.
- Evaluation tests are added, so each version is scored before release.
- Guardrails and access policies are applied.
- The agent is deployed with monitoring and a trace of every run.
On HPE infrastructure
On HPE Private Cloud AI
HPE Private Cloud AI is HPE's turnkey private AI system, co-engineered with NVIDIA. Katonic AI is a partner in the HPE Unleash AI program. On Private Cloud AI, Katonic connects to the models the system serves and adds the agent platform on top. The agent calls the models served on the system, in place of a public model API.
On HPE GPU servers
Customers who build their own estate on HPE GPU servers, as part of an HPE AI Factory design, get compute, storage and networking from HPE. Katonic supplies the software layer above the hardware: open-weight models served on the customer's GPUs, the AI gateway, the agent platform and governance. The agent, its models and its traces run on the company's own GPU servers.
More on both routes: Katonic AI on HPE.
Where it stands
Available in the Katonic platform today.
Questions and answers
Do we have to rewrite the agent?
No. Agents built on open-source frameworks are brought in as they are, through the NVIDIA Agent Toolkit.
What does production add that the pilot lacks?
Evaluation before release, guardrails, access control, monitoring and a trace of every run.
Can the agent keep using a public model API?
It can, but the purpose of the move is to run it on models served on the company's own GPUs.
Who owns the agent after the move?
The company does. The code, the prompts and the traces stay on its own infrastructure.