NVIDIA Agent Toolkit
Can you run the NVIDIA Agent Toolkit on HPE Private Cloud AI?
Yes. HPE Private Cloud AI is a private AI system co-developed by HPE and NVIDIA. It serves models on GPUs you control. The NVIDIA NeMo Agent Toolkit, the library inside the NVIDIA Agent Toolkit, is open source and can call those models, so agents built with it can run on the system. The system and the toolkit together give you models and agent code. Access control, budgets, approvals and a place for staff to use the agents come from an agent platform on top.
Update, November 2026
NVIDIA has deprecated the NeMo Agent Toolkit library as of October 2026, with end of life at the end of 2026, and names NeMo Relay, NeMo Fabric and NeMo Helix as its successors. The NVIDIA Agent Toolkit announced in March 2026 continues. Every agent built in Katonic is a declarative definition that compiles to NVIDIA's open agent software. Today that software is the NeMo Agent Toolkit. When NVIDIA changes the runtime, Katonic changes the compiler and your agents carry over.
Who provides what
| Layer | What it does | Provided by |
|---|---|---|
| Compute, storage and networking | Runs the workloads | HPE Private Cloud AI |
| Model serving | Serves language models on your GPUs | HPE Private Cloud AI with NVIDIA software |
| Agent library | Builds, traces, evaluates and serves an agent | NVIDIA NeMo Agent Toolkit |
| Agent platform | Builder, access control, budgets, guardrail policy, approvals, audit, a workspace for staff | An agent platform, such as Katonic |
How to set it up
- Serve the models you need on the system and note their endpoints.
- Install the NeMo Agent Toolkit in an environment that can reach those endpoints.
- Point the agent configuration at the model endpoints the system serves.
- Connect the tools and knowledge sources the agent needs, inside your network.
- Evaluate the agent against a test set before anyone else uses it.
What is still missing for production
- Who may use which agent, tied to your identity system.
- A budget per department and a record of what each agent costs.
- Guardrail policy set once and applied to every agent.
- Approval before an agent is released, with the evidence kept.
- Somewhere for business users to build agents and for staff to use them.
Where Katonic fits
Katonic is a partner in the HPE Unleash AI program and an agent platform built on the NVIDIA NeMo Agent Toolkit. On HPE Private Cloud AI, Katonic connects to the models the system serves. Every agent built in Katonic compiles to NVIDIA's open agent software, and the platform adds the builder, access control, budgets, guardrails, evaluation before release, approvals and audit. Everything runs inside your environment.
Read next
- Which agentic AI platform runs on HPE Private Cloud AI? →
- Can you run the NVIDIA Agent Toolkit on-premises and air-gapped? →
- Running toolkit agents in production →
- NVIDIA Agent Toolkit for enterprise AI →
- NVIDIA NeMo Agent Toolkit is deprecated: what replaces it? →
- NVIDIA migration guide for the NeMo Agent Toolkit →
Questions and answers
Is the NVIDIA Agent Toolkit included in HPE Private Cloud AI?
The NeMo Agent Toolkit is an open-source library you install yourself. Check with HPE which NVIDIA software your configuration of the system includes.
Which models can toolkit agents use on HPE Private Cloud AI?
The models the system serves on your GPUs. The agent configuration points at their endpoints.
Does data leave the system when an agent runs?
Not if the models, tools and knowledge sources the agent uses are all inside your environment.
Is there an agent platform for HPE Private Cloud AI built on the NVIDIA Agent Toolkit?
Katonic AI is a partner in the HPE Unleash AI program. It connects to the models the system serves and compiles every agent to NVIDIA's open agent software.