Cohesix Documentation
Documentation View source

NeMo Agent Toolkit kit (selected Linux AArch64 profile)

This kit pins NeMo Agent Toolkit 1.9.0, MCP SDK 1.29.1 and A2A SDK 0.3.26. It runs on the Jetson client host, beside but outside the GPU provider runtime. Hive Gateway connects to the exact-source Queen and owns Cohesix admission, standing budgets and receipts. NeMo calls the gateway through its native MCP and A2A clients; the model endpoint only plans calls. It is never an authority or a substitute for native provider verification. Apple, NIM, NeMo Framework, Triton and Kubernetes are not required by this selection.

Install from a sealed candidate

Build the wheel once on a trusted packaging host:

python3 -m pip wheel --no-deps --wheel-dir dist integrations/nemo-agent-toolkit
shasum -a 256 dist/cohesix_nemo_kit-1.2.0-py3-none-any.whl
shasum -a 256 integrations/nemo-agent-toolkit/requirements-linux-aarch64.lock

Copy the wheel, requirements-linux-aarch64.lock and scripts/install/install_nemo_agent_toolkit.py to the Linux AArch64 client. Verify their recorded digests through the chosen artifact channel. The installer accepts absolute paths and creates a new venv; it never installs from an editable checkout:

python3 install_nemo_agent_toolkit.py \
  --venv /absolute/private/nemo-client-1.9.0 \
  --wheel /absolute/kit/cohesix_nemo_kit-1.2.0-py3-none-any.whl \
  --sha256 EXPECTED_WHEEL_SHA256 \
  --lock /absolute/kit/requirements-linux-aarch64.lock \
  --lock-sha256 EXPECTED_LOCK_SHA256

The lock names every selected transitive distribution at its tested version. Use a new venv for a different Linux, Python or Toolkit release and rerun the native compatibility checks. pip check and the installed console command must pass before connecting to Cohesix. Keep the accepted CUDA/PEFT container and the Toolkit client venv separate.

One subject per process

Give each Toolkit process a private settings JSON file, mode 0600, and a gateway credential pair minted for the same delegated subject. Example values below are references only:

{
  "gateway_url": "http://127.0.0.1:8415",
  "subject": "operator-a",
  "request_auth_ref": "file:/absolute/private/request-auth",
  "delegated_ticket_ref": "file:/absolute/private/operator-a.ticket"
}

The gateway URL must be HTTPS or loopback HTTP over a protected host channel. subject labels NeMo’s local per-user context; the verified delegated ticket decides Cohesix authority. Use a second process, credential pair and settings file for a second user. Client-supplied user IDs cannot grant a Cohesix scope. Do not forward a gateway ticket, request token or model key to the CUDA executor. Do not put credentials in tracked YAML or shell arguments.

Run cohesix-nemo doctor --settings /absolute/private/settings.json and the read-only cohesix-nemo mcp and cohesix-nemo a2a discovery modes first. The MCP path requires the selected available_selected_jobs, preflight, submit, inspect and recover tools. The A2A path requires a scoped Agent Card with the selected CUDA or PEFT skill. An optional service is unavailable when absent; discovery never invents it.

Native-client workflows

Prepare a private request JSON with scope_id and a complete immutable host-ticket/v2 object from the selected Cohesix workload or PEFT recipe. The same ticket ID and idempotency key must be used through recovery. Then:

cohesix-nemo mcp --settings /absolute/private/settings.json \
  --request /absolute/private/cuda-request.json \
  --output /absolute/private/mcp-cuda-submit.json
cohesix-nemo mcp --settings /absolute/private/settings.json \
  --recover-id ORIGINAL_TICKET_ID \
  --output /absolute/private/mcp-cuda-recovered.json

cohesix-nemo a2a --settings /absolute/private/settings.json \
  --request /absolute/private/peft-request.json \
  --output /absolute/private/a2a-peft-submit.json
cohesix-nemo a2a --settings /absolute/private/settings.json \
  --get-task-id ORIGINAL_TICKET_ID \
  --output /absolute/private/a2a-peft-recovered.json

The MCP path preflights, submits once and reconnects for the same admission. The A2A path sends one data part and uses NeMo’s native get_task helper. --cancel-id issues NeMo’s native cancellation helper for the inspected task; it does not claim native termination. Timeout or lost ACK is uncertain until the original ID is reconciled. Output files are create-only and 0600; repeat a read with a fresh output path. Typed client observations carry the original ID and result digest, but provider_verified remains false for A2A. Run the independent CUDA byte verifier or coh peft release --deployment ... verify against the native record before reporting success. A failed candidate or rollback remains recovered_failure under the shared verifier.

Model-backed NeMo agents

The MCP configuration uses Toolkit’s tool_calling_agent; the A2A configuration uses its per_user_react_agent because a per-user function group requires a per-user workflow in Toolkit 1.9.0. Both use a pinned OpenAI-compatible model endpoint. mcp-agent.yaml uses the native mcp_client function group with the selected tool list, 30 second call timeout and one bounded reconnect. a2a-agent.yaml uses a per-user native A2A group. Toolkit 1.9.0 fetches its Agent Card before applying its auth interceptor, and its generic text sender cannot produce Cohesix’s required data part. The installed cohesix_a2a_client plugin supplies only those two wire adaptations while retaining Toolkit’s own card, task lookup and cancellation helpers. This kit uses separate protected MCP and A2A processes. A combined process is not advertised without its own authenticated startup test.

Make a private 0600 model settings file after pinning and loading the chosen model, for example:

{
  "base_url": "http://127.0.0.1:8080/v1",
  "model_name": "SELECTED_LOADED_MODEL_ID",
  "api_key_ref": "file:/absolute/private/model-api-key"
}

The launcher probes /v1/models for that exact name. Provide a private 0600 input file containing one fixed task and ticket identity, then run cohesix-nemo agent-mcp or cohesix-nemo agent-a2a with --settings, --model-settings, --input-file and an unused absolute --trace-log. The trace is private because it may contain the task and model output. These model traces explain planner behavior; they do not replace Cohesix’s signed native outcome. The launcher exposes Toolkit’s nat eval and profiler through three modes. Prepare a private 0600 JSON array of one to sixteen {"id":"...","question":"...","answer":"..."} rows, then run the same file and model settings through direct and governed configurations:

cohesix-nemo eval-direct --settings /absolute/private/settings.json \
  --model-settings /absolute/private/model.json \
  --dataset /absolute/private/fixed-task.json \
  --eval-dir /absolute/private/eval-direct-01
cohesix-nemo eval-a2a --settings /absolute/private/settings.json \
  --model-settings /absolute/private/model.json \
  --dataset /absolute/private/fixed-task.json \
  --eval-dir /absolute/private/eval-a2a-01

eval-mcp is also available for a fixed MCP question. Each output directory must be unused and private; it retains NeMo’s native .tmp/nat/examples/default/standardized_data_all.csv, profiler traces and a private nat-eval.log. Inspect WORKFLOW_START/END and TOOL_START/END in the CSV and independently check the task ID and signed result. A zero eval exit code is insufficient if no dataset row or tool call ran. The kit also requires a completed native model response for every dataset row and exits with an incomplete-evaluation error when Toolkit returns zero without one. This check does not verify the answer or any provider effect. Use the fixed quality and latency budgets in BENCHMARKS.md.

For two subjects, run two isolated Toolkit processes and repeat both protocol discovery and cross-subject lookups. The gateway must refuse another subject’s task, adapter and evidence even if the caller changes NeMo’s user_id.

Reproduction record

Record the wheel and lock SHA-256, native package versions, gateway/Queen source and image identity, selected manifest, model name and endpoint, subject labels and credential modes, exact ticket IDs, request/response trace paths, native verifier result, denied action, timeout and restart observations. Report setup effort, credential exposure, control latency, manual recovery and evidence utility for identical direct and governed jobs. Retain failures and unknowns. The M28f conformance case consumes those records; package presence, mock completion and model text are insufficient.