Cohesix Python Support

The cohesix Python package is a host-side, non-authoritative client for the existing Cohesix file and console contracts. It validates inputs, applies generated bounds, and offers typed helpers; it does not add protocol verbs, roles, paths, or authority.

The package source is under tools/cohesix-py/. REST semantics are defined in API_GUIDELINES.md, control schemas in INTERFACES.md, and live topology in HOST_TOOLS.md.

The local CohesixClient.peft_activate and peft_rollback helpers update only the host registry and report execution_location=host projection=pending. They make no backend write to /gpu/models/active, which is read-only. Publish the same registry with the configured gpu-bridge-host --registry <root> --publish workflow to update the target snapshot. Serialize publication with other publishers. A successful local call does not provide a Worker receipt, target publication proof, or inference reload.

See the Glossary for Cohesix-specific role, namespace, and evidence terms.

Python support consumes the compiler-owned host-integration-dependency/v1 graph. Its python-sdk-projection row is release-required, while each external systemd, Docker, Kubernetes, GPU, or PEFT provider remains an independent row. The generated support table is the shared vocabulary: a Python object, successful dry run, mock probe, or package install cannot create Worker READY, live-provider status, or use-case acceptance.

Requirements and installation

Python 3.11 or later is required. Use an isolated environment:

# Created by either canonical host installer:
# ./toolchain/setup_macos_arm64.sh
# ./toolchain/setup_linux_arm64.sh
source .venv/bin/activate

The installers populate .venv from the hash-locked host-test requirements and install tools/cohesix-py in editable mode without optional integrations. To create only that environment with an already installed Python 3.11 or later, run ./toolchain/setup_repo_venv.sh --python python3.

The editable path above is for a source checkout. From a release-bundle root, the runtime setup creates .venv and installs the bundle’s exact wheel without resolving optional dependencies:

./scripts/setup_environment.sh
source .venv/bin/activate

For a Python-only integration with Release A, install the 1.1.0b1 distribution from PyPI in an activated Python 3.11+ environment:

python3 -m pip install 'cohesix==1.1.0b1'

This version uses the 1.1.0-beta tagged SDK code with corrected Python package metadata. It installs the client, not the native host tools or target runtime; use the matching host bundle for those components. The sealed Release A archives retain their original bundled wheel.

The M28g source selects cohesix==1.2.0 and cohesix-nemo-kit==1.2.0 for the unqualified 1.2.0 candidate. Both wheels are built from the compiler’s exact source inventory and must be installed with the matching native host candidate for Release B testing. They do not replace the published Release A client until the assembled release qualifies and is approved for publication.

Optional dependency groups are explicit:

# Docker, Kubernetes, and NVML probes
python3 -m pip install -e 'tools/cohesix-py[integrations]'

# PEFT and Transformers probes
python3 -m pip install -e 'tools/cohesix-py[ml]'

# Package tests
python3 -m pip install -e 'tools/cohesix-py[dev]'

The Python package versions are defined in tools/cohesix-py/pyproject.toml and integrations/nemo-agent-toolkit/pyproject.toml. For M28g, both use the stable Release B version 1.2.0. The installer requires both selected wheel versions to match that candidate identity.

Backends

Use REST when Python must coexist with cohsh, SwarmUI, or continuous publishers. Do not open a TcpBackend while another direct client owns the target console.

Explicit backend construction

Use explicit construction when application configuration already selected one topology. These are the minimal constructors for all four public backends:

import os

from cohesix import FilesystemBackend, MockBackend, RestBackend, TcpBackend

mock = MockBackend(root="out/examples/mockfs")
mounted = FilesystemBackend("/absolute/path/to/cohesix-mount")
rest = RestBackend(
    base_url=os.environ["COH_REST_URL"],
    request_auth_token=os.environ.get("HIVE_GATEWAY_REQUEST_AUTH_TOKEN"),
)
tcp = TcpBackend(
    host=os.environ.get("COH_TCP_HOST", "127.0.0.1"),
    port=int(os.environ.get("COH_TCP_PORT", "31337")),
    auth_token=os.environ["COH_AUTH_TOKEN"],
    role="queen",
    ticket=os.environ.get("COH_TICKET"),
)
try:
    print(tcp.list_dir("/"))
finally:
    tcp.close()

FilesystemBackend requires an already active coh mount; it neither creates nor owns that mount. MockBackend seeds a deterministic local tree and may reuse prior state at the selected root. When a CohesixClient binds a generated profile, future mock Worker observations use that profile’s exact shard width. The REST and TCP constructors perform live operations and must follow the ownership rules in HOST_TOOLS.md#choose-one-live-topology.

Backend selection from the environment

CohesixOrchestrator.from_env() selects the first configured backend in this order:

1. COHESIX_MOCK=1; optional root COHESIX_MOCK_ROOT.
2. COHESIX_MOUNT_ROOT.
3. COH_REST_URL, HIVE_GATEWAY_URL, or COHESIX_REST_URL.
4. Direct TCP using COH_TCP_HOST/COHSH_TCP_HOST, COH_TCP_PORT/COHSH_TCP_PORT, COH_ROLE/COHSH_ROLE, and optional COH_TICKET/COHSH_TICKET.

The common request timeout is COHESIX_TIMEOUT_S; direct TCP retry count is COHESIX_MAX_RETRIES.

Example read-only inspection through a configured backend:

from cohesix import CohesixOrchestrator

with CohesixOrchestrator.from_env() as cohesix:
    entries = cohesix.backend.list_dir("/")
    snapshot = cohesix.read_proc_snapshot()

print(entries)
print(snapshot.schedule_summary)
print(snapshot.lease_summary)

Authentication

REST

cohesix.exchange_identity performs one bounded, authenticated exchange using credential references and an independently enrolled provider graph digest. It returns an IdentityTicket for explicit RestBackend delegation. It follows no redirects and does not retry or refresh authority. cohesix.local_subject reads only the local kernel euid. See identity mapping.

The SDK provides QueenIntent and AdmissionCorrelation, delegated REST write credentials, strict-intent and writer-epoch validation, and bounded authority/dedupe snapshots in evidence packs. Generated cohesix-python-profile/v2 includes selected authority policy; consumers require a generated profile matching the selected target. Neither Python objects nor profile metadata constitute a VM admission grant. Missing production epoch/delegation fails before mutation, and REST writes have no automatic retry. See M27a authority.

RestBackend accepts an explicit request_auth_token. Otherwise it resolves, in order, HIVE_GATEWAY_REQUEST_AUTH_TOKEN, COHSH_REST_AUTH_TOKEN, then COH_REST_AUTH_TOKEN. The backend sends both accepted gateway headers for authenticated requests. Non-public reads, including gateway status and namespace reads, require request auth and a delegated Read or ReadWrite ticket through delegated_ticket= or COH_REST_TICKET; public classification is generated. They remain bounded by the gateway’s upstream target authority. The explicit single-caller read-compatibility option is not enabled by default. See read authentication.

RestBackend.submit_selected_job(binding, ticket) posts the same versioned binding used by coh job submit. The retained admission id is the recovery key: selected_job_status(id) reads execution and independent result delivery; reconcile_selected_job(id) checks the target result without another effect; request_selected_job_cancel(id) only requests cancellation. An uncertain submission response requires status or reconcile with the original id, never a new ticket or idempotency key. inspect_standing_scope(id) and revoke_standing_scope(id) require a separate delegated admin ticket. Standing authority and native outcome remain gateway/agent contracts, not Python grants.

cohesix.workload.batch_edges_expected and verify_batch_edges provide an independent bounded CPU verifier for the registered CUDA batch-edge example. The checked preparation script freezes package, input and expected output digests before a selected job is admitted. Its second dataset is an adaptation using the same installed package, without a Cohesix source change. recovery_guidance(record) explains the original job’s execution and delivery states, retained uncertainty and lack of checkpoint resume; it cannot cancel, settle or authorize a restart. A successful pixel check remains host evidence until correlated with the root admission, native observation and signed terminal. See registered CUDA workloads for installation and recovery.

import os

from cohesix import RestBackend

backend = RestBackend(
    base_url=os.environ["COH_REST_URL"],
    request_auth_token=os.environ.get("HIVE_GATEWAY_REQUEST_AUTH_TOKEN"),
)
print(backend.read_file("/proc/lifecycle/state", 128).decode("utf-8"))

Direct TCP

TcpBackend requires a console auth token, role, and optional ticket:

import os

from cohesix import TcpBackend

backend = TcpBackend(
    host=os.environ.get("COH_TCP_HOST", "127.0.0.1"),
    port=int(os.environ.get("COH_TCP_PORT", "31337")),
    auth_token=os.environ["COH_AUTH_TOKEN"],
    role="queen",
    ticket=os.environ.get("COH_TICKET"),
)
try:
    print(backend.list_dir("/"))
finally:
    backend.close()

For checkout-based development, resolve_tcp_auth_token() can read a Queen ticket secret from an explicitly selected or default source manifest before it falls back to COH_AUTH_TOKEN and COHSH_AUTH_TOKEN. This is a local configuration convenience, not delegated identity. Production callers should pass the intended secret explicitly through their secret-management boundary so manifest discovery cannot select an unexpected credential. Missing, empty, and placeholder tokens are rejected.

Never log tokens, embed them in source, or store populated credential files in the repository.

Public API

The provider helpers cohesix.providers.registry, provider, action, profile, surface, and validate_target resolve compiler-generated contracts and stable integration IDs, including admission, target grammar, exact host profiles and read visibility. Returned dictionaries are detached projections; changing them cannot alter admission policy or establish live provider support.

The package exports these primary surfaces from cohesix:

Target contracts and Worker API

The wheel is target-neutral. Its generated DEFAULTS object contains bounded fallback expectations, manifest_sha256=None, and execution_proof="none". It cannot identify a running target. Live Worker calls require one explicit, regular, non-symlink compiler output:

• configs/generated/cohesix_python_qemu_smp_production.json for the selected qemu_smp_production Mac or qemu_smp_kvm_production Linux build;
• configs/generated/cohesix_python_pi4_production.json for pi4_production.

Both use cohesix-python-profile/v2, but each binds its own resolved-manifest hash. They are generated independently; one target contract must never be copied, renamed, or inferred from the other. A parsed in-memory mapping is useful for validation tests but is marked source="mapping" and does not establish target identity.

from cohesix import CohesixClient, MockBackend, load_profile_contract

profile = load_profile_contract(
    "configs/generated/cohesix_python_qemu_smp_production.json",
    expected_target="qemu",
)
client = CohesixClient(
    MockBackend("out/examples/worker-model"),
    profile_contract=profile,
)

admission = client.worker_spawn("heartbeat", "heartbeat-1")
assert admission.lifecycle == "queued"  # the control write was admitted
ready = client.worker_wait_ready("heartbeat", "heartbeat-1")
assert ready.state.lifecycle == "ready"  # separately observed telemetry
client.worker_teardown("heartbeat", "heartbeat-1")

The mock in this example reports execution_proof="host-model". It exercises the API but is never QEMU or Pi proof. The three executable roles are worker-heartbeat, worker-gpu, and worker-lora; their combined generated maximum is 256 simultaneous tasks for both QEMU and Pi 4 (1 Heartbeat, 127 GPU, and 128 LoRA). worker-bus remains model-only, and spawn or teardown returns a deterministic CohesixError before any backend write.

Worker telemetry uses /shard/<label>/worker/<id>/telemetry, where the selected QEMU contract keeps the top six digest bits and the Pi contract keeps all eight. The legacy /worker/<id>/telemetry path is returned only when the selected contract enables legacy_worker_alias. No client should infer a role from an instance-id prefix.

Lifecycle, receipt, and proof boundaries

The API preserves these independent axes:

Local lease and command summaries now emit cohesix-operation-report/v1 with authoritative=false, proof_class=operation_report, mode=client_local, and source_identity=client-local. Use gpu_lease_with_report and run_command_with_report; the previous *_with_receipt method names and receipt_out argument remain compatibility aliases. Historical cohesix-receipt-v1 files are read as non-authoritative compatibility input and are never emitted by these helpers. An operation report cannot establish GPU execution, Worker completion, or provider receipt authority. Receipt-bearing host-ticket work uses the accepted host-ticket/v2/host-ticket-result/v2 pair and the bounded worker-gpu-receipt/v1 or worker-lora-receipt/v1 telemetry encoding. Python requires the caller to classify such bytes as source="local-admitted"; remote or unclassified version-2 data is rejected. Parsing still sets authoritative=False: only root admission and matching target evidence can establish authority outside the Python object.

The exact receipt actions are:

• GPU: gpu.lease.grant, gpu.lease.renew, gpu.lease.release, gpu.workload.submit, gpu.workload.cancel, gpu.workload.observe;
• PEFT: peft.export, peft.import, peft.activate, peft.rollback, and explicitly selected peft.release. Existing profiles retain the four-action PEFT contract until their manifest selects native release.

The native release recipe uses cohesix.playbooks.run_peft_release; Python validates and projects the same CLI report and never emits training scores or Worker receipts. For an application that needs an explicit release decision, use the higher-level PeftReleaseClient from a cohesix wheel built from the matching M28b source:

from pathlib import Path
from cohesix import PeftReleaseClient

release = PeftReleaseClient(
    coh_binary=Path("/opt/cohesix/bin/coh"),
    deployment=Path("/srv/private-model/release-deployment.json"),
)
planned = release.plan()
# After reviewing the immutable request and delegated ticket:
release.apply(
    rest_url="https://queen.example.invalid",
    auth_ref="file:/run/secrets/coh-auth",
    ticket_ref="file:/run/secrets/coh-ticket",
)
outcome = release.inspect()
if outcome.requested_outcome_verified:
    print(outcome.operation_id, outcome.graph_sha256)
else:
    print(outcome.state, outcome.ambiguous)

plan does not submit work; apply is the only effectful method and never retries on a lost response. After interruption, use recover or inspect with the same deployment. inspect calls the shared CLI verifier only when its read-only observation reports success. The typed status binds operation and request digests and shows not_submitted, outcome_unknown, succeeded, failed, recovered_failure, or rollback_failed without inventing a new receipt. requested_outcome_verified is true only after successful CLI verification. The status also projects signed held-out losses and sample count, comparison policy digest, frozen baseline generation, observed served generation and observed restored generation. Missing or malformed serving observations cannot pass a successful outcome. It is a non-authoritative Python projection; exact source, serving generation and application-client observations remain separate release evidence. MCP and A2A views use the same underlying identity and verifier result, not this Python object as authority. For an operation admitted externally through MCP or REST, submitted=False describes this CLI controller’s journal. A signed terminal result can still be verified under the same operation ID and request digest; verify() calls the shared coh peft release verify path before setting requested_outcome_verified=True. The selected M28d installed-wheel check observed the same MCP PEFT operation, signed graph and accepted generation. The selected configurable LoRA guide uses the installed native Python environment to prepare a pinned profile, independent import, fresh checkpoint-resume request and application endpoint observation. These preparation and observation helpers never submit a Queen ticket. run_peft_release still uses the admitted coh path and its signed outcome; an adapter-only restart has no optimizer, scheduler or RNG resume claim.

The receipt identity is the full role, slot, lease epoch, supervisor generation, and capability generation. A mismatch with the expected identity or public instance is stale; it is never rebound to a newer Worker.

Backend compatibility

All four backends use the same Worker payload validation and canonical paths. Mock returns host-model; direct TCP reports console-projection as a backend class but not target proof; filesystem is unknown unless separately bound; REST treats absent optional worker_runtime_bounds or backend_class metadata as None/unknown. Metadata is declaration-only. A connected gateway, mount, or console is not READY and cannot create QEMU or Pi proof.

Typed control example

The following performs a real scheduler write. Use it only where the mutation is intended and where the gateway’s upstream authority permits it:

from cohesix import CohesixOrchestrator, ScheduleRequest

request = ScheduleRequest(
    request_id="python-schedule-1",
    role="worker-gpu",
    priority=2,
    ticks=3,
    budget_ms=120,
)

with CohesixOrchestrator.from_env() as cohesix:
    results = cohesix.enqueue_schedule([request])

for result in results:
    print(result.path, result.bytes_written)

If the target is policy-gated, queue an ApprovalRequest for the exact target before the control record. Approvals do not broaden the gateway role or ticket and are consumed according to the target policy contract.

Typed Kubernetes intent example

K8sRbacIntent is a less obvious typed surface: it validates a Kubernetes coexistence request, converts it to an allowlisted host-ticket/v1 record, and appends that record to /host/tickets/spec. It does not call Kubernetes directly. The write below is real; run it only after reviewing the active host ticket policy and intended node:

from cohesix import CohesixOrchestrator, K8sRbacIntent

intent = K8sRbacIntent(
    intent_id="maint-node-1",
    subject="ops-user",
    namespace="edge-a",
    node="node-1",
    verb="cordon",
    reason="planned-maintenance",
    ttl_s=900,
)

with CohesixOrchestrator.from_env() as cohesix:
    results = cohesix.enqueue_k8s_rbac_tickets([intent])

for result in results:
    print(result.path, result.bytes_written)

Execution and lifecycle receipts belong to host-ticket-agent; request and result schemas are owned by INTERFACES.md#host-tickets-and-federation.

Bounds and generated defaults

The package imports generated client policy from cohesix/generated.py. That file is a coh-rtc output and must not be edited by hand. The default-profile summary is in snippets/cohesix_py_defaults.md.

Generated defaults provide target-neutral offline fallback bounds. Worker APIs instead require the explicit target-qualified contract described above. A REST client can read the gateway’s /v1/meta/bounds; evidence should retain its manifest_sha256. That response describes the gateway’s compiled generated policy, not a manifest queried from the target. Optional Worker runtime bounds are declarations only. Match all identities to /proc/boot and accepted target evidence before claiming gateway-target manifest parity.

The package validates absolute paths, rejects .., enforces component and payload limits, requires UTF-8 single-line writes, and preserves strict control schemas. Client validation improves diagnostics but never replaces target-side validation.

REST retry behavior

RestBackend uses generated retry defaults unless overridden with max_attempts, backoff_ms, and backoff_ceiling_ms. It retries HTTP 429, 500, 502, 503, and 504, plus selected transient URL failures.

For mutations, a transport loss can leave completion ambiguous. Verify the read-only target state before manually retrying an append. Stable request IDs help the target reject duplicates but do not make every operation idempotent. See FAILURE_MODES.md.

Playbooks and integrations

cohesix-playbook executes checked-in, bounded control-model templates. The templates expose approvals, schedules, leases, exports, and selected local provider probes for deployment design. They do not execute the named industry application, train or serve a model, or establish provider, target, sector, safety, or compliance acceptance.

cohesix-playbook --list
cohesix-playbook --playbook mixed-closed-loop-ai-factory --dry-run --mock
jq '{workflow_kind, use_case_id, plan_summary, production_use_case_accepted}' \
  out/examples/playbooks/mixed-closed-loop-ai-factory/report.json

The list output names each linked use-case id, capability boundary, selected provider probes, and planned control counts. A dry-run report says workflow_kind: control-model, retains production_use_case_accepted: false, and emits no control writes. Remove --dry-run --mock only after selecting a live backend, reviewing the plan, and confirming the intended authority and side effects. A live generic control plan also requires --rehearsal (rehearsal=True in execute_playbook); it remains a control rehearsal and does not execute the sector application.

Generated workflow lifecycle commands are available through cohesix-playbook plan|explain|apply|watch|verify|recover --playbook ID --coh-binary /absolute/coh. Mutating/verification operations require an explicit --deployment file. The SDK delegates to the installed Rust coh verifier; it cannot issue its own authoritative receipt. Connection secrets use --auth-ref and a protected --ticket-ref file:/absolute/path. The generated stages, unresolved external owners and exact package/topology inputs remain visible. Current domain workflows with undeployed external applications refuse apply; none is production-qualified. See HOST_TOOLS.md.

Optional probes cover systemd, Docker, Kubernetes, NVML, and PEFT package state. They run on the host, validate and bound collected data, and do not grant control authority. Evidence and receipt helpers write deterministic local artifacts without creating a new target evidence channel.

Examples live in tools/cohesix-py/examples/. Treat mock examples as functional demonstrations, not live acceptance proof.

Canonical evidence consumers

CohesixClient.evidence_case(pack_dir) and CohesixClient.attestation_result(pack_dir) read bounded canonical Rust-produced artifacts offline. cohesix.operator.trace_reference supplies only an opaque hash/length reference. The generated diagnostic bounds and shared fixtures in tests/fixtures/operator govern path confinement, source-event identity, redaction (including embedded JSON), scenario and non-authoritative outcomes. Unknown additive fields are sanitized; malformed records fail deterministically. Missing optional records raise CohesixError. There is no Python trace parser or attestation verifier. PASS records require matching evidence class, proof scope and complete verification metadata; this is structural projection only. See the signed evidence contract. The existing Python evidence producer remains compatible with legacy timeline consumers; generate the canonical case using coh evidence timeline. See the operator contract.

Testing

Run the package tests from the repository root:

python3 -m pytest tools/cohesix-py/tests
python3 -m pytest -k cohesix_parity tools/cohesix-py/tests/test_parity.py
python3 -m pip wheel --no-deps --wheel-dir out/python-wheels tools/cohesix-py
scripts/ci/python_compat_run.sh \
  --wheel-smoke \
  --wheel-dir out/python-wheels \
  --package-manifest out/python-compat/m26e-python-package.json \
  --state-dir out/python-compat/m26e-wheel

The wheel gate requires both CPython 3.11 and 3.13, installs the same wheel without dependencies into isolated environments, checks the public entry point, verifies the declared apple-mlx, integrations, ml, and dev extras, and emits cohesix-python-package/v1. That manifest binds the wheel hash and both target contract hashes. dev remains test-only. Missing optional providers return a typed skipped/degraded probe and never select MockBackend implicitly.

After a direct target runner has emitted an accepted worker-control, gpu-receipt-path, or peft-receipt-path record, the matrix mode consumes that record by reference and emits python-sdk-projection.json:

scripts/ci/python_compat_run.sh \
  --python-matrix 3.11,3.13 \
  --target qemu \
  --profile-contract configs/generated/cohesix_python_qemu_smp_production.json \
  --package-manifest out/python-compat/m26e-python-package.json \
  --wheel-dir out/python-wheels \
  --matrix configs/host_integration_acceptance.toml \
  --target-session out/host-integration/m26e-qemu/integration/worker-control.json \
  --state-dir out/python-compat/m26e-qemu

The result is release-required projection evidence. It references the direct target session, exact wheel, graph, source matrix, manifest, contract, host, and interpreter hashes; it does not replace target evidence or promote any external CUDA/NVML, PEFT, FUSE, systemd, Docker, Kubernetes, federation, or use-case row.

Changes to a backend or public helper require tests for successful use, validation failure, server refusal, and relevant retry/authentication behavior. Changes to generated defaults must be made in manifest/IR inputs and refreshed with the repository generation workflow.

Related documentation

• API_GUIDELINES.md — REST contract and status handling.
• HOST_TOOLS.md — live topology and host-tool ownership.
• INTERFACES.md — control and observability schemas.
• OPERATOR_WALKTHROUGH.md — canonical live setup.
• OPERATOR_RECIPES.md — evidence, mount, ticket, lifecycle, and PEFT procedures.