v0.7.13 — The Serve Layer

validation.external and mutation.external

External HTTP calls can now be declared under validation: and mutation: in addition to onReconcile:. Calls declared here fire before the corresponding rule loop — at admission webhook time and, by default, at every reconcile.

validation:
  external:
    - name: healthCheck
      url: "{{ .spec.healthCheckUrl }}/health"
      expectedStatus: 200
      continueOnError: true
      fires:
        reconcile: false   # admission-only

  rules:
    - field: "{{ .external.healthCheck.status }}"
      equals: "200"
      action: deny
      message: "health check failed — CR rejected"

fires.reconcile: false marks a call as admission-only. The reconciler skips it on resyncs; the webhook always runs it. When omitted, the call fires at both sites.

Results from validation.external and mutation.external are now propagated back into the main reconcile resolver — status fields, resource templates, and subsequent steps can reference .external.<name>.* just like onReconcile.external results. This holds even on the denial path: when validation denies, status is patched with the enriched resolver, so phase and error fields reflect the external call outcome.

The external call runner is now in pkg/external — shared by the reconciler and the gateway webhook.

include: in external call lists

All external: lists now support per-item include: entries. An item with include: set is replaced in-place by the calls: list from the referenced file. Works in onReconcile.external, onCreate.external, hooks.external, validation.external, and mutation.external.

onReconcile:
  external:
    - include: ./shared/auth-calls.yaml
    - name: healthCheck
      url: "{{ .spec.serviceUrl }}/health"

./shared/auth-calls.yaml:

calls:
  - name: tokenFetch
    url: "{{ .spec.authUrl }}/token"
    method: POST

Protocol clients

external: blocks now support a protocol: field selecting a native client instead of HTTP.

All protocols resolve before deployment evaluation and return results at external.<name>.*. continueOnError: true is supported on all of them.

Fix: Kafka consumer group lag

fetchGroupLag now uses kafka.Client.OffsetFetch + ListOffsets to compute latest − committed per partition. Previously it called Reader.ReadLag, which returns lag from offset 0, not the group’s committed position.

Fix: file: paths in included e2e steps

Relative file: paths in kubectl.apply steps loaded via include: now resolve against the include file’s own directory, not the root e2e.yaml directory.

Prometheus notes

Seven template notes for working with Prometheus external results:

• promValue — extract the first scalar value from a Prometheus instant query result
• promSum / promMax — aggregate across multiple series
• promAboveThreshold / promBelowThreshold — boolean threshold checks for when: conditions
• promSeriesCount — number of series returned
• promLabelValues — extract a label value from the first series

exists and notExists e2e assertions

- name: field is populated
  kubectl:
    get:
      - kind: MyApp
        name: my-app
        namespace: default
        field: .status.connectionCount
        exists: true

exists: true fails when the field is empty or missing. notExists: true fails when the field is present. Both work in all e2e assertion blocks (kubectl, exec, http, expect).

Runtime RBAC — automatic secrets get for auth.secretRef

When any external: block in a Katalog uses auth.secretRef, the generated runtime RBAC now automatically includes a secrets get rule. No manual RBAC annotation required.

GET /api/v1/query endpoint

The runtime now exposes GET /api/v1/query?expr=<promql> — a passthrough Prometheus query endpoint backed by the operator’s own metrics. Useful for dashboards and autoscaler signals without a separate Prometheus instance.

Breaking: labels/annotations move to native map syntax

labels: and annotations: move from a list of {key, value} pairs to a plain map — the shape every Kubernetes user already expects.

# before
labels:
  - key: app
    value: "{{ .metadata.name }}"
  - key: tier
    value: backend

# after
labels:
  app: "{{ .metadata.name }}"
  tier: backend

This also applies to selector:, labelSelector:, and fieldSelector: fields, which shared the same list shape under the internal SelectorMap type — now unified with labels/annotations under one Labels type, since both were already map[string]string underneath.

Kubernetes notes — single-key label, annotation, and status accessors

Nine new notes complement the existing whole-map labels/annotations/status accessors with single-key equivalents:

• getLabel / getLabelInt / hasLabel — read or check a single label key
• getAnnotation / getAnnotationInt / hasAnnotation — read or check a single annotation key
• getStatus / hasStatus — read or check a single status field, scalar or structured
• labelMatches — check whether an object’s labels contain every given key/value pair

# value: '{{ getLabel .children.deployment "app.kubernetes.io/name" }}'
# value: '{{ hasAnnotation . "autoscale/enabled" }}'
# value: '{{ labelMatches .children.deployment "app" "frontend" "env" "prod" }}'

Breaking: envFrom.secretRef/configMapRef move from a name list to a struct

envFrom.secretRef and envFrom.configMapRef move from a plain list of names to a list of {name, prefix, optional, keys, suffix} refs. prefix and optional map directly onto Kubernetes’ own EnvFromSource/SecretEnvSource/ConfigMapEnvSource fields — no behavior change for anyone only using those. keys and suffix are Orkestra additions with no Kubernetes equivalent: Kubernetes’ envFrom is a blanket import with no per-key rename mechanism, so a ref that sets keys is expanded into individual env: entries instead of a native envFrom source. suffix without keys is now a validation error — there’s nothing for it to rename during a blanket import.

# before
envFrom:
  secretRef:
    - myapp-creds
  configMapRef:
    - myapp-config

# after
envFrom:
  secretRef:
    - name: myapp-creds
      prefix: "DB_"
      optional: true
    - name: myapp-feature-flags
      keys: [ENABLE_BETA, ROLLOUT_PCT]
      suffix: "_FLAG"
  configMapRef:
    - name: myapp-config

SecretKeyRef/ConfigMapKeyRef (used under env.valueFrom) also gain an optional field, mirroring corev1.SecretKeySelector/ConfigMapKeySelector.

serve labels/annotations — labels and annotations as self-service form fields

serve.fields exposes spec.* to the Serve form — but team, environment, and feature flags are usually metadata, not spec data. serve labels/annotations exposes label and annotation keys the same way, written to metadata.labels/metadata.annotations on apply instead of spec. Each entry needs an explicit type (string default, integer, number, boolean, enum) since labels/annotations have no CRD schema to infer it from.

serve:
  enabled: true
  fields:
    image:
      label: "Container Image"
  labels:
    team:
      label: "Team"
      required: true
  annotations:
    canary.myorg.io:
      label: "Enable canary rollout"
      type: boolean

Validated at ork validate time: every key must be a syntactically valid Kubernetes label/annotation key, and no key may collide with serve.fields or the other bucket. serve.include now also merges labels: and annotations: blocks from the included file, the same way it already merged fields:.

serve.fields.<name>.required — enforced server-side, for every client

required: true on an serve.fields or serve labels/annotations entry now synthesizes an implicit exists validation rule at katalog load time, with message: matching the field’s label: automatically. This is enforced at the API server — the Control Center form, curl, a CI pipeline, kubectl apply, any Gateway API client — not only the one that renders a required-field asterisk.

serve:
  fields:
    targetRevision:
      label: "Branch / Tag"
      required: true
# → synthesizes: { field: spec.targetRevision, operator: exists,
#                  message: "Branch / Tag is required", action: deny }

The synthesized rule inherits the field’s own when:/or:, so a field required only under one branch of a discriminator (e.g. workloadType: app) stays conditionally required — not unconditionally — matching what a static CRD schema’s required: [...] list can’t express.

Fix: operator: in was never evaluated in validation.rules

operator: in was defined for when:/or: conditions but missing from the separate rule-evaluation switch in both the reconciler and the admission webhook — a validation.rules entry using it silently always passed instead of checking comma-separated membership. Both now evaluate it.

The reconciler and the webhook no longer maintain separate copies of validation-rule evaluation, shorthand resolution, and field lookup — all now shared from pkg/types (EvaluateValidationRule, ResolveValidationOp, ResolveScalarField). That duplication is exactly how operator: in went unimplemented in both places at once.

serve.fields.<name>.type: enum — membership validated automatically

type: enum on an serve.fields/serve labels/annotations entry now synthesizes an implicit in validation rule, the same way required: true synthesizes exists. Membership is checked only when the field has a value — an enum field that isn’t also required: true can still be omitted, it just can’t be set to something outside the declared list.

serve:
  fields:
    workloadType:
      label: "Workload Type"
      type: enum
      enum: [app, cert, monitoring, infra]
# → synthesizes: { field: spec.workloadType, operator: in,
#                  value: "app,cert,monitoring,infra",
#                  message: "Workload Type must be one of: app, cert, monitoring, infra" }

ork serve CLI

Added ork serve command for inspecting and validating Serve configurations.

Subcommands:

• validate — Validate Serve configuration in a Katalog (--full for detailed breakdown)
• schema — Show the flat schema for a Serve target (--target, --kind, --name)
• fields — List serve fields with their paths and types (--target, --kind, --name)
• tokens — Show token permissions for a CRD (--target, --kind, --name)
• targets — List all serve targets in a Katalog
• can-i — Check if a token can perform an operation (--token, --target, --operation, --namespace)
• response — Show the serve response configuration (--target, --preview)

JSON object/array values in onCreate custom resource templates

A resolved template value that looks like a JSON object or array (e.g. a Serve form field collecting raw JSON) now coerces into a real map/slice instead of being embedded as a literal string — matchLabels: "{{ .spec.serviceSelector }}" now produces a structured selector, not a JSON-string value in a field that expects a map. Same mechanism that already coerced resolved templates into int/float/bool (TryCoerceString, now shared from pkg/types instead of duplicated across the custom-resource resolver, the forEach-expansion path, and the conversion webhook — the forEach path had no coercion at all, a real pre-existing gap this closes).

New notes: Kubernetes and general input-format validation

Two new note domains, exposed for use in validation.rules and when: conditions:

• Kubernetes format — isValidLabelValue, isValidLabelKey, isValidAnnotationKey, isDNS1123Subdomain, wrapping the same k8s.io/apimachinery checks the API server itself uses.
• General input format — isValidEmail, isValidGitRepository, isValidURL, isValidImageRef, isValidJSON, isValidPort.

validation:
  rules:
    - field: "{{ isValidGitRepository .spec.repoURL }}"
      equals: "true"
      message: "Repository URL must be a valid git repository"
      action: deny

Resource schema reference — one page per resource kind

documentation/reference/schema/06-resources/ documents every Kubernetes built-in and custom resource declarable under onCreate/onReconcile/onDelete — fields, types, worked YAML examples, and lifecycle semantics (reconcile: true, onDelete cleanup). There was previously no reference for this at all. The *TemplateSource structs’ Go doc comments are now the single source of truth, rendered by hack/generate-resource-docs (make generate-resource-docs, wired into ork: and validated in CI).

Breaking: version removed from every resource’s *TemplateSource

Every onCreate/onReconcile/onDelete resource declaration (deployments, services, secrets, etc.) had a version: field intended for pinning a specific OrkestraRegistry implementation per resource — a feature that was never built. It was accepted and silently discarded everywhere; nothing ever read it.

Helm and ORAS excluded from the runtime and gateway binaries

helm.sh/helm/v3 and oras.land/oras-go are gone entirely from ork run (runtime) and ork gat (gateway) — both are build-tag excluded (!runtime && !gateway) rather than just documented as unreachable. Both were only ever used for authoring-time Katalog imports (imports.helm:, imports.registry:, motif imports) — the runtime and gateway only ever read an already-merged katalog.yaml key from a ConfigMap (ork generate bundle resolves everything ahead of time), so neither binary needs them. This also removes the previously-accepted GO-2026-5932 (openpgp) and GO-2026-5622/5338/5064 (containerd) findings from vuln-runtime/vuln-gateway — they’re gone, not just excused, so that check is now a hard gate (no continue-on-error). vuln-orkestra (the broad, dev-CLI-scoped scan where those findings still apply) moved to its own manually-triggered workflow.

New condition operators: gte, lte, between, notBetween, notIn, notContains, regex

Available in both when:/or: and validation.rules, with shorthand fields matching each operator name. Also fixes a real bug: validation.rules’ gt/lt were accidentally inclusive when used explicitly (Min/Max shared their evaluation case) — Min/Max now resolve to the new gte/lte unchanged, and explicit gt/lt are properly strict. An unknown operator: value in validation.rules/mutation.rules is now rejected at katalog-load time instead of silently never matching.

operator: unique — designed and deferred until stable, now implemented

unique was designed and declared as a valid operator from its introduction, with enforcement deliberately deferred until it could be checked safely — a rule using it silently always passed in the meantime, in both validation.rules and when:/or:. Now implemented at both enforcement points:

• Reconcile time — the reconciler injects a live checker (template.Resolver.WithUniquenessChecker) that lists other instances of the CRD via the API server and denies/gates on a matching field value, excluding the CR under evaluation. Authoritative — immune to cache staleness.
• Admission time — the gateway injects its own checker (pkg/gateway/webhook/uniqueness.go), backed by an HTTP call to the runtime’s own GET /katalog/{crd}/cr?field=<dot-path> endpoint (new ?field= support) instead of a live List() — the runtime already has this data in its informer cache. Deliberately a fast, best-effort early-rejection layer, not a second source of truth: the cache can be momentarily stale, so a duplicate can still slip past admission in a race, but it’s caught on the next reconcile regardless — the reconcile-time guarantee never depends on admission catching it first.

ork simulate can exercise the reconcile-time path: a CR file with two documents of the CRD’s own kind reconciles the first and seeds the second into the fake dynamic client as a pre-existing instance (pkg/registry/simulate/fixture/unique/). The admission-time path needs a real cluster — see ork e2e.

e2e output assertions now use the stable Condition evaluator — formerly deferred until stable

e2e’s shell-command and kubectl-output assertions (equals, contains, regex, oneOf, …) were kept hand-rolled and separate from when:/or: until the shared Condition/EvaluateOneCond evaluator stabilized. Now unified — pkg/registry/e2e’s assertion logic delegates to EvaluateOneCond per field, so e2e assertions gain every when:/or: operator (gte, between, regex, …) for free and can no longer drift from that behavior. Field names and error messages are unchanged.

serve.fields/serve labels/annotations link: — a clean display field, decoupled from the evaluation expression

A validation rule’s field: is often a template expression once it targets serve labels/annotations — e.g. {{ getLabel . "team" }} rather than a plain spec.team. Clients that highlight the offending form field from a violation’s Field had nothing usable to match against in that case. link: fixes this: a plain, non-template field name naming the serve.fields/serve labels/annotations key a rule concerns, used as the violation’s reported field instead of the raw expression. Validated at load time — link: must match a real serve field, and is rejected as redundant if it just repeats an already-clean spec.<name> field.

validation:
  rules:
    - field: '{{ getLabel . "team" }}'
      link: team
      operator: exists
      message: "Team is required"
      action: deny

Synthesized required/enum rules set link: automatically.

order now also decides validation priority, not just form layout: synthesized rules are prepended ahead of hand-written ones, so when a field fails both a missing-value check and a content check at once, DenialMessage() (which reports only the first violation) leads with “required” rather than a less useful content error. Field order itself is deterministic now too — allServeFieldRefs sorts by order instead of ranging over a Go map — and two fields on one CRD sharing a non-zero order: is a load-time error.

kubectl.apply gains exitCode and full assertions — for admission-rejection e2e tests

kubectl.apply previously only ran to completion or failed the test — there was no way to assert an apply should be rejected (e.g. by an admission webhook) without dropping to a raw commands: run: kubectl apply ... block. It now accepts the same exitCode/assertion fields as commands: (equals, outputContains, regex, between, …): default exitCode: 0 means success is expected; a non-zero value asserts the apply must fail, and outputContains/etc. can check the denial message. stdout+stderr are captured unconditionally now, regardless of exitCode.

kubectl:
  apply:
    - file: ./cr-duplicate.yaml
      exitCode: 1
      outputContains: "spec.domain must be unique"

kubectl.port-forward gains headers/body — for testing token-gated endpoints like the gateway Gateway API

Previously every kubectl.port-forward request was an unauthenticated GET. headers (a string map) and body now let an entry send an authenticated POST/PUT/PATCH — e.g. asserting on the gateway’s Gateway API. Both go through os.ExpandEnv (${VAR} syntax, same convention as gateway.api.auth.tokens.token), so a CI secret never has to be written into the e2e file:

kubectl:
  port-forward:
    - service: orkestra-gateway
      namespace: orkestra-system
      port: 8443
      path: /api/v1/apply
      method: POST
      headers:
        Authorization: "Bearer ${ORK_CI_TOKEN}"
      body: '{"apiVersion":"platform.myorg.io/v1","kind":"AppRequest", ...}'
      outputContains: '"accepted":false'

serve.name — server-side name resolution for the Gateway API

metadata.name exists on every CR regardless of scope, so unlike serve.namespace (below), serve.name isn’t scope-dependent — but it is optional, since most CRDs still want the caller to choose a name (multiple concurrent instances of one repo — PR previews, ephemeral environments). Set serve.name only when instances are 1:1 with some identity the caller already supplies, and a redeploy should update that same CR in place rather than create a new one — a stable environment where only the image tag changes between deploys:

serve:
  enabled: true
  name: '{{ repoSlug .spec.repository }}'

serve.name is a template expression the gateway’s Gateway API resolves server-side, against exactly what the caller submitted, and always wins over whatever (if anything) the caller sent — the same in full CR mode (POST /api/v1/apply with a complete CR) and target mode ({"target": ..., ...fields}). When not set, a name is required from the caller instead — metadata.name in full CR mode, a flat "name" field in target mode — and the Gateway API rejects a request with an empty one immediately, as a structured violation (metadata.name is required), instead of letting the SSA patch fail with a raw Kubernetes error.

CRDEntry.RequireServeName() (true unless serve.name is declared) flows through the runtime’s /katalog response as requireServeName and into the Control Center’s Serve form — the Name field is only rendered when requireServeName is true.

→ Target Mode — serve.name and serve.namespace

serve.namespace — server-side namespace resolution for the Gateway API

A namespaced CRD needs a namespace on every CR it creates — but a browser form or a CI curl has no business deciding which one. serve.namespace works the same way as serve.name above — a template expression the gateway’s Gateway API resolves server-side against exactly what the caller submitted, always winning over whatever (if anything) the caller sent, the same in full CR mode and target mode — but only applies to namespaced CRDs, and unlike serve.name, is required rather than optional:

serve:
  enabled: true
  namespace: '{{ teamName }}'

Once set, no Gateway API caller — Control Center, curl, CI — needs to know or send namespace at all; serve.name plus serve.namespace (when both are set) is the whole contract. The Control Center form no longer renders a namespace field for any CRD. serve.namespace routes into a namespace, it doesn’t create one — the platform team provisions it ahead of time, the same way a namespaced CRD already requires. Only affects the Gateway API: a raw kubectl apply is unaffected, since kubectl always resolves some namespace client-side before a request reaches the API server, so there’s never a genuinely empty namespace for a webhook to fill in the way an omitted JSON field lets the Gateway API detect intent — deliberately not implemented as a mutating webhook default for that reason.

Three checks at ork validate time: required on a namespaced CRD with serve.enabled: true; rejected on a cluster-scoped one (namespaced: false) — nothing to resolve into; rejected when templated and the CRD’s informer is pinned to one fixed namespace (allowedNamespaces with exactly one entry, or the legacy namespace: field) — a CR resolved outside that one namespace would exist but never be reconciled, silently. No equivalent checks exist for serve.name — there’s no cluster-scoped/pinned-namespace-style conflict for a name to run into.

serve.fields.path — nested spec paths

serve.fields entries now support a path: field mapping a flat field name to a nested dot-notation path in the CRD spec. Callers submit flat fields; the gateway maps them to nested locations.

serve:
  fields:
    cpu:
      path: app.resources.cpu
      label: "CPU Request"

ork validate ensures paths are unique, formatted correctly, and warns on nested paths (schema existence validation coming later).

→ Nested fields with path reference

Control Center: the [+ Create] form now uses target mode

The Serve form built a full Kubernetes CR client-side — the browser knew which submitted field belonged in spec versus metadata.labels/metadata.annotations, and the server reassembled apiVersion/kind/metadata/spec before forwarding to the gateway. It now submits the same flat {"target": "...", ...fields} shape any other caller would, and the gateway builds the CR — Control Center no longer constructs one. The schema fetch that feeds the form moved with it: GET /api/v1/schema/{kind} (a path shape the gateway never actually served — this was silently 404ing) became GET /api/v1/schema?target=<target>. The runtime’s /katalog response now carries a target field per CRD alongside serveEnabled, so Control Center never has to derive one from Kind/GVK.

Fix: serve.tokens warnings were never surfaced

Three validation warnings — a token’s global permissions containing an operation invalid for schema endpoints, a token with no permissions declared, namespace restrictions on a cluster-scoped CRD — were attached to a loop-local copy of the CRD entry and never written back to the Katalog, so they silently never appeared anywhere. Fixed by writing the mutated entry back to the Katalog’s CRD map after each warning.

New: serve.tokens security page

serve.tokens is a real, separate authorization layer — per-token operation and namespace scoping on the Gateway API — not a variant of the existing CRD-level allowedNamespaces/restrictedNamespaces (which governs informer/admission topology the same way for every caller). Documented at security/serve-permissions.

POST /api/v1/apply response: pollUrl replaces resourceVersion

A successful apply now returns pollUrl — the exact GET /api/v1/resources/{kind}/{namespace}/{name} path for the CR just applied — instead of resourceVersion, which nothing consumed. Callers can jq -r '.pollUrl' straight into a poll loop instead of hand-assembling the path from kind/namespace/name. Cluster-scoped CRDs get an empty namespace segment (/api/v1/resources/AppRequest//payments-api), matching the existing GET/DELETE path convention.

repoSlug and lookup notes

repoSlug extracts a Kubernetes-safe name from a repository reference — the last path segment, .git stripped, then slugified. Works with git URLs, org/repo shorthand, or a bare name, so a platform-curated repository enum doesn’t need pre-cleaned values: {{ repoSlug "myorg/payments-api" }} → "payments-api".

lookup returns the value paired with a key in a flat list of alternating key/value pairs — a derive-team-from-repository switch without a long if/eq chain:

notes:
  functions:
    - name: teamName
      expression: |
        {{ lookup .spec.repository
             "myorg/payments-api"  "team-payments"
             "myorg/orders-api"    "team-orders" }}

notPrefix/notSuffix condition operators

when:/or:/validation.rules had prefix/suffix and notEquals/notContains/notIn, but no negated prefix/suffix — a real gap, since RE2 (Go’s regex engine) can’t express “does not end with X” as a pattern either (no lookahead/lookbehind), leaving no shorthand way to write a rule like “reject :latest image tags”. notPrefix/notSuffix close it, evaluated identically everywhere Condition/ValidationRule already are.

gateway.api.auth.include: — external token file

include: is now supported in gateway.api.auth, following the same pattern as status.include, validation.include, and serve.include. References a YAML file containing a tokens: list. Inline tokens override included tokens with the same name.

gateway:
  api:
    auth:
      include: ./shared/tokens.yaml
      tokens:
        - name: control-center
          secretRef:
            name: ork-apply-token
            key: token

Included entries are loaded first, then inline entries override by name.

Serve enhancements: target, polling, token permissions, schema/raw APIs

serve.target — caller-facing identifier

Decouples the caller-facing identifier from the Kubernetes kind. Defaults to lowercased kind. Validated for uniqueness at ork validate time. serve.target gives platform teams a stable, caller-facing identifier that can evolve independently of the underlying CRD kind. With this, you can swap the underlying CRDs, and callers never know. When omitted, defaults to the lowercased kind. Targets are validated for uniqueness at ork validate time

Gateway API — target mode

Callers can now submit {"target": "smartapp", "fields...} instead of a full CR. The gateway builds the CR from Serve fields. Full CR mode remains supported.

Schema API — flat fields

GET /api/v1/schema?target=<t> returns a flat map of all Serve fields (spec, labels, annotations). Callers no longer need to know Kubernetes structure.

Raw Schema API

GET /api/v1/raw-schema?kind=<kind>&apiVersion=<version> returns the raw OpenAPI spec from the CRD.

Polling URL configuration

serve.config.response.poll.field appends ?field=<value> to the resolved poll URL. poll.url replaces the default URL entirely with a custom template. Both support templating.

Token permissions — scoped

allowedTokens now supports global, schema, and resources permission lists. Schema permissions only allow get/list. Validated at ork validate time.

exclude as a list

serve.config.response.exclude is a list of paths to be removed from the response.

toList note

Converts a comma-separated string to a list. Essential for dynamic exclusion lists.

API Surface Changes

Blog: There Is No Kubernetes Expression Language

New post — blog/KEL. Covers KEL as a composable vocabulary of Go template functions, how notes build on it, and why Helm proved the pattern worked.