Attribute Capability Status Troubleshooting

Symptom

An attribute administration request succeeds, the KEOPS runtime capability endpoint reports runtimeState=DEPLOYED, or an image test passes, but the product capability still reports productStatus=BLOCKED. A screen or integration may also show a state that does not match the immutable evidence record.

Likely Causes

The operator mistook runtime component availability for product deployment or activation.
The latest currentDeliveryProgress checkpoint is still BLOCKED: Security v2026.10.29001 and the Product #667 Java SDK slice v2026.10.29 are released. KEOPS #826 / !1259 is also terminal in immutable v2026.10.30, and JavaScript !93 / !126 are terminal in v2026.10.29 / v2026.10.29001; none is deployment or final compatible-set evidence. Access !189 is terminal in v2026.10.31002; Access !190 at exact head 9618e16530fa4a54eb071c1908045097d1b7a4f0 has a successful pipeline but remains Draft. KEOPS #2309/#2310 is immutable in v2026.10.31002; merged Image !398/!399, Nightly 3012891634, and release pipeline 3012906124 published v2026.10.31 at digest sha256:db3aa2e897e2c95e9a30f56156fbf32a1d9a0ca10480e658b75184feff8da020. Trivy's raw block_on=never inventory is CRITICAL 6 / HIGH 7 / MEDIUM 20 / LOW 1, so it is not finding-free; deployment remains unobserved. The remaining chain is KEOPS #1333 / !1261 post-Image digest-equality qualification -> Gateway #713 / !188 at be067a74c3b9409fbede8eaec4cd1920a02d9219 -> Product #668 K15 positive/negative E2E -> SDK JS !129 immutable-set requalification. Admin #4769/#4770/#4771 is Draft !1418 at 768f54daa4e00179b55ff1f51fe9a13008154ff0; pipeline 3008952380 passed with warnings because its allowed-failure E2E job failed, and it remains CODEOWNER-gated and unreleased.
A release, test pipeline, package identifier, artifact hash, source tree, or image digest belongs to another cohort.
Image #392 closure is still blocked by Migration #287 operational acceptance, Image #217 final exact-image acceptance, or GitOps #419 existing-realm reconciliation. Closed Image #395 and #577 are candidate evidence under #217. The corrected same-digest deploy-free run passes, but its Ready qualification MR and #217 final capability-status criterion remain open.
The candidate-qualified signed attestation, Console #4279 consumption, or Product Documentation gate remains incomplete. GitOps #429's producer and the KEOPS #894 consumer are delivered; neither supplies the candidate attestation itself, so the authenticated result remains UNAVAILABLE.
A signed ACTIVE claim lacks an authorized matching same-digest deployment observation or live #684 runtimeState=DEPLOYED. Deployment evidence is not required for signed VERIFIED.
The isolated-local T901 result was mistaken for a shared-environment or image deployment record.
The manager-only Product Security decision was mistaken for enabled CONFIDENTIAL projection.
A browser calculated inheritance, precedence, filtering totals, provenance, or conflicts from incomplete direct-value or policy-input data.
A partial local source snapshot was mistaken for a released final tuple. The 2026-09-13 currentPreparation record intentionally has aggregateStatus=BLOCKED and finalImmutableTuple=null.
A stale issue description was treated as live delivery state. Admin !966 and !1052 are merged and green source integrations, while #4279 remains open for lifecycle/checklist reconciliation and reopened #4953 / Ready !1064 is exact-head green and discussion-clean, but still waits for EC07 exact-head CODEOWNER approval, merge and release. Separate Ready !1069 adds exact-head browser acceptance for #4279/#5214, but is also CODEOWNER-gated and proves no backend token issuance.
The configured promotion-enforcement mode was ignored. ADVISORY must show non-ACTIVE evidence as a warning without turning it into ACTIVE or an unconditional mutation block; ENFORCED must block the same non-ACTIVE result.

How to Diagnose

Check 1: Separate runtime state from product status

Read Attribute Capability Status. Confirm whether the value comes from the KEOPS runtimeState vocabulary or the immutable productStatus vocabulary. A runtime DEPLOYED result cannot promote product status.

Check 2: Compare one immutable evidence cohort

For each released backend row, compare the project, release, reviewed source commit, release-tag commit, release-pipeline source and aggregate status, validation-job result, test result, package ID, and POM/JAR SHA-256 values. The release-pipeline source must equal the release-tag commit. Do not promote a failed aggregate pipeline to passed because one job passed. For image evidence, compare the source tree and image digest as well. Stop if any identity is missing or belongs to another release.

If inspecting the current preparation record, verify that Attribute #180's bounded provider change is released and adopted by Migration while product has deferred AC01 in the current phase. The candidate 6,000 ms request budget and 3,769 ms database-query ceiling are not approved; approved values remain blank. #180 stays open at 11/12 with native status Blocked. Migration #209 may show 41/41 checked description items while remaining native Blocked, and #287 stays open at 10/11 native Blocked; do not reinterpret those checkboxes as EC11 acceptance. Existing evidence is reusable unless its inputs change or AC01 is reopened. KEOPS #1004 is terminal with released consumer v2026.9.12001 and successful registry read-back; #1030 is a separate adjacent follow-up. Technical Docs !27 is a merged preparation baseline while #121 remains open at 0/7. Admin Console !966 remains requested-changes, diverged from current develop, and waits for a same-digest real-environment signed ACTIVE receipt. Component releases and source pipelines do not supply the missing final image or cross-repository acceptance identity. Likewise, merged catalog-only Image !218 and its passed automatic mandatory jobs do not make aggregate MANUAL pipeline 2842775693 an image build, release, deployment, #392 completion, or GİB manifest mutation. Admin #5079 / MR !991, #5078 / MR !992, and #5082 / MR !994 are terminal merged source integrations with passed final pipelines; their retained local, browser, Sonar, AI, and independent-review evidence remains valid. They are not Admin Console release or activation evidence. #5146 is a non-blocking follow-up. Image #395 AC-09 is profile-scoped by note 3824621450; never reuse AKS evidence to mark GİB #443 verified.

For the current Phase 2 layer, verify immutable Attribute v2026.9.19001/v2026.9.19002 and KEOPS v2026.9.20003/v2026.9.21001/v2026.9.21002 independently. Preserve K2's failed post-publication-helper pipeline status even though its release exists. #1134 / !724 is immutable in v2026.9.23002, #1169 / !758 is immutable in v2026.9.23004, and K4 #1062 / !714 is immutable in v2026.9.23005 after authenticated package read-back. Image !258 has merged that K4 coordinate into the AKS provider catalog, with required source and merge-SHA jobs passed. Treat this as catalog adoption only: release-branch jobs were not triggered, there is no final OCI digest, and env/providers-gib.yaml was not changed. Image #395/#577 later produced immutable candidate v2026.9.24 at digest sha256:9da0c5ca77b77140dc71b6b38aa9cb342f2cd56e2451792be0d37b165676c004, with no deployment. The first deploy-free #217 attempt failed closed because the runtime-authority document still named the historical cohort and the predecessor fixture was incoherent. Ready MR !263 corrects those acceptance inputs without changing the image. Exact source 4fd2467e24cf1c41b93aa7fdbf13213555126dff passed pipeline 2879454266, all five jobs, 29/29 exact-image native tests, five smoke checks and all six lifecycle phases against predecessor v2026.9.23001. Do not promote this deploy-free result into the accepted final tuple: #217 remains 11/12, #392 remains 6/9, and no deployment observation exists. Admin #4953 / !1064 exact source b838542d1f61e84910ed49ed3c45536e4e2ca7d7 passed pipeline 2879428625: 950 files / 9,481 tests, production build, TypeScript, Sonar Quality Gate (86.4% scoped coverage), zero-finding AI review and 43/43 authenticated Chromium E2E in job 16714129415. GitNexus is exact/current at 49,795 nodes and 121,445 edges; all discussions are resolved and the MR is mergeable. CODEOWNER review was re-requested; EC07 exact-head human approval remains open. GitOps #419 / !218 exact source 448f3d4fddbaa2f1c7239b0e491311525e5287ab passed 48 projection, 9 workload-isolation and 31 ASSOS reconciliation tests, all 327 YAML files, compile-all, Helm lint and diff-check; exact-head GitNexus is current across all 491 files and pipeline 2879395856 passed. Sonar Quality Gate passed with 80.9% changed-source coverage and zero new findings, vulnerabilities or security hotspots; AI review has no open finding. Six-phase native reconciliation against released image v2026.9.24 remains bounded to prior semantic source b7983da1a2f551b05cbdce401086c314a39f4ee3. Ali/DevOps review, merge and activation remain pending; the feature remains disabled and undeployed. Do not use Admin !966 or !1052 merges, !1064 source validation, GitOps preparation, or the K4 release as a final image, deployment, VERIFIED, or ACTIVE receipt.

Admin #4279/#5214 browser-acceptance MR !1069 is separately Ready at source 4e0e8c249293b27d729f004ac3bddd03d3ef8fb3; exact pipeline 2879482114 passed 947 files / 9,426 tests, AI and 42/42 authenticated E2E, while the isolated Chromium harness passed 2/2 for membership switch/page two and selected-only client UUID submission. This remains an Admin request-boundary test, not evidence that the selected claim appears in an immutable-image token or that the unselected claim is absent. #5214 is 5/5 in In review; #4279 remains 7/8 in Acceptance. CODEOWNER approval, merge and release remain open.

Check 3: Inspect blockers without exposing values

Use bounded capability reason codes and the per-definition projection configuration read. Do not copy attribute values, tokens, credentials, user identifiers, or raw provider exceptions into a ticket or log.

Check 4: Verify the security and environment scope

Security #257 historical evidence records APPROVED_CURRENT_MANAGER_ONLY_PHASE for its tested digest. It is not candidate-specific authority and cannot replace Image #217 final exact-image acceptance. Closed Image #395 is evidence under #217, not an active #392 blocker. CONFIDENTIAL projection remains disabled under Security #290. The isolated-local T901 evidence proves its realm-local technical matrix only; it is not a shared-environment or same-digest image observation.

Check 5: Verify promotion ownership and phase

Read Image #392's closureDAG first. It must retain exactly the active Migration #287, Image #217, and GitOps #419 blockers; treat closed #395/#577 as #217 candidate evidence. Do not use the closed Security #257 historical decision to clear a current candidate gate.

GitOps #429 owns the signed keops.attribute-projection-promotion/v1 attestation. KEOPS #894 owns the authenticated fail-closed UNAVAILABLE|VERIFIED|ACTIVE read model, and Console #4279 consumes it. Image #392 supplies immutable assembly/handoff evidence; it is not a signer or promoter.

Then inspect the released #1071 enforcement mode. In default ADVISORY, an unavailable, malformed, expired, or otherwise non-ACTIVE evidence result remains visible and auditable but does not add a promotion-evidence block to an independently authorized non-promotion mutation. ADVISORY never authorizes a mutation or weakens its ordinary authorization checks. In explicitly configured ENFORCED, every non-ACTIVE result blocks. ACTIVE remains fail-closed in both modes and requires authenticated and signed ACTIVE, a matching authorized observation, and runtime DEPLOYED. Candidate-specific Product Security/T901 qualification is deferred from the initial ADVISORY closure and retained as later signed-evidence hardening for explicit ENFORCED. An unknown or malformed mode must fail closed. Do not fix either mode by fabricating signed evidence, changing an environment, or relabelling API success as ACTIVE. Admin #5166 / !1009 only aligns the warning presentation with this contract; it does not supply promotion evidence.

The completed producer delivery does not clear the candidate attestation gate. Check the matching signed evidence and read-model result, not whether the producer's issue is closed or its source pipeline passed.

Accept a signed VERIFIED phase without deployment evidence as a valid pre-deployment state. Accept ACTIVE only when the signed phase is ACTIVE, its authorized deployment observation matches the same digest, and the live #684 runtime state is DEPLOYED. Do not turn the historical aggregate BLOCKED/NOT_OBSERVED record into a current promotion state.

How to Resolve

Keep the narrowest evidence-backed productStatus; do not override BLOCKED in a browser or support workflow.
Select one immutable backend cohort and one immutable image record. Discard mixed-tuple evidence.
Complete each blocker at its owning repository: Migration #287 supplies operational acceptance, Image #217 owns final exact-image acceptance, GitOps #419 owns existing-realm reconciliation, GitOps #429 produces signed promotion evidence, KEOPS #894 validates and exposes it, and Console #4279 consumes the KEOPS result. Do not manufacture a release coordinate, fill an unpublished row with branch data, or make the browser reconstruct state from Image/GitOps/runtime sources.
Use server-returned Direct Values, Defaults and Policies, and Effective Values as distinct views. Never calculate an effective winner or authoritative total in the browser.
Repeat product-state evaluation only after the immutable evidence record changes.

Signals to Inspect

Logs: bounded capability ID, component status, and reason code; no raw values or tokens.
Metrics: request outcome and latency without realm, user, client, or attribute-value dimensions.
Traces: the authorized read boundary and bounded provider status, with sensitive detail removed.
Audit events: who performed the administrative read and which immutable release/digest was evaluated, subject to the platform audit policy.

Escalation Notes

Escalate when evidence for the same capability mixes releases, a runtime response contains an unknown state, a UI promotes runtimeState=DEPLOYED, or a server response omits stable identity, version, provenance, or typed failure fields promised by its released contract.

Attach only the capability ID, bounded states/reasons, exact release or digest, relevant pipeline links, and reproduction steps. Route backend compatibility and migration details to Technical Docs #121.

danger

Do not expose credentials, tokens, raw attribute values, sensitive provenance, or unnecessary user identifiers while collecting evidence.

Attribute Capability Verification

Evaluate runtime and product states without inference.