note-cbt...

## CBTA Competency Categories as Normative Artifact **The problem:** The 9 ICAO PANS-TRG competency categories and their Observable Behaviors were not derived from performance data. They were constructed via expert consensus — domain experts divided the space of piloting behavior into categories, then sorted OBs into pre-existing buckets. The category structure is institutional, not empirical. **Epistemological consequence:** This is upstream of the latent variable inference problem. The standard framing of CBTA assumes: competency is a latent variable; OBs are proxies; we infer the latent variable from proxy observations. But if the category structure itself was assumed rather than discovered, then the inference chain has a deeper problem: the assumed latent structure may not correspond to the actual dimensional structure of expert piloting performance. **Structural risk:** The 9 competency categories may be correlated in ways that weren't modeled (committee members lacked the data to detect covariance structure). The real competency space might have fewer dimensions, more, or a non-orthogonal structure. Any downstream measurement and inference built on these categories inherits this foundational uncertainty. **Cross-domain observation:** Different domains define different OB sets for structurally similar learning problems (aviation: 9 categories; some school frameworks: 4). The variation suggests the category counts are artifacts of how specific expert groups divided the problem, not reflections of a domain-invariant underlying structure. **Feedback loop quality:** The current assessment model closes the loop through highly manual processes — instructor observation, OB-graded evaluation forms. This is low bandwidth and susceptible to inter-rater variability at the category level, not just the OB level. --- **Open questions:** 1. How was the ICAO PANS-TRG competency framework actually constructed? What is the documented methodology — expert panels, Delphi method, literature synthesis, operational incident data? Was any data-driven validation of the category structure performed before or after publication? 2. Is there published psychometric evidence for the independence (or interdependence) of the 9 ICAO competencies? Factor analyses, inter-rater reliability studies that tested category structure, not just OB scoring? 3. If the category structure is normative, does this invalidate the CBTA framework operationally, or does it only become a problem at the extremes (Stage 5 assessment, cross-context transfer)? Where exactly does the normative origin matter? 4. Are there alternative competency frameworks (military aviation, medical simulation, high-reliability industries) that were derived differently — and do they converge or diverge on the category count and structure?

Permalink: http://localhost:3000/sensemaking?noteId=note-cbta-ontology-normative-artifact