AC-2026-METH

Methodology.

The Acta score is built on a published competency framework, realistic work scenarios, and a calibrated-trust composite that measures judgment in realistic conditions, not vibe.

Four principles.

  1. 01 · Competency model

    Map every signal to a published framework.

    Acta scores against AICOS, the AI Competency Objective Scale (Markus, Carolus & Wienrich 2025). We do not invent new categories. Five sub-scores (output validation, prompt quality, adaptability, task efficiency, technical fluency) map to AICOS sub-competencies; two cross-cutting composites (calibrated trust, ethics override) are reported separately.

  2. 02 · Real workflows

    Test the work, not a test of the work.

    Every scenario derives from a real workflow we observed in the wild: a Q3 earnings summary, an Australian enterprise-software market sizing, a product spec for a dashboard feature. Each is realistic, demanding work in which the AI makes the kinds of mistakes production AI makes. We measure what the candidate catches, ignores, and trusts.

  3. 03 · Calibrated trust

    Reward the right disagreement, not the right answer.

    The calibrated-trust coefficient (Bansal et al. 2019; Buçinca 2025) is the Pearson correlation between a candidate’s accept/reject decisions and the ground-truth correctness of each AI claim. It punishes over-trust and over-rejection equally. A high Acta score with a low calibrated-trust number is a flag, not a credential.

  4. 04 · Validation by design

    Bake the audit hooks in from day one.

    Test-retest reliability, convergent validity against AICOS short-form and MAILS, and NYC Local Law 144 bias-audit support are first-class concerns in the Acta data model, not bolt-ons. The ValidationArtifact table exists so claims about predictive validity can be checked, not asserted.

The radar, not a number.

Single-number scores collapse signal. Acta surfaces a six-axis radar so a hiring decision can be made against the shape of a candidate’s AI work, not a brittle composite.

Illustrative, not a real candidate

References.

Every [n] in an Acta research article resolves here.

  1. [1]2025

    Markus, J., Carolus, A., & Wienrich, C.. Objective Measurement of AI Literacy: Development and Validation of the AI Competency Objective Scale (AICOS). arXiv:2503.12921. Read source ↗

  2. [2]2023

    Carolus, A., Koch, M., Straka, S., Latoschik, M. E., & Wienrich, C.. MAILS, Meta AI Literacy Scale: Development and testing of an AI literacy questionnaire. Computers in Human Behavior: Artificial Humans, 1, 100014. Read source ↗

  3. [3]2020

    Long, D., & Magerko, B.. What is AI Literacy? Competencies and Design Considerations. CHI ’20: Proceedings of the 2020 CHI Conference on Human Factors. Read source ↗

  4. [4]2019

    Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E.. Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance. Proceedings of the AAAI HCOMP 2019. Read source ↗

  5. [5]2025

    Buçinca, Z.. Worker-Centric AI for Decision Support. Doctoral dissertation, Harvard University.

  6. [6]2025

    McKinsey & Company. The State of AI in 2025. McKinsey Global Survey. Read source ↗

  7. [7]2025

    Brynjolfsson, E., Li, D., & Raymond, L.. Generative AI at Work. Quarterly Journal of Economics, 140(2), 889–942. Read source ↗

  8. [8]2023

    New York City. Local Law 144, Automated Employment Decision Tools. NYC Department of Consumer and Worker Protection. Read source ↗

  9. [9]2025

    World Economic Forum. Future of Jobs Report 2025. World Economic Forum. Read source ↗

  10. [10]2025

    PwC. Global AI Jobs Barometer. PwC. Read source ↗

  11. [11]2025

    Microsoft and LinkedIn. Work Trend Index 2025: The Year the Frontier Firm Is Born. Microsoft WorkLab. Read source ↗

  12. [12]2025

    Pew Research Center. About 1 in 5 U.S. workers now use AI in their job. Pew Research Center. Read source ↗

  13. [13]2025

    Gallup. AI in the Workplace. Gallup. Read source ↗

  14. [14]2025

    Indeed Hiring Lab. AI at Work Report 2025. Indeed Hiring Lab. Read source ↗

  15. [15]2025

    Stanford HAI. The 2025 AI Index Report. Stanford Institute for Human-Centered AI. Read source ↗

  16. [16]2004

    Lee, J. D., & See, K. A.. Trust in Automation: Designing for Appropriate Reliance. Human Factors, 46(1), 50-80. Read source ↗

  17. [17]1997

    Parasuraman, R., & Riley, V.. Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors, 39(2), 230-253. Read source ↗

  18. [18]2024

    Federiakin, D., Molerov, D., Zlatkin-Troitschanskaia, O., & Maur, A.. Prompt Engineering as a New 21st Century Skill. Frontiers in Education, 9, 1366434. Read source ↗

  19. [19]2024

    UNESCO. AI Competency Framework for Students. UNESCO. Read source ↗

  20. [20]2024

    European Union. Regulation (EU) 2024/1689 (AI Act), Article 4 on AI literacy. Official Journal of the European Union. Read source ↗