Benchmark evaluating AI agents on end-to-end, long-horizon, policy-rich healthcare workflow automation. Covers three domains: provider prior authorisation, payer utilisation management, and population care management — 75 single-domain tasks plus 23 end-to-end provider-payer tasks grounded in a 1,279-document Managed Care Operations Handbook and 20 simulated healthcare applications. The best-performing agent (Claude Opus) achieved 28.0% pass@1, reflecting the genuine complexity of real-world healthcare administrative workflows. Code is Apache-2.0; the policy handbook dataset is gated on Hugging Face and requires access approval.
Assess — committed within 3 months; permissive licence (Apache-2.0).
Derived deterministically by the MACH scorer from the signals below — not written prose.
Active — committed within the last month; has tests, CI
★ 79 stars · 21 forks · 10 open issues
Python 99% Jinja 1%
Every value records the source it was harvested from.
| Field | Value | Source |
|---|---|---|
| description | Χ-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows? | github_api |
| discovered_conda | not_found | discovery |
| discovered_cran | not_found | discovery |
| discovered_doi | not_found | discovery |
| discovered_hf | actava/chi-bench,actava/managed-care-operations-handbook | discovery |
| discovered_npm | not_found | discovery |
| discovered_pypi | chi-bench | discovery |
| has CI | True | github_api |
| has docs | True | github_api |
| has releases | False | github_api |
| has tests | True | github_api |
| HuggingFace downloads (monthly) | not_found | huggingface |
| judgement | Assess | mach_scorer |
| judgement reason | Assess — committed within 3 months; permissive licence (Apache-2.0). | mach_scorer |
| license | Apache-2.0 | github_api |
| PyPI downloads (monthly) | not_found | pypi |
| archived | False | github_api |
| repository created | 2026-05-07T17:12:15Z | github_api |
| GitHub forks | 21 | github_api |
| repo health | Active | mach_health |
| repo health reason | committed within the last month; has tests, CI | mach_health |
| homepage | https://actava.ai/benchmarks | github_api |
| languages | Python:99,Jinja:1 | github_api |
| last commit | 2026-11-14T02:26:19Z | github_api |
| open issues | 10 | github_api |
| primary language | Python | github_api |
| repo_size_kb | 3461 | github_api |
| GitHub stars | 79 | github_api |
| topics | benchmark,care-management,healthcare-ai,prior-authorization | github_api |
Recorded deviations — when a harvested value changed between runs.
| When | Field | Was | Now | Source |
|---|---|---|---|---|
| 2026-11-20T21:53:44.593669 | repo_health | — | Active | mach_health |
| 2026-11-20T21:53:44.593669 | repo_health_reason | — | committed within the last month; has tests, CI | mach_health |
| 2026-11-20T21:53:44.593577 | judgement | — | Assess | mach_scorer |
| 2026-11-20T21:53:44.593577 | judgement_reason | — | Assess — committed within 3 months; permissive licence (Apache-2.0). | mach_scorer |
| 2026-11-20T21:45:34.913253 | hf_downloads | — | — | huggingface |
| 2026-11-20T21:45:34.856020 | pypi_downloads | — | — | pypi |
| 2026-11-20T21:45:34.855966 | discovered_conda | — | — | discovery |
| 2026-11-20T21:45:34.855966 | discovered_cran | — | — | discovery |
| 2026-11-20T21:45:34.855966 | discovered_doi | — | — | discovery |
| 2026-11-20T21:45:34.855966 | discovered_hf | — | actava/chi-bench,actava/managed-care-operations-handbook | discovery |
| 2026-11-20T21:45:34.855966 | discovered_npm | — | — | discovery |
| 2026-11-20T21:45:34.855966 | discovered_pypi | — | chi-bench | discovery |
| 2026-11-20T21:45:33.099874 | has_ci | — | True | github_api |
| 2026-11-20T21:45:33.099874 | has_docs | — | True | github_api |
| 2026-11-20T21:45:33.099874 | has_releases | — | False | github_api |
| 2026-11-20T21:45:33.099874 | has_tests | — | True | github_api |
| 2026-11-20T21:45:33.099874 | repo_languages | — | Python:99,Jinja:1 | github_api |
| 2026-11-20T21:45:33.099805 | description | — | Χ-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows? | github_api |
| 2026-11-20T21:45:33.099805 | license | — | Apache-2.0 | github_api |
| 2026-11-20T21:45:33.099805 | repo_archived | — | False | github_api |
| 2026-11-20T21:45:33.099805 | repo_created | — | 2026-05-07T17:12:15Z | github_api |
| 2026-11-20T21:45:33.099805 | repo_forks | — | 21 | github_api |
| 2026-11-20T21:45:33.099805 | repo_homepage | — | https://actava.ai/benchmarks | github_api |
| 2026-11-20T21:45:33.099805 | repo_last_commit | — | 2026-11-14T02:26:19Z | github_api |
| 2026-11-20T21:45:33.099805 | repo_open_issues | — | 10 | github_api |
| 2026-11-20T21:45:33.099805 | repo_primary_language | — | Python | github_api |
| 2026-11-20T21:45:33.099805 | repo_size_kb | — | 3461 | github_api |
| 2026-11-20T21:45:33.099805 | repo_stars | — | 79 | github_api |
| 2026-11-20T21:45:33.099805 | repo_topics | — | benchmark,care-management,healthcare-ai,prior-authorization | github_api |