← Back to catalog

Chi-Bench Assess

Benchmark evaluating AI agents on end-to-end, long-horizon, policy-rich healthcare workflow automation. Covers three domains: provider prior authorisation, payer utilisation management, and population care management — 75 single-domain tasks plus 23 end-to-end provider-payer tasks grounded in a 1,279-document Managed Care Operations Handbook and 20 simulated healthcare applications. The best-performing agent (Claude Opus) achieved 28.0% pass@1, reflecting the genuine complexity of real-world healthcare administrative workflows. Code is Apache-2.0; the policy handbook dataset is gated on Hugging Face and requires access approval.

Judgement: Assess

Assess — committed within 3 months; permissive licence (Apache-2.0).

Derived deterministically by the MACH scorer from the signals below — not written prose.

Repository

Active — committed within the last month; has tests, CI

★ 79 stars · 21 forks · 10 open issues

Python 99% Jinja 1%

Signals & lineage

Every value records the source it was harvested from.

FieldValueSource
descriptionΧ-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows?github_api
discovered_condanot_founddiscovery
discovered_crannot_founddiscovery
discovered_doinot_founddiscovery
discovered_hfactava/chi-bench,actava/managed-care-operations-handbookdiscovery
discovered_npmnot_founddiscovery
discovered_pypichi-benchdiscovery
has CITruegithub_api
has docsTruegithub_api
has releasesFalsegithub_api
has testsTruegithub_api
HuggingFace downloads (monthly)not_foundhuggingface
judgementAssessmach_scorer
judgement reasonAssess — committed within 3 months; permissive licence (Apache-2.0).mach_scorer
licenseApache-2.0github_api
PyPI downloads (monthly)not_foundpypi
archivedFalsegithub_api
repository created2026-05-07T17:12:15Zgithub_api
GitHub forks21github_api
repo healthActivemach_health
repo health reasoncommitted within the last month; has tests, CImach_health
homepagehttps://actava.ai/benchmarksgithub_api
languagesPython:99,Jinja:1github_api
last commit2026-11-14T02:26:19Zgithub_api
open issues10github_api
primary languagePythongithub_api
repo_size_kb3461github_api
GitHub stars79github_api
topicsbenchmark,care-management,healthcare-ai,prior-authorizationgithub_api

Change history

Recorded deviations — when a harvested value changed between runs.

WhenFieldWasNowSource
2026-11-20T21:53:44.593669repo_health—Activemach_health
2026-11-20T21:53:44.593669repo_health_reason—committed within the last month; has tests, CImach_health
2026-11-20T21:53:44.593577judgement—Assessmach_scorer
2026-11-20T21:53:44.593577judgement_reason—Assess — committed within 3 months; permissive licence (Apache-2.0).mach_scorer
2026-11-20T21:45:34.913253hf_downloads——huggingface
2026-11-20T21:45:34.856020pypi_downloads——pypi
2026-11-20T21:45:34.855966discovered_conda——discovery
2026-11-20T21:45:34.855966discovered_cran——discovery
2026-11-20T21:45:34.855966discovered_doi——discovery
2026-11-20T21:45:34.855966discovered_hf—actava/chi-bench,actava/managed-care-operations-handbookdiscovery
2026-11-20T21:45:34.855966discovered_npm——discovery
2026-11-20T21:45:34.855966discovered_pypi—chi-benchdiscovery
2026-11-20T21:45:33.099874has_ci—Truegithub_api
2026-11-20T21:45:33.099874has_docs—Truegithub_api
2026-11-20T21:45:33.099874has_releases—Falsegithub_api
2026-11-20T21:45:33.099874has_tests—Truegithub_api
2026-11-20T21:45:33.099874repo_languages—Python:99,Jinja:1github_api
2026-11-20T21:45:33.099805description—Χ-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows?github_api
2026-11-20T21:45:33.099805license—Apache-2.0github_api
2026-11-20T21:45:33.099805repo_archived—Falsegithub_api
2026-11-20T21:45:33.099805repo_created—2026-05-07T17:12:15Zgithub_api
2026-11-20T21:45:33.099805repo_forks—21github_api
2026-11-20T21:45:33.099805repo_homepage—https://actava.ai/benchmarksgithub_api
2026-11-20T21:45:33.099805repo_last_commit—2026-11-14T02:26:19Zgithub_api
2026-11-20T21:45:33.099805repo_open_issues—10github_api
2026-11-20T21:45:33.099805repo_primary_language—Pythongithub_api
2026-11-20T21:45:33.099805repo_size_kb—3461github_api
2026-11-20T21:45:33.099805repo_stars—79github_api
2026-11-20T21:45:33.099805repo_topics—benchmark,care-management,healthcare-ai,prior-authorizationgithub_api