← Back to catalog

MedCalc-Bench Assess

Benchmark dataset for evaluating LLM ability to perform clinical calculator tasks — a common but under-tested dimension of medical reasoning. Covers 55 distinct clinical scoring systems and calculators (e.g., SOFA, CHA2DS2-VASc, Wells criteria), with 1,100 test instances and 10,543 training instances generated from real patient notes. NeurIPS 2024 Oral. Developed at NCBI/NLM, NIH.

Judgement: Assess

Assess — last commit ~11 months ago; no clear open licence. Capped: no clear open licence.

Derived deterministically by the MACH scorer from the signals below — not written prose.

Repository

Maintained — last commit ~11 mo ago; has releases

★ 97 stars · 21 forks · 2 open issues

Python 99% Jinja 1%

Signals & lineage

Every value records the source it was harvested from.

FieldValueSource
description[NeurIPS 2024 Datasets and Benchmark Track Oral] MedCalc-Bench: Evaluating Large Language Models for Medical Calculationsgithub_api
discovered_condanot_founddiscovery
discovered_crannot_founddiscovery
discovered_doinot_founddiscovery
discovered_hfnsk7153/MedCalc-Bench-Verifieddiscovery
discovered_npmnot_founddiscovery
discovered_pypinot_founddiscovery
has CIFalsegithub_api
has docsTruegithub_api
has releasesTruegithub_api
has testsFalsegithub_api
HuggingFace downloads (monthly)not_foundhuggingface
judgementAssessmach_scorer
judgement reasonAssess — last commit ~11 months ago; no clear open licence. Capped: no clear open licence.mach_scorer
licenseokgithub_api
PyPI downloads (monthly)61pypi
archivedFalsegithub_api
repository created2024-06-07T18:07:27Zgithub_api
GitHub forks21github_api
repo healthMaintainedmach_health
repo health reasonlast commit ~11 mo ago; has releasesmach_health
homepagehttps://arxiv.org/abs/2406.12036github_api
languagesPython:99,Jinja:1github_api
last commit2025-12-18T20:35:27Zgithub_api
open issues2github_api
primary languagePythongithub_api
repo_size_kb118971github_api
GitHub stars97github_api
topicsokgithub_api

Change history

Recorded deviations — when a harvested value changed between runs.

WhenFieldWasNowSource
2026-11-20T21:53:46.085446repo_health—Maintainedmach_health
2026-11-20T21:53:46.085446repo_health_reason—last commit ~11 mo ago; has releasesmach_health
2026-11-20T21:53:46.085385judgement—Assessmach_scorer
2026-11-20T21:53:46.085385judgement_reason—Assess — last commit ~11 months ago; no clear open licence. Capped: no clear open licence.mach_scorer
2026-11-20T21:45:49.067974hf_downloads——huggingface
2026-11-20T21:45:49.010879pypi_downloads—61pypi
2026-11-20T21:45:49.010824discovered_conda——discovery
2026-11-20T21:45:49.010824discovered_cran——discovery
2026-11-20T21:45:49.010824discovered_doi——discovery
2026-11-20T21:45:49.010824discovered_hf—nsk7153/MedCalc-Bench-Verifieddiscovery
2026-11-20T21:45:49.010824discovered_npm——discovery
2026-11-20T21:45:49.010824discovered_pypi——discovery
2026-11-20T21:45:47.222189has_ci—Falsegithub_api
2026-11-20T21:45:47.222189has_docs—Truegithub_api
2026-11-20T21:45:47.222189has_releases—Truegithub_api
2026-11-20T21:45:47.222189has_tests—Falsegithub_api
2026-11-20T21:45:47.222189repo_languages—Python:99,Jinja:1github_api
2026-11-20T21:45:47.032212description—[NeurIPS 2024 Datasets and Benchmark Track Oral] MedCalc-Bench: Evaluating Large Language Models for Medical Calculationsgithub_api
2026-11-20T21:45:47.032212license——github_api
2026-11-20T21:45:47.032212repo_archived—Falsegithub_api
2026-11-20T21:45:47.032212repo_created—2024-06-07T18:07:27Zgithub_api
2026-11-20T21:45:47.032212repo_forks—21github_api
2026-11-20T21:45:47.032212repo_homepage—https://arxiv.org/abs/2406.12036github_api
2026-11-20T21:45:47.032212repo_last_commit—2025-12-18T20:35:27Zgithub_api
2026-11-20T21:45:47.032212repo_open_issues—2github_api
2026-11-20T21:45:47.032212repo_primary_language—Pythongithub_api
2026-11-20T21:45:47.032212repo_size_kb—118971github_api
2026-11-20T21:45:47.032212repo_stars—97github_api
2026-11-20T21:45:47.032212repo_topics——github_api