Benchmark dataset for evaluating LLM ability to perform clinical calculator tasks — a common but under-tested dimension of medical reasoning. Covers 55 distinct clinical scoring systems and calculators (e.g., SOFA, CHA2DS2-VASc, Wells criteria), with 1,100 test instances and 10,543 training instances generated from real patient notes. NeurIPS 2024 Oral. Developed at NCBI/NLM, NIH.
Assess — last commit ~11 months ago; no clear open licence. Capped: no clear open licence.
Derived deterministically by the MACH scorer from the signals below — not written prose.
Maintained — last commit ~11 mo ago; has releases
★ 97 stars · 21 forks · 2 open issues
Python 99% Jinja 1%
Every value records the source it was harvested from.
| Field | Value | Source |
|---|---|---|
| description | [NeurIPS 2024 Datasets and Benchmark Track Oral] MedCalc-Bench: Evaluating Large Language Models for Medical Calculations | github_api |
| discovered_conda | not_found | discovery |
| discovered_cran | not_found | discovery |
| discovered_doi | not_found | discovery |
| discovered_hf | nsk7153/MedCalc-Bench-Verified | discovery |
| discovered_npm | not_found | discovery |
| discovered_pypi | not_found | discovery |
| has CI | False | github_api |
| has docs | True | github_api |
| has releases | True | github_api |
| has tests | False | github_api |
| HuggingFace downloads (monthly) | not_found | huggingface |
| judgement | Assess | mach_scorer |
| judgement reason | Assess — last commit ~11 months ago; no clear open licence. Capped: no clear open licence. | mach_scorer |
| license | ok | github_api |
| PyPI downloads (monthly) | 61 | pypi |
| archived | False | github_api |
| repository created | 2024-06-07T18:07:27Z | github_api |
| GitHub forks | 21 | github_api |
| repo health | Maintained | mach_health |
| repo health reason | last commit ~11 mo ago; has releases | mach_health |
| homepage | https://arxiv.org/abs/2406.12036 | github_api |
| languages | Python:99,Jinja:1 | github_api |
| last commit | 2025-12-18T20:35:27Z | github_api |
| open issues | 2 | github_api |
| primary language | Python | github_api |
| repo_size_kb | 118971 | github_api |
| GitHub stars | 97 | github_api |
| topics | ok | github_api |
Recorded deviations — when a harvested value changed between runs.
| When | Field | Was | Now | Source |
|---|---|---|---|---|
| 2026-11-20T21:53:46.085446 | repo_health | — | Maintained | mach_health |
| 2026-11-20T21:53:46.085446 | repo_health_reason | — | last commit ~11 mo ago; has releases | mach_health |
| 2026-11-20T21:53:46.085385 | judgement | — | Assess | mach_scorer |
| 2026-11-20T21:53:46.085385 | judgement_reason | — | Assess — last commit ~11 months ago; no clear open licence. Capped: no clear open licence. | mach_scorer |
| 2026-11-20T21:45:49.067974 | hf_downloads | — | — | huggingface |
| 2026-11-20T21:45:49.010879 | pypi_downloads | — | 61 | pypi |
| 2026-11-20T21:45:49.010824 | discovered_conda | — | — | discovery |
| 2026-11-20T21:45:49.010824 | discovered_cran | — | — | discovery |
| 2026-11-20T21:45:49.010824 | discovered_doi | — | — | discovery |
| 2026-11-20T21:45:49.010824 | discovered_hf | — | nsk7153/MedCalc-Bench-Verified | discovery |
| 2026-11-20T21:45:49.010824 | discovered_npm | — | — | discovery |
| 2026-11-20T21:45:49.010824 | discovered_pypi | — | — | discovery |
| 2026-11-20T21:45:47.222189 | has_ci | — | False | github_api |
| 2026-11-20T21:45:47.222189 | has_docs | — | True | github_api |
| 2026-11-20T21:45:47.222189 | has_releases | — | True | github_api |
| 2026-11-20T21:45:47.222189 | has_tests | — | False | github_api |
| 2026-11-20T21:45:47.222189 | repo_languages | — | Python:99,Jinja:1 | github_api |
| 2026-11-20T21:45:47.032212 | description | — | [NeurIPS 2024 Datasets and Benchmark Track Oral] MedCalc-Bench: Evaluating Large Language Models for Medical Calculations | github_api |
| 2026-11-20T21:45:47.032212 | license | — | — | github_api |
| 2026-11-20T21:45:47.032212 | repo_archived | — | False | github_api |
| 2026-11-20T21:45:47.032212 | repo_created | — | 2024-06-07T18:07:27Z | github_api |
| 2026-11-20T21:45:47.032212 | repo_forks | — | 21 | github_api |
| 2026-11-20T21:45:47.032212 | repo_homepage | — | https://arxiv.org/abs/2406.12036 | github_api |
| 2026-11-20T21:45:47.032212 | repo_last_commit | — | 2025-12-18T20:35:27Z | github_api |
| 2026-11-20T21:45:47.032212 | repo_open_issues | — | 2 | github_api |
| 2026-11-20T21:45:47.032212 | repo_primary_language | — | Python | github_api |
| 2026-11-20T21:45:47.032212 | repo_size_kb | — | 118971 | github_api |
| 2026-11-20T21:45:47.032212 | repo_stars | — | 97 | github_api |
| 2026-11-20T21:45:47.032212 | repo_topics | — | — | github_api |