Standardised evaluation framework for oncology foundation models, covering both pathology and radiology. Benchmarks patch-level, slide-level, and scan-level classification, 2D/3D semantic segmentation, and visual question answering (VQA) tasks across multiple foundation model encoders. Supports offline (pre-extracted embeddings) and online (end-to-end) evaluation modes, enabling reproducible comparison of FMs such as UNI, CONCH, Virchow, and others on standardised task suites. Fills the evaluation layer in the pathology AI stack that sits alongside TRIDENT (feature extraction) and the foundation models themselves. Apache-2.0, 156 stars, v0.4.5 (April 2026). Published at MIDL 2024. Developed by kaiko.ai.
Assess — committed within 3 months; permissive licence (Apache-2.0).
Derived deterministically by the MACH scorer from the signals below — not written prose.
Active — last commit ~3 mo ago; has tests, CI, releases
★ 176 stars · 42 forks · 33 open issues
Python 100%
Every value records the source it was harvested from.
| Field | Value | Source |
|---|---|---|
| description | Evaluation framework for oncology foundation models (FMs) | github_api |
| discovered_conda | not_found | discovery |
| discovered_cran | not_found | discovery |
| discovered_doi | not_found | discovery |
| discovered_hf | not_found | discovery |
| discovered_npm | not_found | discovery |
| discovered_pypi | kaiko-eva | discovery |
| has CI | True | github_api |
| has docs | True | github_api |
| has releases | True | github_api |
| has tests | True | github_api |
| HuggingFace downloads (monthly) | not_found | huggingface |
| judgement | Assess | mach_scorer |
| judgement reason | Assess — committed within 3 months; permissive licence (Apache-2.0). | mach_scorer |
| license | Apache-2.0 | github_api |
| PyPI downloads (monthly) | 702 | pypi |
| archived | False | github_api |
| repository created | 2024-01-16T12:05:03Z | github_api |
| GitHub forks | 42 | github_api |
| repo health | Active | mach_health |
| repo health reason | last commit ~3 mo ago; has tests, CI, releases | mach_health |
| homepage | https://kaiko-ai.github.io/eva/ | github_api |
| languages | Python:100 | github_api |
| last commit | 2026-09-02T04:34:16Z | github_api |
| open issues | 33 | github_api |
| primary language | Python | github_api |
| repo_size_kb | 18077 | github_api |
| GitHub stars | 176 | github_api |
| topics | evaluation-framework,foundation-models,machine-learning,oncology,pathology,radiology | github_api |
Recorded deviations — when a harvested value changed between runs.
| When | Field | Was | Now | Source |
|---|---|---|---|---|
| 2026-11-20T21:53:45.156584 | repo_health | — | Active | mach_health |
| 2026-11-20T21:53:45.156584 | repo_health_reason | — | last commit ~3 mo ago; has tests, CI, releases | mach_health |
| 2026-11-20T21:53:45.156481 | judgement | — | Assess | mach_scorer |
| 2026-11-20T21:53:45.156481 | judgement_reason | — | Assess — committed within 3 months; permissive licence (Apache-2.0). | mach_scorer |
| 2026-11-20T21:48:34.144089 | hf_downloads | — | — | huggingface |
| 2026-11-20T21:48:34.087193 | pypi_downloads | — | 702 | pypi |
| 2026-11-20T21:48:34.087140 | discovered_conda | — | — | discovery |
| 2026-11-20T21:48:34.087140 | discovered_cran | — | — | discovery |
| 2026-11-20T21:48:34.087140 | discovered_doi | — | — | discovery |
| 2026-11-20T21:48:34.087140 | discovered_hf | — | — | discovery |
| 2026-11-20T21:48:34.087140 | discovered_npm | — | — | discovery |
| 2026-11-20T21:48:34.087140 | discovered_pypi | — | kaiko-eva | discovery |
| 2026-11-20T21:48:32.435600 | has_ci | — | True | github_api |
| 2026-11-20T21:48:32.435600 | has_docs | — | True | github_api |
| 2026-11-20T21:48:32.435600 | has_releases | — | True | github_api |
| 2026-11-20T21:48:32.435600 | has_tests | — | True | github_api |
| 2026-11-20T21:48:32.435600 | repo_languages | — | Python:100 | github_api |
| 2026-11-20T21:48:32.435530 | description | — | Evaluation framework for oncology foundation models (FMs) | github_api |
| 2026-11-20T21:48:32.435530 | license | — | Apache-2.0 | github_api |
| 2026-11-20T21:48:32.435530 | repo_archived | — | False | github_api |
| 2026-11-20T21:48:32.435530 | repo_created | — | 2024-01-16T12:05:03Z | github_api |
| 2026-11-20T21:48:32.435530 | repo_forks | — | 42 | github_api |
| 2026-11-20T21:48:32.435530 | repo_homepage | — | https://kaiko-ai.github.io/eva/ | github_api |
| 2026-11-20T21:48:32.435530 | repo_last_commit | — | 2026-09-02T04:34:16Z | github_api |
| 2026-11-20T21:48:32.435530 | repo_open_issues | — | 33 | github_api |
| 2026-11-20T21:48:32.435530 | repo_primary_language | — | Python | github_api |
| 2026-11-20T21:48:32.435530 | repo_size_kb | — | 18077 | github_api |
| 2026-11-20T21:48:32.435530 | repo_stars | — | 176 | github_api |
| 2026-11-20T21:48:32.435530 | repo_topics | — | evaluation-framework,foundation-models,machine-learning,oncology,pathology,radiology | github_api |