Toward Computational Assessment of
Scientific Justification
Across Biomedical Research Lifecycles
1Centre for Biomedical Ethics, National University of Singapore 2Center for Bioethics, Harvard Medical School 3The Hastings Center for Bioethics
Journal of Medical Artificial Intelligence 2026;9:52 ·
doi:10.21037/jmai-2025-1-246
Received 18 November 2025 · Accepted 12 May 2026 · Published online 22 June 2026
Key findings
- Propose a multi-dimensional rubric for structured evaluation of scientific justification across biomedical research lifecycles.
- Develop an end-to-end computational architecture for executing the rubric and implement it in SciJust.AI, a prototype we make available for testing and feedback. This allows for both manual and artificial intelligence (AI)-aided use, utilizing large language models (LLMs) to evaluate research inputs to produce structured justification evaluation outputs.
- This includes 67 unique criteria scores, across seven levels and seven components of scientific justification along the biomedical research lifecycle, as well as generated explanations for the score of each. These are presented in an interactive interface that displays each score organized by lifecycle stage, along with dynamically updated visual plots and score tables. These materials are additionally available for user download as structured report.
- We include a set of illustrative evaluations in aim of generating discussion and gathering feedback for future work and validation trials.
Abstract
Background: Scientific justification is the foundation of institutional science, ensuring that research and experimentation are pursued on the basis of knowledge contribution, methodological rigor, ethical responsibility, and public benefit. In biomedicine, research investigators, academic departments, funders, and oversight bodies including Institutional Review Boards (IRBs), Institutional Animal Care and Use Committees (IACUCs), Scientific Review Boards (SRBs), and Stem Cell Research Oversight (SCRO) committees each take part in this process—yet evidence shows wide variability in how justification standards are applied across biomedical research lifecycles, with significant consequences for the practice of the scientific method and public commitments to it. Accordingly, this study aims to develop a more precise and practically applicable account of scientific justification.
Methods: This study explores whether computational tools can assist in structuring and evaluating aspects of scientific justification in biomedical research. We propose a categorized and quantified rubric for evaluating justification across research lifecycles, broken down into seven stages of experimental and institutional operation, each assessed against seven core components of justificatory relevance. We develop a data architecture and processing pipeline for computing this in a parameterized schema with prompt-based workflows.
Results: This computable framework is implemented in SciJust.AI, an artificial intelligence (AI)-enabled prototype that uses large language models (LLMs) to score research-related inputs, generate structured feedback, and visualize justification profiles. Constructed to support rather than replace human judgment, SciJust.AI advances a model in which investigators and reviewers can be equipped with an experimental assessment tool that could be used at key stages, or on an ad hoc basis, throughout the design, execution, and evaluation of biomedical research—with the aim of promoting greater clarity, consistency, and comparability from protocol to practice. Both manual and AI-assisted evaluation are made publicly available for early-stage testing, and we present illustrative example evaluations with the goal of generating discussion and gathering feedback for future improvements and validation efforts.
Conclusions: By reframing scientific justification as a measurable, improvable dimension of the scientific method, this approach puts forward a practical path for better aligning experimental rigor with confidence in results.
Keywords: Scientific justification; biomedical research lifecycle; artificial intelligence (AI); AI for Science
What should change now
The instrument
| SVR | FRU | IIP | EIP | APP | TAC | SRT | Σ | |
|---|---|---|---|---|---|---|---|---|
| Σ componentΣ | 95 | 95 | 96 | 118 | 101 | 93 | 102 | 700 |
Conception Initial Research Conceptualization · Approval Ethical and Regulatory Approval · Funding Funding and Resource Allocation · Trials Preclinical and Clinical Trials · Data Data Collection and Analysis · Publication Peer Review and Publication · Policy Public Communication and Policy Impact
Figure 1 of the paper, generated here from the same rubric the applications
score against. Points available at each level × component intersection, shaded by weight
relative to the heaviest cell — these are maxima, not scores. The distribution is deliberately
non-uniform — ethical integrity carries regulatory approval (33), feasibility carries funding
(40), and scientific rigor carries both conceptualization and peer review (24 each).
The paper is explicit that these maxima are preliminary, “representing possible rather
than validated importance of different components per level” — a theoretically motivated starting
point for demonstrating the method. Identifying weights appropriate to institutional use “would
require extensive stakeholder deliberation and iteration”. Only the
high-level component totals are shown here; each cell decomposes further into the individual
criteria the tools score.
Components of justification
Aggregate rating bands
The paper ascribes these high-level judgements to the aggregate score, and is careful about their standing: they “are intended as descriptive orientation aids and should be interpreted in the context of the full criterion-level breakdown rather than as standalone verdicts”. The criterion-level profile, not the band, is the object of interest.
Applications
The instrument as a working tool for use across the lifecycle, by investigators and reviewers alike. Scores a study stage by stage in lifecycle order — each stage informed by the ones already scored — streaming the justification matrix as it goes. Includes rubric authoring, reviewer adjustments, iteration tracing across protocol revisions, run comparison, projects and sharing.
→ Open the web application at SciJust.AIThe prototype described in the paper. Supports manual self-evaluation criterion by criterion as well as AI-assisted scoring of entered text, uploaded documents or URLs, with a dynamically updated justification plot, score table and downloadable report.
↗ Open the v1 prototypeMethods
What does the instrument actually score?
A protocol — typed text, an uploaded document, or a URL — is presented to a language model together with the rubric, and the model returns a score and a written justification for every one of the 67 criteria. Each explanation states why the score was given and what would raise it, so the output is a set of specific, addressable findings. Scores aggregate to per-level and per-component profiles and to the 700-point total.
Why score the lifecycle sequentially?
The levels are the stages of the research process in temporal order. The web application evaluates them one after another, each stage's prompt carrying a summary of the stages already scored, followed by a synthesis pass over the whole rubric. This lets later stages be assessed in light of what earlier ones established, and lets results populate live as the evaluation proceeds. Single-pass evaluation, in one holistic call, is also available.
Human oversight
The instrument supports the deliberative process and the applications are built to keep it that way. A reviewer can rescore any criterion and attach a note; the model's score and its reasoning are never overwritten, and both values plus the delta travel through every view and export. Every run records an audit trail — model, decoding settings, prompt hash, raw response — and freezes a copy of the rubric version it was scored against, so a result stays reproducible and an edited rubric cannot silently change what an earlier evaluation meant.
Can the rubric itself be changed?
Yes. The published rubric is the built-in default and is read-only, but it can be duplicated and edited, and new rubrics can be generated for other research-lifecycle purposes — clinical trials, preclinical and animal research, clinical AI/ML, public health, grant review, data governance. Generation can proceed layer by layer, designing the stages and components first and then each stage's criteria in lifecycle order, so the instrument is built the same way it is applied. Because scores are only commensurable within one rubric version, every evaluation is stamped with a structural hash of the instrument that produced it, and comparisons across versions are flagged as such.
Which models run it?
Nine models across three providers are selectable — OpenAI (GPT-5.1, GPT-5-mini), Anthropic (Claude Fable 5, Opus 5, Sonnet 5, Haiku 4.5), and Google (Gemini 2.5 Pro, Flash, Flash-Lite) — with structured output enforced at the API level in each case. Deterministic-capable models can be run at temperature 0 with a fixed seed. Because the same protocol can be scored by different models under the same rubric, the tool can be used to examine where models agree and where they diverge, criterion by criterion.
Limitations
Stated in the paper
AI-assisted scoring depends on both the precision of the rubric and the model's capacity to interpret research inputs, and the paper notes the balance is delicate: “overly rigid evaluative criteria might risk stifling methodological innovation, just as overly broad criteria may leave inconsistency unresolved”. Model outputs may reflect biases embedded in training data or in rubric design, which is precisely why human oversight is retained.
Run-to-run variation may occur in scoring. The paper's guidance is that “repeated runs should be conducted and comparatively tracked to identify broad evaluation patterns”, and it names multi-run comparability as “a top priority in future work of developing the prototype toward product”.
Finally, it flags the risk its own outputs carry: normalized level scores, a 0–700 aggregate, and interpretive labels “could be treated as shorthand for … reductive thinking or even decision-making”. The rubric does not resolve the incommensurability of values across justification factors; it makes the appeal to them explicit.
Multi-run comparability
The paper names multi-run comparability as a priority for further development, and the web application implements it: any evaluation can be re-run or scored by a different model under the same rubric, and two to four runs can be compared side by side down to the individual criteria where they disagree, with each run's reasoning shown. Protocol revisions form an iteration chain carrying the score trace across versions. Rubric editing and generation follow the paper's point that “the specific scoring breakdowns and the criteria that comprise different components … can be updated and adapted”, and every run is stamped with the rubric version that produced it, so an adaptation leaves earlier evaluations legible as what they were.
Cite this work
@article{ghose2026scijust,
title = {Toward computational assessment of scientific justification
across biomedical research lifecycles},
author = {Ghose, Sankalpa and Hyun, Insoo},
journal = {Journal of Medical Artificial Intelligence},
volume = {9},
pages = {52},
year = {2026},
doi = {10.21037/jmai-2025-1-246},
note = {Applications: https://scijust.ai}
}