When the AI Says "Normal": Who Is Responsible for the Judgment?

Lead

You photograph your baby's diaper and an AI tells you it's "probably normal." You record a cry and the app returns "68% hunger." A growth tracking screen displays the label "within standard range."

When one of those judgments turns out to be wrong, who is accountable?

That question is not a technical one. It is an ethical one. As machine learning has spread through clinical and consumer health applications, researchers have been confronting directly the question of what happens when a person accepts an algorithmic output — who the decision-maker actually is, and where responsibility lies. The parenting app context raises exactly the same questions, and the answers are not as obvious as they might appear.

Between the estimate and the judgment

A 2018 commentary in the New England Journal of Medicine by Char, Shah, and Magnus laid out three ethical challenges in deploying machine learning in clinical practice: bias from unrepresentative training data, lack of explainability, and the erosion of human oversight [1].

The third of these bears directly on parenting AI. When an AI presents an estimate, what is the person receiving that estimate supposed to do with it?

The AI4People initiative, led by Floridi and colleagues, proposed in 2018 adding explicability to the existing four principles of AI ethics (beneficence, non-maleficence, autonomy, and justice) [2]. Explicability means that an AI should be able to make its reasoning understandable to the person relying on it. Without that, the recipient is left with two options: accept the output uncritically or reject it blindly. Neither constitutes a genuine judgment.

In parenting app terms: "probably normal" is not enough. "This color and consistency, at this feeding frequency, is typical for a breast-fed infant — with the exception of white or pale grey stools" is a statement that a parent can actually incorporate into her own assessment. The first output substitutes for judgment; the second supports it.

When a miss happens

If an AI estimate is wrong and a parent acts on it — or fails to act — where does responsibility lie?

The US Food and Drug Administration, in its framework for AI/ML-based SaMD, distinguishes between software that supports human decision-making and software that acts autonomously [3]. In the first category, the human retains final decision-making responsibility. That is an explicit design premise.

Parenting record apps, at least as currently built, fall into the first category. A function that assesses stool color, classifies cry patterns, or evaluates growth trajectory is designed as a reference tool — not a diagnosis, not a prescription. The parent is the decision-maker.

The problem is whether parents actually understand this. Research consistently shows that once someone receives a confident-seeming output from an automated system, they become less likely to notice information that contradicts it — a phenomenon known as automation bias. Goddard and colleagues' systematic review, published in the Journal of the American Medical Informatics Association, documented this effect across 70 studies of clinical decision support systems, finding that it is a reliable feature of human–automation interaction, not an occasional anomaly [5].

The practical consequence: "The app said it was fine, so I didn't call the doctor" is a sentence that represents a real pattern of behavior. Legally and ethically, under current frameworks, the parent who made that decision retains the decision-making responsibility [1, 3]. Whether she had adequate information to understand that she was the decision-maker is a separate and harder question.

Designs that augment judgment versus designs that replace it

Whether AI in a parenting context supports or erodes parental judgment is substantially a product of design choices.

The contrast is straightforward. A system that displays "Normal" transfers nothing to the parent except a verdict. A system that displays the reasoning — what pattern was observed, what it typically indicates, what exceptions should trigger a different response — gives the parent something she can integrate with her own observation.

Topol, in Deep Medicine, argued that AI in medicine should return time to physicians: by automating processing tasks, AI frees clinicians to attend more carefully to the human in front of them [4]. In parenting terms, the parallel would be: AI that handles routine pattern-recognition should free parents to observe their child more attentively, not less. An app whose output causes a parent to stop watching her child in order to watch the screen has failed the design test.

What parents can actually do

The question of AI ethics in parenting apps is not only for the engineers. The user's relationship with the tool matters too.

First, treat AI outputs as one input among several. When an app flags something as within normal range, that assessment is probabilistic, based on a training data set that may or may not resemble your child well. The parent's own observations — is the baby lethargic? feeding adequately? otherwise acting like herself? — remain valid and independent information. An AI calling something normal does not invalidate a parent's sense that something is off.

Second, use the app as preparation for consultation, not as a substitute for it. If you're considering whether to call your pediatrician, the appropriate use of an AI assessment is to inform what you describe in that call — not to resolve whether the call is necessary. The app was not designed to make the latter judgment.

Third, know what the tool is. None of the AI features in consumer parenting apps are approved medical devices, and none are designed to function as clinical diagnostic tools. They are pattern-matching systems built on aggregate data. The gap between "statistically common for this age group" and "appropriate for this child right now" is the gap where parental judgment lives.

Summary

Under current ethical and regulatory frameworks, when an AI offers an estimate and a parent acts on it, responsibility remains with the parent who made the final call [1, 3]. That is a coherent framework — but it rests on the assumption that the parent understood she was the decision-maker, and that the tool was designed to support rather than replace her judgment.

AI developers carry the responsibility to make reasoning visible and to design for augmentation rather than substitution. Parents carry the responsibility to keep their own observation in the loop. In a domain where errors can carry serious consequences, neither side of that responsibility can be delegated away.

References

1. Char DS, Shah NH, Magnus D. Implementing machine learning in health care — addressing ethical challenges. N Engl J Med. 2018;378(11):981–983. doi:10.1056/NEJMp1714229. PMID: 29539284.
2. Floridi L, Cowls J, Beltrametti M, et al. AI4People — an ethical framework for a good AI society: opportunities, risks, principles, and recommendations. Minds Mach. 2018;28(4):689–707. doi:10.1007/s11023-018-9482-5. PMID: 30930541.
3. US Food and Drug Administration. Artificial Intelligence/Machine Learning (AI/ML)-based Software as a Medical Device (SaMD) Action Plan. FDA; 2021. Updated 2023. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-software-medical-device
4. Topol EJ. Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again. Basic Books; 2019.
5. Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc. 2012;19(1):121–127. doi:10.1136/amiajnl-2011-000089. PMID: 21685142.
6. American Academy of Pediatrics Council on Quality Improvement and Patient Safety. Pediatric clinical decision support: guiding principles. Pediatrics. 2024;153(2):e2023062505. doi:10.1542/peds.2023-062505. [unverified]