Three things. First, why accountability in AI systems is more complex than in traditional software — using a concrete hospital scenario that reveals the gap clearly. Second, the three core ethical dimensions that apply to every agentic system: privacy and consent, bias and fairness, and transparency. Third, that ethical decisions are not separate from technical decisions — every architecture choice has ethical implications.
The Accountability Gap — A Real Scenario
Scenario
A hospital deploys an agent that reads discharge letters and sends patients a plain-language checklist of follow-up actions. The agent misreads one letter and tells a patient they can stop taking a medication when the letter said the opposite. The patient follows the instruction and is readmitted two weeks later.
When something goes wrong in a system like this, the natural response is to ask who is responsible. In an AI agent system, the answer is genuinely complicated.
The hospital
deployed the agent and chose to use it for patient communication without adequate clinical review.
The developers
built the system knowing the model could hallucinate. They did not build a clinical review step before output delivery.
The AI company
provided the model whose training data and error rate are now operating in a clinical setting.
The patient
trusted the output without checking it against the original letter. Is any blame here? This is genuinely contested.
Key Insight
AI agents make decisions that were not explicitly programmed. When they cause harm, it can be impossible to trace exactly which training data or probabilistic step led to the error. This is not an excuse to avoid accountability — it is a reason to design accountability in from the start.
Ethics of AI - Challenges and Responsibilities | The Knowledge Academy
Three Core Ethical Dimensions
1. Privacy and Consent
Every agent that processes personal data creates a privacy obligation. The questions are: What data does the agent actually need? Who consented to it being used? How long is it retained? Can users request deletion? Privacy is not just about preventing data breaches — it is about using only what you need for only as long as you need it.
The Hospital Data Problem
Hospitals are one of the richest sources of training data for AI systems — patient records, imaging scans, blood test results, clinical notes, and surgical outcomes accumulated over decades. This data is valuable. It is also deeply personal. In many documented cases, patient data has been shared with AI companies for model training without patients being explicitly informed, let alone having given meaningful consent. The argument made is usually that the data was "de-identified" — names and obvious identifiers removed. But re-identification from de-identified medical records has been demonstrated repeatedly in research. A patient's age, postcode, diagnosis, and admission date can be enough to uniquely identify them. When you build an agent that processes patient data — even indirectly — you are operating in this landscape. The consent question is not bureaucratic. It is substantive.
2. Bias and Fairness
LLMs are trained on human-generated data — which contains human biases. An agent trained on English-language medical literature may perform worse for patients whose conditions are underrepresented in that literature. A hiring agent trained on historical data may systematically disadvantage candidates from certain backgrounds. The obligation is to identify which groups your agent affects and test whether its outputs are equitable across them.
3. Transparency
Can a user understand what the agent did and why? If your agent sends a message, makes a recommendation, or takes an action that affects someone — can that person request an explanation? Transparency is the bridge between AI capability and human trust. Without it, even a high-performing agent is difficult to use responsibly in high-stakes settings.
Watch: Crash Course AI — five types of bias explained (12 min)
✏️ Student Reflection
Think about an agent you could imagine building for a domain you care about. Name one specific privacy concern it raises, one potential bias in its data or training, and one transparency question a user might reasonably ask. For each — describe one design decision you could make to address it. Write 2–3 sentences per dimension.
📝 Your Lesson Submission
Share your reflection, ask a question, or leave a comment. Your mentor will respond.
🔐
You need to be logged in to submit your reflection.
Next →2.2 — The Business of AI: How Agentic Tools Create Economic Value🔒 Submit reflection to unlock
🔐
Create a Free Account to Continue
Lesson 1.1 and beyond are available to enrolled members. Join the program for free to access all 21+ lessons, mentor sessions, and your completion certificate.
