Lesson 5 of 6 · 49 min

Responsible & accessible AI design

Responsibility is a front-page concern, not a footer; accessibility is the gate for probabilistic UIs, not an after-market patch. IBM’s six pillars and Carbon’s shipped AI-label component, the probabilistic-UI accessibility checklist, the bias mechanism (Amazon’s scrapped recruiter), and the data-rights and human-escape-hatch moves — grounded in alt-text generation’s win-and-limit and ChatGPT’s screen-reader regressions.

Two failures that compound

Responsible AI design is not a suffix you add at the end — IBM puts six pillars (Everyday ethics, Accountability, Value alignment, Explainability, Fairness, User data rights) on the front page of its Design for AI hub, not in an appendix. And accessibility for probabilistic UIs has a vicious property: two failures compound. Accessibility: AI-generated content (alt text, captions) opens new opportunities and new regressions — if a model trains on under-described data, its output is under-described too. Responsibility: opaque automation shifts blame from user to vendor; users can’t audit what they never saw. The intersection is the worst case: a screen-reader user relying on AI-generated alt text is most exposed exactly when the model hallucinates. This lesson is how you make responsibility a default, not a patch.

The mechanism behind responsible-design failures is that opacity transfers accountability. When an automation acts silently, the user can’t see what it did, so they can’t be responsible for it — and neither, in practice, is anyone, until a tribunal assigns it (Air Canada). The design response is to put accountability in the product surface: link every AI action to a “who is responsible” indicator — model name, training cutoff, vendor — and bake the six pillars into the spec template, not the PR template. Interview angle. “Whose responsibility is a wrong AI output?” A strong answer treats it as a design question: surface provenance and accountability in the UI so the user can audit, rather than treating responsibility as a legal afterthought.

Accessibility and inclusive designBryce Johnson

Accessibility for probabilistic UIs — the checklist

Accessibility for AI is doubly fragile. The interface often fails standard WCAG conformance — and the output can violate WCAG in non-obvious ways. The breakthrough realization for an AI designer is that the new UI primitives — confidence, regenerate, streaming tokens, citations — each create new accessibility requirements that a normal a11y pass doesn’t cover. You can’t bolt accessibility on after; it has to be in how you spec each probabilistic component.

code

1PROBABILISTIC-UI ACCESSIBILITY CHECKLIST (designer-facing)23  Concern                 WCAG ref          Requirement4  ----------------------  ----------------  ---------------------------------------5  Confidence disclosure   1.3.1, 4.1.2      label as READABLE TEXT, not color alone6  Regenerate / undo ctrl  2.1.1, 4.1.2      keyboard-focusable + accessible NAME7  Conversation turns      1.3.2, 2.4.6      exposed as a logical list, not just8                                            visual bubbles9  Source citations        1.1.1            chip announces "Citation, link, verify"10  Low-confidence / error  3.3.1            aria-live (polite/assertive) on the update11  Streaming tokens        2.2.2, 2.3.1      respect prefers-reduced-motion; pausable12  Generated alt text      1.1.1, 3.1.1      declare language; include what AI claimed13                                            AND how confident1415  The new AI primitives each add an a11y requirement -- spec them per component,16  don't rely on the model (or a late audit) to be accessible by default.

The two failure cases keep this concrete. The win: tools like AltText.ai auto-generate WCAG-compliant alt text “in 130+ languages,” solving the manual-authoring bottleneck — screen-reader users now get descriptions that often didn’t exist before. The limit: generated alt text is a downstream patch; the primary UI still has to be readable. The OpenAI community documented “major screen reader accessibility regressions in the latest iOS ChatGPT app” — controls to attach content, type, and send became inaccessible. The same company that publishes a GPT for “Learn NVDA” shipped a UI screen readers couldn’t parse. WCAG compliance for AI products is mostly a markup, focus, and label discipline problem that AI tooling cannot fix after the fact. Generated alt text should also carry a “report incorrect alt” affordance, because a confident wrong description is worse than none for the user who can’t see the image.

Bias: a fairness failure designers can surface

Bias is the fairness pillar’s sharp edge, and the canonical case is Amazon’s scrapped AI recruiting tool: trained on a decade of resumes from a male-dominated applicant pool, it learned to penalize resumes containing “women’s” (as in “women’s chess club captain”) and downgrade graduates of two all-women’s colleges. Amazon scrapped it. The mechanism every designer should be able to state: a model trained on biased historical data reproduces and amplifies that bias, and a fluent, confident UI launders it — the output looks neutral and authoritative, so the bias is invisible at the point of use. Stanford HAI’s 2025 work shows AI hiring tools can still “yield racial bias and systemic rejection.”

A designer can’t fix the training data, but the interaction design is not powerless: surface which factors influenced a decision (PAIR’s “explainable factors”), make outcomes auditable rather than opaque, provide a contest/override path, and avoid UI that implies a probabilistic ranking is an objective verdict. Inclusive design goes further upstream — design with the affected population, test across the long tail, and treat the edge cases (names, dialects, disabilities, non-English) as first-class, not as a fairness afterthought. Interview angle. “Your AI feature might be biased — what do you do as the designer?” A strong answer names the data-bias mechanism, then proposes surfacing influencing factors, an audit trail, a contest path, and inclusive testing across the long tail — not “we’ll fix the model.”

Everyday ethics: the visible constitution

IBM’s sixth pillar — “everyday ethics” — and Anthropic’s Constitutional AI point at the same designer move: make your product’s AI values inspectable rather than a hidden style guide. Constitutional AI trains models on “a detailed description of … intentions for the model’s values and behavior,” and the point for a designer is that an explicit, composed, auditable set of value statements beats a pile of one-off behavior overrides nobody can see or reason about. Microsoft’s Responsible AI principles (fairness, reliability & safety, privacy & security, inclusiveness, transparency, accountability) play the same role at the org level. The takeaway: write a visible constitution for your product’s AI behavior — what it will refuse, whose interests it prioritizes, what it discloses — and surface it where users and reviewers can read it.

This is also where the four frameworks’ divergences matter, not just their agreement. On reasoning visibility, PAIR says “explain for understanding, not completeness,” o1 hides reasoning, and Claude shows it — there is no consensus, so you must choose per product. On labeling, NN/g critiques the sparkle while IBM ships a system-level AI label. On confidence, PAIR prefers numeric scores while IBM’s generative principles lean toward textual hedging. The mature stance is to treat the frameworks as complements with real tensions — adopt HAX as your audit checklist, PAIR as your experience recipe, NN/g as your anti-pattern red-team, and Carbon as your component source — and to document which side of each tension your product took and why. That documented choice is itself an everyday-ethics artifact.

Labeling, data rights, and the human escape hatch

Three responsible-design surfaces round out the lesson. Labeling: NN/g critiques the ubiquitous sparkles icon as ambiguity-prone — it signals “magic,” not “AI-generated, may be wrong.” IBM’s Carbon for AI ships a system-level alternative: an actual AI label component that is legible, accessible, and structurally tied to the system, with distinct visual behavior for AI vs human-generated content. The lesson: adopt or build an AI-label design-system primitive — never reinvent it per feature, and don’t rely on a sparkle to carry the meaning. (Note the honest tension: arXiv research finds “labeling messages as AI-generated does not reduce persuasiveness,” so labeling is necessary for transparency but is not, by itself, a safety control — move from page-level to element-level labeling by risk.)

Data rights: regulation now makes this a design constraint, not a legal-team problem — California’s AB 2013 requires generative-AI developers to publicly post a “high-level summary of the datasets used,” and the EU AI Act (Art. 13) mandates transparency for high-risk systems. The product move is an in-app, discoverable “How was I trained?” link next to the confidence signal, plus a per-user, per-feature data-port and opt-out flow (Microsoft documents deleting your Microsoft 365 Copilot activity history) — made findable in-app, not buried in legal docs. The human escape hatch: the single most reliable responsible-design pattern from the Air Canada lesson — always offer “talk to a human,” especially when confidence is low or stakes are high. And for any feedback that becomes training data, require explicit opt-in, made visible in the UI.

code

1RESPONSIBLE-DESIGN SURFACES the user can actually reach23  AI label (not a bare sparkle):4     [ AI ]  Generated by , trained through    [ how it works ]5              ^ legible text, distinct from human content, screen-reader announced67  Data rights, in-app (not buried in a privacy PDF):8     "How was I trained?"   ->  dataset summary (AB 2013)9     "Delete my AI activity" /  "Don't use my edits to train"  [ explicit opt-in ]1011  Accountability + escape hatch, at the point of action:12     "I'm not confident here."   ->  [ Talk to a human ]   (low conf / high stakes)13     every AI action -> who's responsible: model name + cutoff + vendor

IBM’s Carbon for AI is the structural model for all of this — it ships the AI label, accessibility-aware chat components, and an explicit “AI explainability” pattern, so responsibility is encoded in reusable components rather than re-invented per product. That is what “responsible design as a default” looks like: not a principle in a deck, but a primitive in the design system. Anthropic’s Constitutional AI offers the same idea at the behavior layer — an inspectable, composed set of value statements rather than a hidden style guide. The designer takeaway: build a visible constitution for your product’s AI behavior, and ship the responsibility UI as components.

Responsibility is a primitive, not a principle. The teams that ship it as components — an AI label, an accessible regenerate, a “talk to a human,” a data-rights link — are the ones whose probabilistic UIs are responsible by default rather than by audit.

Interview & portfolio prep

Responsibility and accessibility show up in portfolio walkthroughs (“how did you incorporate accessibility?” is a verbatim Google IXD question), the design-challenge round, and behavioral “responsible AI” rounds. The signal: you treat both as front-loaded design constraints with concrete component-level moves, and you can name the bias and accessibility mechanisms. Drill these.

01“How do you make a probabilistic UI accessible?” → spec each new primitive: confidence as readable text (not color), regenerate/undo keyboard-focusable with a name, conversation as a logical list, aria-live on low-confidence/errors, prefers-reduced-motion on streaming.
02“Whose responsibility is a wrong AI output?” → a design question: surface provenance + accountability (model, cutoff, vendor) in the UI so it’s auditable; opacity transfers blame and helps no one.
03“Your AI feature might be biased — what do you do?” → name the data-bias mechanism (Amazon recruiter), then surface explainable factors, an audit trail, a contest/override path, and inclusive testing across the long tail.
04“What’s wrong with the sparkles icon?” → it signals magic, not ‘AI-generated, may be wrong’ (NN/g); adopt a system-level AI-label primitive (Carbon), label by risk at the element level.
05“How do you handle AI-generated alt text?” → a real win at scale (130+ languages) but a downstream patch; the primary UI must still be readable, declare language, and offer a ‘report incorrect alt’ affordance.
06“Does labeling content as AI make it safe?” → no — labeling aids transparency but doesn’t reduce persuasiveness (arXiv); it’s necessary, not sufficient. Pair with verifiability and an escape hatch.
07“How do you address data rights in the UI?” → discoverable in-app ‘How was I trained?’ (AB 2013 dataset summary) + per-feature opt-out and delete-activity flow + explicit opt-in for training on user data.
08“What’s the most reliable responsible-design pattern?” → the human escape hatch — ‘talk to a human’ on low confidence / high stakes (the Air Canada lesson).

Follow-ups go deep on craft and evidence: “How would you test the accessibility of an AI chat surface?” (screen readers — NVDA/JAWS/VoiceOver — plus keyboard-only and high-contrast passes on each new primitive; don’t trust the model to be accessible). “Show me where responsibility lives in your portfolio piece.” A standout portfolio piece treats accessibility and responsibility as designed-in, not bolted-on: it shows the AI-label component, the accessible confidence/regenerate treatment, the contest/data-rights path, and ideally inclusive-testing evidence across the long tail — exactly the “design for AI, documenting iterations and decisions” bar that strong AI portfolios hit.

docsCarbon for AI (the AI label, accessibility, AI explainability components)IBM Carbon Design System articleHow WCAG Guidelines Apply to AI-Generated ContentAudioEye articleAmazon scraps secret AI recruiting tool that showed bias against womenReuters articleDesign Principles for Generative AI Applications (IBM’s mental-models + responsibility principles)IBM Design

Checkpoint

Your AI assistant shows answer confidence as a green / amber / red colored dot. An accessibility reviewer flags it. What’s the core problem and fix?

AConfidence is an accessibility element: encode it as readable text (e.g. “Low confidence — verify”), not color alone, so screen-reader and low-vision users get the signal tooBNo problem — colored dots are a common, recognizable patternCMake the dots larger so they’re easier to seeDRemove the confidence indicator to avoid the accessibility issue

Checkpoint

You ship AI-generated alt text for user-uploaded images to improve accessibility. What’s the most important accompanying design decision?

ANothing more is needed — auto-generated alt text makes the product WCAG-compliantBTreat it as a patch, not a guarantee: declare the content language, keep the surrounding UI fully accessible, and add a “report incorrect alt” affordance so users can flag hallucinated descriptionsCGenerate the alt text only in English to keep it consistentDHide the alt text from sighted users so the UI stays clean

Checkpoint

Your team is adding an AI-assisted candidate-screening feature. Given Amazon’s scrapped biased recruiter, what’s the strongest design-side safeguard you can own?

ATrust the model vendor’s fairness claims and shipBMake the ranking look more authoritative so recruiters act decisivelyCSurface the factors that influenced each ranking, make outcomes auditable, provide a contest/override path, and push for inclusive testing across the long tail — so bias is visible and challengeable at the point of useDRemove the candidates’ names to eliminate bias entirely

Checkpoint

A regulator asks how your AI product handles transparency and user data rights. Which in-product design satisfies the intent best?

AA detailed privacy policy PDF linked from the website footerBAn “AI” sparkle icon on every generated elementCNothing in-product — compliance is handled by the legal team’s documentationDA discoverable in-app “How was I trained?” (dataset summary) next to the AI’s output, plus a findable per-feature opt-out / delete-activity flow and explicit opt-in to train on user data

Checkpoint

You’re labeling AI-generated content in a high-stakes product. A teammate says “a sparkle icon and a banner saying ‘AI-generated’ will make it safe.” Best response?

AAgree — labeling content as AI-generated makes users appropriately skeptical, so it’s safeBLabeling is necessary for transparency but not sufficient for safety: adopt a legible, accessible AI-label primitive (not a bare sparkle), label by risk at the element level, and pair it with verifiability and a human escape hatchCSkip labeling entirely since it doesn’t reduce persuasivenessDUse the sparkle icon but make it bigger and brighter

Could you make a probabilistic UI accessible component-by-component, surface bias and accountability in the interface, and defend data-rights and escape-hatch design in a responsible-AI round?

New to itGetting thereConfident

Takeaways

Responsibility is front-loaded: IBM’s six pillars belong in the spec template, not the PR template; opacity transfers accountability, so surface provenance (model, cutoff, vendor) in the UI.
Accessibility for probabilistic UIs is per-component: confidence as readable text (not color), regenerate/undo keyboard-operable, conversation as a list, aria-live on errors, prefers-reduced-motion on streaming.
Generated alt text is a real win (130+ languages) but a downstream patch — the primary UI must still be readable; ChatGPT shipped screen-reader regressions, so never trust the model to be accessible.
Bias is fairness-by-design: a model launders biased data through a fluent UI (Amazon recruiter). Surface explainable factors, an audit trail, a contest path, and inclusive long-tail testing.
Ship responsibility as primitives: an AI-label component (not a bare sparkle, NN/g), in-app data-rights and “How was I trained?”, and a human escape hatch on low confidence / high stakes.
Labeling is necessary but not sufficient (it doesn’t reduce persuasiveness) — label by risk at the element level and pair with verifiability and escalation.

Finally: the capstone — audit a real AI feature and redesign it against every principle in this track.

Sources

Free to read · better with Enzo

Learn it with Enzo

Save your progress, answer the checkpoints, and let Enzo quiz you on what you just read.

Continue with Enzo Sign up free