Designing a more honest ChatGPT
An experiment in making an ethical problem legible to a business.
Developed during the 10-week Product Management Bootcamp at allWomen [April-June 2025 cohort]
Role: Fictional Product Manager for ChatGPT | Scope: Discovery, business case, and roadmap planning
Demo video of honest chatgpt's inline verification feature. Prototyped with Claude Code. try it out on vercel.Contents
Defining the problem: how chatbots design trust the model can't back, and what it costs a paying user.
Solution space: four features aimed at limits that won't go away, one prototyped, and a roadmap that tests before it builds.
Lessons Learned: what a fictional brief can prove, what it can't, and one thing I got wrong.
AT A GLANCEStarted from an ethical worry about mental health and reframed it as retention in the paid tier.
User wellbeing = good business
Users didn’t want a better model.
Three interviews with ex-Plus subscribers. What they wanted was a tool that's honest about what it can't do.
An honesty-focused feature set, a three-phase roadmap, and a working demo of the feature.
Four features, one prototyped
It all started with a mental health concern.
INTRODUCTIONThis project was inspired by reading reports of people experiencing symptoms of psychosis (such as paranoia or delusions) after long conversations with AI chatbots like ChatGPT (see: AI-induced psychosis). I care deeply about mental health and I’m fascinated by new technology. So I decided to play pretend: if I were an actual Product Manager for ChatGPT, what would it take for me to secure the resources needed to tackle this problem?
As a designer, I would ground my ideas in user research. But in this new role as Product Manager, I needed to think in business terms.
Could I frame an ethical problem as a business problem?
Defining the problem: thinking in systems
Sycophancy sets the direction, hallucinations supply the content, chat interfaces set the expectations, and a freemium model depends on people trusting the answers enough to keep paying.
AI Chatbots are built to be believed.
DESK RESEARCH - UNDERSTANDING THE TECHAI-induced psychosis is an extreme consequence of ordinary design decisions. My research pointed to three, and none of them is a bug.
Sycophancy: the tendency to affirm users' beliefs. It's a side effect of personalisation. Products serving users worldwide are designed to take cues from each one, and personality builds trust. It can be tuned down, never removed.
Hallucinations: false information presented as fact. LLMs generate plausible continuations, not verified facts, with no internal step separating "I know this" from "this sounds right." Backstage mitigations (RAG, RLHF) reduce it; nothing eliminates it.
Chat interfaces: back-and-forth messages, like texting. We've used chat to talk to humans since the early internet, so we expect one at the other end (see: ELIZA Effect).
Each of these is doing its job. Together they produce a product that asks to be trusted more than it can justify.
How does this manifest in real users?
USER RESEARCHThis was a hard problem to research with users. It is a sensitive subject. Most people are not willing to talk about how their mental health might be negatively impacted due to their use of AI chatbots. And even if they were, I’m not a mental health expert, so I wouldn't know how to sustain these conversations.
It felt risky, so I decided to make the research focus more pragmatic: I wanted to explore user experiences with undeserved trust. I decided to screen for users who had cancelled their Plus subscription to ChatGPT (€20/month) due to unreliable answers.
The screener, and who it found. Bilingual, shared with my own network, three qualifiers. All former paid subscribers, all men. A pattern worth investigating.User research
I shared a screener survey with my social circle (creative, digitally fluent people, ages 25-45) and found 3 qualified users to interview. They were all former Plus subscribers who had already tried advanced workarounds (custom GPTs, prompt engineering and rival models) before cancelling. Their workarounds had turned into a trial-and-error routine that cost more time and money than the subscription saved.
Alex, a composite persona. Built from three interviews with former Plus subscribers, deliberately weighted toward the high-skill end: if hallucinations beat the most capable user in the sample, they beat everyone below him. Portrait AI-generated.Where the ethical problem becomes a business problem
DESK RESEARCH - UNDERSTANDING THE BUSINESS In Dec 2024, OpenAI announced plans to restructure from a non-profit into a for-profit. This meant that the previously free-for-everyone-product would start to have limitations. They decided on a freemium model.
Under freemium, paid subscriptions fund the free tier, and a subscription is a bet that the answers are worth twenty euros a month. Reliability complaints are therefore a retention risk, not just a support cost. The number I'd want on day one: what share of Plus cancellations cite unreliable answers.
Slide from my original presentation. Info taken from OpenAi.com (June 2025)THE PROBLEMTrust is designed in. Verification is left to the user.
The interface earns trust the model can't back.
A confident voice, a human-shaped exchange, no signal when it's inventing.
Alex extends that trust, then catches the errors late.
Usually after the answer is already inside his work.
The product silently offloads verification onto him.
This is risky, extra work. Without actual help from what is expected to work as a tool.
Alex cancels his paid Plus subscription.
He wants Plus to be worth it: reliable outputs, less stress, and more time for what he pays.
Solution space
How might we deliver a user experience that wins back Alex’s trust in ChatGPT Plus?
THE INSIGHTAs a creative, Alex believes a “perfect” model could potentially replace him. What he wants is a tool that collaborates with him.
None of my interviewees wanted a more powerful model; they suspected a perfect one would replace them. They wanted a tool, and a tool has to be honest and clear about what it can and cannot do.
Since there is no way to fully avoid hallucinations, sycophancy, and the empathy-inspiring effects of a chat-based interface, the best way forward was to deliver an experience that clearly signals uncertainty so that users don’t mistakenly trust the product too much.
BRAINSTORMING FEATURESDesigning around limits that aren't going away.
Four features, aimed at the culprits from the research. All of them work inside the chat, for a more honest user experience.
Critical tone toggle. Switches the model from its usual confident voice into a cautious one that hedges, states uncertainty, and makes the limits of an answer visible, so errors are easier to catch. Aimed at sycophancy.
Inline verification. Highlight any fragment of a response to see the reasoning behind it or ask a follow-up, without breaking the flow of the conversation. Aimed at hallucinations.
Prompt template library. Customisable templates for frequent jobs (summarising a text, analysing information, drafting a professional email) so you can start well without mastering prompting. Aimed at the expectation gap.
Prompt helper. Suggests improvements to your query as you type, like autocorrect for prompts, guiding you toward formulations the model answers well. Aimed at the expectation gap.
The four features, sketched.DEFINING AN MVPI prototyped one feature: Inline Verification
Four features on a slide is informative, but I wanted to inspire. I wanted to show my fictional team that it was possible to do things differently. We just had to figure out how. So I picked the one easiest to quickly vibe code: inline verification.
Critical tone toggle. Defining what sycophancy looks like is a research problem before it's a build problem, and the answer differs per user. Not prototypable yet.
Prompt template library. Technically the easiest, but closer to a community feature than a chat feature, so building it inside a chat prototype wouldn't test the actual idea.
Live prompt helper. Buildable, but it borrows a pattern people already know from autocorrect. A demo would mostly confirm what they can already picture.
Inline verification was the opposite: the least familiar interaction and the clearest user value, so it was the one worth showing rather than describing.
THE ROADMAPThree phases, starting with a chat with the dev team for feasibility.
Phase 1 is discovery, not building: Wizard of Oz prototypes of the top three features, usability testing with Alex-style users, and a RICE prioritisation run with the dev team for feasibility. Phase 2 builds what survives and A/B tests confident against critical tone. Phase 3 combines the winners into one UI and beta tests it with freelancers.
The roadmap, sequenced to de-risk rather than to build. Phase 1 spends nothing on engineering: fake the features, test them, then find out from the dev team what's feasible. Each phase has an exit condition, so only what earns its place moves forward.
So, is an honest ChatGPT good for business?
LESSONS LEARNEDHonestly, I don't know. The bootcamp ended before I could test any of it. But here’s four things I'd take to a real product role:
I think in systems, not symptoms. Each feature answers one link in the chain above, which is what keeps a solution from being decorative.
I can make an ethical case in commercial terms."This is harmful" alone doesn't get the work prioritised. "Paying users cancel because they stopped believing the answers" does. Same problem, same fix, argued in the language the company already uses.
I research what I'm qualified to research. I'm not a clinician, so I couldn't safely interview people whose mental health had been affected. I recruited at the opposite extreme, and built Alex from it. Alex is as skilled and skeptical as ChatGPT users get, and hallucinations still beat him. That proves the problem is real. It doesn't prove the fix reaches the users I was worried about, which is the gap I'd research next.
I work iteratively from what I know. I consulted no engineers, I don't know how to add friction to a chat without making it worse, and a model that hedges might read as a weaker model. So Phase 1 is a RICE run and Wizard of Oz tests, and Phase 2 A/B tests tone. Naming a gap and sequencing the work to close it is the job.
An ironic lesson, found while editing this page (July 2026). The churn statistic I originally built the business case on was given to me by ChatGPT from a source I could not later find. I had trusted a confident answer without verifying it, which is the exact failure this project is about. 😅
Note: Months after I finished, NN/g published Explainable AI in Chat Interfaces and arrived at the same place from usability research: users place undeserved trust in confidently presented answers, disclaimers only work if people read them, and citations belong next to the claim they support. I’m proud to have been thinking in the same direction.