Back to Projects

Insight AI

Helping people judge health information—not just receive answers

Product Lead (Individual Project)|AI Agent Design & Prototype Validation Sep 2024 – Present Individual Graduation Project In Progress 2024-09-01
LangChain Prompt Engineering RAG Chroma DashScope Embedding FastAPI React
AI Agent AI Product Design Information Behavior
Visual material pendingInsight AIAI Agent · AI Product Design · Information Behavior
01

30-second summary

Insight AI does more than judge one health claim. It connects evidence explanation, short practice, and progress review so users can build a judgment method they can reuse.

02

Three key numbers

3
Product modesQuick Check, Guided Practice, and Progress Review
5
Judgment dimensionsSource, evidence, causality, data presentation, and risk framing
4
Agent capability layersUnderstanding, Retrieval, Reasoning, and Reflection
03

Product flow

  1. 01
    Enter a claim

    The user submits a health claim or content excerpt.

  2. 02
    Judge first

    The product records credibility, reasoning, and confidence.

  3. 03
    Retrieve evidence

    Trusted sources provide traceable supporting material.

  4. 04
    Explain reasoning

    Feedback identifies signals the user noticed or missed.

  5. 05
    Practice transfer

    A new scenario tests the same judgment method.

  6. 06
    Update the profile

    Weak dimensions inform the next review suggestion.

04

Two key decisions

01

Turned a one-off verdict into judgment-skill practice

Why this choice
The user needs to understand the evidence and handle the next similar claim, not only resolve the current one.
Alternative not chosen
Provide a verdict, citations, and a single explanation
Tradeoff
The flow is longer, so the first-session burden and willingness to practice still require validation.
02

Kept the multi-layer Agent architecture behind a continuous task flow

Why this choice
Understanding, Retrieval, Reasoning, and Reflection support the experience; they are not the user's primary mental model.
Alternative not chosen
Expose the system as a sequence of Agent nodes
Tradeoff
The technical architecture is less visible, but the user can follow the current task and next action more easily.
05

One bad case

The answer was right, but the method did not stick

Symptom
A user accepted the verdict but could not explain why the evidence was reliable and still depended on AI for the next similar claim.
Diagnosis
A one-turn answer resolved the current item without eliciting reasoning or testing transfer in a new situation.
Response
Added a short transfer exercise after evidence feedback and used weak dimensions to adjust hints and review.
Outcome
The training loop and validation framework are defined; durable transfer with real users remains under validation.
06

Result and validation status

validating

The product concept, three modes, Agent workflow, grounded RAG flow, and controlled validation plan are complete; user outcomes are not yet quantified.

07

Evidence links

  • documentPublic case study
    available
  • demoRunnable product demo
    pending
08

Full retrospective

Project Overview

When users encounter claims such as “this supplement prevents disease” or “this remedy works in three days,” they usually want a quick answer: can I trust it? A general-purpose AI can provide a verdict, but often leaves users unsure which evidence matters, where the reasoning breaks, or what to do when the next claim appears. Insight AI is an AI product for evaluating health information. It helps users check the claim in front of them and turns the reasoning process into short, practical exercises so they can build a reusable judgment method. I independently own the product positioning, needs analysis, core features, Agent workflow, prototype design, and product-validation plan.

Pain Points & Opportunities

The target users regularly encounter health content on social platforms, care about making sound decisions, and lack specialist medical knowledge. They face three connected problems: they do not know whether a claim deserves trust; even after seeing a verdict, they cannot tell which sources, evidence, or reasoning are reliable; and they remain dependent on someone else when the next similar claim appears. Conventional fact-checking and general AI answers usually solve only the first problem and explain every claim in the same way. Yet one user may be influenced by expert-like authority cues while another responds more strongly to emotion, data presentation, or a persuasive causal story. The product opportunity is to connect “check the current claim,” “practice the method,” and “track personal weak spots” in one experience.

Goals & Metrics

The product has three levels of value. Immediate value: help users assess a health claim faster and understand where the evidence comes from. Capability value: help users state the basis for their judgment and reuse the method on unfamiliar content. Long-term value: remember which judgment dimensions a user tends to miss and recommend practice at the right level. Product validation will track judgment accuracy, completeness of reasoning, willingness to share, confidence, and follow-up practice completion. These are target measures for validation, not completed performance improvements.

User Research

The real job to be done is not “decide whether this one claim is true for me,” but “help me quickly judge whether a health claim is credible and know how to verify it next.” I translated common difficulties into five product dimensions: source and credentials, evidence quality, causal reasoning, data presentation, and emotional or risk framing. The experience must do two things well: immediately show users which signals they noticed or missed, and let them apply the same method to new content instead of merely reading an explanation. This led Insight AI away from a one-turn Q&A and toward a product structure combining real-claim checks, guided practice, and progress review.

Solution Design

The product offers three entry points. Quick Check lets users paste a real health claim and receive source checks, evidence grading, reasoning breakdowns, and suggested verification steps. Guided Practice presents a controlled health-information scenario: users rate credibility and explain why, mark suspicious signals, receive immediate feedback, complete a rewrite, rebuttal, or verification-path task, and then apply the method to a new scenario. Progress Review summarizes judgment dimensions the user frequently misses and recommends the next practice session. A complete practice loop is “initial judgment → explain the reason → identify a weak spot → practice in a short scenario → receive instant feedback → apply the method to new content → update the profile and review suggestion.” Understanding, Retrieval, Reasoning, and Reflection support this flow behind the scenes; users experience a coherent task rather than a technical pipeline.

Execution Plan

The MVP focuses on the shortest useful path: enter a health claim → record the user’s initial judgment and confidence → retrieve trusted material → explain the evidence and reasoning issue → complete one transfer exercise → collect feedback. The first version covers quick checks, evidence explanation, short scenario practice, and basic weak-spot tracking. Progress trends, automatic review, and expansion beyond health content come later. LangChain orchestrates the workflow, Prompt Engineering controls questions, hints, and feedback structure, Chroma with DashScope Embedding supports retrieval, and FastAPI plus React provide the service and interaction layers. Coze is used to validate selected flows quickly. RAG does not make the final decision for the user; it gives every explanation a traceable evidence base.

Results Review

Work completed so far includes the product positioning, target-user and job definition, three product modes, core usage loop, five judgment dimensions, four-layer Agent model, grounded RAG flow, and a prototype plan ready for further validation. The next product-validation phase will compare personalized practice, a standard AI answer, and an unguided experience. It will assess whether users judge more accurately, explain their reasoning more clearly, share more cautiously, and reuse the method in a new scenario. The project remains in progress, so this page does not present those outcomes as already achieved.

Iteration Reflection

Three product areas still need refinement. First, the weak-spot profile currently depends on explicit choices and answers; future versions should update it from real usage behavior with less setup effort. Second, the practice path is primarily rule-matched and should eventually adapt task difficulty, hint strength, and scenario type to performance. Third, a correct answer immediately after practice does not prove that the method will last; short review sessions and delayed check-ins are needed. My product roadmap is to make “quick check + evidence explanation + one practice task” work smoothly for health information first, then add progress review, proactive refreshers, and browser or social-information scenarios.