Catch an error in a 4th-grader's textbook answer key, which ChatGPT first got wrong too (failure)
- Posted
- by Carlos Mendoza (@carlosatal)
- Last checked
- Method:
- How we verify
Evidence: Screenshots or logsThe source shows screenshots, a transcript or the agent's output.
FailureThe source says the agent did not get the job done.
What went wrong: ChatGPT agreed with the wrong answer key until the parent pointed to the exact word.
Outcome
Carlos Mendoza's 4th-grade son marked only B and F as correct in a Santillana exercise on numbers up to the billions. The platform's solution marked B, D and F, but D said '9,615 rounded to the billions is 10,000', which is false. When Mendoza asked ChatGPT to review it, it first said Santillana was right, reading 'billions' as if it were 'thousands'. Only when Mendoza pointed to the exact word did ChatGPT acknowledge the error. The son had been right from the beginning.
Agent
Prompt Recreated prompt
The source didn't share its exact prompt, so we wrote this from what it describes.
Run this task now, exactly as written below. Everything you need is here, so start straight away and only stop if something required is missing or a rule below tells you to. Goal: Check whether my child's homework answer or the answer key is right, reading the question word for word. Fill in before running: - Photo or text of the question and all options: [ATTACH / PASTE] - My child's answer and the official answer: [DETAILS] Rules: - Quote the exact wording of each option before judging it. - Work each one out step by step, paying attention to units and words like thousands, millions or billions. - Say clearly which answers are right and which are wrong, and why. - If the wording is ambiguous, say so instead of picking one reading. - Check facts that can change (prices, opening hours, dates, availability, rules) against the official source, and tell me anything you couldn't verify. Output: A short, option-by-option check with the working shown. You'll know it worked when: I know whether to tell the teacher the key is wrong.
Customize this prompt
Answer these questions and we'll put your answers into the prompt for you. You can leave any of them empty.
Your prompt
Run this task now, exactly as written below. Everything you need is here, so start straight away and only stop if something required is missing or a rule below tells you to. Goal: Check whether my child's homework answer or the answer key is right, reading the question word for word. Fill in before running: - Photo or text of the question and all options: [ATTACH / PASTE] - My child's answer and the official answer: [DETAILS] Rules: - Quote the exact wording of each option before judging it. - Work each one out step by step, paying attention to units and words like thousands, millions or billions. - Say clearly which answers are right and which are wrong, and why. - If the wording is ambiguous, say so instead of picking one reading. - Check facts that can change (prices, opening hours, dates, availability, rules) against the official source, and tell me anything you couldn't verify. Output: A short, option-by-option check with the working shown. You'll know it worked when: I know whether to tell the teacher the key is wrong.
Scroll to see the whole prompt. Your answers show in green.
2 blanks are still empty: Photo or text of the question and all options; My child's answer and the official answer. They'll stay in brackets in your prompt, so the chat tool may ask you about them.
What you type stays in this browser tab. We don't send it or save it, and it's gone when you leave the page. If you use a "Try this in" button, your answers go to that tool as part of the prompt.
Try this in:
This prompt has blanks to fill in, like [ATTACH / PASTE]. We'll copy it instead of filling it in, so you can edit it before you send.
Opens in a new tab. We don't see what you send.
These links are only a shortcut. We're not affiliated with OpenAI, Google or xAI, and your results may not match the source.
Optional stronger version
Our rewrite with extra checks. The poster didn't test it.
Run this task now, exactly as written below. Everything you need is here, so start straight away and only stop if something required is missing or a rule below tells you to. Goal: Check whether my child's homework answer or the answer key is right, reading the question word for word. Fill in before running: - Photo or text of the question and all options: [ATTACH / PASTE] - My child's answer and the official answer: [DETAILS] Rules: - Quote the exact wording of each option before judging it. - Work each one out step by step, paying attention to units and words like thousands, millions or billions. - Say clearly which answers are right and which are wrong, and why. - If the wording is ambiguous, say so instead of picking one reading. - Check facts that can change (prices, opening hours, dates, availability, rules) against the official source, and tell me anything you couldn't verify. - End by restating the exact word that decides each option, so I can double-check it myself. Output: A short, option-by-option check with the working shown. You'll know it worked when: I know whether to tell the teacher the key is wrong. Before you finish: re-read my goal and constraints, check your answer against each one, and list anything you assumed or couldn't confirm.
Source
https://x.com/carlosatal/status/2111517945386156108
Related cases
- Catch a wrong multiplication answer from ChatGPT while checking homeworkRecreated promptEvidence: Screenshots or logsFailure
Helping their son with homework, the poster had ChatGPT (GPT-5) do a simple multiplication problem. They couldn't reproduce its answer, checked with a calculator, and found ChatGPT was wrong.
ChatGPT · Personal · Source date:
- Test six AI homework tutors on one maths problem with a 14-year-old and pick ChatGPT Study ModeRecreated promptEvidence: Described by posterBeginner-friendly
During back-to-school prep, Laura Cioffi and their 14-year-old daughter gave the same cone surface-area problem to ChatGPT Study Mode, Photomath, Socratic by Google, Claude, Brainly AI and Duolingo Math. Photomath got it wrong; Claude and Socratic were right but too academic; Brainly gave clear steps; Duolingo felt like a game. ChatGPT Study Mode asked her for the values, walked her through the formula and checked her understanding; she said it didn't make her feel 'thick'.
ChatGPT, Claude, Photomath, Socratic, Brainly AI, Duolingo Math · Personal · Source date:
- Revise for a school exam without the textbook by giving ChatGPT the official PDF to answer the exercisesEvidence: Described by poster
SriSakthi Saravana Prabhu's younger child had a Kannada exam the next day after a week off school with fever, and the textbook was likely left at school. The parent can speak Kannada but can't read or write it, and the exam questions come from the exercises at the end of each lesson. They found the Grade 3 textbook as a PDF on the Karnataka education portal and uploaded it to ChatGPT. Within seconds it gave answers, and the child confirmed they were the ones needed. Some pages came back without answers, but uploading those pages separately filled the gaps.
ChatGPT · Personal · Source date: