Real Agent Cases

Gemini vs ChatGPT on a check-engine EVAP code: replace the sensor or clear the code?

Posted
by Cody Hoeft (LinkedIn)
Last checked
Method:
How we verify

Evidence: Described by posterThe person describes what happened in words, without screenshots or video.

Outcome

After a check-engine light on his pickup, a driver read one evaporative-system code and the sensor readings, then asked Gemini and ChatGPT what they meant. Gemini suggested replacing the sensor; ChatGPT said to clear the codes and see if they came back. He cleared them, and after a long trip the code hadn't returned, avoiding a costly, labour-heavy part replacement for now.

Agent

Gemini, ChatGPT

Prompt Recreated prompt

The source didn't share its exact prompt, so we wrote this from what it describes.

Run this task now, exactly as written below. Everything you need is here, so start straight away and only stop if something required is missing or a rule below tells you to.

Goal: Interpret a check-engine code and sensor readings, and tell me the least-cost safe next step.

Fill in before running:
- Vehicle: [MAKE MODEL YEAR]
- Code(s) and freeze-frame or sensor readings: [PASTE]
- Basic checks done (oil, coolant, gas cap): [LIST]

Safety first: anything touching brakes, steering, airbags, fuel lines or lifting the car is a job for a qualified mechanic unless I confirm I have the tools and experience. Say so clearly.

Rules:
- Explain what the code means and how serious it is.
- Give the cheapest reasonable next step first (e.g. clear and monitor) and when that's not safe.
- List what would justify replacing a part.
- Tell me what to watch for after clearing the code.
- If a site asks for a login, a CAPTCHA or payment details, stop and hand control back to me.
- Ask for my OK before you pay, book, send, submit or cancel anything.
- Check facts that can change (prices, opening hours, dates, availability, rules) against the official source, and tell me anything you couldn't verify.

Output: Code explanation, risk level and a step-by-step plan.

You'll know it worked when: the light stays off, or I know exactly which part to test or replace.
Optional stronger version

Our rewrite with extra checks. The poster didn't test it.

Run this task now, exactly as written below. Everything you need is here, so start straight away and only stop if something required is missing or a rule below tells you to.

Goal: Interpret a check-engine code and sensor readings, and tell me the least-cost safe next step.

Fill in before running:
- Vehicle: [MAKE MODEL YEAR]
- Code(s) and freeze-frame or sensor readings: [PASTE]
- Basic checks done (oil, coolant, gas cap): [LIST]

Safety first: anything touching brakes, steering, airbags, fuel lines or lifting the car is a job for a qualified mechanic unless I confirm I have the tools and experience. Say so clearly.

Rules:
- Explain what the code means and how serious it is.
- Give the cheapest reasonable next step first (e.g. clear and monitor) and when that's not safe.
- List what would justify replacing a part.
- Tell me what to watch for after clearing the code.
- If a site asks for a login, a CAPTCHA or payment details, stop and hand control back to me.
- Ask for my OK before you pay, book, send, submit or cancel anything.
- Check facts that can change (prices, opening hours, dates, availability, rules) against the official source, and tell me anything you couldn't verify.
- Ask a second assistant or a mechanic when the answers disagree, as this poster did.

Output: Code explanation, risk level and a step-by-step plan.

You'll know it worked when: the light stays off, or I know exactly which part to test or replace.

Before you finish: re-read my goal and constraints, check your answer against each one, and list anything you assumed or couldn't confirm.

Source

https://www.linkedin.com/posts/cody-hoeft-87920ab8_day-54-anecdotal-win-for-ai-i-had-a-issue-activity-7488618964512624640--nz1

  • A driver replacing a sensor to clear a check engine light says Grok led them astray for 40 minutes; they had to use Google to find where the sensor was. Once they found it the job was easy, and the light went off. The poster's conclusion: AI is not good at fixing cars.

    Grok · Personal · Source date:

  • Work out what check-engine codes mean and what to test, with ChatGPTRecreated promptEvidence: Shown on videoPartial

    SlashGear describes YouTuber Watch Wes Work asking ChatGPT about three codes (P0171, P0420, P1101) on a 2015 Chevrolet Trax. ChatGPT suggested checking for exhaust leaks, then inspecting the mass airflow sensor and air intake, and gave theoretically accurate test methods, but its normal airflow reading was vague. The YouTuber found vacuum leaks in a PCV hose and the valve cover and, asked whether ChatGPT diagnosed the issue, said it's a matter of perspective, but ultimately it did not; SlashGear says it lacked hands-on context for this model.

    ChatGPT · Personal · Source date:

  • After their mechanic swapped 3 parts without fixing a check engine light and was about to replace another, a driver stopped the mechanic and used ChatGPT and Grok instead. The driver says they found the air leak on their own and fixed the light.

    Grok, ChatGPT · Personal · Source date: