Real Agent Cases

Test five web tasks with Project Mariner

Posted
by @JulianGoldieSEO
Last checked
Method:
How we verify

Evidence: Shown on videoThe source includes a video or screen recording of the agent doing the task.

PartialThe source says it only partly worked, or had a significant problem a person had to deal with.

What went wrong: 1 of 5 tasks succeeded.

Outcome

In a test of 5 tasks, Project Mariner found YouTube video view counts, but failed to generate relevant video ideas, couldn't preview HTML properly, refused to log into email and couldn't send emails.

Agent

Project Mariner

Prompt Improved prompt

Strengthened by us to avoid the problem in the source. Not tested by the poster. It doesn't change what happened in the source.

Let's work this out together. I'll tell you what I'm after; help me sharpen the goal before you dive in, and check in with me before anything you can't undo.

Goal: Collect view counts for a set of YouTube videos and turn the top performers into new video ideas.

Questions to explore first: Which videos or channel, and what kind of ideas do you want (topics, titles, formats)?

What I need from you:
- Video or channel links ([URLS])
- Your niche and audience

Rules:
- Start by restating my goal and suggesting 2–3 ways to approach it; recommend one.
- Collect view counts first, with the date you checked and a link for each video.
- Base every idea on a specific top video and say which one inspired it.
- Keep ideas relevant to my niche; drop anything generic.
- Don't log into any account or send anything.
- If a step fails, say so and move on rather than stopping the whole task.

Output: A table of videos with view counts and dates, then 10 ideas each tied to a source video, plus 2–3 suggested next steps.

If any of this is unclear, tell me what you have and I'll help fill the gaps.

You'll know it worked when: the counts match YouTube and every idea fits your channel.
The prompt they actually used (our rebuild)

The source didn't share its exact prompt, so we wrote this from what it describes.

Find the view counts for these YouTube videos and generate video ideas.

Source

https://x.com/JulianGoldieSEO/status/1927077282041999503

Related cases

  • Summarize English and Tamil YouTube videos with CometImproved promptEvidence: Described by posterPartial

    Comet Assistant summarized English YouTube conversations quickly and well, but on a Tamil discussion it had to play the whole video and made a few mistakes, missing some crucial points.

    Perplexity Comet Assistant · Research · Source date:

  • Find links to content with Project Mariner (no links returned)Improved promptEvidence: Described by posterFailure

    Asked to find links to content, Project Mariner found the content but wouldn't return the links, which the poster said made the result not very helpful.

    Project Mariner · Research · Source date:

  • Report global NAND market share with Gemini Deep ResearchImproved promptEvidence: Screenshots or logsFailure

    Asked for a simple report on global NAND market share, Gemini Deep Research returned figures the poster called an incredible failure when compared with Sandisk's investor-day slide and Claude's answer to the same query.

    Gemini Deep Research · Research · Source date: