1. Habit
  2. Skills
  3. Compare Models skill
replicate · 2K installs

Compare Models skill
Runs by itself, monthly model bake-off.

Your model choice re-tested each month.

Install the Compare Models skill and make it a habit: follow https://gethabit.dev/skills/replicate/compare-models.md

  1. 1Copy
  2. 2Paste into Claude Code, Codex or Cursor
  3. 3It runs by itself

2K installs136 starsApache-2.0 licenceSource on GitHub ↗

Compare Models

by replicate · 2K installs

The skill

The compare models skill helps developers compare Replicate models on speed, quality and capabilities by shortlisting, reading schemas and running the same small test batch through each.

Only when you ask.

The habit

Your model choice re-tested each month.

↻ Runs by itselfmonthly model bake-off

How the habit works.
You don't have to.

Three questionsWhen does it start? How does it check its work? What does it remember?

01

It starts by itself

monthly model bake-off. Runs in Claude Code routine.

  • ↻ monthly model bake-off
  • 0 9 1 * *

02

It checks its work

same prompts, metrics recorded. If a check fails, it tries again.

  • ✓every candidate ran the same prompt set
  • ✓metrics.predict_time is recorded for each completed prediction
  • ✓every candidate's schema was fetched this round

03

It remembers

bake-off results. The next run reads it first: last round's winner, prompts and timings.

loops/replicate-model-bakeoff-loop/results.md

One run, step by step.
Then it stops.

Stop ruleStops after 4 candidates and 5 prompts per candidate per run.

  1. 01

    Shortlist

    Search or browse collections for candidate models.

  2. 02

    Compare schemas

    Fetch each schema and compare inputs, outputs and features.

  3. 03

    Prepare prompts

    Use the prompt-images or prompt-videos skill to write the shared prompt set.

  4. 04

    Run a small batch

    Run the same prompts through each model and collect outputs.

  5. 05

    Compare

    Compare speed from metrics, output quality and capabilities, noting official versus community.

  6. 06

    Record the pick

    Write the winner and reasons to the results file.

What it may do.
Nothing more.

Safe by defaultIt writes only in its own folder and asks you before anything goes out.

Is it for you?
Use it when it fits.

Quick checkRead both lists. If the left one sounds like you, the Compare Models skill helps.

Use it when

  • ✓ You choose between several models
  • ✓ You want to check if a newer model is better
  • ✓ You compare official and community models

Skip it when

  • – You already have a fixed model
  • – You do not use Replicate

Questions

Compare Models skill, answered.

Why test with my own prompts?

The skill says quality is subjective and should match your use case, not a leaderboard.

Why prefer official models?

The skill notes they are always warm and have stable APIs; community models can cold-boot.

How do I install the Compare Models skill?

Copy the prompt at the top into your agent. It installs the skill from replicate/skills and sets up the habit. Or run npx skills add replicate/skills --skill compare-models.

Is the Compare Models skill free?

Yes. It is published under Apache-2.0. Habit is free too.

Compare Models. Now as a habit.

Install the Compare Models skill and make it a habit: follow https://gethabit.dev/skills/replicate/compare-models.md

  • Free
  • No account
  • Undo in one line

All replicate skills →More planning skills →

Works in

  • Claude Code
  • Codex
  • Cursor
  • ChatGPT
  • Claude app

Made by replicate, published under Apache-2.0. Habit links to the original and never changes it. Is this your skill?

Copying didn't work here. Select the line and copy it by hand.