roodit-scraper · how it works

The kitchen

roodit-scraper is a restaurant kitchen. Reddit posts are the ingredients, trending posts are the dishes. The head chef decides what gets cooked, and nothing touches a burner unless there's a free one. Click any part of the kitchen to see what it is in the code.

Discover Track Score Baseline Dedupe (add-on)
Run a service

Live kitchen

Raise the subreddits to send in more orders. Shrink the burners to tighten the budget. Then flip to the old kitchen and watch it melt.

Kitchen
30
8
Service
Ticket rail Queue + events · 0 waiting
No tickets waiting.
Burners Budget · 0 / 8 in use
Follow one post

One dish, start to finish

Click a step to light up where it happens in the kitchen above.

Glossary

Kitchen words, code words

Use either column when talking to the AI or reviewing a PR. They mean the same thing.

KitchenSystemIn the codeIts one job
Why we're rebuilding

Old kitchen, new kitchen

Old kitchen (v1)

  • Three alarm clocks (crons) shout orders every minute, whether or not the last ones are done.
  • No head chef. Every order goes straight onto a burner, even when none are free, so everything cooks slower.
  • Prep cooks chase deliveries page after page (deep pagination) and duplicate each other's work.
  • The taste test re-reads a week of notes for every subreddit, every minute (heavy analytics).
  • Knobs are hidden in drawers: env vars, DB docs, Redis keys, scripts.

New kitchen (v2)

  • One head chef is the only one who calls tickets.
  • Every recipe says how many burners it needs. A ticket waits on the rail until there's room.
  • Prep takes only what's new since last delivery, usually one request.
  • The reference recipe (baseline) is updated hourly, not every minute.
  • Every knob is on the wall: burners, open / last orders / closed, each station on or off.
Reviewing a PR

Three questions for every PR

Ask these before reading any code. If the PR description can't answer them, ask the author (or the AI) first.

1

Which station?

Which module / package does it change?

A good PR works in one station. Changes spread across many stations mean the PR should be split.

Red flag: a station reaching into another station's fridge, like Score reading Track's tables directly.

2

What goes in, what comes out?

Does it change a contract: Stage, Source, Sink, Scorer, or an event?

Recipes inside a station can change freely. Changing what a station receives or hands over affects every station around it.

Red flag: a ticket format changed without updating the stations that read it.

3

How many burners?

Does it add requests or DB work? Is it declared in cost()?

Every new job must say what it costs, so the head chef can budget it.

Red flag: anything that starts cooking without asking the head chef, like a job enqueued outside the scheduler, or a setInterval.