How it works

Architecture & key decisions

PlayIT is a hybrid Steam recommender: a gradient-boosted model predicts whether you’ll enjoy a game and for how many hours, and a fine-tuned language model explains why, with a cost-per-hour figure. This page is the design history — the pipeline, the deployment, and the decisions (and a couple of failures) along the way.

Sample output

real run, not staged

No Steam account needed to see what it produces — 3 of the 10 recommendations one real profile got back. Every profile gets its own set; this is what the model and the explainer actually wrote, unedited.

1Rust

predicted like

price: $19.99
est.: 27 hrs
value: $0.75/hr

Based on your taste in action and adventure games, you might really dig Rust. The game has a lot of depth — especially if you’re into building, fighting, and surviving — but it can be quite challenging. The negative reviews about cheating and frustration might make things tough, though. Given the estimated hours of enjoyment and the cost per hour, it’s definitely worth the investment. The game costs around $0.75 per hour of playtime, which seems fair considering how much time you’ll invest in it.

2Baldur's Gate 3

predicted like

price: $44.99
est.: 29 hrs
value: $1.57/hr

Based on your taste for adventure and strategy games, you're likely to deeply enjoy Baldur's Gate 3. The game offers a vast world filled with mysteries and unexpected twists and engaging narratives that keep you invested throughout hundreds of hours. The positive reviews highlight how immersive and captivating the game can be once you get into it. Considering the cost per hour is only $1.57, you'll get great value out of each dollar spent, making it worth every penny!

3Garry's Mod

predicted like

price: $9.99
est.: 15 hrs
value: $0.69/hr

Based on your taste for simulation games and the positive reviews mentioning creative building and experimentation with physics, you'll likely find Garry's Mod both engaging and enjoyable — especially if you like exploring different scenarios and mods. The variety of gameplay modes and the creative freedom should keep you hooked. At $0.69 per hour of playtime, it’s quite a good deal for the amount of joy you’ll get out of it!

Training & recommendation pipeline

One shared data pipeline feeds two deliberately different models. They meet only at serving time — predicting a rating and generating a paragraph are separate problems, and a purpose-built model wins each one (ADR 0001).

The five-minute version

Don't force everything through an LLM

The obvious move is “hand the reviews to a chat model and ask what to play.” This splits the problem instead. Predicting whether someone enjoys a game and for how long is a recommender-systems regression over structured signals — playtime, review scores, price, genre fit — and a gradient-boosted model beats a language model there on accuracy and on cost. The language model earns its place on the other half: turning raw user reviews into a short, personal explanation with a cost-per-hour figure.

The hard part was the label, not the model

Steam gives you playtime, not a thumbs-up. Raw hours is a bad target — 40 hours in a story RPG and 40 hours in an idle game mean opposite things. The target here is an engagement ratio: a user's hours in a game over the median hours that user spends in that genre, clipped and log-transformed, plus a per-user calibration step since “liked” sits at a different threshold for different players.

Teacher–student distillation for the explanation layer

A 7-billion-parameter model writes good recommendation copy but is too slow to run on every request. It plays teacher instead: it generated roughly 2,800 training examples, and a LoRA fine-tune taught a 1.5-billion-parameter student to reproduce the style at a fraction of the inference cost. Held-out loss fell from 1.73 to 1.10 as the training set grew, tracked every epoch with no divergence.

Two evaluation metrics — report the one that looks worse

Standard cross-validation measures fit for a returning user whose other games the model has already seen: about 89%. Leave-one-user-out holds back every row belonging to a person and measures the prediction for a genuine stranger: about 65%. Three experiments — capacity tuning, review-sentiment features, more genres — produced clean null results, and each is written up as a decision record.

A data-quality failure, caught and fixed

Growing the pool meant crawling friends-of-friends across the Steam social graph. The first attempt regressed the honest metric — ranking candidates by playtime pulled in accounts averaging 11,000-plus lifetime hours: bots and idle-farmers, not players. The fix was an account-quality “decider” (excludes no-playtime, bot-intensity, and single-game accounts) plus representative sampling instead of playtime ranking. The pool went from 23 real users to 393; cold-start recovered.

What’s serving this page right now

The model can’t run on Vercel — no GPU, no model weights, and the Steam key must never leave the box. So this site is a frontend plus a thin, bearer-token-authenticated proxy; the actual inference runs on a home GPU behind an outbound-only Cloudflare Tunnel (ADR 0015).

Key decisions, from the record

ADR 0001

Hybrid architecture. Gradient boosting for scoring, a distilled small LLM for explanation — not one large model doing both.

ADR 0002

Engagement-ratio label. Playtime normalized against the user's own genre-cohort median, so the target reflects enjoyment rather than genre length.

ADR 0005

One shared, personalized model. Pooled across users, conditioned on per-user genre-affinity features computed leave-one-out at training time.

ADR 0003 / 0008

Baseline before boosting. Started with a Ridge baseline; moved to gradient boosting only when more data justified it, and re-tuned capacity as the pool grew.

ADR 0013

Social-graph expansion + quality decider. The regression, the diagnosis, and the recovery are all in the record, not just the win.

ADR 0014

Next: collaborative filtering. Scoped, not built — game-level signal to close the cold-start gap, with the open questions written down first.

ADR 0015

This deployment. The model can't run serverless, so this site is a thin proxy: Vercel → Cloudflare Tunnel → a home GPU box, with a passphrase gate and an async job queue.

ADR 0016

Bad reviews could rank above good ones — found and fixed. A cold-start account with zero Sports history got 4 sports-sim titles in its top 10, one with 31% positive reviews outranking a 97%-positive RPG. Permutation importance showed review sentiment barely mattered to the model while review volume did; a monotonic constraint (better reviews can never lower a score) fixed it and nudged cold-start accuracy up.

ADR 0017

A wallpaper tool was in the recommendations. Wallpaper Engine — a desktop live-wallpaper app, not a game — showed up in a top 10. Steam's own type field calls it "game" too; only its full genre list ("Utilities") gives it away. Candidates are now checked against that list and excluded if they're not really games, backfilled so the pool doesn't shrink.

By the numbers

89% / 65%

Returning-user vs. cold-start liked-accuracy

393

Pooled training users, from a social-graph crawl

~55,000

(user, game) training rows

Model features — structured, sentiment, per-user affinity

1.73 → 1.10

Explainer held-out loss after data scaling

Architecture decision records + a running performance log

Built with an AI agent

Every design decision, experiment, and dead end lives in a numbered decision record and a running performance log — an ADR-first workflow run with Claude Code as a pair. The agent handled breadth: parallel hyperparameter sweeps, GPU training runs as background jobs, the data-pipeline plumbing, this deployment, and keeping the docs in sync with the code.

The judgment calls stayed with a person — the hybrid split, the label design, stopping to diagnose a regression instead of pushing past it, widening a data filter because the users behind it were personally known to be legitimate. The result is a working end-to-end ML system, built and documented in the time it usually takes to build one component of one.