EasyEnv

A pull request lands. It is clean. Good names, consistent style, tests included, a tidy description. You approve it.

Six weeks later something breaks in a way nobody can explain, and it turns out the fix is in a file that three people have now read and none of them really understand.

This is the new review problem. Code arrives faster than anyone can verify it, and it arrives looking finished. The old habit, reading until something feels off, does not work, because nothing feels off.

Why your review habits miss it

Human mistakes cluster around the hard parts. The author struggled with the tricky bit, so the tricky bit is where the bug is. Reviewers learned to read that way, and it worked.

Generated code fails in the opposite pattern. The mistakes are in the parts that look easiest: confident, ordinary, well-formatted code that happens to be wrong.

These are the ones that keep coming up:

  • A function that does not exist. A method, a flag, a config key. It reads exactly like the real thing, and it is from a different version or from nowhere.
  • Right shape, wrong meaning. Does something next to what you asked. A filter applied before instead of after. A boundary off by one.
  • The edge case quietly gone. Empty input, nulls, timezones, the concurrent case. The happy path is flawless.
  • Errors swallowed. A catch that logs and carries on. A retry with no backoff.
  • Correct but foreign. It does not use your helper, your error type, your logging, your transaction boundary. Not a bug. Still a cost, because your codebase slowly becomes five codebases.
  • Out of date. A pattern that was right three library versions ago.
  • Unsafe defaults. Queries built from strings, permissive CORS, weak randomness. It reproduces the most common pattern, and the most common pattern online is often the insecure one.

There is an eighth that is worse than any of these: a confident explanation that is wrong. The code may be fine and the reasoning beside it teaches the reader something false. That one spreads.

Review in this order

The first three steps catch most of it, and they take about five minutes.

1. Check every symbol is real. Before reading for logic, jump to the definition of anything unfamiliar. Check the version you actually have installed, not the docs the model half-remembered. Ninety seconds, and it catches the most embarrassing failures.

2. Run it against the real case. Not the test that came with it. Model-written tests routinely assert the behavior the model implemented, bug included. Take one case from the actual requirement and run it.

This is the highest-value step and the one people skip, because the code looks done.

3. Say the edges out loud. Empty. One. Many. Null. Zero. Negative. Huge. Concurrent. Unicode. Crossing midnight. For each, ask whether this code does something sensible. You will find the missing one in under a minute or confirm there is not one.

4. Read the error paths first. Skip the happy path on the first pass. It is the part the model is best at. Go to the catches, the retries, the timeouts, the cleanup. What happens when the dependency is slow, down, or half-answers?

5. Ask if it belongs here. Does it use your utilities, your conventions, your logging? Correct-but-foreign is a real cost even though nothing is broken.

6. Check the security defaults. Query building, input validation at the edges, anything touching secrets or crypto.

7. Ask if you could maintain it. The last gate, and the strict one: can you explain every line to someone else? If not, you are not reviewing it. You are accepting it.

What to skip: formatting, naming style, micro-optimizations. That is what generated code is already good at. Every minute spent there is a minute not spent on step 2.

The same thing, as a hiring task

Deciding what to accept is now a distinct skill, and it varies wildly between candidates. It is also harder to fake than writing code from scratch, since writing from scratch is the part a model can do for them.

The setup. A real repository, a plausible ticket, and a pull request that was produced by a model. Tell the candidate exactly that: this was AI generated, decide what to do with it.

What to plant. Four defects, at different depths:

  1. Something the compiler or linter catches. Free, and it tells you whether they ran anything.
  2. A call to a function that does not exist. Catchable in ninety seconds by anyone who checks.
  3. An edge case that only fails on one real input. Needs someone to think about inputs.
  4. Something that is not a bug at all, just foreign to the codebase. This is the good one. It separates people who can tell wrong from does not belong.

Give them the environment with the dependencies installed and let them use anything, including a model. You are testing whether they verify, not whether they can spot bugs by staring.

What actually separates people. From sessions we have reviewed, the split shows up early:

  • Strong candidates run something in the first few minutes. Weak candidates read for fifteen and comment on naming.
  • Strong candidates check the unfamiliar call against the installed version. Weak candidates trust it because it looks right.
  • Strong candidates find the edge case by naming input classes, not by luck.
  • Strong candidates say plainly what they would not merge, including "this works and it still does not belong here."

Scoring. Four rows: defects found, whether they ran it or only read it, the quality of the merge or reject decision, and whether they can explain the code they would keep. Use the same rows for everyone, in the shape from how to write a technical interview rubric. Do not score speed on its own.

Where EasyEnv is different

A review task only works if the code can actually run. If the candidate cannot install the dependency, hit the endpoint, or execute the failing case, then "check it against reality" is not available to them and you are back to grading whether they read carefully.

EasyEnv gives them a real Linux machine with the repo cloned, dependencies installed and the service running before they join. So step 2 is one command away, and whether they took it is the clearest signal in the session. The terminal and screen are recorded, so in the debrief you can see the exact moment they either ran the code or decided they did not need to. Written answers get an AI-graded first pass so the reviewer starts from evidence.

The trade-off is real. Authoring a task like this takes an afternoon: you have to plant defects that fail the same way for every candidate, and someone has to review the session afterwards. There is no green checkmark. If you need a cheap automated filter over thousands of applicants, this is not that, and an auto-scored screen is the honest answer for that step.

The short version

Check the symbols exist. Run it against the real case. Name the edges out loud. Then read the error paths. Style comes last, if at all.

And refuse to merge anything you cannot explain. That single rule catches more long-term damage than the rest of the list combined, and it is the one worth putting in your review guidelines this week.

For hiring teams

Hire engineers with real-world interviews

Run live coding sessions and take-home challenges in real production environments. Watch sessions back, score consistently, and hire with confidence.

Continue reading

More posts you might like

Forward Deployed Engineer (FDE) Interview: How to Hire FDEs
November 16, 2026·17 min read

Forward Deployed Engineer (FDE) Interview: How to Hire FDEs

Hiring forward deployed engineers (FDEs)? An FDE is judged inside a customer's environment, by the customer's rules. What the interview needs, the 60-minute loop we run for it, what to watch in each stage, and a rubric you can use tomorrow.

Read more
System Design Interview Tool: A Canvas Your Reviewer Can Read, Push Traffic Through and Replay
November 13, 2026·6 min read

System Design Interview Tool: A Canvas Your Reviewer Can Read, Push Traffic Through and Replay

EasyEnv now has system design questions and a System Design Lab. Candidates build the architecture layer by layer on a typed canvas, and your reviewer reads what was built, the reasoning next to it, and how it got there. Your team can push traffic through a design to find what breaks first.

Read more