How to Use AI Coding Agents Without Losing Engineering Judgment

Posted on in Technology Management / Leadership

AI coding agents are useful in the same way junior engineers, build scripts, and sharp shell aliases are useful: they can remove friction, accelerate boring work, and occasionally surprise you with a clever path through a problem. They are not a replacement for engineering judgment.

That distinction matters.

The strongest engineers I know do not treat tools as magic. They build a mental model of what the tool is good at, where it fails, and what kind of supervision it needs. AI coding agents deserve the same treatment. If you let one roam through a repository with vague instructions and then rubber-stamp the diff because the tests passed, you have not improved your engineering process. You have merely added a faster way to ship confusion.

Used well, though, coding agents can be a real productivity multiplier. They can trace unfamiliar code paths, make mechanical changes, draft tests, summarize diffs, and handle the dull connective tissue around implementation. The trick is to keep the human in the role that matters most: setting intent, evaluating tradeoffs, and deciding what "correct" means.

Start With The Engineering Task, Not The Agent

The most common mistake is asking an AI coding agent to "fix this" before you have decided what "fixed" means.

That is backwards.

Before handing work to an agent, write down the engineering task in plain language:

  • What user-visible behavior should change?
  • What files or systems are likely involved?
  • What constraints should not be violated?
  • What tests or checks would prove the change is acceptable?
  • What would make the solution too risky, too broad, or too clever?

This does not need to be a full design document. Often a short paragraph is enough. The point is to force your own thinking into the open before the model starts producing plausible code.

For example, this is weak:

Fix the login bug.

This is much better:

Users are being redirected to /login after a successful SSO callback when the
session cookie already exists. Find the code path responsible for callback
handling, explain the likely cause, and make the smallest change that preserves
existing local-password login behavior. Add or update a regression test.

The second prompt gives the agent boundaries. More importantly, it gives you something to measure against when the diff comes back.

Use Agents For Exploration, But Own The Conclusion

One of the best uses for an AI coding agent is codebase reconnaissance.

Ask it to find where a concept lives. Ask it to trace a request path. Ask it to identify likely ownership boundaries. Ask it to summarize the tests that already cover a behavior. This is often faster than manually spelunking through a large repository, especially when the naming is inconsistent or the architecture has several historical layers.

But do not confuse a confident map with the territory.

When an agent tells you, "The bug is probably in SessionCallbackHandler," that is a hypothesis. Treat it like one. Open the file. Read the surrounding code. Look at the call sites. Check whether the test it found is actually testing the behavior you care about.

Good engineering judgment is not the ability to type every line yourself. It is the ability to evaluate whether the proposed line belongs in this system.

I like a workflow that separates exploration from implementation:

  1. Ask the agent to inspect the code and report likely approaches.
  2. Read the relevant files yourself.
  3. Pick the approach and constraints.
  4. Ask the agent to implement within those constraints.
  5. Review the diff like you would review a teammate's pull request.

That middle step is where judgment lives. Skipping it is how teams end up with changes that are locally reasonable and globally weird.

Keep The Diff Small Enough To Review

AI coding agents are very good at making broad changes. That is not always a compliment.

A human engineer usually feels the pain of a large diff while making it. An agent does not. It can rename a helper, adjust a dozen call sites, rewrite tests, and "clean up" unrelated code without any emotional resistance at all. That can be useful during deliberate refactors, but it is dangerous during ordinary feature or bug work.

Set expectations early:

Make the smallest change that solves the bug. Do not refactor unrelated code.
Do not change public behavior outside this path. If you think a broader cleanup
is warranted, describe it separately instead of implementing it.

Then enforce that boundary during review. If the agent changed ten files when two would do, ask why. If the answer is not compelling, trim the change.

Small diffs are not just easier to review. They are easier to roll back, easier to reason about in production, and easier to explain to the next person who has to debug the system at 2:00 AM.

Make Tests Part Of The Contract

An agent-generated change without tests is not automatically bad, but it should make you pause.

Tests are one of the best ways to keep the conversation grounded. Instead of asking for "working code," ask for:

  • A regression test that fails before the fix.
  • A unit test for the edge case being changed.
  • An integration test if the behavior crosses boundaries.
  • A short explanation of which existing tests were not sufficient.

This is especially useful because AI agents can be overly satisfied with their own implementation. They may update a test to match the new behavior without proving that the old behavior was wrong. They may mock away the very integration you needed to exercise. They may add coverage that looks respectable but never asserts the thing you care about.

Review tests with the same suspicion you bring to code. Ask:

  • Would this test fail against the previous bug?
  • Does it assert behavior or merely execution?
  • Does it encode the public contract?
  • Is it too tightly coupled to implementation details?

If the test does not protect the behavior, it is decoration.

Do Not Outsource Architecture

AI coding agents are particularly tempting when you are faced with architecture work: "Design the new plugin system," "Migrate this service to event-driven processing," or "Replace this homegrown auth flow."

They can help. They should not decide.

Architecture is mostly tradeoffs, and tradeoffs are rooted in context: team skill, operational maturity, product direction, compliance constraints, latency budgets, deployment habits, and the scars of previous decisions. The agent can describe patterns. It can sketch interfaces. It can compare options. It cannot know which tradeoff your organization is willing to live with unless you tell it.

A better architecture prompt looks like this:

Compare three approaches for adding async job processing to this Django app:
Celery with Redis, a managed queue, and a simple database-backed job table.
Evaluate operational complexity, failure modes, observability, local
development, and migration risk. Do not implement yet.

That keeps the agent in the role of analyst. You remain the engineer.

Once you choose a direction, you can have the agent help with the first slice: interface definitions, a thin adapter, a migration plan, or a test harness. The important part is that the decision belongs to someone accountable for the system after the pull request merges.

Watch For Plausible Nonsense

AI agents rarely fail by saying, "I have no idea." They fail by producing something that looks normal.

That is the hard part.

Plausible nonsense in code often takes a few familiar forms:

  • Calling APIs that do not exist in the version you use.
  • Handling the happy path while ignoring retry, timeout, or rollback behavior.
  • Treating a distributed systems problem like a local function call.
  • Adding configuration without documenting how it is deployed.
  • Introducing hidden coupling between modules.
  • Deleting "unused" code that is reached dynamically.
  • Making tests pass by weakening assertions.

This is where experience matters. A senior engineer reviewing an AI-generated diff should be asking the same questions they would ask of any substantial pull request:

  • What assumptions does this change make?
  • What happens when the dependency is slow, unavailable, or returns malformed data?
  • What is the migration story?
  • How do we observe this in production?
  • Does this make the next change easier or harder?

If the agent cannot answer those questions, the diff is not done.

Treat Prompting As Engineering Surface Area

If your team uses coding agents regularly, prompts become part of your engineering process. That means they deserve the same care as other developer tooling.

At minimum, teams should agree on a few reusable prompts:

  • Bug investigation prompt.
  • Small implementation prompt.
  • Test-writing prompt.
  • Code-review prompt.
  • Documentation update prompt.
  • Refactor planning prompt.

Those prompts should include expectations around scope, tests, security, and review. This is closely related to secure prompt design, which I covered in How to Write Secure Prompts for AI-Driven Developer Workflows. The same principle applies here: clear inputs, clear boundaries, and clear output expectations reduce chaos.

You do not need a giant prompt framework on day one. A versioned prompts/engineering/ directory can be enough:

prompts/
  engineering/
    investigate_bug.md
    implement_small_change.md
    review_diff.md
    write_regression_test.md

The goal is not ceremony. The goal is to stop every engineer from rediscovering the same prompt hygiene lessons the hard way.

Use Agents To Improve The Pull Request, Not Hide It

A good AI-assisted pull request should be easier to review, not harder.

Use the agent to generate a crisp summary:

  • What changed?
  • Why did it change?
  • What tests were run?
  • What risks remain?
  • What follow-up work was intentionally left out?

Use it to update docs. Use it to add comments where the code is genuinely non-obvious. Use it to find call sites you may have missed. Use it to draft a rollback note for operational changes.

But do not let the agent bury review risk under a polished paragraph. The PR description should make the change more inspectable. It should not become a sales pitch for the diff.

This is also where internal developer tooling can help. If your organization is building AI into portals, review workflows, or service catalogs, connect those systems to real metadata rather than vibes. I wrote more about that in Beyond Git: Using LLMs to Power Your Internal Developer Portals. Agents become much more useful when they can see ownership, deployment history, runbooks, and service boundaries.

A Practical Team Policy

If I were introducing AI coding agents to an engineering team, I would start with a lightweight policy:

  • Agents may inspect code, propose plans, implement scoped changes, and draft tests.
  • Humans must approve the intended approach before broad refactors or architecture changes.
  • Agent-generated code requires the same review standard as human-written code.
  • Security-sensitive, data-handling, authentication, authorization, billing, and infrastructure changes need extra scrutiny.
  • Every non-trivial agent-assisted change should include tests or explain why tests are not appropriate.
  • Pull requests should disclose meaningful AI assistance when it affects review expectations.
  • Agents should not be given secrets, private keys, production credentials, or broad access they do not need.

That policy is intentionally boring. Boring is good here. The point is to make AI assistance normal enough to use and constrained enough to trust.

The Judgment Loop

The best mental model I have for AI coding agents is a judgment loop:

  1. Human sets intent.
  2. Agent explores or implements.
  3. Human reviews the reasoning and diff.
  4. Tests and tools provide independent feedback.
  5. Human decides whether the result belongs in the system.

If that loop is healthy, agents can speed up real work. If that loop collapses, the team starts confusing generated output with engineering progress.

And that is the line worth defending.

The future of software engineering is not humans typing every character by hand. It also is not agents spraying code across repositories while engineers become professional approvers. The useful middle is more disciplined than the hype and more interesting than the fear.

Use the agent. Keep your hands on the judgment.

For more practical engineering leadership and developer tooling notes, visit Slaptijack.

Slaptijack's Koding Kraken