Testing starts before any code runs against a live situation. We map the manual process the system is replacing first, so we know what a correct decision actually looks like before we can grade the system against it. Then we build guardrails around it: which tools it's allowed to touch on its own, and which actions still need a person to approve before they go out.
Once it's built, it runs in shadow mode against real or recent data. It makes its decisions and would take its actions, but instead of sending the email or placing the order, the action gets logged instead of executed. We push it through cases meant to break it: a tool that times out, input it wasn't expecting, a decision that's genuinely ambiguous. Then we watch what happens: does it retry sensibly, escalate to a person, or fail without telling anyone. Silent failure is disqualifying.
Go-live is gradual, not a switch flip. We start it on a narrow slice, one channel or one type of decision, and widen its scope once it's run clean for a while. Because we operate the systems we build instead of handing over code and leaving, monitoring doesn't stop at launch: we keep watching the logs, and anything the system isn't confident about keeps routing to a person rather than getting guessed at. We run this same process on our own systems too. Precipitate currently operates 88 systems and more than 110 scheduled jobs across 40-plus integrations, and each one went through a dry run before it touched anything live.