When the DOM Lies, Read the Network
A Skynet field note on why CDP network evidence sits beside DOM automation, and why observation, interception, and route replay require different controls.
Key Takeaways
- DOM stays primary. User-visible DOM state remains the proof surface for browser actions; network evidence is a second witness, not a replacement.
- Observation is not interception. CDP Network telemetry and CDP Fetch interception are different power levels and need different permissions and failure assumptions.
- A route is not an API. Sessions, nonces, signed parameters, side effects, service workers, and schema drift can make discovered request reuse unsafe.
- Escalate deliberately. Use DOM first, passive network observation second, scoped interception when justified, and replay only after the invariants are proven.
A browser agent does not have one source of truth. It has at least two.
The first is the page the user can see: headings, buttons, forms, dialogs, focus, accessible names, and the state represented in the DOM. That is still where an agent should prove that a user-visible action actually happened.
The second is the browser’s network layer: the requests that left, the responses that came back, the status codes, headers, timing metadata, and, when permitted, the response bodies behind the interface.
The useful shift for us was not to abandon the DOM. It was to stop treating the DOM as the only evidence surface.
The page can be right while the workflow is wrong
A green button, a toast, or a changed label can look convincing and still be weak proof. The UI may have changed before a server-side operation completed. A page can render stale cached state. A client can optimistically update before a request fails. Conversely, the network can show that a request succeeded while the page remains in the wrong user-visible state.
That is why our browser tooling already listens below the page when a workflow needs stronger evidence. In the Skynet codebase, we use Chrome DevTools Protocol network primitives including Network.enable, request and response events, and Network.getResponseBody in scoped browser-control paths. Those primitives give us another receipt to compare with what the page claims happened.
The important word is compare. Network evidence is not a replacement for the page. It is a second witness.
Observation, interception, and replay are not the same thing
Deep Research forced a useful distinction that is easy to blur when people talk about “using CDP.” There are three materially different operations.
Passive observation is the least invasive. Chrome DevTools Protocol’s Network domain can expose lifecycle events such as requests being sent and responses being received, along with response metadata and, for eligible requests, response bodies. The browser continues doing what it was already doing. We are observing the transaction, not rewriting it.
Interception is stronger. CDP’s Fetch domain can pause matching requests. Once paused, the controller has to make an explicit choice: continue the request, fail it, or fulfill it with a synthetic response. That is not just telemetry. The automation system has inserted itself into the browser’s execution path.
Replay or route reuse is a third step again. Here the system takes knowledge learned from browser traffic and tries to issue or reuse a request directly. This can be attractive for deterministic read paths, but it changes the trust boundary. The request may depend on browser cookies, short-lived tokens, CSRF values, signed URLs, ephemeral parameters, client-side state, or a sequence of earlier actions. A discovered endpoint is not automatically a stable API.
Those three layers deserve different permissions, tests, and failure handling. Treating them as one feature is how a debugging aid quietly turns into an unsafe execution path.
Why “just call the endpoint” is a trap
The network view makes modern web applications look simpler than they are. You click a control, see an XHR or fetch request, inspect a JSON response, and it is tempting to conclude that the UI can now be skipped.
Sometimes it can. But the request alone does not tell you which invariants made that request valid.
Authentication may be bound to a browser session. A nonce may expire. A signed URL may be good for one operation or one minute. A request body may contain an opaque value created by JavaScript. A service worker may sit between the page and the network. An endpoint may be safe to read repeatedly but dangerous to replay if it creates, deletes, purchases, sends, or publishes something. An undocumented route can also change without notice because the site never promised it as a public API.
CDP itself carries a versioning warning here: the tip-of-tree protocol changes frequently and does not promise backward compatibility. If the debugging protocol can drift, an undocumented application route certainly can.
So the engineering question is not “Can we replay this request?” It is “What state makes this request valid, is the operation idempotent, and what independent evidence proves the result?”
The DOM still matters
It would be a mistake to swing too far the other way.
Modern browser automation frameworks have become better at surviving UI change when locators are based on user-facing semantics rather than brittle CSS or XPath structure. Playwright, for example, recommends locators built around roles, labels, text, and test IDs, and its locators re-resolve elements as the DOM changes. That does not make every UI stable, but it means “DOM automation is inherently brittle” is too crude a claim.
More importantly, some truths only exist at the user-facing layer. Is the dialog actually open? Is the destructive action visibly confirmed? Did the editor retain the text after navigation? Is the control disabled? Did the page route to the intended record? The network can help explain those outcomes, but it cannot substitute for proving them.
For us, the stronger pattern is layered evidence: use semantic DOM interaction for the user-visible workflow, then consult protocol-level network evidence when the operation needs a harder receipt or when the interface is ambiguous.
A practical escalation rule
Our field rule is becoming simpler:
DOM first. Network observation second. Interception only when justified. Replay only when the invariants are proven.
That ordering matters because each step increases power and risk.
A DOM action stays closest to what a human did. Passive observation adds evidence without changing execution. Interception changes execution and therefore needs tighter scope. Replay can bypass parts of the browser workflow entirely, so it should be reserved for deterministic operations whose authentication, side effects, expiry behavior, and failure recovery are understood.
This is also why we are not claiming a production “route cache” milestone or a universal performance gain here. We have real CDP network-observation primitives in our tooling. The broader route-reuse pattern is an architectural option that should earn its way into production through first-party tests, not through an attractive diagram.
The browser is not one interface
A browser agent that only sees pixels is missing structure. An agent that only sees the DOM is missing transport. An agent that only sees the network is missing the user experience.
Reliable browser automation needs to know which surface is answering which question.
The DOM tells us what the user can interact with and what state the interface presents. The network tells us what transactions the browser attempted and what came back. Interception lets us deliberately alter that path. Replay steps outside the normal page flow and therefore carries a different class of assumptions.
That separation is more useful than declaring one layer superior.
When the page is ambiguous, read the network. When the network says success, still check the page. And when a discovered route looks reusable, treat that as a hypothesis to prove, not a shortcut to trust.
