DeepView: Cutting AI-Accelerator Debugging From Weeks to Minutes

Last week I presented DeepView [1], a paper I co-authored with my team at IBM Research, at the IEEE International Conference on Software Services Engineering in Sydney, part of the IEEE World Congress on Services. It comes out of a problem we hit again and again, one I think is among the least glamorous and most expensive in AI infrastructure today. This is the practitioner's version of that talk. The code is open source, and the slides and repository are linked at the end.

The challenges of enabling a model onto a new chip

Every few months a new AI accelerator arrives. Dataflow processors, neuromorphic chips, analog in-memory compute, and a growing list of inference ASICs are all competing to run large language models for less money and less power than a GPU. IBM's own Spyre accelerator is one of them, a 5nm inference card that shipped for commercial availability in late 2025 [2]. The pitch is always some version of cheaper tokens.

But before any of that is real, someone has to get your model running on the new chip. That step sounds like a formality. It is where the weeks go.

PyTorch made it look easy on paper. Since PyTorch 2.0, a hardware vendor can register a custom backend through torch.compile without asking you to rewrite anything [3]. You move the model to the new device, you call torch.compile, and in theory the same model code that ran on a GPU now runs on the accelerator. These vendor backends that live outside the main PyTorch tree are called out-of-tree backends, and they plug in through a dispatch mechanism PyTorch calls PrivateUse1 [4].

In practice the model breaks. It breaks in a way that is hard to locate, hard to reproduce, and hard to hand to the team that can actually fix it. A senior engineer opens a log, sees a compiler error from deep inside the vendor stack, and starts the slow work of guessing which part of a model with hundreds of submodules set it off.

And here is why that cost never goes away. It is not one bug you fix once. It is a grid. Every model you care about has to be validated against every backend you care about, and every new backend restarts the whole column. We call it the models-by-backends matrix, and it is where a lot of AI hardware budgets are spent.

Fig. 1: Each new model must be checked on every backend, and each new backend re-checks every model. The expensive part is not any single failure. It is the same debugging work, repeated across the whole grid.

Three ways a model fails on new hardware

When we looked at what actually goes wrong, the correctness-blocking failures fell into three categories. They map to three stages of getting a model to run, and you have to clear them in order, because each one hides the next.

Fig. 2: A model cannot run until its operators are supported, cannot be checked until it runs, and cannot be trusted until its numbers match a reference. DeepView gives one diagnostic mode per stage, and each mode emits a concrete artifact you can act on.

Unsupported operators, at compile time. A backend implements a subset of PyTorch's operators. When your model reaches for one the backend has not built yet, compilation fails. A state-space model, for example, uses operators that a transformer-only backend has never needed.

Layer-level failure, at run time. The operators all exist, the model compiles, and then it dies during execution with an error that points at the compiler's own internals and tells you nothing about your model. Something like a failed C++ assertion inside the vendor's lowering code. Which of your model's layers triggered it? The error will not say.

Numerical divergence, at verify time. This is the dangerous one. The model compiles, runs, and returns an answer. Nothing crashes. But somewhere inside, reduced precision or a different kernel implementation has pushed an intermediate result off from what a CPU or GPU would produce. The final output still looks plausible, so nobody notices until quality quietly degrades in production.

What DeepView actually does

DeepView is a debugging framework that targets those three failures directly, and it is built on a single PyTorch feature: the forward hook.

A forward hook is a callback you attach to any module. After that module computes its output, PyTorch hands your callback the module, its inputs, and its outputs [5]. DeepView attaches one to every submodule in the model, watches the whole forward pass, records what it needs, and removes the hooks when it is done. It never modifies your model's source and it never reaches into the backend's internals. That is the whole mechanism, and it is what makes DeepView work on any model expressible as a PyTorch module and any backend reachable through torch.compile.

In code, the core of it is short enough to read in one sitting:

def capture(name):
    def hook(module, inputs, output):
        records[name] = {
            "shape": output.shape,
            "dtype": output.dtype,
            "output": output,   # kept for the numerical comparison
        }
    return hook

# Attach a hook to every submodule. The model itself is not modified.
hooks = [m.register_forward_hook(capture(n)) for n, m in model.named_modules()]

with torch.no_grad():
    model(**inputs)          # one normal forward pass on the target backend

for h in hooks:
    h.remove()               # clean up; the model is left exactly as it was

Each mode reads from those records. Mode 1 inspects the failure the compiler raised before a module could run, Mode 2 finds the last module that ran before execution stopped, and Mode 3 compares each module's stored output against a reference. Same hooks, three different questions.

Fig. 3: DeepView registers a forward hook on every submodule. Each hook sees that module's inputs and outputs during a normal forward pass, with no edits to model code and no backend-specific instrumentation. All three modes read from this one mechanism.

On top of that shared mechanism sit three modes, one per failure stage. Each one emits an artifact the backend team can act on without re-running your whole environment. We validated all of it on IBM's Spyre as a representative out-of-tree backend, but nothing in the approach is tied to Spyre.

Mode 1: find the missing operators

When compilation hits an operator the backend does not support, DeepView catches the error at the graph level and captures everything about it: the operator name, the input shapes and data types, the exact submodule it came from, and the stack trace. Then it auto-generates a small, standalone Python script that reproduces that one operator failure on a synthetic input. A compiler engineer can run that script on any machine with the backend installed, without touching the full model.

We ran this on Bamba-9B, IBM's open-source hybrid model that pairs transformer layers with a state-space component [6]. Because of that state-space part, it reaches for operators outside the usual transformer set. In a single automated run, DeepView surfaced all seven missing operators on the Spyre backend at the time: softplus, constant_pad_nd, copy, exp, roll, slice_scatter, and select_scatter. Each came with its own reproduction script.

Doing that by hand had taken engineers around five days of investigation. DeepView brought it down to minutes. Generating the reproduction case for each operator dropped from over thirty minutes to about thirty seconds. And because the output is a structured list, you can aggregate it across many models and rank operators by how many models each one is blocking, so the backend team builds the most valuable operator first instead of guessing.

Mode 2: find the layer that dies

Sometimes every operator exists, the model compiles, and it still fails at runtime. The error you get is from inside the compiler. In one real case on Spyre it was a failed C++ assertion in the vendor's layout code, with no hint of which model layer was responsible.

DeepView's layer-debugging mode runs the model under the backend with hooks on every submodule, walks the hierarchy, and records where execution stops. For Granite-3.2-2B it pinned the failure to model.base_model.layers[0].attn and generated a reproduction script targeting just that attention submodule. Localizing the fault went from one to two weeks of manual hunting to minutes.

Mode 3: catch the silent drift

The third mode is the one I care about most, because the failure it catches is silent. The model runs and produces a reasonable answer while an intermediate layer is quietly wrong.

To measure "wrong," DeepView compares each layer's output against a trusted CPU or GPU reference using two numbers. The first is mean absolute difference, the average size of the element-wise gap, which tells you how big the error is. The second is cosine similarity, which measures whether two vectors still point the same direction regardless of their magnitude. You need both. In the high-dimensional hidden states of a transformer, direction is what carries meaning, so two activations can differ in size while still meaning the same thing, and they can also look close in raw magnitude while pointing somewhere else entirely. Distance metrics behave in unintuitive ways once you are in hundreds of dimensions, which is a well-studied result [7], and it is exactly why one metric alone will mislead you.

The harder question is what counts as too much drift. A single fixed tolerance does not work, because the natural, acceptable variation between two correct pieces of hardware differs from layer to layer and model to model. So DeepView derives the thresholds from data. It runs the model on CPU and GPU, two references everyone already trusts, measures the layer-by-layer difference between them, and takes a high percentile of that as the expected variation for each layer. Any new accelerator is then checked against each layer's own threshold.

Fig. 4: First DeepView measures the per-layer difference between CPU and GPU, two references known to be correct, and stores a threshold for each layer. Then it runs the accelerator and flags any layer whose deviation exceeds its own threshold. The thresholds are derived once per model and reused on every backend.

Run across three production models on Spyre, this mode found the same feedforward normalization layer failing in all three. That kind of pattern is a strong signal for the compiler team, because it points at one shared root cause rather than three unrelated bugs.

Layer layers[0].ff_ln on IBM Spyre. The observed gap is orders of magnitude over threshold and cosine similarity falls below 1.0 in every case, yet all three models still produced sensible-looking output.

Here is the part that should worry anyone shipping models on new hardware. While that layer was failing its threshold, the model still returned plausible answers. Ask it to write a sorting routine and you get a working one. Nothing looks wrong from the outside, partly because later layers absorb some of the error, which is exactly what hides it. Without a tool comparing layer by layer, you would not know the drift was there until it showed up as degraded quality somewhere downstream, long after it would have been cheap to catch.

Results: from weeks to minutes

Across all three modes, the story is the same. Diagnosis that used to take days or weeks takes minutes.

Fig. 5: Order-of-magnitude reductions across the three failure categories. The "before" figures come from internal issue trackers and engineering reports across real bring-up campaigns, not a controlled study, so read them as evidence of scale rather than precise measurements.

I want to be honest about what those numbers are and are not. They are snapshots of the Spyre stack at one point in time, and the stack has moved on since; many of the operators and layers we flagged in early evaluations have been fixed. The results are illustrations of what the diagnostic surfaces and how much engineering effort it saves, not a scorecard on any chip. That distinction matters, because the value of DeepView is not that it found these particular bugs. It is that it finds this class of bug quickly, on any backend, every time a new one arrives.

DeepView runs automatically in a CI/CD pipeline

The reason we presented this at a software-services venue and not only a hardware one is that the interesting part is making the diagnosis repeatable. DeepView ships as a pip-installable command-line tool. It is flag-driven and non-interactive and writes its results to a file, which means you can drop it into a continuous-integration pipeline and check model-backend compatibility automatically, the same way you already run tests. The first two modes add under five percent to a single inference pass. The divergence mode costs more, roughly two to three times a normal run, because it needs a reference pass plus a per-layer re-run, but it is a bring-up diagnostic you run on demand, not something on the serving path.

It also generalizes further than it might sound. There are more than a million models on Hugging Face, but for the purpose of this kind of debugging they collapse into a handful of architectural families: transformer decoders, mixture-of-experts, state-space models, vision transformers, and a few others. Because DeepView only relies on the module hook mechanism that every one of them shares, it already ran on a vision model and a multimodal model with no model-specific work at all.

My take, from the systems side

The hardware conversation is loud and the tooling conversation is silent, and I think that is backwards. We spend enormous effort designing accelerators and comparatively little on the unglamorous work of getting real models onto them and proving the numbers are right. That gap is not a modeling problem or a chip problem. It is a systems and software-engineering problem, and it compounds: every backend without good diagnostics slows down every model that has to run on it. What this work shows me is that a small, backend-agnostic tool built on a stable interface will outlast any single chip, because these failures are structural, not specific to one vendor.

What comes next

DeepView today localizes failures to a layer. The next step is to go deeper into the compiler. We want to map a failing layer to the exact generated kernel that produced it, and to bisect the compiler's own transformation passes to find the first one that introduces a bad graph or a numerical error, which is essentially git bisect applied to a compiler pipeline. Beyond correctness, the same hook-based approach can be pointed at memory allocation failures, recompilation, and distributed-execution bugs, which are the next set of problems you hit after the model is merely running.

None of that changes the core idea, which is small enough to state in a sentence. If you instrument the boundaries between a model's modules with something that does not care which backend is underneath, you can turn the vague, expensive failures of hardware bring-up into localized, reproducible, automatable ones. That is the whole point of DeepView, and it is why I think this part of AI infrastructure deserves more attention than it gets.

DeepView is open source. You can read the paper on IBM Research, see the slides from the IEEE SSE 2026 talk, and get the code at github.com/IBM/deepview. It will also be on arXiv shortly and in the IEEE Xplore proceedings once it is published.

---

References

[1] Kaoutar El Maghraoui, Jordan Sullivan, Seep Goel, Aishwariya Chakraborty, Priyanka Naik, Sahil Suneja, Flavia Janine Rosante Beo, Rashed Bhatti, Pablo Gonzalez, Todd Deshane, Borja Godoy, Praveen Jayachandran, Alaa Youssef, Raghu Kiran Ganti, "DeepView: A Generalizable Debugging Methodology for Enabling Large Language Models on Emerging AI Accelerators," IEEE International Conference on Software Services Engineering (SSE 2026), IEEE World Congress on Services, Sydney, Australia, July 2026.

[2] IBM, "IBM Introduces the Spyre Accelerator for Commercial Availability," IBM Newsroom, October 7, 2025. Spyre is part of IBM Research's AIU (Artificial Intelligence Unit) family.

[3] PyTorch Foundation, "PyTorch 2.x: faster, more pythonic and as dynamic as ever." The torch.compile stack (TorchDynamo, AOTAutograd, backend interface) is what vendors register against.

[4] PyTorch, "Facilitating New Backend Integration by PrivateUse1," PyTorch Tutorials, 2023. The dispatch pathway used by out-of-tree accelerator backends.

[5] PyTorch, "torch.nn.Module.register_forward_hook," PyTorch documentation. The hook receives the module, its inputs, and its outputs after the forward pass.

[6] Raghu Ganti et al., "Meet Bamba, IBM's new attention-state space model," IBM Research, April 2025. Model card: ibm-ai-platform/Bamba-9B-v1 on Hugging Face.

[7] Charu C. Aggarwal, Alexander Hinneburg, Daniel A. Keim, "On the Surprising Behavior of Distance Metrics in High Dimensional Space," ICDT 2001, LNCS 1973, Springer, pp. 420-434.