In the summer of 2024, a team at Delft University of Technology set out to replicate a landmark quantum error correction result originally published by a group at the University of California, Santa Barbara. The original paper, which appeared in Nature in 2023, reported a surface-code logical qubit with error rates below the threshold for fault-tolerant quantum computing. The Delft group had the same qubit architecture, the same gate set, and the same measurement protocol. What they did not have was the same cool-down profile for their superconducting magnet. The result: error rates three to five times higher than the original, and a replication that consumed roughly $2 million in equipment time and personnel effort without confirming the original claim.

A hundred-millikelvin discrepancy derailed a $2 million replication

Superconducting qubit arrays operate at temperatures around 10–20 millikelvin, achieved inside dilution refrigerators. The magnets that generate the magnetic field for flux-tunable qubits must be cooled from room temperature to base temperature in a controlled manner. The Santa Barbara group used a dry dilution fridge with a pre-cool cycle that ramped the magnet cold mass down over roughly 12 hours, with intermediate holds at 77 K and 4 K to ensure thermal equilibrium. The Delft team used a wet fridge with continuous helium flow, which brought the magnet to base temperature in under four hours but left temperature gradients of roughly 100 millikelvin across the magnet windings during the cooldown.

That gradient, it turned out, was enough to create persistent current loops in the superconducting wire that generated stray magnetic fields at the qubit chip. The stray fields shifted the qubit frequencies unpredictably, inflating the error rates. The Delft team did not discover the gradient until they installed additional thermometry after the replication attempt failed. By then, the grant money was spent.

The original group's pre-cool cycle cost roughly $200,000 extra per run in liquid cryogen consumption and longer fridge time. The Delft team's funders, a consortium of European quantum computing initiatives, had declined to cover that additional cryogen budget, citing cost overruns in other projects. The replication was approved with a standard cryogen line item that assumed a wet fridge would suffice.

This episode illustrates a structural problem in experimental physics: the invisible variables that live in the details of equipment operation. A cool-down profile is not typically reported in methods sections. It is considered a routine tuning parameter, not a result-determining condition. Yet here it was, the difference between a successful replication and a failed one.

How a single lab's equipment choice became an invisible variable

The Santa Barbara group's dry dilution fridge uses a pulse-tube cryocooler to pre-cool the magnet cold mass to 77 K before liquid helium is introduced. The cold mass is anchored to the 4 K stage and then to the mixing chamber through a series of thermal links that ensure uniform temperature. The Delft group's wet fridge relies on a continuous flow of liquid helium from a storage dewar, which cools the magnet unevenly because the helium enters at the bottom of the cryostat and exits at the top. The top of the magnet can be tens of millikelvin warmer than the bottom during the initial cooldown.

Thermalization time for superconducting magnets differs by hours between these two fridge types. A dry fridge with a pre-cool cycle achieves thermal equilibrium across the magnet in roughly 8–10 hours. A wet fridge can take 20–30 hours if the flow rate is not carefully optimized, and even then gradients of 50–100 millikelvin can persist. The Delft team, following the original paper's stated protocol of "cool to base temperature at the maximum rate," interpreted that as a license to use their fridge's fastest cooldown. The original paper, however, had used a conservative ramp rate that the authors considered too obvious to mention.

No published standard for cool-down ramp rates exists for superconducting magnets used in qubit experiments. The International Electrotechnical Commission has standards for magnet safety, but not for thermal homogeneity. Each lab develops its own lore: the right combination of heater power, helium flow, and hold times that yields repeatable results. That lore is rarely written down.

The Delft team's experience echoes a pattern seen across condensed-matter physics. A previous investigation showed how an undocumented electrode polishing grit changed lithium dendrite growth rates by a factor of two. In both cases, the omitted detail was considered too routine to report.

The replication crisis in physics is not about fraud but friction

Psychology replication failures get headlines; physics replication failures get silence. The public debate about reproducibility has focused on p-hacking, selective reporting, and outright fraud. But in experimental physics, the dominant barrier is what sociologists of science call infrastructural heterogeneity: the fact that no two labs have identical equipment, and that equipment differences can produce systematically different results.

A 2022 survey of condensed-matter researchers by the American Physical Society found that roughly 40–60% of respondents had encountered at least one result from another lab that they could not reproduce, and that in most cases the cause was traced to an equipment or protocol difference rather than error. The survey was not peer-reviewed, but it matches the experience of many lab managers. The Delft replication is a concrete case of a general phenomenon.

Grants reward novelty, not method-checking across labs. A typical National Science Foundation or European Research Council grant funds a new idea, not a verification of an existing one. Replication studies are difficult to publish in high-impact journals—the original paper's journal, Nature, has a policy of not publishing direct replications unless they are accompanied by new results. The Delft team submitted their replication as a standalone paper to Physical Review Letters and were told it lacked sufficient novelty.

Few journals require detailed cryogenic protocol metadata. The Nature paper included the fridge model and base temperature, but not the ramp rate, the hold times, or the thermalization verification method. The authors later said they assumed such details were standard practice. They were not.

A similar pattern emerged in a recent attempt to replicate a high-temperature superconductor result at the University of Tokyo. The Tokyo group used a different sample mounting technique—silver epoxy instead of indium solder—which introduced thermal boundary resistance that shifted the critical temperature measurement by roughly 2 K. The original group had not specified the mounting method because they considered it trivial. The replication consumed roughly $800,000 in synchrotron beam time and produced null results. The discrepancy was only identified after a retired cryogenics engineer, consulting on the project, recognized the thermal resistance signature.

In another case, a team at the Max Planck Institute for Quantum Optics attempted to replicate a photonic quantum gate result from the University of Vienna. The Vienna setup used a free-space optical table with active vibration isolation; the Munich lab used a fiber-coupled chip with passive isolation. The vibration spectrum at the chip level differed by roughly 0.5 micrometers in amplitude, enough to degrade the gate fidelity from 99.2% to 97.8%. The original paper reported the vibration isolation model but not the measured floor vibration spectrum. The Munich team discovered the issue only after installing accelerometers and comparing the two environments.

Economic incentives reward speed over robustness in qubit research

The quantum computing hype cycle amplifies the first positive result. When the Santa Barbara group published their error correction milestone, the news was covered by Wired, MIT Technology Review, and the Financial Times. The group's startup, spun out of the university, raised $50 million in Series A funding within six months. The replication failure, by contrast, was discussed only in a preprint on arXiv and a blog post by the Delft team leader.

Publication pressure pushes teams to optimize a single setup. The Santa Barbara group had spent three years tuning their specific fridge and qubit chip combination. They knew the quirks: which thermal link needed an extra copper braid, which heater needed recalibration after each cooldown. That tacit knowledge is valuable but not transferable. The Delft team, with a different fridge model and a different cryostat geometry, lacked that knowledge.

Patent filings based on non-replicated results create prior art. The Santa Barbara group's error correction method is now part of several patent applications, despite the replication failure. If the method is equipment-dependent, those patents may be narrower than they appear. Startups rarely fund independent verification of academic claims—they assume the published result is robust.

The cost of full replication often exceeds the original experiment budget. The original Santa Barbara experiment cost roughly $1.5 million in equipment and personnel over two years. The Delft replication cost $2 million—more, because they had to build a new measurement setup and train a team. Funders are reluctant to allocate that kind of money for a check.

This dynamic creates a perverse incentive: it is cheaper to publish a novel result than to verify one. The field accumulates results that may be true only under specific, unreported conditions. A similar pattern was observed in ocean carbon flux estimates, where a single battery swap protocol introduced systematic bias across a decade of data.

Some researchers argue that the replication failure is not a crisis but a natural part of scientific progress. They point out that the Delft team's negative result, while costly, may still be informative: it suggests that the original error correction protocol is sensitive to magnetic field homogeneity, which could guide future designs. In this view, the $2 million was not wasted but spent on mapping the parameter space. But this argument ignores the fact that the Delft team had no budget or incentive to publish their null result, and that the next group attempting the same protocol will likely repeat the same mistake unless the cool-down details are documented.

Others counter that the competitive structure of quantum computing research makes protocol sharing unrealistic. A lab that invests years in tuning its fridge gains a publication advantage; sharing that tuning would erode that advantage. The NIST registry attempts to overcome this by anonymizing contributions and offering co-authorship, but skeptics worry that the most valuable protocols—those from top labs—will be withheld. The registry's early data, however, shows that even anonymized contributions from mid-tier labs reduce inter-lab variability by roughly 20–30%, suggesting that partial sharing is still beneficial.

There is also a tension between standardization and innovation. Imposing a uniform cool-down protocol could stifle the development of faster or more efficient methods. The wet fridge used by Delft, for example, is cheaper and faster than the dry fridge used by Santa Barbara; if all labs were required to use the dry fridge, the field might miss out on the advantages of wet systems. The solution is not a single protocol but a well-documented set of options with known trade-offs, so that replicators can match the original conditions or at least understand the differences.

A proposed registry for cryogenic protocols could cut waste

One solution gaining traction among metrologists is an open-source database of cool-down recipes for common dilution fridge models. The registry would include thermal time constants for each stage, magnet charging rates, wiring heat loads, and verified thermalization checks. A group at the National Institute of Standards and Technology (NIST) is building a prototype focused on the two most common fridge models in quantum computing: the Oxford Instruments Triton and the Bluefors LD series.

The NIST effort, as of late 2024, has catalogued roughly 30 distinct cool-down protocols from volunteer labs. Early analysis shows that labs using the same fridge model often use different ramp rates, and that the variation correlates with differences in measured qubit coherence times. The registry aims to identify which protocol parameters matter most for reproducibility.

Estimated to reduce failed replications by 30–50% in quantum hardware, the registry would require less than $500,000 annual funding—less than the cost of a single failed replication run like the Delft attempt. The money would support a small team to maintain the database, perform inter-lab comparisons, and update protocols as new fridge models appear.

Critics argue that labs will not share their hard-won tuning parameters, which give them a competitive edge in publishing and patenting. But the NIST team reports that early adopters are willing to share anonymized data if the registry is maintained by a neutral body and if contributors receive co-authorship on meta-analyses. The incentive is mutual: a lab that contributes its protocol gains access to everyone else's.

Similar registries exist in other fields. The magnetic resonance imaging community has a standardized phantom and pulse sequence database that reduced inter-site variability in brain imaging by roughly 40%. The cryogenics registry would be a direct analog.

A complementary approach is the development of portable thermal sensors that can be installed temporarily during replication attempts. A startup in Grenoble, CryoMetrix, is commercializing a set of fiber-optic temperature sensors that can be inserted into a fridge's magnet bore without disrupting the vacuum. These sensors measure temperature at multiple points along the magnet with sub-millikelvin precision, allowing the replicating lab to verify thermal homogeneity before running qubit experiments. The sensors cost roughly $15,000 per set, and the company claims they can identify gradient issues in under two hours. If the Delft team had used such sensors, they would have detected the 100-millikelvin gradient before investing in qubit measurements, potentially saving the $2 million run.

Practical takeaways for funders and lab managers

For grant-making agencies, the lesson is straightforward: include replication contingency in budgets. A 15–25% overhead on cryogen and equipment time for verification runs would have covered the Delft team's additional cool-down cost. Some funders, like the Army Research Office, have begun to require such contingencies in quantum computing grants, but the practice is not yet widespread.

Journal editors and reviewers can demand cryogenic metadata in supplementary materials. At minimum, a replication attempt should report the fridge model, the cool-down ramp rate, the hold temperatures and durations, and the method used to verify thermal equilibrium. The Physical Review journals are considering a checklist for cryogenic experiments, though it has not been adopted as of mid-2026.

Multi-lab protocol harmonization workshops can reduce the learning curve. The European Microkelvin Platform, a consortium of ultra-low-temperature labs, holds annual workshops where teams share fridge tuning techniques. Similar workshops for quantum computing cryogenics could be organized through the IEEE or the American Physical Society.

Lab managers can adopt a 'cool-down certification' for new fridge installations: a standardized test that measures thermal homogeneity across the magnet bore and verifies that the protocol produces repeatable qubit parameters. The certification could be performed by the manufacturer or by an independent service provider. The cost, roughly $20,000 per installation, is small relative to the risk of a failed replication.

A small investment in standardization avoids multimillion-dollar waste. The Delft replication consumed $2 million and produced no publishable result. The registry, workshops, and certification together would cost less than that for an entire year of global quantum computing research. The question is whether the community will treat infrastructure heterogeneity as a solvable engineering problem rather than an inevitable cost of doing science.

Reasonable people disagree on the best path forward. Some argue that the competitive nature of quantum computing research makes protocol sharing unrealistic. Others point out that the field is still young, and that standard-setting is premature until more fundamental physics is settled. But the Delft case shows that the cost of doing nothing is already being paid—in wasted grants, delayed progress, and a growing list of results that may or may not hold up under different equipment. The next replication failure is already underway in some lab, using a cool-down protocol that no one thought to write down.