Starting point: Earlier programming chapters; relevant constructions from Quantum Computing and Quantum Algorithms.
Anchor example: Find a circuit bug that passes basis-probability tests but fails on superpositions.
This lesson includes explanations, worked examples, and problems with solutions. Advanced results whose proofs are outside the chapter are identified explicitly.
Every program so far came with checks, and they were not equally strong. Counts showed only which outcomes occurred. A full matrix caught every sign. A comparison of amplitudes on a superposition found errors that a histogram could not see. We chose each check for its example, without asking in general what a check must contain.
What evidence establishes that a quantum program implements its specification? We start from a bug that passes every test on basis inputs, and it forces tests on superpositions. Comparing on superpositions raises the question of which phase differences count, and the answer depends on whether the component will be controlled. A component also makes promises about wire order and workspace, and checks of inverses and composition test some of these promises but not others. Finally, every comparison needs a tolerance, and sampled comparisons need statistics.
The aim is a set of tests in which each one rules out a specific kind of error.
9.1 Functional Tests
Suppose a component should do nothing to its qubit, and by mistake it applies Z. Which test catches the mistake? The obvious tests do not, and seeing why tells us what a functional test must compare. We find the failing test first, then decide which differences between two operations should count as errors.
9.1.1 Basis inputs and superpositions
The first attempt runs both circuits on the basis inputs \(\ket0\) and \(\ket1\) and compares the output probabilities. Both tests pass. Since \(Z\ket0=\ket0\) and \(Z\ket1=-\ket1\), the outputs differ only by a sign, and a sign changes no probability.
What input would expose the difference? One in which the sign becomes a relative phase between two amplitudes. On \(\ket+\), the identity gives \(\ket+\), while \(Z\) gives \(\ket-\). These two states are orthogonal, the most different two states can be. A Z-basis histogram still cannot tell them apart, but a comparison of amplitudes can, and so can a measurement in the X basis.
The companion makes this argument executable. Statevector.from_label("0") builds a basis state from its label, and "+" builds \(\ket+\). The method evolve applies a circuit to a state. A loop over the tuple ("0", "1") shows that the basis probabilities agree. The final checks use not to assert that the two outputs on \(\ket+\), and the two operators, are not equivalent. So a coherent contract needs tests on superpositions, comparing amplitudes, not only labels.
9.1.2 Equivalence and global phase
If we compare amplitudes, should every difference count as a bug? Take a circuit whose only content is a global phase of 0.3, set through phase.global_phase = 0.3. On its own, it makes the same predictions as the identity, so a test that rejects it would be wrong. Operator.equiv accepts two operators that differ by one overall phase factor, and the companion asserts that the identity and this circuit pass.
Two kinds of difference are not covered by that freedom. A separate phase on each basis column is not global: \(Z=\operatorname{diag}(1,-1)\) differs from \(I\) by a phase on one column only, and the equivalence check rejects it. And under control, even a global phase becomes a relative phase on the control, as we derived for controlled operations. So the companion also checks that the controlled identity and the controlled phase circuit are not equivalent.
The rule for a test, then, is to allow a global phase only for a component that will never be controlled, and to record that assumption.
Problem 9.1. Why does testing Z on zero miss the bug?
Worked solution
Zero is an eigenstate of Z with eigenvalue one. Test plus to expose the relative sign between zero and one.
9.2 State and Workspace Checks
Functional tests check the action on the useful registers. A component promises more than that: its wires are attached in a stated order, its workspace starts and ends clean, and it can be inverted and combined with others. Which tests check these promises, and which only seem to? We take wire order and workspace first, then inverses and composition.
9.2.1 Register ordering and ancilla cleanup
Which input exposes two wires attached in the wrong order? Not a symmetric one: the Bell pair and the all-zero input look the same either way round. An asymmetric input, such as X on qubit 0 alone, makes the two orders give different outputs.
For workspace, we have three checks of increasing strength. The weakest measures the ancilla and requires outcome 0 with probability 1. A stronger one computes the reduced state of the useful registers and compares it with the predicted pure state. The strongest, possible when the system is small, compares the whole joint state with the predicted useful state times \(\ket0\) on the ancilla. A clean workspace must factor away from the useful registers in this way.
Suppose a contract promises that the ancilla starts in \(\ket0\). Then we test only inputs with the ancilla in \(\ket0\). Such tests say nothing about other ancilla states. That is acceptable, because the contract never promised anything about them.
9.2.2 Inverse and composition tests
Is “\(U\) followed by its inverse gives the identity” a good test of \(U\)? Recall that inverse() computes the inverse from \(U\)’s own gates. So the product is the identity whatever \(U\) does, right or wrong. The test checks the inverse construction, not the specification, and we pair it with a test against an independent prediction.
Composition tests have a different value. Joining two components can reveal a wire contract that one of them breaks, or a phase convention that one of them omitted. Feeding in a randomly chosen superposition is another useful check, because it exercises all the amplitudes and phases at once. For a small component, though, the complete matrix remains the decisive test. A random input gives evidence about one input. The matrix covers all of them.
Problem 9.2. Can two individually global-equivalent gates always be substituted under control?
Worked solution
No. Their differing phase acts only in the control-one branch and becomes observable relative to the control-zero branch.
9.3 Statistical Tests
Every comparison so far used a tolerance, and every sampled one used a statistical window. A tolerance that is too loose accepts bugs, and one that is too tight rejects correct programs. How do we choose these numbers rather than guess them? We treat rounding and approximation first, then sampling.
9.3.1 Numerical tolerances
How large are rounding errors in an exact calculation? A computer stores numbers with about 16 significant digits, so each operation can introduce an error near \(10^{-16}\). A small gate sequence accumulates a few of these. An absolute tolerance such as \(10^{-12}\) therefore leaves a wide margin above rounding while remaining far below any real mistake.
A circuit that is approximate by design is different. Its error is part of the method, so the tolerance must come from an analytic bound on that error. We will meet such a bound when we approximate a time evolution by a product of simpler steps.
Why not use a relative tolerance alone, accepting \(a\) when \(|a-b|\le r|b|\)? When the expected value \(b\) is 0, the condition becomes \(|a|\le0\). An amplitude that should vanish but comes out as \(10^{-17}\) then fails. So near zero we need an absolute tolerance, and a test should state which tolerance it uses. It should also state which notion of equality it uses, exact or up to global phase, and which distance it measures, entry by entry or over the whole matrix.
9.3.2 Sampling uncertainty and regression tests
A sampled test can fail even when the program is correct. For the coin we chose a window of 6.4 standard deviations, which a correct program leaves with probability about \(2\times10^{-10}\). A suite of many such tests multiplies that chance, so windows must be set with the whole suite in mind. Demanding exact counts, as we saw, rejects most correct runs.
Fixed seeds help in another way. With a seed, a sampled test gives the same result on every run, so a change signals a change in the code or environment. But a seed can also hide a test that passes only for that seed. So sampled tests sit beside deterministic checks of states and operators, which have no randomness at all.
Finally, how do we know a test can fail? We give it a wrong program. The companion includes the Z bug deliberately and asserts that the coherent test rejects it.
"""Runnable companion for Quantum Programming, Chapter 9."""
import numpy as np
from qiskit import QuantumCircuit
from qiskit.quantum_info import Operator, Statevector
identity = QuantumCircuit(1)
wrong = QuantumCircuit(1)
wrong.z(0)
# Basis output probabilities match, but relative phase differs.
for label in ("0", "1"):
state = Statevector.from_label(label)
assert np.allclose(state.evolve(identity).probabilities(), state.evolve(wrong).probabilities())
assert not Statevector.from_label("+").evolve(identity).equiv(Statevector.from_label("+").evolve(wrong))
assert not Operator(identity).equiv(Operator(wrong))
phase = QuantumCircuit(1)
phase.global_phase = 0.3
assert Operator(identity).equiv(Operator(phase))
assert not Operator(identity.to_gate().control()).equiv(Operator(phase.to_gate().control()))
print("Relative-phase bug detected; controlled global phase also checked")
It reduces false failures but can accept real bugs. Choose it from the numerical and approximation budgets, and include known incorrect examples that should fail.
Our tests compare a program with its logical specification. A device, though, will not run our circuit as written. A compiler rewrites it into the device’s own gates and wiring. How does a logical program become executable on a specific device, and how do we check that the rewritten circuit still means the same thing?
Summary
Quantum software testing combines deterministic matrix/statevector unit tests on small widths with statistical hypothesis tests on sampled shot distributions.
Adjoint testing (\(U U^\dagger = I\)) and ancilla-purity checks verify that subroutines leave scratch registers unentangled and return to \(|0\rangle\).
Statistical tests of sampled outputs must account for finite-shot variance, multiple-testing corrections, and explicit failure tolerances.
Questions and worked problems
These exercises draw on the whole chapter. Attempt each question before opening a hint, explain your reasoning, then compare it with the worked solution.
A circuit should do nothing to one qubit, but a bug inserts an S gate. Which of these tests detects the bug: (a) basis-state output probabilities for \(\ket0\) and \(\ket1\); (b) statevector equivalence on \(\ket+\); (c) Operator equivalence?
Hint 1
S changes only the phase of \(\ket1\).
Hint 2
\(S\ket+=(\ket0+i\ket1)/\sqrt2\).
Worked solution
(a) Misses it: S leaves the basis probabilities unchanged. (b) Catches it: \(S\ket+=(\ket0+i\ket1)/\sqrt2\) is not equivalent to \(\ket+\). (c) Catches it: diag\((1,i)\) is not the identity up to a global phase. Tests of coherent components must include superposition inputs or full operators.
import numpy as np
from qiskit import QuantumCircuit
from qiskit.quantum_info import Operator, Statevector
identity = QuantumCircuit(1)
buggy = QuantumCircuit(1)
buggy.s(0)
for label in ("0", "1"):
state = Statevector.from_label(label)
assert np.allclose(state.evolve(identity).probabilities(), state.evolve(buggy).probabilities())
plus = Statevector.from_label("+")
assert not plus.evolve(identity).equiv(plus.evolve(buggy))
assert not Operator(identity).equiv(Operator(buggy))
Why does np.isclose(1e-17, 0, rtol=1e-9, atol=0) return False, while np.isclose(1e-17, 0, rtol=0, atol=1e-12) returns True? Which tolerance should an amplitude test use?
Hint 1
A relative tolerance scales with the size of the reference value.
Hint 2
The reference value here is \(0\).
Worked solution
With atol=0, the allowed difference is rtol times the magnitude of the reference value, which is \(0\), so even \(10^{-17}\) fails. An absolute tolerance of \(10^{-12}\) accepts floating-point noise. Amplitude tests should include an absolute tolerance, because many correct amplitudes are exactly zero and their computed values are tiny nonzero numbers (such as the \(10^{-33}\) entries seen in earlier chapters).
import numpy as np
assert not np.isclose(1e-17, 0, rtol=1e-9, atol=0)
assert np.isclose(1e-17, 0, rtol=0, atol=1e-12)
rz(0.4) and p(0.4) differ by a global phase. Does np.allclose on their Operator matrices succeed? Does Operator.equiv? Which comparison answers “are these the same standalone gate?”
Hint 1
\(R_z(\lambda)=e^{-i\lambda/2}P(\lambda)\).
Hint 2
equiv ignores a global phase.
Worked solution
np.allclose fails, because every entry differs by the factor \(e^{-0.2i}\). Operator.equiv succeeds, because it allows a global phase. equiv answers the standalone question; allclose answers the stricter question that matters when the gate will be controlled, where the two are genuinely different. A test must use the notion of equality that matches the gate’s intended use.
import numpy as np
from qiskit import QuantumCircuit
from qiskit.quantum_info import Operator
rz = QuantumCircuit(1)
rz.rz(0.4, 0)
p = QuantumCircuit(1)
p.p(0.4, 0)
assert not np.allclose(Operator(rz).data, Operator(p).data)
assert Operator(rz).equiv(Operator(p))
Write a test that compares an implementation with its specification on a few random input states, using random_statevector with fixed seeds. Apply it to the S-gate bug of the quick check. Why are random states useful, and what do they not replace?
Hint 1
random_statevector(2, seed=...) gives a reproducible random state.
Hint 2
Compare state.evolve(implementation) with state.evolve(specification).
Worked solution
For each seeded random state, evolve it through both circuits and require equivalence; the S bug fails on every generic input, since a random state has both basis components. Random inputs exercise all relative phases at once and catch errors that special inputs such as \(\ket0\) hide. They do not replace an exhaustive operator check for small components, and they are not a proof: a bug confined to a tiny part of a large space may need many samples or a structured test.
from qiskit import QuantumCircuit
from qiskit.quantum_info import random_statevector
def agrees(implementation, specification, trials=5):
n = specification.num_qubits
for seed in range(trials):
state = random_statevector(2 ** n, seed=seed)
if not state.evolve(implementation).equiv(state.evolve(specification)):
return False
return True
specification = QuantumCircuit(1)
buggy = QuantumCircuit(1)
buggy.s(0)
assert agrees(specification, specification)
assert not agrees(buggy, specification)
A test suite runs \(20\) independent sampling tests, each with a two-sided tolerance of \(z\) standard deviations. (a) For \(z=3\), each correct test fails with probability about \(0.0027\). What is the chance that a run of the whole suite has at least one false failure? (b) Repeat for \(z=4\) (each \(6.3\times10^{-5}\)). (c) What else should be done instead of simply raising \(z\)?
Hint 1
\(P(\text{some failure})=1-(1-p)^{20}\).
Hint 2
Higher \(z\) also makes real bugs harder to detect.
Worked solution
(a) \(1-0.9973^{20}\approx5.3\%\): one in twenty runs of a correct suite fails. (b) \(1-(1-6.3\times10^{-5})^{20}\approx0.13\%\). (c) Larger tolerances reduce false alarms but also hide small bugs. Better: test deterministic properties exactly with Statevector or Operator, use fixed seeds for regression tests of sampling code, and size the shot counts so that the bugs that matter remain detectable at the chosen \(z\).
A product-formula simulation of \(e^{-i(X+Z)t}\) for \(t=1\) with \(r=10\) first-order steps has the error bound \(t^2\|[X,Z]\|/(2r)=0.1\). Write a test that compares the circuit’s operator with the exact exponential and uses this analytic bound as the tolerance. Check the actual error.
Hint 1
Build each step as rx(\(2t/r\)) followed by rz(\(2t/r\)) or equivalent.
Hint 2
Compute the exact unitary from the eigen-decomposition of \(X+Z\).
Worked solution
The circuit repeats ten steps of \(e^{-iX\delta}e^{-iZ\delta}\), implemented as rz(\(2\delta\)) then rx(\(2\delta\)), since \(R_x(\theta)=e^{-i\theta X/2}\). The exact unitary comes from diagonalizing \(X+Z\). The operator-norm error is about \(0.07\), within the bound \(0.1\), so the test passes. A tight tolerance such as \(10^{-12}\) would wrongly fail this correct approximate circuit; the tolerance must come from the approximation’s error budget.
import numpy as np
from qiskit import QuantumCircuit
from qiskit.quantum_info import Operator
t, r = 1.0, 10
delta = t / r
circuit = QuantumCircuit(1)
for _ in range(r):
circuit.rz(2 * delta, 0) # exp(-i Z delta) acts first
circuit.rx(2 * delta, 0) # then exp(-i X delta)
H = np.array([[1, 1], [1, -1]], dtype=complex)
values, vectors = np.linalg.eigh(H)
exact = vectors @ np.diag(np.exp(-1j * values * t)) @ vectors.conj().T
error = np.linalg.norm(Operator(circuit).data - exact, 2)
bound = t ** 2 * 2 / (2 * r)
assert error <= bound and error > 0.05
A biased-coin bug shifts \(P(1)\) from \(0.50\) to \(0.48\). How many shots are needed for a test with a three-standard-deviation false-alarm threshold to detect this bug with \(90\%\) probability? Use \(N\approx\big((z_\alpha+z_\beta)\sigma/\delta\big)^2\) with \(z_\alpha=3\), \(z_\beta=1.28\), \(\sigma=0.5\), \(\delta=0.02\).
Hint 1
\((3+1.28)\times0.5/0.02\approx107\).
Hint 2
Square it.
Worked solution
\(N\approx107^2\approx1.1\times10^4\) shots. With the \(4096\) shots of earlier examples, the test would usually miss a \(2\%\) bias. Detection power must be planned: decide the smallest error that matters, then choose shots. When the property is deterministic, an exact Statevector check detects the bias without any shots.
A student argues: “If a test passes with a large tolerance, the circuit is correct; larger tolerances are just safer.” Using the S-gate bug, show a tolerance at which a statevector comparison on \(\ket+\) would pass, and explain why “safer” is the wrong word.
Hint 1
The S bug changes the \(\ket+\) output by a vector of length \(|1-i|/\sqrt2=1\).
Hint 2
Fidelity \(|\langle+|S|+\rangle|^2=\frac12\).
Worked solution
The outputs \(\ket+\) and \(S\ket+\) have fidelity \(\frac12\); a test that accepts any fidelity above \(0.4\), or any amplitude difference below \(1\), would pass the bug. A tolerance is not a safety margin: it defines which differences count as errors. It must be larger than numerical noise or approximation error, and much smaller than the smallest bug the test exists to catch. Too large is as wrong as too small.
A subroutine is supposed to act only on qubits 0 and 1 of a three-qubit register. Write a test that verifies its Operator equals \(I\otimes U_{\rm sub}\), so that qubit 2 is untouched. What kinds of bugs does this catch that tests of the two-qubit block alone would not?
Hint 1
Operator(QuantumCircuit(1)).tensor(Operator(sub)) places the identity on qubit 2.
Hint 2
In Qiskit, the left tensor factor acts on the higher qubit index.
Worked solution
Comparing Operator(big) with Operator(I).tensor(Operator(sub)) checks that the embedded circuit acts as the subroutine on qubits 0, 1 and as the identity on qubit 2. It catches wrong wire mappings, gates accidentally applied to spectator qubits, and entanglement with the spectator (for example a stray CNOT onto qubit 2), none of which a test of the subroutine in isolation would see. The tensor order follows Qiskit’s convention that the left factor acts on the higher index.
from qiskit import QuantumCircuit
from qiskit.quantum_info import Operator
sub = QuantumCircuit(2)
sub.h(0)
sub.cx(0, 1)
big = QuantumCircuit(3)
big.compose(sub, qubits=[0, 1], inplace=True)
assert Operator(big).equiv(Operator(QuantumCircuit(1)).tensor(Operator(sub)))
leaky = big.copy()
leaky.cx(1, 2)
assert not Operator(leaky).equiv(Operator(QuantumCircuit(1)).tensor(Operator(sub)))
Mutation testing measures a test suite by injecting small bugs. For the Bell circuit, consider four mutants: swap control and target; append Z on qubit 0; replace H by SX; drop the CNOT. Compare two tests: (a) “the counts contain only 00 and 11”; (b) statevector equivalence with \((\ket{00}+\ket{11})/\sqrt2\). How many mutants does each detect?
Hint 1
Appending Z gives \((\ket{00}-\ket{11})/\sqrt2\).
Hint 2
SX then CNOT gives \((\ket{00}-i\ket{11})/\sqrt2\).
Worked solution
Swapping control and target, or dropping the CNOT, produces outcomes 00 and 01, so test (a) catches both. Appending Z gives \((\ket{00}-\ket{11})/\sqrt2\), and SX gives \((\ket{00}-i\ket{11})/\sqrt2\): both still give only 00 and 11, so test (a) misses them. Test (b) catches all four. Mutation testing turns “my tests pass” into a measurable statement about which bugs they would catch, and shows that phase errors need phase-sensitive tests.
from qiskit import QuantumCircuit
from qiskit.quantum_info import Statevector
def bell(mutation=None):
circuit = QuantumCircuit(2)
circuit.sx(0) if mutation == "sx" else circuit.h(0)
if mutation == "swap":
circuit.cx(1, 0)
elif mutation != "drop":
circuit.cx(0, 1)
if mutation == "z":
circuit.z(0)
return circuit
target = Statevector([2 ** -0.5, 0, 0, 2 ** -0.5])
def support_test(c):
probabilities = Statevector.from_instruction(c).probabilities_dict()
return all(p < 1e-12 for k, p in probabilities.items() if k not in ("00", "11"))
def state_test(c):
return Statevector.from_instruction(c).equiv(target)
mutants = ["swap", "z", "sx", "drop"]
assert support_test(bell()) and state_test(bell())
assert sum(not support_test(bell(m)) for m in mutants) == 2
assert sum(not state_test(bell(m)) for m in mutants) == 4