Data science & ML
Zero-Shot Super-Resolution
Learning to upscale from the image itself
A super-resolution network trained at test time on nothing but the low-resolution image itself, measured honestly against bicubic upscaling.
Key result Trained only on the low-resolution input, the network beats bicubic upscaling by 1.16 dB on Coins and 0.48 dB on Cameraman: one seed, two images, 2x, under a corrected zero-shot protocol.
Research study Status: Completed Academic · team of 2
Workflow diagram
Research question
Most super-resolution networks learn from large external datasets. ZSSR asks whether a network can learn to upscale one image using nothing but that image, because small patches recur across scales within a natural image.
My Role & Leadership
Project Owner · Technical Lead · Developer. Owned and implemented ZSSR in a course assignment: the residual network, the single-image training set with optional augmentation, the training and evaluation loops, an eight-view self-ensemble with correct inverse transforms, and iterative back-projection. The assignment was done in a course team of two; this implementation is my hands-on part.
- Led the implementation choices: a network that learns only the residual over bicubic interpolation, and a self-ensemble that inverts every rotation and flip before averaging.
Key findings
- Coins
- +1.16 dB (28.83 against 27.67)
- Cameraman
- +0.48 dB (30.55 against 30.06)
Source: Independent rerun of the owner's unchanged code, zero-shot protocol, seed 0, 2x.
Coins, with its strongly repeating patterns, gains more than Cameraman, as the method predicts: it relies on small patches that recur across scales within one image.
Evaluation
- PSNR at 2x, zero-shot (Coins)
- 28.83 dB
- PSNR at 2x, zero-shot (Cameraman)
- 30.55 dB
Method
Train on the image's own downscaled copies, then apply the trained network to the image itself; the network only learns the residual over bicubic interpolation.
Downscale the low-resolution input once more, so the input and its own downscale form training pairs.
Train a small residual network to undo that downscale, on random crops of the input only.
Apply the trained network to the input itself to produce the 2x result.
Score against the high-resolution original, which the network never saw.
What the team built
- A residual CNN (8 layers, 64 channels, 222,785 parameters) that refines a bicubic upsampling instead of predicting the full image.
- A single-image training set: random crops of the input paired with their own 2x downscale, with optional rotation and flip augmentation.
- An eight-view geometric self-ensemble that inverts each rotation and flip before averaging.
- Iterative back-projection that enforces consistency with the low-resolution input.
Methodological choices
- Learn the residual over bicubic, not the whole image, so a small network trained on one image has less to learn.
- Invert every rotation and flip before averaging the self-ensemble, so the eight views agree pixel for pixel.
- Keep the image being scored out of training. The original notebook trained on it, which inflated its gains about fivefold; only the corrected protocol is reported here.
Built on
- The method is Shocher et al.'s (CVPR 2018); this is an implementation of it.
- The self-check and the experiment template came with the assignment.
- The test images are scikit-image sample images.
Limitations
One small evaluation: two test images, one seed, 2x scale, PSNR only. The original notebook trained on the ground-truth image it was scored against; its numbers are not used anywhere on this page.
Key technologies
Evidence
- The assignment notebook.
- An independent rerun of the unchanged code under the zero-shot protocol (one seed, CPU), which produced every number and the figure on this page.