Patch-PODiff-ViT | Onkar Jadhav

Abstract

Diffusion models enable probabilistic super-resolution and conditional generation, but pixel-space methods are computationally expensive and learned latent spaces often lack interpretable uncertainty quantification. We introduce Patch-PODiff-ViT, a structured latent diffusion framework in which the latent space is defined by patchwise Proper Orthogonal Decomposition (POD), a fixed linear orthonormal basis over local patches, rather than learned by a nonlinear autoencoder. This yields low-dimensional, variance-ordered tokens that preserve spatial structure and enable efficient diffusion in a structured low-dimensional latent space with a Vision Transformer. Because the decoder is fixed, linear, and orthonormal, latent coefficient uncertainty can be propagated directly to physical-space predictive variance, enabling analytic propagation of predictive variance through the linear decoder without Monte Carlo estimation in pixel space. Across sea surface temperature, medical imaging, and natural images, the method achieves strong reconstruction with fewer parameters and lower memory, while producing well-calibrated spatial uncertainty that closely matches empirical ensembles.

Introduction

A single global POD basis is efficient but can under-represent local structures when datasets contain heterogeneous spatial patterns. Scientific and medical images also require uncertainty at the pixel or physical-location level, where repeated Monte Carlo decoding can become expensive.

Patch-PODiff-ViT pipeline from low-resolution inputs through patchwise POD encoding, latent diffusion and decoding — Patch-PODiff-ViT conditional super-resolution pipeline. Source: Figure 1 of the open arXiv preprint.

Goal: Preserve local image structure while making uncertainty-aware super-resolution compact enough for scientific and medical applications.
Key idea: Represent local patches as variance-ordered orthonormal tokens and model their interactions with a conditional Vision Transformer.
Takeaway: The fixed linear decoder makes it possible to propagate predictive variance analytically from latent coefficients into physical space.

Methodology

Patch-PODiff-ViT divides each high-resolution field into local patches and constructs an orthonormal POD basis for the patch representation. The resulting low-dimensional tokens retain local spatial information. A conditional Vision Transformer models interactions between tokens and performs diffusion in this structured latent space.

Because reconstruction uses a fixed linear orthonormal decoder, coefficient-space predictive variance can be propagated analytically into pixel space. This provides spatial uncertainty maps without estimating every statistic through repeated nonlinear decoding.

Results and significance

Patchwise, variance-ordered latent tokens that preserve local structure.
A Vision Transformer denoiser for interactions across spatial patches.
Analytical propagation of predictive variance through the fixed decoder.
A unified evaluation across sea-surface temperature, medical imaging and natural images.

The method achieved strong reconstruction with fewer parameters and lower memory use while producing spatial uncertainty that closely followed empirical ensembles. It extends structured scientific diffusion beyond a single domain and supports applications where calibrated uncertainty is essential.

Reconstruction across domains

Qualitative super-resolution comparison across sea-surface temperature, chest X-ray and face-image datasets — **Figure 2.** Qualitative comparison across three domains. Columns show the low-resolution input, U-Net, PixelDiff, VAE-LDM, Patch-PODiff-ViT and ground truth. Rows compare full reconstructions and absolute errors for SST, Chest X-ray and FFHQ. Source.

**Table 1.** Reconstruction quality across three datasets. Lower is better for RMSE, LPIPS and FID; higher is better for PSNR and SSIM.
Metric	Patch-PODiff-ViT	VAE-LDM	DiT	PixelDiff	U-Net	Fullfield-PODiff
SST (Sea Surface Temperature)
RMSE ↓	0.0030	0.0041	0.0040	0.0049	0.0093	0.0049
PSNR ↑	50.43	47.73	47.95	46.23	40.63	46.22
SSIM ↑	0.9888	0.9799	0.9707	0.9713	0.9589	0.9690
LPIPS ↓	0.0131	0.0201	0.0169	0.0234	0.0295	0.0267
FID ↓	3.986	5.90	5.01	8.46	11.96	8.49
Chest X-ray
RMSE ↓	0.0065	0.0087	0.0081	0.0098	0.0133	0.0092
PSNR ↑	42.98	39.11	41.21	37.64	34.52	35.75
SSIM ↑	0.9885	0.9790	0.9632	0.9743	0.9423	0.9699
LPIPS ↓	0.0201	0.0298	0.0281	0.0310	0.0422	0.0307
FID ↓	6.0152	8.17	7.01	11.44	13.65	9.26
FFHQ (Face Images)
RMSE ↓	0.0109	0.0178	0.0121	0.0147	0.0361	0.0154
PSNR ↑	39.15	36.44	38.04	37.87	32.54	37.15
SSIM ↑	0.9522	0.9321	0.9432	0.9411	0.9255	0.9358
LPIPS ↓	0.0300	0.0561	0.0377	0.0427	0.0584	0.0471
FID ↓	9.168	11.98	10.04	10.11	16.18	12.08

Results are averages over three seed runs; the paper reports standard deviations separately in Appendix Table 5. Bold values reproduce the best entries in the original table. Source: paper Table 1.

Uncertainty quantification

Analytic and empirical predictive standard deviation maps for Patch-PODiff-ViT on sea-surface temperature with their difference — **Figure 3.** Analytic versus empirical uncertainty on SST. The analytic standard deviation from the linear propagation formula closely matches the empirical standard deviation from 100 samples (Pearson correlation r = 0.983); the right panel shows their difference. Source.

Reliability diagrams for Patch-PODiff-ViT and comparison models on SST, FFHQ and Chest X-ray — **Figure 4.** Reliability diagrams comparing empirical and nominal coverage for SST, FFHQ and Chest X-ray. The dashed line denotes perfect calibration. Source.

Ground-truth SST field, Patch-PODiff-ViT ensemble mean and predictive standard deviation — **Figure 5.** SST ensemble application: ground truth, Patch-PODiff-ViT ensemble mean and predictive standard deviation in °C. Higher uncertainty is concentrated along coastal areas and strong thermal gradients. Source.

Computational efficiency

**Table 2.** Computational cost comparison across diffusion models.
Model	Parameters (M)	Training (h)	GPU memory (GB)	Inference (s)	M=100 ensemble (s)
Patch-PODiff-ViT	70	8.7	8.6	0.109	11.036
VAE-LDM	220	30.6	16.1	0.255	26.038
DiT	75	10.2	9.2	0.112	11.855
PixelDiff	160	26.8	12.3	1.800	189.530

The final column reports generation of a 100-member ensemble. Source: paper Table 2.

Technology and data

Python, PyTorch, Vision Transformers, patchwise POD, diffusion models, sea-surface-temperature datasets, MRI/X-ray imagery and natural-image benchmarks.

Research outputs

Preprint GitHub Live demo

BibTeX

Cite the arXiv preprint:

@misc{jadhav2026patchpodiffvit,
  title         = {{Patch-PODiff-ViT}: Structured Latent Diffusion
                   with Patchwise POD for Super-Resolution and
                   Uncertainty Quantification},
  author        = {Jadhav, Onkar and French, Tim and Rayson, Matthew
                   and Jones, Nicole L.},
  year          = {2026},
  eprint        = {2606.31290},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  doi           = {10.48550/arXiv.2606.31290},
  url           = {https://arxiv.org/abs/2606.31290}
}