--- category: literaturenote citekey: zhangunsuperviseddeformableimageregistration2025 title: Unsupervised Deformable Image Registration with Structural Nonparametric Smoothing authors: "Zhang, Hang; Chen, Xiang; Hu, Renjiu; Wang, Rongguang; Zhang, Jinwei; Liu, Min; Wang, Yaonan; Li, Gaolei; Cheng, Xinxing; Duan, Jinming" year: 2025 date: 2025-00-00 2025 url: "https://arxiv.org/abs/2506.10813v1" zotero_key: 5EHJ5PJV zotero_storage: CGN695EM collections: transpordi_planeerimine folder: 001_artiklid firstAuthor: "Zhang, Hang" status: converted --- # Unsupervised Deformable Image Registration with Structural Nonparametric Smoothing Hang Zhang1 , Renjiu Hu1 , Xiang Chen2 , Min Liu2 , Yaonan Wang2 , Rongguang Wang3 , Jinwei Zhang4 , Gaolei Li5 , Xinxing Cheng6 , and Jinming Duan6,7 () - 1 Cornell University, Ithaca, NY, USA - 2 Hunan University, Changsha, China - 3 University of Pennsylvania, Philadelphia, PA, USA - 4 Johns Hopkins University, Baltimore, MD, USA - 5 Shanghai Jiao Tong University, Shanghai, China 6 University of Birmingham, Birmingham, UK - 7 University of Manchester, Manchester, UK Jinming.Duan@manchester.ac.uk Abstract. Learning-based deformable image registration (DIR) accelerates alignment by amortizing traditional optimization via neural networks. Label supervision further enhances accuracy, enabling efficient and precise nonlinear alignment of unseen scans. However, images with sparse features amid large smooth regions, such as retinal vessels, introduce aperture and large-displacement challenges that unsupervised DIR methods struggle to address. This limitation occurs because neural networks predict deformation fields in a single forward pass, leaving fields unconstrained post-training and shifting the regularization burden entirely to network weights. To address these issues, we introduce SmoothProper, a plug-and-play neural module enforcing smoothness and promoting message passing within the network's forward pass. By integrating a duality-based optimization layer with tailored interaction terms, SmoothProper efficiently propagates flow signals across spatial locations, enforces smoothness, and preserves structural consistency. It is model-agnostic, seamlessly integrates into existing registration frameworks with minimal parameter overhead, and eliminates regularizer hyperparameter tuning. Preliminary results on a retinal vessel dataset exhibiting aperture and large-displacement challenges demonstrate our method reduces registration error to 1.88 pixels on 2912×2912 images, marking the first unsupervised DIR approach to effectively address both challenges. The source code will be available at [https:](https://github.com/tinymilky/SmoothProper) [//github.com/tinymilky/SmoothProper](https://github.com/tinymilky/SmoothProper). Keywords: Nonparametric smoothing · Deformable image registration · Neural networks · Plug-and-play · Aperture problem · Large displacement # 1 Introduction In recent years, learning-based deformable image registration (DIR) methods, fueled by progress in neural networks [\[16,](#page-12-0) [27,](#page-13-0) [67\]](#page-16-0), have become mainstream for ![](_page_1_Picture_2.jpeg) Fig. 1: Visual illustration of aperture and large displacement challenges. (a) shows a retinal vessel patch from the FIRE dataset, with blue, green, and yellow dashed boxes highlighting local vessel structures at different positions. The green and yellow boxes are horizontal and vertical translations of the blue box, respectively. (b), (c), and (d) are zoomed-in views of the blue, green, and yellow boxes, demonstrating that local information alone is insufficient to reconstruct their displacements. Furthermore, the displacements in (b)-(c) and (b)-(d) exceed the vessel diameter. various registration tasks. Compared to classical iterative optimization methods [\[3,](#page-12-1) [4,](#page-12-2) [29,](#page-13-1) [47,](#page-14-0) [68\]](#page-16-1), learning-based approaches [\[5,](#page-12-3) [10,](#page-12-4) [11,](#page-12-5) [48,](#page-14-1) [50\]](#page-15-0) achieve faster performance through amortized optimization and offer potentially higher accuracy with the integration of label supervision (e.g., segmentation, keypoints). Despite promising performance, unsupervised DIR methods struggle with tasks involving both the aperture problem and the large displacement problem. The aperture problem arises in homogeneous or textureless regions, where limited local evidence within the network's effective receptive field (ERF) [\[45\]](#page-14-2) hinders accurate displacement estimation. The large displacement problem occurs when structural displacement between image pairs exceeds the structure's size. While many registration tasks involve either issue separately, retinal vessel datasets often present both simultaneously (see Fig. [1\)](#page-1-0), significantly complicating unsupervised DIR. Current methods primarily rely on descriptor matching approaches [\[42,](#page-14-3)[43\]](#page-14-4), and to our knowledge, no existing unsupervised DIR methods have successfully handle both challenges in retinal vessel registration. We hypothesize that the impact of the pairwise regularization term in the loss function has been underestimated. As demonstrated by the Horn-Schunck optical flow equation [\[34\]](#page-14-5), this regularizer not only smooths the flow field but also facilitates message passing [\[25,](#page-13-2)[26,](#page-13-3)[28,](#page-13-4)[75\]](#page-16-2), propagating flow information from regions with strong signals (e.g., boundaries, texture-rich areas) to those with ambiguous or missing signals (e.g., large homogeneous areas). This mechanism is essential for addressing the aperture and large displacement problems. However, most unsupervised DIR methods incorporate this regularizer only as a loss term, shifting the burden for smoothness and signal propagation entirely to the network weights. See Fig. [2](#page-2-0) for a visual illustration. To address these challenges, we introduce SmoothProper, a plug-and-play unrolled neural network layer seamlessly integrating into existing frameworks with minimal parameter overhead (see Fig. [2,](#page-2-0) removing (b) or (c) for illustration). Our approach extends the Horn-Schunck variational formulation [\[34\]](#page-14-5), employing a duality-based optimization framework [\[8,](#page-12-6) [57,](#page-15-1) [73\]](#page-16-3) with quadratic re- ![](_page_2_Figure_2.jpeg) Fig. 2: Illustration of the registration framework and integration of Smooth-Proper into learning-based frameworks. Black arrows represent the forward pass, while yellow arrows indicate the backward pass. Dashed components are optional: removing (a), (b), and (c) reverts to a classic iterative approach where loss gradients directly update the flow field u, similar to ConvexAdam [\[59\]](#page-15-2), which follows a modern learning framework but aligns with this strategy. Removing (c) leads to a conventional learning-based approach, where the r(u) affects the flow field only through network weights via amortized optimization. Most learningbased frameworks, including pyramid or multi-scale models (except unrolling methods [\[28,](#page-13-4) [38,](#page-14-6) [54\]](#page-15-3)), use a backbone network followed by convolution (b) to generate the flow field. Removing (b) allows SmoothProper with a basis vector generator to replace this convolution layer, adding minimal parameters. laxation [\[60\]](#page-15-4) to decouple data fidelity from smoothness regularization. Unlike prior unrolling methods [\[7,](#page-12-7)[35,](#page-14-7)[38,](#page-14-6)[54\]](#page-15-3) emphasizing raw data consistency or neural network-based denoising, SmoothProper introduces adaptive, conditional basis vectors and tailored interaction terms in each unrolled iteration. This structure enables message passing, preserves local structural consistency, and eliminates the need for tuning regularization hyperparameters. SmoothProper resembles MoDL's unrolled architecture [\[1\]](#page-12-8) and incorporates mechanisms analogous to loopy belief propagation [\[21\]](#page-13-5) and mean-field message passing [\[40\]](#page-14-8). The main contributions of this work are as follows: - We introduce SmoothProper, an unrolled neural network layer that enforces smoothness and message passing with local structural consistency during the network forward pass, addressing the aperture and large-displacement challenges in unsupervised DIR. - SmoothProper is lightweight, model-agnostic, integrates seamlessly into unsupervised DIR frameworks with minimal parameter overhead, and removes the need for regularizer hyperparameter tuning. - We have achieved the first unsupervised learning-based retinal vessel registration, reducing the average error to 1.88 pixels on 2912 × 2912 images. ## 2 Related Work Preliminaries on Learning-based DIR Methods: Classical iterative methods calculate a flow field by solving a pairwise variational optimization problem [\[34\]](#page-14-5). This allows direct updates to the flow field u using loss gradients (see Fig. [2](#page-2-0) for a visual example). In contrast, learning-based methods employ amortized optimization over network parameters θ across a cohort of image pairs D: $$\min_{\theta} \mathbb{E}_{(\mathbf{I}_f, \mathbf{I}_m) \sim D}[s(\mathbf{I}_f, \mathbf{I}_m, \mathbf{u}) + \lambda r(\mathbf{u})], \text{ s.t. } \mathbf{u} = g_{\theta}(\mathbf{I}_f, \mathbf{I}_m).$$ (1) Here, If and Im denote the fixed and moving images, respectively; u is the displacement field output by a neural network g parameterized by θ; λ is a hyperparameter; s is the dissimilarity function; and r is typically a pairwise diffusive regularizer. Training can be accomplished by accumulating gradient backflows with respect to θ: $$\frac{\partial \mathcal{L}}{\partial \theta} = \sum_{f,m} \frac{\partial s}{\partial g_{\theta}} \frac{\partial g_{\theta}}{\partial \theta} + \lambda \sum_{f,m} \frac{\partial r}{\partial g_{\theta}} \frac{\partial g_{\theta}}{\partial \theta}$$ (2) Here, L denotes the total loss, with the Jacobians mapping flow gradients to network weights. As noted by Hansen and Heinrich [\[26\]](#page-13-3), and shown in recent work [\[36\]](#page-14-9), unsupervised DIR methods relying on amortized optimization yield limited gains in label matching (e.g., Dice score), since the first Jacobian in Eq. [\(2\)](#page-3-0) contains no more information than raw intensity-label mutual information. We hypothesize that while the first Jacobian fails to improve label matching, the second Jacobian, beyond enforcing smoothness, can function as an effective message passing mechanism [\[21,](#page-13-5) [26\]](#page-13-3) if properly utilized. This potential has been largely overlooked, and our SmoothProper addresses it via a cost-efficient, easily integrated architecture grounded in modern neural network design. Learning-based and Deep Unrolling DIR Methods: To address aperture and large-displacement challenges, increasing the network ERF is common, involving methods like transformers [\[10,](#page-12-4)[58,](#page-15-5)[67\]](#page-16-0), large-kernel convolutions [\[15,](#page-12-9)[37\]](#page-14-10), Laplacian pyramids [\[52,](#page-15-6) [70,](#page-16-4) [74\]](#page-16-5), and coarse-to-fine designs [\[46,](#page-14-11) [48\]](#page-14-1). However, relying solely on the loss term and network weights for message passing may risk the network acting as a low-pass filter [\[55,](#page-15-7) [72\]](#page-16-6), leading to oversmoothing that obscures structural details. To overcome these limitations, deep unrolling [\[32\]](#page-13-6) integrates model-based methods [\[1,](#page-12-8) [76–](#page-16-7)[78\]](#page-16-8) and iterative algorithms [\[12,](#page-12-10) [23,](#page-13-7) [63\]](#page-15-8) directly into neural network architectures. Recent studies [\[7,](#page-12-7) [28,](#page-13-4) [35,](#page-14-7) [38,](#page-14-6) [54\]](#page-15-3) have applied deep unrolling to image registration, enforcing both dissimilarity and regularity functions during the network forward pass. Among these works, most focus on either raw data consistency or neural network denoisers as regularizers, while PDD-Net [\[28\]](#page-13-4) remains the only one that explicitly enforces smoothness, though it relies on a costly 6D dissimilarity map and underutilizes neural network capacity for message passing. GraDIRN [\[54\]](#page-15-3) uses unrolled gradient descent for data consistency, PRIVATE [\[35\]](#page-14-7) employs a plug-and-play denoiser with instance-wise optimization, and SUITS [\[7\]](#page-12-7) and VR-Net [\[38\]](#page-14-6) rely on separate feature extraction or alternating data consistency and denoising steps. In contrast, SmoothProper directly integrates smoothness regularization into the network forward pass, enabling efficient message passing without label supervision or heavy computational overhead. ## 3 Method This section introduces the bi-level optimization framework, defines Smooth-Proper and its optimization, presents a conditional automatic adjustment for λ, and outlines the overall framework integrating SmoothProper. #### 3.1 The SmoothProper Bi-level Optimization framework: The proposed SmoothProper adds a suboptimization problem as a constraint to Eq. [\(1\)](#page-2-1): $$\min_{\theta} \mathbb{E}_{(\mathbf{I}_f, \mathbf{I}_m) \sim D}[\mathcal{L}(\mathbf{I}_f, \mathbf{I}_m, \mathbf{u}^*(\theta))], \tag{3a}$$ s.t. $$\mathbf{u}^*(\theta) = \underset{\mathbf{u}}{\operatorname{arg\,min}} \mathcal{S}(\mathbf{I}_f, \mathbf{I}_m, \mathbf{u}, \theta).$$ (3b) Eq. [\(3b\)](#page-4-0) constrains the expected loss in Eq. [\(3a\)](#page-4-1), optimizing the displacement field by minimizing the SmoothProper function S based on input images and network parameters. The optimization S is embedded in the registration network by unrolling its iterative steps into the forward pass, with trainable parameters θ. Given the solution u ∗ for each image pair in the training set D, Eq. [\(3a\)](#page-4-1) minimizes the expected loss L(If , Im, u ∗ (θ)) to optimize θ. Unlike GraDIRN [\[54\]](#page-15-3), which unrolls its energy function to enforce data consistency, our approach unrolls S to shift the regularization burden away from network weights. Quadratic Relaxation: Following prior works [\[8,](#page-12-6) [60,](#page-15-4) [68,](#page-16-1) [73\]](#page-16-3), we adopt a quadratic relaxation of Eq. [\(1\)](#page-2-1) to decouple the data consistency term and the regularizer: $$\min_{\mathbf{u}} \int_{\Omega} \left( \rho(\mathbf{v}) + \frac{1}{2\alpha} ||\mathbf{v} - \mathbf{u}||^2 + \beta r(\mathbf{u}) \right) d\Omega. \tag{4}$$ Here, ρ(v) measures the cost of applying the flow field v to the input image pair, while βr(u) represents the smoothness regularizer with strength β. It has been shown [\[8\]](#page-12-6) that as α → 0, minimizing Eq. [\(4\)](#page-4-2) with the same data and regularization terms is equivalent to minimizing it without the interaction term. Although non-quadratic relaxation terms exist [\[6\]](#page-12-11), we empirically found the suboptimization problem easier to solve, whereas using an L1 term requires proximal gradients, adding complexity. Many gradient-descent methods rely on first-order Taylor expansions to linearize ρ(v), limiting them to small steps and requiring numerous iterations, which hinders handling large deformations. Deep unrolled DIR methods [\[7,](#page-12-7) [38,](#page-14-6) [54\]](#page-15-3), built on gradient-based optimization, inherit these limitations. In contrast, discrete optimization approaches [\[29,](#page-13-1)[30,](#page-13-8)[59\]](#page-15-2) rely on predefined cost volumes, allowing each subproblem for v and u to be convex and optimally solvable, thus better accommodating large deformations. However, their computational cost rises sharply when expanding the discrete space to cover larger displacements. Objective Function: SmoothProper leverages neural networks to adaptively linearize local intensities using a basis-constrained formulation. We discretize Eq. [\(4\)](#page-4-2) as a sum of voxel- and pair-wise energies, with the basis matrix b ∈ R m×d , where b = [b1, b2, . . . , bm] ⊤ and each bi ∈ R d is a basis vector (d = 2 for 2D images). Given the neural network output p = gθ(If , Im), where p(x) ∈ R m is a non-negative coefficient vector (from ReLU), representing the flow signal strength at x ∈ Ω, the basis vectors are linearly combined to produce the displacement fields. This leads to the following energy optimization: $$\min_{\mathbf{q}, \mathbf{v}} \sum_{x \in \Omega} \|\mathbf{p}(x) - \mathbf{q}(x)\|^2 + \sum_{x \in \Omega} \sum_{i=1}^{m} \frac{1}{2\alpha} \mathbf{q}_i(x) \|\mathbf{v}(x) - \mathbf{b}_i\|^2 + \sum_{x \in \Omega} \frac{1}{2\alpha} \|\mathbf{q}(x)\mathbf{b} - \mathbf{v}(x)\|^2 + \beta \mathbf{r}(\mathbf{v}).$$ (5) Here, q(x)b computes the displacement at location x, while v(x) represents the auxiliary displacement vector at x. The first term aligns q(x) with the network's initial prediction p(x), while the fourth term r(v) = ∥∇v∥ is a diffusive regularizer. The second term, unique to this work, introduces a directional bias between q(x) and bi , ensuring the displacement can be closely represented by a linear combination of the basis vectors bi , thereby guiding the reconstruction of displacement field from q. The third coupling term pulls v(x) toward the linear combination q(x)b. This creates a feedback loop that enables two key properties: - Message Passing: The term ∥∇v∥ 2 encourages gradual changes in v, smoothing q as qb aligns with v. This results in q serving as a smooth approximation of p, effectively passing flow signals across regions. - Structural Consistency: As q(x) and v(x) iteratively align with p(x) and the basis bi , the equation ensures that regions with strong flow signals (large q(x)) remain anchored to their representative basis patterns bi . This basis-constrained formulation employs a "smooth-reinforce" interplay: The smoothing step diffuses signals, slightly diminishing strong ones and enhancing weaker ones. The reinforcement step then "locks in" strong signals: when p(x) is large, the second term keeps v(x) close to the corresponding basis pattern, preventing dilution. Meanwhile, weak signals, associated with smaller q(x),can be influenced by stronger signals spreading through message passing of p(x). This iterative smooth-reinforce cycle promotes signal propagation across regions, preserves strong signals, and effectively extends them into weaker areas, addressing the aperture problem and mitigating oversmoothing. Unlike previous methods that rely on manual linearization [\[7,](#page-12-7)[38\]](#page-14-6) or omit it entirely [\[35,](#page-14-7)[54\]](#page-15-3), our proposed Eq. [\(5\)](#page-5-0) adaptively linearizes image intensities within the network's effective receptive field (ERF) into learned coefficients. In contrast to PDD-Net [\[28\]](#page-13-4), which uses fixed displacement vectors and pre-computed cost volumes, SmoothProper jointly learns both basis vectors and coefficients during training. These learned basis vectors serve as "dictionary atoms" [\[61,](#page-15-9) [74\]](#page-16-5) or clustering centers [\[19,](#page-13-9)[66\]](#page-16-9), providing an adaptive representation that directly influences field smoothness. This approach achieves higher accuracy without computing a full cost volume, instead encoding disparity context directly within the ERF. Figure [3b](#page-6-0) illustrates the recursive SmoothProper with its feedback loop. Unrolling the Recursive Networks. To solve Eq. [\(5\)](#page-5-0), we employ coordinate descent, alternating between q (k) and v (k) while progressively reducing α to achieve q (K)b ≈ v (K) , thus approximating the solution of Eq. [\(5\)](#page-5-0). Here, K is ![](_page_6_Figure_2.jpeg) Fig. 3: (a) is the schematic of the SmoothProper (SP) framework. The yellow box represents SP optimization, where p = gθ(If , Im) and b = Φl(Φ0(λ)), with λ randomly sampled from (0, λmax). SP iteratively updates q k to q k+1 until the maximum iteration count K is reached. (b) and (c) are visualizations of the SmoothProper module architecture in recursive and unrolled forms. Basis vectors b and α are omitted for brevity (details in §[3.1\)](#page-5-1). the total number of iterations, and k denotes the kth iteration. Following prior work [\[30\]](#page-13-8), we set α = {150, 50, 15, 5, 1.5, 0.5} for K = 6 alternating iterations. Let the directional bias and coupling terms be represented by C(q, v, b), we decompose the optimization of Eq. [\(5\)](#page-5-0) into two subproblems as follows: $$\mathbf{q}^{(k)} = \underset{\mathbf{q}}{\arg\min} \ \frac{1}{2\alpha} \mathcal{C}(\mathbf{q}, \mathbf{v}^{(k-1)}, \mathbf{b}) + \sum_{x \in \Omega} \|\mathbf{p}(x) - \mathbf{q}(x)\|^2;$$ $$\mathbf{v}^{(k)} = \underset{\mathbf{v}}{\arg\min} \ \frac{1}{2\alpha} \mathcal{C}(\mathbf{q}^{(k)}, \mathbf{v}, \mathbf{b}) + \beta \|\nabla \mathbf{v}\|^2.$$ (6) The initial q (0) is set to p = gθ(If , Im), the output from the backbone network gθ. The q subproblem can be solved optimally and pointwise across x ∈ Ω in closed form by taking the derivative of C(q, v (k−1) , b) with respect to q(x) and setting it to zero: $$\frac{\partial}{\partial \mathbf{q}(x)} \mathcal{C}(\mathbf{q}, \mathbf{v}^{(k-1)}, \mathbf{b}) = \alpha (\mathbf{p}(x) - \mathbf{q}(x)). \tag{7}$$ The v subproblem is solved through fixed-point iteration [\[8\]](#page-12-6), involving neighboring values of v(x) and q(x)b, yielding v as a smoothed version of qb. This can be approximated by applying Gaussian blurring to qb as v (k) = K ∗ q (k)b across spatial locations, with the blurring strength determined by β. A similar approach for optimizing v is discussed in Diffeomorphic Demons [\[68\]](#page-16-1). The unrolled SmoothProper architecture is illustrated in Fig. [3c.](#page-6-0) #### 3.2 Conditional Learning of Basis Vectors SmoothProper serves as the constraint in Eq. [\(3\)](#page-4-3), providing a smoothed version of p = gθ(If , Im) with q. The learnable basis vectors b link the smoothness of v to q, adjusting the weight of q based on its proximity to v. This enables adaptive regularization strength in the outer loop of Eq. [\(3a\)](#page-4-1) through b. We use two MLPs, Φ0 and Φl : c ← Φ0(λ) generates a latent code c, and b ← Φl(c) produces conditional basis vectors. Using two MLPs instead of one provides model-agnostic flexibility, allowing c to be shared across all pyramid or scale levels if the model is multi-scale-based, while each level has its own Φl During training, we randomly sample λ ∈ (0, λmax) at each forward pass, enabling b and the from-matching strength λ to adapt via Φl(Φ0(λ)). This eliminates the need for manual λ-tuning and allows smoothness strength adjustments after amortized optimization. Prior works (e.g., HyperMorph [\[33\]](#page-14-12) and cLapIRN [\[51\]](#page-15-10)) also use hypernetworks [\[24\]](#page-13-10), but they rely on inefficient, nonscalable designs: HyperMorph conditions the entire network, and cLapIRN requires extensive architectural changes. In contrast, our method is lightweight and efficient, requiring only a single convolution layer replacement with a Smooth-Proper layer at the backbone's end, leaving the backbone unchanged (Fig. [2\)](#page-2-0). #### 3.3 Overall Framework Fig. [3a](#page-6-0) shows the SmoothProper framework, which consists of three components: a backbone network gθ for initial coefficient estimation, MLPs Φ0 and Φl for latent code and conditional basis vector generation, and unrolled SmoothProper layer. The SmoothProper module processes the network-predicted coefficients p and the MLP-predicted basis vectors b through K iterations to produce q (K) , a smooth and structurally consistent version of p. The flow field u is obtained by linearly combining q (K) with b and is integrated using scaling and squaring [\[2\]](#page-12-12) to generate a diffeomorphic deformation field ϕ. As in prior work [\[13\]](#page-12-13), the number of integration steps is fixed to 7. The deformation field ϕ is used for the dissimilarity term, while u is used for the regularization term in the loss function. In summary, SmoothProper not only smooths the displacement field u through the diffusive regularizer but also enables message passing, where flow signals (represented by coefficient vectors p) propagate with local structural consistency preserved via the feedback loop formed by the interaction term C(q, v, b). ## 4 Experiments & Results #### 4.1 Datasets & Evaluation Metrics We evaluated our method on the public Fundus Image Registration (FIRE) dataset [\[31\]](#page-13-11), which contains 134 image pairs from 39 subjects. Captured at 2912 × 2912 resolution with a 45◦ field of view using a Nidek AFC-210 fundus camera, the images were collected at Papageorgiou Hospital, Aristotle University of Thessaloniki. Each pair includes 10 annotated landmarks. To enhance Category Methods TRE ↓ AUC ↑ Group-wise AUC@25 ↑ mAUC@15 mAUC@25 mAUC@50 mAUC@A mAUC@P mAUC@S Detector-based Methods XFeat [CVPR'24] [\[53\]](#page-15-11) 10.858 0.560 0.637 0.794 0.853 0.102 0.915 R2D2 [NeurIPS'19] [\[56\]](#page-15-12) 7.926 0.553 0.701 0.850 0.813 0.333 0.899 LightGlue [ICCV'23] [\[41\]](#page-14-13) 7.802 0.575 0.710 0.855 0.853 0.338 0.904 SuperPoint [CVPR'18] [\[14\]](#page-12-14) 6.641 0.612 0.757 0.879 0.813 0.453 0.928 Glampoints [ICCV'19] [\[64\]](#page-15-13) 6.608 0.595 0.757 0.879 0.733 0.560 0.880 SuperRetina [ECCV'22] [\[43\]](#page-14-4) 6.382 0.622 0.767 0.884 0.813 0.516 0.909 RetinaIPA [MICCAI'24] [\[71\]](#page-16-10) 5.750 0.657 0.774 0.885 0.799 0.599 0.886 Detector-free Methods LoFTR [CVPR'21] [\[62\]](#page-15-14) 7.638 0.526 0.716 0.858 0.693 0.564 0.811 DKM [CVPR'23] [\[17\]](#page-13-12) 6.493 0.610 0.760 0.880 0.800 0.529 0.891 Aspanformer [ECCV'22] [\[9\]](#page-12-15) 6.415 0.602 0.761 0.881 0.800 0.556 0.877 RoMa [CVPR'24] [\[18\]](#page-13-13) 6.388 0.605 0.763 0.881 0.800 0.565 0.875 GeoFormer [ICCV'23] [\[42\]](#page-14-3) 6.201 0.625 0.770 0.887 0.813 0.587 0.925 Learning-based Methods w/o Deep Unrolling KeyMorph [MIDL'22] [\[20\]](#page-13-14) 5.947 0.640 0.784 0.892 0.813 0.551 0.917 C2FViT [CVPR'22] [\[49\]](#page-14-14) 5.842 0.642 0.786 0.893 0.827 0.547 0.920 GAMorph [BIBM'24] [\[44\]](#page-14-15) 3.081 0.825 0.895 0.945 0.906 0.800 0.949 Learning-based Methods w/ Deep Unrolling GraDIRN [MICCAI'22] [\[54\]](#page-15-3) 6.344 0.657 0.774 0.885 0.799 0.599 0.886 PDD-Net [MICCAI'19] [\[28\]](#page-13-4) 5.765 0.688 0.792 0.893 0.819 0.598 0.915 VR-Net [TMI'21] [\[38\]](#page-14-6) 4.974 0.705 0.823 0.911 0.827 0.660 0.931 Table 1: Quantitative comparison on the FIRE dataset. Best results are in bold. ↓ indicates lower is better, and ↑ indicates higher is better. The initial alignment method for learning-based approaches is highlighted in gray. details, we extract the green channel (richest in structural information due to red-free photography), apply Contrast Limited Adaptive Histogram Equalization (CLAHE) for local contrast enhancement, and use gamma correction (γ = 1.2) for better visibility. The dataset is divided into three categories: S (71 pairs, > 75% overlap, no anatomical changes), P (49 pairs, < 75% overlap, no anatomical changes), and A (14 pairs, > 75% overlap, anatomical changes). It was stratified by category and split into train, val, and test sets in a 7:1:2 ratio. SmoothProper (Ours) 1.879 0.920 0.951 0.974 0.937 0.920 0.974 We evaluated model performance using Target Registration Error (TRE) and Area Under the Curve (AUC) for TRE thresholds. TRE measures the L2 distance between corresponding points in the fixed and warped images, with lower values indicating better alignment. AUC (AUC@15, AUC@25, AUC@50) represents the percentage of TRE values below specific thresholds, providing a normalized measure of registration success. We also report multiply-add operations (G) and parameter size (MB) to assess model complexity. #### 4.2 Baseline Methods & Implementation Details For retinal vessel datasets, most studies focus on descriptor matching methods using homography transformations. We benchmarked both detector-based methods, including SuperPoint [\[14\]](#page-12-14), Glampoints [\[64\]](#page-15-13), R2D2 [\[56\]](#page-15-12), SuperRetina [\[43\]](#page-14-4), LightGlue [\[41\]](#page-14-13) with DISK [\[65\]](#page-15-15), SiLK [\[22\]](#page-13-15), XFeat [\[53\]](#page-15-11), and RetinaIPA [\[71\]](#page-16-10), and detector-free methods, such as LoFTR [\[62\]](#page-15-14), DKM [\[17\]](#page-13-12), AspanFormer [\[9\]](#page-12-15), RoMa [\[18\]](#page-13-13), and GeoFormer [\[42\]](#page-14-3). We also evaluated learning-based methods, including KeyMorph [\[69\]](#page-16-11) and C2FViT [\[49\]](#page-14-14), which claim to address large misalignments and arbitrary rotations ([−180◦ , 180◦ ]). Additionally, we included deep unrolling methods, such as VR-Net [\[38\]](#page-14-6), GraDIRN [\[54\]](#page-15-3), and PDD-Net [\[28\]](#page-13-4), as well as GAMorph [\[44\]](#page-14-15), the only known method co-training descriptor matching with deformable registration, which shows promising results. All experiments, including SmoothProper and baseline models, were conducted on a machine with an NVIDIA A100 GPU (80GB), a 16-core CPU, and ![](_page_9_Figure_2.jpeg) Fig. 4: Qualitative comparison. Fixed images are shown with the 'inferno' colormap, and moving images are overlaid with the 'viridis' colormap for clarity. Highlighted shadow regions show challenging cases with local anatomical changes, geometric shifts, or image shadowing. 32GB RAM, using Python 3.7 and PyTorch 1.9.0. We used the Adam optimizer [\[39\]](#page-14-16) with a polynomially decayed learning rate starting at 4 × 10−4 , a batch size of 1, 100 epochs, and a regularization strength λ = 1 in Eq. [\(1\)](#page-2-1) for all learning-based methods across both datasets. Local normalized cross-correlation served as the dissimilarity metric, with r(u) = ∥∇u∥ 2 as the regularirizer for learning based methods as well as outer loop regularizer for SmoothProper. For descriptor matching methods, only GeoFormer [\[42\]](#page-14-3), SuperRetina [\[43\]](#page-14-4), and RetinaIPA [\[71\]](#page-16-10), designed specifically for retinal vessel registration, were trained from scratch on the FIRE dataset. For other models, fine-tuning on FIRE resulted in poorer performance compared to their pre-trained weights, likely due to the dataset's small size. For all learning-based methods, we initialized alignment using the best-performing descriptor matching method. This initial alignment is crucial for unsupervised DIR, as extreme conditions (e.g., 180° rotation) may surpass the capability of deformable-based methods. While SmoothProper is model-agnostic, we employ a simple three-level Laplacian image pyramid as its backbone network. It uses three 2D convoloution, instance normalization and ReLU activations blocks (Conv-Block) as encoder to extract moving and fixed feature maps, followed by bilinear downsampling to form a 3-level image pyramid. At each pyramid level, the fixed feature map along with the moving feature map wapred by the previous level's deformation field (if any) concatenated through channels, followed by three 2D Conv-Blocks for coefficient vector extraction, and finalized with our SmoothProper module to produce the final field. The result of SmoothProper in Table [1](#page-8-0) uses the parameters m = 4×3 , K = 6, and an input image size of 1024×1024. The effects of these parameters are discussed in §[4.3](#page-9-0) and shown in Fig. [6.](#page-11-0) ### 4.3 Results & Analysis Quantitative Results: Table [1](#page-8-0) presents the quantitative results on the FIRE dataset, where the proposed SmoothProper outperforms all other methods across ![](_page_10_Figure_2.jpeg) Fig. 5: Qualitative comparison of deformation fields: (a) Original SmoothProper, (b) SmoothProper without conditional basis vectors, and (c) GAMorph. Darker displacement vectors indicate larger magnitudes. all metrics. This marks the first successful application of unsupervised DIR for retinal vessel registration, achieving a TRE of 1.88 pixels on 2912 × 2912 images. Specifically, SmoothProper reduces TRE by 67.32% over RetinaIPA (detector-based), 69.68% over GeoFormer (detector-free), 39.01% over GAMorph (learning-based w/o unrolling), and 62.20% over VR-Net (learning-based w/ unrolling). SmoothProper also achieves the highest mAUC@25, the key metric for validating registrations with TRE within 25 pixels. Notably, in the breakdown of mAUC@25 into categories A, P, and S, learning-based methods, especially GAMorph and SmoothProper, outperform descriptor-based methods significantly. This reflects the limitations of homography-based descriptor matching in longitudinal retinal imaging, where local deformations occur over time. In addition, most learning-based DIR methods, except GAMorph and VR-Net, perform worse than their initial alignment from descriptor matching method RetinalIPA. This likely occurs because the vessel regions occupy a small portion of the image, causing loss-term-based smooth regularization or message passing to dilute vessel displacements within large, smooth, signal-less regions. Qualitative Results: Fig. [4](#page-9-1) shows the discrepancy between warped moving images and fixed images for exemplar methods in each category. Descriptor matching methods struggle with local deformations, and GAMorph shows limited adaptability. In contrast, SmoothProper best adapts to local deformations and is less affected by shadow artifacts. Fig. [5](#page-10-0) presents local image patches and their displacement fields. Comparing (a) and (c), SmoothProper better preserves local vessel structures, avoids shadowing artifacts, and maintains tissue integrity with a regular field that prevents oversmoothing or over-deemphasis. Complexity and Smoothness Analysis: The SmoothProper (SP) counterpart without SP and CBV has 0.32 MB parameters and 31.84 G multi-adds. With SP (no CBV), it uses 0.32 MB and 31.95 G multi-adds. With SP and CBV, it increases to 41.01 MB and 50.18 G multi-adds. Despite the parameterheavy MLPs for CBV, the SP module remains highly efficient. Readers can decide if the MLPs are worth using based on specific applications. As shown in Fig. [5](#page-10-0) (a) and (b), CBV produces a more regular and plausible displacement field without tuning the λ-parameter, improving TRE slightly from 1.93 to 1.88. All baseline descriptor matching and learning-based methods, including initial alignment runtime, processed each scan pair in under one second. ![](_page_11_Figure_2.jpeg) Fig. 6: Ablation analysis: boxplots and line plots showing the effect of basis vector count (a), inner-loop iterations (b), and input image size (c) on TRE (pixels). Red dashed lines indicate the lowest median TRE in each plot. Ablation Analysis on the Number of Basis Vectors m: In Fig. [6a,](#page-11-0) we vary m from 1×3 2 to 5×3 2 , where 32 mimics the local 3×3 pixel neighborhood in discrete optimization. This neighborhood does not reflect actual displacements but corresponds to learned displacement atoms based on the local disparity context. Larger m allows for richer information querying and better local structural consistency. However, increasing m too much can lead to limitations due to the available training samples. As shown, in FIRE, TRE decreases with m initially, reaching a minimum at 4 × 3 2 before rising again. Ablation Analysis on Iterations K: We vary K from 1 to 9, as shown in Fig. [6b,](#page-11-0) and find that the model achieves optimal performance at K = 6. This result aligns with prior work [\[79\]](#page-16-12) using mean-field inference [\[40\]](#page-14-8) with 5 iterations, where further increasing K yields limited improvement. Ablation Analysis on Image Size: We varied the squared input image size from 128 to 1024, doubling at each step, and included 1456 (half of the original size). As shown in Fig. [6c,](#page-11-0) 128 yields worse performance than the initial alignment. TRE decreases steadily from 256 to 1024 but increases at 1456. Smaller sizes improve efficiency but reduce accuracy, likely due to vessels becoming less visible. Larger sizes can introduce local noise, limiting accuracy gains. ## 5 Conclusions We introduce SmoothProper, a plug-and-play neural network module that enforces smoothness and message passing within the forward pass. By integrating a duality-based optimization layer with tailored interaction terms, it efficiently propagates flow signals while preserving structural consistency. SmoothProper addresses aperture and large displacement challenges in retinal vessel datasets, achieving a TRE of 1.88 pixels in unsupervised DIR for the first time. Although this study focuses on the 2D problem, extending it to 3D is as simple as replacing the 2D convolution with a 3D one. We hope our findings on utilizing the regularization term inspire new milestones and reignite interest in unsupervised DIR techniques. ## References - 1. Aggarwal, H.K., Mani, M.P., Jacob, M.: Modl: Model-based deep learning architecture for inverse problems. IEEE transactions on medical imaging 38(2), 394–405 (2018) - 2. Arsigny, V., Commowick, O., Pennec, X., Ayache, N.: A log-euclidean framework for statistics on diffeomorphisms. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2006: 9th International Conference, Copenhagen, Denmark, October 1-6, 2006. Proceedings, Part I 9. pp. 924–931. Springer (2006) - 3. Ashburner, J.: A fast diffeomorphic image registration algorithm. Neuroimage 38(1), 95–113 (2007) - 4. Avants, B.B., Tustison, N.J., Song, G., Cook, P.A., Klein, A., Gee, J.C.: A Reproducible Evaluation of ANTs Similarity Metric Performance in Brain Image Registration. Neuroimage 54(3), 2033–2044 (2011) - 5. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: Voxelmorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging 38(8), 1788–1800 (2019) - 6. Black, M.J., Anandan, P.: A framework for the robust estimation of optical flow. In: 1993 (4th) International Conference on Computer Vision. pp. 231–236. IEEE (1993) - 7. Blendowski, M., Hansen, L., Heinrich, M.P.: Weakly-supervised learning of multimodal features for regularised iterative descent in 3d image registration. Medical image analysis 67, 101822 (2021) - 8. Chambolle, A.: An algorithm for total variation minimization and applications. Journal of Mathematical imaging and vision 20, 89–97 (2004) - 9. Chen, H., Luo, Z., Zhou, L., Tian, Y., Zhen, M., Fang, T., Mckinnon, D., Tsin, Y., Quan, L.: Aspanformer: Detector-free image matching with adaptive span transformer. In: European Conference on Computer Vision. pp. 20–36. Springer (2022) - 10. Chen, J., Frey, E.C., He, Y., Segars, W.P., Li, Y., Du, Y.: Transmorph: Transformer for unsupervised medical image registration. Medical image analysis 82, 102615 (2022) - 11. Chen, X., Liu, M., Wang, R., Hu, R., Liu, D., Li, G., Zhang, H.: Spatially covariant image registration with text prompts. IEEE Transactions on Neural Networks and Learning Systems pp. 1–11 (2024) - 12. Chen, X., Liu, J., Wang, Z., Yin, W.: Theoretical linear convergence of unfolded ista and its practical weights and thresholds. Advances in Neural Information Processing Systems 31 (2018) - 13. Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces. Medical Image Analysis 57, 226–236 (2019) - 14. DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: Self-supervised interest point detection and description. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pp. 224–236 (2018) - 15. Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., Sun, J.: Repvgg: Making vgg-style convnets great again. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 13733–13742 (2021) - 16. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. In: International Conference on Learning Representations - 17. Edstedt, J., Athanasiadis, I., Wadenb¨ack, M., Felsberg, M.: Dkm: Dense kernelized feature matching for geometry estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17765–17775 (2023) - 18. Edstedt, J., Sun, Q., B¨okman, G., Wadenb¨ack, M., Felsberg, M.: Roma: Revisiting robust losses for dense feature matching. arXiv preprint arXiv:2305.15404 (2023) - 19. Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12873–12883 (2021) - 20. Evan, M.Y., Wang, A.Q., Dalca, A.V., Sabuncu, M.R.: Keymorph: Robust multimodal affine registration via unsupervised keypoint detection. In: Medical imaging with deep learning (2022) - 21. Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. International journal of computer vision 70, 41–54 (2006) - 22. Gleize, P., Wang, W., Feiszli, M.: Silk: Simple learned keypoints. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 22499–22508 (2023) - 23. Gregor, K., LeCun, Y.: Learning fast approximations of sparse coding. In: Proceedings of the 27th international conference on international conference on machine learning. pp. 399–406 (2010) - 24. Ha, D., Dai, A.M., Le, Q.V.: Hypernetworks. In: International Conference on Learning Representations (2017), [https://openreview.net/forum?id=](https://openreview.net/forum?id=rkpACe1lx) [rkpACe1lx](https://openreview.net/forum?id=rkpACe1lx) - 25. Hansen, L., Heinrich, M.P.: Deep learning based geometric registration for medical images: How accurate can we get without visual features? In: Information Processing in Medical Imaging: 27th International Conference, IPMI 2021, Virtual Event, June 28–June 30, 2021, Proceedings 27. pp. 18–30. Springer (2021) - 26. Hansen, L., Heinrich, M.P.: Revisiting iterative highly efficient optimisation schemes in medical image registration. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24. pp. 203– 212. Springer (2021) - 27. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016) - 28. Heinrich, M.P.: Closing the gap between deep and conventional image registration using probabilistic dense displacement networks. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part VI 22. pp. 50–58. Springer (2019) - 29. Heinrich, M.P., Jenkinson, M., Brady, M., Schnabel, J.A.: Mrf-based deformable registration and ventilation estimation of lung ct. IEEE transactions on medical imaging 32(7), 1239–1248 (2013) - 30. Heinrich, M.P., Papie˙z, B.W., Schnabel, J.A., Handels, H.: Non-parametric discrete registration with convex optimisation. In: Biomedical Image Registration: 6th International Workshop, WBIR 2014, London, UK, July 7-8, 2014. Proceedings 6. pp. 51–61. Springer (2014) - 31. Hernandez-Matas, C., Zabulis, X., Triantafyllou, A., Anyfanti, P., Douma, S., Argyros, A.A.: Fire: fundus image registration dataset. Modeling and Artificial Intelligence in Ophthalmology 1(4), 16–28 (2017) - 32. Hershey, J.R., Roux, J.L., Weninger, F.: Deep unfolding: Model-based inspiration of novel deep architectures. arXiv preprint arXiv:1409.2574 (2014) - 33. Hoopes, A., Hoffmann, M., Fischl, B., Guttag, J., Dalca, A.V.: Hypermorph: Amortized hyperparameter learning for image registration. In: Information Processing in Medical Imaging: 27th International Conference, IPMI 2021, Virtual Event, June 28–June 30, 2021, Proceedings 27. pp. 3–17. Springer (2021) - 34. Horn, B.K., Schunck, B.G.: Determining optical flow. Artificial intelligence 17(1-3), 185–203 (1981) - 35. Hu, J., Gan, W., Sun, Z., An, H., Kamilov, U.: A plug-and-play image registration network. In: The Twelfth International Conference on Learning Representations (2024), - 36. Jena, R., Sethi, D., Chaudhari, P., Gee, J.C.: Deep learning in medical image registration: Magic or mirage? arXiv preprint arXiv:2408.05839 (2024) - 37. Jia, X., Bartlett, J., Zhang, T., Lu, W., Qiu, Z., Duan, J.: U-net vs transformer: Is u-net outdated in medical image registration? In: International Workshop on Machine Learning in Medical Imaging. pp. 151–160. Springer (2022) - 38. Jia, X., Thorley, A., Chen, W., Qiu, H., Shen, L., Styles, I.B., Chang, H.J., Leonardis, A., De Marvao, A., O'Regan, D.P., et al.: Learning a model-driven variational network for deformable image registration. IEEE Transactions on Medical Imaging 41(1), 199–212 (2021) - 39. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) - 40. Kr¨ahenb¨uhl, P., Koltun, V.: Efficient inference in fully connected crfs with gaussian edge potentials. Advances in neural information processing systems 24 (2011) - 41. Lindenberger, P., Sarlin, P.E., Pollefeys, M.: Lightglue: Local feature matching at light speed. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 17627–17638 (2023) - 42. Liu, J., Li, X.: Geometrized transformer for self-supervised homography estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 9556–9565 (2023) - 43. Liu, J., Li, X., Wei, Q., Xu, J., Ding, D.: Semi-supervised keypoint detector and descriptor for retinal image matching. In: European Conference on Computer Vision. pp. 593–609. Springer (2022) - 44. Liu, Y., Yu, B., Chen, T., Gu, Y., Du, B., Xu, Y., Cheng, J.: Progressive retinal image registration via global and local deformable transformations. arXiv preprint arXiv:2409.01068 (2024) - 45. Luo, W., Li, Y., Urtasun, R., Zemel, R.: Understanding the effective receptive field in deep convolutional neural networks. Advances in neural information processing systems 29 (2016) - 46. Ma, T., Zhang, S., Li, J., Wen, Y.: Iirp-net: Iterative inference residual pyramid network for enhanced image registration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11546–11555 (2024) - 47. Marstal, K., Berendsen, F., Staring, M., Klein, S.: SimpleElastix: A User-friendly, Multi-lingual Library for Medical Image Registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. pp. 134–142 (2016) - 48. Meng, M., Feng, D., Bi, L., Kim, J.: Correlation-aware coarse-to-fine mlps for deformable medical image registration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9645–9654 (2024) - 49. Mok, T.C., Chung, A.: Affine medical image registration with coarse-to-fine vision transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20835–20844 (2022) - 50. Mok, T.C., Chung, A.C.: Large deformation diffeomorphic image registration with laplacian pyramid networks. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23. pp. 211–221. Springer (2020) - 51. Mok, T.C., Chung, A.C.: Conditional deformable image registration with convolutional neural network. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24. pp. 35–45. Springer (2021) - 52. Mok, T.C., Chung, A.C.: Large deformation image registration with anatomyaware laplacian pyramid networks. In: Segmentation, Classification, and Registration of Multi-modality Medical Imaging Data: MICCAI 2020 Challenges, ABCs 2020, L2R 2020, TN-SCUI 2020, Held in Conjunction with MICCAI 2020, Lima, - 53. Potje, G., Cadar, F., Araujo, A., Martins, R., Nascimento, E.R.: Xfeat: Accelerated features for lightweight image matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2682–2691 (2024) Peru, October 4–8, 2020, Proceedings 23. pp. 61–67. Springer (2021) - 54. Qiu, H., Hammernik, K., Qin, C., Chen, C., Rueckert, D.: Embedding gradientbased optimization in image registration networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 56–65. Springer (2022) - 55. Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Bengio, Y., Courville, A.: On the spectral bias of neural networks. In: International conference on machine learning. pp. 5301–5310. PMLR (2019) - 56. Revaud, J., De Souza, C., Humenberger, M., Weinzaepfel, P.: R2d2: Reliable and repeatable detector and descriptor. Advances in neural information processing systems 32 (2019) - 57. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D: nonlinear phenomena 60(1-4), 259–268 (1992) - 58. Shi, J., He, Y., Kong, Y., Coatrieux, J.L., Shu, H., Yang, G., Li, S.: Xmorpher: Full transformer for deformable medical image registration via cross attention. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 217–226. Springer (2022) - 59. Siebert, H., Großbr¨ohmer, C., Hansen, L., Heinrich, M.P.: Convexadam: Selfconfiguring dual-optimisation-based 3d multitask medical image registration. IEEE Transactions on Medical Imaging (2024) - 60. Steinbr¨ucker, F., Pock, T., Cremers, D.: Large displacement optical flow computation without warping. In: 2009 IEEE 12th International Conference on Computer Vision. pp. 1609–1614. IEEE (2009) - 61. Sukhbaatar, S., Weston, J., Fergus, R., et al.: End-to-end memory networks. Advances in neural information processing systems 28 (2015) - 62. Sun, J., Shen, Z., Wang, Y., Bao, H., Zhou, X.: Loftr: Detector-free local feature matching with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8922–8931 (2021) - 63. Sun, J., Li, H., Xu, Z., et al.: Deep admm-net for compressive sensing mri. Advances in neural information processing systems 29 (2016) - 64. Truong, P., Apostolopoulos, S., Mosinska, A., Stucky, S., Ciller, C., Zanet, S.D.: Glampoints: Greedily learned accurate match points. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 10732–10741 (2019) - 65. Tyszkiewicz, M., Fua, P., Trulls, E.: Disk: Learning local features with policy gradient. Advances in Neural Information Processing Systems 33, 14254–14265 (2020) - 66. Van Den Oord, A., Vinyals, O., et al.: Neural discrete representation learning. Advances in neural information processing systems 30 (2017) - 67. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) - 68. Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Diffeomorphic demons: Efficient non-parametric image registration. NeuroImage 45(1), S61–S72 (2009) - 69. Wang, A.Q., Evan, M.Y., Dalca, A.V., Sabuncu, M.R.: A robust and interpretable deep learning framework for multi-modal registration via keypoints. Medical Image Analysis 90, 102962 (2023) - 70. Wang, H., Ni, D., Wang, Y.: Recursive deformable pyramid network for unsupervised medical image registration. IEEE Transactions on Medical Imaging (2024) - 71. Wang, J., Li, H., Hu, D., Xu, R., Yao, X., Tao, Y.K., Oguz, I.: Retinal ipa: Iterative keypoints alignment for multimodal retinal imaging. arXiv preprint arXiv:2407.18362 (2024) - 72. Wang, P., Zheng, W., Chen, T., Wang, Z.: Anti-oversmoothing in deep vision transformers via the fourier domain analysis: From theory to practice. arXiv preprint arXiv:2203.05962 (2022) - 73. Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime tv-l 1 optical flow. In: Pattern Recognition: 29th DAGM Symposium, Heidelberg, Germany, September 12-14, 2007. Proceedings 29. pp. 214–223. Springer (2007) - 74. Zhang, H., Chen, X., Hu, R., Liu, D., Li, G., Wang, R.: Memwarp: Discontinuitypreserving cardiac registration with memorized anatomical filters. In: Linguraru, M.G., Dou, Q., Feragen, A., Giannarou, S., Glocker, B., Lekadir, K., Schnabel, J.A. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. pp. 671–681. Springer Nature Switzerland, Cham (2024) - 75. Zhang, H., Chen, X., Wang, R., Hu, R., Liu, D., Li, G.: Slicer networks. arXiv preprint arXiv:2401.09833 (2024) - 76. Zhang, H., Wang, R., Hu, R., Zhang, J., Li, J.: Deda: Deep directed accumulator. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 765–775. Springer (2023) - 77. Zhang, J., Liu, Z., Zhang, S., Zhang, H., Spincemaille, P., Nguyen, T.D., Sabuncu, M.R., Wang, Y.: Fidelity imposed network edit (fine) for solving ill-posed image reconstruction. Neuroimage 211, 116579 (2020) - 78. Zhang, J., Spincemaille, P., Zhang, H., Nguyen, T.D., Li, C., Li, J., Kovanlikaya, I., Sabuncu, M.R., Wang, Y.: Laro: Learned acquisition and reconstruction optimization to accelerate quantitative susceptibility mapping. NeuroImage p. 119886 (2023) - 79. Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., Torr, P.H.: Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE international conference on computer vision. pp. 1529–1537 (2015)