Add support for second-order low-rank inducing bilinear optimization frameworks (and related penalties)

Aperta
  • Qualcosa non va?
#50 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
15/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Ferma
Stack tecnologico
python
Ambito
machine-learning

Direzione di ricerca

The issue points to pyproximal/pyproximal/utils/bilinear.py and the PALM optimizer as existing entry points. Read those first, then investigate how second-order methods such as Levenberg-Marquardt and Variable Projection could fit the bilinear framework. Done would mean adding support for the proposed methods and, as a later step, related regularizers; the issue does not specify implementation details or tests.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

There is already support for BilinearOperator in pyproximal/pyproximal/utils/bilinear.py and the PALM optimizer; however,
they do not scale to second-order methods such as Levenberg-Marquardt (LM) and Variable Projection (VarPro). In recent years, such methods have been become popular alternatives to splitting methods such as ADMM for certain types of problems (e.g. image analysis and computer vision).

Examples

As an example the nuclear norm penalty has a variational formulation
image
which was utilized in [1] using a bilinear framework (no second order method involved, though).

This also scales to the weighted nuclear norm, and it has been successfully implemented for vision tasks using second-order methods in e.g. [2, 7, 8]. Other non-convex penalties have been studied as well, e.g. [3, 4, 5, 6].

Suggestion on enhancement

I suggest adding support for Levenberg-Marquardt (LM) and Variable Projection (VarPro). As a second step I suggest adding some of the mentioned regularizers.

References

[1] Cabral et al. "Unifying Nuclear Norm and Bilinear Factorization Approaches for Low-Rank Matrix Decomposition", In the Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2013
[2] Iglesias et al. "Accurate Optimization of Weighted Nuclear Norm for Non-Rigid Structure from Motion", In the Proceedings of the European Conference on Computer Vision (ECCV), 2020
[3] Valtonen Ornhag et al. "Bilinear parameterization for differentiable rank-regularization", In the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPR), 2020
[4] Valtonen Ornhag et al. "Differentiable Fixed-Rank Regularisation using Bilinear Parameterisation", In the Proceedings of the British Machine Vision Conference (BMVC), 2019
[5] Valtonen Ornhag et al. "Bilinear Parameterization for Non-Separable Singular Value Penalties", In the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[6] Xu et al. "A unified convex surrogate for the schatten-p norm". In Proceedings of the Conference on Artificial Intelligence (AAAI), 2017
[7] Kumar "Non-rigid structure from motion: Prior-free factorization method revisited" In IEEE Winter Conference on Applications of Computer Vision (WACV), 2020
[8] McDonald et al. "Spectral k-support norm regularization", In Advances in Neural Information Processing Systems (NIPS), 2014

Lingua principale
Python
Stelle
85
Fork
22
Merge medio
2h 11m
PR unite (30g)
7

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di PyLops/pyproximal

Tutte le issue di PyLops/pyproximal

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.