Skip to content

Commit fe20bc6

Browse files
authored
[WIP] Add new features to unbalanced solvers (#551)
* add new features to unbalanced solvers * add new features to unbalanced solvers * fix bug in test * remove stab_sinkhorn * remove kl * fix bug in lbfgsb_unbalanced * fix bug in lbfgsb_unbalanced * fix bug in KL in sinkhorn_unbalanced * edit release.md * fix test * add test and rearrange arguments * fix test * fix test * fix test * fix bug in test * fix bug in doctest * fix bug in doctest * add test for more coverage
1 parent 1071759 commit fe20bc6

File tree

3 files changed

+386
-282
lines changed

3 files changed

+386
-282
lines changed

RELEASES.md

Lines changed: 15 additions & 148 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,7 @@
1313
+ Update wheels to Python 3.12 and remove old i686 arch that do not have scipy wheels (PR #543)
1414
+ Upgraded unbalanced OT solvers for more flexibility (PR #539)
1515
+ Add LazyTensor for modeling plans and low rank tensor in large scale OT (PR #544)
16-
+ Add exact line-search for `gromov_wasserstein` and `fused_gromov_wasserstein` with KL loss (PR #556)
17-
+ Add KL loss to all semi-relaxed (Fused) Gromov-Wasserstein solvers (PR #559)
16+
+ Further upgraded unbalanced OT solvers for more flexibility and future use (PR #551)
1817

1918
#### Closed issues
2019
- Fix line search evaluating cost outside of the interpolation range (Issue #502, PR #504)
@@ -36,143 +35,13 @@ Many other bugs and issues have been fixed and we want to thank all the contribu
3635

3736

3837
#### New features
39-
- Gaussian Gromov Wasserstein loss and mapping (PR #498)
40-
- Template-based Fused Gromov Wasserstein GNN layer in `ot.gnn` (PR #488)
41-
- Make alpha parameter in semi-relaxed Fused Gromov Wasserstein differentiable (PR #483)
42-
- Make alpha parameter in Fused Gromov Wasserstein differentiable (PR #463)
43-
- Added the sparsity-constrained OT solver to `ot.smooth` and added `projection_sparse_simplex` to `ot.utils` (PR #459)
44-
- Add tests on GPU for master branch and approved PR (PR #473)
45-
- Add `median` method to all inherited classes of `backend.Backend` (PR #472)
46-
- Update tests for macOS and Windows, speedup documentation (PR #484)
47-
- Added Proximal Point algorithm to solve GW problems via a new parameter `solver="PPA"` in `ot.gromov.entropic_gromov_wasserstein` + examples (PR #455)
48-
- Added features `warmstart` and `kwargs` in `ot.gromov.entropic_gromov_wasserstein` to respectively perform warmstart on dual potentials and pass parameters to `ot.sinkhorn` (PR #455)
49-
- Added sinkhorn projection based solvers for FGW `ot.gromov.entropic_fused_gromov_wasserstein` and entropic FGW barycenters + examples (PR #455)
50-
- Added features `warmstartT` and `kwargs` to all CG and entropic (F)GW barycenter solvers (PR #455)
51-
- Added entropic semi-relaxed (Fused) Gromov-Wasserstein solvers in `ot.gromov` + examples (PR #455)
52-
- Make marginal parameters optional for (F)GW solvers in `._gw`, `._bregman` and `._semirelaxed` (PR #455)
53-
- Add Entropic Wasserstein Component Analysis (ECWA) in ot.dr (PR #486)
54-
- Added feature Efficient Discrete Multi Marginal Optimal Transport Regularization + examples (PR #454)
5538

56-
#### Closed issues
57-
58-
- Fix gromov conventions (PR #497)
59-
- Fix change in scipy API for `cdist` (PR #487)
60-
- More permissive check_backend (PR #494)
61-
- Fix circleci-redirector action and codecov (PR #460)
62-
- Fix issues with cuda for ot.binary_search_circle and with gradients for ot.sliced_wasserstein_sphere (PR #457)
63-
- Major documentation cleanup (PR #462, PR #467, PR #475)
64-
- Fix gradients for "Wasserstein2 Minibatch GAN" example (PR #466)
65-
- Faster Bures-Wasserstein distance with NumPy backend (PR #468)
66-
- Fix issue backend for ot.sliced_wasserstein_sphere ot.sliced_wasserstein_sphere_unif (PR #471)
67-
- Fix issue with ot.barycenter_stabilized when used with PyTorch tensors and log=True (PR #474)
68-
- Fix `utils.cost_normalization` function issue to work with multiple backends (PR #472)
69-
- Fix pression error on marginal sums and (Issue #429, PR #496)
70-
71-
#### New Contributors
72-
* @kachayev made their first contribution in PR #462
73-
* @liutianlin0121 made their first contribution in PR #459
74-
* @francois-rozet made their first contribution in PR #468
75-
* @framunoz made their first contribution in PR #472
76-
* @SoniaMaz8 made their first contribution in PR #483
77-
* @tomMoral made their first contribution in PR #494
78-
* @12hengyu made their first contribution in PR #454
79-
80-
## 0.9.0
81-
*April 2023*
82-
83-
This new release contains so many new features and bug fixes since 0.8.2 that we
84-
decided to make it a new minor release at 0.9.0.
85-
86-
The release contains many new features. First we did a major
87-
update of all Gromov-Wasserstein solvers that brings up to 30% gain in
88-
computation time (see PR #431) and allows the GW solvers to work on non symmetric
89-
matrices. It also brings novel solvers for the very
90-
efficient [semi-relaxed GW problem
91-
](https://pythonot.github.io/master/auto_examples/gromov/plot_semirelaxed_fgw.html#sphx-glr-auto-examples-gromov-plot-semirelaxed-fgw-py)
92-
that can be used to find the best re-weighting for one of the distributions. We
93-
also now have fast and differentiable solvers for [Wasserstein on the circle](https://pythonot.github.io/master/auto_examples/plot_compute_wasserstein_circle.html#sphx-glr-auto-examples-plot-compute-wasserstein-circle-py) and
94-
[sliced Wasserstein on the
95-
sphere](https://pythonot.github.io/master/auto_examples/backends/plot_ssw_unif_torch.html#sphx-glr-auto-examples-backends-plot-ssw-unif-torch-py).
96-
We are also very happy to provide new OT barycenter solvers such as the [Free
97-
support Sinkhorn
98-
barycenter](https://pythonot.github.io/master/auto_examples/barycenters/plot_free_support_sinkhorn_barycenter.html#sphx-glr-auto-examples-barycenters-plot-free-support-sinkhorn-barycenter-py)
99-
and the [Generalized Wasserstein
100-
barycenter](https://pythonot.github.io/master/auto_examples/barycenters/plot_generalized_free_support_barycenter.html#sphx-glr-auto-examples-barycenters-plot-generalized-free-support-barycenter-py).
101-
A new differentiable solver for OT across spaces that provides OT plans
102-
between samples and features simultaneously and
103-
called [Co-Optimal
104-
Transport](https://pythonot.github.io/master/auto_examples/others/plot_COOT.html)
105-
has also been implemented. Finally we began working on OT between Gaussian distributions and
106-
now provide differentiable estimation for the Bures-Wasserstein [divergence](https://pythonot.github.io/master/gen_modules/ot.gaussian.html#ot.gaussian.bures_wasserstein_distance) and
107-
[mappings](https://pythonot.github.io/master/auto_examples/domain-adaptation/plot_otda_linear_mapping.html#sphx-glr-auto-examples-domain-adaptation-plot-otda-linear-mapping-py).
108-
109-
Another important first step toward POT 1.0 is the
110-
implementation of a unified API for OT solvers with introduction of [`ot.solve`](https://pythonot.github.io/master/all.html#ot.solve)
111-
function that can solve (depending on parameters) exact, regularized and
112-
unbalanced OT and return a new
113-
[`OTResult`](https://pythonot.github.io/master/gen_modules/ot.utils.html#ot.utils.OTResult)
114-
object. The idea behind this new API is to facilitate exploring different solvers
115-
with just a change of parameter and get a more unified API for them. We will keep
116-
the old solvers API for power users but it will be the preferred way to solve
117-
problems starting from release 1.0.0.
118-
We provide below some examples of use for the new function and how to
119-
recover different aspects of the solution (OT plan, full loss, linear part of the
120-
loss, dual variables) :
121-
```python
122-
#Solve exact ot
123-
sol = ot.solve(M)
124-
125-
# get the results
126-
G = sol.plan # OT plan
127-
ot_loss = sol.value # OT value (full loss for regularized and unbalanced)
128-
ot_loss_linear = sol.value_linear # OT value for linear term np.sum(sol.plan*M)
129-
alpha, beta = sol.potentials # dual potentials
130-
131-
# direct plan and loss computation
132-
G = ot.solve(M).plan
133-
ot_loss = ot.solve(M).value
134-
135-
# OT exact with marginals a/b
136-
sol2 = ot.solve(M, a, b)
137-
138-
# regularized and unbalanced OT
139-
sol_rkl = ot.solve(M, a, b, reg=1) # KL regularization
140-
sol_rl2 = ot.solve(M, a, b, reg=1, reg_type='L2')
141-
sol_ul2 = ot.solve(M, a, b, unbalanced=10, unbalanced_type='L2')
142-
sol_rkl_ukl = ot.solve(M, a, b, reg=10, unbalanced=10) # KL + KL
143-
144-
```
145-
The function is fully compatible with backends and will be implemented for
146-
different types of distribution support (empirical distributions, grids) and OT
147-
problems (Gromov-Wasserstein) in the new releases. This new API is not yet
148-
presented in the kickstart part of the documentation as there is a small change
149-
that it might change
150-
when implementing new solvers but we encourage users to play with it.
151-
152-
Finally, in addition to those many new this release fixes 20 issues (some long
153-
standing) and we want to thank all the contributors who made this release so
154-
big. More details below.
155-
156-
157-
#### New features
158-
- Added feature to (Fused) Gromov-Wasserstein solvers inherited from `ot.optim` to support relative and absolute loss variations as stopping criterions (PR #431)
159-
- Added feature to (Fused) Gromov-Wasserstein solvers to handle asymmetric matrices (PR #431)
160-
- Added semi-relaxed (Fused) Gromov-Wasserstein solvers in `ot.gromov` + examples (PR #431)
161-
- Added the spherical sliced-Wasserstein discrepancy in `ot.sliced.sliced_wasserstein_sphere` and `ot.sliced.sliced_wasserstein_sphere_unif` + examples (PR #434)
162-
- Added the Wasserstein distance on the circle in ``ot.lp.solver_1d.wasserstein_circle`` (PR #434)
163-
- Added the Wasserstein distance on the circle (for p>=1) in `ot.lp.solver_1d.binary_search_circle` + examples (PR #434)
164-
- Added the 2-Wasserstein distance on the circle w.r.t a uniform distribution in `ot.lp.solver_1d.semidiscrete_wasserstein2_unif_circle` (PR #434)
16539
- Added Bures Wasserstein distance in `ot.gaussian` (PR ##428)
16640
- Added Generalized Wasserstein Barycenter solver + example (PR #372), fixed graphical details on the example (PR #376)
16741
- Added Free Support Sinkhorn Barycenter + example (PR #387)
16842
- New API for OT solver using function `ot.solve` (PR #388)
169-
- Backend version of `ot.partial` and `ot.smooth` (PR #388 and #449)
170-
- Added argument for warmstart of dual potentials in Sinkhorn-based methods in `ot.bregman` (PR #437)
171-
- Added parameters method in `ot.da.SinkhornTransport` (PR #440)
172-
- `ot.dr` now uses the new Pymanopt API and POT is compatible with current
173-
Pymanopt (PR #443)
174-
- Added CO-Optimal Transport solver + examples (PR #447)
175-
- Remove the redundant `nx.abs()` at the end of `wasserstein_1d()` (PR #448)
43+
- Backend version of `ot.partial` and `ot.smooth` (PR #388)
44+
- Added argument for warmstart of dual vectors in Sinkhorn-based methods in `ot.bregman` (PR #437)
17645

17746
#### Closed issues
17847

@@ -200,11 +69,9 @@ PR #413)
20069
- Fix an issue where the parameter `stopThr` in `empirical_sinkhorn_divergence` was rendered useless by subcalls
20170
that explicitly specified `stopThr=1e-9` (Issue #421, PR #422).
20271
- Fixed a bug breaking an example where we would try to make an array of arrays of different shapes (Issue #424, PR #425)
203-
- Fixed an issue with the documentation gallery section (PR #444)
204-
- Fixed issues with cuda variables for `line_search_armijo` and `entropic_gromov_wasserstein` (Issue #445, #PR 446)
72+
20573

20674
## 0.8.2
207-
*April 2022*
20875

20976
This releases introduces several new notable features. The less important
21077
but most exiting one being that we now have a logo for the toolbox (color
@@ -348,7 +215,7 @@ a [Generative Network
348215
(GAN)](https://PythonOT.github.io/auto_examples/backends/plot_wass2_gan_torch.html),
349216
for a [sliced Wasserstein gradient
350217
flow](https://PythonOT.github.io/auto_examples/backends/plot_sliced_wass_grad_flow_pytorch.html)
351-
and [optimizing the Gromov-Wasserstein distance](https://PythonOT.github.io/auto_examples/backends/plot_optim_gromov_pytorch.html). Note that the Jax backend is still in early development and quite
218+
and [optimizing the Gromov-Wassersein distance](https://PythonOT.github.io/auto_examples/backends/plot_optim_gromov_pytorch.html). Note that the Jax backend is still in early development and quite
352219
slow at the moment, we strongly recommend for Jax users to use the [OTT
353220
toolbox](https://github.com/google-research/ott) when possible.
354221
As a result of this new feature,
@@ -360,7 +227,7 @@ Pointwise Gromov
360227
Wasserstein](https://PythonOT.github.io/auto_examples/gromov/plot_gromov.html#compute-gw-with-a-scalable-stochastic-method-with-any-loss-function),
361228
Sinkhorn in log space with `method='sinkhorn_log'`, [Projection Robust
362229
Wasserstein](https://PythonOT.github.io/gen_modules/ot.dr.html?highlight=robust#ot.dr.projection_robust_wasserstein),
363-
ans [debiased Sinkhorn barycenters](https://PythonOT.github.ioauto_examples/barycenters/plot_debiased_barycenter.html).
230+
ans [deviased Sinkorn barycenters](https://PythonOT.github.ioauto_examples/barycenters/plot_debiased_barycenter.html).
364231

365232
This release will also simplify the installation process. We have now a
366233
`pyproject.toml` that defines the build dependency and POT should now build even
@@ -501,15 +368,15 @@ are coming for the next versions.
501368

502369
#### Closed issues
503370

504-
- Add JMLR paper to the readme and Mathieu Blondel to the Acknowledgments (PR
371+
- Add JMLR paper to the readme and Mathieu Blondel to the Acknoledgments (PR
505372
#231, #232)
506373
- Bug in Unbalanced OT example (Issue #127)
507374
- Clean Cython output when calling setup.py clean (Issue #122)
508375
- Various Macosx compilation problems (Issue #113, Issue #118, PR#130)
509376
- EMD dimension mismatch (Issue #114, Fixed in PR #116)
510377
- 2D barycenter bug for non square images (Issue #124, fixed in PR #132)
511378
- Bad value in EMD 1D (Issue #138, fixed in PR #139)
512-
- Log bugs for Gromov-Wasserstein solver (Issue #107, fixed in PR #108)
379+
- Log bugs for Gromov-Wassertein solver (Issue #107, fixed in PR #108)
513380
- Weight issues in barycenter function (PR #106)
514381

515382
## 0.6.0
@@ -540,9 +407,9 @@ a solver for [Unbalanced OT
540407
barycenters](https://github.com/rflamary/POT/blob/master/notebooks/plot_UOT_barycenter_1D.ipynb).
541408
A new variant of Gromov-Wasserstein divergence called [Fused
542409
Gromov-Wasserstein](https://pot.readthedocs.io/en/latest/all.html?highlight=fused_#ot.gromov.fused_gromov_wasserstein)
543-
has been also contributed with examples of use on [structured
410+
has been also contributed with exemples of use on [structured
544411
data](https://github.com/rflamary/POT/blob/master/notebooks/plot_fgw.ipynb) and
545-
computing [barycenters of labeled
412+
computing [barycenters of labeld
546413
graphs](https://github.com/rflamary/POT/blob/master/notebooks/plot_barycenter_fgw.ipynb).
547414

548415

@@ -603,7 +470,7 @@ and [free support](https://github.com/rflamary/POT/blob/master/notebooks/plot_fr
603470
implementation of entropic OT.
604471

605472
POT 0.5 also comes with a rewriting of ot.gpu using the cupy framework instead of
606-
the unmaintained cudamat. Note that while we tried to keep changes to the
473+
the unmaintained cudamat. Note that while we tried to keed changes to the
607474
minimum, the OTDA classes were deprecated. If you are happy with the cudamat
608475
implementation, we recommend you stay with stable release 0.4 for now.
609476

@@ -627,7 +494,7 @@ and new POT contributors (you can see the list in the [readme](https://github.co
627494
* Stochastic OT in the dual and semi-dual (PR #52 and PR #62)
628495
* Free support barycenters (PR #56)
629496
* Speed-up Sinkhorn function (PR #57 and PR #58)
630-
* Add convolutional Wasserstein barycenters for 2D images (PR #64)
497+
* Add convolutional Wassersein barycenters for 2D images (PR #64)
631498
* Add Greedy Sinkhorn variant (Greenkhorn) (PR #66)
632499
* Big ot.gpu update with cupy implementation (instead of un-maintained cudamat) (PR #67)
633500

@@ -678,15 +545,15 @@ This release contains a lot of contribution from new contributors.
678545
* new notebooks for emd computation and Wasserstein Discriminant Analysis
679546
* relocate notebooks
680547
* update documentation
681-
* clean_zeros(a,b,M) for removing zeros in sparse distributions
548+
* clean_zeros(a,b,M) for removimg zeros in sparse distributions
682549
* GPU implementations for sinkhorn and group lasso regularization
683550

684551

685552
## V0.2
686553
*7 Apr 2017*
687554

688555
* New dimensionality reduction method (WDA)
689-
* Efficient method emd2 returns only transport (in parallel if several histograms given)
556+
* Efficient method emd2 returns only tarnsport (in paralell if several histograms given)
690557

691558

692559

@@ -727,4 +594,4 @@ It provides the following solvers:
727594
* Optimal transport for domain adaptation with group lasso regularization
728595
* Conditional gradient and Generalized conditional gradient for regularized OT.
729596

730-
Some demonstrations (both in Python and Jupyter Notebook format) are available in the examples folder.
597+
Some demonstrations (both in Python and Jupyter Notebook format) are available in the examples folder.

0 commit comments

Comments
 (0)