Nonmonotone variant of linesearch methods #96
Conversation
|
@aldma thanks, I should be able to review this over the weekend. I’m wondering about the test failures: the test dependency on AbstractOperators we could remove; regarding RecursiveArrayTools, not sure, I’ll need to look into it. |
src/algorithms/panoc.jl
Outdated
| # retrieve merit and set threshold | ||
| FBE_x = state.merit | ||
| sigma = iter.beta * (0.5 / state.gamma) * (1 - iter.alpha) | ||
| tol = 10 * eps(R) * (1 + abs(FBE_x)) | ||
| threshold = FBE_x - sigma * norm(state.res)^2 + tol |
There was a problem hiding this comment.
Why do the assignment FBE_x = state.merit here? It seems to be giving an inappropriate name to this quantity. Should you not directly use state.merit where needed in the lines that follow?
There was a problem hiding this comment.
Also wondering whether merit is a good name here: this quantity appears to be an exponential moving average of the values of the merit function we're minimizing, is that correct?
There was a problem hiding this comment.
You are right, I just wanted to minimize the changes relative to the monotone version. The assignment FBE_x = state.merit has been removed now with dbdd5c1, where possible.
In ZeroFPR and PANOCplus I left an assignment to FBE_x before the loop because the tests where failing otherwise; I did not understand why the variable FBE_x was not available at the final update of state.merit.
There was a problem hiding this comment.
Yes, and the value of merit is the one considered as baseline for accepting or rejecting an update in the linesearch. Also the objective function and the envelope are used as merit functions, but they can decrease nonmonotonically, whereas what is called merit decreases monotonically also in the nonmonotone variant.
I am happy to change nomenclature if you have a more appropriate one.
|
I hope I didn't mess things up with the rebasing. I am still learning gitology. |
This PR adds the possibility to run algorithms with a linesearch procedure in a nonmonotone variant. In particular,
PANOC,PANOCplus,ZeroFPRandDRLS;monotonicityis added to control the level of monotonicity required, with0 < monotonicity <= 1;monotonicity = 1coincides with the current implementation (monotone descent);meritis included in thestate, monitoring progress instead of objective or envelope values.