survival analysis

[OriolAbril]

File: https://github.com/pymc-devs/pymc-examples/blob/main/examples/survival_analysis/survival_analysis.ipynb
Reviewers:

The sections below may still be pending. If so, the issue is still available, it simply doesn't
have specific guidance yet. Please refer to this overview of updates

Known changes needed

Changes listed in this section should all be done at some point in order to get this
notebook to a "Best Practices" state. However, these are probably not enough!
Make sure to thoroughly review the notebook and search for other updates.

General updates

• fix typos
• fix math renderend incorrectly
• update notation to use equal tailed interval (or ETI) instead of HPD (or HDI), both are valid credible intervals, but they are different, so we should not write HPD if using ETI

ArviZ related

• Use arviz (or only import? I don't really understand how that notebook runs)
• Use InferenceData and xarray for exploratory analysis of the results

Changes for discussion

Changes listed in this section are up for discussion, these are ideas on how to improve
the notebook but may not have a clear implementation, or fix some know issue only partially.

General updates

ArviZ related

Notes

Exotic dependencies

Computing requirements

[olgadk7]

In addition, should I update the notebook as mentioned in pymc-devs/pymc#4027?

[fonnesbeck]

@olgadk7 That would be terrific, thanks!

[OriolAbril]

That would be amazing indeed, thanks! Just to clarify, we don't expect you to fix all the bullet points above, choose which ones you want to address and let us know so we can review based on that and force extra work on you after reviewing.

[olgadk7]

Regarding ArviZ, in the survival_analysis notebook it doesn’t do anything that pymc3 couldn’t, does it make sense then to only use pymc3 perhaps? Unless I’m missing some rules that govern how to chose between the two (didn’t find this info after a quick search, just that you seemingly always need to set the ArviZ style, as per the Notebook Style Guide).

Related to that, I’m confused how and why the notebook’s source code https://github.com/pymc-devs/pymc-examples/blob/main/examples/survival_analysis/survival_analysis.ipynb calls to az. (without even importing it), but the published notebook https://docs.pymc.io/notebooks/survival_analysis.html uses pymc3 only.

On another note, Can I add the “with model” context argument to get rid of the corresponding FutureWarning of deprecation?

Regarding the other needed changes, once I deal with the low-hanging fruit, I’d be happy to look into it and report back asap but most likely I wouldn’t trust myself with the more consequential stuff just yet (only if you “force” me) :)

[OriolAbril]

Regarding ArviZ, in the survival_analysis notebook it doesn’t do anything that pymc3 couldn’t, does it make sense then to only use pymc3 perhaps?

PyMC3 has ArviZ as a dependency, and delegates plotting, stats and diagnostics to it. However, for convenience most ArviZ functions are also available as part of the of the pymc3 namespace, pm.plot_trace or pm.summary are aliases to the corresponding ArviZ function. While you are free to use those aliases, we want to encourage using ArviZ directly.

Related to that, I’m confused how and why the notebook’s source code https://github.com/pymc-devs/pymc-examples/blob/main/examples/survival_analysis/survival_analysis.ipynb calls to az. (without even importing it),

I have no idea myself, and what is worse, the notebook looks executed sequentially unless I counted wrong.

but the published notebook https://docs.pymc.io/notebooks/survival_analysis.html uses pymc3 only.

Docs are build only with releases, and those changes were added to pymc-examples after the last release, they will be published with the next release

On another note, Can I add the “with model” context argument to get rid of the corresponding FutureWarning of deprecation?

which FutureWarning? We are currently working on the next major version of PyMC3 so there are several laying around 😅

Regarding the other needed changes, once I deal with the low-hanging fruit, I’d be happy to look into it and report back asap but most likely I wouldn’t trust myself with the more consequential stuff just yet (only if you “force” me) :)

If you are interested in working on that go for it! It's only a matter of we'll be here to provide any help necessary

[olgadk7]

Thank you! And, got it, will use ArviZ directly.

Re FutureWarning: it seems like where the trace is called without the context of the model object ("with model:”) you get a FutureWarning saying "Using from_pymc3 without the model will be deprecated in a future release. Not using the model will return less accurate and less useful results. Make sure you use the model argument or call from_pymc3 within a model context." (adding "with model:" gets rid of the warning.)

[OriolAbril]

Re FutureWarning: it seems like where the trace is called without the context of the model object ("with model:”) you get a FutureWarning saying "Using from_pymc3 without the model will be deprecated in a future release. Not using the model will return less accurate and less useful results. Make sure you use the model argument or call from_pymc3 within a model context." (adding "with model:" gets rid of the warning.)

Yes, that warning is probably triggered when plotting right? What is happening is that to plot, ArviZ first checks if the input is an InferenceData object, if it isn't, it tries to convert the input to inferencedata automatically. Converting pm.MultiTrace to InferenceData works with only the info in the trace in many cases, but in some others info from the model instance is also needed, so the converter needs both. Calling the plotting functions from within the model context make the model available to the converter, but this is not ideal.

The recommended solution is to first convert to inferencedata or directly use return_inferencedata=True in pm.sample and then use an inferencedata for plotting.

[olgadk7]

That worked, thank you for explaining! And you're right: triggered when plotting. Should I then add return_inferencedata=True in every pm.sample in the notebook so that there aren't any of those FutureWarnings?

[OriolAbril]

Should I then add return_inferencedata=True in every pm.sample in the notebook so that there aren't any of those FutureWarnings?

Yes, moreover, return_inferencedata=True will be the default in next version so if at some place you wanted to avoid inferencedata, you'd need to be explicit and manually pass False. Docs should show how to use inferencedata and other best practices though

[olgadk7]

After a cascade of changes resulting from a simple inferencedata change, including updating pymc3, theano-pymc3, as well as my whole environment, and conda-removing theano, I’m up against the non-descriptive “NotImplementedError:” when calling aesara methods listed in its docs. Looked for solutions everywhere, please direct.

This is the line causing it: “lambda_ = pm.Deterministic("lambda_", T.outer(T.exp(beta * df.metastasized), lambda0))” except I replaced “T.” with “at.” as there was a big messy compilation error after removing theano. Am I off-track?

[OriolAbril]

Which versions do you have installed/are using? You shouldn't be using Aesara yet, the latest pymc3 stable release (3.11.2) runs on theano-pymc only: https://github.com/pymc-devs/pymc3/blob/v3.11.2/requirements.txt

[olgadk7]

oh ok - i was under the impression that theano-pymc is aesara, as that's where the theano-pymc link leads to on the installation guide, as well as on the "PyMC3 and Theano" page. So to run theano-pymc, i use "import theano"? Versions:

Python implementation: CPython
Python version       : 3.7.10
IPython version      : 7.22.0

arviz     : 0.11.2
cachetools : 4.2.1
dill : 0.3.3
numpy     : 1.19.2
pandas    : 1.2.3
patsy : 0.5.1
scipy : 1.6.2
theano-pymc : 1.1.2
typing_extensions : 3.7.4.3
pymc3     : 3.11.1
aesara    : 2.0.6

Thanks!

[OriolAbril]

So to run theano-pymc, i use "import theano"?

Yes! I think that should be enough for everything to work.