Add a notebook for Bayesian Factor Analysis

[chartl]

Fulfilling the request here:

https://discourse.pymc.io/t/unique-solution-for-probabilistic-pca/1324/48

This PR adds a notebook that features Bayesian factor analysis, addressing issues both of model factor identification:

https://discourse.pymc.io/t/unique-solution-for-probabilistic-pca/1324

as well as scalability:

https://discourse.pymc.io/t/large-scale-factor-analysis-with-minibatch-advi/246/10

[junpenglao]

A few comments:

• Could you also add the other cases and solution discussed in https://discourse.pymc.io/t/unique-solution-for-probabilistic-pca/1324?
• We recently change the plotting backend to arviz, could you please upgrade your pymc3 to master and rerun the notebook? Note that because of Traceplot parity arviz-devs/arviz#94 you can not plot all RVs, so you need to choose a few dimension to make your point.
• tips: when you are doing plotting, placing a ; at the end suppress the text output in jupyter notebook:

pm.traceplot(trace);

[chartl]

• Could you also add the other cases and solution discussed in https://discourse.pymc.io/t/unique-solution-for-probabilistic-pca/1324?

By "the other solution" I assume you mean the sign-flip symmetrized ADVI? It's a nice example of the group API, but rotation and permutation symmetries remain using this method.

And is "the other case" a reference to a different synthetic dataset? (the obs here: https://discourse.pymc.io/t/unique-solution-for-probabilistic-pca/1324/14)?

[junpenglao]

By "the other solution" I assume you mean the sign-flip symmetrized ADVI? It's a nice example of the group API, but rotation and permutation symmetries remain using this method.

Oh I see - thanks for the information. In that case if you just fix the plotting that would be sufficient.

[chartl]

I've installed a fresh pull of master with python setup.py develop, but I still seem to be getting the same plots. I also switched from python2 to python3 -- is there a way to check i have the right branch version? (pm.version gives 3.6)

[lucianopaz]

@chartl, I think that this problem is similar to what you came across on discourse earlier. Now your repo should be up to date with master for you to re run the notebook.

[fonnesbeck]

A few recommendations:

• when plotting a large number of variables, a forest plot is more appropriate than a trace plot, which gets awfully crowded.
• rather than plotting every variable, perhaps just pick a few illustrative ones.
• either use a semicolon after the plot call, or assign the return values to variables, to suppress the figure and axes objects from being printed
• remove stray line at bottom of notebook

[twiecki]

ping @chartl

[chartl]

I couldn't find documentation in arviz on how to plot just a few components of a matrix or vector. Is there a proper way to do this?

For now I used pm.Deterministic to pull a few into the trace that I could reference with traceplot(varnames=foo)

[chartl]

ping @fonnesbeck

[fonnesbeck]

Looks really good! I was wondering the same thing regarding Arviz, particularly for forest plots, actually. Might be worth a feature request.

[ColCarroll]

For the example you provided, the equivalent call would be:

az.plot_trace(trace, ('W', 'F', 'psi'), 
              coords={'W_dim_0': range(1, 4), 
                      'W_dim_1': 0, 
                      'F_dim_0': 0, 
                      'F_dim_1': range(1, 4)});

Note that we do not yet have "compact" as an option for the traceplot, so the dimensions are spread out:

[ColCarroll]

(er, and note that az.plot_trace is aliased to pm.traceplot. also, i figured this out by manually calling data = az.from_pymc3(trace), and inspecting data.posterior to figure out the coordinates to slice)