DOC: Remove docs code header and use explicit imports in the user guide

[datapythonista]

I've been discussing with the Binder team, and looks like it'll be possible soon to make the pandas examples in the docs runnable directly from the docs.

I think this will be amazing, but I think it's one more reason to start being explicit and self-contained with the docs. One option is of course that the trickiness we have now on importing things translate to Binder, and we automatically inject code for imports (and may be seeds...).

But my opinion is that the right thing is to start being explicit on what we run in the examples.

[TomAugspurger]

we automatically inject code for imports

+1 for removing the {{ header }} stuff in favor of explicit imports.

And also avoid aliasing our own module

I disagree here. Let's keep that as a separate issue.

[jorisvandenbossche]

Also +1 on explicitly doing and showing the imports.
We could also see if they can be simplified. Eg randn = np.random.randn could be fixed for all occurences to use the full version. And maybe pd.options.display.max_rows = 15 can be removed with the shorter default nowadays (although if the examples use a length of eg 50, that would still be shown).

And also agree with keeping the alias discussion for a separate issue. That will be easier for this one ;)

[datapythonista]

Sorry for mixing things. I think there is agreement on this one, for the user guide. I was also considering the API, but I'll also open a separate issue for that, so we keep the discussions focused. Thanks for the feedback.

[jreback]

why the move to remove header macro? IIRC this was put in place to make sure that all of the imports are exactly the same everywhere on each doc section

this seems like churn for no gain

[datapythonista]

The code in the header is hidden, and we implemented the union of all the blocks with imports, seeds... to avoid duplication, and because having a huge header file in every file was transparent to the user.

The rendered header added a problem that is that Sphinx doesn't report the right line for the errors anymore (it reports the line on the rendered file, but we look for the error in the template). It was already under discussion whether we wanted to keep it or not.

But the reason for proposing this now, is that the header adds extra complexity to make the code runnable in Binder. In two different ways:

• Technical complexity in our side (it may be easy to address, but in any case it's added complexity)
• For the user, if we let them open the code in a Binder notebook, since they may end up with a lot of code that we don't show in the docs, and that in many cases may not even be relevant

The header surely provides value in avoiding repetition, but IMO it's not worth. It's better to repeat the needed parts, and also be explicit about all what we're doing, since with Binder we'd give the users the change to play with the code, and any magic there will be confusing and unappreciated.

[jreback]

we’ll i still would have a fixed header of imports; having consistency in the docs overrides all other issues ; it easily becomes a mess without enforcement via CI

[datapythonista]

Now we validate in the CI that Sphinx doesn't generate warnings, so if an import is missing the CI should fail.

[jorisvandenbossche]

having consistency in the docs overrides all other issues

Note that this is only for the doc build, as it is about a hidden code block right now. And then @datapythonista is correct that this is now validated in CI that the doc build runs correctly.

Part of the proposal is also to make it no longer a hidden code block. So there we also need to ensure that what we show to the users is somewhat consistent throughout the user guide. But I think we can certainly achieve that without inserting a common header as well.

[jreback]

having consistency in the docs overrides all other issues

Note that this is only for the doc build, as it is about a hidden code block right now. And then @datapythonista is correct that this is now validated in CI that the doc build runs correctly.

Part of the proposal is also to make it no longer a hidden code block. So there we also need to ensure that what we show to the users is somewhat consistent throughout the user guide. But I think we can certainly achieve that without inserting a common header as well.

well this was just changed to make it consistent for each file
the concern here is the same as the original

it will get inconsistent thru time across files

so if possible to have a single file with consistent imports that is then objected to the build would be preferable

[jorisvandenbossche]

Your last sentence is not very clear to me.