Add support for a pandasrc

[jtratner]

Just using the existing configuration framework, but with a file format like matplotlib uses... See how they do it here: http://matplotlib.org/users/customizing.html.

Plus, we can document all the config options in a single file.

related #2452, #3046

[cpcloud]

+1 for a matplotlibish format

both ipython and python itself have existing startup file mechanisms in place,
I'd still like to hear a good argument explaining what a .pandasrc file provides
that those do not.

[jtratner]

@y-p lets you distribute a project with settings options in the directory to make it work the way you expect and allow you to set something for one project that you aren't doing for others. You can't unilaterally overwrite or append to people's python startup files. This is a way to get granular formatting without needing to alter those things or even know where they are. And this doesn't prevent you from overriding those settings in a startup file.

I see, so it's not about an addition to a user's dotfiles but about making a project self-contained.
In that case, how would such a file improve on just putting some plain ol' initialization code in
the source files?

[jtratner]

@y-p requires imports or setting environment variables. It's also a pretty minor addition - what's below plus a bit of searching for the path to the config file:

def from_object(obj):
    if hasattr('items'):
        set_option(*obj.items())
    else:
        for k in obj:
            set_option(k, obj[k])

def from_file(path_or_buf):
    from pandas.core.common import _get_handle
    option_splitter = re.compile('\s*[:=]\s*').split
    f = _get_handle(path_or_buf)
    errors = []
    for i, line in enumerate(f):
        # allow for comments
        line = line.split('#')[0].strip()
        if line:
            try:
                split = option_splitter(line)
                if len(split) == 2:
                    option, value = split
                    set_option(option, value)
                else:
                    raise ValueError("Malformed option")
            except (KeyError, ValueError) as e:
                errors.append("%d: %s" % (i, e))
    print errors

[jtratner]

and clearly better errors, etc.

[jtratner]

Plus, this separates config options from actual code, which is a net gain.

[cpcloud]

Personally, I'd rather set some config file than have a bunch of calls to pd.set_config() in a startup file. And like @jtratner says, supporting different configs for different projects by e.g., reading a single from the current directory is easier to keep track of than a bunch of calls to pd.set_config().

Nope, initialization code does not requires either envars or imports.
Don't see how the errors would be clearly better. If it is, why not improve the existing
error messages?
If you want to seperate config options from actual code, use another source file, hey presto.
Configuraion is code last time I checked, and seems to be elsewhere as well.
I don't think I've worked on a project that does not have some sort of settings.py, that's
familiar to most people (certainly django folk).

I don't follow what @cpcloud means by "keeping track of" at all. configuration is
just code. I don't think that holds up.

re verbosity of pd.set_config. yep. big deal. How many LOC are we talking? meh.

[cpcloud]

Nope, initialization code does not requires either envars or imports.

I don't see how that's true re imports.

I don't think I've worked on a project that does not have some sort of settings.py

Lucky you. I have, and it's not fun.

I don't follow what @cpcloud means by "keeping track of" at all.

I just meant that I'd rather have a single config file per project than having to copy paste pd.set_config() and tweak the calls, but a settings.py-like file works just fine too. It's not a big deal, I actually use the defaults for almost everything so this doesn't really bother me.

And that's about all I'll say on that.

[jtratner]

okay, that's fine. I'd like to add from_object (to allow you to create a dict and set a bunch of options at once) and make options support __getitem__ and __setitem__ just to clean things up.

In other words, allows you to do this (which is nearly equivalent to what the pandasrc would do in syntax, etc):

config.from_object({
'io.excel.xlsx.writer': 'openpyxl',
'display.max_rows': 80
})

and this

config.options['io.excel.xlsx.writer'] = 'openpyxl'
config.options['display.max_rows'] = 80

Is that okay? feels cleaner to me than a series of function calls.

[jreback]

look ok by me....only requrest I have is to make setting the docs for the option easier (not sure how as you generally need/want a multi-line, so end up creating a variable to hold it.....)

Yeah, I like that too. The python logging module supports a from_dict for configuration,
which comes in handy in that situation. I find I need to change options so I don't miss
this in practice, but if you feel there's a need then from_dict is a good way to do it.

re supporting set/getitem, You can already get/set values directly, e.g. display.foo =1.
"Cleaner" is partly a matter of taste and that form isn't more concise then the
existing set_option mechanism, nor more convenient then the existing options. way of doing it
which also provides tab-completion. Do you feel strongly that adding a 3rd way to do the exact
same thing is worth it?