Skip to content

BUG: fixed old version compatibility warnings #2607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Jan 6, 2013
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions RELEASE.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ pandas 0.10.1
**Improvements to existing features**

- ``HDFStore``

- enables storing of multi-index dataframes (closes GH1277_)
- support data column indexing and selection, via ``data_columns`` keyword
in append
- support data column indexing and selection, via ``data_columns`` keyword in append
- support write chunking to reduce memory footprint, via ``chunksize``
keyword to append
- support automagic indexing via ``index`` keywork to append
Expand All @@ -50,18 +50,22 @@ pandas 0.10.1
to do multiple-table append/selection
- added support for datetime64 in columns
- added method ``unique`` to select the unique values in an indexable or data column
- added method ``copy`` to copy an existing store (and possibly upgrade)
- show the shape of the data on disk for non-table stores when printing the store
- Add ``logx`` option to DataFrame/Series.plot (GH2327_, #2565)
- Support reading gzipped data from file-like object
- ``pivot_table`` aggfunc can be anything used in GroupBy.aggregate (GH2643_)

**Bug fixes**

- ``HDFStore``

- correctly handle ``nan`` elements in string columns; serialize via the
``nan_rep`` keyword to append
- raise correctly on non-implemented column types (unicode/date)
- handle correctly ``Term`` passed types (e.g. ``index<1000``, when index
is ``Int64``), (closes GH512_)
- handle Timestamp correctly in data_columns (closes GH2637_)
- Fix DataFrame.info bug with UTF8-encoded columns. (GH2576_)
- Fix DatetimeIndex handling of FixedOffset tz (GH2604_)
- More robust detection of being in IPython session for wide DataFrame
Expand All @@ -76,18 +80,21 @@ pandas 0.10.1
**API Changes**

- ``HDFStore``

- refactored HFDStore to deal with non-table stores as objects, will allow future enhancements
- removed keyword ``compression`` from ``put`` (replaced by keyword
``complib`` to be consistent across library)

.. _GH512: https://github.com/pydata/pandas/issues/512
.. _GH1277: https://github.com/pydata/pandas/issues/1277
.. _GH2327: https://github.com/pydata/pandas/issues/2327
.. _GH2576: https://github.com/pydata/pandas/issues/2576
.. _GH2585: https://github.com/pydata/pandas/issues/2585
.. _GH2604: https://github.com/pydata/pandas/issues/2604
.. _GH2576: https://github.com/pydata/pandas/issues/2576
.. _GH2616: https://github.com/pydata/pandas/issues/2616
.. _GH2625: https://github.com/pydata/pandas/issues/2625
.. _GH2643: https://github.com/pydata/pandas/issues/2643
.. _GH2637: https://github.com/pydata/pandas/issues/2637

pandas 0.10.0
=============
Expand Down
Binary file added doc/source/_static/legacy_0.10.h5
Binary file not shown.
37 changes: 25 additions & 12 deletions doc/source/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1106,7 +1106,7 @@ Storing Mixed Types in a Table
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Storing mixed-dtype data is supported. Strings are store as a fixed-width using the maximum size of the appended column. Subsequent appends will truncate strings at this length.
Passing ``min_itemsize = { `values` : size }`` as a parameter to append will set a larger minimum for the string columns. Storing ``floats, strings, ints, bools, datetime64`` are currently supported. For string columns, passing ``nan_rep = 'my_nan_rep'`` to append will change the default nan representation on disk (which converts to/from `np.nan`), this defaults to `nan`.
Passing ``min_itemsize = { `values` : size }`` as a parameter to append will set a larger minimum for the string columns. Storing ``floats, strings, ints, bools, datetime64`` are currently supported. For string columns, passing ``nan_rep = 'nan'`` to append will change the default nan representation on disk (which converts to/from `np.nan`), this defaults to `nan`.

.. ipython:: python

Expand All @@ -1115,9 +1115,6 @@ Passing ``min_itemsize = { `values` : size }`` as a parameter to append will set
df_mixed['int'] = 1
df_mixed['bool'] = True
df_mixed['datetime64'] = Timestamp('20010102')

# make sure that we have datetime64[ns] types
df_mixed = df_mixed.convert_objects()
df_mixed.ix[3:5,['A','B','string','datetime64']] = np.nan

store.append('df_mixed', df_mixed, min_itemsize = { 'values' : 50 })
Expand All @@ -1128,8 +1125,6 @@ Passing ``min_itemsize = { `values` : size }`` as a parameter to append will set
# we have provided a minimum string column size
store.root.df_mixed.table

It is ok to store ``np.nan`` in a ``float or string``. Make sure to do a ``convert_objects()`` on the frame before storing a ``np.nan`` in a datetime64 column. Storing a column with a ``np.nan`` in a ``int, bool`` will currently throw an ``Exception`` as these columns will have converted to ``object`` type.

Storing Multi-Index DataFrames
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -1268,11 +1263,11 @@ To retrieve the *unique* values of an indexable or data column, use the method `

**Table Object**

If you want to inspect the table object, retrieve via ``get_table``. You could use this progamatically to say get the number of rows in the table.
If you want to inspect the stored object, retrieve via ``get_storer``. You could use this progamatically to say get the number of rows in an object.

.. ipython:: python

store.get_table('df_dc').nrows
store.get_storer('df_dc').nrows

Multiple Table Queries
~~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -1348,7 +1343,7 @@ Or on-the-fly compression (this only applies to tables). You can turn off file c

- ``ptrepack --chunkshape=auto --propindexes --complevel=9 --complib=blosc in.h5 out.h5``

Furthermore ``ptrepack in.h5 out.h5`` will *repack* the file to allow you to reuse previously deleted space (alternatively, one can simply remove the file and write again).
Furthermore ``ptrepack in.h5 out.h5`` will *repack* the file to allow you to reuse previously deleted space. Aalternatively, one can simply remove the file and write again, or use the ``copy`` method.

Notes & Caveats
~~~~~~~~~~~~~~~
Expand All @@ -1372,10 +1367,28 @@ Notes & Caveats
Compatibility
~~~~~~~~~~~~~

0.10 of ``HDFStore`` is backwards compatible for reading tables created in a prior version of pandas,
however, query terms using the prior (undocumented) methodology are unsupported. ``HDFStore`` will issue a warning if you try to use a prior-version format file. You must read in the entire
file and write it out using the new format to take advantage of the updates. The group attribute ``pandas_version`` contains the version information.
0.10.1 of ``HDFStore`` is backwards compatible for reading tables created in a prior version of pandas however, query terms using the prior (undocumented) methodology are unsupported. ``HDFStore`` will issue a warning if you try to use a prior-version format file. You must read in the entire file and write it out using the new format, using the method ``copy`` to take advantage of the updates. The group attribute ``pandas_version`` contains the version information. ``copy`` takes a number of options, please see the docstring.


.. ipython:: python

# a legacy store
import os
legacy_store = HDFStore('legacy_0.10.h5', 'r')
legacy_store

# copy (and return the new handle)
new_store = legacy_store.copy('store_new.h5')
new_store
new_store.close()

.. ipython:: python
:suppress:

legacy_store.close()
import os
os.remove('store_new.h5')


Performance
~~~~~~~~~~~
Expand Down
3 changes: 3 additions & 0 deletions doc/source/v0.10.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ New features
HDFStore
~~~~~~~~

You may need to upgrade your existing data files. Please visit the **compatibility** section in the main docs.


.. ipython:: python
:suppress:
:okexcept:
Expand Down
Loading