diff --git a/doc/source/user_guide/io.rst b/doc/source/user_guide/io.rst index 1b05c53df4516..98cba6969587b 100644 --- a/doc/source/user_guide/io.rst +++ b/doc/source/user_guide/io.rst @@ -3565,13 +3565,6 @@ HDFStore will by default not drop rows that are all missing. This behavior can b os.remove('file.h5') -.. ipython:: python - :suppress: - - os.remove('file.h5') - - - .. _io.hdf5-fixed: Fixed Format diff --git a/doc/source/user_guide/timedeltas.rst b/doc/source/user_guide/timedeltas.rst index 37cf6afcb96a3..40a8fd3101409 100644 --- a/doc/source/user_guide/timedeltas.rst +++ b/doc/source/user_guide/timedeltas.rst @@ -191,13 +191,12 @@ Operands can also appear in a reversed order (a singular object operated with a df.min().idxmax() df.min(axis=1).idxmin() -You can fillna on timedeltas. Integers will be interpreted as seconds. You can -pass a timedelta to get a particular value. +You can fillna on timedeltas, passing a timedelta to get a particular value. .. ipython:: python - y.fillna(0) - y.fillna(10) + y.fillna(pd.Timedelta(0)) + y.fillna(pd.Timedelta(10, unit='s')) y.fillna(pd.Timedelta('-1 days, 00:00:05')) You can also negate, multiply and use ``abs`` on ``Timedeltas``: diff --git a/doc/source/user_guide/timeseries.rst b/doc/source/user_guide/timeseries.rst index 590fde2aaccf8..f559b0d073320 100644 --- a/doc/source/user_guide/timeseries.rst +++ b/doc/source/user_guide/timeseries.rst @@ -322,13 +322,16 @@ which can be specified. These are computed from the starting point specified by 1349720105400, 1349720105500], unit='ms') Constructing a :class:`Timestamp` or :class:`DatetimeIndex` with an epoch timestamp -with the ``tz`` argument specified will localize the epoch timestamps to UTC -first then convert the result to the specified time zone. +with the ``tz`` argument specified will currently localize the epoch timestamps to UTC +first then convert the result to the specified time zone. However, this behavior +is :ref:`deprecated `, and if you have +epochs in wall time in another timezone, it is recommended to read the epochs +as timezone-naive timestamps and then localize to the appropriate timezone: .. ipython:: python - pd.Timestamp(1262347200000000000, tz='US/Pacific') - pd.DatetimeIndex([1262347200000000000], tz='US/Pacific') + pd.Timestamp(1262347200000000000).tz_localize('US/Pacific') + pd.DatetimeIndex([1262347200000000000]).tz_localize('US/Pacific') .. note:: diff --git a/doc/source/whatsnew/v0.10.0.rst b/doc/source/whatsnew/v0.10.0.rst index 2d6550bb6888d..3671a5e24bdaa 100644 --- a/doc/source/whatsnew/v0.10.0.rst +++ b/doc/source/whatsnew/v0.10.0.rst @@ -295,79 +295,171 @@ Updated PyTables Support :ref:`Docs ` for PyTables ``Table`` format & several enhancements to the api. Here is a taste of what to expect. -.. ipython:: python - :suppress: - :okexcept: +.. code-block:: ipython - import os + In [41]: store = pd.HDFStore('store.h5') - os.remove('store.h5') + In [42]: df = pd.DataFrame(np.random.randn(8, 3), + ....: index=pd.date_range('1/1/2000', periods=8), + ....: columns=['A', 'B', 'C']) -.. ipython:: python + In [43]: df + Out[43]: + A B C + 2000-01-01 -2.036047 0.000830 -0.955697 + 2000-01-02 -0.898872 -0.725411 0.059904 + 2000-01-03 -0.449644 1.082900 -1.221265 + 2000-01-04 0.361078 1.330704 0.855932 + 2000-01-05 -1.216718 1.488887 0.018993 + 2000-01-06 -0.877046 0.045976 0.437274 + 2000-01-07 -0.567182 -0.888657 -0.556383 + 2000-01-08 0.655457 1.117949 -2.782376 - store = pd.HDFStore('store.h5') - df = pd.DataFrame(np.random.randn(8, 3), - index=pd.date_range('1/1/2000', periods=8), - columns=['A', 'B', 'C']) - df + [8 rows x 3 columns] - # appending data frames - df1 = df[0:4] - df2 = df[4:] - store.append('df', df1) - store.append('df', df2) - store + # appending data frames + In [44]: df1 = df[0:4] - # selecting the entire store - store.select('df') + In [45]: df2 = df[4:] -.. ipython:: python - :okwarning: + In [46]: store.append('df', df1) - wp = pd.Panel(np.random.randn(2, 5, 4), items=['Item1', 'Item2'], - major_axis=pd.date_range('1/1/2000', periods=5), - minor_axis=['A', 'B', 'C', 'D']) - wp + In [47]: store.append('df', df2) - # storing a panel - store.append('wp', wp) + In [48]: store + Out[48]: + + File path: store.h5 + /df frame_table (typ->appendable,nrows->8,ncols->3,indexers->[index]) - # selecting via A QUERY - store.select('wp', "major_axis>20000102 and minor_axis=['A','B']") + # selecting the entire store + In [49]: store.select('df') + Out[49]: + A B C + 2000-01-01 -2.036047 0.000830 -0.955697 + 2000-01-02 -0.898872 -0.725411 0.059904 + 2000-01-03 -0.449644 1.082900 -1.221265 + 2000-01-04 0.361078 1.330704 0.855932 + 2000-01-05 -1.216718 1.488887 0.018993 + 2000-01-06 -0.877046 0.045976 0.437274 + 2000-01-07 -0.567182 -0.888657 -0.556383 + 2000-01-08 0.655457 1.117949 -2.782376 - # removing data from tables - store.remove('wp', "major_axis>20000103") - store.select('wp') + [8 rows x 3 columns] + +.. code-block:: ipython + + In [50]: wp = pd.Panel(np.random.randn(2, 5, 4), items=['Item1', 'Item2'], + ....: major_axis=pd.date_range('1/1/2000', periods=5), + ....: minor_axis=['A', 'B', 'C', 'D']) + + In [51]: wp + Out[51]: + + Dimensions: 2 (items) x 5 (major_axis) x 4 (minor_axis) + Items axis: Item1 to Item2 + Major_axis axis: 2000-01-01 00:00:00 to 2000-01-05 00:00:00 + Minor_axis axis: A to D + + # storing a panel + In [52]: store.append('wp', wp) + + # selecting via A QUERY + In [53]: store.select('wp', [pd.Term('major_axis>20000102'), + ....: pd.Term('minor_axis', '=', ['A', 'B'])]) + ....: + Out[53]: + + Dimensions: 2 (items) x 3 (major_axis) x 2 (minor_axis) + Items axis: Item1 to Item2 + Major_axis axis: 2000-01-03 00:00:00 to 2000-01-05 00:00:00 + Minor_axis axis: A to B + + # removing data from tables + In [54]: store.remove('wp', pd.Term('major_axis>20000103')) + Out[54]: 8 + + In [55]: store.select('wp') + Out[55]: + + Dimensions: 2 (items) x 3 (major_axis) x 4 (minor_axis) + Items axis: Item1 to Item2 + Major_axis axis: 2000-01-01 00:00:00 to 2000-01-03 00:00:00 + Minor_axis axis: A to D + + # deleting a store + In [56]: del store['df'] + + In [57]: store + Out[57]: + + File path: store.h5 + /wp wide_table (typ->appendable,nrows->12,ncols->2,indexers->[major_axis,minor_axis]) - # deleting a store - del store['df'] - store **Enhancements** - added ability to hierarchical keys - .. ipython:: python + .. code-block:: ipython + + In [58]: store.put('foo/bar/bah', df) + + In [59]: store.append('food/orange', df) - store.put('foo/bar/bah', df) - store.append('food/orange', df) - store.append('food/apple', df) - store + In [60]: store.append('food/apple', df) - # remove all nodes under this level - store.remove('food') - store + In [61]: store + Out[61]: + + File path: store.h5 + /foo/bar/bah frame (shape->[8,3]) + /food/apple frame_table (typ->appendable,nrows->8,ncols->3,indexers->[index]) + /food/orange frame_table (typ->appendable,nrows->8,ncols->3,indexers->[index]) + /wp wide_table (typ->appendable,nrows->12,ncols->2,indexers->[major_axis,minor_axis]) + + # remove all nodes under this level + In [62]: store.remove('food') + + In [63]: store + Out[63]: + + File path: store.h5 + /foo/bar/bah frame (shape->[8,3]) + /wp wide_table (typ->appendable,nrows->12,ncols->2,indexers->[major_axis,minor_axis]) - added mixed-dtype support! .. ipython:: python - df['string'] = 'string' - df['int'] = 1 - store.append('df', df) - df1 = store.select('df') - df1 - df1.get_dtype_counts() + In [64]: df['string'] = 'string' + + In [65]: df['int'] = 1 + + In [66]: store.append('df', df) + + In [67]: df1 = store.select('df') + + In [68]: df1 + Out[68]: + A B C string int + 2000-01-01 -2.036047 0.000830 -0.955697 string 1 + 2000-01-02 -0.898872 -0.725411 0.059904 string 1 + 2000-01-03 -0.449644 1.082900 -1.221265 string 1 + 2000-01-04 0.361078 1.330704 0.855932 string 1 + 2000-01-05 -1.216718 1.488887 0.018993 string 1 + 2000-01-06 -0.877046 0.045976 0.437274 string 1 + 2000-01-07 -0.567182 -0.888657 -0.556383 string 1 + 2000-01-08 0.655457 1.117949 -2.782376 string 1 + + [8 rows x 5 columns] + + In [69]: df1.get_dtype_counts() + Out[69]: + float64 3 + int64 1 + object 1 + dtype: int64 - performance improvements on table writing - support for arbitrarily indexed dimensions @@ -392,13 +484,6 @@ Updated PyTables Support - minor change to select and remove: require a table ONLY if where is also provided (and not None) -.. ipython:: python - :suppress: - - store.close() - import os - os.remove('store.h5') - **Compatibility** 0.10 of ``HDFStore`` is backwards compatible for reading tables created in a prior version of pandas, diff --git a/doc/source/whatsnew/v0.11.0.rst b/doc/source/whatsnew/v0.11.0.rst index baa464fe842d3..26b97268038e9 100644 --- a/doc/source/whatsnew/v0.11.0.rst +++ b/doc/source/whatsnew/v0.11.0.rst @@ -278,6 +278,7 @@ Enhancements - ``Squeeze`` to possibly remove length 1 dimensions from an object. .. ipython:: python + :okwarning: p = pd.Panel(np.random.randn(3, 4, 4), items=['ItemA', 'ItemB', 'ItemC'], major_axis=pd.date_range('20010102', periods=4), diff --git a/doc/source/whatsnew/v0.12.0.rst b/doc/source/whatsnew/v0.12.0.rst index b2dd8229c91f3..5dbfb2c728f06 100644 --- a/doc/source/whatsnew/v0.12.0.rst +++ b/doc/source/whatsnew/v0.12.0.rst @@ -317,22 +317,24 @@ Other Enhancements - ``pd.set_option()`` now allows N option, value pairs (:issue:`3667`). - Let's say that we had an option ``'a.b'`` and another option ``'b.c'``. - We can set them at the same time: + Let's say that we had an option ``'a.b'`` and another option ``'b.c'``. + We can set them at the same time: - .. ipython:: python - :suppress: + .. ipython:: python - pd.core.config.register_option('a.b', 2, 'ay dot bee') - pd.core.config.register_option('b.c', 3, 'bee dot cee') + In [31]: pd.get_option('a.b') + Out[31]: 2 - .. ipython:: python + In [32]: pd.get_option('b.c') + Out[32]: 3 + + In [33]: pd.set_option('a.b', 1, 'b.c', 4) + + In [34]: pd.get_option('a.b') + Out[34]: 1 - pd.get_option('a.b') - pd.get_option('b.c') - pd.set_option('a.b', 1, 'b.c', 4) - pd.get_option('a.b') - pd.get_option('b.c') + In [35]: pd.get_option('b.c') + Out[35]: 4 - The ``filter`` method for group objects returns a subset of the original object. Suppose we want to take only elements that belong to groups with a diff --git a/doc/source/whatsnew/v0.13.0.rst b/doc/source/whatsnew/v0.13.0.rst index 0f799c069f494..13a2f879211b3 100644 --- a/doc/source/whatsnew/v0.13.0.rst +++ b/doc/source/whatsnew/v0.13.0.rst @@ -272,6 +272,7 @@ This is like an ``append`` operation. A Panel setting operation on an arbitrary axis aligns the input to the Panel .. ipython:: python + :okwarning: p = pd.Panel(np.arange(16).reshape(2, 4, 2), items=['Item1', 'Item2'], @@ -543,7 +544,7 @@ Enhancements .. ipython:: python - td.fillna(0) + td.fillna(pd.Timedelta(0)) td.fillna(datetime.timedelta(days=1, seconds=5)) You can do numeric reduction operations on timedeltas. diff --git a/doc/source/whatsnew/v0.15.0.rst b/doc/source/whatsnew/v0.15.0.rst index 7b9a8ba082411..f9e47b45f498d 100644 --- a/doc/source/whatsnew/v0.15.0.rst +++ b/doc/source/whatsnew/v0.15.0.rst @@ -702,6 +702,7 @@ Other notable API changes: This can also be seen in multi-axis indexing with a ``Panel``. .. ipython:: python + :okwarning: p = pd.Panel(np.arange(2 * 3 * 4).reshape(2, 3, 4), items=['ItemA', 'ItemB'], diff --git a/doc/source/whatsnew/v0.15.2.rst b/doc/source/whatsnew/v0.15.2.rst index dabdcd1ab76c3..9f0449d6a1754 100644 --- a/doc/source/whatsnew/v0.15.2.rst +++ b/doc/source/whatsnew/v0.15.2.rst @@ -161,6 +161,7 @@ Other enhancements: - ``Panel`` now supports the ``all`` and ``any`` aggregation functions. (:issue:`8302`): .. ipython:: python + :okwarning: p = pd.Panel(np.random.rand(2, 5, 4) > 0.1) p.all() diff --git a/doc/source/whatsnew/v0.24.0.rst b/doc/source/whatsnew/v0.24.0.rst index eb0b522e08a32..05d6a03639a2d 100644 --- a/doc/source/whatsnew/v0.24.0.rst +++ b/doc/source/whatsnew/v0.24.0.rst @@ -567,7 +567,7 @@ missing indicator, ``np.nan``. (:issue:`20377`) .. ipython:: python :suppress: - from pandas.io import StringIO + from io import StringIO *Previous Behavior*: