From a169f6427db559708aadbbf06ba723bf143eadae Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Thu, 21 Jan 2021 12:59:11 +0100 Subject: [PATCH 01/12] bug: cell ids included classes bug: set_td_classes loop ignored entries bug: tooltips indexing was dropped doc: added documentation for methods --- doc/source/user_guide/style.ipynb | 267 ++++++++++++++++++++++++++---- pandas/io/formats/style.py | 23 +-- 2 files changed, 244 insertions(+), 46 deletions(-) diff --git a/doc/source/user_guide/style.ipynb b/doc/source/user_guide/style.ipynb index 24f344488d1ca..2793bb1dc58d6 100644 --- a/doc/source/user_guide/style.ipynb +++ b/doc/source/user_guide/style.ipynb @@ -15,30 +15,19 @@ "\n", "The styling is accomplished using CSS.\n", "You write \"style functions\" that take scalars, `DataFrame`s or `Series`, and return *like-indexed* DataFrames or Series with CSS `\"attribute: value\"` pairs for the values.\n", - "These functions can be incrementally passed to the `Styler` which collects the styles before rendering." + "These functions can be incrementally passed to the `Styler` which collects the styles before rendering.\n", + "\n", + "CSS is a flexible language and as such there may be multiple ways of achieving the same result, with potential\n", + "advantages or disadvantages, which we try to illustrate." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "## Building styles\n", - "\n", - "Pass your style functions into one of the following methods:\n", - "\n", - "- ``Styler.applymap``: elementwise\n", - "- ``Styler.apply``: column-/row-/table-wise\n", - "\n", - "Both of those methods take a function (and some other keyword arguments) and applies your function to the DataFrame in a certain way.\n", - "`Styler.applymap` works through the DataFrame elementwise.\n", - "`Styler.apply` passes each column or row into your DataFrame one-at-a-time or the entire table at once, depending on the `axis` keyword argument.\n", - "For columnwise use `axis=0`, rowwise use `axis=1`, and for the entire table at once use `axis=None`.\n", + "## Styler Object\n", "\n", - "For `Styler.applymap` your function should take a scalar and return a single string with the CSS attribute-value pair.\n", - "\n", - "For `Styler.apply` your function should take a Series or DataFrame (depending on the axis parameter), and return a Series or DataFrame with an identical shape where each value is a string with a CSS attribute-value pair.\n", - "\n", - "Let's see some examples." + "The `DataFrame.style` attribute is a property that returns a `Styler` object. `Styler` has a `_repr_html_` method defined on it so they are rendered automatically. If you want the actual HTML back for further processing or for writing to file call the `.render()` method which returns a string." ] }, { @@ -61,6 +50,7 @@ "outputs": [], "source": [ "import pandas as pd\n", + "from pandas.io.formats.style import Styler\n", "import numpy as np\n", "\n", "np.random.seed(24)\n", @@ -68,14 +58,15 @@ "df = pd.concat([df, pd.DataFrame(np.random.randn(10, 4), columns=list('BCDE'))],\n", " axis=1)\n", "df.iloc[3, 3] = np.nan\n", - "df.iloc[0, 2] = np.nan" + "df.iloc[0, 2] = np.nan\n", + "df.style" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Here's a boring example of rendering a DataFrame, without any (visible) styles:" + "The above output looks very similar to the standard DataFrame HTML representation. But we've done some work behind the scenes to attach CSS classes to each cell. We can view these by calling the `.render` method." ] }, { @@ -84,16 +75,40 @@ "metadata": {}, "outputs": [], "source": [ - "df.style" + "df.style.render().split('\\n')[:10]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `row0_col2` is the identifier for that particular cell. We've also prepended each row/column identifier with a UUID unique to each DataFrame so that the style from one doesn't collide with the styling from another within the same notebook or page (you can set the `uuid` if you'd like to tie together the styling of two DataFrames, or remove it if you want to optimise HTML transfer for larger tables)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "*Note*: The `DataFrame.style` attribute is a property that returns a `Styler` object. `Styler` has a `_repr_html_` method defined on it so they are rendered automatically. If you want the actual HTML back for further processing or for writing to file call the `.render()` method which returns a string.\n", + "## Building styles\n", "\n", - "The above output looks very similar to the standard DataFrame HTML representation. But we've done some work behind the scenes to attach CSS classes to each cell. We can view these by calling the `.render` method." + "There are 3 primary methods of adding custom styles to DataFrames using CSS and matching it to cells:\n", + "\n", + "- Directly linking external CSS classes to your individual cells using `Styler.set_td_classes`.\n", + "- Using `table_styles` to control broader areas of the DataFrame with internal CSS.\n", + "- Using the `Styler.apply` and `Styler.applymap` functions for more specific control with internal CSS. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Linking External CSS\n", + "\n", + "*New in version 1.2.0*\n", + "\n", + "If you have designed a website then it is likely you will already have an external CSS file that controls the styling of table and cell objects within your website.\n", + "\n", + "For example, suppose we have an external CSS which controls table properties and has some additional classes to style individual elements (here we manually add one to this notebook):" ] }, { @@ -102,16 +117,110 @@ "metadata": {}, "outputs": [], "source": [ - "df.style.highlight_null().render().split('\\n')[:10]" + "from IPython.display import HTML\n", + "style = \\\n", + "\"\"\n", + "HTML(style)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The `row0_col2` is the identifier for that particular cell. We've also prepended each row/column identifier with a UUID unique to each DataFrame so that the style from one doesn't collide with the styling from another within the same notebook or page (you can set the `uuid` if you'd like to tie together the styling of two DataFrames).\n", + "Now we can manually link these to our DataFrame using the `Styler.set_table_attributes` and `Styler.set_td_classes` methods (note that table level 'table-cls' is overwritten here by Jupyters own CSS, but in HTML the default text color will be grey)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "s = df.style.set_table_attributes('class=\"table-cls\"')\n", + "cls = pd.DataFrame(data=[['cls1', None], ['cls3', 'cls2 cls3']], index=[0,2], columns=['A', 'C'])\n", + "s.set_td_classes(cls)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The **advantage** of linking to external CSS is that it can be applied very easily. One can build a DataFrame of (multiple) CSS classes to add to each cell dynamically using traditional `DataFrame.apply` and `DataFrame.applymap` methods, or otherwise, and then add those to the Styler. It will integrate with your website's existing CSS styling.\n", + "\n", + "The **disadvantage** of this approach is that it is not easy to transmit files standalone. For example the external CSS must be included or the styling will simply be lost. It is also, as this example shows, not well suited (at a table level) for Jupyter Notebooks. Also this method cannot be used for exporting to Excel, for example, since the external CSS cannot be referenced either by the exporters or by Excel itself." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Using Table Styles\n", + "\n", + "Table styles allow you to control broader areas of the DataFrame styling with minimal HTML transfer. Much of the functionality of `Styler` uses individual HTML id tags to manipulate the output, which may be inefficient for very large tables. Using `table_styles` and otherwise avoiding using id tags can greatly reduce the rendered HTML.\n", + "\n", + "Table styles are also used to control features which can apply to the whole table at once such as greating a generic hover functionality:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "s = Styler(df, cell_ids=False, uuid_len=1)\n", + "s.set_table_styles([{'selector': 'tr:hover',\n", + " 'props': [('background-color', 'yellow')]}])\n", + "s.set_table_styles({\n", + " 'A': [{'selector': '',\n", + " 'props': [('color', 'red')]}],\n", + " 'B': [{'selector': 'td',\n", + " 'props': [('color', 'blue')]}]\n", + "}, axis=0, overwrite=False)\n", + "s.set_table_styles({\n", + " 3: [{'selector': 'td',\n", + " 'props': [('color', 'green')]}]\n", + "}, axis=1, overwrite=False)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The **advantage** of table styles is obviously the reduced HTML that it can create and the relative ease with which more general parts of the table can be quickly styled, e.g. by applying a generic hover, rather than having to apply a hover to each cell individually. Rows and columns as individual objects can only be styled in this way.\n", + "\n", + "The **disadvantage** of being restricted solely to table styles is that you have very limited ability to target and style individual cells based on dynamic criteria. For this one must use either of the other two methods. Also table level styles cannot be exported to Excel: to format cells for Excel output you must use the Styler Functions method below." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Styler Functions\n", + "\n", + "Thirdly we can use the method to pass your style functions into one of the following methods:\n", "\n", - "When writing style functions, you take care of producing the CSS attribute / value pairs you want. Pandas matches those up with the CSS classes that identify each cell." + "- ``Styler.applymap``: elementwise\n", + "- ``Styler.apply``: column-/row-/table-wise\n", + "\n", + "Both of those methods take a function (and some other keyword arguments) and applies your function to the DataFrame in a certain way.\n", + "`Styler.applymap` works through the DataFrame elementwise.\n", + "`Styler.apply` passes each column or row into your DataFrame one-at-a-time or the entire table at once, depending on the `axis` keyword argument.\n", + "For columnwise use `axis=0`, rowwise use `axis=1`, and for the entire table at once use `axis=None`.\n", + "\n", + "For `Styler.applymap` your function should take a scalar and return a single string with the CSS attribute-value pair.\n", + "\n", + "For `Styler.apply` your function should take a Series or DataFrame (depending on the axis parameter), and return a Series or DataFrame with an identical shape where each value is a string with a CSS attribute-value pair.\n", + "\n", + "The **advantage** of this method is that there is full granular control and the output is isolated and easily transferrable, especially in Jupyter Notebooks.\n", + "\n", + "The **disadvantage** is that the HTML/CSS required to produce this needs to be directly generated from the Python code and it can lead to inefficient data transfer for large tables.\n", + "\n", + "Let's see some examples." ] }, { @@ -210,7 +319,34 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We encourage you to use method chains to build up a style piecewise, before finally rending at the end of the chain." + "A common use case is also to highlight values based on comparison between columns. Suppose we wish to highlight those cells in columns 'B' and 'C' which are lower than respective values in 'E' then we can write a comparator function. (You can read a little more below in 'Finer Control: Slicing')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def compare_col(s, comparator=None):\n", + " attr = 'background-color: #00BFFF;'\n", + " return np.where(s < comparator, attr, '')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "df.style.apply(compare_col, subset=['B', 'C'], comparator=df['E'])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We encourage you to use method chains to build up a style piecewise, before finally rending at the end of the chain. Note the ordering of application will affect styles that overlap." ] }, { @@ -220,6 +356,7 @@ "outputs": [], "source": [ "df.style.\\\n", + " apply(compare_col, subset=['B', 'C'], comparator=df['E']).\\\n", " applymap(color_negative_red).\\\n", " apply(highlight_max)" ] @@ -271,7 +408,8 @@ "metadata": {}, "outputs": [], "source": [ - "df.style.apply(highlight_max, color='darkorange', axis=None)" + "s = df.style.apply(highlight_max, color='darkorange', axis=None)\n", + "s" ] }, { @@ -290,6 +428,62 @@ "And crucially the input and output shapes of `func` must match. If `x` is the input then ``func(x).shape == x.shape``." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Tooltips\n", + "\n", + "*New in version 1.3.0*\n", + "\n", + "You can now add tooltips in the same way you can add external CSS classes to datacells by providing a string based DataFrame with intersecting indices and columns." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "tt = pd.DataFrame(data=[[None, 'No Data', None], \n", + " [None, None, 'Missing Data'], \n", + " ['Maximum value across entire DataFrame', None, None]], \n", + " index=[0, 3, 9], \n", + " columns=['A', 'C', 'D'])\n", + "s.set_tooltips(tt)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The tooltips are added with a default CSS styling, however, you have full control of the tooltips in the following way. The name of the class can be integrated with your existing website's CSS so you do not need to set any properties within Python if you have the external CSS files. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "s.set_tooltips_class(name='pd-tt', properties=[\n", + " ('visibility', 'hidden'),\n", + " ('position', 'absolute'),\n", + " ('z-index', '1'),\n", + " ('background-color', 'blue'),\n", + " ('color', 'white'),\n", + " ('font-size', '1.5em'),\n", + " ('transform', 'translate(3px, -11px)'),\n", + " ('padding', '0.5em'),\n", + " ('border', '1px solid red'),\n", + " ('border-radius', '0.5em')\n", + "])" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -843,6 +1037,11 @@ { "cell_type": "code", "execution_count": null, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, "outputs": [], "source": [ "html = html.set_table_styles({\n", @@ -850,13 +1049,7 @@ " 'C': [dict(selector='td', props=[('color', 'red')])], \n", " }, overwrite=False)\n", "html" - ], - "metadata": { - "collapsed": false, - "pycharm": { - "name": "#%%\n" - } - } + ] }, { "cell_type": "markdown", @@ -1053,7 +1246,9 @@ "\n", "- Only CSS2 named colors and hex colors of the form `#rgb` or `#rrggbb` are currently supported.\n", "- The following pseudo CSS properties are also available to set excel specific style properties:\n", - " - `number-format`\n" + " - `number-format`\n", + "\n", + "Table level styles are not included in the export to Excel: individual cells must have their properties mapped by the `Styler.apply` and/or `Styler.applymap` methods." ] }, { @@ -1262,7 +1457,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.0" + "version": "3.8.6" } }, "nbformat": 4, diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py index 782562f455607..2776c680ae68c 100644 --- a/pandas/io/formats/style.py +++ b/pandas/io/formats/style.py @@ -511,16 +511,15 @@ def format_attr(pair): for c, col in enumerate(self.data.columns): cs = [DATA_CLASS, f"row{r}", f"col{c}"] - cs.extend(cell_context.get("data", {}).get(r, {}).get(c, [])) formatter = self._display_funcs[(r, c)] value = self.data.iloc[r, c] row_dict = { "type": "td", "value": value, - "class": " ".join(cs), "display_value": formatter(value), "is_visible": (c not in hidden_columns), } + # only add an id if the cell has a style props = [] if self.cell_ids or (r, c) in ctx: @@ -531,6 +530,11 @@ def format_attr(pair): props.append(tuple(x.split(":"))) else: props.append(("", "")) + + # add custom classes from cell context + cs.extend(cell_context.get("data", {}).get(r, {}).get(c, [])) + row_dict["class"] = " ".join(cs) + row_es.append(row_dict) cellstyle_map[tuple(props)].append(f"row{r}_col{c}") body.append(row_es) @@ -690,10 +694,12 @@ def set_td_classes(self, classes: DataFrame) -> Styler: mask = (classes.isna()) | (classes.eq("")) self.cell_context["data"] = { - r: {c: [str(classes.iloc[r, c])]} + r: { + c: [str(classes.iloc[r, c])] + for c, cn in enumerate(classes.columns) + if not mask.iloc[r, c] + } for r, rn in enumerate(classes.index) - for c, cn in enumerate(classes.columns) - if not mask.iloc[r, c] } return self @@ -1914,11 +1920,8 @@ def _translate(self, styler_data: FrameOrSeriesUnion, uuid: str, d: Dict): ------- render_dict : Dict """ - self.tt_data = ( - self.tt_data.reindex_like(styler_data) - .dropna(how="all", axis=0) - .dropna(how="all", axis=1) - ) + self.tt_data = self.tt_data.reindex_like(styler_data) + if self.tt_data.empty: return d From 847ec69f113814eb271bf93996c275b26f9f5188 Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Thu, 21 Jan 2021 21:26:58 +0100 Subject: [PATCH 02/12] add tests --- pandas/tests/io/formats/test_style.py | 41 ++++++++++++++++++++++++++- 1 file changed, 40 insertions(+), 1 deletion(-) diff --git a/pandas/tests/io/formats/test_style.py b/pandas/tests/io/formats/test_style.py index c61d81d565459..51810786395b5 100644 --- a/pandas/tests/io/formats/test_style.py +++ b/pandas/tests/io/formats/test_style.py @@ -1706,11 +1706,34 @@ def test_no_cell_ids(self): def test_set_data_classes(self, classes): # GH 36159 df = DataFrame(data=[[0, 1], [2, 3]], columns=["A", "B"], index=["a", "b"]) - s = Styler(df, uuid="_", cell_ids=False).set_td_classes(classes).render() + s = Styler(df, uuid_len=0, cell_ids=False).set_td_classes(classes).render() assert '0' in s assert '1' in s assert '2' in s assert '3' in s + # GH 39317 + s = Styler(df, uuid_len=0, cell_ids=True).set_td_classes(classes).render() + assert '0' in s + assert '1' in s + assert '2' in s + assert '3' in s + + def test_set_data_classes_reindex(self): + # GH 39317 + df = DataFrame( + data=[[0, 1, 2], [3, 4, 5], [6, 7, 8]], columns=[0, 1, 2], index=[0, 1, 2] + ) + classes = DataFrame( + data=[["mi", "ma"], ["mu", "mo"]], + columns=[0, 2], + index=[0, 2], + ) + s = Styler(df, uuid_len=0).set_td_classes(classes).render() + assert '0' in s + assert '2' in s + assert '4' in s + assert '6' in s + assert '8' in s def test_chaining_table_styles(self): # GH 35607 @@ -1815,6 +1838,22 @@ def test_tooltip_render(self, ttips): in s ) + def test_tooltip_reindex(self): + # GH 39317 + df = DataFrame( + data=[[0, 1, 2], [3, 4, 5], [6, 7, 8]], columns=[0, 1, 2], index=[0, 1, 2] + ) + ttips = DataFrame( + data=[["Mi", "Ma"], ["Mu", "Mo"]], + columns=[0, 2], + index=[0, 2], + ) + s = Styler(df, uuid_len=0).set_tooltips(DataFrame(ttips)).render() + assert '#T__ #T__row0_col0 .pd-t::after {\n content: "Mi";\n }' in s + assert '#T__ #T__row0_col2 .pd-t::after {\n content: "Ma";\n }' in s + assert '#T__ #T__row2_col0 .pd-t::after {\n content: "Mu";\n }' in s + assert '#T__ #T__row2_col2 .pd-t::after {\n content: "Mo";\n }' in s + def test_tooltip_ignored(self): # GH 21266 df = DataFrame(data=[[0, 1], [2, 3]]) From e436a95a9ae4e95943542b49719a4330a60d40b6 Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Fri, 22 Jan 2021 09:53:40 +0100 Subject: [PATCH 03/12] doc fixes --- doc/source/user_guide/style.ipynb | 182 +++++++++++++++--------------- 1 file changed, 93 insertions(+), 89 deletions(-) diff --git a/doc/source/user_guide/style.ipynb b/doc/source/user_guide/style.ipynb index 2793bb1dc58d6..114b4688fffaf 100644 --- a/doc/source/user_guide/style.ipynb +++ b/doc/source/user_guide/style.ipynb @@ -50,7 +50,6 @@ "outputs": [], "source": [ "import pandas as pd\n", - "from pandas.io.formats.style import Styler\n", "import numpy as np\n", "\n", "np.random.seed(24)\n", @@ -161,9 +160,38 @@ "source": [ "### Using Table Styles\n", "\n", - "Table styles allow you to control broader areas of the DataFrame styling with minimal HTML transfer. Much of the functionality of `Styler` uses individual HTML id tags to manipulate the output, which may be inefficient for very large tables. Using `table_styles` and otherwise avoiding using id tags can greatly reduce the rendered HTML.\n", + "Table styles allow you to control broader areas of the DataFrame, i.e. the whole table or specific columns or rows, with minimal HTML transfer. Much of the functionality of `Styler` uses individual HTML id tags to manipulate the output, which may be inefficient for very large tables. Using `table_styles` and otherwise avoiding using id tags in data cells can greatly reduce the rendered HTML.\n", + "\n", + "Table styles are also used to control features which can apply to the whole table at once such as greating a generic hover functionality. This `:hover` pseudo-selectors, as well as others, can only be used this way.\n", + "\n", + "`table_styles` are extremely flexible, but not as fun to type out by hand.\n", + "We hope to collect some useful ones either in pandas, or preferable in a new package that [builds on top](#Extensibility) the tools here." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def hover(hover_color=\"#ffff99\"):\n", + " return {'selector': \"tr:hover\",\n", + " 'props': [(\"background-color\", \"%s\" % hover_color)]}\n", + "\n", + "styles = [\n", + " hover(),\n", + " {'selector': \"th\", 'props': [(\"font-size\", \"150%\"),\n", + " (\"text-align\", \"center\")]}\n", + "]\n", "\n", - "Table styles are also used to control features which can apply to the whole table at once such as greating a generic hover functionality:" + "df.style.set_table_styles(styles)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If `table_styles` is given as a dictionary each key should be a specified column or index value and this will map to specific class CSS selectors of the given column or row." ] }, { @@ -172,19 +200,69 @@ "metadata": {}, "outputs": [], "source": [ - "s = Styler(df, cell_ids=False, uuid_len=1)\n", - "s.set_table_styles([{'selector': 'tr:hover',\n", - " 'props': [('background-color', 'yellow')]}])\n", - "s.set_table_styles({\n", + "df.style.set_table_styles({\n", " 'A': [{'selector': '',\n", " 'props': [('color', 'red')]}],\n", " 'B': [{'selector': 'td',\n", " 'props': [('color', 'blue')]}]\n", - "}, axis=0, overwrite=False)\n", - "s.set_table_styles({\n", + "}, axis=0)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "df.style.set_table_styles({\n", " 3: [{'selector': 'td',\n", " 'props': [('color', 'green')]}]\n", - "}, axis=1, overwrite=False)" + "}, axis=1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also chain all of the above by setting the `overwrite` argument to `False` so that it preserves previous settings." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from pandas.io.formats.style import Styler\n", + "s = Styler(df, cell_ids=False, uuid_len=0).\\\n", + " set_table_styles(styles).\\\n", + " set_table_styles({\n", + " 'A': [{'selector': '',\n", + " 'props': [('color', 'red')]}],\n", + " 'B': [{'selector': 'td',\n", + " 'props': [('color', 'blue')]}]\n", + " }, axis=0, overwrite=False).\\\n", + " set_table_styles({\n", + " 3: [{'selector': 'td',\n", + " 'props': [('color', 'green')]}]\n", + " }, axis=1, overwrite=False)\n", + "s" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "By using these `table_styles` and the additional `Styler` arguments to optimize the HTML we have compressed these styles to only a few lines withing the \\ tags and none of the \\ cells require any `id` attributes. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "s.render().split('\\n')[:16]" ] }, { @@ -193,7 +271,7 @@ "source": [ "The **advantage** of table styles is obviously the reduced HTML that it can create and the relative ease with which more general parts of the table can be quickly styled, e.g. by applying a generic hover, rather than having to apply a hover to each cell individually. Rows and columns as individual objects can only be styled in this way.\n", "\n", - "The **disadvantage** of being restricted solely to table styles is that you have very limited ability to target and style individual cells based on dynamic criteria. For this one must use either of the other two methods. Also table level styles cannot be exported to Excel: to format cells for Excel output you must use the Styler Functions method below." + "The **disadvantage** of being restricted solely to table styles is that you have very limited ability to target and style individual cells based on dynamic criteria. For this, one must use either of the other two methods. Also table level styles cannot be exported to Excel: to format cells for Excel output you must use the Styler Functions method below." ] }, { @@ -961,7 +1039,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Regular table captions can be added in a few ways." + "Regular table captions can be added and, if necessary, controlled with CSS." ] }, { @@ -971,86 +1049,12 @@ "outputs": [], "source": [ "df.style.set_caption('Colormaps, with a caption.')\\\n", + " .set_table_styles([{\n", + " 'selector': \"caption\", 'props': [(\"caption-side\", \"bottom\")]\n", + " }])\\\n", " .background_gradient(cmap=cm)" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Table styles" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The next option you have are \"table styles\".\n", - "These are styles that apply to the table as a whole, but don't look at the data.\n", - "Certain stylings, including pseudo-selectors like `:hover` can only be used this way.\n", - "These can also be used to set specific row or column based class selectors, as will be shown." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from IPython.display import HTML\n", - "\n", - "def hover(hover_color=\"#ffff99\"):\n", - " return dict(selector=\"tr:hover\",\n", - " props=[(\"background-color\", \"%s\" % hover_color)])\n", - "\n", - "styles = [\n", - " hover(),\n", - " dict(selector=\"th\", props=[(\"font-size\", \"150%\"),\n", - " (\"text-align\", \"center\")]),\n", - " dict(selector=\"caption\", props=[(\"caption-side\", \"bottom\")])\n", - "]\n", - "html = (df.style.set_table_styles(styles)\n", - " .set_caption(\"Hover to highlight.\"))\n", - "html" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "`table_styles` should be a list of dictionaries.\n", - "Each dictionary should have the `selector` and `props` keys.\n", - "The value for `selector` should be a valid CSS selector.\n", - "Recall that all the styles are already attached to an `id`, unique to\n", - "each `Styler`. This selector is in addition to that `id`.\n", - "The value for `props` should be a list of tuples of `('attribute', 'value')`.\n", - "\n", - "`table_styles` are extremely flexible, but not as fun to type out by hand.\n", - "We hope to collect some useful ones either in pandas, or preferable in a new package that [builds on top](#Extensibility) the tools here.\n", - "\n", - "`table_styles` can be used to add column and row based class descriptors. For large tables this can increase performance by avoiding repetitive individual css for each cell, and it can also simplify style construction in some cases.\n", - "If `table_styles` is given as a dictionary each key should be a specified column or index value and this will map to specific class CSS selectors of the given column or row.\n", - "\n", - "Note that `Styler.set_table_styles` will overwrite existing styles but can be chained by setting the `overwrite` argument to `False`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "pycharm": { - "name": "#%%\n" - } - }, - "outputs": [], - "source": [ - "html = html.set_table_styles({\n", - " 'B': [dict(selector='', props=[('color', 'green')])],\n", - " 'C': [dict(selector='td', props=[('color', 'red')])], \n", - " }, overwrite=False)\n", - "html" - ] - }, { "cell_type": "markdown", "metadata": {}, From 48790941b6e18f3202e5b6b88e891e0459b008db Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Sat, 23 Jan 2021 15:27:05 +0100 Subject: [PATCH 04/12] Styler.apply permits ndarray with axis=None --- doc/source/user_guide/style.ipynb | 63 +++++++-------------------- pandas/io/formats/style.py | 15 +++++-- pandas/tests/io/formats/test_style.py | 5 ++- 3 files changed, 30 insertions(+), 53 deletions(-) diff --git a/doc/source/user_guide/style.ipynb b/doc/source/user_guide/style.ipynb index 114b4688fffaf..f9197e0789499 100644 --- a/doc/source/user_guide/style.ipynb +++ b/doc/source/user_guide/style.ipynb @@ -140,9 +140,10 @@ "metadata": {}, "outputs": [], "source": [ - "s = df.style.set_table_attributes('class=\"table-cls\"')\n", - "cls = pd.DataFrame(data=[['cls1', None], ['cls3', 'cls2 cls3']], index=[0,2], columns=['A', 'C'])\n", - "s.set_td_classes(cls)" + "css_classes = pd.DataFrame(data=[['cls1', None], ['cls3', 'cls2 cls3']], index=[0,2], columns=['A', 'C'])\n", + "df.style.\\\n", + " set_table_attributes('class=\"table-cls\"').\\\n", + " set_td_classes(css_classes)" ] }, { @@ -315,13 +316,10 @@ "outputs": [], "source": [ "def color_negative_red(val):\n", - " \"\"\"\n", - " Takes a scalar and returns a string with\n", - " the css property `'color: red'` for negative\n", - " strings, black otherwise.\n", - " \"\"\"\n", - " color = 'red' if val < 0 else 'black'\n", - " return 'color: %s' % color" + " \"\"\"Color negative scalars red.\"\"\"\n", + " css = 'color: red;'\n", + " if val < 0: return css\n", + " return None" ] }, { @@ -369,11 +367,9 @@ "outputs": [], "source": [ "def highlight_max(s):\n", - " '''\n", - " highlight the maximum in a Series yellow.\n", - " '''\n", - " is_max = s == s.max()\n", - " return ['background-color: yellow' if v else '' for v in is_max]" + " \"\"\"Highlight the maximum in a Series bold-orange.\"\"\"\n", + " css = 'background-color: orange; font-weight: bold;'\n", + " return np.where(s == np.nanmax(s.values), css, None)" ] }, { @@ -407,8 +403,8 @@ "outputs": [], "source": [ "def compare_col(s, comparator=None):\n", - " attr = 'background-color: #00BFFF;'\n", - " return np.where(s < comparator, attr, '')" + " css = 'background-color: #00BFFF;'\n", + " return np.where(s < comparator, css, None)" ] }, { @@ -448,36 +444,7 @@ "*Debugging Tip*: If you're having trouble writing your style function, try just passing it into DataFrame.apply. Internally, Styler.apply uses DataFrame.apply so the result should be the same.\n", "\n", "What if you wanted to highlight just the maximum value in the entire table?\n", - "Use `.apply(function, axis=None)` to indicate that your function wants the entire table, not one column or row at a time. Let's try that next.\n", - "\n", - "We'll rewrite our `highlight-max` to handle either Series (from `.apply(axis=0 or 1)`) or DataFrames (from `.apply(axis=None)`). We'll also allow the color to be adjustable, to demonstrate that `.apply`, and `.applymap` pass along keyword arguments." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "def highlight_max(data, color='yellow'):\n", - " '''\n", - " highlight the maximum in a Series or DataFrame\n", - " '''\n", - " attr = 'background-color: {}'.format(color)\n", - " if data.ndim == 1: # Series from .apply(axis=0) or axis=1\n", - " is_max = data == data.max()\n", - " return [attr if v else '' for v in is_max]\n", - " else: # from .apply(axis=None)\n", - " is_max = data == data.max().max()\n", - " return pd.DataFrame(np.where(is_max, attr, ''),\n", - " index=data.index, columns=data.columns)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "When using ``Styler.apply(func, axis=None)``, the function must return a DataFrame with the same index and column labels." + "Use `.apply(function, axis=None)` to indicate that your function wants the entire table, not one column or row at a time. Let's try that next." ] }, { @@ -486,7 +453,7 @@ "metadata": {}, "outputs": [], "source": [ - "s = df.style.apply(highlight_max, color='darkorange', axis=None)\n", + "s = df.style.apply(highlight_max, axis=None)\n", "s" ] }, diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py index 2776c680ae68c..ba486ddbf5bc8 100644 --- a/pandas/io/formats/style.py +++ b/pandas/io/formats/style.py @@ -848,10 +848,17 @@ def _apply( else: result = func(data, **kwargs) if not isinstance(result, pd.DataFrame): - raise TypeError( - f"Function {repr(func)} must return a DataFrame when " - f"passed to `Styler.apply` with axis=None" - ) + if not isinstance(result, np.ndarray): + raise TypeError( + f"Function {repr(func)} must return a DataFrame or ndarray when" + f" passed to `Styler.apply` with axis=None" + ) + if not (data.shape == result.shape): + raise TypeError( + f"Function {repr(func)} returned ndarray with shape " + f"{result.shape} but {data.shape} was expected" + ) + result = DataFrame(result, index=data.index, columns=data.columns) if not ( result.index.equals(data.index) and result.columns.equals(data.columns) ): diff --git a/pandas/tests/io/formats/test_style.py b/pandas/tests/io/formats/test_style.py index 51810786395b5..428861217d635 100644 --- a/pandas/tests/io/formats/test_style.py +++ b/pandas/tests/io/formats/test_style.py @@ -1357,7 +1357,10 @@ def f(x): return "" df = DataFrame([[1, 2], [3, 4]]) - msg = "must return a DataFrame when passed to `Styler.apply` with axis=None" + msg = ( + "must return a DataFrame or ndarray when passed to `Styler.apply` " + "with axis=None" + ) with pytest.raises(TypeError, match=msg): df.style._apply(f, axis=None) From 1aa097e14b3707e567c28628154873fd7a8a3947 Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Sat, 23 Jan 2021 15:43:57 +0100 Subject: [PATCH 05/12] apply with row --- doc/source/user_guide/style.ipynb | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/doc/source/user_guide/style.ipynb b/doc/source/user_guide/style.ipynb index f9197e0789499..ec251d22f2d28 100644 --- a/doc/source/user_guide/style.ipynb +++ b/doc/source/user_guide/style.ipynb @@ -381,11 +381,20 @@ "df.style.apply(highlight_max)" ] }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "df.style.apply(highlight_max, axis=1)" + ] + }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In this case the input is a `Series`, one column at a time.\n", + "In this case the input is a `Series`, one column (or row) at a time.\n", "Notice that the output shape of `highlight_max` matches the input shape, an array with `len(s)` items." ] }, @@ -439,12 +448,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Above we used `Styler.apply` to pass in each column one at a time.\n", + "Above we used `Styler.apply` to pass in each column (or row) one at a time.\n", "\n", "*Debugging Tip*: If you're having trouble writing your style function, try just passing it into DataFrame.apply. Internally, Styler.apply uses DataFrame.apply so the result should be the same.\n", "\n", "What if you wanted to highlight just the maximum value in the entire table?\n", - "Use `.apply(function, axis=None)` to indicate that your function wants the entire table, not one column or row at a time. Let's try that next." + "Use `.apply(function, axis=None)` to indicate that your function wants the entire table, not one column or row at a time. In this case the return must be a DataFrame or ndarray of the same shape as the input. Let's try that next. " ] }, { From 228f619a5833c749eb01957ac7c941c07c810772 Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Sun, 24 Jan 2021 09:32:24 +0100 Subject: [PATCH 06/12] doc updates --- pandas/io/formats/style.py | 34 ++++++++++++++++++++++++---------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py index ba486ddbf5bc8..3120988625768 100644 --- a/pandas/io/formats/style.py +++ b/pandas/io/formats/style.py @@ -886,7 +886,7 @@ def apply( **kwargs, ) -> Styler: """ - Apply a function column-wise, row-wise, or table-wise. + Apply a CSS-styling function column-wise, row-wise, or table-wise. Updates the HTML representation with the result. @@ -896,7 +896,7 @@ def apply( ``func`` should take a Series or DataFrame (depending on ``axis``), and return an object with the same shape. Must return a DataFrame with identical index and - column labels when ``axis=None``. + column labels or an ndarray of appropriate shape when ``axis=None``. axis : {0 or 'index', 1 or 'columns', None}, default 0 Apply to each column (``axis=0`` or ``'index'``), to each row (``axis=1`` or ``'columns'``), or to the entire DataFrame at once @@ -913,9 +913,11 @@ def apply( Notes ----- - The output shape of ``func`` should match the input, i.e. if + The output of ``func`` should be elements having CSS style as string or, + if nothing is to be applied to that element, an empty string or ``None``. + The output shape must match the input, i.e. if ``x`` is the input row, column, or table (depending on ``axis``), - then ``func(x).shape == x.shape`` should be true. + then ``func(x).shape == x.shape`` should be ``True``. This is similar to ``DataFrame.apply``, except that ``axis=None`` applies the function to the entire DataFrame at once, @@ -923,12 +925,12 @@ def apply( Examples -------- - >>> def highlight_max(x): - ... return ['background-color: yellow' if v == x.max() else '' - for v in x] - ... + >>> def highlight_max(x, color): + ... return np.where(x == np.nanmax(x.values), f"color: {color};", None) >>> df = pd.DataFrame(np.random.randn(5, 2)) - >>> df.style.apply(highlight_max) + >>> df.style.apply(highlight_max, color='red') + >>> df.style.apply(highlight_max, color='blue', axis=1) + >>> df.style.apply(highlight_max, color='green', axis=None) """ self._todo.append( (lambda instance: getattr(instance, "_apply"), (func, axis, subset), kwargs) @@ -946,7 +948,7 @@ def _applymap(self, func: Callable, subset=None, **kwargs) -> Styler: def applymap(self, func: Callable, subset=None, **kwargs) -> Styler: """ - Apply a function elementwise. + Apply a CSS-styling function elementwise. Updates the HTML representation with the result. @@ -964,10 +966,22 @@ def applymap(self, func: Callable, subset=None, **kwargs) -> Styler: ------- self : Styler + Notes + ----- + The output of ``func`` should be a CSS style as string or, if nothing is to be + applied, an empty string or ``None``. + See Also -------- Styler.where: Updates the HTML representation with a style which is selected in accordance with the return value of a function. + + Examples + -------- + >>> def color_negative(v, color): + ... return f"color: {color};" if v < 0 else None + >>> df = pd.DataFrame(np.random.randn(5, 2)) + >>> df.style.applymap(color_negative, color='red') """ self._todo.append( (lambda instance: getattr(instance, "_applymap"), (func, subset), kwargs) From 259ca047688da19e2553254f65238963521c50c0 Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Mon, 25 Jan 2021 08:29:20 +0100 Subject: [PATCH 07/12] tests --- pandas/io/formats/style.py | 13 ++++++------- pandas/tests/io/formats/test_style.py | 4 ++++ 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py index 22df17299c69c..e7ab810ff1361 100644 --- a/pandas/io/formats/style.py +++ b/pandas/io/formats/style.py @@ -854,9 +854,10 @@ def _apply( f" passed to `Styler.apply` with axis=None" ) if not (data.shape == result.shape): - raise TypeError( - f"Function {repr(func)} returned ndarray with shape " - f"{result.shape} but {data.shape} was expected" + raise ValueError( + f"Function {repr(func)} returned ndarray with wrong shape.\n" + f"Result has shape: {result.shape}\n" + f"Expected shape: {data.shape}" ) result = DataFrame(result, index=data.index, columns=data.columns) if not ( @@ -867,13 +868,11 @@ def _apply( f"index and columns as the input" ) - result_shape = result.shape - expected_shape = self.data.loc[subset].shape - if result_shape != expected_shape: + if result.shape != data.shape: raise ValueError( f"Function {repr(func)} returned the wrong shape.\n" f"Result has shape: {result.shape}\n" - f"Expected shape: {expected_shape}" + f"Expected shape: {data.shape}" ) self._update_ctx(result) return self diff --git a/pandas/tests/io/formats/test_style.py b/pandas/tests/io/formats/test_style.py index 428861217d635..d4e1e3a036cee 100644 --- a/pandas/tests/io/formats/test_style.py +++ b/pandas/tests/io/formats/test_style.py @@ -1352,6 +1352,10 @@ def test_bad_apply_shape(self): with pytest.raises(ValueError, match=msg): df.style._apply(lambda x: ["", "", ""], axis=1) + msg = "returned ndarray with wrong shape" + with pytest.raises(ValueError, match=msg): + df.style._apply(lambda x: np.array([[""], [""]]), axis=None) + def test_apply_bad_return(self): def f(x): return "" From ea6ad925534b1c90510e214e53d83e1560d40ba4 Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Mon, 25 Jan 2021 09:44:23 +0100 Subject: [PATCH 08/12] whats new --- doc/source/whatsnew/v1.3.0.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/source/whatsnew/v1.3.0.rst b/doc/source/whatsnew/v1.3.0.rst index 67f27a75c7071..610e88db86a93 100644 --- a/doc/source/whatsnew/v1.3.0.rst +++ b/doc/source/whatsnew/v1.3.0.rst @@ -53,6 +53,7 @@ Other enhancements - :meth:`DataFrame.apply` can now accept non-callable DataFrame properties as strings, e.g. ``df.apply("size")``, which was already the case for :meth:`Series.apply` (:issue:`39116`) - :meth:`Series.apply` can now accept list-like or dictionary-like arguments that aren't lists or dictionaries, e.g. ``ser.apply(np.array(["sum", "mean"]))``, which was already the case for :meth:`DataFrame.apply` (:issue:`39140`) - :meth:`.Styler.set_tooltips` allows on hover tooltips to be added to styled HTML dataframes. +- :meth:`.Styler.apply` now more consistently accepts ndarray function returns, i.e. in all cases for ``axis`` is ``0, 1 or None``. .. --------------------------------------------------------------------------- From 09d6302ffb03c870800863c24d92623752682794 Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Mon, 25 Jan 2021 09:50:10 +0100 Subject: [PATCH 09/12] op --- pandas/io/formats/style.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py index e7ab810ff1361..3a21984a9cca7 100644 --- a/pandas/io/formats/style.py +++ b/pandas/io/formats/style.py @@ -860,7 +860,7 @@ def _apply( f"Expected shape: {data.shape}" ) result = DataFrame(result, index=data.index, columns=data.columns) - if not ( + elif not ( result.index.equals(data.index) and result.columns.equals(data.columns) ): raise ValueError( From b9715c9e4d06422f5fa6c6bc105e72f9b570d2f1 Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Mon, 25 Jan 2021 10:56:28 +0100 Subject: [PATCH 10/12] doc fix --- pandas/io/formats/style.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py index 3a21984a9cca7..9c16ebf9bf47b 100644 --- a/pandas/io/formats/style.py +++ b/pandas/io/formats/style.py @@ -965,16 +965,16 @@ def applymap(self, func: Callable, subset=None, **kwargs) -> Styler: ------- self : Styler - Notes - ----- - The output of ``func`` should be a CSS style as string or, if nothing is to be - applied, an empty string or ``None``. - See Also -------- Styler.where: Updates the HTML representation with a style which is selected in accordance with the return value of a function. + Notes + ----- + The output of ``func`` should be a CSS style as string or, if nothing is to be + applied, an empty string or ``None``. + Examples -------- >>> def color_negative(v, color): From c6173b5d9ecff3cda5d0d18f4f6d1fc2a8cb8c6e Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Mon, 25 Jan 2021 13:08:20 +0100 Subject: [PATCH 11/12] text fix --- pandas/io/formats/style.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py index 9c16ebf9bf47b..b44bc40b4ccd8 100644 --- a/pandas/io/formats/style.py +++ b/pandas/io/formats/style.py @@ -850,8 +850,8 @@ def _apply( if not isinstance(result, pd.DataFrame): if not isinstance(result, np.ndarray): raise TypeError( - f"Function {repr(func)} must return a DataFrame or ndarray when" - f" passed to `Styler.apply` with axis=None" + f"Function {repr(func)} must return a DataFrame or ndarray " + f"when passed to `Styler.apply` with axis=None" ) if not (data.shape == result.shape): raise ValueError( From a463a53f365f4d61ed6af3ad567409bac22aad4c Mon Sep 17 00:00:00 2001 From: "JHM Darbyshire (iMac)" Date: Mon, 8 Feb 2021 07:45:32 +0100 Subject: [PATCH 12/12] docs --- pandas/io/formats/style.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/pandas/io/formats/style.py b/pandas/io/formats/style.py index e0ea4e722f8bb..8db067cbc4e8f 100644 --- a/pandas/io/formats/style.py +++ b/pandas/io/formats/style.py @@ -893,7 +893,10 @@ def apply( ``func`` should take a Series or DataFrame (depending on ``axis``), and return an object with the same shape. Must return a DataFrame with identical index and - column labels or an ndarray of appropriate shape when ``axis=None``. + column labels or an ndarray with same shape as input when ``axis=None``. + + .. versionchanged:: 1.3.0 + axis : {0 or 'index', 1 or 'columns', None}, default 0 Apply to each column (``axis=0`` or ``'index'``), to each row (``axis=1`` or ``'columns'``), or to the entire DataFrame at once