Merge branch 'master' of https://github.com/pandas-dev/pandas into doc-decorator

Author: HH-MWB

.github/CONTRIBUTING.md

@@ -2,7 +2,7 @@
 
 Whether you are a novice or experienced software developer, all contributions and suggestions are welcome!
 
-Our main contributing guide can be found [in this repo](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst) or [on the website](https://pandas-docs.github.io/pandas-docs-travis/development/contributing.html). If you do not want to read it in its entirety, we will summarize the main ways in which you can contribute and point to relevant sections of that document for further information.
+Our main contributing guide can be found [in this repo](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst) or [on the website](https://pandas.pydata.org/docs/dev/development/contributing.html). If you do not want to read it in its entirety, we will summarize the main ways in which you can contribute and point to relevant sections of that document for further information.
 
 ## Getting Started
 

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,39 @@
+---
+
+name: Bug Report
+about: Create a bug report to help us improve pandas
+title: "BUG:"
+labels: "Bug, Needs Triage"
+
+---
+
+- [ ] I have checked that this issue has not already been reported.
+
+- [ ] I have confirmed this bug exists on the latest version of pandas.
+
+- [ ] (optional) I have confirmed this bug exists on the master branch of pandas.
+
+---
+
+**Note**: Please read [this guide](https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports) detailing how to provide the necessary information for us to reproduce your bug.
+
+#### Code Sample, a copy-pastable example
+
+```python
+# Your code here
+
+```
+
+#### Problem description
+
+[this should explain **why** the current behaviour is a problem and why the expected output is a better solution]
+
+#### Expected Output
+
+#### Output of ``pd.show_versions()``
+
+<details>
+
+[paste the output of ``pd.show_versions()`` here leaving a blank line after the details tag]
+
+</details>

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,22 @@
+---
+
+name: Documentation Improvement
+about: Report wrong or missing documentation
+title: "DOC:"
+labels: "Docs, Needs Triage"
+
+---
+
+#### Location of the documentation
+
+[this should provide the location of the documentation, e.g. "pandas.read_csv" or the URL of the documentation, e.g. "https://dev.pandas.io/docs/reference/api/pandas.read_csv.html"]
+
+**Note**: You can check the latest versions of the docs on `master` [here](https://pandas.pydata.org/docs/dev/).
+
+#### Documentation problem
+
+[this should provide a description of what documentation you believe needs to be fixed/improved]
+
+#### Suggested fix for documentation
+
+[this should explain the suggested fix and **why** it's better than the existing documentation]

.github/ISSUE_TEMPLATE/documentation_improvement.md

@@ -0,0 +1,33 @@
+---
+
+name: Feature Request
+about: Suggest an idea for pandas
+title: "ENH:"
+labels: "Enhancement, Needs Triage"
+
+---
+
+#### Is your feature request related to a problem?
+
+[this should provide a description of what the problem is, e.g. "I wish I could use pandas to do [...]"]
+
+#### Describe the solution you'd like
+
+[this should provide a description of the feature request, e.g. "`DataFrame.foo` should get a new parameter `bar` that [...]", try to write a docstring for the desired feature]
+
+#### API breaking implications
+
+[this should provide a description of how this feature will affect the API]
+
+#### Describe alternatives you've considered
+
+[this should provide a description of any alternative solutions or features you've considered]
+
+#### Additional context
+
+[add any other context, code examples, or references to existing implementations about the feature request here]
+
+```python
+# Your code here, if applicable
+
+```

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,24 @@
+---
+
+name: Submit Question
+about: Ask a general question about pandas
+title: "QST:"
+labels: "Usage Question, Needs Triage"
+
+---
+
+- [ ] I have searched the [[pandas] tag](https://stackoverflow.com/questions/tagged/pandas) on StackOverflow for similar questions.
+
+- [ ] I have asked my usage related question on [StackOverflow](https://stackoverflow.com).
+
+---
+
+#### Question about pandas
+
+**Note**: If you'd still like to submit a question, please read [this guide](
+https://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports) detailing how to provide the necessary information for us to reproduce your question.
+
+```python
+# Your code here, if applicable
+
+```

.github/ISSUE_TEMPLATE/submit_question.md

@@ -125,32 +125,18 @@ jobs:
     - name: Check ipython directive errors
       run: "! grep -B1 \"^<<<-------------------------------------------------------------------------$\" sphinx.log"
 
-    - name: Install Rclone
-      run: sudo apt install rclone -y
-      if: github.event_name == 'push'
-
-    - name: Set up Rclone
+    - name: Install ssh key
       run: |
-        CONF=$HOME/.config/rclone/rclone.conf
-        mkdir -p `dirname $CONF`
-        echo "[ovh_host]" > $CONF
-        echo "type = swift" >> $CONF
-        echo "env_auth = false" >> $CONF
-        echo "auth_version = 3" >> $CONF
-        echo "auth = https://auth.cloud.ovh.net/v3/" >> $CONF
-        echo "endpoint_type = public" >> $CONF
-        echo "tenant_domain = default" >> $CONF
-        echo "tenant = 2977553886518025" >> $CONF
-        echo "domain = default" >> $CONF
-        echo "user = w4KGs3pmDxpd" >> $CONF
-        echo "key = ${{ secrets.ovh_object_store_key }}" >> $CONF
-        echo "region = BHS" >> $CONF
+        mkdir -m 700 -p ~/.ssh
+        echo "${{ secrets.server_ssh_key }}" > ~/.ssh/id_rsa
+        chmod 600 ~/.ssh/id_rsa
+        echo "${{ secrets.server_ip }} ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE1Kkopomm7FHG5enATf7SgnpICZ4W2bw+Ho+afqin+w7sMcrsa0je7sbztFAV8YchDkiBKnWTG4cRT+KZgZCaY=" > ~/.ssh/known_hosts
       if: github.event_name == 'push'
 
-    - name: Sync web with OVH
-      run: rclone sync --exclude pandas-docs/** web/build ovh_host:prod
+    - name: Upload web
+      run: rsync -az --delete --exclude='pandas-docs' --exclude='docs' --exclude='Pandas_Cheat_Sheet*' web/build/ docs@${{ secrets.server_ip }}:/usr/share/nginx/pandas
       if: github.event_name == 'push'
 
-    - name: Sync dev docs with OVH
-      run: rclone sync doc/build/html ovh_host:prod/pandas-docs/dev
+    - name: Upload dev docs
+      run: rsync -az --delete doc/build/html/ docs@${{ secrets.server_ip }}:/usr/share/nginx/pandas/pandas-docs/dev
       if: github.event_name == 'push'

.github/workflows/ci.yml

@@ -7,6 +7,7 @@
 # pandas: powerful Python data analysis toolkit
 [![PyPI Latest Release](https://img.shields.io/pypi/v/pandas.svg)](https://pypi.org/project/pandas/)
 [![Conda Latest Release](https://anaconda.org/conda-forge/pandas/badges/version.svg)](https://anaconda.org/anaconda/pandas/)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3509134.svg)](https://doi.org/10.5281/zenodo.3509134)
 [![Package Status](https://img.shields.io/pypi/status/pandas.svg)](https://pypi.org/project/pandas/)
 [![License](https://img.shields.io/pypi/l/pandas.svg)](https://github.com/pandas-dev/pandas/blob/master/LICENSE)
 [![Travis Build Status](https://travis-ci.org/pandas-dev/pandas.svg?branch=master)](https://travis-ci.org/pandas-dev/pandas)
@@ -158,7 +159,7 @@ Most development discussion is taking place on github in this repo. Further, the
 
 All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome.
 
-A detailed overview on how to contribute can be found in the **[contributing guide](https://dev.pandas.io/docs/contributing.html)**. There is also an [overview](.github/CONTRIBUTING.md) on GitHub.
+A detailed overview on how to contribute can be found in the **[contributing guide](https://pandas.pydata.org/docs/dev/development/contributing.html)**. There is also an [overview](.github/CONTRIBUTING.md) on GitHub.
 
 If you are simply looking to start working with the pandas codebase, navigate to the [GitHub "issues" tab](https://github.com/pandas-dev/pandas/issues) and start looking through interesting issues. There are a number of issues listed under [Docs](https://github.com/pandas-dev/pandas/issues?labels=Docs&sort=updated&state=open) and [good first issue](https://github.com/pandas-dev/pandas/issues?labels=good+first+issue&sort=updated&state=open) where you could start out.
 

README.md

@@ -39,7 +39,7 @@
     // followed by the pip installed packages).
     "matrix": {
         "numpy": [],
-        "Cython": [],
+        "Cython": ["0.29.16"],
         "matplotlib": [],
         "sqlalchemy": [],
         "scipy": [],

asv_bench/asv.conf.json

@@ -50,6 +50,23 @@ def time_frame_op_with_scalar(self, dtype, scalar, op):
         op(self.df, scalar)
 
 
+class OpWithFillValue:
+    def setup(self):
+        # GH#31300
+        arr = np.arange(10 ** 6)
+        df = DataFrame({"A": arr})
+        ser = df["A"]
+
+        self.df = df
+        self.ser = ser
+
+    def time_frame_op_with_fill_value_no_nas(self):
+        self.df.add(self.df, fill_value=4)
+
+    def time_series_op_with_fill_value_no_nas(self):
+        self.ser.add(self.ser, fill_value=4)
+
+
 class MixedFrameWithSeriesAxis0:
     params = [
         [

asv_bench/benchmarks/arithmetic.py

@@ -9,6 +9,11 @@ def setup(self):
         self.values_float = np.array([1.0, 0.0, 1.0, 0.0])
         self.values_integer = np.array([1, 0, 1, 0])
         self.values_integer_like = [1, 0, 1, 0]
+        self.data = np.array([True, False, True, False])
+        self.mask = np.array([False, False, True, False])
+
+    def time_constructor(self):
+        pd.arrays.BooleanArray(self.data, self.mask)
 
     def time_from_bool_array(self):
         pd.array(self.values_bool, dtype="boolean")
@@ -21,3 +26,16 @@ def time_from_integer_like(self):
 
     def time_from_float_array(self):
         pd.array(self.values_float, dtype="boolean")
+
+
+class IntegerArray:
+    def setup(self):
+        self.values_integer = np.array([1, 0, 1, 0])
+        self.data = np.array([1, 2, 3, 4], dtype="int64")
+        self.mask = np.array([False, False, True, False])
+
+    def time_constructor(self):
+        pd.arrays.IntegerArray(self.data, self.mask)
+
+    def time_from_integer_array(self):
+        pd.array(self.values_integer, dtype="Int64")

asv_bench/benchmarks/array.py

@@ -0,0 +1,16 @@
+import pandas as pd
+
+
+class Finalize:
+    param_names = ["series", "frame"]
+    params = [pd.Series, pd.DataFrame]
+
+    def setup(self, param):
+        N = 1000
+        obj = param(dtype=float)
+        for i in range(N):
+            obj.attrs[i] = i
+        self.obj = obj
+
+    def time_finalize_micro(self, param):
+        self.obj.__finalize__(self.obj, method="__finalize__")

asv_bench/benchmarks/finalize.py

@@ -626,4 +626,38 @@ def time_first(self):
         self.df_nans.groupby("key").transform("first")
 
 
+class TransformEngine:
+    def setup(self):
+        N = 10 ** 3
+        data = DataFrame(
+            {0: [str(i) for i in range(100)] * N, 1: list(range(100)) * N},
+            columns=[0, 1],
+        )
+        self.grouper = data.groupby(0)
+
+    def time_series_numba(self):
+        def function(values, index):
+            return values * 5
+
+        self.grouper[1].transform(function, engine="numba")
+
+    def time_series_cython(self):
+        def function(values):
+            return values * 5
+
+        self.grouper[1].transform(function, engine="cython")
+
+    def time_dataframe_numba(self):
+        def function(values, index):
+            return values * 5
+
+        self.grouper.transform(function, engine="numba")
+
+    def time_dataframe_cython(self):
+        def function(values):
+            return values * 5
+
+        self.grouper.transform(function, engine="cython")
+
+
 from .pandas_vb_common import setup  # noqa: F401 isort:skip

asv_bench/benchmarks/groupby.py

@@ -74,10 +74,38 @@ def setup(self):
             ],
             dtype=object,
         )
+        self.other_mi_many_mismatches = MultiIndex.from_tuples(
+            [
+                (-7, 41),
+                (-2, 3),
+                (-0.7, 5),
+                (0, 0),
+                (0, 1.5),
+                (0, 340),
+                (0, 1001),
+                (1, -4),
+                (1, 20),
+                (1, 1040),
+                (432, -5),
+                (432, 17),
+                (439, 165.5),
+                (998, -4),
+                (998, 24065),
+                (999, 865.2),
+                (999, 1000),
+                (1045, -843),
+            ]
+        )
 
     def time_get_indexer(self):
         self.mi_int.get_indexer(self.obj_index)
 
+    def time_get_indexer_and_backfill(self):
+        self.mi_int.get_indexer(self.other_mi_many_mismatches, method="backfill")
+
+    def time_get_indexer_and_pad(self):
+        self.mi_int.get_indexer(self.other_mi_many_mismatches, method="pad")
+
     def time_is_monotonic(self):
         self.mi_int.is_monotonic