DOC: ecosystem: Vaex, Pandas on Ray, alphabetization, pandas-datareader #20345

westurner · 2018-03-14T14:32:48Z

closes #20334 - Pandas on Ray
closes #20335 - Vaex

kylebarron · 2018-03-14T17:14:15Z

It might be worthwhile to also reference https://github.com/altair-viz/altair in the visualization section, since Vincent refers people to Altair: https://github.com/wrobstory/vincent#2016-06-18-update

westurner · 2018-03-15T00:04:27Z

Good call. #20355 "DOC: ecosystem.rst: Altair"

westurner · 2018-03-15T00:05:29Z

Pending a response to #20355, I'll go ahead and also alphabetize the Visualization Tools section and then this PR will be ready to merge.

TomAugspurger · 2018-03-15T12:52:40Z

doc/source/ecosystem.rst

+`Vaex <https://docs.vaex.io/>`_
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Increasingly, packages are being built on top of pandas to address specific needs in data preparation, analysis and visualization. Vaex is a python library for Out-of-Core DataFrames (similar to Pandas), to visualize and explore big tabular datasets. It can calculate statistics such as mean, sum, count, standard deviation etc, on an N-dimensional grid up to a billion (10\ :sup:`9`) objects/rows per second. Visualization is done using histograms, density plots and 3d volume rendering, allowing interactive exploration of big data. Vaex uses memory mapping, zero memory copy policy and lazy computations for best performance (no memory wasted).


Is Vaex built on top of pandas?

Vaex is not built on top of Pandas; which is why I asked whether it's appropriate for this list in #20335.

We could also list PyArrow, which also, in relation to Pandas, only does to/from:
https://arrow.apache.org/docs/python/pandas.html

jorisvandenbossche · 2018-03-15T16:30:15Z

doc/source/ecosystem.rst

+
+Blaze provides a standard API for doing computations with various
+in-memory and on-disk backends: NumPy, Pandas, SQLAlchemy, MongoDB, PyTables,
+PySpark.


I know you are only putting them in alphabetical order, but I not sure to what extent the blaze library itself is still actively developed (in contrast to things that grew out of it, like dask)

I think @llllllllll is still doing some work on it?

we could remove blaze I think. Should add Ibis though:http://docs.ibis-project.org/

Ibis also 'only' does to/from in relation to pandas?
http://docs.ibis-project.org/impala.html#ingesting-data-from-pandas

Would we add ibis under 'Out of Core' or a new 'Similar API' section?

I, and a few others, are still using blaze, but it is not very active. We are still using blaze at @quantopian in production, and it is still maintained. Given its current state, I am fine with either leaving it or removing it.

jreback · 2018-03-15T23:23:53Z

doc/source/ecosystem.rst

@@ -252,6 +259,26 @@ PyTables, h5py, and pymongo to move data between non pandas formats. Its graph
 based approach is also extensible by end users for custom formats that may be
 too specific for the core of odo.

+`Ray <https://ray.readthedocs.io/en/latest/pandas_on_ray.html>`_
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


line is too long here

jreback · 2018-03-15T23:24:03Z

doc/source/ecosystem.rst

+
+
+`Vaex <https://docs.vaex.io/>`_
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


needs to be the same length

codecov · 2018-03-16T04:37:20Z

Codecov Report

❗ No coverage uploaded for pull request base (master@20ca108). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master   #20345   +/-   ##
=========================================
  Coverage          ?   91.84%           
=========================================
  Files             ?      153           
  Lines             ?    49279           
  Branches          ?        0           
=========================================
  Hits              ?    45259           
  Misses            ?     4020           
  Partials          ?        0

Flag	Coverage Δ
#multiple	`90.23% <ø> (?)`
#single	`41.9% <ø> (?)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 20ca108...94f0cdd. Read the comment docs.

https://libraries.io/pypi/pandas/usage

…vailable

jreback · 2018-03-25T14:48:10Z

doc/source/ecosystem.rst

 `Dask <https://dask.readthedocs.io/en/latest/>`__
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


think length here needs to be edited

jreback · 2018-03-25T14:48:15Z

doc/source/ecosystem.rst


 Dask is a flexible parallel computing library for analytics. Dask
 provides a familiar ``DataFrame`` interface for out-of-core, parallel and distributed computing.

 `Dask-ML <https://dask-ml.readthedocs.io/en/latest/>`__
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


The line lengths appear to be equal in my editor, and the output from rst2html.py doesn't show an error.

jreback · 2018-03-30T21:28:47Z

can you update

jreback · 2018-04-14T13:42:49Z

@westurner can you update

westurner · 2018-04-15T22:19:48Z

What needs to be updated here?

jorisvandenbossche · 2018-07-08T15:14:52Z

Rebased, thanks @westurner !

…er (pandas-dev#20345)

westurner added 5 commits March 14, 2018 10:21

DOC: ecosystem.rst: Vaex (closes pandas-dev#20335)

bfc1c57

DOC: ecosystem.rst: Pandas on Ray (closes pandas-dev#20334)

08d3116

DOC: ecosystem.rst: alphabetize Out-of-Core section

f9b3e98

DOC: ecosystem.rst: update pandas-datareader data sources

cfb5262

DOC: ecosystem.rst: Vaex from_pandas, to_pandas_df

a07dd94

sinhrks added the Docs label Mar 15, 2018

TomAugspurger reviewed Mar 15, 2018

View reviewed changes

jorisvandenbossche reviewed Mar 15, 2018

View reviewed changes

jreback requested changes Mar 15, 2018

View reviewed changes

DOC: ecosystem.rst: remove Vincent per pandas-dev#20355

77edcd0

westurner and others added 6 commits March 16, 2018 00:39

DOC: ecosystem.rst: ipyvega wording

26c4c9e

DOC: ecosystem.rst: header line lengths

035dd40

DOC: ecosystem.rst: link to libraries.io and pypi

dfc129b

https://libraries.io/pypi/pandas/usage

DOC: ecosystem.rst: ggplot -> ggpy

0feb269

DOC: ecosystem.rst: PyCharm

8f6cf0c

DOC: ecosystem.rst: Spyder note re: # In[0] prompts

7f9c00b

westurner mentioned this pull request Mar 16, 2018

Potential documentation clarifications regarding IPython/Jupyter notebook support? spyder-ide/spyder#6774

Closed

DOC: ecosystem.rst: IPython/Jupyter Notebook, _repr_latex_, options.a…

ffdd5e9

…vailable

jreback requested changes Mar 25, 2018

View reviewed changes

jreback added this to the 0.23.0 milestone Mar 25, 2018

jreback mentioned this pull request Mar 25, 2018

DOC: add ibis to pandas ecosystem #20483

Closed

jreback removed this from the 0.23.0 milestone Apr 14, 2018

DOC: ecosystem.rst: fix header underlines

42438c7

Wes Turner and others added 3 commits April 15, 2018 18:25

DOC: ecosystem.rst: Altair. closes pandas-dev#20355

25f6ae6

Merge remote-tracking branch 'upstream/master' into westurner-patch-7

6fe2b2f

remove pycharm

94f0cdd

jorisvandenbossche merged commit f90aa44 into pandas-dev:master Jul 8, 2018

jorisvandenbossche modified the milestones: 0.23.4, 0.24.0 Jul 8, 2018

Sup3rGeo pushed a commit to Sup3rGeo/pandas that referenced this pull request Oct 1, 2018

DOC: ecosystem: Vaex, Pandas on Ray, alphabetization, pandas-dataread…

aa1732c

…er (pandas-dev#20345)



		`Vaex <https://docs.vaex.io/>`_
		~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Uh oh!

DOC: ecosystem: Vaex, Pandas on Ray, alphabetization, pandas-datareader #20345

DOC: ecosystem: Vaex, Pandas on Ray, alphabetization, pandas-datareader #20345

Uh oh!

Conversation

westurner commented Mar 14, 2018 • edited by jorisvandenbossche Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kylebarron commented Mar 14, 2018

Uh oh!

westurner commented Mar 15, 2018

Uh oh!

westurner commented Mar 15, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Mar 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jreback commented Mar 30, 2018

Uh oh!

jreback commented Apr 14, 2018

Uh oh!

westurner commented Apr 15, 2018

Uh oh!

jorisvandenbossche commented Jul 8, 2018

Uh oh!

Uh oh!

westurner commented Mar 14, 2018 •

edited by jorisvandenbossche

Loading

codecov bot commented Mar 16, 2018 •

edited

Loading