Skip to content
This repository was archived by the owner on Dec 22, 2022. It is now read-only.

Merging in remote master #1

Merged
merged 59 commits into from
May 10, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
e30bf51
Update Table and Patch Table support has been added.
Sep 4, 2015
5b7eabc
Add more syntax support for SQL query generation
sagarrakshe Oct 14, 2015
87d053c
Make the SELECT alias 'AS' to lower case
sagarrakshe Oct 14, 2015
9b6493e
Added tags to the .gitignore
sagarrakshe Oct 14, 2015
b3ec17f
Modify tests
sagarrakshe Oct 14, 2015
d615e12
Merge pull request #1 from sarang4/table_update_patch
sagarrakshe Oct 14, 2015
bfb88ac
Render conditions in HAVING clause
sagarrakshe Oct 14, 2015
1249147
Modify the BETWEEN condition logic
sagarrakshe Oct 15, 2015
ec85547
Unit Test to check the BETWEEN operator
sagarrakshe Oct 15, 2015
2150806
Unit Test for HAVING Clause
sagarrakshe Oct 15, 2015
c25ead6
Add Having clause in complete render query
sagarrakshe Oct 15, 2015
cdfbac8
Unit Test for selecting tables in date range
sagarrakshe Oct 15, 2015
005ae94
Minor error handling in _render_sources
sagarrakshe Oct 15, 2015
cfb5d29
Updated README.md according to new changes
sagarrakshe Oct 15, 2015
90d6643
Minor bug fix: BETWEEN Operator
sagarrakshe Oct 16, 2015
7c0167e
Take into account limit argument when returning rows
mdanieli Dec 22, 2015
d454a3d
Fix stopping condition
mdanieli Dec 22, 2015
337bf6e
Merge pull request #76 from BendingSpoons/master
tylertreat Dec 22, 2015
117b916
Bump version to 1.5.0
tylertreat Dec 22, 2015
26241da
Add an update table method to client
Jan 18, 2016
5c6afe7
Add an patch table method to client
Jan 18, 2016
eecd8f4
Merge pull request #78 from izberg-marketplace/feat_table_update_patch
tylertreat Jan 18, 2016
8f365c6
added udf support to client.write_table and tests fixed
sshrestha-datalicious Jan 21, 2016
e547196
Update README.md to include example for UDF support in write_to_table
sshrestha-datalicious Jan 21, 2016
9ed0969
docstring for external_udf_uris in client.write_to_table
sshrestha-datalicious Jan 21, 2016
970546f
Update README.md to include missing comma
sshrestha-datalicious Jan 21, 2016
9deda4e
Merge pull request #79 from sshrestha-datalicious/add_udf_support_to_…
tylertreat Jan 21, 2016
2d3a08b
Removed un-neccessary tags file
sagarrakshe Feb 2, 2016
f09f4da
Merge branch 'master' into render_query_modifications
sagarrakshe Feb 2, 2016
12156ca
Corrected BETWEEN comparator test
sagarrakshe Feb 2, 2016
bd7ce78
Remove duplicate methods 'patch_table', 'update_table'
sagarrakshe Feb 3, 2016
0221215
Fix: Deleted the un-correct code for patch_table
sagarrakshe Feb 4, 2016
4ab6674
Merge pull request #80 from sagarrakshe/render_query_modifications
tylertreat Feb 10, 2016
d45009a
Add service_url parameter to get_client and _get_bq_service
andrewgardners Feb 11, 2016
12231de
Modify test_client tests to take service_url parameter into acccount
andrewgardners Feb 11, 2016
15b066c
Merge pull request #81 from andrewgardners/master
tylertreat Feb 11, 2016
3f90480
Bump version to 1.6.0
tylertreat-wf Mar 15, 2016
117af09
Changed all docstrings to NumPy Style Python Docstrings
nickstanisha Apr 9, 2016
35d784d
Added Sphinx and Numpydoc tools for automated documentation
nickstanisha Apr 9, 2016
65896b3
Merge pull request #88 from nickstanisha/master
tylertreat Apr 10, 2016
44e0b78
Add documentation link to readme
tylertreat Apr 10, 2016
218b842
Add stream insert options
hagino3000 Apr 14, 2016
bfbfde6
Upgrade to latest OAuth library(2.0.2) and Google Python library (1.5.0)
jasonab Apr 17, 2016
a1af7de
Merge pull request #91 from jasonab/google-credentials
tylertreat Apr 17, 2016
5a9290a
Add tests
hagino3000 Apr 18, 2016
2b43879
Merge pull request #90 from hagino3000/add_stream_insert_options
tylertreat Apr 18, 2016
6436432
Use package logger instead of root logger
hagino3000 Apr 22, 2016
9678944
Revert comment line
hagino3000 Apr 22, 2016
22d3e58
Revart unnecesarry change
hagino3000 Apr 22, 2016
4042057
Merge pull request #92 from hagino3000/use_logger
tylertreat Apr 22, 2016
7ce159a
Read project_id from JSON key file.
orangain Apr 23, 2016
2904a1b
Merge pull request #93 from orangain/use-project-id-in-json-key
tylertreat Apr 23, 2016
890270a
Bump version to 1.7.0
tylertreat Apr 23, 2016
4f84b83
Various formatting fixes and fix import issue
tylertreat Apr 23, 2016
7da5061
Fix version
tylertreat Apr 23, 2016
3a52987
Merge pull request #94 from tylertreat/fixes
tylertreat Apr 23, 2016
c9eb5e4
PEP8 formatting fixes
tylertreat Apr 23, 2016
bab1c99
Add template_suffix option support
hagino3000 Apr 28, 2016
97a55bc
Merge pull request #95 from hagino3000/add_template_suffix_option
tylertreat Apr 28, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 54 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ Simple Python client for interacting with Google BigQuery.

This client provides an API for retrieving and inserting BigQuery data by wrapping Google's low-level API client library. It also provides facilities that make it convenient to access data that is tied to an App Engine appspot, such as request logs.

[Documentation](http://tylertreat.github.io/BigQuery-Python/)

# Installation

`pip install bigquery-python`
Expand All @@ -29,7 +31,7 @@ client = get_client(project_id, service_account=service_account,
# JSON key provided by Google
json_key = 'key.json'

client = get_client(project_id, json_key_file=json_key, readonly=True)
client = get_client(json_key_file=json_key, readonly=True)

# Submit an async query.
job_id, _results = client.query('SELECT * FROM dataset.my_table LIMIT 1000')
Expand Down Expand Up @@ -99,13 +101,32 @@ conditions = [
}
]

grouping = ['Timestamp']

having = [
{
'field': 'Timestamp',
'type': 'INTEGER',
'comparators': [
{
'condition': '==',
'negate': False,
'value': 1399478981
}
]
}
]

order_by ={'fields': ['Timestamp'], 'direction': 'desc'}

query = render_query(
'dataset',
['table'],
select=selects,
conditions=conditions,
groupings=['Timestamp'],
order_by={'field': 'Timestamp', 'direction': 'desc'}
groupings=grouping,
having=having,
order_by=order_by
)

job_id, _ = client.query(query)
Expand Down Expand Up @@ -168,6 +189,34 @@ try:
except BigQueryTimeoutException:
print "Timeout"

# write to permanent table with UDF in query string
external_udf_uris = ["gs://bigquery-sandbox-udf/url_decode.js"]
query = """SELECT requests, title
FROM
urlDecode(
SELECT
title, sum(requests) AS num_requests
FROM
[fh-bigquery:wikipedia.pagecounts_201504]
WHERE language = 'fr'
GROUP EACH BY title
)
WHERE title LIKE '%ç%'
ORDER BY requests DESC
LIMIT 100
"""
job = client.write_to_table(
query,
'dataset',
'table',
external_udf_uris=external_udf_uris
)

try:
job_resource = client.wait_for_job(job, timeout=60)
print job_resource
except BigQueryTimeoutException:
print "Timeout"

# write to temporary table
job = client.write_to_table('SELECT * FROM dataset.original_table LIMIT 100')
Expand All @@ -176,6 +225,8 @@ try:
print job_resource
except BigQueryTimeoutException:
print "Timeout"


```

# Import data from Google cloud storage
Expand Down
3 changes: 3 additions & 0 deletions bigquery/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
from __future__ import absolute_import

from .version import __version__

from .client import get_client
from .client import (
BIGQUERY_SCOPE,
Expand Down
Loading