-
Notifications
You must be signed in to change notification settings - Fork 1.1k
iotools: reader for maccrad #279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this is much easier for me to review now.
We need a much smaller sample file (it's currently 44k lines and 5 MB). I think only a day of data would be fine for the sample file.
Please also remove all of the code that you've commented out.
if "Time reference" in line: | ||
if "Universal time (UT)" in line: | ||
tz_raw = 'UTC' | ||
else: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can these files ever have a time zone different from UTC or can they ever have no time zone information?
name = file_csv.split('.')[0] | ||
|
||
## if file is on local drive | ||
f = open(file_csv) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need to use a context manager to guarantee that the file will be closed.
|
||
|
||
|
||
location = Location(lat, lon, name=name, altitude=alt, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Location
is part of the higher level API, so we don't want to force people into it in a low level reader. Instead, build and return a dict. You can add a constructor method to Location
that consumes that dict.
tz=tz_loc) | ||
|
||
#XXX | ||
metadata = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what's this?
|
||
## if file is on local drive | ||
f = open(file_csv) | ||
for line in f: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to add a way to stop the iteration once we reach the end of the header.
# if line.startswith( "# Longitude"): | ||
lon_line = line | ||
lon = float(lon_line.split(':')[1]) | ||
lon = float(line.split(':')[1]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
duplicate
# latitude and longitude coordinates? | ||
# http://stackoverflow.com/a/16086964 | ||
# upstream: https://github.com/MrMinimal64/timezonefinder | ||
from timezonefinder import TimezoneFinder |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
see comment here: #274 (comment)
|
||
return tz | ||
|
||
def localise_df(df_notz, tz_source_str='UTC', tz_target_str=None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need this function? Seems more clear and robust to only call the pandas methods.
Retrieve timezone | ||
""" | ||
if not name: | ||
name = file_csv.split('.')[0] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sounds fragile. Check out the os
module for how to separate the file name from the full path. Next, are you trying to drop .csv
from the name with the split? If so, do that explicitly.
|
||
def read_maccrad_metadata(file_csv, name='maccrad'): | ||
""" | ||
Read metadata from commented lines of the file |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
numpydoc format
Still interested in the feature, but this PR would need to be reopened against the master branch. |
a separate PR for MACC-RAD as requested in #274 (comment)
Discussion in #271