Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

Gangsta Lean: Dec 3, 2001; Calm, relaxed...what could be more fulfilling?

SirPablo posted:

This might be in left-field/hail mary territory because it is such a niche use, but any suggestion how to open grib2 files using the PythonX,Y framework? I was using python 2.5 before and had a mix of library that worked for what I do (working with a lot of weather model data that is stored in the grib/grib2 format) but decided to try x,y to just have as many tools available as I could. Well, now I'm missing this one small but important piece. I found this but I got the same error indicated in the later comments. Any suggestions (besides reverting back to my 2.5 stuff)?

I don't know anything about pythonxy, never heard of it until now, but have you looked into using GDAL? I also use lots of weather model data and GDAL is my goto library for reading grib2. It's a C++ library, but it has bindings to many languages including python by using SWIG. With python, you can read each timestep's data as a numpy array.

GDAL is very powerful and can reproject data, load and save many file formats through a common API, and is used by many mapping tools under the hood since it has such a large number of standard capabilities.

I don't know about its python 2.5 support, I use it with 2.7.

code:

import osgeo.gdal
osgeo.gdal.UseExceptions()
osgeo.gdal.SetConfigOption('GRIB_NORMALIZE_UNITS', 'NO')

dataset = osgeo.gdal.Open('my filename')
for band_index in xrange(dataset.RasterCount):
    band = dataset.GetRasterBand(band_index + 1)
    md = band.GetMetadata()
    geotransform = dataset.GetGeoTransform()
    wkt = dataset.GetProjection()
    data = band.ReadAsArray()

# ¿ Aug 27, 2013 05:36

Adbot: ADBOT LOVES YOU

# ¿ May 19, 2024 18:39

Gangsta Lean: Dec 3, 2001; Calm, relaxed...what could be more fulfilling?

Your file isn't closed. Unindent the last two lines of your program so that they are outside the scope of the "with" line (which automatically closes your file when the block's scope ends).

Edit: But you'd have to open the file again, first. Or do f.seek(0) before Image.open(f) I guess.

Gangsta Lean fucked around with this message at 02:03 on Jun 6, 2014

# ¿ Jun 6, 2014 02:01

Gangsta Lean: Dec 3, 2001; Calm, relaxed...what could be more fulfilling?

I'm late and you said you got it already, but I'd do something like this, looping over the formats in order of which ever is expected to occur more often. I threw in a bad value to show that it raises the most recent exception if it can't find a match.

code:

#!/usr/local/bin/python2.7
    
import datetime


def parse_time(time_str):
    formats = ['%M:%S.%f', '%H:%M:%S.%f']
    
    for fmt in formats:
        try:
            return datetime.datetime.strptime(time_str, fmt).time()
        except ValueError:
            continue
    raise


times = ['23:32.6', '1:02:55.7', '1']
for t in times:
    print 'parsed is {}'.format(parse_time(t))

# ¿ Jul 17, 2014 03:14

Gangsta Lean: Dec 3, 2001; Calm, relaxed...what could be more fulfilling?

You should at least take the repetitive list creation out of the comprehension.

# ¿ Dec 6, 2014 14:42

Gangsta Lean: Dec 3, 2001; Calm, relaxed...what could be more fulfilling?

That isn't code golf at all. That is indeed the most straightforward way to test a palindrome in any language.
1. Create a reversed copy of the original string.
2. Compare the copy to the original, testing for equality.

It just so happens that the idiomatic way to reverse a string in Python is short and somewhat cryptic.

# ¿ Oct 15, 2015 02:03

Gangsta Lean: Dec 3, 2001; Calm, relaxed...what could be more fulfilling?

SurgicalOntologist posted:

Is there a preferred python library for dealing with GPS coordinates? Specifically I'll be converting them to a coordinate system in meters.

I think pyproj is what you're asking for. It's for converting a point from one coordinate system to another, for example I use it for converting from EPSG:4326 (your standard lat/lon projection) to EPSG:3857 (Web Mercator, where the coordinates are in meters). If that's what you're asking for, this is the recommended library. If you're worried about accuracy issues, be sure you use the right projection info for your source and destination projections when you call the Proj() constructor.

sund posted:

Might be tons of overkill, but I used the GDAL python bindings (http://gdal.org/) in the past and it worked well.

This works, too, is a VERY useful library, and is also using Proj.4 behind the scenes to do reprojections, but if you don't need the other stuff it supplies, pyproj is going to be much easier to install.

# ¿ Dec 5, 2015 01:39

Gangsta Lean: Dec 3, 2001; Calm, relaxed...what could be more fulfilling?

Pumpy Dumper posted:

Question: I don't really understand how zip() works. I managed to find an example to create tuples of n=2 with the following code:
code:
	
def pairwise(self, iterable):
    a, b = tee(iterable)
    next(b, None)
    return zip(a, b)
But I want to change it so that it creates tuples of three, and I can't really figure out how to do that. If I just add another variable c it prints tuples of 3 but the first two items repeat themselves.

edit: was able to get it working with the following code:
code:
	
def pairwise(self, iterable):
    a = iter(iterable)
    b = iter(iterable[1::])
    c = iter(iterable[2::])		
    return zip(a, b, c)
And it seems to work great

Your original question is, How does zip() work?
It takes any number of iterables, and iterates over all of them simultaneously. A list is an iterable, a tuple is an iterable, a dictionary is iterable, a set is iterable, a string is iterable. Even classes you create can be iterable. Here's an example using two lists.

code:

items1 = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
items2 = [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]

for x in zip(items1, items2):
	print(x)

#Result is:
# (0, 10)
# (1, 11)
# (2, 12)
# (3, 13)
# (4, 14)
# (5, 15)
# (6, 16)
# (7, 17)
# (8, 18)
# (9, 19)

Your edited definition of pairwise may work for your use case, but
1. It no longer produces a pair, it is a triple - so, pairwise is a bad name for the method.
2. Your method argument is named iterable, but by using slicing, you prevent iterables that don't support slicing.
3. You created multiple (shallow) copies of the input (see further below).

An input that is iterable but doesn't support slicing, such as a dictionary (iterating over a dictionary gives you the keys), does this when using your pairwise function:

code:

items3 = {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'e': 4, 'f': 5, 'g': 6, 'h': 7}

def pairwise(iterable):
    a = iter(iterable)
    b = iter(iterable[1::])
    c = iter(iterable[2::])		
    return zip(a, b, c)

for x in pairwise(items3):
	print(x)

# Traceback (most recent call last):
#   File "python", line 51, in <module>
#   File "python", line 41, in pairwise
# TypeError: unhashable type: 'slice'

When you used the [1::] syntax, you created a duplicate of your input. You can see this here, because only the first list is affected.

code:

items1 = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
v = items1[1::]
items1[2] = 10000

print(items1)
print(v)

[0, 1, 10000, 3, 4, 5, 6, 7, 8, 9]
[1, 2, 3, 4, 5, 6, 7, 8, 9]

If you want to support all types of iterables, you want to use something more like this:

code:

import itertools

items1 = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
items3 = {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'e': 4, 'f': 5, 'g': 6, 'h': 7}

def threewise(iterable):
	a, b = itertools.tee(iterable)
	a, c = itertools.tee(iterable)
	next(b, None)
	next(c, None)
	next(c, None)
	return zip(a, b, c)


print('threewise a list')
for x in threewise(items1):
	print(x)
	
print('threewise a dictionary')
for x in threewise(items3):
	print(x)


# threewise a list
# (0, 1, 2)
# (1, 2, 3)
# (2, 3, 4)
# (3, 4, 5)
# (4, 5, 6)
# (5, 6, 7)
# (6, 7, 8)
# (7, 8, 9)
# threewise a dictionary
# ('f', 'c', 'h')
# ('c', 'h', 'a')
# ('h', 'a', 'd')
# ('a', 'd', 'b')
# ('d', 'b', 'g')
# ('b', 'g', 'e')

* itertools.tee() returns two independent iterators from a single iterable, so we can iterate more than once from the same input iterator. We need to do this twice so that we get three independent iterators: a, b, and c. Tee does not duplicate the input. By using it, it allows each of a, b, and c to iterate over the values of the input one item at a time, without producing three complete copies of the input.

* next() advances an iterator by one item. To set the desired starting points, we need to do this once to skip one item in b, and twice to skip two items in c.

* zip() was explained earlier.

Does this demonstrate it well enough?

# ¿ Apr 22, 2016 01:19

Adbot: ADBOT LOVES YOU

# ¿ May 19, 2024 18:39

Gangsta Lean: Dec 3, 2001; Calm, relaxed...what could be more fulfilling?

Lpzie, I see you're using numpy. If m and r in your real data are large numpy arrays, neither your original code nor list comprehensions are efficient. You can do it all with numpy, where it will be much faster since all the calculations take place in non-interpreted code.

code:

import numpy

G = 6.67e-8
m = numpy.array([1.90e33, 2.00e33, 2.10e33, 2.20e33])
r = numpy.array([5.00e10, 5.10e10, 5.20e10, 5.30e10])

logg = numpy.log10(G * m / numpy.power(r, 2))

# ¿ Apr 22, 2016 23:55

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.