Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Nippashish: Nov 2, 2005; Let me see you dance!

Captain von Trapp posted:

What's the Pythonic way to do this? At first I tried:
code:
print [x for x in range(1000,10000) if list(str(x)).sort() == list(str(4*x)).sort()]

You can use sorted which creates a generator that yields elements in sorted order, so your one liner becomes:

code:

print [x for x in range(1000,10000) if sorted(str(x)) == sorted(str(4*x))]

# ? Aug 12, 2012 21:39

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 16:46

Captain von Trapp: Jan 23, 2006; I don't like it, and I'm sorry I ever had anything to do with it.

Thanks both of you, that works beautifully.

# ? Aug 12, 2012 21:41

leterip: Aug 25, 2004

Suspicious Dish posted:

That would make bar(None) behave wrong. I'm not sure what Bodhi wants, though.

How so? Seems like bar(None) would behave just like foo(None) would.

# ? Aug 12, 2012 22:08

Cat Plus Plus: Apr 8, 2011

leterip posted:

How so? Seems like bar(None) would behave just like foo(None) would.

bar uses None as default argument, which is supposed to correspond to calling foo with a defaulted argument. But foo doesn't use None for that, so now semantics have changed.

# ? Aug 12, 2012 22:41

leterip: Aug 25, 2004

PiotrLegnica posted:

bar uses None as default argument, which is supposed to correspond to calling foo with a defaulted argument. But foo doesn't use None for that, so now semantics have changed.

Uh, in the code he was responding to, bar was defined as def bar(*args, **kwargs). I don't see a default argument there. :confused:

# ? Aug 12, 2012 22:56

Cat Plus Plus: Apr 8, 2011

leterip posted:

Uh, in the code he was responding to, bar was defined as def bar(*args, **kwargs). I don't see a default argument there.

That code is a response to this:

Bodhi Tea posted:

Say I have a function with an optional argument which I don't know the default value of:
code:
def foo (max_id=<mystery value>)
and another function, that will call foo(). Is there a more succinct way of doing this?
code:
def bar(max_id=None):
    if max_id:
          foo(max_id)
    else:
          foo()

# ? Aug 12, 2012 23:00

leterip: Aug 25, 2004

PiotrLegnica posted:

That code is a response to this:

Yeah but that code was a completely different definition. I guess I just got confused because I thought the code he quoted didn't have the problem he was describing.

# ? Aug 12, 2012 23:25

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

I don't see the point in

Python code:

def bar(*args, **kwargs):
    foo(*args, **kwargs)

You lose the return value and docstring, and gain what, exactly?

As far as I understood, the point of:

Python code:

def bar(a_thing=None):
    if a_thing is None:
        foo()
    else
        foo(a_thing)

was to make bar(None) consistent, no matter what random default value might have been placed in foo's signature. I think it's fairly clear that the only place that this changes is when a_thing is None.

I'm not sure why Bodhi wants this, and if he gives his overall goal, I'm sure we'll find a more elegant solution. XY problem and all that.

# ? Aug 13, 2012 00:20

Bodhi Tea: Oct 2, 2006; seconds are secular, moments are mine, self is illusion, music's divine.

Suspicious Dish posted:

I don't see the point in
Python code:
def bar(*args, **kwargs):
    foo(*args, **kwargs)
You lose the return value and docstring, and gain what, exactly?

As far as I understood, the point of:
Python code:
def bar(a_thing=None):
    if a_thing is None:
        foo()
    else
        foo(a_thing)
was to make bar(None) consistent, no matter what random default value might have been placed in foo's signature. I think it's fairly clear that the only place that this changes is when a_thing is None.

I'm not sure why Bodhi wants this, and if he gives his overall goal, I'm sure we'll find a more elegant solution. XY problem and all that.

Yup I think you got my intention here. I'm using the Twitter api with Twython. There is a "max_id" argument that you can pass to getHomeTimeline() in order to properly parse a Twitter timeline. However, this argument should not be passed on the first call.

code:

    if session.get("max_id"):
        tweetq=g.t.getHomeTimeline(include_entities=True,count=5,max_id=session["max_id"])
    else:
        tweetq=g.t.getHomeTimeline(include_entities=True,count=5)

I don't know what Twitter uses as the default for max_id.

I'm using Flask and I know I'm probably doing a million things wrong, but one thing at a time and all that

# ? Aug 13, 2012 01:13

Scaevolus: Apr 16, 2007

Bodhi Tea posted:

Yup I think you got my intention here. I'm using the Twitter api with Twython. There is a "max_id" argument that you can pass to getHomeTimeline() in order to properly parse a Twitter timeline. However, this argument should not be passed on the first call.
code:
    if session.get("max_id"):
        tweetq=g.t.getHomeTimeline(include_entities=True,count=5,max_id=session["max_id"])
    else:
        tweetq=g.t.getHomeTimeline(include_entities=True,count=5)
I don't know what Twitter uses as the default for max_id.

I'm using Flask and I know I'm probably doing a million things wrong, but one thing at a time and all that

This should work:

code:

tweetq = g.t.getHomeTimeline(include_entities=True, count=5, max_id=session.get("max_id", None))

Scaevolus fucked around with this message at 01:20 on Aug 13, 2012

# ? Aug 13, 2012 01:18

Bodhi Tea: Oct 2, 2006; seconds are secular, moments are mine, self is illusion, music's divine.

Scaevolus posted:

This should work:

code:

tweetq = g.t.getHomeTimeline(include_entities=True, count=5, max_id=session.get("max_id", None))

I thought of that, but as best I can tell, when I pass max_id=None, no tweets are returned. I just double checked by replacing my code with yours and it seems tweetq is empty.

# ? Aug 13, 2012 01:28

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Python code:

kwargs = {}

if 'max_id' in session:
    kwargs['max_id'] = session['max_id']

tweetq = g.t.getHomeTimezone(include_entities=True, count=5, **kwargs)

# ? Aug 13, 2012 01:38

Bodhi Tea: Oct 2, 2006; seconds are secular, moments are mine, self is illusion, music's divine.

Suspicious Dish posted:

Python code:

kwargs = {}

if 'max_id' in session:
    kwargs['max_id'] = session['max_id']

tweetq = g.t.getHomeTimezone(include_entities=True, count=5, **kwargs)

That works, thanks!

# ? Aug 13, 2012 02:44

The King of Swag: Nov 10, 2005; To escape the closure,
is to become the God of Swag.

As a long-time (and happy) C and Objective-C programmer, I just recently had to learn Python so I could use the panda3d engine. I will give Python this: it isn't C++ (which is quite possibly the worst language ever designed), but Python does some really stupid stuff that I need help figuring out.

Tabs: Spaces are for weirdos who like to wear out their space and delete keys particularly quickly, but Python seems to choke on tabs used as the leading whitespace. Is there a solution for this, or will I just have to deal?

Empty Methods: The first thing I do when I create a new class in Objective-C is to design the API and put in all the methods, so that I have a shell that can be later fleshed out. This allows me to design other parts of the code that utilizes the class before fully fleshing it out. Python has a fit over empty methods though; is there a solution other than throwing in garbage code just to appease the parser?

Serialization: I realize that Python has the pickling functionality, but it's inherently unsafe and apparently does way more than I actually want it to (just save the values of the instance variables; I don't care about restoring the object itself). In the end I've resorted to writing my own serialization code utilizing the struct class--unfortunately the lack of fine-grained data control and the beauty that are void pointers actually makes this much more complicated than the equivalent C code.

I realize that some of this is probably because I'm coming from a C background, and that the overwhelming response will be "well that's just not how we do it in Python", but despite being an overall useful and nice language, it makes some really poor design decisions (the only other language I actively use that cares anything about white-space is Fortran 77, which should tell you something about the decision to use it in Python).

# ? Aug 13, 2012 07:43

fart simpson: Jul 2, 2005; DEATH TO AMERICA

1. Python only cares if you mix tabs and spaces. If you don't want to force yourself to adhere to one or the other, probably use a text editor that can autoreplace tabs to spaces. I mostly use Python on Windows, and I have Notepad++ set up to automatically type 4 spaces whenever I press tab

2. Use pass:

Python code:

class Foo:
    def bar(self):
        pass

3. Can't help you here

# ? Aug 13, 2012 08:04

The King of Swag: Nov 10, 2005; To escape the closure,
is to become the God of Swag.

MeramJert posted:

1. Python only cares if you mix tabs and spaces. If you don't want to force yourself to adhere to one or the other, probably use a text editor that can autoreplace tabs to spaces. I mostly use Python on Windows, and I have Notepad++ set up to automatically type 4 spaces whenever I press tab

2. Use pass:
Python code:
class Foo:
    def bar(self):
        pass
3. Can't help you here

Thanks for the help; that makes at least the first two problems a littler simpler.

# ? Aug 13, 2012 08:18

Maluco Marinero: Jan 18, 2001; Damn that's a
fine elephant.

Not to get in a holy war, but your rationale for effective whitespace as bad design is somewhat silly. It produces very readable code, making it very easy to discern the structure and flow, whether you wrote it or not.

Just because it isn't the norm doesn't make it inherently bad, and using tools such as the standard library itertools, list comprehensions and the like will remove the need for heavily nested structures.

# ? Aug 13, 2012 08:44

The King of Swag: Nov 10, 2005; To escape the closure,
is to become the God of Swag.

Maluco Marinero posted:

Not to get in a holy war, but your rationale for effective whitespace as bad design is somewhat silly. It produces very readable code, making it very easy to discern the structure and flow, whether you wrote it or not.

Just because it isn't the norm doesn't make it inherently bad, and using tools such as the standard library itertools, list comprehensions and the like will remove the need for heavily nested structures.

Actually I'll say that neither C nor Python is right in this situation; braces should be optional but completely valid code structures. Reliance on whitespace to properly parse code, by necessity limits your options on code style. Worst of all, I've never seen a white-space related error thrown that actually relates it to white-space at all, leading to sometimes ridiculous amounts of time spent trying to track a bug that shouldn't be a bug to begin with.

In short, I agree with you: whitespace can be a very effective tool to structure code and define flow, but a fundamental reliance on it to function is ridiculous.

# ? Aug 13, 2012 09:09

fart simpson: Jul 2, 2005; DEATH TO AMERICA

The King of Swag posted:

Worst of all, I've never seen a white-space related error thrown that actually relates it to white-space at all, leading to sometimes ridiculous amounts of time spent trying to track a bug that shouldn't be a bug to begin with.

What do you mean by this?

Python code:

>>> for i in range(10):
print i
  File "<pyshell#88>", line 2
    print i
        ^
IndentationError: expected an indented block

>>> def f(x):
	x += 1
	 print x
	 
  File "<pyshell#97>", line 3
    print x
   ^
IndentationError: unexpected indent

Python code:

#error.py
def f(x):
[tab]x += 1 #[tab] literally refers to using tab here in place of spaces
     print x
f(1)

code:

>python error.py
  File "error.py", line 3
    print x
          ^
IndentationError: unindent does not match any outer indentation level

# ? Aug 13, 2012 09:59

Maluco Marinero: Jan 18, 2001; Damn that's a
fine elephant.

I am admittedly no expert but...

The King of Swag posted:

Reliance on whitespace to properly parse code, by necessity limits your options on code style.

Is entirely the point in the design of Python, as is the introduction of PEP8. You can always backslash a line end to carry a statement over to the next line if you need to as well.

quote:

Worst of all, I've never seen a white-space related error thrown that actually relates it to white-space at all, leading to sometimes ridiculous amounts of time spent trying to track a bug that shouldn't be a bug to begin with.

Can't the same be said for braces, spotting and counting braces is not always easy either, and is highly dependant on the mercy of the previous author.

quote:

In short, I agree with you: whitespace can be a very effective tool to structure code and define flow, but a fundamental reliance on it to function is ridiculous.

Fair enough if that's your opinion. In the end of the day though, there's no such thing as one true language, just different languages with different goals, and they should be evaluated on the basis of those goals.

# ? Aug 13, 2012 10:04

fart simpson: Jul 2, 2005; DEATH TO AMERICA

Maluco Marinero posted:

Is entirely the point in the design of Python, as is the introduction of PEP8. You can always backslash a line end to carry a statement over to the next line if you need to as well.

This is true:

Python code:

>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Particularly look at the line that says "There should be one-- and preferably only one --obvious way to do it." It's a design goal of Python

# ? Aug 13, 2012 10:12

evilentity: Jun 25, 2010

The King of Swag posted:

Serialization: I realize that Python has the pickling functionality, but it's inherently unsafe and apparently does way more than I actually want it to (just save the values of the instance variables; I don't care about restoring the object itself). In the end I've resorted to writing my own serialization code utilizing the struct class--unfortunately the lack of fine-grained data control and the beauty that are void pointers actually makes this much more complicated than the equivalent C code.

This should do the trick.

Python code:

def save_instance_vars(object, path):
    with open(path, 'w') as f:
    	variables = vars(object)
   	for (variable, value) in variables.items():
            line = 'variable:{variable}, value:{value}'.format(variable=variable, value=value)
	        f.write(line)

Src

# ? Aug 13, 2012 11:21

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Python will always expand tab characters to 8 spaces, because tab characters are always 8 spaces (if you set your editor to anything else source will look funny)

That competes with PEP8's style of 4 spaces.

Just configure your editor to emit 4 spaces when you hit the tab key. Every programmer editor can do it.

# ? Aug 13, 2012 15:45

JetsGuy: Sep 17, 2003; science + hockey
=
LASER SKATES

Also echoing the make tab = 4 spaces method in your editor. It makes life a million times easier.

# ? Aug 13, 2012 17:36

Rohaq: Aug 11, 2006

My personal favourite is tab = 2 spaces, but I may go up to 4; I'm used to coding with Perl, where curlies are rife, and code blocks can be huge, so indentation with 2 spaces isn't too bad, but 4 spaces with Python would probably be better to maintain code readability.

# ? Aug 13, 2012 18:49

good jovi: Dec 11, 2000; 'm pro-dickgirl, and I VOTE!

I think the only time I've ever seen Python that wasn't 4-space indented was in terrible ActiveState code snippets.

# ? Aug 13, 2012 19:10

NadaTooma: Aug 24, 2004; The good thing is that everyone around you has more critical failures in combat, the bad thing is - so do you!

evilentity posted:

This should do the trick.

When I was looking around for a good explanation on why pickle is insecure, I found a module called Cerealizer, which was written as a safe replacement. I figured a link to it might be useful here.

# ? Aug 13, 2012 19:14

The King of Swag: Nov 10, 2005; To escape the closure,
is to become the God of Swag.

MeramJert posted:

Indentation Error examples

I just tried this and I only get the carat; no notice about the indentation error. What it looks like may be happening is the output window in Komodo is dropping the last line or so of output, as I've noticed some other errors have strangely abrupt descriptions. Thanks for pointing it out though, I would have never known otherwise.

evilentity posted:

This should do the trick.

Python code:

def save_instance_vars(object, path):
    with open(path, 'w') as f:
    	variables = vars(object)
   	for (variable, value) in variables.items():
            line = 'variable:{variable}, value:{value}'.format(variable=variable, value=value)
	        f.write(line)

Src

I've seen that before and the problem is that I need to save my data out as binary and be able to easily read it all back. Using that trick, saving would be easy but loading would require a lot of parsing. My current method of an explicit file format with run lengths embedded for blocks of data is a lot more succinct with no need for parsing of data beyond reading the length of a block, loading that block into memory and reading the next. Although I'm all ears if there is an easy Pythonesque way to load your example.

Suspicious Dish posted:

Python will always expand tab characters to 8 spaces, because tab characters are always 8 spaces

Since when? Tab on every computer I've ever used has been the equivalent of 4 spaces.

# ? Aug 13, 2012 22:50

evilentity: Jun 25, 2010

The King of Swag posted:

I've seen that before and the problem is that I need to save my data out as binary and be able to easily read it all back. Using that trick, saving would be easy but loading would require a lot of parsing. My current method of an explicit file format with run lengths embedded for blocks of data is a lot more succinct with no need for parsing of data beyond reading the length of a block, loading that block into memory and reading the next. Although I'm all ears if there is an easy Pythonesque way to load your example.

Parsing bad? Json to the rescue!

Python code:

import json

class Foo(object):
    def __init__(self):
        self.a = 1
        self.b = 2

def save_vars(object, path):
    with open(path, 'w') as f:
        json.dump(vars(object), f)
        
def load_vars(path):
    with open(path, 'r') as f:
        return json.load(f)            
            
if __name__ == '__main__':
    foo = Foo()
    print vars(foo)
    save_vars(foo, r'c:\temp.json')
    print load_vars(r'c:\temp.json')

# ? Aug 14, 2012 00:12

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Please don't use vars like this. It will break in the unintentional case that you get cyclic values or anything like this.

The correct thing to do is to define a list of variables that can be exported/imported as JSON, or to define a toJSON method that returns something that can be validly passed to json.dumps.

# ? Aug 14, 2012 00:20

Emacs Headroom: Aug 2, 2003

The King of Swag posted:

Serialization: I realize that Python has the pickling functionality, but it's inherently unsafe and apparently does way more than I actually want it to (just save the values of the instance variables; I don't care about restoring the object itself). In the end I've resorted to writing my own serialization code utilizing the struct class--unfortunately the lack of fine-grained data control and the beauty that are void pointers actually makes this much more complicated than the equivalent C code.

A labmate of mine uses Panda3D extensively, and wrote some sort of methods for his classes that make them pickle in a way that he specifies.

I know it's pretty useless of me to say "hey you can make your classes behave how you want with respect to pickling" without saying how, but at least you know it's possible so you can look up how?

edit: looks like you want to write your own __getstate__() and __setstate__(), though I have no idea what side-effects this might have

Emacs Headroom fucked around with this message at 02:11 on Aug 14, 2012

# ? Aug 14, 2012 02:07

Captain Capacitor: Jan 21, 2008; The code you say?

Or you could use Protobuf.

# ? Aug 14, 2012 02:11

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Emacs Headroom posted:

A labmate of mine uses Panda3D extensively, and wrote some sort of methods for his classes that make them pickle in a way that he specifies.

I know it's pretty useless of me to say "hey you can make your classes behave how you want with respect to pickling" without saying how, but at least you know it's possible so you can look up how?

edit: looks like you want to write your own __getstate__() and __setstate__(), though I have no idea what side-effects this might have

Pickle breaks when you sneeze on it. I'd highly suggest having a better serialization method.

# ? Aug 14, 2012 02:24

Mrs. Wynand: Nov 23, 2002; DLT 4EVA

The King of Swag posted:

I've seen that before and the problem is that I need to save my data out as binary and be able to easily read it all back. Using that trick, saving would be easy but loading would require a lot of parsing. My current method of an explicit file format with run lengths embedded for blocks of data is a lot more succinct with no need for parsing of data beyond reading the length of a block, loading that block into memory and reading the next. Although I'm all ears if there is an easy Pythonesque way to load your example.

Firstlies, have you tried cjson for your dataset? I'm not saying fo-shizie it will be fast enough because there are indeed some very large datasets out there but unless your dataset is remarkably large cjson is very likely to be fast 'nuff.

If not, you definitely want protobuffs. Unless your dataset is extremely specialized as well as large you are extremely unlikely to come up with something better loving around struct.pack.

Anyway, json is amazing and I love it to pieces - it just really hits the sweet spot in terms of what a serialization format should do. ;-*

# ? Aug 14, 2012 02:36

tef: May 30, 2004; -> some l-system crap ->

Mr. Wynand posted:

Anyway, json is amazing and I love it to pieces - it just really hits the sweet spot in terms of what a serialization format should do.

Just as long as you use unicode within the BOM, floating point, and no binary data, integers, or dates. :3:

# ? Aug 14, 2012 12:40

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

within the BOM?

did you mean the BMP?

# ? Aug 14, 2012 17:27

Mrs. Wynand: Nov 23, 2002; DLT 4EVA

tef posted:

Just as long as you use unicode within the BOM, floating point, and no binary data, integers, or dates.

UTF-8 until death

base64 for occasional binary derp

are numbers not good enough for you

dates are just strings with app specific meaning! :p

Ok of course you shouldn't actually use json for literally everything. As I said it's just a "sweet spot" for a huge number of application, primarily because it supports what are probably the most useful data structures across just about any language or platform while still remaining really really really simple.

(TBH, I agree with you about dates. Fancy date formats can get complicated sure, but we could probably just have picked one and stuck with it. I'd nominate RFC3339 since that's what <time datetime> uses, and json is all web-y and poo poo anyway. OTOH I can also empathize with the choice against it since now you need a way to delimit string representations of non-string objects and then people will be tempted to allow arbitrary class markers and suddenly we are in YAML clusterfuckland)

# ? Aug 14, 2012 19:12

accipter: Sep 12, 2003

tef posted:

Just as long as you use unicode within the BOM, floating point, and no binary data, integers, or dates.

Why not integers?

# ? Aug 14, 2012 20:13

tef: May 30, 2004; -> some l-system crap ->

Suspicious Dish posted:

within the BOM?

did you mean the BMP?

Yes. I'm losing my mind

# ? Aug 14, 2012 21:42

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 16:46

tef: May 30, 2004; -> some l-system crap ->

accipter posted:

Why not integers?

javascript/json only has double precision floating point

# ? Aug 14, 2012 21:51

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »