Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Please don't use pickle in user-facing code! pickle is absolutely insecure and I can hose your computer with it. It uses a bunch of hacks so if you use it with too complex of a Python object, it can fall down and break.

Basically, a pickle object is only valid if you're running the *exact* same code that created the pickle. If you change a class, or even move it around, pickle will fail to load.

# ? Apr 12, 2015 16:47

Adbot: ADBOT LOVES YOU

# ? May 24, 2024 01:12

Hed: Mar 31, 2004; Fun Shoe

I just write out repr() to a file and then run eval() on it :grin:

# ? Apr 12, 2015 16:49

hooah: Feb 6, 2006; WTF?

Suspicious Dish posted:

Please don't use pickle in user-facing code! pickle is absolutely insecure and I can hose your computer with it. It uses a bunch of hacks so if you use it with too complex of a Python object, it can fall down and break.

Basically, a pickle object is only valid if you're running the *exact* same code that created the pickle. If you change a class, or even move it around, pickle will fail to load.

They definitely warn about the insecurity in the library documentation. In my case, this is for my own project for school, so no one else is going to be seeing it.

# ? Apr 12, 2015 16:54

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Same goes for shelve as it also uses pickle behind the scenes. It also yells about the security implications in the docs.

# ? Apr 12, 2015 17:21

evensevenone: May 12, 2001; Glass is a solid.

Nippashish posted:

The file gets closed anyway when python exits with an exception, you don't need a context manager to make that happen. The context manager auto-closing is useful if you catch that exception further up the callstack though.

That code is reading several pickled objects out of a single file.

Context managers are nice if you ever have to write code that actually handles exceptions instead of unceremoniously crashing.

# ? Apr 12, 2015 17:41

Fergus Mac Roich: Nov 5, 2008; Soiled Meat

let's say i have a function that i want to use a particular set of values that i want to generate just once. is it appropriate to do something like this?

Python code:

def outer():
    d = {'x': 3}
    def inner():
	#let's assume I did something useful with an argument and this value instead of just returning it
        return d['x'] 
    return inner

outer = outer()

It makes sense to me, but it seems like maybe I should use a decorator?

# ? Apr 12, 2015 23:17

good jovi: Dec 11, 2000; 'm pro-dickgirl, and I VOTE!

Fergus Mac Roich posted:

let's say i have a function that i want to use a particular set of values that i want to generate just once. is it appropriate to do something like this?
Python code:
def outer():
    d = {'x': 3}
    def inner():
	#let's assume I did something useful with an argument and this value instead of just returning it
        return d['x'] 
    return inner

outer = outer()
It makes sense to me, but it seems like maybe I should use a decorator?

If you're just doing it once, why wrap the function at all?

Python code:

d = {'x': 3}
def outer():
    return d['x']

That said, assuming your example is a gross simplification, then yes, it's fine to build functions that way. It just only really makes sense if the wrapper function is parameterized in some way.

# ? Apr 13, 2015 01:22

Fergus Mac Roich: Nov 5, 2008; Soiled Meat

Yes, perhaps I should have used a more complex example, but both functions would be parameterized.

# ? Apr 13, 2015 01:37

duck monster: Dec 15, 2004

Suspicious Dish posted:

Please don't use pickle in user-facing code! pickle is absolutely insecure and I can hose your computer with it. It uses a bunch of hacks so if you use it with too complex of a Python object, it can fall down and break.

Basically, a pickle object is only valid if you're running the *exact* same code that created the pickle. If you change a class, or even move it around, pickle will fail to load.

Fun fact: Eve Online either does or did use pickle in its network code. Don't ask me how I know or I'll be forced to eat this cyanide tablet! Its pretty much public knowledge

# ? Apr 13, 2015 01:40

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

This is why I emphasize it every single time, because there's somebody who does it even though the docs clearly say never to do it.

Pickle is *not* a safe, secure or stable data storage or wire format.

# ? Apr 13, 2015 01:55

duck monster: Dec 15, 2004

Its spectacularly stupid, and in the case of Eve Online, I believe it was behind various code injection attacks (and the eventual bannings that ensued)

# ? Apr 13, 2015 03:12

hooah: Feb 6, 2006; WTF?

I have a Python proper question and a PyCharm question.

The first: why the hell is Counter capitalized when everything else I've run across so far is lower-case?

The second: occasionally when I try to start PyCharm, I get an error window saying something about JVM and error 4. I looked it up and this can happen because there isn't enough memory for what PyCharm's requesting, so I close pretty much everything, open it successfully, then re-open everything without a problem. How can I avoid going through this rigamarole?

# ? Apr 15, 2015 00:34

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

hooah posted:

I have a Python proper question and a PyCharm question.

The first: why the hell is Counter capitalized when everything else I've run across so far is lower-case?

My guess, without ever having used Counter, is that the answer to these sort of questions is usually: historical baggage.

hooah posted:

The second: occasionally when I try to start PyCharm, I get an error window saying something about JVM and error 4. I looked it up and this can happen because there isn't enough memory for what PyCharm's requesting, so I close pretty much everything, open it successfully, then re-open everything without a problem. How can I avoid going through this rigamarole?

I seem to recall that some memory problems with PyCharm can be fixed by editing the Java startup Xmm or Xms or something parameters. I'm sorry that's not more helpful, but maybe that will give you something to Google.

# ? Apr 15, 2015 00:38

Fergus Mac Roich: Nov 5, 2008; Soiled Meat

Class names are supposed to use CamelCase according to PEP8. I assume that dict and list are lowercase because they're built in types, but why collections.deque is lowercase and collections.Counter isn't is beyond me. nametuple() is a function that produces classes so that one's different.

For another CamelCase example, collections.ChainMap.

# ? Apr 15, 2015 00:40

EAT THE EGGS RICOLA: May 29, 2008

Thermopyle posted:

I seem to recall that some memory problems with PyCharm can be fixed by editing the Java startup Xmm or Xms or something parameters. I'm sorry that's not more helpful, but maybe that will give you something to Google.

I had to do this too, it's a pain in the rear end when my work machine only has 4GB ram.

# ? Apr 15, 2015 03:33

Hughmoris: Apr 21, 2007; Let's go to the abyss!

For those who don't know, there is website to learn a little python and solve some riddles here. A little like project euler except nowhere near as math heavy.

I am stuck on a problem, as this is the only information given:
http://www.pythonchallenge.com/pc/def/linkedlist.html

Anyone suggestions on how to tackle this one?

# ? Apr 15, 2015 03:53

KICK BAMA KICK: Mar 2, 2009

Change the extension in the url as indicated.

I remember the site. It's fun for a while and takes you to some interesting (if maybe not all that useful) parts of the standard library but eventually, instead of Project Euler's math I think it relies on you to do stuff like recognize the hexadecimal headers of various file formats to figure out how to decode whatever they've given you. Hardly beyond the skills of someone more clever than me but came a point when I called it a day and started Googling the solutions just out of curiosity.

# ? Apr 15, 2015 04:03

Hughmoris: Apr 21, 2007; Let's go to the abyss!

KICK BAMA KICK posted:

Change the extension in the url as indicated.

Clearly, I need some sleep.

# ? Apr 15, 2015 04:17

SelfOM: Jun 15, 2010

Is there a way to have variable axes in matplotlib, ie from 1-100 scaled 10:1 and then from 100-110 be scale 1:1 then back to 10:1. I could scale the underlying data, but I want to avoid this.

# ? Apr 16, 2015 15:21

Cingulate: Oct 23, 2012; by Fluffdaddy

I'm not sure I correctly understand the question, but have you tried help(matplotlib.pyplot.axes)?

If the first arg to plt.axes is a tuple of 4 numbers, it's setting the axis size (as left, bottom, width, height).

# ? Apr 16, 2015 15:50

SurgicalOntologist: Jun 17, 2004

Do you have an example of something professionally done that looks like what you're going for? Might give insight into how it was done. Seems like a strange thing to do.

IIUC, I doubt it's possible without getting down and dirty into the internals, maybe subclassing some matplotlib class or another. Another hack would be to use several subplots all right next to each other. Both options seem to be a huge PITA.

# ? Apr 16, 2015 17:09

EAT THE EGGS RICOLA: May 29, 2008

I have a dict of dicts I need to sort based on a key in the dicts. It's kind of like this:

Python code:

{
{"fieldname1": {"name": "name1", "order": 2},
{"fieldname2": {"name": "name2", "order": 1}
}

(obviously much more complicated than that)

I can't use collections.OrderedDict (I'm stuck on 2.6).

Any suggestions on how to do it in a way that doesn't feel like I'm kludging it together?

# ? Apr 16, 2015 18:00

Jose Cuervo: Aug 25, 2004

EAT THE EGGS RICOLA posted:

I have a dict of dicts I need to sort based on a key in the dicts. It's kind of like this:
Python code:
{
{"fieldname1": {"name": "name1", "order": 2},
{"fieldname2": {"name": "name2", "order": 1}
}
(obviously much more complicated than that)

I can't use collections.OrderedDict (I'm stuck on 2.6).

Any suggestions on how to do it in a way that doesn't feel like I'm kludging it together?

Can you sort dictionaries? I thought the idea with dictionaries was to be able to locate things in constant time using the key, and not worrying about 'where' in the dictionary the value is located.

EDIT: In fact it says here that keys and values in a dictionary are located in a random order.

Jose Cuervo fucked around with this message at 18:18 on Apr 16, 2015

# ? Apr 16, 2015 18:15

Dominoes: Sep 20, 2007

Post a better example of your data.

# ? Apr 16, 2015 18:16

OnceIWasAnOstrich: Jul 22, 2006

Python code:

sorted(nested_dict.iteritems(), key=lambda sub_dict: sub_dict[1]['order'])

This gives you a list of dicts, which is really the best you can get without an ordered dictionary class of some kind. There is always pip install ordereddict

# ? Apr 16, 2015 18:16

EAT THE EGGS RICOLA: May 29, 2008

Jose Cuervo posted:

Can you sort dictionaries? I thought the idea with dictionaries was to be able to locate things in constant time using the key, and not worrying about 'where' in the dictionary the value is located.

Right, I don't need a sorted dict, I just need a sorted representation of the data in the dict.

Dominoes posted:

Post a better example of your data.

What's unclear about that data? There's a key in the inner dict that has an integer that corresponds to the ordering.

OnceIWasAnOstrich posted:

Python code:
sorted(nested_dict.iteritems(), key=lambda sub_dict: sub_dict[1]['order'])
This gives you a list of dicts, which is really the best you can get without an ordered dictionary class of some kind. There is always pip install ordereddict

lol locked down and airgapped servers

I think this is about as good as things will be able to get, though, thank you.

# ? Apr 16, 2015 18:20

QuarkJets: Sep 8, 2008

SelfOM posted:

Is there a way to have variable axes in matplotlib, ie from 1-100 scaled 10:1 and then from 100-110 be scale 1:1 then back to 10:1. I could scale the underlying data, but I want to avoid this.

You want to squish the data around another set of data, basically? I don't think that you should do this for many reasons that are non-programmatic, such as it could cause misinterpretation of your results. Make a second graph of the range that you're interested in, you could even plot it in the same figure, just please don't go loving with axes in weird ways like this.

# ? Apr 16, 2015 18:21

EAT THE EGGS RICOLA: May 29, 2008

EAT THE EGGS RICOLA posted:

I have a dict of dicts I need to sort based on a key in the dicts. It's kind of like this:
Python code:
{
{"fieldname1": {"name": "name1", "order": 2},
{"fieldname2": {"name": "name2", "order": 1}
}
(obviously much more complicated than that)

I can't use collections.OrderedDict (I'm stuck on 2.6).

Any suggestions on how to do it in a way that doesn't feel like I'm kludging it together?

aag i also need to sort the records with no "order" key in alpha order, which makes this way more annoying.

# ? Apr 16, 2015 20:20

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

Do items without an order sort before or after those with one? Alphabetical order by what? Field name in the outer dict (e.g. 'fieldname1'), or the value of the 'name' key in the inner one (e.g. 'name1')?

EDIT: whatever, here's stuff for all 4 possibilities

Python code:

d = {
    'fieldname1': {'name': 'name1', 'order': 2},
    'fieldname2': {'name': 'name2', 'order': 1},
    # Note the flip between field name and inner dict name
    'fieldname3': {'name': 'name4'},
    'fieldname4': {'name': 'name3'},
}

def sort_key_order_name_missing_order_last(key_value):
    key, value = key_value
    return value.get('order', float('inf')), value['name']

def sort_key_order_name_missing_order_first(key_value):
    key, value = key_value
    return value.get('order', float('-inf')), value['name']

def sort_key_order_fieldname_missing_order_last(key_value):
    key, value = key_value
    return value.get('order', float('inf')), key

def sort_key_order_fieldname_missing_order_first(key_value):
    key, value = key_value
    return value.get('order', float('-inf')), key

Python code:

>>> from pprint import pprint
>>> pprint(sorted(d.iteritems(), key=sort_key_order_name_missing_order_last))
[('fieldname2', {'name': 'name2', 'order': 1}),
 ('fieldname1', {'name': 'name1', 'order': 2}),
 ('fieldname4', {'name': 'name3'}),
 ('fieldname3', {'name': 'name4'})]
>>> pprint(sorted(d.iteritems(), key=sort_key_order_name_missing_order_first))
[('fieldname4', {'name': 'name3'}),
 ('fieldname3', {'name': 'name4'}),
 ('fieldname2', {'name': 'name2', 'order': 1}),
 ('fieldname1', {'name': 'name1', 'order': 2})]
>>> pprint(sorted(d.iteritems(), key=sort_key_order_fieldname_missing_order_last))
[('fieldname2', {'name': 'name2', 'order': 1}),
 ('fieldname1', {'name': 'name1', 'order': 2}),
 ('fieldname3', {'name': 'name4'}),
 ('fieldname4', {'name': 'name3'})]
>>> pprint(sorted(d.iteritems(), key=sort_key_order_fieldname_missing_order_first))
[('fieldname3', {'name': 'name4'}),
 ('fieldname4', {'name': 'name3'}),
 ('fieldname2', {'name': 'name2', 'order': 1}),
 ('fieldname1', {'name': 'name1', 'order': 2})]

I feel so dirty typing iteritems.

EDIT 2: a bit shorter

Lysidas fucked around with this message at 21:24 on Apr 16, 2015

# ? Apr 16, 2015 21:09

Crack: Apr 10, 2009

Has anything changed in terms of starting out since the OP (last edited 2011)? I'm going to be learning python 2, and I think maybe a combination of a book, MOOC and challenges (e.g. Rosalind).

Is there a go to standard for any of these? I tried a MOOC once and I would really want one without someone waffling for 30 minutes in a video lecture and wasting my time.

I've heard some good things about Learning Python the Hard way and Think Python but I don't know what's really the best resources to use in 2015.

I'll be doing most of my programming in some linux distro (debian or a ubuntu fork quite likely), and as a backup I have a macbook to use. Should I use vim, or what?

# ? Apr 16, 2015 22:38

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Crack posted:

I've heard some good things about Learning Python the Hard way and Think Python but I don't know what's really the best resources to use in 2015.

I haven't heard of anything better than these two.

Crack posted:

I'll be doing most of my programming in some linux distro (debian or a ubuntu fork quite likely), and as a backup I have a macbook to use. Should I use vim, or what?

Of the set of all IDE/editors that people talk about liking in this thread, and without having done any sort of count, my gut tells me that PyCharm probably gets brought up the most.

# ? Apr 17, 2015 03:03

Emacs Headroom: Aug 2, 2003

PyCharm is so much better than anything else it's not even close. There's an VIM plugin that works well for it (even does block select!). Bonus points that if you end up ever trying out JVM languages, IntelliJ (made by the same people with a lot of similar functionality) is the best IDE there.

# ? Apr 17, 2015 05:57

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Emacs Headroom posted:

PyCharm is so much better than anything else it's not even close. There's an VIM plugin that works well for it (even does block select!). Bonus points that if you end up ever trying out JVM languages, IntelliJ (made by the same people with a lot of similar functionality) is the best IDE there.

I use and love both PyCharm and IntelliJ but I was wondering...

Can you load up IntelliJ with plugins to get parity with PyCharm? Or does PyCharm still do stuff that IntelliJ does not?

# ? Apr 17, 2015 06:07

KICK BAMA KICK: Mar 2, 2009

The only beefs I ever have with PyCharm are poo poo that it probably just can't do because of dynamic typing. And even those are mitigated if you take the time to run your scripts in debug mode with the "collect run-time type information" option checked, but go grab a drink cause that's mad slow.

# ? Apr 17, 2015 06:20

accipter: Sep 12, 2003

Crack posted:

Has anything changed in terms of starting out since the OP (last edited 2011)? I'm going to be learning python 2, and I think maybe a combination of a book, MOOC and challenges (e.g. Rosalind).

Is there a go to standard for any of these? I tried a MOOC once and I would really want one without someone waffling for 30 minutes in a video lecture and wasting my time.

I've heard some good things about Learning Python the Hard way and Think Python but I don't know what's really the best resources to use in 2015.

I'll be doing most of my programming in some linux distro (debian or a ubuntu fork quite likely), and as a backup I have a macbook to use. Should I use vim, or what?

Any reason you are going to learn Python2 instead of Python3? If you have the choice, I would recommend Python3 since that is the present and future of the language. However, the differences are relatively minor.

As others have mentioned, PyCharm is really quite nice. If you set on using Vim look at the jedi-vim completion library.

# ? Apr 17, 2015 14:36

namaste friends: Sep 18, 2004; by Smythe

Python 2 is still the default interpreter for a lot of base Linux installs. If you're a computer janitor, this could be important to you.

# ? Apr 17, 2015 14:52

SelfOM: Jun 15, 2010

QuarkJets posted:

You want to squish the data around another set of data, basically? I don't think that you should do this for many reasons that are non-programmatic, such as it could cause misinterpretation of your results. Make a second graph of the range that you're interested in, you could even plot it in the same figure, just please don't go loving with axes in weird ways like this.

People that don't understand variable scaling of an axis isn't the audience I care about. I've done the latter and the graphs already take up too much space.

# ? Apr 17, 2015 14:58

Hed: Mar 31, 2004; Fun Shoe

I bought PyCharm 2.6 a while ago but I didn't like it as much because doing remote environment execution and setup seemed like a ballache but maybe it has gotten better in the later versions (or I wouldn't be too lazy to figure it out)

On my personal projects at home I mainly dick around with Django/DRF sites and keep up with what's hot in front end land (Angular to Ember and now React). Would that still be helpful? It looked like I'd need web storm / IntelliJ to get the JS piece.

Currently my workflow is I develop from either desktop (win) or laptop (OS x) at home and just ssh -> tmux into some panels that have vim and alongside that a prompt / unit test results / whatever. I just really like that it is portable from anywhere but I would like vim to like highlight unused imports and stuff on the Python side.

# ? Apr 17, 2015 16:13

accipter: Sep 12, 2003

Hed posted:

...but I would like vim to like highlight unused imports and stuff on the Python side.

Have you used the syntastic plugin for vim? It will highlight things like that.

# ? Apr 17, 2015 18:08

Adbot: ADBOT LOVES YOU

# ? May 24, 2024 01:12

QuarkJets: Sep 8, 2008

SelfOM posted:

People that don't understand variable scaling of an axis isn't the audience I care about. I've done the latter and the graphs already take up too much space.

Have you tried creating two subplots in the same figure? Or how about a small plot that's a zoomed portion of the plot that you want, like what's done here? There are many ways to have two plots in the same figure size, if you're worried about space

Variable scaling of an axis is such a horror that I feel the need to continue badgering you into doing literally anything else. Please don't try to modify your axis so that the tick mark distances correspond to different quantities

# ? Apr 17, 2015 18:14

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »