Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Dominoes
Sep 20, 2007

accipter posted:

I just wanted to point out Pendulum: http://pendulum.eustace.io/ . It looks like it solves a number of datetime's issues without some of arrow's issues.
Looks cool. I'm partial to the syntax I put in the saturn module; Feels easier to me.

Adbot
ADBOT LOVES YOU

cinci zoo sniper
Mar 15, 2013




Would anyone reasonably familiar with python's (2) struct module mind looking into my problem? I'll post the quote describing it below, sorry for poor formatting and such. Spent better half of this day breaking spears against it at no avail, so likely it's something dumb and obvious.

kalstrams posted:

ffs this struct thing is getting annoying
code:
. [80, 584024144, 1048576, 524288, '=524288h', 1, 706772912, 122748768, 1048576, 122748768]
. [80, 585072720, 1048576, 524288, '=524288h', 1, 706772912, 121700192, 1048576, 121700192]
. [80, 586121296, 1048576, 524288, '=524288h', 1, 706772912, 120651616, 1048576, 120651616]
. [80, 587169872, 1048576, 524288, '=524288h', 1, 706772912, 119603040, 1048576, 119603040]
. [80, 588218448, 1048576, 524288, '=524288h', 1, 706772912, 118554464, 1048576, 118554464]
Traceback (most recent call last):
  File "script.py", line 128, in <module>
    real=real, datafile=datafile)
  File "/path/codebase.py", line 125, in binary
    data.append(np.array(unpacker.unpack(storage.read(line_bytes)), dtype=np.float32))
struct.error: unpack requires a string argument of length 1048576
so, in that list that's being printed, we have
header in bytes, pointer in bytes, data size to read in bytes (line_bytes), number samples to read (line_samples), sample format (form), irrelevant variable, effective size of the current file in bytes, unread amount of data in the current file in bytes, irrelevant variable, and effective remaining amount of data in bytes

code where it breaks
code:
storage.seek(header + pointer)
data.append(np.array(unpacker.unpack(storage.read(line_bytes)), dtype=np.float32))
where
code:
form = format % (line_samples)
unpacker = struct.Struct(form)
and
code:
storage = io.open(self.source[datafile], 'rb', buffering=0)
for the life of me i cant figure out why it seemingly randomly has broken at that point, if, as you see from earlier in the log (new line = successful read) - it did work just fine before that
Basically, I have a method inside class that reads numerical data from binary files. In the executable script there are two for loops, outer that initialises sets of input arguments, and inner that runs bunch of methods in the workflow.

I'm struggling to understand the problem since I can't easily reproduce it on the machine where I am able to use debugger. I can get the exact same error on the exact same data, but it's on a remote workstation, and my personal computer might not be able to handle this exact computation in the event that I had data in situ.

QuarkJets
Sep 8, 2008

kalstrams posted:

Would anyone reasonably familiar with python's (2) struct module mind looking into my problem? I'll post the quote describing it below, sorry for poor formatting and such. Spent better half of this day breaking spears against it at no avail, so likely it's something dumb and obvious.
Basically, I have a method inside class that reads numerical data from binary files. In the executable script there are two for loops, outer that initialises sets of input arguments, and inner that runs bunch of methods in the workflow.

I'm struggling to understand the problem since I can't easily reproduce it on the machine where I am able to use debugger. I can get the exact same error on the exact same data, but it's on a remote workstation, and my personal computer might not be able to handle this exact computation in the event that I had data in situ.

Unpack is being given an argument of the wrong length. Try running that long set of calls line-by-line and check that the object being returned by storage.read(line_bytes) is actually line_bytes in length.

cinci zoo sniper
Mar 15, 2013




QuarkJets posted:

Unpack is being given an argument of the wrong length. Try running that long set of calls line-by-line and check that the object being returned by storage.read(line_bytes) is actually line_bytes in length.
Beats me how that could be possible but at least now I know where and how to look. Cheers!

yippee cahier
Mar 28, 2005

kalstrams posted:

Beats me how that could be possible but at least now I know where and how to look. Cheers!

Not sure about your file format, but one reason could be your seek statement... Do you want to skip the pointer or to where your pointer is pointing?

cinci zoo sniper
Mar 15, 2013




yippee cahier posted:

Not sure about your file format, but one reason could be your seek statement... Do you want to skip the pointer or to where your pointer is pointing?
Data is a continuous stream of numbers. My understanding of seek statement is that next read operation will begin with the first byte after the pointer, so I've written the code in a way that pointer equals to an integer amount of data read during the previous cycle added to the old pointer, since I often have to introduce artificial gaps in the data read to cope with massive amounts of data available in a reasonably timely fashion.

Tigren
Oct 3, 2003
I'd be willing to call myself an advanced beginner Python programmer, and I just completed a small project that someone actually paid me for!

Iterating through the project made me realize I need to start implementing better testing into my design. I've already completed the project and I want to go back and build it again using TDD.

For the life of me I can't figure out where to start. Does anyone have any good reads or tutorials on test-driven design? I've found a couple that show TDD with Django, but I'm not really looking to get into the whole Django workflow while learning TDD.

onionradish
Jul 6, 2006

That's spicy.

Tigren posted:

For the life of me I can't figure out where to start. Does anyone have any good reads or tutorials on test-driven design? I've found a couple that show TDD with Django, but I'm not really looking to get into the whole Django workflow while learning TDD.
If you're ok with "watching" rather than just reading, PyCon hosts all their conference videos on YouTube. Here are the test-specific videos from the last couple years (excluding Django/webapps). Note that some are long hands-on sessions and some are just presentations. Ned Batchelder would be a good starting point

PyCon 2014

PyCon 2015

PyCon 2016

Emmideer
Oct 20, 2011

Lovely night, no?
Grimey Drawer
This is a pretty basic question, but can I get a good tutorial to help me understand in the ins and outs of classes in python? Something about them makes my eyes bounce off them and brain refuse to understand how they work, no matter how many pages about them that I google and read. This is despite understanding the concept of objects (or so I think).

QuarkJets
Sep 8, 2008

jon joe posted:

This is a pretty basic question, but can I get a good tutorial to help me understand in the ins and outs of classes in python? Something about them makes my eyes bounce off them and brain refuse to understand how they work, no matter how many pages about them that I google and read. This is despite understanding the concept of objects (or so I think).

Is this your first time using classes in general, or just classes in python?

Emmideer
Oct 20, 2011

Lovely night, no?
Grimey Drawer

QuarkJets posted:

Is this your first time using classes in general, or just classes in python?

For functional purposes, let's say in general, but I'm interested in python specifically. I've used them before in various online courses, some python and some not, but I just can't wrap my head around writing my own from scratch from an idea in my head. It just feels like there's so much going on, and trying to read examples of them is resistant to my attempts to understand.

tef
May 30, 2004

-> some l-system crap ->
i can tell you a few unhelpful lies, but the nature objects is finicky when you consider EOL'd python (2.7 and below), so i'll cover how objects work in python 3 and new style objects in python 2. just use class Foo(object): in python 2, don't ask.

so, in python we have dictionaries.

we can get from them: del d["key"], we can set them d["key"]=1 and we can get them d["key"]

objects sorta work the same way, they're kinda like dictionaries, except we use foo.bar instead of foo["bar"], and python does *magic* behind the scenes when you lookup an object's attribute

code:
>>> class Example:
...     pass
... 
>>> e = Example()
>>> e.a = 1
>>> e.a
1
>>> del e.a
>>> e.a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Example' object has no attribute 'a'
>>> 
we can assign functions to instances too:

code:
>>> def same(x):
...     return x
... 
>>> e.same = same
>>> e.same(1)
1
although I said foo.bar does magic in python, it doesn't really do it for instances.

instances are really just a wrapper around a class, an a dictionary for attributes

code:
>> e.__class__
<class '__main__.Example'>
>>> e.__dict__
{'same': <function same at 0x104b33730>}
>>> 
for instances, foo.bar is really foo.__dict__['bar']

if python can't find 'bar' in the instance dictionary, it looks it up in the class instead, but classes are where the magic starts to happen for foo.bar

code:
>>> class Example:
...     a = 1
... 
>>> Example.a
1
>>> Example.__dict__
mappingproxy({..... some poo poo that's too long but it's a dict ok ? .... })
>>> e = Example()
>>> e.a
1
>>> e.a = 2 # We'd want to do Example.a = 2 to set that one
>>> e.__dict__
{'a': 2}
>>> e.a
2
>>> Example.a
1
>>>
in python, classes are just templates for instances, which contain functions.

code:
>>> class Another:
...    def example(self): 
...        return "nice"
... 
>>> a = Another()
>>> a.example()
'nice'
>>> Another.example
<function Another.example at 0x104b33840>
>>> def example2(self, a):
...     return a
... 
>>> Another.example2 = example2
>>> a.example2(1)
1
when you call foo.bar, if python finds a bar function in the class, it binds it - it returns a new function, calling the original, but passing foo as self.

you can think of foo.bar being partially appliying foo.__class__.bar(self=foo), if you want to, i guess

code:
>>> Another.example2
<function example2 at 0x104b338c8>
>>> a.example2
<bound method Another.example2 of <__main__.Another object at 0x104bc1fd0>>
>>> 
this is actually a huge difference between python and many of it's peers: in ruby, you always call a method, in javascript, if you lookup the method you must bind it manually

there is no special this or self keyword, python's self is just named thay by convention, and it's a lexically scoped variable too, so no more "var that=this" as seen in other languages

recap:

so, objects all wrap some dictionary (o.__dict__) which stores attributes.

when you call foo.bar, you're looking up bar inside foo's dictionary

when you lookup an attribute of a class directly, or lookup something inside an instance, no real magic is performed

classes are just wrapped dictionaries of functions and values too.

when an instance calls up to the class to find an attribute, it perfoms binding, turning the function into a bound method - where the first argument is the instance itself


i'm too tired and i dunno how accurate my memory is but from here you can start to explore the metaobject protocol: a class with __getattr__/__setattr__ methods can start to change the behaviour of `foo.bar`

it is worth shouting out to descriptors, which allow you to overwrite foo.bar by setting methods on bar, not on foo.

tef
May 30, 2004

-> some l-system crap ->
there are plenty of places where this is not true but it's a reasonable set of lies to go on i hope

Dominoes
Sep 20, 2007

jon joe posted:

This is a pretty basic question, but can I get a good tutorial to help me understand in the ins and outs of classes in python? Something about them makes my eyes bounce off them and brain refuse to understand how they work, no matter how many pages about them that I google and read. This is despite understanding the concept of objects (or so I think).

Forget objects - you don't need them!

QuarkJets
Sep 8, 2008

It's true that you don't necessarily need them but objects be useful sometimes

tef
May 30, 2004

-> some l-system crap ->
there's a reason i didn't cover super.

Emmideer
Oct 20, 2011

Lovely night, no?
Grimey Drawer

Dominoes posted:

Forget objects - you don't need them!

That's another thing I don't get. Do objects really do anything besides organize code and change the syntax? I like to be able to go dictionary[dictionary][thing], can I go class.class.thing? Do I -want- to go class.class.thing? Will a function I call from a class be different from a function I call from a dictionary? Can I even pass variables into class.class.thing like I can dictionary[dictionary][thing], ala class.class.x, x = thing? And what's the deal with airline food?


All signs point to no, and also airlinefood.deal points to crashing the IDE.

Emmideer fucked around with this message at 03:20 on Jul 16, 2016

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Objects are useful some percentage of the time.

a large part of the rest of the time they don't really bring anything to the party, but they don't hurt either unless you don't understand them.

The rest of the time they actively make code worse.

Emmideer
Oct 20, 2011

Lovely night, no?
Grimey Drawer

Thermopyle posted:

Objects are useful some percentage of the time.

Could you give some examples of when and why this would be?

Thermopyle posted:

The rest of the time they actively make code worse.

Examples of this would may also be interesting, if only for entertainment purposes.

SurgicalOntologist
Jun 17, 2004

jon joe posted:

Examples of this would may also be interesting, if only for entertainment purposes.
Obligatory
https://www.youtube.com/watch?v=o9pEzgHorH0

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

jon joe posted:

That's another thing I don't get. Do objects really do anything besides organize code and change the syntax?

There are a few extra features, like inheritance, but otherwise, nope! They're just organizing code and changing the syntax.

jon joe posted:

I like to be able to go dictionary[dictionary][thing], can I go class.class.thing? Do I -want- to go class.class.thing? Will a function I call from a class be different from a function I call from a dictionary? Can I even pass variables into class.class.thing like I can dictionary[dictionary][thing], ala class.class.x, x = thing?

You can do that if you want, but some people consider more than two dots like that to be a sign of bad style. The interesting thing about functions you call from classes is that they will automatically have the class passed to them. For instance, if you do this right now:

Python code:
foo = {}

def airline_food(my_dict, the_deal):
    if the_deal > 10:
        return my_dict['chicken']
    else:
        return my_dict['fish']

foo['airline_food'] = airline_food
foo['chicken'] = "yes, i remember, i had the lasagna"
foo['fish'] = "don't call me shirley"
Then you have to call it like foo['airline_food'](foo, 15). That is, you have to pass the dictionary in twice. With classes, foo.airline_food(15) will automatically pass in your class. This is very convenient when trying to make clean, structured "building blocks" for your code.

It might not seem like much to you right now. That's OK. You don't have to use classes any time soon. Come back in three months and you might understand better.

SurgicalOntologist
Jun 17, 2004

You technically can't do dictionary[dictionary][thing] because dictionaries aren't hashable. But I don't think you really intended to do that, you just meant dictionary[thing][another_thing], so I apologize for the pedantry.

Emmideer
Oct 20, 2011

Lovely night, no?
Grimey Drawer

SurgicalOntologist posted:

You technically can't do dictionary[dictionary][thing] because dictionaries aren't hashable. But I don't think you really intended to do that, you just meant dictionary[thing][another_thing], so I apologize for the pedantry.

yeah, where thing in dictionary[thing][another_thing] is a key pointing to dictionary containing the key of another_thing

I think

If this is also wrong I'm sorry

SurgicalOntologist
Jun 17, 2004

No that's not wrong. Nested dictionaries are a thing. I was just making sure you weren't asking about passing a dictionary to itself.

QuarkJets
Sep 8, 2008

It's also interesting to note that everything in Python is an object.

Python code:
x = 5+4j
print(x.imag)
print(x.__add__)
So pedantically speaking, you do need objects in order to use Python. What you don't really need is to define your own classes. It can occasionally be useful for organizing your code in a more logical way. It's really going to be a mix of personal preference on a case-by-case basis

Dominoes
Sep 20, 2007

jon joe posted:

Examples of [OOP making things worse] would may also be interesting, if only for entertainment purposes.
https://github.com/Jefferson-Henrique/GetOldTweets-python/tree/master/got/manager

https://github.com/crsmithdev/arrow/pull/327/files


Technique-only- Times I use objects:
Something that can be thought of concretely as an object in the world
Something that's like a new datatype
I have a pile of related functions that take a large number of args.
An API requires them, like GUIs or ORMs.
NamedTuples are nice and have object-like syntax.

If a class ends in 'er', watch out!

Dominoes fucked around with this message at 11:21 on Jul 16, 2016

SurgicalOntologist
Jun 17, 2004

Just a quick PSA that the talks at Scipy 2016 are going up on YouTube right away and seem to be unusually good.
Not that they're not usually good. Just that I'm more impressed than usual. JupyterLab(https://www.youtube.com/watch?v=Ejh0ftSjk6g) looks awesome.

I know we have some posters there who may also be giving talks, godspeed!

Emmideer
Oct 20, 2011

Lovely night, no?
Grimey Drawer
This is going to sound rather silly, but how do I do this:

code:
for (player1, action1), (player2, action2) in action_dict:
Where action_dict is a dictionary containing two (key, value)s, and I need to use all that information within the same function?

edit: removed .items() because apparently that works different in python 3.5 which is what I am using. Still same problem, though.

editedit: Got what I wanted. Behold my terror:

code:
temp_dict = action_dict
player_action1 = temp_dict.popitem()
player_action2 = temp_dict.popitem()
player1 = player_action1[0]
action1 = player_action1[1]
player2 = player_action2[0]
action2 = player_action2[1]
There has to be a more pythonic way to do it, though...

Emmideer fucked around with this message at 17:01 on Jul 16, 2016

SurgicalOntologist
Jun 17, 2004

There's no actual loop because you're only doing an operation once. The operation is "tuple unpacking". Pretty sure this should work in 3.5:

Python code:
(player1, action1), (player2, action2) = action_list.items()
which is short for
Python code:
item1, item2 = action_list.items()
player1, action1 = item1
player2, action2 = item2
Finally, for your own sanity I would recommend not using _list in the variable name for a dictionary.

Emmideer
Oct 20, 2011

Lovely night, no?
Grimey Drawer

SurgicalOntologist posted:

There's no actual loop because you're only doing an operation once. The operation is "tuple unpacking". Pretty sure this should work in 3.5:

Python code:
(player1, action1), (player2, action2) = action_list.items()
which is short for
Python code:
item1, item2 = action_list.items()
player1, action1 = item1
player2, action2 = item2
Finally, for your own sanity I would recommend not using _list in the variable name for a dictionary.

Oh that's much better than my horrible solution

(and yeah, I figured out _list was a terrible idea, haha)

Emmideer
Oct 20, 2011

Lovely night, no?
Grimey Drawer
Which is better:

code:
dict = {}
def populate(key, tuple):
    if key in dict:
        dict[key][tuple[0]] = tuple[1]
    else:
        dict[key] = {}
        dict[key][tuple[0]] = tuple[1]
vs

code:
dict = {}
def populate(key, tuple):
    try:
        dict[key][tuple[0]] = tuple[1]
    except KeyError:
        dict[key] = {}
        dict[key][tuple[0]] = tuple[1]

SurgicalOntologist
Jun 17, 2004

jon joe posted:

Which is better:

code:
dict = {}
def populate(key, tuple):
    if key in dict:
        dict[key][tuple[0]] = tuple[1]
    else:
        dict[key] = {}
        dict[key][tuple[0]] = tuple[1]
vs

code:
dict = {}
def populate(key, tuple):
    try:
        dict[key][tuple[0]] = tuple[1]
    except KeyError:
        dict[key] = {}
        dict[key][tuple[0]] = tuple[1]

Not bad solutions. But I think you'll be pleased to learn about defaultdict.

Python code:
from collections import defaultdict

nested_dict = defaultdict(dict)
nested_dict['outer']['inner'] = 72 
Basically, when a new item is accessed it automatically runs the default that was passed in. Here, dict. So, it sets nested_dict['outer'] = dict().

\/\/ Agreed.

SurgicalOntologist fucked around with this message at 05:17 on Jul 17, 2016

Space Kablooey
May 6, 2009


jon joe posted:

Which is better:

code:
dict = {}
def populate(key, tuple):
    if key in dict:
        dict[key][tuple[0]] = tuple[1]
    else:
        dict[key] = {}
        dict[key][tuple[0]] = tuple[1]
vs

code:
dict = {}
def populate(key, tuple):
    try:
        dict[key][tuple[0]] = tuple[1]
    except KeyError:
        dict[key] = {}
        dict[key][tuple[0]] = tuple[1]

If you don't want to go with defaultdict (why wouldn't you), it think the snippet below is nicer.

Python code:
def populate(key, tuple):
    if key not in dict:
        dict[key] = {}
    dict[key][tuple[0]] = tuple[1]

Emmideer
Oct 20, 2011

Lovely night, no?
Grimey Drawer
Aha, yes, default dict does make it much shorter/intuitive/(faster?). Though, interesting enough, it also does this:

defaultdict(<class 'dict'>, {})

Which, I'm not sure the full implications of.

SurgicalOntologist
Jun 17, 2004

That's just defaultdict's string representation. In this case it's saying "I'm a defaultdict, my default is value is the class 'dict', and I'm empty". Try adding some items and looking at it again, and try other defaults as well, e.g. defaultdict(list), to get a better sense of what it's showing you there.

Dominoes
Sep 20, 2007

Any word on if Tensorflow is coming to Windows? Docker's a PITA I couldn't get working.

At the bottom of the last page, I posted a library I'm working on for plotting functions; a thin wrapper for matplotlib that has easy syntax and sensible defaults. Any suggestions? It's helping me visualize things from Khan Acad's multivariate calc class, and I'd like to get it in a state where others can use it. Need feedback since I've only tailored it to my use case, and I'm not sure what others might use it for that I haven't covered.


Here's a specific one: Is there a way to draw a 2d graph on the axis of a 3d one? Like have a surface plot, with its direction derivative vector field on the x/y plane?

Would it be weird having a general plotting function that analyzes the number of inputs and outputs of your function-to-plot, then automatically figures out a type of plot to draw? Or is that too 'arrow.get' ? (ie, if there's 1 in, 1 out, draw a 2d graph; 2 in 1 out, draw a surface or contour, 2 in 2 out, draw a vector field.)

Dominoes fucked around with this message at 11:41 on Jul 17, 2016

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

defaultdict is the right answer, but you should google LBYL and EAFP and their pros/cons.

Dominoes
Sep 20, 2007

Thermopyle posted:

LBYL and EAFP and their pros/cons.
I have no preference on this - it's a case-by-case basis, and [usually] doesn't matter much.

QuarkJets
Sep 8, 2008

Hey so I was updating my anaconda's pyside today and anaconda appears to have completely hosed itself; the Python and conda binaries are completely gone. Anyone else ever have this issue (on a centos system)?

Adbot
ADBOT LOVES YOU

BigRedDot
Mar 6, 2008

QuarkJets posted:

Hey so I was updating my anaconda's pyside today and anaconda appears to have completely hosed itself; the Python and conda binaries are completely gone. Anyone else ever have this issue (on a centos system)?

I haven't heard of or seen anything like that. Feel free to shoot me details in a PM if you like, and I will try to see if anyone here has anything to say.

Edit:

My SciPy tutorial is up!

https://www.youtube.com/watch?v=LXLQTuSSKfY

Speaking of JupyterLab, I am so infinitely much more excited by it already than I have ever been by the notebook. The cross-org team working on it is pretty phenomenal.

BigRedDot fucked around with this message at 16:01 on Jul 19, 2016

  • Locked thread