Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›6 »

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Drunk Badger posted:

I think I need to add some more information now that I have a better idea what I'm doing.

In each function, that function returns a random number itself. So what I have is a random number deciding what function gets called that produces another random number. So function1 randomly returns it's own set of random numbers, function2 randomly returns a different set of numbers, etc.

While the examples do produce a result, I'm finding that every function ends up returning the same result as it's called. function1 returns 1 the entire time on one run of the script, and always returns 2 on another run of the same script.

It seems to run the function once, save that value as a permanent result, and just return that number. Instead, I want it to run the function and possibly give me something different.

I'm not sure what it is you're trying to work towards, but here is something silly I put together quickly. Maybe it will help you, IDK.

Python code:

import random

def functionmaker (myfunction):
    def returnvalue ():
        return myfunction(random.random())
    return returnvalue

def f0 (x):
    return 0 if x < 0.2 else 1 if x < 0.4 else 2

def f1 (x):
    return 0 if x < 0.5 else 1 if x < 0.75 else 2

def f2 (x):
    return 0 if x < 0.3 else 1 if x < 0.8 else 2

funcs = list(map(functionmaker, [f0, f1, f2]))

def silly ():
    y = 0
    for i in range(10):
        mylist = []
        for j in range(10):
            y = funcs[y]()
            mylist.append(str(y))
        print(' '.join(mylist))

# ¿ Feb 14, 2013 04:10

Adbot: ADBOT LOVES YOU

# ¿ May 9, 2024 00:58

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

The notion of having an "official style guide" is awful. The idea that you're not free to adopt your own set of formatting conventions for your project, no matter that they're consistent and work well for you, because some busybodies decided to codify what they think everyone else's code should look like.

# ¿ Feb 15, 2013 02:13

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Thermopyle posted:

I agree if you're not doing open source work.

Why on Earth would that make a difference to anything?

I think if I were to have read PEP 8 and discovered that quite by coincidence my preferred formatting conventions matched its prescriptions to a tee, I should have chosen to alter key elements so that they no longer matched, purely out of principle.

# ¿ Feb 15, 2013 02:19

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Lysidas posted:

What is wrong with you?

A lot of the responses to what I posted (or comments that may have been partially prompted by what I wrote) have been along the lines of "look guy, you aren't obliged to follow the suggestions in PEP 8. They're a default for people to start out with". Or similar. If that's all it was then I wouldn't have a problem with it. But that's not all it is, because as soon as you endorse a default style you get people saying snarky things like

xtal posted:

Special cases aren't special enough to break the rules.

My problem isn't with anyone who really does just see it as a suggestion. My problem is that people think it's a set of rules and that it's wrong to "break" those rules (as opposed to them merely being guidelines that you can disregard if you like). If you're not one of those people then great.

Maybe my post was immature, but I guess I have a streak of bloody-mindedness that prompts me to resent being told I have to format my code in a certain way when I like another way better!

# ¿ Feb 15, 2013 05:12

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Innocent Bystander posted:

Can someone explain to me the way that Python variables behave in closure like situations?
code:
def function():
    myLat = []
    a = 1
    myLat.append(lambda: someFunc(a))
    myLat.append(lamdda: someOtherFunc(a))
    a = 2
    return myLat
When this gets called, someFunc and someOtherFunc get called with a = 2, rather than a = 1, and I consider my understanding of when Python uses pointer-like variable and when it doesn't my weak suit. Anyone care to explain?

It looks like the functions stored in myLat evaluate their return values when called, not when defined. Since the variable "a" referenced in each function is still needed, a reference to that variable sticks around, and of course it has the last value it was given, which is 2.

It looks like there is an accepted way to get Python to accept the value of "a" that was in place at the time the function was defined. You declare it as an argument with a default value. The default value is evaluated at function definition time. So your function would become

code:

def function():
    myLat = []
    a = 1
    myLat.append(lambda a = a: someFunc(a))
    myLat.append(lamdda a = a: someOtherFunc(a))
    a = 2
    return myLat

which has the same effect as

code:

def function():
    return [lambda a = 1: someFunc(a), lamdda a = 2: someOtherFunc(a)]

# ¿ Feb 15, 2013 06:29

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Suspicious Dish posted:

No, it's:

Oh, yeah you're right. It was late

# ¿ Feb 15, 2013 11:54

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

JetsGuy posted:

I still don't understand why. What is the advantage of doing:

"if not os.path.exists(sys.argv[1])"

instead of

"if os.path.exists(sys.argv[1]) is False"

or

"if os.path.exists(sys.argv[1]) == False"

Is this an elegance thing or is this something about how the guts of Python works? I don't see an advantage over any of those short of "it's tradition". "It's tradition" is a perfectly acceptable answer too, Lord knows its often the answer to why we do things in astro.

I don't really know what the Pythonesque (TM) answer is but to me saying "if not <thing>" does a better job of expressing your intentions than "if <thing> == False" (or "is"). The "not" version makes it clear (in part because it reads like English) that you're checking for a negative response; whereas the "False" version ties us up in detail (that a negative response is given in the form of the constant False). Of course there are much worse ways you can obfuscate what code is doing but it's the same sort of thing, just on a trivial level.

Edit: Suspicious Dish's post suggests a related point, which is that you wouldn't write "if x == True:"; you'd just write "if x:". By analogy you shouldn't write "if x == False:", just "if not x:".

Hammerite fucked around with this message at 22:16 on Mar 4, 2013

# ¿ Mar 4, 2013 22:12

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

JetsGuy posted:

EDIT:
Here's a dumb example of usage of lambda that's not a mathematical function. I pulled it from my plotter program that I made so that I can keep matplotlib separate from my data analysis routines.
code:
    master_data = col.OrderedDict(sorted(master_data.items(),
                                         key=lambda k: k[0]))
EDIT2:
col is collections

code:

master_data = col.OrderedDict(sorted(master_data.items(), key = operator.itemgetter(0)))

edit: bastard

# ¿ Mar 11, 2013 22:22

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Love Stole the Day posted:

Hey again,

Back on page 247 or whatever I asked about a library or module or something for me to expedite the process of making an IRC bot.

I decided to go with "ircutils": http://dev.guardedcode.com/docs/ircutils/index.html

It was exactly what I was looking for and it's a very small, well-enough documented library and a good enough tutorial to do what you need to do.

Just wanted to share here in case anyone from the future searches this old post of mine here looking for how to make an IRC bot.

PS: Hi, future goon!

I have a project that I kind of go back to every often that uses this too, I agree it gives you a nice way to build simple IRC bots. Watch out if Python3 support matters to you though, it isn't compatible and doesn't appear to be actively developed any longer.

# ¿ Mar 15, 2013 02:56

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Dominoes posted:

Is there a way to procedurally change a function based on variables, like in a loop? The best way I can describe it is with psuedocode:
Python code:
for n in range(:10):
            self.ui.checkBox_%s.stateChanged.connect(self.check_%s) % (n, n)
In this example, I'd like to create signals for 10 QT checkboxes, without writing each one out.

There's probably a better way to do this, and I'm tired or I would think about it and suggest one. But yes you can do that using __getattribute__.

Python code:

for n in range(:10):
    ( self.ui
          .__getattribute__('checkBox_%s' % n)
          .stateChanged
          .connect(self.__getattribute__('check_%s' % n))
          )

Totally untested, so there is probably a mistake in there that stops it working.

# ¿ Apr 10, 2013 01:53

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

yaoi prophet posted:

To expand, what it will actually do is evaluate as
Python code:
(foo == 1) or 2 or 3
which will obviously always be True.

It will be either True (if indeed foo == 1) or 2 (which is a "true" value in a Boolean context) (otherwise).

# ¿ Apr 28, 2013 23:05

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Popper posted:

Even if foo is not 1 it will return True, it's completely useless.

in on the other hand is the keyword you want.

No, you have not understood what I said. If foo != 1 then it will return 2, which is not True, although it is a "true" value in a Boolean context.

code:

Python 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> x = 4
>>> y = (x == 1) or 2 or 3
>>> y
2
>>> y is True
False
>>> x = 1
>>> y = (x == 1) or 2 or 3
>>> y
True
>>> y is True
True
>>>

# ¿ Apr 29, 2013 15:32

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

You can encode a set of Booleans as a number using something like this, though there might often be a better way of doing whatever you are trying to do:

code:

return self.MyBoolean + 2 * self.MyOtherBoolean + 4 * self.YetAnotherBoolean + 8 * self.AndSoOn

# ¿ Apr 29, 2013 21:34

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

If all else fails, you can just write scripts and run them from the Python interactive command-line interpreter using import. You will want to add the directory where you are saving the scripts to your PYTHONPATH so that Python can find them. (On my version of Windows I can do this by going Control Panel -> Classic View -> System -> Advanced system settings -> Advanced -> Environment Variables and adding/editing the PYTHONPATH variable under "System variables".)

# ¿ May 1, 2013 21:08

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

You are already using the Python interactive interpreter, you are not using the OS's command line. You need to type Python commands, not commands that would be understood by Windows's command line. So to run the code in hello.py you need to type "import hello.py"

"python hello.py" is not valid Python syntax, which is why you see that error.

# ¿ May 1, 2013 21:44

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

I realised that the correct version of the command is "import hello" (without the .py). However, I do not know whether that is the cause of your most recent error message. Try "import hello" and see whether that works better.

# ¿ May 1, 2013 21:51

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

dantheman650 posted:

Well, no errors this time! However, it also had no output of any sort.

This is probably because you entered the command in the same command-line window as before, and the hello module had been imported already. Once a module has been imported once, importing it again only serves to define variable names in the namespace in which the import statement appears. If you use "reload hello" you will see output, because that re-executes the script in full.

edit: You should probably not get used to using reload, because once you are writing larger scripts that themselves import other scripts of yours, using reload will only re-execute the named script, and not the others nested inside. This is a downside to using the interactive command line.

Hammerite fucked around with this message at 22:02 on May 1, 2013

# ¿ May 1, 2013 21:56

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

dantheman650 posted:

Are you talking about the Windows command prompt or the cmd.py that is in my Python intall directory?

If the former, I get an error that it is not recognized as an intrernal or external command. If the latter, it pops open for a second and then immediately closes.

A friend I'm chatting with recommended changing my script to a function:
def hi():
print ("Hello")
Then in the terminal doing:
import hello
hello.hi()

This also did not work, just for the record, and produced "AttributeError: 'module' object has no attribute 'hi'"

Use the [ code ] tags when posting code, because otherwise people who read your post may not see spacing correctly. This is particularly important for Python because whitespace is syntactically significant.

Did you close and reopen the interpreter, or use reload, as I mentioned?

You might be getting that error from the Windows command line because Python is not in your PATH and so Windows can't find it. Follow the same steps you used to edit the PYTHONPATH variable but this time change the PATH variable (it might be capitalised differently) by adding the path to the directory where python.exe is at the end, after a semicolon.

# ¿ May 1, 2013 22:09

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

I feel like this is the sort of thing I'm going to slap myself on the forehead for not remembering by myself, but suppose I have a dictionary and I want to transform the values of that dictionary in some way while keeping their keys the same... how do I do that? I mean, obviously I can come up with a way to do that using a loop, but what's the idiomatic way?

What I want to do is make a dictionary whose values are tuples to be passed to a class constructor, and I then want to run the class constructor across the dictionary so as to produce a dictionary of objects of the class.

Python 3 if it matters.

Python code:

mydict = {
    'a': (2, 'cat', 42),
    'b': (3, 'dog', 420),
    'c': (5, 'mouse', 1234),
    'd': (7, 'guinea pig', 5678),
}

DO_SOMETHING(MyClass, mydict)

What I'm looking for is something basically similar to map(), but of course map() returns an iterable "map object", and I want to preserve the keys of my dictionary.

# ¿ May 7, 2013 03:19

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

I see, I felt sure there must be a builtin that does this, but I guess I was mistaken.

The following is what I have:

Python code:

def dictArgsMap (myFunction, myDict):
    for item in myDict:
        myDict[item] = myFunction(*myDict[item])

But I suppose I could go the extra mile and allow it to cope with dictionaries, as well.

Python code:

def dictArgsMap (myFunction, myDict):
    for item in myDict:
        if isinstance(myDict[item], dict):
            myDict[item] = myFunction(**myDict[item])
        else:
            myDict[item] = myFunction(*myDict[item])

Does this seem reasonable? I don't like that "isinstance" very much.

edit: less misleading variable names

Python code:

def mutatingStarmap (myFunction, myCollection):
    for item in myCollection:
        if isinstance(myCollection[item], dict):
            myCollection[item] = myFunction(**myCollection[item])
        else:
            myCollection[item] = myFunction(*myCollection[item])

Hammerite fucked around with this message at 04:01 on May 7, 2013

# ¿ May 7, 2013 03:46

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Lysidas posted:

Hammerite, if you want to update your dict in place there probably isn't much room for improvement in your functions/loops. A few suggestions though: you may as well directly loop over key, value in my_dict.items() and you should check for isinstance(value, collections.abc.Mapping) instead of dict.

I concluded that I had no need to update the dict in place after all. So I ended up with more or less what Dren suggested in the first place. I also decided I don't really need that code to cope with dictionaries right now, so it's out.

Python code:

def dictStarmap (myFunction, myCollection):
    return {k: myFunction(*v) for (k, v) in myCollection.items()}

But if I decided I did want to handle dictionaries as well, I could do:

Python code:

from collections.abc import Mapping as MappingABC

def dictStarmap (myFunction, myCollection):
    return {
        k: myFunction(**v) if isinstance(v, MappingABC) else myFunction(*v)
        for (k, v) in myCollection.items()
    }

I'm confused about this collections.abc business though. This module has a dot in the name?? Python 3.2 seemed to find this very confusing, but I installed 3.3 and it loves it.

# ¿ May 7, 2013 08:43

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Master_Odin posted:

Yeah, I just used self.deckStart[:] which makes a copy as I understand it. I guess I'm a bit confused as to when to know when something is passed by reference and when it's not. Running:
code:
a = 0
def update(temp):
    temp += 1

update(a)
print(a)
So is it just a list thing where it gets passed by reference while ints and strings and other basic types are just passed regularly?

I feel like I really should read a solid book on Python instead of haphazardly learning it.

Everything is passed by reference, absolutely everything. But ints and strings are immutable - can't be changed. Once you have a value of 4, that's it; that 4 can't be changed (to 5 say), though it can be replaced by a new value 5.

The reason why += changes a list in place whereas for your integer it acts like "temp = temp + 1" is that the behaviour of += is dependent on the type on its left. It behaves in the one way for lists and in the other for ints, strings and so on. This might seem confusing at first and almost like "cheating", but in fact you can set the behaviour of += for your own types however you want (by defining the __iadd__() method).

# ¿ May 9, 2013 08:45

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

tef posted:

It's more accurate to say that python uses call by object semantics. Every object is stored on the heap, and variables contain a reference to that object. When you pass in a list to a function, the reference is copied.

Yeah, I knew I was most likely not using the correct terminology.

I hoped to just get across the idea of how Python behaves and what's going on with Master_Odin's code.

# ¿ May 10, 2013 03:29

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

BeefofAges posted:

Even if you import numpy twice, it's still just happening twice while loading your program. This shouldn't have any noticeable speed impact. It's not like you're reimporting numpy over and over in a loop.

Correct me if I'm wrong, but the second time aren't you just adding a name to the namespace of the module you're in at the time? That's essentially no work.

# ¿ May 13, 2013 16:49

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Is it considered poor form to use the fact that loop variables are still set after the loop?

I have to repeatedly loop over elements of a dictionary and unset an element at each iteration, but can't unset in the loop because that's not allowed. So I came up with

Python code:

while len(myDict):
    for k in myDict:
        if condition(myDict[k]):
            do_things()
            break
    else:
        raise MyCustomException('badly formed dict!')
    del myDict[k]

Here I use the fact that k is still set after the loop is broken out of.

# ¿ May 14, 2013 21:25

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Misogynist posted:

Why are you using an inner loop when each run through the loop will execute 0 or 1 times?
Python code:
for k in myDict.keys():
    if condition(myDict[k]):
        do_things()
    else:
        raise MyCustomException('badly formed dict!')

    del myDict[k]

At each step in the process, at least one of the elements of the dictionary should satisfy the if clause (otherwise the dictionary shall be considered badly-formed by definition). However, I don't know which one(s).

# ¿ May 14, 2013 21:51

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Lichtenstein posted:

2) If I do the "from myModule import *", everything from the first point still applies to variables, except I'm spared the effort of writing moduleName. before them? What would happen if I did it while loving up and having a variable that's named exactly the same in each file?

You would clobber the existing variable in the module where the import statement is.

# ¿ May 15, 2013 09:22

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Dren posted:

This seems like a job for a heapsort.

I'm not sorting it. Here, it will be more straightforward if I just post my actual code.

http://pastebin.com/ikaisXAA

The relevant function is getPatternDict().

# ¿ May 15, 2013 10:44

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Dren posted:

I think it's a little strange that you transform the dictionary in place but if you're after efficiency I suppose it makes sense. Then again, if you're after efficiency you probably ought to write it all in C.

Any particular reason you have these pre-schema files instead of plain-ol' json schema?

The choice to transform the dictionary in place was not based on anything in particular.

The main motivation is the following note from http://json-schema.org/latest/json-schema-validation.html#anchor6

quote:

Furthermore, given the high disparity in regular expression constructs support, schema authors SHOULD limit themselves to the following regular expression tokens:
individual Unicode characters, as defined by the JSON specification [RFC4627];
simple character classes ([abc]), range character classes ([a-z]);
complemented character classes ([^abc], [^a-z]);
simple quantifiers: "+" (one or more), "*" (zero or more), "?" (zero or one), and their lazy versions ("+?", "*?", "??");
range quantifiers: "{x}" (exactly x occurrences), "{x,y}" (at least x, at most y, occurrences), {x,} (x occurrences or more), and their lazy versions;
the beginning-of-input ("^") and end-of-input ("$") anchors;
simple grouping ("(...)") and alternation ("|").

I wanted to write regular expressions using named subexpressions and without significant whitespace. The algorithm that prompted me to ask the question is related to the construction of a dictionary of named subexpressions which might initially refer to one another. These references first need to be resolved so that each subexpression stands on its own. It could happen that some subexpression references by name another subexpression that does not in fact exist, or that a circular relationship exists between subexpressions. Hopefully it is obvious that collections of subexpressions like this are no good. The whitespace thing is permitted in a primitive way by allowing the pattern property to be a list, which is ''.join()'d.

I took the opportunity to add a few bells and whistles, like the ability to write (for example) "_INTEGER": [4, 8] and have it be interpreted as "type": "integer", "minimum": 4, "maximum": 8.

A regular expression in a preschema:

code:

"_TRANSFORMPATTERN": [
   "^(",
      "<abstractPointIdentifier>|",
      "<locationidentifier>:(",
         "[1-4]:(<intercardinalOrMiddle>|<cardinal>(:<parameter48%/>)?)|",
         "name:(<intercardinalOrMiddle>|<cardinal>(:<parameter300%/>)?)|",
         "<intercardinalOrMiddle>|",
         "<cardinal>(:<parameter196%/>)?",
      ")|",
      "<linkIdentifier>:(",
         "<locationIdentifier>|",
         "start|end|midway|",
         "<intercardinalOrMiddle>|",
         "<cardinal>(:<parameter800%/>)?|",
         "[1-4]:(<locationIdentifier>|start|end|midway|<parameter400%/>)|",
         "marker(:(",
            "<intercardinalOrMiddle>|",
            "<cardinal>(:<parameter18%bad/>)?",
         "))?|",
         "<parameter1500%/>",
      ")|",
      "<widgetIdentifier>:(",
         "<intercardinalOrMiddle>|",
         "<cardinal>(:<parameter500%/>?)",
      ")",
   ")",
   "(\\+\\(",
      "-?<positiveparameter200>,",
      "-?<positiveparameter200>",
   "\\))?$"
]

The regular expression in the processed schema (with line breaks added for tables):

code:

"pattern":
"^((\\.[a-zA-Z0-9]{3,12})|(@[a-zA-Z0-9]{3,12}):([1-4]:(((northeast|ne|southeast
|se|southwest|sw|northwest|nw)|middle|m)|(north|n|east|e|south|s|west|w)(:(-?([
1-3]?[0-9]?|4[0-8])|(-?([1-9]?[0-9]?|100)%)|(-?(1/2|[12]/3|[123]/4|[1-5]/6|[1-7
]/8|([1-9]|10|11)/12))))?)|name:(((northeast|ne|southeast|se|southwest|sw|north
west|nw)|middle|m)|(north|n|east|e|south|s|west|w)(:(-?([1-9]?[0-9]?|[12][0-9]{
2}|300)|(-?([1-9]?[0-9]?|100)%)|(-?(1/2|[12]/3|[123]/4|[1-5]/6|[1-7]/8|([1-9]|1
0|11)/12))))?)|((northeast|ne|southeast|se|southwest|sw|northwest|nw)|middle|m)
|(north|n|east|e|south|s|west|w)(:(-?([1-9]?[0-9]?|1[0-8][0-9]|19[0-6])|(-?([1-
9]?[0-9]?|100)%)|(-?(1/2|[12]/3|[123]/4|[1-5]/6|[1-7]/8|([1-9]|10|11)/12))))?)|
(\\^[a-zA-Z0-9]{3,12}):((@[a-zA-Z0-9]{3,12})|start|end|midway|((northeast|ne|so
utheast|se|southwest|sw|northwest|nw)|middle|m)|(north|n|east|e|south|s|west|w)
(:(-?([1-9]?[0-9]?|[1-7][0-9]{2}|800)|(-?([1-9]?[0-9]?|100)%)|(-?(1/2|[12]/3|[1
23]/4|[1-5]/6|[1-7]/8|([1-9]|10|11)/12))))?|[1-4]:((@[a-zA-Z0-9]{3,12})|start|e
nd|midway|(-?([1-9]?[0-9]?|[123][0-9]{2}|400)|(-?([1-9]?[0-9]?|100)%)|(-?(1/2|[
12]/3|[123]/4|[1-5]/6|[1-7]/8|([1-9]|10|11)/12))))|marker(:(((northeast|ne|sout
heast|se|southwest|sw|northwest|nw)|middle|m)|(north|n|east|e|south|s|west|w)(:
(-?([0-9]|1[0-8])|(-?([1-9]?[0-9]?|100)%)|(-?(1/2|[12]/3|[123]/4|[1-5]/6))))?))
?|(-?([1-9]?[0-9]?|([1-9]|1[0-4])[0-9]{2}|1500)|(-?([1-9]?[0-9]?|100)%)|(-?(1/2
|[12]/3|[123]/4|[1-5]/6|[1-7]/8|([1-9]|10|11)/12))))|(%[a-zA-Z0-9]{3,12}):(((no
rtheast|ne|southeast|se|southwest|sw|northwest|nw)|middle|m)|(north|n|east|e|so
uth|s|west|w)(:(-?([1-9]?[0-9]?|[1-4][0-9]{2}|500)|(-?([1-9]?[0-9]?|100)%)|(-?(
1/2|[12]/3|[123]/4|[1-5]/6|[1-7]/8|([1-9]|10|11)/12)))?)))(\\+\\(-?([1-9]?[0-9]
?|1[0-9]{2}|200),-?([1-9]?[0-9]?|1[0-9]{2}|200)\\))?$"

You can see that this regular expression would be tiresome to write by hand, notwithstanding that its length is inflated slightly by superfluous pairs of brackets.

Hammerite fucked around with this message at 17:31 on May 15, 2013

# ¿ May 15, 2013 16:51

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Followup question to what I was talking about before: what is the correct abstract base class to use for lists (in Python 3)?

I was using Sequence as an abstract base class for lists, but then ran into a bug where my recursing function would go beyond the recursion limit. I eventually worked out it was because Sequence matches strings as well as lists, so I changed it to MutableSequence.

# ¿ May 16, 2013 18:09

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

JetsGuy posted:

Hey y'all, I understand that PEP8 is a guideline and all, but I was just wondering if there was an accepted way to fix this. I have a feeling this just may be one of those cases where I have to just let it be.

I have lines in a code that is very long, and really can't be cut down too much further without affecting the readability of the var names. E.g.:
Python code:
marker = usr_deflts["markers_list"][key_cnt%len(usr_deflts["marker_list"])]
which is like 75 characters to begin with. This is in a loop such that it's already 12 chars indented, and I'm really just defining marker because it originally was in a function call which would have made the line like 105-127 depending on how I indent the args in the function.

Again, I realize that PEP8 is a style thing, but I was just curious if there was some pythonic trick to avoiding this. I can't really see much of a workaround.

In addition to the ideas already posted,

Python code:

marker = (
   usr_deflts ["markers_list"]
              [key_cnt % len(usr_deflts["marker_list"])]
)

Although I would be more put off by the variable name "usr_deflts" personally.

# ¿ May 16, 2013 18:59

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

I wrote some code that involves searching through directories and doing stuff to any files found whose filenames match a regular expression. I'm interested to know whether I've done it "the right way" or what I should have done differently.

I wanted to provide the option to either (1) not search subdirectories, (2) search all subdirectories up to a specified depth, or (3) search all subdirectories regardless of depth. It seemed like the way you're supposed to navigate a file system is using os.walk(), so that's what I did.

http://pastebin.com/Dzy8mNPs

The relevant code is lines 202 through 258.

In particular, I'm not sure what to do if an IOError gets thrown as a result of something the function tries to do. I mean, at the moment they aren't caught, but with the errors that I'm raising myself I attach information about which file was being processed and how many files were previously processed. It would be nice to have escaping IOErrors have that information attached to them, too, but I don't know what's the "correct" way to do such a thing.

# ¿ May 21, 2013 00:16

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

No-one answered my question but I think maybe I was overly complicating it. Anyway here's my question distilled into a more basic form.

Suppose I have a function that makes a change or a series of changes to the filesystem. Some of the changes it attempts to make might result in an exception being raised (for example, it might try to write a file somewhere, but Python might not have the necessary privileges to do so). If that happens, I want to just allow the caller to deal with the exception, so I allow it to propagate out of the function. But, by the time an exception is raised, the function might already have made some changes to the filesystem, and it would be nice to have a way of communicating back to the caller what those changes were. What is the best way to do this?

# ¿ May 21, 2013 22:51

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Haystack posted:

Well, there's always the option of attaching something to the exception. You catch the exception, and then either raise your own custom exception or monkeypatch and re-raise the original exception.

See, both those things occurred to me, but I have no idea if there's a Correct Way of doing things that I don't know about because I'm a know-nothing newbie :kiddo:

It occurred to me that I could raise my own exception and have it say something like 'While writing file "%s" after writing %d files already, this happened: ' followed by the __str__ of the existing exception, or the __str__ preceded by the class name, as when the exception is printed. But for all I knew that might discard some useful information that is carried by the original exception.

It also occurred to me to interfere with the original exception, but then that might also have been frowned upon for reasons entirely unknown to me.

# ¿ May 22, 2013 01:08

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Dren posted:

Hammerite I think the thing to do in your case is to store the state of the directory walk outside of the scope of the function doing the work. Something like this:

...

If you need to you could even do something like pickle the current state and load it when the app comes back up to allow the user to resume from where they left off.

I actually prefer writing processing pipelines like this as coroutines but that's opening a whole can of worms.

Thanks, that's really interesting. I hadn't heard of coroutines before and they seem complicated to wrap one's head around, but interesting.

# ¿ May 22, 2013 03:20

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Sab669 posted:

So a buddy of mine is going back to college to get a BS in Comp Sci in a few months, and I have a really dumb question. He decided to start with the MIT lecture last night and I guess it starts off having him use Python. I don't know the first thing about Python or the Python Shell application it was having him use (Or any scripting language for that matter, I'm a newbie C# guy). He was having a problem trying to "understand" how he would actually use this in the real world. By that I mean, if he saved a file and ran it, a command window would pop up and immediately disappear, even though the script requests user input.

I also think part of it for him is he expects to be creating full blown desktop applications, but you need to learn to walk before you can run. Most day-to-day users would poop themselves if they had to deal with the command line in any way.

He was just texting me about it this morning as I was getting ready for work, so I didn't have a chance to sit down and try to learn anything, though I figured while he's taking this course I probably should too

Sounds like he is working on Windows and is running into a limitation of the Windows command line. Basically, when you run a command line application on Windows, it will close the window as soon as the program exits, which is inconvenient if you wanted to see what the program did.

If your program is waiting for user input then it will not close until it gets it (and goes on to exit), but probably your friend's newbie code contains a syntax error or hits some other error condition which causes an exception to be thrown, which isn't caught and causes the program to exit with a printed error message, which counts as exiting and causes Windows to close the window.

There are ways around this. One way is to open a windows command line from the Start menu and run python from there, by typing "python <name of script>.py". If you are changing the script as you go, you can do this repeatedly without having to reopen the window. Note that for maximum convenience, the python install directory should be added to the Windows PATH environment variable, and the directory where he is saving his scripts should be added to the PYTHONPATH environment variable. This prevents errors where Windows can't find Python, or Python can't find the scripts.

Your friend should also acquaint himself with the Python command line and how it differs from the Windows command line.

# ¿ May 22, 2013 15:30

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Dren posted:

Hammerite I think the thing to do in your case is to store the state of the directory walk outside of the scope of the function doing the work. Something like this:
Python code:
import os

class WalkState(object):
    def __init__(self):
        self.processed = []

def do_thing_to_file(path):
    pass

def walk_and_do_thing(dirpath, func, context=None):
    for root, dirs, files in os.walk(dirpath):
        for filename in files:
            func(os.path.join(root, filename))
            if context is not None:
                context.processed.append(os.path.join(root, filename))

def main():
    walk_state = WalkState()
    try:
        walk_and_do_thing('/', do_thing_to_file, walk_state)
    except IOError as e:
        print str(e)
        print 'Processed files before error: %s' % ', '.join(walk_state.processed)

if __name__ == '__main__':
    main()
If you need to you could even do something like pickle the current state and load it when the app comes back up to allow the user to resume from where they left off.

I actually prefer writing processing pipelines like this as coroutines but that's opening a whole can of worms.

So now suppose I want to do it like this:

Python code:

def depthCheckingDirectoryWalk (
   myGenerator, directory, subdirectories, outdata = None, **kwargs
):
   if outdata is None: outdata = {}
   if subdirectories >= 0:
      depthDict = {os.path.abspath(directory): 0}
      myGenerator = myGenerator(outdata, **kwargs)
      next(myGenerator)
      for (currentDir, subdirs, files) in os.walk(directory):
         myGenerator.send(currentDir, subdirs, files)
         if subdirectories is not True and subdirectories <= depthDict[currentDir]:
            subdirs.clear()
         else:
            subdirDepth = depthDict[searchdir[0]] + 1
            for d in subdirs:
               depthDict[os.path.join(searchdir[0], d)] = subdirDepth
   return outdata

And for the functionality that handles a given directory, I need to write a generator function that first sets up its environment, then repeatedly receives (currentDir, subdirs, files) tuples using the yield keyword. But I also want to write a convenience function that just calls depthCheckingDirectoryWalk() with the appropriate arguments. My question is, I want the generator function to accept specific keyword arguments. Where should I declare those? I guess I need to declare them for the generator function so that they are available in the function. But it feels like they should be declared for the convenience function too, so that they appear in help() output and suchlike; and yet then I'm putting the same thing in two places, which seems off.

# ¿ May 22, 2013 19:57

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Thanks for the thoughts on coroutines, Dren. I did it that way and just put a brief explanation in the docstring. Added to that there is the fact that a ready made example is present as well due to the fact that I was writing it for a purpose (had a specific application for walking the filesystem).

Question: Should a module export its custom exceptions? I found out that the help() function respects __all__. But it seems to add unnecessary clutter to have the help list the custom exception class and a long list of double-underscore methods it inherits. Blah, blah, blah. It is a class of exceptions. Who would want to read that? So I thought perhaps __all__ should exclude the exceptions.

# ¿ May 23, 2013 04:06

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

I have a Python module, let us call it x.py, I wrote for myself that I now want to be in version control. So I made a directory called simply x, moved x.py in there, and initialised a git repository. So far so simple. But now in order to import that module I have to type

import x.x as x

Which is kind of onerous. I could add the directory x to my PYTHONPATH, but I don't want to do that for every module I create. I could put a copy of x in my Python install directory, but I'd have to update it each time I make a change. But I see that I can rename x.py to __init__.py and I can once again just "import x". So now I have a directory containing a single file, "__init__.py" (and a .gitignore file and git/pycache nonsense). Is there a better way?

# ¿ May 28, 2013 07:09

Adbot: ADBOT LOVES YOU

# ¿ May 9, 2024 00:58

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Thanks, I decided to bite the bullet and split the thing up into a half dozen files. After a bit of faffing around with import statements and so on the organisation of the code is now improved.

# ¿ May 28, 2013 10:47

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›6 »