Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Helado posted:

I'll admit I'm not terribly familiar with the requests module, so just did a quick lookup on the post method. Back to the encoding issue, when you use io.StringIO it is basically a string in memory. If you check the parent class for StringIO and you see that StringIO doesn't really have a buffered backing (https://docs.python.org/3/library/io.html#io.TextIOBase.detach). Which is why the getvalue() is necessary for the post method. So, encoding is none because stringIO is just a string stored in memory, there is no encoding. It's stored as a unicode string in some format. When you need to write the file out (in this case sending bytes to a POST), you need to convert it to a stream of bytes encoded in the proper format. We need to convert the unicode string to a utf-8 stream of bytes (https://docs.python.org/3/howto/unicode.html#converting-to-bytes). If you were to read out the data from your file as bytes and into a io.BytesIO object, it would come out already encoded because it was never decoded to a string.

Ahhhh that makes sense. Thank you so much for the explanation!

# ? Jul 9, 2015 21:54

Adbot: ADBOT LOVES YOU

# ? May 20, 2024 02:47

Harriet Carker: Jun 2, 2009

How do you guys deal with saving different versions of a program you're working on? Is there any way to automate it? For instance, if I'm working on memory.py, when I save, I want it to save as memory1.py, then memory2.py the next time and so on, so I can easily revert to an older version if I break it beyond comprehension.

# ? Jul 10, 2015 05:05

The March Hare: Oct 15, 2006; _{Je r�ve d'un}
Wayne's World 3; Buglord

dantheman650 posted:

How do you guys deal with saving different versions of a program you're working on? Is there any way to automate it? For instance, if I'm working on memory.py, when I save, I want it to save as memory1.py, then memory2.py the next time and so on, so I can easily revert to an older version if I break it beyond comprehension.

https://help.github.com/articles/good-resources-for-learning-git-and-github/

# ? Jul 10, 2015 05:08

FoiledAgain: May 6, 2007

dantheman650 posted:

How do you guys deal with saving different versions of a program you're working on? Is there any way to automate it? For instance, if I'm working on memory.py, when I save, I want it to save as memory1.py, then memory2.py the next time and so on, so I can easily revert to an older version if I break it beyond comprehension.

The term you're looking for is "version control system". Google around. The most popular is Git. Check out the Coding Horrors thread to find out why all VCS suck.

# ? Jul 10, 2015 05:29

QuarkJets: Sep 8, 2008

dantheman650 posted:

How do you guys deal with saving different versions of a program you're working on? Is there any way to automate it? For instance, if I'm working on memory.py, when I save, I want it to save as memory1.py, then memory2.py the next time and so on, so I can easily revert to an older version if I break it beyond comprehension.

Yeah definitely try git, it's a lot cleaner than the interface that you've imagined.

The bare basics are that you tell git about your project, and then you commit changes to your project whenever you feel like it, including a short message describing the changes that you made. If you want to revert the project to some older version, you can do that easily. If you want to revert just one file to some older version, you can do that too. If you want to add new files to your project you just use "git add filename.py" and then do a new commit. If you want to freeze your project in its current configuration before working on some new feature, sometimes working on the original configuration, sometimes working on the new configuration, whenever you want, eventually choosing (or not, up to you) to merge the two configurations... you can do that, too. Git has a lot of complex features but if you just want basic version control then it's really easy to use.

If you're using PyCharm, it has git integration that you can enable.

# ? Jul 10, 2015 05:40

pmchem: Jan 22, 2010

Does anyone have a favorite package or function in numpy/scipy that they use often? I feel like branching out.

My contribution is np.correlate for time series analysis.

# ? Jul 10, 2015 15:04

Helado: Mar 7, 2004

There is also a thread for questions and discussing how much all VCS sucks. http://forums.somethingawful.com/showthread.php?threadid=3113983

You can use git locally so you don't have to rush out for a github/bitbucket account.

# ? Jul 10, 2015 15:06

karms: Jan 22, 2006; by Nyc_Tattoo; Yam Slacker

Don't use git, use mercurial.

# ? Jul 10, 2015 20:39

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Got a PyCharm/Django question about logging....how do I get log messages to show up in the PyCharm debug console?

I've got the following in my settings.py:

code:

LOGGING = {
    'version': 1,
    'formatters': {
        'simple': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        }
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        }
    },
    'loggers': {
        'root': {
            'handlers': ['console'],
            'level': 'DEBUG',
            'propagate': True
        }
    }
}

LOGGING_CONFIG = None

import logging.config
logging.config.dictConfig(LOGGING)

Then in views.py:

code:

logger = logging.getLogger(__name__)

def index(request):
    logger.info('hello world')
    return render(request, 'index.html')

I see the HTTP request in the PyCharm console, but no 'hello world' message:

code:

[10/Jul/2015 22:56:03] "GET / HTTP/1.1" 200 1946

If I do a print('hello') I do see the message in the console tho

# ? Jul 10, 2015 23:59

mrbotus: Apr 7, 2009; Patron of the Pants

Hi, total beginner here. Been trying for about 2 hours to figure out how to install and use modules with no success. Actually, I did use the Python command prompt to install 'pyperclip' (at least it gave me a message saying so), but I've been trying to use it in PyCharm and the command shell with no results.

This is what I tried:

import pyperclip
pyperclip.copy('Hello, world!')
pyperclip.paste()

But nothing.

# ? Jul 11, 2015 00:07

QuarkJets: Sep 8, 2008

nickmeister posted:

Hi, total beginner here. Been trying for about 2 hours to figure out how to install and use modules with no success. Actually, I did use the Python command prompt to install 'pyperclip' (at least it gave me a message saying so), but I've been trying to use it in PyCharm and the command shell with no results.

This is what I tried:

import pyperclip
pyperclip.copy('Hello, world!')
pyperclip.paste()

But nothing.

Linux or Windows?

# ? Jul 11, 2015 00:28

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

nickmeister posted:

Hi, total beginner here. Been trying for about 2 hours to figure out how to install and use modules with no success. Actually, I did use the Python command prompt to install 'pyperclip' (at least it gave me a message saying so), but I've been trying to use it in PyCharm and the command shell with no results.

This is what I tried:

import pyperclip
pyperclip.copy('Hello, world!')
pyperclip.paste()

But nothing.

Go to File -> Settings and under Project select Project Interpreter

The Project Interpreter should be set to a virtualenv. If it's set to your global python installation like /usr/local/bin/python then click the little gear icon and select Create VirtualEnv.

Also from that Project Interpreter you can click the green + button on the right side and install the pyperclip package.

If you installed pyperclip in your global python installation, you should uninstall it. (just do "pip uninstall pyperclip" from command line)

In general you always want to work in virtualenvs and not pollute your global installation with various packages. The virtualenv website can probably do a better job than me explaining why this is important.

Personally, I use virtualenv-burrito to manage my virtual environments since I like the command line, but for a beginner such as yourself I don't see any reason why you can't do it from PyCharm like I've described.

# ? Jul 11, 2015 00:34

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Hmmm regarding my logging question from a couple posts up if I change 'root' in the logging dict to 'myapp.views' then the log message shows as expected...

Shouldn't setting it on the 'root' logger do it for every logging.getLogger('whatever') though?

# ? Jul 11, 2015 00:40

Dominoes: Sep 20, 2007

nickmeister posted:

Hi, total beginner here. Been trying for about 2 hours to figure out how to install and use modules with no success. Actually, I did use the Python command prompt to install 'pyperclip' (at least it gave me a message saying so), but I've been trying to use it in PyCharm and the command shell with no results.

This is what I tried:

import pyperclip
pyperclip.copy('Hello, world!')
pyperclip.paste()

But nothing.

Install modules that have been uploaded to pypi by running pip install pyperclip

If the module uses code from a different language and you're using Windows , download and install a file from this site. If you're on Linux, use the package manager.

A nice alternative is to download Anaconda to replace your normal python. It's like normal python, but includes a bunch of packages by default, and has a lot of pre-compiled ones you can download: conda install pyperclip.

# ? Jul 11, 2015 00:45

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

fletcher posted:

Hmmm regarding my logging question from a couple posts up if I change 'root' in the logging dict to 'myapp.views' then the log message shows as expected...

Shouldn't setting it on the 'root' logger do it for every logging.getLogger('whatever') though?

Ugh I just had to change 'root' to '' in the logging dict and now it works like I expect it to

code:

LOGGING = {
    'version': 1,
    'formatters': {
        'simple': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        }
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        }
    },
    'loggers': {
        '': {
            'handlers': ['console'],
            'level': 'DEBUG',
            'propagate': True
        }
    }
}

# ? Jul 11, 2015 00:47

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Ok on to the next logging question....

There's a portion of the log that I want to capture and make available to the user. So I do the following right before performing that action:

code:

            user_log = NamedTemporaryFile()
            formatter = logging.Formatter("%(asctime)-15s %(levelname)-8s %(message)s")
            file_handler = logging.FileHandler(user_log.name)
            file_handler.setFormatter(formatter)
            file_handler.setLevel(logging.INFO)
            logger.addHandler(file_handler)

            logger.info('Starting process xyz')

This seems to capture the 'Starting process xyz' message just fine, but any logger.info() calls that are in other packages are not captured. How do I get it to capture logging info from other packages?

# ? Jul 11, 2015 01:24

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

^ Had to add it to the root logger i.e. replace logger.addHandler(file_handler) with logging.getLogger('').addHandler(file_handler)

Although now I'm wondering if this approach is flawed...it seems like the log file for one user could pick up log entries meant for another user if they are being executed simultaneously

# ? Jul 11, 2015 01:49

Plasmafountain: Jun 17, 2008

Edit: Derp. Ignore me.

Plasmafountain fucked around with this message at 11:06 on Jul 13, 2015

# ? Jul 13, 2015 10:52

Edison was a dick: Apr 3, 2010; direct current only

Zero Gravitas posted:

Quick question about while loops. Im trying to get a solution to an equation that can only be evaluated with any ease in one direction.
code:
gamma = 1.4
last_circ_y = 0.546
throat_y_max = 0.5

def throat_mach(last_circ_y,throat_y_max,gamma):
    
    aratio = last_circ_y/throat_y_max
    test = 0.0
    mtest = 1.0
    
    
    while abs(aratio-test) >0.001:
        miteration = mtest
        atest = (1/mtest)*(((2/gamma+1)+(((gamma-1)/(gamma+1))*(mtest**2))))**((gamma+1)/(2(gamma-1)))
        test = atest
        mtest = miteration + 0.001
        
    return mtest
Python spits out: "TypeError: 'int' object is not callable" about the atest line,

It's doing this because of your (2(gamma-1)), which says call 2 with the parameter gamma-1.

quote:

but am I wrong in thinking everything I'm including are floats? So why is it giving me poo poo about ints?

Underneath it may be, but Python will promote a number to a greater precision storage format when it reaches the bounds of its current representation.

# ? Jul 13, 2015 11:08

Plasmafountain: Jun 17, 2008

Thanks for that, I realised only a couple of minutes before you posted.

Well now I've got an issue that isnt simply down to poor proofreading.

I have two large arrays (25x25) of coordinates - one for x, one for y. So a point is defined by x[i,j], y[i,j]. Matplotlib will gladly draw lines column by column, but I need point-to-point along the lines of x[i,j],y[i,j] - x[i+1,j+1],y[i+1,j+1].

Can this kind of thing simply be done with a for loop or two?

# ? Jul 13, 2015 17:56

QuarkJets: Sep 8, 2008

You can feed line coordinates directly to plot(), and it'll draw a line between those coordinates. For instance:

Python code:

from matplotlib import pyplot as plt
plt.plot([10, 5], [12, 11], 'k-', lw=2)
plt.show()

From there you just need to set up your axes to whatever you want

# ? Jul 13, 2015 18:32

SurgicalOntologist: Jun 17, 2004

Zero Gravitas posted:

Thanks for that, I realised only a couple of minutes before you posted.

Well now I've got an issue that isnt simply down to poor proofreading.

I have two large arrays (25x25) of coordinates - one for x, one for y. So a point is defined by x[i,j], y[i,j]. Matplotlib will gladly draw lines column by column, but I need point-to-point along the lines of x[i,j],y[i,j] - x[i+1,j+1],y[i+1,j+1].

Can this kind of thing simply be done with a for loop or two?

This isn't really clear. You can plot multiple lines in the same call, but they will connect by column. If you want them to connect by row, transpose the data. But based on what you posted it looks like you want the lines to connect on the diagonal, which makes little sense.

# ? Jul 13, 2015 19:08

Plasmafountain: Jun 17, 2008

SurgicalOntologist posted:

This isn't really clear. You can plot multiple lines in the same call, but they will connect by column. If you want them to connect by row, transpose the data. But based on what you posted it looks like you want the lines to connect on the diagonal, which makes little sense.

Thats exactly it. This script I've been writing solves for the method of characteristics, a way of collapsing complicated differential equations to linear ones, resulting in positive characteristic lines (diagonal to upper right) and negative characteristic lines (diagonal to lower right). The coordinate arrays I have essentially mark where these two families cross, but I would like to plot the actual +/- characteristics to display their paths of propagation.

# ? Jul 13, 2015 19:34

SurgicalOntologist: Jun 17, 2004

Ah, okay. Is it just the main diagonals or all of them? In any case this can probably get you there without a loop: http://docs.scipy.org/doc/numpy/reference/generated/numpy.diagonal.html#numpy.diagonal

E: you might still need a loop if you're doing all the diagonals, but at least you won't need an inner loop to actually construct each diagonal.

SurgicalOntologist fucked around with this message at 19:41 on Jul 13, 2015

# ? Jul 13, 2015 19:37

Stringent: Dec 22, 2004; image text goes here

Bit of a design question here.

I've got a few Tastypie resources which need to be limited in what they can edit according to the Role assigned to the ApiKey accessing them. I want to do this through the Authorization, but the problem is that the Resource and it's Authorization are instantiated before the request which has the ApiKey information is available. When the request hits the Resource it populates a property in the Authorization called resource_meta which contains the resource_name which is needed for figuring out if the editing limitations need to be applied.

So, what I've got now is basically this:

Python code:

class Resource(ModelResource):
    ...stuff...
    class Meta:
        ...stuff...
        resource_name = 'buttes'
        authorization = MultiRoleAuthorization()
    ...more_stuff...

class SnowflakeAuthorization(PreviousAuthorization):
    def update_detail(self, object_list, bundle):
        if self.resource_meta.resource_name == 'buttes':
            return the_limited_version
        return regular_version

class MultiRoleAuthorization(Authorization):
    AUTHORIZATION_CLASSES = { 'snowflake': SnowflakeAuthorization, }

    def _authorization_by(role):
        authorization_class = self.AUTHORIZATION_CLASSES[role]
        # probably no reason to make a copy here
        authorization = copy.copy(self)

        ### THIS DOESN'T SEEM RIGHT ###
        authorization.__class__ = authorization_class
        return authorization

    def update_detail(self, object_list, bundle):
        role = bundle.request.api_key.role 
        authorization = self._authorization_by(role)
        return authorization.update_detail(object_list, bundle)

So basically I'm having to do that role check for every method I override, and setting the __class__ like that seems like a huge hack that's going to bite me in the rear end later. I initially tried to compose this in a way that'd allow me to use super to pass the resource_meta into a new instance of SnowflakeAuthorization, but that basically ends up duplicating my PreviousAuthorization which I'd like to avoid if possible. Coming into this I'd intended to just do a factory, but the ApiKey information not being available at instantiation time seems to have thrown a wrench in that. Is there some other pattern that would be appropriate for this?

# ? Jul 14, 2015 02:59

The March Hare: Oct 15, 2006; _{Je r�ve d'un}
Wayne's World 3; Buglord

If I'm trying to make a class for subscribers (I'm making an API wrapper, so I'm just converting a 3rd party JSON response into something more useable), and my data must look like:

code:

{
    "id": id,
    "name", name,
    "active", active,
    "custom_fields":
    {
        "unknown": unknown,
        "another": another
    }
}

The unknown values in custom fields are user defined and necessarily unique. Ideally, they are each accessible as attributes on the class, so just whoever.unknown, not whoever.custom_fields.unknown. Is it a miserable idea, given that, to just init them by iterating through the dict and creating them from there?

# ? Jul 14, 2015 17:20

Space Kablooey: May 6, 2009

Something like this?

code:

def __init__(self, butt):
    custom_fields = butt['custom_fields']
    for k, v in custom_fields.iteritems():
        setattr(self, k, v)

That has the problem that the fields that aren't in the custom_fields won't exist in the instance, I don't know how much of a problem it will be.

# ? Jul 14, 2015 17:39

The March Hare: Oct 15, 2006; _{Je r�ve d'un}
Wayne's World 3; Buglord

Yeah, I'd need the other fields but those are static and known so it would be something like:

code:

class Subscriber(object):
    def __init__(self, **kwargs):
        defaults = {
            'name': None,
            'id': None,
            'email': None,
            'active': None,
        }

        for (param, default) in defaults.iteritems():
            setattr(self, param, kwargs.get(param, default))

        custom_fields = butt['custom_fields']
        for k, v in custom_fields.iteritems():
            setattr(self, k, v)

I only ask because I've never done it, and SO seems to be of two minds on it being a good/bad idea, so I figured I'd go for a 2nd opinion.

The March Hare fucked around with this message at 18:06 on Jul 14, 2015

# ? Jul 14, 2015 18:03

Space Kablooey: May 6, 2009

Personally, if I was sure that those fixed fields won't change any time soon, I would do

code:

    def __init__(self, **kwargs):
        self.name = kwargs.get('name')
        self.id = kwargs.get('id')
        self.email = kwargs.get('email')
        self.active = kwargs.get('active')

# ? Jul 14, 2015 18:48

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Stringent posted:

Tastypie

I'm just throwing this out there but if you're still in early stages with your design, you might want want to look at using django-rest-framework instead. I always found it much better than tastypie.

Don't ask me to defend that because I can't remember my reasons! But you should evaluate it if you can.

# ? Jul 14, 2015 20:06

Stringent: Dec 22, 2004; image text goes here

Thermopyle posted:

I'm just throwing this out there but if you're still in early stages with your design, you might want want to look at using django-rest-framework instead. I always found it much better than tastypie.

Don't ask me to defend that because I can't remember my reasons! But you should evaluate it if you can.

Nah, that ship has long since sailed. This is a few years old, several tens of thousands of lines application.

# ? Jul 14, 2015 23:30

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

fletcher posted:

^ Had to add it to the root logger i.e. replace logger.addHandler(file_handler) with logging.getLogger('').addHandler(file_handler)

Although now I'm wondering if this approach is flawed...it seems like the log file for one user could pick up log entries meant for another user if they are being executed simultaneously

I think I was thinking about this the wrong way...adding the logger during execution of a script is only going to change the logging config within that script. It's not some global thing that is going to impact the logging config used by other executions of that script. I was thinking about it more like a Java webapp.

# ? Jul 15, 2015 00:11

oxidation: May 22, 2011

I'm having trouble getting LiveEdit to work properly with PyCharm and Chrome on Windows. It technically works, but not how well as I had hoped.

If I type something, it looks like Chrome is doing an actual refresh, because it reloads the whole page, versus just changing whatever I just typed. I also can't get external css to work at all -- if i do it inline, it's still the same deal where if i say, want to change the background color, i need to select the color, hit enter, then wait for the page to refresh to even see if I like it.

In the video tutorial I watched, his goes through instantaneously, and he can just hit the up and down keys to instantly see how the color works (Start at 19:13):
https://www.youtube.com/watch?v=2geC50roans

I'm not sure how to achieve that affect.

He is using an older version of PyCharm and the video is 2 years old, but I can't imagine this was taken out.

# ? Jul 15, 2015 19:22

Plasmafountain: Jun 17, 2008

SurgicalOntologist posted:

Ah, okay. Is it just the main diagonals or all of them? In any case this can probably get you there without a loop: http://docs.scipy.org/doc/numpy/reference/generated/numpy.diagonal.html#numpy.diagonal

E: you might still need a loop if you're doing all the diagonals, but at least you won't need an inner loop to actually construct each diagonal.

I need all of them. I know that I can use the same method for the right to left diagonals by simply flipping the array with np.fliplr, but still not confident about finding all of the diagonals in a nice simple method. I've been dicking around a little bit with it, but Im not sure that I can plot them without creating a mass of new arrays to store each diagonal - I had hoped that I could avoid doing that by doing things inside a function but Im not sure thats the case.

# ? Jul 15, 2015 22:38

Dominoes: Sep 20, 2007

What's the best way to send a list of namedtuples [or objects] into a Jinja or Django template for use in Javascript? Ideally, it would turn into a list of JS objects.

This seems to work:

Python code:

labels_jssafe = [[label.text, label.lat, label.lon] for label in labels]

ie passing this instead of the list of namedtuples. |safe required in the template.

Dominoes fucked around with this message at 23:50 on Jul 18, 2015

# ? Jul 18, 2015 23:26

EAT THE EGGS RICOLA: May 29, 2008

Dominoes posted:

What's the best way to send a list of namedtuples [or objects] into a Jinja or Django template for use in Javascript? Ideally, it would turn into a list of JS objects.

These aren't my words, but here's how I did it last time I had to do this:

View:

code:


import json

@app.route('/')
def my_view():
    data = [1, "foo"]
    return render_template('index.html', data=map(json.dumps, data))

Template

code:

{% for item in data %}
    <span onclick=someFunction({{ item|safe }});>{{ item }}</span>
{% endfor %}

# ? Jul 18, 2015 23:33

Dominoes: Sep 20, 2007

That's clever.

# ? Jul 18, 2015 23:34

Cingulate: Oct 23, 2012; by Fluffdaddy

Things I do:
- few, usually medium-length, loops
- list and dict comprehensions - a lot, often nested
- functions small and large, even sometimes recursive
- a common pattern is to create a function or a dict for the purpose of using it for a list or dict comprehension

Things I know I don't do:
- classes and objects
- decorators

Things I don't know I don't do:
- ...

Most of what I do is scientific python, so usually numpy and scipy based, with a lot of pandas and matplotlib. I also operate with strings (e.g., cleaning up text input) a lot. So nothing exciting.

What's the next concept I should focus on learning and understanding so as to become slightly less awful at Python?

# ? Jul 19, 2015 00:11

Dominoes: Sep 20, 2007

Lazy evaluation. Ie generators and genexps.

# ? Jul 19, 2015 00:17

Adbot: ADBOT LOVES YOU

# ? May 20, 2024 02:47

Cingulate: Oct 23, 2012; by Fluffdaddy

I actually do that - not much, but sometimes.
Though one thing I do a lot is
var = (foo if condition else bar)

That counts right?

I do this whenever var is required and there is no clear default. If there is a clear default, or if var is usually set by an external input, I'll usually rather use an if clause for the rarer case.

# ? Jul 19, 2015 00:23

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »