Python information and short questions megathread.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »

tef: May 30, 2004; -> some l-system crap ->

asaf posted:

I am a translator by day and so I spend most of my time in Microsoft Word--Word 2011 on OS X and 2012 on Windows 7. There are a lot of repetitive things I could stand to automate; for example, before I send in a file I usually run a search-and-replace of doubled whitespace. I'm able to automate some of these things using macros, but anything more complicated seems like it would require the use of Visual Basic (gross).

VBA is painful, but trying to hook up python interactively to Word is likely to be far, far, more painful.

quote:

Or perhaps I should just start doing my work in Markdown, processing it in plain text, and generating Word-compatible files from there. My work uses simple formatting so that's not a problem, although itwould interfere with the workflow for any document that I'm not writing from scratch.

I hacked up some python code to build word documents years ago, using COM http://printf.net/~tef/pyword/ (warning - it's terrible python code, really, really, bad python code). But it is possible.

edit: it is really bad python :smith:

tef fucked around with this message at 06:54 on Oct 20, 2012

# ? Oct 20, 2012 06:51

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 00:53

M31: Jun 12, 2012

At least it enforces proper spelling :britain:

Python code:

        def color(self,r,g,b):
                raise "american"

# ? Oct 20, 2012 10:59

wither: Jun 23, 2004; I have a turn both for observation and for deduction.

tef posted:

VBA is painful, but trying to hook up python interactively to Word is likely to be far, far, more painful.

An alternative option worth entertaining depending on 1) the complexity of what you're doing, and 2) your comfort level with other languages, is looking at the VSTO interop stuff. C#4 has dynamic and late-binding, so you won't be lusting for anything that VB.NET had.

Another alternative option would be to save as docx, close out of Word, then run some non-interactive processing on your files in Python. There are some fantastic libs out there with tons of functionality.

Finally, (though this isn't really helpful now) if/when you move to Office 13, you'll have a way easier time since it was designed with extensibility 100% in mind.

# ? Oct 20, 2012 17:27

how!!: Nov 19, 2011; by angerbot

For the past few weeks I've been working on a python web framework. I'm only about half way finished with everything, but if anyone want to look at what I have already, go here: http://giotto.readthedocs.org/en/latest/

Basically my idea is to force developers to keep one model/one view per page, and to write code that is decoupled from the controller. You can create applications with the framework, and port them from HTTP to IRC with hardly any modifications.

I know it will never be as popular as Django and Flask, but its been fun and educational building so far.

# ? Oct 21, 2012 05:46

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

how!! posted:

You can create applications with the framework, and port them from HTTP to IRC with hardly any modifications.

Oh how!!, never change :allears:

# ? Oct 21, 2012 07:27

Opinion Haver: Apr 9, 2007

how!! posted:

For the past few weeks I've been working on a python web framework. I'm only about half way finished with everything, but if anyone want to look at what I have already, go here: http://giotto.readthedocs.org/en/latest/

Basically my idea is to force developers to keep one model/one view per page, and to write code that is decoupled from the controller. You can create applications with the framework, and port them from HTTP to IRC with hardly any modifications.

I know it will never be as popular as Django and Flask, but its been fun and educational building so far.

I have to ask what's up with that title.

# ? Oct 21, 2012 07:31

tef: May 30, 2004; -> some l-system crap ->

how!! posted:

For the past few weeks I've been working on a python web framework.

http://www.slideshare.net/priestc/visualizing-mvc-and-an-introduction-to-giotto your presentation is very illuminating.

# ? Oct 21, 2012 12:08

My Rhythmic Crotch: Jan 13, 2011

Giotto posted:

Giotto does force users to do things the �Giotto way�.

Sounds fun, can't wait to get started :downs:

# ? Oct 21, 2012 12:31

tef: May 30, 2004; -> some l-system crap ->

how!! posted:

For the past few weeks I've been working on a python web framework. I'm only about half way finished with everything, but if anyone want to look at what I have already, go here: http://giotto.readthedocs.org/en/latest/

From a brief inspection, here are some things you'll want to change.

https://github.com/priestc/giotto/blob/master/giotto/exceptions.py

These should really inherit from StandardError rather than Exception.

https://github.com/priestc/giotto/tree/master/giotto/contrib/auth

You shouldn't be storing the hashed password in the cookie. There is a plethora of documentation on how to do this.

https://github.com/priestc/giotto/blob/master/giotto/controllers/irc.py#L122

looks_legit is not a good name for a function.

https://github.com/priestc/giotto/blob/master/giotto/views/__init__.py

Calling arbitrary methods based upon user input is also not the best idea.

# ? Oct 21, 2012 12:41

Jewel: May 2, 2009

tef posted:

https://github.com/priestc/giotto/blob/master/giotto/controllers/irc.py#L122

looks_legit is not a good name for a function.

What do you mean? That's the absolute best name for a function. :colbert:

# ? Oct 21, 2012 12:47

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

tef posted:

https://github.com/priestc/giotto/blob/master/giotto/controllers/irc.py#L122

looks_legit is not a good name for a function.

Also, it looks like you're trying to shoehorn in IRC stuff to a web library. I've not really had an in-depth look so I guess you might have a valid reason, but if you're going to do that, use an already existing IRC library or, if you really must, write your own and use that from your web framework code. Don't dirty your framework code with all the nasty specifics of doing IRC (just like you wouldn't dirty every single other class with the specifics of doing HTTP (hopefully.))

# ? Oct 21, 2012 13:44

tef: May 30, 2004; -> some l-system crap ->

The entire framework seems like a series of boilerplate to allow you to hook up a trivial function to a bunch of interfaces. Complete with an ascii-art renderer for images.

tef fucked around with this message at 14:52 on Oct 21, 2012

# ? Oct 21, 2012 14:35

The Gripper: Sep 14, 2004; i am winner

Yeah I think the phrase web framework is used to mean "thing on the internet framework", so you can just make controllers for whatever.

Is the repo missing anything? pip install just spews errors (mostly related to packages=['giotto'] in setup.py not including giotto.controllers|contrib|views, and even with that modified it's still missing giotto.core (I was thinking of moving irc.py to use python-irclib if possible).

Edit; oh it was because of restructuring and a typo, submitted a pr to fix it (though I'm sure it would have been soon on your list!).

The Gripper fucked around with this message at 15:12 on Oct 21, 2012

# ? Oct 21, 2012 14:41

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

yaoi prophet posted:

I have to ask what's up with that title.

http://forums.somethingawful.com/showthread.php?threadid=3469049&userid=182061

# ? Oct 21, 2012 15:57

tef: May 30, 2004; -> some l-system crap ->

how!! posted:

For the past few weeks I've been working on a python web framework. I'm only about half way finished with everything, but if anyone want to look at what I have already, go here: http://giotto.readthedocs.org/en/latest/

Basically my idea is to force developers to keep one model/one view per page, and to write code that is decoupled from the controller. You can create applications with the framework, and port them from HTTP to IRC with hardly any modifications.

I know it will never be as popular as Django and Flask, but its been fun and educational building so far.

As a final word of advice, successful frameworks are normally extracted from an application, rather than built in a vacuum.

# ? Oct 21, 2012 16:43

how!!: Nov 19, 2011; by angerbot

tef posted:

From a brief inspection, here are some things you'll want to change.

https://github.com/priestc/giotto/blob/master/giotto/exceptions.py

These should really inherit from StandardError rather than Exception.

makes sense.

quote:

https://github.com/priestc/giotto/tree/master/giotto/contrib/auth

You shouldn't be storing the hashed password in the cookie. There is a plethora of documentation on how to do this.

You're right about the hashed password being stored in the cookie. I'll probably change it later, but there are still a bunch of design decisions I need to make. For instance I don't want to implement any kind of session functionality...

quote:

https://github.com/priestc/giotto/blob/master/giotto/controllers/irc.py#L122

looks_legit is not a good name for a function.

The irc controller is pretty barebones and really only for demonstration purposes. I only wrote it because I realized "oh hay this can be done". I tried looking for a python IRC library that does what I'm trying to do, but couldn't find anything...

quote:

https://github.com/priestc/giotto/blob/master/giotto/views/__init__.py

Calling arbitrary methods based upon user input is also not the best idea.

Why? It's not evaling anything... getattr is safe, isn't it? If the method doesn't exist, it raises an AttributeError. How could that be abused?

# ? Oct 21, 2012 16:51

how!!: Nov 19, 2011; by angerbot

Jonnty posted:

Also, it looks like you're trying to shoehorn in IRC stuff to a web library. I've not really had an in-depth look so I guess you might have a valid reason, but if you're going to do that, use an already existing IRC library or, if you really must, write your own and use that from your web framework code. Don't dirty your framework code with all the nasty specifics of doing IRC (just like you wouldn't dirty every single other class with the specifics of doing HTTP (hopefully.))

Its really not just an HTTP framework. It will (I imagine) be used primarily for web development, with the IRC and command line controllers used for debugging and development. There really isn't a python IRC library, except for one that comes with twisted, which I may switch to at some time.

# ? Oct 21, 2012 16:55

how!!: Nov 19, 2011; by angerbot

tef posted:

The entire framework seems like a series of boilerplate to allow you to hook up a trivial function to a bunch of interfaces. Complete with an ascii-art renderer for images.

So basically just like any other framework?

The Gripper posted:

Yeah I think the phrase web framework is used to mean "thing on the internet framework", so you can just make controllers for whatever.

Is the repo missing anything? pip install just spews errors (mostly related to packages=['giotto'] in setup.py not including giotto.controllers|contrib|views, and even with that modified it's still missing giotto.core (I was thinking of moving irc.py to use python-irclib if possible).

Edit; oh it was because of restructuring and a typo, submitted a pr to fix it (though I'm sure it would have been soon on your list!).

Thanks for the Pull Request. Just merged it. setup.py errors are easy to slip by while developing.

quote:

As a final word of advice, successful frameworks are normally extracted from an application, rather than built in a vacuum.

I'm not creating this in a vacuum. I'm developing an app alongside the framework. Its a blog hosting site that works like gist.io except instead of gists, you plug it into your "thoughts" repo.

edit:

My Rhythmic Crotch posted:

Sounds fun, can't wait to get started

All good frameworks and languages are opinionated. Django wants you to do things the Django way. Rails wants you to do things the Rails way. You can have a Django app where all your data is crammed into one model (I've seen it), and you can have an entire Rails app served from one big gigantic 2000 line controller (I've seen that too). Giotto makes it as hard as possible to do it the wrong way. Its sort of like Lisp, in the sense that it's actually easier to write code the right way, than it is to do things wrong.

how!! fucked around with this message at 17:10 on Oct 21, 2012

# ? Oct 21, 2012 17:00

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

how!! posted:

Its really not just an HTTP framework. It will (I imagine) be used primarily for web development, with the IRC and command line controllers used for debugging and development. There really isn't a python IRC library, except for one that comes with twisted, which I may switch to at some time.

This is literally the first result on google for "python irc library." I've used it a few times before and never had a problem with it. (I imagine if you do come across a problem with it you'd be much better forking and fixing it than just starting from scratch.)

That's pretty much irrelevant though, you should separate off your low-level IRC stuff anyway as it's distracting from the actual logic of what your IRC controller is actually logic of what your IRC controller is actaully for.

# ? Oct 21, 2012 17:09

The Gripper: Sep 14, 2004; i am winner

Yeah python-irclib is *ok*, my local copy of Giotto is almost done using that library other than some weirdness with parse_kwargs (because I'm not sure what format messages need to be when coming from IRC).

# ? Oct 21, 2012 17:10

tef: May 30, 2004; -> some l-system crap ->

how!! posted:

So basically just like any other framework?

Other popular frameworks (django, rails), were stripped out of an application, which was deployed and in use. I'm yet to see bits of django that handle irc, or converting a png into ascii art. :3:

# ? Oct 21, 2012 17:13

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

I still don't see why "you can have both an HTTP interface and an IRC interface with minimal changes of code" is a good idea. The interfaces of both systems are completely different; you should want to write different front-end code for each, or you'll end up with a UI that fits neither.

# ? Oct 21, 2012 17:16

how!!: Nov 19, 2011; by angerbot

Jonnty posted:

This is literally the first result on google for "python irc library." I've used it a few times before and never had a problem with it. (I imagine if you do come across a problem with it you'd be much better forking and fixing it than just starting from scratch.)

That's pretty much irrelevant though, you should separate off your low-level IRC stuff anyway as it's distracting from the actual logic of what your IRC controller is actually logic of what your IRC controller is actaully for.

Te night I wrote the irc controller I must have came across that page (your link is purple), but for some reason I decided not to use it. I don't remember why.

quote:

I still don't see why "you can have both an HTTP interface and an IRC interface with minimal changes of code" is a good idea. The interfaces of both systems are completely different; you should want to write different front-end code for each, or you'll end up with a UI that fits neither.

Giotto lets you do that. The model is the same for the IRC and HTTP controllers. Its not like the IRC controller spits out HTML. You write a "text/irc" renderer for your model that is used for rendering the data for IRC, and a "text/html" renderer for rendering for the browser. Lets say you want to writing a blog application. You make a POST request that contains all your blog data. Blog title, blog body, etc. The model function takes in this information, validates it, saves it to the database, then the controller sends that blog data to the view where it is rendered into HTML. If you submitted that blog data via IRC, the model is exactly the same. The same validation logic, the same database tables, etc. The only difference is that the data came in through an IRC bot, and will be returned through IRC, rather than through an HTTP client.

# ? Oct 21, 2012 17:34

how!!: Nov 19, 2011; by angerbot

The Gripper posted:

Yeah python-irclib is *ok*, my local copy of Giotto is almost done using that library other than some weirdness with parse_kwargs (because I'm not sure what format messages need to be when coming from IRC).

make a pull request when youre done dog :0

# ? Oct 21, 2012 17:36

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

how!! posted:

You write a "text/irc" renderer for your model that is used for rendering the data for IRC

I didn't say anything about rendering...

Also, that's a non-standard MIME type. The standard says that any non-standard MIME types need to start with an "x-", so that should be text/x-irc, although I say that message/x-irc is more appropriate.

# ? Oct 21, 2012 17:41

Opinion Haver: Apr 9, 2007

how!! posted:

Why? It's not evaling anything... getattr is safe, isn't it? If the method doesn't exist, it raises an AttributeError. How could that be abused?

It means that anything that subclasses GiottoView has to ensure that none of its methods can do bad things if called by a malicious user, and none of its variables contain sensitive data. This is an awful idea.

Suspicious Dish posted:

http://forums.somethingawful.com/showthread.php?threadid=3469049&userid=182061

quote:

Basically I'm trying to see how attached the company is to their code. The best companies see the data as the important part, bad companies see the code as the important part.

Holy poo poo are you for real.

Opinion Haver fucked around with this message at 17:58 on Oct 21, 2012

# ? Oct 21, 2012 17:51

tef: May 30, 2004; -> some l-system crap ->

Suspicious Dish posted:

Also, that's a non-standard MIME type. The standard says that any non-standard MIME types need to start with an "x-", so that should be text/x-irc, although I say that message/x-irc is more appropriate.

Although I seem to recall a recent draft suggesting the 'x-' prefix is a waste of time, because it never gets removed in practice.

I do find the idea of using a different mime type for rendering 'text in irc' vs 'text over http' a little bit disgusting.

# ? Oct 21, 2012 18:03

how!!: Nov 19, 2011; by angerbot

tef posted:

Although I seem to recall a recent draft suggesting the 'x-' prefix is a waste of time, because it never gets removed in practice.

I do find the idea of using a different mime type for rendering 'text in irc' vs 'text over http' a little bit disgusting.

Why? IRC clients render text differently than the command line and browsers. For one, color codes are different. Anyways, the text/cmd and text/irc mimetypes never leak away from the internals of the framework.

quote:

It means that anything that subclasses GiottoView has to ensure that none of its methods can do bad things if called by a malicious user, and none of its variables contain sensitive data. This is an awful idea.

View classes shouldn't be doing "bad things". All views do is take in raw data, then return that data in a visual format. No database calls or anything like that to be abused. Anyways, how would you dispatch the correct render method?

how!! fucked around with this message at 18:33 on Oct 21, 2012

# ? Oct 21, 2012 18:18

Opinion Haver: Apr 9, 2007

how!! posted:

View classes shouldn't be doing "bad things". All views do is take in raw data, then return that data in a visual format. No database calls or anything like that to be abused. Anyways, how would you dispatch the correct render method?

code:

>>> GiottoView("render", None).render("render")
RuntimeError: maximum recursion depth exceeded

Whoops!

I'm not sure how I'd do method dispatch, since my Python is a little rusty. Maybe a whitelist of permitted render methods?

# ? Oct 21, 2012 19:15

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Python code:

@mime_renderer("text/html")
def renderer(self):
    # ...

# ? Oct 21, 2012 19:29

baquerd: Jul 2, 2007; by FactsAreUseless

Suppose I have a list of values and a list that represent the indices of the values that I want to slice:

code:

#core
values = ["Blue", "Green", "Yellow", "Red"]
indices = [1,3]
sliced = [values[i] for i in indices]

#numpy
values = array(["Blue", "Green", "Yellow", "Red"])
sliced = values[:,[1,3]]

Is this reasonable? I'm very unfamiliar with python.

# ? Oct 25, 2012 19:23

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

baquerd posted:

Suppose I have a list of values and a list that represent the indices of the values that I want to slice:
code:
#core
values = ["Blue", "Green", "Yellow", "Red"]
indices = [1, 3]
sliced = [values[i] for i in indices]
Is this reasonable? I'm very unfamiliar with python.

I don't know anything about numpy so I won't comment on that. The above should set sliced to be ["Green", "Red"]. Since you are talking about slicing I assume what you expected was ["Green", "Yellow"]. You could do that like this:

code:

#core
values = ["Blue", "Green", "Yellow", "Red"]
indices = [1, 3]
sliced = values[slice(*indices)]

# ? Oct 25, 2012 20:01

baquerd: Jul 2, 2007; by FactsAreUseless

Hammerite posted:

I don't know anything about numpy so I won't comment on that. The above should set sliced to be ["Green", "Red"]. Since you are talking about slicing I assume what you expected was ["Green", "Yellow"]. You could do that like this:
code:
#core
values = ["Blue", "Green", "Yellow", "Red"]
indices = [1, 3]
sliced = values[slice(*indices)]

I was specifically looking for ["Green", "Red"], but you just taught me about the slice function. I suppose there would be a different terminology used for a discontinuous slice?

It look me some time to find the official reference on the "*" operator, but I found it here: http://docs.python.org/reference/expressions.html#calls interesting stuff.

# ? Oct 26, 2012 14:39

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

baquerd posted:

I was specifically looking for ["Green", "Red"], but you just taught me about the slice function. I suppose there would be a different terminology used for a discontinuous slice?

Do you mean a slice with a step parameter? It should work fine for that as well.

Python code:

values = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
indices = [1, 8, 2]
sliced = values[slice(*indices)] # [1, 3, 5, 7]

# ? Oct 26, 2012 16:41

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

baquerd posted:

Suppose I have a list of values and a list that represent the indices of the values that I want to slice:
code:
#core
values = ["Blue", "Green", "Yellow", "Red"]
indices = [1,3]
sliced = [values[i] for i in indices]

There's nothing wrong with this. I don't know why Hammerite is trying to convince you differently. Note that it's very unusual you'll have something like this, so if you do have this exact problem, please post more context so we can lead you to a better solution for your specific problem.

# ? Oct 26, 2012 17:39

Modern Pragmatist: Aug 20, 2008

Hammerite posted:

Do you mean a slice with a step parameter? It should work fine for that as well.
Python code:
values = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
indices = [1, 8, 2]
sliced = values[slice(*indices)] # [1, 3, 5, 7]

Look at his initial question. He is attempting to retrieve the elements at all positions designated by indices which has nothing to do with slicing. He just used a bad variable name.

baquerd, I don't believe there is a more elegant solution to your problem than the initial code you posted, although the colon is unnecessary in the numpy solution.

Python code:

#core
values = ["Blue", "Green", "Yellow", "Red"]
indices = [1,3]
sliced = [values[i] for i in indices]

#numpy
values = array(["Blue", "Green", "Yellow", "Red"])
sliced = values[[1,3]]

^^^ I agree with Suspicious Dish. I can't think of a scenario where you would need this so posting more code could help.

# ? Oct 26, 2012 17:45

baquerd: Jul 2, 2007; by FactsAreUseless

Modern Pragmatist posted:

baquerd, I don't believe there is a more elegant solution to your problem than the initial code you posted, although the colon is unnecessary in the numpy solution.

^^^ I agree with Suspicious Dish. I can't think of a scenario where you would need this so posting more code could help.

Interesting that the colon is unnecessary in that example, I thought numpy put it in a 2D array automatically. For a 2D array, it's definitely needed.

I have a symbol list that I use to get a 2D array back from numpy. For testing, I then want to take a selection of those symbols and graph them:

code:

import matplotlib.pyplot as plt

dates = #list of dates
symbols = ["APPL", "GOOG", "SPY", "$SPX", "AA", "C"]
prices = getPriceData(dates, symbols) #ndarray

colsToGraph = [2,3]
plt.clf()
plt.plot(dates, prices[:,colsToGraph])
plt.legend(symbols[i] for i in colsToGraph)
plt.savefig("test.pdf",format="pdf")

baquerd fucked around with this message at 19:07 on Oct 26, 2012

# ? Oct 26, 2012 19:02

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

If that code is exactly as is, why get all this data just to throw it out? I'm assuming that your column list comes from somewhere else, like from the user. What I would do is only fetch data as required:

Python code:

dates = get_me_some_dates()
symbols = some_user_data.split(";")
# ...

# ? Oct 26, 2012 19:26

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

Suspicious Dish posted:

There's nothing wrong with this. I don't know why Hammerite is trying to convince you differently. Note that it's very unusual you'll have something like this, so if you do have this exact problem, please post more context so we can lead you to a better solution for your specific problem.

Modern Pragmatist posted:

Look at his initial question. He is attempting to retrieve the elements at all positions designated by indices which has nothing to do with slicing. He just used a bad variable name.

I am aware now, after reading his second post, that he is just trying to retrieve arbitrary elements from a list of their offsets. When I responded to his first post, I misunderstood what he wanted, because he used the term "slice". In his second post, he used the term "slice" again, and I presumed that on that occasion he had used the term in that way intentionally and was asking me a follow-up question about slicing.

# ? Oct 26, 2012 19:29

Adbot: ADBOT LOVES YOU

# ? May 9, 2024 00:53

Lumpy: Apr 26, 2002; La! La! La! Laaaa!; College Slice

Hammerite posted:

I am aware now, after reading his second post, that he is just trying to retrieve arbitrary elements from a list of their offsets. When I responded to his first post, I misunderstood what he wanted, because he used the term "slice". In his second post, he used the term "slice" again, and I presumed that on that occasion he had used the term in that way intentionally and was asking me a follow-up question about slicing.

I for one am glad you answerd the "wrong" question, as I learned something from it. And while it is important to point out when an answer is not correct, or to the wrong question, it is too bad people are frequently more harsh about it than they need be. (I am guilty of this as well, sadly)

# ? Oct 26, 2012 19:36

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Python information and short questions megathread.

«‹›484 »