Django Web Framework: The 'D' is silent dumbass

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›122 »

duck monster: Dec 15, 2004

yoyo

http://code.djangoproject.com/wiki/ProfilingDjango

Run your code through that bad boy and try and figure out where its lingering.

http://www.rkblog.rk.edu.pl/w/p/django-profiling-hotshot-and-kcachegrind/

If you dont have access to kCacheGrind, WinCacheGrind should provide a pretty good profiling client

kcachegrind is phenomenal though.

Also, try using SQLite and see if the problem persists. I realise you don't want to use sqlite production, but it'll give a reasonable comparison to confirm against re your performance issues.

Also read this:
http://code.google.com/p/django-pyodbc/wiki/Settings

And try and work out whats your connection pooling and cacheing policy. You dont want it having to re-establish a connection each page load, you'll want a reasonable sized pool of connections with as much persistance as possible

duck monster fucked around with this message at 07:13 on Dec 15, 2009

# ? Dec 15, 2009 07:07

Adbot: ADBOT LOVES YOU

# ? Jun 7, 2024 08:47

nbv4: Aug 21, 2002; by Duchess Gummybuns

does anybody know what the deal is with the djangocon 2009 videos? Did they not record them? It's been over a quarter of a year and they still aren't up anywhere... Did it take this long for the '08 videos to get posted?

# ? Dec 15, 2009 10:02

king_kilr: May 25, 2007

They were recorded, and they've been edited (and really well too), however they can't be released for some reason. All I know is that the reason isn't that the speakers didn't sign release forms, I have no more information, but Karl is working to get them released.

# ? Dec 15, 2009 18:43

Doh004: Apr 22, 2007; Mmmmm Donuts...

Hey guys,

I have a quick question about where I should go with a project of mine.

I've created an online portal for my fraternity that provides a bunch of tools and functionality that works great for us. I've been working on it for about a year now, constantly adding new features as I went along. I'm not beginning to look over my code and I realized that a lot of my code is just horrendous and it's starting to hold me back now. It's in PHP and MySQL.

That's where Django comes into play. I've learned a lot of Python this past semester, and I've done a couple small things with Django in the past. I can picture exactly how I'd recreate my portal in Django, but I'm hesitant. I'm a very competent web developer and am comfortable in coding so that's not an issue.

Do I just scrap my thousands of lines of code and move onto a new format? What if I've never implemented a production ready project with Django before? My Centos webserver probably isn't properly confered for Django, so that'd take some setting up.

Has anyone else converted previous applications to a Django created website? If so, what would you suggest and what did you do to ease your problems?

# ? Dec 17, 2009 01:48

SlightlyMadman: Jan 14, 2005

The more you try to migrate, the more your rewrite will have the same problems. You know how it works, so take everything you learned and rewrite it from the ground up to be awesome.

# ? Dec 17, 2009 02:08

nbv4: Aug 21, 2002; by Duchess Gummybuns

Doh004 posted:

Hey guys,

I have a quick question about where I should go with a project of mine.

I've created an online portal for my fraternity that provides a bunch of tools and functionality that works great for us. I've been working on it for about a year now, constantly adding new features as I went along. I'm not beginning to look over my code and I realized that a lot of my code is just horrendous and it's starting to hold me back now. It's in PHP and MySQL.

That's where Django comes into play. I've learned a lot of Python this past semester, and I've done a couple small things with Django in the past. I can picture exactly how I'd recreate my portal in Django, but I'm hesitant. I'm a very competent web developer and am comfortable in coding so that's not an issue.

Do I just scrap my thousands of lines of code and move onto a new format? What if I've never implemented a production ready project with Django before? My Centos webserver probably isn't properly confered for Django, so that'd take some setting up.

Has anyone else converted previous applications to a Django created website? If so, what would you suggest and what did you do to ease your problems?

I was in the same boat a few months ago. I have a huge 10,000 LOC project in PHP/MySQL that was getting so ugly I needed to do a rewrite. I decided to start over from scratch, and haven't looked back since. If you're like me, you'll have a hell of a hard time at first, trying to wrap your head around doing it the django way, as django's URL dispatcher/view/template system is very different from PHP's way of doing things. But eventually you'll get it and you'll be glad you made the plunge.

# ? Dec 17, 2009 02:21

LuckySevens: Feb 16, 2004; fear not failure, fear only the limitations of our dreams

Once you 'get' django (which isn't that hard), and you understand that you only have to go as deep as you want to go, the whole thing becomes very awesome. If you want to be clever, there's ways to do that. If you just want basic functionality and don't mind if its a little unscalable or something, that's also easy.

And then you have all the python documentation with a million ideas on how to do XYZ in a clean way which feels like a bonus to be honest.

# ? Dec 17, 2009 05:10

avidal: May 19, 2006; bawl-some

avidal posted:

Anyone currently using Django on Windows (or even Linux) with pyodbc, django-pyodbc, and SQL Server have any experience with lovely performance?

I've been trying to figure out this problem for several days now, but I can't manage it. I created a barebones Django app to try and replicate the problem. If I don't involve the database at all, I get acceptable performance, roughly 400-500 requests/second according to Apache Bench. However, as soon as I involve the db, my performance drops to about 2-5 requests per second. Each individual query completes very quickly (less than a millisecond in some cases), but I still have horrid performance. I've tried with new tables with only one row, tables with 150,000+ rows, it doesn't matter at all.

I'm currently using Django 1.1.1 (but I was using Django 1.1b0 with no difference) as well as the latest pyodbc and django-pyodbc (as well as the legacy django-pyodbc branch) with no differences at all. The problem exists even on CentOS with the same packages.

I'm at my wits end here, I really need to solve this. We are 2 weeks behind deadline because of this problem. I will PayPal anyone that can offer a solution $100.

Well, we migrated everyone back over to the new system again to see how it would perform, and it's working just fine now. The only thing we changed was bypassing our ISA server (Microsoft Firewall/Proxy) for the intranet site. So I guess it was ISA that was causing it oddly enough.

# ? Dec 17, 2009 17:58

MonkeyMaker: May 22, 2006; What's your poison, sir?

One big point, though, that I'm having to drill into my co-worker, is that you've bought into a framework, so use it.

He was wanting to do a form that listed choices for a user to search on from a given table (like meal times, since it's a recipe site). His idea was to pull in all the entries in his view, format it how he wanted, output that to the template and then wrap it in the appropriate markup.

Thankfully he listened to me and just used Django's forms to do the whole thing in 10 lines of code.

You're using a very in-depth, full-featured framework so don't go reinventing the wheels or making your own axle and you'll have a much better time.

# ? Dec 17, 2009 17:59

Sock on a Fish: Jul 17, 2004; What if that thing I said?

I'm trying to use a ModelChoice field in one of my forms and I need to use a variable in my queryset arg that I'd pass to the form init in a view.

I was struggling with this last night and am still struggling with it today. It looks like **kwargs are out unless I start mucking about with Django internals, they're available for the field init but not the form init. It looks like neither the form or field class has any knowledge of the request.

How do I do this? There's got to be a way, the ModelChoice field is going to be fairly useless if you can't use variables in your queryset filters.

# ? Dec 18, 2009 20:22

king_kilr: May 25, 2007

Take a look at slide 17/18: http://www.slideshare.net/kingkilr/forms-getting-your-moneys-worth

# ? Dec 18, 2009 20:27

nbv4: Aug 21, 2002; by Duchess Gummybuns

Sock on a Fish posted:

I'm trying to use a ModelChoice field in one of my forms and I need to use a variable in my queryset arg that I'd pass to the form init in a view.

I was struggling with this last night and am still struggling with it today. It looks like **kwargs are out unless I start mucking about with Django internals, they're available for the field init but not the form init. It looks like neither the form or field class has any knowledge of the request.

How do I do this? There's got to be a way, the ModelChoice field is going to be fairly useless if you can't use variables in your queryset filters.

you can add in a new form init argument and then call super(). Heres a snippet of some of my code where I do this:

code:

class FlightForm(ModelForm):
    def __init__(self, *args, **kwargs):  
        super(FlightForm, self).__init__(*args, **kwargs)
        
        from share.middleware import share
        from django.db.models import Max
        self.fields['date'].widget = widgets.AdminDateWidget()
        self.fields['plane'].queryset = \
                    Plane.objects\
                    .user_common(share.get_display_user())\
                    .annotate(fd=Max('flight__date')).order_by('-fd')
        self.fields['plane'].default=1
        self.fields['plane'].blank=False
        self.fields['plane'].null=False

# ? Dec 18, 2009 20:29

Sock on a Fish: Jul 17, 2004; What if that thing I said?

nbv4 posted:

you can add in a new form init argument and then call super(). Heres a snippet of some of my code where I do this:

code:

class FlightForm(ModelForm):
    def __init__(self, *args, **kwargs):  
        super(FlightForm, self).__init__(*args, **kwargs)
        
        from share.middleware import share
        from django.db.models import Max
        self.fields['date'].widget = widgets.AdminDateWidget()
        self.fields['plane'].queryset = \
                    Plane.objects\
                    .user_common(share.get_display_user())\
                    .annotate(fd=Max('flight__date')).order_by('-fd')
        self.fields['plane'].default=1
        self.fields['plane'].blank=False
        self.fields['plane'].null=False

I've done this:

code:

class EventRegForm(forms.Form):
	def __init__(self, *args, **kwargs):  
		super(EventRegForm, self).__init__(*args, **kwargs)
		self.event_id = kwargs['event_id']
	meal_choice = forms.ModelChoiceField(queryset=EventMealChoice.objects.filter(event__id=self.event_id),required=False)
	stay_at_hotel = forms.BooleanField(label='stay at hotel?',required=False)
	confirm_reg = forms.BooleanField(label='confirm registration')

But I'm now being told that 'self' is not defined on the line where I initialize meal_choice.

# ? Dec 18, 2009 20:53

nbv4: Aug 21, 2002; by Duchess Gummybuns

thats because you need to set the queryset from the init function instead of from the ModelChoiceField:

code:

class EventRegForm(forms.Form):
	def __init__(self, *args, **kwargs):  
		super(EventRegForm, self).__init__(*args, **kwargs)
		self.fields['meal_choice'].queryset = EventMealChoice.objects.filter(event__id=kwargs.pop('event_id')
	meal_choice = forms.ModelChoiceField(required=False)
	stay_at_hotel = forms.BooleanField(label='stay at hotel?',required=False)
	confirm_reg = forms.BooleanField(label='confirm registration')

# ? Dec 18, 2009 22:11

king_kilr: May 25, 2007

pop event_id out *before* you call super().

# ? Dec 18, 2009 22:25

Sock on a Fish: Jul 17, 2004; What if that thing I said?

king_kilr posted:

pop event_id out *before* you call super().

You're both right! Thanks!

# ? Dec 19, 2009 00:44

SlightlyMadman: Jan 14, 2005

Do I have to do something special to call a custom function of a method from a template? I have a class like this:

code:

class Purchase(models.Model):
    ...
    def get_price(self):
        price = 0
        for ticket in self.ticket_set:
            price += ticket.tier.price
        return price

and a method like:

code:

{% for purchase in purchases %}
    <br/>Total Price: {{ purchase.get_price }}
{% endfor %}

It iterates the purchase just fine (I'm actually printing other information in my real code so I see it), but the purchase.get_price reference doesn't return anything, not even the "0" it would return if self.ticket_set were empty.

# ? Dec 20, 2009 02:21

Sock on a Fish: Jul 17, 2004; What if that thing I said?

SlightlyMadman posted:

Do I have to do something special to call a custom function of a method from a template? I have a class like this:
code:
class Purchase(models.Model):
    ...
    def get_price(self):
        price = 0
        for ticket in self.ticket_set:
            price += ticket.tier.price
        return price
and a method like:
code:
{% for purchase in purchases %}
    <br/>Total Price: {{ purchase.get_price }}
{% endfor %}
It iterates the purchase just fine (I'm actually printing other information in my real code so I see it), but the purchase.get_price reference doesn't return anything, not even the "0" it would return if self.ticket_set were empty.

The templating language doesn't supporting calling methods. Call it in your view and assign its output to a variable that you'll pass to the template.

# ? Dec 20, 2009 02:36

SlightlyMadman: Jan 14, 2005

Huh, ok it's in a loop so I can't really do that but I guess I could write a template tag that gets passed the object and handles that.

I'm confused though, because the documentation says:

quote:

Technically, when the template system encounters a dot, it tries the following lookups, in this order:

* Dictionary lookup
* Attribute lookup
* Method call
* List-index lookup

http://docs.djangoproject.com/en/1.1/topics/templates/#topics-templates

# ? Dec 20, 2009 02:43

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

Sock on a Fish posted:

The templating language doesn't supporting calling methods. Call it in your view and assign its output to a variable that you'll pass to the template.

Yes it does, they just can't have any arguments. As for the original problem, I can't see anything obviously wrong with it. Make sure there's actually something in purchases, maybe? Although that would be an obvious issue, because no "Total Price:"s would show.

# ? Dec 20, 2009 02:50

duck monster: Dec 15, 2004

Big protip: Just stumbled across context processors. Makes life a HELL of a lot easier filling the template stuff thats persistant across all pages.

Unless theres an easier way to do the

{% extends 'blah.html' %}

and actually fill the poo poo in with blah without having to pass the same variables in every drat view.

# ? Dec 20, 2009 05:28

SlightlyMadman: Jan 14, 2005

Jonnty posted:

Yes it does, they just can't have any arguments. As for the original problem, I can't see anything obviously wrong with it. Make sure there's actually something in purchases, maybe? Although that would be an obvious issue, because no "Total Price:"s would show.

Yeah, there's other code in there that I didn't show that outputs various properties from the purchase objects, so I know it's populated and being iterated properly. I'm hosting the project on Dreahmost, so I'm not sure what version they're running. Was method access maybe added in a recent patch? I do also access purchase.ticket_set.count with no problem, and I think that's a method.

# ? Dec 20, 2009 06:21

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

duck monster posted:

Big protip: Just stumbled across context processors. Makes life a HELL of a lot easier filling the template stuff thats persistant across all pages.

Unless theres an easier way to do the

{% extends 'blah.html' %}

and actually fill the poo poo in with blah without having to pass the same variables in every drat view.

Yep, that's pretty drat useful for variables in your master template and stuff like that, and context processors are indeed useful. There some useful default ones if you use RequestContext - check them out.

# ? Dec 20, 2009 06:22

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

SlightlyMadman posted:

Yeah, there's other code in there that I didn't show that outputs various properties from the purchase objects, so I know it's populated and being iterated properly. I'm hosting the project on Dreahmost, so I'm not sure what version they're running. Was method access maybe added in a recent patch? I do also access purchase.ticket_set.count with no problem, and I think that's a method.

Method access has been around for a while and count is indeed a method, so that can't be the problem. I'm completely stumped.

However, I've just noticed you can replace that with a call to the annotate method. For example, doing this:

code:

from django.db.models import Sum
purchases = Purchase.objects.all().annotate(price=Sum('ticket_set__tier__price'))

will mean that for every purchase you'll get a purchase.price attribute which you can use in just the same way. Obviously that works for every queryset, not just the one returned by all(). At the very least, if something's wrong, that ought to give you a more descriptive error.

Jonnty fucked around with this message at 06:35 on Dec 20, 2009

# ? Dec 20, 2009 06:33

tehk: Mar 10, 2006; [-4] Flaw: Heart Broken - Tehk is extremely lonely. The Gay Empire's ultimate weapon finds it hard to have time for love.

I also do not see why the method is not working, but you can make it a property using the property built-in if you'd like. Still the first method should work.

code:

class Purchase(models.Model):
    ...
    price = property(lambda self: sum((ticket.tier.price for ticket in self.ticket_set)))

# ? Dec 20, 2009 06:50

SlightlyMadman: Jan 14, 2005

Gah! You guys were right, there was actually just an error in the get_price method (I should have been iterating "ticket_set.all()", not just "ticket_set"). Why do template calls sometimes ignore errors like that?

edit: I have a follow-up problem. I'm trying to call another method that does a "urllib.urlopen" call but is throwing an error which seems to indicate it's not returning successfully. When I cann it manually in the view and pass it to the template though, it works fine.

Is there some reason I can't do a urlopen in a method from a template call? I guess I could build an array or tuple in the view with all the attributes I need from the purchase objects and pass that to the template, but that would be really annoying since I'd need to edit my view any time I add fields to my presentation.

edit: And nevermind again, in putting it into a template tag I noticed that it was just one of the iterated purchases that was throwing the error.

SlightlyMadman fucked around with this message at 18:25 on Dec 20, 2009

# ? Dec 20, 2009 17:45

king_kilr: May 25, 2007

No, there is no reason that you would be unable to call urlopen.

# ? Dec 20, 2009 18:24

Jonnty: Aug 2, 2007; The enemy has become a flaming star!

SlightlyMadman posted:

Gah! You guys were right, there was actually just an error in the get_price method (I should have been iterating "ticket_set.all()", not just "ticket_set"). Why do template calls sometimes ignore errors like that?

Whoops, forgot that related names are Managers so have to be called like that. Sorry.

# ? Dec 20, 2009 18:30

Profane Obituary!: May 19, 2009; This Motherfucker is Dead

SlightlyMadman posted:

Why do template calls sometimes ignore errors like that?

Because originally Django had some functions that would return an empty string on an error, and it became the "correct" way of doing things and can't change because a lot of sites depend on that functionality now.

# ? Dec 20, 2009 19:15

king_kilr: May 25, 2007

Profane Obituary! posted:

Because originally Django had some functions that would return an empty string on an error, and it became the "correct" way of doing things and can't change because a lot of sites depend on that functionality now.

Wrong.

Because that's the philosophy of the template language. Errors should never propagate (alternatively: your designers should not be able to trigger a 500).

# ? Dec 20, 2009 19:35

Sock on a Fish: Jul 17, 2004; What if that thing I said?

What's your preferred way of handling payments? I've looked at PayPal, Google Checkout, and Amazon FPS and none of them official Python libraries. The third-party libraries I've found don't look very well maintained. Do most people resort to manually crafting JSON and SOAP requests for these services?

# ? Dec 20, 2009 19:50

jupo: Jun 12, 2007; Time flies like an arrow, fruit flies like a banana.

Satchmo (an awful Django ecommerce app) has a handful of payment modules built in that'd serve as good examples.

# ? Dec 21, 2009 01:29

SlightlyMadman: Jan 14, 2005

Heh, the code I've been asking for help on is actually to handle a checkout system using the paypal express checkout API. I'm using this code:
http://www.djangosnippets.org/snippets/1181/
with a nice usage example here:
http://uswaretech.com/blog/2008/11/using-paypal-with-django/

# ? Dec 21, 2009 04:19

king_kilr: May 25, 2007

Ugh, old style class, ignoring pep8, pointless class attrs, CameCase mtehods. My rage is strong.

# ? Dec 21, 2009 04:54

SlightlyMadman: Jan 14, 2005

Yeah, that bothered me a little too, but I just stuck it all in a separate file so I only have to deal with the few places I call it looking like java. It's a pretty simple class though, so if it really gets to you, you could make it pep8 compliant in 15 minutes, I'm sure.

# ? Dec 21, 2009 05:35

Sock on a Fish: Jul 17, 2004; What if that thing I said?

I've been having good luck with django-paypal. It has a few forks, this one seems to work best. It comes with forms that you can use to generate buttons to send users over to PayPal to make their payment, as well as a form to handle the POST from PayPal that they use to indicate payment success.

It looks like you'll need a pro account to avoid sending users offsite, but that's okay for my requirements.

Googling 'paypal python' didn't turn that up in the first few pages of results, I had to search for 'django paypal' to find it.

# ? Dec 21, 2009 05:59

Captain Capacitor: Jan 21, 2008; The code you say?

We use django-paypal in our site and it's been nothing but easy to use. Took a bit of head-scratching to figure out what some of the variables meant but the setup itself was easy.

Edit:
\/\/\/\/ Yep, I'm dumb.

Captain Capacitor fucked around with this message at 09:50 on Dec 23, 2009

# ? Dec 22, 2009 20:23

pokeyman: Nov 26, 2006; That elephant ate my entire platoon.

Captain Capacitor posted:

We use django-python in our site

haha what

# ? Dec 23, 2009 02:11

SlightlyMadman: Jan 14, 2005

I think he meant "django-paypal," judging from the context.

# ? Dec 23, 2009 03:12

Adbot: ADBOT LOVES YOU

# ? Jun 7, 2024 08:47

LuckySevens: Feb 16, 2004; fear not failure, fear only the limitations of our dreams

I'm a moron. I just spent 2 hours trying to get a custom comment system preview to display properly. I couldn't work out why it was giving me a 404. I tried everything, even ported it into another project and somehow it worked there. I made a post here with all the info, and it wasn't until I was seconds away from posting that I saw it.

I put the urlpattern under my flatpages. please tell me im not the only one to have done this

# ? Dec 23, 2009 07:36

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›122 »