Django Web Framework: The 'D' is silent dumbass

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›122 »

Captain Capacitor: Jan 21, 2008; The code you say?

king_kilr posted:

There's a better source than adrian: http://twitter.com/TheOnion/status/10921296161 If anyone has any questions about it I can probably answer them, I interned there last summer

Are they using something like Pinax or EveryBlock? Or did they roll their own? I know you said you interned last year but maybe you could find out.

# ? Mar 23, 2010 18:29

Adbot: ADBOT LOVES YOU

# ? Jun 8, 2024 05:07

ATLbeer: Sep 26, 2004; Über nerd

king_kilr posted:

There's a better source than adrian: http://twitter.com/TheOnion/status/10921296161 If anyone has any questions about it I can probably answer them, I interned there last summer

I'd be curious on URL migration. Did they decide to just maintain their old URL structure from Drupal?

# ? Mar 23, 2010 19:37

Yay: Aug 4, 2007

ATLbeer posted:

I'd be curious on URL migration. Did they decide to just maintain their old URL structure from Drupal?

One would hope so; anything else would be barbarous (yes, even redirects). So sign me up as curious, too.

# ? Mar 23, 2010 19:46

ATLbeer: Sep 26, 2004; Über nerd

Yay posted:

One would hope so; anything else would be barbarous (yes, even redirects). So sign me up as curious, too.

I probably should have expanded by asking if they were planning on staying with the same URL pattern in perpetuity for a while, are they just supporting old URLs in place with no desire to change, migrating the old URLs to the the new URLs with 302 redirects, etc?

# ? Mar 23, 2010 20:01

king_kilr: May 25, 2007

Captain Capacitor posted:

Are they using something like Pinax or EveryBlock? Or did they roll their own? I know you said you interned last year but maybe you could find out.

No, no external tools (that I can recall, we may have used some external apps). And yes, the old URLs appear to have been migrated.

# ? Mar 24, 2010 03:36

king_kilr: May 25, 2007

ATLbeer posted:

I probably should have expanded by asking if they were planning on staying with the same URL pattern in perpetuity for a while, are they just supporting old URLs in place with no desire to change, migrating the old URLs to the the new URLs with 302 redirects, etc?

Yes, there are new URLs, with 302s, etc.

# ? Mar 24, 2010 03:37

deimos: Nov 30, 2006; Forget it man this bat is whack, it's got poobrain!

king_kilr posted:

There's a better source than adrian: http://twitter.com/TheOnion/status/10921296161 If anyone has any questions about it I can probably answer them, I interned there last summer

I really wish they'd do decent explanation of their rationale, my work is looking at Drupal for a bunch of poo poo which I think it'll be a terrible fit for which a framework (I am partial to django but I've said use anything) would be a much better fit. If I could point to an article or blog post or blurb with their rationale I would have some ground on my arguments.

# ? Mar 24, 2010 05:33

bitprophet: Jul 22, 2004; Taco Defender

deimos posted:

I really wish they'd do decent explanation of their rationale, my work is looking at Drupal for a bunch of poo poo which I think it'll be a terrible fit for which a framework (I am partial to django but I've said use anything) would be a much better fit. If I could point to an article or blog post or blurb with their rationale I would have some ground on my arguments.

A year old, but this post goes into some high level info about that. It's from back when they were "just" doing the AV Club port, but I assume the same general ideas applied to the main site's switcheroo as well.

# ? Mar 24, 2010 15:01

Yay: Aug 4, 2007

king_kilr posted:

Yes, there are new URLs, with 302s, etc.

Couple of questions, then:

Why 302s?
Why are the new URLs still kinda bad, using both a slug and (what looks like) a pk, when its only the pk that is being looked up?

Unrelated: Like the django redevelopment of Michael Moore's website, I fear I'm going to be hearing about this from slower people (and twitter) for weeks to come. Sigh.

# ? Mar 24, 2010 19:42

king_kilr: May 25, 2007

Yay posted:

Couple of questions, then:

Why 302s?

Why are the new URLs still kinda bad, using both a slug and (what looks like) a pk, when its only the pk that is being looked up?

Unrelated: Like the django redevelopment of Michael Moore's website, I fear I'm going to be hearing about this from slower people (and twitter) for weeks to come. Sigh.

Ugh, sorry meant a 301 (obvious reasons). Not sure what the origin of the slug comma pk URLs is, but they're used on all the onion sites, for whatever reason.

# ? Mar 24, 2010 21:35

MonkeyMaker: May 22, 2006; What's your poison, sir?

I'm trying to make a generic region model using django_countries and django.contrib.localflavor.us.us_states to supply the choices for two fields. I have them showing up and saving but when I go to edit a saved region, the selections are gone. How would I provide an initial for this?

Here's my admin and form:

code:

class RegionForm(forms.ModelForm):
    states = forms.MultipleChoiceField(choices=STATE_CHOICES)
    countries = forms.MultipleChoiceField(choices=COUNTRIES)

    class Meta:
        model = Region

class RegionAdmin(admin.ModelAdmin):
    form = RegionForm
    list_display = ['name', 'states', 'countries']

edit:
Actually, another issue came up. When I save the model, the two fields save like this: '{AZ,CA}'. Is there a way to make it save as a list or tuple?

MonkeyMaker fucked around with this message at 22:56 on Mar 24, 2010

# ? Mar 24, 2010 22:50

Lamacq: Jun 15, 2001; Breezeblock RIP

MonkeyMaker posted:

I'm trying to make a generic region model using django_countries and django.contrib.localflavor.us.us_states to supply the choices for two fields. I have them showing up and saving but when I go to edit a saved region, the selections are gone. How would I provide an initial for this?

Here's my admin and form:
code:
class RegionForm(forms.ModelForm):
    states = forms.MultipleChoiceField(choices=STATE_CHOICES)
    countries = forms.MultipleChoiceField(choices=COUNTRIES)

    class Meta:
        model = Region

class RegionAdmin(admin.ModelAdmin):
    form = RegionForm
    list_display = ['name', 'states', 'countries']
edit:
Actually, another issue came up. When I save the model, the two fields save like this: '{AZ,CA}'. Is there a way to make it save as a list or tuple?

Code for the model? I'm wondering why you don't have states be a ManyToManyField on your region model, and let the framework just handle it all for you.

# ? Mar 25, 2010 01:31

MonkeyMaker: May 22, 2006; What's your poison, sir?

Lamacq posted:

Code for the model? I'm wondering why you don't have states be a ManyToManyField on your region model, and let the framework just handle it all for you.

States aren't in a model, they're just choices pre-built for localflavor.us.

The model is just three CharFields: name, states, countries

# ? Mar 25, 2010 01:59

Lamacq: Jun 15, 2001; Breezeblock RIP

MonkeyMaker posted:

States aren't in a model, they're just choices pre-built for localflavor.us.

The model is just three CharFields: name, states, countries

That's true, but you can just make your own State model that has a USStateField attribute. Then the regions model has a manytomanyfield to that. Roughly:

code:

class State(models.Model):
    state = USStateField()

    def __unicode__(self):
        return "%s" % (self.state,)

class Country(models.Model):
    name = models.CharField(max_length=50)

    def __unicode__(self):
        return self.name

class Region(models.Model):
    name = models.CharField(max_length=50)
    states = models.ManyToManyField(State)
    countries = models.ManyToManyField(Country)

    def __unicode__(self):
        return self.name

I think that would give you a lot more mileage out of your DB than packing the states for each region into a single comma-separated field. And you can use your initial_data fixture to load the countries and states.

# ? Mar 25, 2010 02:38

ATLbeer: Sep 26, 2004; Über nerd

Random post on the internet speaking from the Onion Tech team's point of view. Looks legit but, no way to really verify I guess

king_kllr? Any opinion on the validity of their statements?

http://www.reddit.com/r/django/comments/bhvhz/the_onion_uses_django_and_why_it_matters_to_us/

# ? Mar 25, 2010 12:28

Sock on a Fish: Jul 17, 2004; What if that thing I said?

I'd love to read more about the environment they're running in. Looks like nginx is their public-facing web server.

# ? Mar 25, 2010 17:39

MonkeyMaker: May 22, 2006; What's your poison, sir?

Lamacq posted:

That's true, but you can just make your own State model that has a USStateField attribute. Then the regions model has a manytomanyfield to that. Roughly:
code:
class State(models.Model):
    state = USStateField()

    def __unicode__(self):
        return "%s" % (self.state,)

class Country(models.Model):
    name = models.CharField(max_length=50)

    def __unicode__(self):
        return self.name

class Region(models.Model):
    name = models.CharField(max_length=50)
    states = models.ManyToManyField(State)
    countries = models.ManyToManyField(Country)

    def __unicode__(self):
        return self.name
I think that would give you a lot more mileage out of your DB than packing the states for each region into a single comma-separated field. And you can use your initial_data fixture to load the countries and states.

But then I have to create and maintain a list of countries and states simply to be able to make a list for these arbitrary regions. I'm not complaining about the work but this has to be handed off to some relatively clueless editors at a non-profit. Yes, they can do that work but they'll complain again and again about it.

I'd really rather find a smarter solution.

# ? Mar 25, 2010 18:24

king_kilr: May 25, 2007

ATLbeer posted:

Random post on the internet speaking from the Onion Tech team's point of view. Looks legit but, no way to really verify I guess

king_kllr? Any opinion on the validity of their statements?

http://www.reddit.com/r/django/comments/bhvhz/the_onion_uses_django_and_why_it_matters_to_us/

Yeah, tomxtobin, john_onion, alexkillough, all work for the Onion.

When I was there we ran mod_wsgi behind nginx.

# ? Mar 25, 2010 19:00

Lamacq: Jun 15, 2001; Breezeblock RIP

MonkeyMaker posted:

But then I have to create and maintain a list of countries and states simply to be able to make a list for these arbitrary regions. I'm not complaining about the work but this has to be handed off to some relatively clueless editors at a non-profit. Yes, they can do that work but they'll complain again and again about it.

I'd really rather find a smarter solution.

How often does the list of countries and states change?

# ? Mar 25, 2010 19:06

ATLbeer: Sep 26, 2004; Über nerd

Lamacq posted:

How often does the list of countries and states change?

Tangentially realated: Google Maps have had a few problems, specifically with India : Kashmir, and China : Taiwan. They resolved it by showing different maps, with different boundaries and names in different geo-ip located regions.

# ? Mar 25, 2010 19:19

ATLbeer: Sep 26, 2004; Über nerd

From the reddit onion thread

quote:

And the biggest performance boost of all: caching 404s and sending Cache-Control headers to the CDN on 404. Upwards of 66% of our server time is spent on serving 404s from spiders crawling invalid urls and from urls that exist out in the wild from 6-10 years ago. [Edit: We dropped our outgoing bandwidth by about 66% and our load average on our web server cluster by about 50% after implementing that change]

wow...

# ? Mar 25, 2010 19:30

ErIog: Jul 11, 2001

I have a newbie question. So I'm getting around to writing pretty views based on the admin for all the data I've put into my app so far. I've got 12 models, but most of them are separate tables for foreign keys and m2m fields.

I really only need 5 views, and so I went about creating my first one. I'm having no trouble pulling my foreign key and m2m data into my template. It displays great, and looks just like the default admin panel. It was pretty drat easy.

Now the problem comes in with implementing the sorting logic. When I sort by my m2m fields they produce duplicates in the QuerySet despite the fact that I'm using distinct. That's fine. I understand why that happens. It's documented in the docs that it will happen.

My problem is that it doesn't provide any sort of hint as to how to make these things work. Right now my view for all this fits in 15 lines, and my template is also extremely simple. I'd like to keep it that way, but I can't think of a way around. It looks like I'm going to need to dump the information from my QuerySet into a Python set to clear duplicates, but this means performance is going to take a gigantic hit since, in doing so, I'm making QuerySets no longer lazy.

# ? Mar 30, 2010 18:18

Captain Capacitor: Jan 21, 2008; The code you say?

ErIog posted:

I have a newbie question. So I'm getting around to writing pretty views based on the admin for all the data I've put into my app so far. I've got 12 models, but most of them are separate tables for foreign keys and m2m fields.

I really only need 5 views, and so I went about creating my first one. I'm having no trouble pulling my foreign key and m2m data into my template. It displays great, and looks just like the default admin panel. It was pretty drat easy.

Now the problem comes in with implementing the sorting logic. When I sort by my m2m fields they produce duplicates in the QuerySet despite the fact that I'm using distinct. That's fine. I understand why that happens. It's documented in the docs that it will happen.

My problem is that it doesn't provide any sort of hint as to how to make these things work. Right now my view for all this fits in 15 lines, and my template is also extremely simple. I'd like to keep it that way, but I can't think of a way around. It looks like I'm going to need to dump the information from my QuerySet into a Python set to clear duplicates, but this means performance is going to take a gigantic hit since, in doing so, I'm making QuerySets no longer lazy.

Well if you're going to render it to the template, it's going to be unlazyfied at some point. The only other option that I could foresee helping you is creating some sort of custom Manager methods and refining/filtering the search to make the duplicates go away.

# ? Mar 30, 2010 19:30

ErIog: Jul 11, 2001

Captain Capacitor posted:

Well if you're going to render it to the template, it's going to be unlazyfied at some point. The only other option that I could foresee helping you is creating some sort of custom Manager methods and refining/filtering the search to make the duplicates go away.

I know it's going to be unlazyfied at some point, but I can't even optimize it by paginating it because there's no way to paginate until I can kill the duplicates. Otherwise I could end up with 17 elements on a page when making slices of 20 items on the original queryset.

After doing some thinking I've decided that it doesn't make sense to be able to sort by those fields anyway. What I'm really trying to do, I think, is make up for the fact that I don't have any way to search for data in my app yet. Once I write my search page it'll all make a lot more sense.

edit: Now I have a side question. How do I get a list of the models in my application?

ErIog fucked around with this message at 23:11 on Mar 30, 2010

# ? Mar 30, 2010 22:07

nbv4: Aug 21, 2002; by Duchess Gummybuns

ErIog posted:

I have a newbie question. So I'm getting around to writing pretty views based on the admin for all the data I've put into my app so far. I've got 12 models, but most of them are separate tables for foreign keys and m2m fields.

I really only need 5 views, and so I went about creating my first one. I'm having no trouble pulling my foreign key and m2m data into my template. It displays great, and looks just like the default admin panel. It was pretty drat easy.

Now the problem comes in with implementing the sorting logic. When I sort by my m2m fields they produce duplicates in the QuerySet despite the fact that I'm using distinct. That's fine. I understand why that happens. It's documented in the docs that it will happen.

My problem is that it doesn't provide any sort of hint as to how to make these things work. Right now my view for all this fits in 15 lines, and my template is also extremely simple. I'd like to keep it that way, but I can't think of a way around. It looks like I'm going to need to dump the information from my QuerySet into a Python set to clear duplicates, but this means performance is going to take a gigantic hit since, in doing so, I'm making QuerySets no longer lazy.

give us an example of a queryset that you can't get unique. Most likely theres some way to get it the way you want.

# ? Mar 31, 2010 01:16

ErIog: Jul 11, 2001

nbv4 posted:

give us an example of a queryset that you can't get unique. Most likely theres some way to get it the way you want.

Here's a line that produces duplicates. I actually want it to produce duplicates in this case because it helps when there's multiple authors. The ones where I don't want duplicates are of the same pattern, though.

code:

editorialobjects = Editorial.objects.all().order_by(columnorderfield['Authors__LastName')

Here are the relevant models to go along with that.

code:

class Editorial(models.Model):
	id = models.AutoField(primary_key=True)
	Type = models.ForeignKey(EditorialType)
	Title = models.CharField(max_length=150)
	Authors = models.ManyToManyField(Author)
	SerialPart = models.IntegerField(default=0)
	SerialTotal = models.IntegerField(default=0)
	DatePages = models.ManyToManyField(Page)
	def __unicode__(self):
		return "%s; %s" % (self.Title)
	class Meta:
		pass

class Author(models.Model):
	id = models.AutoField(primary_key=True)
	FirstName = models.CharField(max_length=100)
	LastName = models.CharField(max_length=100)
	def __unicode__(self):
		return "%s, %s" % (self.LastName, self.FirstName)
	class Meta:
		pass

Here's what my data returns:

code:


ID: 16; Type: NonFiction; Title: I Park a Thousand Cars a Day; Authors: [<Author: Borden, Hank>, <Author: Jones, Stacy V.>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--032>, <Page: 1949-01-22--033>, <Page: 1949-01-22--040>, <Page: 1949-01-22--045>, <Page: 1949-01-22--047>, <Page: 1949-01-22--049>]
ID: 14; Type: Fiction; Title: Unwelcome Guest; Authors: [<Author: Brooks, George S.>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--029>]
ID: 20; Type: NonFiction; Title: What Would You Have Done?; Authors: [<Author: De Mesa, Benjamin>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--058>]
ID: 7; Type: Fiction; Title: The Price of Divorce; Authors: [<Author: Duganne, Phyllis>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--018>, <Page: 1949-01-22--019>, <Page: 1949-01-22--049>, <Page: 1949-01-22--052>, <Page: 1949-01-22--054>]
ID: 3; Type: NonFiction; Title: State Medicine Hasn't Worked Any Miracles; Authors: [<Author: Editors, Post>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--010>]
ID: 4; Type: NonFiction; Title: How Many Know Why They Think What They Think?; Authors: [<Author: Editors, Post>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--010>, <Page: 1949-01-22--012>]
ID: 6; Type: NonFiction; Title: Moscow Herds Intellectuals Into Pro-Stalinist Fronts; Authors: [<Author: Editors, Post>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--012>]
ID: 17; Type: Fiction; Title: Date With Death; Authors: [<Author: Ford, Leslie>]; 
     SerialPart: 2; SerialTotal: 8; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--034>, <Page: 1949-01-22--035>, <Page: 1949-01-22--064>, <Page: 1949-01-22--066>, <Page: 1949-01-22--068>]
ID: 11; Type: NonFiction; Title: Look at That Girl Go!; Authors: [<Author: Hauser, Ernest O.>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--024>, <Page: 1949-01-22--025>]
ID: 16; Type: NonFiction; Title: I Park a Thousand Cars a Day; Authors: [<Author: Borden, Hank>, <Author: Jones, Stacy V.>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--032>, <Page: 1949-01-22--033>, <Page: 1949-01-22--040>, <Page: 1949-01-22--045>, <Page: 1949-01-22--047>, <Page: 1949-01-22--049>]
ID: 21; Type: Humor; Title: Put It This Way; Authors: [<Author: Jones, Franklin P.>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--061>]
ID: 12; Type: NonFiction; Title: A Man's Reputation; Authors: [<Author: Lamson, David>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--026>, <Page: 1949-01-22--027>]
ID: 5; Type: NonFiction; Title: Who Really Elected Truman?; Authors: [<Author: Lubell, Samuel>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--015>, <Page: 1949-01-22--016>, <Page: 1949-01-22--017>, <Page: 1949-01-22--054>, <Page: 1949-01-22--056>, <Page: 1949-01-22--058>, <Page: 1949-01-22--061>, <Page: 1949-01-22--064>]
ID: 13; Type: NonFiction; Title: My Fifty Years in the White House; Authors: [<Author: Smith, Ira R.T.>, <Author: Morris, Joe Alex>]; 
     SerialPart: 6; SerialTotal: 8; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--028>]
ID: 10; Type: NonFiction; Title: Tales of the Indian Devil; Authors: [<Author: Murphy, Robert>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--023>]
ID: 18; Type: Fiction; Title: Ambush; Authors: [<Author: Short, Luke>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--038>]
ID: 9; Type: Fiction; Title: Contraband; Authors: [<Author: Small, Sidney Herschel>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--022>]
ID: 13; Type: NonFiction; Title: My Fifty Years in the White House; Authors: [<Author: Smith, Ira R.T.>, <Author: Morris, Joe Alex>]; 
     SerialPart: 6; SerialTotal: 8; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--028>]
ID: 8; Type: NonFiction; Title: Key West; Authors: [<Author: Thielen, Benedict>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--020>, <Page: 1949-01-22--021>]
ID: 19; Type: Humor; Title: The Perfect Squelch; Authors: [<Author: Yoder, Robert M.>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--056>]
ID: 15; Type: NonFiction; Title: Didn't Anybody See Me in Television?; Authors: [<Author: Yoder, Robert M.>]; 
     SerialPart: 0; SerialTotal: 0; 
      IssueDate: 1949-01-22; Pages: [<Page: 1949-01-22--030>, <Page: 1949-01-22--031>]

This data was printed with this line of python which mirrors how I'm pulling the data in my template:

code:

for query in editorialobjects:
	print "ID: %s; Type: %s; Title: %s; Authors: %s; SerialPart: %s; SerialTotal: %s; IssueDate: %s; Pages: %s" % (query.id, query.Type, query.Title, query.Authors.all(), query.SerialPart, query.SerialTotal, query.DatePages.all()[0].IssueDate, query.DatePages.all())

ErIog fucked around with this message at 01:42 on Mar 31, 2010

# ? Mar 31, 2010 01:27

nbv4: Aug 21, 2002; by Duchess Gummybuns

try:

code:

editorialobjects = Editorial.objects.defer('author').order_by(columnorderfield['Authors__LastName')

this way it doesn't do the author table join on the original queryset. All objects will be distinct. Then when you loop through the queryset, it will execute a separate query to get the author set.

btw, you don't need all() if you are going to do more queryset methods.

# ? Mar 31, 2010 02:03

Lamacq: Jun 15, 2001; Breezeblock RIP

I'm pretty sure model instances are hashable, so one way to do this (I think) would be to stick all the instances returned by your evaluated queryset into this OrderedSet recipe:

code:

editorialobjectset = OrderedSet()
for e in Editorial.objects.order_by('Authors__LastName'):
    editorialobjectset.add(e)

And then you could iterate over editorialobjectset the same way you did the editorialobject list. I don't have any django environments handy to test that in ATM, but give it a shot.

The Django docs pretty much tell you straight up though that ordering by a manytomanyfield doesn't work in the way you expect, as you're seeing. I think you need to either do it by hand (SQL + filtering results) with a custom manager or try to remove the duplicate instances from the evaluated queryset using an ordered dictionary or set of some kind.

fake edit: ^^^^^^ or do it nbv4's way, which results in more queries but is certainly a lot simpler. I wasn't aware defer() worked like that, cool.

real edit: that OrderedSet thing is only for 2.6 or later

Lamacq fucked around with this message at 02:11 on Mar 31, 2010

# ? Mar 31, 2010 02:09

ErIog: Jul 11, 2001

nbv4 posted:

try:
code:
editorialobjects = Editorial.objects.defer('author').order_by(columnorderfield['Authors__LastName')
this way it doesn't do the author table join on the original queryset. All objects will be distinct. Then when you loop through the queryset, it will execute a separate query to get the author set.

btw, you don't need all() if you are going to do more queryset methods.

Lamacq posted:

I'm pretty sure model instances are hashable, so one way to do this (I think) would be to stick all the instances returned by your evaluated queryset into this OrderedSet recipe:
code:
editorialobjectset = OrderedSet()
for e in Editorial.objects.order_by('Authors__LastName'):
    editorialobjectset.add(e)
fake edit: ^^^^^^ or do it nbv4's way, which results in more queries but is certainly a lot simpler. I wasn't aware defer() worked like that, cool.

real edit: that OrderedSet thing is only for 2.6 or later

Thanks for the help. Like I said, I read what it said in the documentation, but the documentation didn't mention any real direction to start looking for a workaround. I don't really know anything about SQL, and very little about Django. So I thought I'd ask since this seems like something lots of people must have dealt with before, but I couldn't find any real help when I Googled.

# ? Mar 31, 2010 02:31

nbv4: Aug 21, 2002; by Duchess Gummybuns

Lamacq posted:

OrderedSet

I don't think that'll work. Each object returned from that query is unique. It's just that some of the values are the same. If you take the SQL that the queryset generates and feed it into PHPMyAdmin or something, you'll see it returns a bunch of rows, but none of them are complete duplicates. If you have one object that has 3 ManyToMany objects, you'll get three of the first object, each with a different ManyToMany object. I hope that made sense. At least thats how I think it'll work.

edit: I just realized something. Since you're ordering along the 'Author' relation, you're forcing the join to occur. defering the author field will do nothing. I suggest you use .annotate() somehow to create the value you want, then order along that instead. Maybe something like

code:

Editorial.objects.annotate(ord=Max('author__LastName')).order_by('ord')

nbv4 fucked around with this message at 02:58 on Mar 31, 2010

# ? Mar 31, 2010 02:53

Lamacq: Jun 15, 2001; Breezeblock RIP

nbv4 posted:

I don't think that'll work. Each object returned from that query is unique. It's just that some of the values are the same. If you take the SQL that the queryset generates and feed it into PHPMyAdmin or something, you'll see it returns a bunch of rows, but none of them are complete duplicates. If you have one object that has 3 ManyToMany objects, you'll get three of the first object, each with a different ManyToMany object. I hope that made sense. At least thats how I think it'll work.

edit: I just realized something. Since you're ordering along the 'Author' relation, you're forcing the join to occur. defering the author field will do nothing. I suggest you use .annotate() somehow to create the value you want, then order along that instead. Maybe something like
code:
Editorial.objects.annotate(ord=Max('author__LastName')).order_by('ord')

I think it should work. Here's django.db.model.Model's implementation of __hash__():

code:

def __hash__(self):
        return hash(self._get_pk_val())

# ? Mar 31, 2010 03:06

Captain Capacitor: Jan 21, 2008; The code you say?

For those that might have missed it, Pydev's most recent update now includes support for Django. I haven't had a chance to check it out myself, but I'm a big fan of Pydev and I'm eager to see how it works out.

# ? Apr 2, 2010 14:54

blugbee: Mar 1, 2004; hi c-fut

I've got multiple modelforms (each in it's own view) linked up step by step like a form wizard. Is it possible to make a DB transaction span all of these views? I don't want any of the models saved to the DB unless all of the steps are complete. I've run into a lot of problems trying just 'save(commit=False)' because some of the forms have admin-like pop-up foreign key model and stacked inline model creation.

# ? Apr 5, 2010 16:25

king_kilr: May 25, 2007

blugbee posted:

I've got multiple modelforms (each in it's own view) linked up step by step like a form wizard. Is it possible to make a DB transaction span all of these views? I don't want any of the models saved to the DB unless all of the steps are complete. I've run into a lot of problems trying just 'save(commit=False)' because some of the forms have admin-like pop-up foreign key model and stacked inline model creation.

No, it's not. The usual correct route is to store the data somewhere auxilliary (e.g. the session) and only save it to the DB once all stages of the formwizard are complete.

# ? Apr 5, 2010 16:49

MonkeyMaker: May 22, 2006; What's your poison, sir?

I think I found a bug in Django?

Trying to do GenericModelAdminInlines in my admin (obviously. Actually using generic.GenericStackedInline). I'd also like to have save_as = True so I can do save as new. Apparently this is a fixed problem in the past with inlines but it was patched a year ago and the patched code is in both Django 1.1.1 and 1.2 beta.

However, every time I try to save_as a model with these generic inlines, it throws an IndexError. Turning off the inlines makes it work.

Any ideas how I can fix it?

# ? Apr 6, 2010 00:56

ErIog: Jul 11, 2001

I'm working on an archives project that's going to involve a lot of XML because of standards that must be complied to. We're going to need to present the data from this XML on the web. Some of the stuff I'm reading suggests storing the millions of tiny XML files individually, and having your CMS work with them directly. Because of the size of this project I'm thinking this is going to be a gigantic pain.

Is there some standard workflow somebody has written down for importing standardized XML data into a database format that can be used efficiently for presentation on the web?

Here's how this relates to Django. I would love to do this in Django. Is it reasonable for me to do this in Django?

# ? Apr 7, 2010 17:36

Lamacq: Jun 15, 2001; Breezeblock RIP

ErIog posted:

I'm working on an archives project that's going to involve a lot of XML because of standards that must be complied to. We're going to need to present the data from this XML on the web. Some of the stuff I'm reading suggests storing the millions of tiny XML files individually, and having your CMS work with them directly. Because of the size of this project I'm thinking this is going to be a gigantic pain.

Is there some standard workflow somebody has written down for importing standardized XML data into a database format that can be used efficiently for presentation on the web?

Here's how this relates to Django. I would love to do this in Django. Is it reasonable for me to do this in Django?

IMO it just depends on how structured the data it is and (most importantly) how often it will change. If the archives are going to be constantly updated and changed, with records being edited and/or deleted frequently, then I think Django is probably not a good solution and you'll want to work with the XML more directly in a system that's better suited for that. If on the other hand it's the kind of thing where you get a gigantic dump of data to load one time (with perhaps periodic additions in the future) then I think you can just create models that correspond to the structure of the XML data and write a management command that does whatever transformations you need to load the data. I do that sort of thing periodically (management commands that load XML, JSON, or CSVs into django databases) in my job and it's pretty simple.

I also had an idea for your problem before about ordering by a M2M field -- I was thinking you could add a non-editable "order" field on the main Editorial model, and a post-save hook on the main model and the related Author model that every time one of those is saved, sorts all Editorial objects by their related authors and updates the Editorial.order field with the result. Then when you need to display them in the public-facing view, just order them by that new "order" field (or whatever you want to call it) instead of the related model.

# ? Apr 7, 2010 18:52

ErIog: Jul 11, 2001

Lamacq posted:

IMO it just depends on how structured the data it is and (most importantly) how often it will change. If the archives are going to be constantly updated and changed, with records being edited and/or deleted frequently, then I think Django is probably not a good solution and you'll want to work with the XML more directly in a system that's better suited for that. If on the other hand it's the kind of thing where you get a gigantic dump of data to load one time (with perhaps periodic additions in the future) then I think you can just create models that correspond to the structure of the XML data and write a management command that does whatever transformations you need to load the data. I do that sort of thing periodically (management commands that load XML, JSON, or CSVs into django databases) in my job and it's pretty simple.

I also had an idea for your problem before about ordering by a M2M field -- I was thinking you could add a non-editable "order" field on the main Editorial model, and a post-save hook on the main model and the related Author model that every time one of those is saved, sorts all Editorial objects by their related authors and updates the Editorial.order field with the result. Then when you need to display them in the public-facing view, just order them by that new "order" field (or whatever you want to call it) instead of the related model.

Thanks, do you have any suggestions on places to start looking for solutions that work with XML more directly?

The XML really isn't going to be updated a lot, but I would like to minimize the opportunity for bugs by cutting down on transforming the data. The structure of the data shouldn't be a problem because we're choosing software that is known for adhering to a specific XML Schema. I won't have to deal with creating any documents in that format since the tools in our workflow will do that for us, I'll just have to deal with parsing that standardized XML data.

Even though the data won't be changing, due to the size of the project there will be a constant stream of new XML data for the next 2 or 3 years.

# ? Apr 7, 2010 19:09

Lamacq: Jun 15, 2001; Breezeblock RIP

I just mean, if you're going to be really working with the XML on the fly -- XQuery or XSLT wise -- you'd want to store it directly in an XML database, which probably puts you in Java or .NET land. Not a lot of fun, in other words.

If you're just adding new records from time to time and the schema won't be changing, I stand by my recommendation to use a management command to transform the XML records into records in your Django database. It's really not hard (basically you'd just load up the XML and loop through it, using XPath expressions to pull out the field values to populate and save your django model instances). Then whenever you get new records to import, you copy that file to your server and run a

code:

./manage.py import_my_xml_data /path/to/file.xml

And the management command lives at /path/to/django/app/management/commands/import_my_xml_data.py. You'll want to subclass LabelCommand for your command, see James Bennett's post on the topic. As for the XML parsing part, there's lots of python libraries for that.

# ? Apr 7, 2010 20:25

Adbot: ADBOT LOVES YOU

# ? Jun 8, 2024 05:07

mystes: May 31, 2006

Lamacq posted:

I just mean, if you're going to be really working with the XML on the fly -- XQuery or XSLT wise -- you'd want to store it directly in an XML database, which probably puts you in Java or .NET land. Not a lot of fun, in other words.

Actually I believe Berkeley DB XML (which has no Java or .NET involved) has python bindings. I don't know how scalable it is but if the alternative is a ton of little XML files it's probably worth looking into.

# ? Apr 7, 2010 21:25

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›122 »