Django Web Framework: The 'D' is silent dumbass

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›122 »

Opulent Ceremony: Feb 22, 2012

What is the easiest way for me to self-host a Django site? My only experience making it run at this point is the local Django dev server but I need to make it an available intranet site. I've been given access to a server computer that already has Apache serving a couple internal PHP sites. Should I just follow these steps https://docs.djangoproject.com/en/1.8/howto/deployment/wsgi/modwsgi/ and hope I don't take down the other sites? Is there something simpler?

# ? Jun 1, 2015 14:58

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 16:58

epswing: Nov 4, 2003; Soiled Meat

Opulent Ceremony posted:

What is the easiest way for me to self-host a Django site? My only experience making it run at this point is the local Django dev server but I need to make it an available intranet site. I've been given access to a server computer that already has Apache serving a couple internal PHP sites. Should I just follow these steps https://docs.djangoproject.com/en/1.8/howto/deployment/wsgi/modwsgi/ and hope I don't take down the other sites? Is there something simpler?

You could run a Bitnami stack, which comes with everything you need to run Django in one container https://bitnami.com/stack/django

You can do a local install, or easily host it on Amazon, Google, or Microsoft infrastructure.

# ? Jun 1, 2015 15:08

Razzled: Feb 3, 2011; MY HARLEY IS COOL

I did it with a VPS from digitalocean, on just the 5 dollar a month droplet size. I was lazy and just accessed it via ip though

# ? Jun 1, 2015 15:59

Opulent Ceremony: Feb 22, 2012

epalm posted:

You could run a Bitnami stack, which comes with everything you need to run Django in one container https://bitnami.com/stack/django

You can do a local install, or easily host it on Amazon, Google, or Microsoft infrastructure.

Thanks for this suggestion, I think I'll end up using this so I can isolate the Django site from the other junk they've got going on.

# ? Jun 1, 2015 17:21

IAmKale: Jun 7, 2007; やらないか; Fun Shoe

I'm having a problem with my serializers. Right now I have the following serializer:

Python code:

class ShiftSerializer(serializers.ModelSerializer):
    confirmed_location = LocationSerializer(required=False)

    class Meta:
        model = Shift

When I create a new shift I can pass along a basic object like this:

code:

{date_requested: "2015-06-11", employee: 4, shift_requested: 2}

And the object gets created.

In an effort to make shifts a bit more verbose about who requested them, I declared a Serializer for the Employee field so that my GETs would include information about the employee:

Python code:

class ShiftSerializer(serializers.ModelSerializer):
    confirmed_location = LocationSerializer(required=False)
    employee = EmployeeSerializer(required=False)

    class Meta:
        model = Shift

That produced the desired effect - now I can see who requested the shift without making a second query to the server for every single user. However, after doing that I started getting HTTP 400 errors whenever I try to POST that same object to make a new shift:

quote:

Invalid data. Expected a dictionary, but got int.

I'm sure the serializer is freaking out because I'm not passing in an entire Employee object. Is it possible to include the entire Employee object when GETting Shift objects but still accept just a single integer when creating a new Shift object?

# ? Jun 2, 2015 23:30

Lumpy: Apr 26, 2002; La! La! La! Laaaa!; College Slice

Karthe posted:

I'm having a problem with my serializers. Right now I have the following serializer:

I'm sure the serializer is freaking out because I'm not passing in an entire Employee object. Is it possible to include the entire Employee object when GETting Shift objects but still accept just a single integer when creating a new Shift object?

I too ran into this problem and "solved" it by having a serializer for create and one for get. I would imagine there is a way to avoid that, but I can't seem to figure it out.

# ? Jun 3, 2015 12:45

Data Graham: Dec 28, 2009; 📈📊🍪😋

rest_framework question.

Is there some reason why the REST API should be dog-slow compared to just creating a basic view method and outputting the JSON dump of the object (or a serializer'd version of it)?

I've inherited a fully developed Django system and I'm trying to find some way of profiling this, which takes upwards of 10 seconds to query a single object (i.e. with the /widgets/256/ URL form):

code:

urls.py:

...
router.register(r'widgets', api.WidgetViewSet)
...

api.py:

from rest_framework import viewsets
from rest_framework.response import Response
from rest_framework import status
from rest_framework import permissions
from rest_framework_extensions.decorators import action
from rest_framework_extensions.cache.mixins import ListCacheResponseMixin as CacheMixin
from rest_framework_extensions.key_constructor.constructors import KeyConstructor
from rest_framework_extensions.key_constructor import bits
from rest_framework_extensions.cache.decorators import cache_response

class WidgetViewSet(viewsets.ModelViewSet):
    queryset = models.Widget.objects.filter(hidden=False).order_by('name')
    serializer_class = serializers.WidgetSerializer
    filter_fields = models.Widget._meta.get_all_field_names()

If I throw logging into that class, all the lag time occurs after the class attributes are all set (i.e. if I log after the "filter_fields" line, it appears almost instantly, but then the browser doesn't render the result for another 8-10 seconds).

If instead I do this:

code:

views.py:

def widget_test(request):

    w = Widget.objects.get(pk=256)

    widget = {
        'id': w.id,
        'name': w.name,
        'subname': w.subname,
        ...
    }

    return HttpResponse(json.dumps(widget))

...Then the response comes back almost immediately.

The Occam's Razor solution to this would, of course, be to just make my own views like the above, but the REST API is fully fleshed out with dozens of endpoints and serializers and so on, all using the built-in browseable interface. (I've already eliminated the serializers as being the issue.) I don't want to throw away what appears to be a system built the "right" way in favor of my kludgey but demonstrably much much faster one. I'd love to know if this situation rings any bells with anyone. Thanks...

# ? Jun 3, 2015 21:29

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

My DRF endpoints are really fast, something screwy is going on. Logging in WidgetViewSet isn't going to tell you much, you need to get down into the DRF internals to see what is going on.

It's almost always going to be something related to the database, so I'd start by just looking at the mysql query log (or whatever the equivalent is for you) and see what queries it's executing when you hit that endpoint.

# ? Jun 3, 2015 23:17

Hed: Mar 31, 2004; Fun Shoe

Use django-debug-toolbar with something approximating real data on your dev/test server

# ? Jun 4, 2015 00:47

Data Graham: Dec 28, 2009; 📈📊🍪😋

Hed posted:

Use django-debug-toolbar with something approximating real data on your dev/test server

+1 this, thanks.

Turns out the DRF is doing a complete separate query for every model that has a ForeignKey relation to the Widget model (e.g. WidgetComponent, of which there are an assload of records and the slowness is purely from volume). Of course I'm not doing that for my own homegrown endpoint because I'm not making any calls to widgetcomponent_set or anything, and so it's super fast.

Is there a way to make the DRF not make those queries? I've tried select_related, but it doesn't seem to have any effect (I kinda want the opposite).

# ? Jun 4, 2015 16:52

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Data Graham posted:

+1 this, thanks.

Turns out the DRF is doing a complete separate query for every model that has a ForeignKey relation to the Widget model (e.g. WidgetComponent, of which there are an assload of records and the slowness is purely from volume). Of course I'm not doing that for my own homegrown endpoint because I'm not making any calls to widgetcomponent_set or anything, and so it's super fast.

Is there a way to make the DRF not make those queries? I've tried select_related, but it doesn't seem to have any effect (I kinda want the opposite).

I think this is more of a Django issue than with DRF. Are you sure select_related isn't what you're looking for?

# ? Jun 4, 2015 18:56

Data Graham: Dec 28, 2009; 📈📊🍪😋

Well, like I said, I tried adding select_related('widgetcomponent') to the "queryset =" in the API view, but it didn't make any difference; it still made the separate query for all WidgetComponents (completely unconstrained), right before it selects the Widget for the ID I want. From the toolbar:

code:

SELECT ��� FROM `django_session` WHERE (`django_session`.`session_key` = '...' AND `django_session`.`expire_date` > '2015-06-04 18:28:38' )
SELECT ��� FROM `catalog_widgetcomponent`
SELECT ��� FROM `catalog_widget` WHERE (`catalog_widget`.`hidden` = 0 AND `catalog_widget`.`id` = 1234 )

Am I doing it right, though? I can pass totally invalid arguments to select_related and it doesn't seem to care. Is this the right syntax? WidgetComponent has a widget attribute which is a ForeignKey to Widget.

code:

    queryset = models.Widget.objects.select_related('widgetcomponent').filter(hidden=False).order_by('name')

    queryset = models.Widget.objects.select_related('widgetcomponentblah').filter(hidden=False).order_by('name')

And again, it doesn't do any of those queries for widgetcomponent if I just fetch the object directly like in my views.py above.

E: select_related seems to be for going the other direction, i.e. if I were looking up WidgetComponents and wanted to join the related Widgets into the query. For going the direction I want, i.e. selecting Widgets and trying to avoid looking up all WidgetComponents, the docs suggest that that only works with OneToOneFields with a related_name defined.

E2: Okay, adding related_name='+' to the ForeignKey field on the WidgetComponent model did the trick. Now I just hope nothing else was depending on that backward relation being there.

Data Graham fucked around with this message at 19:46 on Jun 4, 2015

# ? Jun 4, 2015 19:35

Gounads: Mar 13, 2013; Where am I?
How did I get here?

Check out prefetch_related

It does an extra query, but just one for all the related objects instead of one query per object.

E: Saw your edit... I was making an assumption your components were many to one to widgets.

# ? Jun 4, 2015 19:47

Data Graham: Dec 28, 2009; 📈📊🍪😋

They are.

# ? Jun 4, 2015 19:51

porksmash: Sep 30, 2008

Oops, stupid mistake

porksmash fucked around with this message at 23:46 on Jun 4, 2015

# ? Jun 4, 2015 23:42

Heisenberg1276: Apr 13, 2007

I'm using django-storages to use S3 as my media host but because of the project I'd rather keep the file access as backend-agnostic as possible.

I'm using "default_storage.open('generated.css', 'wb+')" to write to a .css file but I can't for the life of me figure out how to set the content-type to text/css - it always ends up as binary/octet-stream - which means it won't render on my page when included.

Does anyone know how to approach this?

EDIT: Got it

I solved it by adding

if hasattr(o, "_storage"):
o._storage.headers['Content-Type'] = 'text/css'

(where o is the File returned by default_storage.open)

Heisenberg1276 fucked around with this message at 16:14 on Jun 8, 2015

# ? Jun 8, 2015 11:37

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Heisenberg1276 posted:

I'm using django-storages to use S3 as my media host but because of the project I'd rather keep the file access as backend-agnostic as possible.

I'm using default_storage.open('generated.css', 'wb+') to write to a .css file but I can't for the life of me figure out how to set the content-type to text/css - it always ends up as binary/octet-stream - which means it won't render on my page when included.

Does anyone know how to approach this?

EDIT: Got it

I solved it by adding
Python code:
if hasattr(o, "_storage"):
    o._storage.headers['Content-Type'] = 'text/css'
(where o is the File returned by default_storage.open)

Use code tags when you post on SA about code.

I fixed your post with code tags and fixed tags in the above quote.

# ? Jun 8, 2015 18:10

Jo: Jan 24, 2005; Soiled Meat

I've not yet decided if this would be better placed in a dedicated SQL thread, but...

I'm using Psycopg2 + Django 1.7. Our connections to Postgres still take between 500ms (when the connection is fresh) and 6000ms (when the connection is stale). I might be a luddite, but I'm gobsmacked a connection to a database would take more than 100ms on the same host. I _thought_ we were connection pooling. Where can I start digging to figure out what the hell is going on with this connection latency or why connection pooling isn't working?

# ? Jun 12, 2015 00:23

The March Hare: Oct 15, 2006; _{Je r�ve d'un}
Wayne's World 3; Buglord

I'm trying to make a shop of sorts, and want to make sure I'm not loving Up� here. If I have products, say shirts, which have some base model that has their name and image and description but then also different sizes and color options for each shirt, I'm best off making a shirt size and shirt color model and then just M2Ming them together right? There is no better/easier way of doing this particular thing?

# ? Jun 16, 2015 01:57

MonkeyMaker: May 22, 2006; What's your poison, sir?

The March Hare posted:

I'm trying to make a shop of sorts, and want to make sure I'm not loving Up� here. If I have products, say shirts, which have some base model that has their name and image and description but then also different sizes and color options for each shirt, I'm best off making a shirt size and shirt color model and then just M2Ming them together right? There is no better/easier way of doing this particular thing?

I mean, you could do some horribleness of picking apart a couple of strings but, yeah.

One Shirt model and then either a Stock model that lists color and size (likely just text attributes)

or

One Shirt model, one Size model, one Color model, and a Stock model that combines all three into a single product

I'd go with the former since it should require less database work to do searching, etc.

# ? Jun 16, 2015 02:44

Paper With Lines: Aug 21, 2013; The snozzberries taste like snozzberries!

Are there any resources for making (what was) a PHP style game in Django? As in, a player logs in, do some stuff and presumably a cron job (in php) would replenish your skills every X time. Maybe a decent example of what I mean would be something like urbandead.

edit: I'm fairly new to Django though have run through the tutorials. I'm probably intermediate to slightly advanced in python.

# ? Jun 17, 2015 00:52

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Paper With Lines posted:

Are there any resources for making (what was) a PHP style game in Django? As in, a player logs in, do some stuff and presumably a cron job (in php) would replenish your skills every X time. Maybe a decent example of what I mean would be something like urbandead.

edit: I'm fairly new to Django though have run through the tutorials. I'm probably intermediate to slightly advanced in python.

I was just looking at Huey the other day, it seems like it might be suited for that type of thing.

# ? Jun 17, 2015 01:05

Paper With Lines: Aug 21, 2013; The snozzberries taste like snozzberries!

I think that will help a lot with replacing cron jobs but I was kind of looking for a pseudo tutorial.

Don't get me wrong, I'll use the hell out of that if a tutorial doesn't exist -- but I was looking for a more gradual path into this biz-nass.

# ? Jun 17, 2015 06:13

MonkeyMaker: May 22, 2006; What's your poison, sir?

Do your cron jobs w/ something like Celery. The rest should be handled through basic Django views, models, etc.

# ? Jun 17, 2015 17:26

Fluue: Jan 2, 2008

Is there any preferred method to install a Django skeleton in Vagrant using Windows? The guides I've been able to pull up require me to install Django on Windows or are written with Linx/OSX in mind rather than Windows. Would rolling my own VM and building from scratch be the way to go if my host machine is Windows?

# ? Jun 22, 2015 01:49

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Fluue posted:

Is there any preferred method to install a Django skeleton in Vagrant using Windows? The guides I've been able to pull up require me to install Django on Windows or are written with Linx/OSX in mind rather than Windows. Would rolling my own VM and building from scratch be the way to go if my host machine is Windows?

How come you want it in a Windows VM? Linux is such a nicer development environment!

# ? Jun 22, 2015 02:17

Fluue: Jan 2, 2008

fletcher posted:

How come you want it in a Windows VM? Linux is such a nicer development environment!

Ah sorry, meant that the VM will be a Linux distro but I'm running Virtualbox on a Windows machine and putting the Django install in a shared folder so I can run an IDE on Windows.

# ? Jun 22, 2015 04:01

Blinkz0rz: May 27, 2001; MY CONTEMPT FOR MY OWN EMPLOYEES IS ONLY MATCHED BY MY LOVE FOR TOM BRADY'S SWEATY MAGA BALLS

Yeah, so create your virtualenv and run pip in the VM and write your actual code in Windows.

# ? Jun 22, 2015 04:17

IAmKale: Jun 7, 2007; やらないか; Fun Shoe

I need help narrowing down a list of objects. For simplicity's sake let's say I have the following two models:

Python code:

class Employee(models.Model):
    email = models.EmailField()

class Submission(models.Model):
    employee = models.ForeignKey(Employee)

Is there a Django-esque way to retrieve all Employees that DON'T currently have a Submission pointing to them? I can't think of how to do it other than retrieve all employees and then iterate through all submissions and pop off employee objects but that sounds really stupid to me.

# ? Jul 16, 2015 23:14

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Karthe posted:

I need help narrowing down a list of objects. For simplicity's sake let's say I have the following two models:
Python code:
class Employee(models.Model):
    email = models.EmailField()

class Submission(models.Model):
    employee = models.ForeignKey(Employee)
Is there a Django-esque way to retrieve all Employees that DON'T currently have a Submission pointing to them? I can't think of how to do it other than retrieve all employees and then iterate through all submissions and pop off employee objects but that sounds really stupid to me.

I haven't worked on Django much in a year, but does something like this work?

Python code:

Employee.objects.filter(submission_set=None)

# ? Jul 16, 2015 23:54

porksmash: Sep 30, 2008

Python code:

Employee.objects.filter(submission=None)

This should work. (just no _set after submission). Here's the relevant doc page: https://docs.djangoproject.com/en/1.8/topics/db/queries/#lookups-that-span-relationships

# ? Jul 17, 2015 01:45

IAmKale: Jun 7, 2007; やらないか; Fun Shoe

Thermopyle posted:

I haven't worked on Django much in a year, but does something like this work?

porksmash posted:

This should work. (just no _set after submission). Here's the relevant doc page: https://docs.djangoproject.com/en/1.8/topics/db/queries/#lookups-that-span-relationships

Thanks for pointing this out to me, I combined it with an .exclude() to narrow down an existing set of Employees:

Python code:

employees = Employee.objects.all().exclude(submission__date_requested=request.data['date'])

But now I have an additional question along these lines. If I want to exclude a range of dates, should I make an .exclude() for each date?

# ? Jul 21, 2015 01:53

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Karthe posted:

Thanks for pointing this out to me, I combined it with an .exclude() to narrow down an existing set of Employees:
Python code:
employees = Employee.objects.all().exclude(submission__date_requested=request.data['date'])
But now I have an additional question along these lines. If I want to exclude a range of dates, should I make an .exclude() for each date?

Naw dawg. Use the range filter thing: https://docs.djangoproject.com/en/1.8/ref/models/querysets/#range

# ? Jul 21, 2015 02:04

Lumpy: Apr 26, 2002; La! La! La! Laaaa!; College Slice

I'm having one of those "I know this is so easy but my brain is broken today and google is giving me lots of results for a slightly different question..." moments.

Given this:

Python code:

class Farm(models.Model):
    ...

class  Cow(models.Model):
    farm = models.ForeignKey(Farm, related_name='cows')
    cowthing = models.ForeignKey(CowThing, null=True, blank=True)

class CowThing(models.Model):
    ...

When I have an instance of a farm, I want to get all the cows that don't have cowthings:

Python code:

farm = get_object_or_404(Farm, pk=1)
all_cows = farm.cows  # neat, but not what I want
thingless_cows = farm.:iiam:  # something like farm.cows.filter(cowthing=None)

I know this is easy, but I have been away from python / Django for a while.

# ? Jul 31, 2015 15:56

Gounads: Mar 13, 2013; Where am I?
How did I get here?

I don't see why your comment wouldn't work...

farm.cows.filter(cowthing=None)

The sql does do a join on CowThing which isn't necessary.

I thought these might fix it:

farm.cows.filter(cowthing_id=None)
farm.cows.filter(cowthing__isnull=True)
farm.cows.filter(cowthing_id__isnull=True)

But all had the same sql... I'm using an older Django, so your results may vary.

They all give the correct result.

# ? Jul 31, 2015 16:11

Lumpy: Apr 26, 2002; La! La! La! Laaaa!; College Slice

Gounads posted:

I don't see why your comment wouldn't work...

farm.cows.filter(cowthing=None)

The sql does do a join on CowThing which isn't necessary.

I thought these might fix it:

farm.cows.filter(cowthing_id=None)
farm.cows.filter(cowthing__isnull=True)
farm.cows.filter(cowthing_id__isnull=True)

But all had the same sql... I'm using an older Django, so your results may vary.

They all give the correct result.

I told you it would be simple.... :smith:

Thanks.

# ? Jul 31, 2015 16:21

IAmKale: Jun 7, 2007; やらないか; Fun Shoe

I'm going to allow end users to POST CSV text to the server to enable bulk user creation. For this I'm going to have a CreateAPIView that takes in the CSV, sanitizes it, then validates each line before iterating through and creating each user. What's the best way of sanitizing the input before doing anything with it? Is this a case for some basic regex? Does Django include anything that could help, something akin to PHP's htmlspecialchars()?

I'm going to set it up so that the end user submits a file containing lines in this format:

pre:

first name, last name, email, phone number, location, station, pay rate, skills
Alma,Bailey,abailey@domain.com,7145550189,West Los Angeles,Cashier,10.50,Books

The only special characters I can think (aside from the commas) that might be necessary are @, . (period), and -. Everything else should be covered by [A-Za-z0-9].

# ? Aug 6, 2015 22:33

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Karthe posted:

I'm going to allow end users to POST CSV text to the server to enable bulk user creation. For this I'm going to have a CreateAPIView that takes in the CSV, sanitizes it, then validates each line before iterating through and creating each user. What's the best way of sanitizing the input before doing anything with it? Is this a case for some basic regex? Does Django include anything that could help, something akin to PHP's htmlspecialchars()?

I'm going to set it up so that the end user submits a file containing lines in this format:
pre:
first name, last name, email, phone number, location, station, pay rate, skills
Alma,Bailey,abailey@domain.com,7145550189,West Los Angeles,Cashier,10.50,Books
The only special characters I can think (aside from the commas) that might be necessary are @, . (period), and -. Everything else should be covered by [A-Za-z0-9].

What exactly are you trying to sanitize out? Django automatically handles what htmlspecialchars does when content is rendered, unless you use the [safe|https://docs.djangoproject.com/en/1.8/ref/templates/builtins/#safe] tag. You shouldn't be storing sanitized input in the database though, just store whatever they give you and then you escape it on the way out.

# ? Aug 6, 2015 22:57

IAmKale: Jun 7, 2007; やらないか; Fun Shoe

fletcher posted:

What exactly are you trying to sanitize out? Django automatically handles what htmlspecialchars does when content is rendered, unless you use the [safe|https://docs.djangoproject.com/en/1.8/ref/templates/builtins/#safe] tag. You shouldn't be storing sanitized input in the database though, just store whatever they give you and then you escape it on the way out.

Django isn't rendering anything here, it's strictly a prereq for Rest Framework do its thing. The View I'm creating is only an endpoint for me to POST CSV content to. The View will insert records into the database using the Serializer for the User class. It'll then return a Response object populated with an object of A) Users that were successfully created or B) Users that had issues that need to be resolved before anyone can be inserted into the database.

I was playing around and I'll probably use something like this to verify each line before anything gets processed: ([^\w\d\s\@\,\.\-])

IAmKale fucked around with this message at 00:05 on Aug 7, 2015

# ? Aug 7, 2015 00:01

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 16:58

Jo: Jan 24, 2005; Soiled Meat

I've got a Postgres database and Django 1.7. I'm adding a non-null foreign key to auth_group to my model Thingy. When makemigrations runs, I see the AddField with correct attributes. Unfortunately, it looks like Postgres is just entirely ignoring the migration for reasons I don't understand. It completes, but afterwards it seems like the changes haven't actually taken place. When I do 'sqlmigrate', I don't see the default group in the migration. Am I SOL as far as that's concerned? I can get around it by just writing the raw SQL, but I'd rather not.

Jo fucked around with this message at 20:10 on Aug 10, 2015

# ? Aug 10, 2015 18:15

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›122 »