Django Web Framework: The 'D' is silent dumbass

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

Worth mentioning that if you're using virtualenv (which is very good idea) to develop Django apps, you specify the Python version with a flag when you create the env.

# ¿ Jan 2, 2019 19:30

Adbot: ADBOT LOVES YOU

# ¿ Apr 29, 2024 10:09

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

I use WhiteNoise for local static files.

# ¿ Sep 14, 2019 22:31

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

I need a page to send an AJAX request to add a task to a queue (Celery, Huey, etc), wait for the task to complete, then grab the results and display it. What's the best Django-friendly way to let the page know that the task is done?

Looks like my options are to:
- periodically poll the server for an update
- use Comet "long polling" (which seems obsolete)
- use WebSockets so that the server can just send a signal to tell me when it's done.

I've never done any of these before. What's best practice, are there any helpful libraries, and any pitfalls to watch out for?

# ¿ Mar 28, 2020 00:47

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

The first option seems like the simplest. I don't want to run into hard-to-debug connection issues with WebSockets and Comet.

Did you roll the AsyncTask model yourself, or is that in some library somewhere? This kind of thing seems like a super common use-case but I can't seem to find much sample code or best practices out there.

Edit: Also how do your tasks communicate with the AsyncTask model to update their progress?

"Channels" seems to be the go-to Django library for doing WebSockets and general ASGI stuff, but I think that might be overkill for my simple use-case.

# ¿ Mar 28, 2020 01:01

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

Oh that's really nice. Generic, simple, doesn't require the worker to do any crazy bus messaging to signal that it's done. And it'd be trivial to write a task monitor page. Thanks for the write up!

# ¿ Mar 28, 2020 02:29

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

How are the pages grabbing the CSRF value? The vanilla way to do it is for each form to have a hidden CSRF <input> field. But AJAX requests typically pull that value out of the CSRF cookie. If the failures are only on AJAX requests, it could be something messing with the value of that cookie; either the AJAX library itself, or some ad-blocker or other filter. I think (without looking it up) that per-session CSRFs are stored in each user's Django session, so double check that's working correctly too - if the persistent session storage area is full then it won't be able to persist the CSRF session data (doesn't explain why it's only happening to a subset of users though).

# ¿ May 6, 2020 17:39

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

Next thing I'd dig into then would be the nature of each rejection. Like, is the CSRF value not present, or corrupted, or invalid? And if it's invalid, why exactly? Because it got replaced by another one, because it expired, because it doesn't match the server key?

# ¿ May 6, 2020 18:14

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

Point it at an empty database and run:

manage.py makemigrations
manage.py migrate

and let Django populate the schema of the empty database. Then compare the 2 schemas in Postgres and see what you're missing.

Worse comes to worst, just manually copy over the data from the existing DB into the new empty one.

# ¿ May 19, 2020 06:23

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

Are you logging the request & response headers? (If that's too verbose, you could only do it from the IP/UserAgent of the problematic users).

It's a long-shot, but if your site's cookies go over 8kB total then the browser won't send them.

# ¿ May 25, 2020 00:19

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

I'd look at Cookie: and Set-Cookie: since those are the ones that are going to be storing your sessionid and CSRF tokens. If either of those change between the first POST and the 2nd then that might explain the behavior, if not the root cause.

Edit: yeah and Location too. A blank Location: is definitely a server-side problem, no idea how a browser would react to that.

Also I think I asked this before, but did you ever determine exactly how the CSRF was failing? Missing, corrupted, expired, etc?

minato fucked around with this message at 01:42 on May 26, 2020

# ¿ May 26, 2020 01:39

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

This sounds like the TCP connection is getting reset. Server receives + processes request, sends response, but the client only receives a TCP RST. Browser shrugs and assumes the POST never got through, so it sends it again, according to HTTP 1.1 8.2.4 Client Behavior if Server Prematurely Closes Connection. This would explain why it's not consistently reproducible, and only occurs to certain groups of users.

The RST could be getting sent by the server, a load balancer, a TLS terminator, NAT gateway, or a corporate web proxy. Wiresharking would prove the theory, but you'd have to be tapping the packets right in the middle; wiresharking on the server wouldn't necessarily show any RSTs getting sent.

Assuming that's the cause, you can try to solve it at the network level or the software level. As you may not have much control over the network level, you might just have to adapt the CSRF checker to allow dupes. You don't want replay attacks, so maybe get the CSRF checker to somehow add the sessionid into the CSRF token. That way an attacker would have to steal both the sessionid and the CSRF token to successfully execute an XSS.

# ¿ May 26, 2020 05:33

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

No, you can't just skip the CSRF check if the user is authenticated. CSRF attacks rely on the user being authenticated. CSRF is just a way of saying "the form this user just POSTed was created by the server, not by some hacker" so whatever you change has to fundamentally support that.

A quick look at Django's CSRF FAQ says this:

quote:

Why might a user encounter a CSRF validation failure after logging in?
For security reasons, CSRF tokens are rotated each time a user logs in. Any page with a form generated before a login will have an old, invalid CSRF token and need to be reloaded. This might happen if a user uses the back button after a login or if they log in a different browser tab.

The default django.contrib.auth code calls rotate_token. If you didn't rotate the token after login then this would solve your problem, but reduce security a little bit. For an intranet app or an app where people must login to access any functionality, this is probably fine.

# ¿ May 26, 2020 14:24

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

And just a word of warning; removing "rotate_token" on login will "fix" this problem but it won't stop double-POSTing due to wonky TCP connections. So on every other form on your site that results in creation of some resource, you'll need to add code to detect the double-POST.

One technique; if the POST is going to create something with a unique ID, use a UUID and insert it into the form itself, don't just use the DB's next available sequence number. The server's POST processor should pre-check to see if the UUID already exists in the DB, and if so just redirects the user to the resource's page. That way if someone double-POSTs, the 2nd request won't create a 2nd resource.

# ¿ May 26, 2020 14:44

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

I would create a manage.py command to create the user (there's already a built-in one that creates a superuser, you could base your implementation off that sample code), and the built-in "changepassword" management command to set their new password, and then tie it all together with a local bash script that would SSH into the server:

code:

#!/bin/bash
new_user=blah
new_password=blah
ssh someuser@myserver "/bin/bash" <<EOF
  set -o errexit
  manage.py create_user "${new_user}"
  manage.py changepassword "${new_user}" "${new_password}"
EOF

# ¿ Jun 5, 2020 20:37

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

I have what I think is a common problem, but I can't seem to find a solution.

I want to show 2 form fields where the first decides whether the 2nd is displayed. e.g.

code:


Field 1: (Radio buttons)
( ) Foo
( ) Bar
( ) Baz

Field 2: (only visible if "Baz" is selected)
[ <some CharField> ]

The standard solution is to use Javascript to show/hide Field 2 when "Baz" is selected/de-selected. But this seems fragile, because now I have to write JS code in the template that's tightly-coupled to the Form definition but yet is in a different file. And if someone renames "Baz" or adds a new item to Field 1, it might break that JS code. Without comprehensive tests, it's going to be easy to break this code.

I have many forms like this so I'd like a generic way to solve it. What I'd like to do is programmatically define the relationship between the Field 1 & 2 in the Form definition, and have some general-purpose JS parse that relationship and handle the client-side show/hiding. It feels like this problem is common enough that someone must have solved it, but I can't seem to find any libraries that would assist here. Any ideas?

# ¿ Jun 10, 2020 02:01

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

It's also used as the seed for the built-in CRSF protection that prevents phishing attacks.

# ¿ Sep 12, 2020 20:37

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

Django is designed so that static assets (JavaScript, CSS, images) can be served by a CDN, i.e. a separate host to your webserver so that it's not bogged down with small easily-cacheable requests. However not everyone has a CDN so a lot of people use a plugin like whitenoise to serve these assets from the same web server as the Django app itself.

For media uploaded via users, you need to build a view to actually serve those; i.e. something that will parse a URL to determine which file to serve, pull the data from wherever it's stored, and send it to the user. Django has a built-in view for this that's convenient for development but isn't suitable for production for various reasons.

But you could write your own. E.g. If you have a model "MyImage", you might write a view with url pattern "/view_image/<int:model_id>/", and the view might look something like:

code:

import mimetypes
from django.http import FileResponse

def view_image(request: HttpRequest, model_id: int) -> FileResponse:
    model = get_object_or_404(MyImage, model_id)
    fullpath = MEDIA_PATH + model.file_name   # This is more psuedo-code than actual code
    content_type, encoding = mimetypes.guess_type(str(fullpath))
    content_type = content_type or 'application/octet-stream'
    response = FileResponse(fullpath.open('rb'), content_type=content_type)
    if encoding:
        response.headers["Content-Encoding"] = encoding
    return response

That's really basic, there's probably a better way to do it; almost certainly there's some Django plugin or utility that can do a better job, handling cache headers, Last-Modified headers, etc.

# ¿ May 4, 2022 15:50

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

I'd be a little nervous about that, because Django implements some funky magic with its app-discovery / import system, and lots of places import the settings.py file so anything non-standard there might be tricky to debug.

It also smells bad to me that __init__.py is being used for anything except making it easier for other modules to import whatever "public functions/classes/types" exist in the __init__.py's directory. __init__.py is the last place I expect to see code, and it always irks me whenever I find significant code and constants defined there.

I might be wrong on this, maybe there's some popular alternative way of managing Django settings and this PR is just following best practices. But I'd feel more comfortable if they could justify that decision by pointing to some docs that explain the alternative and why it's better.

# ¿ Mar 30, 2023 23:54

Adbot: ADBOT LOVES YOU

# ¿ Apr 29, 2024 10:09

minato: Jun 7, 2004; cutty cain't hang, say 7-up.; Taco Defender

Jose Cuervo posted:

I believe the code does not work because I am nesting the "{{ }}" (once around the url_for function and once around the jsFilename). Is this the issue? If this is the issue, how do I deal with it?

Try:

code:

... src={{ url_for('static', filenamt='js/reports/' ~ jsFilename ~ '.js') }}

and maybe this will work too:

code:

... src={{ url_for('static', filenamt='js/reports/' ~ (jsFilename | safe) ~ '.js') }}

# ¿ Oct 19, 2023 15:58

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass