Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Portland Sucks
Dec 21, 2004
༼ つ ◕_◕ ༽つ
Having some 403 issues with attempting to pass a CSRF token. I've run through every article and discussion I can find on SO about it but I'm running into a brick wall.

According to the django docs I've done this bit regarding creation of the token and passing it into the header

code:
function getCookie(name) {
            var cookieValue = null;
            if (document.cookie && document.cookie !== '') {
                var cookies = document.cookie.split(';');
                for (var i = 0; i < cookies.length; i++) {
                    var cookie = jQuery.trim(cookies[i]);
                    // Does this cookie string begin with the name we want?
                    if (cookie.substring(0, name.length + 1) === (name + '=')) {
                        cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
                        break;
                    }
                }
            }
            return cookieValue;
        }
        var csrftoken = getCookie('csrftoken');

        function csrfSafeMethod(method) {
            // these HTTP methods do not require CSRF protection
            return (/^(GET|HEAD|OPTIONS|TRACE)$/.test(method));
        }
        $.ajaxSetup({
            beforeSend: function(xhr, settings) {
                if (!csrfSafeMethod(settings.type) && !this.crossDomain) {
                    xhr.setRequestHeader("X-CSRFToken", csrftoken);
                }
            }
        });
then my little ajax function to just test this out looks like

code:
        function update_table(){
            $.ajax({
                method: 'POST',
                contentType: 'application/json',
                url: "get_table",
                data: {search_results: search_results },
                success: function(data){
                    alert("ajax success");
                    console.log(data)
                }
            })
        }
the associated html is just

code:
{% csrf_token %}
 <input id="testButton" type="submit" value="testButton" onclick="update_table();"/>
and i've given the view being called a @ensure_csrf_cookie decorator

When I inspect the 403 error I can see the csrftoken being generated in the cookies request header cookie.

Is there something I'm overlooking? From what I understand the csrf middleware should be handling the validation, and I don't need to worry about receiving and processing the cookie do I?

The specific error I'm getting is just the "Forbidden (CSRF token missing or incorrect.)" error.

Adbot
ADBOT LOVES YOU

PT6A
Jan 5, 2006

Public school teachers are callous dictators who won't lift a finger to stop children from peeing in my plane

Portland Sucks posted:

Having some 403 issues with attempting to pass a CSRF token. I've run through every article and discussion I can find on SO about it but I'm running into a brick wall.

According to the django docs I've done this bit regarding creation of the token and passing it into the header

code:
function getCookie(name) {
            var cookieValue = null;
            if (document.cookie && document.cookie !== '') {
                var cookies = document.cookie.split(';');
                for (var i = 0; i < cookies.length; i++) {
                    var cookie = jQuery.trim(cookies[i]);
                    // Does this cookie string begin with the name we want?
                    if (cookie.substring(0, name.length + 1) === (name + '=')) {
                        cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
                        break;
                    }
                }
            }
            return cookieValue;
        }
        var csrftoken = getCookie('csrftoken');

        function csrfSafeMethod(method) {
            // these HTTP methods do not require CSRF protection
            return (/^(GET|HEAD|OPTIONS|TRACE)$/.test(method));
        }
        $.ajaxSetup({
            beforeSend: function(xhr, settings) {
                if (!csrfSafeMethod(settings.type) && !this.crossDomain) {
                    xhr.setRequestHeader("X-CSRFToken", csrftoken);
                }
            }
        });
then my little ajax function to just test this out looks like

code:
        function update_table(){
            $.ajax({
                method: 'POST',
                contentType: 'application/json',
                url: "get_table",
                data: {search_results: search_results },
                success: function(data){
                    alert("ajax success");
                    console.log(data)
                }
            })
        }
the associated html is just

code:
{% csrf_token %}
 <input id="testButton" type="submit" value="testButton" onclick="update_table();"/>
and i've given the view being called a @ensure_csrf_cookie decorator

When I inspect the 403 error I can see the csrftoken being generated in the cookies request header cookie.

Is there something I'm overlooking? From what I understand the csrf middleware should be handling the validation, and I don't need to worry about receiving and processing the cookie do I?

The specific error I'm getting is just the "Forbidden (CSRF token missing or incorrect.)" error.

I'm not sure what's going wrong, but the alternate way to do that (and easier, IMHO), is to use javascript to retrieve the token from the element created by {% csrf_token %} and then add it to the data parameter in your AJAX call manually.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb
I want to generate a CSV and then write it to a Django FileField (which is using django-storages to upload to s3):

code:
from django.core.files import File
from django.core.files.temp import NamedTemporaryFile

with NamedTemporaryFile(mode='w') as f:
    writer = csv.DictWriter(f, fieldnames=['one', 'two', 'three'], delimiter=',')
    writer.writeheader()
    f.flush()
    my_model = MyModel()
    my_model.my_field.save('whatever.csv', File(f))
Tried that but I get:
code:
RuntimeError: Input <tempfile._TemporaryFileWrapper object at 0x7fa904bb7390> of type: <class 'tempfile._TemporaryFileWrapper'> is not supported.

Hed
Mar 31, 2004

Fun Shoe
I think you need to use one of the components underneath the wrapper to use it that way. I remember the docs being tricky with that. I can look tomorrow, but also try doing a dir() on that wrapper object to see what your options are.

Eleeleth
Jun 21, 2009

Damn, that is one suave eel.

Hed posted:

I think you need to use one of the components underneath the wrapper to use it that way. I remember the docs being tricky with that. I can look tomorrow, but also try doing a dir() on that wrapper object to see what your options are.

Fairly sure you're correct and he just wants to change the last line to

code:
my_model.my_field.save('whatever.csv', File(f.file))
To unwrap the object and get the underlying file. From the tempfile docs: "The returned object is always a file-like object whose file attribute is the underlying true file object."

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

Eleeleth posted:

Fairly sure you're correct and he just wants to change the last line to

code:
my_model.my_field.save('whatever.csv', File(f.file))
To unwrap the object and get the underlying file. From the tempfile docs: "The returned object is always a file-like object whose file attribute is the underlying true file object."

Thanks for the responses guys. I tried that one and got:
code:
.../lib/python3.4/site-packages/storages/backends/s3boto3.py", line 466, in _save_content
AttributeError: 'str' object has no attribute 'seek'
What did end up working was:
code:
File(open(f.name, "rb"))
I mean it works but this seems waaay too convoluted

Portland Sucks
Dec 21, 2004
༼ つ ◕_◕ ༽つ
Anyone have experience setting up Django/RabbitMQ/Celery on a windows server? I think I've got everything set up as it should be, but attempting to start workers gives me a WinError 10042 response. I've looked around on google and people are just sort of abstractly saying it may be an issue running on windows even though all three say they supposedly support being ran under windows?

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. Bertrand Russell

fletcher posted:

I mean it works but this seems waaay too convoluted

I haven't actually paid much attention to what your problem here is, but I just wanted to say that in Django writing to filefields is way too convoluted. It's a weird and bad API.

PT6A
Jan 5, 2006

Public school teachers are callous dictators who won't lift a finger to stop children from peeing in my plane

Thermopyle posted:

I haven't actually paid much attention to what your problem here is, but I just wanted to say that in Django writing to filefields is way too convoluted. It's a weird and bad API.

God yes. It feels like the behaviour is way too inconsistent to actually make sense on a regular basis.

IAmKale
Jun 7, 2007

やらないか

Fun Shoe
Has anyone here used django-treebeard in a project? I'm needing to go down this route for a personal project and am looking for any caveats from anyone who's practically incorporated it.

hanyolo
Jul 18, 2013
I am an employee of the Microsoft Gaming Division and they pay me to defend the Xbox One on the Something Awful Forums
I have a reasonably simple model which I'm having trouble trying to display/edit in a form correctly. For example:

code:
class Host (models.Model):
    system = models.ForeignKey(System, on_delete=models.CASCADE)
    name_text = models.CharField(max_length=255, verbose_name='Host')

class System (models.Model):
    name_text = models.CharField(max_length=255, verbose_name='Sub-System')

class Ticket(models.Model):
    system = models.ForeignKey(System, on_delete=models.CASCADE)
    host = models.ManyToManyField(Host)
    description_text = models.CharField(max_length=255, verbose_name='Description')
I've built a crispy form page which I want to use to edit a ticket, but what happens is that all the hosts in all the systems are displayed. I just want to see the particular systems hosts for the ticket instance I'm editing.

Is there a way to display/update the form just based on the system in the ticket instance? Or is my modelling incorrect?

I've created a function in the Ticket model to get the relevant hosts and if they're currently selected as well, is there any way to pass and read that to/from my crispy form?

code:
    def get_host_list(self):
        hosts = self.system.host_set.all()
        hostlist = {}
        for i in hosts:
            exists = False
            for j in self.host.all():
                if (i == j):
                    hostlist[i] = True
                    exists = True
                    break
            if (exists == False):
                hostlist[i] = False
        return hostlist

hanyolo fucked around with this message at 06:25 on Mar 2, 2018

Data Graham
Dec 28, 2009

📈📊🍪😋



What does your form class look like?

What I typically do is specify the form field with a "queryset=None" attribute, and then in the __init__ method for the form I set the queryset for the field based on extra code, like this:

code:
class TicketForm(forms.ModelForm):
    host = forms.ModelChoiceField(required=False, queryset=None, empty_label='(Select host)')

    def __init__(self, *args, **kwargs):
        super(TicketForm, self).__init__(*args, **kwargs)
        self.fields['host'].queryset = Host.objects.filter(system=self.system)
Something like that? Crispy should use this same form code, right? Shooting from the hip here.

hanyolo
Jul 18, 2013
I am an employee of the Microsoft Gaming Division and they pay me to defend the Xbox One on the Something Awful Forums
Ah nice yeah that looks good and makes sense, ill give it a go when im back at work, cheers!

Edit: didn't work for me, was getting "TicketForm has no attribute 'system' error". Googled around on stack overflow and found a solution:

code:
self.fields['host'].queryset = Host.objects.filter(system=kwargs['instance'].system.pk)

hanyolo fucked around with this message at 04:15 on Mar 5, 2018

Data Graham
Dec 28, 2009

📈📊🍪😋



I finally got sick of telling clients look youll probably have to flush your cache before the new feature will work, hold down shift and refresh and decided to buckle down and write some kind of static file caching busting thing. Figured I would store a time stamp of the last modification date of every important js and css file in the settings, maybe pass them to all the templates through a context processor. Pain in the rear end but I knew it would be worth it in the end.

Ten minutes of reading later and Im using https://docs.djangoproject.com/en/2.0/ref/contrib/staticfiles/#manifeststaticfilesstorage

drat I love batteries included :allears:


E: wtf, preview ate all my quotes and apostrophes

epswing
Nov 4, 2003

Soiled Meat
For some reason I can't seem to google the right words.

If I have a url like
Python code:
url(r'^blog/$', views.blog, name='blog'),
and I hit http://127.0.0.1/blog (without trailing slash), Django will issue a 301 and redirect me to http://127.0.0.1/blog/ (with trailing slash).

How do I get Django to not do that, and just return a 404 instead?

Edit: Oh I think I'm just looking for APPEND_SLASH = False

epswing fucked around with this message at 00:32 on Mar 7, 2018

The Fool
Oct 16, 2003


https://docs.djangoproject.com/en/2.0/ref/settings/#append-slash

Hed
Mar 31, 2004

Fun Shoe
Should be able to set this: https://docs.djangoproject.com/en/2.0/ref/settings/#append-slash

epswing
Nov 4, 2003

Soiled Meat

Yep thanks, I was looking all over for stuff about redirection and I missed the obvious setting.

huhu
Feb 24, 2006
Why would I need to specify a queryset and serializer_class for a viewset that only accepts POST? Or should I not be using a viewset in this situation?

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

huhu posted:

Why would I need to specify a queryset and serializer_class for a viewset that only accepts POST? Or should I not be using a viewset in this situation?

I think you may just want to use a CreateAPIView if it only accepts POST: http://www.django-rest-framework.org/api-guide/generic-views/#createapiview

NtotheTC
Dec 31, 2007


If we assume the benefits of turning a Django app into a separate DRF backend and <js framework of choice> frontend outweigh the cons in general- what's a good way to handle site configuration?

So I have a site that I want to reuse for lots of different clients, but the clients have different requirements, say for example that on one form, a client wants the telephone number field to be mandatory whereas another doesn't. A simple flag for the form / DRF view validation is easy enough but what's a smart (and scalable ) way of passing that sane information to the front end without tightly coupling anything together? Obviously I want to avoid creating a new git repository for each client.

This is more of an architecture question than django specific I guess

huhu
Feb 24, 2006
Why not just create a template repo and fork it for each site? I imagine you'd spend so much more time on architecture.

NtotheTC
Dec 31, 2007


huhu posted:

Why not just create a template repo and fork it for each site? I imagine you'd spend so much more time on architecture.

Do you mean a Django template repo? I don't really like the django templating language. For a few simple forms it's fine but when you want client side validation etc it quickly becomes unmanageable.

Data Graham
Dec 28, 2009

📈📊🍪😋



No, like a basic common project repo that you keep cloning new projects from, like a template.

Thats how I usually go about it, but of course if you have common development you want to feed into all your extant projects it can get out of hand pretty quick.

Ahz
Jun 17, 2001
PUT MY CART BACK? I'M BETTER THAN THAT AND YOU! WHERE IS MY BUTLER?!
I wouldn't go multi-tenant architecture unless that's a key requirement for your system and you are designing this business around this concept.

If you just want some deployment conveniences and think multi-tenant will go, It will probably cause you more problems in the long run. What do you do when one client hogs all your I/O or CPU?

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

Ahz posted:

I wouldn't go multi-tenant architecture unless that's a key requirement for your system and you are designing this business around this concept.

If you just want some deployment conveniences and think multi-tenant will go, It will probably cause you more problems in the long run. What do you do when one client hogs all your I/O or CPU?

Another thing to consider would be that when you add some feature for one customer, you need to make sure it doesn't introduce regressions for other customers.

I think the common project repo to use as a starting point would be the way to go.

porksmash
Sep 30, 2008
Does anyone know a way to coerce django-filter to use a certain lookup expression without that lookup expression being part of the filter key when passing it into the filter? For example, a model with a CharField of name. I want the lookup to always be name__icontains, but I don't want people putting ?name__icontains=fred in the URL query parameters. I it to be ?name=fred, while django-filter performs name__icontains='fred' in the background.

I know I could just resort to a method to perform the filter but that seems like a last resort.

huhu
Feb 24, 2006
Anyone ever experience the Django manage.py runserver stop updating content? If I go into the Django Admin, change an entry, and save it, it doesn't show up in the DRF api until I restart runserver.

fletcher
Jun 27, 2003

ken park is my favorite movie

Cybernetic Crumb

huhu posted:

Anyone ever experience the Django manage.py runserver stop updating content? If I go into the Django Admin, change an entry, and save it, it doesn't show up in the DRF api until I restart runserver.

Never had that issue before, that sounds very very odd.

porksmash
Sep 30, 2008
Cache issue maybe? I try to always use the dummy cache in dev environment

Data Graham
Dec 28, 2009

📈📊🍪😋



Does the same thing happen with templated data? What about through the manage.py shell?

Would want to narrow it down to a DRF problem, a webserver problem, or a data layer problem...

huhu
Feb 24, 2006
Like the rubber duck, I posted here and the problem resolved itself. At least for now.

Another question, I have a bunch of portfolio projects, that already exist in the DB, that need images. Going manually one by one and uploading images is kind of annoying. However, everything I'm finding on Google is confusing or horribly outdated. Any suggestions?

Data Graham
Dec 28, 2009

📈📊🍪😋



Can you upload them all into a directory, write a mapping text file listing which ones go where/with which records, and then write a Django management script to move them into the appropriate places in your media dir (if necessary) and update the image field (and width/height) in your model?

Its roll-your-own but its how Ive been spending my week so

Data Graham
Dec 28, 2009

📈📊🍪😋



Okay seriously, why does the awful app keep eating my quotes and apostrophes? In this thread only?

Its maddening

E: okay so if I look at those same posts on desktop it's fine. Smart quotes getting displayed as blanks in the awful app only I guess?

Data Graham fucked around with this message at 19:25 on Mar 16, 2018

huhu
Feb 24, 2006

Data Graham posted:

Can you upload them all into a directory, write a mapping text file listing which ones go where/with which records, and then write a Django management script to move them into the appropriate places in your media dir (if necessary) and update the image field (and width/height) in your model?

Its roll-your-own but its how Ive been spending my week so

Ooh this is a brilliant idea. Thanks!

huhu
Feb 24, 2006
Edit: When you solve something and there isn't really an aha moment. Working Code:
code:
        if options['upload']:
            root_photos_dir = os.path.join(BASE_DIR, INPUT_DIR)
            projects = Project.objects.all()
            p = projects[0]
            project_dir = os.path.join(root_photos_dir, p.name)
            for img_name in os.listdir(project_dir):
                file_name = os.path.join(project_dir, img_name)
                file_data = open(file_name, 'rb')

                f = File(name=img_name, file=file_data, )
                i = Image(name=img_name, src=f)
                i.save()
                p.image.add(i)
            p.save()
So I've been chugging along with your suggestion. However I am a bit stuck.

I've got a file dir:

code:
BASE_DIR/
    /inputs
        /project1
            img.jpg
            img2.jpg
Where project1 is the name used in the Project model.

code:
import os

from django.core.files import File
from django.core.management.base import BaseCommand
from api.models import Project, Image
from eng40.settings import BASE_DIR

INPUT_DIR = 'inputs'


class Command(BaseCommand):
	# ...

    def handle(self, *args, **options):
        if options['upload']:
            root_photos_dir = os.path.join(BASE_DIR, INPUT_DIR)
            projects = Project.objects.all()
            p = projects[0] # just trying to get this working on one project first
            project_dir = os.path.join(root_photos_dir, p.name)
            for img_name in os.listdir(project_dir):
                file_name = os.path.join(project_dir, img_name)
                f = File(open(file_name, 'rb'))
                i = Image(name=img_name, src=f)
                i.save()
                p.image.add(i)
            p.save()
Project Model:
code:
class Project(models.Model):
    name = models.CharField(max_length=200, unique=True)

Image Model:
code:
class Image(models.Model):
    name = models.CharField(max_length=200)
    src = models.ImageField(blank=True, null=True)
    project = models.ForeignKey(Project, on_delete=models.CASCADE, blank=True, null=True, related_name='image')
Which results in
code:
django.core.exceptions.SuspiciousFileOperation: The joined path (/Users/huhu/Documents/Programming/eng40-api/inputs/Water System/ewb3_TUCqUWh.jpg) 
is located outside of the base path component (/Users/huhu/Documents/Programming/eng40-api/media)
I can't really tell, and I'm not finding much help with Google, on why I can't upload images from outside the project dir (obviously something security related but beyond that no idea).

huhu fucked around with this message at 23:52 on Mar 16, 2018

Data Graham
Dec 28, 2009

📈📊🍪😋



The media/ dir is special; ImageField expects a value which is a filename relative to your MEDIA_ROOT, which is media/. So assuming youre keeping all the project images in a single directory (which you may or may not want to rethink later on, depending on volume and security and other factors), you should set the value of your src field to just be the bare name of the file, no path. Move the file into the media dir, use PIL.Image to get its width and height (your Image class should have width and height fields too) and set those, and then save.

Phone posting from the hip but that should get it going.

huhu
Feb 24, 2006

Data Graham posted:

Does the same thing happen with templated data? What about through the manage.py shell?

Would want to narrow it down to a DRF problem, a webserver problem, or a data layer problem...

Alright so it appears to be an issue with DRF. Django Admin and Django shell both show the updated content but not DRF.

Data Graham posted:

The media/ dir is special; ImageField expects a value which is a filename relative to your MEDIA_ROOT, which is media/. So assuming you’re keeping all the project images in a single directory (which you may or may not want to rethink later on, depending on volume and security and other factors), you should set the value of your src field to just be the bare name of the file, no path. Move the file into the media dir, use PIL.Image to get its width and height (your Image class should have width and height fields too) and set those, and then save.

Phone posting from the hip but that should get it going.


Data Graham posted:

The media/ dir is special; ImageField expects a value which is a filename relative to your MEDIA_ROOT, which is media/. So assuming you’re keeping all the project images in a single directory (which you may or may not want to rethink later on, depending on volume and security and other factors), you should set the value of your src field to just be the bare name of the file, no path. Move the file into the media dir, use PIL.Image to get its width and height (your Image class should have width and height fields too) and set those, and then save.

Phone posting from the hip but that should get it going.

Good to know, thanks.

epswing
Nov 4, 2003

Soiled Meat
We're building a DRF app with a <js framework> frontend. We also want the nice browsable API and API Index (e.g. Swagger) to be accessible.

In settings.py, we have both TokenAuthentication and SessionAuthentication available:

Python code:
REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework.authentication.TokenAuthentication',
        'rest_framework.authentication.SessionAuthentication',
    ),
    'DEFAULT_PERMISSION_CLASSES': [
        'rest_framework.permissions.IsAuthenticated',
    ],
}
In urls.py we have url patterns for rest_framework views, and swagger views (along with other patterns, not shown here, that expose the api of the actual app):

Python code:
from rest_framework.schemas import get_schema_view
from rest_framework_swagger.views import get_swagger_view
from rest_framework.authtoken import views as authtoken_views

urlpatterns = [
    url(r'^schema/$', get_schema_view(title='Schema Index')),
    url(r'^auth/', include('rest_framework.urls', namespace='rest_framework')),
    url(r'^api/$', get_swagger_view(title='API Index')),
    url(r'^api/token/$', authtoken_views.obtain_auth_token),
"Sometimes", the frontend, when POSTing to the api (in fact, sometimes when POSTing to "api/token/" to acquire a token to use in future requests) the server responds with

quote:

Forbidden (403) CSRF verification failed. Request aborted. Reason given for failure: CSRF token missing or incorrect."

If you open up a different browser, or an incognito tab, or clear your cache/cookies, and try again, the problem goes away.

I know Token auth is meant for communication from a totally separate frontend. And we are building a separate frontend, however... the JS frontend happens to be used in the same browser with the same session variables sitting around that the same user is using to access the default DRF browsable API, and the Swagger API Index.

In the reddit post DRF's TokenAuthentication and SessionAuthentication can't co-exist? Tom Christie says

quote:

You should only be seeing that error if you're making a request from a session authenticated client. Try making the token request from the command line or native app (eg not in the context of an AJAX request by a signed-in user)

Based on the StackOverflow post Django REST Framework: using TokenAuthentication with browsable API
and my own testing, it appears that the browsable API cannot be used without enabling SessionAuthentication.

Question 1: Is the act of simply using the default DRF browsable API and/or Swagger's API Index causing later requests from a JS frontend (running in the same browser) to inadvertently go down DRF's SessionAuthentication pipeline (instead of using TokenAuthentication, as intended)?

Question 2: Should a JS frontend accessing DRF via TokenAuthentication be expected to co-exist with the default DRF browsable API and/or Swagger's API Index using SessionAuthentication in the same browser?

Adbot
ADBOT LOVES YOU

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. Bertrand Russell

I'm busy right now so I didn't read your post in detail, but...

If your SPA request doesn't send cookies, then SessionAuthentication will be avoided.

If you're using fetch to do your requests on the frontend look into the credentials config on requests.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply