Django Web Framework: The 'D' is silent dumbass

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›122 »

epswing: Nov 4, 2003; Soiled Meat

Ahz posted:

Nginx is likely forwarding requests as itself (127.x) so you can tell ngnix to maintain the original host header the request comes in as (domain.com).

OK, interesting. Nginx is still basically magic to me. Is there a keyword I should be looking for in my config file?

Also, any idea why this might be happening "once in a while"?

# ? Apr 26, 2017 16:57

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 23:34

Dominoes: Sep 20, 2007

Hey dudes. Do y'all use Django's forms, or make your own in HTML/JS?

Django recommends you use their forms when able due to validation etc, but it seems like if you're doing something other than a very basic form that matches a model directly, they're a PITA. Ie: Want a date/time picker widget? Dynamic behaviour? A customisable layout? Doable, but a pain.

Dominoes fucked around with this message at 18:11 on Apr 28, 2017

# ? Apr 28, 2017 17:46

Data Graham: Dec 28, 2009; 📈📊🍪😋

Forms are like the last item on my "Django things to make friends with" list.

So much form handling logic these days wants to be ajaxy, or like you say, do more than basic REST/CRUD stuff. But in those cases where what I want to do does fit nicely with the classical pattern, it can be a joy. I love the idea of all my validation errors being returned in a nice marshaled way so I can style them how I want, and the back-end form_valid method being something I can just trust to only get fired when I want it to. And then there's all the niceties of localization.

Though... when you say "make your own in HTML/JS", you don't mean you're doing all your form validation logic in the front-end, do you? :raise:

One of Django forms' biggest benefits is that it makes it a low bar to doing your logic in the back-end and keeping you from being tempted to move it into JS, with all the security issues that implies. When I struggle with whether to use Django forms, it's "as opposed to building my own form handling logic in views" versus "as opposed to using a front-end solution".

Apologies if I'm jumping to conclusions.

# ? Apr 28, 2017 18:03

a witch: Jan 12, 2017

You can do ajaxy stuff and still have the data validated by django forms. Every piece of data coming into your application should be going through a form.

# ? Apr 28, 2017 18:15

Dominoes: Sep 20, 2007

That sounds familiar. It's been a different reason that drives it each time. For example, today I switched from a Django model form to an HTML form submitted via AJAX since I wanted date/time widgets, a checkbox that automatically sets the time to a certain thing and disables the inputs, and customized dropdowns. Another case was having items be read-only on the page, but have a button that turns them into dropdowns and input fields in an 'edit' mode. I suppose this all depends on the use case.

These things could be handled by Django, but AFAIK, the API's not there. (It wouldn't surprise me if we get elegant input-widget handling in a future release; it seems so obvious.)

# ? Apr 28, 2017 18:17

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

a witch posted:

You can do ajaxy stuff and still have the data validated by django forms. Every piece of data coming into your application should be going through a form.

I've never used Django forms before. Been doing everything with Django Rest Framework & React. Should I still be using Django forms for that?

# ? Apr 28, 2017 19:59

a witch: Jan 12, 2017

I don't have any experience with Django Rest Framework sorry.

# ? Apr 28, 2017 20:04

Data Graham: Dec 28, 2009; 📈📊🍪😋

DRF and Django Forms don't really coexist super well. You can build forms in the DRF-endorsed way, but it's all built on its own serializers and doesn't give you validators like the Django native forms do.

You can try to stitch them together but it's kind of unsatisfying: https://ejosh.co/de/2014/02/django-rest-framework-and-forms/

For user-facing stuff I usually stick with regular Form views, and keep my DRF views for building API endpoints.

# ? Apr 28, 2017 20:17

MonkeyMaker: May 22, 2006; What's your poison, sir?

DRF's serializers basically _are_ Django forms. You're good.

RE: forms and AJAX-y stuff. Django forms have had their own media for a long time and you can set custom widget types per field or per field type. Look that up, it can save you a lot of work. *And* with Django 1.11, form widget rendering is now done through HTML and not Python so it should be a lot easier to tweak. I haven't played with that any myself, though, so don't quote me.

# ? Apr 29, 2017 05:03

Maluco Marinero: Jan 18, 2001; Damn that's a
fine elephant.

We use a node server to render all our templates with json as data fed to React, so we had to figure out what to do with forms a fair while back. It's pretty straightforward to make a mixin class that turns a form into a JSON representation, so you can use Django to be the single source of truth and then client/template rendering can use that truth as basis for its own rendering or client validation rules.

Maluco Marinero fucked around with this message at 05:56 on May 1, 2017

# ? Apr 30, 2017 05:34

Dominoes: Sep 20, 2007

Hey dudes, I'm looking for info on what Django Rest framework is used for. Context: I was reading about using Django and React together (AFAICT it's straightforward; just put a <div> in your template that ReactDOM's render func points to), and almost all results pointed to Django Rest Framework. From skimming their site, it seems like it's supposed to clean up messes when passing AJAX data to/from the front end?

ie avoid awkward deserializing code like this?

Python code:

    data = dict(request.POST)

    date = saturn.from_iso(data['date'][0])
    lines = json.loads(data['lines'][0])
    sims = json.loads(data['sims'][0])
    duties = json.loads(data['duties'][0])
    meetings = json.loads(data['meetings'][0])

    data_py = {'date': date, 'lines': lines, 'sims': sims, 'duties': duties, 'meetings': meetings}

So you could call a serializer from DRF to just import the POST data as a dict? And maybe more functionality than django.http's JsonResponse for sending serialized data?

# ? May 1, 2017 00:56

xpander: Sep 2, 2004

Dominoes posted:

Hey dudes, I'm looking for info on what Django Rest framework is used for. Context: I was reading about using Django and React together (AFAICT it's straightforward; just put a <div> in your template that ReactDOM's render func points to), and almost all results pointed to Django Rest Framework. From skimming their site, it seems like it's supposed to clean up messes when passing AJAX data to/from the front end?

ie avoid awkward deserializing code like this?
Python code:
    data = dict(request.POST)

    date = saturn.from_iso(data['date'][0])
    lines = json.loads(data['lines'][0])
    sims = json.loads(data['sims'][0])
    duties = json.loads(data['duties'][0])
    meetings = json.loads(data['meetings'][0])

    data_py = {'date': date, 'lines': lines, 'sims': sims, 'duties': duties, 'meetings': meetings}
So you could call a serializer from DRF to just import the POST data as a dict? And maybe more functionality than django.http's JsonResponse for sending serialized data?

Serialization is just one aspect of DRF and similar packages. In general, the idea is to offer a REST API and separate the presentation from your data layer while still taking advantage of Django's many strengths(ORM, user system, etc). In the endgame of this setup, you would likely keep your HTML/CSS/JS as its own separate application that merely talks to the API served up by Django/DRF. There are no HTML templates anywhere in existence in Django, all it does is respond to requests with serialized JSON(this is basically what a REST API is). You can choose your own adventure and make use of DRF to whatever extent you want though. Disclaimer: I haven't used it all that much so I might be missing out on some of its intricacies.

# ? May 1, 2017 19:51

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I'm sure most of you know about Classy Class Based Views, the site that helps you understand how to use CBV's by giving a clearer picture of the methods and inheritance structure.

I've often wished for something similar for DRF's views and serializers...and somehow I never found that there is such a thing!

# ? May 29, 2017 21:21

Data Graham: Dec 28, 2009; 📈📊🍪😋

Thermopyle posted:

I'm sure most of you know about Classy Class Based Views, the site that helps you understand how to use CBV's by giving a clearer picture of the methods and inheritance structure.

I didn't, but I'm sure glad I do now!

# ? May 29, 2017 21:31

Dominoes: Sep 20, 2007

I'm looking for wisdom on encrypting DB fields. I stumbled acrossthis module called Django extensions, which offers an EncryptedCharField doing what I'd like; however, it requires on a package called KeyCzar, which no longer works. Does anyone know how to handle encrypting DB fields with Django?

# ? Jun 13, 2017 16:58

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Are you sure you want to encrypt them? I only ask because 9 out of 10 times someone wants to do something like this it turns out that encrypting a db field doesn't actually meet their security requirements.

A big reason this is often the case is that if someone has access to your database, they probably have access to the means to decrypt the data.

# ? Jun 13, 2017 17:53

Dominoes: Sep 20, 2007

I'm working on a scheduling webapp for a government agency as part of a quasi-official function at work. I'd just like to encrypt first/last and organization names. I'm forcing an HTTPS connection, and it seems like the Heroku/AmazonAWS database setup is fairly secure. This is unknown territory, and is a gray area. ie software like this is traditionally hosted locally or semi-locally, takes years to make, costs millions of dollars, sucks, and the devs don't know what users want. The info's not particularly sensitive, but the names are considered PII, especially when associated with the org. Do you have any recommendations, encryption or otherwise?

Dominoes fucked around with this message at 20:51 on Jun 13, 2017

# ? Jun 13, 2017 20:37

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

If you encrypt the field, you have to be able to decrypt it to serve it in views or whatever. This means that the key for decryption is stored alongside the database.

If your threat model is a targeted attack on your application to get such data, then you're basically gaining nothing by encrypting the field. If they access the data through your website, the data is already decrypted at that point. Seeing as how the database is managed by Heroku/Amazon, the likely way they'd gain access is by compromising your access credentials to Heroku at which point they've also got access to your decryption key.

If your threat model includes someone indiscriminately vacuuming up your database by compromising Heroku/Amazon without actually your site being targeted then I suppose you gain something by encrypting your fields. Its of note, that on production tiers Heroku/Amazon already encrypts the data volumes including your database...so what you're wanting to do with a single field is basically already done for your whole db. A compromise would have to be specific enough to catch the data on the fly or getting access to Amazon's key storage.

I don't think you gain any worthwhile security by encrypting the fields yourself. However, you really need to know what the relevant specifications require of you.

Also, you shouldn't take my advice on this because I'm not an information security expert. Maybe my rambling will help you ask better questions of whoever you need to ask about what your actual requirements or responsibilities actually are.

All that being said...I have no advice on the best way to actually encrypt the fields if you decide you need to. If it was me I'd probably just manually encrypt and decrypt in my views. I've done this before when a client required me to encrypt the data and it was pretty easy.

# ? Jun 13, 2017 21:57

Dominoes: Sep 20, 2007

Thermopyle posted:

Its of note, that on production tiers Heroku/Amazon already encrypts the data volumes including your database...so what you're wanting to do with a single field is basically already done for your whole db. A compromise would have to be specific enough to catch the data on the fly or getting access to Amazon's key storage.

...

Maybe my rambling will help you ask better questions of whoever you need to ask about what your actual requirements or responsibilities actually are.

Thank you v much for the detailed explanation. It sounds like this may be the heart of your answer. I'd mainly like to be able to answer questions from higher-ups when they ask about security/PII. So far, I've answered by saying the server's secure, and the connection is encrypted, which seemed to be acceptable thus far.

# ? Jun 13, 2017 22:06

NtotheTC: Dec 31, 2007

I've inherited an old Django 1.6 project that uses Rest Framework for it's API and the response times for some of the larger queries seem absolutely ridiculous. We're talking 10+ seconds to fetch ~3000 records. I initially thought that under the hood there must be some horrible nested many-to-many clusterfuck with no prefetching but the nesting is just two foreign keys deep and it doesn't really hit the database that hard, the majority of the time is spent within the view/serializer code.

The time appears to be related to building hyperlinks, most of the serializers are HyperlinkedModelSerializers, which I haven't used much of but I can understand why they're useful for interacting with the API in the front end JS. I'm surprised that a few calls to reverse() is adding 6+ seconds on to a query time though, and even more surprised that this isn't a problem that occurs more often. Has anyone had more experience than me with larger querysets using DRF? Do you have to abandon any attempts at convenience and scrap all nesting/hyperlinks when you reach 1000+ records? I could switch to using .values() but that again is taking a massive chunk of the convenience out of using DRF in the first place.

# ? Jun 14, 2017 16:26

Ahz: Jun 17, 2001; PUT MY CART BACK? I'M BETTER THAN THAT AND YOU! WHERE IS MY BUTLER?!

NtotheTC posted:

I've inherited an old Django 1.6 project that uses Rest Framework for it's API and the response times for some of the larger queries seem absolutely ridiculous. We're talking 10+ seconds to fetch ~3000 records. I initially thought that under the hood there must be some horrible nested many-to-many clusterfuck with no prefetching but the nesting is just two foreign keys deep and it doesn't really hit the database that hard, the majority of the time is spent within the view/serializer code.

The time appears to be related to building hyperlinks, most of the serializers are HyperlinkedModelSerializers, which I haven't used much of but I can understand why they're useful for interacting with the API in the front end JS. I'm surprised that a few calls to reverse() is adding 6+ seconds on to a query time though, and even more surprised that this isn't a problem that occurs more often. Has anyone had more experience than me with larger querysets using DRF? Do you have to abandon any attempts at convenience and scrap all nesting/hyperlinks when you reach 1000+ records? I could switch to using .values() but that again is taking a massive chunk of the convenience out of using DRF in the first place.

I found that when I need high performance with DRF, I drop all convenience features and keep my queries raw if necessary and serializers as manual as possible. I find DRF often likes to run with n² or worse complexity on joins/linked data when you can restructure and serialize data yourself with manual data linking/joining @ 2n complexity or better.

# ? Jun 14, 2017 16:52

porksmash: Sep 30, 2008

N+1 query orgies can be easily solved with a select_related or prefetch_related. I have a related URL field that requires the username of the person who owns an object related to the object I'm serializing and it's 4 levels deep, but it has minimal performance impact doing it this way:

code:

queryset = MonsterInstance.objects.filter(owner__public=True).select_related(
        'monster',
        'owner__user',
    ).prefetch_related(
        'runeinstance_set',
        'runeinstance_set__owner__user',
    )

# ? Jun 14, 2017 17:34

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

NtotheTC posted:

I've inherited an old Django 1.6 project that uses Rest Framework for it's API and the response times for some of the larger queries seem absolutely ridiculous. We're talking 10+ seconds to fetch ~3000 records. I initially thought that under the hood there must be some horrible nested many-to-many clusterfuck with no prefetching but the nesting is just two foreign keys deep and it doesn't really hit the database that hard, the majority of the time is spent within the view/serializer code.

The time appears to be related to building hyperlinks, most of the serializers are HyperlinkedModelSerializers, which I haven't used much of but I can understand why they're useful for interacting with the API in the front end JS. I'm surprised that a few calls to reverse() is adding 6+ seconds on to a query time though, and even more surprised that this isn't a problem that occurs more often. Has anyone had more experience than me with larger querysets using DRF? Do you have to abandon any attempts at convenience and scrap all nesting/hyperlinks when you reach 1000+ records? I could switch to using .values() but that again is taking a massive chunk of the convenience out of using DRF in the first place.

In addition to the previous suggestions, might also be worth trying to update to the latest Django, DRF, and Python versions to see if has any improvement.

# ? Jun 14, 2017 18:01

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

fletcher posted:

In addition to the previous suggestions, might also be worth trying to update to the latest Django, DRF, and Python versions to see if has any improvement.

This is one of the first things I do when I inherit a project if the environment and client supports it.

Upgrading Django versions are easy.
DRF pre-3.0 to post-3.0 can be hard depending on features in use.
Python same as DRF.

# ? Jun 14, 2017 18:14

icantfindaname: Jul 1, 2008

Is there any way in DRF Docs to format the docstring instead of just spitting it out on a single line?

http://www.django-rest-framework.org/topics/documenting-your-api/

http://demo.drfdocs.com/

So for example for the first endpoint, when it says "A view that allows users to login providing their username and password.", can you add like a line break to that? I see you can edit the template in teh library itself to turn off raw HTML cleaning, but that can't be deployed.

# ? Jun 20, 2017 21:13

icantfindaname: Jul 1, 2008

Okay, no takers for that it seems. Does anyone have any help for using the built-in documentation feature added in 3.6?

http://www.django-rest-framework.org/topics/3.6-announcement/

I'm not sure how to get it to recognize endpoints besides the first one available

This is what I get when I just do the quickstart steps and add these two lines

code:

from rest_framework.documentation import include_docs_urls

urlpatterns = [
    ...
    url(r'^docs/', include_docs_urls(title='My API title'))
]

I have a bunch of other endpoints though, under the v1 header. It's supposed to automatically recognize them, right? How do I do that?

edit: OK, I figured out it recognizes URL endpoints only if they are linked from the top level api/views.py file. How do you get endpoint functions in other files and subdirectories to be recognized?

I have an api/urls.py file that handles the first level of the file path, including the docs/ endpoint, and this then includes another urls.py file, apps/main/urls.py for the v1/... path.

icantfindaname fucked around with this message at 19:08 on Jun 22, 2017

# ? Jun 22, 2017 18:15

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

I haven't messed around with the auto doc features, but I seem to recall they only work for Viewsets, not functions.

# ? Jun 22, 2017 20:20

icantfindaname: Jul 1, 2008

They are viewsets, they're accessed with as_view(). It's a bunch of these

code:

apps/main/urls.py

...
urlpatterns = [
     url(r'device/list', api.devices.DeviceList.as_view(), name='get_all_devices'),
...

# ? Jun 22, 2017 22:38

huhu: Feb 24, 2006

I'm trying to run Django locally and access a database hosted at webfaction.com.

code:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'test',
        'USER': 'test',
        'PASSWORD': ' test',
        'HOST': 'web123.webfaction.com',
        'PORT': '3306'
    }
}

Resulting in:

code:

django.db.utils.OperationalError: (1045, "Access denied for user 'test'@'c-65-96-169-6.hsd1.ma.comcast.net' (using password: YES)")

# ? Jun 22, 2017 23:59

Tigren: Oct 3, 2003

huhu posted:

I'm trying to run Django locally and access a database hosted at webfaction.com.

code:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'test',
        'USER': 'test',
        'PASSWORD': ' test',
        'HOST': 'web123.webfaction.com',
        'PORT': '3306'
    }
}

Resulting in:

code:

django.db.utils.OperationalError: (1045, "Access denied for user 'test'@'c-65-96-169-6.hsd1.ma.comcast.net' (using password: YES)")

Looks like access is denied for user test. Check permissions on your webfaction database?

Edit:

A very quick google turns up:

quote:

If you need remote access enabled for your database, you just need to send us a ticket requesting remote access and the database name you need it enabled for.

Otherwise, they suggest using an SSH tunnel to connect "locally"

https://docs.webfaction.com/user-guide/databases.html#databases-accessing-remotely

Tigren fucked around with this message at 00:10 on Jun 23, 2017

# ? Jun 23, 2017 00:02

huhu: Feb 24, 2006

Tigren posted:

Looks like access is denied for user test. Check permissions on your webfaction database?

Edit:

A very quick google turns up:

Otherwise, they suggest using an SSH tunnel to connect "locally"

https://docs.webfaction.com/user-guide/databases.html#databases-accessing-remotely

But why is it trying to connect to "c-65-96-169-6.hsd1.ma.comcast.net"?

# ? Jun 23, 2017 00:28

The Fool: Oct 16, 2003

huhu posted:

But why is it trying to connect to "c-65-96-169-6.hsd1.ma.comcast.net"?

It's not, that's the host you're connecting from.

# ? Jun 23, 2017 00:36

huhu: Feb 24, 2006

The Fool posted:

It's not, that's the host you're connecting from.

Ah ok. That's where my confusion came from. Thanks for clarifying.

# ? Jun 23, 2017 00:42

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Personally I would opt for the ssh tunnel rather than enabling remote access for the DB. No sense in increasing your attack surface unnecessarily.

# ? Jun 23, 2017 01:01

Data Graham: Dec 28, 2009; 📈📊🍪😋

Especially since we all now have your IP for as long as your lease lasts :nsa:

# ? Jun 23, 2017 01:36

epswing: Nov 4, 2003; Soiled Meat

I'm reading about python-social-auth and I think it's what I want, but I'm not sure. I'd like to use it with G Suite (formerly Google Apps for Business), to allow employees to login to a webapp with the same credentials they'd use for email.

Anyone used python-social-auth before, and know if this is possible?

# ? Jun 26, 2017 21:13

Prancing Shoes: Jul 8, 2008

epalm posted:

I'm reading about python-social-auth and I think it's what I want, but I'm not sure. I'd like to use it with G Suite (formerly Google Apps for Business), to allow employees to login to a webapp with the same credentials they'd use for email.

Anyone used python-social-auth before, and know if this is possible?

I haven't used python-social-auth before, but this sounds like something I would just use django-allauth along with the Google social account provider for.

https://django-allauth.readthedocs.io/en/latest/providers.html#google

# ? Jun 26, 2017 21:39

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

epalm posted:

I'm reading about python-social-auth and I think it's what I want, but I'm not sure. I'd like to use it with G Suite (formerly Google Apps for Business), to allow employees to login to a webapp with the same credentials they'd use for email.

Anyone used python-social-auth before, and know if this is possible?

I use that library for exactly that purpose. Last week I migrated from the old python-social-auth to the new social-app-django (which took like 5 minutes, very painless). My experience with that library has been very positive and I highly recommend it.

django-allauth looks pretty good too, I have not used it before though.

# ? Jun 26, 2017 22:19

PT6A: Jan 5, 2006; Public school teachers are callous dictators who won't lift a finger to stop children from peeing in my plane

I used allauth for a project and I can confirm it works very painlessly.

# ? Jun 26, 2017 22:30

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 23:34

epswing: Nov 4, 2003; Soiled Meat

I'd already taken a few steps in the python-social-auth direction, I continued and got it working within 30 minutes. Quite painless, although if I wanted to nitpick, the documentation is a bit daunting (there are a ton of options and tweaks) and lacks "flow", and could benefit from a minimum working example. (If there is one, I didn't see it.)

# ? Jun 27, 2017 01:23

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›122 »