Django Web Framework: The 'D' is silent dumbass

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›4 »

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

I've got a simple little Django app and I want to switch out MySQL for PostgreSQL. I've been fiddling around with mysqldump, psql, and a bunch of random migration scripts I found through google to try and get this working, but it's not going very well so far.

I don't really give a poo poo about migrating the schema, I just want the data. I was thinking about just making a little python script that dumps all my model fields to a CSV, and then another little script to read the CSV and save the new records.

Is there something I can use to make this easier? It seems like a simple script in itself, just wanted to double check before I went down that path.

I don't care about migrating the auth/user/etc tables, just my custom models.

# ¿ Nov 3, 2016 00:38

Adbot: ADBOT LOVES YOU

# ¿ May 15, 2024 10:09

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Maluco Marinero posted:

You could try using manage.py dumpdata. That dumps all your data into JSON files that can then run into manage.py loaddata.

That way you avoid having to worry about conversion from MySQL to Postgres, however be aware constraints are not relaxed while you use loaddata, so it's important to segment your loaddata json payloads sometimes so you don't get integrity errors on foreign keys. As long as you have a nice environment to run some tests you can just smash away at it til you find a sequence that works.

https://docs.djangoproject.com/en/1.10/ref/django-admin/#dumpdata

Note that you can be very specific dumping single models and apps and things.

Glad I asked, that looks like exactly what I was looking for

Thanks Maluco!

e: quoted since we're on a new page

# ¿ Nov 3, 2016 00:49

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Dr. Dos posted:

I used the excuse of having to change hosting providers to start porting all my sites to use Python 3 instead of 2 and get them all running on the latest version of Django and so far everything has gone very smoothly with one exception.

I can't for the life of me find an openid library that supports Python 3 and is up to date enough to not call a bunch of deprecated/removed django functions. Does anybody know of one that's up to date (for a consumer only, specifically logging in to a site via Steam)

I use python-social-auth with Python 3 and the latest version of Django & Django Rest Framework. It "just works" and I have never had to fiddle with it.

# ¿ Nov 12, 2016 01:47

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Where do you guys put your signals code and how do you make sure it gets imported?

I want to stick mine in a separate signals.py file and came across this solution but wasn't sure if that's the right way to go

# ¿ Nov 16, 2016 20:50

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Thermopyle posted:

Yeah, this is the way to go.

Make sure you follow all the steps! I often forget to put the default_app_config part in app/__init__.py

Apparently they tell you to avoid setting default_app_config now, and instead just list it in INSTALLED_APPLICATIONS: https://docs.djangoproject.com/en/1.9/ref/applications/#configuring-applications

It seems to all be working now...just seems a little hacky

And my signals are definitely being registered multiple times now (as warned in the docs), fortunately they are idempotent though.

# ¿ Nov 17, 2016 02:12

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Thermopyle posted:

I just came across intercooler.js, which Django users might be interested in.

This is pretty cool, thanks for sharing! Kinda reminds me of JSF

# ¿ Jan 18, 2017 19:43

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

a witch posted:

You can do ajaxy stuff and still have the data validated by django forms. Every piece of data coming into your application should be going through a form.

I've never used Django forms before. Been doing everything with Django Rest Framework & React. Should I still be using Django forms for that?

# ¿ Apr 28, 2017 19:59

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

NtotheTC posted:

I've inherited an old Django 1.6 project that uses Rest Framework for it's API and the response times for some of the larger queries seem absolutely ridiculous. We're talking 10+ seconds to fetch ~3000 records. I initially thought that under the hood there must be some horrible nested many-to-many clusterfuck with no prefetching but the nesting is just two foreign keys deep and it doesn't really hit the database that hard, the majority of the time is spent within the view/serializer code.

The time appears to be related to building hyperlinks, most of the serializers are HyperlinkedModelSerializers, which I haven't used much of but I can understand why they're useful for interacting with the API in the front end JS. I'm surprised that a few calls to reverse() is adding 6+ seconds on to a query time though, and even more surprised that this isn't a problem that occurs more often. Has anyone had more experience than me with larger querysets using DRF? Do you have to abandon any attempts at convenience and scrap all nesting/hyperlinks when you reach 1000+ records? I could switch to using .values() but that again is taking a massive chunk of the convenience out of using DRF in the first place.

In addition to the previous suggestions, might also be worth trying to update to the latest Django, DRF, and Python versions to see if has any improvement.

# ¿ Jun 14, 2017 18:01

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Personally I would opt for the ssh tunnel rather than enabling remote access for the DB. No sense in increasing your attack surface unnecessarily.

# ¿ Jun 23, 2017 01:01

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

epalm posted:

I'm reading about python-social-auth and I think it's what I want, but I'm not sure. I'd like to use it with G Suite (formerly Google Apps for Business), to allow employees to login to a webapp with the same credentials they'd use for email.

Anyone used python-social-auth before, and know if this is possible?

I use that library for exactly that purpose. Last week I migrated from the old python-social-auth to the new social-app-django (which took like 5 minutes, very painless). My experience with that library has been very positive and I highly recommend it.

django-allauth looks pretty good too, I have not used it before though.

# ¿ Jun 26, 2017 22:19

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Lumpy posted:

Has anyone integrated these social auth things with Django Rest Framework? Does it "just work"? Any experiences with social-app-django or allauth along side DRF that would steer me one way or the other?

There was some discussion in the thread recently about this: https://forums.somethingawful.com/showthread.php?threadid=2790475&pagenumber=102&perpage=40#post466375208

For me python-social-auth "just works" along side DRF with almost no futzing around.

# ¿ Jul 17, 2017 22:34

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Harik posted:

Anyone develop django inside visual studio? I'm trying to figure out how to run it external to VS, and still attach the debugger to it. Apparently that's possible, and it makes things like auto-reload work properly so development is faster.

Alternately, is there a better IDE that integrates well with django? VS can't follow the model magic so none of the fields show up on intellisense. I'm not an IDE guy normally (they make me want to break things), but I'm trying to use it enough to be able to help the new guys get up-to-speed.

PyCharm (JetBrains) is a fantastic IDE for Python & Django. Definitely check it out.

# ¿ Jul 21, 2017 09:44

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

epalm posted:

I was going to get PyCharm with that 30% discount where proceeds go to the Django project, until I remembered that PyCharm is now a "subscription service". I seem to remember the Internet crying about this a year or two ago, and I thought JetBrains went back on it and said "ok ok we'll still offer standalone licenses" but I'm not seeing that option on their site.

Is everyone still outraged about this, or is this just how things are nowadays.

You only have to keep paying the subscription fee if you want new versions. Otherwise you can continue to use whatever old version you have. Read more about it here: https://sales.jetbrains.com/hc/en-gb/articles/207240845-What-is-perpetual-fallback-license-

It seems like a perfectly reasonable subscription model to me.

# ¿ Sep 5, 2017 22:49

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Thermopyle posted:

Personally, I'd just go with python-rq because redis is easy to install and maintain and I have a strong bias against small projects (python-41 has 4k+ stars on github, django-background-tasks has 117). They're more likely to become un-maintained, less likely to have security issues found and patched, you're less likely to be able to get help, etc...

python-rq looks pretty cool.

I agree with the github stars thing most of the time. Occasionally there is a small project that is very simple and does what I need, like this one I've been using for years that has 66 stars and hasn't been updated since 2015: https://github.com/defrex/django-after-response

# ¿ Oct 4, 2017 19:09

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

I want to generate a CSV and then write it to a Django FileField (which is using django-storages to upload to s3):

code:

from django.core.files import File
from django.core.files.temp import NamedTemporaryFile

with NamedTemporaryFile(mode='w') as f:
    writer = csv.DictWriter(f, fieldnames=['one', 'two', 'three'], delimiter=',')
    writer.writeheader()
    f.flush()
    my_model = MyModel()
    my_model.my_field.save('whatever.csv', File(f))

Tried that but I get:

code:

RuntimeError: Input <tempfile._TemporaryFileWrapper object at 0x7fa904bb7390> of type: <class 'tempfile._TemporaryFileWrapper'> is not supported.

# ¿ Feb 22, 2018 02:19

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Eleeleth posted:

Fairly sure you're correct and he just wants to change the last line to
code:
my_model.my_field.save('whatever.csv', File(f.file))
To unwrap the object and get the underlying file. From the tempfile docs: "The returned object is always a file-like object whose file attribute is the underlying true file object."

Thanks for the responses guys. I tried that one and got:

code:

.../lib/python3.4/site-packages/storages/backends/s3boto3.py", line 466, in _save_content
AttributeError: 'str' object has no attribute 'seek'

What did end up working was:

code:

File(open(f.name, "rb"))

I mean it works but this seems waaay too convoluted

# ¿ Feb 22, 2018 20:53

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

huhu posted:

Why would I need to specify a queryset and serializer_class for a viewset that only accepts POST? Or should I not be using a viewset in this situation?

I think you may just want to use a CreateAPIView if it only accepts POST: http://www.django-rest-framework.org/api-guide/generic-views/#createapiview

# ¿ Mar 10, 2018 21:26

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Ahz posted:

I wouldn't go multi-tenant architecture unless that's a key requirement for your system and you are designing this business around this concept.

If you just want some deployment conveniences and think multi-tenant will go, It will probably cause you more problems in the long run. What do you do when one client hogs all your I/O or CPU?

Another thing to consider would be that when you add some feature for one customer, you need to make sure it doesn't introduce regressions for other customers.

I think the common project repo to use as a starting point would be the way to go.

# ¿ Mar 13, 2018 20:57

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

huhu posted:

Anyone ever experience the Django manage.py runserver stop updating content? If I go into the Django Admin, change an entry, and save it, it doesn't show up in the DRF api until I restart runserver.

Never had that issue before, that sounds very very odd.

# ¿ Mar 16, 2018 02:26

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Is it possible to package up an entire django project?

In the docs I saw this page about packaging an app: https://docs.djangoproject.com/en/2.0/intro/reusable-apps/#packaging-your-app

In that case it seems like it's just packaging up "polls" - I want to package up all of "mysite" though, settings, templates, etc

I was thinking this would simplify deployment a bit, turning it into just a "pip install my_site" followed by a collectstatic & migrate

# ¿ May 9, 2018 02:38

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Lumpy posted:

git?

Yeah that's what I'm currently doing, would prefer to pull a build artifact from internal pypi server rather than source control though.

# ¿ May 9, 2018 08:08

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

I'm thinking that given how little information I am finding out there about how to do this, it may be a sign. I am surprised nobody else is doing this though...

# ¿ May 10, 2018 01:09

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Lumpy posted:

Yeah, I mean:
code:
git clone
pip install -r reqs.txt
./manage.py migrate
Isn't exactly tedious.

Yeah, agreed it's definitely not tedious the existing way I'm doing it.

I guess part of the problem is that deployments are usually done during off hours, and occasionally this means that our git server is also undergoing maintenance at the same time we want to do a deploy to production. It'd be nice to eliminate that dependency, with an s3 backed pypi repo I wouldn't have to worry about it. I can achieve the same goal just my creating an archive and uploading it somewhere, it just seemed like something that setup.py would have been able to handle for me.

fletcher fucked around with this message at 06:24 on May 10, 2018

# ¿ May 10, 2018 06:17

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Thermopyle posted:

Should I just start out geodjango, or just migrate to it when I need more capabilities?

I'm in kind of the same boat right now, but with a Java project that we started out with storing GeoJSON in a char field. Now we are migrating it to PostGIS for those additional capabilities, I wish we had just started out with PostGIS to begin with, would have eliminated some of this extra overhead of doing the migration. It's not a huge issue, but I'd say maybe just start out with geodjango if you know that's where it is headed anyways.

I had never heard of geodjango before, looks pretty neat.

# ¿ May 24, 2018 21:25

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

epalm posted:

I keep reading horror stories about those dev contractor sites. Where would be a reasonable place to look for a django dev to convert an old site from Python 2 and Django 1.11 using function-based views to Python 3 and Django 2.2 using class-based views? I'd like to jump from one LTS version to the next, and 2.2 has been pre-released (aiming to be fully released this April). It's a 24 page internal site, mostly just a LOB app with lists of records users can open, view, and edit.

I know exactly what I want, and is something I could do myself, but right now I have more money than time.

It's intern season! Hire yourself an intern from a decent school and have them bang it out over the summer under your supervision.

# ¿ Feb 1, 2019 23:42

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

minato posted:

The standard solution is to use Javascript to show/hide Field 2 when "Baz" is selected/de-selected. But this seems fragile, because now I have to write JS code in the template that's tightly-coupled to the Form definition but yet is in a different file. And if someone renames "Baz" or adds a new item to Field 1, it might break that JS code. Without comprehensive tests, it's going to be easy to break this code.

I have many forms like this so I'd like a generic way to solve it. What I'd like to do is programmatically define the relationship between the Field 1 & 2 in the Form definition, and have some general-purpose JS parse that relationship and handle the client-side show/hiding. It feels like this problem is common enough that someone must have solved it, but I can't seem to find any libraries that would assist here. Any ideas?

Django Forms are great for their simplicity, but I don't think they are really meant for anything more complex than the absolute basics. For that sort of behavior you would definitely need some client side code. Looks like django-dynamic-form-fields would at least abstract it away from you.

# ¿ Jun 10, 2020 08:05

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Sulla Faex posted:

Say I want to track Presents and Wishlists, which is a list of Presents. Presents can be created without being on a Wishlist (sorry kids), but I also want the possibility to, when creating a Present, automatically add it to a Wishlist.

I would switch up the schema a bit, the list of Presents seems odd to me. Do you need a Present to be part of multiple Wishlists?

I was thinking it should be along the lines of:

code:

class WishList(models.Model):
	name = models.CharField(max_length=100)

class Present(models.Model):
	name = models.CharField(max_length=100)
        wishlist = models.ForeignKey(Wishlist, related_name='presents', blank=True, null=True)

In your UI code I would just do two separate API calls on the flow where they want to create a Present and add it to a new Wishlist, one to create the WishList and then one to create the Present associated with that WishList. Or if they are associating the present to an existing WishList, just hit the endpoint to create the Present and specify the WishList value. Easy access to wishlist.presents in the ORM if you want the whole wishlist.

# ¿ Nov 30, 2020 21:22

Adbot: ADBOT LOVES YOU

# ¿ May 15, 2024 10:09

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

lazerwolf posted:

I will organize my settings files in a folder called settings with a pattern like:
code:
settings
- __init__.py  # this is empty
- base.py
- local.py
- staging.py
- production.py
Base contains general settings across all environments. Then I separate out environment specific settings in each other file.

This is the way

# ¿ Mar 31, 2023 05:36

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Django Web Framework: The 'D' is silent dumbass

«‹›4 »