Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

I just skimmed your post, because I've been doing that a lot lately for some reason, and it's been awhile since I've done something requiring celery, but...

You've got to get the task id from where you instantiate it to where you want to query the task state. django-celery doesn't really have any mechanism to tie tasks or task results to a specific user.

So yes, you need to store task ids belonging to users in some manner.

Then, when the user uses a view that needs the task results or status, you can do this:

http://docs.celeryproject.org/en/latest/faq.html#how-do-i-get-the-result-of-a-task-if-i-have-the-id-that-points-there

Adbot
ADBOT LOVES YOU

Monkey Fury
Jul 10, 2001

Thermopyle posted:

Anybody in here tried to set up and use any of the various solutions out there for Websockets + Django? How'd it go?

It seems like everything is kinda hacky and a lot of effort so I haven't actually tried anything yet...

I'm actually leaning towards just writing something completely separate from my Django project that interacts with its already existing REST API, and serves it out out over its own websocket implementation.

Have you looked at Pusher at all? Their free tier includes 200k messages a day. We use it + DRF at work to do some data streaming with Kinesis, and it's pretty great.

Also found this on their site: https://blog.pusher.com/django-pusherable/

darthbob88
Oct 13, 2011

YOSPOS
Working on my first Django project, after years of C# and ASP.NET, and I've got two questions that feel stupid.
1. Where do I put the program state? Working in ASP.NET, it was easy enough to add a simple C# object to handle data structures that didn't really belong in the database, but I can't find anything in Django docs on how to do that. Specifically, I need some place to put the Primary Data Structures listed on that README; order of jobs to be done, the calculated average time each person spends in the queue, and how much money is in the machine. The sort of data that does belong in the backend model, but not in the existing model classes, and which will be referenced enough places that I'm hesitant to just put it in the views. I'm considering using this dude's singleton model, but want a second opinion and/or other advice.
2. I think I just found the answer, but how do I attach additional data to the authentication user model? In that project, I need a Customer class with a time value and some other info, and I would like to connect that to the authentication system and their User class, if only to avoid having an "And which customer are you again?" field on some of the forms. The linked section is probably the way to do it, but not 100% sure.

Data Graham
Dec 28, 2009

📈📊🍪😋



Yeah, that's pretty much what you do. An extension model with a OneToOneField that extends the base User class, and then you'll have those fields whenever you have a request.user object.

Sharktopus
Aug 9, 2006

https://www.youtube.com/watch?v=EP6l1SZwF7Q

I made a short 15m video explaining a recent django CVE yall might like

Any and all feedback is really appreciated!

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Sharktopus posted:

https://www.youtube.com/watch?v=EP6l1SZwF7Q

I made a short 15m video explaining a recent django CVE yall might like

Any and all feedback is really appreciated!

I didn't watch it all because I'm a child of the internet, but the first half was informative!

(I also find myself searching a page of code on github not infrequently...try searching "def get_format" to bypass all the spurious results you get when you search for "get_format")

German Joey
Dec 18, 2004

Eleeleth posted:

This is what I did, it works a lot better than trying to integrate websockets into django, in my limited experience wrestling with things like swampdragon.

what problems have you been having with swampdragon? I'm using it for my current project, and I like it quite a lot so far.

IAmKale
Jun 7, 2007

やらないか

Fun Shoe
What would be a good way to manage a collection of temporary session hashes? I've started working on a goon auth server that will require goons to post an auth hash to their profile. What I want to do, though, is create an eligible hash/username combo (defined in a Model) around only for a short period of a time, a few minutes, before it gets cleared from the database. Is there a way to automate the nuking of the record after X minutes? Would it be easier to just make a cronjob that calls a python script that Model.objects.filter()s anything later than time + X minutes and deletes them?

Edit: the correct answer is a cronjob and a custom management command: https://docs.djangoproject.com/en/1.9/howto/custom-management-commands/

IAmKale fucked around with this message at 08:40 on Mar 5, 2016

Gounads
Mar 13, 2013

Where am I?
How did I get here?
A cron job that runs every few minutes is probably overkill.

Don't use a database, use a caching server with an expiration set for that. Memcache or redis are great.

Alternatively, in your model have an expiration field and alter the default query set so it only ever gets unexpired ones. Something like:
http://stackoverflow.com/questions/2494501/modify-default-queryset-in-django

Then you can have a daily or weekly job that cleans up.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Yeah, I wouldn't use the database.

That's being said, I always first think of task queues like Python-RQ and Celery when I need to do Django stuff outside of the normal request-response cycle.

IAmKale
Jun 7, 2007

やらないか

Fun Shoe

Gounads posted:

Don't use a database, use a caching server with an expiration set for that. Memcache or redis are great.

Thermopyle posted:

Yeah, I wouldn't use the database.

That's being said, I always first think of task queues like Python-RQ and Celery when I need to do Django stuff outside of the normal request-response cycle.
Talk about a hammer making everything look like a nail :unsmith:

Thanks for pointing me towards Redis, I've never used it before but it looks simple to use (it's basically in-memory JSON?) and I like the ability to set expirations on keys to have them disappear automatically. That should make it really easy to keep things tidy.

WINNINGHARD
Oct 4, 2014

Demonachizer posted:

Do any of you know if there is a way to change the order than django does a clean() on form data? Essentially I have a form in the order I want and it validates email to come from a certain domain. I would like it to skip that validation if the user is from outside as a guest but the 'position' field is must lower in the form. When I do position = self.cleaned_data.get('position') it stores None to position. I have tested tossing position ahead of the email field and everything processes through just fine.

If there is a way to clean() position earlier somehow it would be super helpful.

Maybe I'm misunderstanding - you can validate form fields individually before they're validated with clean. It's especially useful when composing together form mixins.
code:
class EmailMixin(forms.Form):
    email = forms.EmailField()

    def clean_email(self):
        data = self.cleaned_data['email']
        domains = ['bad webzone']
        if data in domains:
            raise forms.ValidationError('error message')
        return data

class PositionMixin(forms.Form):
    position = forms.CharField()

    def clean_position(self):
       #etc etc

class FakeForm(PositionMixin, EmailMixin):

    def clean(self):
          #do more stuff here

e: I didn't explain that fully - to validate an individual field before the form is validated, you have to name the method in a specific way: clean_FIELDNAME. https://docs.djangoproject.com/en/1.9/ref/forms/validation/#cleaning-a-specific-field-attribute

WINNINGHARD fucked around with this message at 12:51 on Mar 6, 2016

epswing
Nov 4, 2003

Soiled Meat
Back in 2006 between semesters at school, I wrote a web app for a small company. It's a LOB, standard parent-child table relationships (maybe 30 tables), simple log in/out system, written in Java for Tomcat using JSPs, database is MS Access (was a requirement, lol). The drat thing has been in production ever since, and is still ticking (though after moving to a new office, they're not sure which office computer is actually running the web server).

The plan is to rewrite it in Django (I'm currently maintaining a half dozen Django sites, all using the latest 1.8 release).

1) Should I stick with 1.8 because LTS?

2) I need to write a one-time migration program to move data from Access to Postgres. In the old system the "id" primary key value matters, on some tables. For example, they use it as a reference number when printing (and later billing) service reports, so I need to keep this value around. If I want to continue using it as a pk, I need to somehow migrate the data to the target DB with a non-auto-incrementing pk, and then post-migration set it to auto-incrementing. Does that sound about right?

At least I'm rewriting my own legacy system, rather than some other legacy system written by some other crazy person :v:

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

epalm posted:

Back in 2006 between semesters at school, I wrote a web app for a small company. It's a LOB, standard parent-child table relationships (maybe 30 tables), simple log in/out system, written in Java for Tomcat using JSPs, database is MS Access (was a requirement, lol). The drat thing has been in production ever since, and is still ticking (though after moving to a new office, they're not sure which office computer is actually running the web server).

The plan is to rewrite it in Django (I'm currently maintaining a half dozen Django sites, all using the latest 1.8 release).

1) Should I stick with 1.8 because LTS?

2) I need to write a one-time migration program to move data from Access to Postgres. In the old system the "id" primary key value matters, on some tables. For example, they use it as a reference number when printing (and later billing) service reports, so I need to keep this value around. If I want to continue using it as a pk, I need to somehow migrate the data to the target DB with a non-auto-incrementing pk, and then post-migration set it to auto-incrementing. Does that sound about right?

At least I'm rewriting my own legacy system, rather than some other legacy system written by some other crazy person :v:

1.8 vs 1.9 is really up to you. You get an extra year of support out of 1.8, but miss out on whatever features of 1.9. You'll have to decide which you value more. Personally, I always use the latest but I'm a wild and crazy person.

And yeah, I think you've got the data migration plan right.

hitze
Aug 28, 2007
Give me a dollar. No, the twenty. This is gonna blow your mind...

I'm having some weird problem with an update form using UpdateView. It seems to only happen on ManyToMany and ForeignKey fields. The ForeignKey fields never seem to be the right values (but on some items they are?), and the ManyToMany field seem to at least have all the right values but then it adds another one. Anyone have any idea what is going on here? Maybe I'm doing something really dumb.

The two lines are the correct values dumped into context.


E: It was something really dumb. For whatever reason I had this sitting in the page
code:
$("select option[value=" + {{ object.product.id }} + "]").attr("selected","selected");
Super Dumb :haw:

hitze fucked around with this message at 03:10 on Mar 17, 2016

reitetsu
Sep 27, 2009

Should you find yourself here one day... In accordance with your crimes, you can rest assured I will give you the treatment you deserve.
I'm picking back up my attempt at writing a character generator for a tabletop roleplaying game, and one piece in particular is driving me up a wall. I'm trying to replicate this game's Merit system, which works thusly:
- A Character can add Merits to their sheet
- Each individual Merit has a different number of options for the level of said Merit
- These options consist of 1, 2, 3, 4, 5. A large portion of Merits can be selected at levels 1 through 5, but all other combinations are also possible (2 or 3; 1, 2, or 3; etc.)

There's other stuff like certain Merits needing others as prerequisites, but I'm not worried about that at this stage. This initial model structure is what's throwing me off - I'm not sure how to set this up such that when I pick a Merit, the level field will display only the options entered into the database as true.

Since the Merits are their own thing, I have another model, CharacterMerit, that stores the Merit name and level for a given Character. I tried setting up the CharacterMerit and Merit models like so...
code:
MERIT_CHOICES = (
	(1, 'O'),
	(2, 'OO'),
	(3, 'OOO'),
	(4, 'OOOO'),
	(5, 'OOOOO'),
)


class Merit(models.Model):
	name = models.CharField(max_length=100)
	one = models.BooleanField(default=False)
	two = models.BooleanField(default=False)
	three = models.BooleanField(default=False)
	four = models.BooleanField(default=False)
	five = models.BooleanField(default=False)
	effect = models.TextField()


class CharacterMerit(models.Model):
	character = models.ForeignKey(Character)
	merit = models.ForeignKey(Merit)
	level = models.IntegerField(choices=MERIT_CHOICES)
	details = models.CharField(
		max_length = 100,
		blank = True
	)
...But the level part of CharacterMerit is where I'm stumbling. Right now I've just got level set to pick 1-5, but the choices for that field should be whichever of the five Booleans of that Merit are set to True. Is what I'm trying here even possible, and/or does anyone know of a better way to tackle this?

porksmash
Sep 30, 2008
I've just about thrown myself off a cliff trying to figure out some weird environmental issue I ran into. On my laptop, I added a small model, and then a M2M field on another model that uses the smaller one. Migrate, add some admin juice, and it's working great.

Later, when I pull it to my main computer and try to run, Django just can't see the new model. It literally ignores the fact that it exists. Any imports result in 'ImportError: cannot import name MonsterTag', which Google says is a circular reference. The problem is, the same code works on my laptop just fine. What's extra weird is that if I run another makemigrations, it produces something despite there being no code changes since the last makemigrations. It produces:

  • Removed field <m2mfield> from PreExistingModel
  • Delete model <SmallModel>

It's like it can't even tell the goddamn model is in my code. I'm staring RIGHT at it. I reduced it down to just a single character field in case the custom fields (taggit, and ColorField) were not playing nice. I even went so far to run a diff on the entire project from both computers and they're identical. They connect to the same development DB. The virtual environments are identical as far as packages and version numbers.

Gounads
Mar 13, 2013

Where am I?
How did I get here?
This is one of those step-back and check the basics situations.

Make sure the file actually exists.
Delete all .pyc files to make sure your not getting an old precompiled version
Put a breakpoint in the model file, does it hit it?
Make sure the app is listed in INSTALLED_APPS

etc.

porksmash
Sep 30, 2008
It turned out to be the old .pyc! Thank you.

Jo
Jan 24, 2005

:allears:
Soiled Meat
I'm stuck with another lovely use case.

We have objects, call them Books. We have Groups. We have Corrections which are usually specific to Groups, the one exception being Group 1. Everybody can see Group 1's corrections. We want to select all Books and get related corrections. If there are no corrections, we want to return just the Book with no extra values. If there are corrections by Group 1, we want to show Group 1's corrections. If there are corrections by Group n, we want to show only Group n's corrections to members of Group n (otherwise, show Group 1's corrections).

We did this originally with some horrifying DB fuckery: sort by Group ID descending and do distinct on Correction.BookID before joining with Books. Our data team yelled at us, though, because this is really inefficient for 14 million books. I'm trying to think of another way to do it OR a way to redesign the tables to make this easier (other than denormalizing because then we get 10M * # groups).

Is there a way to make IF x THEN x ELSE IF y THEN y ELSE z happen on the database level in the Django ORM? Hell, is there a way with SQL?

IAmKale
Jun 7, 2007

やらないか

Fun Shoe
I've been experimenting with Django Rest Framework lately, becoming more comfortable with it and experimenting with some of its third-party authentication packages. Somewhere along the way I became obsessed with the idea of getting the default settings.py file down to just the essentials. I wanted to see what parts of Django I could remove and still have Rest Framework work at full capacity.

Right now I think I've found the absolute minimum number of Django-related apps that need to be included in INSTALLED_APPS, and I've nuked all the MIDDLEWARE_CLASSES and most of the default values set in TEMPLATES:

Python code:
INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.staticfiles',

    'rest_framework',
    'knox',
    'djoser',

    'api_app'
]	

MIDDLEWARE_CLASSES = [
]

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'APP_DIRS': True
    },
]
Rest Framework appears to run and handle requests just fine, but I'm wondering if perhaps I've gone too far and should just leave everything as it was prior to my pruning. Am I just asking for trouble by trying to pare down the output of the Django project generator? I'm afraid I might be overeager to get thing down to the bare essentials and will just make more work for myself later on down the line.

Hed
Mar 31, 2004

Fun Shoe
It's hard to tell if you've gone too far without knowing the reason for your optimizing. Really major sites that use Django, you can bet everything in settings has earned its place there. If it's just a fun obsession, then that's fine. But it's the equivalent of those people who remove Aero or whatever theme Windows ships with because "that stuff is slowing my computer down :argh:"

IAmKale
Jun 7, 2007

やらないか

Fun Shoe

Hed posted:

It's hard to tell if you've gone too far without knowing the reason for your optimizing. Really major sites that use Django, you can bet everything in settings has earned its place there. If it's just a fun obsession, then that's fine. But it's the equivalent of those people who remove Aero or whatever theme Windows ships with because "that stuff is slowing my computer down :argh:"
It was pure curiosity. I've only ever used Django as a platform for Rest Framework so I wanted to see how much of the Django defaults were actually necessary when I'm hardly using any of the default functionality. It seems SOP is to just leave things as-is so I won't take this too far.

porksmash
Sep 30, 2008
I'm curious how things like sessions and CSRF work without a SECRET_KEY.

IAmKale
Jun 7, 2007

やらないか

Fun Shoe

porksmash posted:

I'm curious how things like sessions and CSRF work without a SECRET_KEY.
Oh, that wasn't the entirety of my settings file. Those were just the three default Django sections that suffered the most pruning. You can't even runserver if SECRET_KEY is missing.

Hed
Mar 31, 2004

Fun Shoe
Some of the defaults for middleware handle sessions, the redirection to have slashes at the end (which is always a style question when running DRF), and some caching stuff that the client can pay attention to. I do think CSRF protection won't work without the middleware there injecting into the response and processing the requests. I'd be interested if all of that works as expected still. Still, it's kind of fun to see what can work.

Jo
Jan 24, 2005

:allears:
Soiled Meat
New question because I'm stupid, overwhelmed, and a bit rushed after a day of not being able to solve this simple problem:

Here's my data model:

code:
Cluster (id, name)

AClusterMap (cluster_id, A_id, parent_id)

A (id, name)

Collection (id, name)

ACollectionMap (A_id, collection_id)
I have an endpoint which, given a collection ID, should return the clusters for that collection. That part is pretty easy.

code:
return SomeSerializer(AClusterMap.objects.filter(A__in=Collection.objects.get(id=whatever)))
# Where SomeSerializer pulls out the correct cluster info.
The problem I'm having is returning the COUNT of the A's in the Collection. I'd love to have something like this:

code:
[
  { "name": "asdf", "count": 10, "id":1 },
  { "name": "ClusterOne", "count": 1, "id":5 },
  { "name": "PinkFloyd", "count": 3, "id":6 },
]
The problem I have is the annotation AClusterMap.annotate(count=Count('A', distinct=True)) OR Cluster.annotate(count='AClusterMap__patent') both seem to produce the wrong result -- they are aggregating across _ALL_ the clusters instead of just aggregating inside the results that are returned. Is it possible to annotate/filter on Count inside the collection? Do I need to spin up the .extra(select={...}) stuff?

Also, I'm on 1.7 still. No Case/Where/When.

EDIT: gently caress me running. What is this bullshit?

code:
    def filter_queryset(self, queryset):
        p_id = self.request.query_params.get('p', None)
        p = P.objects.get(id=p_id)

        queryset = Cluster.objects.filter(aclustermap__parent__isnull=True)
        queryset = queryset.filter(aclustermap__a__in=p.as.all())
        queryset = queryset.annotate(count=Count('aclustermap__a_id'))

        return queryset

[
    {
        "id": 13840,
        "name": "acoustics",
        "count": 31488
    },
    {
        "id": 13815,
        "name": "general mechanics",
        "count": 79775
    }
]

===

    def filter_queryset(self, queryset):
        p_id = self.request.query_params.get('p', None)
        p = P.objects.get(id=p_id)

        queryset = Cluster.objects.filter(
            aclustermap__parent__isnull=True,
            aclustermap__a__in=p.as.all()
        )
        queryset = queryset.annotate(count=Count('patentclustermap__patent_id'))

        return queryset

[
    {
        "id": 13840,
        "name": "acoustics",
        "count": 2
    },
    {
        "id": 13815,
        "name": "general mechanics",
        "count": 1
    }
]
Examining the SQL output, queryset = Foo.objects.filter(a, b) DOES NOT EQUAL queryset = Foo.objects.filter(a); queryset = queryset.filter(b).

Jo fucked around with this message at 23:57 on Mar 29, 2016

Dominoes
Sep 20, 2007

Does anyone know how to make date / time / datetime widgets work in Django forms? I've found nothing but broken addons and ancient admin hacks.

Jo
Jan 24, 2005

:allears:
Soiled Meat
Why does Django's migration generate

code:
CREATE COLUMN xyz DEFAULT 0;
UPDATE COLUMN xyz DROP DEFAULT;
When I've got default=0 in my model?

Mulozon Empuri
Jan 23, 2006

It looks like a lot of the links in the first post are broken/out of date. Which is fair enough since it's from 2008.

It might be time to update it, maybe add stuff like

The Djangogirls Tutorial http://tutorial.djangogirls.org/en
Some Other Tutorial for experienced developers
Cookiecutter Django https://github.com/pydanny/cookiecutter-django
Awesome Django http://awesome-django.com/
Latest Djangocon videos https://www.youtube.com/playlist?list=PLE7tQUdRKcyaRCK5zIQFW-5XcPZOE-y9t
Latest Djangocon eu videos https://opbeat.com/events/djangocon-eu-2016/

etc

epswing
Nov 4, 2003

Soiled Meat

Mulozon Empuri posted:

It looks like a lot of the links in the first post are broken/out of date.

The Djangogirls Tutorial http://tutorial.djangogirls.org/en

This link is broken ;) missing a trailing slash

Cock Democracy
Jan 1, 2003

Now that is the finest piece of chilean sea bass I have ever smelled
Anyone else going to DjangoCon in July? I just bought my ticket. I'm mainly interested in hearing about channels and any kind of advice from high-traffic Django sites about scaling and caching.

MonkeyMaker
May 22, 2006

What's your poison, sir?
I'll be there!

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

For like 4 years I'm always like "I've got to remember to go to DjangoCon next year."

Then I'm always surprised when I read somewhere that DjangoCon is coming up shortly and I've already got plans. :(

NtotheTC
Dec 31, 2007


How widespread is cookiecutter-django? I came across it over the weekend and I really like it. It was also my first foray into using docker and docker-compose, which is neat. My major gripe is that running it on linux means running it via he root user, which means that any files you generate with django's makemigrations or dumpdata are chown'ed by root by default and for the life of me I couldn't find a neat solution to that other than chown'ing them back after I run the command. How does that transfer over to production though?

Gounads
Mar 13, 2013

Where am I?
How did I get here?
I haven't used it, but I have heard of it several times. I can't imagine why you'd have to run it as root. I mean, sure, run the docker daemon as root, but I *think* everything else should work in user space?

NtotheTC
Dec 31, 2007


Gounads posted:

I haven't used it, but I have heard of it several times. I can't imagine why you'd have to run it as root. I mean, sure, run the docker daemon as root, but I *think* everything else should work in user space?

Yeah I'm not sure either, on the docker-compose django docs it just says:

quote:

If you are running Docker on Linux, the files django-admin created are owned by root. This happens because the container runs as the root user. Change the ownership of the the new files.

code:
sudo chown -R $USER:$USER .

Which isn't particularly helpful. I made a stack overflow post here about this but no replies yet. I think I've managed to confuse myself and everyone else. It just seems like this would be such a common issue to come across that I'm amazed that noone else has had a similar problem.

The March Hare
Oct 15, 2006

Je rêve d'un
Wayne's World 3
Buglord
Celery question! I'm working on a legacy app that has all of its tasks (there are many) in their own named queue. I just set up the quickstart guide's auto task discovery mechanism, but my issue is that they don't seem to run unless I specify -Q queue_name for each of the tasks that I am testing in the celery worker command.

Is there some way to autofire queues that I don't know about? I thought CELERY_CREATE_MISSING_QUEUES = True would do it (True is the default but I tried anyway...) but it doesn't seem to do the thing.

MonkeyMaker
May 22, 2006

What's your poison, sir?
Anyone at or coming to DjangoCon US this week? If so, say 'hi'. Look for a guy with a big beard and a bright green "organizer" ribbon.

Adbot
ADBOT LOVES YOU

ohrwurm
Jun 25, 2003

MonkeyMaker posted:

Anyone at or coming to DjangoCon US this week? If so, say 'hi'. Look for a guy with a big beard and a bright green "organizer" ribbon.

Yup, I will be there. I will say hello to all bearded organizers.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply