Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
bitprophet
Jul 22, 2004
Taco Defender

MetaKnyght posted:

I've searched for information on this, but can't really seem to find anything useful. I think my Google-fu has gotten weak.

What is the best way to run something automatically every so often? I want to go through my database and update it every 15 minutes or so. Is there any way to do this from within the main Django application, or does it need to be a seperate process or cron job? And finally, what is the best method for locking the database while the update is running so anybody accessing the web interface isn't going to receive partially updated data? Thanks!

Well, consider that a Web framework is inherently limited to responding to browser requests and I think you'll realize why a script run by a cron job is the way to go here :) All you should have to do for a Python script to load up your Django models/views/whatever, is to ensure the cron has DJANGO_SETTINGS_MODULE set, just like you have to do for your Web server that hooks into Django. E.g. for a nightly cron task in the root crontab,

code:
0 3 * * * DJANGO_SETTINGS_MODULE='myproj.settings' /srv/django/myproj/scripts/update_db_lol
For DB locking, that's actually more of an application level thing: what you want is for no users to be able to visit parts of the site that let them alter the database. What those parts are is different for every project.

What I've seen done in the Rails world, and it makes some sense, is to construct a static HTML page that lives outside your Apache docroot somewhere, and to symlink it into the docroot while you need the site to be down (i.e. your cron job script would take care of putting it up, then taking it down at the end). You then pair that with some Apache rewrite rules/conditions which essentially say "if this file exists, redirect any and all requests to it".

End result is that when that symlink exists, Apache sends all traffic to that "lol we're updating sorry!" page, and once your process is done and the symlink is removed, the site goes back to normal.

bitprophet fucked around with this message at 20:25 on Feb 14, 2009

Adbot
ADBOT LOVES YOU

Crazak P
Apr 11, 2003

PUNISHER > SPIDERMAN

bitprophet posted:

Unless you're doing something really funky with your PythonPath when Django loads up, Django is just Python, and should always have full access to anything on your global PythonPath.

Given what you said about lib/site-packages, I'm guessing that you may have 2 Python versions set up? That, or some other aspect of the configuration is messed up; typically, a python setup.py install SHOULD put stuff into your site-packages directory, and site-packages should always be on your PythonPath (and it sounds like it is).

Long story short: tell us exactly what OS you're using, what exact version you're on, and if possible, what version(s) of Python you've got and how they were installed (e.g. via MacPorts, via apt-get, from source, whatever). That should help figure out what's going on.

I think you figured it out. I have two installs of Python, 2.4 and 2.5. Sorry I forgot to put the necessary information. I'm on Windows XP and I just used the installers for everything.

bitprophet
Jul 22, 2004
Taco Defender

Crazak P posted:

I think you figured it out. I have two installs of Python, 2.4 and 2.5. Sorry I forgot to put the necessary information. I'm on Windows XP and I just used the installers for everything.

Yea, that would do it. If you find aforementioned lib/site-packages dir, it should be inside some directory hierarchy which, at some point, mentions the Python version, i.e. C:\something_or_other\Python2.4\Lib\Site-Packages. You likely have two identical trees like this, one for Python2.4 and one for Python2.5.

I don't use Windows so I can't help you with the specifics, but your various configs for Apache or Django or the command line Python binary, should all be pointing at one or the other of the two versions; you want to try and make sure that everything is using 2.5 if possible (or at least make sure they are consistent; 2.5 is better because it's newer).

Or, ideally, just remove Python 2.4, if there's an uninstaller that will target just that version.

hitze
Aug 28, 2007
Give me a dollar. No, the twenty. This is gonna blow your mind...

I was wondering if anyone knew how to do a simple for loop that iterates the number it gets from say "bingodingo.number". I've searched djangoproject.com and found a ticket about it. I can't seem to find any documentation on how to make the for loops in django work like that :suicide:

Wedge of Lime
Sep 4, 2003

I lack indie hair superpowers.

hitze posted:

I was wondering if anyone knew how to do a simple for loop that iterates the number it gets from say "bingodingo.number". I've searched djangoproject.com and found a ticket about it. I can't seem to find any documentation on how to make the for loops in django work like that :suicide:

This is more to do with python than with Django its self, in python you are encouraged to loop through all elements of a list (or in Django a query result set) instead of using an iterator (as I'm sure you've found). So first I'd look at avoiding this in some way.

However, if you want to perform a loop x times you could use something like the built-in xrange function in your view. Then pass the resulting xrange object to your template.

code:
number = 5
for i in xrange(number):
    print i
    # do something else...
Though, I find that in most situations I can avoid using xrange and end up with simpler to read / maintain code.

Wedge of Lime fucked around with this message at 23:42 on Feb 18, 2009

ATLbeer
Sep 26, 2004
Über nerd

MetaKnyght posted:

I've searched for information on this, but can't really seem to find anything useful. I think my Google-fu has gotten weak.

What is the best way to run something automatically every so often? I want to go through my database and update it every 15 minutes or so. Is there any way to do this from within the main Django application, or does it need to be a seperate process or cron job? And finally, what is the best method for locking the database while the update is running so anybody accessing the web interface isn't going to receive partially updated data? Thanks!
The first part of your question is easy. You can't do it solely within Django. You'll have to set up a cron job that calls a script to run every 15 minutes. I've done it before and it's quite easy to do really. I use Django Bootstrap so that I can simply just code in my cron script just like you can in a normal view. Import all your models make your DB calls and exit out

Your second question is pretty hard. There are a few ways to skin this cat and the first thing that pops in my head is to have a DB table that has a "lock" boolean to be true when the cron is running and use a decorator around views that alter the DB. So when a view is called alters the DB you can redirect to a polite message to try again in a bit.

Unfortunately this isn't a good user experience. I'm not sure of the exact situation that your working in that needs a full lock on the tables but, there's few options to really use here. If your user experience is critical and you don't want 1 minute down times (which is essentially what your doing) every 15 minutes you need to refactor this somehow. Is there anyway you can simple store up an array of updates that you want to do as your analyzing your data and then run through them while locked as opposed to locking doing all the queries analysis and updating then unlocking, etc...

king_kilr
May 25, 2007
2 pieces to the puzzle, 1 a management command for the script itself, 2 python lockfile for the lock: http://pypi.python.org/pypi/lockfile

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!
Washington Times just released some pretty awesome django projects:
http://opensource.washingtontimes.com/blog/post/coordt/2009/02/washington-times-releases-open-source-projects/

edit: Forgot to mention that I started playing with django-projectmgr and it is pure awesome.

deimos fucked around with this message at 02:05 on Feb 20, 2009

bitprophet
Jul 22, 2004
Taco Defender

deimos posted:

Washington Times just released some pretty awesome django projects:
http://opensource.washingtontimes.com/blog/post/coordt/2009/02/washington-times-releases-open-source-projects/

edit: Forgot to mention that I started playing with django-projectmgr and it is pure awesome.

I saw the feature list for that and basically creamed myself, it's got a LOT of ideas I was hoping to put into a tiny fledgling Django app I have going in the same vein. Plan to check out the source sometime soon.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!
As a sidenote, what's the state of django on Windows? I haven't tried it at all. I need to connect to SQL Server and last I checked the only reliable way was running django on Windows through Apache and :downs: mod_python.

TOO SCSI FOR MY CAT
Oct 12, 2008

this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

deimos posted:

As a sidenote, what's the state of django on Windows? I haven't tried it at all. I need to connect to SQL Server and last I checked the only reliable way was running django on Windows through Apache and :downs: mod_python.

Get a WSGI plugin for IIS or Apache.

king_kilr
May 25, 2007

deimos posted:

As a sidenote, what's the state of django on Windows? I haven't tried it at all. I need to connect to SQL Server and last I checked the only reliable way was running django on Windows through Apache and :downs: mod_python.

On the MSSQL front django-pyodbc and django-mssql are both decent, though I don't know which is actually better.

Mashi
Aug 15, 2005

Just wanted you to know your dinner's cold and the children all agree you're a shitheel!
Anyone got any recomendations for database schema migration tools for Django? Anyone use and are happy with it? I'm talking about the sort of tool that allows me to type "manage.py evolvedb" and it will print SQL to sync the differences between the models and the DB structure.

king_kilr
May 25, 2007

Mashi posted:

Anyone got any recomendations for database schema migration tools for Django? Anyone use and are happy with it? I'm talking about the sort of tool that allows me to type "manage.py evolvedb" and it will print SQL to sync the differences between the models and the DB structure.

I like django-evolution: http://code.google.com/p/django-evolution/
south is also popular: http://south.aeracode.org/

mwarkentin
Oct 26, 2004
I've used django-evolution.. worked well for what I used it for (really simple migrations.. adding / removing db columns). Didn't need to get into anything too advanced.

Crazak P
Apr 11, 2003

PUNISHER > SPIDERMAN
django templating question:
How do I insert some html after every third iteration through a loop?

example:
code:
{% for prods in prodlist %}

<div>
{{ prods.name }}
</div>

{% if this is the third time %}
<br />
{% endif %}

{% endfor %}

tankadillo
Aug 15, 2006

Crazak P posted:

django templating question:
How do I insert some html after every third iteration through a loop?

example:
code:
{% for prods in prodlist %}

<div>
{{ prods.name }}
</div>

{% if this is the third time %}
<br />
{% endif %}

{% endfor %}
You could try cycle with two blank variables.

edit: see the post below this for a much better answer!

tankadillo fucked around with this message at 23:47 on Feb 20, 2009

king_kilr
May 25, 2007
code:
{% for prods in prodlist %}

<div>
{{ prods.name }}
</div>

{% if forloop.counter|divisibleby:"3" %}
<br />
{% endif %}

{% endfor %}

should do what you want

Xenos
Jun 17, 2005

king_kilr posted:

On the MSSQL front django-pyodbc and django-mssql are both decent, though I don't know which is actually better.

I can tell you from experience that pyodbc is the best free MSSQL driver for python, and that django-mssql is Windows-only (it requires PyWin32). I've never developed against either, though.

Crazak P
Apr 11, 2003

PUNISHER > SPIDERMAN

king_kilr posted:

code

Awesome, thanks man. I knew I had to use divisbleby somewhere, but I was thinking more along the lines of declaring a variable, which I don't think you can using the template language.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

Crazak P posted:

Awesome, thanks man. I knew I had to use divisbleby somewhere, but I was thinking more along the lines of declaring a variable, which I don't think you can using the template language.

The loop variables are really great, make sure you learn them because they're very useful here.

Crazak P
Apr 11, 2003

PUNISHER > SPIDERMAN
I completely missed that while looking through the documentation.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

bitprophet posted:

I saw the feature list for that and basically creamed myself, it's got a LOT of ideas I was hoping to put into a tiny fledgling Django app I have going in the same vein. Plan to check out the source sometime soon.

The source is gorgeous. It's really clean code and from what I have browsed so far, I've already learned a couple of new things.

You actually need to dive into the code to see some of the really neat stuff it does like notifications using django-notification.

Ohh and MPTT is my new most favorite thing ever.

deimos fucked around with this message at 05:45 on Feb 21, 2009

tankadillo
Aug 15, 2006

This is probably an easy problem to solve, but I haven't seen anything helpful yet. Basically I have a base template that several views use, and there are a few variables that I want to always get passed to that template. Right now I just include them in my render_to_response() function at the end of each view, but it seems like there would be an easier way that didn't require repeating code.

bitprophet
Jul 22, 2004
Taco Defender

raezr posted:

This is probably an easy problem to solve, but I haven't seen anything helpful yet. Basically I have a base template that several views use, and there are a few variables that I want to always get passed to that template. Right now I just include them in my render_to_response() function at the end of each view, but it seems like there would be an easier way that didn't require repeating code.

You probably want to write a custom context processor, which is just a function that can add key/value pairs to the context dict.

Basics of how to enable context processors in general: http://docs.djangoproject.com/en/dev/ref/templates/api/#id1

How to write your own context processors: http://docs.djangoproject.com/en/dev/ref/templates/api/#writing-your-own-context-processors

Only caveat is that, as in the first link, you need to ensure that you render templates with that special RequestContext class for the processors to work. (You can write a one-line function to replace render and/or render_to_response, that uses RequestContext, if you find yourself needing your context processor across the entire site.)

tankadillo
Aug 15, 2006

bitprophet posted:

You probably want to write a custom context processor, which is just a function that can add key/value pairs to the context dict.

Basics of how to enable context processors in general: http://docs.djangoproject.com/en/dev/ref/templates/api/#id1

How to write your own context processors: http://docs.djangoproject.com/en/dev/ref/templates/api/#writing-your-own-context-processors

Only caveat is that, as in the first link, you need to ensure that you render templates with that special RequestContext class for the processors to work. (You can write a one-line function to replace render and/or render_to_response, that uses RequestContext, if you find yourself needing your context processor across the entire site.)
I'm not sure how I managed to miss this in the documentation before, but it's pretty awesome. It does exactly what I want, thanks!

hitze
Aug 28, 2007
Give me a dollar. No, the twenty. This is gonna blow your mind...

I just finished setting up this (post save?) deal. It renames the photo and moves it into a new folder and it updates the photos path. Everything seems to work properly with it, adding a new entry, and editing an existing one.
code:
	def save(self, force_insert=False, force_update=False):
		alpha       = self.Slug[:1]
		destination = 'C:/Users/Chris/Desktop/Django/django/Lyrics/www/%s/%s/' % (alpha, self.Slug)
		renamed     = '%s%s-(artist).jpg' % (destination, self.Slug)
		
		if not os.path.exists(destination):
			os.mkdir(destination)
		shutil.move(self.Photo.path, renamed)
		self.Photo = renamed
		super(Artist, self).save(force_insert, force_update)
Is this the proper way to update fields?

Mashi
Aug 15, 2005

Just wanted you to know your dinner's cold and the children all agree you're a shitheel!
I've got a question about template inheritance. I have a common setup, one parent template for the site header / footer, and another one that extends it for the section content.

What I want to do is have the parent template include another template for the login block. There isn't an elegant way to do it using normal inheritance, it would have to go "section extends login extends site_outer", which is wrong.

To demonstrate in markup:

code:
{# site_outer.html #}
<html><body>
    {% block login %}{# this is where I want login.html included #}{% endblock %}
    {% block content %}{% endblock %}
</body></html>
---------------------------------------
{# some_section.html #}
{% extends site_outer.html %}
{% block content %} hello user {% endblock %}
---------------------------------------
# views.py
def someview(request):
    return render_to_response('some_section.html', Context())

Git
Aug 21, 2004

Mashi posted:

{# this is where I want login.html included #}

Sounds like you want to {% include login.html %}

Mashi
Aug 15, 2005

Just wanted you to know your dinner's cold and the children all agree you're a shitheel!

Git posted:

Sounds like you want to {% include login.html %}

Yes.. I don't know how I missed that. Thank you.

tayl0r
Oct 3, 2002
This post brought to you by SOE
I'm using Django with Google App Engine and I can't seem to figure out how to display a specific element from a list, when the element index is contained in another variable.

I have a list called contentType. It's a simple list with elements 0 through 4.
I have a variable (I guess a dict?) called adventure. It's from my datamodel and has a property called contentType, so adventure.contentType is the integer of the index I want to display.

I basically want this: contentType[adventure.contentType]
But that doesn't work. I can't figure what does.

I tried {{ contentType|splice:"3:"|first }}
This works and gives me back the 4th element.
When I changed it to {{ contentType|splice:"adventure.contentType:"|first }} it just gives me back the 1st element.

I can also do {{ contentType.3 }}, which gives me back the 4th element. How can I use a variable there instead of hard coding the number?

I guess I could just do this little bit of logic in the python and pass that variable into the template but it seems like there should be a way to do it here, in the template.

tayl0r
Oct 3, 2002
This post brought to you by SOE
Another question:

I'd like to build a quick and dirty administration tool for 3 database tables with foreign keys to each other. It's a pretty standard relationship.

The database is oracle, and the tables are already created by our database admins. I only have read and write access to it, not any admin permissions.

What do I do at this point, in regards to the data model?
Do I just setup my data model to mimic what already exists?

Will Django just start reading and writing to it without it actually running in any SQL code to modify the tables?

ATLbeer
Sep 26, 2004
Über nerd

tayl0r posted:

Another question:

I'd like to build a quick and dirty administration tool for 3 database tables with foreign keys to each other. It's a pretty standard relationship.

The database is oracle, and the tables are already created by our database admins. I only have read and write access to it, not any admin permissions.

What do I do at this point, in regards to the data model?
Do I just setup my data model to mimic what already exists?

Will Django just start reading and writing to it without it actually running in any SQL code to modify the tables?

You still have to generate the models.py file for an application then fire up your app.

You can either do it by hand or use a built in Django tool. I recommend a bit of both

Configure all your database settings like normal (DATABASE_NAME, etc) then run python manage.py inspectdb > appname/models.py and Django will automatically construct the models.py file for you in your application. I highly recommend going into that models file and just checking it to make sure that it actually makes sense and has identified the relationships correctly.

If you run syncdb after this thought Django will add some tables to your database for user administration and permissions like it always does for every app.

tayl0r
Oct 3, 2002
This post brought to you by SOE
Oh nice, I didn't realize it had an inspectdb command. I will get on that.
Hopefully I can limit it to just the couple of tables I want to mess with, and not all 50+ of them.

Thanks!


--- edit ---
drat, looks like this isn't quite going to work, since the tables I want to work with are in other schemas (not mine). I'm going to setup the synonyms but that will break the management tools apparently, which is what I wanted to use.

http://code.djangoproject.com/ticket/6148

"There is frequently a need for Django to access data from tables in other schemas; this is especially true when building Django apps on top of legacy databases.

A current workaround in Oracle is to manually create private synonyms inside the Django schema, referencing the necessary tables from other schemas. However, the management commands do not currently recognize the existence of synonyms via introspection. Additionally, the synonym solution may not be sufficiently generic in the long term."

tayl0r fucked around with this message at 18:51 on Mar 4, 2009

Mashi
Aug 15, 2005

Just wanted you to know your dinner's cold and the children all agree you're a shitheel!
In a site I'm making I have a model called Feed. Some types of content are associated with a Feed, and the name of the feed is prefixed to every path on the site (ie /feed1/whatever).

When a user makes a request I have to determine the feed they are requesting by looking at request.path. That then needs to be stored somewhere so it can be added to template context and made available to models (I really want to have managers that pick it up automatically).

Since this is per request data, it makes sense to store it as a property of the request, but this is less than ideal because then I can't have model managers that pick it up automatically. If I store it as a property of the settings module then sometimes I can't access it later.

The most basic way to do it is to match the feed in the URLs and pass it in to every view, but then there needs to be boilerplate code in every view. It is the simplest way though. It would be nice to be able to match it in the URLConf, pass it to a function which stores it, then pass execution back to the urlresolvers.

Anyone have suggestions / preferred methods of doing something like this?

Mashi fucked around with this message at 13:03 on Mar 5, 2009

tayl0r
Oct 3, 2002
This post brought to you by SOE
So I have the Django admin tools and data model definitions setup and it works pretty well, even though I'm using Oracle and my tables are in different schemas.
The synonym trick seems to work.

The only problem I'm having is with one of my tables that has a primary key of 2 columns.

code:
class Leaderboards(models.Model):
    leaderboard_type = models.ForeignKey(RefLeaderboardTypes, db_column='LEADERBOARD_TYPE', related_name='leaderboards', primary_key=True)
    subtype_id = models.IntegerField(db_column='SUBTYPE_ID', primary_key=True)
When I click on a Leaderboards object in the admin tool, I get this error:
code:
Exception Type:	MultipleObjectsReturned
Exception Value:	
get() returned more than one Leaderboards -- it returned 2! Lookup parameters were {'pk': u'1'}
It doesn't appear to be using both columns from the primary key, even though I have the primary_key=true flag set on both of the columns in the data model.

Any ideas?


edit-
gently caress. another thing that isn't supported out of the box!

"Currently, you can "fake" it by declaring one of the keys to be primary in Django and adding a unique constraint to the model. You'd have to do without the auto-generated SQL that Django gives you, but if you're smart enough to know how and why to use multiple primary keys, you can probably write the schema by hand. Still, this is less than ideal."

http://code.djangoproject.com/ticket/373

tayl0r fucked around with this message at 23:00 on Mar 6, 2009

Mashi
Aug 15, 2005

Just wanted you to know your dinner's cold and the children all agree you're a shitheel!
When using mod_wsgi in daemon mode with apache, should i feel comfortable running multiple threads? I don't know how to check my application for thread safety any further than just doing a bit of stress testing and seeing if it falls over. It's a basic database driven Django site without any special 3rd party modules.

I'm asking because I always see setup tutorials showing examples where there is a thread count of 15 or 25, and sometimes not warning that the application should be checked for thread safety first.

Ned
May 23, 2002

by Hand Knit
I just started working on a Django based site and I was wondering what the best practices are for using jQuery based AJAX with Django?

Allie
Jan 17, 2004

Mashi posted:

When using mod_wsgi in daemon mode with apache, should i feel comfortable running multiple threads? I don't know how to check my application for thread safety any further than just doing a bit of stress testing and seeing if it falls over. It's a basic database driven Django site without any special 3rd party modules.

I'm asking because I always see setup tutorials showing examples where there is a thread count of 15 or 25, and sometimes not warning that the application should be checked for thread safety first.

The threads can't share anything, so you shouldn't have a problem. The option is available more as a matter of performance.

The only time I could imagine it being an issue is if you're using a C extension that isn't coded properly. Issues with concurrent access to shared resources outside of Python would also present themselves using multiple processes, so you should keep that in mind if you're doing anything like that (e.g., modifying files, in which case you'd use file locks).

I haven't had any issues doing this with Django and MySQLdb.

Adbot
ADBOT LOVES YOU

Mashi
Aug 15, 2005

Just wanted you to know your dinner's cold and the children all agree you're a shitheel!

Milde posted:

The threads can't share anything, so you shouldn't have a problem. The option is available more as a matter of performance.

The only time I could imagine it being an issue is if you're using a C extension that isn't coded properly. Issues with concurrent access to shared resources outside of Python would also present themselves using multiple processes, so you should keep that in mind if you're doing anything like that (e.g., modifying files, in which case you'd use file locks).

I haven't had any issues doing this with Django and MySQLdb.

Great. I'm gonna give it a shot, then.

Ned posted:

I just started working on a Django based site and I was wondering what the best practices are for using jQuery based AJAX with Django?

Here's one I do:

When I have a clickable element that can update the page via ajax, I use a normal link with an event handler that returns false. If the client has javascript, the return false in the event handler will stop the client from following the link. I use the same view for ajax and normal requests, but I detect if the request is an ajax request and return the same data either in JSON or a normal template. Perhaps if the request is an ajax request you can skip some context processors that won't be required, since the whole page is not reloading.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply