Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
TOO SCSI FOR MY CAT
Oct 12, 2008

this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

duck monster posted:

Actually I'd strongly recomend not letting your mysql listen on a public port. Some dudes gunna scan it down and brutalize it.

If you must, set up some sort of SSH tunnel or something.

Or better still just put the server on the same box as the machine and only accept local connections.

What? There's nothing insecure about a public MySQL server. Just use a password other than the combo to your luggage and you'll be fine.

Adbot
ADBOT LOVES YOU

bitprophet
Jul 22, 2004
Taco Defender

Janin posted:

What? There's nothing insecure about a public MySQL server. Just use a password other than the combo to your luggage and you'll be fine.

General rule of thumb: never leave a daemon bound to external addresses unless you have an explicit need to serve connections to external clients. Doing otherwise is just stupid and leaving yourself open to the various kinds of remote exploits that are a fact of life.

For MySQL specifically, just looking at RedHat's advisories from the past few years, I've found two remote exploits that don't require any sort of login:

https://rhn.redhat.com/errata/RHSA-2009-0259.html
https://rhn.redhat.com/errata/RHSA-2007-0875.html

That's for a single vendor, I'm sure there's more.

Despite your offhand luggage password comment, all you need is one DB account with a crackable password -- even if that account only has limited access -- and that account can then be used to exploit the other 3 or 4 advisories I found that required an (any!) authenticated user. Betting your data and your server on being able to keep tabs on database account passwords is, again, stupid.

tl;dr: "lol just have a safe password" is not a valid security policy :)

ATLbeer
Sep 26, 2004
Über nerd

Janin posted:

What? There's nothing insecure about a public MySQL server. Just use a password other than the combo to your luggage and you'll be fine.

This is wrong. Never leave any service running publicly available unless it has to be accessed by something external. When it does need to be accessed externally enforce the tightest access rules possible on multiple layers when possible. If your backup server needs to get access to your MySQL server and an SSH tunnel or VPN isn't possible 3306 should be explicitly blocked in your firewall from all IPs EXCEPT your backup machine's IP and then the backup user should ONLY be able to connect from that IP.

Never just leave a service running publicly available unless it 100% needs to be

TOO SCSI FOR MY CAT
Oct 12, 2008

this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

bitprophet posted:

General rule of thumb: never leave a daemon bound to external addresses unless you have an explicit need to serve connections to external clients. Doing otherwise is just stupid and leaving yourself open to the various kinds of remote exploits that are a fact of life.

For MySQL specifically, just looking at RedHat's advisories from the past few years, I've found two remote exploits that don't require any sort of login:

https://rhn.redhat.com/errata/RHSA-2009-0259.html
https://rhn.redhat.com/errata/RHSA-2007-0875.html

That's for a single vendor, I'm sure there's more.

Despite your offhand luggage password comment, all you need is one DB account with a crackable password -- even if that account only has limited access -- and that account can then be used to exploit the other 3 or 4 advisories I found that required an (any!) authenticated user. Betting your data and your server on being able to keep tabs on database account passwords is, again, stupid.

tl;dr: "lol just have a safe password" is not a valid security policy :)

The first advisory wouldn't be solved by moving MySQL to an internal port, and the second doesn't expose any data. Forcing all requests to a database server through a proxy is just introducing another layer that could, and probably does, contain security vulnerabilities. Any vulnerability exploitable by a logged-in user could be performed just as easily over an SSH tunnel.

I maintain that Client -> public SSH -> private MySQL is just as insecure as Client -> public MySQL.

TOO SCSI FOR MY CAT fucked around with this message at 18:25 on Jun 21, 2009

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

Janin posted:

I maintain that Client -> public SSH -> private MySQL is just as insecure as Client -> public MySQL.

Then you are stupid.

As a sidenote if you do indeed chose to go with an outside facing MySQL please have the good conscience to use SSL.

Zombywuf
Mar 29, 2008

How do I get python to deal with datetimes with timezones?

code:
>>> from datetime import *
>>> d = datetime.strptime("2009-06-22T21:00:00.000BST", "%Y-%m-%dT%H:%M:%S.%f%Z")
>>> d.tzinfo
>>> d.strftime("%Z")
''
>>> d = datetime.strptime("2009-06-22T21:00:00.000+0000", "%Y-%m-%dT%H:%M:%S.%f%z")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/_strptime.py", line 317, in _strptime
    (bad_directive, format))
ValueError: 'z' is a bad directive in format '%Y-%m-%dT%H:%M:%S.%f%z'
I'm using python 2.6.2.

Python Docs posted:

%z UTC offset in the form +HHMM or -HHMM (empty string if the the object is naive).
%Z Time zone name (empty string if the object is naive).

TOO SCSI FOR MY CAT
Oct 12, 2008

this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"
pytz is your best chance, but I don't know if it supports parsing strings with tz information.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

Zombywuf posted:

How do I get python to deal with datetimes with timezones?

code:
>>> from datetime import *
>>> d = datetime.strptime("2009-06-22T21:00:00.000BST", "%Y-%m-%dT%H:%M:%S.%f%Z")
>>> d.tzinfo
>>> d.strftime("%Z")
''
>>> d = datetime.strptime("2009-06-22T21:00:00.000+0000", "%Y-%m-%dT%H:%M:%S.%f%z")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/_strptime.py", line 317, in _strptime
    (bad_directive, format))
ValueError: 'z' is a bad directive in format '%Y-%m-%dT%H:%M:%S.%f%z'
I'm using python 2.6.2.

http://pypi.python.org/pypi/iso8601/

duck monster
Dec 15, 2004

m0nk3yz posted:

You should see this:

Python 2.5.1 (r251:54863, Nov 12 2008, 17:08:51)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Are you not on leopard?

Definately on leopard.

duck monster
Dec 15, 2004

Janin posted:

What? There's nothing insecure about a public MySQL server. Just use a password other than the combo to your luggage and you'll be fine.

Keep on believing bro. Keep on believing. I've seen first hand MySQL databases raped from somewhat determined brute force attacks. And There have been brute forcer distributed worms for MS-SQL leaving me to suspect a MySQL brute worm is a matter of when, not if.

duck monster
Dec 15, 2004

Janin posted:

The first advisory wouldn't be solved by moving MySQL to an internal port, and the second doesn't expose any data. Forcing all requests to a database server through a proxy is just introducing another layer that could, and probably does, contain security vulnerabilities. Any vulnerability exploitable by a logged-in user could be performed just as easily over an SSH tunnel.

I maintain that Client -> public SSH -> private MySQL is just as insecure as Client -> public MySQL.

What. No thats an absurdity. Two iron gates is harder to crack than one, always.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

duck monster posted:

What. No thats an absurdity. Two iron gates is harder to crack than one, always.

Not to mention that everyone runs outside facing SSH, flaws in it get detected near real time of attacks. No one in their right mind runs a public facing MySQL, so chances of detecting an exploit on it are very low.

bitprophet
Jul 22, 2004
Taco Defender

deimos posted:

Not to mention that everyone runs outside facing SSH
I actually had to lock our network down with a VPN because of SSH bruteforce attacks. Successful ones. Because highly placed nontechnical staff (read: ones I cannot say "no" to) with logins didn't want to use secure passwords. Moral of story: never rely solely on the strength of a single level of password authentication if your user base consists of people who are not you.


To ask a Python question: anyone know of any decent "CLI UI" (and not curses-based, I don't think) libraries for Python? I'm thinking of commonish stuff like progress bars, spinners, formatting data into columns, simple "choose option A B or C" menus, and so forth. (I could swear I saw a PyMOTW posting on that last one, but can't find it now. Maybe I should ask Doug...)

Yes, many of these tasks are relatively simple, but it's still annoying to try and reinvent the wheel and it feels like the sort of thing that someone would have written a lib for.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

bitprophet posted:

To ask a Python question: anyone know of any decent "CLI UI" (and not curses-based, I don't think) libraries for Python? I'm thinking of commonish stuff like progress bars, spinners, formatting data into columns, simple "choose option A B or C" menus, and so forth. (I could swear I saw a PyMOTW posting on that last one, but can't find it now. Maybe I should ask Doug...)

Yes, many of these tasks are relatively simple, but it's still annoying to try and reinvent the wheel and it feels like the sort of thing that someone would have written a lib for.

If this exists, it would be cool. I'm always writing progress bars and the like.

deimos
Nov 30, 2006

Forget it man this bat is whack, it's got poobrain!

bitprophet posted:

I actually had to lock our network down with a VPN because of SSH bruteforce attacks. Successful ones. Because highly placed nontechnical staff (read: ones I cannot say "no" to) with logins didn't want to use secure passwords. Moral of story: never rely solely on the strength of a single level of password authentication if your user base consists of people who are not you.

To continue with the derail, your fault for not using private keys.

To answer your question maybe python-gnt? (It is curses though)

hlfrk414
Dec 31, 2008
This might be what you need. Seems to implement a windowing system on top of curses. Forgot I still had that link saved.

bitprophet
Jul 22, 2004
Taco Defender

deimos posted:

To continue with the derail, your fault for not using private keys.
It would be, if that hadn't been a setup I inherited, with management taking a typical "don't waste time fixing what's not (100%) broken" stance until the breakins occurred :sigh: Plus, again, nontechnical users. If only dev types needed S(SH|FTP) access, I could probably have gotten away with forcing a move to key-based auth, but when it involves company partners...

quote:

To answer your question maybe python-gnt? (It is curses though)

hlfrk414 posted:

This might be what you need. Seems to implement a windowing system on top of curses. Forgot I still had that link saved.

Yea, those are both curses, which I was hoping to avoid. Just want simple, no frills generic implementations of stuff like wget/curl type progress bars and etc.

I'll do more looking on my own; was mostly looking for a "oh yea, package X is a well known standard in this area" which doesn't seem to be the case. Not asking you guys to do my research for me :)

LuckySevens
Feb 16, 2004

fear not failure, fear only the limitations of our dreams

Is there an alternative to mod_python, since I'm running 2.6? I found some vague info from google on how to alter the install, but I can't get it to work.

tef
May 30, 2004

-> some l-system crap ->

LuckySevens posted:

Is there an alternative to mod_python, since I'm running 2.6? I found some vague info from google on how to alter the install, but I can't get it to work.

mod_wsgi if you're lucky

duck monster
Dec 15, 2004

bitprophet posted:

I actually had to lock our network down with a VPN because of SSH bruteforce attacks. Successful ones. Because highly placed nontechnical staff (read: ones I cannot say "no" to) with logins didn't want to use secure passwords. Moral of story: never rely solely on the strength of a single level of password authentication if your user base consists of people who are not you.
Yeah, theres brute forcer worms out there alright. Use fail2ban, it basically sits over ssh (and a few other things, I think) and if it gets more than 3 or 4 consecutive login atempts from an IP, it drops it at the firewall for 15 minutes, and a bunch of other techniques. A friend was talking the other day about a hack that'd implement teergrubbing , basically rather than drop the connection, just fall comatose and try and exhaust the worms threads (Thus crashing it or grieviously slowing it down)

quote:

To ask a Python question: anyone know of any decent "CLI UI" (and not curses-based, I don't think) libraries for Python? I'm thinking of commonish stuff like progress bars, spinners, formatting data into columns, simple "choose option A B or C" menus, and so forth. (I could swear I saw a PyMOTW posting on that last one, but can't find it now. Maybe I should ask Doug...)
For what its worth, curses is really nice to play with, and "just works". I was writing a script a while back to scrape SA and present it in a pine/mutt like format for slacking off at work. Lost interest, but the UI was pretty easy to hack.

Zombywuf
Mar 29, 2008

duck monster posted:

Yeah, theres brute forcer worms out there alright. Use fail2ban, it basically sits over ssh (and a few other things, I think) and if it gets more than 3 or 4 consecutive login atempts from an IP, it drops it at the firewall for 15 minutes, and a bunch of other techniques. A friend was talking the other day about a hack that'd implement teergrubbing , basically rather than drop the connection, just fall comatose and try and exhaust the worms threads (Thus crashing it or grieviously slowing it down)

You just need a few iptables rules. If you have the TARPIT module (unmaintained for ages) you can tarpit the connection, i.e. set the TCP window to 0, but keep responding to SYNs (If you don't know what that means, basically its what you describe as teergrubbing (where the hell did that term come from?)).

http://paste2.org/p/278718 for anyone interested.

code:
 258K   15M            all  --  any    any     anywhere             anywhere            recent: SET name: SSH side: source 
 257K   15M BLOCK      all  --  any    any     anywhere             anywhere            recent: CHECK seconds: 60 hit_count: 5 name: SSH side: source 
 1237 72848 ACCEPT     all  --  any    any     anywhere             anywhere            
257K of blocked scan packets...

duck monster
Dec 15, 2004

Uh. Old anti spam trick. MTA , when it detects spam ,rather than hanging up it just sits there holding the spammers hand and falls asleep. The old spam bots where pretty much single threaded, so it'd kill it dead.

Old web trick that used to work awesome for web spam was to create a gzip file with a html header and 2 gigs of zeros. 2meg file.

Hide it on a <div style=display:none> type link and for added measure put it in robots.txt (to filter out good bots like google) and serve it as a gzipped html file. the thing would send the 2meg file, which would unpack inside the bots brain, flooding it with 2 gigabytes worth of zeros. Make sure apache knows not to unpack it under any costs and ONLY serve it if the bot accepts gzip html

It does horrifying things to browsers too.

deedee megadoodoo
Sep 28, 2000
Two roads diverged in a wood, and I, I took the one to Flavortown, and that has made all the difference.


LuckySevens posted:

Is there an alternative to mod_python, since I'm running 2.6? I found some vague info from google on how to alter the install, but I can't get it to work.

It depends on what you're trying to do with it. If you're looking to find something like the python publisher handler you can probably use zope but I've never used it personally so I can't say what the real differences are. If you wanted to use mod_python to access apache via handlers for each step I don't think there is an alternative to mod_python. But if all you're trying to do is write CGI scripts, you can just stick a "#!/bin/env python" at the top of your script and drop them in a directory with +ExecCGI.

tef
May 30, 2004

-> some l-system crap ->

Zombywuf posted:

teergrubbing (where the hell did that term come from?)).

A tarpit (also known as Teergrube, the German word for tarpit)...

Zombywuf
Mar 29, 2008

tef posted:

A tarpit (also known as Teergrube, the German word for tarpit)...

Crazy Germans, munging all their words together to make new words. Oh wait...

tef
May 30, 2004

-> some l-system crap ->
Is there good documentation on python internals or any advice for diving in?

I've been looking at the gil talk http://blip.tv/file/2232410 and I've been wondering if there might be some simple improvements that could mititage some of the gil problems on multicore.


Or at least make Control-C work in a mutlithreaded program.

Habnabit
Dec 30, 2007

lift your skinny fists like
antennas in germany.

bitprophet posted:

Yea, those are both curses, which I was hoping to avoid. Just want simple, no frills generic implementations of stuff like wget/curl type progress bars and etc.

I'll do more looking on my own; was mostly looking for a "oh yea, package X is a well known standard in this area" which doesn't seem to be the case. Not asking you guys to do my research for me :)

Uh what no. urwid is a general console UI library. One of its backends is curses, but it doesn't depend on curses or require you to use curses.

LuckySevens posted:

Is there an alternative to mod_python, since I'm running 2.6? I found some vague info from google on how to alter the install, but I can't get it to work.

This question makes no sense. You're going to have to explain what it is you want.
mod_python also sucks, for reasons I've already explained several times in this very thread.

m0nk3yz
Mar 13, 2002

Behold the power of cheese!

tef posted:

Is there good documentation on python internals or any advice for diving in?

I've been looking at the gil talk http://blip.tv/file/2232410 and I've been wondering if there might be some simple improvements that could mititage some of the gil problems on multicore.


Or at least make Control-C work in a mutlithreaded program.

Ugh. That talk.

Here's one link for you: http://www.python.org/dev/ - unfortunately, I don't know of a doc on interpreter internals, or if I did, I've forgotten

king_kilr
May 25, 2007

m0nk3yz posted:

Ugh. That talk.

Here's one link for you: http://www.python.org/dev/ - unfortunately, I don't know of a doc on interpreter internals, or if I did, I've forgotten

ceval.c, it really is remarkably readable. For unladen swallow eval.cc and llvm_fbuilder.cc, unladen-swallow might be a good place to look just because they do have the man power and the strong inclination to look at these issues (no offense monkeyz).

Scaevolus
Apr 16, 2007

king_kilr posted:

ceval.c, it really is remarkably readable. For unladen swallow eval.cc and llvm_fbuilder.cc, unladen-swallow might be a good place to look just because they do have the man power and the strong inclination to look at these issues (no offense monkeyz).
It's funny how in the OSS world "man-power" means 3 people.

m0nk3yz
Mar 13, 2002

Behold the power of cheese!

king_kilr posted:

ceval.c, it really is remarkably readable. For unladen swallow eval.cc and llvm_fbuilder.cc, unladen-swallow might be a good place to look just because they do have the man power and the strong inclination to look at these issues (no offense monkeyz).

What? No worries, I'd drop everything and work on unladen swallow if I had the time. The more core-devs that work on it the happier I am. I'd really like to see unladen become core sooner rather than later. Now, I'd bitch myself blue if someone was suggesting dump time into pypy, because that's a science project, and has been for awhile. Depending on how the next milestone go, I'm considering cutting over to running it alongside 2.6 in production.

quote:

It's funny how in the OSS world "man-power" means 3 people.

Well, it's 3 almost-full time people whose job it is to kick rear end on the project. For example, I don't know of anyone who works on python-core who get paid close to full time to work on it. Most of the people I speak to get *maybe* 25% of their time at paying jobs to work on it. I get around 20% usually.

Also, it's not just Collin/Jeffery/Thomas - they've actually got 4 or 5 other people helping out, as well as LLVM people pitching in. Well, that and they have an actual project plan, milestones and focus. It's not often you get something like that in OSS.

m0nk3yz fucked around with this message at 12:59 on Jun 27, 2009

TOO SCSI FOR MY CAT
Oct 12, 2008

this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

m0nk3yz posted:

Now, I'd bitch myself blue if someone was suggesting dump time into pypy, because that's a science project, and has been for awhile. Depending on how the next milestone go, I'm considering cutting over to running it alongside 2.6 in production.
PyPy is very compatible with CPython now, and is currently faster than Unladen.

Scaevolus
Apr 16, 2007

Janin posted:

PyPy is currently faster than Unladen.

on some very synthetic benchmarks

king_kilr
May 25, 2007

Janin posted:

PyPy is very compatible with CPython now, and is currently faster than Unladen.

I'm not sure where you're getting your numbers from:

PyPy Guys posted:

Python interpreter is now between 0.8 and 2x (and in some corner case 3-4x) slower than CPython

If we assume they're refering to CPython 2.5 (which is the version they're compatible with) then they'll be maybe 5% slower (above this) than 2.6, and over 20+% slower than unladen swallow (above and beyond the other numbers).

m0nk3yz
Mar 13, 2002

Behold the power of cheese!

Janin posted:

PyPy is very compatible with CPython now, and is currently faster than Unladen.

It is... with very selective benchmarks - namely long running/hot loops, and I don't know anyone who would run it in production. Run the unladen swallow benchmarks (real world, no bullshit ones) and you'll see the real performance.

I'm sure PyPy, when it gets focus, or commercial backing/sponsorship could very well become a better platform than CPython. The JIT work it's doing is pretty cool, from a "hey that's neat" type of perspective, but the difference between what they've been doing, and the unladen swallow is vast.

Not to mention unladen swallow is running youtube. That's real-world.

TOO SCSI FOR MY CAT
Oct 12, 2008

this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"
http://morepypy.blogspot.com/2009/06/jit-progress.html

quote:

The performance of the resulting code is quite good: even with Boehm (the GC that is easy to compile to but gives a slowish pypy-c), a long-running loop typically runs 50% faster than CPython. That's "baseline" speed, moreover: we will get better speed-ups by applying optimizations on the generated code.

Even unoptimized, with the less-performant GC option, it's faster than CPython. 50% is obviously a corner case, but I have seen 5-8% improvements over 2.x trunk. Once it achieves 3.x compatibility, I plan to begin testing it for use in personal projects.

king_kilr
May 25, 2007

Janin posted:

http://morepypy.blogspot.com/2009/06/jit-progress.html


Even unoptimized, with the less-performant GC option, it's faster than CPython. 50% is obviously a corner case, but I have seen 5-8% improvements over 2.x trunk. Once it achieves 3.x compatibility, I plan to begin testing it for use in personal projects.

... the post even specifically says it's for a very specific case, and not for general code. More importantly if the improvements you've seen are 5-8% (which is inconsistant with the PyPy guys' own statements) then that's 15% slower than Unladen swallow's Q1 release at a minimum.

TOO SCSI FOR MY CAT
Oct 12, 2008

this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

king_kilr posted:

... the post even specifically says it's for a very specific case, and not for general code. More importantly if the improvements you've seen are 5-8% (which is inconsistant with the PyPy guys' own statements) then that's 15% slower than Unladen swallow's Q1 release at a minimum.

Python trunk, which contains the Unladen improvements.

m0nk3yz
Mar 13, 2002

Behold the power of cheese!

Janin posted:

Python trunk, which contains the Unladen improvements.

Actually, it doesn't - it only contains some of the "easy to upstream" ones.

Adbot
ADBOT LOVES YOU

TOO SCSI FOR MY CAT
Oct 12, 2008

this is what happens when you take UI design away from engineers and give it to a bunch of hipster art student "designers"

m0nk3yz posted:

Actually, it doesn't - it only contains some of the "easy to upstream" ones.

From http://code.google.com/p/unladen-swallow/wiki/Releases

quote:

2009Q1

The first release. Unladen Swallow 2009Q1 is based on CPython 2.6.1. This release is primarily a collection of tweaks to 2.6.1, most of which are now available in mainline Python trunk. What are not currently available are in the process of being pushed upstream.

  • Locked thread