Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
tef
May 30, 2004

-> some l-system crap ->
basically tcp is a skeuomorph for the phone network :q:

Adbot
ADBOT LOVES YOU

Socracheese
Oct 20, 2008

JewKiller 3000 posted:

you're welcome to your opinion, but loving say something, not just "HURR PYTHON GOOD LANGUAGE DURR SPAM EGGS HEIL PEP 8"

perhaps you could list the things you like about the python language that other languages do not provide?

python is cool because:
- its really easy
- its fast enough for your dumb bullshit
- it has a lot of libraries for most common stuff you wanna do
- its not php

Socracheese fucked around with this message at 06:03 on Jun 28, 2013

uG
Apr 23, 2003

by Ralp
thats every plang that isnt php

Bloody
Mar 3, 2013

Nomnom Cookie posted:

go does closures properly *and* has a really novel and useful take on OOP. you can think of it as "C done right" or alternately "javascript made by people who have a clue"

it even has language support for maps (that's a computer scientist way of saying "hash table" for you p-langers) so if you're coming from a bad language you can just dive right in and poo poo out code like nobody's business

counterpoint: but then you would be using go

i like that there are two programming languages named go

there's go
and there's go!

thats some ruby style bullshit right there

X-BUM-RAIDER-X
May 7, 2008
cURLing in the tcp stack

prefect
Sep 11, 2001

No one, Woodhouse.
No one.




Dead Man’s Band

uG posted:

thats every plang that isnt php

cpan owns

Symbolic Butt
Mar 22, 2009

(_!_)
Buglord

uG posted:

all you python scrubs are going to be programming perl6 in 10 years (if you havent killed urself by then)

eh I'm ok with this. like it could be way worse.

luchadornado
Oct 7, 2004

A boombox is not a toy!

the thing that sucks the most about javascript for me is that it is totally ok with you writing really lovely code because of no type safety, several confusing ways of declaring functions, etc. working on your own poo poo isn't so bad. working with other developers across large projects some of which really suck at coding is where the weaknesses really stand out.

Elder Postsman
Aug 30, 2000


i used hot bot to search for "teens"

JewKiller 3000 posted:

you're welcome to your opinion, but loving say something, not just "HURR PYTHON GOOD LANGUAGE DURR SPAM EGGS HEIL PEP 8"

perhaps you could list the things you like about the python language that other languages do not provide?

i like python because it's really well integrated into arcgis (which is what i use p much all day every day my job) so it's super easy to automate all these dumb things that need to be done. also it was easy to learn, i'm a big dumb idiot moron (as evidenced by the code i've posted in this thread) and even i was able to pick it up really quickly

FamDav
Mar 29, 2008

Nomnom Cookie posted:

it even has language support for maps (that's a computer scientist way of saying "hash table" for you p-langers) so if you're coming from a bad language you can just dive right in and poo poo out code like nobody's business

a) what language implements an associative container and calls it an hash table instead of a map?

b) all hash tables are maps, but not all maps are hash tables

c) i was curious what go's underlying implementation for map is and the spec doesnt even give algorithmic complexity for its operations. thanks rob.

prefect
Sep 11, 2001

No one, Woodhouse.
No one.




Dead Man’s Band

FamDav posted:

a) what language implements an associative container and calls it an hash table instead of a map?

in perl they're officially named "associative arrays", but everybody calls them "hashes" :shrug:

tef
May 30, 2004

-> some l-system crap ->

FamDav posted:

a) what language implements an associative container and calls it an hash table instead of a map?

perl, ruby.

python calls it a dict, not a hash or a map.

quote:

c) i was curious what go's underlying implementation for map is and the spec doesnt even give algorithmic complexity for its operations. thanks rob.


quote:

hashmap.c is a multilevel hash table (a short tree so that it doesn't
have to be contiguous in memory; the tree grows as the table needs
to). Collisions are handled by linear probing.

http://golang.org/src/pkg/runtime/hashmap.c

uG
Apr 23, 2003

by Ralp

perlguts posted:

A stash is a hash table (associative array) that contains all of the different objects that are contained within a package.

of course if you are reading perlguts im so sorry

FamDav
Mar 29, 2008
why cant they put that in their spec? i realize that the implementation of their map type is changing every day but i would appreciate it if i could just go read through the spec and have a general understanding of the performance characteristics of basic types.

Bloody
Mar 3, 2013

FamDav posted:

why cant they put that in their spec? i realize that the implementation of their map type is changing every day but i would appreciate it if i could just go read through the spec and have a general understanding of the performance characteristics of basic types.

if you cared about performance at all you wouldnt be using a plang duh

tef
May 30, 2004

-> some l-system crap ->

FamDav posted:

why cant they put that in their spec? i realize that the implementation of their map type is changing every day but i would appreciate it if i could just go read through the spec and have a general understanding of the performance characteristics of basic types.

because it might be a property of the implementation, rather than the design :q:

FamDav
Mar 29, 2008

tef posted:

because it might be a property of the implementation, rather than the design :q:

Plastic Snake
Mar 2, 2005
For Halloween or scaring people.

qft and also god drat that movie was horrific

Doc Block
Apr 15, 2003
Fun Shoe
what moobie is dat?

MeruFM
Jul 27, 2010
my favorite reason to use python
it doesn't make me want to kill myself but can still get me a job

Bloody
Mar 3, 2013

MeruFM posted:

my favorite reason to use python
it doesn't make me want to kill myself but can still get me a job

same except hardware engineering instead of plang

MeruFM
Jul 27, 2010
hardware specific C makes me want to kill myself so I still take issue with that
especially if the predecessor felt the need to write in assembly

Greed is eternal
Jun 8, 2008
I actually used Clojure for something useful today. It felt like I was hacking the matrix.

Plastic Snake
Mar 2, 2005
For Halloween or scaring people.

Doc Block posted:

what moobie is dat?

antichrist

DONT THREAD ON ME
Oct 1, 2002

by Nyc_Tattoo
Floss Finder
so i'm working on a thing to do loudness monitoring for broadcast TV. basically i'm just trying to implement this algorithm:

http://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1770-3-201208-I!!PDF-E.pdf

pretty sure i've got the algorithm figured out but holy christ my program takes like 10 minutes to go through a 20 minute AIFF file. it should be near instant. i've never dealt with inputs of this nature.

basically i'm just using libsndfile to read all of the PCM signals into a really huge array (48000 samples/second * 2 channels * 60 seconds * 20 minutes). clearly this is not optimal. how should i go about optimizing this?

Doc Block
Apr 15, 2003
Fun Shoe
If it's 16 bits per sample then remember to multiply that number by 2, giving you about ~220MB of samples to process. If it's 24-bit multiply by 3, giving you about ~660MB of samples data.

If you can, load it as-needed instead of loading the whole thing first. Maybe have a thread that does the loading and another that processes each chunk once it's loaded (so while the loading thread is blocked waiting for the OS to load more data, the processing thread can be chewing away on the previous chunk). Or, even better, do one processing thread per core, breaking the data up into as many pieces as you have processing threads. Assuming the algorithm you're using isn't dependent on the results of the previous operation or whatever.

edit: and that's just brute-force stuff. Make sure your implementation of the algorithm is well optimized. Run it through a profiler and see where your bottlenecks are.

Doc Block fucked around with this message at 03:05 on Jul 4, 2013

DONT THREAD ON ME
Oct 1, 2002

by Nyc_Tattoo
Floss Finder

Doc Block posted:


If you can, load it as-needed instead of loading the whole thing first. Maybe have a thread that does the loading and another that does the processing. Or, even better, break the thing up into chunks and have each chunk processed in a separate thread if you can.

Yeah, this was my first thought, only because i've seen it suggested elsewhere. it's pretty easy to understand why breaking it into chunks and multithreading it will make it a lot faster, but the idea that loading as-needed will help is hard for me to grasp.

is there any reason to spawn more threads than I have cores?

luchadornado
Oct 7, 2004

A boombox is not a toy!

i needed to setup an admin site with some configuration stuff today, and started setting up entity framework to hook into an existing MS SQL Server instance. thought I should try something new - so I tried RavenDB in embedded mode. digging the simplicity and design of it so far.

Doc Block
Apr 15, 2003
Fun Shoe

chumpchous posted:

Yeah, this was my first thought, only because i've seen it suggested elsewhere. it's pretty easy to understand why breaking it into chunks and multithreading it will make it a lot faster, but the idea that loading as-needed will help is hard for me to grasp.

is there any reason to spawn more threads than I have cores?

read my edit. when you load and convert the entire thing into raw samples before you can even start processing, your program is doing almost nothing during that time. so you load it in chunks, with another thread that does the processing (with a processing queue). so while the loading thread is blocked, waiting for the OS to load more data off the disk (or is converting it into raw samples), the processing thread is chewing on the previous chunk, and when the loading thread has loaded another chunk it adds it to the "to be processed" queue. this is assuming many chunks (i.e. not just 3 big chunks or whatever, unless the file size is really small).

or load and convert it to raw samples all in one go, and then do the "one chunk and thread per core" thing. and if the processor supports hyperthreading then you can spawn even more threads (a 4 core CPU with hyperthreading means 8 logical cores, so you spawn 8 threads). the reason you probably don't want more threads than cores is that, once each core is near 100% utilization, adding more threads doesn't make things go any faster.

plus you have to think about cache utilization, which is partly a data structure issue. unnecessary pointer dereferences causing cache misses, etc.

right now, though, you need to figure out where your program's time is being spent so that you don't waste time blindly making useless optimizations. do that by profiling the code. the exact method depends on the language & compiler you're using. and, by the way, what language and compiler are you using?

Doc Block fucked around with this message at 03:51 on Jul 4, 2013

DONT THREAD ON ME
Oct 1, 2002

by Nyc_Tattoo
Floss Finder
the slow as poo poo version was a rough out i did in ruby, just to make sure i understood the process -- which obviously accounts for a lot of the slowness. now i'm redoing it in straight C with LLVM, but i havent started yet. i did quite a bit of profiling when i was doing objective C stuff, so I'm fairly familiar with those tools.

thanks for the help. i'm officially drinking now, but i'll respond in more detail tomorrow

Malcolm XML
Aug 8, 2009

I always knew it would end like this.

chumpchous posted:

Yeah, this was my first thought, only because i've seen it suggested elsewhere. it's pretty easy to understand why breaking it into chunks and multithreading it will make it a lot faster, but the idea that loading as-needed will help is hard for me to grasp.

is there any reason to spawn more threads than I have cores?

Basically:

Read in X bytes at a time on IO thread (asynchronously) into a processing channel

Have N available computation thread(s) process chunks as available from channel

Challenge: making sure chunk boundary edge cases are handled properly (probably not a huge deal in your case)

Tune X and N for your workload/computer it's not usually as simple as N = number of cores since getting 100% utilization is rare

Or implement this on a dsp whatever floats your boat

Test Measure and Iterate

Malcolm XML fucked around with this message at 04:02 on Jul 4, 2013

Malcolm XML
Aug 8, 2009

I always knew it would end like this.

chumpchous posted:

the slow as poo poo version was a rough out i did in ruby, just to make sure i understood the process -- which obviously accounts for a lot of the slowness. now i'm redoing it in straight C with LLVM, but i havent started yet. i did quite a bit of profiling when i was doing objective C stuff, so I'm fairly familiar with those tools.

thanks for the help. i'm officially drinking now, but i'll respond in more detail tomorrow

Might want to check if you can have something like MATLAB generate the code for you actually

Doc Block
Apr 15, 2003
Fun Shoe
if you're doing it on a mac, you can use Instruments to help profile your code, and you can take advantage of grand central dispatch (even in C) to make the threading easier.

Bloody
Mar 3, 2013

just use boost

uG
Apr 23, 2003

by Ralp
but c++ is terrible

Bloody
Mar 3, 2013

yeah but its also just c so you can do everything in c but still use c++ libs

its a beautiful abomination mlmp dsyp

uG
Apr 23, 2003

by Ralp
yes that is something a terrible programmer would think and say

uG
Apr 23, 2003

by Ralp
420 write my own string libraries every day

Bloody
Mar 3, 2013

uG posted:

yes that is something a terrible programmer would think and say

let me show you my thesis code :getin:

Adbot
ADBOT LOVES YOU

Bloody
Mar 3, 2013

Bloody posted:

let me show you my thesis code :getin:

which i now have to rewrite a significant portion of because its poorly designed and a new batch of input data is formatted rather differently than the old batch

  • Locked thread