Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »

Shaocaholica: Oct 29, 2002; Fig. 5E

defmacro posted:

If you're rather stick to the shell, give parallel a try.

This could be fun, thanks.

# ? Oct 25, 2011 07:25

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 12:58

floWenoL: Oct 23, 2002

Shaocaholica posted:

Does anyone know how I can something similar to this but instead of a single sequential process, spawn multiple processes/threads?
code:
#! /usr/local/bin/tcsh -fb

foreach some_file ( `find . -type f -name \*.poo` )
    echo $some_file
    convert $some_file $some_file:r.pee
end
Doesn't have to be tcsh.

xargs with -P may also work.

# ? Oct 25, 2011 09:07

Zombywuf: Mar 29, 2008

code:

find . -type f -name \*.poo | sed 's/\(.*\)\.poo/\1.poo \1.pee/ | xargs -P0 convert

To be precise.

Might want to check your find command outputs one line per file and -P0 in xargs implies -L1.

# ? Oct 25, 2011 12:31

Uziel: Jun 28, 2004; Ask me about losing 200lbs, and becoming the Viking God of W&W.

I'm looking for recommendations as to where to store an encrypted password for an application.

The users of the application all share a login that can change so we need to house the password in a central location on an intranet server. We don't want to keep it in a database though.

Any ideas? Or directions we can look in?

# ? Oct 25, 2011 13:06

baquerd: Jul 2, 2007; by FactsAreUseless

Uziel posted:

The users of the application all share a login that can change so we need to house the password in a central location on an intranet server. We don't want to keep it in a database though.

Why do they all need the same login? That's just not good from a security or accountability standpoint.

Storing it in a database with application server login specific permissions for access would be good.

What sort of an application requires the password to be available over the network? Or do you mean you want users to be able to access the encrypted password, decrypt it, and then enter it on their machines?

# ? Oct 25, 2011 13:13

Uziel: Jun 28, 2004; Ask me about losing 200lbs, and becoming the Viking God of W&W.

baquerd posted:

Why do they all need the same login? That's just not good from a security or accountability standpoint.

Storing it in a database with application server login specific permissions for access would be good.

What sort of an application requires the password to be available over the network? Or do you mean you want users to be able to access the encrypted password, decrypt it, and then enter it on their machines?

It is actually specifically for security/accountability. The application does terminal emulation automation in a billing system. The central login's profile only has access to a specific screen and can only read/write information on certain fields on that one screen. Users of the application have their own billing system logins that aren't as restricted and we want to be able to differentiate what was done on that screen by our application vs what was done on that screen by a regular user that normally has/needs access to it outside of said application's scope.

The user will never know what the username or password is. During login, the application and the server exchange encryption keys and the username and password are decrypted and entered without being drawn on the screen. When prompted for the password to change by the terminal session, the change is handled, encrypted and updated on the server. On next login, other application users get the new password.

# ? Oct 25, 2011 13:30

baquerd: Jul 2, 2007; by FactsAreUseless

Uziel posted:

The user will never know what the username or password is. During login, the application and the server exchange encryption keys and the username and password are decrypted and entered without being drawn on the screen. When prompted for the password to change by the terminal session, the change is handled, encrypted and updated on the server. On next login, other application users get the new password.

And how is the application run? Is this a portable binary, full install on a local machine, application/web server based, run over remote windowing on a specific machine...?

A database with secured permissions on the table would really be a good way to go in most any case, why don't you want to use one? There are lightweight databases that can be hosted with minimal overhead.

# ? Oct 25, 2011 14:26

NextTime000: Feb 3, 2011; bweeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
<----------------------------

anyone know a good place I can learn up on "general architecture ideas (in terms of general OOP)"? I was told to look that up (along with a couple other things that I already figured out) so I will have a much firmer grasp of fundamentals and junk so a company I have been applying to for the last 8 months will finally hire me.

# ? Oct 25, 2011 14:51

Uziel: Jun 28, 2004; Ask me about losing 200lbs, and becoming the Viking God of W&W.

baquerd posted:

And how is the application run? Is this a portable binary, full install on a local machine, application/web server based, run over remote windowing on a specific machine...?

A database with secured permissions on the table would really be a good way to go in most any case, why don't you want to use one? There are lightweight databases that can be hosted with minimal overhead.

Its just an exe with a config file deployed via ClickOnce. It already does pull and send data from a SQL server. It's mostly wanting to know if there are better options so when our security audit comes up, we have the most secure setup as possible.

# ? Oct 25, 2011 15:19

baquerd: Jul 2, 2007; by FactsAreUseless

Uziel posted:

Its just an exe with a config file deployed via ClickOnce. It already does pull and send data from a SQL server. It's mostly wanting to know if there are better options so when our security audit comes up, we have the most secure setup as possible.

Ideally, you wouldn't ever expose a login via the application, but you likely have limits on what you can modify with the billing system. The database solution with permissions is secure unless the database login itself is exposed (check against plaintext strings in your binary file and obfuscate or use an NT integrated security solution) or non-SSL TCP connections are used.

Adding NT authentication and possible machine level whitelisting would provide additional security.

# ? Oct 25, 2011 15:58

gonadic io: Feb 16, 2011; >>=

I'm just starting to use LaTeX and while finding out how to do a particular thing doesn't seem that hard, knowing what to do is a lot less certain.

I'm essentially writing a report containing code snippets (Haskell if it matters). Would the best way to have verbatim (or alltt) blocks around the code, make it a monospaced font and then format it myself, or use a library like listings to format it for me?

I'm pretty close to just giving up and slapping it into OpenOffice but I wanted to try and learn something new. Any tips?

# ? Oct 25, 2011 17:57

nielsm: Jun 1, 2009

AlsoD posted:

I'm essentially writing a report containing code snippets (Haskell if it matters). Would the best way to have verbatim (or alltt) blocks around the code, make it a monospaced font and then format it myself, or use a library like listings to format it for me?

Listings is cool poo poo, use it. Also, its "aligned proportional width text" stuff is great, proportional width text is easier to read and listings can solve the problem of proportional width fonts not always lining up as intended.

# ? Oct 25, 2011 18:33

ToxicFrog: Apr 26, 2008

floWenoL posted:

xargs -P

holy poo poo

AlsoD posted:

I'm just starting to use LaTeX and while finding out how to do a particular thing doesn't seem that hard, knowing what to do is a lot less certain.

I'm essentially writing a report containing code snippets (Haskell if it matters). Would the best way to have verbatim (or alltt) blocks around the code, make it a monospaced font and then format it myself, or use a library like listings to format it for me?

Looking at my previous papers, I've just used the verbatim environment and \verbatiminput, but I should probably have been using listings.

quote:

I'm pretty close to just giving up and slapping it into OpenOffice but I wanted to try and learn something new. Any tips?

Best general LaTeX tip I can give is to use a LaTeX-aware IDE like Kile, which automates the entire build process and has built-in help and autocompletion; it makes learning LaTeX much easier by doing the ugliest part (the build) for you and making it easier to discover what you can do as well as how.

# ? Oct 25, 2011 18:58

TheReverend: Jun 21, 2005

Is there any information out there anyone can recommend regarding designing some mapping software? Right now I use third party SDKs and they aren't cutting the mustard. The map's features will be contained in a database and need to be drawn on-the-fly so it's more complicated than just a bitmap viewer.

# ? Oct 25, 2011 21:46

defmacro: Sep 27, 2005; cacio e ping pong

AlsoD posted:

I'm just starting to use LaTeX and while finding out how to do a particular thing doesn't seem that hard, knowing what to do is a lot less certain.

I'm essentially writing a report containing code snippets (Haskell if it matters). Would the best way to have verbatim (or alltt) blocks around the code, make it a monospaced font and then format it myself, or use a library like listings to format it for me?

I'm pretty close to just giving up and slapping it into OpenOffice but I wanted to try and learn something new. Any tips?

listings sucks, I'd recommend you use minted/pygments if you can get it to work. The results are head and shoulders above what listings can do. This will require some more upfront time getting used to the library, but it's well worth it.

Also, make sure you're using an editor with some kind of snippet functionality (vim/emacs/TextMate/whatever). This allows you to type sec<TAB> and it autofills the markup for a section declaration and let's you tab through the fields. An absolute must if you become a heavy LaTeXer. And if you get stuck, you'll probably get better answers in the LaTeX Megathread.

edit: This is a good general LaTeX guide. I'd read at least the first two sections, add in the third if you'll be doing lots of math. I still pull this up every so often as a reference.
edit2: VVV that rules

defmacro fucked around with this message at 05:04 on Oct 26, 2011

# ? Oct 25, 2011 22:29

ultrafilter: Aug 23, 2007; It's okay if you have any questions.

AlsoD posted:

I'm just starting to use LaTeX and while finding out how to do a particular thing doesn't seem that hard, knowing what to do is a lot less certain.

I'm essentially writing a report containing code snippets (Haskell if it matters). Would the best way to have verbatim (or alltt) blocks around the code, make it a monospaced font and then format it myself, or use a library like listings to format it for me?

I'm pretty close to just giving up and slapping it into OpenOffice but I wanted to try and learn something new. Any tips?

Haskell actually supports LaTeX annotations. I recommend doing it that way.

# ? Oct 25, 2011 23:04

zapateria: Feb 16, 2003

This is probably a very basic question but my mind is fried right now and I can't think straight.

I have a dbtable VALUES with the columns "key", "datetime", "value".
There is one value every hour of the day, so example data:

code:

KEY   DATETIME              VALUE
42    26.10.2011 00:00.00   16
42    26.10.2011 00:01:00   17
42    26.10.2011 00:02:00   17

etc. for all 24 hours.

Now I want to display this in a htmltable that has the DATE on the Y axis and HOUR on the X axis, like this:

code:

DATE          HOUR0 HOUR1 HOUR2...HOUR23
26.10.2011    16    17    17      29
27.10.2011    15    14    13      18
:
:

What would be the most efficient way of doing this? (using PHP/SQL)

# ? Oct 26, 2011 08:04

NextTime000: Feb 3, 2011; bweeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
<----------------------------

NextTime000 posted:

anyone know a good place I can learn up on "general architecture ideas (in terms of general OOP)"? I was told to look that up (along with a couple other things that I already figured out) so I will have a much firmer grasp of fundamentals and junk so a company I have been applying to for the last 8 months will finally hire me.

alright, well I guess I know that all he was meaning was understanding the whole concept of having web server / app server / database server to handle applications, and perhaps splitting some services to be handled by other servers sometimes.

that is about all I can find on the subject so did I miss anything?

# ? Oct 26, 2011 15:00

rolleyes: Nov 16, 2006; Sometimes you have to roll the hard... two?

"Architecture" is a bit of an ambiguous statement without further context. I mean they could be talking about design patterns, i.e. the structure/architecture of the code within OOP software.

# ? Oct 26, 2011 15:34

NextTime000: Feb 3, 2011; bweeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
<----------------------------

well here is a straight quote from my contact's last e-mail for context:

NextTime000's Contact posted:

...Take some time looking over OOP, order notation for performance improvements and general architecture ideas, and revisit the test. Expand on it if you want. Don�t worry about time this go around; I want to see your best. If it takes 1-2 months or more is not a problem (I realize what I�m saying to go study is a set of very broad topics)...

I may not really spend a MONTH going over those things (and re-doing the test) as I already get Order Notation, and over in the Flash Questions thread I already learned about stuff that will make my test run a whole lot better than before. my goal is to re-do my test this weekend and send it back next week. I just wanna make sure I understand as much as I can before then.

# ? Oct 26, 2011 16:02

rolleyes: Nov 16, 2006; Sometimes you have to roll the hard... two?

Ok, so in that context OOP wasn't being linked to architecture really, they're just two topics on a list of things to look at. Since you seem to be going for a Flash job then yes, I think you're probably on the right track in terms of interpreting "architecture" to mean, roughly, "hosting architecture".

# ? Oct 26, 2011 16:57

qntm: Jun 17, 2009

rolleyes posted:

"Architecture" is a bit of an ambiguous statement without further context. I mean they could be talking about design patterns, i.e. the structure/architecture of the code within OOP software.

Today I was on a training call for 20 minutes before I realised that the woman was talking about solution architecture - as in, putting lots of software together to solve some client's business problems - rather than software architecture.

# ? Oct 26, 2011 19:31

more like dICK: Feb 15, 2010; This is inevitable.

Are there any good resources for people who are new to software licensing/redistribution?

# ? Oct 26, 2011 22:32

the talent deficit: Dec 20, 2003; self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture

ICK posted:

Are there any good resources for people who are new to software licensing/redistribution?

Licensing is only ever really simple or terrifyingly complex. Do you have specific questions? (Please don't ask, 'what license is best?')

# ? Oct 27, 2011 08:03

pokeyman: Nov 26, 2006; That elephant ate my entire platoon.

ICK posted:

Are there any good resources for people who are new to software licensing/redistribution?

A lawyer.

# ? Oct 27, 2011 17:33

more like dICK: Feb 15, 2010; This is inevitable.

pokeyman posted:

A lawyer.

After looking into it further, this is the best option in my case, thanks.
:shobon:

# ? Oct 27, 2011 20:16

Gxp: Jan 23, 2004; Son I'm afraid this is a stereo disaster.

Pre-summary: Is there a quicker non-lovely way to import (lots of) XML into a MySQL database than insertion?

I have a large number of XML files with entries I'm trying to get into a MySQL database, but the XML is formatted kind of strangely. The data I want to put in individual columns are properties of each node instead of being properly nested. So at least some processing needs to be done to get it into order. It's like this:

code:

<customers>
	<customer id="3240" name="Mike" Timestamp="123456789">
	<customer id="3241" name="Tony" Timestamp="223456789">
	...
</customers>

I wrote a .net program for this already, parsing the files that way is pretty quick, but the insertion is a huge bottleneck and I may have to do the whole thing multiple times in the future, for slightly over a million records. Is there a way to do this at mysql's command line or something? I wanted to avoid having to convert 900 megs of XML to CSV for importation or something but if that's the fastest way then so be it. I know phpmyadmin supports xml import but only in it's own format and that might be just as slow.

# ? Oct 28, 2011 01:57

Zombywuf: Mar 29, 2008

Gxp posted:

I wrote a .net program for this already, parsing the files that way is pretty quick, but the insertion is a huge bottleneck and I may have to do the whole thing multiple times in the future, for slightly over a million records. Is there a way to do this at mysql's command line or something? I wanted to avoid having to convert 900 megs of XML to CSV for importation or something but if that's the fastest way then so be it. I know phpmyadmin supports xml import but only in it's own format and that might be just as slow.

If you've already written a program to do the parsing (although a lovely command line tool I wrote called Xawk (although not the one I just found a manual for when googling it which looks like exactly the same idea) would do the job), outputting it as CSV shouldn't be hard. Failing that look at BULK INSERTs.

# ? Oct 28, 2011 02:04

Gxp: Jan 23, 2004; Son I'm afraid this is a stereo disaster.

Zombywuf posted:

If you've already written a program to do the parsing (although a lovely command line tool I wrote called Xawk (although not the one I just found a manual for when googling it which looks like exactly the same idea) would do the job), outputting it as CSV shouldn't be hard. Failing that look at BULK INSERTs.

Searching for bulk inserts lead me to find LOAD DATA INFILE which is pretty much what I was looking for, so thank you! Dumping all that xml into a giant csv file only takes about 40 seconds, and insertion takes about 10 seconds. This is substantially better than the 70+ minutes insertion was taking. :barf:

# ? Oct 28, 2011 02:55

NextTime000: Feb 3, 2011; bweeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
<----------------------------

rolleyes posted:

Ok, so in that context OOP wasn't being linked to architecture really, they're just two topics on a list of things to look at. Since you seem to be going for a Flash job then yes, I think you're probably on the right track in terms of interpreting "architecture" to mean, roughly, "hosting architecture".

well just to be sure, I asked him whether he meant hosting or OOP architecture and here was his reply:

quote:

Yes I meant coding architecture for your application as a whole, such as good OOP practices, when to/not to use singletons and other data structures, etc.

he wants me to look over OOP in general it seems :v:

anyone got a good idea of a good place look that up? I wanna check another source than just whatever I can find in a quick google search.

# ? Oct 29, 2011 07:25

rolleyes: Nov 16, 2006; Sometimes you have to roll the hard... two?

NextTime000 posted:

well just to be sure, I asked him whether he meant hosting or OOP architecture and here was his reply:

he wants me to look over OOP in general it seems anyone got a good idea of a good place look that up? I wanna check another source than just whatever I can find in a quick google search.

drat, good job you checked! That wasn't well articulated on his part.

Does all this relate to ActionScript or are you looking at it from a generic viewpoint? I wouldn't really know where to look for AS but in terms of general explanation you can do a lot worse than wiki's design patterns article: http://en.wikipedia.org/wiki/Design_pattern_(computer_science)

The thing is, you can buy entire books on this subject alone.

# ? Oct 29, 2011 12:15

NextTime000: Feb 3, 2011; bweeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
eeeeeeeeeeeeeeeeeeeee
<----------------------------

rolleyes posted:

drat, good job you checked! That wasn't well articulated on his part.

Does all this relate to ActionScript or are you looking at it from a generic viewpoint? I wouldn't really know where to look for AS but in terms of general explanation you can do a lot worse than wiki's design patterns article: http://en.wikipedia.org/wiki/Design_pattern_(computer_science)

The thing is, you can buy entire books on this subject alone.

Thanks! as for relating back to what I am doing, here was another note from the previous e-mail he sent to me:

quote:

A general note, you should look into Object Oriented Programming and how you can use it to simplify your code a bit. Your architecture is very procedural and you�d save a lot of headache with making things more object based.

in the test I sent back, he liked that I was able to solve the problem very well, however I was using 2 arrays with nested for-loops to look-up the data I wanted. but that problem was solved over here :ms:

# ? Oct 29, 2011 19:20

etcetera08: Sep 11, 2008

Anyone help with building MacVim with python 3.2+ support? I found this answer on StackOverflow, which seems to be on the right track but I tried his "patched" MacVim download and tried to build it configured with the option he has listed but to no avail. I may be missing something obvious?

I'm on OS X Lion and have Python 3.2.2 installed, if that helps. On a sidenote, anyone with tips on getting a good+easy MacVim + IPython (0.11) workflow working would be a huge help too.

# ? Oct 30, 2011 04:43

Shaocaholica: Oct 29, 2002; Fig. 5E

Are there any fundamental differences between popular OS schedulers/memory management/etc that would greatly affect the speed at which the same code is executed on the same machine barring the use of GPUs and their driver differences?

# ? Nov 1, 2011 00:01

shrughes: Oct 11, 2008; (call/cc call/cc)

Shaocaholica posted:

Are there any fundamental differences between popular OS schedulers/memory management/etc that would greatly affect the speed at which the same code is executed on the same machine barring the use of GPUs and their driver differences?

In average case scenarios or worst case scenarios? In worst case scenarios, the OS scheduler can be very important.

What about memory management? What kind of memory management are you talking about? In-process memory management (i.e. malloc?) or something else? If you're asking whether the malloc implementation is important, well hell yeah, it's very important and you can get very big speedups. It's practically not optional to swap out the default malloc with a choice of your own.

# ? Nov 1, 2011 00:10

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

shrughes posted:

It's practically not optional to swap out the default malloc with a choice of your own.

This is the part where we mention that glibc malloc is a loving Liar in addition to its other faults, right?

# ? Nov 1, 2011 02:44

shrughes: Oct 11, 2008; (call/cc call/cc)

Otto Skorzeny posted:

This is the part where we mention that glibc malloc is a loving Liar in addition to its other faults, right?

It is? Sorry, how?

# ? Nov 1, 2011 04:56

Blotto Skorzany: Nov 7, 2008; He's a PSoC, loose and runnin'
came the whisper from each lip
And he's here to do some business with
the bad ADC on his chip
bad ADC on his chiiiiip

shrughes posted:

It is? Sorry, how?

It doesn't always return NULL when it can't actually allocate the amount of memory you requested.

# ? Nov 1, 2011 05:23

ShoulderDaemon: Oct 9, 2003; support goon fund; Taco Defender

Otto Skorzeny posted:

It doesn't always return NULL when it can't actually allocate the amount of memory you requested.

That's not really glibc's fault. Linux is optimistic when allocating pages to processes; any allocation will succeed as long as there's address space available, because what with swapping and processes constantly starting and stopping and all allocating memory at various times it's very hard to tell if there is "really" available memory.

And honestly, what else are you going to do? If some random malloc call in some random process returns NULL, you're just going to abort and terminate the program because essentially nobody seriously tries to handle that case. If it returns a valid pointer, then maybe by the time your program actually touches that page the kernel will be able to allocate it. Otherwise, the kernel will kill some random memory-intensive process, which is pretty much the same as if some random memory-intensive process had a failed malloc and decided to abort. There are even mechanisms provided to safeguard certain processes from being terminated, to nominate some processes as more expendable, and for processes to determine what proportion of the problem the kernel currently believes them to be, allowing them to manage caches and such in a system-aware manner if they're so capable.

For the very very few people who are building systems where running out of memory is a situation they can actually handle gracefully and who don't want any overcommit at all, it's easy to disable this behaviour and make Linux operate in a paranoid mode. These people will then have to deal with the large number of garbage-collected languages whose runtimes simply allocate an enormous heap right off the bat at program start and manage their own allocations within that space, never touching a majority of it (the JVM, in particular, is notorious for this). For the majority of people, overcommit is almost certainly the correct behaviour and shouldn't be disabled.

As far as programmers are concerned, it's probably best to imagine that memory allocation always succeeds, and if they are concerned about using too much memory, they should just occasionally ask the system if they are a high-weight process, and if so run a heap compression pass and return pages to the kernel. There's a reason that high level programming languages don't have a concept of "failed to allocate this object"; it's vastly preferable to handle "low available memory" at a broad scale, instead of trying to handle "zero available memory" at the scale of an individual allocation.

# ? Nov 1, 2011 06:44

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 12:58

shrughes: Oct 11, 2008; (call/cc call/cc)

Indeed, it's very sensible to have your own malloc linked in that never ever ever returns NULL and just terminates the process. The last thing you need is to explicitly handle allocation failures in your C program.

# ? Nov 1, 2011 18:54

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »