Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »

floWenoL: Oct 23, 2002

clay posted:

I have an 2d array set up like this in C:
code:
int *ary[1000];

for (i; i < 1000; ++i) {
    ary[i] = malloc(other_ary[i]);
}
And I want to set up the exact same thing using mmap. Any pointers on this?

What exactly are you trying to do? mmap is used to treat files as if they were loaded in memory; the closest thing would be to mmap a region that is backed by an anonymous file.

# ? May 4, 2008 23:12

Adbot: ADBOT LOVES YOU

# ? May 13, 2024 21:55

clay: Feb 4, 2003; brakes? unnecessary

floWenoL posted:

What exactly are you trying to do? mmap is used to treat files as if they were loaded in memory; the closest thing would be to mmap a region that is backed by an anonymous file.

I'm loading the netflix dataset to memory. Right now I have it load the entire dataset into memory each pass which is pretty silly and takes about 4 minutes. I want to mmap it all once so I can just load the mmaped file.

clay fucked around with this message at 23:53 on May 4, 2008

# ? May 4, 2008 23:39

poopiehead: Oct 6, 2004

clay posted:

I'm loading the netflix dataset to memory. Right now I have it load the entire dataset into memory each pass which is pretty silly and takes about 4 minutes. I want to mmap it all once so I can just load the mmaped file.

Are you trying to load multiple times in the same process or running multiple processes. If it's the former, just allocate the array as an int** and pass it around.

code:

int **ary = malloc(sizeof(int*)*1000); 
for (i; i < 1000; ++i) { 
  // this seems weird, does this array hold the size that you need to allocate 
  // or the numder of ints?
  ary[i] = malloc(other_ary[i]); 
}

If it's the latter, you probably should wait for someone that knows C much better than me to answer, but this would be how I'd try to get it into a bunch of memory. You need a block of memory big enough to hold the array of int* and the data that they point to. So find out how big it needs to be, allocate the space and build up your int*[]. I suspect this depends on assumptions that I'm not allowed to make though. Memory alignment probably gets in the way.....

code:

int size = sizeof(int*)*1000; 
for (i; i < 1000; ++i) { 
  size += other_ary[i];
}

int **ary = malloc(size); // or mmap equivalent
int * data = (int*)(ary + 1000);
for (i; i < 1000; ++i) { 
  ary[i] = data;
  data += other_ar[i];
}

edit: this is why I shouldn't attempt to answer C questions. fixed a bug in the second code block.

poopiehead fucked around with this message at 12:21 on May 5, 2008

# ? May 5, 2008 01:41

floWenoL: Oct 23, 2002

clay posted:

I'm loading the netflix dataset to memory. Right now I have it load the entire dataset into memory each pass which is pretty silly and takes about 4 minutes. I want to mmap it all once so I can just load the mmaped file.

I'm still not sure why loading in a file requires a 2-d array, but the canonical code to mmap a file for reading is:

code:

#include <fcntl.h> /* open() */
#include <sys/mman.h> /* mmap()/munmap() */
#include <sys/stat.h> /* stat() */
#include <unistd.h> /* close() */

...

int fdin;
struct stat buf;
char *bufin;

...

fdin = open("myfile", O_RDONLY);
fstat(fdin, &buf);
bufin = mmap(NULL, buf.st_size, PROT_READ, MAP_SHARED, fdin, 0);

...
munmap(bufin, buf.st_size);
close(fdin);

# ? May 5, 2008 09:14

notMordecai: Mar 4, 2007; Gay Boy Suicide Pact?
Sucking Dick For Satan??

edit: nvm wrong place to put this!

notMordecai fucked around with this message at 14:48 on May 5, 2008

# ? May 5, 2008 14:41

Crush: Jan 18, 2004; jot bought me this account, I now have to suck him off.

Does anyone know of a Greasemonkey script for YouTube that will:

* Not automatically start the video, but load it
* Have everything on the page expanded (More From:, More Info:, etc.)

I have tried looking around on Google for such a script and of course have looked at YouTube's settings and can not seem to find anything that does these things.

Thanks!

# ? May 5, 2008 22:52

graffin: Sep 28, 2002

I'm sure this has been covered before, but the search function is down for the forum, so I will ask.

I was looking into working on open source projects to get experience for my job search. Is there any good information out for those looking to get into open source projects? I'm looking for some basic entry level info.

# ? May 6, 2008 18:16

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

Well most open source programs are written in Linux so you should familiarize yourself with the build environment and tools. The languages vary but I'd say that the majority are written in C/C++.

Other than that you can get started by just checking out the source code and looking at their data structures. A lot of OS projects have developer wikis which can steer you in the right direction.

# ? May 6, 2008 18:21

wolf_man: Oct 5, 2005; Nunez?

This a javascript issue and hopefully someone can help me,

~~I have a page with a frame. . .~~

btw: The "programmer" who designed/created the peice of poo poo software/web interface I was put incharge to "patch" together to fix, was a complete moron (I just had to say that somewhere).

EDIT:

I was able to fix it, turns out it wasent a frame but a DIV with overflow set to scroll. Just needed to implemnenet: document.getElementById("div id").scrollTop=y;

I loath the man who orinigally created this garbled mess of code I have to fix.

wolf_man fucked around with this message at 14:44 on May 7, 2008

# ? May 6, 2008 20:37

Viper_3000: Apr 26, 2005; I could give a shit about all that.

I'm not exactly sure where to put this, and I'm hesitant to make a thread in SH/SC about it so I'm going to try here first.

I'm working with an excel spreadsheet that has to be formatted a certain way in order to integrate with our database, and thanks to the stupidness of another staff member of the library, I have a column of 40,000 items that need to be reformatted. Normally, if it's a short list I do it by hand but since it's so large I figured I'd ask here for help writing a script/macro/whatever it is I need.

So basically the column looks like this:

JA1304B37
B375F93
TAE236A32

And I need it to magically transform into this:

JA 1304 B37
B 375 F93
TAE 236 A32

So Basically inserting spaces after the first letters, and then in between the letters and numbers, but not between the last set. So if someone can point me in the right direction I'd be very appreciative.

# ? May 7, 2008 06:46

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

A regular expression can do this easily. You just need a text editor that lets you replace using a regular expression.

This will match the test and separate it into 3 tags.

code:

([A-Z]+)([0-9]+)([A-Z]*)

The [A-Z]+ will match at least the first letter and any subsequent letters and place them into the first tag. The [0-9]+ will match the first number after the letters and any additional numbers and place them into tag 2. The final part [A-Z]* will match the first letter after the last tag and anything else after that letter.

Then all you have to do is replace using the 3 tags. This is how you would do it in notepad++. Notice the spaces between the 3 classes.

code:

\1 \2 \3

I'm not sure if you can do this directly in Excel, but if you can copy that column in a text editor it should work.

Janitor Prime fucked around with this message at 08:12 on May 7, 2008

# ? May 7, 2008 08:02

Viper_3000: Apr 26, 2005; I could give a shit about all that.

MEAT TREAT posted:

A regular expression can do this easily. You just need a text editor that lets you replace using a regular expression.

This will match the test and separate it into 3 tags.
code:
([A-Z]+)([0-9]+)([A-Z]*)
The [A-Z]+ will match at least the first letter and any subsequent letters and place them into the first tag. The [0-9]+ will match the first number after the letters and any additional numbers and place them into tag 2. The final part [A-Z]* will match the first letter after the last tag and anything else after that letter.

Then all you have to do is replace using the 3 tags. This is how you would do it in notepad++. Notice the spaces between the 3 classes.
code:
\1 \2 \3
I'm not sure if you can do this directly in Excel, but if you can copy that column in a text editor it should work.

Ok, so I understand why and how this works, I just can't get it to work in notepad ++. I've hit ctl+r to bring up the F/R box, imputed this, and nothing seems to happens. It'll either say, not found, or 0 found depending on whether I pick find or count. Am I missing something obvious here?

# ? May 7, 2008 09:43

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

Yes you need to switch to the replace tab and then you push the replace all button.

# ? May 7, 2008 11:17

graffin: Sep 28, 2002

Does anybody have experience doing open source work with freshmeat.net? I'm looking for some experience, and I am looking to find a good site to do it.

# ? May 7, 2008 16:53

CrusaderSean: Jun 10, 2002; AwakenedDreamer

This is more of an algorithm question than programming.. but is there a more efficient random permutation/shuffling algorithm than Fisher-Yates shuffle? That's what I'm using currently to shuffle >200K entries in a table.

# ? May 7, 2008 17:22

tef: May 30, 2004; -> some l-system crap ->

It is an in-place O(n) shuffle when implemented correctly, so finding something faster might be a challenge.

Why are you shuffling the table?

# ? May 7, 2008 17:32

graffin: Sep 28, 2002

CrusaderSean posted:

This is more of an algorithm question than programming.. but is there a more efficient random permutation/shuffling algorithm than Fisher-Yates shuffle? That's what I'm using currently to shuffle >200K entries in a table.

It looks like that algorithm works in O(n) time. If I'm right, I can't imagine a shiffle algorithm that would work faster. Of course, I'm new to this so I could be wrong.

edit: beaten by above post

# ? May 7, 2008 17:36

CrusaderSean: Jun 10, 2002; AwakenedDreamer

tef posted:

It is an in-place O(n) shuffle when implemented correctly, so finding something faster might be a challenge.

Why are you shuffling the table?

Whoops, didn't see it was O(n). I'm not really shuffling a table. I have a large array of strings and I'm permuting the order of each string.

# ? May 7, 2008 19:40

darfur: Nov 1, 2003

I've searched the web for this, but can't really find a good answer.

What good is an interface? I understand what one is, and how I would implement one, but I don't understand the benefit of using one instead of just writing the classes that would use it by themselves without using the interface. It seems like if your class "inherits" and interface, it must include all of the methods in the interface, but why not just write the class with all the methods anyway?

Can anyone clarify this for me?

darfur fucked around with this message at 19:48 on May 7, 2008

# ? May 7, 2008 19:46

csammis: Aug 26, 2003; Mental Institution

darfur posted:

I've searched the web for this, but can't really find a good answer.

What good is an interface? I understand what one is, and how I would implement one, but I don't understand the benefit of using one instead of just writing the classes that would use it by themselves without using the interface. It seems like if your class "inherits" and interface, it must include all of the methods in the interface, but why not just write the class with all the methods anyway?

Can anyone clarify this for me?

Hiding methods that consumers don't need to know about, for one. Here's a quick example of a really good reason to use an interface:

code:

public class SomeData
{
  public void setData(String data);
  public String getData();
}

But what if you don't want your consumers to set the data, because it's something done by your business logic only? Changing the visibility of the setData method won't cut it, because you still want to be able to set it, so you create an interface that only exposes the getData method:

code:

public interface IData
{
  public String getData();
}

public class SomeData implements IData
{
  public void setData(String data);
  public String getData();
}

Give your consumers IData objects instead of SomeData objects, and bam, safe encapsulation.

Another good reason is being able to change implementations under the hood without the consumer having to be aware of it. Let's say you have a connection class. Could be a connection to a database, could be a connection to a network socket, could be a connection to some dank nugz from the kid your parents warn you away from all the time, but all your consumer cares about is that it's a connection to some resource. If you give them a DatabaseConnection, they have to write new code when you decide that databases are stupid and you'd rather have them use a NetworkConnection. If these connections implement a common interface, however, the consumer won't have to change anything in order to get your data from a new source.

edit: continued, more formally and less contrived:

Interfaces are contracts: they guarantee that a class has methods X, Y, and Z. It doesn't specify how that contract will be fulfilled, that's the job of the class that implements the interface. Interfaces allow a programmer to separate the design (the interface) from the implementation (the class).

csammis fucked around with this message at 20:21 on May 7, 2008

# ? May 7, 2008 20:17

Incoherence: May 22, 2004; POYO AND TEAR

darfur posted:

I've searched the web for this, but can't really find a good answer.

What good is an interface? I understand what one is, and how I would implement one, but I don't understand the benefit of using one instead of just writing the classes that would use it by themselves without using the interface. It seems like if your class "inherits" and interface, it must include all of the methods in the interface, but why not just write the class with all the methods anyway?

Can anyone clarify this for me?

An interface allows multiple implementations of the same methods; any member of an interface can be used as a variable of the interface type.

Consider the Java List interface: http://java.sun.com/j2se/1.5.0/docs/api/java/util/List.html
You can see that among the implementations give there are ArrayList, LinkedList, and Vector. So anything that takes a List, and uses the List methods, can be called on any of these implementations. From a consumer's point of view, if they just want to get some data, the fact that ArrayList.get() and LinkedList.get() behave entirely differently under the hood is irrelevant.

Another example: event handlers. All event handlers for a given event implement the same interface, but they may (and hopefully do) have different behavior, and you may have multiple different event handlers for a particular event.

Incoherence fucked around with this message at 20:22 on May 7, 2008

# ? May 7, 2008 20:20

csammis: Aug 26, 2003; Mental Institution

Incoherence posted:

Another example: event handlers. All event handlers for a given event implement the same interface, but they may (and hopefully do) have different behavior, and you may have multiple different event handlers for a particular event.

Huh? Event handlers (callbacks, whatever you call them in your language) are methods, not interfaces. The fact that you can have multiple event handlers for a single event has zilch to do with interfaces, it has to do with how events are implemented in your architecture.

# ? May 7, 2008 20:23

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

Interfaces allow you to have more freedom than simple inheritance while at the same time giving you all the benefits like polymorphism.

I'll give you a good example of one I did recently. I was writing a B+ tree data structure. If you've ever studied the B+ tree you know that it has 2 different kinds of nodes. It has index nodes which are really just 2 arrays with one being an array of pointers to other nodes and it has the leaf nodes which is where the data is actually stored. Now both these classes share a lot of methods like insert, search and delete but each one has to implement it differently since they have different internal structures. So I created a Node interface with the above methods which these two classes implement. Now I can pass around Node objects without having to worry if they are leaf nodes or index nodes*.

Now I could have used normal inheritance with abstract functions but then if I ever wanted to inherit from another class I couldn't.

e: csammis where those methods come from? You have to inherit them from some place and in languages that don't allow multiple inheritance you have to use interfaces so that you can have multiple even handlers for one object. I think Incoherence said it wrong, I'm sure he meant that one object can have multiple even't handlers like keyboard and mouse, not that many objects can share the keyboard event handler. Because in that case you are correct, interfaces have nothing to do with it.

Janitor Prime fucked around with this message at 20:43 on May 7, 2008

# ? May 7, 2008 20:27

Anode: Dec 24, 2005; Nail me to my car and I'll tell you who you are

I've just started using SVN for a personal project a few weeks ago and made a branch to add a couple of features. When I try to merge my branch back to the trunk (using the dry run option), it skips all the files I added even though they're versioned in the branch. The SVN docs/book are no help, any ideas?

EDIT: Nevermind, apparently "From" and "To" mean "To" and "From" in SVN lingo.

Anode fucked around with this message at 21:16 on May 7, 2008

# ? May 7, 2008 21:10

csammis: Aug 26, 2003; Mental Institution

MEAT TREAT posted:

e: csammis where those methods come from? You have to inherit them from some place and in languages that don't allow multiple inheritance you have to use interfaces so that you can have multiple even handlers for one object. I think Incoherence said it wrong, I'm sure he meant that one object can have multiple even't handlers like keyboard and mouse, not that many objects can share the keyboard event handler. Because in that case you are correct, interfaces have nothing to do with it.

I know what Incoherence meant about one event having multiple handlers, but I'm not sure where you're coming from. Inheriting a method? :psyduck:

Where are you inheriting the method from when you use a delegate in C#? How about registering a method callback in C++? Event handling is a concept that has absolutely nothing to do with interfaces. Interfaces only come into it as an implementation detail of Java's Listener pattern.

# ? May 7, 2008 21:31

floWenoL: Oct 23, 2002

CrusaderSean posted:

Whoops, didn't see it was O(n). I'm not really shuffling a table. I have a large array of strings and I'm permuting the order of each string.

Not sure how you're storing the strings, but if it is by value you'd probably get a huge speed up by shuffling references/pointers to the strings instead.

# ? May 7, 2008 21:53

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

csammis posted:

I know what Incoherence meant about one event having multiple handlers, but I'm not sure where you're coming from. Inheriting a method? Where are you inheriting the method from when you use a delegate in C#? How about registering a method callback in C++? Event handling is a concept that has absolutely nothing to do with interfaces. Interfaces only come into it as an implementation detail of Java's Listener pattern.

Well that shows how much I know about either of those languages. :downs:

# ? May 7, 2008 21:54

Incoherence: May 22, 2004; POYO AND TEAR

csammis posted:

Huh? Event handlers (callbacks, whatever you call them in your language) are methods, not interfaces. The fact that you can have multiple event handlers for a single event has zilch to do with interfaces, it has to do with how events are implemented in your architecture.

Well, yes, that particular implementation is Java's. Conceptually, a C# delegate is equivalent to (but much less wordy than) a single-method interface.

I was trying to think of an example where you would have multiple implementations of an interface and want to hold them all in a list and run them all in sequence, without using the one on the project I work on (I can't figure out a good way to generalize/anonymize it).

# ? May 7, 2008 23:08

haveblue: Aug 15, 2005; Toilet Rascal

Incoherence posted:

I was trying to think of an example where you would have multiple implementations of an interface and want to hold them all in a list and run them all in sequence, without using the one on the project I work on (I can't figure out a good way to generalize/anonymize it).

There's a concept in Cocoa/ObjC called class clusters, in which a type has a bunch of private subtypes which are implemented independently and never exposed to the user. That page has a more complete and in-depth justification than any forum post could.

# ? May 7, 2008 23:12

darfur: Nov 1, 2003

Awesome replies! They make a lot of sense to me, at least I think.

In MEAT TREAT's tree, for the insert function, would it just take an INode as the parameter, and then you could pass it any class which inherits the INode interface and that class' specific implementation of insert would be what gets called? That is my understanding as to how an interface would work there, is this correct? Would you have to make sure you don't link different types of INodes together or is that what this enables you to do?

darfur fucked around with this message at 09:54 on May 8, 2008

# ? May 8, 2008 09:50

Rohaq: Aug 11, 2006

So I have a structured array in C++, compiling for Windows under VS2005, and I'd like to be able to save it and load it to a file at will. I hear that there's a way to essentially save/load the entire thing to and from memory in one go somehow, instead of doing retarded looping, but I can't see much info on it. Can anybody suggest as to how I would go about this?

# ? May 8, 2008 23:33

Incoherence: May 22, 2004; POYO AND TEAR

darfur posted:

In MEAT TREAT's tree, for the insert function, would it just take an INode as the parameter, and then you could pass it any class which inherits the INode interface and that class' specific implementation of insert would be what gets called? That is my understanding as to how an interface would work there, is this correct? Would you have to make sure you don't link different types of INodes together or is that what this enables you to do?

You would have your other methods take an INode parameter. If you have INode n, and you call n.insert(), it will call the insert() method corresponding to the actual implementation. You could even have a list of INodes of different types and calling insert() on all of them in sequence will do the right thing.

(Insert yammering about the C++ virtual keyword here, but if you're talking about interfaces you assume the methods are virtual. Also insert yammering about how this is actually an instance of the Composite Design Pattern, and that if you're in academia there are all sorts of neat things you can add to this like the Visitor Pattern.)

# ? May 9, 2008 00:36

peak debt: Mar 11, 2001; b& :(; Nap Ghost

Did Microsoft change anything about how Visual Studio handles exceptions with version 2008 64bit? If you had an uncaught exception in 2005 and had run the application in the debugger, you used to get a popup with the exception and the debugger automatically opened on the offending line.

But I can't seem to get the same working in 2008. Even a completely new project whose only line is "throw new Exception()" simply throws the exception then ignores it.

# ? May 11, 2008 16:40

mdemone: Mar 14, 2001

Hi guys. I am bad at shell scripting. Please help me.

I have a set of data files I want to repeatedly run a program on, with both input and output files marked by padded integers, i.e. they go from 00000 in increments of 20 all the way up to 02000 (or whatever min/max).

So I need a little loop that uses the padded integers:

./program < input.00000 > 00000.out
./program < input.00020 > 00020.out
...
etc. (the < and > characters denote input and output and are actually necessary in the program's call line)

Can anybody help me out real quick? You'll probably save me a couple hours of looking up how to do this...thanks in advance!

# ? May 13, 2008 01:27

poopiehead: Oct 6, 2004

heres a really lovely way that would probably work. But I know for sure there's a much easier way to do it, Just don't know linux well enough. the syntax is close but definitely wrong, but until someone answers for real, that should work

code:

for i in 1 to 100000
  if [ -f input.000$i] 
  then
    ./program < input.000$i > output.000$i
  fi
  if [ -f input.00$i] 
  then
    ./program < input.00$i > output.00$i
  fi
  if [ -f input.0$i] 
  then
    ./program < input.0$i > output.0$i
  fi
  if [ -f input.$i] 
  then
    ./program < input.$i > output.$i
  fi
next

another option would be to iterate all the files that ls returns. again pseudo shell.

code:

for f in `ls input.*`
  output=`echo $f | sed -e "s/input/output/g"`
  ./program <$f >$output 
next

poopiehead fucked around with this message at 02:02 on May 13, 2008

# ? May 13, 2008 01:55

10011: Jul 22, 2007

The first line of the loop is for i in `seq 0 1000000`, and you need spaces before the ]s (also, you might want to use elif instead if fi/if). Otherwise, what poopiehead said.

Edit: I can't read.

Edit 2: vv Use `seq 0 20 1000000`.

10011 fucked around with this message at 02:07 on May 13, 2008

# ? May 13, 2008 02:03

mdemone: Mar 14, 2001

That looks just fine, but how do I make the step-size 20 in the for call?

Edit: Thanks guys!

~~Edit 2: Not so fast. It says there is an unexpected token near the if statement. I don't know what that means.~~

~~Edit 3: I fixed it! Thanks again!~~

Edit 4: I didn't fix it.

Still have bad syntax near the unexpected token if. Waaah...

mdemone fucked around with this message at 20:13 on May 13, 2008

# ? May 13, 2008 02:03

Moof Strydar: Jul 13, 2001

How would I make a piano-roll style GUI, like for use with MIDI?

I could probably get as far as drawing some lines in OpenGL, but I don't really know how to proceed from there, or how to make a piano roll that fits in with an existing GUI framework. Are there any references out there on writing custom GUIs/widgets or whatever? Preferably in Python (a long shot, I know)?

# ? May 13, 2008 13:08

JoeNotCharles: Mar 3, 2005; Yet beyond each tree there are only more trees.

Moof Strydar posted:

I could probably get as far as drawing some lines in OpenGL, but I don't really know how to proceed from there, or how to make a piano roll that fits in with an existing GUI framework. Are there any references out there on writing custom GUIs/widgets or whatever? Preferably in Python (a long shot, I know)?

Absolutely - both wxPython and PyQT are fantastic GUI frameworks (I prefer PyQT). Google will find you lots of good stuff.

# ? May 13, 2008 18:19

Adbot: ADBOT LOVES YOU

# ? May 13, 2024 21:55

Acer Pilot: Feb 17, 2007; put the 'the' in therapist

Do we have an Actionscript thread? My head is going to explode trying to sort out that language.

# ? May 14, 2008 04:09

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »