Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

Would anyone have any idea why a SSH key that works fine with a general SFTP tool like PuTTy or WinSCP would give a "packet too short" error on another application? It's a proprietary software tool and it's driving me nuts because the same tool works with another SFTP server/key just fine using the same process.

# ? Jul 31, 2019 00:50

Adbot: ADBOT LOVES YOU

# ? May 30, 2024 23:58

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

PierreTheMime posted:

Would anyone have any idea why a SSH key that works fine with a general SFTP tool like PuTTy or WinSCP would give a "packet too short" error on another application? It's a proprietary software tool and it's driving me nuts because the same tool works with another SFTP server/key just fine using the same process.

What format is the key in? If I remember correctly, PuTTy uses ppk format, but pem file would be the standard for a linux tool. Don't remember what WinSCP uses off hand. Could also be permissions, ssh command on linux will throw an error unless your private key is 0600.

# ? Jul 31, 2019 01:02

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

fletcher posted:

What format is the key in? If I remember correctly, PuTTy uses ppk format, but pem file would be the standard for a linux tool. Don't remember what WinSCP uses off hand. Could also be permissions, ssh command on linux will throw an error unless your private key is 0600.

It�s a .pem that was converted to .ppk (WinSCP uses PuTTy to convert it). It�s unencrypted with full control on a Windows file system.

# ? Jul 31, 2019 01:21

leper khan: Dec 28, 2010; Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

Pollyanna posted:

We�re looking to replace Mongo with DocumentDB. As part of this, we want to do some performance testing against an endpoint backed alternately by Mongo or DocDB. We can do this by continually generating faked data and sending a request to the endpoint over and over.

We want a structured and easily replicable performance testing plan, such that we can get a good idea of our expected performance if/when we cut over. What tools are out there that can do this, and are recommended?

http://tsung.erlang-projects.org will press things to the limits.

# ? Jul 31, 2019 13:12

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

PierreTheMime posted:

It�s a .pem that was converted to .ppk (WinSCP uses PuTTy to convert it). It�s unencrypted with full control on a Windows file system.

Maybe see if there's a new line at the end and try adding one if there isn't?

Heck, maybe it expects different newlines than PuTTy uses?

Munkeymon fucked around with this message at 14:14 on Jul 31, 2019

# ? Jul 31, 2019 13:59

Janitor Prime: Jan 22, 2004; PC LOAD LETTER

What da fuck does that mean; Fun Shoe

PierreTheMime posted:

Would anyone have any idea why a SSH key that works fine with a general SFTP tool like PuTTy or WinSCP would give a "packet too short" error on another application? It's a proprietary software tool and it's driving me nuts because the same tool works with another SFTP server/key just fine using the same process.

Maybe the key size is too big for the app or it's using a different SSH key-type like ECDSA

# ? Jul 31, 2019 15:12

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

The same process takes another .ppk in the exact same format, spacing, etc. just fine. The only difference I can see is that the private key is 10 characters shorter, but is the same number of linse and that same key works if fed to WinSCP so :shrug:

. I've got a ticket open with the vendor about it, and in the meanwhile had to script a super round-about way using PowerShell and OpenSSH batch calls.

# ? Jul 31, 2019 16:49

FredMSloniker: Jan 2, 2008; Why, yes, I do like Kirby games.

I'm looking to do some procedural generation, and I need a function that will take an integer in a given range (0-65535 for now, but if the function can deal with arbitrary powers of two, that'd be great) and map it to an integer in the same range. The mapping should have the following properties:

The mapping is 1:1; each integer in the new range has a corresponding integer in the old range.
The mapping should appear random. For instance, y = (x + prime) % range won't do.
The function should be inexpensive, both in terms of execution time and in terms of memory. I can't, for instance, shuffle a 64k table. (Given this, the previous property isn't terribly strict, but the player will be able to enter the input number, and I don't want them to see obvious patterns in the output.)

I'm going to be implementing this in a few different ways, so pseudocode is fine; I'll also accept a Lua function. Can you help me out?

FredMSloniker fucked around with this message at 22:07 on Aug 4, 2019

# ? Aug 4, 2019 17:27

Xerophyte: Mar 17, 2008; This space intentionally left blank

~~If it's just 65k you could always make an array of all the integers and shuffle it. This doesn't generalize all that well.~~ Oh hey, you said you couldn't do this and I apparently chose not to read that.

For larger sets you're basically looking for an RNG where the period is equal to the state size, i.e. the RNG will enter each state. This property is called being full cycle.

Extremely basic LCGs on the type of state = (state + prime) % period are full cycle if the prime and period are coprime.
The other major category of RNGs that can be -- but aren't always -- known to be full cycle are LFSRs. I recommend Xorshift, which has known full cycle versions for 32, 64 and 128 bit state and is known to have a not-awful distribution, if less good than a more complex RNG.

E: I should also mention that the way you use a full cycle RNG as a function that maps integers to "random" intergers in the same range is to use the input integer as the state.

E2: You can additionally construct full cycle xorshift versions for any arbitrary power of 2 as long as you use bit rotations instead of bit shifts which ensures that the new state is guaranteed unique from the input. The paper found ones with shifts instead on the grounds that they were faster.

Xerophyte fucked around with this message at 20:47 on Aug 4, 2019

# ? Aug 4, 2019 20:32

leper khan: Dec 28, 2010; Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

FredMSloniker posted:

I'm looking to do some procedural generation, and I need a function that will take an integer in a given range (0-65535 for now, but if the function can deal with arbitrary powers of two, that'd be great) and map it to an integer in the same range. The mapping should have the following properties:

The mapping is 1:1; each integer in the new range has a corresponding integer in the old range.

The mapping should appear random. For instance, y = (x + prime) % range won't do.

The function should be inexpensive, both in terms of execution time and in terms of memory. I can't, for instance, shuffle a 64k table. (Given this, the previous property isn't terribly strict, but the player will be able to enter the input number, and I don't want them to see obvious patterns in the output.)

I'm going to be implementing this in a few different ways, so pseudocode is fine; I'll also accept a Lua function. Can you help me out?

https://en.m.wikipedia.org/wiki/Xorshift

# ? Aug 4, 2019 22:16

FredMSloniker: Jan 2, 2008; Why, yes, I do like Kirby games.

Xerophyte posted:

Xorshift

That looks like just what I wanted. Thanks! Now to find the right shift constants to get a full-cycle result for my desired state size. (I've got a program testing various ones.)

# ? Aug 4, 2019 23:57

taqueso: Mar 8, 2004

You might check out LFSR counters if you didn't end up seeing it already.

# ? Aug 5, 2019 00:06

fritz: Jul 26, 2003

FredMSloniker posted:

I'm looking to do some procedural generation, and I need a function that will take an integer in a given range (0-65535 for now, but if the function can deal with arbitrary powers of two, that'd be great) and map it to an integer in the same range. The mapping should have the following properties:

The mapping is 1:1; each integer in the new range has a corresponding integer in the old range.

The mapping should appear random. For instance, y = (x + prime) % range won't do.

The function should be inexpensive, both in terms of execution time and in terms of memory. I can't, for instance, shuffle a 64k table. (Given this, the previous property isn't terribly strict, but the player will be able to enter the input number, and I don't want them to see obvious patterns in the output.)

I'm going to be implementing this in a few different ways, so pseudocode is fine; I'll also accept a Lua function. Can you help me out?

Just out of curiosity, what kind of weird constraints are you under that you can't afford 128kb of table?

# ? Aug 5, 2019 00:29

FredMSloniker: Jan 2, 2008; Why, yes, I do like Kirby games.

fritz posted:

Just out of curiosity, what kind of weird constraints are you under that you can't afford 128kb of table?

Developing for a Commodore 64.

# ? Aug 5, 2019 00:40

ultrafilter: Aug 23, 2007; It's okay if you have any questions.

Can you save state across function calls?

# ? Aug 5, 2019 00:57

FredMSloniker: Jan 2, 2008; Why, yes, I do like Kirby games.

ultrafilter posted:

Can you save state across function calls?

Yes, but not 64k of state. Does that help somehow?

# ? Aug 5, 2019 05:40

Boris Galerkin: Dec 17, 2011; I don't understand why I can't harass people online. Seriously, somebody please explain why I shouldn't be allowed to stalk others on social media!

There's a person's personal project on github that was last committed in 2018 and has an attached MIT license.

I forked it and modified some of the code, and then ended up rewriting most of the code in a different language. Before I push my changes back to my github repo what do I need to change/do to keep all the licensing stuff kosher? Do I need to rename the project/repo? Do I just change the copyright year/name in the LICENSE.txt file to reflect the current year/name or do I need to keep the original copy as-is and add my own?

# ? Aug 5, 2019 11:14

leper khan: Dec 28, 2010; Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

Boris Galerkin posted:

There's a person's personal project on github that was last committed in 2018 and has an attached MIT license.

I forked it and modified some of the code, and then ended up rewriting most of the code in a different language. Before I push my changes back to my github repo what do I need to change/do to keep all the licensing stuff kosher? Do I need to rename the project/repo? Do I just change the copyright year/name in the LICENSE.txt file to reflect the current year/name or do I need to keep the original copy as-is and add my own?

The thing I normally see is the single copyright year/name line for edits listed above the original.

IANAL so no idea on the correctness of that. It�s probably fine as the original license remains below your tag. :shrug:

# ? Aug 5, 2019 11:21

ultrafilter: Aug 23, 2007; It's okay if you have any questions.

FredMSloniker posted:

Yes, but not 64k of state. Does that help somehow?

That lets you implement pretty much any standard PRNG, so yeah, there's probably something you can do with it.

# ? Aug 5, 2019 14:54

TheKingofSprings: Oct 9, 2012

E: Ended up solving my question about an hour or so after I asked it, removing it!

TheKingofSprings fucked around with this message at 22:48 on Aug 6, 2019

# ? Aug 6, 2019 20:20

AgentCow007: May 20, 2004; TITLE TEXT

So I am in my last year of my CompSci degree and am realizing I don't know poo poo about poo poo. I had an idea for an on-demand, drop-shipping business, kind of like a Teespring... but all I've learned is using some Python, Java, and C on a local machine. I have no idea about web frameworks, "front end", "back end", etc. What do I need to Google to learn enough to start prototyping something like this? I've found a lot of Youtube videos sperging out over what the new hotness.JS is, but I really need some fundamentals.

# ? Aug 7, 2019 00:16

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

AgentCow007 posted:

So I am in my last year of my CompSci degree and am realizing I don't know poo poo about poo poo. I had an idea for an on-demand, drop-shipping business, kind of like a Teespring... but all I've learned is using some Python, Java, and C on a local machine. I have no idea about web frameworks, "front end", "back end", etc. What do I need to Google to learn enough to start prototyping something like this? I've found a lot of Youtube videos sperging out over what the new hotness.JS is, but I really need some fundamentals.

I give this advice a lot because it helped me back in the day, so maybe it'll help you:

Write a web server in Python or whatever. It'll really help you understand a lot. Don't actually use the web server you write. Throw it away and then use Django or whatever. You'll be more able to evaluate stuff after you've done this.

When someone gave me this advice a long time ago it sounded like someone told me to write some insane thing like...I dunno...a whole operating system or something. But, actually a web server is pretty simple and there's lots of tutorials out there.

And some more info:

Web backends are basically big functions that taking a request from the browser, using that to build a big string of HTML/CSS/JS, and then sending it to the browser as a response. Web frontends are about running that JS in the browser to manipulate the HTML and CSS to build interactive applications. (or, less and less commonly, the JS does very little and every "screen" of the web site is a request<>response with the web backend which does the manipulation of the HTML/CSS to change what you see)

# ? Aug 7, 2019 01:07

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

Yeah, the best way to broaden your experience is to build a "real" and reasonably complete product or tool of some kind. Odds are excellent that you'll get halfway in, discover major structural issues, and have to scrap it and start over, but since your primary goal is education, that shouldn't be a huge problem.

# ? Aug 7, 2019 01:39

leper khan: Dec 28, 2010; Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

Thermopyle posted:

I give this advice a lot because it helped me back in the day, so maybe it'll help you:

Write a web server in Python or whatever. It'll really help you understand a lot. Don't actually use the web server you write. Throw it away and then use Django or whatever. You'll be more able to evaluate stuff after you've done this.

When someone gave me this advice a long time ago it sounded like someone told me to write some insane thing like...I dunno...a whole operating system or something. But, actually a web server is pretty simple and there's lots of tutorials out there.

And some more info:

Web backends are basically big functions that taking a request from the browser, using that to build a big string of HTML/CSS/JS, and then sending it to the browser as a response. Web frontends are about running that JS in the browser to manipulate the HTML and CSS to build interactive applications. (or, less and less commonly, the JS does very little and every "screen" of the web site is a request<>response with the web backend which does the manipulation of the HTML/CSS to change what you see)

Write an operating system. Simple ones aren�t that hard; undergraduates can get things running within a single semester.

Much like the web server, never use it for anything other than running the half dozen toy programs you decide to write.

# ? Aug 7, 2019 02:53

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

Yeah, I wasn't clear on what I meant by "whole", but I meant like a full OS with all the trappings...a production-ready thingamajig. AKA, not the equivalent of writing a toy web server.

Like, it's quite doable to write a 3d renderer, it's another thing to create Unreal engine.

# ? Aug 7, 2019 03:06

FreshFeesh: Jun 3, 2007; Drum Solo

Currently I have a program which outputs XML in the following wonderful format:

code:

<?xml version="1.0" encoding="utf-8" ?><Reports>
<Detail>
  <_x003C_b_x003E_CLIENT-NAME-A_x003C__x002F_b_x003E_>
    <ShortDateFormat:Date xmlns:ShortDateFormat="datetime">7/15/2019</ShortDateFormat:Date>
    <ShortTimeFormat:Start_x0040_Time xmlns:ShortTimeFormat="datetime">12:55 PM</ShortTimeFormat:Start_x0040_Time>
    <ShortTimeFormat:End_x0040_Time xmlns:ShortTimeFormat="datetime">1:45 PM</ShortTimeFormat:End_x0040_Time>
    <Charge_x0020_To_x002F_Project_x002F_Service>Ticket 856667</Charge_x0020_To_x002F_Project_x002F_Service>
 </_x003C_b_x003E_CLIENT-NAME-A_x003C__x002F_b_x003E_>
  <_x003C_b_x003E_CLIENT-NAME-A_x003C__x002F_b_x003E_>
    <ShortDateFormat:Date xmlns:ShortDateFormat="datetime">7/29/2019</ShortDateFormat:Date>
    <ShortTimeFormat:Start_x0040_Time xmlns:ShortTimeFormat="datetime">12:30 PM</ShortTimeFormat:Start_x0040_Time>
    <ShortTimeFormat:End_x0040_Time xmlns:ShortTimeFormat="datetime">2:45 PM</ShortTimeFormat:End_x0040_Time>
    <Charge_x0020_To_x002F_Project_x002F_Service>Ticket 859511</Charge_x0020_To_x002F_Project_x002F_Service>
</Detail>
</Reports>

My aim is to extract the client name and other fields for presentation in a much more useful output. Currently I'm using perl with XML::LibXML but I am also open to other suggestions that will be workable on Windows. The difficulties I'm having are that 1) the CLIENT-NAME is not an id but rather the name of an entire node (i.e. not unique across entries), and 2) I can't figure out how to correctly extract the date or time.

Here is my quick code to iterate through all records and grab the CLIENT-NAME and service date:

code:

for my $node ($dom->findnodes('//Reports/Detail/*[text()]')) {
	my $client = $node->nodeName();
	my $date = $node->find('ShortDateFormat:Date');
	say $client -- $date;
}

While nodeName() correctly returns the name, any combination I try to extract ShortDateFormat:Date fails, I'm assuming because I'm not in the right context for it to know what information to look for? Using the code above yields an "Undefined namespace prefix" XPath error, which I understand, but leaving off ":Date" or altering it in any other way yields a blank response (just as if I put in a knowingly missing find).

Given the structure of this data, is there a good/better way for me to grab this information? My end goal is to pump out information in a CSV

Bonus Question: Is there an easy way to clean up the special character codes (_x003C_ and the like) without coding up a number of manual regular expressions?

# ? Aug 7, 2019 23:18

Scaramouche: Mar 26, 2001; SPACE FACE! SPACE FACE!

Don't know if its necessarily better than your current approach but the last time I had to do something like that I imported it all into an mssql express instance and used xml path/for xml/toxml to do some heavy lifting.

# ? Aug 8, 2019 00:32

redleader: Aug 18, 2005; Engage according to operational parameters

My recommendation would be to find the person who created that XML, and beat them senseless with the nearest fire extinguisher.

Have you tried registering the ShortDateFormat namespace in an XPathContext, a la this?

# ? Aug 8, 2019 02:18

FreshFeesh: Jun 3, 2007; Drum Solo

Thank you both. Believe me, if I had the power to make some changes, XML formatting would be the least of them. I'll see what I can wrangle up with those suggestions and see if something fits the bill. The use case here is for an end-user to get the raw XML report, run my translator program, and then make use of the resulting CSV. It's far from a great system.

# ? Aug 8, 2019 16:06

nielsm: Jun 1, 2009

redleader posted:

My recommendation would be to find the person who created that XML, and beat them senseless with the nearest fire extinguisher.

Have you tried registering the ShortDateFormat namespace in an XPathContext, a la this?

Wholeheartedly agree on both points.

Probably something like this:

Perl code:

my $dom = ...;
my $xpc = XML::LibXML::XPathContext->new($dom);
$xpc->registerNs('dt',  'datetime');

$xpc->findnodes('//Reports/Detail/*/dt:Date');

# ? Aug 8, 2019 17:19

Gyshall: Feb 24, 2009; Had a couple of drinks.
Saw a couple of things.

Hey goons. Can anyone recommend me some web framework for sorting/managing table data? My use case is fantasy football rankings, mostly for personal use.

Hopefully the right place to ask. Thanks!

# ? Aug 8, 2019 21:50

fletcher: Jun 27, 2003; ken park is my favorite movie; Cybernetic Crumb

Gyshall posted:

Hey goons. Can anyone recommend me some web framework for sorting/managing table data? My use case is fantasy football rankings, mostly for personal use.

Hopefully the right place to ask. Thanks!

Google Sheets? What is it that you are trying to do? How come you need a web framework for it?

If you want to get up and running very quickly with a web based CRUD app, Django all the way! PyCharm + Django = Fun

# ? Aug 8, 2019 23:23

necrotic: Aug 2, 2005; I owe my brother big time for this!

Toss datatables on a table and boom. https://datatables.net/

# ? Aug 8, 2019 23:28

Pollyanna: Mar 5, 2005; Milk's on them.

What exactly are .map files used for, and how can I process them/link them?

Twilight Princess has some .map files that supposedly match up to where functions and objects and such are located in memory (which I need for a project I'm working on), and they look like this as plaintext:

code:

.text section layout
  00000000 000068 800056c0  4 version_check__Fv 	m_Do_main.o 
  00000068 000064 80005728  4 CheckHeap1__9HeapCheckFv 	m_Do_main.o 
  000000cc 0000bc 8000578c  4 CheckHeap__FUl 	m_Do_main.o 
  00000188 000058 80005848  4 countUsed__FP10JKRExpHeap 	m_Do_main.o 
  000001e0 000024 800058a0  4 getUsedCount__9HeapCheckCFv 	m_Do_main.o 
  00000204 000214 800058c4  4 heapDisplay__9HeapCheckCFv 	m_Do_main.o 
  00000418 000274 80005ad8  4 debugDisplay__Fv 	m_Do_main.o 
  0000068c 000400 80005d4c  4 Debug_console__FUl 	m_Do_main.o 
  00000a8c 00007c 8000614c  4 LOAD_COPYDATE__FPv 	m_Do_main.o 
  00000b08 0000c4 800061c8  4 debug__Fv 	m_Do_main.o 
  00000bcc 0001c8 8000628c  4 main01__Fv 	m_Do_main.o 
  00000d94 000184 80006454  4 main 	m_Do_main.o 
  00000f18 000008 800065d8  4 dump_sort__7JKRHeapFv 	m_Do_main.o 
  00000f20 0001b8 800065e0  4 __sinit_m_Do_main_cpp 	m_Do_main.o 
  000010d8 000030 80006798  4 OSSwitchFiberEx__FUlUlUlUlUlUl 	m_Do_printf.o 
  ...

code:

.text section layout
  00000000 00002c 00000000  4 _prolog 	executor.o 
  0000002c 00002c 0000002c  4 _epilog 	executor.o 
  00000058 000020 00000058  4 _unresolved 	executor.o 
  00000078 00001c 00000078  4 __register_global_object 	global_destructor_chain.o 
  00000094 000058 00000094  4 __destroy_global_chain 	global_destructor_chain.o 
  000000ec 000440 000000ec  4 setReinPos__13daHorseRein_cFi 	d_a_horse.o 
...
  0000d570 00004c 0000d570  4 __ct__16daPy_frameCtrl_cFv 	d_a_horse.o 
  0000d5bc 00003c 0000d5bc  4 __dt__19mDoExt_AnmRatioPackFv 	d_a_horse.o 
  0000d5f8 000018 0000d5f8  4 __ct__19mDoExt_AnmRatioPackFv 	d_a_horse.o 
  0000d610 000048 0000d610  4 __dt__12J3DFrameCtrlFv 	d_a_horse.o 
.ctors section layout
  00000000 000000 00000000  4 _ctors 	Linker Generated Symbol File 
  00000004 000004 00000004  4 _ctors$99 	Linker Generated Symbol File 
.dtors section layout
  00000000 000000 00000000  4 _dtors 	Linker Generated Symbol File 
  00000000 000004 00000000  4 __destroy_global_chain_reference 	global_destructor_chain.o 
  00000004 000004 00000004  4 __destroy_global_chain_reference 	Linker Generated Symbol File 
  00000008 000004 00000008  4 _dtors$99 	Linker Generated Symbol File 
.rodata section layout
  00000000 000000 00000000  4 ...rodata.0 (entry of .rodata) 	d_a_horse.o 
  00000000 00000c 00000000  4 @3894 	d_a_horse.o 
  0000000c 000006 0000000c  4 l_arcName 	d_a_horse.o 
  00000014 000008 00000014  4 m_footJointTable__9daHorse_c 	d_a_horse.o 
  0000001c 000004 0000001c  4 m_callLimitDistance2__9daHorse_c 	d_a_horse.o 
  00000020 000124 00000020  4 m__14daHorse_hio_c0 	d_a_horse.o 
...

I know they have something to do with static linking, but exactly what, I don't know, and I wouldn't know where to start looking to figure out how to work with them. Does anyone know what these are? And if so, are there programs I can use that will take all of these files and give me the memory ranges that each function is located at?

# ? Aug 14, 2019 23:52

JawnV6: Jul 4, 2004; So hot ...

map files are an output of the linker, describing where it put things. I think the columns in the one you're showing are offset, size, location in memory. Is that all you need?

So for example

code:

  00000d94 000184 80006454  4 main 	m_Do_main.o

the main() function starts 0xd94 bytes into the text section, is 0x184 bytes long, and would be loaded at the memory address 0x80006454. Does that cover what you need? My guess for the other stuff is 4 is alignment and the last bit is the object file that the function came from.

ctors - constructors, dtors - destructors, rodata - read-only data

# ? Aug 15, 2019 00:15

robostac: Sep 23, 2009

They show the global symbols, where they are in memory and how big they are. I've not seen that exact format before but in my experience the format is entirely dependant on the linker used. You'd probably have to search for tools specifically for your linker if they exist.

When compiling symbols are put into sections depending on the type of symbol (eg .text is code, .data is global initialised variables, .bss is global variables set to 0 and .rodata are constants).

My guess for that file would be:
<Adresss in section> <size> <Address in memory> <?> <symbol> <file>

The second file is probably relocatable sections so the address in memory isn't known until that section has been loaded (eg dynamic linking).

Eg your main function is at address 0x80006454 and is 0x184 bytes long.

# ? Aug 15, 2019 00:24

Pollyanna: Mar 5, 2005; Milk's on them.

Oh, okay - so those are in fact the exact locations in memory where those functions are located, and I don't have to munge the files in any way to get that information. I was confused since all the map files seemed to start from 0x00000000, with that _prolog function.

In that case, I should be able to write a script to parse those files and list the memory range for each function. Thanks!!!!

robostac posted:

The second file is probably relocatable sections so the address in memory isn't known until that section has been loaded (eg dynamic linking).

Oh, drat, okay. I got some more info from someone working on a similar project, and supposedly they're statically linked. I guess I might have to go looking for that tool after all.

EDIT: Or maybe I actually can't, cause their location is only known when the game is actually running...dammit.

Pollyanna fucked around with this message at 00:40 on Aug 15, 2019

# ? Aug 15, 2019 00:28

csammis: Aug 26, 2003; Mental Institution

Do you have any more of the mapfiles which specify where each of those sections is placed in the binary? Your .rodata might be specified as being placed at 0x34000000 so offsets of symbols listed in the .rodata section would be relative to that address.

# ? Aug 15, 2019 01:11

Pollyanna: Mar 5, 2005; Milk's on them.

Oh yeah, there's like a "master" mapfile called frameworkF. All the sections and the first line under them look like this:

code:

.text section layout
  00000000 000068 800056c0  4 version_check__Fv 	m_Do_main.o 

.ctors section layout
  00000000 000000 803737c0  4 _ctors 	Linker Generated Symbol File 

.dtors section layout
  00000000 000000 80373980  4 _dtors 	Linker Generated Symbol File 

.rodata section layout
  00000000 000310 803739a0  4 @stringBase0 	m_Do_main.o 

.data section layout
  00000000 000000 803a2ee0  4 ...data.0 (entry of .data) 	m_Do_main.o 

.bss section layout
  00000000 000000 803d32e0  4 ...bss.0 (entry of .bss) 	m_Do_main.o 

.sdata section layout
  00000004 000004 80450584  4 memMargin__7mDoMain 	m_Do_main.o 

.sbss section layout
  00000008 000008 80450b08  4 sPowerOnTime__7mDoMain 	m_Do_main.o 

.sdata2 section layout
  00000000 000004 80451a00  4 @3884 	m_Do_main.o 

.sbss2 section layout
  00000000 000004 80456b60  4 @4530 	m_Do_graphic.o 

.stack section layout

.extabindex section layout
  00000000 00000c 80005660  4 @206 	Runtime.PPCEABI.H.a NMWException.o

It looks like all the other .data, .rodata etc entries in the other map files are relative to that first line's memory location, the third field. That seems to make sense. It might be as simple as reading that master mapfile, recording where those sections start, and all the functions just derive their locations from their offset and their size.

# ? Aug 15, 2019 01:24

Adbot: ADBOT LOVES YOU

# ? May 30, 2024 23:58

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

They're a linker map output by the CodeWarrior toolchain. They're left on the disk for the crash handler, which has a very hacky parser for them. You can load them in, say, Ghidra, with the GameCube loader: https://github.com/Cuyler36/Ghidra-GameCube-Loader

frameworkF.map refers to the main.dol binary, the REL objects are dynamically linked at runtime upon room transitions and other things -- it's a technique to save on memory, which was fairly limited on the GC/Wii. They're relocatable, which means that the game will dynamically choose a base address at runtime for them, load them in, patch up all relative pointers, link up imports, etc.

What are you looking to do with this? FYI, the Wind Waker and Twilight Princess codebases are pretty impossible to read for a non-Japanese speaker.

# ? Aug 15, 2019 03:06

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »