Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Graphics formats have a lot of this. A recent format from a tiled GPU has every 8 bytes being backwards, for some reason.

GIF is also really dumb with this too. Comment from a codebase I have:

code:

            // A bitstream is something that takes a bytestream and
            // tries to read the individual bits in it. The GIF format
            // tries to read the rightmost (least significant) bit
            // first, and tries to read the code in reverse order,
            // which is quite annoying. With two bytes and three codes
            // of size 5, the packing looks like:
            //
            // byte 0 : bbbaaaaa
            // byte 1 : cccccbbb
            //
            // Note that the bits are in reverse order, with the
            // most significant bit being at the right-hand side.
            // This means that it isn't just a simple mask and shift,
            // but requires reversing all the bits in a tight loop.

# ? Apr 6, 2016 01:54

Adbot: ADBOT LOVES YOU

# ? May 18, 2024 06:41

Ciaphas: Nov 20, 2005; > BEWARE, COWARD

JawnV6 posted:

Endianness is the right term. From wikipedia:

I use the term "swizzle" for any arbitrary byte shuffling that lacks a concise name. I think it's from graphics.

I'm sorta assuming "including values not on a byte border" you mean word, or are there actual cases where you're swapping nybbles around?

Thanks, I didn't know that kind of byte shuffling fuckery was all that common. I'll have to look up the history of this data and see why it's set up this way.

(And yeah I meant word, sorry. I was just trying to say that some of the data < 8 bits long gets sliced up by the swaps, like in the posted examples.)

# ? Apr 6, 2016 04:53

Cryolite: Oct 2, 2006; sodium aluminum fluoride

edit: nevermind, the Windows thread in SH/SC is a better place for this question.

# ? Apr 6, 2016 23:47

Fergus Mac Roich: Nov 5, 2008; Soiled Meat

Is it necessary to parameterize an SQL statement that contains no user input? For example, I have setup code in my app for a SQLite database that creates some tables and views.

# ? Apr 6, 2016 23:55

Skandranon: Sep 6, 2008; fucking stupid, dont listen to me

Fergus Mac Roich posted:

Is it necessary to parameterize an SQL statement that contains no user input? For example, I have setup code in my app for a SQLite database that creates some tables and views.

If the library you are using supports parameters, then use it. There's no good reason to concat SQL unless there literally is no other way.

# ? Apr 7, 2016 00:29

piratepilates: Mar 28, 2004; So I will learn to live with it. Because I can live with it. I can live with it.

Also just because it's not coming straight from a user doesn't mean it's necessarily safe. Maybe it's derived from something a user entered down the line, maybe it's scraped from an untrusted source, etc. Either way it's just not worth it when it's easy enough to use parameterized statements and not have to worry.

# ? Apr 7, 2016 00:37

Sedro: Dec 31, 2008

Fergus Mac Roich posted:

Is it necessary to parameterize an SQL statement that contains no user input? For example, I have setup code in my app for a SQLite database that creates some tables and views.

You don't need to parameterize a statement which has no parameters, if that's what you're asking.

Otherwise, always use a parameterized query. If nothing else, you will help the database cache plans for identical queries with different parameters.

# ? Apr 7, 2016 01:49

Chuu: Sep 11, 2004; Grimey Drawer

Is there a good cross-platform (specifically, C++, C#, Python) toolkit that can generate human-readable serialized messages based on a defined protocol? Essentially something like Thrift or Protobuf, except encodes to JSON, XML, etc. instead of a binary format.

I know Protobuf v3 can do this -- but I am explicitly not allowed to use this because of (unfound?) paranoia that it might cause issues with the protobuf v2 code already floating around in our codebase.

# ? Apr 7, 2016 07:15

Gravity Pike: Feb 8, 2009; I find this discussion incredibly bland and disinteresting.

I'm not quite sure what you're asking. Each of those languages has a good JSON de/serialization library. Are you looking for something that is one "project," but releases libraries for multiple languages?

# ? Apr 7, 2016 08:17

Chuu: Sep 11, 2004; Grimey Drawer

Gravity Pike posted:

I'm not quite sure what you're asking. Each of those languages has a good JSON de/serialization library. Are you looking for something that is one "project," but releases libraries for multiple languages?

I have a series of messages that I want to send between apps on different platforms. I have definitions of all the messages, I want a tool that will take those definitions and generate the serialization/deserialization/validation code for me. There are tons of tools that do this (protobuf, thrift, someone in the C++ thread just pointed me to cap'n'proto) *but* I want the generated messages to be human readable.

Protobuf3 can do this, since JSON serialization is defined as part of the spec*, but as I said I can't use it. Theoretically JSON Schemas could be used to solve this problem, but all the JSON Schema libraries out there look incredibly immature.

I am unsure if Avro can do this, but I am shying away from Avro because of how heavyweight it is**.

(*protobuf2 does have a human readable serialization format, but it's mainly for logging so it's hard to tell if it is stable across releases. It's also not avaliable on all platforms -- specifically it's not avaliable in any of the C# ports of protobufv2)

(**Actually . . . so much of the docs focus on RPC which is necessarily heavyweight that maybe I am getting the wrong impression if just using it for messaging? If you have experience with Avro and think it's suitable, please advise!)

Chuu fucked around with this message at 09:04 on Apr 7, 2016

# ? Apr 7, 2016 08:51

nielsm: Jun 1, 2009

Do you need the actual wire data to be JSON/whatever, or do you just need a mechanism to dump messages for debugging purposes once in a while?

I'm pretty sure that at least with Python, it should be quite simple to massage e.g. a Protobuf object so you can feed it straight to a JSON serializer, and then just dump it to a log file or whatever.

# ? Apr 7, 2016 09:37

Bob Morales: Aug 18, 2006; ~~Just wear the fucking mask, Bob~~

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Skandranon posted:

If the library you are using supports parameters, then use it. There's no good reason to concat SQL unless there literally is no other way.

*peeks inside legacy codebase at work*

At least the second one is escape'd

# ? Apr 7, 2016 13:38

Chuu: Sep 11, 2004; Grimey Drawer

nielsm posted:

Do you need the actual wire data to be JSON/whatever, or do you just need a mechanism to dump messages for debugging purposes once in a while?

I'm pretty sure that at least with Python, it should be quite simple to massage e.g. a Protobuf object so you can feed it straight to a JSON serializer, and then just dump it to a log file or whatever.

Wire. I am explicitly trying to avoid massaging anything because when a message is added or changed, I just want to update something like a .proto file and have all platforms automatically bring in the changes via their build process. Also it means not having to write and debug the same serialization and validation code in multiple languages for every change.

Chuu fucked around with this message at 15:14 on Apr 7, 2016

# ? Apr 7, 2016 15:12

Ciaphas: Nov 20, 2005; > BEWARE, COWARD

Ciaphas posted:

Thanks, I didn't know that kind of byte shuffling fuckery was all that common. I'll have to look up the history of this data and see why it's set up this way.

(And yeah I meant word, sorry. I was just trying to say that some of the data < 8 bits long gets sliced up by the swaps, like in the posted examples.)

Replying to myself as a continuation to the story, turns out this mixed endian swizzling is only through PART of the file, and the documentation incorrectly identified which parts

I want to find the idiothitler hellfucker who wrote these docs so I can do something meanspirited to them :mad:

# ? Apr 7, 2016 23:25

P-Value Hack: Apr 4, 2016

I know in the real database world, MS Access is poo poo on a lot for being bloated and dumb, but nevertheless, I have no programming skills and all I need is a simple database (not even really relational in structure, there's only one table) to store contact information. So, is there a thread in Cavern of COBOL for MS Access questions?

I've tried googling but no results, I want to make a simple .pdf form where the fieldds (like name, phone number, address) etc. can be filled out, and then I can automatically import that completed .pdf into my MS Access when I receive it back from them, just so I don't have to retype it into the data entry form in MS Access that I created. I can't seem to word my Google correctly to find a website with an instruction on this, even though it seems like a fairly simple request, I keep getting other results.

# ? Apr 8, 2016 08:01

Linear Zoetrope: Nov 28, 2011; A hero must cook

I don't think you can really parse PDF files that easily, I assume you can get someone to write a database import script pretty simply, but you probably want to use anything other than a PDF for your form. Even a .docx is probably better, as bad as those can be to parse, because I would guess Microsoft Access, being an MS product, probably has some tools to work with the format.

On Googling, I find a bunch of really technical sites that try and invent PDF parsing from scratch, and the only library I'm finding requires you to work directly with the Acrobat API. Since you have no programming experience, this probably isn't something you want to monkey with.

# ? Apr 8, 2016 08:10

Hadlock: Nov 9, 2004

re: P-value

If it's a single table, you could just store everything in a CSV file. There's no need to get hyper-fancy with Access unless you need the features available.

# ? Apr 8, 2016 09:05

Lysidas: Jul 26, 2002; John Diefenbaker is a madman who thinks he's John Diefenbaker.; Pillbug

Jsor posted:

I don't think you can really parse PDF files that easily, I assume you can get someone to write a database import script pretty simply, but you probably want to use anything other than a PDF for your form. Even a .docx is probably better, as bad as those can be to parse, because I would guess Microsoft Access, being an MS product, probably has some tools to work with the format.

On Googling, I find a bunch of really technical sites that try and invent PDF parsing from scratch, and the only library I'm finding requires you to work directly with the Acrobat API. Since you have no programming experience, this probably isn't something you want to monkey with.

Getting data out of a PDF form is radically different from parsing PDFs from scratch.

# ? Apr 8, 2016 12:46

mystes: May 31, 2006

P-Value Hack posted:

I know in the real database world, MS Access is poo poo on a lot for being bloated and dumb, but nevertheless, I have no programming skills and all I need is a simple database (not even really relational in structure, there's only one table) to store contact information. So, is there a thread in Cavern of COBOL for MS Access questions?

I've tried googling but no results, I want to make a simple .pdf form where the fieldds (like name, phone number, address) etc. can be filled out, and then I can automatically import that completed .pdf into my MS Access when I receive it back from them, just so I don't have to retype it into the data entry form in MS Access that I created. I can't seem to word my Google correctly to find a website with an instruction on this, even though it seems like a fairly simple request, I keep getting other results.

I haven't tried either of these personally, but:
Option 1 ($): Use Acrobat (probably pro) to create form, and then use Acrobat to export the form data into CSV files, which you will be able to import into Access with a little work (the easiest way is to create a table with columns that exactly match the names of the columns in the CSV file).
Option 2: Use libreoffice to make the form and then use any of various programs/libraries. Pdftk is a simple command-line program that can dump pdf form data, but it's not in a csv format, so you might need to write or find a really simple program to fix this.

# ? Apr 8, 2016 13:38

ulmont: Sep 15, 2010; IF I EVER MISS VOTING IN AN ELECTION (EVEN AMERICAN IDOL) ,OR HAVE UNPAID PARKING TICKETS, PLEASE TAKE AWAY MY FRANCHISE

Jsor posted:

On Googling, I find a bunch of really technical sites that try and invent PDF parsing from scratch, and the only library I'm finding requires you to work directly with the Acrobat API. Since you have no programming experience, this probably isn't something you want to monkey with.

You're right since the OP has no programming experience, but if you ever want to read text data out of a PDF in Java, Apache PDFBox is pretty good at it.
https://pdfbox.apache.org/

# ? Apr 8, 2016 13:50

Steve French: Sep 8, 2003

Chuu posted:

I have a series of messages that I want to send between apps on different platforms. I have definitions of all the messages, I want a tool that will take those definitions and generate the serialization/deserialization/validation code for me. There are tons of tools that do this (protobuf, thrift, someone in the C++ thread just pointed me to cap'n'proto) *but* I want the generated messages to be human readable.

Protobuf3 can do this, since JSON serialization is defined as part of the spec*, but as I said I can't use it. Theoretically JSON Schemas could be used to solve this problem, but all the JSON Schema libraries out there look incredibly immature.

I am unsure if Avro can do this, but I am shying away from Avro because of how heavyweight it is**.

(*protobuf2 does have a human readable serialization format, but it's mainly for logging so it's hard to tell if it is stable across releases. It's also not avaliable on all platforms -- specifically it's not avaliable in any of the C# ports of protobufv2)

(**Actually . . . so much of the docs focus on RPC which is necessarily heavyweight that maybe I am getting the wrong impression if just using it for messaging? If you have experience with Avro and think it's suitable, please advise!)

Thrift supports a variety of pluggable transport protocols, not just the thrift binary format. One of the protocols that it supports is JSON:

https://github.com/apache/thrift/blob/master/lib/json/schema.json

Maybe the resulting JSON doesn't meet your requirements for readability? But perhaps worth looking into.

# ? Apr 8, 2016 15:56

P-Value Hack: Apr 4, 2016

Hadlock posted:

re: P-value

If it's a single table, you could just store everything in a CSV file. There's no need to get hyper-fancy with Access unless you need the features available.

Well I use Access like a horrible frankenstein "case management" tool where I have a Form with both all their info, as well as radio buttons to click on what point they are along in our process (received documents, documents sent out, awaiting response etc.). Then I can generate reports to colour code all of the cases to show what stage they are at in the process. So, I'd like to keep Access for now.

I'm thinking of just making it a Word file as it seems those are easier to import into Access, but a .pdf just seems more "professional" I guess.

mystes posted:

I haven't tried either of these personally, but:
Option 1 ($): Use Acrobat (probably pro) to create form, and then use Acrobat to export the form data into CSV files, which you will be able to import into Access with a little work (the easiest way is to create a table with columns that exactly match the names of the columns in the CSV file).
Option 2: Use libreoffice to make the form and then use any of various programs/libraries. Pdftk is a simple command-line program that can dump pdf form data, but it's not in a csv format, so you might need to write or find a really simple program to fix this.

Yes, I think I'll either try the first one (since I do have Acrobat Pro I believe) or just use a Word document instead.

# ? Apr 8, 2016 17:36

JawnV6: Jul 4, 2004; So hot ...

You could also use Google Forms to put up and online survey that would be easy to save out as a .csv and pull into Access. If your users are fine with PDF forms that may not be an issue.

# ? Apr 8, 2016 18:08

mystes: May 31, 2006

ulmont posted:

You're right since the OP has no programming experience, but if you ever want to read text data out of a PDF in Java, Apache PDFBox is pretty good at it.
https://pdfbox.apache.org/

I think in most cases, if just working with just the dumped text out of pdftotext or whatever isn't enough, you're probably better off reconsidering whether you really need to do what you are planning on doing before resorting to this.

P-Value Hack posted:

Yes, I think I'll either try the first one (since I do have Acrobat Pro I believe) or just use a Word document instead.

If there's no reason you specifically need it to be a PDF, you might want to consider other options. Google forms as suggested by JawnV6 is probably the best idea.

Otherwise, there are lots of ways if you can program, but if you can't, it's sort of ugly, but one option is to create an excel file with two tabs. The first will be a "form" for entering data with all the cells that aren't fillable being locked. The second will be a hidden tab that uses formulas to convert the data from the first tab into a proper format to import into access.

Office might have some sort of built in form filling thing, but you probably wouldn't want to actually use it even if it exists (or still exists).

mystes fucked around with this message at 23:53 on Apr 8, 2016

# ? Apr 8, 2016 23:45

ufarn: May 30, 2009

As part of a CI test, I need to run a bash script that does something along the lines of:

npm runserver && stopserver

What's the best way to this idiomatically with bash/Travis CI since you can't && an on-going process?

# ? Apr 9, 2016 23:22

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

ufarn posted:

As part of a CI test, I need to run a bash script that does something along the lines of:

npm runserver && stopserver

What's the best way to this idiomatically with bash/Travis CI since you can't && an on-going process?

I don't think you can. && only makes sense when the first program you invoke has a return code, so the boolean logic can evaluate it. You could invoke npm as a backgrounded program:

code:

npm runserver& stopserver

But that of course would always run stopserver even if runserver failed for some reason, and could also run stopserver before runserver was "up and running".

# ? Apr 9, 2016 23:47

Maluco Marinero: Jan 18, 2001; Damn that's a
fine elephant.

Yeah. This is really dependant on what the server itself is and what the tests are hoping to accomplish.

# ? Apr 9, 2016 23:53

Gravity Pike: Feb 8, 2009; I find this discussion incredibly bland and disinteresting.

ufarn posted:

As part of a CI test, I need to run a bash script that does something along the lines of:

npm runserver && stopserver

What's the best way to this idiomatically with bash/Travis CI since you can't && an on-going process?

Maybe

Bash code:

npm runserver &
NPMPID=$!
sleep 1
ps -p $NPMPID
NPMRUNNING=$?

if [ $NPMRUNNING -eq 0 ]
then stopserver
else echo "Server not running, the world is on fire"
fi

My bash is terrible, I'm sure that someone else could clean this up...

# ? Apr 10, 2016 09:45

ufarn: May 30, 2009

I don't really have an idea either. I just have a node project I forked, which is started with npm start. I guess I could write a quick JS linter for the few files I need instead.

# ? Apr 10, 2016 10:03

22 Eargesplitten: Oct 10, 2010

Is it poor form to check if an integer is 0 by using something like this? I guess there's always the outside chance of it somehow becoming negative. I need to get better at exception checking.

code:

int foo = bar;

if(!foo)

# ? Apr 10, 2016 19:02

Fergus Mac Roich: Nov 5, 2008; Soiled Meat

22 Eargesplitten posted:

Is it poor form to check if an integer is 0 by using something like this? I guess there's always the outside chance of it somehow becoming negative. I need to get better at exception checking.
code:
int foo = bar;

if(!foo)

That's common, you can do it if you like. Negative integers are also considered to be "true". If you need to check for them, you want if (foo <= 0).

# ? Apr 10, 2016 19:05

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

Personally I prefer to be explicit instead of relying on implicit typecasting, if only because typecasting has a tendency to change from one language to the next. This is probably not a big deal for ints (are there any languages where all non-null ints are true?) but can definitely catch you out for strings, lists, etc.

# ? Apr 10, 2016 19:06

Dr. Stab: Sep 12, 2010; 👨🏻‍⚕️🩺🔪🙀😱🙀

I would just go with the one that conveys the intended meaning the best, which will almost always be "if (foo == 0)"

# ? Apr 10, 2016 19:08

Skandranon: Sep 6, 2008; fucking stupid, dont listen to me

Dr. Stab posted:

I would just go with the one that conveys the intended meaning the best, which will almost always be "if (foo == 0)"

Agreed. While sometimes JavaScripts type coercion works in your favour, it usually does not and ends up confusing things. I prefer to keep my conditions strictly operating on Boolean types.

# ? Apr 10, 2016 19:19

The Laplace Demon: Jul 23, 2009; "Oh dear! Oh dear! Heisenberg is a douche!"

TooMuchAbstraction posted:

Personally I prefer to be explicit instead of relying on implicit typecasting, if only because typecasting has a tendency to change from one language to the next. This is probably not a big deal for ints (are there any languages where all non-null ints are true?) but can definitely catch you out for strings, lists, etc.

Lisps typically treat any value other than false or nil (for the ones that have it) as true, e.g. Common Lisp, Clojure(script), Scheme.

# ? Apr 10, 2016 19:29

ToxicFrog: Apr 26, 2008

TooMuchAbstraction posted:

Personally I prefer to be explicit instead of relying on implicit typecasting, if only because typecasting has a tendency to change from one language to the next. This is probably not a big deal for ints (are there any languages where all non-null ints are true?) but can definitely catch you out for strings, lists, etc.

Lua and most (all?) Lisp dialects consider all numeric values, including 0, to be logical true.

# ? Apr 10, 2016 20:23

Steve French: Sep 8, 2003

Also, Ruby. I'm a firm believer that this is the correct behavior, assuming treating non-boolean values as boolean is allowed, which it shouldn't be. :colbert:

# ? Apr 10, 2016 21:40

sarehu: Apr 20, 2007; (call/cc call/cc)

22 Eargesplitten posted:

Is it poor form to check if an integer is 0 by using something like this? I guess there's always the outside chance of it somehow becoming negative. I need to get better at exception checking.
code:
int foo = bar;

if(!foo)

The hard version of this question is when foo and bar are bools and you're deciding between foo ^ bar and foo != bar.

# ? Apr 11, 2016 01:04

22 Eargesplitten: Oct 10, 2010

That just tells me I need to brush up on my discrete mathematics, I barely remembered that ^ is XOR.

# ? Apr 11, 2016 01:30

Adbot: ADBOT LOVES YOU

# ? May 18, 2024 06:41

TheresaJayne: Jul 1, 2011

Lysidas posted:

Getting data out of a PDF form is radically different from parsing PDFs from scratch.

Should be fairly simple
http://www.ccma.cat/llibredestil/sites/www.ccma.cat.llibredestil/files/GuiaEditorialENG.pdf
http://partners.adobe.com/public/developer/tips/topic_tip31.html

# ? Apr 11, 2016 07:58

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »