Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
MrMoo
Sep 14, 2000

Only secure web sockets, aka wss:// can make it through a proxy. Waiting for "Let's Encrypt" project to start before I bother looking at it.

Adbot
ADBOT LOVES YOU

Knifegrab
Jul 30, 2014

Gadzooks! I'm terrified of this little child who is going to stab me with a knife. I must wrest the knife away from his control and therefore gain the upperhand.

TheresaJayne posted:

Make sure your virtual machine gets its own IP address on the lan , also make sure that the fedora firewall is turned off.

Thank you it was a firewall issue.

hyphz
Aug 5, 2003

Number 1 Nerd Tear Farmer 2022.

Keep it up, champ.

Also you're a skeleton warrior now. Kree.
Unlockable Ben
Is there any language in which just reading in a binary file, decoding data in it, providing a UI editor for it, and saving it back isn't a soul-destroying exercise that leaves me miserable for the entire evolution of computing?

I've tried a whole bunch of them so far, and inevitably I end up having to write tons and tons of repeated "ceremonial" code to either
- Store the data from the file in an internal format that's readable with vaguely readable code.
- Copy the data back and forth from the secondary containers that the UI library insists on using.
- Build the UI bit-by-bit constantly repeating configuration for things that ought to be obvious (no, I do not want the label to be too small to read; no, I do not want the contents of a tab to overlap the tab bar; no, I want neither no space at all nor a cavernous yawning chasm between the label and the editor)

So many of the experimental geek-type languages end up having weak UI support or no support for Windows. Sometimes languages claim to have UIs even though they have no field editor gadget which would make me laugh if it didn't make me bang my head on the desk.

So is there any language or system that can do this even vaguely neatly?

The MUMPSorceress
Jan 6, 2012


^SHTPSTS

Gary’s Answer

hyphz posted:

Is there any language in which just reading in a binary file, decoding data in it, providing a UI editor for it, and saving it back isn't a soul-destroying exercise that leaves me miserable for the entire evolution of computing?

I've tried a whole bunch of them so far, and inevitably I end up having to write tons and tons of repeated "ceremonial" code to either
- Store the data from the file in an internal format that's readable with vaguely readable code.
- Copy the data back and forth from the secondary containers that the UI library insists on using.
- Build the UI bit-by-bit constantly repeating configuration for things that ought to be obvious (no, I do not want the label to be too small to read; no, I do not want the contents of a tab to overlap the tab bar; no, I want neither no space at all nor a cavernous yawning chasm between the label and the editor)

So many of the experimental geek-type languages end up having weak UI support or no support for Windows. Sometimes languages claim to have UIs even though they have no field editor gadget which would make me laugh if it didn't make me bang my head on the desk.

So is there any language or system that can do this even vaguely neatly?

Does the file have a predictable and repeated format? You could use mmap in c to map it to memory and point a struct pointer that represents its internal data structure at it, and use pointer arithmetic to traverse.

Like, if your file is just a repeating list of
String integer integer

and those represent some data type, you could just do something like:
code:
struct filestuff {
   char[5] title;
   int count;
   int value;
}
(Been a couple months, this is probably not quite the correct syntax for declaring a C struct).

Then you open your file, use mmap to map it to memory, and assign a struct filestuff * to the beginning of that mmap'd region. You can then iterate over it in chunks of sizeof(struct filestuff) to retrieve each individual piece.

This is how I did my big homework project to parse filesystem structures on a disk image, and I found it very easy to work with.

raminasi
Jan 25, 2005

a last drink with no ice

hyphz posted:

Is there any language in which just reading in a binary file, decoding data in it, providing a UI editor for it, and saving it back isn't a soul-destroying exercise that leaves me miserable for the entire evolution of computing?

I've tried a whole bunch of them so far, and inevitably I end up having to write tons and tons of repeated "ceremonial" code to either
- Store the data from the file in an internal format that's readable with vaguely readable code.
- Copy the data back and forth from the secondary containers that the UI library insists on using.
- Build the UI bit-by-bit constantly repeating configuration for things that ought to be obvious (no, I do not want the label to be too small to read; no, I do not want the contents of a tab to overlap the tab bar; no, I want neither no space at all nor a cavernous yawning chasm between the label and the editor)

So many of the experimental geek-type languages end up having weak UI support or no support for Windows. Sometimes languages claim to have UIs even though they have no field editor gadget which would make me laugh if it didn't make me bang my head on the desk.

So is there any language or system that can do this even vaguely neatly?

C# with WPF? If you have total control over the binary format spec and you're the only one who's ever going to be reading it, you can just serialize your UI's model object. It's not exactly good practice (those layers of indirection do have some benefits to go with their costs) but it would be fast to set up.

ExcessBLarg!
Sep 1, 2001

LeftistMuslimObama posted:

Does the file have a predictable and repeated format? You could use mmap in c to map it to memory and point a struct pointer that represents its internal data structure at it, and use pointer arithmetic to traverse.
If you're going to do this, make sure to use the correct packing attribute that your compiler supports, so that the compiler doesn't add padding underneath you. Also beware that the code may not be fully portable to architectures that don't support (or have to emulate) unaligned accesses. Fortunately if you only care about i386 and amd64 you're fine there.

Ruby is a good language for avoiding ceremonial code since you can metaprogram the poo poo out of it. I don't know about UI toolkit support though.

fritz
Jul 26, 2003

LeftistMuslimObama posted:

Does the file have a predictable and repeated format? You could use mmap in c to map it to memory and point a struct pointer that represents its internal data structure at it, and use pointer arithmetic to traverse.
...
Then you open your file, use mmap to map it to memory, and assign a struct filestuff * to the beginning of that mmap'd region. You can then iterate over it in chunks of sizeof(struct filestuff) to retrieve each individual piece.

This is how I did my big homework project to parse filesystem structures on a disk image, and I found it very easy to work with.

If you're going to do that, just use protobuf.

rjmccall
Sep 7, 2007

no worries friend
Fun Shoe

ExcessBLarg! posted:

If you're going to do this, make sure to use the correct packing attribute that your compiler supports, so that the compiler doesn't add padding underneath you. Also beware that the code may not be fully portable to architectures that don't support (or have to emulate) unaligned accesses. Fortunately if you only care about i386 and amd64 you're fine there.

The packing attribute should also take care of the alignment issue for you.

hyphz
Aug 5, 2003

Number 1 Nerd Tear Farmer 2022.

Keep it up, champ.

Also you're a skeleton warrior now. Kree.
Unlockable Ben

GrumpyDoctor posted:

C# with WPF? If you have total control over the binary format spec and you're the only one who's ever going to be reading it, you can just serialize your UI's model object. It's not exactly good practice (those layers of indirection do have some benefits to go with their costs) but it would be fast to set up.

I don't have control over the binary format. It's a large binary format with multiple records and packed in a 7-bit encoding because the creator hates everyone.

Hadlock
Nov 9, 2004

What's the term for why it takes so long to develop something from scratch, and people look back and go, "but that's so simple! why would that take three years to develop when I can crank out something identical in 20 minutes" when in fact they were the first ones to develop it

It's called something like, "the cost of first discovery process" or something like that. There's a specific term for it, but it's eluding me at the moment.

MrMoo
Sep 14, 2000

Hadlock posted:

What's the term for why it takes so long to develop something from scratch, and people look back and go, "but that's so simple! why would that take three years to develop when I can crank out something identical in 20 minutes" when in fact they were the first ones to develop it

It's called something like, "the cost of first discovery process" or something like that. There's a specific term for it, but it's eluding me at the moment.

Isn't it just R&D costs, everything is more obvious with hindsight. Also, especially in wedev land there is enormous enamor for reinventing the wheel and taking the same costs or even more than first discovery.

hyphz
Aug 5, 2003

Number 1 Nerd Tear Farmer 2022.

Keep it up, champ.

Also you're a skeleton warrior now. Kree.
Unlockable Ben

Hadlock posted:

What's the term for why it takes so long to develop something from scratch, and people look back and go, "but that's so simple! why would that take three years to develop when I can crank out something identical in 20 minutes" when in fact they were the first ones to develop it

It's called something like, "the cost of first discovery process" or something like that. There's a specific term for it, but it's eluding me at the moment.

Either Bleeding Edge or Pioneer Plague?

(Yes, I know Pioneer Plague was an Amiga game. It was the first game to use HAM graphics, which were a pain to use, so the title presumably followed logically.. I believed it originally meant diseases suffered by actual pioneers from encountering people they'd always been separate from before, but it might be a myth..)

raminasi
Jan 25, 2005

a last drink with no ice

hyphz posted:

I don't have control over the binary format. It's a large binary format with multiple records and packed in a 7-bit encoding because the creator hates everyone.

There's no language or platform that won't make this part a pain in the rear end, then.

Linear Zoetrope
Nov 28, 2011

A hero must cook
Speaking of binary file formats, is there any reason why some formats switch their endianness? A while ago I was writing a .wav decoder and for whatever reason parts of the format are defined to be stored in little endian, and parts in big endian.

The Laplace Demon
Jul 23, 2009

"Oh dear! Oh dear! Heisenberg is a douche!"

Jsor posted:

Speaking of binary file formats, is there any reason why some formats switch their endianness? A while ago I was writing a .wav decoder and for whatever reason parts of the format are defined to be stored in little endian, and parts in big endian.

To make sure everybody suffers equally.

ExcessBLarg!
Sep 1, 2001

Jsor posted:

Speaking of binary file formats, is there any reason why some formats switch their endianness?
Often files ("binary" or not) will contain blobs or chunks of data that are, internally, governed by a different format and so may use different endianness from other parts of the file. For example, WAV files are RIFF containers (with integers in the RIFF header stored little endian) where the actual audio sample data may be encoded with either endianness. You'd think the entire file would be written as little endian for consistency purposes, but the code that handles the audio samples may not know anything about the container it's placed in and vice versa.

If you're referring to the actual RIFF header format, integers are stored little endian, but you might see tag IDs that appear to be big-endian integers. By spec, though, the tag IDs are opaque four-byte sequences (effectively strings) and so you shouldn't try to interpret them as integers in a decoder.

program666
Aug 22, 2013

A giant carnivorous dinosaur
Is it too much of a hassle to use linux for android coding? I use vim and would prefer to work like I did with j2me, and that means just having a minimal window with the project open, where you manage project settings and that is capable of opening a emulator with my new code running already, and I tend to prefer command line over IDEs in general, but I'm afraid it could be much harder in other aspects and could switch to windows if it's much easier (and I may install command line and vim on Windows anyway).

Plorkyeran
Mar 22, 2007

To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed
The All of the standard Android developer tools are cross-platform and if anything Windows would be the worst platform to use them on. The only reason to specifically choose Windows for Android dev would be to use Visual Studio.

program666
Aug 22, 2013

A giant carnivorous dinosaur
Thanks.

Pie Colony
Dec 8, 2006
I AM SUCH A FUCKUP THAT I CAN'T EVEN POST IN AN E/N THREAD I STARTED
Can any of you Smart Guys recommend some resources for learning about emulation + virtualization (or point me to the thread where I should be asking this question)? The extent of my knowledge is that I tried to make a NES emulator once but got bored after making the 6502 part of it.

Storgar
Oct 31, 2011
^^ whoa, personally interested in the answer to that too.

If I have a function pointer declared as a member variable to a C++ class that I have, do I need to free it's memory in the destructor?

code:
class Butts {
public:
  ~Butts() {
    delete this->fart; // or something, I'm not actually sure...
  };
protected:
  void (*fart) ();
};
Also, has this entire thing been replaced with some sort of Boost/C++11/C++14 smart pointer setup?

nielsm
Jun 1, 2009



Storgar posted:

^^ whoa, personally interested in the answer to that too.

If I have a function pointer declared as a member variable to a C++ class that I have, do I need to free it's memory in the destructor?

code:
class Butts {
public:
  ~Butts() {
    delete this->fart; // or something, I'm not actually sure...
  };
protected:
  void (*fart) ();
};
Also, has this entire thing been replaced with some sort of Boost/C++11/C++14 smart pointer setup?

No, the function is stored in static memory. The function pointer points directly into the code (text) segment of your program, if you try to free it you'll most likely crash.

Although I'm not entirely sure what would be appropriate if you assigned a lambda function with bound variables to it... you should probably be using std::function instead then.

midnightclimax
Dec 3, 2011

by XyloJW
If I want to take a closer look at shell scripting, should I bother with zsh? I'm a total newb, so I'm not sure if maybe bash or fish or something would be better.

Storgar
Oct 31, 2011

nielsm posted:

No, the function is stored in static memory. The function pointer points directly into the code (text) segment of your program, if you try to free it you'll most likely crash.

Although I'm not entirely sure what would be appropriate if you assigned a lambda function with bound variables to it... you should probably be using std::function instead then.

Ah that makes a lot more sense than what I was thinking. I ended up using std::function, like you said. Thanks!

Ekster
Jul 18, 2013

Any good references for batch programming in windows? I've been checking a couple of tutorials via google but they are kinda sparse in their explanation.

It would be nice if it focuses on how to manipulate files and/or setting up for compiling and such.

nielsm
Jun 1, 2009



Ekster posted:

Any good references for batch programming in windows? I've been checking a couple of tutorials via google but they are kinda sparse in their explanation.

It would be nice if it focuses on how to manipulate files and/or setting up for compiling and such.

Any good reason to not use PowerShell? It's much more suited for advanced scripting, can do a whole lot more, and it's installed by default in all newer Windows versions.

Ekster
Jul 18, 2013

nielsm posted:

Any good reason to not use PowerShell? It's much more suited for advanced scripting, can do a whole lot more, and it's installed by default in all newer Windows versions.

I'm kinda new to programming, and especially batch files, so I wasn't even aware that exists. I'll look into it, thanks.

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.

midnightclimax posted:

If I want to take a closer look at shell scripting, should I bother with zsh? I'm a total newb, so I'm not sure if maybe bash or fish or something would be better.

I can't imagine writing a zsh-specific script outside of my .zshrc. Stick to bash, or even sh.

edit: in terms of writing scripts. Feel free to use either bash or zsh as your main shell.

pokeyman fucked around with this message at 17:55 on Aug 2, 2015

darthbob88
Oct 13, 2011

YOSPOS
I'm writing documentation for a company project, to train my replacement. What elements do I really need to include? I've gone over every source file with JSDoc, am in the process of writing up something on the path(s) of execution, and have written something simple which I should probably expand on "How to Install Our System". What else should I do? I'm getting paid hourly for this, so thoroughness is not an issue.

midnightclimax
Dec 3, 2011

by XyloJW

pokeyman posted:

I can't imagine writing a zsh-specific script outside of my .zshrc. Stick to bash, or even sh.

edit: in terms of writing scripts. Feel free to use either bash or zsh as your main shell.

Yeah I just thought maybe zsh has a different syntax that's more elegant (maybe) than ba/sh. Also portability is not really an issue.

nielsm
Jun 1, 2009



darthbob88 posted:

I'm writing documentation for a company project, to train my replacement. What elements do I really need to include? I've gone over every source file with JSDoc, am in the process of writing up something on the path(s) of execution, and have written something simple which I should probably expand on "How to Install Our System". What else should I do? I'm getting paid hourly for this, so thoroughness is not an issue.

What things look like when it's running correctly. Processes running, what gets written to logs during normal operation, etc. Knowing how "normal" looks makes it easier to identify abnormal situations. It's typically also easier than to enumerate every failure mode.

A high-level overview of components and their purposes, and the separation of concerns. Also a reverse index of functionality. E.g. you could look up "data validation" and read a description of where validation is fitted into the system, so the new guy can find the relevant code to fix bugs and add features, and will have less risk of accidentally building new code where it doesn't belong.

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.

midnightclimax posted:

Yeah I just thought maybe zsh has a different syntax that's more elegant (maybe) than ba/sh. Also portability is not really an issue.

There's some random options and builtins but nothing comes to mind for what you're talking about.

Other shells like fish are very different, and if you're not worried about portability that could be an interesting way to go.

pram
Jun 10, 2001
What is the best database/store for something like visitor statistics such as ip, time, page, referer, etc

I'd preferably like to have native queries and typical reduce stuff (avg, sum, min, max) and for it to not be super complicated to maintain. Maybe sqlite, or rethinkdb?

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.
Whatever relational database you like will almost certainly suit your needs.

sarehu
Apr 20, 2007

(call/cc call/cc)

nielsm posted:

Although I'm not entirely sure what would be appropriate if you assigned a lambda function with bound variables to it... you should probably be using std::function instead then.

It wouldn't compile.

nielsm
Jun 1, 2009



I don't think there is any cryptography-related thread, but I have a problem that probably involves it and I'd like some advice/suggestions for stuff to read up on.

I will probably be building a system that will need to collect some personal, sensitive information from users. The system would be used by a community group and the users would be volunteers who need to make this information available to function as volunteers. Either way, the legal stuff should be in order as long as the data security is sound.

The system should be collecting a small amount of sensitive information from each user. The data in an ASCII or simple structured format would be less than 128 bytes for most users.
Requirements I think I have:
1. The user must be able to see and modify his own data at a later data (this is strongly desirable)
2. One or two persons of managerial duty will need to extract and process the data from all users
3. The data must be encrypted on the wise (HTTPS to the front end, data store depends)
4. The data must be encrypted at rest (not be readable by a simple database query)
5. Breaking the secrecy of one user's data should not break the secrecy of any other users' data

I have come up with something that might be a solution, but I doubt it. I don't have good knowledge of what sorts of e.g. known-plaintext attacks might be possible.
Either way:
1. Each user has a symmetric key assigned, called Kd.
2. Kd is used to encrypt the sensitive data for the user.
3. Kd is not stored directly.
4. The user's password is used to derive another key, called Ku.
5. For the user to access and modify his data, the system stores Kd encrypted by Ku, for each user.
6. Ku is not stored.
7. An asymmetric key pair exists for managerial access, public key Kmp and private key Km.
8. Kmp is stored once for the system.
9. For managerial access to all user data, Kd encrypted by Kmp is stored for each user.
10. Km is stored offline and managers will download the encrypted data and process offline.
11. When a user retrieves his data, user enters password, Ku is derived, Kd is decrypted with Ku, data is decrypted with Kd, data is transferred to user via a secured connection.
12. When a user stores his data, user enters password, Ku is derived, Kd is decrypted with Ku, data is encrypted with Kd.

Things I'm not sure about:
A) Can block size matter for safety, i.e. if the data is in a common format might it be necessary to pad the data with randomness in one or both ends? Pad to a fixed size? Can that improve anything?
B) If an attacker can create a user on the system and create a password used to derive one Ku key, could that possibly be used to derive other keys apart from Kd for that one user?
C) ??? I don't know crypto

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
Don't roll your own cryptographic scheme. Don't roll your own key management scheme.

--

With that out of the way, I think it's worth taking a step back and figuring out what your actual objectives are. I'll just assume you're not legally obligated to do anything in particular, and this is about generally protecting your system and not ending up looking like a fool if someone hacks into it - if there are legal requirements around this stuff, it gets a fair bit more complex unless you just do the easy thing and do exactly what the law tells you to do.

The absolute first thing to do is start with threat modelling - figure out what sort of attacks you're trying to defend against. Do you care about :
- Someone running Firesheep on the Starbucks wifi?
- Someone who's pwned your web server and can issue requests against the database?
- A rogue employee deciding to look through the database for anything juicy?
- An outsider who's managed to hack in and get a full database dump?
- Someone at your hosting provider who can snoop on all your servers and their traffic?
- A rogue manager deciding to download everything and sell the information for cash?
- A government sending you a subpoena for information on one of your users?
- A three-letter agency who can break into your workplace and carry everyone off to a black site for interrogation?

Encryption helps against some of these attacks, but not others. Focusing on a complex encryption scheme without knowing what you're actually defending against practically always means you're leaving glaring weaknesses in your security in some places, while spending a lot of time and effort on security theater that doesn't actually accomplish much.

ExcessBLarg!
Sep 1, 2001

nielsm posted:

I don't think there is any cryptography-related thread, but I have a problem that probably involves it and I'd like some advice/suggestions for stuff to read up on.
This amount of specialized crypto in the data store is unusual. That doesn't necessarily mean it's unwarranted, but it's not yet clearly justified.

More commonly you'd find that user passwords are used for access control, but not direct encryption of the data. The data itself might reside on an encrypted store, but one protected by a single symmetric key and not-per row keys. So long as the only people who have non-API access to the server/store are trusted, then the common approach is sufficient. If the folks who do have access (login, physical, etc.) to the server/store are not trusted, then you need to consider specific attack scenarios and see what you're actually able to protect against.

For example, if I have root login access on the server, I can just dump and save all the user data as it comes in after TLS termination, or MITM the TLS connections using the TLS private key stored unencrypted on the server. At present, those are the weakest links in the system.

As for your questions:

nielsm posted:

A) Can block size matter for safety, i.e. if the data is in a common format might it be necessary to pad the data with randomness in one or both ends? Pad to a fixed size? Can that improve anything?
Strong ciphers, when used appropriately should not depend on randomness or other properties of the plaintext for security. You do need to pad plaintext for use with block ciphers, simply because they operate on specific block sizes, but the padding scheme is more about implementation compatibility and recovery of the original plaintext length. What's important is to use an appropriate block cipher mode of operation, and to make sure that any cipher dependencies (e.g., the initialization vector) are satisfied appropriately, as doing things inappropriately here (e.g., zero-filled IVs) will render the system insecure.

nielsm posted:

B) If an attacker can create a user on the system and create a password used to derive one Ku key, could that possibly be used to derive other keys apart from Kd for that one user?
Probably not, but what do you assume the attacker has access to?

ExcessBLarg! fucked around with this message at 17:21 on Aug 3, 2015

nielsm
Jun 1, 2009



I think these are the main purposes:
1. Regulatory compliance
2. Security for users on untrusted networks
3. Reduced risk of getting dumps of sensitive data through application bugs
4. Not seeing sensitive data if having to much about in the DB would be nice

3) and 4) are arguably handled just by separating the sensitive data into a separate table/data store.
2) by regular HTTPS.

It really the 1) that's the big issue, and I've just sent a letter to the national data protection agency asking for some specific guidelines on the types of data.

The actual data would most likely be "small fish", not something anyone would bother mounting an attack for. The purpose would largely be allowing the org to say they're taking reasonable steps for protecting the data submitted.

Adbot
ADBOT LOVES YOU

the talent deficit
Dec 20, 2003

self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture





This book is old but incredibly good and relevant to what you are asking: http://www.wayner.org/node/39

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply