Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »

TheresaJayne: Jul 1, 2011

Thermopyle posted:

Looking over my oldest questions on Stack Overflow this morning. I started asking questions on there within months of the site opening in 2008 and it's funny how wet behind the ears I was

Funnily enough one of my highest rated questions is one of those newbie questions from 8 years ago and it still continually racks up votes.

I find myself not asking questions on there nowadays because my questions are usually a little more...specialized? I dunno how to describe the nature of the questions exactly but once I started asking questions that would get zero answers I kind of stopped trying to even ask.

I guess my question is: does this mirror anyone else's experience or did I just get bad at asking questions?

My top answer still gets upvotes all the time, its an answer for this question.

http://stackoverflow.com/a/10108667/621567

# ? Nov 28, 2016 13:22

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 08:54

Star War Sex Parrot: Oct 2, 2003

Bob Morales posted:

What is a good example of a simple CPU to use if you want to experiment with writing an emulator or assembler or something?

From an academic and instructional standpoint, MIPS is probably one of the most documented. That said, CHIP-8 as people said is pretty easy, while Z80 and 6502 are well-tread at this point with plenty of other work to look at.

# ? Nov 28, 2016 21:32

nielsm: Jun 1, 2009

Yeah in the hardware architecture course I took, the major project was to implement a subset of the MIPS instruction set in a circuit simulator. One of the big advantages IIRC is that all instructions are exactly one word long, simplifying decoding a bunch.

# ? Nov 28, 2016 21:58

ulmont: Sep 15, 2010; IF I EVER MISS VOTING IN AN ELECTION (EVEN AMERICAN IDOL) ,OR HAVE UNPAID PARKING TICKETS, PLEASE TAKE AWAY MY FRANCHISE

nielsm posted:

Yeah in the hardware architecture course I took, the major project was to implement a subset of the MIPS instruction set in a circuit simulator.

Considering the many many versions of the MIPS instruction set, you might actually have implemented all of say MIPS I.

# ? Nov 28, 2016 22:26

JawnV6: Jul 4, 2004; So hot ...

ulmont posted:

Considering the many many versions of the MIPS instruction set, you might actually have implemented all of say MIPS I.

Probably not. The parts that most arch courses drop off are branch flavors and the litigious instructions that require barrel shifters to do correctly.

# ? Nov 28, 2016 23:01

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

You could try to find someone's 'mini' version of some architecture on opencores.org, as another option.

I'd go with z80 though, but mostly because I owned a calculator in high school so I'm already familiar.

# ? Nov 29, 2016 14:25

Linear Zoetrope: Nov 28, 2011; A hero must cook

Boris Galerkin posted:

Thanks you two. I'm from an aero/mech engineering background now doing a lot of HPC stuff. It's just I've always worked with proprietary/in house codes so there's never been an instance where I didn't need to (re)compile my code so the whole splitting of blas and blas-devel is new to me, though I get it now.

About the compiling OpenBLAS myself:

In general I always heard that this was a bad idea to use your own compiled software vs getting it from a package manager? I mean I have nothing to back this up or know where I even heard this from. It's just always been one of those "things" I've heard. Have I been completely misguided this entire time?

I know that compiling OpenBLAS myself and throwing it into /usr/lib64 or whatever is a bad idea but I thought that kind of extended to "compiling OpenBLAS yourself when it's available in the package manager for your distribution is a bad idea as well because ______." I mean if that's not the case then why should I not compile my own gcc/gfortran with "-O2 -march=native" and then use that to compile my own OpenBLAS, mpich, boost, etc. etc, and throw it all into ~/home/local/ ?

Sorry I didn't see this earlier. It's usually better to use the distribution version for various reasons:

1. You don't have to deal with dependency hell building it yourself
2. It's easy to (un)install and takes care of updating itself with a simple command (including security updates!)
3. It tends to automatically put thing like pkgconfig files in the right place, which various libraries' make install sometimes fails to do.

99% of the time, use the package manager. BLAS specifically tends to use a prebuilt Atlas BLAS in the package manager. Atlas BLAS itself isn't known for having great performance, and the apt-get version may not be optimized for your computer. Generally OpenBLAS is considered a far better BLAS distribution if you're doing anything that requires performance, and isn't really that hard to build.

The only other times you tend to avoid the package manager are if you don't have sudo privileges and are dealing with hostile IT staff that doesn't like installing/updating packages, or you need to run a beta or experimental release/patch that's not on the package manager yet. At my university, for instance, I have my own ecosystem set up in my home directory because half of their libraries are hopelessly out of date. Which is fine for undergrads learning what a for loop is, but less ideal for me when I need bleeding edge SciKit-Learn features or whatever.

Linear Zoetrope fucked around with this message at 03:36 on Nov 30, 2016

# ? Nov 30, 2016 03:22

Knyteguy: Jul 6, 2005; YES to love
NO to shirts; Toilet Rascal

I'm mostly a C# guy trying to dig a little into C++. This is like day 1 newbie stuff.

When do you use pointers and what does the & symbol do before a variable?

What is the difference between:

C++ code:

gotDirectorySuccess = FileDownloader::getFile(targetUrl, fileContents, &updateProgress);

and

C++ code:

gotDirectorySuccess = FileDownloader::getFile(targetUrl, fileContents, updateProgress);

C++ code:

char * repoUrl = "192.168.1.104:8000";

and a string?

Thanks.

# ? Dec 2, 2016 23:38

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

Do you understand what pointers are? That's important to establish before we get into when they're used.

When you put & before a variable in a variable declaration (e.g. "int& foo"), you are saying "this variable is a reference type". I don't trust myself to give an accurate and concise definition of references here.

When you put a & before a variable name outside of the variable declaration, you are applying the & operator, which returns a pointer whose value is the address of the variable. This is the opposite of the * operator, which dereferences the pointer, returning the value found there. In other words:

code:

int foo;
int* foo_address_in_memory = &foo;
foo = 5;
printf("%d\n", *foo_address_in_memory); // prints "5"

In your example calling getFile(), the first is passing in a pointer to the local variable updateProgress, and the second is passing in the value of updateProgress. Unless there are two definitions of the function, one of which takes a pointer and one of which doesn't, only one of these is valid. Functions that take pointers to variablse usually do so because they want to modify the values of those variables. For example, a memory-copying function would ask you to allocate a block of memory, and then pass in the pointer to that block, so it can copy some data into it.

The difference between char* and std::string is that char* should never be used for string handling. It's the old, "C-style string" and is literally just an array of chars. There are old, C-style functions for working with these, but they're tricky to use and you can easily end up getting into buffer overrun situations or otherwise introducing crashes or security issues in your code if you use them. std::string is much more pleasant to work with and correspondingly harder to shoot yourself in the foot with.

# ? Dec 3, 2016 00:11

Knyteguy: Jul 6, 2005; YES to love
NO to shirts; Toilet Rascal

TooMuchAbstraction posted:

Do you understand what pointers are? That's important to establish before we get into when they're used.

When you put & before a variable in a variable declaration (e.g. "int& foo"), you are saying "this variable is a reference type". I don't trust myself to give an accurate and concise definition of references here.

When you put a & before a variable name outside of the variable declaration, you are applying the & operator, which returns a pointer whose value is the address of the variable. This is the opposite of the * operator, which dereferences the pointer, returning the value found there. In other words:
code:
int foo;
int* foo_address_in_memory = &foo;
foo = 5;
printf("%d\n", *foo_address_in_memory); // prints "5"
In your example calling getFile(), the first is passing in a pointer to the local variable updateProgress, and the second is passing in the value of updateProgress. Unless there are two definitions of the function, one of which takes a pointer and one of which doesn't, only one of these is valid. Functions that take pointers to variablse usually do so because they want to modify the values of those variables. For example, a memory-copying function would ask you to allocate a block of memory, and then pass in the pointer to that block, so it can copy some data into it.

The difference between char* and std::string is that char* should never be used for string handling. It's the old, "C-style string" and is literally just an array of chars. There are old, C-style functions for working with these, but they're tricky to use and you can easily end up getting into buffer overrun situations or otherwise introducing crashes or security issues in your code if you use them. std::string is much more pleasant to work with and correspondingly harder to shoot yourself in the foot with.

e: Thanks for the example that really helps. I don't really understand what pointers are, no. I was thinking it was related to scope, but that seems wrong. For example why does this function need a pointer declaration: http://stackoverflow.com/questions/8958044/expected-constructor-destructor-or-type-conversion-before-token ?

Knyteguy fucked around with this message at 02:00 on Dec 3, 2016

# ? Dec 3, 2016 01:20

dupersaurus: Aug 1, 2012; Futurism was an art movement where dudes were all 'CARS ARE COOL AND THE PAST IS FOR CHUMPS. LET'S DRAW SOME CARS.'

Modest Mouse cover band posted:

e: Thanks for the example that really helps. I don't really understand what pointers are, no. I was thinking it was related to scope, but that seems wrong. For example why does this function need a pointer declaration: http://stackoverflow.com/questions/8958044/expected-constructor-destructor-or-type-conversion-before-token ?

Massive oversimplification, but in c# whenever you pass a non-primitive value to a function, you're essentially passing a pointer (the location of the item in memory rather than creating a copy). While c# assumes the pointering, in c++ you have to explicitly declare it.

# ? Dec 3, 2016 02:06

leper khan: Dec 28, 2010; Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

dupersaurus posted:

Massive oversimplification, but in c# whenever you pass a non-primitive value to a function, you're essentially passing a pointer (the location of the item in memory rather than creating a copy). While c# assumes the pointering, in c++ you have to explicitly declare it.

that's true for 'class'es, but 'struct's are value types in c#

# ? Dec 3, 2016 02:22

Klades: Sep 8, 2011

Modest Mouse cover band posted:

e: Thanks for the example that really helps. I don't really understand what pointers are, no. I was thinking it was related to scope, but that seems wrong. For example why does this function need a pointer declaration: http://stackoverflow.com/questions/8958044/expected-constructor-destructor-or-type-conversion-before-token ?

CS undergrad who likes C++ too much post ahead:

C++ code:

void func(std::string* name); //Function taking a pointer to a string
void func2(const std::string& name); //Function taking a const reference to a string

So, pointers are variables that hold a memory address. You can dereference them by putting * before them to access the thing at the memory address they're holding. There's basically three main reasons to use pointers: declaring things on the heap (with new but don't actually use new we have better things now), passing things that would be expensive to copy (since a pointer just needs to be big enough to hold a memory address while your string might be a hundred bytes long), and passing things that you want to modify in the function (and have that modification be reflected outside).
References are a bit like pointers, in that they basically point you at something somewhere in memory. The difference is that references can't point to nothing (nullptr in modern C++), and you can't change what they point to. They can still point to things on the heap (I don't think you can use them to declare something on the heap, or at least if you can you shouldn't), and passing a reference to something is (probably) as cheap as passing a pointer. You also use them like you would a normal variable, so

C++ code:

void func(std::string* name) {
  std::cout << *name << '\n';
}

void func2(std::string& name) {
  std::cout << name << '\n'
}

Passing something by const ref is basically saying "I don't want this entire thing to get copied so give me a reference, but I promise that I'm not going to modify it inside the function". One benefit of this is that it lets you pass in a temporary value; void func(std::string str); won't work with func("Hello"); but void func(const std::string& str); is fine.

# ? Dec 3, 2016 03:43

lunar detritus: May 6, 2009

I have been playing around with Google's Cloud Vision's Face Recognition and it's pretty fun. When you analize a face it gives you a ton of information including the face's orientation (pan, tilt and roll).

I want to take a random image and transform it to fit the face's perspective but I have no idea how to do it, any geometry I knew is long forgotten. Google gives me stuff like transformation matrixes and euler angles but I don't know enough to use that theory in practice. Assuming I know nothing more advanced than arithmetics , where should I start (books, free courses, etc)?

# ? Dec 3, 2016 17:58

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

Transformations are done by multiplying what are called Homogeneous Matrices together. The first three columns of a 4x4 matrix describe the direction of rotation in the x, y, and z axis (pan, tilt and roll, respectively); and the fourth column describes the position. By setting up your matrices correctly, you can perform matrix multiplication that has the desired effect ...

I think this is a good reference from a programming standpoint: http://www.opengl-tutorial.org/beginners-tutorials/tutorial-3-matrices/

This covers specific translations point by point, and describes how rotation works: http://www.j3d.org/matrix_faq/matrfaq_latest.html

On that note, I was working on a GUI in opengl 4. Is it better to use opengl's coordinate origin of the bottom-left, to be consistent, or to use the top-left, which I think is more common for GUIs?

# ? Dec 3, 2016 18:55

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

a matrix is simple. if i have 2x2 matrix:

code:

a b
c d

then I can multiply it against vector x y and get:

code:

x' = a*x + b*y
y' = c*x + d*y

each row corresponds to an output, each column is a weight for one of the other inputs. this naturally extends to higher dimensions like 3x3 and an x y z vector.

we often use 4x4 matrices because there's no constant factor and we often want translation. this is known as an "affine transformation" rather than just a "linear one". so we hack up our input vector to have x y z w, and we set the w to be 1. that effectively gives us:

code:

x' = a*x + b*y + c*z + d*1

which gives us translation.

for more complex topics, this series is good: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab

# ? Dec 3, 2016 19:17

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

dougdrums posted:

This covers specific translations point by point, and describes how rotation works: http://www.j3d.org/matrix_faq/matrfaq_latest.html

This document is hideous - why is this person trying to represent matrix mathematics in plain ASCII text? It's nigh-unreadable.

I also spotted a few things which strike me as mathematically dubious; for example it claims that you can't transpose a non-square matrix, and it claims that "Attempting to calculate the reciprocal of zero, generates a value of infinity." which is a good sign of someone who thinks they understand mathematics but doesn't.

# ? Dec 3, 2016 20:50

SurgicalOntologist: Jun 17, 2004

I'm building my first web app (using Flask but posting here instead of the Python thread because this is a more general question) and I'm trying to figure out how to handle registration. The issue is that we don't want just anyone to be able to register. Users are associated with organizations, and we want to give users at each organization the ability to register, under their organization. We also don't want them to be able to see the other organizations in the database.

I have several ideas here. One option would be some kind of two-stage login, where we give each organization a password which provides access to their register page. A second option would to give each organization their own register URL. This is nice from an end-user perspective but I don't think it's truly secure. I doubt this is a novel problem...but is there any option which is secure besides giving each organization a password separate from the user password?

# ? Dec 3, 2016 21:23

fritz: Jul 26, 2003

Does anybody have some good docs for SSE/AVX programming? I can look at lists of intrinsics / opcodes and bang together a working thing from that, but it would be cool to see some best practices guides and'post-mortem reports.

# ? Dec 3, 2016 21:48

nielsm: Jun 1, 2009

SurgicalOntologist posted:

I'm building my first web app (using Flask but posting here instead of the Python thread because this is a more general question) and I'm trying to figure out how to handle registration. The issue is that we don't want just anyone to be able to register. Users are associated with organizations, and we want to give users at each organization the ability to register, under their organization. We also don't want them to be able to see the other organizations in the database.

I have several ideas here. One option would be some kind of two-stage login, where we give each organization a password which provides access to their register page. A second option would to give each organization their own register URL. This is nice from an end-user perspective but I don't think it's truly secure. I doubt this is a novel problem...but is there any option which is secure besides giving each organization a password separate from the user password?

Some simple but limited options are to put some trust into the network, and either let orgs have one or more registered gateway IPs all their employees connect from, or you can sort users into orgs based on email address domain. If j.smith@contoso.com signs up and can receive your registration mail, you can probably assume he's an employee at Contoso.

Other, more secure and flexible, solutions for self-sign-up will include some kind of SSO protocol, OAuth and ADFS could be options. Those can also require significant configuration in both ends.

The final option would be purely manual, simply require org members to be registered by a designated contact person inside the org.

# ? Dec 3, 2016 21:49

SurgicalOntologist: Jun 17, 2004

Thanks, the domain name idea is pretty simple, should have though of that. Hopefully that will suffice.

# ? Dec 4, 2016 04:24

Vier: Aug 5, 2007

There is a sale website that I use that displays 50 items at a time, with no way of choosing to display more than that, this results in around 15 pages for each auction day.
What I would like to do is get all of these pages to display at once in a single window. what type of knowledge would I need to achieve this? I have had a look at the source code of the site and tried doing simple things like finding a value of 50 and changing it to something higher but that does not seem to work.
I have also tried URL manipulation to try and load more at a time without any success.

Any advice would be appreciated.

# ? Dec 4, 2016 15:02

mystes: May 31, 2006

Vier posted:

There is a sale website that I use that displays 50 items at a time, with no way of choosing to display more than that, this results in around 15 pages for each auction day.
What I would like to do is get all of these pages to display at once in a single window. what type of knowledge would I need to achieve this? I have had a look at the source code of the site and tried doing simple things like finding a value of 50 and changing it to something higher but that does not seem to work.
I have also tried URL manipulation to try and load more at a time without any success.

Any advice would be appreciated.

If you can only get the server to display 50 items per page, you will have no choice except to obtain all the pages and assemble them yourself. You can either:
1) If you want to do this in the browser: load the first page and then make an extension/userscript that will use javascript to load the subsequent pages and append their content to the end of the body element of the page
2) Skip the browser and use any programming language to request all the pages and extract the data.

Do you have any programming background?

You could also try something like a free account on scrapinghub.com which is a hosted version of Portia, a gui for web scraping which doesn't require programming, but may still be somewhat complicated (and I'm not quite clear on what the free account limitations are so you might have to actually install Portia after you got it working, which may require linux or something).

mystes fucked around with this message at 15:47 on Dec 4, 2016

# ? Dec 4, 2016 15:44

Vier: Aug 5, 2007

mystes posted:

If you can only get the server to display 50 items per page, you will have no choice except to obtain all the pages and assemble them yourself. You can either:
1) If you want to do this in the browser: load the first page and then make an extension/userscript that will use javascript to load the subsequent pages and append their content to the end of the body element of the page
2) Skip the browser and use any programming language to request all the pages and extract the data.

Do you have any programming background?

You could also try something like a free account on scrapinghub.com which is a hosted version of Portia, a gui for web scraping which doesn't require programming, but may still be somewhat complicated (and I'm not quite clear on what the free account limitations are so you might have to actually install Portia after you got it working, which may require linux or something).

Thanks, I have 0 real programming skills outside of editing other peoples javascript, I took a look at scrapinghub and then ended up using something called webscraper.io which I nearly have working, I will stick with it.

# ? Dec 4, 2016 20:54

peak debt: Mar 11, 2001; b& :(; Nap Ghost

Is there a common standard about how to normalize "weird" unicode characters?

If I have a c�t�ʂtr�phı� string like this one, is there a function or table that translates those letters into their visually or phonetically closest 8 bit letter?

# ? Dec 5, 2016 16:45

Symbolic Butt: Mar 22, 2009; (_!_); Buglord

peak debt posted:

Is there a common standard about how to normalize "weird" unicode characters?

If I have a c�t�ʂtr�phı� string like this one, is there a function or table that translates those letters into their visually or phonetically closest 8 bit letter?

This is kind of a hairy problem but one way to start tackling it is to to convert the unicode text into normal form so you can break up the combining characters and then extract only the ASCII characters.

Here's how you can do it in python:

Python code:

import unicodedata

result = ''
for c in unicodedata.normalize('NFKD', 'c�t�&#642;tr�ph&#305;�'): # pretend SA didn't mangle your original string here e_e
    if ord(c) < 256:
        result += c

print(result)
# output: catatrophc

But notice how you lost "ʂ" and "ı" because they're not made up of combinations of ASCII characters so yeah... I hope this is a start at least.

# ? Dec 5, 2016 17:28

nielsm: Jun 1, 2009

peak debt posted:

Is there a common standard about how to normalize "weird" unicode characters?

If I have a c�t�ʂtr�phı� string like this one, is there a function or table that translates those letters into their visually or phonetically closest 8 bit letter?

Yeah that's a bad problem.

What are you actually trying to accomplish?

# ? Dec 5, 2016 17:44

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

peak debt posted:

Is there a common standard about how to normalize "weird" unicode characters?

If I have a c�t�ʂtr�phı� string like this one, is there a function or table that translates those letters into their visually or phonetically closest 8 bit letter?

What do you mean by "8 bit letter"? ASCII? Latin-1? (Which?)

# ? Dec 5, 2016 17:53

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

I don't think it would be possible to do this at any level of certainty, since there's nothing that relates any other unicode glyph to latin glyphs, or frankly any other language's glyphs to another. I mean, if I wanted to use the glyph '十' as a latin 'T' , what can you do? And what if a glyph makes up two latin letters, like "⠟叶"?

The only thing I can think of is to filter it through the categories, like: http://www.fileformat.info/info/unicode/category/Lu/list.htm has a list of all upper case letters ... While some may not be latin or renderable, you can match the unicode description up with its letter fairly simply. Here is a big ol' csv file that can help you sort through them: http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt

Are you trying to make a sort of 8-bit pidgin language? Or are you trying to filter out strings you don't want? Or something else?

dougdrums fucked around with this message at 18:36 on Dec 5, 2016

# ? Dec 5, 2016 18:18

peak debt: Mar 11, 2001; b& :(; Nap Ghost

The reason I'm asking is that we have a lovely embedded device that has to display names, and it only supports "normal" letters. If it grabs a string with extended unicode letters from the database it garbles them into a series of "å" replacements or question marks, depending on the exact letter. Currently this is handled by having about 40 lines of string.replace() exchange all the letters we know about but of course every now and again, some new letter shows up that we haven't met yet, and the display looks like poo poo again until somebody takes the time to fix the code.

We've done it this way ever since before I started working here but this morning I saw the display again with a garbled string, and thought that this problem must be rather common, and that the Unicode standard must offer some kind of solution. I see that Python at least did, and unicodedata.normalize() at least seems to go towards what I was looking for. Or we could parse that CSV for "letter ([a-z]+)" and use that?

It'll be hacky, but better than what we have now.

peak debt fucked around with this message at 19:14 on Dec 5, 2016

# ? Dec 5, 2016 19:08

dougdrums: Feb 25, 2005; CLIENT REQUESTED ELECTRONIC FUNDING RECEIPT (FUNDS NOW)

Ah, that's is a common, sensible, and difficult problem. I guess the first thing I would do (and you've probably already tried) is figure out what encoding the device qualifies as "normal", and see if there's something to go from utf-8 to moon encoding. If this isn't possible, or the encoding is really proprietary, you're left with simply making a big lookup table like you're already doing.

It is definitely impossible to shoehorn every, if not even most western names into the latin alphabet, and there's nothing unicode or anything else can really do. I think most users with non-latin names know best how to convert their names. Your best bet might be to allow the user to enter a latin display name wherever, along with their utf8 name, and display the latin name on the device. If that is not an option, as you don't have access to the input form, then I think you just have to keep entering the names manually.

Maybe if you posted utf-8 with the garbled result, we could take a stab at guessing the encoding?

Names are a strange thing. I believe Stallman is attributed with saying, "names are just names" when asked about the name of emacs. If I recall the story correctly, ed and emacs were named after an ice cream store, which was named after two deceased homeless men. My last name, despite being four letters, is utterly and hilariously unpronounceable for Mandarin speakers (also for Spanish and some English speakers ...). When I started learning Mandarin I was pretty stubborn about using my roman name, but it soon became clear that using my roman name was way too distracting and confusing, and I needed to use the proper syllabic Mandarin version instead. There are a ton of people who are still not able to represent their own names on a computer, it is an issue that runs pretty deep ... imo people are too arbitrarily attached to their own names. If your device somehow managed to mangle my name, I don't see how I could get too upset as long as I can still go "ya! that's me!".

dougdrums fucked around with this message at 19:47 on Dec 5, 2016

# ? Dec 5, 2016 19:27

nielsm: Jun 1, 2009

Apart from Unicode normalization data, the collation data can also be interesting for gelding text like that, since it will generally put "supposed to be similar/related" characters into the same collation groups or series.

# ? Dec 5, 2016 19:47

huhu: Feb 24, 2006

I'm beginning to learn Angular2 and I originally looked up a Django vs Angular blog post since I already know Django and my takeaway was that Django would do the back end and Angular would do the front end. I'm starting to wonder if that was a bad blog post I stumbled upon because it seems like everything I've learned in the first few hours of this Angular course I'm watching could just be done in Django instead. My question is then, is there a lot more to Angular that would make it worth learning to supplement my Django projects, is it a tool for completely different projects?

# ? Dec 5, 2016 21:43

ToxicFrog: Apr 26, 2008

dougdrums posted:

Ah, that's is a common, sensible, and difficult problem. I guess the first thing I would do (and you've probably already tried) is figure out what encoding the device qualifies as "normal", and see if there's something to go from utf-8 to moon encoding. If this isn't possible, or the encoding is really proprietary, you're left with simply making a big lookup table like you're already doing.

At a guess (having worked on similar problems before) it's not that "normal" is some weird character encoding, it's "the device only supports (and more importantly only has fonts for) ASCII, or perhaps Latin-1 if you're lucky" -- so you need to have some kind of mapping so that you don't end up trying to display names as "B????? F???".

# ? Dec 5, 2016 21:59

Ciaphas: Nov 20, 2005; > BEWARE, COWARD

I want to work on the side on a project for work, but it's entering into several new-to-me territories and I need to work out what I need to know.

We have Solaris SPARC (still working on the upgrade to Linux/Intel) servers that run some mathematical software, one process per dataset. We have operators that manually queue up runs; several can run simultaneously but after that the system runs out of RAM, so some 15+ years ago the then developers set up a GUI in Tcl/Tk that the computer operators (who use Windows machines and are, as a rule, not actually adept with computers at all--don't ask) can run via an X Windows session of some sort (currently Exceed). Besides this GUI being old and filled with code rot, Exceed is a crash-happy bastard, and when it crashes it takes the GUI--and any processes it was controlling--with it.

So partly as a learning thing and partly to help out, I want to redo the whole thing. I want to set up a control server--daemon, I guess--that runs on the UNIX box (be it Solaris or Linux) that spawns these data runs when asked, and a client--probably in C#/WPF--that connects to this server over TCP or whatever and issues commands/receives status updates for operator display.

Over the last couple work days I got as far as setting up an echo server on the UNIX side in C++ using UNIX sockets and threading it (using boost::thread) so that it can handle multiple concurrent sockets--main thread for the listener and a new thread for each client. Beyond this point though I'm entering unknown territory.

Where should I start? I need to learn best practices for client/server communication, how to use UNIX sockets more in depth than an echo server (both from the C/C++ server side and the C# client side), and probably a lot of other things I don't know that I don't know.

(ed) For what it's worth I should note that access to external/downloaded libraries on the server boxes is limited for security reasons, so probably whatever capabilities are built into standard C/C++ are all I have.

# ? Dec 5, 2016 22:17

Peristalsis: Apr 5, 2004; Move along.

Ciaphas posted:

I want to work on the side on a project for work, but it's entering into several new-to-me territories and I need to work out what I need to know.

We have Solaris SPARC (still working on the upgrade to Linux/Intel) servers that run some mathematical software, one process per dataset. We have operators that manually queue up runs; several can run simultaneously but after that the system runs out of RAM, so some 15+ years ago the then developers set up a GUI in Tcl/Tk that the computer operators (who use Windows machines and are, as a rule, not actually adept with computers at all--don't ask) can run via an X Windows session of some sort (currently Exceed). Besides this GUI being old and filled with code rot, Exceed is a crash-happy bastard, and when it crashes it takes the GUI--and any processes it was controlling--with it.

So partly as a learning thing and partly to help out, I want to redo the whole thing. I want to set up a control server--daemon, I guess--that runs on the UNIX box (be it Solaris or Linux) that spawns these data runs when asked, and a client--probably in C#/WPF--that connects to this server over TCP or whatever and issues commands/receives status updates for operator display.

Over the last couple work days I got as far as setting up an echo server on the UNIX side in C++ using UNIX sockets and threading it (using boost::thread) so that it can handle multiple concurrent sockets--main thread for the listener and a new thread for each client. Beyond this point though I'm entering unknown territory.

Where should I start? I need to learn best practices for client/server communication, how to use UNIX sockets more in depth than an echo server (both from the C/C++ server side and the C# client side), and probably a lot of other things I don't know that I don't know.

(ed) For what it's worth I should note that access to external/downloaded libraries on the server boxes is limited for security reasons, so probably whatever capabilities are built into standard C/C++ are all I have.

Forgive me if this is a stupid question, but it sounds like you want to manage a queue of parallel jobs on networked computers - have you considered using HTCondor?

# ? Dec 5, 2016 22:25

Ciaphas: Nov 20, 2005; > BEWARE, COWARD

Never heard of it, but I'll take a look. We only currently have the one production server running at a time, though. (Potentially multiple clients watching the server, but only one production server. A dev server too but that'd be a completely separate entity.)

Doesn't surprise me to realize that this is A Thing What's Been Done Before :v:

Ciaphas fucked around with this message at 22:33 on Dec 5, 2016

# ? Dec 5, 2016 22:30

Hammerite: Mar 9, 2007; And you don't remember what I said here, either, but it was pompous and stupid.; Jade Ear Joe

peak debt posted:

The reason I'm asking is that we have a lovely embedded device that has to display names, and it only supports "normal" letters. If it grabs a string with extended unicode letters from the database it garbles them into a series of "å" replacements or question marks, depending on the exact letter. Currently this is handled by having about 40 lines of string.replace() exchange all the letters we know about but of course every now and again, some new letter shows up that we haven't met yet, and the display looks like poo poo again until somebody takes the time to fix the code.

We've done it this way ever since before I started working here but this morning I saw the display again with a garbled string, and thought that this problem must be rather common, and that the Unicode standard must offer some kind of solution. I see that Python at least did, and unicodedata.normalize() at least seems to go towards what I was looking for. Or we could parse that CSV for "letter ([a-z]+)" and use that?

It'll be hacky, but better than what we have now.

Symbolic Butt's approach involving normalizing the input is probably a good start, you might still occasionally see cases that don't work and that you have to manually add as special cases though (as already remarked). I'm guessing there would be fewer special cases required than your current approach, because most (?) accented Latin characters can be decomposed into ASCII letters and combining accents.

The "å" stuff is because the device is expecting some flavour of Latin-1 and you are feeding it UTF-8 I guess.

# ? Dec 5, 2016 22:44

Thermopyle: Jul 1, 2003; ...the stupid are cocksure while the intelligent are full of doubt. �Bertrand Russell

huhu posted:

I'm beginning to learn Angular2 and I originally looked up a Django vs Angular blog post since I already know Django and my takeaway was that Django would do the back end and Angular would do the front end. I'm starting to wonder if that was a bad blog post I stumbled upon because it seems like everything I've learned in the first few hours of this Angular course I'm watching could just be done in Django instead. My question is then, is there a lot more to Angular that would make it worth learning to supplement my Django projects, is it a tool for completely different projects?

Django vs Angular doesn't exactly really make sense. Generally speaking you don't have an Angular site without Django (or some other backend). What Angular brings to the table is doing more stuff in the users web browser instead of on your own servers.

There's pros and cons of this approach, but one pro is that frontend frameworks like React and Angular can make your site feel more performant. So, like if you have a TODO list web site and the user creates a new TODO item when you're using Django the form gets submitted to your server, the page you're on gets redirected and they download a new page with the TODO list updated with the new item. With a frontend framework, they submit the new item, and the TODO list gets updated immediately right on the same page with no page refreshes or anything and the frontend framework updates the server with the new TODO item in the background.

I'm not completely convinced of the advice I'm about to give you, but if you're at the stage in the learning process it sounds like, you might be better off using React than Angular. React's documentation is a lot easier to grasp and has a smaller "surface area" if that makes any sense. Like Angular, React is also insanely popular and widely used.

# ? Dec 5, 2016 22:54

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 08:54

Extortionist: Aug 31, 2001; Leave the gun. Take the cannoli.

peak debt posted:

The reason I'm asking is that we have a lovely embedded device that has to display names, and it only supports "normal" letters. If it grabs a string with extended unicode letters from the database it garbles them into a series of "å" replacements or question marks, depending on the exact letter. Currently this is handled by having about 40 lines of string.replace() exchange all the letters we know about but of course every now and again, some new letter shows up that we haven't met yet, and the display looks like poo poo again until somebody takes the time to fix the code.

We've done it this way ever since before I started working here but this morning I saw the display again with a garbled string, and thought that this problem must be rather common, and that the Unicode standard must offer some kind of solution. I see that Python at least did, and unicodedata.normalize() at least seems to go towards what I was looking for. Or we could parse that CSV for "letter ([a-z]+)" and use that?

It'll be hacky, but better than what we have now.

If you're using python, there is the unidecode module: https://pypi.python.org/pypi/Unidecode/

Alternatively, the tar of that module includes the mappings they use, so you may be able to convert those into something you can use.

# ? Dec 5, 2016 22:58

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »