Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

Agreed with the Python recommendation. All of the scientific Python software I've written has used scipy, which is a superset of numpy. numpy is basically your vector math library (and it gives you the detailed datatype control you need); on top of that scipy provides a ton of algorithms that work with those vectors (FFTs, Simplex methods, linear regressions, all kinds of statistics, etc.), as well as matplotlib (a fairly capable plotting library), and a few other things I never had reason to work extensively with.

# ? Apr 27, 2016 20:59

Adbot: ADBOT LOVES YOU

# ? May 30, 2024 20:42

baka kaba: Jul 19, 2003; PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

Der Shovel posted:

Follow up question, though. Let's say instead of just picking out names for a separate list I want to replace a found bad guy's name with, say "TERRORIST ALERT: *NAME*" on a list.

Replacing substrings and strings in Haskell seems easy enough, but if I do it as an extension of what I've got going right now (1. find matches on lists, 2. add matches to a third list, 3. go through lists again replacing hits I found earlier with updated text) that seems insanely wasteful.

Is there a cooler way of doing that?

Couldn't you just iterate over your lists, and use a map function that spits out the original name or the TERROR DETECTED version, depending on whether the name is found in your watchlist?

e- like this, since I'm not on my phone now

code:

detectTerror:: [String] -> String -> String
detectTerror watchlist name =
	case elem name watchlist of
	True -> "WOOP WOOP: " ++ name
	False -> name

main = do
	let passengers = ["Ajay","Carmen","Annika"]
	let knownTerrors = ["Bob","Carmen"]
	print $ map (detectTerror knownTerrors) passengers

code:

["Ajay","WOOP WOOP: Carmen","Annika"]

baka kaba fucked around with this message at 17:32 on Apr 28, 2016

# ? Apr 28, 2016 14:16

Shaman Tank Spec: Dec 26, 2003; *blep*

Walh Hara posted:

It's probably possible to simplify this a lot by using monads, but I don't know which one to use and didn't want to bother writing one myself.

Note: the type of "foldr (\ (x, b1) (xs, b2) -> (x:xs, b1 || b2) ) ([], False)" is [(a, Bool)] -> ([a], Bool) and it probably also could have been written as "\list -> (map fst list, List.any snd list)" .

Thanks, this is actually helpful. I'll look into it

Right now I'm having the stupidest problem, though. I'm trying to write files. I have a list of Strings, which I want to write into a text file. If I just have one String I want to write, that's easy enough and

code:

writeFile outputFile stringToWrite

works fine. But when I have a list of files, it doesn't work. Apparently this is because writeFile doesn't want a [String] as its argument. OK, so we can probably get past this with map, right? Well... can we? If we can, I can't figure it out, but it's probably due to some stupid formatting error I'm making.

Here's what I've tried:

code:

map (appendFile outputFile) processedText

-- And

map (\x -> appendFile outputFile x) processedText

Both produce the same error: "Couldn't match expected type `IO b' with actual type `[IO ()]'"

Googling does not really help me here.

# ? Apr 28, 2016 18:21

Asymmetrikon: Oct 30, 2009; I believe you're a big dork!

You can use "sequence" to run a list of IO () (that is, to go [IO a] -> IO [a]):

code:

sequence $ map (appendFile outputFile) processedText

# ? Apr 28, 2016 19:00

Shaman Tank Spec: Dec 26, 2003; *blep*

God damnit, I should've remembered that. Thanks

EDIT: Good GOD this is slow! I mean I'm not surprised it's slow, since I'm doing discrete IO operations while writing a massive .txt file one line at a time. Is there a better way of doing all the writing at once through newlines or something?

For the record, I did a simple program that accepts as its input a .txt file with columns of census data, strips away all but the first column, makes sure everything is written with proper cases and then outputs the results into a separate file. It works fine but JESUS this output is taking forever. Possibly because the source file is three megs :v:

E2: Welp, that was easy enough!

code:

outputFile <- openFile outputPath WriteMode
mapM (hPutStrLn outputFile) processed
hClose outputFile

Now the file writing took less than a second :v:

Shaman Tank Spec fucked around with this message at 20:19 on Apr 28, 2016

# ? Apr 28, 2016 19:57

LOOK I AM A TURTLE: May 22, 2003; "I'm actually a tortoise."; Grimey Drawer

Even more succinct:

code:

mapM_ (appendFile outputFile) processedText

To write Haskell code is to forever search for a way to reduce your entire program to a single library function call that does all the stuff you just spent the last hour working out.

mapM_ discards the results of the operation, which are just a list of empty tuples in this case. If you want the results for some reason (can't imagine why) you can use the regular mapM.

# ? Apr 28, 2016 19:59

Walh Hara: May 11, 2012

LOOK I AM A TURTLE posted:

To write Haskell code is to forever search for a way to reduce your entire program to a single library function call that does all the stuff you just spent the last hour working out.

My main problem with Haskell is that reading code from other people is often really difficult as you keep have to look up what some weird library function does. In any code that's not trivially simply I also often end up wondering whether a certain function is defined somewhere else in the code or is from a library.

I stopped programming in Haskell when I encountered this function in somebody else's code: https://hackage.haskell.org/package/base-4.8.2.0/docs/Control-Arrow.html#v:second
It's probably a really easy concept once you understand it, but I just found it incredible frustrating that I kept having to spend so much time researching weird library functions.

# ? Apr 28, 2016 20:19

Eela6: May 25, 2007; Shredded Hen

Just checking in to say I've starting my migration to a python-based approach and enjoying it so far. Matlab will remain in my toolbox, but having explicit dictionaries, etc alongside numpy's tools is really nice. Thanks again!

# ? Apr 28, 2016 21:50

John Brown: Jul 10, 2009

Hi, I was looking for some information about programming and this thread seems like a good resource. I have a few questions I think would help me prepare for a new gig I've landed. I'm a Digital Project Manager with an emphasis on managing what I assume will be the implementation, and/or maintenance, of a site that offers services such as the refinancing of vehicles and applying for auto loans. Since I only have less than a year of experience managing these types of projects I'm hoping you guys could give me some tips or things to consider to refresh my general knowledge.

Here are some questions I have:
-What are the most common types of CMSs used for sites that offer these services, or for newly built/updated sites with similar purposes?
-In regards to those CMSs/sites, what are some of pointers I need to be aware of? (i.e. most common maintenance issues, compatibility with other types of software/programming languages, potential security risks)
-Are there any references (preferably free) I could seek out that pertain to programming for sites and apps? Like a beginner's level intro that explains the basic principles of web and mobile development (I see on the first page you guys mentioned The Practice of Programming and Code Complete so this may already be answered).

Any help would be much appreciated.

# ? Apr 29, 2016 17:24

Mr. Crow: May 22, 2008; Snap City mayor for life

Can anybody give me some direction on getting two PCs to communicate via USB? Or at the least one end needs to be USB. Needs to be a Linux and C++ solution (working with an existing simulator but it only runs on the same machine using a virtual usb solution).

Been looking at USB OTG and USB networking among other things but I'm not familiar with USB so I'm just kind of shooting in the dark.

# ? Apr 29, 2016 18:29

Bob Morales: Aug 18, 2006; ~~Just wear the fucking mask, Bob~~

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Mr. Crow posted:

Can anybody give me some direction on getting two PCs to communicate via USB? Or at the least one end needs to be USB. Needs to be a Linux and C++ solution (working with an existing simulator but it only runs on the same machine using a virtual usb solution).

Been looking at USB OTG and USB networking among other things but I'm not familiar with USB so I'm just kind of shooting in the dark.

You'll need to use a 'bridge cable' AFAIK because USB works on the host/target model, not like a serial cable does between two computers. In fact you'll cross power lines or something and could damage the ports.

They used to sell data transfer kits back in the Windows 98/XP days that came with a special cable and special software to do the transfer.

# ? Apr 29, 2016 19:30

Mr. Crow: May 22, 2008; Snap City mayor for life

Bob Morales posted:

You'll need to use a 'bridge cable' AFAIK because USB works on the host/target model, not like a serial cable does between two computers. In fact you'll cross power lines or something and could damage the ports.

They used to sell data transfer kits back in the Windows 98/XP days that came with a special cable and special software to do the transfer.

Right, but I'm having trouble finding anything that doesn't have some propriety software attached to it making it useless for my purposes.

If you or anyone else know of a cable with an API that would be great?

# ? Apr 29, 2016 19:34

JawnV6: Jul 4, 2004; So hot ...

Mr. Crow posted:

Right, but I'm having trouble finding anything that doesn't have some propriety software attached to it making it useless for my purposes.

If you or anyone else know of a cable with an API that would be great?

Would you be comfortable with a COM port on both ends? http://www.ftdichip.com/Products/Cables/USBtoUSB.htm

Honestly I'm curious what you simulator 'device' side looks like if not that.

# ? Apr 29, 2016 19:36

nielsm: Jun 1, 2009

Mr. Crow posted:

Right, but I'm having trouble finding anything that doesn't have some propriety software attached to it making it useless for my purposes.

If you or anyone else know of a cable with an API that would be great?

Your best bet is an USB Ethernet adapter, I'd say.

# ? Apr 29, 2016 19:36

Mr. Crow: May 22, 2008; Snap City mayor for life

JawnV6 posted:

Would you be comfortable with a COM port on both ends? http://www.ftdichip.com/Products/Cables/USBtoUSB.htm

Honestly I'm curious what you simulator 'device' side looks like if not that.

This might work, I'll need to look into it.

Basically on one machine we have app communicating with some device with USB and/or serial connection. For testing the app somebody wrote a simulator, my job was to throw it into a docker container and provide it to the testing department. Unfortunately I guess the serial communication was to slow and it doesn't really behave like the actual device, so they created a virtual USB solution (because of my current problem) but it has to run on the same machine, which was acceptable at the time for developer testing but now it needs to work without modifying the host machine at all, so here I am.

# ? Apr 29, 2016 19:50

nielsm: Jun 1, 2009

I think Linux also has a "USB Gadget" driver which allows you to configure the system as a USB device rather than host, but I don't really know anything about it other than that it exists. It probably requires special hardware; I don't think normal host controllers can be reprogrammed to act as targets instead.

# ? Apr 29, 2016 20:14

Mr. Crow: May 22, 2008; Snap City mayor for life

The virtual USB device was using that but it didn't appear to be on my Ubuntu VM, the app is on a in-house spin of Ubuntu so I thought it was some proprietary thing, didn't look into it.

But the null modem cable solution should work so cheers.

# ? Apr 30, 2016 00:34

22 Eargesplitten: Oct 10, 2010

I'm doing some research on unit testing, and I'm seeing a lot of people talking about how good TDD is. Is it universally accepted as the way to go, or is it another one of the things that people argue over, like most things I ask about?

While I'm asking, I'm working on a game mod. I'm not sure if it's how patched together the engine was from 2001-2010, or if it's the other mods included in the code I'm building off of, but it's taking about an hour to compile. Are there any guides on reducing compile time on a big project like that? Or is it a matter of optimizing each file and project one at a time?

22 Eargesplitten fucked around with this message at 03:17 on May 2, 2016

# ? May 2, 2016 03:14

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

22 Eargesplitten posted:

I'm doing some research on unit testing, and I'm seeing a lot of people talking about how good TDD is. Is it universally accepted as the way to go, or is it another one of the things that people argue over, like most things I ask about?

It's one of those things people argue about. It definitely can help make your development more rigorous (not to mention help ensure that you have unit tests), but it requires a certain mentality of development where you can know that the tests and the spec and what is actually needed are all in agreement. Which can be hard to achieve in many software shops.

# ? May 2, 2016 03:39

raminasi: Jan 25, 2005; a last drink with no ice

22 Eargesplitten posted:

While I'm asking, I'm working on a game mod. I'm not sure if it's how patched together the engine was from 2001-2010, or if it's the other mods included in the code I'm building off of, but it's taking about an hour to compile. Are there any guides on reducing compile time on a big project like that? Or is it a matter of optimizing each file and project one at a time?

If your toolchain has some way of doing precompiled headers (it should), definitely use them. (I'm assuming this is C++; I don't know what else would take that long to compile.)

# ? May 2, 2016 04:42

Plorkyeran: Mar 22, 2007; To Escape The Shackles Of The Old Forums, We Must Reject The Tribal Negativity He Endorsed

TDD is a great way to get better at writing code that can actually be unit tested, so purely for educational purposes I believe everyone should spend a while doing it. It is not the best way to solve every single problem you will ever encounter. I personally only actually follow TDD when I'm writing something with a clearly defined "correct" result but I have no idea on how to even get started on producing that result.

# ? May 2, 2016 05:04

Shaman Tank Spec: Dec 26, 2003; *blep*

My Haskell trials continue.

I'm still working on parsing text and replacing parts of it with other text. The problem I'm wrestling with right now is this:

1. Find out if substring X appears as part of String Y.
2. If it does, pass it on to another list so I can later replace it with another String.

Now, on its surface this is extremely simple and can be achieved with something like a simple list comprehension. For instance:

code:

[x | x <- inputText, isInfixOf x textToCompareAgainst]

will get me results, but it's not good enough because it doesn't for instance account for cases where x is just part of a string I'd want to pass along. For instance, let's consider this:

code:

inputText = John Smith
textToCompareAgainst = We desperately wanted to get John Smith's hat

My list comprehension will get John Smith as a hit, but will only pass on "John Smith", not "John Smith's", which is what I'd want to pick out. I honestly can't figure out any decent way to do this.

E: I'm not married to list comprehensions here, but I find I'm constantly hitting my head against conventions and practises I learned from OOP and which might not work that well for Haskell. My original plan was to write functions which would do a more thorough job of checking against various edge cases but they became horribly unwieldy and monstrous.

Shaman Tank Spec fucked around with this message at 14:52 on May 2, 2016

# ? May 2, 2016 14:40

tazjin: Jul 24, 2015

It seems like you want to do more than what is listed in your requirements. Lets split the problem up:

1. We have an input list of strings :: [String]
2. We have a predicate to filter these strings on :: String -> Bool -- (this would be your isInfixOf or whatever)

Look at 'filter' to do this bit, no list comprehension required.

3. For all strings that pass the filter, you want to retrieve a certain substring. You have a function f :: String -> String to do this

Use 'map' to apply this function to your strings.

It isn't quite clear from your post what the function in 3. does:

code:

s1 = "I like John Smith's hat"
s2 = "LizardsJohn Smith'a balloon"

If the substring you want from s1 is "John Smith's", what is the substring you want from s2?

# ? May 2, 2016 15:04

Shaman Tank Spec: Dec 26, 2003; *blep*

s1: John Smith's
s2: Nothing

So basically I want any suffixes, like possessives, plurals etc.

And yeah, I was originally messing around with trying to combine filter and map, but I guess I couldn't figure out the precise syntax needed to combine flip, map and filter

E: basically here's the whole thing.

1. We have an arbitrary String of text. I was originally planning on splitting this text into smaller Strings through splitOn("\n") but decided against it since it doesn't really do me much good.

2. We have lists of smaller Strings that might be names, objects or places.

3. I want to run checks to see if any names, objects or places from my lists of Strings appear in the long Strings, and if they do, add them to new lists for later post-processing.

This would be easy as heck if my smaller Strings were single words or if I could be sure that they didn't have possessive suffixes, weren't pluralized etc, but that would be too limiting.

Here's a sample of what I'm trying to do:

code:

Input: "John Smith is a warrior who has a Sword of Magical Goblin Slaying and five apples."
Output: "<PERSON>John Smith</PERSON> is a <CLASS>warrior</CLASS> who has a <WEAPON>Sword of Magical Goblin Slaying</WEAPON> and five <ITEM>apples</ITEM>."

Obviously some of these will be harder to check against, like for instance things like "Sword of Magical Goblin Slaying", but I figure I can just do more in depth pattern matches for those.

Shaman Tank Spec fucked around with this message at 16:03 on May 2, 2016

# ? May 2, 2016 15:51

baquerd: Jul 2, 2007; by FactsAreUseless

Der Shovel posted:

1. We have an arbitrary String of text. I was originally planning on splitting this text into smaller Strings through splitOn("\n") but decided against it since it doesn't really do me much good.

2. We have lists of smaller Strings that might be names, objects or places.

3. I want to run checks to see if any names, objects or places from my lists of Strings appear in the long Strings, and if they do, add them to new lists for later post-processing.

Here's a sample of what I'm trying to do:
code:
Input: "John Smith is a warrior who has a Sword of Magical Goblin Slaying and five apples."
Output: "<PERSON>John Smith</PERSON> is a <CLASS>warrior</CLASS> who has a <WEAPON>Sword of Magical Goblin Slaying</WEAPON> and five <ITEM>apples</ITEM>."
Obviously some of these will be harder to check against, like for instance things like "Sword of Magical Goblin Slaying", but I figure I can just do more in depth pattern matches for those.

Depending on your dictionary and input size, brute forcing it by doing regex searches over the entire string for each dictionary item may be entirely feasible.

# ? May 2, 2016 16:12

Shaman Tank Spec: Dec 26, 2003; *blep*

baquerd posted:

Depending on your dictionary and input size, brute forcing it by doing regex searches over the entire string for each dictionary item may be entirely feasible.

I know of regex but don't really have any experience with it.

E: and honestly I'd prefer to learn the "proper" way of doing it with Haskell, since I figure one of the cases where Haskell might actually be useful for me is for making tools like this.

Shaman Tank Spec fucked around with this message at 16:35 on May 2, 2016

# ? May 2, 2016 16:20

22 Eargesplitten: Oct 10, 2010

raminasi posted:

If your toolchain has some way of doing precompiled headers (it should), definitely use them. (I'm assuming this is C++; I don't know what else would take that long to compile.)

Thanks. Yeah, it's C++. I'm using VS2013, so it looks like it should work.

e: Well, there are 27 .pch files already. But there are also 6462 .h files :aaa:

22 Eargesplitten fucked around with this message at 17:31 on May 2, 2016

# ? May 2, 2016 17:16

Sinestro: Oct 31, 2010; The perfect day needs the perfect set of wheels.

/\s(John Smith\S*)/ is all you need, it's a very simple regex problem. Choosing the right tool isn't a sign of weakness.

# ? May 2, 2016 17:24

JawKnee: Mar 24, 2007; You'll take the ride to leave this town along that yellow line

Der Shovel posted:

I know of regex but don't really have any experience with it.

E: and honestly I'd prefer to learn the "proper" way of doing it with Haskell, since I figure one of the cases where Haskell might actually be useful for me is for making tools like this.

If you have a set of substrings that you're checking for in larger strings then regex would be an incredibly easy and fast way of doing so. That being said, if you don't know all the possible substrings ahead of time, or the specific construction of the larger strings (ie: where substrings might appear, common elements that are always, or are often in place) then I don't know how much use it would be.

# ? May 2, 2016 20:13

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

Regexes are an incredibly powerful tool for string manipulation, searching, and extraction. They are consequently the "correct" way to do pretty much anything more complicated than "is this substring in this string", "take the first/last N characters of this string", and "split this string into tokens based on this delimiter".

It's worth your time to learn to use regexes, just like it's worth your time to learn to use databases, write functional code, write unit tests, etc. Even if they aren't a tool you use often, knowing that they're available and being able to recognize what they can do can/will save you a ton of time.

# ? May 2, 2016 20:22

Shaman Tank Spec: Dec 26, 2003; *blep*

JawKnee posted:

That being said, if you don't know all the possible substrings ahead of time, or the specific construction of the larger strings (ie: where substrings might appear, common elements that are always, or are often in place) then I don't know how much use it would be.

That's part of the problem, I don't know that. I can probably hack together a solution that'll get me 95% of the way there and will just not work for edge cases like plurals and possessives.

I've been thinking about this on and off all evening long, and I'm wondering if it wouldn't be better to make custom functions for every possible type.

# ? May 2, 2016 20:28

dupersaurus: Aug 1, 2012; Futurism was an art movement where dudes were all 'CARS ARE COOL AND THE PAST IS FOR CHUMPS. LET'S DRAW SOME CARS.'

I know I know the answer to this, but I'm drawing a huge blank. What's the name of this pattern?

code:

class A { }

var classReference = A;
var classObject = new classReference();

That is, being able to set a variable to (or make a function that returns) a class type, and then use calls to that variable to make instances of the class?

# ? May 2, 2016 20:40

TooMuchAbstraction: Oct 14, 2012; I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.; Fun Shoe

That's just functions as first-class objects (or "first-class functions"). I guess using that specifically for an object constructor is like having an extremely bare-bones factory.

# ? May 2, 2016 20:43

Shaman Tank Spec: Dec 26, 2003; *blep*

OK, Haskell and -- possibly -- folds.

Let's say I want to apply a function, in this case replace (replace oldValue newValue where_to_replace), several times to the same String with different values for oldValue and newValue, I'd use a fold, right?

Because if I used map, I wouldn't end up making n changes to my target string, I'd end up with n instances of my target string each with one change made to it, right? But is even a fold the right thing to use here, or should I try to make some kind of recursive function that is run n times on the same material, manually iterating through my lists of values?

# ? May 2, 2016 21:44

nielsm: Jun 1, 2009

Yes if you have a list of (oldValue,newValue) tuples, and a single string, and you want to use the tuples as replacements in the string, folding would be a way of doing it.

# ? May 2, 2016 21:57

the talent deficit: Dec 20, 2003; self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture

foldl is almost always implemented recursively. (foldr sometimes gets weird)

# ? May 3, 2016 07:20

TheresaJayne: Jul 1, 2011

Mr. Crow posted:

Can anybody give me some direction on getting two PCs to communicate via USB? Or at the least one end needs to be USB. Needs to be a Linux and C++ solution (working with an existing simulator but it only runs on the same machine using a virtual usb solution).

Been looking at USB OTG and USB networking among other things but I'm not familiar with USB so I'm just kind of shooting in the dark.

You could always get USB-> Serial and then join the 2 with a serial crossover cable

or how about USB-> Ethernet and put a small hub in the middle

# ? May 3, 2016 07:26

feedmegin: Jul 30, 2008

TooMuchAbstraction posted:

Regexes are an incredibly powerful tool for string manipulation, searching, and extraction. They are consequently the "correct" way to do pretty much anything more complicated than "is this substring in this string", "take the first/last N characters of this string", and "split this string into tokens based on this delimiter".

Well, to a point. Insert 'how do I parse HTML with regexes' zalgo page here.

# ? May 3, 2016 14:44

Shaman Tank Spec: Dec 26, 2003; *blep*

Here's a small question.

Does Haskell offer any kind of syntactic sugar for this type of situation:

code:

if (a == b) or (a == c) or (a == d) then x

I'm fairly convinced I saw some way of having both b, c and d represented in a more compact way but I'm not having any luck finding anything on it right now and I'm starting to think I might have been imagining things.

# ? May 3, 2016 14:45

Adbot: ADBOT LOVES YOU

# ? May 30, 2024 20:42

Asymmetrikon: Oct 30, 2009; I believe you're a big dork!

code:

if (a `elem` [b, c, d]) then x

# ? May 3, 2016 14:59

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »