Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »

Hughlander: May 11, 2005

sarehu posted:

I'm pretty sure at my last job at some point we were sending JSON over Protobuf. I think now they're serializing protobufs to JSON.

Probably the horror thread but I've worked on the following: MySQL as a key value store with compressed JSON deserialized to a Java class encoded into a protobuf message. Sent over the wire wrapped in JSON. Deserialized into an NSDictionary, taken out of the protobuf to a C++ class then a Lua table is formed from that where it is consumed.

# ? Sep 11, 2015 05:07

Adbot: ADBOT LOVES YOU

# ? May 31, 2024 10:59

TheresaJayne: Jul 1, 2011

Hughlander posted:

Probably the horror thread but I've worked on the following: MySQL as a key value store with compressed JSON deserialized to a Java class encoded into a protobuf message. Sent over the wire wrapped in JSON. Deserialized into an NSDictionary, taken out of the protobuf to a C++ class then a Lua table is formed from that where it is consumed.

Welcome to corporate development

and you are right the horror thread is a scary read sometimes, (At least i have NEVER seen any of my code in there)(Yet!)

# ? Sep 11, 2015 05:51

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

JawnV6 posted:

What's eating me is C being referenced in the same line as --C. What does a javascript call a sequence point?

JavaScript has strict left-to-right evaluation. --C + --C returns 2 * C - 3 (for the original value of C and ignoring some FP subtlety).

# ? Sep 12, 2015 02:24

Blue On Blue: Nov 14, 2012

I'm really not sure if this is even a programming question, but it seemed like the best place to start

I am looking for a way to setup a 'scrubber' if you will, that searches for news articles, forum posts, and general internet traffic, relying on key phrases.

I'm sure things like this exist for the big guys like the FBI and NSA etc, but I just want something that can tell me if 'John Smith protest tomorrow' was posted in the last 24 hours on any of the local activist forums or newsgroups

# ? Sep 12, 2015 03:47

Snak: Oct 10, 2005; I myself will carry you to the Gates of Valhalla...
You will ride eternal,
shiny and chrome.; Grimey Drawer

Not sure if this is a good place to ask, but I need to "Perform Detailed Requirements Analysis Process" for a school project and I am very lost. Does anyone have pointers or know of good resources that show examples or templates? I tried to read about IEEE standards, but it was very daunting and most of what I found was more about the purposes of the standards rather than how to actually make a document in compliance with them.

If there is a better place for me to ask about this topic, I would appreciate being pointed in that direction.

# ? Sep 12, 2015 03:56

Centripetal Horse: Nov 22, 2009; Fuck money, get GBS

This could have bought you a half a tank of gas, lmfao -
Love, gromdul

Sappo569 posted:

I'm really not sure if this is even a programming question, but it seemed like the best place to start

I am looking for a way to setup a 'scrubber' if you will, that searches for news articles, forum posts, and general internet traffic, relying on key phrases.

I'm sure things like this exist for the big guys like the FBI and NSA etc, but I just want something that can tell me if 'John Smith protest tomorrow' was posted in the last 24 hours on any of the local activist forums or newsgroups

At the simplest, you can just set up a cron job that grabs pages using curl or wget, and then greps them for keywords or phrases. There are a lot of reasons why that's not a great solution. If you want something more powerful, you could look into Selenium, which allows for headless browser interaction. That's definitely on the programming side of things, though.

# ? Sep 12, 2015 05:10

Blue On Blue: Nov 14, 2012

Centripetal Horse posted:

At the simplest, you can just set up a cron job that grabs pages using curl or wget, and then greps them for keywords or phrases. There are a lot of reasons why that's not a great solution. If you want something more powerful, you could look into Selenium, which allows for headless browser interaction. That's definitely on the programming side of things, though.

Ok let me modify my original post a bit

I'm not a programmer, nor do I want to learn how to program. Not my bag.

I guess what I was really meaning to ask, is there a program that does this, or some other solution where I don't have to get my hands into the guts?

# ? Sep 12, 2015 05:20

Linear Zoetrope: Nov 28, 2011; A hero must cook

If the boards have RSS feeds, you could probably just use some sort of RSS filter like this. Disclaimer: I have never used an RSS filter, for all I know that one will infect your computer with every virus known to man and several extra.

# ? Sep 12, 2015 05:23

Impotence: Nov 8, 2010; Lipstick Apathy

Sappo569 posted:

I'm really not sure if this is even a programming question, but it seemed like the best place to start

I am looking for a way to setup a 'scrubber' if you will, that searches for news articles, forum posts, and general internet traffic, relying on key phrases.

I'm sure things like this exist for the big guys like the FBI and NSA etc, but I just want something that can tell me if 'John Smith protest tomorrow' was posted in the last 24 hours on any of the local activist forums or newsgroups

https://www.google.com/alerts with a bunch of inurl: and site: keywords

# ? Sep 12, 2015 05:26

Blue On Blue: Nov 14, 2012

Biowarfare posted:

https://www.google.com/alerts with a bunch of inurl: and site: keywords

That might be the ticket.

Thanks!

# ? Sep 12, 2015 05:31

mystes: May 31, 2006

Biowarfare posted:

https://www.google.com/alerts with a bunch of inurl: and site: keywords

Doesn't Google alerts really only work on Google News now, rather than the web in general?

# ? Sep 12, 2015 05:44

TheresaJayne: Jul 1, 2011

Sappo569 posted:

Ok let me modify my original post a bit

I'm not a programmer, nor do I want to learn how to program. Not my bag.

I guess what I was really meaning to ask, is there a program that does this, or some other solution where I don't have to get my hands into the guts?

the NSA pull a record of every blog post and every page on the internet, they then us something like hadoop to store it all then do data analysis on it for trends and connections.

I worked for a company that looked for telecoms fraud, and I had to maintain and improve their fraud detection software, it was quite simple once you understood what you were looking for.

Sudden increase of calls to certain numbers (competition lines when they knew they couldnt afford the bill)
more than one number on the same hardware,
more than one hardware on the same number - both indicitive of cloned sims - the prior - cloned sims as an international call scam, they would sellotape 2 phones against each other 1 stolen sim and one local sim.

there was also contact tracking which was the scary part,
where they track every contact between their prey and see who gets pulled into the net...

but at the end of it all, its statistics

# ? Sep 12, 2015 06:20

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Does anybody have any recommendations for job or task queues? beanstalkd seems like mostly what we want, but it seems to be very poorly maintained.

# ? Sep 12, 2015 06:27

TheresaJayne: Jul 1, 2011

Suspicious Dish posted:

Does anybody have any recommendations for job or task queues? beanstalkd seems like mostly what we want, but it seems to be very poorly maintained.

What exactly are you trying to do, I have used the Quartz scheduler package in java and have used many Software development management systems like Trello and its ilk

# ? Sep 12, 2015 06:35

Gravity Pike: Feb 8, 2009; I find this discussion incredibly bland and disinteresting.

Yeah, there are lots of options, depending on what you want to do. I have also used Quartz, and, while I'm not always thrilled with it, it does the task adequately. Depending on the character of the work I need to do, I've also used cron, or pulling items off of a message queue and running them as fast as I can. (SQS, ActiveMQ, RabbitMQ, depending.)

# ? Sep 12, 2015 22:14

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

While not a completely original idea, I can't actually find anything out there that does exactly what I need it to. Basically, a job queue which can also collect logs and store everything in a persistent way, so we can look back at old jobs later. Jobs can be handled by multiple workers in any language (we have at least three different languages we want to support), with the option to parallelize this out later. Imagine build system archiving, that sort of thing. Doesn't seem overly ridiculous or complicated, but there isn't anything off-the-shelf I can find.

I might just write one myself (famous last words).

# ? Sep 13, 2015 07:04

Amberskin: Dec 22, 2013; We come in peace! Legit!

Suspicious Dish posted:

Does anybody have any recommendations for job or task queues? beanstalkd seems like mostly what we want, but it seems to be very poorly maintained.

Have you taken a look at torque? (http://www.adaptivecomputing.com/products/open-source/torque/). Depending in what are you actually trying to achieve, it could be overkill, but it works and it is relatively easy to set up.

# ? Sep 13, 2015 12:38

hooah: Feb 6, 2006; WTF?

I couldn't find an image processing/graphics thread, so I'll ask here. I'm taking a computer vision course, and I'm having a bit of a hard time understanding how the book's defining a perimeter:

quote:

...the perimeter of a region is the set of its interior border pixels. A pixel of a region is a border pixel if it has some neighboring pixel that is outside the region. When 8-connectivity is used to determine whether a pixel inside the region is connected to a pixel outside the region, the resulting set of perimeter pixels is 4-connected. [and vice versa]

I don't understand the part about 8-connectivity resulting in a 4-connected perimeter. I tried drawing a couple examples, and all the border pixels were connected to a neighboring pixel outside the region in a 4-connected way; 8-connectivity wasn't required.

# ? Sep 13, 2015 17:43

Nippashish: Nov 2, 2005; Let me see you dance!

e:no this was not right

# ? Sep 13, 2015 18:07

Linear Zoetrope: Nov 28, 2011; A hero must cook

hooah posted:

I couldn't find an image processing/graphics thread, so I'll ask here. I'm taking a computer vision course, and I'm having a bit of a hard time understanding how the book's defining a perimeter:

I don't understand the part about 8-connectivity resulting in a 4-connected perimeter. I tried drawing a couple examples, and all the border pixels were connected to a neighboring pixel outside the region in a 4-connected way; 8-connectivity wasn't required.

Instead of thinking of shapes and perimeters, maybe try and think of it like a tile-based graph. I'm in some room that's surrounded by walls. Since this is tile-based, a wall is simply a tile that's marked "impassable". You can imagine this like an old Zelda game or something, if you're familiar with that.

If I can only move in four directions (which is the same as saying the region is 4-connected), you don't need walls in the corners of the room. Because if I'm in the top left the walls to my left and up are already blocking me, so a wall tile to my upper left would be superfluous. This is the same as saying the perimeter is 8-connected, because the wall to my left is connected diagonally to the wall above me.

This goes the other way as well, if I can move in 8 directions, there needs to be a wall to my upper left or I could magically get out of the room just by moving diagonally. This is the same as saying the wall/perimeter is 4-connected because to fully enclose the room tiles can't rely on you not being able to move diagonally, and thus the wall tiles can only connect in 4 directions (up-down and left-right), so a corner tile needs (at least) one tile that connects horizontally and one tile that connects vertically.

A perimeter can be viewed as the minimum set of tiles required to wall-in a room so that I can't escape. So if I can only move 4 directions, you need fewer tiles because they're 8-connected. But if I can move in 8 directions you need those extra corner tiles so I can't escape because they're 4-connected.

Linear Zoetrope fucked around with this message at 18:30 on Sep 13, 2015

# ? Sep 13, 2015 18:24

nielsm: Jun 1, 2009

hooah posted:

I couldn't find an image processing/graphics thread, so I'll ask here. I'm taking a computer vision course, and I'm having a bit of a hard time understanding how the book's defining a perimeter:

I don't understand the part about 8-connectivity resulting in a 4-connected perimeter. I tried drawing a couple examples, and all the border pixels were connected to a neighboring pixel outside the region in a 4-connected way; 8-connectivity wasn't required.

I think the intended meaning is that within the set of border pixels, every border pixel will have 4-connecticity to another border pixel. Probably unless it's a single pixel shape.

The left is 4-connected perimeter. The right is 8-connected perimeter.
In the left, the perimeter pixels are disjoint if you look at them as 4-connected to each other, but a single shape if you consider them with 8-connectivity.
In the right, the perimeter pixels can be considered a single shape with either 4- or 8-connectivity, except for the two outlier pixels in the top left.

# ? Sep 13, 2015 18:32

Linear Zoetrope: Nov 28, 2011; A hero must cook

Yeah, aside from a couple stray pixels those pictures represent it pretty well. In my post I was trying to make ASCII examples and it just wasn't working, and I was too lazy to get out GIMP/MSPaint.

# ? Sep 13, 2015 18:35

Suspicious Dish: Sep 24, 2011; 2020 is the year of linux on the desktop, bro; Fun Shoe

Amberskin posted:

Have you taken a look at torque? (http://www.adaptivecomputing.com/products/open-source/torque/). Depending in what are you actually trying to achieve, it could be overkill, but it works and it is relatively easy to set up.

That looks a bit like mesos, which is that it's a scheduler system for distributing tasks across a giant cluster. I'm architecting a system that's sort of like a web crawler, however it's (mostly) manually triggered.

A job might be "crawl this blog", which then goes out, fetches the blog contents, which then schedules more jobs for, say, other links in the blog post, or, embedded media. Images might require thumbnailing, videos might require transcoding, etc.. And it would be nice if we could store the logs from each job for eternity, since we might want to investigate the logs for an asset that was produced two years back.

So we want a queue of jobs with workers that can come and mash on incoming jobs, and also the ability to persist and store these jobs and their logs forever.

The word "job" is really confusing, I'm aware, since other people use might it for "long-running process". Find me a better word if you want. It's not necessarily distributing a process across a large cluster of machines.

# ? Sep 13, 2015 18:47

hooah: Feb 6, 2006; WTF?

nielsm posted:

I think the intended meaning is that within the set of border pixels, every border pixel will have 4-connecticity to another border pixel. Probably unless it's a single pixel shape.

The left is 4-connected perimeter. The right is 8-connected perimeter.
In the left, the perimeter pixels are disjoint if you look at them as 4-connected to each other, but a single shape if you consider them with 8-connectivity.
In the right, the perimeter pixels can be considered a single shape with either 4- or 8-connectivity, except for the two outlier pixels in the top left.

Ohh, that helps a lot. Thanks!

# ? Sep 13, 2015 21:54

an skeleton: Apr 23, 2012; scowls @ u

There is some Really Bad Code (TM) that I am refactoring at work. The code is frontend Angular code for a web app that configures a hardware system, in short. There are no great requirements or specs to go buy except an older version of the app and some really old user manual documentation. Mix this with the fact that some modules (directives, services, javascript packages) have changed and refactoring is happening at the same time as transitioning to those changes.

When code/logic/requirements are really bad in this way, I struggle for longer than I would like to in getting the old functions up to snuff. First I usually try to make small changes in hopes that that will fix it. Then I struggle to understand what the code was supposed to do in the first place. Then I make some bigger changes, moving code into subfunctions (the code in its old form is not even close to unit testable) so that each part is easier to understand. Rinse and repeat all of that, over and over again. It ends up working, but I feel like it takes me forever, sometimes up to 1-2 hours to fix one big monster function. Granted, I'm pretty confident in the results and they usually end up being much more unit-testable, modular, etc.

Just wondering if this reflects anyone else's experience, and if they have any experience or advice in this regard they'd like to share. I think I'm working on pace with my coworkers but it just feels bad spending this much time refactoring.

# ? Sep 14, 2015 21:00

raminasi: Jan 25, 2005; a last drink with no ice

I think that experience is pretty typical.

# ? Sep 14, 2015 21:07

Skandranon: Sep 6, 2008; fucking stupid, dont listen to me

I find it helps if I first have an idea how I'd like it to look in the future state, and instead of trying to fix the old code, start from scratch in the new state and copy over the functionality you'd like to keep.

However, maintaining original functionality can be difficult if you don't have any unit tests, and if dealing with pure JS, you also have no compile time checks. In that case, yeah, it can be a real headache, but try not to get bogged down chasing every single thing that sucks, focus on one change at a time, even if it means touching the same piece of code more than once.

# ? Sep 14, 2015 21:57

JawnV6: Jul 4, 2004; So hot ...

an skeleton posted:

It ends up working, but I feel like it takes me forever, sometimes up to 1-2 hours to fix one big monster function. Granted, I'm pretty confident in the results and they usually end up being much more unit-testable, modular, etc.

Is there any possibility of adding precision to "one big monster function"? Is that 1kloc, 1 page, 1 semicolon? For something like code review of C I assume a professional can get through 100~200 loc/hour, with unreviewed code having 1 bug/20loc on average. I assume there's some equivalent known-lovely metrics for JS.

# ? Sep 14, 2015 22:20

an skeleton: Apr 23, 2012; scowls @ u

JawnV6 posted:

Is there any possibility of adding precision to "one big monster function"? Is that 1kloc, 1 page, 1 semicolon? For something like code review of C I assume a professional can get through 100~200 loc/hour, with unreviewed code having 1 bug/20loc on average. I assume there's some equivalent known-lovely metrics for JS.

The function that took me the longest in the past week or so started at around 120loc, but it was tied up in all kinds of state and dependent on various data whose format had changed from the old app to the new app. The data (for every fn on this page) is parsed and formatted in a very particular way, with lots of split()s and nested forEach-es among other things that were redundant and unnecessary.

# ? Sep 15, 2015 00:05

Sex Tragedy: Jan 28, 2007; father of three with an extra large butt

Does anyone know of an introduction to nurbs?

# ? Sep 15, 2015 02:27

Hughmoris: Apr 21, 2007; Let's go to the abyss!

My work has a poo poo application that often times does not fully close itself when the User clicks "close". It will close the application window, but the application process will still be hanging around and causing headaches. I've created an AutoIt script that will look for this specific scenario and kill the application process. A potential problem with deploying my script is that it needs to run locally in the background 24/7 365.

Here is the rough pseudocode:

code:

While True:
     If application.window = CLOSED and application.process = RUNNING:
             kill application.process
     Sleep(10 seconds)
Wend

Having a script like this loop indefinitely, is there something I should be doing about flushing memory or releasing all variables or some sort of maintenance at the end of each loop, so my script process doesn't grow into some monstrosity of a memory leak?

Hughmoris fucked around with this message at 05:27 on Sep 15, 2015

# ? Sep 15, 2015 04:58

piratepilates: Mar 28, 2004; So I will learn to live with it. Because I can live with it. I can live with it.

Hughmoris posted:

My work has a poo poo application that often times does not fully close itself when the User clicks "close". It will close the application window, but the application process will still be hanging around and causing headaches. I've created an AutoIt script that will look for this specific scenario and kill the application process. A potential problem with deploying my script is that it needs to run locally in the background 24/7 365.

Here is the rough pseudocode:
Visual Basic .NET code:
While True:
     If application.window = CLOSED and application.process = RUNNING:
             kill application.process
     Sleep(10 seconds)
Wend
Having a script like this loop indefinitely, is there something I should be doing about flushing memory or releasing all variables or some sort of maintenance at the end of each loop, so my script process doesn't grow into some monstrosity of a memory leak?

If it's in VB (a garbage collected language) and it's not allocating good amounts of memory (it shouldn't be) and it isn't holding on to them indefinitely (why would it be) then it should be fine, memory leaks tend to happen when you hold on to references to memory past when the memory will ever be used. As long as any reference you are holding goes out of scope after the memory it is referencing is not being used then it shouldn't matter.

However that seems like a weird thing to have to do, instead of figuring out why the application doesn't gracefully shut down in the first place. On the other hand it's VB so who knows if you really want to go down that rabbit hole anyway.

# ? Sep 15, 2015 05:21

Linear Zoetrope: Nov 28, 2011; A hero must cook

piratepilates posted:

If it's in VB (a garbage collected language) and it's not allocating good amounts of memory (it shouldn't be) and it isn't holding on to them indefinitely (why would it be) then it should be fine, memory leaks tend to happen when you hold on to references to memory past when the memory will ever be used. As long as any reference you are holding goes out of scope after the memory it is referencing is not being used then it shouldn't matter.

It can depend on the language, though. I don't know if it's still true, but at least in 1.3 Go (a GC'd language) free'd segments of goroutine stack memory locally but never released it back to the OS, though a lot of that is just virtual memory.

It's unlikely that even if that was the case in VB it would allocate enough to be an issue, since it would just re-use the memory it's already allocated. You'd have to allocate slightly more each time or accidentally allocate a giant chunk once for it to be a problem.

# ? Sep 15, 2015 05:27

Hughmoris: Apr 21, 2007; Let's go to the abyss!

piratepilates posted:

If it's in VB (a garbage collected language) and it's not allocating good amounts of memory (it shouldn't be) and it isn't holding on to them indefinitely (why would it be) then it should be fine, memory leaks tend to happen when you hold on to references to memory past when the memory will ever be used. As long as any reference you are holding goes out of scope after the memory it is referencing is not being used then it shouldn't matter.

However that seems like a weird thing to have to do, instead of figuring out why the application doesn't gracefully shut down in the first place. On the other hand it's VB so who knows if you really want to go down that rabbit hole anyway.

Sorry, my script is in AutoIt which is BASIC, not VB. I labeled my code block wrong. I can't say I'm well versed enough in AutoIt or memory management to say if my script releases items at the end of each loop. I'll try and post my code tomorrow when I get to work.

The problematic application is actually a pretty major piece of software. My company has raised a HUGE stink about it but the vendor states that our issue can't be fixed because its "hardcoded", and we essentially just need to deal with it. This entails telling all of our users not to click the shiny CLOSE APPLICATION button, but instead click on FILE->EXIT (no one will do this).

# ? Sep 15, 2015 05:33

Xerophyte: Mar 17, 2008; This space intentionally left blank

Sex Tragedy posted:

Does anyone know of an introduction to nurbs?

I don't know nearly as much about the topic as I probably should, but try the course notes for NURBS: A Primer from siggraph 2002? It looks like it does what it says on the tin, at least.

Looking in Real-Time Rendering and they recommend NURBS: From Projective Geometry to Practical Use (amazon), An Introduction to NURBS: With Historical Perspective (amazon) and The NURBS Book (free, it seems), none of which I've read.

# ? Sep 15, 2015 09:49

The MUMPSorceress: Jan 6, 2012; ^SHTPSTS; Gary’s Answer

Hughmoris posted:

Sorry, my script is in AutoIt which is BASIC, not VB. I labeled my code block wrong. I can't say I'm well versed enough in AutoIt or memory management to say if my script releases items at the end of each loop. I'll try and post my code tomorrow when I get to work.

The problematic application is actually a pretty major piece of software. My company has raised a HUGE stink about it but the vendor states that our issue can't be fixed because its "hardcoded", and we essentially just need to deal with it. This entails telling all of our users not to click the shiny CLOSE APPLICATION button, but instead click on FILE->EXIT (no one will do this).

Lol. Is it like a hard-coded collapse to tray or something? If their app is hard-coded to lose all UI elements but keep running, that's some ghetto poo poo or they're lieing because they don't feel like fixing it. Either way, a call from one of your execs to their exec should sort it out. And if it doesn't, you need a new vendor yesterday.

# ? Sep 15, 2015 13:31

Sex Tragedy: Jan 28, 2007; father of three with an extra large butt

Xerophyte posted:

none of which I've read.

lol

thanks

# ? Sep 16, 2015 07:23

hooah: Feb 6, 2006; WTF?

I have some 2-dimensional data in Matlab. I want to flip it vertically about the "center". That is, if the highest point's y-value is 6 and the lowest point's y-value is 1, a point which has an original y-value of 4 would have its value changed to 3. Is there a non-loop way to do this?

# ? Sep 16, 2015 14:46

nielsm: Jun 1, 2009

hooah posted:

I have some 2-dimensional data in Matlab. I want to flip it vertically about the "center". That is, if the highest point's y-value is 6 and the lowest point's y-value is 1, a point which has an original y-value of 4 would have its value changed to 3. Is there a non-loop way to do this?

y = MAX - x + MIN

# ? Sep 16, 2015 15:17

Adbot: ADBOT LOVES YOU

# ? May 31, 2024 10:59

hooah: Feb 6, 2006; WTF?

nielsm posted:

y = MAX - x + MIN

I don't entirely follow. Say my data is in an array called Data. So, flipped_data = MAX - Data + MIN doesn't seem like it would quite work. I suppose I could do it in two lines, the first copying in the x values and the second doing what you suggest.

# ? Sep 16, 2015 15:32

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »