Ask General Programming Questions Not Worth Their Own Thread

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »

Slimy Hog: Apr 22, 2008

Blinkz0rz posted:

This is the basic use case for an ELK stack

Extract, Load...... Kill?

# ? Jun 4, 2020 03:22

Adbot: ADBOT LOVES YOU

# ? May 30, 2024 22:37

Hadlock: Nov 9, 2004

Elasticsearch, Logstash, and Kibana

Dead reliable but Kibana wants $16K/year for SSO integration at the lowest tier

# ? Jun 4, 2020 03:28

redleader: Aug 18, 2005; Engage according to operational parameters

Slimy Hog posted:

Extract, Load...... Kill?

be still my beating heart

# ? Jun 4, 2020 04:05

Slimy Hog: Apr 22, 2008

redleader posted:

be still my beating heart

I think that's the idea

# ? Jun 4, 2020 07:20

TheIncredulousHulk: Sep 3, 2012

I am just a babby so this might be a real dumb question but does anybody mind explaining how stdin/stdout works to me? I have been practicing algorithms a shitload lately and wanted to try the Google competition next week using Python but when I tried a practice problem, I had no idea how to actually put the input into the function I wrote or presumably return it

# ? Jun 7, 2020 03:22

Look Around You: Jan 19, 2009

TheIncredulousHulk posted:

I am just a babby so this might be a real dumb question but does anybody mind explaining how stdin/stdout works to me? I have been practicing algorithms a shitload lately and wanted to try the Google competition next week using Python but when I tried a practice problem, I had no idea how to actually put the input into the function I wrote or presumably return it

So stdin/stdout/stderr are pre-opened data streams that are hooked up when your program/script runs. By default, stdin comes from the console and stdout outputs to the console. This can be changed on the command line when you run your script.*

I'd take a look at the Python builtin io library documentation to get a feel for how to work with them (the definitions for stdin/stdout/stderr live in the sys package though) , but basically they're handled like regular stream files (that is, you shouldn't be able to seek in them). You can read from stdin and write to stdout/stderr as though they're normal textual files, and can get binary input/output per the instructions in the sys package documentation.

e:
* You can do this by using the > and < characters on the unix/linux/mac shell. Example: python3 myscript.py < my_stdin.txt > my_stdout.txt

# ? Jun 7, 2020 04:00

huhu: Feb 24, 2006

For those that use Windows with the Linux subsystem - have you run into weird file/folder issues with VS Code? I've been developing on Mac + VS Code for the past two years and am moving back over to a Windows machine. I've had things where I've lost complete control of permission for folders. I also had phantom folders - doing an ls said they didn't exist but VS Code wouldn't let me create the folder because it existed. Twice now I've rm -rf'd different repos and started again.

# ? Jun 7, 2020 17:31

TheIncredulousHulk: Sep 3, 2012

Look Around You posted:

So stdin/stdout/stderr are pre-opened data streams that are hooked up when your program/script runs. By default, stdin comes from the console and stdout outputs to the console. This can be changed on the command line when you run your script.*

I'd take a look at the Python builtin io library documentation to get a feel for how to work with them (the definitions for stdin/stdout/stderr live in the sys package though) , but basically they're handled like regular stream files (that is, you shouldn't be able to seek in them). You can read from stdin and write to stdout/stderr as though they're normal textual files, and can get binary input/output per the instructions in the sys package documentation.

e:
* You can do this by using the > and < characters on the unix/linux/mac shell. Example: python3 myscript.py < my_stdin.txt > my_stdout.txt

Thanks. I'm still kind of confused about what to actually do with the input to render it usable in my own code but I think I have enough to go off of now that I can gently caress around for a while and hopefully get it

# ? Jun 7, 2020 23:06

Mr Shiny Pants: Nov 12, 2012

TheIncredulousHulk posted:

Thanks. I'm still kind of confused about what to actually do with the input to render it usable in my own code but I think I have enough to go off of now that I can gently caress around for a while and hopefully get it

Get input from the user is a big one.

# ? Jun 8, 2020 04:50

baka kaba: Jul 19, 2003; PLEASE ASK ME, THE SELF-PROFESSED NO #1 PAUL CATTERMOLE FAN IN THE SOMETHING AWFUL S-CLUB 7 MEGATHREAD, TO NAME A SINGLE SONG BY HIS EXCELLENT NU-METAL SIDE PROJECT, SKUA, AND IF I CAN'T PLEASE TELL ME TO
EAT SHIT

TheIncredulousHulk posted:

Thanks. I'm still kind of confused about what to actually do with the input to render it usable in my own code but I think I have enough to go off of now that I can gently caress around for a while and hopefully get it

stdin and stdout are both file objects - they act like files. You can read from stdin and write to stdout using the usual file access functions - they're the "standard" IO channels for your program. By default stdin is mapped to your keyboard and stdout is your console/terminal window

input and print are convenience functions for reading from stdin and writing to stdout respectively (you can specify another file object to print to though if you want). So if you want to get some user input, call input and the program will block (stop doing anything) until you enter a line. (think of it like it's waiting to read the next line of a file, but that line hasn't been provided yet)

But the thing about having standard IO interfaces like this is you can connect them to other stuff, and so long as it follows the "file-like object" rules it'll all Just Work. Which is why you can call a python script from the command line and direct a file into it, like python coolprogram.py < rad_data.txt, or pipe the output of one program into the input of another

if you want to read keypresses and not block waiting for one, that's trickier and I think you're gonna have to pick from a bunch of solutions!

# ? Jun 8, 2020 19:02

nielsm: Jun 1, 2009

TheIncredulousHulk posted:

Thanks. I'm still kind of confused about what to actually do with the input to render it usable in my own code but I think I have enough to go off of now that I can gently caress around for a while and hopefully get it

If by "render it usable in my own code" you mean parsing input, i.e. turning it into some values in variables, yes that's the big problem with lots of solutions. You mainly need to decide what form you want your input to be in, and then your input parser/reader will depend on that choice. If you need to read an existing format maybe there is a library you can use. (E.g. XML, JSON, YAML, INI, etc.) If you need to process a lot of values in vaguely tabular format, maybe CSV is appropriate. If you need to perform a sequence of actions, maybe a command-parser "conversational style" is needed.

# ? Jun 8, 2020 21:40

TheIncredulousHulk: Sep 3, 2012

Thanks for the tips all, I get what I'm supposed to be doing now

# ? Jun 8, 2020 23:17

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

I have an ANTLR4 question. I finally have run into a code generation problem in my visitor where I need to know the index where my current block of code started. At the point that I need this, I have already generated that so it's been established. The problem with the visitors is I don't know how I can pass that along all the subcontexts that are getting visited from the point where that happened. I haven't had to add any state to the visitor so far and I feel like it would be awkward if I had to add a stack to track these, along with all the trimmings to push and pop it.

When one's mind goes towards a stack then it's a surefire time to recurse, but I don't think I can alter these signatures for all the Visit() calls unless I alter them all and ensure I always follow that chain. There's a lot of them and I expect I'd gently caress that up at some point. I am wondering if I can add some metadata to the contexts when I visit them so I can use the usual visit calls. Is there some aspect to the contexts that are getting passed around I can take advantage of for this?

# ? Jun 9, 2020 08:52

ceebee: Feb 12, 2004

Would anyone be able to point me in the right direction to find a reliable and trustworthy web developer with experience in things like CDNs and NGINX? I'm pretty comfortable with the frontend stuff of webdev but the backend I'll definitely need to find a professional to help me out. Would prefer a referral to somebody rather than just hiring somebody who I don't have a connection to.

# ? Jun 10, 2020 18:48

The Fool: Oct 16, 2003

ceebee posted:

Would anyone be able to point me in the right direction to find a reliable and trustworthy web developer with experience in things like CDNs and NGINX? I'm pretty comfortable with the frontend stuff of webdev but the backend I'll definitely need to find a professional to help me out. Would prefer a referral to somebody rather than just hiring somebody who I don't have a connection to.

Do you just need to ask some questions, or are you looking for a contractor?

If questions, :justpost:

If contractor, :justpost:

but in here: https://forums.somethingawful.com/showthread.php?threadid=3075135

# ? Jun 10, 2020 18:52

Bad Titty Puker: Nov 3, 2007; Soiled Meat

I have a tesselation pattern that I designed based on a simple algorithm. All it needs to do is to draw vertical and horizontal lines on a grid. Looking for recommendations of simple programming languages/libraries/packages etc. I have a lot of experience with Borland Delphi and a little experience using Python 3.

# ? Jun 11, 2020 00:23

The Fool: Oct 16, 2003

Bad Titty Puker posted:

I have a tesselation pattern that I designed based on a simple algorithm. All it needs to do is to draw vertical and horizontal lines on a grid. Looking for recommendations of simple programming languages/libraries/packages etc. I have a lot of experience with Borland Delphi and a little experience using Python 3.

If you want to stick with Python, pycairo is probably a good choice.

If you wanted to explore other languages, using Javascript with a 2d library like PixiJS would work well and be highly portable.

# ? Jun 11, 2020 00:38

KillHour: Oct 28, 2007

The above are good. Here are some more options that might be a little more off the beaten path:

https://processing.org/
https://two.js.org/
https://www.shadertoy.com/
https://fiddle.skia.org/c/@skcanvas_paint
https://github.com/OneLoneCoder/olcPixelGameEngine/wiki

# ? Jun 11, 2020 00:44

Bad Titty Puker: Nov 3, 2007; Soiled Meat

The Fool posted:

If you want to stick with Python, pycairo is probably a good choice.

If you wanted to explore other languages, using Javascript with a 2d library like PixiJS would work well and be highly portable.

KillHour posted:

The above are good. Here are some more options that might be a little more off the beaten path:

https://processing.org/
https://two.js.org/
https://www.shadertoy.com/
https://fiddle.skia.org/c/@skcanvas_paint
https://github.com/OneLoneCoder/olcPixelGameEngine/wiki

Thanks for the recommendations. I've done a small amount of Javascript programming before and it's something I'd like to learn as well.

# ? Jun 11, 2020 01:17

KillHour: Oct 28, 2007

Bad Titty Puker posted:

Thanks for the recommendations. I've done a small amount of Javascript programming before and it's something I'd like to learn as well.

Why? Javascript is terrible.

# ? Jun 11, 2020 01:37

dupersaurus: Aug 1, 2012; Futurism was an art movement where dudes were all 'CARS ARE COOL AND THE PAST IS FOR CHUMPS. LET'S DRAW SOME CARS.'

KillHour posted:

Why? Javascript is ~~terrible~~ a blast.

# ? Jun 11, 2020 01:38

HappyHippo: Nov 19, 2003; Do you have an Air Miles Card?

Javascript is bad but it's nice being able to send people your project as a webpage that they just open

Also I don't see the need for a library, you can use javascript + canvas to draw some lines.

# ? Jun 11, 2020 23:41

Macichne Leainig: Jul 26, 2012; by VG

Modern JS/TypeScript is a lot of fun. I wouldn�t want to be working on a massive Angular app or anything, but small scale projects are definitely a blast.

Fully recommend hopping on the JAMstack train if you can. I�ve been working with Next.js for a while now and fully recommend it if you like working with React at all.

# ? Jun 12, 2020 01:13

Dominoes: Sep 20, 2007

WebAssembly will save us all.

# ? Jun 12, 2020 05:00

Mr Shiny Pants: Nov 12, 2012

Fable is awesome. Too bad it doesn't get used that much.

# ? Jun 12, 2020 07:52

Loezi: Dec 18, 2012; Never buy the cheap stuff

Years since I did time complexity stuff, so really rusty on this.

Assume that I have a set S of n elements. For k in [k_min, k_max], (assume k_min >= 1, k_max <= n) I need to generate all permutations of S of size k and run an algorithm that has time complexity O(k).

So say that I have S = {1,2,3}, k_min = 1, k_max = 3, the desired output is:

code:

[1]
[2]
[3]
[1,2]
[1,3]
[2,1]
[2,3]
[3,1]
[3,2]
[1,2,3]
[1,3,2]
[2,1,3]
[2,3,1]
[3,1,2]
[3,2,1]

And then I'd need to make final, linear, pass through all those.

I think this is called k-permutations of n and thus the whole shebang would be O(sum_{k in [k_min, k_max]} (n!)/((n-k)!)) with some kinda weird added factor from doing a linear pass over all the generated sequences?

E: This is in the context of "this is way too computationally heavy, so we can't just brute force this poo poo", so some kinda lower bound would be fine as well.

# ? Jun 14, 2020 12:43

Jabor: Jul 16, 2010; #1 Loser at SpaceChem

Just call it O(n!).

Ignoring the other possible k values (for reasons which we'll see in a moment), start by figuring how expensive it is to process the most expensive value of k. It's when k = n, and you have n! permutations that each require k processing time, for a total complexity of O(n!n).

This is obviously more than n!, so your overall process cannot have a lower bound better than factorial complexity.

So what's our upper bound? Well, for the purposes of an upper bound, let's assume each other value of k takes just as long. Then our total runtime would be n!*n*n. This is definitely smaller than (n+2)!, so we can at least say that the overall process isn't worse than factorial complexity either.

Factorial complexity is shockingly bad and you're not going to be able to brute-force this for anything more than a tiny problem.

# ? Jun 14, 2020 13:01

Loezi: Dec 18, 2012; Never buy the cheap stuff

Jabor posted:

Just call it O(n!).

Ignoring the other possible k values (for reasons which we'll see in a moment), start by figuring how expensive it is to process the most expensive value of k. It's when k = n, and you have n! permutations that each require k processing time, for a total complexity of O(n!n).

This is obviously more than n!, so your overall process cannot have a lower bound better than factorial complexity.

So what's our upper bound? Well, for the purposes of an upper bound, let's assume each other value of k takes just as long. Then our total runtime would be n!*n*n. This is definitely smaller than (n+2)!, so we can at least say that the overall process isn't worse than factorial complexity either.

Factorial complexity is shockingly bad and you're not going to be able to brute-force this for anything more than a tiny problem.

Thanks. I tried out some of the "these are far from worst case" values I'd seen previously (n = 2500, 5 <= k <= 40) with WolframAlpha and the numbers are so hilariously bad even without liner pass component that I don't think I even need to do account for it. Doubling k_max from 40 to 80 "only" increases the number of permutations from 10⁴⁹⁶³ to 10⁵⁰⁸² so even the summation part is indeed unnecessary for my argument.

# ? Jun 14, 2020 13:11

rjmccall: Sep 7, 2007; no worries friend; Fun Shoe

The number of permutations you�d be considering is a(n)-1, where a(n) is OEIS A000522. It is somewhere between (N!) and (N)(N!). So yeah, if you then do work proportional to the size of the permutation, that is an awful lot of work.

# ? Jun 15, 2020 19:05

Sab669: Sep 24, 2009

Does anyone here have much experience using Google Charts?

My company is looking for an ad hoc report creation tool to add to our software. One of the biggest requirements is that these reports must be able to run without an internet connection.

We thought we found just such a tool which says it's available for offline use. However, they use Google Charts for their tool, and on GC's FAQ website it says:

Google posted:

Can I use charts offline?

Your users' computers must have access to https://www.gstatic.com/charts/loader.js in order to use the interactive features of Google Charts. This is because the visualization libraries that your page requires are loaded dynamically before you use them. The code for loading the appropriate library is part of the included script, and is called when you invoke the google.charts.load() method. Our terms of service do not allow you to download the google.charts.load or google.visualization code to use offline.

Can I download and host the chart code locally, or on an intranet?

Sorry; our terms of service do not allow you to download and save or host the google.charts.load or google.visualization code. However, if you don't need the interactivity of Google Charts, you can screenshot the charts and use them as you wish.

So assuming they aren't breaking the Google ToS and supplying restricted code, I don't understand what functionality I'm actually losing by not having access to https://www.gstatic.com/charts/loader.js ?

# ? Jun 16, 2020 16:48

lifg: Dec 4, 2000; <this tag left blank>; Muldoon

I wouldn�t assume that.

There�s a small ecosystem of companies whose entire business model is �we work around Google�s ToS so you don�t have too.�

# ? Jun 16, 2020 16:59

Macichne Leainig: Jul 26, 2012; by VG

Knowing Google there's likely some analytics component that requires you to load the script from their CDN. They're a bit of a control freak in that regard.

# ? Jun 16, 2020 17:05

Sab669: Sep 24, 2009

lifg posted:

I wouldn’t assume that.

There’s a small ecosystem of companies whose entire business model is “we work around Google’s ToS so you don’t have too.”

They're located in the US so wilful misuse seems unlikely to me :shrug:

but they are a pretty small company, so maybe they just hope no one looks too closely.

# ? Jun 16, 2020 17:18

The Fool: Oct 16, 2003

Sab669 posted:

They're located in the US so wilful misuse seems unlikely to me

lol, that's cute

# ? Jun 16, 2020 17:21

Pollyanna: Mar 5, 2005; Milk's on them.

We have an application that's behind a load balancer. We want to replace this application with one that responds identically, and therefore should behave identically, but that is based off of a completely different codebase and is otherwise a total rewrite. There's a particular approach I want to take in validating that our rewrite is a perfect drop-in replacement, but I don't recall the name.

As an engineer working on the replacement for this application, I want a tool that lets me compare the results of the same response made to two different applications/endpoints, so I can tell if the new solution is a drop-in replacement for the old solution.

Ideally, this could be done at the load balancer level (AWS if it matters) so that we could simply change the load balancer to always return the New Response instead of the Old Response.

Is there a name for this pattern/framework? Something along the lines of "experiment" or whatever? I could have sworn this existed, but I have no idea where to start looking.

# ? Jun 18, 2020 19:48

Hadlock: Nov 9, 2004

Is it A RESTful API or something serving up HTML or what? That will dictate your toolset/methodology

I would write a test harness for the old app in the new framework/language, and out the results in json, and then write a script to boot up both apps as containers, run the harness against both containers, and then diff the JSON output

# ? Jun 18, 2020 19:54

ultrafilter: Aug 23, 2007; It's okay if you have any questions.

Is it stateless?

# ? Jun 18, 2020 19:55

Pollyanna: Mar 5, 2005; Milk's on them.

It is not stateless. The monolith + small helper apps we're replacing interact with a Postgres database and a Mongo database.

The main reason I bring this up is that the grand majority of interaction made with our old codebase is requests made to one of the small helper apps, and ideally what we replace it with should return the exact same request response as the old one. I was hoping to have a top-level way to compare old vs. new at that critical junction to give us more confidence in cutting over to the new services.

Basically I want to make the New API respond the same way to the same production requests as the Old API, and use that as the way to determine "ok we're good to cut over". You should be able to do that by making your load balancer send the request to Old API, record the old response, send the request to the New API, record the new response, save the comparison somewhere, and serve the old response, right?

EDIT: Thinking about it, that might be a problem with regards to latency/stability. Maybe an asynchronous approach would be better, in which case it might require a little more investigation.

Is there a better way to ensure this? Ongoing testing of identical responses/behavior and constant checking against an ideal end state is important for a project like this. I just have no idea how to execute it.

Pollyanna fucked around with this message at 20:07 on Jun 18, 2020

# ? Jun 18, 2020 20:03

Bruegels Fuckbooks: Sep 14, 2004; Now, listen - I know the two of you are very different from each other in a lot of ways, but you have to understand that as far as Grandpa's concerned, you're both pieces of shit! Yeah. I can prove it mathematically.

Pollyanna posted:

It is not stateless. The monolith + small helper apps we're replacing interact with a Postgres database and a Mongo database.

The main reason I bring this up is that the grand majority of interaction made with our old codebase is requests made to one of the small helper apps, and ideally what we replace it with should return the exact same request response as the old one. I was hoping to have a top-level way to compare old vs. new at that critical junction to give us more confidence in cutting over to the new services.

Basically I want to make the New API respond the same way to the same production requests as the Old API, and use that as the way to determine "ok we're good to cut over". You should be able to do that by making your load balancer send the request to Old API, record the old response, send the request to the New API, record the new response, save the comparison somewhere, and serve the old response, right?

EDIT: Thinking about it, that might be a problem with regards to latency/stability. Maybe an asynchronous approach would be better, in which case it might require a little more investigation.

Is there a better way to ensure this? Ongoing testing of identical responses/behavior and constant checking against an ideal end state is important for a project like this. I just have no idea how to execute it.

You could just write a bunch of test scripts for postman that run through your api calls and verify that you're getting the expected results - this could act as an integration level test for your API. There are other tools that work similarly but postman comes to mind as widely available.

# ? Jun 18, 2020 20:56

Adbot: ADBOT LOVES YOU

# ? May 30, 2024 22:37

Munkeymon: Aug 14, 2003; Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.

Pollyanna posted:

Ideally, this could be done at the load balancer level (AWS if it matters) so that we could simply change the load balancer to always return the New Response instead of the Old Response.

I want this but for SQL results

# ? Jun 18, 2020 21:37

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Ask General Programming Questions Not Worth Their Own Thread

«‹›671 »