Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
hooah
Feb 6, 2006
WTF?
I enabled autocomplete for plain text. I have no idea if that's important here.

Adbot
ADBOT LOVES YOU

Ranzear
Jul 25, 2013

Capri Sun Tzu posted:

Type L then press CTRL+Space

Nope, nothing. Am on Linux though.

feedmegin
Jul 30, 2008

ToxicFrog posted:


LISP, not Forth.

Oh yeah right. How did I brain fart and confuse it with PostScript :ohdear:

Ranzear
Jul 25, 2013

Ranzear posted:

Capri Sun Tzu posted:

Type L then press CTRL+Space
Nope, nothing. Am on Linux though.

Remembered to check at home: Yep, Windows only it seems.

It even newlines it at 80 columns. What the gently caress is this poo poo?

NihilCredo
Jun 6, 2011

iram omni possibili modo preme:
plus una illa te diffamabit, quam multæ virtutes commendabunt

Portland Sucks posted:

I'm going to be in the position of needing to redesign a major software platform at work and I'm lacking in some understanding regarding how well designed real time systems work behind the scenes. We have an application that currently just uses a MSSQL Server database as it's backend, the desktop application itself keeps an in memory snapshot of it's current state, and then just has some really poorly written HasChanged() methods to look at the state of the database compared to its cache to determine if another client changed it within X amount of time. It's a horrible mess of race conditions and collisions and requires a lot of supervision.

I've been looking at various messaging and queue systems (RabbitMQ, Kafka, 0MQ, etc.) and I get the feeling that they were designed for use cases like this. Are there any pretty standard references or design patterns in regards to how systems like this are generally setup? Like how should the communication pathways be designed at the high level, how is long term storage handled, how to work with race conditions, etc. would be very helpful.

First question: do you NEED that additional layer? If you just deleted the in-memory cache thing, kept your application stateless, ran queries every time, and trusted MSSQL to do its own caching for you, would things probably Just® Work®?

User
May 3, 2002

by FactsAreUseless
Nap Ghost

NihilCredo posted:

First question: do you NEED that additional layer? If you just deleted the in-memory cache thing, kept your application stateless, ran queries every time, and trusted MSSQL to do its own caching for you, would things probably Just® Work®?

For a true real-time system I would be pretty leery of anything that involved calls to a remote server (unless my organization controlled the link layer end to end and some other things). If it's just buzzword real-time then this is the right answer.

KernelSlanders
May 27, 2013

Rogue operating systems on occasion spread lies and rumors about me.
Is there a coherent definition of "real time" systems or programming? Back when I did robotics, we thought about dedicated OS feature and guaranteed task execution times, but now I hear the term thrown around to describe REST services. Should I just stop thinking of "real time" as meaning real time?

ultrafilter
Aug 23, 2007

It's okay if you have any questions.


I hear people talk about hard and soft real time systems, presumably to make the distinction that was eroded when the term became common.

Also some people are clueless.

Volguus
Mar 3, 2009

KernelSlanders posted:

Is there a coherent definition of "real time" systems or programming? Back when I did robotics, we thought about dedicated OS feature and guaranteed task execution times, but now I hear the term thrown around to describe REST services. Should I just stop thinking of "real time" as meaning real time?

That is the definition that I was familiar with as well: guaranteed task execution time. Lately people just mean: display the data as soon as is available (different values of "soon" are acceptable).

underage at the vape shop
May 11, 2011

by Cyrano4747

TooMuchAbstraction posted:

Dunno what to tell you. Possibly you're benefiting from some kind of runtime optimizations? What language are you using?

Python code:
import random
import time

for n in [100, 1000, 10000, 100000, 1000000, 10000000]:
  ns = [random.randint(-1000, 1000) for i in xrange(n)]
  start = time.time()
  i = 0
  j = n - 1
  while i <= j:
    if ns[i] < 0:
      i += 1
    else:
      ns[i], ns[j] = ns[j], ns[i]
      j -= 1
  end = time.time()
  print("%d: %.4fs" % (n, end - start))

I'm using Java. We can't use python, just C, C Sharp or Java

rjmccall posted:

That is an asympotically linear growth curve. It’s understood that wall-clock measurements won’t necessarily match the curve for small N, for all sorts of fixed-overhead reasons like the time spent waiting for the code to load from disk.

Your argument probably needs to be stronger than “once for every element in the array”, though, since it’s not like the code is obviously advancing through the array one element at a time.

Yeah I'd go into more detail in the report and demonstrate why it goes over each element once, but it does.

User
May 3, 2002

by FactsAreUseless
Nap Ghost

underage at the vape shop posted:

Yeah I'd go into more detail in the report and demonstrate why it goes over each element once, but it does.

It's actually a rather pretty algorithm. Also I have a soft spot for that algolish pseudocode. It's definitely a great little exercise in proving loop termination.

Edit: I'm kind of disappointed that professional programmers almost never get to write even marginally nontrivial loops like this one. I love the fixed point iterator I learned from SICP, but I've never been in a position to get paid to write anything even remotely like it.

User fucked around with this message at 04:53 on Apr 9, 2018

TooMuchAbstraction
Oct 14, 2012

I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.
Fun Shoe

underage at the vape shop posted:

I'm using Java. We can't use python, just C, C Sharp or Java

The Python was just an example because it's easy to write and most people have little trouble reading it. The specific language shouldn't make any difference to asymptotic runtime. If you're seeing runtime variations in your language of choice, then either they're not asymptotic or else your language is detecting and implementing clever optimizations that make the asymptotic runtime not what you think it is. Compilers are really loving smart these days.

User
May 3, 2002

by FactsAreUseless
Nap Ghost

TooMuchAbstraction posted:

The Python was just an example because it's easy to write and most people have little trouble reading it. The specific language shouldn't make any difference to asymptotic runtime. If you're seeing runtime variations in your language of choice, then either they're not asymptotic or else your language is detecting and implementing clever optimizations that make the asymptotic runtime not what you think it is. Compilers are really loving smart these days.

That's the thing about asymptotes, lots of stuff can happen when n isn't "sufficiently large." Software run-time issues and hardware issues will play a huge role. With modern hardware a lot can fit in a cache line for example.

User fucked around with this message at 04:56 on Apr 9, 2018

Munkeymon
Aug 14, 2003

Motherfucker's got an
armor-piercing crowbar! Rigoddamndicu𝜆ous.



Ranzear posted:

What the gently caress is this poo poo?

:same:

To disable it: http://docs.sublimetext.info/en/latest/customization/key_bindings.html

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

I'm trying to reverse engineer the API and figure out sent/received data for a device that communicates with a cloud service via wifi on my home network.

I'm familiar with how to capture packets going to and from my machine with a proxy like Charles and using its MITM SSL certificate thingy, but I'm not really sure how to do the same thing with some external device.

How do I do this?

JawnV6
Jul 4, 2004

So hot ...

Thermopyle posted:

I'm not really sure how to do the same thing with some external device.

How do I do this?

Googling "ARP poisoining" should get you 90% of the way there.

Jabor
Jul 16, 2010

#1 Loser at SpaceChem
Since you mentioned SSL: the point of SSL is to prevent what you're trying to do, so if the device is using it and you have no way to change the certificate then you're a bit out of luck. Sorry.

pokeyman
Nov 26, 2006

That elephant ate my entire platoon.
That's a point of ssl, it's still totally worth a shot because a ton of software dgaf about which cert so long as there's some cert.

The Fool
Oct 16, 2003





Thermopyle posted:

I'm trying to reverse engineer the API and figure out sent/received data for a device that communicates with a cloud service via wifi on my home network.

I'm familiar with how to capture packets going to and from my machine with a proxy like Charles and using its MITM SSL certificate thingy, but I'm not really sure how to do the same thing with some external device.

How do I do this?
You proxy and capturing need to be done on a device that is acting as the router for the device in question. This is how the bigger content filter edge devices work.


This blog has instructions to do it with WiFi sharing on a MacBook. But with a little digging you could do it other network configurations.
https://www.trustwave.com/Resources/SpiderLabs-Blog/Intercepting-SSL-And-HTTPS-Traffic-With-mitmproxy-and-SSLsplit/

underage at the vape shop
May 11, 2011

by Cyrano4747

User posted:

It's actually a rather pretty algorithm. Also I have a soft spot for that algolish pseudocode. It's definitely a great little exercise in proving loop termination.

Edit: I'm kind of disappointed that professional programmers almost never get to write even marginally nontrivial loops like this one. I love the fixed point iterator I learned from SICP, but I've never been in a position to get paid to write anything even remotely like it.

Yeah I was thinking that when I read it, it's pretty cool and really simple.

TooMuchAbstraction posted:

The Python was just an example because it's easy to write and most people have little trouble reading it. The specific language shouldn't make any difference to asymptotic runtime. If you're seeing runtime variations in your language of choice, then either they're not asymptotic or else your language is detecting and implementing clever optimizations that make the asymptotic runtime not what you think it is. Compilers are really loving smart these days.
Yeah true. Are language like python used in business much? I really like all the little things, like print vs Console.WriteLine or System.out.print.
I did some research on it because I wanted to make the report as easy as possible (this is uni coursework), ended up porting it to c# just to try it out, and it's almost perfectly linear. Apparently Java needs a bit of work to get the curve you'd expect, because of exactly what you guys talked about, overheads and optimisations, but it is possible. Thats a lot of extra method justification taking up word limit so I figured I can use that better elsewhere :shrug:.

underage at the vape shop fucked around with this message at 16:44 on Apr 11, 2018

TooMuchAbstraction
Oct 14, 2012

I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.
Fun Shoe

underage at the vape shop posted:

Are language like python used in business much?

Python is I think in the top 5 languages used worldwide? IIRC the top langs these days are something like Java > (C++ and C#, I forget which is more popular) > Javascript > Python, or something like that. Talking about new development here, since there's of course still a poo poo-ton of COBOL and other older languages in active use too.

Python is basically the go-to language for rapid development when you don't care about the language providing safeguards like type-checking.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.
In bash, how do I loop over a list of files, skipping the first n files?


I have a process that generates a log file that rolls every hour and I want to move these files to a central location. The problem is that the log file that is currently in use is not locked so if I just do a "mv *" command It will grab all of the files and the running process will not create a new log file. So I want to write a script that moves all files, skipping the file with the most recent timestamp.

DOS has the FOR IN SKIP DO construct, but I cannot find a similar method for bash. Help?

Kilson
Jan 16, 2003

I EAT LITTLE CHILDREN FOR BREAKFAST !!11!!1!!!!111!

Agrikk posted:

In bash, how do I loop over a list of files, skipping the first n files?

You can use head/tail to chop off some element(s) of your file list.

TooMuchAbstraction
Oct 14, 2012

I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.
Fun Shoe

Kilson posted:

You can use head/tail to chop off some element(s) of your file list.

Irritatingly, there's not a good way using head/tail to say "all but the first/last N rows". As far as I'm aware, you have to count the rows, subtract off the amount you want to exclude, then pass that number to head/tail.

I wish there was a flag like the grep -v flag. :(

Kilson
Jan 16, 2003

I EAT LITTLE CHILDREN FOR BREAKFAST !!11!!1!!!!111!

TooMuchAbstraction posted:

Irritatingly, there's not a good way using head/tail to say "all but the first/last N rows". As far as I'm aware, you have to count the rows, subtract off the amount you want to exclude, then pass that number to head/tail.

I wish there was a flag like the grep -v flag. :(

I'm pretty certain it's possible.
head -n -<number> or tail -n +<number> are supposed to show all lines starting at that number, for example.

Methanar
Sep 26, 2013

by the sex ghost

Agrikk posted:

In bash, how do I loop over a list of files, skipping the first n files?


I have a process that generates a log file that rolls every hour and I want to move these files to a central location. The problem is that the log file that is currently in use is not locked so if I just do a "mv *" command It will grab all of the files and the running process will not create a new log file. So I want to write a script that moves all files, skipping the file with the most recent timestamp.

DOS has the FOR IN SKIP DO construct, but I cannot find a similar method for bash. Help?

This is how I'm doing it right now for a script.

Find all directories
sort
everything but the bottom 4
grep manipulation
apply command to everything remaining

code:
        find /home/ubuntu/apps/code -maxdepth 1 -type d | sort -n | head -n -4 |
        grep -E '[[:digit:]]' | xargs -n 1 rm -rf
A slightly different version that sorts based on creation timestamps. most recent at the bottom.

code:
          #Find all directories in a given path and print the creation timestamp.
          #Sort oldest to newest. Grep out the base directory. only print the file path.
          #Print everything except for the bottom 4 (most recent) directories
          #Delete everything that is printed
          find /home/ubuntu/apps/code -maxdepth 1 -type d -printf '%T+ %p\  n' 2>/dev/null | sort -n | grep -v "code$" | awk '{print $2}' | head -n -4 |   xargs -n 1 rm -rf

Methanar fucked around with this message at 00:58 on Apr 12, 2018

Ghost of Reagan Past
Oct 7, 2003

rock and roll fun

TooMuchAbstraction posted:

Python is I think in the top 5 languages used worldwide? IIRC the top langs these days are something like Java > (C++ and C#, I forget which is more popular) > Javascript > Python, or something like that. Talking about new development here, since there's of course still a poo poo-ton of COBOL and other older languages in active use too.

Python is basically the go-to language for rapid development when you don't care about the language providing safeguards like type-checking.
:eng101: With Python 3.5+ we now have type hinting!

PEP484 defines the spec; mypy is the BDFL-approved (and developed!) tool for type-checking. It's super cool, entirely optional, and if you use Python 3.5+ you can and should be using it. And mypy is compatible with older Python versions, you just have to use type comments, without the convenient syntax.

So you can instill type discipline in your code base, though it will never be mandatory at the language level.

And for what it's worth I don't write ALL my code type-checked. I just do it for some nice guarantees at some places.

Ghost of Reagan Past fucked around with this message at 01:42 on Apr 12, 2018

ultrafilter
Aug 23, 2007

It's okay if you have any questions.


From PEP 484:

quote:

No first-class syntax support for explicitly marking variables as being of a specific type is added by this PEP.
Type hints are a nice idea, but they're pretty far off from actual static type checking.

Eela6
May 25, 2007
Shredded Hen
That's been added since, though.



Python code:

n: int = 5

I use MyPy & type hinting for all of my python code and it does a pretty good job of catching type errors.

Pollyanna
Mar 5, 2005

Milk's on them.


How do you know when a monolith and a service that were split up are better off rolled back into each other? The project I’m on right now has a microservice that shares a LOT of data models and internal knowledge with its larger parent, to the point where there’s tons of duplication between the two. This has made me wonder whether having them split up is making things worse. If they use the same data models that deeply (we often have to synchronize changes between the two projects), are they really properly separated?

ultrafilter
Aug 23, 2007

It's okay if you have any questions.


Is it something that could be fixed by creating a common library/module and having them both use that? In that case, it's almost always worth it if the code is something you have to maintain.

TooMuchAbstraction
Oct 14, 2012

I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.
Fun Shoe
My general approach to refactoring code:
  • First, learn how the existing code works. Write tests, if they do not already exist. Understand where the pain points are in development.
  • Find something that looks easily-separable (a "seam"). Don't worry too much about how big it is. Cut it off of the existing system, and turn it into a library. Its connections to the old code become the library's API.
  • Fix all the build bugs you just introduced. This should teach you a lot about what components rely on what other components.
  • Watch out for circular dependencies. These do not indicate a component that must be part of a monolith; they may just indicate components that you can't split out quite yet because of some shared dependency that also needs to be split out.
  • Repeat the above steps until your brain isn't screaming every time you open your IDE.
The goal is to have a large number of small, narrowly-defined, and stateless libraries, plus an "application" (or central library or whatever) that ties them all together.

Remember, there's absolutely nothing wrong with a library that contains a single file that has a single 10-line function. You wouldn't want to do that for all of your code because it'd be inconvenient to navigate, but it's better to have the "this library contains the "recursively flatten dictionaries" function" library than it is to have that function be part of some other library it's not otherwise related to.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
here's the opposite approach. how to refactor "the wrong abstraction":

* locate an ugly piece of code. perhaps it's in charge of too much. more often it's built of too many ""self-contained"" parts that are intricately connected and brittle because of the interdependencies.

* locate all of its dependencies. perhaps these are utility methods or classes, perhaps it's the web of messy intersecting objects. take those dependencies and copy/paste them into the spot you want to clean up. this way, you can gently caress with them without loving the rest of the codebase.

* merge. don't focus on finding the right abstraction. do not worry about copy/paste. you will have an urge to split things out into common functionality. resist it. the goal in this phase is to repeat yourself a lot. get rid of inheritance and overrides during this stage.

* clean up. often times the abstraction and DRY made it frustratingly brittle in the first place. things shared code because the shape looked similar, not because the functionality was actually similar. as you work with the code and understand what it's doing, you will naturally come across better abstraction ideas. act on them ONLY in ways which are easy to undo. copy/paste is still your friend.

* harden. still don't attempt DRY across modules. when you find yourself fixing the same bug in the same code three or four times, split it out to shared code.

* repeat this process for modules. get them off of shared code into their own sandbox away from everybody else, play around, explore, solve the problems JUST for that module. as you do this, you will find where the real shared code is. don't be afraid to still have copy/paste. duplication is better than the wrong abstraction.

* if you see two pieces of code that are AAAALmost the same but not quite, leave them separate. one special case and flag parameter will grow into two, and two will grow into three. if you see that pattern start to emerge, split them apart. this is much easier when the code is smaller, and when the code was written to be easily to copy/paste and delete.

* write integration documentation. if something is sketchy and annoying, add a TODO or XXX comment. i use the former for tasks i am more sure about, and the latter for documenting questions and ugliness i am unsure about the answers to. the biggest thing that's helpful for a maintenance programmer are open questions the original guy had about the architecture, and places the original programmer feels are the ugliest.

this is my personal preference, but i find a lot of "layered architecture" and "separable utility" stuff to be extremely brittle and hard to work with. each component independently is nice and easy, but the complexity and depth comes from subtle interaction concerns between them.

and finally, my opinion as a jaded low-level programmer: code reuse should hopefully be rare. if all your applications are plumbing code between reusable components, you're not solving interesting enough problems.

redleader
Aug 18, 2005

Engage according to operational parameters

TooMuchAbstraction posted:

Remember, there's absolutely nothing wrong with a library that contains a single file that has a single 10-line function.

Counterpoint: npm

TooMuchAbstraction
Oct 14, 2012

I spent four years making
Waves of Steel
Hell yes I'm going to turn my avatar into an ad for it.
Fun Shoe

redleader posted:

Counterpoint: npm

Pretty sure that was fewer than 10 lines, so my point stands. :v:

leper khan
Dec 28, 2010
Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

Suspicious Dish posted:

and finally, my opinion as a jaded low-level programmer: code reuse should hopefully be rare. if all your applications are plumbing code between reusable components, you're not solving interesting enough problems.

I have not found a strong correlation between technically interesting work and business value. When I can make choices about the work I do for someone else, I tend to focus on the latter.

Pollyanna
Mar 5, 2005

Milk's on them.


Thanks for the advice, guys! I'll keep this all in mind as I'm working.

Refactoring is something I haven't had a lot of opportunities to do, and has never had much focus on - so I'm thinking of how to weave all of this into PRs and ticket work I gotta do. It's also a codebase I've had less than a month of exposure to, so I still need to understand what the gently caress I'm working with (which the rest of my team has little to no understanding of and the people that do have moved to different teams).

I'm beginning to wonder if now that the core problem has more or less been addressed (Mongo performance issues), maybe we shouldn't focus on totally ripping out a core component of it until we fully understand how it works. Then again, how much do I really have to know before I can deliver on my work...? I'll find out.

leper khan posted:

I have not found a strong correlation between technically interesting work and business value. When I can make choices about the work I do for someone else, I tend to focus on the latter.

I agree - I get more out of feeling like I'm doing a good job and bringing value than I do working on something new and interesting but not necessarily appreciated. Plus, I'm still a little early in my career to think of doing The Hard Stuff if I still need to get advice on refactoring...

du -hast
Mar 12, 2003

BEHEAD THOSE WHO INSULT GENTOO
I have a dumb question:

I work at a datacenter and we assign IPs from our online portal contraption.

There is a dropbox with all the available IPs in it. This is the only list I am aware of with all the usable IPs. This dropbox is in a Javascript windows that edits the IPs listed in a database.

So the options are something like.
1.2.3.4/29
69.42.0.0/29
14.88.69.69/26
9.9.9.0/24

Is there a way I can get that list in some sort of text file - ie pull all the entries from the dropdown. If I could do this then I could more easily see which sized IP blocks are available (ie for example I need 4x/26 but it's in a numerically sorted list, rather than sorted by side). I don't even know what language this would be done in but I assume some sort of Javascript.

Any hints would be appreciated x 1000 as it's a pain in the rear end hunting through a dropdown with 800 options searching for the ones that say /26 at the end.

leper khan
Dec 28, 2010
Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

du -hast posted:

I have a dumb question:

I work at a datacenter and we assign IPs from our online portal contraption.

There is a dropbox with all the available IPs in it. This is the only list I am aware of with all the usable IPs. This dropbox is in a Javascript windows that edits the IPs listed in a database.

So the options are something like.
1.2.3.4/29
69.42.0.0/29
14.88.69.69/26
9.9.9.0/24

Is there a way I can get that list in some sort of text file - ie pull all the entries from the dropdown. If I could do this then I could more easily see which sized IP blocks are available (ie for example I need 4x/26 but it's in a numerically sorted list, rather than sorted by side). I don't even know what language this would be done in but I assume some sort of Javascript.

Any hints would be appreciated x 1000 as it's a pain in the rear end hunting through a dropdown with 800 options searching for the ones that say /26 at the end.

Using chrome, open the dev console (ctrl+shift+j; mac = cmd+shift+j). You should see the source of the page with whatever stuff inserted that gets placed at runtime. Scroll through it/search for some ip range you know exists; copy everything to a text file.

Adbot
ADBOT LOVES YOU

du -hast
Mar 12, 2003

BEHEAD THOSE WHO INSULT GENTOO

leper khan posted:

Using chrome, open the dev console (ctrl+shift+j; mac = cmd+shift+j). You should see the source of the page with whatever stuff inserted that gets placed at runtime. Scroll through it/search for some ip range you know exists; copy everything to a text file.

I figured this out before you replied and I cannot believe how complicated I was going to make it. Works perfectly! Thanks a lot, as always you guys are loving fantastic.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply