Working in Development: Leaving sounds like a great decision.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Working in Development: Leaving sounds like a great decision.

«‹›434 »

B-Nasty: May 25, 2005

I would definitely have a scheduled integration test somewhere that calls real API endpoints. I have our integration tests running in TeamCity triggered by a commit, but it doesn't stop the main build/deploy which only runs the unit tests. The full integration test suite might only run 10-15 times a day due to how long they take.

Through testing live APIs, I can name a number of times where something in the API changed (by the vendor, without warning, of course) and the test caught it. It allowed us to make an emergency fix or yell at the vendor immediately. Those tests also alert us to the API temporarily going down so that support can be aware of the possible calls they are about to receive from customers.

# ? Sep 28, 2016 19:07

Adbot: ADBOT LOVES YOU

# ? May 10, 2024 01:42

necrobobsledder: Mar 21, 2005; Lay down your soul to the gods rock 'n roll; Nap Ghost

To add a bit more on testing externalities as more of an SRE, you ultimately need:

means to cross-validate access and latency to your external services (eg. account locked out but AD still works fine, site is up but your LAN died is what died)
a failsafe or circuit breaker mechanism to gracefully fail or re-route. Also needs to consider that you may get rate-limited and to back off before you hit a problem.
logging or notification when an external API has changed perceptibly along with its impact to production and test. This should include non-software-level changes like SSL certificates and DNS changes

# ? Sep 28, 2016 22:12

My Rhythmic Crotch: Jan 13, 2011

I hesitate to post this, because I actually like working here. But my boss is just ... so weird. He's amazing at SQL tuning and linux hacking, but he hates any kind of structure or organization. This has led to all kinds of interesting situations.

- When I first got here (3+ years ago), none of the legacy code was version controlled. To develop, he would make a backup of the file, and just edit the file on the production server. So as people are using stuff, they're seeing his edits in real time. His "version control" was to add a dash onto each backup. So if you did "ls" on the server, you'd see:
index.html
index.html-
index.html--
index.html---
index.html----
index.html-----
The file with the most dashes being the newest "version" (other than the file with no dashes, which is both the production file and the one he's editing).

- There was no backup system either. He had some bash scripts that would rsync files around to the various production servers, so stuff was duplicated I suppose. I have since put a real, daily backup in place... phew.

- We use this awful thing called Bucardo to replicate our data. One of the downsides to replicating is that schema changes are very painful, because <reasons>. So instead of changing schema, he just starts jamming concatenated text into one column. So, made up example here, instead of having a column for first_name and a column for last_name, he just has one column called "name" and you store "Jane:Doe" in it. How do you get data out you may ask?

code:

select unnest(string_to_array(butts::text,',')) from (
    select unnest(string_to_array('A,1:B,2:C,3'::text,':')) as butts
) as doody;

This example is simple compared to some of our queries, which have unnest(string_to_array()) nested 4 layers deep, plus more joins, cases, etc, than you've ever wanted to see.

- Fun side effect of replication is that it really hurts insert performance. Especially in our case, since we are replicating across large distances with really bad latencies.

- There was no test or dev database. There were something like 50-60 junk tables in the production DB, which I've since cleaned up. Without a test DB, all schema changes were done seat-of-the-pants, with no practice, on the production DB. He'd at least dump the table(s) before the migration... but still...

- All projects I take on, I make a concerted effort to get requirements, design the system, write tests, etc. Even now, he's still very much like " :rolleyes:

state diagrams, look at mister fancy pants" or " :rolleyes:

unit tests, hah!" Apparently they were regularly coding and putting stuff in production without having any design at all, no user feedback, no beta testing, just BLAM dump it in production and go do something else.

Don't get me wrong, I enjoy working here and my boss is actually a pretty cool guy. He just comes from a bygone era where hacking was how everything was done I suppose.

# ? Sep 29, 2016 01:01

leper khan: Dec 28, 2010; Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

My Rhythmic Crotch posted:

I hesitate to post this, because I actually like working here. But my boss is just ... so weird. He's amazing at SQL tuning and linux hacking, but he hates any kind of structure or organization. This has led to all kinds of interesting situations.

- When I first got here (3+ years ago), none of the legacy code was version controlled. To develop, he would make a backup of the file, and just edit the file on the production server. So as people are using stuff, they're seeing his edits in real time. His "version control" was to add a dash onto each backup. So if you did "ls" on the server, you'd see:
index.html
index.html-
index.html--
index.html---
index.html----
index.html-----
The file with the most dashes being the newest "version" (other than the file with no dashes, which is both the production file and the one he's editing).

- There was no backup system either. He had some bash scripts that would rsync files around to the various production servers, so stuff was duplicated I suppose. I have since put a real, daily backup in place... phew.

- We use this awful thing called Bucardo to replicate our data. One of the downsides to replicating is that schema changes are very painful, because <reasons>. So instead of changing schema, he just starts jamming concatenated text into one column. So, made up example here, instead of having a column for first_name and a column for last_name, he just has one column called "name" and you store "Jane:Doe" in it. How do you get data out you may ask?
code:
select unnest(string_to_array(butts::text,',')) from (
    select unnest(string_to_array('A,1:B,2:C,3'::text,':')) as butts
) as doody;
This example is simple compared to some of our queries, which have unnest(string_to_array()) nested 4 layers deep, plus more joins, cases, etc, than you've ever wanted to see.

- Fun side effect of replication is that it really hurts insert performance. Especially in our case, since we are replicating across large distances with really bad latencies.

- There was no test or dev database. There were something like 50-60 junk tables in the production DB, which I've since cleaned up. Without a test DB, all schema changes were done seat-of-the-pants, with no practice, on the production DB. He'd at least dump the table(s) before the migration... but still...

- All projects I take on, I make a concerted effort to get requirements, design the system, write tests, etc. Even now, he's still very much like " state diagrams, look at mister fancy pants" or " unit tests, hah!" Apparently they were regularly coding and putting stuff in production without having any design at all, no user feedback, no beta testing, just BLAM dump it in production and go do something else.

Don't get me wrong, I enjoy working here and my boss is actually a pretty cool guy. He just comes from a bygone era where hacking was how everything was done I suppose.

Version control has been a thing since the 80s, and your employer sounds frightful. I hope you make lots of money to deal with it.

# ? Sep 29, 2016 01:15

New Yorp New Yorp: Jul 18, 2003; Only in Kenya.; Pillbug

My Rhythmic Crotch posted:

I hesitate to post this, because I actually like working here. But my boss is just ... so weird. He's amazing at SQL tuning and linux hacking, but he hates any kind of structure or organization. This has led to all kinds of interesting situations.
...
instead of having a column for first_name and a column for last_name, he just has one column called "name" and you store "Jane:Doe" in it. How do you get data out you may ask?
code:
select unnest(string_to_array(butts::text,',')) from (
    select unnest(string_to_array('A,1:B,2:C,3'::text,':')) as butts
) as doody;
This example is simple compared to some of our queries, which have unnest(string_to_array()) nested 4 layers deep, plus more joins, cases, etc, than you've ever wanted to see.

- Fun side effect of replication is that it really hurts insert performance. Especially in our case, since we are replicating across large distances with really bad latencies.

Sounds like he is actually awful at SQL. And everything else related to being a developer.

# ? Sep 29, 2016 01:20

Axiem: Oct 19, 2005; I want to leave my mind blank, but I'm terrified of what will happen if I do

Pollyanna posted:

Related to testing: if your app depends on a connection to an API to work properly, is it better to bypass it/stub it out, or should the test environment enforce a live connection to the API while running tests?

Personally, in an ideal world, some of both. I want to test various normal scenarios and common error conditions against the actual server, because it does catch when APIs suddenly change without documentation, or when the server provides malformed JSON or something. To an extent, it'a a test that covers both the client and the server, to make sure they're talking together correctly.

But I also want to stub out a lot more scenarios that build on what I test against a real server (e.g. the real server is "order an ice cream cone with vanilla", and the fake server gets "order an ice cream cone with chocolate" or something like that), and more importantly, to stub out the scenarios that are difficult or impossible to orchestrate on the server. What happens when the server returns 500? What happens when we hit the server right in-between those two cron jobs that both need to run to get real data (yes, this is a scenario I've had to test)? Things like that.

I'd say it's my ideal; living up to it is a far more difficult thing, because of practical concerns and time pressures.

# ? Sep 29, 2016 04:08

Messyass: Dec 23, 2003

Ithaqua posted:

Sounds like he is actually awful at SQL. And everything else related to being a developer.

Yeah I guess you become amazing at "tuning" SQL when you don't have a sane design in the first place.

# ? Sep 29, 2016 07:14

Cuntpunch: Oct 3, 2003; A monkey in a long line of kings

Reminds me somewhat of a developer I worked with briefly: an 'expert' about performance matters. So yes, his framework's data access and SPs were *lightning fast* - they were! But you had to make 60 calls per pageload to the exact same SP with slightly tweaked parameters - and so any possible performance benefit over just having a larger single call with a table-valued-parameter was completely lost.

# ? Sep 29, 2016 12:37

Docjowles: Apr 9, 2009

Have you checked that you weren't teleported back to the early 90's? Are you perhaps on the worst episode ever of Quantum Leap? Because :lol:

at all of that.

# ? Sep 29, 2016 14:26

kedo: Nov 27, 2007

My Rhythmic Crotch posted:

Don't get me wrong, I enjoy working here and my boss is actually a pretty cool guy. He just comes from a bygone era where hacking was how everything was done I suppose.

Coding things on the internet was a lot more fun when no one knew what they were doing and "whatever hack works without crashing the server or users' browsers" was the most potent and often used weapon in your arsenal.

Come to think of it, we're still in more or less the same situation, everyone just has a lot of opinions.

# ? Sep 29, 2016 14:46

Polio Vax Scene: Apr 5, 2009

I hate inheriting the projects of the senior developer at the place I work. It's the most monstrous inefficient spaghetti code every time and its easier to just rewrite entire blocks of it based on the sparse documentation of what it should be doing than try and organize it.
The reason these projects get passed to me is because he's too busy to maintain them all...well maybe if you took the time to sort the poo poo out in the first place you wouldn't be months deep in help requests for them!
We're a pretty small company with only 4 active developers so things like code review don't even come into question.

# ? Sep 29, 2016 14:50

spiritual bypass: Feb 19, 2008; Grimey Drawer

Seems like code review (or any process change) should be easier to implement for a small team, supposing those people are actually Team Players™

# ? Sep 29, 2016 15:20

revmoo: May 25, 2006; #basta

Polio Vax Scene posted:

I hate inheriting the projects of the senior developer at the place I work. It's the most monstrous inefficient spaghetti code every time and its easier to just rewrite entire blocks of it based on the sparse documentation of what it should be doing than try and organize it.
The reason these projects get passed to me is because he's too busy to maintain them all...well maybe if you took the time to sort the poo poo out in the first place you wouldn't be months deep in help requests for them!
We're a pretty small company with only 4 active developers so things like code review don't even come into question.

This is my job except there's only two of us. We're maintaining an app that streams 200mbit+ 24/7 into a huge mapreduce cluster as the backend. Our oldest commit goes back to like 2006. I do not get paid enough for this.

# ? Sep 29, 2016 15:49

Greatbacon: Apr 9, 2012; by Pragmatica

rt4 posted:

Seems like code review (or any process change) should be easier to implement for a small team, supposing those people are actually Team Players™

Yeah, assuming you use git, you should be able to configure things so that all code going into your develop branch needs to be merged in from a feature branch, and that merge can't occur until at least 1 or 2 other develops have looked at and approved the merge.

Obviously there's nothing stopping folks from just blindly hitting accept but it's better than nothing.

# ? Sep 29, 2016 16:07

Rocko Bonaparte: Mar 12, 2002; Every day is Friday!

Greatbacon posted:

Yeah, assuming you use git, you should be able to configure things so that all code going into your develop branch needs to be merged in from a feature branch, and that merge can't occur until at least 1 or 2 other develops have looked at and approved the merge.

Obviously there's nothing stopping folks from just blindly hitting accept but it's better than nothing.

You can make it a +2 rule with the CI infrastructure providing one of the votes.

# ? Sep 29, 2016 19:22

Docjowles: Apr 9, 2009

rt4 posted:

Seems like code review (or any process change) should be easier to implement for a small team, supposing those people are actually Team Players™

I'm guessing it's more like "there's only 4 of us, we don't have time for code review ... because we are constantly putting out fires and taking 3 times as long to write anything due the codebase being a loving minefield of tech debt and bugs. Which wouldn't have been there if we reviewed the drat code in the first place".

Not at all saying that's Polio's stance, it's probably been that way since before they got there. Just that it's a very very common attitude. I've been on a few Ops teams that never took time to make improvements because they were always firefighting. It was awful, I got burnt out super fast, and noped on out.

# ? Sep 29, 2016 20:03

Polio Vax Scene: Apr 5, 2009

That's right on the money.
Plus, wouldn't want to get on anyone's bad side in a tiny company by telling them their code is poo poo. Insert metaphor about small town relationships here.
I make sure that anything I write myself is properly written to the fullest extent of my knowledge, but I suspect that's what was done here, six years ago. Back then, the guy probably didn't know much about what he was doing, but hasn't had the time or necessity to clean it up. I sometimes think about how the same thing will happen to me several years from now.

# ? Sep 29, 2016 21:39

Skandranon: Sep 6, 2008; fucking stupid, dont listen to me

Polio Vax Scene posted:

That's right on the money.
Plus, wouldn't want to get on anyone's bad side in a tiny company by telling them their code is poo poo. Insert metaphor about small town relationships here.
I make sure that anything I write myself is properly written to the fullest extent of my knowledge, but I suspect that's what was done here, six years ago. Back then, the guy probably didn't know much about what he was doing, but hasn't had the time or necessity to clean it up. I sometimes think about how the same thing will happen to me several years from now.

Even if you are always outputting excellent code as far as you are able to, your excellent code today is (hopefully) better than it was yesterday. Unless you get extra time to go back and fix your code from years ago, you will always have lovely code in your past.

# ? Sep 29, 2016 21:42

revmoo: May 25, 2006; #basta

The worst developer is always:

you - 6months

# ? Sep 29, 2016 21:44

leper khan: Dec 28, 2010; Honest to god thinks Half Life 2 is a bad game. But at least he likes Monster Hunter.

revmoo posted:

The worst developer is always:

you - 6months

Hey, that guy has it out for me.

# ? Sep 29, 2016 21:59

MisterZimbu: Mar 13, 2006

Cuntpunch posted:

Reminds me somewhat of a developer I worked with briefly: an 'expert' about performance matters. So yes, his framework's data access and SPs were *lightning fast* - they were! But you had to make 60 calls per pageload to the exact same SP with slightly tweaked parameters - and so any possible performance benefit over just having a larger single call with a table-valued-parameter was completely lost.

This reminds me of a former DBA I had who refused to put indexes on tables because they slowed down queries too much, and provided us a "data access layer" that was pretty much C# code that concatenated together parts of SQL statements.

Also of the project I inherited, where I was told that the data layer was great, used stored procedures for everything, which ended up being things like:

code:

create procedure UpdateWorkOrderRecord(@itemId nvarchar(2000), @fieldName nvarchar(2000), @newvalue nvarchar(2000)) as
   exec('update WorkOrder set ' + @fieldName + ' = ' + @newvalue + ' where id = ' + @itemId')
go

code:

void btnSubmit_Click(object sender, EventArgs e) {
  cmd = conn.CreateCommand("UpdateWorkOrderRecord")
  cmd.Parameters.Add("@itemId", ItemId);
  cmd.Parameters.Add("@fieldName", "FirstName");
  cmd.Parameters.Add("@newValue", txtFirstName.Text);
  cmd.ExecuteCommand();

  cmd = conn.CreateCommand("UpdateWorkOrderRecord")
  cmd.Parameters.Add("@itemId", ItemId);
  cmd.Parameters.Add("@fieldName", "LastName");
  cmd.Parameters.Add("@newValue", txtLastName.Text);
  cmd.ExecuteCommand();

  etc etc
}

# ? Sep 29, 2016 22:29

vonnegutt: Aug 7, 2006; Hobocamp.

Polio Vax Scene posted:

Plus, wouldn't want to get on anyone's bad side in a tiny company by telling them their code is poo poo.

I'm dealing with this attitude now and I hate it so much. I have had to beg the other devs on the project to review my code properly, but they "don't want to be rude". Which explains why my last review of their code was met with such hostility. I'm not the best dev in the world, I make mistakes, and you have more experience with the codebase, of course I want you to point out where I'm duplicating functionality or whatever.

My most growth as a coder came from a team that had a deeply ingrained code review attitude - no code was accepted until you had at least one and preferably two thorough reviews. Everyone understood how important it was, so it was rare that you would have to wait more than a day for your code to be reviewed, so it didn't really slow things down either. It also had the bonus of everyone on the team having read most of the codebase. You don't end up with a soup of different coding styles and the whole codebase was really easy to jump around in.

# ? Sep 29, 2016 22:37

Cuntpunch: Oct 3, 2003; A monkey in a long line of kings

MisterZimbu posted:

This reminds me of a former DBA I had who refused to put indexes on tables because they slowed down queries too much, and provided us a "data access layer" that was pretty much C# code that concatenated together parts of SQL statements.

See, I've also run into DBAs who go "performance troubles? fuckit, slap another index on that massive table!" and then get a little grumpy and not-my-problem-ish when you tell them that table performance on large bulk modifications is unacceptable

# ? Sep 30, 2016 00:17

My Rhythmic Crotch: Jan 13, 2011

Messyass posted:

Yeah I guess you become amazing at "tuning" SQL when you don't have a sane design in the first place.

Exactly. The planner / optimizer / whatever is frequently confused by the stuff we do.

kedo posted:

Coding things on the internet was a lot more fun when no one knew what they were doing and "whatever hack works without crashing the server or users' browsers" was the most potent and often used weapon in your arsenal.

That sounds like his experience basically.

So when I rebuilt our main app, I very creatively named it "DumbApp V2" and the boss insisted it be called "DumbApp V3" because the "real first version" (not the one I replaced) was some activeX or java plugin or whatever from 1998. Eighteen years ago now, lol.

leper khan posted:

Version control has been a thing since the 80s, and your employer sounds frightful. I hope you make lots of money to deal with it.

It's not frightful. I work in an enormous tech company, and he's just a low level manager, of which there are hordes. So it's not like I'm working side-by-side, 24/7 with the guy. Other than the occasional disagreement over DB schema and stuff, he largely lets me design and build stuff as I see fit. The DB schema is just about the only thing I haven't clawed away from him. And like I said, he's a chill guy, and doesn't yell or scream when we disagree.

I've worked places where people solved disagreements in the parking lot or by screaming behind closed doors for 30 minutes. That was actually scary and retarded.

# ? Sep 30, 2016 02:02

spiritual bypass: Feb 19, 2008; Grimey Drawer

My Rhythmic Crotch posted:

screaming

That's getting into quit-without-notice territory preeeetty quickly

# ? Sep 30, 2016 02:47

My Rhythmic Crotch: Jan 13, 2011

rt4 posted:

That's getting into quit-without-notice territory preeeetty quickly

I have thought about making a thread about that experience. The culture was insane, the technology we were building was exotic, the software stack was so bad I really don't have the right words for it, and I got to live in another country for a few years doing it.

# ? Sep 30, 2016 03:15

spiritual bypass: Feb 19, 2008; Grimey Drawer

I'd read it

# ? Sep 30, 2016 03:21

Greatbacon: Apr 9, 2012; by Pragmatica

rt4 posted:

That's getting into quit-without-notice territory preeeetty quickly

Yeah, there is only one category of person in a company who should get upset enough about stuff to scream (the people with equity) and there is only one place they should actually scream.

# ? Sep 30, 2016 04:37

Gounads: Mar 13, 2013; Where am I?
How did I get here?

revmoo posted:

The worst developer is always:

you - 6months

I've certainly had the experience of seeing my name pop up after angrily typing in 'git blame'

# ? Sep 30, 2016 12:36

Hughlander: May 11, 2005

Greatbacon posted:

Yeah, there is only one category of person in a company who should get upset enough about stuff to scream (the people with equity) and there is only one place they should actually scream.

Being threatened or belittled, or screamed at are pretty much my pure quit on the spot. It's best for you long term and sends the message to everyone else that it's not acceptable in any regard.

# ? Sep 30, 2016 15:23

Che Delilas: Nov 23, 2009; FREE TIBET WEED

Gounads posted:

I've certainly had the experience of seeing my name pop up after angrily typing in 'git blame'

If you're diligent you can prevent that from ever happening.

# ? Sep 30, 2016 17:19

return0: Apr 11, 2007

Might be a language barrier but what do you mean scream/screaming?

The connotations of that here are always to do with pain, anguish, or emotional distress, but I guess you guys mean more aggressive/confrontational shouting and bullying. Lol if someone sticks around for that.

# ? Sep 30, 2016 20:07

necrobobsledder: Mar 21, 2005; Lay down your soul to the gods rock 'n roll; Nap Ghost

Shouting or yelling may be a more appropriate word with more restricted meanings.

Last job I had I was a few steps down from a newly hired executive that seemed to have explicitly hired to be aggressive and to push people around because so much of the company was too lax and nothing got done as a cultural value. Work-life balance doesn't mean that everyone instantly clocks out at 5:30 pm unless your datacenter is literally on fire by my standards, but that's what it seemed to mean for most people.

# ? Sep 30, 2016 20:37

Xarn: Jun 26, 2015

necrobobsledder posted:

Work-life balance doesn't mean that everyone instantly clocks out at 5:30 pm unless your datacenter is literally on fire by my standards, but that's what it seemed to mean for most people.

Assuming you work during the day, I don't see why not.

# ? Sep 30, 2016 20:39

spiritual bypass: Feb 19, 2008; Grimey Drawer

Why would anyone stay at work longer than absolutely necessary??

# ? Sep 30, 2016 21:05

return0: Apr 11, 2007

necrobobsledder posted:

Shouting or yelling may be a more appropriate word with more restricted meanings.

Last job I had I was a few steps down from a newly hired executive that seemed to have explicitly hired to be aggressive and to push people around because so much of the company was too lax and nothing got done as a cultural value. Work-life balance doesn't mean that everyone instantly clocks out at 5:30 pm unless your datacenter is literally on fire by my standards, but that's what it seemed to mean for most people.

In my contract I am salaried based in a 40 hour week, I'll stay more maybe but it has to be give and take, I'm not just going to do it for free. Why would anyone?

# ? Sep 30, 2016 21:10

revmoo: May 25, 2006; #basta

I can't imagine ever working more than 40 hours. I've been doing a 60 hour deathmarch this month and I'm interviewing at other companies as a result.

Anyone working more than 40 hours is literally lowering their market value by cutting their own hourly rate. It's nonsense.

# ? Sep 30, 2016 21:11

Gounads: Mar 13, 2013; Where am I?
How did I get here?

Depends on the environment. When I had a corporate job, it was 40 hour. Lately, in startup environment, I've been doing more than 40 hours. October 18th I switch back to sweet sweet hourly contracting.

# ? Sep 30, 2016 21:33

necrobobsledder: Mar 21, 2005; Lay down your soul to the gods rock 'n roll; Nap Ghost

I was meaning that nothing could get people to be flexible (if you stay later, no big deal coming in later, for example) in an environment where there were constant fires and one of the biggest problems with schedule slip was because nobody had any urgency to finish their tasks and low performers just couldn't get canned. The situation was how people think government employees work, except this is private sector.

# ? Sep 30, 2016 21:33

Adbot: ADBOT LOVES YOU

# ? May 10, 2024 01:42

revmoo: May 25, 2006; #basta

Gounads posted:

Depends on the environment. When I had a corporate job, it was 40 hour. Lately, in startup environment, I've been doing more than 40 hours. October 18th I switch back to sweet sweet hourly contracting.

Any situation where you're taking a percentage of company ownership is going to be different than a normal job though.

# ? Sep 30, 2016 21:47

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Working in Development: Leaving sounds like a great decision.

«‹›434 »