|
Volmarias posted:Living the dream. You have no idea how good it feels to smash "DECLINE" over and over again on meeting requests between 2:30 - 5:30.
|
#
?
Sep 5, 2017 15:16
|
|
|
|
| # ? May 18, 2024 00:07 |
|
|
Keetron posted:You can leave early without anyone knowing! I'm gonna spend that "capital" on the doctor's appointment I have this afternoon, so no dice. KoRMaK posted:Why are you trying to ruin it for your team?? You joke but I've been having nightmares about failing my team recently  Mniot posted:Anticipating the follow-up post where Pollyana's entire team was laid off... Honestly, I'm surprised every morning that I open my inbox and there isn't an announcement that we're all fired.
|
|
#
?
Sep 5, 2017 15:22
|
|
|
Volmarias posted:Living the dream. Except when your boss says everyone should head out a bit early before a long weekend, which ends up meaning 'leave when you normally do'
|
|
#
?
Sep 5, 2017 15:29
|
|
|
Pollyanna posted:Honestly, I'm surprised every morning that I open my inbox and there isn't an announcement that we're all fired. That's... unlikely to be how that'd happen. It'd be an all-team meeting with the leader's direct report and - oh hey what's that HR person doing here? Oh.
|
|
#
?
Sep 5, 2017 15:31
|
|
|
ChickenWing posted:Except when your boss says everyone should head out a bit early before a long weekend, which ends up meaning 'leave when you normally do' Yeah, but that's only like, 4 times a year.
|
|
#
?
Sep 5, 2017 15:35
|
|
|
ChickenWing posted:Except when your boss says everyone should head out a bit early before a long weekend, which ends up meaning 'leave when you normally do' I take that to mean exactly what they said, every time. Hey, they offered  Munkeymon posted:That's... unlikely to be how that'd happen. It'd be an all-team meeting with the leader's direct report and - oh hey what's that HR person doing here? Oh. The company has basically laid off ~1000 employees via rumors and email before, and these are the same people that kicked out our head manager+VP and then refused to refill those positions, soooo...
|
|
#
?
Sep 5, 2017 15:42
|
|
|
ChickenWing posted:Except when your boss says everyone should head out a bit early before a long weekend, which ends up meaning 'leave when you normally do' So, I guess you still end up ahead? But really I was talking about the "there's only a 3 hour block in my day where others may interrupt my productivity, and the rest of it is mine" bit. Someone post that "quick meeting" calendar picture again.
|
|
#
?
Sep 5, 2017 16:12
|
|
|
Volmarias posted:So, I guess you still end up ahead? oh yeah the pros most certainly outweigh the cons, the only reason I mention it is because it was recently a long weekend and I'm still nursing the semi-stale fake resentment 
|
|
#
?
Sep 5, 2017 16:15
|
|
|
ChickenWing posted:Except when your boss says everyone should head out a bit early before a long weekend, which ends up meaning 'leave when you normally do' At least for me, those days are pretty predictable, so I head into work later than I normally would on those days.
|
|
#
?
Sep 5, 2017 16:23
|
|
|
Munkeymon posted:That's... unlikely to be how that'd happen. It'd be an all-team meeting with the leader's direct report and - oh hey what's that HR person doing here? Oh.
|
|
#
?
Sep 5, 2017 16:26
|
|
|
Vulture Culture posted:Unless you work for Oracle, where you might get RIF'd by robocall  and [img-thats-our-oracle] combined somehow
|
|
#
?
Sep 5, 2017 16:30
|
|
|
Man, having to think for myself at work is the worst  I was assigned a project last week to develop a tool to make our testers' jobs easier. They use ReadyAPI to test our endpoints, so I'm developing a Groovy tool to help organize endpoint templates and apply changes in a consistent fashion. There's two problems with this 1) The QA Lead and the Project Lead have incredibly different ideas on how this should work, and didn't really realize this before asking me to work on the project. Until last night, I was getting a conflicting set of requirements from each of them. Of course, they told me that I should be listening to the QA Lead because the end result is ultimately for him, but the Project Lead has very definite opinions about how the end result should look and is also kinda sorta everyone's boss on this project. Luckily, I think this was kinda sorta resolved, and everyone at least knows what everyone else wants. Whether or not this will result in less confusion down the road is, of course, a complete guess 2) Holy poo poo I am not used to having to do all this stuff on my own and I feel like I'm barely treading water. Everyone seems happy with me, so I assume I'm doing something reasonably correctly, but every day before my little demo is 15 minutes of "oh gently caress is today the day I get told that everything is wrong and I've hosed it all up and this was a colossal waste of time?" Please I just want someone to make me a jira ticket that I can complete and then sometimes ask me to help design stuff I don't actually want responsibility aaaaaaaaaaaaaaaaaaaaa 
|
|
#
?
Sep 6, 2017 14:35
|
|
|
ChickenWing posted:Man, having to think for myself at work is the worst
|
|
#
?
Sep 6, 2017 15:05
|
|
|
At least you're hopefully getting a good feedback loop going, with those frequent demos. That should mean that, at the end, they won't suddenly go, "This is all wrong! Start over!"
|
|
#
?
Sep 6, 2017 15:17
|
|
|
Congrats on rising above the junior dev level op!
|
|
#
?
Sep 6, 2017 15:17
|
|
|
Volmarias posted:Living the dream.
|
|
#
?
Sep 6, 2017 15:19
|
|
|
CPColin posted:At least you're hopefully getting a good feedback loop going, with those frequent demos. That should mean that, at the end, they won't suddenly go, "This is all wrong! Start over!" I've finished up the basic framework and am doing a test implementation now, so that'll be the real indicator of how well this turns out. I have run into a brick wall that is an utter lack of motivation though, so I'm currently working on bulling my way through that  Volmarias posted:Congrats on rising above the junior dev level op! Now to work on getting the appropriate job title and salary I've been taking on more responsibility lately and there's a bunch of contractors who keep asking me how the promotion process works here and what my level equates to in the dev hierarchy and why I'm still considered a junior developer, which is nice. Unfortunately, given that I'm still technically a consultant, there's a whole bunch of non-developer hoops I need to jump through.
|
|
#
?
Sep 6, 2017 15:33
|
|
|
ChickenWing posted:I've finished up the basic framework and am doing a test implementation now, so that'll be the real indicator of how well this turns out. I have run into a brick wall that is an utter lack of motivation though, so I'm currently working on bulling my way through that Also if it helps your process, make your own tickets in jira/phone todo list/notebook/whatever. It's the productivity secret everyone will tell you about.
|
|
#
?
Sep 6, 2017 15:50
|
|
|
leper khan posted:Also if it helps your process, make your own tickets in jira/phone todo list/notebook/whatever. This is the most work my notebook has done since I got it. I realized on day two that this was not going to be a 'keep everything organized in my head' thing
|
|
#
?
Sep 6, 2017 16:04
|
|
|
Oh poo poo test implementation looks like it's working the way I wanted it to gently caress yeah demo isn't going to be a complete failure hype hypeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee
|
|
#
?
Sep 6, 2017 19:31
|
|
|
ChickenWing posted:Oh poo poo test implementation looks like it's working the way I wanted it to gently caress yeah demo isn't going to be a complete failure hype hypeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee Good luck!
|
|
#
?
Sep 6, 2017 19:38
|
|
|
ChickenWing posted:Oh poo poo test implementation looks like it's working the way I wanted it to gently caress yeah demo isn't going to be a complete failure hype hypeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee Repeat after me: "Works on my machine."
|
|
#
?
Sep 6, 2017 21:35
|
|
|
Che Delilas posted:Repeat after me: "Works on my machine." My machine is prod*
|
|
#
?
Sep 6, 2017 21:45
|
|
|
I suggested at one point to run lower environments and potentially production on developers' Macbook Pros where everyone runs Kubernetes' kubeadm in a hypervisor with no way to login to avoid some security issues, and we'd reschedule pods after hours into AWS to help cut down on our really stupidly high infrastructure costs. It turns out the primary problem is that we wouldn't have an easy way to ensure solid connectivity over wifi to our monolithic database.
|
|
#
?
Sep 6, 2017 21:49
|
|
|
Another rant. Sorry everybody. I hate Apache Spark so loving much. I'm so sick of it. I spent at least a solid 40-hour week trying to get it to read data from S3. Unfortunately, you need to use v1.7.4 from 2012 of the Hadoop-S3 jar and v1.7.81 of the AWSjdk downloaded from http://fuckery.clownpenis.fart from before Jan 16 and and and. Finally I said gently caress it and dumped a small subset of our data to a CSV and tried to run it, but Spark won't push your local data to the workers. Okay, whatever, I'll read from the DB directly. Now I'm fighting with the JDBC drivers. I already feel like enough of a moron without being forced to set up and use a cluster.
|
|
#
?
Sep 7, 2017 00:19
|
|
|
Jo posted:Another rant. Sorry everybody. We've had some major woes with Spark as well. We have it somewhat stable, but really there's a big lack of a knowledge base with it. We use it on top of Cassandra so that also doesn't help. It also doesn't help that it's 1 part of our tech stack that we touch maybe 4 times a year so it's almost "out of site, out of mind". I don't hate it at the moment, but perhaps I would if I had to work with it every day.
|
|
#
?
Sep 7, 2017 04:25
|
|
|
Apache Spark is the PHP of big data applications. On so many levels. - everyone is using it or used it at some point - it's full of bugs, quirks, and surprises - it tries to do everything, but - it never is the best solution for the particular problem you are trying to solve - it's poo poo - but we're gonna be stuck with it for the foreseeable future.
|
|
#
?
Sep 7, 2017 09:24
|
|
|
Jo posted:Another rant. Sorry everybody. I've been using NiFi for prototyping lately and it's been a fairly positive experience. It has its own quirks, but I was able to pick it up pretty quickly and it comes with a lot of processors out of the box for doing things that you'd have to supply yourself with Spark. Writing more processors is also pretty easy - I was able to write a couple of processors for munging JSON in like a day or two after having not written any Java in over a decade. If you just need a basic ETL pipeline and don't want to fumble with workers and Zookeeper and JAR hell, maybe take a look at it. There's a lot of overlap between it and Spark for simple data manipulation jobs.
|
|
#
?
Sep 7, 2017 14:53
|
|
|
necrobobsledder posted:I suggested at one point to run ... production on developers' Macbook Pros
|
|
#
?
Sep 7, 2017 15:15
|
|
|
Vulture Culture posted:no don't run production this way
|
|
#
?
Sep 7, 2017 15:47
|
|
|
chutwig posted:I've been using NiFi for prototyping lately and it's been a fairly positive experience. It has its own quirks, but I was able to pick it up pretty quickly and it comes with a lot of processors out of the box for doing things that you'd have to supply yourself with Spark. Writing more processors is also pretty easy - I was able to write a couple of processors for munging JSON in like a day or two after having not written any Java in over a decade. If you just need a basic ETL pipeline and don't want to fumble with workers and Zookeeper and JAR hell, maybe take a look at it. There's a lot of overlap between it and Spark for simple data manipulation jobs. I'll second this. I was briefly on a big data project that used a Kafka/NiFi/Hadoop, and NiFi was a blast to work with. Like you I hadn't touched Java since forever, but was writing custom components after a few days. Dunno how it compares to Spark.
|
|
#
?
Sep 7, 2017 15:50
|
|
|
necrobobsledder posted:The place is a complete and utter clusterfuck and our SLAs would probably be met better by developers' laptops than our actual servers that we can't afford anyway. This is horrifying. "Sorry, customerdude, we dropped your production server at the bus stop. Expect a few days of downtime."
|
|
#
?
Sep 7, 2017 16:04
|
|
|
No no, in the world of cloud computed dockerswarm containerization and bespoke artisanal parallelized Hadoop data pipelines backed by Typescript and deep machine learning, I can do literally loving anything and it will still work because our revenue is fueled entirely by buzzwords! 
|
|
#
?
Sep 7, 2017 16:08
|
|
|
a foolish pianist posted:This is horrifying. "Sorry, customerdude, we dropped your production server at the bus stop. Expect a few days of downtime." Pollyanna posted:No no, in the world of cloud computed dockerswarm containerization and bespoke artisanal parallelized Hadoop data pipelines backed by Typescript and deep machine learning, I can do literally loving anything and it will still work because our revenue is fueled entirely by buzzwords!
|
|
#
?
Sep 7, 2017 17:15
|
|
|
BabyFur Denny posted:Apache Spark is the PHP of big data applications. On so many levels. Hmm I dunno. What's better at general etl/analysis than spark?
|
|
#
?
Sep 7, 2017 18:07
|
|
|
return0 posted:Hmm I dunno. What's better at general etl/analysis than spark? I'd take just about anything over Spark at this point. I spent about two hours trying to do a concat/group by of a string in Spark. Last month I had a project I was going to do in it, but gave up and wrote the netcode + job system in the time it has taken me to get Spark to do the concat I mentioned above. It's a surreal, frustrating experience where I feel like I'm in grad school again defending a dissertation and unable to get anything to work or answer questions. It should be so simple! Why do I need a special class of function to do a map? Why can't I use a plain Java lambda on this agg? Why can't map or flatmap take a lambda? Oh. It crashed after 18 hours. Okay. 
|
|
#
?
Sep 8, 2017 00:41
|
|
|
Is there a particular reason they're using Spark?
|
|
#
?
Sep 8, 2017 01:17
|
|
|
Pollyanna posted:Is there a particular reason they're using Spark? Because some exec told some other exec on the golf course that they used Spark.
|
|
#
?
Sep 8, 2017 01:39
|
|
|
GutBomb posted:Because some exec told some other exec on the golf course that they used Spark. In short, this. I've expressed my opposition. We do have a few TB of data to process, so I guess they think it's the only option.
|
|
#
?
Sep 8, 2017 04:14
|
|
|
|
| # ? May 18, 2024 00:07 |
|
|
I use Spark extensively at work to process decent sized datasets, and have found it pretty good tbh. One difference is I've used the Scala API exclusively, and haven't touched the java API. I don't recognise your comments about map/flatmap not taking lambda from my experience, for example. I have read anecdotally the Scala API is more consistent. I've used Hadoop for similar batch ETL in the past (with python over Hadoop streaming) and find Spark to be light years better. Maybe try Scala?
|
|
#
?
Sep 8, 2017 07:21
|
|