Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
MC Fruit Stripe
Nov 26, 2002

around and around we go

CLAM DOWN posted:

I alternate on-call weekends with another guy and get paid for it :smug:
Plot twist - he's talking about Thermopyle and daddy warbucks is paying his moderators now.

e: Yep, that's how it works, you make a throw away joke and you're going right to the top of the page.

Adbot
ADBOT LOVES YOU

dogstile
May 1, 2012

fucking clocks
how do they work?

Heh, this reminds me of something i'm currently doing in addition to my IT responsibilities.

I've been asked to convert a bunch of excel sheets from various companies that we work with into a workable data format and instead i've given them a template to fill out and asked them to fill it in.

Now i'm just converting the excel sheet i'm getting into a CSV and throwing it straight into our data, but i'm about to automate that. As the IT responsibilities right now are really limited (small business, I automated almost everything else) i'm getting really close to the story of that guy who automated his job.

Automation is kinda great tbh

dogstile
May 1, 2012

fucking clocks
how do they work?

Heh, this reminds me of something i'm currently doing in addition to my IT responsibilities.

I've been asked to convert a bunch of excel sheets from various companies that we work with into a workable data format and instead i've given them a template to fill out and asked them to fill it in.

Now i'm just converting the excel sheet i'm getting into a CSV and throwing it straight into our data, but i'm about to automate that. As the IT responsibilities right now are really limited (small business, I automated almost everything else) i'm getting really close to the story of that guy who automated his job.

Automation is kinda great tbh

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer
I’m starting to think that as an IT Director I’ve done all I can for this company and it has been less than a year. We modernized a few systems, set a real help desk ticketing system up, and moved some stuff to Azure, but the big thing staring this company in the face is the need to sanitize the data in their ERP or, more importantly, replace their current ERP. It is an SAP implementation from 1998 that is so heavily customized without documentation that things are a mess. The relatively new CFO and myself are really pushing the CEO to realize this, but CEO refuses to accept the cost of things. CEO had CFO and me get on a call with the CTO at a partner company, because hey thought the other company did things better without an ERP. Talk to the CTO, learn that while they have a system in production for WMS, everything feeds into SAP S/4 HANA. Report these findings to CEO along with how it works and all they respond with is “hm I don’t think that’s how it works”

Why have me here if you’re going to disregard my input in the one field I’m supposed to manage for you?

Thank you for letting me rant IT thread

Zapf Dingbat
Jan 9, 2001


Thanatosian posted:

You... you are helpdesk, but expected to be on-call 24/7?

May I suggest https://www.indeed.com? Or just look for some contract work. Your workplace is dysfunctional as fuuuuuuuuuuuuuuuuuuuuuuuuuck.

Yeah, it's a complex situation, with an admittedly not complex solution.

My life was kind of a wreck even before the recession, then the market tanked, and that began a good 5 years or so of semi-employment, living from paycheck to paycheck, being stuck in a house I bought right before the crash, wondering what the hell to do with myself.In 2011 I finally realized not everyone has clinical depression and meds helped me out of that. I had given up on the original industry I set out to do, broadcast television, because I found out I wasn't really all that interest in it and you have to be god damned psycho to work in that industry, and eventually had the gumption to get up and figure out my career.

So as of about 3 years ago I decide to suck it up and get my A+, and all of a sudden I start getting places in job interviews, instead of negative nowhere. Turns out I actually liked IT work and literally turning my hobby into my career wasn't such a bad thing. I hadn't even considered it due to the emo problems. I eventually land a job in helpdesk at a local company in my lovely little town.

I had fun cutting my teeth in this IT cowboy company. It was bonkers when I first started. But I felt like I could make a whole crapload of mistakes without much consequence, so good place to learn. I eventually got my Net+ and CCENT. I eventually planned on finding a better job because I felt like I was attached to the Tasmanian devil, but I don't regret getting the job.

They allowed me to work from home and move to Atlanta, which was the entire goal for me to begin with. My wife can actually find work up here and I have lots of opportunities, but I'm running into a little trouble finishing my CCNA. The distraction of moving out of my hometown for the first time, being in a place I only vacationed in before, and having to sell a house with an underwater loan has finally subsided and I'm able to concentrate on studying again.

I'm not real confident to go out looking for something until I get that CCNA, though. I'm not sure what being a mid-30s guy with 3 years helpdesk and a CCENT will get me in this city. I probably get paid about average at $40k now that I live where I live. I know I just have to get them certs and I'll be out.

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Kashuno posted:

I’m starting to think that as an IT Director I’ve done all I can for this company and it has been less than a year. We modernized a few systems, set a real help desk ticketing system up, and moved some stuff to Azure, but the big thing staring this company in the face is the need to sanitize the data in their ERP or, more importantly, replace their current ERP. It is an SAP implementation from 1998 that is so heavily customized without documentation that things are a mess. The relatively new CFO and myself are really pushing the CEO to realize this, but CEO refuses to accept the cost of things. CEO had CFO and me get on a call with the CTO at a partner company, because hey thought the other company did things better without an ERP. Talk to the CTO, learn that while they have a system in production for WMS, everything feeds into SAP S/4 HANA. Report these findings to CEO along with how it works and all they respond with is “hm I don’t think that’s how it works”

Why have me here if you’re going to disregard my input in the one field I’m supposed to manage for you?

Thank you for letting me rant IT thread

I want to rant on ERP implementation at clueless companies so bad...

Vargatron
Apr 19, 2008

MRAZZLE DAZZLE


Bob Morales posted:

I want to rant on ERP implementation at clueless companies so bad...

I mean if you want, I can start. I've done a few implementations in a past life and they were all various stages of hosed.

It's why I drink so much these days.

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer

Bob Morales posted:

I want to rant on ERP implementation at clueless companies so bad...

This company tried to implement SAP Warehouse Management last year and spent dozens of thousands on the project, along with a ton of manhours, and ultimately had to scrap it because loving around with the WMS stuff broke so many parts of their SAP system and they had no idea why or how to fix.

Vargatron
Apr 19, 2008

MRAZZLE DAZZLE


Something tells me they failed to do any kind of value stream or process mapping when doing this implementation.

Judge Schnoopy
Nov 2, 2005

dont even TRY it, pal

dogstile posted:

Now i'm just converting the excel sheet i'm getting into a CSV and throwing it straight into our data, but i'm about to automate that. As the IT responsibilities right now are really limited (small business, I automated almost everything else) i'm getting really close to the story of that guy who automated his job.

Automation is kinda great tbh

The most useful thing I've learned this year is working with JSON. Csv is fine for dumping flat output but gets really limiting when you want to read that data back in and make it useful.

I can't recommend enough if you're automating things to learn arrays, hash tables, and JSON. It also pushes you to rethink your code in terms of objects which opens all sorts of opportunities for compares and logic-based processing.

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer

Vargatron posted:

Something tells me they failed to do any kind of value stream or process mapping when doing this implementation.

ASK me about everyone having a role called "SAP_ALL" that gives them access to everything in the ERP at any given time

Thom and the Heads
Oct 27, 2010

Farscape is actually pretty cool.
ive looked up whatever ERP is like 90 times due to this thread and each time, I read the first two sentences of the wikipedia page and my eyes glaze over. I then close the tab.

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Bob Morales posted:

I want to rant on ERP implementation at clueless companies so bad...

The company knows, or at least thinks, they need a new ERP system. Or at least they say they do.

But they don't ever want to understand WHY they need a new one. Or don't need a new one.

The first thing to identify, is what the problems are with the new system.

These are GOOD problems:

It crashes
It's slow
It doesn't interface with X
We can't get the data out of it that we want

These are BAD problems:

The data is all trash
Process X has too many steps/problems
Only a few people know how to use it
There are no good procedures present

The good problems are easy. You can fix them with troubleshooting, buying faster hardware, paying a programmer to fix a problem...these are simple things.

The other problems show deep-rooted problems at the organization level. You aren't going to fix these by spending a ton of money and bringing in consultants to implement magical ERP software.

You have to clean the data up first. Garbage in, garbage out. If you bring all that trash into your new system, it's going run like trash. Data being inconsistent (data in different formats, database/design level issues) isn't going to be easy to export and import into the new one. But the key is, if nobody in the company gives a gently caress about the data, it's just going to turn to poo poo as you let your users back into it. If nobody gave a gently caress before, why would they give a gently caress now?

The data is always such garbage. People develop their own systems over time. If a customer name has a + or * in it, they're behind on credit. Or they closed up shop. Or they moved and have a new address. Who knows except the people who work with that customer. Maybe they have a field called 'Customer Type', and it's being used for 500 things. Discount codes, taxable status...all things that should have had their own fields. The best is determining a value based one 2 or 3 fields. Why not just make one field that gives you this info? Usually the case of 'well, let's do this for this one customer...' and then little bit of laziness creeps into the entire system.

When you get the company together to talk about 'so how does your accounts payable process work? How do you receive and then fill orders?", you're going to get 5 different answers from 5 different people. And technically, none of them are wrong. Why? These people don't know what the next is doing. Part of why the process is such poo poo right now. Once they talk about this and figure out what's going on, they magically fix the process and see the system isn't the problem! "Oh, so your location charges labor by the minute and manually adjusts the price, instead of using '.5' for 30 minutes of labor...no wonder none of your poo poo makes sense..."

"Only a few people know how to use it", that's a huge problem. Perhaps only the people who have been here a long time now how things work. Or they've been working on second, then third, and then fourth-hand information. They know what they do, but they don't know why they do it. The worst part - if you do get new people trained in the 'right way', it might not help because you're using the software WRONG in the first place. "Well, we don't actually do it that way because..." A good example of this was in our old system, nobody did a return or credit. They created invoices for negative amounts. THERE ARE DIFFERENT TYPES OF TRANSACTIONS MAYBE YOU SHOULD TRY NOT RIGGING EVERYTHING AS A SALE. Invoices to dummy accounts and poo poo like that. Absolute loving nightmare when you try to bring just 1 year of transaction history into a new system as the actual types of transactions they should be.

Old habits die hard. Even after we got a new system, people would record poo poo on paper all day and enter things in at the end of the week or end of the day. Why? because they were lazy. That's how they did it before. "We're out of stock on this part..." Which is followed up by "Yea, we never keep those amounts up to date. I'll adjust some into inventory if you want to use some of those parts" WHY CANT YOU JUST DO IT THE RIGHT WAY

Next up, getting the right people involved in the new system. So you want to let the people who basically created this pile of poo poo (or stood by and let it happen), create your new system? BRILLIANT

The first thing they try to do? Re-create your old system, in your new system. Why not!? It sounds like a great idea at first. Less for people to learn! It's familiar. Bad idea. You're just going to create a lovely version of your existing system that people are going to hate even more. Why would Applebee's try to re-create McDonald's menu? McDonald's can do it faster and cheaper. Plus they're already doing it.

You can't re-create the old system. You need to make a new one. A better one. It's work. You have to make all the old stuff work. Hopefully in a better way. The old system has been running pretty smoothly for the last 10 years. Even if it's not smooth, people know how to work around all the bugs. People have ran everything through this system. It's tested. It looks like poo poo, but it works. Somehow. So when the new system is running, 2 months have passed and some little things are still going wrong, new things being discovered, people will get frustrated. Remind them that the old system ran for 10 years of testing, it's only been 2 months, and it's getting better every day.

It's going to be different. It can't be better if it doesn't change. But people have to put some effort into learning a new system. The worst part is they've lived in this system for the last 10-15 years. You're basically dropping them off in Mexico with no translator. People have to understand what they are actually doing, not what they 'think' they do. "I hit menu 4, 5, 3, and then enter a check" This person might not get they go into Accounts Payable, then Customers, then 'Pay Bills'. They've got muscle memory so they can get to any screen in the system in 2 seconds. Now they're using a mouse, choosing menu options, clicking into unknown territory. Plus this is 10 times slower than it used to be, so they're not going to like it.

You've switched. Now the great thing is everyone has so much better access to data. People are making their own reports. People don't need to come to IT. It seems nice. Everything is running smooth. Then the people in sales or accounting want more power. They want to be able to customize the system. They want new fields. They want new layouts. They don't give a gently caress about how it affects anyone else. They don't understand the repercussions of changes. They don't understand databases so they want to make fields like "Commercial Customer" or "Customer has not paid in 30 days", that just don't make any drat sense. They want to override built-in features of the system with their own. Using the systems default way of doing something is so much easier than trying to work around it. But they don't care. Management says IT is being the bad guy and not letting people do their jobs easier (no matter how much time it really costs them). These monkey fucks get access to make changes, or IT is forced to do things no jackass in their right mind would. You remind them that this eerily reminds you of the old system, and it's going to cause the same problems you used to have....

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer
I want to take that entire rant wholesale and present it our CEO. The thing I hear constantly is "SAP sucks." and I have to remind them, repeatedly, that it is one of the top of the line ERPs for a reason. No matter what we go with for another system, none of it matters, at all, if we take the same lovely data, lovely business processes, and lovely documentation issues/customization into a new system. One of the big things I implemented this year was requiring anyone requesting any change to SAP, no matter how small, to fill out an SAP change request form. They needed to spell out what they wanted changed and why they felt the change was necessary. Ever since then, requests for changes to SAP have almost entirely dried up. We've had a few people try and just submit their request to the SAP Programmer directly without submitting the ticket, but those have been met with "If there is no form, I am not authorized to change anything." It won't really help this current system much, but if I get it into people's heads that they are required to fill out paperwork that is well documented before we move to a new system, when we do finally make the jump it'll be easier to force that policy from day 1.

Vargatron
Apr 19, 2008

MRAZZLE DAZZLE



My last company was a tier 2 automotive supplier, which meant that we supplied parts to customers who would assemble them or add their own doodads, and then send those goods up to a car manufacturer or OEM. The operations manager was a Toyota Productions Systems fanatic. Everything had to be done with lean management/kanban systems. Everything was to be sequenced and level loaded so the manufacturing floor could have a steady amount of work with high quality standards all around.

The major problem with this was that our customers did not place orders in a way that facilitated a level loaded schedule. Contractually we agreed to supply a certain quantity of parts per week, with a +/- 20% "flex" value which allowed customers to order over or under by that amount without repercussions. If they wanted to increase order volumes above this, then they would be subject to a "break in fee" to account for the rescheduling. We never actually enforced this. The directive from senior Ops was for the ERP system to automatically "level load" orders to the production floor and distribute them in even buckets. The problem with this was that getting five orders of 2000 parts didn't take into account that some machines would have like a 2 hour changeover time so you would have more downtime than actual productive time if you followed the system's suggestions.

After many months of discussions, it was decided that this level loading style of planning would be done away with in favor of traditional MRP planning. You could still do level loading if you wanted, but this was to be done manually. When we were doing the value stream mapping/training sessions, the we became painfully aware that nobody in the production control department actually knew MRP planning. Everything was "well I do these three things and hit the suggest orders button and that tells me what to run". We had to teach planning professionals how to handle lead times and order fluctuations. Also had to go through the master data cleanup and normalization like you described in your post. This was an absolute loving nightmare.

Maybe the biggest thing that was wrong with the process was that our warehouse/production floor was all ONE inventory location, called WIP. This made it impossible to actually find inventory anywhere in the plant. Not only that, but raw materials were automatically consumed by the inventory system even if they were physically not being used. If there was any kind of inventory discrepancy or arrerage, the system would automatically consume inventory as soon as it was added to the system. This one functional flaw in the system was causing thousands of dollars in inventory adjustments PER WEEK. We had to fight tooth and nail to get discreet inventory locations set up, and even this failed because the production floor REFUSED to perform inventory transfers between the bins, citing "this is too much work and we can't keep up".

I left the company before the actual SAP implementation started going, but I was there when we performed a reimplementation of the current MRP system and as far as I know, the SAP team is having the exact same issues as I had.

Kashuno
Oct 9, 2012

Where the hell is my SWORD?
Grimey Drawer

quote:

well I do these three things and hit the suggest orders button and that tells me what to run

Please stop I beg you I can't take anymore of this

:negative:

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

Kashuno posted:

I want to take that entire rant wholesale and present it our CEO. The thing I hear constantly is "SAP sucks." and I have to remind them, repeatedly, that it is one of the top of the line ERPs for a reason. No matter what we go with for another system, none of it matters, at all, if we take the same lovely data, lovely business processes, and lovely documentation issues/customization into a new system. One of the big things I implemented this year was requiring anyone requesting any change to SAP, no matter how small, to fill out an SAP change request form. They needed to spell out what they wanted changed and why they felt the change was necessary. Ever since then, requests for changes to SAP have almost entirely dried up. We've had a few people try and just submit their request to the SAP Programmer directly without submitting the ticket, but those have been met with "If there is no form, I am not authorized to change anything." It won't really help this current system much, but if I get it into people's heads that they are required to fill out paperwork that is well documented before we move to a new system, when we do finally make the jump it'll be easier to force that policy from day 1.

For just $5,000 I'll put on a suit, visit and do a PowerPoint.

deedee megadoodoo
Sep 28, 2000
Two roads diverged in a wood, and I, I took the one to Flavortown, and that has made all the difference.


22 Eargesplitten posted:

Speaking of programming, I am thinking about creating an AWS instance to pull data from the Steam API into a MongoDB database, export as an Excel file, and load it into my MSSQL database at work for a training exercise using SSIS. I know it seems kind of convoluted, but it’s really a resume builder. Is there anything I need to watch out for?

I know Cavern of COBOL exists, but I want this thread’s experience.

Look into using DynamoDB.

Also you don’t need an entire ec2 instance just for a simple ETL job. You can do that with lambda or batch.

Vargatron
Apr 19, 2008

MRAZZLE DAZZLE


Also every business thinks they're a 100% special snowflake operation and that they need a custom software solution to manage their idiot management team production process.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

very stable genius posted:

Look into using DynamoDB.

Also you don’t need an entire ec2 instance just for a simple ETL job. You can do that with lambda or batch.
Or Glue, if the stuff you're doing maps well to Spark.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.
The best are companies that use a full-fledged on-premises ERP system to manage employee benefits enrollment and literally nothing else.

Vargatron
Apr 19, 2008

MRAZZLE DAZZLE


Vulture Culture posted:

The best are companies that use a full-fledged on-premises ERP system to manage employee benefits enrollment and literally nothing else.

Ah so you too work in academia.

Proteus Jones
Feb 28, 2013



Vulture Culture posted:

The best are companies that use a full-fledged on-premises ERP system to manage employee benefits enrollment and literally nothing else.

We also use it for scheduling time off.

The Iron Rose
May 12, 2012

:minnie: Cat Army :minnie:
It is now the start of my fourth day in my new job, and as I still don't have access into our ticketing system I've literally been paid a grand so far to sit on my rear end and do absolutely nothing other than troubleshoot a few printer problems :v:

consultancy gigs always this ridiculous? Fulltime and I bill by the day too...

orange sky
May 7, 2007

No marketing emails on SA :)

orange sky fucked around with this message at 17:06 on Jul 10, 2018

Zaepho
Oct 31, 2013

22 Eargesplitten posted:

Yeah, I meant .csv when I said Excel file. My mistake, it's just that they open in Excel and one of the weekly scheduled report updates I do involves pulling from one poo poo system as an .xls and then opening it in Excel and saving as a .csv since the lovely system isn't capable of exporting .csv files. Then I put them in the import folder and upload them into the system we should be using for everything in the first place.

install-module PSExcel

seriously, automate that terrible step.

H110Hawk
Dec 28, 2006

Author deleted the post.

H110Hawk fucked around with this message at 17:16 on Jul 10, 2018

orange sky
May 7, 2007

H110Hawk posted:

Yeah but what is full list for the USA commercial market with the common bundle of modules? Is it on your website?

Nope, not on our website (which imo isn't pretty good as of now). I don't have that information but I'd say the full solution is around 50$/device/year list price.

This is in no way official by the way, I still haven't figured out the pricing since I don't deal with it directly and just leave it to sales.

orange sky fucked around with this message at 15:44 on Jul 10, 2018

Ham Equity
Apr 16, 2013

The first thing we do, let's kill all the cars.
Grimey Drawer

Zapf Dingbat posted:

Yeah, it's a complex situation, with an admittedly not complex solution.

My life was kind of a wreck even before the recession, then the market tanked, and that began a good 5 years or so of semi-employment, living from paycheck to paycheck, being stuck in a house I bought right before the crash, wondering what the hell to do with myself.In 2011 I finally realized not everyone has clinical depression and meds helped me out of that. I had given up on the original industry I set out to do, broadcast television, because I found out I wasn't really all that interest in it and you have to be god damned psycho to work in that industry, and eventually had the gumption to get up and figure out my career.

So as of about 3 years ago I decide to suck it up and get my A+, and all of a sudden I start getting places in job interviews, instead of negative nowhere. Turns out I actually liked IT work and literally turning my hobby into my career wasn't such a bad thing. I hadn't even considered it due to the emo problems. I eventually land a job in helpdesk at a local company in my lovely little town.

I had fun cutting my teeth in this IT cowboy company. It was bonkers when I first started. But I felt like I could make a whole crapload of mistakes without much consequence, so good place to learn. I eventually got my Net+ and CCENT. I eventually planned on finding a better job because I felt like I was attached to the Tasmanian devil, but I don't regret getting the job.

They allowed me to work from home and move to Atlanta, which was the entire goal for me to begin with. My wife can actually find work up here and I have lots of opportunities, but I'm running into a little trouble finishing my CCNA. The distraction of moving out of my hometown for the first time, being in a place I only vacationed in before, and having to sell a house with an underwater loan has finally subsided and I'm able to concentrate on studying again.

I'm not real confident to go out looking for something until I get that CCNA, though. I'm not sure what being a mid-30s guy with 3 years helpdesk and a CCENT will get me in this city. I probably get paid about average at $40k now that I live where I live. I know I just have to get them certs and I'll be out.
The worst thing they'll say is "no."

CLAM DOWN
Feb 13, 2007




orange sky posted:

marketing email

Source your quotes/marketing emails.

CLAM DOWN fucked around with this message at 17:48 on Jul 10, 2018

H110Hawk
Dec 28, 2006

Author deleted the post.

H110Hawk fucked around with this message at 17:16 on Jul 10, 2018

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

Vargatron posted:

Ah so you too work in academia.
Not anymore!

orange sky
May 7, 2007

Yeah gently caress it I'll just delete the post

It sounded bad while I was writing it, I'm just pretty excited about it. Sorry guys, won't happen again.

Vargatron
Apr 19, 2008

MRAZZLE DAZZLE


To be honest I wouldn't mind doing ERP consulting, but gently caress they had better be paying me a ton to put up with how dumb C-levels can be about their operations.

22 Eargesplitten
Oct 10, 2010



very stable genius posted:

Look into using DynamoDB.

Also you don’t need an entire ec2 instance just for a simple ETL job. You can do that with lambda or batch.

I need some sort of URL though, Steam requires a URL for you to pull anything from the API. Since I don’t intend to set up a static IP at home using a free hosting instance seems like the best choice.

I was planning on using Mongo because it’s in high demand, should I really use Dynamo?

I’m writing a batch file to run the VBS mentioned in the Stack Overflow link above without us having to type the file names. The import folder is hosted on our cloud provider’s servers, and there’s no way they are going to install an unofficial PS module just for us.

xsf421
Feb 17, 2011

22 Eargesplitten posted:

I need some sort of URL though, Steam requires a URL for you to pull anything from the API. Since I don’t intend to set up a static IP at home using a free hosting instance seems like the best choice.

I was planning on using Mongo because it’s in high demand, should I really use Dynamo?

I’m writing a batch file to run the VBS mentioned in the Stack Overflow link above without us having to type the file names. The import folder is hosted on our cloud provider’s servers, and there’s no way they are going to install an unofficial PS module just for us.

Have you tried something as simple as:

code:
Get-Content excelfile.xls | Export-Csv newfilename.csv
I'm pretty sure we've done that to deal with all the xls files our BI people love to send us.

H110Hawk
Dec 28, 2006

22 Eargesplitten posted:

I was planning on using Mongo because it’s in high demand, should I really use Dynamo?

Do both. The extract part of your job marshals the data into a generic format in memory based on the language you're using. Basically an object per record. Now make your translate portion modular depending on the destination - it consumes the generic object and makes it into a destination specific one. Now load it into Mongo or Dynamo or both.

Want to expand? Duplicate your input data 10,000 times. The goal is to make it larger than is reasonable to block at each step, and consume a huge amount of memory loading the whole file up at once. To accomplish this you convert it to a stream processor, one thread per portion, with a queue in between each one. As your extractor loads objects it pushes them into the queue, then the translator pops them off and translates them, pushing them into the loader queue, and the loader attempts to write them to persistent storage. Congratulations, stream processing is huge.

deedee megadoodoo
Sep 28, 2000
Two roads diverged in a wood, and I, I took the one to Flavortown, and that has made all the difference.


22 Eargesplitten posted:

I need some sort of URL though, Steam requires a URL for you to pull anything from the API. Since I don’t intend to set up a static IP at home using a free hosting instance seems like the best choice.

I’m not familiar with the steam api, but you can use API gateway if you need a callback address. I’m currently doing some things with Slack where I have it pointed at API gateway fronting a step function to handle different events.

Submarine Sandpaper
May 27, 2007


is webdev a worthwhile pursuit or is the work mindbogglingly terrible?

Adbot
ADBOT LOVES YOU

Zil
Jun 4, 2011

Satanically Summoned Citrus


Submarine Sandpaper posted:

is webdev a worthwhile pursuit or is the work mindbogglingly terrible?

Usually depends on the customers in my experience. Also get everything in writing.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply