Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
Literally everyone says "Oh it's form fields but in the cloud? Cool!" because that's what you expect it to be.

Manage your virtual, private, and hybrid cloud infrastructures

Focused on gaining control of your virtualization environment? Need to build and manage a private or hybrid cloud? Red Hat® CloudForms can meet your needs today with a comprehensive management platform to do both.
Future-proof your management investment—evolve without lock-in

As your needs change, CloudForms evolves, protecting your investment and providing a continuum of capabilities as you progress toward Infrastructure-as-a-Service (IaaS) models. Red Hat gives you choice and flexibility along the way so you don’t get locked in to proprietary solutions, while letting you use your existing virtualization and cloud investments from Red Hat, VMware, Microsoft, and Amazon.

Adbot
ADBOT LOVES YOU

MononcQc
May 29, 2007

Suspicious Dish posted:

I thought Heroku was a PaaS.

It is.

Suspicious Dish posted:

But don't you get full VMs on Azure / Heroku? I didn't think they had an app server.

Not exactly. Heroku is based on the idea behind 12 Factor App, which basically has strong opinions about how your dependencies, configuration, logging, etc.

So basically what you get is the equivalent of a very small VM where you're not expected to store stuff to disk (you store stuff in databases, the web front-ends are stateless so they can be scaled out more or less linearly). You're expected to run what is pretty much one process per type of instance (web process, worker process, etc. of a specific type) and get separate isolated containers for each type.

The next step in there is that you want everything to be explicit and declared (dependencies, versions, expected to work, etc.) so the platform does detection of such files or directory structures and auto-configures web containers and whatnot for you. Push in a git repo that has PHP files and .htaccess, and it figures it out. Push out a RoR app, it figures it out. Push something it doesn't understand? Set a configuration (ENV) variable that gives links to scripts on how to detect and build for your type of app (a buildpack).

Stuff will be sent to log routers automatically, and then you have an add-on platform ("oh I want log analytics" or "oh I need a new database for this") that can bind stuff to your apps because they all work on the same conventions and interfaces. At this point there's an entire ecosystem built around it.

IMO it's definitely not for everyone, but when you're a dev who wants to push code and not care about servers (and don't want to hire a sysadmin or devops guy), you can put that stuff on Heroku, pay up, and get it managed that way.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
I think CloudForms is an abstraction over all cloud layers in case you want to run a public site on AWS, and a private site on an internal cloud, and want to manage them the same way.

I'm not sure, because I've asked a lot of people, and they can never give me a straight answer, and when they do I forget it.

more like dICK
Feb 15, 2010

This is inevitable.

Suspicious Dish posted:

Literally everyone says "Oh it's form fields but in the cloud? Cool!" because that's what you expect it to be.

Manage your virtual, private, and hybrid cloud infrastructures

Focused on gaining control of your virtualization environment? Need to build and manage a private or hybrid cloud? Red Hat® CloudForms can meet your needs today with a comprehensive management platform to do both.
Future-proof your management investment—evolve without lock-in

As your needs change, CloudForms evolves, protecting your investment and providing a continuum of capabilities as you progress toward Infrastructure-as-a-Service (IaaS) models. Red Hat gives you choice and flexibility along the way so you don’t get locked in to proprietary solutions, while letting you use your existing virtualization and cloud investments from Red Hat, VMware, Microsoft, and Amazon.

That's pretty awful.

I still don't even know the difference between JBoss AS, JBoss EWS, and Wildfly.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

MononcQc posted:

So basically what you get is the equivalent of a very small VM where you're not expected to store stuff to disk (you store stuff in databases)

I hate this. FS are databases with certain tradeoffs. It's very, very good for certain kinds of data storage (e.g. near-constant lookup, "dumb" data, doesn't need to be flexibly queried), whereas SQL databases have different sets of features with different sets of tradeoffs. They're good for dynamic data and flexible queries like "give me all the users with more than 500 posts".

Why am I allowed to use one type of database, but not the other? What if I make an image hosting service, where I don't need many dynamic queries like that, and I instead want to store a 1-1 mapping of resource ID to image data?

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

more like dICK posted:

That's pretty awful.

I still don't even know the difference between JBoss AS, JBoss EWS, and Wildfly.

Yeah, I gave up trying to explain to the marketing and branding team why nobody knows who the hell we are and what the hell we do. They love being as confusing as hell. I guess some CEOs like this?

Red Hat has a really big enterprise presence, but we're not a hip and up-and-coming startup-friendly company. That's the biggest shift we need to do to our image and our presence, but it means cutting 90% of the bullshit.

more like dICK
Feb 15, 2010

This is inevitable.

Suspicious Dish posted:

Why am I allowed to use one type of database, but not the other? What if I make an image hosting service, where I don't need many dynamic queries like that, and I instead want to store a 1-1 mapping of resource ID to image data?
You pay extra money for a Heroku Add-on like Cloudinary that lets you do that sort of thing, IIRC.

Shaggar
Apr 26, 2006
file system storage is bad if you are using direct access to the file system w/ like hardcoded paths or whatever. you also need to replicate the file system to every host in your cluster. that can be a pain to do right, but its been the way we've been doing it forever.

azure solves this with a component resource specifically for storage. you add a storage project to your solution and set it up and make it available to your other projects. they reference the storage thru that component api, and then the container abstracts it to a file system or whatever other implementation it feels like using. your application doesn't care.

MononcQc
May 29, 2007

FIHGT W HUBBY posted:

It's cool that heroku hosed up their load balancing specifically to accommodate nodejs

RapGenius were still on the old stack that was tailored for Ruby apps afaict. I joined the company after the whole ordeal, so I don't have too many details. In the end the problem ended up more or less being an issue with how little concurrency Rails could handle on the old versions of the framework (like one request at a time per process, 0 concurrency), and issues like noisy neighbours and P99 times eventually making it so all the requests would back up as soon as you had more than 1 load-balancer in front.

The problem there is more or less that for web requests in terms of ms, you don't have the time to share load information across load-balancers for redundancy and whatnot, and the decisions that will be made will always be at risk of being done with inaccurate information (you evaluate instance Y has 3 reqs being served right now, but other load balancers thought the same and now you have 9 reqs going on it).

For this reason, as soon as you have more than 1 load-balancer, unless you implement the Join-Idle-Queue algorithm (which has a token-based dispatch mechanism that requires smart logic on the endpoints and a specifically tailored load-balancer to talk with them), the standard is more or less to either route randomly, or through round-robin. These two later models are probabilistically equivalent (each node gets 1/N requests average).

In these cases, to handle the possible overload, what you then need to do is move the concurrency decision making closer to where the code running takes place: you know load, you know memory, you know disk access, etc.

Basically, it's way easier and way more reliable to have 2-4 concurrent threads or processes on each endpoint than it would be to write a very clever distributed load-balancer that fetches outdated metrics and tries to do remote load analysis (e.g. a request takes on average 50ms it takes 40ms to distribute that information everywhere. You have a window of 10s to make good decisions after which you're making things a lot worse through thundering herds and whatnot). This is why Join-Idle-Queue eventually was invented. They said gently caress it and moved decision-making to the endpoint there.

There were since then releases of larger runtimes (2x, 4x, PX, and other terms that means "more memory, more isolation against noisy neighbors, more CPU allocated") that ended up giving people acceptable response times, because they could more readily handle the eventual slower requests that would before make response times blow up.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
I was disappointed in that whole ordeal because RapGenius was talking about it like they were unleashing a giant scandal, instead of talking privately with the Heroku guys about how their service works, how they're able to scale, and the engineering tradeoffs involved.

Damiya
Jul 3, 2012
Yea I was mostly joking we don't really have enough traffic for it to be an issue.

I kind of wish we were on aws or something more buzzwordy just because picking up some Ops experience would look nice on my resume, but we're super against anything that involves us having to manage our infrastructure due to not wanting to hire an Ops person just yet

MononcQc
May 29, 2007

Suspicious Dish posted:

I hate this. FS are databases with certain tradeoffs. It's very, very good for certain kinds of data storage (e.g. near-constant lookup, "dumb" data, doesn't need to be flexibly queried), whereas SQL databases have different sets of features with different sets of tradeoffs. They're good for dynamic data and flexible queries like "give me all the users with more than 500 posts".

Why am I allowed to use one type of database, but not the other? What if I make an image hosting service, where I don't need many dynamic queries like that, and I instead want to store a 1-1 mapping of resource ID to image data?

more like dICK posted:

You pay extra money for a Heroku Add-on like Cloudinary that lets you do that sort of thing, IIRC.

more like dICK is right on that one. Heroku has a Postgres team that provides Postgres databases. You then have a bunch of add-ons where anyone can decide to offer their own layers for databases of all kinds and whatnot.

Then again if what you need is NFS and distributed access to the file system, I'm guessing Heroku is just not the right platform for the stuff you may need. It's an opinionated platform and when your stuff fits in, it's gonna make your life a lot easier, and when it doesn't, it may be a better time and money investment to look into services or platforms that cater to your specific needs better.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

Shaggar posted:

file system storage is bad if you are using direct access to the file system w/ like hardcoded paths or whatever. you also need to replicate the file system to every host in your cluster. that can be a pain to do right, but its been the way we've been doing it forever.

azure solves this with a component resource specifically for storage. you add a storage project to your solution and set it up and make it available to your other projects. they reference the storage thru that component api, and then the container abstracts it to a file system or whatever other implementation it feels like using. your application doesn't care.

Sure. If there's a flexible, networked database that provides similar benefits and tradeoffs as filesystems, I consider them filesystems. I think the official name for the quasi-filesystems are "object storage systems", and sometimes they're implemented in the VFS layer, e.g. Ceph.

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

MononcQc posted:

more like dICK is right on that one. Heroku has a Postgres team that provides Postgres databases. You then have a bunch of add-ons where anyone can decide to offer their own layers for databases of all kinds and whatnot.

Then again if what you need is NFS and distributed access to the file system, I'm guessing Heroku is just not the right platform for the stuff you may need. It's an opinionated platform and when your stuff fits in, it's gonna make your life a lot easier, and when it doesn't, it may be a better time and money investment to look into services or platforms that cater to your specific needs better.

I don't really need NFS. I think we're a bit too focused on the filesystem as developers sometimes. What I really need for an "image hosting service" in my case is an object store with similar performance benefits and tradeoffs as a traditional filesystem. Resource ID maps to binary blob, and lookup, insertion, and deletion are mostly constant time. That's the best kind of database for my app.

The API and underlying implementation doesn't particularly matter.

MononcQc
May 29, 2007

Suspicious Dish posted:

I don't really need NFS. I think we're a bit too focused on the filesystem as developers sometimes. What I really need for an "image hosting service" in my case is an object store with similar performance benefits and tradeoffs as a traditional filesystem. Resource ID maps to binary blob, and lookup, insertion, and deletion are mostly constant time. That's the best kind of database for my app.

The API and underlying implementation doesn't particularly matter.

That's almost any K/V store that can store binary blobs, unless you need specific searching, indexing, or declared constraints on it.

Shaggar
Apr 26, 2006

Suspicious Dish posted:

I don't really need NFS. I think we're a bit too focused on the filesystem as developers sometimes. What I really need for an "image hosting service" in my case is an object store with similar performance benefits and tradeoffs as a traditional filesystem. Resource ID maps to binary blob, and lookup, insertion, and deletion are mostly constant time. That's the best kind of database for my app.

The API and underlying implementation doesn't particularly matter.

nobody needs nfs

double sulk
Jul 2, 2010

Shaggar posted:

nobody needs nfs

you're right, nobody needs notions from shaggar

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
Nope. I'm an image host, remember. I would be concerned about storing image data contents inside a K/V store for other reasons (namely deletion and insertion not being constant-time, but maybe that's not as important as retrieval).

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe

Shaggar posted:

nobody needs nfs

I was trying to be polite, but yeah, this is correct.

Shaggar
Apr 26, 2006
ive done file storage as blobs in databases before and it works fine as long as you understand the performance impacts on the rest of your system. ex: I just stuck the table hosting the files on separate disk from the main db. worked great.

for clod stuff you have ur s3s and your azure storage and both of those guys have apis available for those systems that you can access from your application. they work like file systems and they are file systems at some level, but you don't really care you just say "hey store this thing w/ this reference" or "hey, delete this thing w/ this reference" and that's what it does. no need to know how its storing or deleting it.

MononcQc
May 29, 2007

One of the web hosts (NearlyFreeSpeech.net) I use for side-projects uses NFS and you know poo poo is going bad when everything that needs to touch the disk apparently takes 10x longer than anything else. Incidentally, their initials are quite literally 'NFS'.

They went down entirely a few week-ends ago because one of the servers went down, and they're apparently working to remove all the needs for it entirely. It didn't look very good from here.

Shaggar
Apr 26, 2006
the real problem w/ file storage wrt ur app is that your data about whats stored becomes out of sync with what is stored. this always happens if you use normal file storage cause some fucker will move the files outside of your application. this is why no one should have direct access to the files. store them relationally in the db itself, or store them in an opaque store thats only accessible to your app where you can be sure they will be safe.

Malcolm XML
Aug 8, 2009

I always knew it would end like this.
let me tell u about azure,,,,,,



actually the blob-queue-table abstraction is pretty dece and covers virtually all of the nosql poo poo youd ever need

Notorious b.s.d.
Jan 25, 2003

by Reene

Suspicious Dish posted:

"platform as a service" is the new name for "shared hosting", right?

yes.

(shaggar was wrong)

Notorious b.s.d.
Jan 25, 2003

by Reene
PaaS: shared hosting with a double dose of vendor lockin

Notorious b.s.d.
Jan 25, 2003

by Reene
hey you know what my enterprise cloud solution was missing? the rollercoaster performance of a plesk account from 2001

Shaggar
Apr 26, 2006

Notorious b.s.d. posted:

yes.

(shaggar was wrong)

shared hosting isn't anything like paas.

Damiya
Jul 3, 2012
yea i kind of hate heroku. ohwell :D

Notorious b.s.d.
Jan 25, 2003

by Reene

Shaggar posted:

shared hosting isn't anything like paas.

paas is literally shared hosting in every example i know of

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
IaaS was called "VPS hosting", and before that, "shell logins"

PaaS was called "shared hosting", and before that, "homepage.rr.com"

Shaggar
Apr 26, 2006

Notorious b.s.d. posted:

paas is literally shared hosting in every example i know of

then you've never seen an actual example of paas

Notorious b.s.d.
Jan 25, 2003

by Reene

Shaggar posted:

then you've never seen an actual example of paas

lol no true scotsman

Suspicious Dish
Sep 24, 2011

2020 is the year of linux on the desktop, bro
Fun Shoe
That's why we're asking for examples. Your example seems to be Azure, which seems to either take VM images and or a webhost where you can upload CGI files.

Shaggar
Apr 26, 2006
shared hosting is the same as iaas/vps except you're limited to only a few specific janitoring abilities of the host.

in paas theres no concept of a host.

Shaggar
Apr 26, 2006
uploading some php to an ftp site is not paas.

MononcQc
May 29, 2007

There's an overlap between Shared Hosting and PaaS in that resources are shared and a subset of whatever, but usually the hosting solution will let you set your poo poo up on your own, and PaaS will do a huge part (if not all) of it for you. PaaS should be able to have you go from 0 to app deployed and running without you ever having to log on to a remote server and run command lines and see things like where files go.

It's like when you call your sysadmin and say "Hey I have this app to deploy can you do it for me" except your replaced a CJ by software (the platform). Some shared hosts come close to it when they have stuff like one click installers to run wordpress and whatnot, but then they also come close to Software as a Service. On the other end you've got Infrastructure as a Service.

It's basically just a big map where one axis is 'how much control do you get on the host' and the other axis is 'how much decision power do you have on the [likely virtualized] hardware resources available to you'.

All of the systems are on that map, but there are different levels of controls and transparency you can get. SaaS is basically "I don't give a poo poo about how any of this stuff works, just give me the website". PaaS is "I just want to put code online and make it work, I'm not a CJ", IaaS is like "I want to control everything but I'm not going in a server room/data center ever", and shared hosts is like "I want to order my fake machines manually and put stuff on them through (S)FTP".

MononcQc fucked around with this message at 17:11 on May 5, 2014

Bloody
Mar 3, 2013

MononcQc posted:

PaaS should be able to have you go from 0 to app deployed and running without you ever having to log on to a remote server and run command lines and see things like where files go.

this is what azure does and i lovelovelove it, can make and deploy a website without ever leaving visual studio or even thinking at all

Sapozhnik
Jan 2, 2005

Nap Ghost
I bet that integrates terrificaly with your source control system (as long as it is Microsoft Visual Studio Team Foundation Server Professional 2013 for Cloud Application Architects Professional Edition (probably not even then)) or with unit testing or with continuous integration or with anything automated whatsoever.

Sapozhnik
Jan 2, 2005

Nap Ghost
"Professional" aka "lmao they actually paid us HOW much for that hill of poo poo?"

Adbot
ADBOT LOVES YOU

Shaggar
Apr 26, 2006

Mr Dog posted:

I bet that integrates terrificaly with your source control system (as long as it is Microsoft Visual Studio Team Foundation Server Professional 2013 for Cloud Application Architects Professional Edition (probably not even then)) or with unit testing or with continuous integration or with anything automated whatsoever.

because it uses the existing vs solution layout it works fine w/ whatever source control plugin ur using.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply