Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
teen phone cutie
Jun 18, 2012

last year i rewrote something awful from scratch because i hate myself
i have a question a couple questions about shared CPU instances. i was reading this Linode guide:

https://www.linode.com/docs/products/compute/compute-instances/plans/choosing-a-plan/#shared-cpu-instances

and they basically explain how shared CPU instances are good for "low to medium" traffic, so my first question is what exactly does "medium" traffic mean?

i had a spike in users yesterday and my site didn't do great during it, so i'm considering moving to a dedicated CPU instance, but i'm not sure if there's other things i can do to mitigate the latency without necessarily spending more on my own box. my other question is, say I was using about ~50% of my CPU during the spike on the shared CPU box, and my response times for pages were about 1-1.2 seconds long, is there any way i could go back to the spike and see if my shared box was being throttled?

essentially what i'm asking is, how exactly do i know when i need to go from shared -> dedicated CPU?

(i had about 8000 users that didn't have adblocker yesterday come to my site throughout the day)

Adbot
ADBOT LOVES YOU

teen phone cutie
Jun 18, 2012

last year i rewrote something awful from scratch because i hate myself
okay i know nobody answered my last question, but i have another one god damnit

last night i just tried out using object storage on my staging server for user avatars and icons that my site uses - all very small files (less than 200kB). when i deployed the code to upload the files to object storage and then read from it, the downloads were SOOOO slow. like 500ms to download a 150kB .jpg in the browser. what's the deal with that? is object storage just slow and i'm trying to use it for a use-case it wasn't designed for?

previously, i was uploading the files through SFTP and reading them from just normal file storage. it seems like it's faster, but i'd like to know if i should keep with this or was there something i was not understanding about object storage.

The Object Storage service I was using was Linode which uses s3 under the hood.

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


You have literally not said anything that we can help you with.

Ask a question, not "How to I make my site faster?"

First off, CDN for your AV's. that's just loving common knowledge by now.


e: if your poo poo is Wordpress just go sign up with loving Wordpress.

wait is it this https://basementcommunity.com?

jaegerx fucked around with this message at 06:03 on Feb 27, 2023

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.

teen phone cutie posted:

okay i know nobody answered my last question, but i have another one god damnit

last night i just tried out using object storage on my staging server for user avatars and icons that my site uses - all very small files (less than 200kB). when i deployed the code to upload the files to object storage and then read from it, the downloads were SOOOO slow. like 500ms to download a 150kB .jpg in the browser. what's the deal with that? is object storage just slow and i'm trying to use it for a use-case it wasn't designed for?

previously, i was uploading the files through SFTP and reading them from just normal file storage. it seems like it's faster, but i'd like to know if i should keep with this or was there something i was not understanding about object storage.

The Object Storage service I was using was Linode which uses s3 under the hood.

Object storage is a heck of lot higher latency than the disk of a server, whether real, virtual, on a SAN or whatever. Typically, when you're serving something from object storage to users browsers, you'll put a CDN in front of the object storage to cache things closer to users. Object storage latency is generally a bit better when your servers are hitting the object storage, because you're already in the providers network and there's probably even a connection open already. The expectation is still that you will cache things you need with low latency.

Linode doesn't use S3 under the hood, Linode built their own object storage which is S3 compatible, meaning that it implements some of the server API that S3 uses. The S3 API has become something of a standard that lots of solutions implement: https://min.io/

Re: Your performance question:

I would get more shared cores before I paid for dedicated cores instead. You will need to profile your site to figure out why it is slow, is this thing PHP? Is the database on the same box? What's your disk I/O situation looking like?

With Linodes pricing the way it is, I would look at the dedicated CPU cores as something you do at the point where you're unwilling to move up to the next tier of shared CPU pricing.

teen phone cutie
Jun 18, 2012

last year i rewrote something awful from scratch because i hate myself

jaegerx posted:

You have literally not said anything that we can help you with.

Ask a question, not "How to I make my site faster?"

First off, CDN for your AV's. that's just loving common knowledge by now.


e: if your poo poo is Wordpress just go sign up with loving Wordpress.

wait is it this https://basementcommunity.com?

lol shut the gently caress up loser

there were 3 different questions in there, you dumb bitch

teen phone cutie posted:

what exactly does "medium" traffic mean?

is there any way i could go back to the spike and see if my shared box was being throttled?

essentially what i'm asking is, how exactly do i know when i need to go from shared -> dedicated CPU?

teen phone cutie fucked around with this message at 07:55 on Feb 27, 2023

teen phone cutie
Jun 18, 2012

last year i rewrote something awful from scratch because i hate myself

Twerk from Home posted:

Object storage is a heck of lot higher latency than the disk of a server, whether real, virtual, on a SAN or whatever. Typically, when you're serving something from object storage to users browsers, you'll put a CDN in front of the object storage to cache things closer to users. Object storage latency is generally a bit better when your servers are hitting the object storage, because you're already in the providers network and there's probably even a connection open already. The expectation is still that you will cache things you need with low latency.

Linode doesn't use S3 under the hood, Linode built their own object storage which is S3 compatible, meaning that it implements some of the server API that S3 uses. The S3 API has become something of a standard that lots of solutions implement: https://min.io/

Re: Your performance question:

I would get more shared cores before I paid for dedicated cores instead. You will need to profile your site to figure out why it is slow, is this thing PHP? Is the database on the same box? What's your disk I/O situation looking like?

With Linodes pricing the way it is, I would look at the dedicated CPU cores as something you do at the point where you're unwilling to move up to the next tier of shared CPU pricing.

good stuff thanks for this! i'm not exactly sure if i have a CDN in my budget but it's good to know all this. i recently added performance monitoring actually so i'm starting to get a better picture of where the the problems lie. the backend API is on the same box as the DB and the frotend is on a different box in the same datacenter

astral
Apr 26, 2004

Cloudflare has a comprehensive free tier.

teen phone cutie
Jun 18, 2012

last year i rewrote something awful from scratch because i hate myself

astral posted:

Cloudflare has a comprehensive free tier.

oooo yeah i actually do have cloudflare set up. for some reason i guess i thought image CDN caching was a separate service, so i guess i'm good on that regard.

but yeah, either way the benchmark testing i was doing on object storage vs. disk storage was loading up my page with caching turned off, so i'll guess i'll just stick with disk storage if it's objectively the better option for my use case

giogadi
Oct 27, 2009

yergacheffe posted:

Not traditional webhosting, but I've gotten away with tiny projects hosted on a free tier compute engine on google cloud platform. If you can make do what amounts to a VM with

-half a cpu core
-1gb RAM
-10gb hard drive
-1gb of outgoing traffic per month

I've tried doing this in the past, but I found that sometimes my compute server would just go down for seemingly no reason; is this a thing that "just happens", or did I maybe have something setup incorrectly?

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


giogadi posted:

I've tried doing this in the past, but I found that sometimes my compute server would just go down for seemingly no reason; is this a thing that "just happens", or did I maybe have something setup incorrectly?

Just search for Digitalocean codes. You can usually get a free $100 to use. Then just use a different email.

M_Gargantua
Oct 16, 2006

STOMP'N ON INTO THE POWERLINES

Exciting Lemon
Whats the Cost/Effort balance like for having an email server on a shared host vs. just using google workspace? I've been trying to get away from cloud based solutions and the whole google ecosystem for a while now.

Twerk from Home
Jan 17, 2009

This avatar brought to you by the 'save our dead gay forums' foundation.

M_Gargantua posted:

Whats the Cost/Effort balance like for having an email server on a shared host vs. just using google workspace? I've been trying to get away from cloud based solutions and the whole google ecosystem for a while now.

Hosting your own email is consistently a giant pain in the rear end, and you're likely to have to troubleshoot deliverability problems even if you do everything right.

Your choices for hosted solutions range from the cloud giants to struggling companies with dim futures who I would not tie my boat to. If you specifically want to stay away from Google only, you can get email from Microsoft or even Amazon if you really want. The other big hosts that do email are companies like Rackspace, who I recommend you stay far, far away from.

M_Gargantua
Oct 16, 2006

STOMP'N ON INTO THE POWERLINES

Exciting Lemon
That kinda aligns with what I read - Even if I already have a domain and a host, it's not a reliable solution to just have some sort of email server running in a VM.

CarForumPoster
Jun 26, 2013

⚡POWER⚡

M_Gargantua posted:

Whats the Cost/Effort balance like for having an email server on a shared host vs. just using google workspace? I've been trying to get away from cloud based solutions and the whole google ecosystem for a while now.

You emails won't reach the inbox way too often to be worth it. Go with MS/Google/Whoever and make sure to set up SPF/DKIM/DMARC.

PleasantDilemma
Dec 5, 2006

The Last Hope for Peace
I've heard fastmail is good for hosting email and they aren't one of the big clouds. Today I was actually trying to sign up for icloud email but the app kept giving me an error. They support custom domain and the price looks really good. I'm looking to switch off runbox.com since I've had some delivery issues.

fuf
Sep 12, 2004

haha
I'm gonna disagree with the normal consensus slightly and say that hosting your email on a good old standard shared hosting package, as long as it's from a good hosting provider, will probably be fine.

cPanel is actually pretty good these days at setting up SPF, DKIM, and PTR records for you. And good hosts even route their email through a service like SendGrid or Mailgun to improve deliverability (which mitigates the main risk of doing email on shared hosting, which is that your IP might get blacklisted through no fault of your own).

Definitely wouldn't recommend running your own email server from scratch though, unless you really want to get into the nuts and bolts of deliverability.

I have been using Fastmail for about 8 years, and the service is great and always reliable, but the price can rocket pretty quickly. They have pretty cheap 750mb and 2gb mailboxes, but as soon as someone fills that (which they will pretty quickly if they're actually using it) you have to bump them up to the 30gb package, which is $72/year.

Fastmail also has an annoying system where your main account password is not the same as the password you use to access the mailbox from a phone / device, which can be a pain to explain to users/clients.

A couple of years ago I started creating mailboxes in cPanel instead, and I've been surprised how well it's worked out. Barely any deliverability issues even from mailboxes that are getting used all day every day. The main advantage for me is that I'm paying for the whole package, rather than per mailbox, so it works out a lot cheaper.

eightysixed
Sep 23, 2004

I always tell the truth. Even when I lie.

fuf posted:

cPanel is actually pretty good these days
This is an amazingly fun post :v:

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


eightysixed posted:

This is an amazingly fun post :v:

Does it still compile everything from source?

eightysixed
Sep 23, 2004

I always tell the truth. Even when I lie.

jaegerx posted:

Does it still compile everything from source?

Yes.
https://docs.cpanel.net/installation-guide/install/

My favorite part is “We do not provide an uninstaller. If you wish to remove our software, you must reformat the server.”

nielsm
Jun 1, 2009



Without having read it, all I can think is why not just make a straight up OS distro instead of consuming an existing one??

fuf
Sep 12, 2004

haha

eightysixed posted:

This is an amazingly fun post :v:

Aren't we too old for this kind of thing? I'm only trying to be helpful. :(

A reliable hosting provider using cPanel seems like the obvious middle ground between running your own email server and putting everything on MS/Google/Fastmail, so I was just pointing out how it has been working out for me.

jaegerx
Sep 10, 2012

Maybe this post will get me on your ignore list!


eightysixed posted:

Yes.
https://docs.cpanel.net/installation-guide/install/

My favorite part is “We do not provide an uninstaller. If you wish to remove our software, you must reformat the server.”

Yeah that's about right from them.

Aware
Nov 18, 2003
I just use CloudFlare free tier to setup email@domain.com addresses and forward them to a Gmail account and setup the send-as feature. Works great but I only need like 3 emails per domain max so YMMV. I used to just do this with dreamhost when my domains were there too.

Macatt
May 3, 2005
I just found out about Google Domains being sold off to Squarespace. I'm not well versed in this stuff. My needs are wildcard email forwarding (using just the domain name, no hosting package) and private registration.

I'm a little overwhelmed by the recommendations. Namecheap, NameSilo, NearlyFreeSpeech, Cloudflare, Fabulous, Dynadot, WordPress, Porkbun. It seems like they're all fairly comparable in most respects. Should I just figure out which ones offer catch-all forwarding, then select based on prices?

Macatt fucked around with this message at 08:27 on Aug 17, 2023

Rufus Ping
Dec 27, 2006





I'm a Friend of Rodney Nano
Cloudflare are a domain registrar, DNS provider, and also offer free email forwarding (including wildcard) who sell domains at-cost and likely won't disappear any time soon

If they support the TLDs you want, they're the easiest choice

Macatt
May 3, 2005
Thanks, Cloudflare does look to be the easiest and cheapest route for my email needs. However, I neglected to mention that I also have a separate domain name pointing to an actual site. And I guess Cloudflare doesn't allow changing nameservers without upgrading to a business plan.

So what I'm thinking is maybe using Cloudflare for my email-forwarding domain, and someone else for my site's domain. This might also possibly address an issue I've been wondering about -- which email provider should you use for actually registering a domain (both your account with the registrar and the whois contact email)?

I imagine it's unwise to use email@yourdomain.tld for registering yourdomain.tld ... but I'm also seeing people say not to use free email accounts either. Would using two registrars and cross-listing an email address from each be good practice?

Macatt fucked around with this message at 03:12 on Aug 18, 2023

Rufus Ping
Dec 27, 2006





I'm a Friend of Rodney Nano

Macatt posted:

However, I neglected to mention that I also have a separate domain name pointing to an actual site. And I guess Cloudflare doesn't allow changing nameservers without upgrading to a business plan.

Cloudflare's business plan still doesn't offer what you want actually - you can't point a cloudflare-registered domain at a non-cloudflare nameserver.

(The feature you've seen is actually for vanity nameservers - where yourdomain.com uses ns1.yourdomain.com as an alias for something.ns.cloudflare.com, but cloudflare still host the DNS behind the scenes.)

Fortunately you are not obliged to use your web hosting provider's DNS. You can move both the domain and DNS to cloudflare, and then use an A Record to point the DNS to your hosting provider's IP.

Macatt posted:

So what I'm thinking is maybe using Cloudflare for my email-forwarding domain, and someone else for my site's domain.
personally I'd move both domains to cloudflare, and also use them for DNS and email forwarding

Macatt posted:

This might also possibly address an issue I've been wondering about -- which email provider should you use for actually registering a domain (both your account with the registrar and the whois contact email)?

I imagine it's unwise to use email@yourdomain.tld for registering yourdomain.tld ...
I agree this is unwise because if there's a problem with the domain, you won't be able to receive email about it and run the risk of losing access completely

Macatt posted:

but I'm also seeing people say not to use free email accounts either.
I don't agree with this and personally use my gmail. It may be free, but it's good enough for every other aspect of my personal and professional life

Macatt posted:

Would using two registrars and cross-listing an email address from each be good practice?
I think this is unnecessarily convoluted

Macatt
May 3, 2005
Awesome, thank you so much for that detailed breakdown. I feel good about this now. :cheers:

Pablo Bluth
Sep 7, 2007

I've made a huge mistake.

Twerk from Home posted:

Hosting your own email is consistently a giant pain in the rear end, and you're likely to have to troubleshoot deliverability problems even if you do everything right.

Your choices for hosted solutions range from the cloud giants to struggling companies with dim futures who I would not tie my boat to. If you specifically want to stay away from Google only, you can get email from Microsoft or even Amazon if you really want. The other big hosts that do email are companies like Rackspace, who I recommend you stay far, far away from.
I gather these days it's not even just a pain in the usual home-brewing sense, but that it's becoming increasingly impossible for anyone but the small number of mega-providers to deliver email reliably. Google, Microsoft et al will just randomly break the ability for organisations to send email and it' impossible to talk to anyone.

nem
Jan 4, 2003

panel.dev
apnscp: cPanel evolved
Not only IP reputation that makes it impossible to relay mail unless you own a /24 network but also insurance. Many business insurance policies require MFA, which IMAP/SMTP was never built with consideration in mind. You're stuck with web-based solutions like O365 and Google Workspace.

istewart
Apr 13, 2005

Still contemplating why I didn't register here under a clever pseudonym

So Cloudflare is definitely my best bet for a long-term registrar? I have a .net that I just reupped with GoDaddy for their max term (7 years) and a domain that HostGator gave me for 1 year free, that I've been renewing year-by-year since then. I've been thinking about getting a VPS instead of the shared hosting I signed up for and never used. I never got around to setting up WordPress, and at this point would rather run a simple static-site generator without fancy templates, comments, etc. I'm also considering toying around with a single-user Mastodon instance.

So in that case, transfer domain to Cloudflare -> point at VPS provider's IP?

Rufus Ping
Dec 27, 2006





I'm a Friend of Rodney Nano
Yes imho

Macatt
May 3, 2005
I've only just started tinkering with my new Cloudflare setup, but I'm really liking everything about it so far. Big thanks to Rufus for the recommendation. At this point I wouldn't go back to Google Domains even if I could.

To be fair, much of what they offer is actually available just by using their free DNS services, regardless of if they're your registrar. Even the email forwarding function I was so concerned about, I had that working before transferring my domain (which minimized my downtime to just a few minutes).

Since you just renewed at GoDaddy, though, it's my understanding that when you pay to transfer your domain, you'll forfeit your additional year of renewal (because it was already renewed in the last 45 days). So you'll have to contact GoDaddy and request a refund for one of those seven years of renewal.

Boywhiz88
Sep 11, 2005

floating 26" off da ground. BURR!
I'm helping a local non-profit with their mass e-mails.

They have their host, then DNS through another service, setup for Google Workspace, and use SendGrid for the big e-mails.

The e-mails go straight to Junk because it fails DKIM/SPF.

I included the DKIM records provided by SendGrid, and added SendGrid to the SPF record, but it still fails.

I suspect it's because of how things are routing w/ regards to Google. I'm working on getting the DNS Records so I can better show what a mess they are.

They have like 10 different "staging" records for Google.

Anyone have thoughts on where I should start? The big issue seems like SendGrid is not being recognized as being authorized through the domain, that the SPF and DKIM records aren't actually being pulled from the domain host.

unknown
Nov 16, 2002
Ain't got no stinking title yet!


Setup some kind of dmarc analysis - there's lots of ones out there and most have a free-tier (especially for non-profits) and it'll say what's failing for who (in a generic setting at least, not specific email addresses).

But to figure out what's wrong, you'll need the full headers from a failed message.

Boywhiz88
Sep 11, 2005

floating 26" off da ground. BURR!

unknown posted:

Setup some kind of dmarc analysis - there's lots of ones out there and most have a free-tier (especially for non-profits) and it'll say what's failing for who (in a generic setting at least, not specific email addresses).

But to figure out what's wrong, you'll need the full headers from a failed message.

Authentication-Results: spf=none (sender IP is 167.89.40.80) smtp.mailfrom=em6847.DOMAIN.org; dkim=fail (no key for signature) header.d=DOMAIN.org;dmarc=fail action=none header.from=DOMAIN.org;compauth=fail reason=001

I did throw on a basic DMARC policy for the domain, Allow, with a 100 pct. But no reporting e-mail because I didn't want anyone getting overwhelmed by a bunch of failures at some point.

Rawrbomb
Mar 11, 2011

rawrrrrr

Boywhiz88 posted:

Authentication-Results: spf=none (sender IP is 167.89.40.80) smtp.mailfrom=em6847.DOMAIN.org; dkim=fail (no key for signature) header.d=DOMAIN.org;dmarc=fail action=none header.from=DOMAIN.org;compauth=fail reason=001

I did throw on a basic DMARC policy for the domain, Allow, with a 100 pct. But no reporting e-mail because I didn't want anyone getting overwhelmed by a bunch of failures at some point.

If that is the result, you have no SPF allow from configured, and you have no DKIM keys configured on the DNS side of things.

You need to add your DKIM records to your DNS from SendGrid, and then make sure your SPF allows that sending IP (or block of IPs). It almost sounds like you haven't gone through the Domain Auth process at SendGrid yet: https://docs.sendgrid.com/ui/account-and-settings/how-to-set-up-domain-authentication

unknown
Nov 16, 2002
Ain't got no stinking title yet!


If you've actually configured all those details in your DNS, and google isn't finding them - check to make sure you've made the changes on the actual DNS provider. See many times where someone (usually a web developer) moved the DNS servers to a 3rd party hosting. If that's the case and you're not changing the 3rd party provider, then anything you do won't get picked up "live" and all your work is for nothing.

Boywhiz88
Sep 11, 2005

floating 26" off da ground. BURR!
Sorry for the delay. I got COVID last week and it kicked my rear end.

Thankfully prior to that, I asked for teh DNS records.

As I came out of my fugue state, I realized that the DKIM and NAME entries managed to create redundant parts. So they show up like em6847.DOMAIN.org.DOMAIN.org.

I assume i just have to edit and remove the second DOMAIN.org parts on those and try again. Does that seem like the best next step, and then re-assess if it continues?

Adbot
ADBOT LOVES YOU

Rawrbomb
Mar 11, 2011

rawrrrrr

Boywhiz88 posted:

Sorry for the delay. I got COVID last week and it kicked my rear end.

Thankfully prior to that, I asked for teh DNS records.

As I came out of my fugue state, I realized that the DKIM and NAME entries managed to create redundant parts. So they show up like em6847.DOMAIN.org.DOMAIN.org.

I assume i just have to edit and remove the second DOMAIN.org parts on those and try again. Does that seem like the best next step, and then re-assess if it continues?

Yeah, those entries should be em6847.domain.org without the secondary hostname. The sender auth process should go into a "validated" state, once you get it setup.

You can use something like mxtoolbox or digwebinterface to query the DNS directly and verify your records match what they gave you as well.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply