Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
stevewm
May 10, 2005
Yesterday they took their DNS provider out.. So they posted all the IPs for their POPs in a Twitter post so you could get your phones back up. But I guess earlier today the DDoS was pointed directly at them.

It appears they are moving some things over to CloudFlare.. Their DNS is being provided by CloudFlare now. But of course all of their IP addresses are unreachable now.

Adbot
ADBOT LOVES YOU

kensei
Dec 27, 2007

He has come home, where he belongs. The Ancient Mariner returns to lead his first team to glory, forever and ever. Amen!


Thanks Ants posted:

[Biden voice]
C'mon, man!





Maybe use the dates set on the auto-responder or the week-long calendar entry to determine that someone isn't at work, and stop nudging for a response?

I get this from our outsourced help desk and it pisses me off

dragonshardz
May 2, 2017


stevewm posted:

they posted all the IPs for their POPs in a Twitter post

hmm, yes, i too post my IP address in public while being DDoSed.

:owned:

Impotence
Nov 8, 2010
Lipstick Apathy

dragonshardz posted:

hmm, yes, i too post my IP address in public while being DDoSed.

:owned:

i don't think it would have made a difference here because they were just the same thing as resolving voip.ms hostnames anyway

KozmoNaut
Apr 23, 2008

Happiness is a warm
Turbo Plasma Rifle


The Claptain posted:

There's also ctrl + win + shift + b, which resets the graphics driver

:aaaaa:

stevewm
May 10, 2005
There are rumours flying that a group has claimed responsibility for the VoIP.MS DDoS and is asking for 100 BTC ransom.

And they are still mostly down. Ugh..

D34THROW
Jan 29, 2012

RETAIL RETAIL LISTEN TO ME BITCH ABOUT RETAIL
:rant:
Reminds me of a couple of years ago, when our company network got absolutely hosed by Ryuk, from a phishing email or someone on a porn site or some drat thing. That insidious little bastard ruined the network, and we lost about 9 months of work because they backup so infrequently.

Altogether, from the time the network went down to the time functionality was restored - including a team of temps re-entering jobs and jobs and jobs into the ERP - took seven or eight months. During that time, we went back to the early '90s with paper, faxes, and phone calls instead of emails.

stevewm
May 10, 2005
It appears they acknowledged it just minutes ago in a Facebook post. They mentioned it is a "ransom DDoS".

Thanks Ants
May 21, 2004

#essereFerrari


stevewm posted:

There are rumours flying that a group has claimed responsibility for the VoIP.MS DDoS and is asking for 100 BTC ransom.

And they are still mostly down. Ugh..

Easy for me to say as I have no skin in the game but people can't start paying ransoms to make DDoS go away - another group will just line up to start a new attack once the payment is made

stevewm
May 10, 2005
Looks like they are putting everything behind CloudFlare. At least their main website is now.

The IP addresses on most of their POPs have changed. And several POPs are now stable again. At least I found one that is consistent and we have calls working again. (Chicago2 and Chicago4 if anyone here is using VoIP.MS).

Edit: And they are ALL dead now, including their website. loving fantastic.

stevewm fucked around with this message at 17:41 on Sep 20, 2021

klosterdev
Oct 10, 2006

Na na na na na na na na Batman!
Pissing me off: Having to cover my rear end/babysit a user by sending an email every day reminding them to please read the email I sent ten days ago and reply back with the information I need to build $resource

Kyrosiris
May 24, 2006

You try to be happy when everyone is summoning you everywhere to "be their friend".



stevewm posted:

Edit: And they are ALL dead now, including their website. loving fantastic.

Can't help but wonder if their new IP space got leaked or something.

stevewm
May 10, 2005

Kyrosiris posted:

Can't help but wonder if their new IP space got leaked or something.

Wouldn't matter. The pop hostnames are all public.

It is definitely whack-a-mole right now. A group of POPs will work for a while and then degrade to the point where they stop registering.

I'm looking at other providers to port out to. This situation doesn't appear to be improving at all.

SyNack Sassimov
May 4, 2006

Let the robot win.
            --Captain James T. Vader


stevewm posted:

Wouldn't matter. The pop hostnames are all public.

It is definitely whack-a-mole right now. A group of POPs will work for a while and then degrade to the point where they stop registering.

I'm looking at other providers to port out to. This situation doesn't appear to be improving at all.

Going through the same thing here. At least their website is mostly up now (behind heavy Cloudflare protection), so once we figure out a new provider we should be able to forward numbers temporarily until the port completes. What a clusterfuck.

If anyone has any thoughts or recommendations for reliable SIP trunk providers with strong DDOS protection, that aren't Flowroute because my boss hates them (which annoys me because in my experience they've been very solid), I'm all ears. stevewm I'd also be interested to hear who you end up going with.

stevewm
May 10, 2005

SyNack Sassimov posted:

Going through the same thing here. At least their website is mostly up now (behind heavy Cloudflare protection), so once we figure out a new provider we should be able to forward numbers temporarily until the port completes. What a clusterfuck.

If anyone has any thoughts or recommendations for reliable SIP trunk providers with strong DDOS protection, that aren't Flowroute because my boss hates them (which annoys me because in my experience they've been very solid), I'm all ears. stevewm I'd also be interested to hear who you end up going with.

I am looking at Skyetel. They where one of the ones that came up as being often recommended when I was researching SIP trunk providers about a year ago. Waiting for them to confirm my DID can be ported. The only reason I ended up with VoIP.MS in this case was because they where the only one that had local DIDs for that particular store location. The CEO/owner always wants our store locations to have a local phone number.

Silly Newbie
Jul 25, 2007
How do I?
I've been with 8x8 for a couple years and found them easy to work with, if we're talking about similar services.

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.
Certificates. Again. Still.

Goddamn if all I want to do is be able to access a couple of network appliances and my vSphere server without them throwing ssl errors in a browser.

I tried, once again, to set up a certificate authority in my AD and this time I couldn’t even manage to get the /certsrv site to work over https.

Is it possible to actually get dumber at something because I think I’m getting dumber at this.

Impotence
Nov 8, 2010
Lipstick Apathy
my solution to everything is always to give everything a publicly trusted certificate chaining to a proper audited ca, the hostnames don't even need to resolve publicly if you DV the domain in other ways like a txt dns record

the only time i'd want to touch a private ca is for like, mtls or hating my employees and wanting to dig around in their https traffic or something

Agrikk
Oct 17, 2003

Take care with that! We have not fully ascertained its function, and the ticking is accelerating.

Biowarfare posted:

my solution to everything is always to give everything a blah de blah bla blah blah de bla. the hostnames don't even need to resolve publicly if you bleh bla blargh bla blargle glaggla dns record

Yes, this exactly.

Impotence
Nov 8, 2010
Lipstick Apathy
that great, slouching beast: self signed certificates that expire in 2060

CitizenKain
May 27, 2001

That was Gary Cooper, asshole.

Nap Ghost

CitizenKain posted:

There is a everyone in IT meeting planned for Monday. As far as I can remember working here, this is the first time that has happened.
I expect it is the results of the consultants brought in to figure why projects are slipping and we are having difficulties hiring people. So instead of hiring people and raising wages, its time to shuffle departments around. Sure that doesn't actually solve problems, but if you move them around, it looks like you are doing something.

So, big meeting happened, here are the results.
1. Still no WFH policy, do whatever we guess.
2. No covid policy, continue to do whatever.
3. Project management reports to someone else now, this is exciting for them for some reason. Even they don't know what this means.
4. We are going to look into a department that works on projects and isn't hampered by just KLO activity. No further information.
5. We will make these fancy new positions to lure in people from an acquisition that weren't planning on staying. No you can't apply for these positions, or know what they are.

That was it. Kinda what I expected I guess, but no surprise how any details were glossed over, no big issues were even touched on.

Happy Litterbox
Jan 2, 2010
How do you guys deal with customers who call you directly three to four times per day just to ask what the current status of a ticket is and when it will be solved? I am short of just throwing my phone out of the window. And before you ask, no I am not the helpdesk. Asked my boss already and he is said I need to be available for the customers at all times.

Weedle
May 31, 2006




“Your ticket is in our queue. We do not have an ETA at this time.” repeat until they stop bothering you

xzzy
Mar 5, 2009

When I was a young buck working at a tier 1 isp we'd have customers that demand we stay on the phone with them the entire time we were fixing their T1 lines. It was always the ones that could barely afford it and were trying to stay afloat by oversubscribing their bandwidth. The instant they saw a dropped packet on a ping test they'd be on the horn demanding we make it perfect.

The "we don't have an ETA" line never worked because I was like 20 years old at the time I didn't have the confidence to take a firm stance. They'd reply with "okay I'll wait on the phone" and I'd suck it up.

Polio Vax Scene
Apr 5, 2009



Biowarfare posted:

that great, slouching beast: self signed certificates that expire in 2060

Clever move, making them last until retirement. :hmmyes:

Thanks Ants
May 21, 2004

#essereFerrari


I just don't answer my phone

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

On the phone with the company that does our server warranties etc

Boss keeps asking if they buy used equipment. Do you buy these do you buy this do you buy that

NO
ITS ALL OLD poo poo
YOU'RE LUCKY IF THEY WILL PICK IT UP AND DISPOSE FOR FREE
gently caress

stevewm
May 10, 2005
Either they are giving up on the DDoS, or VoIP.MS has it somewhat under control.

Our trunks have stayed up since yesterday afternoon and calls have been working fine using NY#4. Despite the portal showing "No Registration" our PBXs have remained registered and working.

SyNack Sassimov
May 4, 2006

Let the robot win.
            --Captain James T. Vader


stevewm posted:

Either they are giving up on the DDoS, or VoIP.MS has it somewhat under control.

Our trunks have stayed up since yesterday afternoon and calls have been working fine using NY#4. Despite the portal showing "No Registration" our PBXs have remained registered and working.

Yeah things have been OK for us as well.

We're still gonna switch as soon as loving possible because jesus christ that was not acceptable.

stevewm
May 10, 2005

SyNack Sassimov posted:

Yeah things have been OK for us as well.

We're still gonna switch as soon as loving possible because jesus christ that was not acceptable.

I spoke too soon. Even their website is dead again.

Wibla
Feb 16, 2011

They're providing a service over the internet.

This is a downside of that.

If you can't bear downtime, call your local telco or something I guess :haw:

Fart Amplifier
Apr 12, 2003

We have extensive issues with our team providing vendors with way more goddamned access than they should have during product setup. I just discovered that our MSSQL server has a local login that's been granted SA privileges and executes queries from the frontend web server.

When we contacted the software vendor to say "please let us know what permissions the front end web server actually needs to the DB" they responded

quote:

Hi _____ – in general that is the account level desired. The account used not only runs sql scripts/stored proceedures for normal operation, but also allows us to modify settings on the _______ databases, run commands like sp_updatestats, dbcc freeproccache, etc.

I don't know how we haven't been catastrophically affected by ransomware yet.

wolrah
May 8, 2006
what?

Biowarfare posted:

my solution to everything is always to give everything a publicly trusted certificate chaining to a proper audited ca, the hostnames don't even need to resolve publicly if you DV the domain in other ways like a txt dns record

the only time i'd want to touch a private ca is for like, mtls or hating my employees and wanting to dig around in their https traffic or something
A significant downside of this is that those internal names now become part of public CT logs forever, which may reveal internal details you may prefer to keep private depending on how you do your DNS. It's not the worst thing and you can minimize the amount of information revealed if you want to with careful naming, but just something to keep in mind.

Wibla posted:

They're providing a service over the internet.

This is a downside of that.

If you can't bear downtime, call your local telco or something I guess :haw:
Yeah, as someone whose been working in VoIP I take solace in the fact that my company is too small of a target for that level of attacks, but I have been collateral damage from DDoSes aimed at our hosts or other tenants of our hosts.

DDoS protection on SIP is a hard problem, and the attackers these days can generate absolute fucktons of traffic that basically no one other than the biggest players on the internet really have the capacity to absorb.

Unfortunately the way the PSTN works and the fact that ENUM never caught on combine to prevent any kind of real carrier redundancy on local DIDs. Toll free numbers have more flexibility, but local numbers exist in one place and that one place will always be a potential point of failure. As with your internet connection, if your phones are really important to you get and advertise numbers from multiple providers.

Thanks Ants
May 21, 2004

#essereFerrari


If your phones are really important then you can get your SIP over a private fibre or another VLAN on your existing ethernet service, and just hope that whoever your provider is keeps their public and private endpoints separate from each other.

Impotence
Nov 8, 2010
Lipstick Apathy

wolrah posted:

A significant downside of this is that those internal names now become part of public CT logs forever, which may reveal internal details you may prefer to keep private depending on how you do your DNS. It's not the worst thing and you can minimize the amount of information revealed if you want to with careful naming, but just something to keep in mind.


hey, wildcard certificates are free with LE

there's also no real guarantee that your private certificates or CAs are totally private either given that half of your employees probably have chrome extensions that have hidden adware/malware in them, referrer headers to public sites that get logged in analytics, etc - if you care a lot about keeping hostnames private there's a lot of other work to do. isps and google and the like probably have failed resolution attempts from DoH/DoT after they've been idle kicked off vpn


Impotence fucked around with this message at 05:53 on Sep 22, 2021

tactlessbastard
Feb 4, 2001

Godspeed, post
Fun Shoe
BOW HOWDY I SURE DO LOVE COMING IN AN HOUR EARLY FOR A MEETING THAT GOT CANCELLED

Wibla
Feb 16, 2011

:yotj:

angry armadillo
Jul 26, 2010

tactlessbastard posted:

BOW HOWDY I SURE DO LOVE COMING IN AN HOUR EARLY FOR A MEETING THAT GOT CANCELLED

eh, going home an hour early is kinda fun too

Bob Morales
Aug 18, 2006


Just wear the fucking mask, Bob

I don't care how many people I probably infected with COVID-19 while refusing to wear a mask, my comfort is far more important than the health and safety of everyone around me!

tactlessbastard posted:

BOW HOWDY I SURE DO LOVE COMING IN AN HOUR EARLY FOR A MEETING THAT GOT CANCELLED

Did you get an email notification that the meeting was canceled, or did you sit on a zoom call with 4 other people and the host never showed up?

Adbot
ADBOT LOVES YOU

Kyrosiris
May 24, 2006

You try to be happy when everyone is summoning you everywhere to "be their friend".



Bob Morales posted:

Did you get an email notification that the meeting was canceled, or did you sit on a zoom call with 4 other people and the host never showed up?

I had a customer piss and moan about how we didn't show up to a meeting when I was in a Teams link they provided for nearly 20 minutes.

Turns out they canceled and used a different Teams meeting but neglected to email me or anyone on my team. When they did forward a copy of the message they got super mad when I pointed out that no one from my team was in the To or CC and how were we expected to know it had changed without being notified?

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply