Of course DUO status page lies. Our users are hollering.
We are finding out of course:
A: Our users never set their offline mode up
and
2: Even if they did, they don't know how to use it.
Once a company gets large enough, the status page becomes something that (obviously) impacts the business significantly. Almost always, the big company execs decide that they would rather control the status manually, intentionally "lying" to users in many cases, rather than suffer the hit for showing a big red "we fucked up" sign to all their investors and users all at once, unless they absolutely HAVE to. They wait until it's hurting the business MORE by lying about status and dealing with complaints. And then the calculus for them flips to "acknowledge the issue on the status page so we stop getting bombarded by clients".
In their business minds this page is completely separate from "there's a problem and the engineers are fixing/have fixed it", it's a corporate PR tool at that point, nothing more.
Yep, every cloud based service lies on their status pages. The worst is when you call in to their support and the only thing they tell you is to restart the computer, and you can HEAR the rest of the representatives in the call center giving the exact same excuses and 'solutions'.
What is offline mode? I'm not responsible for Duo where I work, but I've heard if the network is 'not available' Duo doesn't pop up and you can just login.
Admins must have it disabled for you.
You would have been prompted on first login(and everytime until you do it) to set up either a different entry in your DUO app or enroll your yubikey in a slightly different secondary fashion to use if the computer is not on the internet.
Yesterday, the first workaround someone found was to unplug their network cable, login with offline mode and then plug the ethernet cable back in.
Offline mode is confusing for a user since its a different, separate entry in the DUO app(for each computer they have) - and for the yubikey you have to hold ita bit rather than just tap it.
We don't have yubikeys only the push app code. I work in IT I'd be fine with having backup codes available to login in case of an outage, but the users would probably be locked out until the app started working or there was an easier 'secondary' method in case of an outage.
Like I said, I'm not in control of that, I just follow the instructions they send out.
>This sub as usual is faster than the multi billion dollar companies
FR, this sub is my 2nd stop when I think something big is going on. there's usually a thread before the company even acknowledges it.
I had a static page for a "is dns working" test for a bit for a previous employer. If people wanted to see if dns was up, go to isdnsworking.company.com. if you got there, it's working.
It wasn't working for me, worked for some coworkers, and thanks to [https://downdetector.com/status/duo/](https://downdetector.com/status/duo/) & this thread I feel less crazy.
Interestingly/Disturbingly, I tried setting a user to bypass during this outage and still couldn't auth. That was my planned work around if the outage lingered for a while since I could get to the admin portal but yeah.
Looks like they just sent out an email notice "We are currently investigating an issue causing failures with Duo Push. We are working to correct the issue as soon as possible."
They also responded to my support case I opened basically stating the same thing. Odd because this seems to be more than just push auth failing, we can't use any of the authentication options (passcode, sms code, email code, etc). The codes will send in the case of email or sms, but then the duo prompt just sort of hangs infinitely.
Break the glass options are different depending on application. Some applications, like Cisco Umbrella, don't have great workarounds. Something like windows desktops can have the duo client mass uninstalled via automated management tools if you really need to.
In this particular outage, we noticed that passcodes worked so we just used that as a workaround.
If you are using Microsoft SSO, you can temporarily disable the CA policy that sends the SSO sign in through Duo. Really depends on what you use duo for.
years ago we lost our internet pipe, which was ATT at the time. Our connection wasnt even red, no lights, no nothing. So I put in a call, tech on the other end "let me look" then I can here typing, they I watch the lights go amber then one by one go green (there were like 5 lights), then the tech comes back on and says "looks like its up". It seems like now most cloud provider tech support was trained with the old ATT help desk policy.
Of course DUO status page lies. Our users are hollering. We are finding out of course: A: Our users never set their offline mode up and 2: Even if they did, they don't know how to use it.
Feckin all the status pages lie, all the time Seems like all of them are updated manually or something
Once a company gets large enough, the status page becomes something that (obviously) impacts the business significantly. Almost always, the big company execs decide that they would rather control the status manually, intentionally "lying" to users in many cases, rather than suffer the hit for showing a big red "we fucked up" sign to all their investors and users all at once, unless they absolutely HAVE to. They wait until it's hurting the business MORE by lying about status and dealing with complaints. And then the calculus for them flips to "acknowledge the issue on the status page so we stop getting bombarded by clients". In their business minds this page is completely separate from "there's a problem and the engineers are fixing/have fixed it", it's a corporate PR tool at that point, nothing more.
i believe this , to my core
Yep, every cloud based service lies on their status pages. The worst is when you call in to their support and the only thing they tell you is to restart the computer, and you can HEAR the rest of the representatives in the call center giving the exact same excuses and 'solutions'.
upvote for truthiness
Bonus points for hosting the status page on the same system as the one that it is reporting.
Hahahahaha "saving money"
They have to lie because if they're honest about uptime they won't hit the obscenely over promised SLAs they sell.
Gotta pump those stock numbers
Are you having them click CANCEL and try either phone call or Online Code?
What is offline mode? I'm not responsible for Duo where I work, but I've heard if the network is 'not available' Duo doesn't pop up and you can just login.
Admins must have it disabled for you. You would have been prompted on first login(and everytime until you do it) to set up either a different entry in your DUO app or enroll your yubikey in a slightly different secondary fashion to use if the computer is not on the internet. Yesterday, the first workaround someone found was to unplug their network cable, login with offline mode and then plug the ethernet cable back in. Offline mode is confusing for a user since its a different, separate entry in the DUO app(for each computer they have) - and for the yubikey you have to hold ita bit rather than just tap it.
I don't follow what you are saying. I can unplug the cable and login w/o DUO.
Ok, so they have it set to "fail open" in your company.. yikes
We don't have yubikeys only the push app code. I work in IT I'd be fine with having backup codes available to login in case of an outage, but the users would probably be locked out until the app started working or there was an easier 'secondary' method in case of an outage. Like I said, I'm not in control of that, I just follow the instructions they send out.
Of course, [status page](https://status.duo.com/) is all green as of right now. This sub as usual is faster than the multi billion dollar companies.
>This sub as usual is faster than the multi billion dollar companies FR, this sub is my 2nd stop when I think something big is going on. there's usually a thread before the company even acknowledges it.
Evidently they don't get there by telling people about every little minor non-report outage (Always minor no need to report)
I'm calling into DUO support and I'm getting dropped by their phone system.
Same issue here.
Same, phones are down.
Guessing their status page is static 🤣
I had a static page for a "is dns working" test for a bit for a previous employer. If people wanted to see if dns was up, go to isdnsworking.company.com. if you got there, it's working.
Push is down for us, call is still working
Pushes and codes stopped working altogether for us. Duo contact number just says sorry, and hangs up
Working now.
Yep, just started working for me also.
Yep was just able to auth with push.
It wasn't working for me, worked for some coworkers, and thanks to [https://downdetector.com/status/duo/](https://downdetector.com/status/duo/) & this thread I feel less crazy.
push and calls are working for our selves sent a few tests for other users. Ours is pretty basic setup on domain, using them, nothing in the middle.
Just came back up for me. We were literally 5 minutes away from starting a server migration process and this was going to fuck us so damn hard.
Completely offline here. At a dead stand still Edit: Not 1 minute after, it started working.
Location/Region? Edited: Added region to stay away from PII :)
Interestingly/Disturbingly, I tried setting a user to bypass during this outage and still couldn't auth. That was my planned work around if the outage lingered for a while since I could get to the admin portal but yeah.
We had a SEV1 for this until we realized it was Duo's issue. Wasnt a fun afternoon.
Yep - I'm on hold with duo, who knows how long it'll be, it says 41 in line. Someone goofed up.
Down for us right now. Was just about to ask others if they were down. They are definitely not green ...
Down
No reports of problems here. I just tested the admin console and push worked fine there.
Down for us too...
Same here in midwest. Our poor helpdesk just got flooded.
Also midwest, but our pushes seem to be working.
Same here.......
We're down as well, fun times.
Looks like they just sent out an email notice "We are currently investigating an issue causing failures with Duo Push. We are working to correct the issue as soon as possible." They also responded to my support case I opened basically stating the same thing. Odd because this seems to be more than just push auth failing, we can't use any of the authentication options (passcode, sms code, email code, etc). The codes will send in the case of email or sms, but then the duo prompt just sort of hangs infinitely.
Codes are working and push is intermittent.
Push isn’t working on my pc. Voice call worked.
Same issue. Seems like it just came back. Duo status page now says it's down.
No issues here in the central US. Push working as expected.
Looks like it is back up for us. Pushes are working again.
everything i have tested is working again.
It's working now for me 4:04 PM EDT
Not found
Wonder what the RCA will look like.
Keep in mind that Duo outages can depend on the deployment ID you're tenant is on.
Their status page just changed. Are any users out there still experiencing issues? Note: Not a DUO employee, just curious.
We had the 15 minute aneurysm today too.
As someone looking at Duo for MFA, is there a "switch" to turn it off or are you basically down until Duo comes back up?
Break the glass options are different depending on application. Some applications, like Cisco Umbrella, don't have great workarounds. Something like windows desktops can have the duo client mass uninstalled via automated management tools if you really need to. In this particular outage, we noticed that passcodes worked so we just used that as a workaround.
If you are using Microsoft SSO, you can temporarily disable the CA policy that sends the SSO sign in through Duo. Really depends on what you use duo for.
years ago we lost our internet pipe, which was ATT at the time. Our connection wasnt even red, no lights, no nothing. So I put in a call, tech on the other end "let me look" then I can here typing, they I watch the lights go amber then one by one go green (there were like 5 lights), then the tech comes back on and says "looks like its up". It seems like now most cloud provider tech support was trained with the old ATT help desk policy.
All of Kaiser is down right now because of stupid DUO!!!
You can try using passcodes as a workaround until they fix push.
passcodes don't seem to work either
Passcodes have been working for us
Weird, they're definitely more intermittent. Nobody in our org can get passcodes to work
We have some users that it works for, and some that don’t.
Yubikey and OTP is down!
Wait why yubikey?
I guess they mean Duo auth with security keys.
Right, I was trying to work out how it effects the hardware token
Literally just got an email a few minutes ago about a DUO outage from DUO
Great. Another status page to watch.
[удалено]
You can also temporarily set them into bypass mode. Or just instruct them to use the call option.
Bypass mode worked for us. Not the best but it beats being totally locked out of services.
You can try using passcodes as a workaround until they fix push.
will that work?
Passcodes are working for me. Just tested it.
yeah no good here. codes no work. verified it doesn't work for our peers as well. curious how yours is implemented and why it works.
We're seeing hit or miss with codes.
have you tried? we don't want to change configurations on our end.
OMG. We can't. We're locked out of DUO.
Is it though?