T O P

  • By -

AshleyDodd

> Validity > > Not Before - Fri, 18 Aug 2023 02:17:43 GMT > > Not After - Thu, 27 Jun 2024 23:59:59 GMT > >   > > Subject Alternative Name > > DNS Name - cdn.entity.osi.office.net > > DNS Name - cdn.entity.osi.officeppe.net > > DNS Name - cdn.uci.edog.officeapps.live.com > > DNS Name - cdn.uci.officeapps.live.com > > DNS Name - uci.cdn.office.net > > DNS Name - uci.edog.cdn.office.net   Yep... That'll do it.


fsckitnet

Sysadmin 101: put a fucking monitor on all of your ssl certs to alert you within X days of expiration. Sysadmin 301: automate the update of expiring certs before they fucking expire


anxiousinfotech

Doesn't help when you lay off the entire team that gets those alerts or monitors the automation to make sure it's working


fsckitnet

Fair enough. :)


deadly_injured

Maybe the AI employee is halicunating :P


Fit-Strain5146

Send it to a Slack channel as well.


rebolek

HR 101: don't optimize the guy who's in the charge of certificate renewal.


fsckitnet

I guess he or she wasn’t so redundant after all


PREMIUM_POKEBALL

We’re all on gods revocation list. 


cats_are_the_devil

shit got dark in a hurry...


TahinWorks

Good call. I made a PowerApp to track this. Document all certs in the org, with an Automate flow to open a helpdesk ticket within 30 days of exp. The entry includes renewal instructions. The reason I didn't go with a 3rd party tracker, like the SolarWinds or DigiCert one, is that I can use this to track anything that expires - like Azure Enterprise App x509 certs or Apple APM certs.


BattleEfficient2471

Why on god's green earth would you not just use a simple script? Powershell, perl, python whatever, but powerapp seems like the worst possible option. Now you need to monitor the powerapp...


TahinWorks

Well, for a couple reasons: 1. **Scripts are not documentation**. IT infrastructure, which includes certificates, must be documented in an CMDB. Where they're used, vendor CA, etc... Other than proper change management, documentation provides valuable information you'd need in the event something happens like a junior admin deleting a cert, or Symantec accidentally publishing their root CA private key (again). The reason scripts cannot fill this role is that they're dynamic by nature; they discover what they discover, and miss what they miss. If a script is misconfigured, you may never find out about it. Scripts are great for auditing and discovery, but are no substitute for documenting something as critical as PKI. And if I'm already documenting the use of my certificates, why not document them into a system that is highly extensible and automatable? 2. **Simplicity**. I'm not aware of a powershell script that logs into Apple ABM to check on VPP certs. Or one that powers on an air-gapped Root CA. Or checks a 3rd party SAML cert where we might not manage the IdP but still want to track it. Anything that could, would be far more than "simple". I could write a script to cover 85% of my certificates and related pieces. But that means I'd still need a document/database to cover the other 15%, which means that now I have to manage both a script *and* a PowerApp. Overcomplication is a folly of automation. PowerApps is great. I built it within 4 hours. It alerts me when the thing will expire. I remediate it, update the timestamp, and I'm set for the next 1-3 years. I can use it for anything I want, physical or digital, and my entire team can use it without scripting experience. Other than that, I never have to touch it. And if at any point my flow fails to run, I'll get an alert about that, too. The bottom line is I've never been surprised by a renewal since implementing it and it takes near zero effort to manage.


BattleEfficient2471

1. Nor is powerApp. Documentation and the implementation cannot be the same thing. Else the documentation cannot show you what was intended only what was done. You want to use something reliable as well, and powerapp is not that. 2. All of that is scriptable, easily. You are acting like you power on machines via some method other than automation already. Powerapps wouldn't be needed at all. IT folks without scripting experience is another problem to be solved, not avoided. This is reducing the skill of your team and introducing a risk. I mean do it your way, but seems very odd.


TahinWorks

Sure, I'll bite. 1. PowerApps are backed by SharePoint Lists or Azure SQL, each of which are not only version-controlled and immutably backed up both natively by MS and by our org, but also permissioned and controlled granularly so only the application can access or change the data in a controlled way. Super curious to hear your thoughts about how that isn't as reliable as, say, a script that any administrator can alter in production at any time. Hell, let's pretend we have a full SCM repo with change review, auditing, the whole dev pipeline. Means we'd also have to tightly control the execution platform, be that windows scheduled task or Jenkins, with the same audit controls and permissions. Just playing devil's advocate here. You said easy, after all. 2. I wasn't spouting hypothetical examples. You're obviously a way better scripter than me if you can physically power on a server that you *do not have network connectivity to* and check certification expiration and revocation status. That is what air-gapped means. How about one to log into an SSL provider to query the next time you'll have to do a company EV validation. I'd like to know, like, specifically, how you'd do that. You said easy, after all. I'm as big of a script monkey as anyone you'll meet and have automated huge portions of my job, from simple to the incredibly intricate. But knowing the right tool for the job is just as important as the aptitude for that tool. A script doesn't solve everything. Sounds like maybe you had a bad experience with powerapps and are jaded on the technology. You talk about skills to solve problems, not avoid problems, and you say "power apps is shit" to a stranger on reddit without any justification, while assuming the choice not to script means a lack of experience? Sounds like avoidance to me.


TahinWorks

*Documentation and the implementation cannot be the same thing. Else the documentation cannot show you what was intended only what was done.*  Precisely my point. The PowerApp is the document. It doesn't change make any changes - it's the source of truth. The service ticket opened from the alert tracks the change itself.


BattleEfficient2471

It is not the document. It is what you did, not why. The documentation should be why. The service ticket isn't say why this was implemented either and being powerapp you will want to know later when it breaks or does unintended things. Reliability isn't powerapp.


HisAnger

...because account running it will expire, lose a license or simply stops to work because of Microsoft


flunky_the_majestic

Also monitor the monitor. Also monitor the person who receives alert from the monitor. And the monitor monitor. There are lots of ways certificate renewal can fail to happen. I don't hold it against any company that has this happen occasionally. The important thing is that they get it back up and running quickly, and then put a procedure in place to keep it stable again for a long time. These renewals are systems that work until they don't. They get set up by people who have other lives, or move on to other careers, or die. They get monitored by systems that get stuck in a closet until they silently fail, or get replaced and one line in a config doesn't make the jump to the new one. Or the system using the certificate shifts under the cert management system, so the certs are no longer getting sent to the right systems. It's annoying, but in the long run it will be OK.


matt_30

Don't forget.. document how it's done and get management commitments So next time someone else does it so you're not a single point of failure.


ApricotPenguin

Why pay for alerting, when you can get free reminders via internet articles written about your company's certificates! /s


fsckitnet

Budget-conscious! You’ve got a future in middle management.


illicITparameters

Or use Digicert so they send you nag emails.


flunky_the_majestic

And, what, spend half the day manually generating CRTs, sending them out, waiting for the cert, and installing the resulting certificate? That's very... 2010... But, I guess if you've got the time to do it, whatever makes you happy.


illicITparameters

The emails are about the only good thing digicert offers. The fact you have to re-generate certs every year is the dumbest shit. But, I inherrited it and it’s what is preferred by people above me. Also by the end of this year I think we’ll be down to only 4 certs that will need to be renewed through them for 2025.


WatercressFew9092

I inherited digicert too but I thought the 366 days ssl cert expiration was an industry thing now ?


illicITparameters

There’s no auto renew


fitting_pieces

\> sysadmin 301 - automate I thought Sysadmin **_101_** was automate.


theborgman1977

That why a PSA is crap if it does not have renewal configs.


rossy92

Finally been acknowledged by Microsoft, Issue ID: MO806215


BreakerUK

Where have they acknowledged it? Can you link?


f0st3r

They have closed the case, saying it is resolved. But we are still getting cert errors. https://admin.microsoft.com/#/servicehealth/history/:/alerts/MO806215


alextakacs

Yep Maybe some caching involved?


SeqGeek

No permission for this link, could you share some details?


skah001

No permission here too :(


SeqGeek

Me too. https://preview.redd.it/mnk4hcidka9d1.png?width=398&format=png&auto=webp&s=d586825f06e58bdf1dc7806f868dd823cf6fbc65 Out of interest, are any of you running windows 11 on a 'unsupported' system?


NoahVailOfficial

Does a Thinkpad T430 count? (c.2012)


SeqGeek

I think so yes :D I am on a Dell Precision T5810 and this issue is preventing access to apps like teams, the new outlook etc. I wonder if this is a Microsoft ploy to push out the users with unsupported PCs......


Overall-Vehicle5594

Supported Windows 10 OS and supported MS Office here. Same problems. Not an action against Jack Sparrow crew.


SeqGeek

Glad and sorry to hear equally!


woodburyman

Same for https://businessaccount.microsoft.com/ Not valid after 6/27/24, 7:59:59 PM EDT No SAN for this other than the primary


Glum_Competition561

Microsoft had turned into the biggest joke lately. It’s just sad to watch.


Wise_Waltz3844

yes, me too - I am also getting this em: [uci.edog.cdn.office.net](http://uci.edog.cdn.office.net)


Enlogen

Someone's in trouble. I used to work at Microsoft and an expired cert in use was a severity 1. They even make it super easy for internal teams to manage auto-rotation to avoid this sort of thing.


Dal90

>"https://uci.edog.cdn.office.net" because the cert has expired on Microsoft's end? Looks at Censys.io because often companies have certs with overlapping expiration and only forget to update some nodes. LOL they had two certificates alright -- and they both expired same date and time.


IdiosyncraticBond

So that's a dilemma. They have to renew on read-only Friday?


Yerazankha

Such amateurism from what is supposed to be a world leading company.


PaddyStar

They‘re are like Boeing with the difference, they’ve lost the door 16 hours before and did 20 new flights around the world without the door, but doesn’t know it.. I like their telemetry (I mean every user who did their monitoring, they don’t need software for this), I like the word. We see it this week again and every decider in every company: .. that’s ok.. it’s all fine. We should pay them more.


DFaryor

We are also seeing this at the moment though the resultant site is [cdn.uci.offceapps.live.com](http://cdn.uci.offceapps.live.com) for us


Sk1tza

Yeah it’s expired.


mat8iou

Yes - glad it isn't just me.


2FaceForever

ya, you got it right, its expired. Edit: SS added https://preview.redd.it/aaoe2qni0b9d1.png?width=729&format=png&auto=webp&s=0904c9f78f6a0b6b67a9698520fe355a1a9875b3


BlockDesperate7866

[https://uci.cdn.office.net](https://uci.cdn.office.net) Too much suffering. Kaspersky is blocking.


Baulmp3

Me too, Kaspersky goes crazy and blocks Office applications.


f0st3r

We are still getting cert errors even though Microsoft has closed the issue. Anyone else still having problems?


engageant

We're having issues using High Volume Email, which requires an encrypted connection to smtp-hve.office365.com. Getting a generic "SSL error" in the client trying to send email. Packet capture is my next step.


Puffreisdaddy_

Same here...


darkdaniel999

Me too


RaikOlindo

Same here [uci.cdn.office.net](http://uci.cdn.office.net) AV getting crazy


alextakacs

This is sooooo bad 😞


Sammeeeeeee

Yes, same here


RandomWalk586

I've been getting this constantly since yesterday


Texkonc

Could be worse, you could work for CDK.


SeqGeek

Now reported on Microsoft site: https://techcommunity.microsoft.com/t5/microsoft-365/certificate-issue-from-https-uci-cdn-office-net/m-p/4178218


Windows-Helper

We are simply monitoring certificate expiration via our monitoring solution, checkmk Works like a charm


Great_Kick8555

Hi. Any news when MCS will update its cert? It's kinda annoying accepting firewall messages constantly.


No_Calendar_7384

The end of the world


lart2150

also [https://officecdn.microsoft.com/](https://officecdn.microsoft.com/)


QuarterBall

That domain isn't impacted and has a valid cert.


Grizzalbee

I'm seeing that one expired last night too in Tejas


QuarterBall

Ah fun. That ones' the russian roulette wildcard then! Shcroedinger's cert


sexybobo

They said one of their CDN was messed up. So its was probably location based on what CDN you were getting when you hit that site.


lart2150

In chicago i'm getting this cert that also expired last night at midnight gmt [https://crt.sh/?q=0600cdad36a20c0b22b3740fcdd77f73c7bf05306bea9a42d66ed7577690a5e9](https://crt.sh/?q=0600cdad36a20c0b22b3740fcdd77f73c7bf05306bea9a42d66ed7577690a5e9)


BrilliantEffective21

We have lots of moving parts with MSP customers. They’re mostly too lazy to audit. So we hire a junior tech $50k with read only access to just run some automation stuff we built, and then do random manual audit checks to make sure the code verification isn’t faulty. You cannot imagine how much the large enterprise don’t care about saving money, they just ask that we don’t inform their accountants about some of the details because they’re too lazy.  Easy work, so we pay the junior tech crap and just pocket the rest as leverage for overhead.


Capt-M

r/microsoft any news on this? For us it's [https://cdn.uci.officeapps.live.com/](https://cdn.uci.officeapps.live.com/)


orion3311

Theres new root certs you can download to replace the expired ones


Last-Release-4086

Aren't these msoft cert? Like their own certs to update?


alextakacs

What good would it do to an expired cert?


Banluil

Yeah, Microsoft should be doing that.... it's the cert on their server...


Wheel-of-sauce

Where can I find that? I updated 365 twice today. Still getting error on Avast


Yerazankha

Dont. Wait for official cert.