Technology

59589 readers

2910 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

830

CrowdStrike downtime apparently caused by update that replaced a file with 42kb of zeroes (twiiit.com)

submitted 4 months ago* (last edited 4 months ago) by Aatube@kbin.melroy.org to c/technology@lemmy.world

190 comments fedilink hide all child comments

…according to a Twitter post by the Chief Informational Security Officer of Grand Canyon Education.

So, does anyone else find it odd that the file that caused everything CrowdStrike to freak out, C-00000291-
00000000-00000032.sys was 42KB of blank/null values, while the replacement file C-00000291-00000000-
00000.033.sys was 35KB and looked like a normal, if not obfuscated sys/.conf file?

Also, apparently CrowdStrike had at least 5 hours to work on the problem between the time it was discovered and the time it was fixed.

you are viewing a single comment's thread
view the rest of the comments

[–] diffusive@lemmy.world 151 points 4 months ago (33 children)

If I had to bet my money, a bad machine with corrupted memory pushed the file at a very final stage of the release.

The astonishing fact is that for a security software I would expect all files being verified against a signature (that would have prevented this issue and some kinds of attacks

[–] BossDj@lemm.ee 51 points 4 months ago (9 children)

So here's my uneducated question: Don't huge software companies like this usually do updates in "rollouts" to a small portion of users (companies) at a time?

[–] deegeese@sopuli.xyz 2 points 4 months ago

When I worked at a different enterprise IT company, we published updates like this to our customers and strongly recommended they all have a dedicated pool of canary machines to test the update in their own environment first.

I wonder if CRWD advised their customers to do the same, or soft-pedaled the practice because it’s an admission there could be bugs in the updates.

I know the suggestion of keeping a stage environment was off putting to smaller customers.

load more comments (8 replies)

load more comments (31 replies)