A few years ago I designed a way to detect bit-flips in Firefox crash reports and last year we deployed an actual memory tester that runs on user machines after the browser crashes. Today I was looking at the data that comes out of these tests and now I'm 100% positive that the heuristic is sound and a lot of the crashes we see are from users with bad memory or similarly flaky hardware. Here's a few numbers to give you an idea of how large the problem is. 🧵 1/5
You can’t effect the number of bit flips your users hardware has, but you can affect how often buggy code corrupts their memory or otherwise crashes your program.
Let’s say any app will crash about once a year on my machine due to a bit flip. If the app is crap and crashes hundreds of times for other reasons, the bit flip is irrelevant. If the app is robust enough that the bit flip accounts for 10 % of the crashes, that basically means the app is pretty much never crashing due to poor code.
As a long time Firefox user, I believe Firefox sees orders of magnitude more RAM issues than other apps because it is using orders of magnitude more RAM than other apps.
You’re assuming that app quality is constant. But if I made an app that crashes on launch, I can confidently say 0% of those crashes would be from bitflips.
Firefox isn’t special in some way that could cause bitflips, but it’s 1) where this data was collected (and why this post isnt talking about some other product) and 2) speaks to the quality of FF, because crashes are rare enough for bit flips to be a significant crash factor.
The takeaway is that for the FF team, and anyone using ram (everyone), bitflips are more common than expected
It would be interesting to see how this works in Chrome. I would guess that it could be the same - people tend to leave their browsers open with hundreds of tabs and will never reboot their laptops. If you play a random game for 2 hours, bit flips shouldn’t be a problem. But if you keep your browser open for weeks or months with hundreds of tabs, that may cause problems.
… I can’t imagine having a browser with hundreds of open tabs. That would tend me of the old days of Netscape Navigator and all the popups and browser add on cancer.
Ahh the nostalgic days of the early Dotcom era. I sometimes miss you geocities
Wouldn’t that mean ten percent of all crashes in all apps would be caused by bit flips? What makes Firefox special?
You can’t effect the number of bit flips your users hardware has, but you can affect how often buggy code corrupts their memory or otherwise crashes your program.
Let’s say any app will crash about once a year on my machine due to a bit flip. If the app is crap and crashes hundreds of times for other reasons, the bit flip is irrelevant. If the app is robust enough that the bit flip accounts for 10 % of the crashes, that basically means the app is pretty much never crashing due to poor code.
As a long time Firefox user, I believe Firefox sees orders of magnitude more RAM issues than other apps because it is using orders of magnitude more RAM than other apps.
You Firefox also hoards RAM? I thought it’s just mine.
You’re assuming that app quality is constant. But if I made an app that crashes on launch, I can confidently say 0% of those crashes would be from bitflips.
Firefox isn’t special in some way that could cause bitflips, but it’s 1) where this data was collected (and why this post isnt talking about some other product) and 2) speaks to the quality of FF, because crashes are rare enough for bit flips to be a significant crash factor.
The takeaway is that for the FF team, and anyone using ram (everyone), bitflips are more common than expected
It would be interesting to see how this works in Chrome. I would guess that it could be the same - people tend to leave their browsers open with hundreds of tabs and will never reboot their laptops. If you play a random game for 2 hours, bit flips shouldn’t be a problem. But if you keep your browser open for weeks or months with hundreds of tabs, that may cause problems.
… I can’t imagine having a browser with hundreds of open tabs. That would tend me of the old days of Netscape Navigator and all the popups and browser add on cancer.
Ahh the nostalgic days of the early Dotcom era. I sometimes miss you geocities