Bug Story Time: Firefox Edition

Back when I was fairly new to testing I had an interesting situation come up.

At the time I was working for a company that ran an energy supplier comparison website for a major UK brand.

One piece of data that was vital for a user to get a quote was their address, so we had a form that allowed them to enter this information.

It was a required field in the form, so there was absolutely no way a user would be able to avoid entering this information.

Apart from us only using client side validation on this piece of data, that is. At the time I wasn’t aware there was a lack of server side validation for this field. Like I said, I was new to testing.

So one day we received feedback from our client that some users were reporting they couldn’t use the energy comparison site because they were running into an error.

We checked those users and the error messages they were encountering, and found they’d somehow managed to bypass the client side validation of the address field, and were nullpointering when trying to get quotes.

We quickly put their data back to a state where they could enter their details again, but how had they managed it?

I played around with theories for a couple of days, and through devtools in Chrome confirmed that the validation could be removed by users and this broken state could be achieved. But why would a bunch of users suddenly decide this was a good idea?

We slapped some server side validation onto it, and moved on.

Except that I hate not knowing. So while waiting for the client to approve the timeline for putting the patch live, I continued to ponder.

Eventually I looked up each users browser data in the system, and there was an interesting pattern. They were all Firefox users. Every. Single. One.

From the system logs we learned that every user that ended up in the broken state did it within a one week period. We hadn’t done any updates to the site in that timeframe, so it seemed impossible for us to have broken and then fixed it within that time.

For another day or two I spent any spare time I had playing around in Firefox trying to find what these users had done. It just wasn’t working though. Other than intentionally removing the validation there didn’t seem to be a way that this could have happened.

And then I accidentally crashed Firefox. I don’t remember how anymore, but it had crashed while our address form was open and filled in. For some context, it was that kind of address form where you put in the postcode/zip code, and are then presented with a list of addresses that match. This data was then stored in a read-only part of the form, and the user was unable to alter it directly.

When I opened Firefox again it restored the tabs that were open prior to it crashing and… It only loaded in half the data in the address form, but it looked like the form was in a state that could be submitted.

I pressed the submit button and sure enough everything broke! But still… Why would so many users suddenly crash, and only over a period of about a week?

I had a reasonable suspicion, so I set about trying to confirm it. What else might cause tabs to be restored other than a random crash? Automatic updates.

I searched for a list of Firefox versions and when they released. Sure enough there was a major version put out the week of the errors!

So it turns out that a bunch of people started filling in our form, probably left it to do something else for a while, and came back to Firefox having updated and failed to properly restore our form.

This is the bug that’s stayed in my mind all these years. Partly because of the deep investigation, and partly because of the sheer luck of having Firefox crash at just the right time to discover a likely cause.