Dear Facebook: You Lost My What?

By M. David Peterson
November 25, 2008 | Comments: 44

So the following represents an email that recently arrived in my inbox:

To: xxxxxxxxx@gmail.com
From: Facebook <xxxx+xxxxxxx@facebookmail.com>
Subject: Please reset your email notification settings.

Unfortunately, the settings that control which email notifications get sent to you were lost. We're sorry for the inconvenience.

To reset your email notification settings, go to:

http://www.facebook.com/editaccount.php?notifications

Thanks,
The Facebook Team

At first I thought this /had/ to be a hoax. But in fact, after logging into my Facebook account, lo-and-behold right there at the top of my home page was:

FacebookNotificationsWTF.png

Facebook? Are you serious?! How exactly do you "lose" the email notifications settings? In a day and age where system backup has become a basic and ongoing operation that in many cases is an integrated part of each application process (e.g. someone uploads a file, it gets saved to a local disk cache, that then invokes a background process that stores the file on an archive "disk" which then replicates itself in multiple locations across the globe automagically), how do you "lose" the settings for /anything/?

Disks get fried all the time**, that's understandable. DB files get corrupted often enough to provide justifiable reasons to find "We are currently undergoing [scheduled] maintenance. Please come back later"-type messages when attempting to visit any given site. Hell, systems get hacked often enough to recognize there are more than just system software upgrades and corrupted DB's that sit as the root cause/reason for these same messages.

But lost data? /LOST/?! If your system is designed in such a way that there's reasonable possibility the phrase "lost data" can be used for reasons /other/ than than /massive/, global wide natural disasters or nuclear holocaust (neither of which I've heard anything about, though I guess if everyone else is dead, that would be a good reason why I haven't heard anything about it. ;-)) it's not only time to redesign the system:

It's time to fire the entire engineering staff.

And then go out and find software engineers who understand what the phrases "data redundancy", "system and data backup", and "system failover" actually mean. Because any systems engineer whose worth even close to what you pay him/her understands that while system failure is an expected occurrence, its an occurrence in which the word "lost" only applies to things like "lost power" or "lost network connections". Not "lost the whole damn data center and every single backup file we've made and stored in nuclear proof storage containers across the globe just in case we /do/ lose the whole damn data center."

Anyone at Facebook care to step up to the plate and tell us what happened? Is this an isolated incident or is this a system wide thing? Maybe I'm the only one who got the above email and home page notification? If yes, while it's still unacceptable to lose /anyone's/ data, maybe who ever sent the message and notification did so with the idea:

"It will be easier just to send him a message and notification telling him to reset his settings than it will be to get off my lazy a$$ and restore his settings from backup."

If this is an isolated incident or simply a case of laziness on the part of some engineer, then maybe firing the entire engineering staff is a bit hasty.

But if not? It's not.

Seriously.

Facebook?

--
** Something I've painfully learned /twice/ in the past three weeks (though I only have pics from one***, I have plenty of witnesses from last weeks MSFT WebDev Summit for the other ;-)). Yes, I wish I was kidding. And no, I'm not, though fortunately there was very little overall loss due to a little somethin' somethin' I like to call /BACKUPS/ (Oh, and to give MSFT some /HUGE/ props, it was because of Windows Live Mesh and the Mac OS X client application that saved my a$$. Otherwise I would have been re-writing a whole hell of a lot more than a few code files that hadn't been synched up yet.)

*** Yes, I too noticed the recursive irony in the fact that I am linking to Facebook-hosted pics of "your hard drive is dead, sucka!" messages on the screen of my MacBook Pro while bemoaning Facebook for "losing" my email notifications settings data.

If irony wasn't bad enough, we have to throw recursion into the mix?

Damn recursion. ;-)


You might also be interested in:

44 Comments

You weren't the only person to get the email.

Question: were almost all of your notifications set to off, before they 'lost' the settings?

@Michael,

>> You weren't the only person to get the email.

Interesting. And based on your email address sent with your comment notification, you're in the UK which means this is more than a regional thing (I can only assume that based on both it's original focus and ongoing growth pattern based primarily on individual universities, cities, regions, etc., the DB cluster expanded, grew, and is structured accordingly.)

>> Question: were almost all of your notifications set to off, before they 'lost' the settings?

Not almost. /ALL/ of them were. Fortunately, the interface for application settings is fairly well designed from a usability standpoint, so for people like you (I assume?) and me we can switch them all to off w/o much concern. But for those with 100's of applications with notification settings which /were/ in various states of on and off, coupled with the fact that each application can gain access to multiple notification types (i.e. email, notification feed, personal feed, etc.) this could turn into a /real/ pain in the a$$ for a lot folks.

This could get ugly /fast/ if this is as widespread as it seems it may be, especially considering FB has such a /huge/ base of college students who use FB and FB apps as their primary source of communication and interaction with their peers.

What strikes me as odd is that there was no attempt in the email sent out to state:

"we really messed up, we're working round the clock to remedy the situation and ensure we understand what happened and why so we can fix it. We're really sorry for any pain and interference this has caused with your use of FB and FB applications. Here, have a Ben & Jerry's on us. Oh, and we've set up this FAQ and support forum for anyone who has any further questions or concerns."

Hell, I'd bet they could have even made a phone call and found ways to get Ben&Jerry's to pay for the increased exposure and additional sales to help alleviate the obvious expense that's going to hit their bottom line pretty hard.

The way this was handled, you'd think some 24 year old college drop out** was running the company.

Oh, wait... ;-)

--
** No disrespect meant to college drop outs. I'm one of them. But I'm not the CEO of a social platform company with ~150 million members and I haven't been 24 for a /LONG/ time (though that's obviously a moot point. I could be a 20 year college drop out. I'm still not the CEO of the above aforementioned social platform.)

@Michael,

Having had too many other things to attend to first, I hadn't yet gone in to change the notification settings, and now taking a quick moment to verify the interface was as easy as I remember**, from what I can tell, non-FB application settings weren't effected. Still a pain, and something that shouldn't have happened, but at least it seems the overall damage is less than first thought.

** It is, though it seems they've simplified the application settings such that they inherit the system settings for each notification type, having only an on/off switch of their own.

Hey Facebook: Could you do us all a favor and quickly script up a "Set /ALL/ notifications to off" radio button/switch? For those of us who replaced email with Atom/RSS feeds /long/ ago and don't /ever/ want to be emailed for /anything/, having to go and click what felt like about 40 radio buttons to accomplish what could /easily/ be accomplished with one is aggravating.

Thanks!

Obviously this is just a cheap way for them to re-enable email notifications for people like me who receive way too much from them... Actually it will boost their ad revenue probably by forcing people like me to log in and change the settings all while displaying a couple more ads. There is NO WAY facebook can lose just the notification settings. Genius really... Maybe their having a bad quarter in ad revenue?

@Obviously,

Haha! You could be onto something here. I agree with the idea that all of that data suddenly going *POOF*, simultaneously disappearing from all known backup locations on the planet seems over the top. And when companies like Microsoft, Google, Yahoo!, etc. are spending /millions/ of dollars each year to determine how much click-through rates increase or decrease based on moving text and lines a single pixel in any direction, this could easily be a realistic scenario.

When you've got ~150 million members, I can easily see internal analysts crunching everything from whether adding the ability to edit a comment increases or decreases revenue (e.g. if we allow people to edit a comment, do they spend more time editing and less time clicking ads?) to comparing the cost of damage for the "event" of "losing data" with the expected increase in ad revenue from the number of people who will log in to fix the "damage".

Crazier things both /have/ and /do/ happen quite often, so you never know.

I have to say I thought exactly that.

I've encountered several firms who have used the cover of a 'regular update' to change my defaults to receive more email than I wanted.

I guess we'll know that's what it was about when they begin to put ads in the emails.

Ditto - same message to a Canadian Email Address!

Lousy!, Just Lousy..

I agree,

Fire them all

DR Smith

>> I guess we'll know that's what it was about when they begin to put ads in the emails.

Haha! Yup, that's about when the proof will arrive, though as per a recent tweet:

"Laurent Lellouch++ proved more than e=mc2; They proved intelligence requires a heart+soul to know true what can't yet be proven by science."

IOW, sometimes scientific proof is little more than creature comforts for things we already know are true but lack physical evidence.

Actually, the moment we'll know the smoking gun has been found is the day Facebook retains the services of F. Lee Bailey to cross-examine whomever it was that found it. ;-)

@DR Smith:

>> Ditto - same message to a Canadian Email Address!

Eveel bastards. ;-)

>> Lousy!, Just Lousy..

So it just occurred to me that this so called "lost data" event took place two and half days before Thanksgiving, a week that the press is preoccupied with prepping people (wow, that's a lot of "p[r]" words in the same sentence. Trying saying it 10 times fast ;-)) for what to buy and not buy on Black Friday, which is why I'm guessing we're not seeing a lot of extended coverage on the topic from anywhere else.

By the time people return to the office next Monday they'll be /FAR/ more interesting and recent news to write about, so for all intents and purposes, this whole "data loss" thing will go unnoticed during the same week people who rarely use their Facebook accounts -- the exception being during and just after the holidays -- will be uploading and/or viewing /HOARDS/ of family photos to/from Facebook, the "lost data" notification having arrived at a time they were too preoccupied with preparations for Thanksgiving to even take notice and therefore receiving email notifications for new photos/messages/etc. for the simple fact that they haven't reversed any changes to the email notification settings that Facebook "lost".

>> Fire them all

Well, in rethinking this through, it really depends on whom "them" refers to. If this was /truly/ an engineering error, then yeah, fire them all.

But I think we (referring to us engineers) all know the chances of this data having actually been lost are basically zero. You can pretty much stop crunching on the "justifiable" additional revenue figures once you multiply the millions of additional folks receiving email notifications of holiday photo uploads to Facebook by the number of people uploading pics over that same period.

Still being a private corporation, I have no clue what kind of revenue Facebook is generating these days (anyone have the numbers?), but you don't have to have a Ph.D. in Physics to know that when families and friends come together for the holidays the end result is a pile of incriminating evidence called photos, and an entirely /new/ phorm of phamilies and phriends now bound together by this evidence.

Of course, when those photos wind up on Facebook (and they /will/ wind up on Facebook at some point ;-)) between the end of November and beginning of January you get what can only -- and obviously -- be termed: The PHacebook PHolidays.

Now multiply the PHacebook PHolidays with that BIG /PHAT/ number of additional notifications being sent out and you wind up with an additional page impression multiplier of a /minimum/ of 1-2 billion, potentially quite a bit more.

In this regard, while I hate to both admit that I'm wrong while accepting defeat, you don't have to get past the first verse of the Ph'ified version of "Deck the Halls" before realizing whatever the number of additional PHranklin's trimming the PHacebook PHolidays Tree, that numbers is going to be /PHAT/ and /PHUGLY/ (the good, attractive kind ;-))

Something tells me there's going to be a lot of:

Deck the phalls with boughs of pholly,

PHa la la la la, la la la la.

Tis the season to be jolly, 


PHa la la la la, la la la la.

... being sung through the phalls of PHacebook over the coming PHolidays. ;-)

PHUCK. I've been phwnd by PHacebook. But I have to admit that the PHacebook PHat Cats are phreakin' /BRILLIANT/!

EVEEL!

But /BRILLIANT/!

Yeah, I was very dubious when I read the message.

And seriously, can we get a show of hands of how many people who had their setting lost (passive voice on purpose) had a majority of them set to off? Mine were.

But what REALLY irked me was the passive voice they used. They warned "... the notifications were lost ... " Who lost them? How were they lost? Why can we rest assured it won't happen anywhere else? Why were backups not possible?

They make it sound like the new cleaning crew mistakenly threw out the boxes they stored them in.

It reflects very poorly on Facebook.

Sounds like phishing to me.

I experienced the same problem

I also thought it was a hoax, because how the heck do you lose secured personal information?

and its no hoax.
it looks i also lost most of my settings.

i just want a good explanation from facebook, not just "We're sorry for the inconvenience"

I too received the email, and suspiciously I had all my emails turned to off also. Facebook has blatantly found a new way to spam its user-base.

I really thought it was phishing, i've taken lots of steps to remove junk emails, ad's and unimportant emails, and seeing a random email telling me they lost my settings and wanted me to click a link blew my mind, even more so because i had clearly clicked every botton asking them never to fucking email me again! Phishing phishing phishing!!! But no, it was real. Fuckers!

Yes, I just received the same e-mail, which brought me to this site when I checked out its veracity on Firefox. I'm in BC, Canada.

Hi,

I'd the same problem, but what I've found out is that contrary to other times I've logged in into facebook, there's a java-script coming from msn.com. After noticing this, a natural question arise:

Does microsoft has anything to do with this "lost" data?

Anyway, from now on I'll think twice before adding personal data into facebook.... maybe they will "lose" it again :S

M. David,

You really should consider actually getting in touch with a human at Facebook for a better explanation. If this was an engineered 'loss' to pump up ad revenue by forcing users to visit the site, it would be very big news indeed.

Also interesting would be finding out if all users who received this message had their settings originally defaulted to no mail or 'none' for every option, and they were reset to a default other than none.

This happened to be today. But a similar thing happened a few weeks ago - with an email and a notification box, but it was my PRIVACY SETTINGS that were lost. That was even scarier, that that could happen. And now my notification settings were lost.. What's next?

yup, I got it too... I couldn't understand how they lost data - when ti should have been backed up - and couldn't understand why they emailed everyone like fools to admit it LOL~!

I think it's because they're developing the new look for Facebook and probably not backing that up as much.

you guys have to remember that doing a backup for such a system is pretty difficult considering how many people are logged in at any point in time and how many are doing changes, etc. There is NO downtime... so even if they didn't backup for 5 minutes, they would have lost a lot of data! That's cause they probably don't know how many users have changed their email notification settings in the 5 minutes span. That's my time on what has happened anyway.

We're all very familiar with backup procedures, but I am sure it is much more difficult in a rapidly changing environment like that of Facebook's.

My thoughts anyway and I could be wrong, but... :)

Today this happened to me, most notifications were set to off too! that is weird

Yup, this happened to me, too.
and it's a bluewin.ch mail address, that means it reached Switzerland.

And yes, as you can guess, most of my notification settings were turned off. And are again. Even more of them.

But now I understand what they mean with
"Even if you turn off all notifications, we may sometimes need to email you important notices about your account." ;-(

Yeah same here in Kosovo (SE Europe). My nofications were mostly OFF.
what bothers me is the content of the email that i have received from FB. it doest say a thing about what happened or how it happened or will it happen again, and also it makes you belive that this is a everyday situation (we loose the data all the time, (its that lazy John again, not clicking the backup button) so, sorry but just click here and reset your notifications...)...
Its shoking...

BTW I totally agree with Felipe:
"Anyway, from now on I'll think twice before adding personal data into facebook.... maybe they will "lose" it again :S"

Yes, it was really something that look bogus. It was so suspicious that I have even reported it to gmail. :) (Poor gmail should have got a lot of these records)

There are a few important issues here:
1- How come a site like Facebook lose data like this? Security problems? Technical incompetence? Are our other personal data safe with them?
2- How come a site like Facebook sends such a stupid email that everybody thinks its bogus?

The problem with the email is they don't even address you personally which is something that every genuine email from a web service you use would do in this day and age given the amount of spam. They don't even address you personally but then can't even bother to even start off the email with a "dear user" etc.


They also don't give any explanation as to what happened and what they plan to do to address it. It is such a poorly worded email that I really thought it was spam when I received it. This isn't really any surprise however since Facebook are very poor with customer communication. They cared when they were small and now that they are bigger than even MySpace - they do what any company their size do and just don't give a shit.

HOW CAN THEY SEND THE ABOVE MENTIONED MESSAGE IF THEY HAVE LOST THE INFO WHERE TO SEND IT TO ???

Somethings aint right about that... but I really dont care to much, I get a sence that its a spamming thing.. "just to see how many of those ppl are dumb enouh to comply" or any other (commercial?) reason..

Havent logged in to my facebook for weeks and after this it will happen even less frequent than now :D


I'm in the UK and just received the same email - thought it was a phishing scam so googled it and found your blog - it's a joke, what a bunch of amateurs.

Im in Uruguay and i just got the same email. I think I will cancel my fb account---- i dont trust them anymore

In Italy and received the same e-mail.

I'm in the U.S. and just got this message too. Sucks!!

"Unfortunately, the settings that control *which* email notifications get sent to you were lost. We're sorry for the inconvenience."
They didn't lose our email addresses, just our preferences about what will trigger email notifications. My settings got turned off, so I went in to the notifications preferences panel and clicked on the radio buttons that turned them back on.


Grab a Netbook and Go!
http://www.NetbookComputers.com

I'm in Finland, got this email, and had turned off all email from Facebook.

Dear Facebook,

No need to fire your employees for this. Simply lose them accidentally!

Got the same message here on Columbus Ohio.

I too had all my email settings set to "off".

I hope facebook doesn't loose my pics!

i got that email and notification too, this morning.

Yes, this just happened to me yesterday, and I too had most of my notifications set to "off." I do not think they actually "lost" any data; I fully believe this was just an excuse to turn my notifications back on and drive me to the site more frequently. It's a shame because I'd just started using Facebook more than ever, but now want to exit the site.

Yeah, same thing here. In fact I got to this blog post when googling to try and figure out how to delete my goddamn facebook account entirely. Anyone know how to do that? I don't just mean "deactivate", I mean "you slimy bastards don't get to count me as a user at all."

After this "data lost" thing happens the first thing that user does (I assume) goest to search the help of the FB. After not finding any thing then they are googling the error and are being directed to this (wonderfull) site.
Therefore i suggest M. David Peterson to open a group on the FB or even a cause and post all the posts from this site... maybe this way FB will take it more seriously our problem.

Ha, I'm not the only one. So not only did I get that email, I went and reset my notifications to none for everything and everything was good. Next day I logged in, I have a message at the top of my home page saying my notifications were lost. Seriously, a company the size of Facebook and stuff is getting lost? I can't wait to get a good laugh when I hear the explanation.

Oddly enough, all of my notification settings were also lost, but I didn't get an email. I feel left out. :)

hi facebook doesnt sends me msgs anymore like when someone comments on my picz or sends me a msg it doesnt sends me a msg to my email and all my setting are on dont know what happen to it can someone help me please?

I STIL HAVNT GOTN MY E-MAIL

Facebook is definitely in the doghouse for me too. We're launching a new online company and a huge part of our online marketing strategy has been through Facebook -- using our facebook group for promotions about the upcoming launch. We're in the middle of huge promotion (that has cost us a ton of money, including ads via Facebook Ads). About 2 1/2 weeks ago, our group just disappeared. Poof! Gone! We've been emailing constantly to their help link and to info@facebook.com and tried phoning their offices, but, they don't have any customer support. This is very frustrating! How can our group just disappear. Try looking for it in the search...it doesn't exist! The group is Wicount Joburg ( http://www.facebook.com/home.php#/group.php?gid=185931708954&ref=ts)

News Topics

Recommended for You

Got a Question?