Facebook moderators break NDAs to expose working conditions (opens in new tab)

This problem applies to other forms of sites and content as well. The app store gets hundreds of submissions in the game category per day. Hosting and distributing that content is easy. Only a tiny fraction of those games are going to be played and rated enough times to show up in a recommendation engine. The bulk of the incoming content stream isn't being matched to interested people at all (sorting by interesting is the same curation issue as filtering by offensive).

Literally every content platform is going to have some problem similar to Facebook. We can blame Facebook, but the reality is no one has a good solution. Not even me.

Maybe the best solution is for the moderaters to unionize.

anecdotal, friend who streams on twitch also mods for a few streamers and they even have issues when a stream is in subscriber mode only. simply because anonymity and distance from those being affected empower people to do bad things

edit : I am really surprised there is/are not companies which would spring up to provide these services seeing how most of these activities are required by law.

however before bemoaning what they are paid, just go look at your local 911 operators who are government employees. just because we think it should be paid more doesn't mean others do.

I think for most people that I'd want to be moderating content (e.g. not sociopaths) making them richer isn't going to do anything to deal with the real reason this is worse than a call centre - the content itself. I'd rather see facebook put more effort into protecting moderators from the content they're viewing. I realise this is a non-trivial problem, but here's a few ideas of things they could do which may help:

* Once a video is confirmed bad, fingerprint it and block future uploads.

* Provide colour-inversion buttons to help reduce visual impact

* Rotate people between areas, so they aren't looking at the same class of bad content constantly

* Use ML training to detect which content is likely to be the most severe and for this content, reduce minimum viewing interval to ~ 5 seconds, and ensure that a single individual doesn't get too many of these.

* Flag accounts who post bad content too often, and shadow-ban them so their posts are not visible by anyone else (this could go in stages, starting off by blocking re-shares of their posts or something)

Also in terms of the reputation of Facebook: any issue with moderation failures and Facebook can easily says that it's the contractor's fault

I want to believe the content moderators only end up seeing the 1% of content that is dubious, or brand new content (not published and hashed earlier). I don't understand in that regard why the one video mentioned in the article (about the iguana) wasn't marked as "ok" in the systems, or marked as "on hold for law enforcement" so the moderators didn't have to see it again.

They are passively aware but mostly don't care. The engineers who express frustration are in the vast minority. Most engineers are just super happy about their own situations, very focused on PSCs, and try to keep to their own stuff. Any internal frustrations are channelled towards appropriate feedback channels where the anger is watered down.

Their profit is not proof they are valuable, because they are willfully abdicating responsibility for the damage they cause, and, further, involving themselves in government debates over their own regulation.

Advertising businesses are inherently suspect, and when one is addictive and potentially enabling election fraud etc, we need to be extra careful in declaring its value.

Anyone participating in the debate should have to disclose if they are a daily user or if their company advertises on any social network. (I'm clean)

Edit: my opinion of course is that FB, Insta, etc (any advertising product with push notifications and/or infinite scroll) ought to be illegal. In 50 or 100 years we'll all recognize that social networks are like internet cigarettes.

Why do you think they don’t get benefits? Facebook contracts out to another company. The company they contract with still is probably providing some benefits

It is a very politically correct statement, which, as it usually happens with politically correct statements, really says nothing. I mean, what exactly do you propose they should do about it?

I am the last person to defend Facebook, but, seriously, they are running a business. That is, essentially, finding a way to do/own/produce some stuff while spending less on it than you receive for it. And cost management is all about supply and demand.

After all, it's not like Facebook wants to pay engineers $150k/year and for them to be happy. It needs good engineers and the only way to get them is to keep them happy, or else they stop coming. Engineers are not really special, they are not even exactly more necessary than cleaners, and loaders, and content moderators. The only difference is, demand for them is higher than supply.

And that would be tragic in a sense, if everyone would be randomly assigned a role and if you draw an "engineer" you are lucky, but if you draw a "content moderator" you are not. Or if we were talking about some factory in some giant village in Vietnam, where no matter how smart you are and no matter what you do, you'll be working much harder job in conditions far, far worse than described for $500/year and you really don't have any choice.

But we are not. No, this is Florida, $30K/year and these are people choosing the job of content moderator over a job of an engineer, either because they don't want to or cannot do otherwise. They while many of them may have stories that can be seen as tragic (because who hasn't, really?), they are no more victims than anyone else born into living a life on the Earth. And thus, hardly entitled to anything.

Let me be clear: I really hate Facebook and I find companies with a modus operandi like Congizant's distasteful to say the least. And I can see how Cognizant can be held accountable for giving somebody a false hope, luring into doing the job while promising something else. But what exactly do you want for Facebook to do? They want a service as cheap as possible, they get asked a given price, they agree. That's it. It won't save their reputation after an article like this (because, well, public relations...), but it seems just fair.

If you go work at Facebook as a engineer chances are you aren't too bothered by ethical questions

> Cognizant also offers a 24/7 hotline, full healthcare benefits, and other wellness programs.

I bet this can be done with ML and IA instead human power. Yet, FB uses their human cattle to censor conservative voices.

Where are the moderators living ?

Or the opposite. People flock to the internet to see this stuff, the material that isn't on TV or even dedicated commercial websites. They get to view it in the privacy of their homes, a secret between them an their trusted "friends" on facebook. Facebook knows that if it really cracks down, if cops bust down doors over this stuff, that people will go elsewhere to share horrible material. So facebook does a poor job of moderation, one just good enough to avoid new regulation but not so good as to actually make a difference.

Advertisers care about eyeballs. They may claim to not want to be associated with X or Y content, but in reality they would rather be in front of an audience than not.

Of course the real answer is that this sort of site is a bad idea and shouldn't exist. Wide-open signup and public visibility of content, or ease of sharing it with strangers. Bad combo, don't care how much money it's making them (and other, similar sites).

Granted, not everyone does that, someone nibbling at the frontend or sitting deep in some distributed DB generally does not care much about data quality.

[Clarity: Am not a Facebook employee.]

If you're making $30K you certainly don't have any kind of buffer to hold you over between jobs, and don't have skills that are attractive to employers. Even basic things like interviewing are much harder. An engineer or data scientist can pretty much disappear from their desk for an entire day and no one will ask questions, you don't need time off for an interview, and if you do you likely have tons of leave. Someone making 30k most likely has to give notice of vacation many weeks in advance, if they even have leave benefits at all. And because your skills are less in demand your interview success rate is going to be much lower so you'll need even more time to interview.

It's likely very hard for these people to find new jobs even if they hate their current one, they are probably happy just to have income.

These people are literally trading dollars for their mental health. That is the choice they are making.

Why is this necessarily true? Couldn't it be the case that minimum wage were $15 but they still couldn't find any other job?

... then low-skilled candidates would NOT be able to find any job at all, because if low skilled candidate's expertise does not support $15/hour salary, and employers who could, potentially, pay less (e.g. $10/hour) are legally prohibited from employing people at that lower rate -- then there are no other jobs available to that low skilled candidate at all. Which means forced and hopeless unemployment.

That one paragraph about organs was enough to ruin my day, and it was just text. I'm surprised such a "rash of videos" wasn't in the news somewhere.

Some of the people interviewed were complaining primarily about the bad working conditions (and their complaints are valid as far as I care). I would wager these people are not as bothered by the content (though I'm sure they don't like it) as the ones who's primary complaints are about the content. They could probably do the job with less burnout if the rest of the job was made to suck less (i.e. it wasn't a shitty call center style job with all the accompanying baggage).

Edit: Why am I getting down-voted? Can people legitimately not fathom that some people would not be seriously bothered by seeing this content? People post violent content to Facebook. Shock and gore sites exist because some people actively seek out(!!) the kind of content that these moderators are being exposed to. It stands to reason that the subset of the population that at least finds that content not mentally damaging is substantially larger than the group that seeks it out.

In the article, one of the contractors says that Cognizant puts up a "dog-and-pony show" whenever FB executives visit. Again, it's ultimately up to FB to decide how much they want to push past the facade.

Not sure how you get around that.

Even if it could (and parallel posters argue it cannot), there is a meta-problem: "Who is allowed to post violent content?". Violence is contextual, and also legality of posting it is contextual. Same goes for other objectionable materials (nudity, extreme ideologies etc.).

Point in case, if all users were subject to the same criteria, the big names in media would quickly get banned. And/or would enter war path with Facebook, accusing it of heavy-handed censorship against the freedom of the press.

Same for reportage of popular events with any degree of nudity. A news piece about a Femen protest, a Pride event, a World Naked Bike Ride, maybe even the No Pants Subway Ride would cause a ban.

Likewise, anybody documenting & discussing historical events, and anybody documenting & discussing present-day uprising, revolutions, civil wars, persecutions, etc., would quickly get banned, essentially sweeping a lot of abuse under the rug. Probably even discussion & documentation of domestic violence would have this problem.

As a lighter aside, cartoonish violence (video games & movies) could also easily fall prey to automated violence take-downs.

All in all, Facebook really really wants to give certain users (mostly press & historians) broad, sweeping exceptions to the general rules.

Nope. It's just not that easy. Why do you think the biggest companies on the planet struggle so mightily to reliably and un-controversially figure out what content to block.

Facebook doesn't have a blanket policy of removing violent content, for wholly legitimate reasons. Many users in Mexico very strongly want videos of cartel murders to stay on the site, because they feel that they are important reportage of something that the mainstream media is unwilling or unable to report. Many users in Syria very strongly want videos of IS murders to remain on the site, because they want the world to see the suffering that IS has wrought on their country.

If Facebook just delete all violent content, they're robbing the victims of violence of their voice - it doesn't feel like Facebook are protecting users of their platform, it feels like Facebook are complicit in covering up the crimes of drug cartels and terrorists.

It's not enough to identify that a video is violent - they need to know context. Is this particular video in this particular context a cry for help or a celebration of evil? Will removing this video silence the propaganda of a murderer, or will it help to conceal their crimes? Does this video show the death of an unidentifiable person in an ungoverned warzone, or is it vital evidence that urgently needs to be forwarded to the relevant authorities? For now, only well-trained human beings are capable of making that call.

This sort of dilemma points to the fundamental impossibility of moderating Facebook to the satisfaction of everyone. Some users complain about pornography, so Facebook make a policy to take down the porn. Other users complain that these policies are sexist and protest outside Facebook's offices to free the nipple. Facebook end up with several pages of rules that allow their army of moderators to consistently distinguish between pornographic and empowering nipples, but nobody's really happy with the outcome.

Facebook are doing a really crappy job of looking after their moderators, but that issue is fixable if we apply enough pressure. There are many other problems with moderating Facebook that are far less tractable and have far wider consequences.

I guess it depends how reliably it could determine between ordinary screaming (especially with children who scream at just about anything that makes them happy or unhappy), and 'terrible things are happening' screaming. Even most humans would probably struggle to tell the difference just based on a short audio clip of a scream, which is what your algo would have to work with.

No. Already filters fuck up way too much and restrict perfectly legal content.

As usual, the author manages to make the employer responsible for employee's life situation. But aside from an impulse of assigning blame on the closest most powerful actor related to situation, I don't see any reason to see Facebook in any kind of negative light for this. They need a service performed, they found people who're willing to do that service for the money offered, they gave them this money. They didn't create neither the job situation, neither the lack of education, neither the whole life of choices that lead these people toward this point: they aren't even in a any condition to control the labor market.

And speaking of labor market, I really don't think that they would do anyone a favor if they pay significantly higher than the market going rate. In my experience, such an experiment creates a very unhealthy office politics: people realize that they won't get this kind of compensation anywhere else, and their concern for not getting fired becomes more influential than personal ethics or professionalism.

The balance between protecting privacy and making abuses public is pretty nuanced and doesn't lend itself to one-bit thinking.

NDAs and non-competes have their uses. It's when they become part of the default boilerplate that everyone signs to get a job that the problems start.

I'm not sure exactly how it would work, and perhaps it wouldn't work in practice, but I imagine it might involve paying the (former) employee a certain sum every month for their continued cooperation, and the employer would reserve the right to unilaterally cancel the arrangement: it would be "discretionary" or whatever. So the employee has a motive to cooperate (unless they're terminally ill ...) but there's nothing to "enforce".

I think Non-competes should be limited to while youre actually working there

I find hard to believe humans are so sensitive that some visual stimulation is going to give them mental illnesses. We evolved in an environment where violence was part of real life and a real threat to our own well being, seeing it on a screen will elicit emotions we may not be used to in modern life, but causing mental illness is a stretch.

This just doesn't work in today's world. If it did, Equifax wouldn't be thriving the way it still does today.

"This, then, is our answer. We have no words to waste on you. When you reach out your vaunted strong hands for our palaces and purpled ease, we will show you what strength is. In roar of shell and shrapnel and in whine of machine-guns will our answer be couched. We will grind you revolutionists down under our heel, and we shall walk upon your faces. The world is ours, we are its lords, and ours it shall remain. As for the host of labor, it has been in the dirt since history began, and I read history aright. And in the dirt it shall remain so long as I and mine and those that come after us have the power. There is the word. It is the king of words--Power. Not God, not Mammon, but Power. Pour it over your tongue till it tingles with it. Power."

[1] - https://en.wikipedia.org/wiki/The_Iron_Heel

I think if FB engineers had to actually interact with the bottom-bucket of the content on a regular basis, Facebook would have stricter rules, and perhaps even prior-restraint filters for image and video uploads.

While putting themselves in the moderator's shoes may prove helpful they aren't comparable in skillsets. If they were given a command line interface for instance most mods would be puzzled. "Produced for self" and "produces for content moderators" are differing needs.

End result: engineers paid $250k, moderators unemployed making $0k.

There should be some disclosure of potential mental health risks in the job description. no? i.e. Screening for this at a later stage in the hiring process would be reasonable (No idea on the legality, however)

My decision to delete my account from this toxic spider web of a platform is affirmed regularly by the frequent revelations of their immoral behavior.

Posting violent, hateful, evil stuff is as old as the internet itself. There will always need to be moderators. The question is how these people are treated.

Do you really think it is _impossible_ for a company as big and valuable as FB to treat these moderators well? Do they not have the money to pay them what they deserve, to give them sick leave, and to offer even basic mental health services?

Imagine if half or even a quarter the money and effort that has gone into their crypto debut had gone into helping build better tools and services for their moderation team.

I'd definitely believe that that is the case, as there's little economic incentive to play the long term game with its uncertain payout if you can just churn through low-wage content moderators instead.

Users of Facebook, but more realistically, employees of Facebook are the ones best situated to pressure Facebook to change that behavior.

in general response to this article there was a thread on twitter with some good links https://twitter.com/max_read/status/1141336189800239107

This moderator labor is (relatively) cheap. It also allows FB to point to a third party if something goes wrong.

Look at their statements in the article. It's easier for them to distance themselves and point to "bad actor" contractors than it is if these were direct FB employees.

The only reasonable explanation I can imagine is that Facebook is doing everything they can to avoid having to implement a "Content ID" system like YouTube. Now why exactly they don't want to do that can only be speculated.

police busting down your door for uploading a video kind of is censorship.

But yeah, I'm now starting to see why more and more people are just wanting to go to the censor and police raid system. A lot of this stuff people are uploading just doesn't belong in a civilized society. What started off as maybe just content on making non violent jokes about blacks or gays has morphed into showing children being disemboweled and videos of little old black or jewish ladies being gunned down in their place of worship. It's just gone too far.

Probably just have to file it under:

"This is why we can't have nice things"

> Under the policy, the [animal cruelty] video was allowed to remain on Facebook. A manager told him that by leaving the video online, authorities would be able to catch the perpetrators. But as the weeks went on, the video continued to reappear in his queue, and Speagle realized that police were unlikely to look into the case.

Once an employee has used their allotment, they receive a write up if they take off again no matter how ill. Too many write ups and you're fired. Managers have no discretion in this process, so they can only mitigate the impact by doing something like bringing the trash can over or buying hospital masks.

Not saying it’s acceptable! But this isn’t a facebook problem specifically.

I also believe this is one of the reasons where Facebook's real name policy comes in - it discourages people from posting the worst of it. Animal brutality could just be kids fooling around on their phone, accidental, but produced child pornography is not, and the people making that shit know really well how they shouldn't put it on sites that require them to use their real names.

Facebook is really broad, so are also online news magazines which are sometimes full of horrible comments (only text of course). Usenet more or less ceased to exist because of that. So in reality this is not a Facebook problem but an online community problem - it just looks like a FB problem because that's the major online community.

I wonder if this could be solved with a 3rd party Facebook integration/startup :-). Anyway, Facebook also has more problems with that, another one being undesired unsolicited contacts which fall into the same category of not so nice stuff about Facebook. People don't see the meat production but the bad smell is really close which is why many leave.

Ban the user and the video, if the same video is posted again, autoban it. If the user is law-enforcent, allow them to see the details of all related content that was banned.

There are neat things called if-statemets that could do that.

People can be awful. So can other organisms (independent of whether they're evil or just immoral). Not facing that fact, at all, isn't obviously best.

https://www.irishtimes.com/culture/tv-radio-web/facebook-s-d...

FB vigorously opposes this.

However, with 1) a staff of 30K++ screeners, 2) who knows how many moderators, etc, 3) many layers of management decisions and policy on what is and is not allowed on the platform and the reasons why/why not, 4) extensive human & algorithmic promotion and demotion of content, they are absolutely actively editing their site. This is arguably even more extensive editing than from any print or media organization.

The only reason FB wants to avoid the "editing" label is so they can edit as they please. I.e., they want to edit for 'engagement' and profit, not for social responsibility.

This cannot remain out of control without consequences, which we are already seeing in society.

There's also the aspect of if we make it search out for distinct frames (like "scene change" type things), there's the possibility that some safe things could become auto-moderated which would be another controversy. Perhaps if they could locate start/stop times for inappropriate elements of the source video and then check for videos containing any of those in a later uploaded video, but then instead of asking human moderators to discern "is this video good or bad" we're now forcing them to get timestamps of when bad elements take place which would force them to really watch the video.

Dunno, I'm of the belief that it's just not an easily solved problem. I hope I'm wrong and they can toss money at the problem to fix it, but I genuinely don't know how they could.

We know that FB has nudity detection systems, due to their past problems with banning breastfeeding discussion groups.

Shutting down FB will just create this problem in the next popular social media platform. This is a problem that needs to be solved, and shutting companies down won't do it.

I have worked for too many startups with inadequate bathroom setups.

It is a big red-flag for me now.

The city of Austin recently passed a mandatory sick leave policy, only to be struck down at the state level after lobbying by employers.

It’s not that companies like Facebook simply fail to protect their people, they’re financially incentivized to undermine their rights. And since money == free speech in the US, it’s perfectly legal for them to do so.

A lot of the bad content is highly desired by criminals and can even be profitable to sell on the dark web (e.g. child abuse).