Its really hard to contextualize these numbers. What are the comparable rates for other media?
What % of hate speech on Mastodon is taken down? Twitter? YouTube? Discord? Comparing with old-school mob formation technology, what about leaflets handed out? Political rallies?
Meta is an obvious target because they have such a high % of total online speech, and by virtue of this make a potentially impactful single intervention point to make things better. And these articles do make it seem they have room at the margin to improve. But imagine a word with multiple social networks per country - do we think they would be better or worse at policing this stuff (and crucially, the places where it matters are the developing less-regulated countries, often with governments that are at least disinterested in preventing the ethnic conflicts, if not actively promoting them)? I can see hand-wavey arguments in both directions but what I really want is data analyzing the question.
You could get an approximation - For resourced languages though.
Meta launched Hindi classifiers in 2021 and Bengali in 2022, covering 0.6bn and 0.24bn people, respectively. I know how poor the lexicons I have seen in the wild are, and how hard it is to stay ahead of terminology.
Meta is probably doing more to stay ahead than others. However the ARPU for a user in the US vs India is stark.
Platforms aren’t going to release the necessary data for even a rough approximation. Reddit was a potential data source to learn from, but now we have the API rules, so that is also closed off. Lucky us.
Not to mention how supremely unprepared society is to actually understand or discuss the data without making it worse.
Meta has more resources and frankly they regained my trust years ago. However, business is business. Twitter and Reddit have shown that in this current cycle, there are are no consequences for neglecting Trust and Safety, as shown by other companies. T&S teams have been reduced globally. T&S teams are being targeted and vilified.
Next year, there are ~50 elections happening, including the USA and India. The network suits certain narratives - the more insular, fear driven ones. Trust and Safety teams are themselves being exposed.
Its already looking like a perfect storm.
> And this method works. In Myanmar, “reshare depth demotion” reduced “viral inflammatory prevalence” by 25% and cut “photo misinformation” almost in half.
> In a reasonable world, I think Meta would have decided to broaden use of this method and work on refining it to make it even more effective. What they did, though, was decide to roll it back within Myanmar as soon as the upcoming elections were over.
First there would be the delay of the rest of the community understanding what was going on in an unusual language. Second there is the difficulty of deciding what to do and implementing an effective filtering policy. Third since there are multiple instances this can't be done in a consistent way across the network. Fourth they can't stop the Myanmar government from setting up its own Mastodon instances, blocking access to outside instances, etc.
You'd better believe that if Modi keeps going down the road he is going on that India will kick out all western social media companies if they try to block his campaign of dehumanizing non-hindus.
From my perspective there is a lot of harmful talk that goes on social media that falls short of hate speech. In terms of my own feed I have keyword blocks against most Republican primary candidates (not Nikki Haley though) and there are many other topics that I think people are not capable of having a civil and productive conversation on or that I just don't want to be exposed to. A lot of stuff on the slippery slope to "hate speech" is blocked by this and it is OK when I do it because it is my own feed, but you'd better believe there are lots of people who think they have something so important to say that it would be crime to "censor" their shit (for lack of a better word) if the block affected more than just me.
Unfortunately the discussion around A.I. is dominated by positions like: (1) ChatGPT is the God at the end of the universe that Teilhard de Chardin told us about, or (2) It was just fine that Google stole everybody's content for years and years but all of a sudden ChatGPT came out and be outraged all of a sudden at this rip-off.
The truth about it is that machine learning algorithms over text have terrible performance, even the good ones. The performance curves look something like
https://scikit-learn.org/0.23/auto_examples/model_selection/...
and for my RSS reader I am happy to approach an AUC of 0.8, an AUC of > 0.95 for most tasks is getting into science fiction territory. And that's nowhere near being able to clean up hate speech at the rate that it should be cleared.
The short of it is that if you want to get rid of a lot more than "5%" of the outright "hate speech" you will also get rid of a lot of "dislike speech" and "angry speech" and "annoyed speech" as well as some "innocent speech".
To be really frank my filter deletes anything about "cis" and "trans" because it I think maybe 10% of what people say on the subject is outright hateful, 80% of it is emotionally negative, maybe 10% of the time somebody has something neutral or positive to say. My take is that losing a small amount of content I might want to see for being free of a lot of negativity is a bargain I am willing to make. A platform that imposed that kind of judgement on the community could be a much more emotionally positive place than anything you've seen but it is also going to get all sorts of criticism from people for doing so.
> First there would be the delay of the rest of the community understanding what was going on in an unusual language. Second there is the difficulty of deciding what to do and implementing an effective filtering policy. Third since there are multiple instances this can't be done in a consistent way across the network.
This (except the last bit) is missing the point about what Mastodon is for.
Why would "the rest of the community" would even be entitled to decide what to do about it ? Not that there would be a lot of overlap with the Burmese community anyway, so that wouldn't have much of an effect.
We already had an example BTW, with Gab splitting out with Mastodon, but then in a heavily English-speaking context.
What would have made at least a symbolic difference, is that some of these Mastodons would have been operated by the anti-hatred groups described in part I, and they would not have needed to unsuccessfully beg Facebook about doing moderation (with 0-4 moderators from other continents).
But in the end this is mostly spurious discussion : hate speech only effectively gets stomped out when a judge hands out sentences to the haters, but when the law enforcement that should handle them are the ones preparing genocide...
(Not that Facebook is not condamnable for heating it up.)
Certainly some kinds of right-wing hate are suppressed but there is very little pushback about widespread misuse of the word "fascist" to describe your local police department, or people in women's sports who use period trackers to prevent ACL tears, mainstream milquetoast politicians, or just about anyone.
On the other hand a lot of it seems like BS, there are two Mastodon instances that seem to be at war with Hacker News, I think I understand why one of them is mad at HN, I don't know if the second is mad out of sympathy with the first or they have some other grievance.
Many mastodonsters were concerned that Threads would be poorly moderated (might allow posts about how period tracking prevents ACL tears in female athletes I guess) but Threads pretty much dropped everybody's radar.
---
One angle is that a system like this should have "defense in depth", like you try to create a culture where hateful speech is not welcome, where people don't try it because they know they'll get disapproval, where people know they'll get confronted if they call Keir Starmer a fascist, where people know you just don't boost hateful things, etc.
In that sense you might just want to cultivate a lot of positive and neutral stuff and encourage people seeking negativity to go elsewhere. I think this is what Instagram was going for with Threads and it might be one reason why people should be asking "whatever happened to Threads?" but they don't because they've forgotten all about it.
The question, of course, is whether there is a political will to "police hate speech". In reality this already dubiously noble goal is as flexible as the definition of "terrorist organisation".
https://www.reuters.com/article/us-myanmar-rohingya/myanmars...
> "“Though not confirmed, there are indications he went to Pakistan and possibly elsewhere, and that he received practical training in modern guerrilla warfare,” the group said. It noted that Ata Ullah was one of 20 Rohingya from Saudi Arabia leading the group’s operations in Rakhine State. Separately, a committee of 20 senior Rohingya emigres oversees the group, which has headquarters in Mecca, the ICG said."
https://en.wikipedia.org/wiki/Ataullah_abu_Ammar_Jununi
That certainly doesn't justify the response of the Myanmar government, but it helps explain it to some degree - imagine the US political response if, say, US border facilities were attacked by an Iran-based group that killed a dozen US border guards.
Entirely neglecting this background is just bad journalism, and also distorts the role of Meta. It's highly unlikely, given this information, that a rigorous censorship strategy on Meta would have changed the outcome much if at all. A more rational conclusion is that the Myanmar government viewed the Rohingya people as a potential base for Saudi/Pakistani-backed terror groups to launch attacks within Myanmar, and that was the root cause of the expulsion, and that social media-based propaganda efforts were not that important in understanding what happened.
I'm also in favor of free speech over censorship - it's very easy to demonstrate in an open conversation that mindless hatred of others based on their ethnic/religious/racial background is simply the product of ignorance and immaturity.
Clearly it's not; this myth needs to stop being pushed. Everything we have observed from the internet age is that such hatred metastasizes even easier because of the internet and social media, and that almost all discourse is worse.
The whole reason Facebook is a useful tool for hate groups is because it allows them to cultivate a fake justification for hate that goes beyond "mindless". Inventing crimes, doctoring photos, promoting false narratives and conspiracy theories, etc. It radicalizes moderate people by making it seem like the out group is an existential threat to the in group. We don't hate them because of their skin color or religion, we hate them because they're trying to kill us first.
Open dialogue can't fix this because it's much harder to disprove bullshit than to generate it.
It's true there are problems with Facebook. But without Facebook the Tatmadaw would still have laws against Rohingya, still have a policy of driving them out, still would engage in fighting ARSA, still would fund and promote groups like Ma Ba Tha, etc.
The Myanmar (Bamar), Karen, Shan, etc populations were all there before the British drew the borders of 'Burma.' The conflict goes back way farther than WWII. The main difference now is that the Myanmar are winning.
Settlers and natives has nothing to do with it. All the major armed ethnic peoples existed in Myanmar since before WWII. The “settlers” were the British and they left.
At issue here is the PROMOTION of misinformation, lies and genocidal speech (in users' feeds), not the fact that they haven't deleted it.
Demonstrating that fact is a really poor substitute for taking some kind of action against an ongoing genocide.
Kind of like the best way to have avoided the war in Ukraine would have been for the USA to NOT have engaged in a regime change op in 2014, and the best way to avoid the current Libyan disaster would have been for NATO to have NOT engaged in a regime change operation in 2011, and so on.
Not an unexpected decision overall, but it did surprise me how low Facebook can stoop. Like, they didn't even bother to fix their stuff in Myanmar after all these years with Myanmar being the poster child of their hate speech problem!
> In 2022, Global Witness came back for one more look at Meta’s operations in Myanmar, this time with eight examples of real hate speech aimed at the Rohingya—actual posts from the period of the genocide, all taken from the UN Human Rights Council findings I’ve been linking to so frequently in this series. They submitted these real-life examples of hate speech to Meta as Burmese-language Facebook advertisements.
> Meta accepted all eight ads.
(Keep in mind after this, Facebook hired 20,000+ content moderators at great expense who they still employ to monitor for stuff like this).
According to the article, Facebook did make the deliberate decision not to dedicate resources to Myanmar and other non-western countries, so I don't think it's negligence. "Greed" is more appropriate.
> Guy Rosen, Facebook’s VP of Integrity, told Sophie Zhang that the only coordinated fake networks Facebook would take down were the ones that affected the US, Western Europe, and “foreign adversaries.”
They did certainly care about the growth of daily active users even if it meant they expanded to markets where they couldn't responsibly moderate their platform.
A company as big as Facebook is going to have different groups with different objectives that often do not see eye to eye or even know about each other. A company as profitable as Facebook is going to have projects that are motivated by things other than profits.
There is most likely a team at Facebook whose goal it is to get as many users as possible in developing markets. These markets do not generate profits from online ads. Facebook will never disclose its cost per user but in general online ads only make money from affluent users. I have heard that even mid income countries like Mexico or Romania are not profitable for ads. A place where people make an average of $3 a day is wildly unprofitable for Facebook and even the most optimistic development projections will not make them profitable for several decades. The OP is spot on that the amount of money that Facebook will ever receive from Myanmar is negligible.
Based on conversations with people at Facebook I generally get the sense that people there (especially at the top) have a strong sense of mission and a belief that their product makes the world a better place (I disagree with this but that doesn't change the fact). I would guess that these the project tasked with increasing users in developing countries is viewed more as philanthropy than business.
Part II of this series addresses this stat. During the peak genocidal years it seems there were only a handful of mods assigned to Myanmar, and none of them lived in Myanmar. To this day it sounds like there still may not be more than a hundred people moderating Myanmar content. Which is inline with the 20K number: 195 countries * 100 moderators per country = 19.5K mods total
I guess my main idea here is that when I look at the back-of-envelope numbers it's easy to see how even 100 moderators would have trouble keeping up with the content that millions of people produce. So you really do need those automated moderation tools to be effective.
Can someone please continue the thought experiment by estimating how much content each mod can review in a day, then extrapolating up to a week or month, and comparing to an estimate of how much content was actually posted on Facebook in the similar timeframe? I believe Part I or II provides an estimate like that.
Of course Facebook didn't fix anything in Myanmar - the new junta government is both the one running the troll farms and responsible for the previous genocide !
> We proactively detect 99 percent of the hate speech removed from Facebook in Myanmar
If the accusation is correct, they are not saying that they capture 99% of all Myanmar hate speech on Facebook. I remember that the big news agencies reported it that way. Meta is actually saying "of the 5% that we do catch, 99% was caught automatically." The key phrasing is "99% of the hate speech removed".
In other words, this number actually means, "We do almost no non-automated detection of hate speech."
Smart spin, huh...
Do they have good estimates or is this an unknown amount?
> …we’re deleting less than 5% of all of the hate speech posted to Facebook. This is actually an optimistic estimate—previous (and more rigorous) iterations of this estimation exercise have put it closer to 3%, and on V&I [violence and incitement] we’re deleting somewhere around 0.6%…we miss 95% of violating hate speech.
https://facebookpapers.com/wp-content/uploads/2021/11/Hate-s...
The reference to Nuremberg was more for an example of an international trial than for the nazis, although faciliting a smaller scale genocide does somehow seem to have a relationship with the nazis...
Given the havoc they willingly caused to nations worldwide, an international trial for Zuckerberg and the other responsibles does not seem so inconceivable to me...
And that's not considering a lot of other probably less important things they did
Is your argument that Facebook as a whole is specifically engineered to incite tribal violence, and so should be banned on the same basis as radar scanners? Because in that case I think you have an argument to make which isn't completely incoherent.
With that being said, are we really supposed to be holding entities like Meta/Facebook accountable for the actions of governments? You can read through my comments here on HN and I'm sure you'll notice I'm not one to generally have nice or even neutral things to say about megacorp entities like Meta, but I just fail to see how any of this is really on them.
If it wasn't Facebook where misinfo was being spread, it would've been twitter, or mastodon, or whatever other social media platform. And even if there were no social media platforms, disinformation, propaganda and unfortunately even genocides have all been happening for a lot longer than Meta's been a thing, and I seriously doubt that the lack of channels like Facebook would've prevented or even changed anything.
Why are we suddenly deciding that Meta/Twitter/Google/Pick your poison are to be the arbiters and keepers of truth and justice?
They are being held responsible for their claims. For their product decisions. For their choices to ignore their own researchers.
No one is “suddenly” anything. People have been crying out about this for a decade+.
Please read the article. The answers are directly there.
It was them who first adopted most of the dynamics that lead to these things
(That quote comes from the description of the mechanics of the military's coordinated propaganda campaign.)
I would like to share a PSA in the truest sense of that acronym. I suggest building an automatic bullshit indicator in your brain whenever you see any content along the lines of "they're harming our women" from anyone. It may literally be the oldest trick in the propaganda book. Both sides used it extensively in WWI [1]. It pops up a lot in the history of American slavery / racism as the idea that often directly sparked lynchings. I'm sure you can think up lots of other examples. Sorry if this is already obvious but over the years I keep seeing more and more examples of it and IMO this tactic doesn't seem to be as well-known as the technique of depicting the victims as an infestation of animal pests.
[1] Example of USA propaganda from WWI https://firstamendment.mtsu.edu/article/committee-on-public-...
Calling out the simple statistic slight of hand would have been a sure fire way of establishing control and the impression of hard on big tech they wanted. It was delivered on a platter.
Instead of that we got “Senator we run ads” and stupid gotcha games trying to prove FB was biased against conservatives when it was painfully clear that Joel Kaplan had fully taken over the bias department in favor of his friends (preventing misinformation action on the news surface because, shocker, it came 90% from conservative staples like Drudge report)
The articles present a lot of evidence to support the notion that Facebook was well aware of clear-cut incitement to violence years before its escalation into large-scale genocide in late 2016. It isn't spelled out explicitly, but I think there are two things that make Facebook's initial apparent non-response easier to understand as incompetence rather than evil:
* In the aftermath of the Arab spring, technology and social networks enjoyed a honeymoon in public perception that is hard to remember over a decade later. People, including (especially?) people in tech, could genuinely believe that making it easier for people to talk to each other was basically good, full stop. I'd suggest that this honeymoon only really ended after the 2016 US presidential election, which caused a lot of soul searching about information and echo chambers, and got even worse after the Cambridge Analytica scandal [3, this is a graph image from NBC news]
* As part I of the article notes, "[t]he information landscape in Myanmar is so unstable that accounts of any given incident conflict, often in major ways". This dynamic seems to appear in a lot of places where well-off westerners interact with much poorer and more chaotic countries (Rwanda is a clear parallel). Part I points out that the "Burmese government" itself seems to make statements both fanning anger against the Rohingya and then trying to walk them back when it boils over into disorder
So I think it was possible to be a basically well-intentioned (naive) Facebook employee in the leadup to 2016 and hear of strange, almost genocidal sounding posts in another language in a place you don't understand and still not think "this is a break-glass moment and I need to escalate and make a new path inside my corporation for this to be taken seriously". In hindsight, this was wrong, but I think we're forgetting that what happened next didn't seem possible, because we still thought of the internet as an exceptional new dawn in mass communication. A lot is necessary for people to process something as a novel, serious emergency.
However, it's hard to maintain this judgment over the ensuing years, as senior figures at Facebook are repeatedly presented with evidence that people are deliberately distributing misinformation (e.g., making fake accounts, and other clear-cut evidence that doesn't require a fact checker and research) and mostly wave it away unless it's big enough to provoke media and investor consternation. By the end of the 2010s, I think Facebook was well past the "this sort of thing wasn't something we could imagine" stage of a crisis, and into a less defensible position along the lines of "this is reality, it cannot really be moderated, but we can pretend to moderate".
One last point re: moderation, I have no idea how Meta hires its Burmese content moderators, but it must be tricky, since as far as I can tell most people in Burma are anti-Rohingya. I would like to be wrong about this, but what should be factually trustworthy sources like the Brookings institution write things like:
> Most of the Myanmar population, especially the Buddhist majority, feels that the Rohingya don’t belong in their country ... [t]he straightforward solution would be to help the Rohingya return to their homes in Myanmar and live in peace and freedom. Sadly, this solution looks impossible in the near term because of the nationalist sentiment of the Buddhist majority
[1] https://erinkissane.com/meta-in-myanmar-part-i-the-setup
[2] https://erinkissane.com/meta-in-myanmar-part-ii-the-crisis
[3] https://media-cldnry.s-nbcnews.com/image/upload/t_fit-560w,f...
[4] https://www.brookings.edu/articles/on-the-ground-in-myanmar-...
