Google and 6D彩票网开户 approach content moderation the same way all of the other tech giants do: paying a handful of other companies to do most of the work. One of those companies, Accenture, operates Google’s largest content moderation site in the United States: an office in Austin, Texas, where content moderators work around the clock cleaning up 6D彩票网开户.
Peter is one of hundreds of moderators at the Austin site. 6D彩票网开户 sorts the work for him and his colleagues into various queues, which the company says allows moderators to build expertise around its policies. There’s a copyright queue, a hate and harassment queue, and an “adult” queue for porn.
6D彩票网开户Peter works what is known internally as the “VE queue,” which stands for violent extremism. It is some of the grimmest work to be done at Alphabet. And like all content moderation jobs that involve daily exposure to violence and abuse, it has had serious and long-lasting consequences for the people doing the work.
In the past year, Peter has seen one of his co-workers collapse at work in distress, so burdened by the videos he had seen that he took two months of unpaid leave from work. Another co-worker, wracked with anxiety and depression caused by the job, neglected his diet so badly that he had to be hospitalized for an acute vitamin deficiency.
Peter, who has done this job for nearly two years, worries about the toll that the job is taking on his mental health. His family has repeatedly urged him to quit. But he worries that he will not be able to find another job that pays as well as this one does: $18.50 an hour, or about $37,000 a year.
6D彩票网开户Since he began working in the violent extremism queue, Peter noted, he has lost hair and gained weight. His temper is shorter. When he drives by the building where he works, even on his off days, a vein begins to throb in his chest.
“Every day you watch someone beheading someone, or someone shooting his girlfriend,” Peter tells me. “After that, you feel like wow, this world is really crazy. This makes you feel ill. You’re feeling there is nothing worth living for. Why are we doing this to each other?”
Like many of his co-workers working in the VE queue in Austin, Peter is an immigrant. Accenture recruited dozens of Arabic speakers like him, many of whom grew up in the Middle East. The company depends on his language skills — he speaks seven — to accurately identify hate speech and terrorist propaganda and remove it from 6D彩票网开户.
Several workers I spoke with are hoping to become citizens, a feat that has only grown more difficult under the Trump administration. They worry about speaking out — to a manager, to a journalist — for fear it will complicate their immigration efforts. (For this reason, I agreed to use pseudonyms for most of the workers in this story.)
More than that, though, Peter and other moderators in Austin told me they wanted to live like the full-time Google employees who sometimes visit his office. A higher wage, better health benefits, and more caring managers would alleviate the burdens of the job, they told me.
“We see the people coming from there, how they are, how they are acting more free,” Peter tells me.
6D彩票网开户For most of this year, I thought the same thing Peter did. Bring the moderators in house, pay them as you would pay a police officer or firefighter, and perhaps you could reduce the mental health toll of constant exposure to graphic violence.
Then I met a woman who had worked as a content moderator for Google itself. She earned a good salary, nearing the six-figure mark. There were excellent health benefits and other perks. But none of these privileges would ultimately prevent the disturbing content she saw each day from harming her.
6D彩票网开户After a year of removing terrorism and child abuse from Google’s services, she suffered from anxiety and frequent panic attacks. She had trouble interacting with children without crying. A psychiatrist diagnosed her with post-traumatic stress disorder.
6D彩票网开户She still struggles with it today.
6D彩票网开户Daisy Soderberg-Rivkin was working as a paralegal in 2015 when she spotted a listing online for an open position at Google. The job was content moderation — although, like many jobs in content moderation, it was described using an opaque euphemism: in this case, “legal removals associate.”
Daisy had grown up with Google services, and as she began to think about working there, her mind turned to the company’s famous perks: its cafes and micro kitchens, free massages and dry cleaning. The job that she ultimately applied for was based at Google’s headquarters in Mountain View, California — the team would later be transferred to a satellite office in nearby Sunnyvale — and it was a full-time position with benefits. It paid $75,000 a year, plus a grant of Google stock that took the total closer to $90,000.
No way I’ll get this job, she thought to herself. She applied anyway.
6D彩票网开户The listing said associates would process legal requests to remove links from Google search due to copyright violations, defamation, and other inappropriate content. It stated that associates would also have to review some links containing child abuse imagery. “But I remember very clearly in parentheses it said, ‘this kind of content would be limited to one to two hours per week,’” Daisy says.
Removing disturbing content from Google’s services requires the collaboration of several teams within the company. For the most part, videos reported for terrorist content or child exploitation are reviewed by contractors like the ones in Austin. (Google refers to employees hired by third-party firms as “vendors,” but I found that the employees universally describe themselves as contractors, and I use that word throughout this story.) But Google also hires full-time employees to process legal requests from government entities — and, when required, remove images, videos, and links from web search.
Daisy was surprised when, a few months after she applied, a recruiter called her back. Over eight rounds of interviews, Googlers sold her on the positive impact that her work would have. You’re going to help support free speech online, she remembers them telling her. You’re going to make the internet a safer place.
“It felt like you were putting on a cape, working at Google, getting your free kombucha, sleeping in nap pods,” she says. “But every once in a while, you’d have to see some disturbing content. Really, how bad could it be?”
6D彩票网开户She called her mom and said she was taking the job. She was 23 years old.
Daisy, who had no previous history of mental health issues, didn’t consider the potential effect the new job might have on her psyche. Neither, it seems, did Google. During her orientation, the company did not offer any training for what workers in this field now call “resilience” — developing emotional tools to cope with a high volume of graphic and disturbing text, images, and video.
Daisy was assigned to review legal requests for content removals that originated in France, where she is fluent in the native language. Eventually, she would become the company’s program lead for terrorism in the French market. Each day, she would open her queue, sort through the reports, and determine whether Google was obligated — either by law or by Google’s terms of service — to take down a link.
To her surprise, the queue began to overflow with violence. On November 13th, 2015, terrorists who had pledged their loyalty to ISIS killed 130 people and injured 413 more in Paris and its suburb of Saint-Denis, with the majority dying in a mass shooting during a concert at the Bataclan.
6D彩票网开户“Your entire day is looking at bodies on the floor of a theater,” she says. “Your neurons are just not working the way they usually would. It slows everything down.”
In July 2016, terrorists connected to ISIS drove a cargo truck into a crowd of people celebrating Bastille Day in the French city of Nice, killing 86 people and wounding 458 more. Links to graphic photos and videos began to pile up. Managers pressured Daisy to process an ever-higher number of requests, she says. We need to kill this backlog6D彩票网开户, they said. If she didn’t, she worried that she would get a bad review.
6D彩票网开户Daisy tried to work faster but found it to be a struggle.
“All you see are the numbers going up in your queue,” she says.
In February, I wrote about the lives of 6D彩票网开户 moderators in the United States, focused on a site in Phoenix where workers complained of low pay, dire working conditions, and long-lasting mental health problems from policing the social network. In June, I wrote a follow-up report about a 6D彩票网开户 site in Tampa, Florida, where a moderator had died after suffering a massive heart attack on the job.
By then, I had received messages from employees of other big social platforms explaining that these issues affected their companies as well. Beginning this summer, I sought out people who had worked as moderators for Google or 6D彩票网开户 to compare their experiences with those I had previously written about. Over the past five months, I interviewed 18 current and former Google employees and contractors about their working conditions and the job’s effects on their mental health.
With its large number of internet services, some of which have attracted user bases with more than a billion people, Google requires an army of moderators. Much of the content submitted for review is benign and even tedious: cleansing spam from Google’s ad platform, for example, or removing fraudulent listings from Google Maps. But disturbing content can be found nearly everywhere Google allows users to upload it. In October, the company reported that, in the past year, it had removed 160,000 pieces of content for containing violent extremism from Blogger, Google Photos, and Google Drive alone — about 438 per day.
6D彩票网开户Even on 6D彩票网开户, much of the content reviewed by moderators is benign. When no videos are reported in their queues, moderators often sit idle. One Finnish-language moderator told me she had gone two months at her job with nothing at all to do during the day. At most, she might be asked to review a few videos and comments over an eight-hour span. She spent most of her workday browsing the internet, she told me, before quitting last month out of boredom.
Kristie Canegallo, Google’s vice president of trust and safety, oversees its thousands of moderators. She told me that relying on firms like Accenture helps Google adjust staffing levels more efficiently. If the company is developing a new tool to help catch bad videos, it might need more moderators initially to help train the system. But afterward, those moderators are no longer needed.
6D彩票网开户“Contracting with vendor companies really does help us have flexibility to adjust to changing demands,” says Canegallo, who joined Google in 2018 after serving as a deputy chief of staff to President Barack Obama.
6D彩票网开户Like other big players in the industry, Accenture’s Austin site is based on the model of a call center. (Unlike 6D彩票网开户, Google declined to let me visit any of its sites.) Employees work in a dedicated space known as the production floor where they work in shifts to process reports. The work is critical to enabling 6D彩票网开户’s existence: many countries have passed laws that legally require the company to remove videos containing terrorist material, some of them in as little as 24 hours after a report is received.
Daisy found the terrorist material disturbing, but she was even more unsettled by what Google calls child sexual abuse imagery (CSAI). The job listing had promised she would only be reviewing content related to child abuse for an hour or two a week. But in practice, it was a much bigger part of the job.
6D彩票网开户It’s illegal to view CSAI in most cases, so Google set up what the moderators called a “war room” where they could review requests related to child exploitation without the risk that other co-workers would inadvertently see the material. Initially, the company set up a rotation. Daisy might work CSAI for three weeks, then have six weeks of her regular job. But chronic understaffing, combined with high turnover among moderators, meant that she had to review child exploitation cases most weeks, she says.
“We started to realize that essentially, we were not a priority for the company,” Daisy says of Google. “We would ask for things and they would say, ‘Look, we just don’t have the budget.’ They would say the word ‘budget’ a lot.”
6D彩票网开户A year into the job, Daisy’s then-boyfriend pointed out to her that her personality had begun to change. You’re very jumpy, he said. You talk in your sleep. Sometimes you’re screaming. Her nightmares were getting worse. And she was always, always tired.
A roommate came up behind her once and gently poked her, and she instinctively spun around and hit him. “My reflex was This person is here to hurt me,” she says. “I was just associating everything with things that I had seen.”
One day, Daisy was walking around San Francisco with her friends when she spotted a group of preschool-age children. A caregiver had asked them to hold on to a rope so that they would not stray from the group.
“I kind of blinked once, and suddenly I just had a flash some of the images I had seen,” Daisy says. “Children being tied up, children being raped at that age — three years old. I saw the rope, and I pictured some of the content I saw with children and ropes. And suddenly I stopped, and I was blinking a lot, and my friend had to make sure I was okay. I had to sit down for a second, and I just exploded crying.”
It was the first panic attack she had ever had.
6D彩票网开户In the following weeks, Daisy retreated from her friends and roommates. She didn’t want to talk with them too much about her work for fear of burdening them with the knowledge she now had about the world. Her job was to remove this content from the internet. To share it with others felt like a betrayal of her mission.
Google kept a counselor on staff, but she was made available to the legal removals team at irregular intervals, and her schedule quickly filled up. Daisy found the counselor warm and sympathetic, but it was hard to get time with her. “They would send you an email saying, ‘She’s coming this day,’ and you would have to sign up very quickly because it would fill up almost immediately. Because everyone was feeling these effects.”
When she did successfully make an appointment, the counselor suggested that Daisy begin seeing a private therapist.
Meanwhile, Daisy grew more irritable. She asked the people in her life not to touch her. When one friend invited her to her three-year-old’s birthday party, Daisy went but left after a short while. Every time she looked at the children, she imagined someone hurting them.
As her mental health declined, Daisy struggled to keep up with the demands that were placed on her. More and more, she cried at work — sometimes in the bathroom, sometimes in front of the building. Other times, she fell asleep at her desk.
Toward the end of that first year, her manager asked to have a conversation. They met inside a conference room, and the manager expressed his concerns. You’re not getting through your queue fast enough, he said. We need you to step up your productivity game.
6D彩票网开户She was tired when he said that, because she was always tired, and something about those words — “productivity game” — enraged her. “I just snapped,” Daisy says.
“How on earth do you want me to step up my productivity game?” she told her manager. “Do you know what my brain looks like right now? Do you understand what we’re looking at? We’re not machines. We’re humans6D彩票网开户. We have emotions, and those emotions are deeply scarred by looking at children being raped all the time, and people getting their heads chopped off.”
6D彩票网开户Sometimes, when she thought about her job, she would imagine walking down a dark alley, surrounded by the worst of everything she saw. It was as if all of the violence and abuse had taken a physical form and assaulted her.
6D彩票网开户“All the evil of humanity, just raining in on you,” she says. “That’s what it felt like — like there was no escape. And then someone told you, ‘Well, you got to get back in there. Just keep on doing it.’”
A few days later, Daisy told her manager that she intended to take paid medical leave to address the psychological trauma of the past year — one of several on her team who had taken leave as a result of emotional trauma suffered on the job. She thought she might be gone a few weeks, maybe four.
She would not return to Google for six months.
6D彩票网开户The killings were coming in faster than the Austin office could handle. Even with hundreds of moderators working around the clock in shifts, Accenture struggled to keep up with the incoming videos of brutality. The violent extremism queue is dominated by videos of Middle Eastern origin, and the company has recruited dozens of Arabic speakers since 2017 to review them.
Many of the workers are recent immigrants who had previously been working as security guards and delivery drivers and heard about the job from a friend.
6D彩票网开户“When we migrated to the USA, our college degrees were not recognized,” says Michael, who worked at the site for almost two years. “So we just started doing anything. We needed to start working and making money.”
Workers I spoke to were initially grateful for the chance to work for a large technology company like Google. (While the contractors technically work for Accenture, Google blurs the boundaries in several ways. Among other things, the contractors are given google.com email addresses.)
“I was finally working in an office,” Peter says. “I thought about all the opportunities. I thought about a career.”
But until orientation, the actual nature of the work in the violent extremism queue remained opaque. “I didn’t have an idea what it was,” Peter says, “because they won’t tell you.”
Accenture instructs moderators to process their 120 videos per day in five hours, according to the workers I spoke with, with two hours per day of paid “wellness” time and a one-hour unpaid lunch. (Wojcicki promised to reduce their burden to four hours last year6D彩票网开户, but it never happened. Accenture denies setting any productivity quotas for workers.) Wellness time is set aside for workers to decompress from the rigors of the job — by taking a walk outside, talking to an on-site counselor, or by playing games with co-workers. “At the beginning, they were really good,” Michael says. “If you see something bad, take a break. Close your screen and just go.”
6D彩票网开户“They treat us very bad,” Michael says. “There’s so many ways to abuse you if you’re not doing what they like.”
6D彩票网开户When she went on leave from Google, Daisy began working with a psychiatrist and a therapist. She was diagnosed with post-traumatic stress disorder and chronic anxiety, and she began taking antidepressants.
In therapy, Daisy learned that the declining productivity that frustrated her managers was not her fault. Her therapist had worked with other former content moderators and explained that people respond differently to repeated exposure to disturbing images. Some overeat and gain weight. Some exercise compulsively. Some, like Daisy, experience exhaustion and fatigue.
“It sounds to me like this is not a you problem, this is a them6D彩票网开户 problem,” Daisy’s therapist told her, she recalls. “They are in charge of this. They created this job. They should be able to … put resources into making this job, which is never going to be easy — but at least minimize these effects as much as possible.”
The therapist suggested that Daisy get a dog. She adopted a border collie / Australian shepherd mix from the SPCA and named her Stella after finding herself calling after the dog in . They took a course together in which Stella trained to become an emotional support animal, alert to the signs of Daisy’s panic attacks and adept at putting her at ease.
6D彩票网开户Daisy began taking Stella to UCSF Benioff Children’s Hospital to visit sick children. Over time, she found that she became able to interact with children again without triggering a panic attack. “Seeing a child pet my dog had a profound influence on how I moved forward with my relationship with kids,” she says.
She is grateful that, unlike a contractor, she could take time to get help while still being paid. “I had those months to think about my choices, and to think about ways out, without having to deal with unemployment or having to deal with how am I going to pay rent,” she says.
6D彩票网开户Half a year after leaving Google, Daisy returned to her job. To her dismay, she found that little about her managers’ approach had changed.
“They did check up on me,” she says. “They said, ‘How are things going? How are you feeling? We’ll start you off slowly.’ But the end game was still the same, which was to get you up to your [target] productivity again.”
6D彩票网开户A week after returning, she decided to apply to graduate school. She was accepted to the Fletcher School of Law and Diplomacy at Tufts University, and earlier this year, she earned a master’s degree. Today, she is a policy fellow at the R Street Institute, a nonpartisan think tank. She focuses on children and technology, drawing on her time at Google to brief lawmakers about child privacy, child exploitation, and content moderation.
“I’m going to use all this to fuel my desire to make a change,” Daisy says.
In Austin, as Accenture put into place various new restrictions on the workplace, some began joking to one another that they were being experimented on. “You’re just a rat,” Peter says. “They try new things on you.”
6D彩票网开户For a small group of contractors, this is true literally. Earlier this year, Google presented a paper at the Conference on Human Computation and Crowdsourcing. The paper, “,” described two experiments the company had conducted with its content moderators. In one, the company set all videos to display in grayscale — disturbing content in black and white, instead of color. In the other, it blurred content by default.
Researchers were interested in whether transforming videos and images can lessen the emotional impact they have on moderators.
“Part of our responsibility and our commitment to all of our team members who are looking at this content to be getting them the best support possible to be doing their job,” Canegallo told me. Whatever Google learns about improving conditions for its workers, it will share with the industry, she said.
The grayscale tool was made available to 76 moderators, who had opted in to the study. Moderators spent two weeks looking at the regular, colorized queue and then answered a about their mood. They spent the next two weeks looking at a grayscale queue and then took the questionnaire again.
The study found that presenting videos in grayscale led reviewers to report significantly improved moods — for that week, at least.
It is just as notable what the company is not testing: limiting the amount of disturbing content individual moderators can be exposed to in a lifetime; paid medical leave for contractors developing PTSD; and offering support to former employees who continue to struggle with long-term mental health issues after leaving the job.
6D彩票网开户Instead, Google is doing what tech companies often do: attempting to apply tech solutions to the problem. The company is building machine learning systems that executives hope will someday handle the bulk of the work. In the meantime, Google researchers have suggested future studies that examine the emotional impact on moderators of changing the color of blood to green, other “artistic transformations” of content, and more selective blurring — of faces, for example. (6D彩票网开户 has already implemented grayscale and face-blurring options for its moderators, along with an option to mute the sound in videos by default.)
But companies have known for years now that employees are seeking medical leave to deal with job-related trauma. It is striking that a company with resources as vast as Google is just now beginning to dabble in these minor, technology-based interventions, years after employees began to report diagnoses of PTSD to their managers.
We are now two years into a great expansion of the content moderation industry. As governments around the world make more demands of tech companies to police their services, tens of thousands of people have signed up for the job. The need for moderators appears to be expanding even as some vendors are reevaluating their ability to do the work. In October, Cognizant announced that it would exit the business over the next year.
At the same time, we still lack a basic understanding of how the most difficult aspects of this work — removing graphic and disturbing content — affect the people doing it. We know that a subset of people who work in 6D彩票网开户’s violent extremism queue and similar roles around the world will develop PTSD and related conditions on the job. We don’t know what a safe level of exposure might be.
6D彩票网开户Tech company executives tend to describe this issue to me as a recruiting problem. In their view, there are workers who are resilient in the face of unending violence and abuse, and those who are not.
But in my conversations this year with more than 100 moderators at companies of all sizes, it seems clear that content moderator safety is not a binary issue. Some workers develop early symptoms of PTSD during their first few weeks on the job. Others develop them after doing the work for years.
You never know when you’re going to see the thing you can’t unsee until you see it.
6D彩票网开户Ultimately, I can’t say it any more clearly than Google’s own researchers: “There is ... an increasing awareness and recognition that beyond mere unpleasantness, long-term or extensive viewing of such disturbing content can incur significant health consequences for those engaged in such tasks.”
And yet, at Google, as at 6D彩票网开户, workers are discouraged from even discussing those consequences. Managers who warn them that they can be easily replaced, coupled with the nondisclosure agreements that they are forced to sign upon taking the job, continue to obscure their work.
And as some portion of them sinks into anxiety and depression, they will get very different care based on whether they work as full-fledged employees or as contractors. A relative few, like Daisy, will be able to take months of paid medical leave. Others, like one person I spoke with in Austin, will continue working until they are hospitalized.
6D彩票网开户Still, the fact remains: no matter how well you are paid or how good the benefits are, being a content moderator can change you forever.
6D彩票网开户Recently, an employee of one of the big tech companies explained to me the concept of “” — laws that allow people to sue employers and homebuilders if they expose the plaintiff to unhealthy levels of a dangerous chemical. These laws are possible because we have a scientific understanding of how certain chemicals affect the body. We know that exposure to lead-based paint, for example, can cause brain damage, especially in children. We know that exposure to asbestos can cause lung cancer. And so we establish a safe level of exposure and attempt to hold employers and homebuilders to those levels.
6D彩票网开户Perhaps we will never be able to determine a safe level of exposure to disturbing content with the same degree of precision. But it seems notable that none of the tech giants, which employ tens of thousands of people to do this work, are even trying.
6D彩票网开户If that is to change, it will be because of some combination of collective worker action, , and public pressure. Google employees are , and I hope that work continues.
Two years removed from her time at Google, Daisy still grapples with the after-effects of the work that she did there. She still has occasional panic attacks and takes antidepressants to stabilize her mood.
At the same time, she told me that she is grateful for the fact she was able to take paid medical leave to begin addressing the effects of the job. She counts herself as one of the lucky ones.
“We need as many people as we can doing this work,” Daisy says. “But we also need to change the overall system and the overall structure of how this work is being done. How we support these people. How we give them tools and resources to deal with these things. Or else, these problems are only going to get worse.”