Pro-eating disorder communities are thriving on Instagram despite content moderation

Pretending a problem isn't there doesn't tend to work

Being online is a great way to join communities of people with a similar mindset to you, with similar interests and ways of looking at the world. But this isn’t always a positive and constructive thing. Pro-eating disorder communities exist across the internet, you’ve probably come across their hashtags or images before; they’re online communities which encourage the adoption and maintenance of disordered eating habits, portraying them as a lifestyle choice rather than a health threat.

Instagram’s reaction to these communities in 2012 was to ban the search terms they used. This meant that anyone on the platform searching for terms like ‘thinspiration’, ‘proana’ (pro-anorexia) or ‘thigh gap’ would find no results. They also implemented content advisory warnings for search terms like ‘ana’ which still allowed users to access the content but also offered to direct them to Instagram’s help page for eating disorders. Recent research from the School of Interactive Comupting at the Georgia Insitute of Technology has found, however, that Instagram’s content moderation approach hasn’t been at all effective in stopping the growth of these communities on their platform or others, largely thanks to use of lexical variation.

Instagram banned the use of 17 pro-eating disorder terms, but according to the researchers “almost 40 variants emerged corresponding to each moderated tag.” For example, ‘Thinspo’ became ‘thinspooo’ or ‘thynsporation’. Not only that, they found that user engagement through likes and comments was 15-30% higher on these variant tags than it was on the originals. Tags which had been banned outright saw a sharp drop in use, on average around a 52% drop, but those that were still open with content advisory warnings actually increased by an average of 22%. After this initial period, variant tags began to appear and though the researchers found that many of the communities adopting variant tags were smaller “certain lexical variations reached dramatic sizes (2 to 40 times larger) relative to the initial tag” and that in fact, “lexical variants of tags with content advisories grew by 22% following Instagram’s moderation of pro-ED content.” They also found that these communities tended to feature even more triggering content.

From their findings, the researchers suggested that not only is Instagram’s approach to content moderation not controlling pro-eating disorder content on the platform, it “might in fact be amplifying the destructive power of pro-ED posts.”

As an alternative to moderation where content is simply banned, the researchers suggest that platforms like Instagram might try a more nuanced approach by issuing public service announcements with pointers to support websites or hotline numbers. The hope is that this might “create less incentive to migrate to different tags” and make reaching these communities with pro-recovery content easier. They also suggest alterations to recommendation algorithms, which would result in pro-recovery content being suggested in the ‘explore’ section of Instagram for those who have previously searched for pro-eating disorder content.

The researchers note that though these platforms have no obligation to intervene in these communities, they say that “eating disorders are unique in that body perception and self-esteem are negatively impacted by social comparison enabled by social platforms” and that “there is a collective opportunity for social media designers and researchers to rethink the affordances around discoverability and sharing of pro-ED content, not only for the dissipation of such behaviours but also to promote recovery and treatment of eating disorders.”

via Wired