RHETORIC

The RHETORiC project aimed to design and evaluate elements of an interface between commenters, journalists and moderators that could reduce the influence of cognitive bias and aid discussion and thus improve the tone, level and social connectedness of online debate. The overall goal was to promote civil participation in online debates by allowing journalists to better empathize with the audience, encouraging people to post high-quality comments and featuring audience feedback to specific news stories.

The outcomes

To overcome the drawbacks of traditional news comment sections and human content moderation, we gathered new knowledge on how to promote civil participation in digital journalism. Based on this insight, we developed and evaluated an interface that includes design interventions to increase the added value of audience participation for both readers and editors supported by semi-automatic moderation of textual and visual comments.

1. Audience conversation and moderation tools

The project developed an audience conversation tool to support people in formulating their opinions and arguments and help them learn how to contribute constructively to online discussions. The tool guides commenters through the commenting process based on conversational prompts, stance taking, live feedback and clustering of similar comments.

In addition, a new moderation dashboard for news publishers and journalists aggregates insights and provides semi-automatic moderation of comments delivered through the audience conversation tool. The dashboard allows journalists to see relevant opinion clusters, comment toxicity and argument levels for a given news topic. Journalists can also use the dashboard to moderate comments based on an automatic pre-moderation step.

2. AI-based text and image analysis algorithms

To enable the above commenter guidance and semi-automated moderation in real time, the project developed novel text and image analysis algorithms based on artificial intelligence (AI). Existing tools for analyzing sentiment and detecting profanity in text were augmented with new tools to identify stance, level of disagreement and the type and target of any toxicity. Furthermore, we investigated and tested the ability to detect toxic memes through image analysis.

3. Real-world evaluation of the interface

These elements were evaluated in a first large-scale test on the website of Belgian newspaper Het Nieuwsblad. The results were promising, with the quality of comments written using the RHETORiC tool significantly higher than those on social media for the same topic. Across all topics, comments in the RHETORiC tool were significantly better argued, more skillfully written, less toxic and more positively worded than comments submitted via Het Nieuwsblad’s Facebook page. Furthermore, more than 8 out of 10 respondents reported (very) positive feelings after using the RHETORiC tool. Over 90% found participating in a debate using the RHETORiC tools worthwhile, including a part of the “silent majority” that does not typically comment on social media.