How can algorithms help journalists judge the credibility of sources? Richard Fletcher, of the University of Oxford, Steve Schifferes, of City University London, and Neil Thurman, of Ludwig Maximilians University of Munich, examine the tool ‘Truthmeter’, created for evaluating Twitter sources.

Truthmeter is “a tool that automatically scores the journalistic credibility of social media contributors in order to inform overall credibility assessments”, the authors write. It is used real-time alongside normal work, assessing Twitter accounts on a scale from 0 to 9. During the research process, the instrument was evaluated in three stages, and the meter’s scores were compared to those of trainee journalists’.

As a result, the Truthmeter generated scores that were close to scores given by journalists. The evaluation highlighted the importance of relational reliability: credibility could be attributed based on networked connections to other credible contributors, the authors write. If the Twitter user was new, the relatively low number of followers and tweets might provide a misleading indication. Despite its limitations and need for improvements, the instrument seems promising.

The article “Building the ‘Truthmeter’” was published in Convergence and is available on the publisher’s website (free abstract), from City University’s repository (open access) or from Neil Thurman’s website (open access).

Picture: Grandalas in Nepal by Solundir, license CC BY-SA 3.0