The study “Attitudes to automated and human written sport journalism” by Sam Duncan, Jessica Kunert from University of Mainz and Adam Karg from Swinburne University examined the differences in perceptions regarding automated and human written sports journalism stories.
Automated content creation in sports journalism allows the writers to cover more events or matches and reduces the time to be spent on formulaic writing of structure. On the flipside, historically sports journalism has been emotive, taking the readers on a journey and creating the perception of heroes and villains – something that AI is currently unequipped to do well.
Sports journalism, on the face, is something where automated content creation has a lot of potential, as it is often filled with data about the event, such as the number of goals and who scored them and when. But, research has shown that the readers rate automated reports as shallow and dull.
Previous research has studied automated sports journalism from the viewpoints of issues such as objectivity, credibility, bias, accuracy, readability, and quality measures. However, emotional responses have been less studied expect as a small part of readability. This study fills the gap be focusing on reader enjoyment.
The design of the study was that of an online survey tool, where two randomized groups of readers were presented an automation-generated or human-written report about a sports event: a match between Melbourne Victory and Melbourne City. The two reports differed in style, length, and formatting.
A pilot survey was conducted before the main study on 30 undergraduate student as the participants. The main study had 266 participants, who were all at least 18 and lived in Melbourne. The number was reduced to 251 after an attention check that removed some participants. The final sample was 62% male and 38% female.
The perceptions of enjoyment and liking differed significantly between the machine written and human written reports: human written reports were rated as more enjoyable, and were rated as enjoyable, entertaining, appealing, exciting, lively, and less boring.
These results support previous research results that finds automated reports as formulaic, shallow, and dull. Enjoyment and liking were differing factors, not arousal or quality. This suggests that the emotive factors were what differed here. This is in line with Duncan’s (2020) research on sports journalism being more affectively impactful than other forms of journalism.
Nevertheless, the automated pieces were considered of high (roughly equal) quality. The author states about this: “The results suggest that while the automated sample was considered clear, coherent, concise, comprehensive, and well-written as the human-written sample, a clear gap remains regarding enjoyment and liking.”.
There were some limitations to the study. First, the media-enjoyment measures here were experimental, and other measures might be more appropriate. Second, the timing of match reports may have influenced the participants’ answers – they were not close to the actual time of the matches.
Third, the automated text-generation tools were of the time the study was made, and they are likely to become more sophisticated in the future. All of the limitations highlight the need for further research on the topic.
The article “Attitudes to automated and human written sport journalism” by Sam Duncan, Jessica Kunert and Adam Karg is in Journalism. (free abstract).
Picture: Football (soccer) training field at Kuantan by Izuddin Helmi Adnan.
License Unpslash.
