BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Google's Vision AI Found Two Hours Of Trump's Tweets In A Week Of Television News

Following
This article is more than 4 years old.

Getty

Twitter has become the defacto communication platform for the world’s heads of state, political leaders, celebrities and journalists to announce major news or issue formal statements. Then-candidate Donald Trump famously embraced Twitter as his primary communications medium during the 2016 campaign and today routinely takes to Twitter to issue statements, proclamations, updates and commentary about the events of the moment. His tweets have become ubiquitous fixtures of the news cycle, raising the question of just how frequently they are cited. Using Google’s Vision AI API to scan a week of television news, tweets from @realDonaldTrump were found onscreen for just over two full hours across the seven stations. Twitter usernames were displayed onscreen for more than 23 hours representing 2.7% of the week’s total airtime. What can we learn about just how important Twitter has become to television news?

Twitter and the news media are often portrayed as opposing forces, but there is actually considerable overlap. In the aftermath of breaking news, Twitter is often used to share news, with first person citizen reporting intermixed with links to mainstream journalism coverage. In turn, mainstream journalists also frequently turn to Twitter, either as a source of first person imagery and video from the scene of a breaking news event or simply to promote their social media handles as a way to engage with them and consume their content.

The use of Twitter-sourced imagery has rapidly increased in online media, slowly displacing the professional photojournalist with crowdsourced photography.

This raises the question of just how often Twitter handles appear on television news. Unlike online news, in which tweets are frequently embedded and thus identifiable from the page HTML code, identifying tweets on television requires performing OCR text recognition on each frame to identify what is seen on the screen. While the contents of a tweet may be read out on air and thus visible in the broadcast’s closed captioning, the information necessary to identify that content as a tweet and its corresponding author is typically exclusively found in the onscreen imagery, requiring OCR.

To explore this further, CNN, MSNBC and Fox News and the morning and evening broadcasts of San Francisco affiliates KGO (ABC), KPIX (CBS), KNTV (NBC) and KQED (PBS) from April 15 to April 22, 2019, totaling 812 hours of television news, were analyzed using Google’s Vision AI image understanding API with all of its features enabled, including OCR text recognition. Each broadcast was converted into a one-frame-per-second sequence of still images that were OCR’d using the Vision AI API.

Across the seven stations, Google’s API recognized more than 371 million characters of onscreen text.

With each video frame converted to a searchable block of ASCII text, a simple keyword search is all that is necessary to flag the appearance of specific Twitter usernames.

In total, just over 2.09 hours of airtime contained President Trump’s Twitter handle “realDonaldTrump” somewhere on the screen across the seven stations during the examined week, accounting for 0.26% of the total airtime. In contrast, President Obama’s Twitter handle appeared for just 4.02 minutes of airtime.

The graph below shows the percentage of each station’s airtime devoted to Trump’s tweets, from a high of 0.33% of CNN’s airtime to just 0.08% of PBS’. All three 24-hour stations leaned heavily on Trump’s tweets in their coverage, with Fox News and MSNBC interestingly referring to them at nearly the same rate, while the networks relied on them far less. One possible explanation is that the compressed time schedule of the morning and evening network broadcasts means they rely more heavily on quick summarization rather than lengthy tweet displays.

Kalev Leetaru

Among the three national stations, all three spent more than half an hour over the course of the week displaying Trump’s tweets, with CNN spending 37.2 minutes showing them, followed by Fox News’ 33.5 minutes and MSNBC’s 33 minutes.

Other than presidential tweets, are there many other tweets and social media usernames that appear on television news?

To examine this in more detail, a simple regular expression was applied to the OCR for each frame, extracting all strings beginning with an “@” symbol and followed by a string matching Twitter’s username requirements. While this undercounts the total number of tweets and username mentions due to OCR error that misses the “@” symbol and overcounts the number of unique usernames due to OCR error, it offers a useful approximation of the density of social media references on television news.

In all, an incredible 2.7% of airtime across the seven stations displayed a social media username onscreen. The graph below shows strong stratification, from a high of 6% for CBS to a low of 0.3% for PBS.

Kalev Leetaru

This graph looks very different from the first one. Upon further inspection the reason appears to be that the local CBS and NBC affiliates display their station’s Twitter handles (@KPIXTV and @NBCBAYAREA, respectively) repeatedly throughout their broadcasts, while Fox News prominently displays several handles including @TuckerCarlson and @foxnews. Many prominent individual journalists and commentators have their Twitter handles displayed at various times when they are speaking or cited and when sourcing imagery and video from the Web, stations also display the handles of their sources.

In all, the OCR identified 3,002 distinct Twitter accounts mentioned onscreen, though many of these are OCR error variants of the same core set of accounts.

Putting this all together, the influence of social media over television news is clearly significant. President Trump’s tweets alone accounted for more than two hours of airtime during the examined week, with social media usernames displayed onscreen for more than 23 hours accounting for 2.7% of the total airtime. Many of these usernames are those of the stations themselves and their most prominent personalities and panelists, illustrating the degree to which even television news pushes its viewers out to social media. Other mentions are citations to the sources of first person social media imagery and video used to illustrate breaking events, while others are actual tweets displayed on the screen.

It is certainly possible that the week examined here was an outlier in terms of its connection to social media. The Notre Dame fire forced television stations to rely heavily on witness reports gathered via social media, while the release of the Mueller report prompted an endless barrage of presidential tweets. Thus, a more ordinary week might not have quite the density of Twitter references, though this week captures the extent to which a news-heavy week can rely on social media.

In the end, it is clear that Twitter and television are not forces in stark opposition to one another – they are symbiotic mediums.

I’d like to thank the Internet Archive and its Television News Archive, especially its Director Roger Macdonald. I’d like to thank Google for the use of its cloud, including its Video AI, Vision AI, Speech-to-Text and Natural Language APIs and their associated teams for their guidance.