Longer online version

Can AI assist journalists?

HOW CAN Artificial Intelligence (AI) actually help journalists, not just put them out of a job? That was the theme of the NUJ London Freelance Branch meeting on 12 November. It was the first LFB meeting in 12 years that was standing room only, including quite a few non-members who'd found out about it and come along. (Some asked about joining the Union.)

We had two speakers, neither of them journalists - both are computer scientists who work with journalists.

Neil Maiden and Toby Abel; Photo: Hazel Dunlop

Neil Maiden (left) and Toby Abel (right)

Neil Maiden, Professor of Digital Creativity at City University, London is co-instigator of Inject - a free to use "discovery" engine funded by the EU - it's currently a prototype, becoming "a product soon." We also heard from Toby Abel, founder - technology of krzana.com. Krzana is a paid-for newsgathering tool, Krzana's clients include BBC (they can't give details) and Reach, formerly Trinity Mirror.

Neil told how Inject trailed first in local newsrooms in Scandinavia, where the journalists they talked to said they now had to write six to ten stories a day. He claims that Inject will "improve productivity so we can actually maintain the workforce", that's how much pressure newsrooms are under these days, in his experience.

Inject has got "pieces" of AI, it's "AI Lite". While it has natural language processing and all the rest of it, it hasn't got "machine learning", it "shouldn't scare you", it's there to help journalists work "quicker and better" - which is vital for newsrooms to stay operating given the economics of current journalism.

Inject is about moving from information search to "information discovery", it's more creative than a search engine. Neil compared it to having to find an idea for a Christmas present for an awkward aunt, you find yourself going onto, say, the John Lewis website just "to get ideas", these search engines don't yet give you ideas, they just find things. Most journalists' news beats can't be reduced to a search term. Inject is a plug in or add-on that you can add to - for example - Google Docs or a content management system, it appears as a side bar that helps you come up with new angles and ideas.

Neil gave an example of three search terms from a then recent story - "Democrats" "House of Representatives" "majority" - and ran all these terms through Inject. The software lists hundreds of thousands of recent news stories from news sites' RSS feeds, from 300 public news outlets globally, translated from six languages.

There are various "tools" with icons that you click on in the Inject sidebar - currently they're called "backing and evidence", "causal", "individuals" "quirky and satire" - the latter is linked to the VJ Movement's Amsterdam-based global community of "visual journalists" including political cartoonists. There are links that lead you to information on how to buy licences for these cartoons.

Inject were at the time of the meeting working on new tools such as "Ramifications," "Twittersphere" and "visualisation" - these should be out now.

Neil Maiden; Photo: Hazel Dunlop

Neil Maiden demonstrates the Inject tool

The news sources that pop up on an Inject search are from a deliberately random list, if you run it again it gives you different results - this is all part of the "inspirational" bit.

Inject picks out organisations, people and locations. It gives you prompts (the "creativity tool") for example, "Who is Richard Cordray?" He is, it quickly turns out, a random Democrat Congresional candidate in the recent mid-term elections, with an interesting background in Federal consumer protection work. He may be an interesting man to watch, making him a possible angle for a story on the recent mid-term elections, a less obvious and therefore more original angle that other news outlets are less likely to have thought of.

Another prompt brought up Pakistan PM Imran Khan and his less than obvious links to the recent mid-term elections in the US.

You can record the references to the stories with the angles that are thrown up as you go, or bring it all up as a word cloud.

Inject is said by its users to be most successful if used for two or three minutes at a time, then you drill down into Google (other search engines are available: see, for example, here and here.) for the detail. It's for the first moments of a story, the first five to ten per cent of the writing, when you have only a topic and are finding an angle for it, that it’s used for.

There's a "control" at the bottom of the Inject sidebar that allows you to tweak the sources that come up, you can tell it how "strict" you want to be with your choice of sources, how far back you go in terms of how old the story is, which original languages the sources are in, and so on. A comparison of news stories written in different languages shows different languages often bring different angles.

Toby Abel; Photo: Hazel Dunlop

Toby Abel talks us through Krzana

Tony and his co-founders started on Krzana four years ago - the first thing they noticed is that journalists are "really, really stretched" these days - they have to make decisions about which bits are vital journalism and "which bits are just the tasks that they need to do because they need to get done... that's what AI is good at doing" - the repetitive tasks.

Talking to journalists, Tony found they spent way too much time doing stuff like looking for trending stories on Tweetdeck and so on. (Tweetdeck is a tool for managing Twitter accounts, m’lud.) Journalists were "finding it too expensive to do all of their newsgathering in person". A lot of their time was spent on keyword searches and trending searches on social media. According to Tony, AI is really good at the bits of the slog of newsgathering that we have to do but don't really regard as journalism.

Keyword searches don't express your beat, what they throw up is mostly "noise". Trending searches for "spikes in activity" on social media are flawed as well - mostly you're too late, they're already talking about it, while there's the "more ideological issue... you're being told what to write by what people are already talking about". Trending search tools encourage people to "just be the loudest voice in the echo chamber". Tools like Krzana are designed to help journalists to "take back control of the narrative."

It's also hard to know whether social media is actually happening locally, 90 per cent of social media output doesn't have a location identifier attached to it. While Google tells you what the top answer is, Krzana allows you to decide yourself what sort of sources you rely on. Krzana currently specialise in local journalism and political stories - particularly elections worldwide.

If you describe you beat, via a search term such as, for example, "violence near polling stations in US" - Krzana will bring up real-time results from blogs, police websites, real-time Wikipedia edits and so on. It provides hyperlocal maps, where you can highlight, for example, particular churches, schools and businesses and show what's being said about them, every time someone talks about them online. There is disambiguation software that works out from context which "Main Street" (in which town) is being talked about.

All this, says Tony, allows journalists "to take these simple tasks... automate those" so we have time to be able to do the stuff we are actually trained to do. While Krzana charge news organisations around £175 per journalist per month for a subscription for a major news group, they would like to talk with the NUJ about coming up with a package for freelances.

Krzana does employ machine learning, it "underpins" everything in the software. How can we avoid just creating a machine that through learning how society is doing things just picks up "its prejudices," asked Freelance editor Mike Holderness. Tony replied that Krzana’s software still allows journalists to think critically about the information that is before them. He says Krzana is not telling us, this is newsworthy, you should go and write about it, but allowing journalists to themselves ask, "What is newsworthy?"

  • Article updated 21 December: we are now spelling Toby Abel's name right