5 takeaways from Scaling Reader Engagement With Machine Learning

Kat Borgerding (@katborgerding), a volunteer with the ONA Resource Team, compiled these key moments from the ONA20 session on Oct. 15, 2020. To view a recording of the session, register for on-demand access to the ONA20 archive. Session participants included:

Speakers:

Keep an editor in the loop. When building a machine learning tool, the technology needs monitoring and editing from newsroom editors. Get input on what the goals are as you develop the tool.
Monitor for bias. Think broadly about what biases could exist in your model and be aware of areas where you could have a bias you wouldn’t expect.
Consider how the tool might be abused. Think about how it responds to abusive language or hate speech, and think about how the tool might be abused or used to harass editors.
User testing can only tell you so much. Be vigilant and open to updating your tool based on how the audience is actually using it
Be transparent about your use of machine learning. We need to be incredibly transparent with our audiences about how we’re using it and its limitations.

“I think it’s really important when developing a tool like this to really consider all the ways in which it can be used to do harm.” —Amelia Pisapia
“Be transparent about your use of machine learning. We made a very deliberate design decision to include a disclaimer on the module saying it was powered by machine learning, which also linked to a piece of reporting that clearly explained what machine learning was.” — Amelia Pisapia on rd.nytimes.com
“When developing a tool like this to really consider all the ways in which it can be used to do harm.” — Amelia Pisapia
“So for us this [tool] proved our hypothesis that we think it’s possible to offer readers the ease of a search engine and the rigor of journalism, as do we kind of mentioned one of the major benefits of this tool is that we have insight into what our readers want to know.” —Dalit Shalom

Additional Content