CrossOver podcast – episode 4 “Algorithm watchers”

This module focuses on how new information technology tools, such as CrossOver’s dashboard, are helping to shed light on the role and impact of algorithms on information and disinformation. The podcast gives the floor to those who work at CrossOver at the technical and pedagogical levels to shed light on the impact of algorithms on information and disinformation, in particular through its dashboard. This role is illustrated through an investigation of the role of predictive algorithms in Google’s “autocomplete” search.

How do developers work with journalists, fact-checkers and MIL experts? What are the benefits and challenges of this complex approach?  How can technical tools such as the one developed in this project be useful to all citizens – information professionals as well as ordinary citizens – to avoid disinformation?

Podcast guests: 

Amaury Lesplingart and Nelly Pailleux (Check First), Divina Frau-Meigs (Savoir Devenir)

00:00:05:17 – 00:00:07:17

Sébastien

You are listening to the Crossover podcast.

00:00:07:17 – 00:00:24:13

Sébastien

Crossover is a European project that explores the backstage  of algorithms through its dashboard, a tool for monitoring the algorithms used in social media, search engines and on certain platforms.

00:00:29:03 – 00:00:47:00

Sébastien

Algorithms influence our relationships, our consumption, our culture and especially the way we inform ourselves and perceive the world. But do we really know what they are? How do they work? What economic or ideological logic underlies them?

00:00:47:11 – 00:01:06:07

Sébastien

And how can we try to master them rather than suffer from them? Understanding algorithms, being able to analyze them critically, grasping them and learning to protect oneself against them, are the objectives of algo-literacy, a field explored by the Crossover project in its informational dimension.

00:01:06:23 – 00:01:26:08

Sébastien

For this episode titled Algorithm Watchers, we talk about all the people who work at Crossover at the technical level to shed light on the impact of algorithms on information and disinformation. How do these developers work with journalists, fact-checkers and media and information literacy experts?

00:01:27:00 – 00:01:50:09

Sébastien

What are the benefits and challenges of this complex approach? How can it be useful to the press, to fact-checkers, to teachers, to citizens? This is what we will see with our three accomplices and algorithm watchers: Nelly PAILLEUX and Amaury LESPLINGART from Check First, and Divina FRAU MEIGS from Savoir Devenir.

00:01:50:09 – 00:02:06:11

Divina

The Crossover project has been going strong since its creation in December 2021. It has relied on an innovative tool, the Dashboard, which observes and analyzes algorithms to fight against disinformation, facilitate fact-checking and train citizens.

00:02:06:11 – 00:02:30:02

Divina

By bringing together developers, journalists, and experts in media and information literacy, it has concretely shown the influence of algorithms on our consumption of news and in particular on social media. investigations and studies were conducted, journalists were trained and a resource kit for educators to tackle the topic of algo-literacy was produced.

00:02:30:20 – 00:02:43:03

Divina

All this thanks to the Dashboard, of course. Amaury LESPLINGART, who worked on the creation and development of this tool is with us to answer our questions about this new algorithmic monitoring tool.

00:02:43:03 – 00:02:43:21

Divina

Hello Amaury.

00:02:43:21 – 00:02:44:16

Amaury

Hello Divina.

00:02:45:00 – 00:02:52:07

Divina

You are co-founder of Check First which develops tools to fight against disinformation. Could you tell us a bit about your job?

00:02:52:07 – 00:02:55:19

Amaury

I have been a developer for 20 years. I have diverse and varied experiences in many fields.

00:02:56:06 – 00:03:07:00

Amaury

Since I was very young, I’ve been trying to understand how things work, especially everything related to the Internet field, in the broadest sense of the term. I would define my job as someone who tries to understand technically the world of today and tomorrow.

00:03:07:00 – 00:03:11:04

Divina

Very good. And how did you come up with the idea of creating this rather special tool?

00:03:11:04 – 00:03:19:16

Amaury

Algorithms are embedded in everyone’s everyday life and we don’t even realize it anymore. No one thinks that there is an algorithm that actually predicts the question they are going to ask Google anymore.

00:03:20:06 – 00:03:33:15

Amaury

There are many initiatives that aim to open up algorithms, the so-called black boxes, but we found that there were few initiatives that were interested in the end users. We conceived Crossover as a project that wants to analyze the impact of algorithms on the life of Mr. and Mrs. Everyone.

00:03:33:21 – 00:03:35:06

Divina

So the basic idea is what?

00:03:35:06 – 00:03:46:00

Amaury

The basic idea is simple: if I do this, what happens? We’ve created systems that allow us to replicate exactly that. If I type “Donbass” on Google, what happens? If I type “Russia” on YouTube, what happens?

00:03:46:12 – 00:03:51:07

Amaury

This information is then compiled in a public dashboard and analyzed by our journalist partners.

00:03:51:07 – 00:03:58:13

Divina

So, it seems to me that there are other dashboards that allow you to see trends online. How is Crossover’s different?

00:03:58:22 – 00:04:07:07

Amaury

So indeed, there are many ways to analyze online algorithms and trends. We tried to bring a touch of originality to the Crossover project with a double approach.

00:04:07:07 – 00:04:23:23

Amaury

First of all, we use tools that are made available by the platforms, their APIs. That is to say, we have developed a system that will query the platforms to retrieve the data. On the other hand, we have developed an autonomous system that has been placed in the homes of people like you, Divina, or like me or like Nelly.

00:04:24:17 – 00:04:39:04

Amaury

It is a kind of robot that will go and make requests on the platforms to query them. This double approach allows us not to be dependent on the platforms to provide us with data, but also to compare the data they officially provide us with the concrete experience of the users.

00:04:39:16 – 00:04:46:12

Amaury

Crossover’s dashboards are not tools for measuring buzz on the Internet. They analyze the behavior of the platforms’ algorithms over the long term.

00:04:46:12 – 00:04:55:03

Divina

And that’s it. Let’s hear it. Do you consider the Crossover dashboard to be a fact-checking tool, a tool for fact-checking?

00:04:55:08 – 00:05:05:14

Amaury

They are rather tools for fact-checking. We have never pretended to take the place of fact-checkers, who are probably the best fact-checking tools, but we can probably help them a little in their work.

00:05:06:04 – 00:05:14:17

Amaury

Thanks to the Crossover dashboards, it is possible to understand what users are facing. In this case, Belgian users. It’s important for fact-checkers to understand this.

00:05:14:17 – 00:05:16:02

Divina

Why is this important?

00:05:16:02 – 00:05:22:15

Amaury

This was demonstrated with our investigation of Google search, which systematically offers users the Insider recommendation when they type Donbass.

00:05:23:00 – 00:05:34:15

Amaury

Donbass Insider is a media that relays Russian communication. It is therefore important to warn users that the direction in which it pushes the algorithms is in fact risky. It is almost a pre-bunk work that fits perfectly in the role of fact-checking.

00:05:34:22 – 00:05:41:18

Divina

And so this dashboard, was it complicated to develop? How did it evolve over time?

00:05:41:18 – 00:05:47:11

Amaury

It’s like asking a baker if making bread is complicated. He will tell you that it is not, yet I am unable to make my own bread.

00:05:48:16 – 00:06:09:23

Amaury

On a more serious note, there were some challenges in development, but nothing insurmountable. The biggest challenge, I would say, was the self-imposed constraints with the dual collection of information. We had to imagine systems that collect both data through the APIs of the platforms, but also data through our robots that we put in people’s homes.

00:06:09:23 – 00:06:24:12

Amaury

It hasn’t been easy. Over time, we’ve evolved the dashboards, added new platforms. Today, Crossover monitors eight different platforms, compared to only three at the beginning of the project. We hope to continue to add more algorithms in the future and make sure this doesn’t cause harm to the company.

00:06:24:23 – 00:06:25:16

Divina

Hello Nelly.

00:06:25:16 – 00:06:26:14

Nelly

Hello Divina.

00:06:26:14 – 00:06:40:23

Divina

Nelly PAILLEUX. You are a journalist by training and also co-founder of Check First. What did you learn during this project by going to look under the hood of the recommendation algorithms of YouTube, Google, etc? Do you really think it plays an important role in disinformation?

00:06:41:09 – 00:07:09:22

Nelly

So, the investigations conducted by Check First, EU DisinfoLab and Apache in the framework of the Crossover project give some answers to this question. For example, in the investigation “Are YouTube’s algorithms addicted to state media?” We saw that for a period of time, YouTube recommended videos from Russia Today France, the French-speaking branch of the state media, Russia Today, to Belgian users who typed the word “Russia” in the YouTube search bar.

00:07:09:22 – 00:07:26:13

Nelly

The interesting phenomenon is that after the banning of Russia Today by European decision, a new actor, another state media, seems to have filled the gap left by its Russian counterpart and this other actor is CGTN, the state channel of the Chinese Communist Party.

00:07:26:24 – 00:07:59:04

Nelly

In another investigation, it was found that Belgian Google users, Google which still represents 86% of the searches done on a desktop computer in 2021; these users who typed the keyword Donbass in the search bar would have their query completed by the word insider. The phrase Donbass Insider links to a pro-Kremlin media outlet that was created in 2016 by Christelle Néant and is accused of, among other things, spreading lies about the war in Ukraine.

00:08:00:03 – 00:08:03:12

Divina

Nelly, so sometimes there is a gap between what happens online and in real life.

00:08:03:24 – 00:08:23:06

Nelly

We suspected that the virtual didn’t always reflect the real, and here you’re referring, Divina, to episode 1 of the Crossover podcast on clicktivists and the Freedom Convoy. In fact, what the dashboard or dashboard allowed us to do was compare other metrics.

00:08:23:16 – 00:08:45:17

Nelly

When we started the Crossover project, the first thing we had to do was to collect data to put them in this dashboard. As Amaury explained earlier, we started by collecting data from the official APIs of the platforms. APIs are applications that allow you to query databases via a programmatic interface provided by the platforms.

00:08:45:21 – 00:08:46:23

Divina

Thank you for this clarification.

00:08:47:07 – 00:09:07:09

Nelly

The problem is that some platforms do not have an API. Moreover, the API is given directly by the platforms. And as we have a project in which we are many journalists, what we wanted to do is to cross our sources to be sure that the information returned by the API of the platforms is correct.

00:09:07:09 – 00:09:27:19

Nelly

To observe the algorithms of platforms that did not have an API and to cross-check our sources, we set up a network of mini computers on which, as Amaury explained, a system that simulates the behavior of a user on these eight platforms is running, and these mini computers are each distributed in a province of Belgium connected to volunteers.

00:09:28:14 – 00:09:44:15

Nelly

This way, it allows us to do three things. The first is to visualize what Belgian users really see in front of their screens. The second is to see if there were differences in the trends between the different provinces of Belgium.

00:09:45:04 – 00:10:01:16

Nelly

And the third is to have a way to compare them. What users see and the official API data. It’s a way to see the gap between what the platform tells us is happening and what’s actually happening in front of the users’ eyes.

00:10:01:21 – 00:10:23:01

Divina

Well, that’s very clear. Thank you for the clarification, which shows how complex it is. The dashboard can also test the temperature of public opinion, it seems to me. With its different platforms and different computers in the home, it’s important for information?

00:10:23:05 – 00:11:02:23

Nelly

This is important for the information, as Amaury explained, it also allows one to have a long term vision of the trends that crossed the news during the year 2022 since the Crossover project runs from December 2021 to February 2023. The dashboard also allows journalists to have a place where the watch is condensed, a tool where the watch is condensed in one place without needing special technical skills, or to go digging in all the platforms to see the different trends that emerge, among others. It is a priori a saving of time and efficiency that they appreciate.

00:11:05:21 – 00:11:24:07

Sébastien

We can now see the interest of the dashboard for professionals. It allows us to analyze and show what the recommendation algorithms do to the information. But how can these technical tools be useful to you, to me, to everyone, to avoid falling into disinformation?

00:11:24:21 – 00:11:34:15

Sébastien

We will discuss this with Divina FRAU MEIGS, specialist in education, media and information within Savoir Devenir.

00:11:34:15 – 00:11:35:15

Sébastien

Hello Divina.

00:11:35:15 – 00:11:36:14

Divina

Hello Sébastien.

00:11:36:14 – 00:11:41:23

Sébastien

So this algorithm thing is not obvious to everyone. So how do you tame them?

00:11:42:06 – 00:11:59:09

Divina

So it’s true, the culture of data and algorithms remains very fuzzy for many people. The whole project of algo-literacy and that we defend in Crossover is to make sure that this changes. Obviously, we cannot reduce media literacy, information literacy, to the question of algorithms. But we can’t ignore it either.

00:11:59:09 – 00:12:06:24

Divina

Algo-literacy is an increasingly important part of IME that allows everyone to master rather than suffer technology.

00:12:06:24 – 00:12:13:23

Sébastien

But then, how do you work on this type of subject? And after all, how do you get over this hurdle because it is quite technical?

00:12:13:23 – 00:12:37:16

Divina

What is exciting is that we work with other actors such as journalists, developers and fact-checkers. This gives depth to the IME, it enhances the multi-actor, multi-partner approach that is fundamental for us, to decompartmentalize teachers, introduce students to other professions, train them, and actively learn in touch with reality in connection with other people.

00:12:37:16 – 00:12:47:23

Sébastien

But at the same time, there’s another point: it’s one thing to understand what algorithms do to information. But it’s quite another to use tools like the famous Dashboard.

00:12:47:23 – 00:13:02:22

Divina

This is true. But in IME, we rarely have a tool approach. We are on the side of uses. If we use the tool, it must be embedded in a broader approach to information literacy, to the fight against disinformation.

00:13:03:11 – 00:13:20:00

Divina

That said, tools like the dashboard are very useful for understanding what is going on without necessarily having to use them yourself. In this project, we are using it first as an indicator. Afterwards, there is also a whole part of the population of all ages, which is quite capable of using it.

00:13:20:19 – 00:13:31:23

Divina

And then there are other simple practices that you can do on the tool side. For example, once you understand the role of auto-completion algorithms on Google, it’s really easy to disable this feature if you don’t like it.

00:13:32:07 – 00:13:42:17

Sébastien

As for the multiple algorithms of recommendation, sorting, prediction, are they always at work with one of its variants, as soon as there is disinformation?

00:13:42:17 – 00:13:52:08

Divina

No, no, of course. One of our missions is to cover the different types of disinformation and to show what mechanisms they rely on in order to find the right solutions to fight them.

00:13:52:08 – 00:14:12:18

Divina

But take for example, fake news which is based on sophism like lazy people don’t work, unemployed people don’t work either. So the unemployed are lazy. In this case, yes, although we’re going to work on the fallacy and not on the algorithms. So there are different mechanisms, different information that call for different actions from the IME.

00:14:12:18 – 00:14:20:02

Sébastien

So what? To conclude, what would you say to those who would like to start this famous algo-literacy?

00:14:20:02 – 00:14:42:23

Divina

Well, first of all, bravo to you! There aren’t many IME resources out there yet, but it’s starting to happen, and we’re doing our part with the IME Crossover kits, which allow for both a better understanding of what’s going on and for easy workshops with high school students and adults. Each kit is built around a podcast like this one and then is based on a concrete situation of investigations, studies.

00:14:43:09 – 00:14:57:16

Divina

It provides a quiz on the topic in question to help students acquire key knowledge. There is also an accompanying document that provides a brief description of the key concepts covered in the theme and a concrete proposal for workshops.

00:14:58:07 – 00:15:02:08

Sébastien

Thank you Divina! And then we leave the last word to Nelly.

00:15:02:08 – 00:15:02:24

Divina

Of course.

00:15:02:24 – 00:15:09:16

Sébastien

Nelly. So the Crossover project is over. So, is it over? Or will you continue to work on it?

00:15:09:16 – 00:15:31:03

Nelly

So the Crossover project is available in different formats. The media and information literacy kits in French and Dutch will remain available free of charge on the Savoir Devenir website. The investigations published by Apache and published by Check First and the EU DisinfoLab will also remain available online.

00:15:31:19 – 00:15:56:15

Nelly

The Crossover project will continue to exist, notably through Com Presse with Apache and Check First, and will focus on images manipulated on social networks. The results will be available and public inside the Dashboard. And to note that the Crossover project focused for fifteen months on Belgium in various provinces of Belgium, but has the vocation to extend potentially in other territories.

00:15:57:03 – 00:16:04:02

Divina

Well, we thank you and we’ll see you on the platforms and on the dashboard.
See you soon.

00:16:04:02 – 00:16:19:06

Sébastien

This podcast was produced by Savoir Devenir and INA within the framework of the Crossover project funded by the European Union. To the conception Sophia HAMADI and Pascale GARREAU.

00:16:19:06 – 00:16:29:12

Sébastien

Technical direction and editing by Gabriel FADAVI, writing assistance by Jean-François GERVAIS and Sébastien GAILLARD.

00:16:29:12 – 00:16:35:13

Sébastien

Find out more about the project on the Crossover.social website.