Childrens tablet banner

Media Literacy Programme

Evaluation of the effectiveness of the Common Sense Digital Citizenship Curriculum

The UK Government’s Department for Culture, Innovation and Technology DCITS (previously sitting in DCMS)’ Media Literacy Programme aims to engage stakeholders working in the field of Media literacy and digital citizenship in processes of rigorous evaluation in line with OFCOM’s paper on Evaluation as a tool of sharpening Media Literacy Interventions. The call requires partnerships between Stakeholders who work in the field of media literacy/digital citizenship and Academic institutions. In this project, our team at the Department of Media and Communications in LSE partners with civil society organisation Common Sense (UK). Our aim in this research is to undertake an independent evaluation of the effectiveness of the Common Sense Digital Citizenship lessons, which have previously been taught in US schools and are now being developed and customised for UK schools through the UK branch of Common Sense Media, which started in the US (https://www.commonsense.org).

Aims

Recent academic research has highlighted the prevalent concerns of adults and parents regarding their children's digital engagement, particularly focusing on social media usage and screen time allocation. Our project aims first and foremost to answer the following research questions:

1) What factors make some school communities more able to recognise and be resilient to Disinformation than others?

2) How impactful are the Common Sense Digital Citizenship resources in improving learners’ digital literacy, including their ability to recognise and resist misinformation and fake news?

Our project also aims to explore how the  Common Sense curriculum materials contribute cumulatively, in smaller units and over time, to nurturing healthy and critical digital habits and dispositions among students, (for instance, responsible behaviours that enhance online safety, informed decision-making, critical thinking, screen time management, mental well-being preservation for self and others, and informed, compassionate digital decision-making. A comprehensive study conducted by Ofcom in 2021 in the UK revealed additional parental worries related to their children's online activities, including cyberbullying, online privacy, exposure to inappropriate content, financial pressures for online purchases, and concerns about online gaming, such as in-game spending, bullying in gaming communities, and game content. In light of such concerns, our project also seeks to better understand children's online gaming habits and the potential value of Common Sense materials as an intervention therein.

Project methods and objectives

In June 2023 we started working in four schools in and around London (2 primary and 2 secondary) to reach our four main objectives: 1) to train teachers of 4 classes in each school to teach the Common Sense Digital Citizenship Curriculum; 2) to produce age-appropriate baseline evaluation materials of no longer than 30 minutes to establish the baseline media literacy and critical digital awareness of the children; 3) to observe the complex interplay of school environment, teaching and learning going on during the media literacy intervention with the use of the Common Sense Digital Citizenship Materials in the classrooms for 6 weeks in each case; and 4) to conduct post-teaching quizzes, interviews and focus groups with the teachers and children to establish how effective the materials are at improving different age cohorts’ critical media literacy in regard to issues such as digital safety and security; online identity protection; being able to distinguish marketing materials online and particular originators and promotors; being able to recognise fake or improper information from a range of sources.

The findings of this evaluation will be of use to parents, children, young people, teachers and other educators working with children and digital environments; its direct utility will, of course, be to the Stakeholder Common Sense in tightening its materials for the UK audience; and to the funders (DSITS) to improve the collective evidence base about effective ways of delivering media literacy interventions, and inform the development of intervention evaluation best practices.

Project team

ShakuntalaBanaji

Professor Shakuntala Banaji
Professor of Media, Culture and Social Change

Interests and expertise: Bollywood; Hindi cinema; South Asia; audiences; children; creativity; film studies; international media; media education; media literacy; misinformation and disinformation; online participation; social media, toxic speech and hate speech; youth civic participation

Fiona-Abades-Barclay

Dr Fiona Abades Barlcay
Research Officer

Interests and expertisemedia and culture; online labour; critical media literacy; digital and popular cultures; postfeminism and neoliberalism; girls and young women; online communities and parasocial relations.

saumyadeepmandal

Saumyadeep Mandal
Research Assistant

Interests and expertise: film studies; subaltern media; caste; dalits; audiences; spectatorship; popular culture


The project team also includes Jenna Khanna from Common Sense Media.

Project phases

Project activities will proceed in 3 phases that enable robust evaluation and analysis as well as dissemination via the project staff.

In phase 1 we will develop the methodological instruments which will include:

  • Tests – age appropriate pre and post lessons, testing for core competencies of digital citizenship
  • Observation in the classroom during all lessons where the curriculum is being taught
  • Teacher focus groups
  • Lesson quizzes (existing resource)
  • Surveys for teachers and learners, post lesson
  • Interviews with teachers (on camera) and learners (voice only, as compatible with ethical protections)
  • Interviews with teachers and leadership teams throughout test period

We will identify 4 schools in inner London, Essex or Kent with diverse demographics, to include: 1 urban/suburban high FSM; 1 urban/suburban affluent or selective independent; 1 rural or peri-urban primary high % FSM; 1 rural or peri-urban primary affluent/ selective

Phase 2 - action research stage (including deployment and data collection):

Each school will:

  • Teach 6 lessons
  • Over the course of 1 term
  • To at least 1 class, (or minimum 50 students)
  • In each Key stage (KS1-KS4) Schools which take part will be offered support to release teachers from their normal duties to participate.

LSE’s RA will work in schools alongside Common Sense representative(s) to support teachers in implementation while LSE’s RO observes, and conducts independent evaluation of quant and qual data as follows:

  • Pre and post lesson quiz (existing)
  • New pre and post-test designed by LSE
  • Teacher interviews and focus groups
  • Observations of delivery of Common sense digital citizenship ‘curriculum’ of 6 lessons per academic year, over the course of one term with a minimum of 4 classes across 2 key stages (KS1&2 or KS3&4) i.e. minimum 50 students in each class in each school
  • ‘Curriculum map’ to support the teachers to implement the lessons

We will endeavour to collect data from a minimum of 400 learners in the course of the evaluation programme.

Phase 3 - data analysis and writing of the report

Results will be broken down by area, key stage, gender, age, and we will collect data relating to socioeconomic macro factors relating to the school & catchment area.

Funding

This project is funded by the Department for Culture, Media and Sport (DCMS).