What process did we go through to rank the supermarkets? A description of PAN UK’s methodology

Below is a description of the process that PAN UK went through to produce our supermarket ranking. It was an extremely detailed process so we have just given an overview but are happy to answer additional questions so please do contact us if you want further information.

Stage 1: Supermarkets selected

PAN UK’s 2021 survey builds on work conducted for our 2019 ranking. For this previous ranking we selected supermarkets based on their share of the UK groceries market. This gave us a list of the top ten supermarkets in order of grocery market share: Tesco, Sainsbury’s, Asda, Morrisons, Aldi, Co-op, Lidl, Waitrose, Iceland, M&S. In 2021, these remain the supermarkets from which the majority of people in the UK buy their weekly groceries and are therefore the ten supermarkets assessed for the ranking.

Stage 2: Survey sent to supermarkets

We sent a survey to each of the ten supermarkets, with a five-week deadline to respond. The survey asked detailed questions about eight topics related to pesticides in supermarkets’ global supply chains. While the key focus was on actions the supermarkets are currently taking, we also took account of any progress made since the 2019 survey and ranking. The eight topics included in the survey were as follows:

• Phasing out the most hazardous pesticides
• Supporting suppliers to reduce pesticide use and adopt non-chemical alternatives
• Monitoring and reducing pesticide residues in food
• Reducing harms caused to bees and pollinators
• Being transparent about pesticides
• Selling pesticide products
• Engaging with consumers on reducing pesticide use
• Boosting organic sales

Download the full PAN UK supermarkets survey here. 

Stage 3: Supermarkets scored and ranked

After five weeks, PAN UK received survey responses from all of the top ten supermarkets. We set about analysing and scoring the responses. Whenever possible, supermarket survey responses were cross-checked against publicly available information.

The majority of the survey questions were qualitative, but some were quantitative. Each answer was scored from low to high. A higher score was received when a supermarket was doing well compared to other supermarkets, and likewise a low score was received if a supermarket was doing comparatively badly.

Based on their scores, supermarkets were then allocated a ranking for each topic of ‘lagging behind’ (1 trolley), ‘could do better’ (2 trolleys), ‘making good progress’ (3 trolleys) or ‘outstanding’ (4 trolleys). Each supermarket’s total number of trolleys was added up and then divided by the total number of relevant topics to reveal the final ranking positions. Most supermarket’s scores were therefore divided by eight because all eight topics are relevant for their supply chains. However, two supermarkets (M&S and Iceland) do not have gardening ranges and therefore don’t stock pesticide products, and so their scores were calculated as an average of how they performed over seven topics. When two supermarkets received the same total number of trolleys, the supermarket which received the highest number of ‘outstanding’ rankings was placed higher up the overall ranking.

Supermarkets were scored against three main criteria:

• Most importantly we assessed to what extent each supermarket is implementing PAN UK’s recommendations.
• Secondly, we compared how supermarkets are performing against each other.
• Thirdly (but least importantly in terms of score), we looked at how much progress each supermarket has made since 2019.

While many supermarkets mentioned measures designed to ensure that they do not break pesticides laws in the countries where they operate, we did not give points for these actions. PAN UK expects all supermarkets to follow national pesticides law and, as such, points were only given for measures that go beyond legal compliance.

Another factor to consider is that as the UK supermarket sector makes progress on pesticide reduction, the measures needed to achieve a score might evolve. So, for example, in the 2019 ranking, a supermarket that published its pesticide policy might have received a high score for transparency since they were one of the few to do so. However, the sector has made progress in transparency over the past two years so the same supermarket might find itself with a lower transparency score in 2021 if they have made no progress.

Stage 4: Supermarkets given an Opportunity to Comment

We sent each supermarket an outline of what we were planning to publish about them and gave them 17 days to respond. We received responses from all supermarkets except for Asda. PAN UK has represented the responses we received as fairly and honestly as possible, while making them comprehensible and engaging to the public.

Next steps

In 2022, PAN UK plans to continue to work closely with as many supermarkets as possible to help them implement our recommendations. In 2023, we will send another survey to UK supermarkets and they will once again be scored and ranked according to a similar methodology. Through our ongoing efforts, we hope to assist UK supermarkets in making considerable headway at reducing pesticide-related harms linked to their supply chains.