“NEW’s Quick Count Result Needs Questioning…” -Dr. Adams B. Steven Asserts

By Adams B. Steven (Ph.D.)

Not many Sierra Leoneans had heard or paid attention to the acronym PVT until the 27th of June 2023. The National Elections Watch (NEW), a network of Civil Society Organizations, through two press statements, reported that going by the data from its Process and Result Verification for Transparency (PRVT), candidate Rtd Brig. Julius Maada Bio would poll between 47% and 53% and therefore would be wrong to call an outright winner of the 2023 presidential election on the first ballot. However, this is not the first time the exercise has been undertaken in Sierra Leone. It was implemented in each of the elections since 2007.

But what is Parallel Voting Tabulation (PVT) or Quick Count? A quick count is an elections monitoring mechanism done through surveying a sample of polling stations on elections day. Elections observers monitor the voting and counting processes at selected polling stations, record key information including polling results and report their findings to a tally center which aggregates and analyses these results. Generalizations are then made based on the findings. The system has been used around the world since 1986 as a mechanism by civil society to enhance the integrity of electoral processes. There are many positives as well as negatives of implementing PVTs which I will not delve into in this piece.

Unlike in other types of surveys, sampling and consequently announcing elections results has serious implications on governance and the citizenry. In politically charged and fiercely contested elections like the just concluded multi-tiered 2023 elections, quick count results can become a flashpoint for controversy and subsequently tensions. For these reasons, it is essential that PVTs be implemented with uncompromisingly high standards, statistically valid methodologies, and highest levels of transparency.

Why should we question the results published by NEW? We should base first on the statistical methodologies used, and the highest levels of opaqueness surrounding the process.

The consequences of inaccurate PVT results can be damning. As a result, the statistical methods used (sample size, sample selection, margins of errors and confidence levels) must be strictly conservative. The standard practice in election observations is to design the sample with parameters that capture the heterogeneity of the voters, the tightest of margins of error, and the highest of confidence levels.

The most important decision in the PVT implementation revolves around the sample size and sample selection. Getting either wrong would result in a bias outcome leading to misleading or outright erroneous conclusions. For PVT purposes, the standard formula used to calculate sample size is:

𝑛 =  𝑁𝑧 ! (1 − 𝑃)

𝑁𝑒 ! + 𝑧 ! (1 − 𝑃)

Where: n = sample size; N = number of registered voters; z is confidence level; P = degree of heterogeneity; and e = margin of error.

Therefore, constructing a quick count sample involves making judgements on heterogeneity of the population, and determining the margin of error and confidence levels.

The NDI, a pioneer of quick counts recommends very conservative parameters be used in determining the sample size because of the importance of elections and the ramifications of wrongly predicting election results. The NDI recommends, in very tight elections, heterogeneity score of 0.5 (the most competitive suggesting that either candidate has a 50% chance of winning), a confidence level of 99%, and a tight design margin of error (ME) of less than 0.5 percent. In practice, most quick counts around the world use ME’s of less than 0.1%.

With these recommended and generally used parameters, let us calculate the ideal sample size for a PVT exercise for the just concluded elections.

A very popular formula for calculation of sample size is:

𝑠𝑎𝑚𝑝𝑙𝑒𝑠𝑖𝑧𝑒 = 𝑁𝑥𝑧 ! (1 − 𝑃)

𝑁𝑥𝐸 ! + 𝑧 ! 𝑥 (1 − 𝑃)

Where,

N = Total registered voters (3,374,258)

P = Heterogeneity score: (0.5, most conservative)

E = Margin of error (0.1%, higher than the usual but still tight enough)

Z = z-score of Confidence level (99 %; 2.58)

𝑠𝑎𝑚𝑝𝑙𝑒𝑠𝑖𝑧𝑒 = 3,374,258 𝑥 2.58! 0.5(1 − 0.5)

3,374,258 𝑥 0.001! + 2.58! 𝑥 0.5(1 − 0.5) = 1,114,471

The ideal sample size should have been 1,114,471 voters, and if there are 300 voters per polling station, then the PVT should have surveyed 3,715 polling stations. However, resource limitations would not easily allow polling such large number of stations. It is acceptable to increase the ME to 0.2%. This should result in 1,235 polling stations. NEW sampled only 750 stations. By polling only this number of stations, under sampling may have occurred which could have led to unreliable results. New needs to explain to the people of Sierra Leone what parameters were used in deciding the sample size and why were these parameters used.

The second source of doubt should be the margin of error published by NEW. Let’s assume for arguments sake that NEW used the most conservative parameters in arriving at their 750-sample size. That is, they used 99% confidence level and a heterogeneity score of about 0.5 (the most competitive). The formula for calculating the Margin of Error is:

𝑀𝐸 =   ?(1 − 𝑃)                                                         √𝑛

Where P = 0.5 (degree of heterogeneity); n = 750 (sample size); z = 2.58 (z-score of 99% confidence level)

𝑀𝐸 = 2.58  ?0.5(1 − 0.5)

√750 = 0.047 𝑜𝑟 4.71%

NEW’s point score for the SLPP candidate is 50.4. With this margin of error, the president’s 99% confidence interval would be 45.69 – 55.11 putting the president in an outright winning range. By reducing the margin of error to 2.7 (which must be explained by NEW), outright winning was unachievable for the president. The people of Sierra Leone have the right to know how NEW arrived at 1.7 or 2.7 margin of error for their analyses.

Third source of doubt is the selection of the 750 polling selections. A major decision is determining the most appropriate type of random sample during the PVT process. A NEW executive was forced to announce the distribution of stations polled on air on Monday July 3rd. Looking at the numbers, the stations were distributed perfectly by share of registered voters. Way too perfectly! The issue of distribution remains, however. Take for instance Kenema district and Kenema town. Results from a

poll from Nyandeyama will be a lot different from another random poll from GorahunTunkia. The urban settlements are more diverse and the rural settlements more homogenous. This can be settled with a disclosure of the exact polling stations sampled. In other words, providing the people of Sierra Leone the raw data with polling station identifiers. Another issue is the heterogeneity of the districts. The districts are not equally competitive. To have an accurate data, a competitive district must have higher number of stations compared less competitive district even if both districts have equal number of registered voters. For example, whereas 10 polling stations in Bombali can capture the sentiments of the entire district, more polling stations will be needed to fully capture the diverse opinions of the people of Kambia district. This Limits generalization of the NEW results.

A fourth source of doubt is the voter turnout rate of 77.3% announced by NEW. Sierra Leoneans love their votes. They love their democracy, and they fully participate in it. Historically, turnout rate hovers around 85%. There was no significant incident which would have suppressed turnout by up to 7%. Clearly, there was enthusiasm gap between strongholds of the SLPP and that of the APC. The turnout rate was lower in APC strongholds. Many districts in the South-East recorded turnout levels in the north of 90%. Maybe the difference in turnout rate may have been missed by NEW and to some degree explains the difference vis the ECSL results. If you take the vote tallies that were announced for 60% of the polling stations which NEW confirmed that were correct, calculate the percentage share for each candidate and divide up the remaining votes at NEW’s turnout rate of 77% for each district, President Bio gets 45.98% and Dr. Samura gets 50.50% of the remaining votes. Cumulatively, this would result in final vote tally for President Bio at 51.16%. However, when the turnout rate is changed to 90% for the south-east, (less than most polling stations in that region), President Bio’s share of the total votes increases to 56.25%, mirroring the ECSL tally. NEW probably got the turnout rate wrong for the South-East. And this is because of bad sampling where I believe most of the sample was collected in Urban areas especially in the South-East.

The fifth source of doubt is the secrecy with which the quick count was done and the refusal to make the data public. In all other countries quick counts are implemented, enough publicity about its merit and methodology is done. In Ghana for instance, videos and audios were made and played over TVs and radio multiple times prior to the elections. Ivory Coast did the same, so was Chile and Cambodia recently. NEW did not only fail to educate the public, but they also kept their methodology away from even those that should be considered partners to NEW or stakeholders in the election process. PVT requires complete knowledge, buy-in, and acceptance by all the stakeholders in the elections. The stakeholders according to NDI are the political parties and the ECSL. Post elections, NEW has so far refused to publish their data even after incessant calls from other CSOs.

And finally, who are the people behind NEW? Are they truly independent? Has anyone of them made a public pronouncement that the president must be a one term president? Or a candidate is a personal project? Is anyone of them a son/daughter of a renowned opposition politician? Has anyone of the executives been fired by the current government from their previous positions? The answers to these questions will water down the apprehensions of most Sierra Leoneans about the NEW results that has the potential to throw the entire country in chaos.

In summary, the international community should look beyond these numbers published by NEW. They must look behind and determine all the moving parts that made up these numbers. Was the sampling size right? Are the calculations consistent with grounded statistical methodologies? Where the correct numbers transmitted from the polling centers? The international community must, together with the people of Sierra Leone ask for the entire raw data and a cross reference made to the elections data from ECSL. Together, we all should question accuracy of NEW’s PVT result because there are unknown unknowns about their methods, about the data, and about the people behind exercise. This election should not be about the ambitions of individual politicians of any color. Nor should it be about any preferences of non-Sierra Leoneans. It is about 8.4 million people who would suffer subsequently any negative consequence of the actions of greedy politicians.