V-Party is likely to produce systematic bias

The purpose of this letter is to share some of my first impressions regarding the V-Dem Party dataset, released recently by the V-Dem project. As I understood, country experts, normally based on the country they code, codify most of the questions related to party positions. I acknowledge how complicated it might be to measure party positions based on expert surveys, but I’m afraid that, looking at the party scores in Spain, the measurement instrument is flawed by strong systematic group bias. Most the of questions are coded from a Spanish-centric perspective that, in the context of high polarization in the national issue, decisively affect party scores representing regional minorities. I base my analysis on the 2019 score of ERC (Esquerra Republicana de Catalunya, a Catalan-nationalist party), since other Catalan/Basque regional parties are not represented in the survey that year.

My concerns are placed especially on the components of the illiberal index. In the first place, the minority rights (v2paminor) question states that the “will of the majority will be implemented, even if it violates minority rights”. All parties score high, except ERC and VOX. Country experts show a clear particular interpretation of minority rights, ignoring that political and civil rights of ERC politicians and citizens have become systematically violated according to many international experts. And ERC’s main leader cannot occupy a seat in the European Parliament despite a favorable sentence of the European Court of Justice. Neither is consistent that minority rights are an issue for C’s (v2pasalie_3, it doesn’t specific if it’s in favor or against) but not for ERC, despite being considered to represent an ethnic and racial group (v2pagroup_5) and despite Catalans are considered a minority group according to the United Nations relator in national minorities. Moreover, coders view ERC as a party that, along with VOX, strongly promotes cultural superiority (v2paculsup; also identified as a salient issue for the party in v2pasalie_6). This particular view of what constitutes a minority and who speaks with cultural superiority is shared especially within Spanish elites but hardly could be sustained outside the Spanish borders.

The other components of the illiberal index follow a similar direction. For example, ERC and Podemos are two of the parties scoring worst in the variable “personal attacks and tactics of demonization against their opponents” (v2paopresp), whereas PP and C’s receive a high rate (!!). A proper content analysis of political leaders' declarations could show the inaccuracy of these perceptions. It seems to me that country experts fail to capture the political outbidding among PP, C’s and VOX, in the last two elections. Another variable that conforms the illiberal index is the rejection of political violence (v2paviol). In general, PSOE, C’s, and PP rank high and ERC ranks relatively low compared to the rest of parties. It is hard to think of a party in Spain which has rejected violence more repeatedly than ERC in the last years. It is not the case of the Spanish mainstream parties, who applauded the violent police methods against Catalan citizens in 2017.

It seems to me that the expert survey shows favorable bias particularly towards C’s (and PP, which probably also shows ideological bias against left-wing parties). A clear example is in the question that asks whether parties are clearly committed with free and fair elections (v2paplur). C’s scores higher than the other parties. Perhaps, the coders ignored the case of Andrés Betancor, paid by C’s at the same time that was member of the Electoral Board that blocked the candidacy of Catalan members in the European Parliament. In a press article, Betancor warned in April 2018 that the law allowed the Catalan leader Puigdemont to run the European elections, and in April 2019 he banned his participation. A court resolution finally allowed the Catalan members to run the elections. Betancor resigned. This is one of the clearest cases against free and fair elections. The worst part is that ERC, one of the victims of this operation, scores worse than VOX.

Other questions that may suggest a systematic bias against some parties and for another are the LGTB social equality (v2palgbt) –seems hard to understand why ERC is below C’s–, or clientelism (v2paclient), where ERC is at a similar level of PP. I exclude other variables and evidence to not overload this article.

To conclude, coding the positioning of political parties in highly polarized political environments is very likely to produce systematic bias. The risk is higher when the coding decisions are taken by a reduced number of five country experts living in the country, the national issue is salient, and some parties to be coded represent national minorities. I haven’t looked at confidence intervals, although it seems to me clear that V-Party indices show a group bias aligned with the Spanish-centric attempts (I would even dare to say officialist-centric) to criminalize regionalist movements. Since ideological preferences of country experts may be revealed in these salient issues, V-Party may compromise the good reputation of V-Dem dataset.