Meeting Report

I. BackgroundII. Event overview  |  III. Session summaries  |  IV. Moving Forward

 

Session summaries

 

 

Defining quality and minimal quality standards for AVMS

The second thematic session examined different aspects of quality standards for observatories, addressing questions such as ‘Quality for whom?’ and ‘Quality for what?’. This session looked at issues such as ‘value for money’ (VfM), and feasibility and usability in challenging settings.

Anna Alvazzi del Frate, Research Director at the Small Arms Survey, set the scene by addressing the linkages between ‘value for money’ and quality [See presentation]. Taking into consideration that participants were from different backgrounds and working in distinct environments, she underlined common thoughts regarding the notion of ‘quality’, a set of characteristics on product or service defined as standards. In various cases, high quality data might not be able to satisfy the quantitative needs. Instead, value for money can refer to the degree of excellence. It is not always about being cost effective, but rather to identify the characteristics of quality that a more expensive project could bring.

Martin Kerkula, shared the experience of the Liberian Armed Violence Observatory (LAVO). Defining  standards was one of LAVO’s main challenges, as they had little experience with this. In a post-conflict setting as Liberia, the challenges of creating an institution like LAVO are many. Some have to do with the poor national infrastructure of roads, internet services, and telecommunications as the data collected are not often not reliable or accessible. Some challenges are more of an internal nature and have to do with the limited resources available (and low salaries) of the organization, the difficult logistics related to data collection and with the still limited capacities to disseminate data in a user-friendly manner. Other challenges stem from the environment in which LAVO operates. For example, police and other data providers request ‘incentives’ and charge money for providing data. More importantly, policy makers need to see the issue of armed violence as a national problem and a priority for intervention. LAVO is working to address all the above mentioned challenges. Getting a legal NGO status was complicated, but was nevertheless important, as it gave credibility to LAVO as an institution. This was an important step to strengthening its cooperation with the national government so as to bring more attention to the issue of armed violence.

Enrico Bisogno, Team Leader of Crime Statistics of the Statistics and Surveys Section at United Nations Office on Drugs and Crime (UNODC), presented the organization’s perspective on quality standards [See presentation]. He first stated that there were often two situations regarding data: either data were considered very scarce, or there was plenty of existing data. National statistical offices provide data on many different topics; however, the key concern is the quality of the data. The processes used to collect the data from one year to another may change, raising therefore the question of quality. Relevance and accuracy were often seen as the most important dimensions to guarantee quality of data collection. Timeliness/punctuality, coherence/clarity, comparability, as well as accessibility were other dimensions which were also seen as very important. Bisogno then made recommendations for each stage of the data process. (1) When deciding whether to collect and produce statistical data, one needs to make sure that resources, finances, skills and infrastructures are available. (2) During data collection, the data must be defined and meet the international or national standards. Currently still under development, the Integrated Criminal Court Survey (ICCS) will help set the standards for crime-related data collection. (3) During data processing, it is vital to develop and implement all possible consistency checks. Errors have to disappear while data processing should not introduce new errors. Bisogno added that data processing was important as “bad data never dies” and advised to be conservative when processing. (4) Finally data dissemination and analysis should guarantee all data users to receive the same data. Data used should always be defendable--broad consultation, good allies, as well as data processing and checking are important parts towards achieving the quality standards.

Sana Jaffrey shared her experience in supporting the Indonesian National Violence Monitoring System (NVMS) [See presentation]. In Indonesia, data have been collected since 1998 and been presented with monthly updates to the public. The NVMS database is hosted by the government, with the support of the World Bank. Two key challenges are data reliability and accessibility. In cases where data are difficult to be found, the selection of sources is particularly critical. In other cases, data are inaccessible and absent from the database; for example, while NVMS uses media as one of its key data sources, homicide statistics are of difficult access. Jaffrey also insisted on the importance of having a clear approach to data dissemination. Addressing the data in the correct manner is crucial in order to reach out even to people who are not fond of quantitative data. We should maintain policy relevance, manage political sensitivities, facilitate broad use of data and enable data users to spotlight ‘hidden’ statistics.

Abdullahi Mohamed Odowa, discussed quality challenges from the perspective of the Hargeisa Observatory of Conflict and Violence Prevention (OCVP) in Somalia. The Observatory, established in 2009, gathers information at the national level mostly by conducting surveys and works with local authorities on data collection and distribution. The Somali communities don’t recognize national authority since the country has been divided in many ways over the last 20 years. Due to a weak national infrastructure and since conflict dynamics are rooted at a district level, the OCVP relies mainly on data at the local level. Furthermore, the history of conflict has led to a breakdown in public trust regarding public data and information. Such a fragile context hinders the work of the OCVP. Additional challenges encountered are: low level of literacy/knowledge/imagination (if interviewees do not understand properly the question, the answer is not reliable); low capacity and skills of staff; dissemination of data (addressing numerous different perspectives in the country); security (influencing both quality and quantity of data collection); and meeting the costs of covering security threats. Odowa explained that the OCVP would continue working at the district level, but in  time  they would like to start collecting and presenting data at the national level.

Maria Cerqueira, from the Pan American Health Organization (PAHO), presented the experience of  the Juarez Citizen Safety and Conviviality Observatory [See presentation]. Initially working on road injuries and safety, when violence started to escalate as of 2008, the Observatory data collection was extended to include information on homicides and violence at home. The first challenges encountered were related to the equipment and information system where the data had to be collected. The participation of the university in this project, that finally provided the equipment, helped add credibility to the data collected. After a first assessment of the reliability of data sources, data is collected from different public health institutions as well as from local (and border) authorities. The Observatory presents its results in a ‘data hub’ which can be accessed by everyone. The data can be mapped on a graph using different indicators, allowing data users to access the required information.

The session highlighted that ‘quality’ is measured by the capacity to take into consideration the local needs for data and by ‘value for money’ considerations. Investment in data need to provide actionable information. All presenters had noted the importance of paying attention to local needs, whether it is traditional community, elders, workers, or children. These needs should be considered when developing armed violence observatories. The session also underlined the importance of platforms for stimulating policy changes. However, such changes will need time since the issues at stake often will require creating political will and sometimes fostering new ideas in the society.