After the end of the On Think Tanks Data Visualisation competition, we were able to catch up with the winner of the Third Round as well as the final overall winner of the competition, Michal Tošovský. Michal led the team at Otevrena spolecnost, o.p.s. -- a Czech think tank focusing on police and public security -- that is behind Mapping Czech Crime, a visualisation that we think does what it says on the tin and does it well! I wanted to dig behind the scenes to understand a bit more the technical nature of how the visualisation was put together, and the challenges they encountered in working with such a potentially sensitive topic.
JK: What was your inspiration for creating this visualisation? Had you seen similar products elsewhere? Was there a particular need in the Czech Republic?
MT: Several years back we were discussing the idea of improving public access to information about police efficiency in controlling crime with Czech police and Ministry of Interior. Both of us knew about similar initiatives, among others, in the UK, US, and later on in Latvia. But while it seemed that police and MoI were ready to take something forward in this field, nothing happened for quite a long time. So we decided to take the initiative.
With regard to the particular need – I think that such a need is embedded in development of recent society. The spread of the Internet brought enormous access to various people’s and institution’s interpretations of any issue – including criminality – and to be qualified to scrutinize these interpretations one needs to have access to source information. Without knowing the data it's nearly impossible to challenge the status-quo, especially one that might be acceptable for the institutions but far from ideal from the perspective of particular citizen. So I don’t think that it is some specificity of Czech Republic that made us working on the project. We may have wished to create a similar application if we lived in any other European country too.
JK: This visualisation seems to have required both a lot of data and also close collaboration with police departments in the Czech Republic. How did you go about obtaining the data?
MT: From the very beginning we knew that our application had to cover the whole country. We weren’t interested in making a nice product for the capital city only to find the problems faced by people in some regions outside Prague are very different and even hard to believe for somebody who doesn’t leave the comfort of a big city. So at the beginning of the project the “lowest common denominator” was equivalent access to crime data all around the country.
With the first version of the website, we decided to use only data that were published by the police at that time. We had two reasons for it. First, we knew that if we start negotiating with police about providing us deeper data we would end up mainly with restrictions and fears about its use. And second, we also wanted to show that providing better information is not only question of its quality but also question of real will to inform with it. In the end, this strategy paid off - after launch of the first version, the MoI contacted us and supported us in negotiation with the police to get deeper data while the website apparently didn’t bring any threat to them.
JK: Working across a number of police departments and the MoI sounds like it must have been a nightmare in terms of reading across data sets. What was the data cleaning and processing like?
MT: Yes, the police provided us data at the lowest organizational level (527 police divisions across the country), but they surprised us by refusing to provide the shapefile of its jurisdiction. Cleaning the data was simple thing compared to re-creating the shapefile. The data were provided in hundreds of excel tables and getting it right was “only” about initial analysis and preparing of a script to read it. On the contrary, creation of the shapefile took several hundred hours of work to have it ready, in proper borderlines and linked to source data about population of the areas (which we had to buy from the Czech Statistical Office).
JK: One of the things that strikes me about this visualisation is just how much information it contains. How did you come to this specific way of organising the information?
MT: Thank you for appreciation. Probably it is not necessary to explain the informative power of map-based data. From the beginning we knew that the visualisation must communicate both to somebody who is not familiar with data at all and to skilled data users as scholars, journalists or policy-makers. To make the data interesting and self-explanatory for the first group of users we simply didn’t see any alternative to map visualisation. We used crime rate (number of crimes per 10 000 inhabitants) as the key indicator for comparison of diverse areas because it is the simplest way to express distribution of any phenomenon in any given population and as such, despite many criticisms, it is recognized internationally too.
We knew that most of the information may be accessible straight from the mapview, so that users didn’t lose visual contact with the context of the area they were just looking into. Also we tried to be empathic with users – we knew that mostly they will be interested in various comparisons of the data – in time, in relation to another area and so on. And we hoped that “more analytical” users would be able to wade through the visual part of the application to detailed data. I must not forget that at the very beginning we had a very useful consultation on data-display with web designer Karel Minařík, who helped put us on the right track right from the beginning.
JK: This is already the second version of a visualisation you did earlier. What sort of upgrades did you make and why?
MT: The biggest upgrade is in much finer granularity of the data. Instead of 80 police divisions we have divided the country in 527 areas. Instead of 9 administrative and non-intuitive categories of crimes the new version distinguishes 201 crime types as they are recognized by the police. Moreover there are other data related to each of the crime types (e.g. damage or some perpetrators’ characteristics) that can be accessed through the new version's API. Also, I have to mention one outstanding fact: with the new visualisation we provide much more statistical information about recorded crime in Czech Republic than the police themselves.
JK: And what sort of things might Mapping Czech Crime 3.0 contain and why?
MT: The “next” version is a big question now. The data we have in the application is, to some extent, the most detail that can be extracted from the aggregated crime data kept by police. Unfortunately, the quality of particular crime records doesn’t allow its unified treatment. On the other hand, the police have recognized the benefits of computerized crime data analysis and in recent years they invest some resources to improve the architecture of recording crime data. However, it will definitely take them some years until they will be ready (both technically and “morally”) to provide detailed, yet anonymous data to an external subject.
Before creating version 3.0 of the application we have to deal with the quality of data that can be provided. We are facing the fact that crime data is only part of the description of the security situation in the Czech Republic. To have a better picture of it we need to add data about offences against public order – there is no central database for that. We plan to make an agreement with several towns on provision of such data and we may be able to add several local case studies into the application in year or two. Also we plan to animate the “crime data community” in order to define some sort of request towards authorities about changes in crime and offence data recording for its future use.
JK: Overall, how long did this visualisation take to develop and what sized team were working on it? Was it all done in house or were you working with outside contractors?
MT: Otevřená společnost is mainly a police watchdog and think tank. We are not programmers or web-designers so we had to hire an external contractor. We were very lucky to choose a great cartography/programming/design studio Geographics.cz. Without the enthusiasm of the team – namely Zdenek Hynek and Martin Pulicar – we would never have got so much from the application in any of its aspects. The simplicity of its control, the intuitiveness of data display and the overall design is of much their credit. In the end, the team stabilized at four people – apart from the guys named above and myself one more colleague was helping us with additional data gathering and analysis.
We started in December 2011 with financial support of Open Society Foundations and we were able to deliver the first version after a year in December 2012. The recent version of the application was launched at the end of November 2013. So, it took two years for the application to get where it is now. Of course things could go faster, but this wasn't the only task for any of us on the team.
If you're curious to find out more from the perspective of the Geographics team, Zdenek has also presented on some of the challenges they faced in this video below.
JK: What has been the public response to this visualisation? Who is using it and for what purposes? Do you think it's helping hold the police to account for their job? What about policy-makers?
MT: Generally we are quite satisfied with the response. The main online and print newspaper, MF Dnes, promoted the new version of the application and thanks to it we had nearly 100,000 unique visits at the website on the first two days after its relaunch. On average, we have about 350 unique visitors daily which is not bad regarding the fact that it is a static application updated monthly. So we can see that people are using it and that there is public demand for the data.
We know that college students are using the data for their schoolwork, journalists are exploring it when reporting about crime and municipalities are getting used to work with the information when forming their crime prevention strategies. We can say therefore that the application delivered the goods we were expecting.
JK: Has this project helped in any ways to raise the profile of Otevřená společnost? Does the organisation see it as an important investment? Will it continue to support and update this visualisation? Or is it still too early to say?
MT: Definitely it helped a lot. The application helped us to become recognized all over the country, especially among municipality officials, which helps them to identify Otevřená společnost when we communicate to them. It improved our fundraising opportunities as well.
The application is an important incentive for other areas of our interest and gives us new resource for our advocacy activities. However the strongest benefit it can bring is a mass of informed citizens. Therefore we need to continue teaching them to use it and raising their awareness of police efficiency.