A small team from the ThoughtWorks Recife office recently built an open data application in response to call for entries by the Ministerio da Justiça of Brazil. The application was awarded second prize in the contest.
The main goal of the app was to expose information related to accidents on federal roads by using data created by the Polícia Rodoviária Federal between 2007 and 2012.
A second goal of the project was to raise awareness among the states about the accidents they had in their roads: e.g. critical times of the day, types of vehicle, any road sections with more accidents, and the gravity of the accidents.
When we started this project the following challenges were laid out:
- Gathering data and generation of information. Starting from the dataset provided for the contest, we faced all sorts of problems dirty data can have. Referential integrity problems, missing data, wrong data, etc. The dataset was massive, so although we ended up losing about 5% of the initial amount of data, it still produced a meaningful set of statistics.
- Usage of maps to show data. The team decided to show maps of the states so it can be easy to see the most accident-prone roads. That created a big challenge:
- Rendering the map using SVG. First problem was where to find the georeferenced open data, then the problem was how to transform it into something the browser could render. We discovered kartograph, a lib that can convert shapefiles into SVG, which every HTML5 browser can render. Then we ended finding out that the National Transit Department (DNIT) has shapefiles that corresponds to every road in the country. So both data for every state were merged and problem solved!
- Mobile devices. The final app should be seen from a mobile device too, either a phone or a tablet. We decided to focus only on the mainstream devices: iOS and Android.
- Deadline. The project had a timeline of a month, so we had to prioritize the main tasks to finish the system.
- People involved. The team had different people along its lifetime. And some people involved were not able to be involved 100% due to their engagement with their projects.
One of the tough calls we had to make was to remove the map when in smaller screens, like smartphones. The rationale we followed was that it would be very difficult for the user to see the whole app, like the roads presented in the maps of these screens.
HTML and CSS Compliance
The markup of the application passed the validation of the W3C organization and can be seen well in browsers that follow the HTML5 specs. The main CSS files we used for the application (main.css, responsive.css) passed most of the validation with just one error.
Data was gathered from the Brazilian Government open data repository (dados.gov.br) and analyzed with the help of MySql. Geospatial data was collected from the National Transit Department (DNIT) and merged with quantitative data with the help of Qgis.
The website is a single page, responsive, client side app using mainly knockout.js and require.js. Charts were drawn with the help of google charts api and chroma.js.
It is an open source application under the AGPLv3 public License.
The technologies used in the project were:
- Google Charts API
- Kartographic.js - Raphael.js
- Semantic Tags
See it in action here: http://analyticsmj.github.io/prf/
And here a few pictures of the app in different devices (picture taken http://modern.ie):