Europe is often regarded as playing somewhat second fiddle to the US when it comes to technology and innovation – with little to show for compared to the dominance of Silicon Valley’s big Internet companies (though China is beginning to catch up) . However there is one area where Europe, and particularly the UK, is leading the way – namely the use and dissemination of Open Data. In this post, I explore what Big Data is, and what are the main benefits and drivers.
UK leads the world
Last week, the World Wide Web Foundation’s Open Data Barometer report put the UK at the top for opening up government data for the social and economic benefit of its citizens, just ahead of the US and Sweden. While the UK government has push a number of digital and open initiatives to achieve this, ‘father of the web’ Tim-Berners-Lee, believes that the UK’s position is as much a reflection of the lack of progress by other, particularly OECD, countries. The UK’s credentials are further burnished by its topping the Global Open Data Index , a crowdsourced tracker of government data initiatives.
Big Data vs Open Data
But what exactly is Open Data, and how does it relate to Big Data? Joel Gurin explains the relationships between three overlapping concepts – Big Data, Open Government and Open Data. Open Data is taken to be all non-personal data that is released by government and non-government organisations to be freely used, interpreted and manipulated by third parties. Examples include scientific data such as from the Human Genome Project, environment data such as weather, seismic, pollution and flooding, geographic data, including addresses, maps, and locations, data from cities and government such as transportation, health data, census statistics, as well as data from private companies, often released to meet regulatory requirements. This data can be ‘big’ or ‘small’, and can exist in several formats – files for downloads, APIs for access.
The Benefits Part 1: Good Governance and Democracy
The primary benefit of Open Data initiatives put forward its numerous champions lies in the value it can bring to the policy-making process and the debate around it. For example, the underlying information underpinning immigration or healthcare policy can be made available to all, making for much more transparent, and hopefully better decision making. The openness of the weekly UK Hospital Accident and Emergency statistics, which are being given similar prominence as football results shows the power such data can have to shaping the debate. The vast amount of data being released brings a proper scientific rigour and discipline to measuring the performance of all layers of government from macro-economic policy to local traffic levels and makes the scrutiny available to everyone, not just a cabal of civil servants. This should not only drive more effective governance, but also provide a transparent environment with fewer shadows for corruption and political abuse to flourish. The arguments on the links or correlation between data openness and democracy remains open. Whilst the usual suspects (US, UK, Scandinavian countries, Australia and Canada) perform well on both counts, some countries, lag significantly behind. Germany, for example has a lower openness score than Russia or China.
The Benefits Part 2: The Economic Argument
In a 2013 report, McKinsey estimated that open data has the potential to unlock $3 trillion in additional economic value across the global education, transportation, consumer products, energy, health care and consumer finance. The $3 trillion figure seems to be designed primarily to attract attention and headlines, but intuitively one can see how economic value can be obtained through being able to better identify inefficiencies in both the public and private sector, give consumers better information to allow them to choose high-performing companies rather than poorly-performing ones, and driving improvements in consumer products through better industry benchmarking.
The Benefits Part 3: Driving Innovation
The area that is most aligned with the topics discussed on this site is the way Open Data creates the opportunities for new services and businesses that would not have been possible previously. In recognition of the fact that government departments are not usually the engines of innovation and creativity, opening up data allows all sorts of players, from students to multinationals to fill in the gap. Currently, there are a multitude of data visualisation sites that operate off open data sources, many of which being student projects. A very good repository of London-centric visualisations can be found at the Mapping London website, which hosts an impressive span of data-driven maps.
Compared to the number of data visualisation sites, the start-up scene seems to be lagging behind, and though is getting pushed along by government assistance. Nevertheless, a number of initiatives are beginning to break through. Spend Network, collates and analyses European and UK prublic procurement, allowing for inefficiencies to be identified. Zoopla, the house buying network makes use of Land Registry information to provide buyers with pricing information, while Mastodon C, a big data company exposed great variations in the prescription of generic and proprietary drugs.
Looking ahead
As we have seen, the Open Data initiative is one of the key trends that is shaping the digitalisation of society. However, simply putting an Excel file on a web site, does not on its own have a transformative effect. The reality is that simply making data open to third parties does not in itself create openness, as only the most numerically-literate among us will be able to make any sense of it. What is required is for the data to be be reliable and machine-readable in the easiest and most autonomous way possible, allowing for companies, individuals, the non-profit sector and government itself to use a host of data analytics and visualisation tools to capture meaning and drive action. Paradoxically, this means that for Open Data platforms to provide people with the best understanding, they must focus on making the data readable by other computers rather than by the end-user.
