Why it matters
Big Data is a consequence of the growth of digital data on the Internet and the number of objects connected to the Internet (see previous article on Internet of Things).
But when is data big? In essence, it is when traditional computing capabilities (storage, analysis, transfer networks and visualisation) can no longer cope with the quantity, speed, complexity or quality of the data which overwhelms us humans beings. Examples include: email, social data, XML data, videos, audio files, photos, GPS, satellite images, sensor data, spreadsheets, web log data, mobile data, RFID tags and PDF docs. This has called for investment in innovative hardware and software architectures (such as open-standard Hadoop Distributed File System and associated application MapReduce). Big Data is being seized upon by the private sector to improve decision-making and predict future events. For example, using Big Data, telecommunications and transport companies can now better predict customer usage, supermarkets can predict what products will sell and car insurance companies understand how well their customers actually drive.
This is important because the ability to harness the ever-expanding amounts of data is transforming our capacity to understand the world and everything within it.
The advances in analysing Big Data allow us to, for example, decode human DNA in minutes, find cures for cancer, accurately predict human behaviour, foil terrorist attacks, pinpoint marketing efforts and prevent diseases. Big Data is used to better understand customers and their behaviours and preferences getting a more complete picture in order to create predictive models. So what does this mean for defence capabilities?