A data paradigm that focuses on collecting and rapidly making sense of large amounts of information from a wide variety of sources.
Big data is defined by the ‘three Vs’: volume, variety and velocity. That means handling huge amounts of data, which is being generated quickly from numerous sources. Big data capabilities are key to enabling your organization to monitor everything going on in your markets and your operations and being able to react.
The analysis of large volumes of data, which is arriving into your enterprise rapidly from a wide variety of sources.
Big data can help you better respond to opportunities and threats. It can also enable you to spot areas for innovation.
The data scientists that you’ll need to really drive value are expensive. Many firms have already made significant data investments without generating the expected return.
Many of the digital champions use big data to improve their insights into operations and their effectiveness.
What is it?
Big data is something of a catch-all term. There’s no single point where data becomes ‘big’, but big data is likely to have some of these characteristics: it is being generated in significant volumes; it is coming into your enterprise at velocity; and it is coming from a variety of sources.
Traditionally, organizations derived performance information from their internal business systems — that typically meant the data was highly structured — so it has a prescribed format, perhaps including a customer name, order number and shipping address. It was likely updated every day. So-called big data can arrive continuously and from external as well as internal sources, and is likely to be unstructured. Such data may include call center transcripts, geospatial images, weblogs or video, which are a far cry from the tabular, structured data traditionally used to gain business intelligence from.
These characteristics of big data mean traditional tools, such as relational databases and data warehouses, are ill-suited to handling big data. Instead, enterprises have turned to a new generation of big data tools, such as Hadoop (used to process large datasets), Spark (platform for large-scale SQL, batch processing, stream processing), Cassandra and other NoSQL technologies. Advances in cloud computing have fueled the big data revolution, thanks to cloud’s ease and relative cheapness when it comes to scaling.
What’s in for you?
The allure of big data is that it promises to give you better information to make decisions. It can enable you to reduce the time taken to respond to threats or opportunities. It can also super-charge your ability to experiment and innovate by using vast quantities of data to discover new patterns of customer behavior or to rapidly spot new trends.
Organizations with powerful big data capabilities can track what’s happening internally with their operations and the market at large.
What are the trade offs?
Becoming adept at handling big data isn’t easy. You’ll likely need to employ highly sought after data scientists, which can be expensive.
You also need to consider how suited your organization is to handling big data. For data-driven internet companies, such as Google, Facebook and Netflix, the need for big data capabilities is a given. If you aspire to become a data-driven enterprise, you’ll need some form of big data platform to improve your analytical capabilities.
But many traditional enterprises run on critical legacy systems that were never intended to use big data. In such cases, a move to become more data-driven may well be desirable but it doesn’t necessitate making significant investments in big data immediately.
How is it being used?
Companies such as Google, Facebook and Amazon use their big data capabilities as a core part of their business. They look to gain competitive advantage by their ability to make sense of vast quantities of data.
At SAGE Publishing, its big data platform is being used to transform research in the social sciences. It set up a new unit, SAGE Ocean to develop resources and practical tools that would enable social science researchers to analyze big data sets and draw out new insights.
And as more organizations explore the possibilities of the Internet of Things — where vast arrays of low-cost sensors are used to monitor business-critical systems — the need to be able to handle big data intensifies.
But like many hyped technologies, business leaders will need to understand their own requirements and potential uses, before making significant big data investments.
Related topics
Would you like to suggest a topic to be decoded?
Just leave your email address and we'll be in touch the moment it's ready.