If you’re not familiar with the term, big data describes data sets that are so large or complex that traditional data processing applications can’t effectively deal with them. The reasons for the inability to deal with Big Data include storage, capture, analysis and visualization of the data. In fact, it has been suggested that big data may transform the way we do business, and impact on society as a whole. Bearing this in mind, big data can provide a useful tool for start-up organizations.
While many organizations (and even governments) have been working on processes to capture, store and analyze huge volumes of data, many of these organizations haven’t realized (or fully understood) the value of the data they are capturing, and as a result are not gaining significant value from their investment in the cost of data collection and analysis.
Some pundits have suggested that big data may take the place of market research, as organizations will no longer have to contract research consultancies to prepare, collate, analyze and report on surveys. The real value of big data can be realized in conjunction with effective marketing research. Big data can be captured in-house, based on all the available transactions and details of your existing customers, and when used in this way it can provide a tool for analysis of what your current customers are purchasing (or doing, watching, etc. subject to your market).
You can use the resulting data to determine trends, the effectiveness of promotion, and other similar valuable tools for optimizing your organization’s ability to engage with current customers. There are some excellent examples of how organizations such as Uber and Netflix have used big data to understand their customers, predict demand for services, and develop a service offering to meet their customers’ needs. The big data set is also used to help determine factors such as the perceived value (and related pricing structure) for services provided. Uber has developed a pricing model based around specific demands identified through big data, known as ‘surge pricing’.
Bearing this in mind, big data cannot provide details regarding how your non-customers may engage with your goods or services. There are big data sources that can provide non-customer insights. These are generally in the social media space, where your products or services are up for open discussion and review in a peer-to-peer communication forum. Unfortunately, the data captured in social media platforms is owned by the platform providers, and there are costs involved in accessing the data.
There will always be a role for market research, either in supporting, or providing more specific detail around issues faced within an organizations. Big data may identify factors that require additional research, such as determining why specific demographic groups are (or aren’t) engaging with your products or services, or identifying gaps in what the existing customers would like to see you provide. By understanding the role of big data, you may well find that your marketing research budget doesn’t change, but that the results provide a more effective and nuanced understanding of how to improve your organization’s competitive edge.