Master
ThoughtWorks
Menu
Close
  • What we do
    • Go to overview
    • Customer Experience, Product and Design
    • Data Strategy, Engineering and Analytics
    • Digital Transformation and Operations
    • Enterprise Modernization, Platforms and Cloud
  • Who we work with
    • Go to overview
    • Automotive
    • Healthcare
    • Public Sector
    • Cleantech, Energy and Utilities
    • Media and Publishing
    • Retail and E-commerce
    • Financial Services and Insurance
    • Not-for-profit
    • Travel and Transport
  • Insights
    • Go to overview
    • Featured

      • Technology

        An in-depth exploration of enterprise technology and engineering excellence

      • Business

        Keep up to date with the latest business and industry insights for digital leaders

      • Culture

        The place for career-building content and tips, and our view on social justice and inclusivity

    • Digital Publications and Tools

      • Technology Radar

        An opinionated guide to technology frontiers

      • Perspectives

        A publication for digital leaders

      • Digital Fluency Model

        A model for prioritizing the digital capabilities needed to navigate uncertainty

      • Decoder

        The business execs' A-Z guide to technology

    • All Insights

      • Articles

        Expert insights to help your business grow

      • Blogs

        Personal perspectives from ThoughtWorkers around the globe

      • Books

        Explore our extensive library

      • Podcasts

        Captivating conversations on the latest in business and tech

  • Careers
    • Go to overview
    • Application process

      What to expect as you interview with us

    • Grads and career changers

      Start your tech career on the right foot

    • Search jobs

      Find open positions in your region

    • Stay connected

      Sign up for our monthly newsletter

  • About
    • Go to overview
    • Our Purpose
    • Awards and Recognition
    • Diversity and Inclusion
    • Our Leaders
    • Partnerships
    • News
    • Conferences and Events
  • Contact
Global | English
  • United States United States
    English
  • China China
    中文 | English
  • India India
    English
  • Canada Canada
    English
  • Singapore Singapore
    English
  • United Kingdom United Kingdom
    English
  • Australia Australia
    English
  • Germany Germany
    English | Deutsch
  • Brazil Brazil
    English | Português
  • Spain Spain
    English | Español
  • Global Global
    English
Blogs
Select a topic
View all topicsClose
Technology 
Agile Project Management Cloud Continuous Delivery  Data Science & Engineering Defending the Free Internet Evolutionary Architecture Experience Design IoT Languages, Tools & Frameworks Legacy Modernization Machine Learning & Artificial Intelligence Microservices Platforms Security Software Testing Technology Strategy 
Business 
Financial Services Global Health Innovation Retail  Transformation 
Careers 
Career Hacks Diversity & Inclusion Social Change 
Blogs

Topics

Choose a topic
  • Technology
    Technology
  • Technology Overview
  • Agile Project Management
  • Cloud
  • Continuous Delivery
  • Data Science & Engineering
  • Defending the Free Internet
  • Evolutionary Architecture
  • Experience Design
  • IoT
  • Languages, Tools & Frameworks
  • Legacy Modernization
  • Machine Learning & Artificial Intelligence
  • Microservices
  • Platforms
  • Security
  • Software Testing
  • Technology Strategy
  • Business
    Business
  • Business Overview
  • Financial Services
  • Global Health
  • Innovation
  • Retail
  • Transformation
  • Careers
    Careers
  • Careers Overview
  • Career Hacks
  • Diversity & Inclusion
  • Social Change
Data Science & EngineeringTechnology

Big Data? More Like Big Headache

Mark Collin Mark Collin

Published: Oct 15, 2013

Is it just me or is everyone just about fed up with this overused, overhyped term…Big Data? Data has always been big; it just got bigger, so deal with it. We didn't go through a period saying we had big software, or big Internet...so why the obsession with Big Data?

At the Big Data Roadshow in Manchester last week, Dave Elliman, principal technology consultant, and I gave the keynote speech to open up what was an interesting conference. We took the 'big' out of big data and focused the audience back onto some core Agile Analytic approaches, using retail experiences to tell the story.

As a technology community we have an accelerating growth of technologies that can deal with the big data conundrum. But before we can go out and use them or select a new tool or platform, we must make the business folks aware of the possibilities, and work together to agree what ‘building the right thing' means. We are not proposing to throw out the legacy or big data warehouses but rather simply figuring out how to now sweat those assets whilst using newer approaches by the side, on top or around to solve the new challenges of volume, variety and velocity.

The entrepreneurial tech communities are already doing this and our next generation of data scientists and analysts are using completely different tool sets to those currently in place or proposed by big vendors. They are using and creating open source tools to solve the problem. Whether it is YARN, Hadoop, cloud, or massive parallel processing, these communities are finding better, more efficient, faster ways of doing things. 

A Dutch haulage firm provided a good example of how to put a value on data. This company needed to focus on fuel/carbon reduction in order to receive tax breaks so it fitted GPS devices to their several hundred vehicles and ended up with a data set that they sold back to and continue to sell to the government. Brilliant.

Which begs an interesting question, if we were to put a value on our data, could we stick it on the balance sheet?  Now that would get the attention of business leaders. 

Dave and I ended our session by asking the audience to reflect on the following - how much of your business today is run on human intuition versus machine learning? This cross-over will happen at some point in the future, maybe in many more years to come, but we must understand ourselves and where we are on this curve to understand the opportunities in data for our business in the future.

What is your answer?

Learn more about an Agile Analytics engagement.

Master
Privacy policy | Modern Slavery statement | Accessibility
Connect with us
×

WeChat

QR code to ThoughtWorks China WeChat subscription account
© 2021 ThoughtWorks, Inc.