Thoughtful vs. Big Data - Insights from the experts

For this year's Perspectives evening, we invited three of the world's most innovative data experts (more on each of them here) to share their views on the debate between big data and what we like to call thoughtful data - human data turned into actionable insights. This was the 4th of our biannual Perspectives events, but the first in the US - specifically Mountain View in the heart of Silicon Valley. I know you're probably busy, so in case you don't have 50 minutes to watch the entire evening's debate unfold, feel free to pick the one(s) you're interested in:

Progress without results? (10 mins)

How far have we come with analysing Big Data? Rajat Monga, Engineering Director of TensorFlow at Google, outlines the tremendous progress machines have made in understanding data at scale.

"Intuition can't be scaled" (10 mins)

Where does intuition fit into a Big Data world? Ashu Garg, CEO of Volkscience, discusses the drawbacks of trusting machines with large amounts of data.

"Thick" data in a world of unknowns (17 mins)

What kind of data are algorithms missing? Tricia Wang, co-founder of Sudden Compass and a technology ethnographer, based our evening’s talk on the one she did with TED last year (in the video). She warns of the dangers of ignoring the unknown.

The final debate (22 mins)

Listen to the panel discussion between Tricia, Ashu, Rajat and the moderator, David Hayes. They address questions like "What's your idea of a power team for maximising your use of data?" (00:19); Can thoughtful data fit into current measures of success, or do businesses need to shift their definitions of success? (6:52); "How do you fill the wide gap between intuition and technology?" (8:41); "What is the difference between context and intuition?" (11:38); "How far away are we from machines developing intuition and understanding context?" (13:49); "When can we start to truly trust the judgments made by machines? At what point can we say that we can stop manually going back to validate the data that machines put out?" (15:12); "What do you think happened this summer with Facebook's chatbots who started talking to each other?" (17:53); "A lot of companies are claiming they're using machine learning, but often looking at past patterns is not a good predictor for the future. What developments do you see in algorithms that attempt to address this?" (19:27).

What's your take?

I feel extremely lucky to have met and learned from these amazing speakers first-hand. I would love to hear your own thoughts on this debate. How do you bring thoughtful data into decisions in the office? When do you think it's most valuable? Please add your comments below.