‘Who’s more excited about the future now?’
This was a question asked at the end of the ASC (Association for Survey Computing) AI conference yesterday. As a crude sum, I’d say around 90% of people said yes. Based on everything they’d heard at the conference, yes they were excited about what’s to come. And that gave me a warm, fuzzy feeling.
As an industry – and further afield – there has been a lot of doom and gloom recently, for obvious reasons. Some big brands and agencies have made some big decisions that ultimately filter through to us insight lot. We’re talking tighter budgets (or budgets completely pulled), democratised research that threatens to cut us out of the picture entirely, or AI taking parts of our jobs.
Uniquely human skills
So my ultimate takeaway from yesterday is that there is genuinely some amazing AI stuff solving actual research challenges. But better than that, most talks in some way addressed the elephant in the room – we (humans) are still crucial in the research process. Which for very selfish reasons, was probably my favourite takeout (namely because I have a mortgage to pay, a son to feed and I happen to love what I do).
From quality checking to analysis, there are some incredible AI tools out there (and a dozen more probably launched while I wrote this sentence). But they’re only as good as the humans driving them and interpreting their outputs. So yes, we’ve seen a lot of big names shun whole teams in favour of AI, but AI will only get them so far. Indeed some have even completely backtracked when they realised they ballsed up. Then of course we’ve all heard about the Deloitte AI blunder. And last week, a newspaper printed an article that ended with the notorious AI sign off ‘if you want, I can create an even snappier front page’ (facepalm). A human checking these things would have spotted them a mile away.
The 80/20 rule
So coming back to the conference, we heard about how AI is great for the mundane, and that’s the stuff it should be sticking to. Whereas we (humans) should be doing the brilliant stuff that we all do so well – the 20% as it’s come to be known. All that critical and creative thinking (though there was a debate about how creative AI is/isn’t). Also interpreting data based not just on the masses, but on our own unique understanding. Bringing cultural nuance and making time for outliers, those who have different – yet equally valid – behaviours. So we come to maybe messy, but oh so powerful insights, rather than rely on AI’s slightly biased view of the world.
Champion the human
That’s why it’s important we champion the human using these AI tools. Use great stuff that works, and gets interesting insights but always monitor it and stay as close to the data as you can.
At the conference, my colleague Andrew and I presented an analysis of our latest innovation, Maya (our WhatsApp AI moderator). We ran an A/B test against a traditional survey to see how conversational AI stacks up. When we asked the room at the start if they thought conversational AI could replace surveys, around 75% people said yes. This was quite a shock, in a room full of experienced quanties.
Let’s do hybrid (for now)
So our summary was that Maya was amazing (we’re biased though) and she got the answers we needed with gusto and richness. But she’s not so good with numbers…just yet. She captures the data for say, crisp brands people are eating. Though she can’t quite quantify responses to tell us for example, that 36% people snack on Walkers crisps most often. And this isn’t a Maya issue – this is an LLM issue that people are looking for workarounds for.
So we don’t think conversational AI can replace traditional surveys right now…but in the future and with improvements, that’ll probably change. For now then, we think a hybrid approach is the best use of tech like this. That means a great user experience (95% of people loved speaking to Maya), but with quant backing from a traditional survey.
Funnily enough, I bumped into some colleagues at the conference from my very first grad job. They won’t mind me calling them veteran data types (I hope). But one of them thought my conclusion quite trad. He could absolutely see the value of conversational AI and was 100% adamant it would replace surveys in the future. I guess as someone who’s semi-retired, he can say that, whereas I still have skin the game.
Power to the people
My colleague Hasdeep also did a great talk yesterday about his agentic work aiming to support our whole workforce. And I really love that we’re looking to empower employees with great AI tools, rather than ram them down their throats. If what we heard yesterday is anything to go by, that message will only get stronger and more vital.
*
In the interests of practising what I preach, AI hasn’t touched any part of this blog 😎
Hit me up if you want to talk about Maya. She’s already doing amazing stuff and she only launched yesterday!
