Qualitative data is OK, but without a really strong understanding of social theory and stuff like desirability bias there's a big risk of not asking the right questions and making misinformed decisions.

One way we've found to limit this risk is by supplementing hypotheses developed through qualitative research with a quantitative approach.

So, ask users what they like/dislike about an experience to formulate an idea of what changes to your app may better the experience, BUT make sure to then TEST those changes using a rigorous method (i.e.: experimentation or A/B testing) to validate that the feedback you're hearing is not just noise...

Agree on the premise. I think theres a spectrum of questions, from those that work great in short simple polls:

"What's frustrating you most right now? Level is boring, level is too difficult, loading time is too slow etc

vs ones that require detail framing and context - for example How would you approach building your army.

The former can work very well, but you need to be careful to get good results out of those less concrete with many factors at play.

As long as the message is nog pushed in people's faces and they can voluntarily give their input this could be interesting for us.

We need to ask our users for which stocks they like to trade in our gamified trading app. And it could also help us learn more about our users (have they trade before? Where?).

You need to know very specifically what you want to find out.

Good to see real numbers in these sort of articles..

But let’s face it - users are never going to tell you what they need...or what you should build.

Shouldn’t they have spent more time building a great product first - before launching it? Those retention numbers to me say the product wasn’t working.

I doubt Apple asked any users for input when coming up with the iPhone.

Well we tried hard to build a great app, going through lots of mockups, user testing, research into how grandparents currently communicate with remote relatives etc.. and genuinely thought it could work based on the feedback.

Maybe there was never a product market fit in the first place, but the early feedback was that there was.

But maybe the product wasn't executed well and that's why it failed. Or there was an audience but it was far too niche.

That's the point, we just didn't even know why it failed.

We wouldn't use it to ask the users what to build necessarily (thought there are use cases for that) but more understanding why they DON'T like it...

even that would have given us a clear idea of what to change first and ignore the rest...

Sometimes its as simple as getting lost the 1st time into the app - what button do I press next?

"Onboarding" is critical, and every single step not only Can lose customers but Will lose a certain percentage. Too many steps and your failure rate is compounded. Even losing 10% at each step means losing half after 6 steps.

In an app that requires at least 2 people to work (I work on one of those for a living), I'd guess that the 1st person tries it, finds nobody else online to interact with, and gives up instantly. We solve that by putting an Invite feature prominently in our onboard process. New users connect with someone almost immediately.

Hey Joe agree about onboarding..

Actually our flow included invite by SMS, email etc really early on and was pretty simple (we did a bunch of early usability testing)...but still we got very little connect rate, less than 10% of users. Never really figured out the main reason why, just speculation.

I'd be curious what you are seeing in your case... maybe we could have improved it in other ways.

Actually we get phenomenal conversion. But we vet our customers (Enterprise) and they're already 'converted' when their boss says "We're trying this collaboration tool" so its fish-in-a-barrel.

I work at Sococo. You can try a trial - its free - and see how it works better than I can describe :)

Thanks Joe - I guess when it's top down it's going to get pretty good conversion rates.. appreciate the share :)

I disagree. I’m sure Apple had plenty of user input in the prototype and test phase.. I’m guessing they a big team of engineers building the thing who were the perfect target audience, and probably had lots more to try it too. The idea that you should build an awesome product in isolation and launch it perfectly seems like very bad advice...

We're meant to be the new product guys and come up with the new ideas, asking users to come up with new product suggestions is a recipe for a product that does a lot and not well.

In a past web forum I used to admin the users would ask for a million different things and if we implemented them all it would have been chaos.

At the end of the day someone has to have a creaive vision and see it executed, even if it only does one thing.. but does it really well..

Looking at the site it’s basically using popups in your app or website and we all know how much we like them.

We built something similar for a photo app and our fashion app. Worked great to get details such as what types of content people were looking for. We also headed towards a slightly different path. We had simple polls, but also built in 1 to 1 communications. Essentially like a built in messaging service on top of the polls. Never pushed it to anyone else and only ever used it for our own apps.

That's pretty cool, especially the 1 to 1 communications.

We're thinking about adding that reply and inbox so you can keep it on the conversation going and help users out with problem or ask for more info..

I think Intercom just added something like that for iOS too..

Yes it is popups, you're right about that.. but we tried to take a better approach:

- single question only - max 4 questions - strict character limits - only one question per poll, no long multi-page surveys (we hate those long survey money surveys too) - etc

Also letting game and app devs customize the look and feel with custom graphics and borders etc so it feels native as possible

ie- better fit for mobile apps and games

So I checked out the code on github and and it looks pretty simple, especially the integration part. But to integrate requires developers to clone the codes in our projects. Might be a hassle if updates and fixes roll out after the integration.

Hey - parts of the updates will be done on our server side, so no extra effort for you guys.

For the SDK we’ll only push out new versions with major updates (bunch of new stuff planned!). We’ll keep the community updated and keep backward compatibility. We are also developers and know the pain..

I could have used the type of feedback you're talking about at my failed startup. We found it very difficult to get an honest answer about why we had no traction. Keen to see where this goes.

Yeah, know the feeling. All the early feedback seemed to be really positive talking one on one to people but then in reality people votes with their feets and never returned.

It's funny, we're getting a lot of really useful feedback ourselves (we're a customer of it ourselves) - why they are interested, where they drop out etc

eg- is it pricing, technical issue, perceived lack of value etc

also why they deploy and what value they see out of it

Hey Simon here, one of the co-founders.. Antony and I would love your feedback, especially the areas for improvement

direct link: https://polljoy.com

At the very least, make the popups non-modal and optional.

There is literally NOTHING I hate more in an app than intrusive modal popups stopping me from getting done what I actually WANT to get done. Unless I really love the app otherwise, they usually generate an urge to insta-delete the app.

Yup, the polls default to optional and if you're a game dev you can give some coins or virtual currency to sweeten the deal for the players.

Badly written multiple choice polls can get as low as a 10% response rate, but well written one (which are optional) can get as high as 70%- to the user it can be fun if written well.

What we're learning is that length (brevity is king) plays a bit part of it, as well as speed to load the question.. we had to change our loading order to get better response rates for our users.

Also we recommend to put questions at the points that makes sense... eg while waiting for a game level to load, or after a big activity is complete and it's a natural break point...

Other point is - we typically recommend questions only go to a small subset of the audience - enough to answer the question you're trying to ask. So most users never see a question...

How do you avoid the bias of people saying one thing but doing another? I’m sceptical.

Good question but same thing goes for focus groups but they have a lot of value - if run well.

There is skill to asking questions that are not leading and can deliver useful findings.

Something like this is just a tool - whether it helps or hinders is up to the person using it after all.

I like the idea but then the challenge becomes how do i know what questions to ask?

brilliant. such a head-slapper idea!

Thanks, we actually looked for a tool like this at the time.. couldn't find something that fit (lots on the web but nothing really for mobile that we could customize etc)