As digital product creators we all know the importance of getting feedback from users throughout the lifecycle of our of product - that goes without saying hopefully, however, when it comes to building mobile apps and getting early feedback throughout the lifecycle of your product there are a few approaches to consider at different phases:

1. Pre-launch: low/high fidelity usability prototype testing (Maze.design, UsabilityHub etc.)
2. Alpha/beta launch: Surveys, user interviews, feedback widgets, TestFlight
3. Post launch: Focus groups, surveys, community forums, user interviews, feedback widgets, TestFlight and .....

In this post I will be discussing a new approach I've been watching closely the last few years relating to post-launch testing and community building for mobile apps - however, it should be noted that the majority of what I mention can be used in web apps too.

A sandbox environment for experimenting with new ideas

I've noticed over the last few years a number of consumer based tech startups launching versions of 'Labs' within their products. These lab features essentially allow users to opt-in to gain early access to new and experimental features within the product - using a feature toggle/flag system.

It's basically like running multiple versions of your app in one build.

This feature allows users to activate the new features and improvements before it's considered for a general release.

One UK based Fintech startup I personally use called 'Monzo' has been doing a fantastic job at this since 2018 when they first launched 'Monzo labs' for their iOS and Android customers.

‍

‍

We believe that the best way to build a brilliant product is by releasing features early, and getting feedback from you - Monzo

‍

Monzo Labs access can be found in the settings - it's not glaringly obvious but again it's not for everyone. From Monzo Labs you can view the new features or improvements and decide which ones you want to opt into - it's as simple as that.

‍

‍

Nurturing a community of early adopters

In an attempt to create an open discussion about the new features within Monzo Labs - they create a new topic for each new feature within their community forum allowing users of the labs features to discuss their initial thoughts as a community. By providing this open forum the product and design team at Monzo can join the discussion and interact with users in an effort to shape the new feature based on community feedback.

Further to that, Monzo also created a twitter handle called 'Making Monzo' - which provides yet another medium for users to see new features coming to Labs and allows users to provide their feedback publically via Twitter.

These community based features provide users and evangelists with the ability to shape and influence the product, providing more buy-in from the community of users and subsequently increasing engagement and providing more value to the product with the overall aim to increase retention.

‍

TestFlight vs Labs

As a Product Manager, I find the idea of creating a sandbox environment to test new and early stage ideas and improvements with a group of early adopters super exciting as previous traditional approaches conducted via TestFlight can be cumbersome for designers, PM's, developers and more importantly the beta testers themselves.

TestFlight can work with very early adopters and evangelist - particularly technical users, however, it's tricky explaining to a non-technical user that they need to install TestFlight first and go back to the email and then install the TestFlight build - it's a big ask. Further to that, in my experience, a lot of confusion can be caused by two builds on a users device for those who haven't used TestFlight before.

That's not to say there isn't a need for TestFlight for internal test purposes or at the early stages during an Alpha release or early Beta - but I personally think the user experience of integrated Labs features within the consumer build is a better strategy. Il touch more on this shortly.

‍

‍

The Technology Adoption Life Cycle

Although Geoffrey Moore's 'Crossing The Chasm' model refers to the Technology Adoption Life Cycle, I think it's a great way to breakdown the types of users who would be willing to adopt each approach.

The types of consumers that will buy a product into five categories:

• Innovators seek out novel technology; it’s like a hobby for them. There aren’t many Innovators, and because they’ll try new things, they are important. Other people see them use new things and feel braver themselves about trying them.
• Early Adopters are quick to understand the benefits of new technology. Unlike the Innovators, they don’t love technology for its own sake.
• Early Majority are practical minded consumers. If a product seems useful, this group will try it. The Early Majority are cautious of fads.
• Late Majority consumers wait for something to become well established. They don’t feel confident in their ability to deal with technology.
• Laggards are those consumers who, for personal and/or economic reasons, are not looking to buy new technology.

Personally I think the types of users that would be interested and willing to enroll into TestFlight builds fall into the Innovators and potentially the Early adopter category. These segments know that being first with new technology likely means that there will be glitches and bugs, but they are comfortable with that and their expectations are aligned.

The types of users who would be encouraged to use a Labs feature would include Innovators and Early adopters but also the 'Early Majority'. They favor evolution over revolution; they want things to work smoothly. Lab features are typically more stable and tend to have less problematic bugs than early TestFlight builds.

It should be noted that Monzo also use TestFlight alongside their 'Labs' feature. I would presume the TestFlight builds are for their super users who want to get super early access to new improvements and features with Labs catering to the general consumers who would prefer a more stable experience with fewer bugs.

‍

The tradeoffs to consider

From my personal experience, there is a trade-off that needs to be addressed. Feature flags within a product need to be carefully considered and can be tricky to manage as you continue to add further features. It also adds more time to your builds compared to a simple TestFlight build.

There is still room for  improvement though - in particular, most lab features tend to neglect a smooth and frictionless feedback loop and add unnecessary friction for users to provide feedback. More can be done to reduce this friction and improve the feedback loop with the end-user.

‍

“As with anything in Monzo Labs, this is a work in progress. It’ll be a little rough around the edges and can’t guarantee it’ll be a 100% smooth experience. But your feedback will help make it great before we finally bring it to everyone!” - Monzo

‍

To improve this, I would consider adding action based notifications - first monitoring users who have opted-in and adopted the new feature by creating an automated action based trigger a day after the action has been complete. This could consist of a popup, push notification or email - prompting the user to provide some quick feedback through a short form or widget.

However, if you working on a more mature product with thousands/ millions of users I would consider the approach Monzo took with their Discourse community forum as it removes the asynchronous feedback and allows users to share their feedback with each other. Perhaps a blend of the two could be considered.

Lastly, creating a lab environment isn't exclusive to mobile apps and can be added to any web based platform too using the same principles as I outlined above.

I'm keen to hear of any other strategies or lab features. If you know of any I might have overlooked please get in-touch - I'm always looking to review new examples.

‍