A/B Testing for Decision of scaling or decommissioning an Human Resources product

Introduction

Today, creating a culture where employees genuinely feel valued is more important than ever. But how do we move beyond standard recognition practices to truly understand what matters most to our team members?

At Nubank, innovation and rethinking what’s possible are at the heart of everything we do—and our People team embodies this philosophy too. We’re continuously pushing past traditional approaches, using advanced technology to enhance the way our employees experience work.

This mindset led to the birth of our “Noodles” project, an initiative aimed at revolutionizing how we recognize and appreciate one another.

The Technical Bet: Integrating Recognition into Everyday Tools

The core idea behind “Noodles” was simple: recognition shouldn’t be something extra or forced; it should seamlessly fit into the everyday tools our teams already use. Given how integral Slack is for our daily communication, we decided it was the perfect place to integrate regular peer-to-peer appreciation. Our hypothesis was straightforward: embedding recognition into Slack would naturally lead to more frequent appreciation, and ultimately, employees would feel more recognized.

To validate this, we applied a rigorous A/B testing framework, commonly used in product development, to assess Noodles’ true impact. We split users into two groups (treatment and control), carefully calculated the necessary sample size to ensure our findings were statistically sound, and conducted surveys before and after implementation to track changes in key perception metrics. We closely monitored structured metrics such as our internal Recognition Index, Product-Market Fit (PMF) Score, and user behavior patterns.

Instead of launching Noodles to everyone and holding back only a small control group, we practiced what we call Smart Efficiency—one of Nu’s core values—by identifying the minimum sample size needed to detect meaningful change within our test window. This helped us limit operational overhead like training and surveys, while still collecting reliable, statistically significant insights.

By treating this internal HR product with the same experimental discipline we apply to customer-facing features, we ensured that our decisions wouldn’t rely on anecdotal feedback or early excitement, but on evidence.

Unpacking the Results: Learning from Data

While the “Noodles” experiment showed promising results in some areas (like tool adoption and active users), the data also surfaced unexpected challenges. Despite achieving a 43% Product-Market Fit, our core Recognition Index actually declined among specific groups.

But thanks to our structured A/B testing, we could isolate the effects of the new tool from external influences and dive deeper into understanding these unexpected results. One insight was that “Noodles” resonated more positively with people giving recognition than with those receiving it. In fact, increased usage didn’t always translate to a stronger feeling of being recognized—a critical insight that might have been overlooked without an experimental design.

This reinforced a fundamental principle we believe at Nubank: data, when approached with honesty and scientific rigor, leads to better decisions—even when the outcome challenges our initial assumptions.

Our culture of experimentation and collaboration

Although we ultimately decided to pause “Noodles” in its current form, the experience was invaluable. It illustrates not only our commitment to continuous improvement, but also how experimentation is central to our People strategy. Rather than scale a solution based on incomplete signals, we chose a disciplined route—testing, measuring, and interpreting results with precision.

At Nubank, we approach People & Culture as a technology-driven domain. The Noodles experiment showcased how cross-functional teams—HR, Product, Data Science—can come together to test hypotheses, build MVPs, and make decisions based on real-world impact. This collaborative model reinforces our belief that HR should be as analytical, experimental, and responsive as any other function of our business.

Looking Forward: The Path to Meaningful Recognition

The learnings from Noodles have clarified the path ahead. Rather than scaling a solution that doesn’t fully solve the core issue, we’re shifting towards qualitative exploration to deeply understand the motivations behind the numbers. Combining quantitative experimentation with human insights is essential to creating meaningful employee experiences.

Moving forward, we want to apply this same philosophy to as many People & Culture initiatives as possible: test rigorously, measure results, learn continuously, and adapt accordingly. Whether a tool is built in-house or externally sourced, our approach remains the same: only scale initiatives that demonstrate clear, positive impacts, letting data guide our actions.

Conclusion

Even though “Noodles” didn’t turn out the way we expected, it illustrates Nubank’s dedication to pushing boundaries within our People team. By integrating data, product development strategies, and an iterative approach, we are revolutionizing how we engage and appreciate our people. 

Our goal remains clear: leverage technology thoughtfully to build a meaningful, human-centered employee experience. At Nubank, we are committed to ensuring that every Nubanker feels genuinely seen, recognized, and ready to build the Purple Future.