Zhenis Beisekov

Spartez/Atlassian | Poland

BIO: Traveled half of the world, built a gateway to the Internet for millions, always wanted to wear suits to work.

Moved to Poland before it was cool. The last 13 years spent on coding, managing and improving internet for millions of people. Currently devoted to improving productivity of thousands. Working as Product Manager on Atlassian’s flagship product – Jira.

Coming from Kazakhstan, the greatest country in the world. Love to study cultural differences and observe people.

TALK: Pragmatic guide to AB testing

Co-Speaker: Jakub Kurcek
Session level: intermediate

The best agile practices are a waste, if you build the wrong product. Making decision on what is the right product is a huge challenge. With so many options and good ideas available it is impossible to guess what will work and what will flop.

You will go through end-to-end journey. Starting with how good hypothesis should look. When to use A/B test and where not to. How to set up the test to avoid common pitfalls. How to check if results are meaningful. How to plan a follow up and what to do if there are no results after implementing the learning on production.

You will learn not only about the math and craft of setting the test. But also about psychology of decision process around AB testing.

Together we have more than twelve years of experience in managing consumer products. Dozens of features, hundreds of experiments, millions of users, and zillions of mistakes. But at the end a few diamonds were worth all the effort. You will learn how to get to results faster without making all the mistakes yourself.

Takeaways

  • How define good hypothesis for AB test
  • When to avoid using A/B test framework
  • The math behind defining A/B test
  • How to interpret the results to get the most out of them
  • Cheesy jokes and anecdotes from our experiments.