Jakub Kurcek

Spartez/Atlassian | Poland

BIO: A corsair of product management, with the right crew on a fast ship can do the impossible.

15 years of experience in design, development, security and product management. Worked with the big and small companies. Built a better home for millions of pop-culture fans with Fandom. Also made Jira faster with Atlassian.

Huge nerd.

TALK: Pragmatic guide to AB testing

Co-Speaker: Zhenis Beisekov
Session level: intermediate

The best agile practices are a waste, if you build the wrong product. Making decision on what is the right product is a huge challenge. With so many options and good ideas available it is impossible to guess what will work and what will flop.

You will go through end-to-end journey. Starting with how good hypothesis should look. When to use A/B test and where not to. How to set up the test to avoid common pitfalls. How to check if results are meaningful. How to plan a follow up and what to do if there are no results after implementing the learning on production.

You will learn not only about the math and craft of setting the test. But also about psychology of decision process around AB testing.

Together we have more than twelve years of experience in managing consumer products. Dozens of features, hundreds of experiments, millions of users, and zillions of mistakes. But at the end a few diamonds were worth all the effort. You will learn how to get to results faster without making all the mistakes yourself.

Takeaways

  • How define good hypothesis for AB test
  • When to avoid using A/B test framework
  • The math behind defining A/B test
  • How to interpret the results to get the most out of them
  • Cheesy jokes and anecdotes from our experiments.