Mykola Gurov

bol.com | the Netherlands

BIO: Mykola is a java backend developer (calls himself full-stack). He has a keen interest in CI/CD, testing, and everything that helps to move faster without breaking too many things. Since 2015 he works at bol.com, one of the biggest online retailers of the Netherlands.

TALK: Testing on Production, Deep Backend Edition

Session level: intermediate

Thorough testing before merging to master is great, but it doesn’t reveal the unknowns. Staging on shared environments tends to be slow, unreliable and costly to support. Why not just learn from the only true environment by conducting safe and efficient experiments?

This talk is based on my experience of increasing the delivery rate within the context of back-end systems of bol.com (one of the biggest online retailers of the Netherlands; logistics and purchasing domains), where correctness is often a bigger concern than performance, and recovery might require a bit more than users hitting the refresh button of their browser. Testing on production is often associated with A/B testing or canary releases, but those aren’t always the best – or even applicable – techniques. We’ll look instead at shadow and dry runs, controlled experiments, survival of the fittest; how to apply these techniques and what to be aware of.

Takeaways

  • Techniques for responsible and safe testing on production e.g. shadow runs
  • Deliberately enabling testing on production actually improves quality
  • Beware the costs of running representative testing environments.