Dimitar Dimitrov
Head of Infrastructure, Dext
Dimitar Dimitrov
Head of Infrastructure, Dext
The speaker
Even though he rarely gets to write code lately, Mitio (as many call him) has been programming for many years and since 2005 is getting paid for it. One of the lecturers of the Programming Ruby course at FMI and a co-founder of the Bulgarian edition of Rails Girls, he's been doing all kinds of things in the last decade or two. Mitio has always had a strong interest in anything technical, starting from hardware and going up the "ladder". Since about two years ago, he's also been learning how to be a parent.
Towards the end of 2016 he joined Dext (formerly Receipt Bank) and now has the honour to lead about a dozen brave souls in the Dext infrastructure team. Together, they're trying to keep all of Dext's systems and apps up and running 24/7.
The talk
Takeaways from 15 years of production incidents
As a lead of the Dext infrastructure team, I feel that I carry personal responsibility for both the swift resolution of all production incidents as well as eliminating potential future incidents (ha-ha, impossible). I was on the front line on many incidents and that has taught me a lot. I will share a few of the most memorable incidents and production issues I have encountered and I will follow up with my takeaways for each of them. Besides the story behind each incident, that I hope will be somewhat amusing, the takeaways will revolve around metrics, alerting and observability techniques, debugging and troubleshooting tools and tips, as well as what is our incident response process, including how we communicate and escalate. I will also touch a bit on resilient architectures and will share a few thoughts on rolling out riskier changes to production.