With the widespread adoption of internet services, today's web sites such as AutoScout24 and ImmobilienScout24 face the challenge of handling more and more web traffic at high levels of availability and speed: today's web surfers have little patience with slow or intermittently broken web sites. How does a data infrastructure architecture look like for a marketplace web platform that satisfies current user's demands, and at the same time, can keep up with the pace of current business development?
Using classical, monolithic relational database infrastructures for web-scale traffic entails looming design pitfalls as well as surprising technical challenges to scaling and business logic evolution; Exploring the aftereffects of using relational databases for web-scale data processing will be the first part of this talk.
In the second part, we will discuss an alternative system architecture that addresses the needs mentioned above. It is a variety of a microservice architecture: self-contained services that communicate via asynchronous data streams instead of synchronous, RPC- or REST-style calls. Done properly, this architecture provide very desirable properties, i.e., loose system coupling, simple scaling, low-latency, and high system resilience.
Kafka is the stream data processing backbone at AutoScout24; We will present how Kafka, a distributed, immutable (append-only) commit log, helps at AutoScout24 to create a system that makes both users happy and allows developers to keep up with today's impressive pace of change.
message of 16.01.2017