Building an Event-oriented Data Platform with Kafka

Building an Event-oriented Data Platform with Kafka

Watch Video

Kafka Summit 2016 | Operations Track

While we frequently talk about how to build interesting products on top of machine and event data, the reality is that collecting, organizing, providing access to, and managing this data is where most people get stuck. Many organizations understand the use cases around their data – fraud detection, quality of service and technical operations, user behavior analysis, for example – but are not necessarily data infrastructure experts. In this session, we’ll follow the flow of data through an end to end system built to handle tens of terabytes an hour of event-oriented data, providing real time streaming, in-memory, SQL, and batch access to this data. We’ll go into detail on how open source systems such as Hadoop, Kafka, Solr, and Impala/Hive are actually stitched together; describe how and where to perform data transformation and aggregation; provide a simple and pragmatic way of managing event metadata; and talk about how applications built on top of this platform get access to data and extend its functionality.

Attendees will leave this session knowing not just which open source projects go into a system such as this, but how they work together, what tradeoffs and decisions need to be addressed, and how to present a single general purpose data platform to multiple applications. This session should be attended by data infrastructure engineers and architects planning, building, or maintaining similar systems.

Speaker:

Eric Sammer, CTO, Rocana

We use cookies to understand how you use our site and to improve your experience. Click here to learn more or change your cookie settings. By continuing to browse, you agree to our use of cookies.

Agenda

GENERAL SESSIONS
08:00 – 09:00 Registration & Partner Showcase
09:00 – 9:10 Welcome
09:10 – 09:45 Keynote: State of Streaming, what’s next
09:45 – 10:15 Keynote: Customer – Euronext ??
10:15 – 10:45 Partner Session
10:45 – 11:15 Coffee Break & Sponsor Showcase
11:15 – 11:45 Keynote: Customer – Criteo ??
11:45 – 12:15 Partner Session
12:15 – 13:00
Lunch
13:00 – 14:00
14:00 – 14:30 CONFLUENT Monitor Kafka Like a Pro with C3
14:30 – 15:00 Customer Use Case – Credit Mutuel ??
15:00 – 15:45 CONFLUENT KSQL in Production
15:45 – 16:15 Coffee Break & Sponsor Showcase
16:15 – 16:35 CONFLUENT : Global Kafka / Kubernetes
16:35 – 16:55 Partner Session
16:55 – 17:15 Customer Case : BNP Secu ?
17:15 – 19:00 Topic Corners & Get Together