Conferences

Confluent is proud to participate in the following conferences, trade shows and meetups.

BrightTALK Summit: Events as a Global API

Speaker: Ben Stopford, Technologist, Office of the CTO, Confluent

Are events the new API? Event driven systems provide some unique properties, particularly for microservice architectures, as they can be used both for notification as well as for state transfer. This lets systems run in a broad range of use cases that cross geographies, clouds and devices. In this talk we will look at what event driven systems are; how they provide a unique contract for services to communicate and share data and how stream processing tools can be used to simplify the interaction between different services, be they closely coupled or largely disconnected.

Ben is a technologist working in the Office of the CTO at Confluent Inc. (the company behind Apache Kafka®). He’s worked on a wide range of projects, from implementing the latest version of Kafka’s replication protocol through to developing strategies for streaming applications. Before Confluent, Ben led the design and build of a company-wide data platform for a large investment bank. His earlier career spanned a variety of projects at Thoughtworks and UK-based enterprise companies. He is the author of the book Designing Event Driven Systems, O’Reilly, 2018.

Register

Leeds Devops Meetup

18:45 - 19:15 Doors open, chat and make some new devops friends
19:15 - 19:25 Group updates and news
19:25 - 20:05 Look Ma, No Code! Apache Kafka® and KSQL
20:05 - 20:20 Intermission
20:20 - 21:00 Ansible Enterprise Grade
21:00 Decamp to the Wardrobe

Speaker: Robin Moffatt, Partner Technology Evangelist, EMEA, ConfluentSession: Look Ma, No Code! Building Streaming Data Pipelines with Apache Kafka® and KSQL

Abstract: Have you ever thought that you needed to be a programmer to do stream processing and build streaming data pipelines? Think again! Companies new and old are all recognising the importance of a low-latency, scalable, fault-tolerant data backbone, in the form of the Apache Kafka® streaming platform. With Kafka, developers can integrate multiple sources and systems, which enables low latency analytics, event driven architectures and the population of multiple downstream systems. These data pipelines can be built using configuration alone. In this talk, we'll see how easy it is to stream data from a database such as MySQL into Kafka using the Kafka Connect API. In addition, we'll use KSQL to filter, aggregate and join it to other data, and then stream this from Kafka out into multiple targets such as Elasticsearch and MySQL. All of this can be accomplished without a single line of code!

About Robin: Robin is a Partner Technology Evangelist at Confluent, the company founded by the creators of Apache Kafka, as well as an Oracle ACE Director and Developer Champion. His career has always involved data, from the old worlds of COBOL and DB2, through the worlds of Oracle and Hadoop, and into the current world with Kafka.

Speaker: Phil Cornelius, Ansible Specialist, EMEA, Red HatSession: Ansible Enterprise Grade

Abstract: As you introduce Ansible into your organisation there are additional requirements to make running Ansible ‘Enterprise Grade’. This session is a live demo of some of key use cases for Ansible in the Enterprise. You will get to see Ansible Tower with a specific focus on Application Lifecycle Management.

About Ansible: He joined Red Hat in 2016 from Credit Suisse where he was responsible for the developer tools and services for 3000 applications, developed by 8000 developers globally. Phil's background is primarily application development, specifically Enterprise Java. Phil brings over 18 years of experience in what is now commonly called DevOps.

Register

Java With The Best

Speaker: Kai Waehner, Technology Evangelist, ConfluentSession: KSQL - An Open Source SQL Streaming Engine for Apache Kafka®

The rapidly expanding world of stream processing can be daunting, with new concepts such as various types of time semantics, windowed aggregates, changelogs, and programming frameworks to master. KSQL is an open-source, Apache 2.0 licensed streaming SQL engine on top of Apache Kafka which aims to simplify all this and make stream processing available to everyone. Benefits of using KSQL include: No coding required; no additional analytics cluster needed; streams and tables as first-class constructs; access to the rich Kafka ecosystem.

Event Details

Confluent & PAYBACK: Apache Kafka, KSQL and Spark

6:30 pm Doors open
6:30 pm - 7:15 pm That's the Point: Lessons Learned of Operating a Real-World Spark Streaming / Kafka application on Hadoop - Rostislaw Krassow
7:15 pm - 8:00 pm KSQL--Streaming SQL for Apache Kafka® - Matthias J. Sax
8:00 pm - 8:45 pm Pizza, Drinks, Networking and Additional Q&A

Speaker: Rostislaw Krassow, Big Data Engineer, PAYBACKSession: That's the Point: Lessons Learned of Operating a Real-World Spark Streaming / Kafka Application on Hadoop

Abstract: As leading marketing platform PAYBACK must be able to provide every customer the individual offer at the right time. In order to process any business transaction or external event a new streaming based infrastructure was set up based on Kafka as data pipeline and Spark Streaming as processing layer. This presentation covers experience with Spark Streaming on YARN after year of productive usage. Especially non-functional aspects like security, resource allocation and isolation, high availability and scalability will be discussed.

About Rostislaw: Rostislaw Krassow works as big data engineer at PAYBACK. His responsibilities include design, implementation and operation of the Hadoop- and Kafka-based big data infrastructure.

Speaker: Matthias J. Sax, Software Engineer, ConfluentSession: KSQL--Streaming SQL for Apache Kafka

Abstract: This talk is about KSQL, an open source streaming SQL engine for Apache Kafka®. KSQL aims to make stream processing available to everybody without the need to write Java or Scala code. Streaming SQL makes it easy to get started with a wide-range of stream processing applications such as real-time ETL, sessionization, monitoring and alerting, or fraud detection. We will give a general introduction to KSQL covering its SQL dialect, core concepts, and architecture including some technical deep-dives how it works under the hood.

About Matthias: Matthias is an Apache Kafka committer working as a Software Engineer at Confluent. His main focus is Kafka’s Streams API and stream processing with KSQL. Prior to Confluent, he was a PhD student at Humboldt-University of Berlin, conducting research on data stream processing system. Matthias is also a committer at Apache Flink and Apache Storm.

Register

Devoxx France

Event Details
Speaker: Florent Ramière
Session: Beyond the Brokers: A Tour of the Kafka Environment (Workshop)18 April, 17:55 - 18:25

Abstract: Apache Kafka® is not just about brokers. There is an open-source ecosystem that revolves around it. I'll help you discover the main components, such as Kafka Streams, KSQL, Kafka Connect, Proxy Rest, Schema Registry, MirrorMaker, Interceptors, etc.

About Florent: Florent Ramière has more than twenty years of experience in software development and IT project management. He spent several years in the United States at a software company. On his return to France after a detour at Capgemini, he co-founded the company Jaxio. After more than 10 years of entrepreneurship, Florent joined the company Confluent the company behind Apache Kafka where he accompanies large accounts in Europe.

Session Details
Speaker: Gwen Shapira, Principal Data Architect, Confluent
Speaker: Xavier Léauté
Session: Monitor Kafka Like a Pro19 April, 12:55 - 13:40

Abstract: Every business has a central nervous system through which all information flows and around which all decisions are made. Sometimes this system is ad-hoc and non-standard, resulting in an architecture that is difficult to reason about and even harder to keep running.

Kafka operators need to provide guarantees to the business that Kafka is working properly and delivering data in real time, and they need to identify and triage problems so they can solve them before end users notice them. This elevates the importance of Kafka monitoring from a nice-to-have to an operational necessity.

In this talk, Kafka operations experts Xavier Léauté and Gwen Shapira share their best practices for monitoring Kafka and the streams of events flowing through it. How to detect duplicates, catch buggy clients, and triage performance issues — in short, how to keep the business’s central nervous system healthy and humming along, all like a Kafka pro.

About Gwen: Gwen has 15 years of experience working with code and customers to build scalable data architectures, integrating relational and big data technologies. Gwen is the author of Kafka - The Definitive Guide and Hadoop Application Architectures, and a frequent presenter at industry conferences. Gwen is a PMC member on the Apache Kafka® project and committer on Apache Sqoop. When Gwen isn't building data pipelines or thinking up new features, you can find her pedaling on her bike exploring the roads and trails of California, and beyond.

About Xavier: One of the first engineers to the Confluent team, Xavier is responsible for analytics infrastructure, including real-time analytics in KafkaStreams. He was previously a quantitative researcher at BlackRock, Barclays Global Investors and MSCI. He holds graduate degrees from ɉcole Centrale Paris and Cornell University.

Session Details
Speaker: Gwen ShapiraSession: Stream All Things - Patterns of Modern Data Integration20 April, 11:15 - 12:00

Abstract: Eighty percent of the time in every project is spent on data integration: getting the data you want the way you want it. This problem remains challenging despite 40 years of attempts to solve it. We want a reliable, low latency system that can handle varied data from wide range of data management systems. We want a solution that is easy to manage and easy to scale. Is it too much to ask?

In this presentation, we’ll discuss the basic challenges of data integration and introduce design and architecture patterns that are used to tackle these challenges. We will explore how these patterns can be implemented using Apache Kafka® and share pragmatic solutions that many engineering organizations used to build fast, scalable and manageable data pipelines.

Session Details
Event Details

Top 10 KSQL FAQs

Speaker: Nick Dearden, Director of Engineering, Confluent
Speaker: Tim Berglund, Senior Director of Developer Experience, Confluent
Speaker: Hojjat Jafarpour, KSQL Project Lead, Confluent
10:00 am - 11:00 am PT | 1:00 - 2:00 pm ET

KSQL, recently announced as generally available, is the streaming SQL engine for Apache Kafka® that’s easier than Java, highly intuitive compared to other stream processing solutions and more accessible for developers and data engineers.

In this interactive discussion, the KSQL team will answer 10 of the toughest, most frequently asked questions about KSQL. These range from technical examples of managing streaming topics to practical applications and common use cases, such as market basket pattern identification and network monitoring patterns.

You can help inform the discussion beforehand by submitting questions when you register. We will also provide a form to submit questions in your registration confirmation email if you have more questions later on. All inquiries will be considered by the panel.

We will also take live questions throughout the entire session.

Register

Meet the Experts: Data-Driven Day with Tim Berglund at codecentric AG

Register
Speaker: Tim Berglund, Senior Director of Developer Experience, ConfluentSession: The Database Unbundled: Commit Logs in Age of Microservices 09:45 - 10:45

In a queue, only a few hundred microseconds later, the first thing you find is a commit log: mutations enter the database, and they are stored as immutable events in a queue various views that the data model demands. These views can be quite handy-graphs, documents, triples, tables-but they are always derived from a stream of changes. Each one of the centralized schema, only a few thousands of microseconds later in the process, is used as a suite of microservices demands. These views can be handy, but they are always derived interpretations of a centralized database.

Wait a minute. It seems like we are repeating ourselves.

Microservice architectures provide a robust challenge to the traditional centralized database. In this talk, we'll explore the notion of unbundling that database, and putting a distributed commit at the center of our information architecture. As events impinge on our system, we store them in a durable, immutable log (happily provided by Apache Kafka®), Allowing each microservice to create a derived view of the data according to the needs of its clients. Event-based integration avoids the now-well-known problems of REST and database-based service integration, and allow the information of the future architecture of the processing system Kafka.

Speaker: Tim Berglund, Senior Director of Developer Experience, ConfluentSession: Streaming Data with Apache Kafka®13:15 - 14:30

When it comes to choosing a distributed messaging system, Apache Kafka. But how about when you're on the hook to choose a world-class, horizontally scalable stream data processing system? When you need not just publish and subscribe messaging, but also long-term storage, a flexible integration framework, and a means of deploying real-time stream processing applications at scale without having to compromise? The answer is still Apache Kafka.

In this talk, we'll make a rapid-fire review of the breadth of Kafka as a streaming data platform. We'll look at its internal architecture, including how it partitions messaging workloads in a fault-tolerant way. We'll learn how it provides message durability. We'll look at its approach to pub / sub messaging. Kafka Connect provides code-free, scalable, fault-tolerant integration, and how the Streams API provides a complete framework for computation across all the streaming data in your cluster.

Speaker: Tim Berglund, Senior Director of Developer Experience, ConfluentSession: Processing Streaming Data with KSQL16:00 - 16:45

Apache Kafka® is a de facto standard streaming data processing platform, which is widely used as a messaging system, and has a robust data integration framework (Kafka Connect) and stream processing API (Kafka Streams) to meet the needs that common attend real-time message processing. But there's more!

Kafka now offers KSQL, a declarative, SQL-like stream processing language that lets you define powerful stream-processing applications easily. What once took some moderately sophisticated Java code can now be used at the command line with a familiar and eminently approachable syntax. Come to talk about KSQL with live coding on live streaming data.

Register

Apache Kafka® Meetup in Beijing

Speaker: Jun Rao, Co-founder, ConfluentSession: Apache Kafka® Past, Present and Future

In 2010, LinkedIn started developing Kafka. In 2011, Kafka became an Apache open source project. Since then, the use of Kafka has grown rapidly across a variety of businesses. Currently, more than 30% of Fortune 500 companies are already using Kafka. This lecture will first introduce why Kafka initially became a very popular release subscription messaging system. Then we will introduce Kafka recently added enterprise-class features. Finally, we will introduce how to build Kafka into a real-time streaming data processing platform in the future.

Jun Rao is Co-founder of Confluent, which provides a streaming data platform based on Apache Kafka. Prior to founding Confluent, he led the development of Kafka at LinkedIn. Jun Rao is the Apache Kafka Project Board Chairman (PMC chair) and a contributor to Apache Cassandra.

Register

Apache Kafka® Shenzhen Meetup

Register

13:30 - 14:00 Admission / opening
14:00 - 14:40 Past, Present and Future of Apache Kafka® - Jun Rao
14:40 - 15:20 Kafka Cluster Automation Operation Practice - Erhui Yan
15:20 - 16:00 Design and Implementation of Kafka Platform Based on Kubernetes - Yuqing Bai
16:00 - 16:20 Coffee break
16:20 - 17:00 Application of CKafka in Face Recognition PASS - Xinyu Chen
17:00 - 17:40 Kafka Performance Demystification and Optimization - Yanfei Yan
17:40 - 17:50 Take a photo with a souvenir!

Speaker: Jun Rao, Co-founder, ConfluentSession: Past, Present and Future of Apache Kafka®

In 2010, LinkedIn started to develop Apache Kafka®. In 2011, Kafka became the Apache open source project. Since then, the use of Kafka has grown rapidly in various companies. At present, more than 30% of the world's top 500 companies are already using Kafka. This lecture will first introduce why Kafka became a very popular publish and subscribe message system. Next, we will introduce Kafka's recent increase in enterprise-level features. Finally, we will introduce how to build Kafka into a real-time streaming data processing platform in the future.

Jun is a co-founder of Confluent. The main business of the company is based on Apache Kafka® provides streaming data platform. Prior to Confluent, he led the development of Kafka at LinkedIn. Jun Rao is chairman of the Apache Kafka Project Committee (PMC chair) and a contributor to Apache Cassandra.

Speaker: Erhui Yan, TencentSession: The Kafka Cluster Automation Operation

The advent of the era of big data has made the use of Apache Kafka® far more than a messaging channel. The Kafka cluster deployed on Tencent Cloud processes more than trillions of messages per day and is still growing at a high rate. Such a large-scale Kafka cluster will face the removal of faulty nodes, the joining of new nodes, the skew of data between nodes, the dynamic lifting and lowering of Kafka instances required by users, the uneven utilization of Borker resources, and the migration of data between different regions. All kinds of requirements, how to detect abnormalities efficiently and intelligently, generate repair tasks and automatically execute them, avoiding completely relying on the maintenance of human resources for operation and maintenance personnel is something Tencent middleware team must consider. This article will focus on how Tencent Cloud CKafka solves problems. The above problems achieve large-scale automated operations.

Erhui joined the Tencent Infrastructure Department in 2012, Tencent middleware team responsible for R & D.

Speaker: Yuqing Bai, Technical Middle Engineer, , Zhihui Session: Design and Implementation of Kafka Platform Based on Kubernetes

Apache Kafka® is the core infrastructure component that knows the technology platform and hosts information and log services. With the continuous development of the business, higher requirements are placed on the stability and ease of use of Kafka clusters, resource isolation, automatic operation and maintenance, and capacity management. We use Kubernetes and container technology to build the Kafka platform and practice scalable stateful services in the Kubernetes scenario. The platform provides high availability solutions from the client to the server through container technology, service registration discovery, monitoring and alerting, and custom client.

This presentation will introduce the knowledge and experience of Kafka platform in Kubernetes stateful services, cluster management, automated operation and maintenance and high availability.

Yuqing is a technical middle and Taiwanese engineer with a deep understanding of technologies such as Kafka and containers.

Speaker: Xinyu Chen, Shen Hitomi Team Engineer, Ge Ling Shen HitomiSession: Application of CKafka in Face Recognition PASS

Apache Kafka® is a key basic component in deep pupil cloud data flow processing, and is the core of the picture flow real-time face recognition platform. The large-scale image face recognition service of Gelling Shenzhen Hitomi uses CKafka optimized by Tencent Cloud. It applies to business data import, cache, output and monitoring and log collection. It links various data processing modules to real-time people. Face features extraction, comparison and analysis provide protection. This time introduces the position and importance of Kafka in the deep learning face recognition platform architecture from the application level, and introduces the application and problems encountered by some Kafka ecosystem components such as Connect.

Xinyu has a deeper understanding and experience in the use of Kafka and Kafka in the application of artificial intelligence service platform.

Speaker: Yanfei YanSession: Kafka Performance Demystification and Optimization

Apache Kafka® benefits from its distributed, high-performance, low-latency, data-ordering and more, and is increasingly selected as the core data pipeline and storage for streaming computing components. Currently, Kafka processing information deployed on the Tencent cloud has reached trillions of dollars. In the long-term continuous operation of Kafka, Tencent middleware team optimizes the technical details of Kafka according to the actual business conditions, and maximizes resource utilization. This time we focused on the key points of Kafka's high performance and the optimization points of the Tencent Cloud Kafka team.

Yanfei joined the Infrastructure Department in 2014 and is a Tencent Cloud senior engineer mainly engaged in middleware related product development. Yanfei is now the Head of Tencent Cloud CKafka R & D.

Register

Kafka Summit London

Discover the World of Streaming Data

As streaming platforms become central to data strategies, companies both small and large are re-thinking their architecture with real-time context at the forefront. Monoliths are evolving into Microservices. Datacenters are moving to the cloud. What was once a ‘batch’ mindset is quickly being replaced with stream processing as the demands of the business impose more and more real-time requirements on developers and architects.

This revolution is transforming industries.

What started at companies like LinkedIn, Uber, Netflix and Yelp has made its way to countless others in a variety of sectors. Today, thousands of companies across the globe build their businesses on top of Apache Kafka®. The developers responsible for this revolution need a place to share their experiences on this journey.

Kafka Summit is the premier event for data architects, engineers, devops professionals, and developers who want to learn about streaming data. It brings the Apache Kafka community together to share best practices, write code, and discuss the future of streaming technologies.

Welcome to Kafka Summit London!

Event Details

Meetup: KSQL and Stream All Things

Register

18:00 Doors open
18:10 pm - 18:55 KSQL--Streaming SQL for Apache Kafka® - Matthias J. Sax
18:55 - 19:35 Stream All Things - Patterns of Modern Data Integration - Gwen Shapira
19:35 - 20:00 Pizza, drinks, networking and additional Q&A

Speaker: Matthias J. Sax, Software Engineer, ConfluentSession: KSQL--Streaming SQL for Apache Kafka®

Abstract: This talk is about KSQL, an open source streaming SQL engine for Apache Kafka®. KSQL aims to make stream processing available to everybody without the need to write Java or Scala code. Streaming SQL makes it easy to get started with a wide-range of stream processing applications such as real-time ETL, sessionization, monitoring and alerting, or fraud detection. We will give a general introduction to KSQL covering its SQL dialect, core concepts, and architecture including some technical deep-dives how it works under the hood.

About Matthias: Matthias is an Apache Kafka® committer working as a Software Engineer at Confluent. His main focus is Kafka’s Streams API and stream processing with KSQL. Prior to Confluent, he was a PhD student at Humboldt-University of Berlin, conducting research on data stream processing system. Matthias is also a committer at Apache Flink and Apache Storm.

Speaker: Gwen Shapira, Principal Data Architect, ConfluentSession: Stream All Things - Patterns of Modern Data Integration

Abstract: 80% of the time in every project is spent on data integration: getting the data you want the way you want it. This problem remains challenging despite 40 years of attempts to solve it. We want a reliable, low latency system that can handle varied data from wide range of data management systems. We want a solution that is easy to manage and easy to scale. Is it too much to ask?

In this presentation, we’ll discuss the basic challenges of data integration and introduce design and architecture patterns that are used to tackle these challenges. We will explore how these patterns can be implemented using Apache Kafka® and share pragmatic solutions that many engineering organizations used to build fast, scalable and manageable data pipelines.

About Gwen: Gwen Shapira is a principal data architect at Confluent, where she helps customers achieve success with their Apache Kafka implementation. She has 15 years of experience working with code and customers to build scalable data architectures, integrating relational and big data technologies. Gwen currently specializes in building real-time reliable data-processing pipelines using Apache Kafka. Gwen is an Oracle Ace Director, the coauthor of Hadoop Application Architectures, and a frequent presenter at industry conferences. She is also a committer on Apache Kafka and Apache Sqoop. When Gwen isn’t coding or building data pipelines, you can find her pedaling her bike, exploring the roads and trails of California and beyond.

Register

CGI Know-how Day

Seit 16 Jahren bietet der CGI Know-how Day Ihnen eine einzigartige Plattform zum Informationsaustausch und Netzwerken in angenehmer Atmosphäre. Dies wollen wir gerne fortsetzen.

Der CGI Know-how Day findet dieses Jahr am Donnerstag, 26. April 2018, ab 15:30 Uhr in der KAMEHA SUITE Frankfurt statt.

Auch dieses Jahr wollen wir die Auswirkungen der Digitalen Transformation auf ihr Unternehmen, auf die Arbeitswelt generell sowie jeden Einzelnen von uns näher beleuchten:

    Wie wird sich in Zeiten von Robotik und künstlicher Intelligenz unser Arbeitsleben verändern?
    Was sind die aktuellen und zukünftigen Megatrends der Digitalen Transformation?
    Wie können Sie und ihr gesamtes Unternehmen diese Veränderungen für sich nutzen?

Diese und weitere Fragen möchten unsere CGI-Spezialisten und Partner mit Ihnen diskutieren und Ihnen unter dem Motto „Digital Transformation - Quo Vadis?“ Ansatzpunkte aufzeigen.

Als Keynote Speaker konnten wir Wolfgang Bosbach, Querdenker und ehemaliger Vorsitzender des Innenausschusses des Deutschen Bundestages, gewinnen, der mit seinem Thema „Deutschland in Zeiten der Globalisierung und Digitalisierung“ einen sehr spannenden Vortrag halten wird.

Die Teilnahme am CGI Know-how Day ist für Sie kostenfrei. Da die Teilnehmerzahl begrenzt ist, empfehlen wir eine rechtzeitige Anmeldung.

Wir würden uns freuen, Sie zum CGI Know-how Day begrüßen zu dürfen.

Anmelden

Full Stack Hack

MongoDB, Confluent and Nearform present you a one day Hackathon, Full Stack Hack.

Full Stack Hack is a Hackathon where you get a chance to build a full stack application in one day using the top technology stack for today’s modern web applications specifically MongoDB, Kafka and Node.js. On the 27th of April 80 people will form up into teams of 3-5 people to compete for the prize of Top Full Stack Hack team.

Register

Interop ITX

Speaker: Gwen Shapira, ConfluentSession: Metrics Are Not Enough: Monitoring Apache Kafka®May 4, 10:00 am - 10:50 am

When you are running systems in production, clearly you want to make sure they are up and running at all times. But in a distributed system such as Apache Kafka… what does "up and running" even mean?

Experienced Apache Kafka users know what is important to monitor, which alerts are critical and how to respond to them. They don't just collect metrics - they go the extra mile and use additional tools to validate availability and performance on both the Kafka cluster and their entire data pipelines.

In this presentation we'll discuss best practices of monitoring Apache Kafka. We'll look at which metrics are critical to alert on, which are useful in troubleshooting and what may actually misleading. We'll review a few "worst practices" - common mistakes that you should avoid. We'll then look at what metrics don't tell you - and how to cover those essential gaps.

Gwen is a systems architect at Confluent. She has 15 years of experience working with code and customers to build scalable data architectures, integrating relational and big data technologies. Gwen is the author of "Kafka - The Definitive Guide" and "Hadoop Application Architectures," and a frequent presenter at industry conferences. Gwen is a PMC member on the Apache Kafka project and committer on Apache Sqoop. When Gwen isn't building data pipelines or thinking up new features, you can find her pedaling on her bike exploring the roads and trails of California, and beyond.

Session Details
Event Details

Codemotion Amsterdam

Speaker: Kai Wähner, Technology Evangelist, ConfluentSession: How to Leverage the Apache Kafka® Ecosystem to Productionize Machine Learning15:10 / 15:50

This talk shows how to productionize Machine Learning models in mission-critical and scalable real time applications by leveraging Apache Kafka® as streaming platform. The talk discusses the relation between Machine Learning frameworks such as TensorFlow, DeepLearning4J or H2O and the Apache Kafka ecosystem. A live demo shows how to build a Machine Learning environment leveraging different Kafka components: Kafka messaging and Kafka Connect for data movement, Kafka Streams for model deployment and inference in real time, and KSQL for real time analytics of predictions, accuracy and alerts.

Kai Wähner works as Technology Evangelist at Confluent. Kai’s main area of expertise lies within the fields of Big Data Analytics, Machine Learning, Integration, Microservices, Internet of Things, Stream Processing and Blockchain. He is regular speaker at international conferences such as JavaOne, O’Reilly Software Architecture or ApacheCon, writes articles for professional journals, and shares his experiences with new technologies on his blog.

Event Details

Apache Kafka® Delivers a Single Source of Truth for The New York Times

Speaker: Boerge Svingen, Director of Engineering, The New York Times

Abstract: With 3.2 million paid print and digital subscriptions last year, how did The New York Times remain a leader in an evolving industry that once relied on print? It fundamentally changed its infrastructure at the core to keep up with the new expectations of the digital age and its consumers. Now every piece of content ever published by The New York Times throughout the past 166 years and counting is stored in Apache Kafka®.

Join The New York Times' Director of Engineering Boerge Svingen to learn how the innovative news giant of America transformed the way it sources content while still maintaining searchability, accuracy and accessibility through a variety of applications and services—all through the power of a real-time streaming platform.

In this talk, Boerge will:

    Provide an overview of what the publishing infrastructure used to look like
    Deep dive into the log-based architecture of The New York Times’ Publishing Pipeline
    Explain the schema, monolog and skinny log used for storing articles
    Share challenges and lessons learned
    Answer live questions submitted by the audience

About Boerge: Boerge Svingen was a founder of Fast Search & Transfer (alltheweb.com, FAST ESP). He was later a founder and CTO of Open AdExchange, doing contextual advertising for online news. He is now working on search and backend platforms at the New York Times.

Register

Gluecon

Speaker: Gwen Shapira, Principal Data Architect, ConfluentSession: Kafka and the Service MeshMay 16, 1:30 pm - 2:10 pm

Service Mesh is an infrastructure layer for microservices communication. It abstracts the underlying network details and provides discovery, routing and a variety of other functionality. Apache Kafka® is a distributed streaming platform with pubsub APIs - also often used to provide an abstract communication layer for microservices. In this talk, we’ll discuss the similarities and differences between the communication layer provided by a service mesh and by Apache Kafka. We’ll discuss the different paradigms they help implement - streaming vs request/response, and how to decide which paradigm fits different requirements. We’ll then discuss a few ways to combine them and to use Apache Kafka within a service-mesh architecture. We’ll conclude with thoughts on how Apache Kafka and its ecosystem can evolve to provide some of the functionality available in service mesh implementations...and vice versa.

Event Details

JFrog swampUP

Speaker: Viktor Gamov, Solutions Architect, Confluent
Speaker: Baruch Sadogursky, Developer Advocate, JFrog
Session: Fight Crime with Kafka Streams and Bintray Firehouse APIRoom: 1May 18, 5:15 pm - 6:00 pm

Abstract: Can you find this malicious activity needle in the haystack of events on one of the busiest distribution hubs in the world? Processing the streaming events from Bintray Firehose API with Kafka Streams can give you the superpower to do that. In this session, we will show a real-life example of how using Kafka KSQL to process and parse huge amounts of data to determine a worrying trend that might be a sign of a malicious activity.

About Viktor: Viktor Gamov is a Solution Architect at Confluent, the company behind the popular Apache Kafka® streaming platform. Viktor has comprehensive knowledge and expertise in enterprise application architecture leveraging open source technologies and enjoys helping different organizations build low latency, scalable and highly available distributed systems.

He is a professional conference speaker on Distributed Systems, Java and JS topics, and is a regular at the most prestigious events including SwampUP, JavaOne, Devoxx, OSCON, Qcon and others , blogging and producing podcasts “Razbor Poletov” (in Russian) and DevRelRad.io.

About Baruch: Baruch Sadogursky (a.k.a JBaruch) is the Developer Advocate at JFrog. For a living he hangs out with JFrog’s tech leaders, writes code around the JFrog Platform and its ecosystem and then speaks and blogs about it all. He has been doing this for the last dozen years or so, and enjoys every minute of it.

Event Details

DataXDay

Session: Kafka Beyond the Brokers: Stream Processing and Monitoring
Speaker: Florent Ramière, Technical Account Manager, Confluent

He is a technical account manager for Confluent. His job is to sit with customers and help them succeed with Apache Kafka®, so he knows a thing or two about Kafka.

Event Details

Strata Data London

Speaker: Michael Noll, Product Manager, ConfluentSession: Unlocking the World of Stream Processing with KSQL, the Streaming SQL Engine for Apache Kafka®May 23, 14:05 – 14:45

We introduce KSQL, the open source streaming SQL engine for Apache Kafka. KSQL makes it easy to get started with a wide range of real-time use cases such as monitoring application behavior and infrastructure, detecting anomalies and fraudulent activities in data feeds, and real-time ETL. We cover how to get up and running with KSQL and also explore the under-the-hood details of how it all works.

Michael Noll is a product manager at Confluent, the company founded by the creators of Apache Kafka. Previously, Michael was the technical lead of DNS operator Verisign’s big data platform, where he grew the Hadoop, Kafka, and Storm-based infrastructure from zero to petabyte-sized production clusters spanning multiple data centers—one of the largest big data infrastructures in Europe at the time. He is a well-known tech blogger in the big data community. In his spare time, Michael serves as a technical reviewer for publishers such as Manning and is a frequent speaker at international conferences, including Strata, ApacheCon, and ACM SIGIR. Michael holds a PhD in computer science.

Session Details
Event Details

Building IoT 2018

Speaker: Kai Wähner, Technology Evangelist, ConfluentSession: Process IoT Data with Apache Kafka, KSQL and Machine LearningJune 5, 10:45 - 11:25

IoT devices generate large amounts of data that must be continuously processed and analyzed. Apache Kafka is a highly scalable open-source streaming platform for reading, storing, processing and routing large amounts of data from thousands of IoT devices. KSQL is an open source streaming SQL engine natively based on Apache Kafka to enable stream processing to anyone using simple SQL commands.

This talk, with a health care scenario, shows how Kafka and KSQL can help to continuously perform health checks on patients. A live demo shows how machine learning models - trainers with frameworks such as TensorFlow, DeepLearning4J or H2O - can be deployed in a time-critical and scalable real-time application.

Previous Knowledge
Knowledge of distributed systems and architectures is helpful. Experience with machine learning is helpful, but not mandatory.

Learning Objectives

    Apache Kafka is a streaming platform for reading, storing, processing and forwarding large volumes of data from thousands of IoT devices.
    KSQL allows continuous integration and analysis without external big-data clusters and without writing source code.
    Machine learning models can be easily trained and used in the Apache Kafka environment.

Kai works as a Technology Evangelist at Confluent. The kangaroo areas are Big Data Analytics, Machine Learning / Deep Learning, Messaging, Integration, Microservices, Stream Processing, Internet of Things and Blockchain.

Session Details
Event Details

Velocity Conference

Event Details
Speaker: Gwen Shapira, Principal Data Architect, Confluent
Speaker: Xavier Léauté, Software Engineer, Confluent
Session: Metrics Are Not Enough: Monitoring Apache Kafka®Room: LL21 A/BJune 13, 11:25 am – 12:05 pm

Prerequisite knowledge: Some knowledge of Apache Kafka is important.

Abstract: When you are running systems in production, clearly you want to make sure they are up and running at all times. But in a distributed system such as Apache Kafka… what does “up and running” even mean?

Experienced Apache Kafka users know what is important to monitor, which alerts are critical and how to respond to them. They don’t just collect metrics – they go the extra mile and use additional tools to validate availability and performance on both the Kafka cluster and their entire data pipelines.

In this presentation we’ll discuss best practices of monitoring Apache Kafka. We’ll look at which metrics are critical to alert on, which are useful in troubleshooting and what may actually misleading. We’ll review a few “worst practices” – common mistakes that you should avoid. We’ll then look at what metrics don’t tell you – and how to cover those essential gaps.

About Gwen: Gwen Shapira is a system architect at Confluent, where she helps customers achieve success with their Apache Kafka implementation. She has 15 years of experience working with code and customers to build scalable data architectures, integrating relational and big data technologies. Gwen currently specializes in building real-time reliable data-processing pipelines using Apache Kafka. Gwen is an Oracle Ace Director, the coauthor of Hadoop Application Architectures, and a frequent presenter at industry conferences. She is also a committer on Apache Kafka and Apache Sqoop. When Gwen isn’t coding or building data pipelines, you can find her pedaling her bike, exploring the roads and trails of California and beyond.

About Xavier: One of the first engineers to Confluent team, Xavier is responsible for analytics infrastructure, including real-time analytics in KafkaStreams. He was previously a quantitative researcher at BlackRock. Prior to that, he held various research and analytics roles at Barclays Global Investors and MSCI. He holds an MEng in Operations Research from Cornell University and a Masters in Engineering from École Centrale Paris.

Session Details
Event Details

Big Data Analytics London

We showcase Big Data use cases and techniques that drive the greatest business value. With an emphasis on real-life implementation of Big Data technologies, this practical business forum will provide bold vision from leading innovators across the data-driven spectrum. Join hundreds of C-suite executives, business strategists, data scientists and analytics professionals to leverage the opportunity to harness your data for competitive advantage.

Event Details

AWS Public Sector Summit

On June 20-21, 2018, global leaders from government, education, and nonprofit organizations will come together for the ninth annual AWS Public Sector Summit in Washington, DC. The move to the cloud is unlike any other technology shift in our lifetime. Don’t miss this opportunity to learn how to use the cloud for complex, innovative, and mission-critical projects. With over 100 breakout sessions led by visionaries, experts, and peers, you’ll take home new strategies and tactics for shaping culture, building new skillsets, saving costs, and achieving your mission. Also, check back soon for an opportunity to register for technical bootcamps and workshops on the Summit Pre-day.

Event Details

Scala Days New York

Speaker: Neha Narkhede, Co-founder and CTO, ConfluentSession: Journey to a Real-Time EnterpriseJune 20, 9:00 am - 10:00 am

There is a monumental shift happening in how data powers a company's core business. This shift is about moving away from batch processing and to real-time data. Apache Kafka® was built with the vision to help companies traverse this change and become the central nervous system that makes data available in real time to all the applications that need to use it.

This talk explains how companies are using the concepts of events and streams to transform their business to meet the demands of this digital future and how Apache Kafka serves as the foundation to streaming data applications. You will learn how KSQL, Connect, and the Streams API with Apache Kafka capture the entire scope of what it means to put real time into practice. Neha Narkhede is co-founder and CTO at Confluent, the company behind the popular Apache Kafka streaming platform. Prior to founding Confluent, Neha led streams infrastructure at LinkedIn, where she was responsible for LinkedIn’s streaming infrastructure built on top of Apache Kafka. She is one of the initial authors of Apache Kafka and a committer and PMC member on the project.

Session Details
Event Details

OSCON Open Source Convention

OSCON has been ground zero of the open source movement. In its 20th year, OSCON continues to be the catalyst for innovation and success for companies.

Event Details

Google Cloud Next

Google Cloud Next ’18 is your chance to unlock new opportunities for your business, uplevel your skills, and uncover what’s next for Cloud.

Event Details

Strata Data East

How do you drive business results with data?

Every year thousands of top data scientists, analysts, engineers, and executives converge at Strata Data Conference—the largest gathering of its kind. It's where technologists and decision makers turn data and algorithms into business advantage.

Event Details

SpringOne

Speaker: Neha Narkhede, Co-founder and CTO, Confluent

Neha is the co-founder of Confluent and one of the initial authors of Apache Kafka®. She’s an expert on modern, stream-based data processing.

Event Details

Kafka Summit San Francisco

Discover the World of Streaming Data

As streaming platforms become central to data strategies, companies both small and large are re-thinking their architecture with real-time context at the forefront. Monoliths are evolving into Microservices. Data-centers are moving to the cloud. What was once a ‘batch’ mindset is quickly being replaced with stream processing as the demands of the business impose more and more real-time requirements on developers and architects.

This revolution is transforming industries. What started at companies like LinkedIn, Uber, Netflix and Yelp has made its way to countless others in a variety of sectors. Today, thousands of companies across the globe build their businesses on top of Apache Kafka®. The developers responsible for this revolution need a place to share their experiences on this journey.

Kafka Summit is the premier event for data architects, engineers, devops professionals, and developers who want to learn about streaming data. It brings the Apache Kafka community together to share best practices, write code, and discuss the future of streaming technologies.

Welcome to Kafka Summit San Francisco!

Event Details

Big Data London

Speaker: Jay Kreps, Co-founder and CEO, ConfluentRoom: Keynote Theater13 November, 09:30

Jay Kreps is the CEO of Confluent, Inc., a company backing the popular Apache Kafka® messaging system. Prior to founding Confluent, he was formerly the lead architect for data infrastructure at LinkedIn. He is among the original authors of several open source projects including Project Voldemort (a key-value store). Apache Kafka (a distributed messaging system) and Apache Samza (a stream processing system).

Event Details

Ready to Talk to Us?

Have someone from Confluent contact you.

Contact Us