Elastic Scaling in the Streams API in Kafka
This blog post is the first in a series about the Streams API of Apache Kafka, the new stream processing library of the Apache Kafka project, which was introduced in Kafka v0.10. Current […]
Distributed, Real-time Joins and Aggregations on User Activity Events using Kafka Streams
In previous blog posts we introduced Kafka Streams and demonstrated an end-to-end Hello World streaming application that analyzes Wikipedia real-time updates through a combination of Kafka Streams and Kafka Connect. […]
Apache Kafka and Kafka Streams at Berlin Buzzwords
At the beginning of June several Confluent team members attended Berlin Buzzwords 2016, where we gave three talks focused on stream processing and distributed computing. These talks, which we summarize […]
Hello World, Kafka Connect + Kafka Streams
Important: The information in this article is outdated. With recent Kafka versions the integration between Kafka Connect and Kafka Streams as well as KSQL has become much simpler and easier. […]
Introducing Kafka Streams: Stream Processing Made Simple
I’m really excited to announce a major new feature in Apache Kafka v0.10: Kafka’s Streams API. The Streams API, available as a Java library that is part of the official […]
Announcing Kafka Connect: Building large-scale low-latency data pipelines
For a long time, a substantial portion of data processing that companies did ran as big batch jobs — CSV files dumped out of databases, log files collected at the […]
Getting started with Kafka in node.js with the Confluent REST Proxy
Previously, I posted about the Kafka REST Proxy from Confluent, which provides easy access to a Kafka cluster from any language. That post focused on the motivation, low-level examples, and […]
Using logs to build a solid data infrastructure (or: why dual writes are a bad idea)
This is an edited transcript of a talk I gave at the Craft Conference 2015. The video and slides are also available.
How I Learned to Stop Worrying and Love the Schema
The rise in schema-free and document-oriented databases has led some to question the value and necessity of schemas. Schemas, in particular those following the relational model, can seem too restrictive, […]
Real-time stream processing: The next step for Apache Flink
This is a guest blog post written by Kostas Tzoumas and Stephan Ewen from data Artisans. Stream processing is becoming very popular with open source projects like Apache Kafka, Apache […]