Getting Started with KSQL Recipes
To accelerate your stream processing project, we've pre-built a series of stream processing use cases and KSQL code samples. Much like a recipe, we provide the steps to help get you started.
Mehr erfahrenLevel Up your KSQL
Whether you are brand new to KSQL or ready to take it to production, now you can dive deep on core KSQL concepts, streams and tables, enriching unbounded data and data aggregations, scalability and security configurations, and more.
-
KSQL Introduction
Get an introduction to the concept of stream processing with Apache Kafka and KSQL.
Watch Now -
KSQL and Core Kafka
Learn about relating KSQL to clients, choosing the right API and how KSQL uses Kafka topics.
Watch Now -
KSQL Use Cases
KSQL use cases include data exploration, arbitrary filtering, streaming ETL and more.
Watch Now -
Installing and Running KSQL
Find out how to get KSQL, start the KSQL server and CLI, along with other syntax basics.
Watch Now -
KSQL Streams and Tables
Distinguish a STREAM from a TABLE, and discover how streaming queries are unbounded.
Video ansehen -
Reading Kafka Data from KSQL
Explore Kafka topic data. Create a STREAM or TABLE. Identify fields, metadata and formats.
Video ansehen -
Streaming and Unbounded Data
Stream queries, read topics, discover persistent and non-persistent queries and more.
Video ansehen -
Enriching data with KSQL
Use scalar functions, change field types, filter and merge data and rekey streams with KSQL.
Video ansehen -
Aggregations in KSQL
Review various aggregate functions (e.g., MAX, MIN), windowing and late-arriving data.
Video ansehen -
Taking KSQL to Production
Build a streaming ETL pipeline, scale processing, secure KSQL and monitor KSQL performance.
Watch Now -
INSERT INTO in KSQL
A brief tutorial on how to use INSERT INTO in KSQL.
Watch Now -
STRUCT in KSQL
A brief tutorial on how to use STRUCT (Nested Data) in KSQL.
Watch Now
Anwendungsfälle und Beispiele
Streaming-ETL
Apache Kafka ist eine beliebte Wahl für den Betrieb von Daten-Pipelines. KSQL vereinfacht die Daten-transformation innerhalb der Pipeline, so dass Nachrichten sauber in einem anderen System ankommen können.
CREATE STREAM vip_actions AS
SELECT userid, page, action FROM clickstream c LEFT JOIN users u ON c.userid = u.user_id
WHERE u.level = 'Platinum';
Erkennung von Anomalien
KSQL eignet sich hervorragend zur Erkennung von Mustern und Anomalien bei Echtzeit-Daten. Durch die sofortige Verarbeitung des Streams, wenn Daten eintreffen, können Sie ungewöhnliche Ereignisse sofort erkennen und richtig damit umgehen – mit einer Latenz im Millisekundenbereich.
CREATE TABLE possible_fraud AS
SELECT card_number, count(*)
FROM authorization_attempts
WINDOW TUMBLING (SIZE 5 SECONDS)
GROUP BY card_number
HAVING count(*) > 3;
Monitoring
Dank seiner Fähigkeit, skalierbare geordnete Nachrichten mit Stream-Verarbeitung bereitzustellen, wird Kafka häufig als Lösung für Log-Daten-Überwachung und -Warnungen eingesetzt. KSQL bietet eine vertraute Syntax zum Tracken, Verstehen und Verwalten von Warnungen.
CREATE TABLE error_counts AS
SELECT error_code, count(*) FROM monitoring_stream WINDOW TUMBLING (SIZE 1 MINUTE) WHERE type = 'ERROR' GROUP BY error_code;