Project Metamorphosis: Unveiling the next-gen event streaming platformLearn More

Enhance Security with Apache Kafka 2.0 and Confluent Platform 5.0

As customers across verticals like finance, healthcare, state and local government and education adopt Confluent Platform for mission-critical data, security becomes more and more important. In the latest release of our streaming platform, Confluent Platform 5.0, we introduced several security enhancements to help you solve your most difficult security problems. In this article, I will provide deeper technical analysis about the most important security enhancements that are part of the Confluent Platform 5.0 release, including those from Apache Kafka® 2.0.

What you’ve been asking for

Over the past several quarters, we have made major security enhancements to Confluent Platform, which have helped many of you safeguard your business-critical applications. With the latest release, we increased the robustness of our security feature set to help with:

  • Using standard and central directory services like Active Directory (AD)/Lightweight Directory Access Protocol (LDAP)
  • Simplifying the management of access control lists (ACLs)
  • Proactive management and monitoring of security configurations to address the gaps as soon as possible

The following new security features are available in both Confluent Platform 5.0 and Apache Kafka 2.0:

  • Support for ACL-prefixed wildcards to simplify the management of access control
  • Kafka Connect password protection with support for externalizing secrets (to “secrets stores,” etc., like Hashicorp Vault)

The following security features are available only in Confluent Platform 5.0:

  • AD/LDAP group support
  • Feature access controls in Confluent Control Center
  • Viewing of broker configurations in Confluent Control Center, including differences in security configurations between brokers

Let’s walk through each of these enhancements in detail.

AD/LDAP group support in Confluent Platform 5.0

The majority of enterprises standardize on AD for identity-related services. Most people who use directory services would also like to be able to use groups for configuring access controls. In the latest release, we added the capability to use AD and LDAP groups for just this purpose. This simplifies access control management, requiring fewer rules for managing access to groups or organizations. The benefits are:

  • You can manage group ACLs with existing tools and APIs, including kafka-acls and AdminClient.
  • A DENY rule for any of the groups that a principal belongs to will deny access to a resource.
  • An ALLOW rule for any of a principal’s groups will allow access if no DENY rule matches the resource.
  • Users and groups are refreshed periodically, as defined by ldap.authorizer.refresh.interval.ms, enabling the user and group information to keep up to date with the directory server.
  • You can limit the users or groups returned from the LDAP server using the ldap.authorizer.group.search.filter configuration parameter. If you have many users or groups in AD/LDAP, this can be a lifesaver.
  • If you’re using AD, we support authentication using Kerberos and authorization using groups.

You can learn more by checking out the Confluent LDAP Authorizer documentation.

Finding broker security configuration mismatches in Confluent Platform 5.0

Confluent Control Center now gives you the ability to compare the configuration of different brokers in a Kafka cluster and identify any differences between them. This is a great way to see if you have misconfigured brokers that could be contributing to security vulnerabilities.

Finding broker security configuration mismatches in Confluent Platform 5.0

Feature access controls in Control Center

Enterprises need to ensure that end users cannot access sensitive data via Control Center for security and compliance reasons. In order to let administrators control application-wide access to features that reveal topic data, we added the capability to control access to topic inspection, schemas and KSQL queries. When you restrict access to one of these features through the configuration file, Control Center’s user interface will reflect this change upon startup.

Simplify ACL management with support for prefixed ACLs

ACLs do a great job of controlling access to resources like topics and consumer groups, but they can sometimes lack flexibility like any other kind of sophisticated pattern matching. A resource name in an ACL definition is either a complete resource name or a wildcard that matches everything—no middle ground exists. Oftentimes, you prefer more flexibility in setting up ACLs so you don’t end up with a huge list of them. In Confluent Platform 5.0 and Apache Kafka 2.0, we extended the concept of wildcard ACL (*) to support prefixed ACLs. For example:

  • FinanceUser has access to all topics that start with Finance_
  • UserN has access to all consumer groups that start with FraudDetection_

Externalizing secrets for Kafka Connect configurations

Today, Kafka Connect keeps configuration information in clear text, including source and sink credentials. This is not ideal from a security standpoint. At the same time, there are third-party secret stores (like HashiCorp Vault) that provide the ability to protect these passwords at rest and in flight, while also maintaining audit logs, versioning and providing high availability.

Apache Kafka 2.0 and Confluent Platform 5.0 include a new framework in Kafka Connect that lets you integrate your preferred secret store and then use placeholders for secrets in the connector configurations. Kafka Connect will keep the placeholders in its stored configurations and in all REST requests and responses. It will automatically use your plugin to resolve the placeholders into secrets immediately before the configuration is passed to the connector upon startup.

Please note that at this point we do not provide built-in secret store plugins—this feature introduces the framework, not the plugins themselves—but you are now able to implement your own.

Streamlining the management of ACLs for KSQL and Kafka Streams

All of the enhancements we’ve made across the platform in 5.0 have enabled us to make further security improvements in KSQL and Kafka Streams applications.

The combination of ACL wildcards, support for AD/LDAP groups and the new configuration setup simplify the security configuration for KSQL and Kafka Streams, which is very helpful particularly for teams who are deploying and operating KSQL and Kafka Streams applications in secured, multi-tenant production environments.

This enables developers faster time to production by streamlining the process of setting access control on source, internal and sink topics.

Give us feedback

As always, we are happy to hear your feedback. Please post your questions and suggestions to the public Confluent Platform mailing list, join our community Slack channel or contact us directly!

Interested in more?

If you’d like to know more, here are some resources for you:

 

Did you like this blog post? Share it now

Subscribe to the Confluent blog

More Articles Like This

Announcing the Elasticsearch Service Sink Connector for Apache Kafka in Confluent Cloud

We are excited to announce the preview release of the fully managed Elasticsearch Service Sink Connector in Confluent Cloud, our fully managed event streaming service based on Apache Kafka®. Our […]

Announcing the Snowflake Sink Connector for Apache Kafka in Confluent Cloud

We are excited to announce the preview release of the fully managed Snowflake sink connector in Confluent Cloud, our fully managed event streaming service based on Apache Kafka®. Our managed […]

Unifying Streams and State: The Seamless Path to Real-Time

More than ever before, people demand immediacy in every aspect of their lives. Expectations for how we shop, bank, and commute have completely evolved over the last decade. When you […]

Jetzt registrieren

Start your 3-month trial. Get up to $200 off on each of your first 3 Confluent Cloud monthly bills

Nur neue Registrierungen.

Wenn Sie oben auf „registrieren“ klicken, erklären Sie sich damit einverstanden, dass wir Ihre personenbezogenen Daten verarbeiten – gemäß unserer und bin damit einverstanden.

Indem Sie oben auf „Registrieren“ klicken, akzeptieren Sie die Nutzungsbedingungen und den gelegentlichen Erhalt von Marketing-E-Mails von Confluent. Zudem ist Ihnen bekannt, dass wir Ihre personenbezogenen Daten gemäß unserer und bin damit einverstanden.

Auf einem einzigen Kafka Broker unbegrenzt kostenlos verfügbar
i

Die Software ermöglicht die unbegrenzte Nutzung der kommerziellen Funktionen auf einem einzelnen Kafka Broker. Nach dem Hinzufügen eines zweiten Brokers startet automatisch ein 30-tägiger Timer für die kommerziellen Funktionen, der auch durch ein erneutes Herunterstufen auf einen einzigen Broker nicht zurückgesetzt werden kann.

Wählen Sie den Implementierungstyp aus
Manuelle Implementierung
  • tar
  • zip
  • deb
  • rpm
  • docker
oder
Automatische Implementierung
  • kubernetes
  • ansible

Wenn Sie oben auf „kostenlos herunterladen“ klicken, erklären Sie sich damit einverstanden, dass wir Ihre personenbezogenen Daten verarbeiten – gemäß unserer Datenschutzerklärung zu.

Indem Sie oben auf „kostenlos herunterladen“ klicken, akzeptieren Sie die Confluent-Lizenzvertrag und den gelegentlichen Erhalt von Marketing-E-Mails von Confluent. Zudem erklären Sie sich damit einverstanden, dass wir Ihre personenbezogenen Daten gemäß unserer Datenschutzerklärung zu.

Diese Website verwendet Cookies zwecks Verbesserung der Benutzererfahrung sowie zur Analyse der Leistung und des Datenverkehrs auf unserer Website. Des Weiteren teilen wir Informationen über Ihre Nutzung unserer Website mit unseren Social-Media-, Werbe- und Analytics-Partnern.