Skip to content

RANGER-5658: Remove obsolete atlas.kafka.zookeeper.connect requirement from Tag Sync#1037

Open
ramackri wants to merge 2 commits into
masterfrom
RANGER-5658-patch
Open

RANGER-5658: Remove obsolete atlas.kafka.zookeeper.connect requirement from Tag Sync#1037
ramackri wants to merge 2 commits into
masterfrom
RANGER-5658-patch

Conversation

@ramackri

Copy link
Copy Markdown
Contributor

Summary

Remove the legacy atlas.kafka.zookeeper.connect configuration requirement from Ranger Tag Sync's Atlas Kafka source (RANGER-5658).

Tag Sync consumes Atlas entity notifications via kafka-clients using atlas.kafka.bootstrap.servers and atlas.kafka.entities.group.id. The zookeeper.connect property was never used by Atlas 2.4 KafkaNotification for consumer creation; Ranger only validated that it was set. With Kafka 3.9.x and KRaft brokers, clients do not use ZooKeeper for consumption.

Changes

  • Drop TAGSYNC_ATLAS_ZOOKEEPER_ENDPOINT constant and startup validation from AtlasTagSource
  • Extract validateRequiredAtlasKafkaProperties() for unit testing
  • Remove TAG_SOURCE_ATLAS_KAFKA_ZOOKEEPER_CONNECT from install.properties, installprop2xml.properties, and setup.py

Out of scope

  • Tag Sync HA (ranger-tagsync.server.ha.zookeeper.*) — unchanged; still requires ZooKeeper when HA is enabled
  • Atlas REST tag source — unaffected

Upgrade note

Existing deployments may still have atlas.kafka.zookeeper.connect in conf/atlas-application.properties; it is harmless and can be removed manually. New installs no longer generate or require it.

Test plan

Unit tests

mvn test -pl tagsync -Drat.skip=true \
  -Dtest=AtlasTagSourceConfigTest \
  -Dsurefire.failIfNoSpecifiedTests=false
Test Verifies
validateRequiredAtlasKafkaProperties_acceptsBootstrapAndGroupWithoutZookeeper No ZK property required when bootstrap + group are set
validateRequiredAtlasKafkaProperties_rejectsMissingBootstrapServers Bootstrap servers still mandatory
validateRequiredAtlasKafkaProperties_rejectsMissingConsumerGroup Consumer group still mandatory
  • AtlasTagSourceConfigTest — 3 tests, 0 failures (local run)
  • Fresh Tag Sync install with Atlas Kafka source — conf/atlas-application.properties has no zookeeper.connect
  • Tag Sync starts and consumes Atlas entity notifications from Kafka 3.9.x (KRaft)
  • Upgrade: existing install with legacy zookeeper.connect still starts
  • Tag Sync HA smoke test (if ranger-tagsync.server.ha.enabled=true) — leader election unchanged

…t from Tag Sync.

Tag Sync consumes Atlas notifications via kafka-clients (bootstrap.servers only).
The legacy zookeeper.connect property was never used by Atlas KafkaNotification
but was still required at startup and in installer templates.

Co-authored-by: Cursor <cursoragent@cursor.com>
@ramackri ramackri requested a review from mneethiraj June 29, 2026 14:07
Co-authored-by: Cursor <cursoragent@cursor.com>
@ramackri ramackri requested review from kumaab and rameeshm June 29, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant