Remoting over Apache Kafka with Kubernetes features

Goal: Enhance the plugin and to provide out-of-the-box management in Kubernetes environment

Status: Completed

Team

Details

Motivation

Previous version of Remoting over Apache Kafka plugin requires users to manually configure the entire system which includes Zookeeper, Apache Kafka and Remoting agents. It also didn’t support dynamic agent provisioning so scalability was harder to achieve. This project aims to solve two problems:

  1. Out-of-the-box solution to provision Apache Kafka cluster.

  2. Dynamic agent provisioning in a Kubernetes cluster.

Phase 1 Summary
Phase 2 Summary
Phase 3 Summary
Features

Here are some screenshots of the implemented features

1. Launch Apache Kafka on Kubernetes

Apache Kafka provisioning in Kubernetes UI

2. Cloud configuration

Remoting Kafka Cloud UI

3. Remoting Kafka Agent get provisioned dynamically

Remoting Kafka Agent get provisioned

4. Helm Chart to bootstrap the whole system on Kubernetes

Kubectl get all to check Helm chart installation