An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.

Zookeeper docker container, ready for deployments on kubernetes and openshift

Dockerfiles & YAML definitions for Apache Drill

Proof of concept open source Amazon Athena

A distributed and highly available URL Shortener built on MERN stack, using Apache Zookeeper, cached using Redis, containerized by Docker.

Helm Charts to Deploy Apache Drill on Kubernetes

Thesis project: topic categorization and sentiment analysis on twitter with Apache Spark

Fully Managed Apache Zookeeper Cluster with Ansible & Terraform.

End-to-end data engineering pipeline with various technologies to ingest real time data.

This project aims to consume a twitter stream via Apache Kafka, apply a sentiment analysis on the tweets with Apache Spark jobs and save the result into Apache HBase.

A Visual Studio Code extension for managing Apache Zookeeper

Real time streaming of a time series with corresponding forecasts.

Apache Drill and Apache Zookeeper helm charts for kubernetes

A demonstration with Spring Batch framework, Kafka, and PostgreSQL

A real-time, dockerized Kafka event processor pipeline (utilizing an Apache Storm topology). The project is running in AWS with automated infrastructure deployment and execution, using Ansible.

Distributed Java Applications at Scale, Parallel Programming, Distributed Computing

Docker image packaging for Apache Zookeeper

An Ansible role which installs and configures Apache Zookeeper on Redhat/Debian based hosts

A realtime ingestion and profiling engine for fast data.