Real-Time Data Ingestion
Client
For a leading computer memory and data storage device manufacturer
Business Problem
- Our client wanted to develop a real-time analytics data pipeline
- They also wanted the capability to monitor and control the pipeline
Abzooba’s Solution:
- Traffic generation and data ingestion through Kafka directly into Druid DB
- Real-time analytics cluster hosted as a microservice which reads Kafka IoT events and converts them into insights and metrics
- A CLI front-end for all components, i.e., traffic generator & analytics cluster
- Kubernetes for container orchestration
- Monitoring and log management
- Microservice framework for traffic generator and data pipeline code
- System/node level instrumentation for resource usage and performance statistics
Business Benefits:
- High-speed traffic generation, ingestion, storage and retrieval capabilities
- Up to 2 million events/second being ingested through Kafka stream
Tech Stack
