Home>Olive

Purpose and Vision

Pain Points

  • Non-existence of a single ingestion engine to ingest all types of data
  • No homogenous codebase
  • Pre-installed cluster needed for the ingestion tool (like Nifi or Sqoop)
  • Lack of a cloud-native dynamic ingestion engine

Our Vision

  • One code for all the data sources and sink (RDBMS, SFTP, Streaming)
  • Spark as the single programming paradigm for all the use-cases
  • No pre-installation of cluster needed and minimal resource footprint
  • User friendly web interface for Data Source registration, Job config and Job runs

Guiding Principles for Olive

Cloud native design
Platform agnostic (any Cloud/On Premise)
One code serves all (homogenous)
Modular design (plug and play model)
Elastic in nature ( auto scale as needed)
Dynamic compute (provision on-demand)
API driven ( UI, APIs, SDK)
Infrastructure as a code (terraform)
Minimum infra footprint (small VM needed for docker based setup)
Minimal configuration needed on client clusters (minimal Interference)

Benefits of Olive

Agility

  • Deploy new use cases in hours…. not days
  • Add new data sources with a single click

Flexibility

  • Choose any big data platform, on-premise or cloud
  • Choose ingestion type like full/incremental, provide analytical queries for ingestion

Cost Efficiency

  • Reduce cost and time of building and running analytics use cases
  • Reduce cost associated with legacy DW’s

Configurability

  • Configurable framework to support any data size​
  • Cluster setup based on configurations

Operating Efficiency

  • Focus resources on generating business value
  • Decrease demand on IT through self-service
  • Improve reliability in production operation of analytics use cases

Logical view of Olive

Architecture and the Tech Stack

Deployment view

Speak to AI expert