An Insight into the World of AI
Introduction: Abzooba being an analytics company, we use spark extensively for machine learning,data ingestion,ETL & large data processing. Spark enables In-memory processing of large-scale data.Spark job can be long running/short-lived/scheduled as per need. Memory requirements also differ to run these kind of jobs.
Introduction: Through this article, I would like to familiarize the readers with some of the basic concepts of Data Lake and also take them through the journey of various flavors of data lake implementations across the industry. I will also deep dive into Data Virtualization concepts and show how a judicious mix of virtualization with the data lake components helps us to get the required agility.