Session: Observability In Data Ingestion
Data has become an essential part of the tech stack of modern organizations. Everyone deals with ingestion, transformation and storage of huge volumes of data which are critical to business analytics operations.
This talk is about how we can observe the ingestion of large datasets and streaming sources, various transformation operations and their storage and why is it important. Observabilty helps to determine completeness, accuracy, efficiency and consistency for any data uploaded through either batch or streaming sources. As well as helps to identify failures. It provides necessary details to customers about ingestion success or failure and enables them to take corrective action. It also reduces the time invested by the tech team to investigate the errors.
- Data ingestion
- Data engineering
- Data monitoring
Hey, This is Nancy Chauhan, software engineer from India. I like hacking through software engineering problems. Over the past 2 years I had been developing solutions for software reliability. In my free time I love writing tech blogs on various engineering topics, making illustrations or go for short cycling trip.