Design a Data Pipeline (ETL at Scale)
Design a Data Pipeline (ETL at Scale)
HardHLD0% accepted
Design a large-scale data pipeline that ingests clickstream events from Flipkart (10B events/day), transforms them, and loads into a data warehouse for analytics. Cover ingestion, batch processing, data quality, and schema evolution.
Approach
Design ETL pipeline: ingestion via Kafka, batch processing via Spark, loading to data warehouse.
Input
Clickstream event schema adds new field 'device_type'
Output
Schema registry with Avro: new field optional with default; producers write new schema; consumers using old schema ignore new field
More HLD problems
Browse all coding problems →Scenarios & Expected Answers
Sign in to continue
Create a free account to view answers and mark progress.
- Free forever — no credit card
- Theory Q&A with detailed answers
- Points, streaks & leaderboards