ETL Schema Validation
ETL Schema Validation
Before loading raw data into a warehouse, ETL pipelines must validate records against a schema to catch corrupt or malformed entries early. Bad data that slips through can silently poison downstream models and analytics.
Given a list of records and a schema definition, validate each record and report which ones pass or fail, along with the specific errors found.
Validation Rules (checked in schema order)
-
Missing column: the column defined in the schema does not exist in the record
-
Null check: if the value is None and the column is not nullable
-
Type check: the value's type must match the expected type. The "float" type accepts both int and float values. Use Python's
type()for checking (so booleans are not treated as ints). -
Range check: if min/max bounds are specified and the value is numeric, it must fall within [min, max] inclusive
If a column is missing, skip remaining checks for it. If a value is None and nullable, skip type and range checks. If type check fails, skip range check.
Error Messages
Use these exact formats: "{column}: missing", "{column}: null", "{column}: expected {type}, got {actual_type}", "{column}: out of range"
Return a list of tuples (record_index, is_valid, errors) where record_index is 0-based, is_valid is a boolean, and errors is a list of error message strings (empty if valid).
Examples
Input:
records = [{"name": "Alice", "age": 30, "score": 95.5}, {"name": "Bob", "age": 25, "score": 88.0}] schema = [ {"column": "name", "type": "str", "nullable": False}, {"column": "age", "type": "int", "nullable": False, "min": 0, "max": 150}, {"column": "score", "type": "float", "nullable": False, "min": 0, "max": 100}, ]
Output:
[(0, True, []), (1, True, [])]
Both records pass all validation checks.
Input:
records = [{"age": "old", "score": None}] schema = (same as above)
Output:
[(0, False, ["name: missing", "age: expected int, got str", "score: null"])]
Three errors found: name column is missing, age has wrong type, score is null but not nullable.
Hint 1
Use type(value) instead of isinstance() to distinguish booleans from integers.
Hint 2
Process checks in order and skip later checks when an earlier one fails for a column.
Requirements
- Validate each record against every column in the schema, in schema order
- The "float" type should accept both Python int and float values
- Nullable columns with None values should skip type and range checks entirely
- Error messages must follow the exact format specified
Constraints
- 1 <= len(records) <= 1000
- 1 <= len(schema) <= 20
- Schema types are "int", "float", or "str"
- Time limit: 300 ms
Log in to take notes on this problem
Accepts: array
Accepts: array
ETL Schema Validation
ETL Schema Validation
Before loading raw data into a warehouse, ETL pipelines must validate records against a schema to catch corrupt or malformed entries early. Bad data that slips through can silently poison downstream models and analytics.
Given a list of records and a schema definition, validate each record and report which ones pass or fail, along with the specific errors found.
Validation Rules (checked in schema order)
-
Missing column: the column defined in the schema does not exist in the record
-
Null check: if the value is None and the column is not nullable
-
Type check: the value's type must match the expected type. The "float" type accepts both int and float values. Use Python's
type()for checking (so booleans are not treated as ints). -
Range check: if min/max bounds are specified and the value is numeric, it must fall within [min, max] inclusive
If a column is missing, skip remaining checks for it. If a value is None and nullable, skip type and range checks. If type check fails, skip range check.
Error Messages
Use these exact formats: "{column}: missing", "{column}: null", "{column}: expected {type}, got {actual_type}", "{column}: out of range"
Return a list of tuples (record_index, is_valid, errors) where record_index is 0-based, is_valid is a boolean, and errors is a list of error message strings (empty if valid).
Examples
Input:
records = [{"name": "Alice", "age": 30, "score": 95.5}, {"name": "Bob", "age": 25, "score": 88.0}] schema = [ {"column": "name", "type": "str", "nullable": False}, {"column": "age", "type": "int", "nullable": False, "min": 0, "max": 150}, {"column": "score", "type": "float", "nullable": False, "min": 0, "max": 100}, ]
Output:
[(0, True, []), (1, True, [])]
Both records pass all validation checks.
Input:
records = [{"age": "old", "score": None}] schema = (same as above)
Output:
[(0, False, ["name: missing", "age: expected int, got str", "score: null"])]
Three errors found: name column is missing, age has wrong type, score is null but not nullable.
Hint 1
Use type(value) instead of isinstance() to distinguish booleans from integers.
Hint 2
Process checks in order and skip later checks when an earlier one fails for a column.
Requirements
- Validate each record against every column in the schema, in schema order
- The "float" type should accept both Python int and float values
- Nullable columns with None values should skip type and range checks entirely
- Error messages must follow the exact format specified
Constraints
- 1 <= len(records) <= 1000
- 1 <= len(schema) <= 20
- Schema types are "int", "float", or "str"
- Time limit: 300 ms
Log in to take notes on this problem
Accepts: array
Accepts: array