AWS Data Engineer

Roles Responsibilities

Requirement Analysis & Solution Design: Collaborate with business and data stakeholders to understand data requirements, transformation rules, and use cases. Translate functional requirements into scalable data engineering and ETL design solutions. Define approach for data ingestion, transformation, orchestration, and access control.

Data Engineering Development & Testing: Design and develop robust ETL pipelines using AWS Glue PySpark. Implement complex data transformations including joins, aggregations, conditional logic, and business rule processing. Build reusable, metadata-driven ETL frameworks where applicable. Optimize data pipelines for performance, scalability, and reliability. Perform unit testing and end-to-end validation of data pipelines. Implement data quality checks including integrity, completeness, and reconciliation. Validate transformation logic and ensure accuracy of aggregated and processed data.

Orchestration & In...