Wipro
5 to 10 Year(s)
Job DescriptionProvides leadership and direction for data engineering and design across multiple projects and functional areas.Advises on performance optimizations and best practices for scalable data models, pipelines and queries.Design robust and automated pipelines to ingest and process structured data from source systems into analytical platforms using batch and streaming mechanisms leveraging cloud native toolsetsDesign reusable Data Pipelines & Wrappers using PySpark, Python, Unix and AirflowExperience with encryption using KMS, CMK as well as custom 3rd party tools.Expert with Glue, Lambda, Lake formation, EMR, RDS and other AWS data tools as required to address batch ingestion and processing needs & use casesAbility to setup Data security and access controls within AWS using Lake Formation, IAM Roles, Bucket Policies, ACL, Security Groups etc.Engineers and supports platform tooling required for data science platform such as Sage maker, Step Functions, Airflow, EKS etc.Leverages the right devops techniques for the right job to deliver testable, maintainable, and modern data solutionsResearches data questions, identifies root causes, and interacts closely with business and technical teamsPerformance tuning, ability to setup monitoring intelligence with Cloud watch and 3rd party toolsReduces technical debt over time with root cause identification and resolution of system problemPreferred Qualifications10+ years of overall experience5+ years of experience in AWS Cloud computing5+ years of experience in Data Engineering3+ years of experience in an Agile delivery environmentAWS certification is a must