Job Details

Email to Friend Save Back

Data Engineer

CHICAGO-60607, IL, US
08/20/2019

-

Required Skills

SQL

Company

Infinity Consulting Solutions, Inc

Experience

Job Description

DATA ENGINEER

An established and growing professional services organization is seeking an experienced Data Engineer to become part of a dedicated, internal Data Analytics group. The ideal Data Engineer will have experience working with AWS and has built a data lake previously. Python and SQL skills are required. As a Data Engineer, you must have experience building and operating data pipelines (both streaming and batch, utilizing both ETL and ELT architectures). You will be building data pipeline solutions by designing, adopting and applying big data strategies and architectures. You must be experienced in scalable system implementations with a focus on complex data processing and analytics pipelines. You must demonstrate an understanding of data integration best practices, and expertise in data integration, data transformation, data modeling and data cleansing. The Data Engineer must be able to demonstrate innovative approaches to complex problems which deliver industry-leading experiences for our clients.

RESPONSIBILITIES

Design and develop datasets with intent to facilitate consistency and efficiency

Work in tandem with analysts and other stakeholders to develop and execute necessary processes and controls around the flow of data to ensure dependability

Innovate scalable solutions and optimize efficiency of data infrastructure

Collaborate with clients to understand and develop on business needs/issues

Test/troubleshoot problems and conduct root cause analysis

Understands and provides input into data governance to enable integrated and consistent application systems

Verifies accuracy of table changes and data transformation processes

Ability and delivery of fully-tested code prior to prod-deployment when appropriate

Recommend and implement enhancements that streamline processes and improve data integrity

REQUIREMENTS

3-5+ years AS A Data Engineer

BA or BS, preferably in Computer Science, Engineering or Science/Technology-based discipline

Motivated, self-starter with ability to learn quickly

AWS required

Previous experience building a data lake is desired

AWS Glue, Redshift or S3 is highly preferred

SQL, python skills (R is a strong plus)

Experience in designing and implementing innovative data integration solutions, utilizing Python with Spark clusters,

Familiarity with architectural patterns for data-intensive solutions

Expertise in real-time streaming and migrating batch-style data processing to streaming and micro-batch solutions

Knowledge of the RDBMS core principles; set up, tune, design, as well as newer unstructured data tools

Familiarity with consulting and traditional application design

excellent written and verbal communication skills

Display sound problem-solving abilities in the face of challenges

Must be a hands-on individual who is comfortable leading by example

Experience with Agile Methodology

Possess excellent interpersonal and organizational skills

Able to manage your own time and work well both independently and as part of a team