Job Details

Data Engineer

ELMHURST-60126, IL, US
10/16/2020

-


Required Skills

    Python, ETL tools
Company

Infinity Consulting Solutions, Inc

Experience

-

Job Description

Data Engineer

We are working with an enterprise company looking for a technical Data Engineer.

Excellent opportunity to be a part of a team to help design, develop, maintain, support the data warehouse environment, utilize new big data technologies and cloud transformation.

Collaborate with the technical team to meet business requirements and organization goals.

Responsibilities

Help develop and maintain data platform models, database architectures, data pipelines, data solutions, cloud platforms.

Work with team to define and develop data ingestion, validation, transformation and data engineering code.

Work with automating and orchestrating jobs on a big data and cloud platforms.

Create and maintain necessary technical documentation, including requirements, design, and test documents. Utulize big data tools

Monitor, develop, support ETL processes to support analytics operations while working on projects to improve the functionality and reliability of the overall data infrastructure

Maintain design / architecture documents, development standards and prioritization plans.

Communicate across technical and business teams, vendors. Continue learning of newest technologies and solutions.

Qualicafations

At least five years of experience working with enterprise data platforms, building and managing data lakes and using big data technologies

Two+ years of experience working with AWS platform. Experience with solutioning on AWS infrastructure using services like AWS S3, Lambda, EMR, Redshift Snowflake etc

Experience with automating and orchestrating jobs on a big data platform using Oozie, Airflow, Jenkins or something similar

Two+ years of experience with Spark using Python. Experience with Spark streaming, building real time data pipelines is preferred

Good understanding and experience working with various products in the Big data ecosystem like Hive, HDFS, Presto, NoSQL databases like Cassandra, DynamoDB

Prior experience with working in a SQL server based environment and using SSIS, SSRS, TSQL

Experience with setting up and using Kafka for real time streaming is a plus

Prior experience with traditional ETL tools like Talend Open Studio, Pentaho or similar

Must be a team player and passionate about newer technologies as well as supporting legacy systems

Excellent communication and professionalism, analytical thinker


Data Architect
Information Technology

No Preference
Contract To Hire
Other
1

Candidate Requirements
-
Bachelors

Walkin Information
-
10/14/2020
-

Recruiter Details
Doug Klares
1350 Broadway, Suite 2205, NEW YORK-10018, NY
-