Jobe Details


Senior Data Platform Engineer Posted on: 10/06/2020

Infinity Consulting Solutions, Inc
  • 2 to 4 Year(s)
  • -
  • BRISTOL-06010, CT, US

Apache Spark

  • Job Description

    Responsibilities:

    Build cool things Build scalable analytics solution, including data processing, storage, and serving large-scale data through batch and stream, analytics for both behavioral & ad revenue through digital & non-digital channels.

    Harness curiosity Change the way how we think, act, and utilize our data by performing exploratory and quantitative analytics, data mining, and discovery.

    Innovate and inspire

    Think of new ways to help make our data platform more scalable, resilient and reliable and then work across our team to put your ideas into action.

    Think at scale - Lead the transformation of a peta-byte scale batch-based processing platform to a near real-time streaming platform using technologies such as Apache Kafka, Cassandra, Spark and other open source frameworks

    Have pride Ensure performance isn’t our weakness by implementing and refining robust data processing using Python, Java, Scala and other database technologies such as RedShift or Snowflake.

    Grow with us Help us stay ahead of the curve by working closely with data architects, stream processing specialists, API developers, our DevOps team, and analysts to design systems which can scale elastically in ways which make other groups jealous.

    Lead and coach Mentor other software engineers by developing re-usable frameworks. Review design and code produced by other engineers.

    Client First - Provide expert level advice to data scientists, data engineers, and operations to deliver high quality analytics via machine learning and deep learning via data pipelines and APIs.

    Build and Support Embrace the DevOps mentality to build, deploy and support applications in cloud with minimal help from other teams

    Have pride Ensure performance isn’t our weakness by implementing and refining robust data processing using Python, Java, Scala and other database technologies such as RedShift or Snowflake.

    Grow with us Help us stay ahead of the curve by working closely with data architects, stream processing specialists, API developers, our DevOps team, and analysts to design systems which can scale elastically in ways which make other groups jealous.

    Lead and coach Mentor other software engineers by developing re-usable frameworks. Review design and code produced by other engineers.

    Client First - Provide expert level advice to data scientists, data engineers, and operations to deliver high quality analytics via machine learning and deep learning via data pipelines and APIs.

    Build and Support Embrace the DevOps mentality to build, deploy and support applications in cloud with minimal help from other teams

    Required Qualifications:

    2+ years of development experience in in Key-Value store databases like DynamoDB, Cassandra, ScyllaDB etc.

    2+ years of development experience in Graph Databases like AWS Neptune, Neo4J, JanusGraph etc.

    4+ years of experience developing data driven application using mix of languages (Java, Scala, Python, SQL etc.) and open source frameworks to implement data ingest, processing, and analytics technologies.

    Very experienced with big data framework such as Hadoop, Apache Spark, No-SQL systems such as
    Cassandra or DynamoDB, Streaming technologies such as Apache Kafka; Understand reactive programming and dependency injection such as Spring to develop REST services.

    Hands-on experience with newer technologies relevant to the data space such as Spark, Airflow, Apache
    Druid, Snowflake (or any other OLAP databases).

    Plenty of experience with developing and deploying in a cloud native environment, preferably AWS cloud

    Preferred Qualifications:

    Prior experience building internet scale platforms handling Peta- byte scale data, operationalizing clusters with hundreds of compute nodes in cloud environment.

    Experience in operationalizing Machine Learning workflows to scale will be a huge plus as well.

    Experience with Content Personalization/Recommendation, Audience Segmentation for Linear to Digital Ad Sales, and/or Analytics

    Experience with open source such as Spring, Hadoop, Spark, Kafka, Druid, Kubernetes.

    Experience in working with Data Scientists to operationalize machine learning models.

    Proficiency with agile development methodologies shipping features every two weeks.

    It would be awesome if you have a robust portfolio on Github and / or open source contributions you are proud to share


  • Data Network Engineer
    Information Technology
  • No Preference
    Contract Only
    Other
    1
  • Candidate Requirements
    -
    Bachelors
  • Walkin Information
    -
    10/5/2020
    -
Recruiter Details
Doug Klares
1350 Broadway, Suite 2205, NEW YORK-10018, NY
- 
Advertise with Us,

Size:120X120