Job Details

Email to Friend Save Back

Big Data QA PerformanceTest Engineer

RESTON-20195, VA, US
02/17/2020

-

Required Skills

programming languages

Company

Infinity Consulting Solutions, Inc

Experience

Job Description

Our client, a leading Data Analytics Organization, is looking to hire a Big Data QA Performance Test Engineer in their Reston headquarters.

As a big data QA performance test engineer, you will be involved in both performance and functional testing of our big data applications and distributed systems.

Testers will be verifying the successful processing of petabytes of data using commodity clusters and other supportive components.

You will be ensuring data quality which involves checking various characteristics like conformity, accuracy, duplication, consistency, validity, data completeness etc.

An ideal candidate should also be familiar with big data architecture testing.

They will be able to identify improperly designed systems and pinpoint performance degradation should the system fail to meet specifications.

Performance testing includes testing of job completion time, memory utilization, data throughput and similar system metrics.

Performance Testing Requirements:

Data Ingestion and Throughput:

You will be verifying how fast the system can get through data from various data sources.

Identifying different messages that the queue can process in a given time frame is involved in testing.

It also comprises of how swiftly data can be inserted into a fundamental data store for example rate of insertion into Mongo and Cassandra database.

Data Processing:

Verifying the speed with which the queried jobs are performed. Testing the data processing in isolation when the underlying data store is occupied within the data sets.

Sub-component Performance:

Multiple components are used for making these systems and it is vital to test each of these components in separation.

You will measure how swiftly messages are indexed and consumed, map reduce jobs, query performance, search and so on.

Qualifications:

Proficiency with Big Data and Cloud technologies

Experience manipulating, testing, and querying large data sets with one or more programming languages such as Scala, Python, or SQL

Experience with big data processing: Batch, Real-Time, or Interactive

Familiar with testing all phases of Big Data: Data staging validation, "MapReduce” validation and Output validation

Data Architecture testing experience

Experienced with testing all phases of Big Data Performance: Data throughput, data processing and sub-component performance

Provide insights into testing challenges in regards to virtualization, test automation and dealing with large datasets

A good understanding of machine learning models

Data analytics/measurement experience

Innate curiosity about data and anomaly detection

Tools:

Automation: QuerySurge

NoSQL: Graph stores (JanusGraph), Elastic, Cloud AWS, Microsoft Azure, GCP, CouchDB, MongoDB, Cassandra, Redis, ZooKeeper, HBase, Time-Series

Container Orchestration: DC/OS, Kubernetes

Storage: S3, HDFS (Hadoop distributed file system)

Servers: Elastic, Heroku, Elastic, Google App Engine, EC2

Processing: Spark, Flink, Kafka, Scala

Version Control: Git, Bitbucket

Team collaboration tools: JIRA, Zephyr