Job Title: Big Data Developer
Location: Dallas, TX
Yearof exp: 5 - 10 years
Weare looking for someone who is good in Kafka Pre-processing using Hive,Impala,Kerberos , LDAPSpark, Pig, SQL, Shell scripting (bash,korn), python and Bigdata-HDFS
Hands on experience with Hadoop, Hive, Pig, Impala, and Spark
H1's and we work on C2C as well
Responsibilitiesinclude:
Translate complex functional and technical requirements into detailed design.
Design for now and future success
Hadoop technical development and implementation.
Loading from disparate data sets. by leveraging various big datatechnology e.g. Kafka
Pre-processing using Hive, Impala, Spark, and Pig
Design and implement data modeling
Maintain security and data privacy in an environment secured using Kerberos andLDAP
High-speed querying using in-memory technologies such as Spark.
Following and contributing best engineering practice for source control,release management, deployment etc
Production support, job scheduling/monitoring, ETL data quality, datafreshness reporting
*Skills Required:
Strong SQL scripting background required, *5-8 years of Python orJava/J2EE development experience is a plus
3+ years ofdemonstrated technical proficiency with Hadoop and big data projects
5-8 years of demonstrated experience and success in data modeling
Fluent in writing shell scripts [bash, korn]
Writing high-performance, reliable and maintainable code.
Ability to write MapReduce jobs
Ability to setup, maintain, and implement Kafka topics and processes
Understanding and implementation of Flume processes
Good knowledge of database structures, theories, principles, and practices.
Understand how to develop code in an environment secured using a local KDC andOpenLDAP.
Familiarity with and implementation knowledge of loading data using Sqoop.
Knowledge and ability to implement workflow/schedulers within Oozie
Experience working with AWS components [EC2, S3, SNS, SQS]
Analytical and problem solving skills, applied to Big Data domain
Proven understanding and hands on experience with Hadoop, Hive, Pig, Impala,and Spark
Good aptitude in multi-threading and concurrency concepts.
B.S. or M.S. in Computer Science or Engineering
Job Type: Contract