Data Engineer - Spark/Hive/AWS (Java or Scala exposure)

Data Engineer - Spark/Hive/AWS (Java or Scala exposure):

- London
- Contract
- £600-£675 per day (negotiable)

Client and role:

I'm currently helping a global, award winning, digital media organisation to hire a Data Engineer. Our client are renowned for their content and their award winning digital products.

A new team has been set up which is looking to create the 'perfect’ Single Customer View in the Data Lake and use that data to drive marketing communication journeys for their customers via their campaign management tool.

You will help evolve the company's Data Lake. The Data Lake, comprising mainly AWS S3 buckets, is their long-term repository of data where disparate data sets can be joined for deeper analysis, e.g. page views, ads data, contributions data, subscription data, discussion data, identity data.

The Data Lake uses:

• Scala.
• AWS’ Elastic Map Reduce, on which we run Hadoop, Spark and Hive.
• AWS Athena.
• EC2.
• Kinesis.
• Cloudformation.
• S3.
• DynamoDB.
• Play framework.

You will be the Data Engineer on the project and will involve the following types of responsibilities and technologies:

• Good experience with a mixture of some of the skills mentioned above.
• Willing to dive into a Scala or Java back-end or work with a variety of database and streaming
• An interest in how data can drive decisions.
• An appreciation of developing sustainable systems.
• A willingness to provide self-service tools to the platform’s users.
• A willingness to learn and experiment with Scala, AWS and data technologies.
• Experience with CMS such as Braze (formerly AppBoy), would be great!
• Any other CMS is desirable
• Marketing domain experience or large scale ecommerce type background.