Bachelor's (Preferred)
SQL: 7 years (Preferred)
Data Warehouse: 8 years (Preferred)
__
_ 1) Python/PySpark_
_ 2) Hadoop tooling - hive, Spark, and Map Reduce_
_ 3) Big Data fundamentals - core understanding of building data pipelines, data modeling, Linux_
_ 4) AWS tooling Glue, EMR, S3 _
The Big Data Engineer is responsible for engaging in the design, development and maintenance of the big data platform and solutions at Quicken Loans. This includes the platform host data sets that support various business operations and enable data-driven decisions as well as the analytical solutions that provide visibility and decision support using big data technologies. The Big Data Engineer is responsible for administering a Hadoop cluster, developing data integration solutions, resolving technical issues, and working with Data Scientists, Business Analysts, System Administrators and Data Architects to ensure the platform meets business demands. This team member also ensures that solutions are scalable, include necessary monitoring, and adhere to best practices and guidelines. The Big Data Engineer helps mentor new team members and continues to grow their knowledge of new technologies.
Responsibilities
Develop ELT processes from various data repositories and APIs across the enterprise, ensuring data quality and process efficiency
Develop data processing scripts using Spark
Develop relational and NoSQL data models to help conform data to meet users’ needs using Hive and HBase
Integrate platform into the existing enterprise data warehouse and various operational systems
Develop administration processes to monitor cluster performance, resource usage, backup and mirroring to ensure a highly available platform
Address performance and scalability issues in a large-scale data lake environment
Provide big data platform support and issue resolutions to Data Scientists and fellow engineers
Requirements
Master's degree in computer science, software engineering or a closely related field
2 years of experience with Hadoop distribution and ecosystem tools such as Hive, Spark, NiFi and Oozie
2 years of experience developing batch and streaming ETL processes
2 years of experience with relational and NoSQL databases, including modeling and writing complex queries
Proficiency in at least one programming language, such as Python or Java
Experience with Linux system administration, scripting and basic network skills
Excellent communication, analytical and problem-solving skills
Subvendor/H1 candidates are fine. Quicken Loans is fine with 100% remote candidates for this engagement, candidates that will eventually relocate are preferred.
Multiple positions open on several different teams.
1) Team is building trusted data products
2) Another team the Big Data Engineer would work very closely with data scientists to help implement their models
3) Platform and Support team could also use people to support DevOps and Performance Tuning activities
Open to Mid and Senior level candidates, most likely additional needs coming beyond this first 4 roles.
Strong Scaled Agile environment, will work on part of Scrum teams within Data Intelligence group. Roughly 380 people in Data Intelligence on over 30 teams. Each Scrum team has 8-15 depending on it's focus.
2 step - Panel interview first step and then second step deep dive coding assessment on Python, Spark, SQL
Internal - variety of business and IT teams that benefit
Better decision making, AI and ML integrated into existing products
Quicken Loans is building new trusted data products for their business teams for data analysis and data science activities. This help them better align resources across the enterprise as the mortgage business stays extremely busy.
We're America's largest mortgage lender, closing loans in all 50 states. J.D. Power ranked Quicken Loans "Highest in Customer Satisfaction in Primary Mortgage Origination" for the past 10 consecutive years, 2010 - 2019. The company was also ranked highest in the nation for client satisfaction among mortgage servicers by J.D. Power for 6 consecutive years, 2014 through 2019, each year the company was eligible. There's a simple reason we've been so successful: We care about the people we work with.
If you're tired of stuffy, bureaucratic workplaces, then you'll be delighted to find something different here. We strive to make a creative, fun and collaborative environment you simply won't find anywhere else. Quicken Loans was named #1 in ESSENCE Magazine's first ever list of "Best Places to Work for African Americans" in 2015. We've been on Computerworld's "Best Places to Work in IT" list for 13 years running, hitting #1 the last 5 years. We were also ranked #14 in FORTUNE Magazine's list of "100 Best Companies to Work For" in 2018, remaining in the top 30 for the past 15 years.
Migrating environment to AWS, so any experience with AWS EMR would be highly preferred
Additional needs
Job Types: Full-time, Contract
Pay: $79.00 - $80.00 per hour
Schedule:
Education:
Experience:
Contract Length:
Contract Renewal:
Work Location:
COVID-19 Precaution(s):