Job Description

The Senior Big Data Platform Engineer is responsible for architecting, engineering, configuring, and deploying advanced big data platforms including technologies such as Hadoop, Spark, Kafka, HUE, R, Solr, and many others. The GM Global Data and Analytics Platform (GDAP) is comprised of an integrated set of platform technologies that provide industry leading data persistence and analysis capabilities. The senior big data platform engineer will be assigned to projects requiring deployment of new big data related technologies, upgrade or enhancement of existing platforms, and to various POC activities designed to test and evaluate new technologies for suitability at GM. This role requires the incumbent to develop technology standards, to provide platform engineering subject matter expertise to the organization, to perform hands on engineering and configuration in complex interdependent environments, and to lead triage, diagnosis and remediation of platform related issues. Additionally, the senior big data platform engineer will be expected to work with our key supplier partners and industry peers to understand and influence industry technical direction in order that GM’s big data requirements are accommodated.

Skill, Experience and Education Requirements

* 10 years of demonstrated expertise in technology selection, architecture design, engineering & configuration, deployment and operational support/issue resolution.
* Full stack technical expertise from OS and configuration management through to Big Data ecosystem components such as Spark, Kafka, Hive, Cassandra, R and Solr.
* Experience as a contributor to open source communities and projects, with a bias toward data and analytics related projects.
* Experience with integration of Hadoop and other big data open source technologies with various commercial platforms including RDBS solutions and off the shelf analytics tools.
* Experience with software defined infrastructure and private cloud environments.
* Experience with development of analytics and data science based solutions using open source tools and libraries is required.
* Expertise and experience with software development and related techniques and methodologies such Agile, Scrum, Extreme Programming, Test Driven Development, Etc.
* Expertise in the area of security assurance engineering in a big data platform environment including application of Kerberos, LDAP based authentication, various encryption technologies, security vulnerability detection and remediation.
* Strong development experience with Python, Ruby, Go, Java, Scala, and related Linux-based scripting tools, including the ability to interpret and contribute improvements to various open source projects.
* Expert in large scale deployment automation and configuration management using Chef or similar tools.
* Experience with Atlassian tools such as Bitbucket and Confluence.
* Demonstrated ability to develop sizing & capacity planning for large scale big data platforms and to periodically evaluate and optimize performance and platform throughput.
* The ability to comprehend business strategy and requirements and to develop corresponding technology roadmaps that integrate supplier technology plans with GM IT standards and requirements.
* Demonstrated ability to work across multiple organizations, including both IT and GMIT’s business partners and suppliers, to ensure that their big data platform related needs are supported, forecasted, incorporated into technology roadmaps and ultimately implemented in a platform as a service model.
* Excellent communications skills including both the ability to address detailed technical communications as well as the ability to distill complex technical status to executive level oral and written communications.