Educational Qualification: B.S. or higher in Computer Science, Engineering
Experience: 7-10 years
The technical lead will be responsible to design and develop the next generation of data processing on Hadoop/ Spark platform. He will work closely with various teams including data acquisition, data products & data sciences to build the web services platform exposing these services to the end customers.
- Leading the team, designing and developing the data processing & ML code on Spark.
- Prototype and program data transformations and entity resolutions on petabyte-scale data on Hadoop/ Spark clusters.
- Understand the data patterns and design and implement the code and debug ML models for entity resolution in Spark. Work on Spark code performance improvement.
- Run along the cutting edge with an all-star lineup of experts in computer science, software engineering, marketing science, operation research and marketing strategy.
- Experience in information technology and information services domain
- 3+ years of experience in the ETL or Analytics domain with significant hands-on experience
- Adept at data architecture, data platforms, and ecosystem tools, technologies and applications. Experience with the latest data and digital ecosystem related to marketing, advertising, security, internet of things and related domains is a plus
- Adept in the use of big data, business intelligence and analytics with significant past deliveries and achievements using SQL, Map-Reduce, Spark and other distributed computing frameworks including cloud (AWS, Azure etc.)
- Adept at data pre-processing, complex data transformations using ETL/ELT technologies and SQL as well as NOSQL technologies, relational as well as graph databases and data API frameworks to query data off these databases
- Good hands-on programming skills in Java, Python and Scala.
- Hands on experience with continuous integration and continuous deployment tools including Chef, Jenkins etc.
- Experience in implementing entity resolution models in Spark/ experience in digital marketing, advertising domains is a plus
- Experience in using Spark graph libraries will be a definite advantage