Educational Qualification: B.S. or higher in Computer Science, Engineering
Experience: 5-10 years
The Senior Hadoop Administrator will manage end to end Big Data platform operations in a highly challenging role supporting numerous critical applications running mixed workload. The individual will be part of Big Data Practice team and will work closely with customers and development teams to create, manage, upgrade, secure, and optimize Hadoop clusters. The individual should have proven experience in automation and system operations. The individual should have experience in performance tuning, diagnosis, troubleshooting, and optimizing applications.
- Installation and configuration of Hadoop cluster (HDP 2.1, HDP 2.5)
- Experience in managing HDP operations using Ambari and native tools.
- Experience setting up secured clusters using KNOX, Kerberos, & Ranger.
- Experience setting up fine-grained access control.
- Experience setting up optimum cluster configurations for MapReduce, Spark, Hive, Hbase, etc.
- Experience configuring Zookeeper, Oozie, Falcon, etc.
- Configuring HDP High Availability.
- Write/ Review Java and Python code for frameworks and applications.
- Debugging knowledge of YARN. Hands-on with analyzing various Hadoop log files.
- Recommends best practices in compression, encoding, file formats, etc.
- Experience in monitoring cluster health by using Ganglia, Nagios, Monit and other tools. Develop automated scripts for holistic monitoring.
- Writing automation scripts to generate cluster and system usage reports to fine tune the configuration
- Implement HDFS Quota, Node Labels, & Fair/ Capacity Scheduler.
- Installing and Configuring Apache Spark
- Experience in 24×7 production on-call rotation and Agile development environment.
- Experience in Linux system administration.
- Experience in Configuration Management tools such as Chef, Cookbooks, Cobbler, etc.
- Experience in configuring separate storage and compute node clusters.
- Experience in setting up heterogeneous storage layers.
- Programming in Spark.
- Experience setting up MPP, In-Memory databases
- Experience in managing big data systems in cloud environments.
- Experience in one of the leading BI reporting/analysis product.