Data Engineer (Reference: Data Engineer)

Job summary

Career Level:
Mid Career (2+ years of experience)
Education:
Bachelor's Degree
Job type:
Full time
Positions:
1
Salary:
Negotiable
Apply before:
30 Nov, 2017

Data Engineer

Skills and Qualifications
Bc's degree, preferably in a related field (e.g., Engineering, Computer Science, Statistics, Applied Math)
2-4 years' of working experience in a similar Job
Experience with integration of data from multiple data sources
Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
Proficiency with Hadoop, Spark , MapReduce, HDFS, distributed computing and clustering
Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
Experience with various messaging systems, such as Kafka or RabbitMQ
Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
Experience with Cloudera/MapR/Hortonworks
Knowledge of various ETL techniques and frameworks and Lambda architecture

Responsibilities
Management of Hadoop cluster with all included services
Collecting, storing, processing, and analyzing of huge sets of data.
Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
Find, maintaining, implementing, and monitoring data applications, and integrate them with all available data sources
Implementing ETL process, and stream processing architecture
Monitoring performance and advising any necessary infrastructure changes
Defining and implementing data retention policies


Subscribe for newsletter and promotions