We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data for autonomous driving algorithm development.
The primary focus will be on choosing optimal solutions to use for these purposes, then implementing, monitoring, profiling and improving them.
You will also be responsible for integrating them with the architecture used across the company.
Responsibilities: Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities Monitoring performance and advising any necessary infrastructure changes Defining data retention policies Combine both automatic and manual steps into data processing pipeline so that it can be scaled efficiently with data size Qualifications: Proficient understanding of distributed computing principle Deep knowledge of app containerization and orchestration Experience of Kubernetes, with all included services Experience/knowledge with Hadoop Experience with integration of data from multiple data sources Experience with Machine Learning toolkits, such as Caffe2, PyTorch Experience with NoSQL databases in general, MongoDB in particular Strong programming skills in CUDA, Python, shell-script