Things you should know when traveling via the Big Data Engineering hype-train

These were also the things I asked candidates for Big Data Engineer position in my previous company.I will try to not only point out relevant questions but also give brief answers to some of them..For mid or junior positions the subset is enough.   It’s a very popular question: what is the difference between data science, data analysis and data engineering..Here is my understanding:For more detailed comparison as well as introduction of “Machine Learning Engineer” you can check out this awesone article.Following paragraphs describe the knowledge needed by data engineers but it’s also relevant to data scientists as they use similar toolset but in a different way.   To efficiently develop data pipelines you need to be proficient with at least one of the following languages: Scala, Java, Python.The choice really depends on your previous experience and on your workplace preference..One of the most critical skills of data engineer is to know when NOT to use BigData and go with much simpler tooling.   This list is not comprehensive but it’s most probably not possible to create a comprehensive one.. More details

Leave a Reply