Don’t worry we can strategically make things a little bit easier by using Python library (Beautiful Soup).
With this library, we can create a web scraper function to retrieve your job search results in Toronto from Indeed.
ca.
This will help us to submit job application strategically and prioritize your goals for learning required toolset in future job submissions.
Before we jump in, let’s start with 5 key questions to get some ideas:1.
What companies are looking to hire these talents (i.
e.
, Data Scientist, Data Engineer, etc.
)2.
What are the average salaries of these different roles?3.
What is the top 5 mandatory skill set that required for these different roles?4.
What is required or recommended education level for these different roles?5.
What are preferred education backgrounds (fields of study) for these different roles?Okay, here is a quick overview of data wrangling pipeline for Indeed Job Scraper function that I built to answer these 5 key questions:Figure 1.
Indeed Scraper Data PipelineStep 1.
Connect to job search URL pageStep 2.
Tokenize, parse words then do some text cleaning.
Step 3.
Create the primary function to collect job page description (job title, location, salary, etc.
)Step 4.
Loop through each page URL and stored info.
Step 5.
Create dictionaries to capture the required skills, education level, etc.
Count the term frequency on each specified term then visualize the results!Let’s dive into the 5 significant vital insights that we gained from the results.
A.
List of Hiring Companies and the Average Salary for Data Scientist Roles:Figure 2.
List of Companies from Indeed.
caFigure 3.
Average Salary Listed from Indeed.
caFirst, data scientist jobs are found in various industries.
Most jobs are in financial or insurance companies like TD, Scotiabank, Sun Life Financial, etc.
For other careers, similar trends are observed.
Except, only different companies are looking to hire talents.
Second, an average annual salary of data scientist is reported as $91,000 in Toronto.
However, this salary is a bit off by 5 to 10K from average salary found from the Indeed.
ca.
This is because some observations included salaries for senior data scientist roles as well.
Realistically, an average annual salary of an entry-level data scientist would be somewhere from 80 to 85K.
B.
Comparison between Data Scientist and Data Engineer by Top Five Required SkillsThis figure represents the top five most demanding skills, DevOps and clouds tool for each profession.
Here is a summary between two positions based on commonality (similarity) and difference.
Commonality:Among top 5 listed skills, both data scientists and data engineers required to know the tools like Python, Spark, Hadoop, and SQL.
In which Python and SQL are the fundamental tools whereas Spark and Hadoop are essential to working for companies with big data storage.
Difference:For data scientist roles, hiring companies are more concentrated on looking at other data analytics tools and data visualization experience (i.
e.
, SAS, Tableau, etc.
).
On the other hand, data engineer roles are heavily focused on tools like cloud platforms (AWS) and DevOps (Jenkins, Kubernetes, Docker).
There are two main reasons for this difference.
First, data engineers create data pipelines to production ML models created by data scientists.
They need to be equipped with DevOps tool experience.
This helps them to implement agile practices of model/code deployment life cycle between different fixes and releases in the most efficient manner.
Second, most start-up or big companies use cloud platforms like AWS over having an on-premise solution (in-house data warehouse) due to its flexibility and cost-effectiveness.
Still, this depends a lot on other factors like the company’s business/strategic road map, system architecture, and the environment.
C.
Education Requirements and Background for Data ScientistsRegarding the education level, a data scientist is the only profession where most hiring companies preferred to hire applicants with Ph.
D.
level education (figure on the left).
For other professions like data engineer, data analyst, and business intelligence roles, the bachelor level education is enough for jobs.
Besides, many job applicants will wonder what kind of an educational background or a field of study is ideal for data scientists?.From the analysis, it seems that most hiring companies want candidates from S.
T.
E.
M field like science, technology, engineering, and math.
Especially for data scientists, a lot of companies want candidates with a math background followed by computer science, engineering and so forth.
So Why Is This Becoming a Trend for Data Scientists?I collected some insights by doing some researches on my own from published journals, data science meetups and chit-chat with data scientist mentor(s).
Let’s start off with a question about why most hiring companies do want Ph.
D.
level candidates.
First, with resurgence of artificial intelligence (AI) trends, there are a lot of companies out there interested in building their own AI/deep learning (DL) research and product development teams.
They want people who possessed knowledge of advanced AI and DL algorithms.
Not only just use existing libraries and packages that are already made available from R and Python.
These data scientists must be able to tweak and implement novel algorithms from scratch for solving specific business problems and build data products.
Second, math seems to be the most famous academic discipline that many companies are looking to hire followed by computer science and engineering degrees.
This is well connected to the nature of data scientist roles.
As a data scientist must be able to understand math well within different areas like linear algebra, calculus, and statistics.
Since all ML algorithms are about understanding how these algorithms can be applied to a data set and formulate the unique solution for solving specific business problems.
Also, data scientists must do programming well.
That is the reason why many companies want applicants with background from computer science or engineering degrees.
Most of code/model development is done on Python/R and it is essential as a data scientist to write efficient and scalable level code within production for getting a job done.
Thanks for reading this article.
I hope many readers find this interesting.
I highly encourage readers from other industries to consider learn Python and build a web scraper function for your industry market insights.