How I Feel About Data Science And The Metis Bootcamp ExperienceRecording my Data Science Related Thoughts as I Approach the Conclusion of the Metis BootcampTony YiuBlockedUnblockFollowFollowingJun 14Summer is here and my data science bootcamp is almost over — it has been extremely fun, insightful, and rewarding and I will dearly miss this type of learning environment and the other members of my Metis cohort (keep in touch guys!).
Before we all ride off into the sunset, I wanted to record for posterity a few key take-aways from the past few months.
So here it comes — my first listicle…1.
Figuring out the right question is the hardest part.
Getting your data in a cleaned and usable format can be a struggle.
Picking the right model and tweaking the hyperparameters just right can sometimes be challenging as well (hello grid search!).
But honestly, many times the hardest part is finding an interesting and worthwhile question to attack.
I worked on four big projects during the bootcamp.
The one I struggled with the most was my project on natural language processing (NLP).
I kicked off the project with the best of intentions but also with no actual idea of what I would do with my data.
“I am going to download a bunch of stuff off Reddit and do something super cool with it!” I thought.
Out of all my projects, this turned out to be the worst one.
For every other project, there was a clear objective:Reliably predict loan defaults in order to construct a portfolio of clean loans that can earn a great return.
Figure out the drivers of a movie’s box office performance so that film studios can more reliably turn a profit.
Predict short term stock returns with neural nets to beat the stock market.
But while I learned topic modeling, sentiment analysis, and recommender systems, I produced no real result of substance.
All because I had no idea what question I was trying to answer — and thus no clear goal to work towards.
Lesson learned — the question should drive what data I collect not the other way around.
Imposter syndrome is intense.
At the bootcamp, you basically have three months to go from data science noob to training deep neural nets.
During which you will be intimidated by the credentials of industry professionals and amazed by the creativity of your classmates.
And throughout it all you will be working hard to ignore that voice whispering —“You will fail to make this career transition.
The companies you want to work for won’t hire you as a data scientist because you don’t have the right credentials or because you lack the relevant experience.
”The truth is that I am worried about interviews.
And until I sign that offer, I will feel a fair amount of stress over finding a job.
But also when I look back, I feel very proud of all that I’ve learned and all the hard work I‘ve put in.
While we may still have a long ways to go, my classmates and I have already come a long way as well.
The imposter syndrome may never fully disappear.
But I am confident that instead of succumbing to these feelings of inadequacy, my classmates and I will use it as fuel to learn faster and work harder.
There is still room for MBAs, even in data science.
MBAs get a bad rap these days, especially in Silicon Valley.
But having an intuitive understanding of how a business makes money and what levers we can pull to make even more money are still critically important.
Exploring data, training models, and predicting things is all great fun.
But most of the time, we still use analytics and data science to support some sort of business goal.
How will insights into this question help my company become more successful?And folks with business backgrounds are relatively adept at and experienced with framing a data science problem in terms of dollars and cents.
So be nice to your local MBA, even if he or she can’t tell the difference between Pandas and pandas.
If there is a gradient, then we can descend it.
Gradient descent is a versatile optimization method for finding minimum valuesI can’t count how many of the bootcamp’s lectures about a particular algorithm ended with:“Finally, we can specify our cost function and use gradient descent to minimize it and estimate the model’s parameters.
”In my opinion, two things tie together almost all of data science:Correlation — data science is all about looking for the connections between various things.
I say things because unlike in traditional statistics, we often deal with messy and unstructured data such as text or images.
But at its heart we are still looking for interesting and insightful connections (the signal) between our messy data and the things we care about explaining or predicting.
Gradient Descent — because we often end up needing to analyze massive amounts of messy, nonlinear data, we require an equally versatile method for minimizing cost functions (we seek to minimize our model’s cost function in order to obtain its optimal parameter values).
Gradient descent is exactly that.
It is fast, effective, and adaptable and is what powers everything from logistic regression to neural nets.
If you want a more in depth explanation of gradient descent, I wrote about it here in my post about neural networks (you will have to scroll down a ways or just read the whole thing hehe).
Collecting the right data and hiring the right people to mine insights from that data CAN be a real and enduring competitive advantage.
I’m a value investor at heart and I’m used to looking for traditional sources of competitive advantage (such as economies of scale) when I analyze a company.
Thus early on, I was definitely not used to thinking about a company’s data and analytics as an enduring source of competitive advantage.
The reason for my previous skepticism was that almost every company has some interesting proprietary data about its customers.
So I thought being able to slice the data better or forecast better was not something sustainable — your competitors could themselves employ better engineers and better models and easily catch up.
But companies like Google, Facebook, and Amazon proved this to be very false.
As far as I can tell, these three companies and others like them built their massive competitive advantages by using the following pipeline:Collect massive amounts of data about literally everything.
Build the infrastructure for storing and cleaning said data.
Employ thousands of really smart and diligent people to build models and mine the data of all available insights.
Use the newly gleaned insights to sell people things or make them buy things (capitalists!).
Collect more data from all the interactions in Step 4 in order to iterate and improve the model and the insights generated.
Keep repeating steps 3, 4, and 5 until competition is left in the dust.
While not all the promised benefits of big data and machine learning may ultimately materialize, smartly run companies have already demonstrated that through data science, they can extract massive amounts of value from data that was previously thought to be worth very little.
One other amazing thing I got out of the Metis bootcamp was this blog.
Without some great examples from previous Metis students and prodding from my instructors, I probably wouldn’t have gotten around to writing so much.
But now I love it and will keep writing through thick and thin.
Wish me luck as a I prepare to launch a new career (I am super excited about it).
Thanks for reading!.Cheers!Check out more from me here.