Well, I learnt on the job.
Credit: Michael Lowe via WrikeCapturing the lessons learnt is one of the most important parts of my job as a project manager, and so is sharing those lessons.
In no particular order, here are most important things I learned about being a project manager for AI — things I didn’t know back in March 2018 when I had started out.
Being realistic about AI capabilitiesTech trends generate hype and chances are high that you have already been bombarded on social media or in your inbox or at some conference about all these “AI transformation” this and “game-changing revolutionary AI” that.
And yet, because the claims of what it will achieve are so grand, businesses risk raising their hopes for AI too high — and wasting money by trying to apply this technology to problems it can’t solve.
But it is important to separate the hype from reality because AI is no silver bullet.
Jean-François Gagné, CEO of the Montreal software startup Element AI, reminds clients that A.
I.
solutions are only as good as the accumulated data being fed into them.
“The opportunity every organization is looking at is the ability to have adaptive systems,” he says.
“It is a journey.
It is not something you can buy and suddenly flip a switch.
By the very definition of A.
I, it takes time to learn.
”Working in a centralized AI teamThe structure of data and analytics in established organizations could be anything: from having multiple data science teams in each business unit, or to a centralized advanced analytics team at a corporate level.
There is no single way of gaining benefits as the structure will depend on the type of industry.
What I learnt working in a centralized AI team are beyond my expectations.
Below is an example of a centralized analytics team working in matrix with other business units.
Credit: AltexsoftConsultant mindset: When compared to consulting firms, working as an internal consultant for an established organized is less hectic with loads of learning along the way.
My typical work-life could be to work on one type of project, calculate business case (mostly by crunching numbers), deliver the insights and later move on to the next project.
This gives an opportunity for all teammates including data scientists, data engineers, machine learning engineers to keep their curiosity alive by providing different types of projects.
Efficient resource allocation: Business Units or functional teams mostly resist centralization of analytics because they would not get dedicated capacity anymore.
But in some cases, that capacity and resources are lying around without real data job while the other team might be in need of such resources.
Having a centralized AI use case backlog enables efficient resource allocation of manpower and computing power, to a place they are most needed (it is product manager’s job to prioritize the backlog).
Removing silos: While analysts in small teams become experts in their own domain, they mostly end up working in silos and run the risk of losing the big picture.
This not only could affect the quality and relevance of the insight generated, but could also lead to lower job satisfaction of the team members.
Working in a centralized AI team could provide big picture with most important processes and data sources around organizations and their cross functionsTrue meaning of “fail fast”For the first 6 months as a project manager, all I did was fail.
I failed at experimenting with 3 different AI projects, failed in discovering new ideas and even failed in efficiently communicating about those failed experiments.
My learning since then is not only about the technology, but also the company culture which was very much different from what I had assumed.
The most important thing here is to know that there is always a way around failure and taking right action after failing is crucial even when the project is closing down (document, feedback etc.
).
I now call it “learn fast” rather than “fail fast”.
Everything is important, but prioritizeCredit: foldingburritos.
comIt is not possible to put out every fire or keeping every business owner happy all the time, be that executive board, designer or engineering.
The most useful and important skill for a AI project manager is to balance between what is urgent v/s what is important v/s what is valuable.
Needless to say, keeping your end goal in mind helps.
Use the word NO whenever needed if it takes your team to the goal you aim for.
Creating success criteria and measuring use case impactA/B testing has been the most useful method to validate a hypothesis, whether an online or offline campaign.
But what is even more important before such testing is to define what success really means to you and to the business.
Is it saving X amount of money, or making X amount of sales or providing better customer service?.(measuring customer service is the most challenging task ever).
It is easier to analyze the impact of an AI use case if a clear success criteria is defined.
It doesn’t matter alone what I thinkMy opinion alone about an identified use case doesn’t matter.
It might not be a problem at all.
Data and research around it is what matter.
Creating a use case in the beginning might be a good way to start but in the end, facts, data, feedback and experiment results matters and every decision for further steps should be made on these.
Along with relevant stakeholders, create business case calculations, use case hypothesis, associated risks, current technology implementation to make a concrete use case which not only motivates the business owners but also your own data science team.
Something shouldn’t be implemented just because it “sounds good”.
We are never really done with a use caseCredit: Carol McDonald via MapRMany algorithms are updated on a regular basis as more training data is made available.
This can be a blessing and a curse when it comes to implementing a ML model, as the models can change significantly over time and need to be monitored to ensure the targets are being met and the metrics for success are sustaining data changes.
Be sure to have data quality audits in place.
If the core data is changing, your algorithms will change and potentially degrade.
This is especially true after you’ve rolled out the machine learning models at full scale and no longer have control groups against which to monitor the models.
Have a rigorous process in place to monitor models, and re-train and measure impact frequently.
The two steps forward, one step back approach is valid here in ensuring that your KPIs (Key Performance Indicators) are moving in the right direction.
Communicate and communicate againOne of the most important task for a AI project manager is to communicate with stakeholders at every stage of the use case experimentation and development.
Communicate with the business owners, the data science team, the senior management and most importantly, with everyone else in the company about what AI is capable of doing.
AI project manager’s role is to be the trusted confidante.
You should be the first one to know if something is not on the decided path; or if a deadline is going to be missed.
Your team needs to know exactly what is expected of them; and why.
Deliver bad news early and upfront.
Be direct.
Ask for and give feedback regularly.
It is really easy to make mistakes since AI is still new and mostly complicated to put into production.
Also, it is pretty different from regular software project where clear procedures are defined.
The key to success is not to rush into implementing the AI but to take a deep breath and find the right problems to solve.
.. More details