Implementing Defensive Design in AI Deployments

The premise of defensive design is to include multiple mitigation layers so that the deployed system does not hang on a very, very thin thread.New machine learning experts and data scientists do not have the same pedigree and battle scars as devops and security engineers, and that can quickly become a problem — especially as more and more machine learning solutions are sown into the business logic of an organization.Medical device design to the rescueLet’s look at an industry with heavy requirements engineering activities that can inspire best practices: medical device design..There’s a lot that can go wrong, and the worst case scenario can be colorful, to say the least..So many things can go wrong, in fact, that we wrote a patent on just the safety systems required for a new neurology device.In terms of designing a new medical device from scratch, there is a recommended methodology that ensures a degree of certainty in verification and traceability, called the V-Model..Is this just a fancy way of drawing the dreaded Waterfall model?.Maybe!.But there are some useful themes and ideas that we can apply to most AI deployments.The V-Model of software development..I also call it “The Waterfall Model’s bastard child”..Although it’s not as cyclical and responsive as Continuous Deployment, it does ensure that there is traceability back to the original design..From Wikimedia.V-Model TakeawaysThe following themes can be pulled out of the V-Model:There is a deliberate reason for doing things..Whether it be client requirements, technology limitations, or performance expectations, there is a reason for every action that can be traced back to the genesis of the project.There is verification of every design versus its implementation..It doesn’t matter if it is a sub-sub-module or the entire system, there is a way to verify the true performance of the system against expected performance.There is a difference between verification and validation..“Did I build the thing right?” is not the same question as “Did I build the right thing?”, and usually the latter is what’s important when working with clients.Implementing defensive designThere’s 3 main stages at which designers would want to ensure that they follow a solid framework for defensive design: at the client discussion/requirements stage, during coding, and during & after deployment.Defensive design at the requirements stageRequirements, requirements, and more requirements — that’s how we ensure good, clean fun at the design stage.. More details

Leave a Reply