Predictive Analytics in Government DecisionsJoel NantaisBlockedUnblockFollowFollowingJun 27Photo by Michael on UnsplashPrivate Sector vs.
Public SectorMany Americans are aware of the pace by which machine learning and predictive analytics are reshaping our personal lives, our jobs, and the world economy.
While recent incidents have caused greater scrutiny on how businesses use and share our data, there are very few people who believe the private sector should not continue to advance their use.
Market forces (i.
customers, public opinions) and government regulations can effect how companies adopt and implement them.
However, the government is outside of the market, and responds to the slower process of legislation and judicial oversight.
As government agencies expand their adoption and implementation of these techniques, how do we ensure that as government agencies adopt machine learning and predictive analytics?Government DecisionsIn many instances, our current laws and regulations require the government to make decisions that directly affect citizens, albeit in varying degrees.
Some of these decisions pertain to the eligibility for benefits, others are whether individuals will receive punishments, or whether individuals’ civil liberties will be limited.
In representative democracies these decisions are often left to a small group of public servants, elected individuals, or even a single individual.
The repercussions of these decisions can vary greatly; from minor inconveniences such as having your bags checked at the airport, to much more impact such as a decision to deprive a citizen of their civil liberties by placing them in prison.
Criminal Justice has multiple examples, including decisions about where and when to send police patrols based on predictions about where crime is likely to occur.
And instances when determining whether to allow for posting of bail by an individual awaiting trial.
Assuming that these government deciders are doing the very best they can in these situations, these deciders still have to deal with the fact that they cannot have perfect information about the situations and individuals they are required to make decisions about.
For instance, recent work has shown the depth of individuals’ biases, whether they are known biases or unknown biases in making decisions.
Further, individuals are not very good at making predictive decisions about future events based on statistics of past events absent a tool to assist them.
Recent research has also suggested that the success rates of individuals making predictions are often not better than random chance.
Combining these limitations and risks, with the requirements of our system to make these important adjudicatory decisions gives rise to the need of establishing better ways to inform these decisions.
The effort to make better decisions, resulting in better policy implementations as well as greater efficiency has led to several different movements.
One off-shoot of these efforts which has been heartily embraced by politicians, public servants, and the public is evidence-based decision-making or performance information- informed decision-making.
Proponents have argued that that better data is needed to be able to make better decisions.
Others are now arguing that government decisions makers also need tools to use better data to make better decisions, and that big data and predictive analytics can achieve these goals .
I share the opinion that predictive analytics is one area where the government can embrace the evidence-based decision-making energy and show increased accuracy and efficiency in decision making, as is prescribed under these movements, thereby gaining public trust and lawmaker backing.
Benefits and Risks of Predictive Analytics in the Public SectorEven if a predictive analytics algorithm was accurate 99 percent of the time, applied to 300 million citizens you would still have 3 million false positives or false negatives.
The potential benefits of predictive analytics in government are many; including more effective decision making, more efficient decision processes, and less biased decision-making.
There are also risks and challenges that must be adequately considered.
Risks exist in implementing any predictive analytics program, but in the government sphere the risks increase as the outcomes of the prediction potentially have greater effects on a citizen.
The risks of false positives and false negatives are heightened.
Additionally, as most practitioners know, these risks are all but certain.
For example, no model is perfect, each has the potential for non-random and random error.
Even if a predictive analytics algorithm was accurate 99 percent of the time, applied to 300 million citizens you would still have 3 million false positives or false negatives.
In recommending movies on Netflix that may be an acceptable error rate, but when recommending whether to launch a surveillance program or arrest a citizen we might think it is not acceptable.
Biases — A predictive algorithm can mistakenly be programmed with inherent discriminatory biases on grounds such as age, race, sex, or sexual preference, which do not represent true causal factors for an outcome.
Privacy — There is concern from about the effect on privacy of citizen’s data when the government or private entities use big data techniques which can more quickly and accurately identify individuals and sensitive attributes about them without their knowledge or consent.
Transparency — Another area of concern is transparency of predictive analytics, or how individuals will have the ability to understand decisions reached about them, dispute the inputs or the algorithm, or understand how their past and future behavior will impact these decisions.
Policies enacted by legislatures are either carried out by publicly funded institutions, public-private partnerships, or by private companies contracted and directed by government staff.
In many areas of the public sector there are values-based and rules-based hierarchies aimed at achieving the outcome with the minimum of costs.
However, due to the public lack of ability to monitor everything, there is an information imbalance in favor of the agency and at the cost of the citizens.
Therefore, there are likely to be information gaps between the public and how the government is using predictive analytics.
This can be compounded by the nature of government classifications and sensitivity of law enforcement and national security information.
Model Risks — Other areas for caution are the risks in the predictive analytics models themselves.
This can occur when they are programmed data which does not accurately account for the breadth of inputs, outputs, and outcomes making up the actual situation.
For example, in predictive policing models, a risk would be using only data provided by police resources as this would only include crimes, arrests, or other actions by police officers.
This could result in an echo chamber of decisions based on past biased decisions such as higher policing and arrests of certain minority groups.
At the very least, the model would suffer from incomplete pictures of past “crime” if it is limited to those crimes that were reported to or discovered by the police.
As we know, not all crimes are reported.
Instead, the predictive analytics model needs to include third party data sources of crimes (as many are under reported by as much as 50 percent) and outcomes that go beyond how police organizations count.
Siloed Data Scientists — Some of these predictive analytics algorithm problems can stem from bad data, but they also can be found when the data scientists don’t necessarily understand the data they are working with and how it was gathered.
This can be common in the government as few agencies have the internal skill sets for big data, machine learning, and predictive analytics and typically rely on outside contractors or off-the-shelf product solutions.
Additionally, the cloistering of specialized data staff (often acquired from outside the operations section of the agency) and the chain of data operations (collections, compiling, cleaning, transforming) that is required for predictive analytics has potential to add biases or inaccuracy for a model.
Due Process — One final area is a concern about how predictive analytics and big data will affect due process for organizations or individuals to seek rectification or inaccurate decisions, or judicial review of an executive branch decision as prescribed in the fifth and 14th amendments of the constitution.
Due process is important to our systems of checks and balances as it provide redress by affected citizens, but also lays the ground work for judicial and legislative oversight and review of the executive branch.
The nature of algorithmic “black boxes” may preclude traditional forms of due process for many citizens.
The essence of Government is power; and power, lodged as it must be in human hands, will ever be liable to abuse.
— James MadisonThese risks and potential pitfalls for government implementation of predictive analytics are not insurmountable.
Because the decisions made by the government have potential to affect citizens significantly, care and oversight is needed.
Additionally, if the public does not have reason to trust government predictive analytic decision-making, then the government does not gain trust and efficiency as possible outcomes of the adoption.
Possible SolutionWhile all of the above risks and concerns are valid in today’s world, I believe there are possible solutions to all.
One way that the government attempts to account for these risks in government policy is through transparency, such as under the Freedom of Information Act, and well as the rules prescribed in the Administrative Procedures Act (APA)and the Paperwork Reduction Act (PRA).
These laws require public posting of draft regulations and rules, allowing public comment and rectification prior to finalizing changes.
Additionally, they prescribe that government agencies must report data and information being requested from applications for services and benefits and how it will be used.
Additionally, these information collections and programs are subject to judicial review and often public review and comment before they are approved to be implemented.
Independent Agency Oversight — My proposal is an independent third-party agency to be created which will be responsible for several areas which will directly mitigate these risks and ensure accurate, effective, and efficient federal government use of predictive analytics.
This policy focuses on mechanisms by which we can ensure public trust in the predictive analytics programs, as well as independent review and auditing of the algorithms for discriminatory biases, inaccuracy, and ensuring that established criteria are indeed utilized as prescribed.
The same public notice and public review and comment would not be possible for predictive analytics algorithms themselves because some of the detailed inputs and outputs of the systems would be too sensitive and not be as helpful as it would seem on first blush.
Scholars have pointed out that most citizens will not have the level of needed expertise to understand and comment constructively on the algorithm.
Additionally, those citizens who do have the level of expertise to understand it would be able to game the predictive analytics model for possible malfeasance and misfeasance of the outputs in order to defraud the system.
Therefore, this independent agency would receive all predictive analytics plans and algorithms for an internal audit, review, and validation.
No agency could implement their predictive analytics program without first receiving approval.
Beyond auditing to ensure the predictive analytics program will yield the intended result, the audit would also look for discriminatory biases and potential privacy harms.
Additionally, the agency would have an official registration of the predictive analytics program which it would retain in order to allow judicial and congressional oversight at a later time if needed.
This would mitigate a concern of public trust to ensure that the proposed predictive analytics system and algorithm are actually being used by the agency, which can guard against claims of agency misbehavior as it would allow comparisons of the proposed system against the systems actually implemented by the agency.
Limitations — Several potential limitations and challenges exist with this proposal.
For one, many new agencies, or offices require legislative approval and funding.
This could conceivably start through executive branch authority, but funding would have to be collected from existing appropriations.
Resources are always needed for any new public administration undertaking.
This could be accomplished by charging agencies wishing to implement predictive analytics a fee to the independent agency, such as required under the Economy Act.
Additionally, the agency charged with providing oversight of these programs will similarly need to be resourced to ensure that agency requests for review and approval are adjudicated quickly to allow the government to realize the effectiveness and efficiency gains of predictive analytics techniques.
The government will also have to invest in the legal resources to devote to this emergent technology and ensure there are legal resources ready to respond to the new challenges and concerns that it will raise.
However, I believe that the accuracy and efficiency gains realized through adoption of predictive analytics in decisions making can more than offset these resource expenditures, as well as contribute to rising rates of public trust in government adjudications.
Other Benefits — This independent agency will be poised to also contribute to the federal government policy of implementing predictive analytics in other ways.
Primarily, it can become the center of excellence for the government, publishing best practices and standards for federal agencies based on the most recent research in the field as well as understanding the needs of the government agencies.
The personnel that conduct the auditing of predictive analytics programs will also be available to consult with agencies on existing government decisions and programs which could benefit from predictive analytics.
This would include advising on information technology needs, infrastructures, and program and process development.
ConclusionUltimately these new technologies are here to stay, and they appear to present numerous opportunities for our government to utilize data to make more accurate, objective, and efficient decisions.
As we have seen in other examples of public adoption of private sector methods and technologies, the ability to adopt and implement these new technologies in ways the do not cause harm or undermine public trust will be the basis for public reaction.
Negative public reaction could cause legislators to implement more severe and further reaching oversight and safe-guards that might inhibit flexibility and agility in taking advantage of these new technologies.
Therefore, proactively taking these steps to adopt policies which mitigate the potential negative outcomes of predictive analytics systems will go a long way to prevent political backlash.
I welcome other thoughts, including disagreement, with these ideas as we discuss the successful implementation by government agencies of these techniques in our democracy.