The Accuracy Para has its roots in imbalanced training datasets.
For instance, in a dataset in which the incidence of category A is dominant, being found in 99% of cases, then predicting that every case is category A will have an accuracy of 99% is completely misleading.
A simpler way to understand the Accuracy Paradox is to find the balance between precision and recall in machine learning models.
In machine learning algorithms, precision is often defined as measuring what fraction of your predictions for the positive class are valid.
It is formulated by (True Positives / True Positives + False Positives).
Complementary, the recall metric measures how often your predictions actually capture the positive class.
It is formulated by (True Positives / True Positives + False Negatives).
In many machine learning models, the balance between precision and recall results a better metric for accuracy.
For instance, in case of an algorithm for fraud detection recall is a more important metric.
It is obviously important to catch every possible fraud even if it means that the authorities might need to go through some false positives.
On the other hand, if the algorithm is created for sentiment analysis and all you need is a high-level idea of emotions indicated in tweets then aiming for precision is the way to go.
The Learnability-Godel ParadoxSaving the most controversial for last, this is a very recent paradox that was published in a research paper earlier this year.
The paradox links the ability of a machine learning model to learn to one of the most controversial theories of mathematics: Gödel’s Incompleteness Theorem.
Kurt Gödel is one of the brightest mathematicians of all time and one that pushed the boundaries of philosophy, physics and mathematics like a few of its predecessors.
In 1931, Gödel published his two incompleteness theorems that essentially say some statements cannot be proved either true or false using standard mathematical language.
In other words, math is an insufficient language to understand some aspects of the universe.
The theorems have come to be known as Gödel ‘s continuum hypothesis.
In a recent work, AI researchers from the Israel Institute of Technology linked Gödel’s continuum hypothesis to the learnability of a machine learning model.
In a paradoxical statement that challenges all common wisdom, the researchers define the notion of a learnability limbo.
Essentially, the researchers go on to show that if the continuum hypothesis is true, a small sample is sufficient to make the extrapolation.
But if it is false, no finite sample can ever be enough.
This way they show that the problem of learnability is equivalent to the continuum hypothesis.
Therefore, the learnability problem, too, is in a state of limbo that can be resolved only by choosing the axiomatic universe.
In simple terms, the mathematical proofs in the study show that AI problems are subjected to Gödel ‘s continuum hypothesis which means that many problems might be effectively unsolvable by AI.
Although this paradox has very little applications to real world AI problems today, it will be paramount to the evolution of the field in the near future.
Paradoxes are omnipresent in machine learning problems in the real world.
You can argue that as, algorithms don’t have a notion of common sense, they might be immune to statistical paradoxes.
However, given that most machine learning problems require human analysis and interventions and are based on human-curated datasets, we are going to live in an universe of paradoxes for quite some time.
.