Visual Agnosia: A Neural Network AnalogyAryan SinghBlockedUnblockFollowFollowingMar 25Visual Agnosia: Intricate Brain ImbroglioThis week I began reading Dr.
Oliver Sacks’s story, The Man Who Mistook His Wife for a Hat, a compelling story of one Dr.
P who has lost the power of visual interpretation known in neurological terms as Visual Agnosia.
The story started off with a cursory description of his symptoms, steadily moving on to a detailed account of his progression into the delusional world of the dysfunctional vision system.
A sort of poetic slump from a man with profound sense of visual imagination and judgement to a persona that sees the world in a very abstract sense like a convolutional neural network would do.
More and more I read this story I could not help but draw parallels of this with a badly performing convolutional/capsule/RNN network and a corresponding generative network, trying to compensate for the deficit with some other imagery.
In this article I present some theories which you may call ramblings of an untrained mind, possibly a long shot, putting together pieces of two different realms in Neuropsychology and deep learning resp.
into one symphony which seems appropriate considering the case of Dr.
P, the musician who inspired this article and the original story.
Although, it is to be noted that all of this article is just theoretic and has no experimental basis so it’s argument should be taken with a lot of salt (not a pinch I mean it).
Theory 1: Theory of missing neuronsSymptom: Some parts of image are missing from the visual cortex esp.
animate ones like faces of people.
Sometimes these faces appear in random places like pillars etc.
Parts missing of a visual are replaced by something else like a hat replacing Dr.
P’s wife’s head.
Hypothesis: Dropout/Deactivation of some neurons in Parietal and Occipital lobes.
Explanation: The exhibited symptoms where in Dr.
P is not able to recognize faces or the instances where these faces are replaced by inanimate objects like a hat or a shoe is a case of some neurons not firing due to lesions/tumor in the right hemisphere perhaps and a desperate attempt of the brain to fill the deficit caused by this dysfunction by generating random imagery/vision.
I would like to illustrate this with a diagram that should simulate some neurons in your pre-frontal cortex, at least I hope so.
Defective visual discriminator and adversaryIf you see the above diagram it is quite conspicuous that there is something wrong in the discriminator network that tries to identify the visuals it receives via the eyes.
The thing that’s wrong might be that some of the critical neurons in the process have stopped functioning.
As a result, the image generated by the visual cortex in the brain is incomplete in that it has got individual pieces almost right but it is missing in some respects which hamper to create the scene as a whole.
In an attempt to reconcile this omission, the brain generates another piece, a face for example and fills in those missing places with this generation.
But the catch is that, this generated face is anticipated as correct with context to the scene because the adversary which is supposed to tell the generative network whether this face is appropriate or not is itself lacking some neurons thus presenting a warped judgment.
Possible Treatment:· One possible treatment could be to prescribe such drugs which could re-trigger or simulate the neurons in the Parietal and Occipital lobes.
· Second way could be to recreate certain scenarios in which the patient has encountered these scenes or people so that the memories that are still intact can be used to rekindle these neurons.
Again all these treatment suggestions are just a shot in the dark and not based on thorough research.
Theory 2: Emotional DeficitSymptom: Dr.
P is not able to recognize his students/people even though he is able to recognize individual features like long nose, curly hair or tallness.
That means there is nothing wrong in pathological part of vision.
In addition to this, he identifies people in a very abstract, cursory form like a machine mainly a CNN with its convolutional layers identifying the features and max-pooling adding the invariance essential for generalization.
Another constraint is that he is able to recognize people on hearing their voice or music they play.
It’s like the lost human touch/emotional part associated with the visual stimuli is regained through the auditory sense.
Hypothesis: The emotions governed by the right hemisphere of the brain as well as the amygdala are amiss.
These emotions, the essential with which we associate people in our life, the bridges that add the facet of intimacy essential to identify our friends and family are like a KPI or an independent variable on which our optical system depends.
The theory I want to present is that these emotions are lacking from the input to the network due to the dysfunction of the amygdala which cause Dr.
P’s inability to recognize people.
Emotional Deficit Compensated By MusicThe reason that music works is that it stimulates the amygdala to evoke the emotions related to a situation or a person and reinforces the input vector so that the visual identification works.
Possible Treatment:· One possible treatment is to again stimulate the neurons/glands in amygdala to generate the emotional input to be entered into the visual recognition mechanism of brain.
These neurons can be activated by narrating stories that can recreate the same visual imagery.
· Another way is to recreate the situations that can bring back the memories related to a person so as to activate the same parts of amygdala and recreating the same emotions.
This way the input will be reinforced and possibly will reconcile the segregated pieces.
Theory 3: The rhythmic conundrumSymptom: Dr.
P seems to lose the context/judgement of the physical activity or action being done if the song he is humming while doing that activity is disturbed.
This is perhaps the most peculiar of symptoms that he exhibited.
Dr.
P seemed to have lost a lot in terms of powers of recognition and judgement but what he has lost, he has gained in terms of musical acumen.
The implication being that if he is humming a particular song while performing an action like shaving, he functions like a fully oiled machine performing the activity like a normal person would do, but as soon as something small like a knock breaks his rhythm, he loses all context and comes to a standstill until the same tune somehow re-kindles the mental progress.
Hypothesis: The sequence of an action has a temporal dimension to it.
The one that is paralleled by a RNN in the neural network world.
In this case this RNN is an encoder decoder network with the encoder encoding the past actions into a context vector and inputting this to a decoder to forecast the next actions.
In case of Dr.
P, this particular context vector seems to be reinforced at each timestep by the music being generated by the brain.
This music sort of act us an attention context that tells the brain to pay attention to what and when.
But as soon as the music stops this context gets lost and the decoder stops getting any information for the next timesteps and just becomes blank.
This leads to loss of context and motion.
Missing context reinforced by music attentionPossible Treatment:· Retrain the mind through voice based instructions.
Repeat them over and over again until the activity is completed.
This might help to activate the dormant neurons.
Then slowly start making the instructions sparse and then wean them away completely allowing the person to establish that context himself.
.. More details