What’s New in Deep Learning ResearchMicrosoft Also Wants to Use Neural Networks to Design Neural NetworksJesus RodriguezBlockedUnblockFollowFollowingDec 3Choosing the right network architecture is a highly subjective aspects of deep learning systems..The most ambitious circles within the deep learning community believe that neural network architecture can be modeled itself as a deep learning problem.The idea of relying on deep learning systems to design deep learning systems is a fascinating area of research..The NAO model is based on three fundamental components:An encoder that maps a discrete neural network architecture into a continuous vector, or embeddingA performance prediction function that takes the vector as input and generates a real value as a prediction of the performance of the architecture (for example, accuracy)A decoder that recovers the architecture from its continuous vectorThe core NAO architecture is illustrated in the following figure:The encoder model is the core component of the NAO architecture..This is achieved by a decoder based on a long-short-term-memory model which “recovers” a discrete network from the continuous representation.Let’s see the NAO architecture in the context of designing a convolutional neural network(CNN)..As shown in the following chart, the encoder was able to achieve a satisfactory accuracy after approximately 500 evaluated architecture while the decoder continuously lower the distance between the chosen architecture and the correct one.The idea of neural networks assisting to design other neural network is will continue to be at the forefront of deep learning research for the next few years.. More details