Story of
Neural Network Console / Libraries

First of all, tell us about the current status and future potentials of deep learning.

As you know, image recognition with deep learning is already outperforming human capability, and it's expanding to other areas like speech recognition. With such high performance, it is certain that it'll be widely spread for practical usage in no time. It looks like we're at the dawn of the age where using deep learning everywhere will be taken for granted.

You have worked on deep learning for quite a while. Could you share with us a brief history of machine learning at Sony?

Sony began R&D on machine learning back in the 90's, and was able to work on autonomous cognitive development intelligence or deep learning from early on. In 1999, Sony released AIBO that can be emotionally attached to the users, which was an outcome of machine learning research. As for deep learning, R&D was started in 2010.

What specific efforts are being made for R&D on deep learning?

For example, we have developed software to make R&D on deep learning more efficient, for example in image recognition. By using this software, a developer working on recognition can develop it more quickly, or implement better performance given the same amount of time.Recently released Neural Networks Console/Libraries are also one of such efforts to support developers, as well as an outcome of our own R&D. Neural Network Console has also been used within Sony since 2015, for development of recognizer, etc. We believe it has now evolved into a highly reliable one that is worth distributing, and decided to make it publicly available in June, 2017.

How are the achievements from deep learning R&D being applied?

Sony's business is spread over a variety of fields, ranging from the products like camera and smart phone, to game consoles like PlayStation®4, even entertainment contents like music and films… In all of these areas, our accomplishments from deep learning R&D are being actively used. More specifically, Digital Paper's hand-written digit recognition, real estimate price prediction engine at Sony Real Estimate, Xperia Ear's gesture recognition, and many other are using deep learning technologies. Deep learning is also used to realize image recognition for user identification, face tracking, charge stand recognition, generic object recognition, etc. using a fisheye camera installed at the nose in the latest entertainment robot "aibo", recently announced in November 2017.

So it is indeed being used in a lot of fields.

People usually think of visual data when it comes to the application of deep learning, but as you can see from these examples, deep learning is also applicable to sensor data, numeric data, and many other fields like speech and text. Moreover, deep learning is not only for data recognition, but also can be applied to anomaly detection, data generation, etc. On top of the products and services directly available to the customers, it is also applicable to a broad range of tasks that improve the efficiency of workplaces, such as factory inspection or anomaly detection.

What's the relationship between Neural Network Console/Libraries and AI?

Neural Network Console/Libraries provide one of the most critical back-end technologies that support AI technology.

What do you mean by back-end technology?

First of all, when we speak of AI market, we usually focus on its applications in autonomous driving, industry, manufacturing etc, but we can also think of it as referring to both the applications and the back-end technology that enables those applications. Back-end technology refers to the technology that operates behind every application and task; for example, processors like GPU or CPU that enables an efficient implementation of deep learning, or cloud environments on top of which training or recognition can be implemented.

So it's a platform for further development of AI technology.

Right, our Neural Network Console/Libraries is a software for development, which is one of those core back-end technologies. It's a very useful software that enables a wide array of tasks, ranging from deep learning R&D to actual embedding into the products.

What type of users were you targeting while developing Neural Network Console/Libraries?

Neural Network Console is a highly innovative tool that enables efficient development of product-level recognizers etc. using GUI, which is unprecedented. It is highly recommended to people who want to focus on the core part of their tasks with high development efficiency, as well as to people who are new to deep learning. On the other hand, Neural Network Libraries belong to the same category as many other deep learning softwares, and are basically for the programmers. It enables programming with flexibility, so it's recommended to people who want to focus on R&D.

What are the merits of Neural Network Console?

Neural Network Console contains a lot of know-hows that we obtained over the years as its functionalities. By utilizing these functionalities, researchers and developers on deep learning can focus on the fundamental aspects of their works. Plus, users that are new to deep learning can learn the essence of deep learning via Neural Network Console. It is also a great tool for the universities and the companies, where the demand for education on deep learning is increasing exponentially.

Now, what are the merits of Neural Network Libraries?

The biggest merit of Neural Network Libraries is that it has been refined continuously since 2011. Not only is it equipped with functionalities to implement most up-to-date R&D works, but it also enables intuitive coding and very fast training and prediction. It is very easy to add new function modules (operators), and portability to various devices has been reflected in its design. Neural Network Libraries' strength comes from having all of these benefits.

What was the primary difficulty during development?

Deep learning is a very active research field world-wide, and new techniques and applications are proposed basically every week. We were very careful in the design process in order to make a tool that is both easy to use and and is compatible with new technologies and applications. This usability, along with flexibility towards a variety of techniques and applications, is one of the biggest merits that characterize Neural Network Console/Libraries.

What is the motivation for making Neural Network Console/Libraries publicly available?

Our goal is to contribute to our society by having developers fully utilize Neural Network Libraries. Considering the wide spectrum of deep learning's applicability, it is impossible for Sony alone to cover the entire fields of application. By providing the users with efficient development tool, we hope to accelerate the application of new deep learning technologies. Also, Sony hasn't really appealed its deep learning technology externally, but hopefully the release of Neural Network Console/Libraries can attract a lot of talents and corporations.

What kind of reactions have you received after the release of Neural Network Console/Libraries?

For Neural Network Console, we're getting lots of positive reactions that it is very easy to understand and use from the users' perspective, since coding is unnecessary and the UI is intuitive. Also, a lot of people are saying that Neural Network Console/Libraries encouraged them to start using deep learning, and our development team is very happy to hear such feedback. Prior to public release, there were already hundreds of users within Sony, but now with the public release, we hope that a lot more new users, including students, can find it useful.

Any particular types of users you would like to recommend Neural Network Console/Libraries to?

We expect it to be of immediate help for those who are already involved in deep learning development and aspire to make it more efficient. Plus, those who are interested in deep learning but are afraid to get started, or those who are just beginning to learn deep learning, will find it helpful as well.