Memory-Centric AI, Part I:
How Kioxia’s Top Engineers Are Developing an AI That Relies on Memory

At Kioxia’s Institute of Memory Technology Research and Development, Jun Deguchi leads a team that is developing something he calls “Memory-Centric AI.” In contrast to conventional AI, which bases its decision-making on algorithms that conduct extensive calculations using immense volumes of training data, Memory-Centric AI stores its training data in external storage media and completes tasks by referencing the relevant knowledge (or memory). This type of artificial intelligence does not require retraining for every task. This not only results in lower energy usage but also has the potential to mitigate problems that arise in conventional AI systems, such as bias and the so-called “black box” problem.

Sputniko!: To start with, could you explain the difference between conventional AI and Memory-Centric AI?

Deguchi: Simply put, conventional AI is like someone taking a test without being able to use any reference materials.

Yoshimizu: To extend the metaphor, it has been trained to memorize only the parts of a textbook that will appear in the test, plus any highlighted keywords. For example, a textbook might tell the whole story of the shogun Toyotomi Hideyoshi, but the only things that the AI takes away are the terms “Toyotomi Hideyoshi” and “Osaka Castle.”

Deguchi: Meanwhile, we are working on Memory-Centric AI that can memorize the entire textbook after reading it just once. So when its knowledge is tested, it can essentially consult the entire textbook to answer the question. This means that the more reference materials the AI has on hand—the more knowledge it absorbs—the smarter it becomes. It’s a rather different approach from conventional AI.

Sputniko!: The conventional approach to AI is to initially train the model using a huge volume of data, right? The AI is “created” through this training. On the other hand, every time Memory-Centric AI does something new or makes a new decision, it does so based not on its training but on its accumulated knowledge, which it can search through and refer to in order to execute its task. Is that correct?

Deguchi: That’s exactly right. The more new knowledge and memories it accumulates, the more the AI matures. And that allows the AI to keep growing indefinitely.

Yoshimizu: To use another metaphor, conventional AI solves arithmetic problems by using a calculator to do the calculations. Memory-Centric AI, on the other hand, has an index listing all of the calculation methods and answers, so it can look up the answers and figure out which page to go to in order to solve any given problem.

Sputniko!: It recognizes that it has solved the problem before, so it can pull up the answer without having to run the calculation all over again.

Yoshimizu: Exactly.

Deguchi: Conventional AI is only able to efficiently memorize information that will be in the test, so it ignores topics that it deems irrelevant. We give it so much data—good learning data—but it can’t use the data effectively since it throws some of it away. I think that’s a major problem.

Sputniko!: The idea of an artificial intelligence capability that “grows” is interesting. As the AI solves problems, it comes into contact with a wide variety of data that it stores and remembers, making the knowledge its own. In other words, the AI is becoming more knowledgeable about the things in its orbit. And you’re saying the more time this kind of Memory-Centric AI spends with a person, the more it learns about them.

Yoshimizu: That’s right. We can think of data and models as akin to “thinking” for artificial intelligence. People grow by learning and memorizing things one at a time. In the computer world as well, storing information in memory is an important part of a machine’s growth.

Deguchi: If you raise an AI capability from a young age, just like you would a human child, the AI will grow up together with you and come to understand you very well. But to use the AI, you have to be able to trust it. Conventional AIs are not trustworthy because they are a black box. We don’t know what is going on inside the neural network—inside the AI. That’s a big problem, and it’s something that we need to figure out how to solve.

Sputniko!: I know people are discussing the issue of bias in artificial intelligence. I myself enjoyed math and computer science when I was a kid, but unfortunately, society as a whole had this stereotype that science and engineering were only for men. So I would feel very disappointed whenever I was given “girly” toys like Barbie dolls. [Laughs.]

I think the stereotypical judgments that all women must like certain things also become a problem in AI. But if I had AI that had learned about me and grown up with me from a young age, it would have bought me the trains or programming toys that I actually wanted.

Deguchi: It’s as you say. With conventional AI, there is no way to know exactly why it suggested a doll, but because a Memory-Centric AI taps into a vast store of memory to give you an answer, you can also study the background information to determine the basis for the answer. That way, if it gives an answer you don’t like, you can delete or rewrite the information easily. So it’s easy to customize and personalize for yourself.

Sputniko!: That’s so important. Black box AI is frustrating because even when you feel confused about the result, there’s nothing you can do about it. But with Memory-Centric AI, you would be able to look at the data and understand the reasoning behind its decisions.

Deguchi: This in turn will help people gradually come to believe in the trustworthiness of this kind of artificial intelligence. And that is an important step toward making personalization viable.

Sputniko!: If an AI facility could search through all of its accumulated data, it would reduce the huge number of calculations required and would use less power, right?

Deguchi: Yes. What we are working on now is developing artificial intelligence that can search through and reference its memory base. Of course that involves some calculations, but the volume is reduced because it can search its memory to do what a conventional AI would need to do with an intensive calculation. And that means that it consumes less power. The question then becomes where to store all this memory, which is where flash memory comes in.

Yoshimizu: We believe that the great advantage of flash memory lies in its ability to store information while using virtually no power.

Sputniko!: Ah! That’s super smart—I wonder why AI hasn’t been doing that all along. Until now, AI had to make huge numbers of calculations for every single decision, in a black box. But if we can accumulate enough data and knowledge (or memory), the calculation process will become more efficient.

Deguchi: In order to use stored data, artificial intelligence has to read it. That means the speed at which the AI can access the flash memory is important. If it is too slow, the AI won’t produce the necessary information. Of course, with access speeds increasing, hardware and AI technology have also evolved. I think that we are proposing our Memory-Centric AI at exactly the right moment.

Sputniko!: When you study and learn something new, it would be nice to be able to easily access that knowledge. It’s so much faster to be able to directly tap into that knowledge than to reestablish the facts from scratch. Is the hardware issue the reason there was no Memory-Centric AI until now?

Yoshimizu: We used to keep photos in albums that took up space in our cabinets and bookshelves. Then we started burning them onto CDs, DVDs, and ROM devices, but you still needed a driver to read the image data and a monitor to display the photos. Since the ability to read large amounts of data is evolving—thanks to flash memory technology—I think that it will become possible to create solutions that store records of a person’s experiences that artificial intelligence can access instantly.

Sputniko!: This whole time, I was wondering why no-one was creating artificial intelligence capable of storing knowledge when it seems to be so obviously superior to conventional AI, from both a theoretical and a practical point of view. Now I get it: it’s only now that evolutions in hardware are making those kinds of algorithms possible.

Deguchi: For conventional AI to learn something new, it needs to acquire vast amounts of data from scratch. That requires enormous computing power at the server level, which makes it difficult to develop the kind of accessible AI that can grow alongside you. Memory-Centric AI, on the other hand, stores externally every new bit of knowledge it acquires, which means someone could theoretically develop artificial intelligence capabilities on their own personal device without the need for a server.

Yoshimizu: There is now a lot of data in the cloud that artificial intelligence is processing as big data, but that data relates to everyone. That means that conventional AI can only produce results that reflect the global average.

Sputniko!: We often see stereotypical recommendations on e-commerce sites, for example.

Yoshimizu: But by communicating with personalized, easily accessible AI, you could shape it into something that suits you.

Sputniko!: That would be revolutionary. The problem of AI only producing results that reflect the average is also connected to the way in which AI generates targeted advertisements on the basis of all kinds of cloud data, whether it’s relevant to you or not. If I had artificial intelligence capabilities that looked at my personal data and used it to get to know me and to make decisions together with me, I would rely on it more.

Deguchi: There is another benefit that comes from having personalized AI. Our personal data—our privacy—is very important. We don’t feel comfortable uploading to the cloud data that we don’t want other people to see. However, in the future, you could store your data on your personal device and create an AI capability that’s based entirely on your data. When it comes to privacy, I think it’s important to have the option of artificial intelligence built around data that you don’t want to be made public.

Sputniko!: We can only use many of the examples of AI we come across if we agree to give our data to a tech giant. I think there has been a deal-with-the-devil aspect to the world of technology: we agree to give them our data in exchange for access to their useful tools. So it would be a positive development for privacy if the data that my AI uses to make decisions was stored on my own device. Also, I think it would give users an increased sense of agency or power to be able to manage and control their own data.

Deguchi: You could also, for example, incorporate and use the knowledge base of an AI capability that someone else had raised or a portion of that AI. Yasuhito’s personal AI might give one answer, but mine would generate something different. I think it will become possible to sell or exchange knowledge between different AIs.

Sputniko!: That’s fascinating! The idea of collective knowledge used to bother me because of the way it averages everyone out, but with a system like that, you could curate your own collective knowledge base. I might be able to add Jun’s perspective to my own, building up knowledge like Lego blocks. Thinking about that makes me more excited about the future!

The content and profile are current as of the time of the interview (March 2022).