Google Titans: AI with Human Memory

Google's new architecture, Titans, features human-like memory and learning processes - big upgrades that will shake up the AI capability.

Google’s latest creation, Titans, reimagines neural network architecture and brings new capabilities to AI systems. A direct evolution of Google’s Transformer architecture, Titans features human-esque memory upgrades, optimising it for incredibly sophisticated data analytics. Many are curious about its potential to form the new backbone of LLMs.

What is an AI architecture?

Both Titans and Transformers are Google AI architectures – but what does that mean? AI architecture is the system’s end-to-end structure – an organised framework of different components and processes that controls how data flows through it. The different components are layered together, and their configuration defines things like how the system should perform certain tasks.

Typically, there are components that handle data collection and pre-processing, feature extraction, training and learning, inference, and evaluation/feedback. The model layer is the core, where particular algorithms (like ML or Deep Learning models) process the data coming into the system.

User interfaces and interaction layers are what allow us to engage with these AI systems so seamlessly, through through chatbots, virtual assistants, or recommendation engines.

Titans VS Transformer

When Google released their Transformer architecture nearly eight years ago, they changed the world. Lying now at the heart of Open AI’s ChatGPT and so many more, the Transformer architecture offered so much to our industry. Their new Titans architecture represents a step forward in the AI evolutionary chain, with the addition of neural long-term memory.

Human thought and reasoning rely on the ability to remember information over extended periods of time. As Google point out in their paper, memory is fundamental to the learning process, and without it our capacities would be incredibly restricted. The Titans architecture replicates the human memory system, giving it an unprecedented recall power that Transformer lacks.

How does Titans work?

Titans architecture combines an attention mechanism with a long-term memory module and a surprise based learning system. The attention mechanism works by “spotlighting” key words and data from input – Transformers have this too. In Titans, the attention mechanism works in tandem with a long-term memory module, which acts as a huge data library for storing important information. This allows Titans to handle massive data flows without losing its thread, and keep focussed on critical points.

Google also gave Titans an intelligent “surprise metric” – a unique approach to memory management. Any data that goes against the system’s expectations is prioritised in the long-term memory module. Not only does this better reflect how human memory works, but it helps avoid the common problems of limited AI memory resources. A “decaying mechanism” then gradually “forgets” less important details over time.

Why give AI human-like memory?

In testing, Titans gives promising results. It outperforms other models in language modelling tasks, keeping consistently accurate output even when increasing input length. Its a versatile and high performing architecture, with potential for broad application across the field of AI.

As of now, Titans is the closest AI replication of human-like cognition. It is paving the way for intuitive, flexible AI systems able to manage the heavy loads of information that humans can. It could revolutionise research, giving AI assistants the capacity to remember years worth of data or literature. For medicine and finance, catching anomalies in huge datasets is crucial, in spotting abnormal scan results or suspicious financial transactions. Titans architecture, with its “surprise metric” and unique memory management function could add incredible value to AI in these industries. The benefit Titans will bring to hundreds of different sectors is going to be revolutionary.

We’re always interested to hear what our network thinks of these topics. What could Google Titans do for your business or industry? Is giving AI human-like memory a step in the right direction, or do you have reservations? Comment below – let us know what you think.

• LinkedIn

Contact us to discuss your AI and Data Analytics project – info@fidra.ai

Back to Insights