Co-Designing NVM-based Systems for Machine Learning Applications

We will discuss employing non-volatile memory technologies like phase change memory (PCM) and memristor  devices to enhance machine learning (ML) capabilities in edge environments. Both software and hardware approaches  covered to implementing these technologies. We explore using PCM's write modes to speed up Convolutional Neural  Networks (CNNs) by dynamically adjusting write modes based on retention time (software approach). Additionally,  we examine how these disruptive memory technologies can impact ML performance and propose architectural and  system-level optimizations to improve efficiency and reduce energy consumption (hardware approach). Our goal is to  optimize ML execution in resource-constrained edge settings by addressing challenges related to non-volatile  memory's write latency and endurance and leveraging their underlying execution properties to improve energy  efficiency.

Large-scale search is prevalent in machine learning applications, but accelerating such search operations faces  challenges due to the data scale and irregular memory accesses. Non-volatile memory based content addressable  memories (CAMs) have emerged as a promising solution for accelerating large-scale search tasks. However, selecting  the most suitable NVM devices and architectures for implementing CAMs is a complex problem. This talk will explore  how different choices of NVM devices and architectures can impact the overall performance of in-memory search  accelerators. The insights gained from this discussion will provide valuable guidance to both device researchers and  architects in their endeavor of developing high-performance accelerators for large-scale search tasks.

Machine Learning research often centers on high-level models, neglecting the impact of underlying execution  properties of memory technologies. Emerging memory technologies bring forth promising features while also  presenting significant challenges in system design, including access-dependent latency (as observed in racetrack  memory), aging, and reliability issues. It is therefore essential to account for such hardware-dependent properties  when deploying machine learning models. By incorporating an understanding of the properties of memory  technologies, we can develop architectural-level solutions and system-level mechanisms to enhance performance  and/or reduce energy consumption. This is particularly vital for sustainability. In this presentation, we will share our  recent insights and experiences in memory-centric deployment tailored for machine learning models.

This talk will provide a review of the current state of various non-volatile memory technologies being investigated in  both industry and academia. The advantages and disadvantages of each NVM device will be analyzed. Factors that  impact their adoption will be discussed.