Samsung claims its HBM-PIM power-in-memory architecture will more than double system performance and reduce power consumption by over 70%.
Samsung adds an AI processor to its high-speed memory to reduce bottlenecks. (Samsung Electronics)
A few months ago, Samsung and Xilinx have co-introduced an SSD with an on-board Xilinx FPGA processor, to bring more processing capacity to storage resources. Flash drive allows data to be processed where it is rather than moving it to and from memory. Today, bothvendors are pushing high bandwidth memory (HBM) integrated with an AI processor, called HBM-PIM. This in-memory processing architecture (PIM) enables the integration of artificial intelligence capabilities into memory rather than moving content to the processor, in order to speed up large-scale processing in data centers, High Performance Computing (HPC) systems and mobile applications using AI.
HBM is a different type of memory than DRAM. It is stacked vertically rather than spread out in 2D, so that data moves less far in memory. As a result, HBM chips are smaller than DRAMs but also more expensive, around 30% more than DRAMs of comparable capacity. HBM components are found mostly on CPUs and GPUs, and they are usually placed physically right next to the chip to reduce data movement. Applications requiringHPC and AI resources are particularly suitable for exploiting these capabilities, as companies' budgets are larger than those of the general public, mainly gaming.
Limit memory bottlenecks
Samsung notes that most of today's computer systems are based on the von Neumann architecture which uses separate processor and memory units to perform data processing tasks. This means that data is moved back and forth between CPU and memory, through the chipset and processor buses. This sequential processing approach requires data to constantly move back and forth, resulting in a bottleneck slowing down the system, especially when processing ever-increasing volumes of data. For years, CPU vendors have tried to compensate by increasing the size of L1 to L3 caches, to the point that they takeent more space than the CPU core.
In contrast, the HBM-PIM architecture brings processing power directly to where data is stored by placing an optimized IA / DRAM engine at the inside each memory bank - a storage subunit - allowing parallel processing and minimizing data movement. Applied to Samsung's HBM2 Aquabolt memory (16 GB with a bandwidth of 3.2 Gbps), this architecture can increase system performance - x2 according to Samsung - while reducing power consumption by over 70 %. HBM-PIMs also do not require any hardware or software modifications, which allows for faster integration into existing systems.
" Our revolutionary HBM-PIM is the first PIM solution programmable industry suitable for
Samsung's HBM-PIM is currently being tested in AI accelerators by major AI platform vendors, and all validations are expected to be completed in the first half of this year.