Micron Introduces High Capacity High Bandwidth Memory (HBM) with Improved Performance to Enable Generative AI Innovation
Views : 132
Update time : 2023-08-02 12:01:30
Micron Technology Inc. today announced that the company has begun sampling the industry's first 8-layer stacked, 24GB capacity, second generation HBM3 memory with more than 1.2TB/s of bandwidth and a pin rate of more than 9.2Gb/s, which can deliver up to a 50 percent performance improvement over existing HBM3 solutions currently available in the market. With a 2.5x performance-per-watt increase compared to the previous generation, Micron's second-generation HBM3 products set new records for performance, capacity and energy efficiency metrics in critical artificial intelligence (AI) data centers, and will help the industry shorten the training time for large-scale language models, such as GPT-4 and higher, provide an efficient infrastructure for AI reasoning, and reduce total cost of ownership (TCO).
Micron introduced a High Bandwidth Memory (HBM) solution based on the industry's leading-edge 1β DRAM process node, which packages 24Gb DRAM die into industry-standard-sized 8-layer stacked modules. In addition, Micron's 12-layer stacked 36GB capacity product will begin sampling in the first quarter of 2024. Micron's HBM3 solution offers a 50 percent increase in capacity compared to other 8-layer stacked solutions currently on the market. The improved performance-to-power ratio and pin speeds of Micron's second-generation HBM3 products are critical to managing the extreme power demands of today's AI data centers. Micron has achieved significant improvements in energy efficiency through technology innovations such as doubling the number of silicon through vias (TSVs), increasing metal density by 5x to reduce thermal resistance, and designing more energy efficient datapaths compared to other HBM3 solutions in the industry. Micron, a longtime storage leader in 2.5D/3D stacking and advanced packaging technologies, is honored to be a partner member of TSMC's 3DFabric Alliance to build the future of semiconductor and system innovation. Micron and TSMC have worked hand-in-hand during the development of the second-generation HBM3 products, laying the groundwork for the smooth introduction and integration of compute systems in AI and high-performance computing (HPC) design applications. TSMC has now received samples of Micron's second-generation HBM3 memory and is working closely with Micron on the next steps of evaluation and testing to help customers innovate their next-generation HPC applications. Micron's second-generation HBM3 solution addresses the growing demand for multimodal, multi-trillion parameter AI models in the generative AI space. With a single module capacity of up to 24GB and a pin rate of more than 9.2Gb/s, the Micron HBM3 can reduce total cost of ownership by reducing training time for large language models by more than 30 percent. In addition, Micron HBM3 will trigger a significant increase in daily queries, resulting in more efficient utilization of trained models. With industry-leading performance per watt, Micron's second-generation HBM3 memory will effectively drive operational savings in the modern AI data center. In 10 million deployed graphics processing unit (GPU) use cases, a single HBM module can save 5W of power consumption, saving up to $550 million in operating expenses over five years. Micron's second-generation HBM3 solutions are designed to deliver superior AI and high-performance computing solutions to our customers and the industry," said Praveen Vaidyanathan, vice president and general manager, Compute Products Business Group, Compute and Networking Division, Micron. A key consideration for us was the ease of integration of the product on our customers' platforms. Micron HBM3 features fully programmable Memory Built-In Self-Test (MBIST), which runs at full specification pin speeds, enabling Micron to provide customers with enhanced test capabilities, enabling efficient collaboration and helping customers reduce time-to-market." Ian Buck, vice president of hyperscale and high-performance computing at NVIDIA, said, "At the heart of generative AI is accelerated computation, where the high bandwidth of HBM is critical and leads to superior power efficiency. We have a long-standing relationship with Micron across a number of product areas and are very much looking forward to continuing to work with Micron on the second generation of HBM3 products to accelerate AI innovation." The second-generation HBM3 is another milestone in Micron's leading-edge technology. Micron previously announced a 96GB capacity DDR5 module for high-capacity server solutions based on a 1-alpha (1-alpha) node with a 24Gb single DRAM die; now it is launching a 24GB capacity HBM3 product based on a 1β node with a 24Gb single DRAM die, and is planning to launch a 128GB capacity DBM3 product in the first half of 2024 based on a 1β node with a 32Gb single DRAM DDR5 modules with 128GB capacity based on 1β node, 32Gb single DRAM die. These new products demonstrate Micron's cutting-edge technology innovation in the AI server space.
The LDK220 series of low-dropout linear regulators (LDOs) is a high-performance device designed specifically for low-power consumption and high-precision voltage regulation, widely used in scenarios such as consumer electronics, industrial control, and po
Xilinx Spartan®-7 FPGA Family stands as a defining solution in the mid-range FPGA landscape, blending high performance, energy efficiency, and cost-effectiveness to redefine versatility for industrial, IoT, and consumer electronics applications. Built on
The Altera FLEX series was more than a lineup of FPGAs—it was a blueprint for how programmable logic devices could evolve to meet diverse market needs. The FLEX 8000 laid the architectural groundwork, the FLEX 10K redefined functionality with embedded mem
Among TI standout offerings, the LM4765 and LM4766 are dual-channel amplifiers designed to cater to diverse audio needs—from compact setups to high-fidelity systems. While sharing the same product lineage, these chips differ significantly in power output,
