The chip giant teams up with LightBits to make faster NVMe/TCP and better storage solution

Intel Corporation and Lightbits Labs announced they would collaborate on developing new "disaggregated storage solutions" for data centers. This announcement came on Tuesday, where the companies expressed their objective to solve the challenges of today's data center operators who are craving improved total-cost-of-ownership (TCO) due to stranded disk capacity and performance amid the pandemic. Intel has acquired an undisclosed equity stake in Lightbits Labs, a developer of composable NVMe-over-TCP storage technology. The latter has developed LightOS, a software-defined block storage platform that pools NVMe storage across multiple storages into a high-performance virtual all-flash storage array.

In its blog post, Intel mentioned that LightOS™ product offers high availability and read-and-write management designed to maximize the value of flash-based storage. Unlike most NVMe storage systems, LightOS connects the NVMe media over industry-standard TCP networking, replacing the need for more specialized storage networks such as Fibre Channel. LightOS, while being fully optimized for Intel® hardware, provides customers with vastly improved storage efficiency and reduces underutilization while maintaining compatibility with existing infrastructure without compromising performance and simplicity.

Eran Kirzner, Lightbits Labs CEO, issued a quote: "We are excited to partner with Intel Corporation, and our joint solutions will set the bar for generating new ROI metrics for enterprise and cloud customers." Moreover, the NVMe/TCP all-flash arrays will be almost as fast as RoCe and InfiniBand options, which require much more expensive cabling. Lightbits and Intel have demonstrated NVMe/TCP with a 146μs access latency using Intel's Ethernet 800 Series network adapters with application device queues (ADQ) technology. As per an official statement, ADQ enables NVMe oF/TCP to achieve distributed storage performance in the same range as RDMA-based protocols, meaning 100 – 120μs. Further, it helps with broad adoption because of its ease of deployment and scalability. Results show an up to 30% improvement in response time predictability as measured by P99.99 tail latency, up to 50% reduction in average latency, and up to 70% throughput increase as measured in IOPS when using ADQ vs. without ADQ.

Besides, the system will also utilize Intel Optane persistent memory and Intel 3D NAND SSDs based on Intel QLC Technology, Intel Xeon Scalable processors with unique built-in artificial intelligence acceleration capabilities apart from the Ethernet 800 network adapters. Intel Optane is required as it provides large volumes of memory at a lower cost than DRAM, which is important for the metadata. These Optane DIMMs will make the array operations faster, while QLC SSDs will send capacity up and cost/GB down. The problem with consumer-grade QLC is that though it provides high-performance reads, it writes data slowly and has issues if data is written to the memory too many times. LightOS completes writes in a nonvolatile buffer and then writes large chunks of data to QLC. This will let customers take full advantage of the low-cost QLC media while extending the media's life. Meanwhile, the TCO variables can include energy usage, cooling requirements, location, and equipment choice.

Josh Goldenhar, vice president of product marketing for Lightbits Labs, says, "The company also expects to add data snapshot capabilities by year-end." He informs that, at present, clusters of up to 16 hosts are supported, and we can expect with 64-host and 128-host clusters since there's no limit on cluster size. LightBits Labs had released LightOS 2.0 during the summer. LightOS 2.0 comes with an updated storage framework that targets Kubernetes deployments as persistent volume support.

The current pandemic has forced businesses to look for innovative ways to survive under constrained budgets without compromising on productivity and delivering secure and compliant services to remote workers across distributed environments. While Intel dominates the server processor industry currently, Advanced Micro Devices (AMD) and NVIDIA have made huge financial profits in the graphics processor units (GPUs) market during this phase. Hence, through this collaboration, Intel aims to protect its business interests and maintain its market dominance while profiting from the massive spike in GPUs demand buy companies trying to accommodate their remote operations.

According to Remi EL-Ouazzane, vice president and Data Platforms Group chief strategy and business development officer at Intel, the data center is being transformed, with disaggregation and composability of resources essential to meet the efficiency requirements needed to address the explosion of data. "Our differentiated hardware capabilities coupled with Lightbits innovative NVMe over Fabrics software, gives our joint customers an exceptional economic solution to address this strategic inflection point," he adds.