SEF project logo
A few days ago the Linux Foundation announced the first release of an open platform for software-driven Flash storage, called SAFE (Software Enabled Flash). The associated SDK is a generous contribution from KIOXIA, a storage company that spun off from Toshiba several years ago.
The Software-Enabled Flash project marks a transformative milestone for developers seeking exceptional access to the creative potential of flash storage applications.
About Software-Enabled Flash
Software-Enabled Flash It is made up of a set of tools which includes a series of patches for the Linux kernel, a block driver designed for SEF devices (Software Enabled Flash), utilities for command line administration, paravirtualized SEF drivers for QEMU, a library that provides an API for application development, patches for nvme-cli and FIO that add support for SEF.
The donated hardware includes the SEF Project SDK, which gives data center storage developers access to sample code and the ability to realize the full potential of flash media control. This encompasses WAF reduction, latency control, support for multiple protocols such as ZNS and FDP, or Block, and promises future capabilities through this dynamic and programmable software-defined flash approach.
Additionally, the SDK has a reference implementation of the FTL software layer (Flash Translation Layer), responsible for translating block device commands into accesses to the real Flash memory chip. This layer is also responsible for garbage collection and data distribution between memory cells. The FTL implements support for flexible data placement (FDP – Flexible Data Placement), zoning (ZNS – Zoned Namespace) and complies with the NVMe specification.
It is mentioned that in the case of conventional Flash storage for external systems, the drive is perceived as a black box where a portion of the memory is allocated for cleaning operations, while performance indicators tend to be inconsistent, and delays are experienced unpredictable due to internal driver activity, such as during garbage collection.
“We are excited to reach this milestone and release the software development kit for the Software-Enabled Flash project,” said Eric Ries, senior vice president of the Memory and Storage Strategy Division at KIOXIA America, Inc. “This release is now "It paves the way for new and innovative uses of flash for storage applications that harness the creativity and inventiveness of users, developers and open source communities."
Unlike conventional flash drives, where data distribution, bad block isolation and garbage collection are handled by the internal controller firmware, Units with SEF support allow job logic to be transferred low level with the Flash memory chip to the software and operating system.
SEF enables direct control of the physical location of data, the modification of load distribution algorithms, priority management and service quality (QoS), the elimination of an additional layer of block translation, the full utilization of available memory, the achievement of predictable operation of the drive, and the division of the drive into isolated virtual devices in input/output operations ( I/O) with different processing priorities. The approach proposed by SEF contributes to reducing overall costs and increasing efficiency in interaction with the drive, adapting garbage collection logic, memory reservation and block allocation to meet current needs.
For example, SEF allows performance to be normalized by using a variety of units purchased at different times with different characteristics. For drives that are primarily read-only, SEF can employ simpler block allocation algorithms and reduce the amount of memory reserved for general purposes.
Finally, it is worth mentioning that the SEF SDK is written in C and is distributed under the BSD license along with the API and command specifications. You can check the code from SEF on GitHub.
Source: https://www.linuxfoundation.org