The Evolution of Storage in Kubernetes

When Kubernetes was created over 10 years ago, its only purpose was to schedule and orchestrate stateless workloads. Since then, a lot has changed, and Kubernetes is increasingly used for stateful workloads. Not just basic ones but mission-critical workloads, like a company’s primary databases. The promise of workload orchestration in infrastructures with growing complexity is too significant.

Anyhow, traditional Kubernetes storage solutions relied upon and still rely on old network-attached storage protocols like iSCSI. Released in 2000, iSCSI was built on the SCSI protocol itself, first introduced in the 1980s. Hence, both protocols are inherently designed for spinning disks with much higher seek times and access latencies. According to our modern understanding of low latency and low complexity, they just can’t keep up.

While these solutions worked well for basic containerized applications, they fall short for high-performance workloads like databases, AI/ML training, and real-time analytics. Let’s look at the NVMe standard, particularly NVMe over TCP, which has transformed our thinking about storage in containerized environments, not just Kubernetes.

Why NVMe and Kubernetes Work So Well Together

The beauty of this combination lies in their complementary architectures. The NVMe protocol and command set were designed from the ground up for parallel, low-latency operations–precisely what modern containerized applications demand. When you combine NVMe’s parallelism with Kubernetes’ orchestration capabilities, you get a system that can efficiently distribute I/O-intensive workloads while maintaining microsecond-level latency. Further, comparing NVMe over TCP vs iSCSI, we see significant improvement in terms of IOPS and latency performance when using NVMe/TCP.

Consider a typical database workload on Kubernetes. Traditional storage might introduce latencies of 2-4ms for read operations. With NVMe over TCP, these same operations complete in under 200 microseconds–a 10-20x improvement. This isn’t just about raw speed; it’s about enabling new classes of applications to run effectively in containerized environments.

The Technical Symphony

The integration of NVMe with Kubernetes is particularly elegant through persistent volumes and the Container Storage Interface (CSI). Modern storage orchestrators like simplyblock leverage this interface to provide seamless NVMe storage provisioning while maintaining Kubernetes’ declarative model. This means development teams can request high-performance storage using familiar Kubernetes constructs while the underlying system handles the complexity of NVMe management, providing fully reliable shared storage.

The NMVe Impact: A Real-World Example

But what does that mean for actual workloads? Our friends over at Percona found in their MongoDB Performance on Kubernetes report that Kubernetes implies no performance penalty. Hence, we can look at the disks’ actual raw performance.

A team of researchers from the University of Southern California, San Jose State University, and Samsung Semiconductor took on the challenge of measuring the implications of NVMe SSDs (over SATA SSD and SATA HDD) for real-world database performance.

The general performance characteristics of their test hardware:

Their resume states, “scale-out systems are driving the need for high-performance storage solutions with high available bandwidth and lower access latencies. To address this need, newer standards are being developed exclusively for non-volatile storage devices like SSDs,” and “NVMe’s hardware and software redesign of the storage subsystem translates into real-world benefits.”

They’re closing with some direct comparisons that claim an 8x performance improvement of NVMe-based SSDs compared to a single SATA-based SSD and still a 5x improvement over a [Hardware] RAID-0 of four SATA-based SSDs.

Transforming Database Operations

Perhaps the most compelling use case for NVMe in Kubernetes is database operations. Typical modern databases process queries significantly faster when storage isn’t the bottleneck. This becomes particularly important in microservices architectures where concurrent database requests and high-load scenarios are the norm.

Traditionally, running stateful services in Kubernetes meant accepting significant performance overhead. With NVMe storage, organizations can now run high-performance databases, caches, and messaging systems with bare-metal-like performance in their Kubernetes clusters.

Dynamic Resource Allocation

One of Kubernetes’ central promises is dynamic resource allocation. That means assigning CPU and memory according to actual application requirements. Furthermore, it also means dynamically allocating storage for stateful workloads. With storage classes, Kubernetes provides the option to assign different types of storage backends to different types of applications. While not strictly necessary, this can be a great application of the “best tool for the job” principle.

That said, for IO-intensive workloads, such as databases, a storage backend providing NVMe storage is essential. NVMe’s ability to handle massive I/O parallelism aligns perfectly with Kubernetes’ scheduling capabilities. Storage resources can be dynamically allocated and deallocated based on workload demands, ensuring optimal resource utilization while maintaining performance guarantees.

Simplified High Availability

The low latency of NVMe over TCP enables new approaches to high availability. Instead of complex database replication schemes, organizations can leverage storage-level replication (or more storage-efficient erasure coding, like in the case of simplyblock) with a negligible performance impact. This significantly simplifies application architecture while improving reliability.

Furthermore, NVMe over TCP utilizes multipathing as an automatic fail-over implementation to protect against network connection issues and sudden connection drops, increasing the high availability of persistent volumes in Kubernetes.

The Physics Behind NVMe Performance

Many teams don’t realize how profoundly storage physics impacts database operations. Traditional storage solutions averaging 2-4ms latency might seem fast, but this translates to a hard limit of about 80 consistent transactions per second, even before considering CPU or network overhead. Each transaction requires multiple storage operations: reading data pages, writing to WAL, updating indexes, and performing one or more fsync() operations. At 3ms per operation, these quickly stack up into significant delays. Many teams spend weeks optimizing queries or adding memory when their real bottleneck is fundamental storage latency.

This is where the NVMe and Kubernetes combination truly shines. With NVMe as your Kubernetes persistent volume storage backend, providing sub-200μs latency, the same database operations can theoretically support over 1,200 transactions per second–a 15x improvement. More importantly, this dramatic reduction in storage latency changes how databases behave under load. Connection pools remain healthy longer, buffer cache decisions become more efficient, and query planners can make better optimization choices. With the storage bottleneck removed, databases can finally operate closer to their theoretical performance limits.

Looking Ahead

The combination of NVMe and Kubernetes is just beginning to show its potential. As more organizations move performance-critical workloads to Kubernetes, we’ll likely see new patterns and use cases that fully take advantage of this powerful combination.

Some areas to watch:

• AI/ML workload optimization through intelligent data placement
• Real-time analytics platforms leveraging NVMe’s parallel access capabilities
• Next-generation database architectures built specifically for NVMe on Kubernetes Persistent Volumes

The marriage of NVMe-based storage and Kubernetes Persistent Volumes represents more than just a performance improvement. It’s a fundamental shift in how we think about storage for containerized environments. Organizations that understand and leverage this combination effectively gain a significant competitive advantage through improved performance, reduced complexity, and better resource utilization.

For a deeper dive into implementing NVMe storage in Kubernetes, visit our guide on optimizing Kubernetes storage performance.

The post NVMe & Kubernetes: Future-Proof Infrastructure appeared first on simplyblock.

In preparation for KubeCon + CloudNativeCon, we at simplyblock figured it was essential to create a guide beyond the conference halls, highlighting the best spots to truly experience Salt Lake City. As a remote only company, we believe this short escapade, with the chance to get the team together, shouldn’t just be about work but also enjoying the local culture, food, and fun. But we’re not the gatekeeping type over here, so we want to share this guide with you in the hope to inspire you to make the most of your time there. After all, work hard, play hard—that’s our motto, and we hope you’ll join us in embracing both during the conference.

Salt Lake City’s TRAX system is your go-to for easily navigating the city. The straightforward routes and convenient stops make it a breeze to get around, much like finding the right Kubernetes pod. Grab a day pass for unlimited travel, and you’ll be ready to explore the city without any detours. To make your journey even smoother, check out this link for all the details on how to ride TRAX, including tips on routes, stops, and purchasing passes.

Here’s a link to the Google Maps location for all the places to make it more convenient.

By the way, you might see us at one of these places, so don’t be shy about greeting us. The worst we can do is improve your storage efficiency. Alright, I’ll stop pitching, read on and make your plans!

🏛️ Landmark Highlights

Salt Lake City is rich in history and architecture, with several landmarks to explore. Don’t miss the Cathedral of the Madeleine (1) ( maps ), a beautiful example of Romanesque and Gothic Revival design. Inside, admire vibrant stained glass, intricate woodwork, and serene murals. Whether attending a service or simply enjoying the peaceful atmosphere, it’s a must-visit spot in the city.

Secondly, don’t miss the Utah State Capitol (2) ( maps ), a stunning architectural landmark perched on Capitol Hill. This neoclassical building offers free guided tours and breathtaking views of Salt Lake City and the surrounding mountains. Inside, explore the rich history of Utah through various exhibits, and enjoy the peaceful ambiance of the beautifully landscaped grounds, perfect for a leisurely stroll.

❄️ Winter Thrills

Salt Lake City isn’t just a hub for tech enthusiasts during KubeCon + CloudNativeCon—it’s also a vibrant winter wonderland that offers something for everyone. After you’ve tackled all the conference sessions, why not unwind by ice skating at the Gallivan Center ? (3) ( Maps ) This charming downtown rink is a perfect way to enjoy the festive atmosphere—just remember to bundle up, because even tech pros feel the chill!

For a more serene experience, visit Temple Square (4) ( Maps ), which turns into a dazzling holiday light display in the winter—think of it as a peaceful stroll through a winter wonderland that’ll definitely brighten your day.

🍽️ Food Adventures Await

Heading to KubeCon + CloudNativeCon in Salt Lake City and eager to dive into the local food scene? We’ve got you covered with some of the city’s top dining spots, each offering a unique taste of Salt Lake’s diverse culinary landscape. For authentic Mexican cuisine, head to Red Iguana **** (5) **** ( Maps ), famous for its rich and flavorful moles that will keep you coming back for more. If you’re in the mood for a contemporary twist on American classics, The Copper Onion (6) ( Maps ) serves up bold, locally sourced dishes like succulent pork belly and house-made pastas. For sushi lovers, Takashi (7) ( Maps ) is a must-visit, offering fresh, innovative rolls and sashimi that rival coastal cities.

HSL (8) ( Maps ) provides a chic yet cozy atmosphere with modern American dishes like roasted chicken with seasonal veggies, perfect for unwinding after a day of sessions. And when you need a coffee break, The Rose Establishment (9) ( Maps ) is your go-to spot for quality coffee and artisanal pastries—a cozy hideaway to recharge between conference activities.

🍻 Don’t Forget to Enjoy the Night

Looking for the perfect spot to unwind after a day at KubeCon + CloudNativeCon in Salt Lake City? Here are some top picks for a fun and relaxing evening. Start with The Red Door (10) ( Maps ), a cozy lounge with a speakeasy vibe, where you can sip on craft cocktails and decompress after all the tech talk.

If you’re in the mood for some nostalgia, head to Quarters Arcade Bar (11) ( Maps ), where you can relive your childhood with classic arcade games while enjoying a drink. For craft beer enthusiasts, Beer Bar (12) ( Maps ) offers a wide selection of local brews in a laid-back atmosphere, making it an ideal spot to kick back with colleagues.

Whiskey Street (13) ( Maps ) is the place for those who appreciate a good whiskey. Its extensive selection, lively atmosphere, and perfect blend of elegance and comfort make it a great spot to enjoy expertly crafted cocktails and delicious food.

⛷ Salt Lake Ski Resorts

Salt Lake City is the ultimate destination for skiing in November, thanks to its unique combination of weather, terrain, and resort accessibility. Nestled at the base of the Wasatch Mountains (worth visiting the area in and of itself), Salt Lake benefits from early-season snowfalls that blanket the region’s world-renowned ski resorts, often making them operational as early as mid-November. The area’s terrain is diverse – ranging from steep and challenging slopes for experts to wide and groomed runs perfect for beginners, ensuring a tailored experience for every skier.

Moreover, Salt Lake City is just a short drive from over 10 major ski resorts , making it incredibly convenient for visitors to access top-tier slopes without long travel times. The efficient transport system, in the form of the ski bus can help you travel to and enjoy all the different ski slopes. If you don’t want to travel by bus, there are multiple options here to pick from. In addition to this, you might also want to check out the bundled pricing (with additional perks) for ski resorts.

This may be subjective, but according to our highly adept (world-class) skiing experts at simplyblock, we have come up with the list of the top 5 resorts that you may want to visit:

These are more for the experienced, and you can get a bundled pass for both here (also known to be in an area called Little Cottonwood):

Alta Ski Area ( Maps ): Alta Ski Area, one of the first ski areas in the U.S., is renowned for its steep terrain and deep powder, averaging 546 inches of snow annually. Covering 2,614 acres with a 2,538-foot vertical drop, 45% of the terrain caters to beginner and intermediate skiers. Six lifts provide access to its varied slopes, making it a top destination for experienced skiers. However, you can find multiple resorts in the Alta ski area, our favorite pick is Snowpine lodge ( Maps ). You can view the prices here . Snowbird Resort ( Maps ): Snowbird’s top elevation is 11,000 feet at Hidden Peak, with a 7,760-foot base at Baby Thunder. The resort’s tram ascends 2,900 vertical feet in just 7 minutes. It gets 500 inches of dry Utah powder annually. SKIING Magazine has ranked Alta and Snowbird the No. 1 resort in the United States. You can view the prices here .

These are more for fun and the beginner (also known to be in an area called Big Cottonwood):

Brighton Resort (Maps): Brighton Resort offers 1,050 acres of skiable terrain with a 1,745-foot vertical drop. Receiving 500 inches of snow annually, Brighton is family-friendly and accessible, with 66 runs and lifts that include five quads, a triple, and a magic carpet. Night skiing is available on 200 acres, making it a well-rounded destination just 35 miles from Salt Lake City. You can view the prices here . Solitude Resort ( Maps ): Solitude Mountain Resort spans 1,200 acres with a 2,494-foot vertical drop and receives 500 inches of snow each year. With 87 runs catering to all skill levels, the resort offers a peaceful skiing experience. Its eight lifts, including four high-speed quads, provide quick access to varied terrain, from groomed runs to powder-filled glades, making Solitude an ideal escape for skiers of all levels. You can view the prices here .

And a bonus tubing and skiing location, the home of the 2002 Winter Olympics:

Soldier hollow Nordic Centre (Maps): Nestled in Wasatch Mountain State Park, Soldier Hollow is famous for its Olympic legacy and offers over 20 miles of cross-country skiing trails through scenic landscapes. It’s a winter paradise for both athletes and the public, with activities like cross-country skiing, snowshoeing, and even public biathlon courses. A highlight is the snow tubing lanes, the longest in Utah, stretching over 1,200 feet. The day lodge provides ski rentals and food, making it an ideal spot for a winter retreat and family fun. We think Zermatt Utah Resort ( Maps ) is relatively close and reasonable. You can find the locations of all of the skiing resorts below and on this map list:

📜 Closing Note

If you’re attending KubeCon + CloudNativeCon NA 2024 in Salt Lake City, we’re excited to let you know that simplyblock will be there too!

Simplyblock offers cutting-edge storage orchestration solutions tailored for IO-intensive stateful workloads in Kubernetes, including databases and analytics. A single system seamlessly connects local NVMe disks, GP3 volumes, and S3, making it easier to manage storage capacity and performance. With smart NVMe caching, thin provisioning, storage tiering, and volume pooling, we enhance database performance while reducing costs—all without requiring changes to your existing AWS infrastructure.

We invite you to visit the simplyblock booth at the event! Swing by to learn more about how we can optimize your storage solutions and pick up some exclusive freebies. We can’t wait to meet you and discuss how we can help improve your Kubernetes experience. See you there!

The post KubeCon + CloudNativeCon NA 2024: Your Salt Lake City Guide appeared first on simplyblock.