Understanding the Challenge of Cloud Infrastructure Growth

Cloud infrastructure footprint extends beyond simple storage consumption. It encompasses the entire ecosystem of resources, including compute instances, databases, storage volumes, networking components, and their complex interactions. It’s not only a problem of companies on-prem. The most significant impact of data center carbon footprint in the public clouds still depends on the end user. Overprovisioning, usage of inefficient technologies, and lack of awareness are some of the problems contributing to the fast-growing cloud’s carbon footprint.

One thing that is often overlooked is storage. Storage, unlike compute, needs to be on at all times. Traditional approaches to storage provisioning usually lead to significant waste. For instance, when deploying databases or other data-intensive applications, it’s common practice to overprovision storage to ensure adequate capacity for future growth. This results in large amounts of unused storage capacity, unnecessarily driving up wastage and carbon emissions.

Environmental Impact of Cloud Data Center Footprint

According to the International Energy Agency, digital infrastructure’s environmental impact has reached staggering levels, with data centers consuming approximately 1% of global electricity use. To put this in perspective, a single petabyte of actively used storage in traditional cloud environments has roughly the same annual carbon footprint as 20 round-trip flights from New York to London. Overall, data centers contribute more to carbon emissions than the whole aviation industry.

Optimizing Compute Resources

Let’s first look at compute, which is the most often mentioned regarding data center footprint. Modern cloud optimization platforms like Cast AI have revolutionized compute resource management with the help of ML-based algorithms. Organizations can significantly reduce compute costs while maintaining performance by analyzing workload patterns and automatically adjusting instance types and sizes. Cast’s customers typically report 50-75% cost savings on their Kubernetes compute costs through automated instance optimization.

For AWS users, some tools enable organizations to leverage lower-cost spot instances effectively. Modern workload orchestrators can automatically handle spot instance interruptions, making them viable even for production workloads. There are also “second-hand” instance marketplaces, where users can “give back” their reserved instances they don’t need anymore. These solutions might have a considerable impact not only on carbon footprint reduction but also on savings.

Modern Approaches to Storage Optimization

Storage optimization has evolved significantly in recent years. Modern solutions like simplyblock have introduced innovative approaches to storage management that dramatically help reduce your cloud footprint while maintaining or even improving performance.

Thin Provisioning as a Rescue

One of the most effective strategies for reducing storage footprint is thin provisioning. Unlike traditional storage allocation, where you must pre-allocate the full volume size, thin provisioning allows you to create volumes of any size while only consuming the actual used space. This approach is compelling for database operations where storage requirements can be challenging to predict.

For example, a database service provider might need to provision 1TB volumes for each customer instance. With traditional provisioning, this would require allocating the full 1TB upfront, even if the customer initially only uses 100GB. Thin provisioning allows the creation of these 1TB volumes while only consuming the actual space used. This typically results in a 60-70% reduction in actual storage consumption and carbon footprint.

Sounds like a no-brainer? Well, cloud providers don’t allow you to use thin provisioning out of the box. The same applies to managed database services such as Amazon RDS or Aurora. The cloud providers use thin provisioning to benefit their own operations, but eventually, it still leads to wastage on the database level. Technologies like simplyblock come in handy for those looking to use thin provisioning in public cloud environments.

Intelligent Data Tiering

Not all data requires high-performance, expensive storage. Modern organizations are increasingly adopting various strategies to optimize storage costs while maintaining performance where it matters. This involves tiering data within a particular storage type (e.g., for object storage between S3 and Glacier) or, as in the case of simplyblock’s intelligent tiering, automatically moving data between different storage tiers and services based on access patterns and business requirements.

Take the example of an observability platform: recent metrics and logs require fast access and are kept in high-performance storage, while historical data can be automatically moved to more cost-effective object storage like Amazon S3. Simplyblock’s approach to tiering is particularly innovative, providing transparent tiering that’s entirely invisible for applications while delivering significant cost savings.

Maximizing Storage Efficiency Through Innovative Technologies

Modern storage solutions offer several powerful technologies for reducing data footprint, including:

• Compression: Reduces data size in-line before writing
• Deduplication: Eliminates redundant copies
• Copy-on-write: Creates space-efficient snapshots and clones

Compression and deduplication work together to minimize the actual storage space required. Copy-on-write technology enables the efficient creation of database copies for development and testing environments without duplicating data.

For instance, when development teams need database copies for testing, traditional approaches would require creating full copies of production data. With copy-on-write technology, these copies can be created instantly with minimal additional storage overhead, only consuming extra space when data is modified.

Integrating with Modern Infrastructure

Kubernetes has introduced new challenges and opportunities for storage optimization. When running databases and other stateful workloads on Kubernetes, one needs cloud-native storage solutions that can efficiently handle dynamic provisioning and scaling while maintaining performance and reliability.

Through Kubernetes integration via CSI drivers, modern storage solutions can provide automated provisioning, scaling, and optimization of storage resources. This integration enables organizations to maintain efficient storage utilization even in highly dynamic environments.

Quantifying Real-World Storage Impact

Let’s break down what storage consumption really means in environmental terms. For this example, we take 100TB of transactional databases like Postgres or MySQL on AWS using gp3 volumes as baseline storage.

Base Assumptions

• AWS gp3 volume running 24/7/365
• Power Usage Effectiveness (PUE) for AWS data centers: 1.15
• Average data center carbon intensity: 0.35 kg CO2e per kWh
• Storage redundancy factor: 3x (EBS maintains 3 copies for durability)
• Average car emission at 4.6 metric tons CO2e annually
• Database high availability configuration (3x replication)
• Development/testing environments (2x copies)

Detailed Calculation

Storage needs:

• Primary database: 100TB
• High availability (HA) replication: 200TB
• Development/testing: 100TB × 2 = 200TB
• Total storage footprint: 500TB

Power Consumption per TB:

• Base power per TB = 0.024 kW
• Daily consumption = 0.024 kW × 24 hours = 0.576 kWh/day
• Annual consumption = 0.576 kWh × 365 = 210.24 kWh/year
• With PUE for AWS: 210.24 kWh × 1.15 = 241.78 kWh/year
• Total annual power consumption with redundancy is 241.78 kWh × 3 = 725.34 kWh/year

Carbon emissions: 725.34 kWh × 0.35 kg CO2e/kWh x 500 TB x 1.15 = 145.9 metric tons CO2e annually

The final calculation assumes additional overhead for networking and management of 15%.

Environmental Impact and Why We Need Carbon Footprint Reduction Strategies

This database setup generates approximately 146 metric tons of CO2e. That’s equivalent to annual emissions coming from 32 cars. The potential for CO2e (carbon footprint) reduction using thin provisioning, tiering, COW, multi-attach, and erasure coding on the storage level would come close to 90%, translating into taking 29 cars off the road. This example is just for a 100TB primary database, a fraction of what many enterprises store.

But that’s only the beginning of the story. According to research by Gartner, enterprise data is growing at 35-40% annually. At this rate, a company storing 100TB today will need over 750TB in just 5 years if growth continues unchecked. This is why we need to consider reducing the cloud’s carbon footprint today.

The Future of Cloud Carbon Footprint Reduction

As cloud infrastructure continues to evolve, the importance of efficient storage management will only grow. While data center operators have taken significant steps to reduce their environmental footprint by adopting green or renewable energy sources, a large portion of the energy used by data centers still comes from fossil fuel-generated electricity. The future of cloud optimization lies in intelligent, automated solutions that dynamically adapt to changing requirements while maintaining optimal resource utilization. Technologies such as AI-driven data placement and advanced compression algorithms will further enhance our ability to minimize storage footprints while maximizing performance.

Reducing your cloud data center footprint through storage optimization isn’t just about cost savings—it’s about environmental responsibility. Organizations can significantly lower their expenses and environmental impact by implementing modern storage optimization strategies and supporting them with efficient compute and network resources.

For more insights on cloud storage optimization and its environmental impact, visit our detailed cloud storage cost optimization guide. Every byte of storage saved contributes to a more sustainable future for cloud computing. Start your optimization journey today with simplyblock and be part of the solution for a more environmentally responsible digital infrastructure.

The post How To Reduce Your Cloud Storage Carbon Footprint appeared first on simplyblock.

Why Run Kubernetes on Bare Metal?

There are several compelling reasons to consider bare-metal Kubernetes deployments.

Virtualized environments like VMware and OpenStack have traditionally been used to run Kubernetes, but they introduce unnecessary overhead, especially for high-performance workloads.

By running Kubernetes on bare metal, businesses can eliminate the virtualization layer and gain direct access to hardware resources. This results in lower latency, higher throughput, and improved overall performance. It is especially beneficial for I/O-intensive applications, such as databases and real-time systems, where every microsecond of delay counts.

Moreover, the recent VMware acquisition by Broadcom has triggered many enterprises to look for alternatives, and Kubernetes has emerged as a top choice for many former VMware users.

Last but not least, bare-metal Kubernetes supports scalability without the complexity of managing virtual machines and containerized infrastructure. Containerization has revolutionized how applications are built and deployed. However, the complexity of managing containerized infrastructures alongside virtual machines has posed challenges for infrastructure teams. A complete transition to Kubernetes offers a unified approach for stateless and stateful workloads.

The Hidden Cost of Virtualization

When they came out, virtualization platforms like VMware and OpenStack were game-changers. They enabled more efficient use of hardware resources such as CPU and RAM for a small overhead cost of virtualization. However, as the adoption of containerized applications and Kubernetes grows, the inefficiencies of these platforms become more apparent.

Every layer of virtualization introduces performance overhead. Each additional abstraction between the application and the physical hardware consumes CPU cycles, adds memory overhead, and introduces latency, particularly in I/O-heavy workloads. While technologies are paravirtualization, enabling drivers inside virtual machines to bypass large parts of the OS driver stack, the overhead is still significant. However, Kubernetes (and containerization in general) is more resource-friendly than traditional hypervisor-based virtualization. Containerization doesn’t require an entire operating system per container and shares most host resources, isolating applications and physical hosts through filters and syscall guards.

From an operation perspective, managing both virtualization layers and Kubernetes orchestration platforms compounds this complexity.

Storage Performance: The Bare-Metal Advantage

One of the key differentiators for bare-metal Kubernetes is its ability to maximize storage performance by eliminating these virtualization layers. Direct hardware access allows NVMe devices to connect directly to containers without complex passthrough configurations. This results in dramatically lower latency and enhanced IOPS. This direct mapping of physical to logical storage resources streamlines the data path and increases performance, particularly for I/O-intensive applications like databases.

Consider a typical storage path in a virtualized environment:

Application → Container → Kubernetes → Virtual Machine → Hypervisor → Physical Storage

Now, compare it to a bare metal environment:

Application → Container → Kubernetes → Physical Storage

By reducing unnecessary layers, organizations see substantial improvements in both latency and throughput, making bare-metal Kubernetes ideal for high-demand environments.

Figure 1: Comparison of system layers for storage access with normal virtualization, virtualization and Kubernetes, and bare metal Kubernetes

NVMe/TCP: The Networking Revolution for Bare-Metal Kubernetes

For bare-metal Kubernetes deployments, the storage performance benefits are further enhanced with NVMe-over-Fabrics (NVMe-oF). Specifically using NVMe/TCP, the successor to iSCSI. This enables organizations to leverage high-speed networking protocols, allowing storage devices to communicate with Kubernetes clusters over a network, providing enterprise-grade performance and scalability. Best of all, NVMe/TCP eliminates the traditional limitations of local storage (physical space limitations) or Fibre Channel and Infiniband solutions (requirement for specialized network hardware).

Simplyblock is built to optimize storage performance in Kubernetes environments using NVMe-oF and NVMe/TCP. Our architecture allows Kubernetes clusters to efficiently scale and use networked NVMe storage devices without sacrificing the low-latency, high-throughput characteristics of direct hardware access.

By integrating NVMe/TCP, simplyblock ensures that storage access, whether local or over the network, remains consistent and fast, allowing for seamless storage expansion as your Kubernetes environment grows. Whether you’re managing databases, real-time analytics, or microservices, NVMe/TCP provides the robust networking backbone that complements the high-performance nature of bare-metal Kubernetes.

Bridging the Gap with Modern Storage Solutions

While the performance benefits of bare metal deployments are clear, enterprises still require advanced storage capabilities. An enterprise-grade storage solution requires high availability, data protection, and efficient resource utilization without sacrificing scalability. Solutions like simplyblock meet these needs without the added complexity of virtualization.

Simplyblock’s NVMe-first architecture directly integrates into Kubernetes environments, providing user-space access to NVMe devices at scale. Simplyblock simplifies persistent volume provisioning with enterprise features such as instant snapshots, clones, and transparent encryption— all without compromising performance.

Discover how Simplyblock optimizes Kubernetes storage.

Future-Proofing Infrastructure with Bare Metal

With advancements in storage and networking technologies, the advantages of bare-metal Kubernetes continue to grow. New NVMe devices are pushing the performance limits, demanding direct hardware access. On the other hand, innovations such as SmartNICs and DPUs require tight integration with the hardware.

In addition, all modern microservices architectures, real-time applications, and high-performance databases benefit from the predictability and low-latency performance offered by bare-metal. It was always something virtualization struggles to deliver.

The Cost Equation

We’ve talked a lot about performance benefits, but that’s not the only reason you should consider adopting bare-metal Kubernetes infrastructures. Kubernetes equally helps to achieve optimal hardware utilization while controlling costs.

Removing the virtualization layer reduces infrastructure and licensing costs, while the streamlined architecture lowers operational overhead. Solutions like simplyblock enable organizations to achieve these financial benefits while maintaining enterprise-grade features like high availability and advanced data protection. The benefits of running on a standard Ethernet network stack and using NVMe over TCP further add to the cost benefits.

Nevertheless, transitioning to bare-metal Kubernetes is not an all-or-nothing process. First, enterprises can begin with I/O-intensive workloads that would benefit the most from performance gains and gradually scale over time. With many Kubernetes operators for databases available, running databases on Kubernetes has never been easier.

Second, enterprises can keep specific applications that may not run well on containers in virtual machines. Kubernetes extensions, such as KubeVirt, help manage virtual machine resources using the same API as containers, simplifying the deployment architecture.

That further confirms the growing adoption of stateful workloads on Kubernetes.

The Future of Kubernetes is Bare Metal

Organizations need infrastructure that taps directly into hardware capabilities while offering robust enterprise features to remain competitive in today’s fast-moving and cloud-native landscape. Bare-metal Kubernetes, combined with cutting-edge storage solutions like simplyblock, represents the ideal balance of performance, scalability, and functionality.

Ready to optimize your Kubernetes deployment with bare-metal storage and networked NVMe solutions? Contact Simplyblock today to learn how our solutions can drive your infrastructure’s performance and reduce your costs.

The post Bare-Metal Kubernetes: Power of Direct Hardware Access appeared first on simplyblock.

That said, as Kubernetes adoption keeps growing in 2025, selecting the right storage solution is more important than ever. Enterprise-level features, data encryption, and high availability are at the forefront of the requirements we want to look into. The same is true for the ability to attach to multiple clients simultaneously and the built-in data loss protection.

Simplyblock: Enterprise Storage Platform for Kubernetes

Simplyblock™ is an enterprise-grade storage platform designed to cater to high-performance and scalability needs for storage in Kubernetes environments. Simplyblock is fully optimized to take advantage of modern NVMe devices. It utilizes the NVMe/TCP protocol to share its shared volumes between the storage cluster and clients, providing superior throughput and lower access latency than the alternatives.

Simplyblock is designed as a cloud-native solution and is highly integrated with Kubernetes through its Simplyblock CSI driver. It supports dynamic provisioning, snapshots, clones, volume resizing, fully integrated encryption at rest, and many more. One benefit of simplyblock is its use of NVMe over TCP which is integrated into the Linux and Windows (Server 2025 or later) kernels, meaning no additional drivers. This also means it is easy to use simplyblock volumes outside of Kubernetes if you also operate virtual machines and would like to unify your storage. Furthermore, simplyblock volumes support read-write multi-attach. That means they can be attached to multiple pods, VMs, or both at the same time, making it easy to share data.

Its scale-out architecture provides full multi-tenant isolation, meaning that many customers can share the same storage backend. Logical volumes can be encrypted at rest either by an encryption key per tenant or even per logical volume, providing the strongest isolation option.

Deployment-wise, simplyblock offers the best of both worlds: disaggregated and hyperconverged setups. Simplyblock’s storage engine can be deployed on either a set of separate storage nodes, building a disaggregated storage cluster, or on Kubernetes worker nodes to utilize the worker node-local storage. Simplyblock also supports a mixed setup, where workloads can benefit from ultra-low latency with worker node-local storage (node-affinity) and the security and “spill-over” of the infinite storage from the disaggregated cluster.

As the only solution presented here, simplyblock favors erasure coding over replication for high availability and fault tolerance. Erasure coding is quite similar to RAID and uses parity information to achieve data loss protection. Simplyblock distributes this parity information across cluster nodes for higher fault tolerance. That said, erasure coding has a configuration similar to a replication factor, defining how many chunks and parity information will be used per calculation. This enables the best trade-off between data protection and storage overhead, enabling secure setups with as little as 50% additional storage requirements.

Furthermore, simplyblock provides a full multi-tier solution that caters to diverse storage needs in a single system. It enables you to utilize ultra-fast flash storage devices such as NVMe alongside slower SATA/SAS SSDs. At the same time, you can manage your long-term (cold) storage with traditional HDDs or QLC-based flash storage (slower but very high capacity).

Simplyblock is a robust choice if you need scalable, high-performance block storage for use cases such as databases, CDNs (Content Delivery Network), analytics solutions, and similar. Furthermore, simplyblock offers high throughput and low access latency. With its use of erasure coding, simplyblock is a great solution for companies seeking cost-effective storage while its ease of use allows organizations to adapt quickly to changing storage demands. For businesses seeking a modern, efficient, and Kubernetes-optimized block storage solution, simplyblock offers a compelling combination of features and performance.

Portworx: Kubernetes Storage and Data Management

Portworx is a cloud-native, software-defined storage platform that is highly integrated with Kubernetes. It is an enterprise-grade, closed-source solution that was acquired and is in active development by Pure Storage. Hence, its integration with the Pure Storage hardware appliances enables a performant, scalable storage option with integrated tiering capabilities.

Portworx integrated with Kubernetes through its native CSI driver and provides important CSI features such as dynamic provisioning, snapshots, clones, resizing, and persistent or ephemeral volumes. Furthermore, Portworx supports data-at-rest encryption and disaster recovery using synchronous and asynchronous cluster replication.

To enable fault tolerance and high availability, Portworx utilizes replicas, storing copies of data on different cluster nodes. This multiplies the required disk space by the replication factor. For the connection between the storage cluster and clients, Portworx provides access via iSCSI, a fairly old protocol that isn’t necessarily optimized for fast flash storage.

For connections between Pure’s FlashArray and Portworx, you can use NVMe/TCP or NVMe-RoCE (NVMe with RDMA over Converged Ethernet) — a mouthful, I know.

Encryption at rest is supported with either a unique key per volume or a cluster-wide encryption key. For storage-client communication, while iSCSI should be separated into its own VLAN, remember that iSCSI itself isn’t encrypted, meaning that encryption in transit isn’t guaranteed (if not pushed through a secured channel).

As mentioned, Portworx distinguishes itself by integrating with Pure Storage appliances. This integration enables organizations to leverage the performance and reliability of Pure’s flash storage systems. This makes Portworx a compelling choice for running critical stateful applications such as databases, message queues, and analytics platforms in Kubernetes, especially if you don’t fear operating hardware appliances. While available as a pure software-defined storage solution, Portworx excels in combination with Pure’s hardware, making it a great choice for databases, high-throughput message queues, and analytical applications on Kubernetes.

Ceph: Open-source, Distributed Storage System

Ceph is a highly scalable and distributed storage solution. Run as a company-backed open-source project, Ceph presents a unified storage platform with support for block, object, and file storage. That makes it a versatile choice for a wide range of Kubernetes applications.

Ceph’s Kubernetes integration is provided through the ceph-csi driver, which brings dynamically provisioned persistent volumes and automatic lifecycle management. CSI features supported by Ceph include snapshotting, cloning, resizing, and encryption.

The architecture of Ceph is built to be self-healing and self-managing, mainly designed to enable infinite disk space scalability. The provided access latency, while not on the top end, is good enough for many use cases. Running workloads like databases, which love high IOPS and low latency, can feel a bit laggy, though. Finally, high availability and fault tolerance are implemented through replication between Ceph cluster nodes.

From the security end, Ceph supports encryption at rest via a few different options. I’d recommend using the LUKS-based (Linux Unified Key Setup) setup as it supports all of the different Ceph storage options. The communication between cluster nodes, as well as storage and client, is not encrypted by default. If you require encryption in transit (and you should), utilize SSH and SSL termination via HAproxy or similar solutions. It’s unfortunate that a storage solution as big as Ceph has no such built-in support. The same goes for multi-tenancy, which can be achieved using RADOS namespaces but isn’t an out-of-the-box solution.

Ceph is an excellent choice as your Kubernetes storage when you are looking for an open-source solution with a proven track record of enterprise deployments, infinite storage scalability, and versatile storage types. It is not a good choice if you are looking for high-performance storage with low latency and millions of IOPS.

Moreover, due to its community-driven development and support, Ceph can be operated as a cost-effective and open-source alternative to proprietary storage solutions. Whether you’re deploying in a private data center, a public cloud, or a hybrid environment, Ceph’s adaptability is a great help for managing storage in containerized ecosystems.

Commercial support for Ceph is available from Red Hat.

Longhorn: Cloud-native Block Storage for Kubernetes

Longhorn is an open-source, cloud-native storage solution specifically designed for Kubernetes. It provides block storage that focuses on flexibility and ease of use. Therefore, Longhorn deploys straight into your Kubernetes cluster, providing worker node-local storage as persistent volumes.

As a cloud-native storage solution, Longhorn provides its own CSI driver, highly integrated with Longhorn and Kubernetes. It enables dynamic provisioning and management of persistent volumes, snapshots, clones, and backups. For the latter, people seem to have some complications with restores, so make sure to test your recovery processes.

For communication between storage and clients, Longhorn uses the iSCSI protocol. A newer version of the storage engine is in the works, which enables NVMe over TCP, however, at the time of writing, this engine isn’t yet production-ready and is not recommended for production use.

Anyhow, Longhorn provides good access latency and throughput, making it a great solution for mid-size databases and similar workloads. Encryption at rest can be set up but isn’t as simple as with some alternatives. High availability and fault tolerance is achieved by replicating data between cluster nodes. That means, as with many other solutions, the required storage is multiplied by the replication factor. However, Longhorn supports incremental backups to external storage systems like S3 for easy data protection and fast recoverability in disaster scenarios.

Longhorn is a full open-source project under the Cloud Native Computing Foundation (CNCF). It was originally developed by Rancher and is backed by SUSE. Hence, it’s commercially available with enterprise support as SUSE Storage.

Longhorn is a good choice if you want a lightweight, cloud-native, open-source solution that runs hyper-converged with your cluster workloads. It is usually used for smaller deployments or home labs — widely discussed on Reddit. Generally, it is not considered as robust as Ceph and, hence, is not recommended for mission-critical enterprise production workloads.

NFS: File Sharing Solution for Enterprises with Heterogeneous Environments

NFS (Network File System) is a well-known and widely adopted file-sharing protocol, inside and outside Kubernetes. That said, NFS has a proven track record showing its simplicity, reliability, and ability to provide shared access to persistent storage.

One of the main features of NFS is its ability to simultaneously attach volumes to many containers (and pods) with read-write access. That enables easy sharing of configuration, training data, or similar shared data sets between many instances or applications.

There are quite a few different options for integrating NFS with Kubernetes. The two main ones are the Kubernetes NFS Subdir External Provisioner. Both automatically create NFS subdirectories when new persistent volumes are requested, and the csi-driver-nfs. In addition, many storage solutions provide optimized NFS CSI drivers designed to provision shares for their respective solutions automatically. Such storage options include TrueNAS, OpenEBS, Dell EMC, and others.

High availability is one of the elements of NFS that isn’t simple, though. To make automatic failover work, additional tools like Corosync or Pacemaker need to be configured. On the client side, automount should be set up to handle automatic failover and reconnection. NFS is an old protocol from a time when those additional steps were commonplace. Today, they feel frumpy and out of place, especially compared to available alternatives.

While multi-tenancy isn’t strictly supported by NFS, using individual shares could be seen as a viable solution. However, remember that shares aren’t secured in any way. Authentication requires additional setups such as Kerberos. File access permissions shouldn’t be used as a sufficient setup for tenant isolation.

Encryption at rest with NFS comes down to the backing storage solution. NFS, as a sharing protocol, doesn’t offer anything by itself. Encryption in transit is supported, either via Kerberos or other means like TLS via stunnel. The implementation details differ per NFS provider, though. You should consult your provider’s manual.

NFS is your Kubernetes storage of choice if you need a simple, scalable, and shared file storage system that integrates seamlessly into your existing infrastructure. In the best case, you already have an NFS server set up and ready to go. Installing the CSI driver and configuring the storage class is all you need. While NFS might be a bottleneck for high-performance systems such as databases, many applications work perfectly fine. Imagine you need to scale out a WordPress-based website. There isn’t an easier way to share the same writable storage to many WP instances. That said, for organizations looking for a mature, battle-tested storage option to deliver shared storage with minimal complexity, NFS is the choice.

Make Your Kubernetes Storage Choice in 2025

Selecting the right storage solution for your Kubernetes persistent volume isn’t easy. It is an important choice to ensure performance, scalability, and reliability for your containerized workloads. Solutions like simplyblock™, Portworx, Ceph, Longhorn, and NFS offer a great set of features and are optimized for different use cases.

NFS is loved for its simplicity and easy multi-attach functionality. It is a great choice for all use cases needing shared write access. It’s not a great fit for high throughput and super-low access latency, though.

Ceph, on the other hand, is great if you need infinite scalability and don’t fear away from a slightly more complicated setup and operation. Ceph provides a robust choice for all use cases, as well as high-performance databases and similar IO-intensive applications.

Longhorn and Portworx are generally good choices for almost all types of applications. Both solutions provide good access latency and throughput. If you tend to buy hardware appliances, Portworx, in combination with Pure Storage, is the way to go. If you prefer pure software-defined storage and want to utilize storage available in your worker nodes, take a look at Longhorn.

Last but not least, simplyblock is your choice when running IO-hungry databases in or outside Kubernetes. Its use of the NVMe/TCP protocol makes it a perfect choice for pure container storage, as well as mixed environments with containers and virtual machines. Due to its low storage overhead for data protection, simplyblock is a great, cost-effective, and fast storage solution. And a bit of capacity always remains for all other storage needs, meaning a single solution will do it for you.

As Kubernetes evolves, leveraging the proper storage solution will significantly improve your application performance and resiliency. To ensure you make an informed decision for your short and long-term storage needs, consider factors like workload complexity, deployment scale, and data management needs.

Whether you are looking for a robust enterprise solution or a more simple and straightforward setup, these five options are all strong contenders to meet your Kubernetes storage demands in 2025.

The post 5 Storage Solutions for Kubernetes in 2025 appeared first on simplyblock.

What are Infrastructure Management Tools?

Infrastructure management tools help businesses efficiently oversee their cloud and bare metal resources and services. These tools simplify tasks such as deploying applications, monitoring performance, optimizing resource usage, and ensuring security across cloud environments.

Managing cloud operations manually can be challenging with the increasing complexity of cloud infrastructures and the wide range of services providers offer. Infrastructure management tools automate processes, improve scalability, and provide valuable insights into resource utilization, helping organizations stay cost-efficient and agile.

These tools allow businesses to automate repetitive tasks, reduce operational costs, prevent downtime, and ensure their cloud environment aligns with their strategic goals. However, to fully leverage their capabilities, organizations need a skilled IT team to implement and maintain them effectively.

What are the Benefits of Infrastructure Management Tools?

Infrastructure management tools offer numerous benefits that help businesses optimize cloud operations and maintain a reliable, efficient IT environment. Some key advantages include:

• Automation and Efficiency: These tools automate routine tasks such as resource provisioning, monitoring, and scaling. They reduce the need for manual intervention and allow teams to focus on strategic initiatives.
• Cost Optimization: By tracking provisioned resources, they prevent dangling and forgotten ones, helping to prevent unexpected cost buildup.
• Improved Performance and Reliability: Continuous monitoring and real-time alerts help detect and resolve issues proactively. They ensure smooth operations and minimal downtime.
• Enhanced Security and Compliance: Infrastructure management tools provide security features such as access controls, encryption, and compliance monitoring. They help businesses meet industry standards and protect sensitive data.
• Scalability and Flexibility: As businesses need to evolve, these tools enable seamless scaling of resources to accommodate growth without significant disruptions. Organizations can leverage infrastructure management tools to enhance operational efficiency, control costs, and maintain a secure and resilient cloud environment.

Kubernetes / Rancher / OpenShift

Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications across clusters of machines. It provides features such as automated load balancing, self-healing, rolling updates, and service discovery. Kubernetes is the de facto standard for container orchestration in cloud-native environments. Additionally, Kubernetes manages all necessary components and resources required for system operation, such as ingress services, container storage, IP addresses, and more. Kubernetes is the basis for many other platforms, such as SUSE Rancher and Red Hat OpenShift.

Simplyblock

Simplyblock is a high-performance, software-defined storage platform optimized for cloud-native environments, particularly Kubernetes. It intelligently manages data across multiple storage tiers, including high-performance NVMe disks, SSDs, HDDs, and object storage, ensuring optimal performance and cost-efficiency for various workloads. By leveraging NVMe over TCP, Simplyblock provides low-latency, high-throughput storage solutions suitable for I/O-intensive and latency-sensitive applications such as databases, AI/ML pipelines, and big data analytics. Its features include intelligent data tiering, thin provisioning, compression, encryption, and cross-region disaster recovery, making it a versatile and resilient choice for modern enterprises seeking scalable and efficient storage solutions.

Terraform / OpenTofu

Terraform is an open-source Infrastructure as Code (IaC) tool developed by HashiCorp that enables users to define, provision, and manage cloud and on-premises infrastructure using declarative configuration files. It supports multiple cloud providers, including AWS, Azure, and Google Cloud, allowing for infrastructure automation, scalability, and consistency across environments. Terraform uses a state management system to track resource changes and provides features like modularity and version control, making it a powerful tool for DevOps teams. In the recent past, concerns arose due to licensing changes from HashiCorp. Terraform was forked out into the OpenTofu project to provide an alternative. Simplyblock uses Terraform or OpenTofu for Linux deployments.

Ansible

Ansible is an open-source automation tool used for configuration management, application deployment, and infrastructure orchestration. It allows users to define system configurations using simple YAML-based files called playbooks, making automating repetitive tasks across cloud, on-premises, and hybrid environments easy. Ansible operates agentlessly, meaning it doesn’t require software installation on target machines, relying instead on SSH or WinRM for communication. With its declarative approach, idempotency, and extensive module ecosystem, Ansible helps streamline IT operations, improve consistency, and accelerate deployments in DevOps workflows.

Puppet

Perforce Puppet, commonly known as Puppet, is an open-source configuration management and automation tool that enables organizations to define infrastructure as code (IaC) and enforce system configurations across large-scale environments. It helps automate repetitive administrative tasks such as software installation, system configuration, and compliance enforcement by using declarative manifests written in Puppet’s domain-specific language (DSL). Puppet operates in agent-based and agentless modes, using a client-server architecture to ensure consistency across servers, workstations, and cloud instances. Acquired by Perforce in 2022, Puppet continues to be widely used in enterprises to improve operational efficiency, compliance, and infrastructure scalability.

Pulumi

Pulumi is an open-source Infrastructure as Code (IaC) platform that allows developers and DevOps teams to define, deploy, and manage cloud infrastructure using familiar programming languages such as Python, JavaScript, TypeScript, Go, and C#. Unlike traditional IaC tools that rely on domain-specific languages (DSLs), Pulumi leverages general-purpose programming languages to provide greater flexibility, code reuse, and integration with existing software development workflows. It supports multiple cloud providers, including AWS, Azure, Google Cloud, and Kubernetes, enabling infrastructure management across hybrid and multi-cloud environments. Pulumi offers features like state management, policy enforcement, and automation, making it a powerful solution for modern cloud-native infrastructure development.

SaltStack

SaltStack, commonly referred to as Salt, is an open-source automation and configuration management tool designed to manage and orchestrate infrastructure at scale. It enables system administrators and DevOps teams to automate provisioning, configuration enforcement, remote execution, and cloud management across large, complex environments. Salt uses a master-minion architecture, where the master node issues commands and configurations to minions (managed nodes) via an efficient, event-driven communication system. It also supports an agentless mode using SSH. SaltStack is known for its speed, scalability, and flexibility, making it ideal for dynamic environments requiring real-time infrastructure automation. Acquired by VMware in 2020, Salt continues to be widely used for IT automation, compliance enforcement, and security management.

OpenStack

OpenStack is a popular open-source cloud computing platform that provides software tools for building and managing public, private, and hybrid clouds. It enables organizations to virtualize and orchestrate pools of compute, storage, and networking resources through a unified dashboard or API. Designed for scalability and flexibility, OpenStack supports a wide range of hypervisors, storage backends, and network technologies, making it adaptable to diverse use cases. Its modular architecture consists of components like Nova (compute), Swift (object storage), Neutron (networking), and others, allowing users to customize their cloud environment. OpenStack is widely used in industries ranging from telecommunications to academia, offering a cost-effective alternative to proprietary cloud solutions.

DataDog

Datadog is a cloud-based monitoring and analytics platform that provides visibility into infrastructure performance, applications, and services. It integrates seamlessly with various technologies, including cloud providers, containers, databases, and more, to collect and analyze real-time metrics, logs, and traces. Datadog’s unified dashboard allows teams to monitor system health, identify performance bottlenecks, and troubleshoot issues efficiently. It also offers powerful alerting and automation capabilities to ensure timely anomaly responses. Designed for modern, dynamic environments, Datadog is widely used by DevOps, IT, and engineering teams to enhance operational efficiency and drive informed decision-making.

Progress Chef

Progress Chef (formerly Chef) is a powerful automation platform that enables organizations to manage and configure their IT infrastructure and applications efficiently. It uses a declarative, code-based approach to define system states, ensuring consistency and reliability across environments. Chef automates tasks such as provisioning servers, deploying applications, and applying security policies, reducing manual effort and minimizing errors. Focusing on DevOps practices, Chef integrates seamlessly with cloud providers, container platforms, and CI/CD pipelines, enabling teams to accelerate deployments and maintain compliance. Businesses widely use it to achieve infrastructure as code (IaC) and drive operational efficiency at scale.

KubeVirt

KubeVirt is an open-source extension to Kubernetes that enables users to run and manage virtual machines (VMs) alongside containerized workloads in a unified platform. By integrating VMs into Kubernetes, KubeVirt allows organizations to modernize their infrastructure by consolidating legacy applications and container-native services within a single orchestration framework. It provides features such as virtual machine lifecycle management, networking, and storage capabilities, leveraging Kubernetes-native concepts like pods, namespaces, and persistent volumes. This hybrid approach simplifies the transition to cloud-native architectures while supporting traditional workloads, making KubeVirt an ideal solution for organizations with diverse application needs.

OpenNebula

OpenNebula is an open-source cloud management platform designed to simplify private, public, and hybrid cloud deployment and operation. It provides a unified interface to orchestrate virtualized resources, including compute, storage, and networking, across various environments such as on-premises data centers, edge locations, and public cloud providers. OpenNebula supports a wide range of hypervisors and container technologies, offering flexibility for diverse workloads. Its lightweight architecture and user-friendly tools make it an efficient solution for small to medium-sized enterprises and edge computing use cases. With features like multi-tenancy, self-service portals, and automation, OpenNebula helps organizations achieve cost-effective and scalable cloud infrastructure.

AppDynamics

AppDynamics is a comprehensive application performance monitoring (APM) and management platform that helps businesses ensure the optimal performance of their applications and infrastructure. It provides deep insights into application behavior by monitoring metrics such as response times, resource utilization, and user interactions in real-time. By using advanced analytics and AI-driven anomaly detection, AppDynamics enables teams to quickly identify and resolve performance bottlenecks, preventing downtime and enhancing user experience. AppDynamics supports distributed applications, cloud environments, and microservices. It is a powerful tool for IT and DevOps teams to maintain application reliability and align performance with business goals.

Foreman

Foreman is an open-source lifecycle management tool that simplifies the provisioning, configuration, and monitoring of physical and virtual servers. Designed to integrate seamlessly with various virtualization platforms, cloud providers, and configuration management tools like Puppet, Ansible, and Chef, Foreman enables organizations to automate repetitive tasks and ensure consistency across their infrastructure. It provides bare-metal provisioning, host management, and detailed reporting through a user-friendly web interface and API. With its modular architecture and support for multi-tenant environments, Foreman is widely used by DevOps and IT teams to streamline server lifecycle management, improve operational efficiency, and maintain infrastructure compliance.

Prometheus

Prometheus is an open-source systems monitoring and alerting toolkit for reliability and scalability in dynamic environments. Originally developed at SoundCloud, Prometheus is now a part of the Cloud Native Computing Foundation (CNCF). It is widely adopted for monitoring cloud-native applications and collects metrics from configured targets. Prometheus stores these metrics in a time-series database and provides a powerful query language (PromQL) for analysis and visualization. With built-in alerting capabilities and seamless integration with tools like Grafana for dashboards, Prometheus monitors complex distributed systems. Its pull-based architecture, support for service discovery, and high modularity make it a popular choice for DevOps teams managing modern infrastructure.

Simplyblock and Infrastructure Management Tools?

Simplyblock is a cloud-native storage solution designed to enhance the performance and efficiency of IO-intensive, latency-sensitive workloads in Kubernetes environments. It complements infrastructure management tools like Kubernetes, OpenShift, Prometheus, OpenNebula, and CloudStack by providing high-performance, scalable block storage.

In Kubernetes deployments, Simplyblock integrates seamlessly as a StorageClass through the Container Storage Interface (CSI), enabling dynamic provisioning of storage resources for stateful applications. This integration ensures that applications requiring high throughput and low latency have access to optimized storage solutions.

When used alongside infrastructure-as-code tools like Terraform, simplyblock allows for the automated deployment and management of storage resources, ensuring consistency and repeatability across various environments. Furthermore, simplyblock uses Terraform or OpenTofu when deployed outside of Kubernetes. For Kubernetes-based deployment, simplyblock uses a helm chart.

For monitoring and observability, Simplyblock’s performance metrics can be integrated with tools like Prometheus or logging solutions such as GreyLog, providing the necessary storage capacity to store the metrics but also enabling real-time insights into storage performance and health. This visibility allows for proactive management and troubleshooting within the infrastructure.

Just as with Kubernetes, in virtualized environments managed by platforms like OpenNebula and KubeVirt, Simplyblock offers scalable and efficient storage solutions, ensuring that virtual machines and container workloads can access the necessary storage performance and capacity.

By incorporating Simplyblock into your infrastructure management stack, you can achieve enhanced storage performance, scalability, and cost-efficiency, ensuring your applications run smoothly and reliably across diverse environments. Get started now.

The post The 15 Top Infrastructure Management Tools appeared first on simplyblock.

With the rise of the cloud-native world of microservices, event-driven architectures, and distributed systems, understanding storage physics has never been more critical. As organizations deploy hundreds of database instances across their infrastructure, the multiplicative impact of storage performance becomes a defining factor in system behavior and database performance metrics, such as queries per second (QPS) and transactions per second (TPS).

While developers obsess over query optimization and index tuning, a more fundamental constraint silently shapes every database operation: the raw physical limits of storage access.

These limits aren’t just academic concerns—they’re affecting your systems right now. Each microservice has its own database, each Kubernetes StatefulSet, and every cloud-native application wrestles with physical boundaries, often without realizing it. When your system spans multiple availability zones, involves event sourcing, or requires real-time data processing, storage physics becomes the hidden multiplier that can either enable or cripple your entire architecture.

In this deep dive, we’ll explain how storage latency and IOPS create performance ceilings that no amount of application-level optimization can break through. More importantly, we’ll explore how understanding these physical boundaries is crucial for building truly high-performance, cloud-native systems that can scale reliably and cost-effectively.

The Latency-IOPS-QPS-TPS Connection

When we look at database and storage performance, there are four essential metrics to understand.

Latency (or access latency) measures how long it takes to complete a single I/O operation from issuing to answering. On the other hand, IOPS (Input/Output Operations Per Second) represents how many operations can be performed per second. Hence, IOPS measures the raw storage throughput for read/write operations.

On the database side, QPS (Queries Per Second) represents the number of query operations that can be executed per second, basically the higher-level application throughput. Last, TPS (Transactions Per Second) defines how many actual database transactions can be executed per second. A single transaction may contain one or more queries.

These metrics have key dependencies:

• Each query typically requires multiple I/O operations.
• As IOPS increases, latency increases due to queuing and resource contention.
• Higher latency constraints maximum achievable IOPS and QPS.
• The ratio between QPS and IOPS varies based on query complexity and access patterns.
• TPS is the higher-level metric of QPS. Both are directly related.

Consider a simple example:
If your storage system has a latency of 1 millisecond per I/O operation, the theoretical maximum IOPS would be 1,000 (assuming perfect conditions). However, increase that latency to 10 milliseconds, and your maximum theoretical IOPS drops to 100. Suppose each query requires an average of 2 I/O operations. In that case, your maximum QPS would be 500 at 1 ms latency but only 50 at 10 ms latency – demonstrating how latency impacts both IOPS and QPS in a cascading fashion.

1 second = 1000ms

1 I/O operation = 10ms
IOPS = 1000 / 10 = 100

1 query = 2 I/O ops
QPS = 100 / 2 = 50

The above is a simplified example. Modern storage devices have parallelism built into them, running multiple I/O operations simultaneously. However, you need a storage engine to make them available, and they only delay the inevitable.

Impact on Database Performance

For database workloads, the relationship between latency and IOPS becomes even more critical. Here’s why:

1. Query Processing Speed: Lower latency means faster individual query execution for data read from storage devices.
2. Concurrent Operations: Higher IOPS enables more simultaneous database operations.
3. Transaction Processing: The combination affects how many transactions per second (TPS) your database can handle.

The Hidden Cost of Latency

Storage latency impacts database operations in subtle but profound ways. Consider a typical PostgreSQL instance running on AWS EBS gp3 storage, which averages 2-4ms latency for read-write operations. While this might seem negligible, let’s break down its real impact:

Transaction Example:

• Single read operation: 3ms
• Write to WAL: 3ms
• Page write: 3ms
• fsync(): 3ms

Total latency: 12ms minimum per transaction
Maximum theoretical transactions per second: ~83

This means even before considering CPU time, memory access, or network latency, storage alone limits your database to fewer than 100 truly consistent transactions per second. Many teams don’t realize they’re hitting this physical limit until they’ve spent weeks optimizing application code with diminishing returns.

The IOPS Dance

IOPS limitations create another subtle challenge. Traditional cloud block storage solutions like Amazon EBS often struggle to simultaneously deliver low latency and high IOPS. This limitation can force organizations to over-provision storage resources, leading to unnecessary costs. For example, when running databases on AWS, many organizations provision multiple high-performance EBS volumes to achieve their required IOPS targets. However, this approach significantly underutilizes storage capacity while still not achieving optimal latency.

A typical gp3 volume provides a baseline of 3,000 IOPS. Let’s see how this plays out in real scenarios:

Common Database Operations IOPS Cost:

• Index scan: 2-5 IOPS per page
• Sequential scan: 1 IOPS per page
• Write operation: 2-4 IOPS (data + WAL)
• Vacuum operation: 10-20 IOPS per second

With just 20 concurrent users performing moderate-complexity queries, you could easily exceed your IOPS budget without realizing it. The database doesn’t stop – it just starts queueing requests, creating a cascading effect of increasing latency.

Real-World Database Performance Implications

Here’s a scenario many teams encounter:
A database server handling 1,000 transactions per minute seems to be performing well, with CPU usage at 40% and plenty of available memory. Yet response times occasionally spike inexplicably. The hidden culprit? Storage queuing:

Storage Queue Analysis:

• Average queue depth: 4
• Peak queue depth: 32
• Additional latency per queued operation: 1ms
• Effective latency during peaks: 35ms

Impact:

• 3x increase in transaction time
• Timeout errors in the application layer
• Connection pool exhaustion

The Ripple Effect

Storage performance limitations create unexpected ripple effects throughout the database system:

Connection Pool Behavior

When storage latency increases, transactions take longer to complete. This leads to connection pool exhaustion, not because of too many users, but because each connection holds onto resources longer than necessary.

Buffer Cache Efficiency

Higher storage latency makes buffer cache misses more expensive. This can cause databases to maintain larger buffer caches than necessary, consuming memory that could be better used elsewhere.

Query Planner Decisions

Most query planners don’t factor in current storage performance when making decisions. A plan that’s optimal under normal conditions might become significantly suboptimal during storage congestion periods.

Breaking Free from Storage Constraints

Modern storage solutions, such as simplyblock, are transforming this landscape. NVMe storage offers sub-200μs latency and millions of IOPS. Hence, databases operate closer to their theoretical limits:

Same Transaction on NVMe:

• Single read operation: 0.2ms
• Write to WAL: 0.2ms
• Page write: 0.2ms
• fsync(): 0.2ms

Total latency: 0.8ms
Theoretical transactions per second: ~1,250

This 15x improvement in theoretical throughput isn’t just about speed – it fundamentally changes how databases can be architected and operated.

New Architectural Possibilities

Understanding these storage physics opens new possibilities for database architecture:

Rethinking Write-Ahead Logging

With sub-millisecond storage latency, the traditional WAL design might be unnecessarily conservative. Some databases are exploring new durability models that take advantage of faster storage.

Dynamic Resource Management

Modern storage orchestrators can provide insights into actual storage performance, enabling databases to adapt their behavior based on current conditions rather than static assumptions.

Query Planning Evolution

Next-generation query planners could incorporate real-time storage performance metrics, making decisions that optimize for current system conditions rather than theoretical models.

How does the future of database performance optimization look like?

Understanding storage physics fundamentally changes how we approach database architecture and optimization. While traditional focus areas like query optimization and indexing remain essential, the emergence of next-generation storage solutions enables paradigm shifts in database design and operation. Modern storage architectures that deliver consistent sub-200μs latency and high IOPS aren’t just incrementally faster – they unlock entirely new possibilities for database architecture:

• True Horizontal Scalability: With storage no longer being the bottleneck, databases can scale more effectively across distributed systems while maintaining consistent performance.
• Predictable Performance: By eliminating storage queuing and latency variation, databases can deliver more consistent response times, even under heavy load.
• Simplified Operations: When storage is no longer a constraint, many traditional database optimization techniques and workarounds become unnecessary, reducing operational complexity.

For example, simplyblock’s NVMe-first architecture delivers consistent sub-200μs latency while maintaining enterprise-grade durability through distributed erasure coding. This enables databases to operate much closer to their theoretical performance limits while reducing complexity and cost through intelligent storage optimization.

As more organizations recognize that storage physics ultimately governs database behavior, we’ll likely see continued innovation in storage architectures and database designs that leverage these capabilities. The future of database performance isn’t just about faster storage – it’s about fundamentally rethinking how databases interact with their storage layer to deliver better performance, reliability, and cost-effectiveness at scale.

FAQ

The post Database Performance: Impact of Storage Limitations appeared first on simplyblock.

When talking to customers and users, this question is frequently left open with the note that they’ll check and tell us later. With storage being one of the main cost factors of cloud services such as Amazon’s AWS, this is not what it should be.

But who could blame them? It’s not like AWS is making it obvious to you how much of your storage resources (not only capacity but especially IOPS and throughput) you really use. It might be bad for AWS’ revenue.

Why We Built This

We believe that there is no reason to pay more than necessary. However, since it’s so hard to get hard facts on storage use, we tend to overprovision—by a lot.

Hence, we decided to do something about it. Today, we’re excited to share our new open-source tool – the AWS EBS Volume Usage Exporter!

What makes this particularly interesting is that, based on our experience, organizations typically utilize only 20-30% of their provisioned AWS EBS volumes. That means 70-80% of provisioned storage is sitting idle, quietly adding to your monthly AWS bill. Making someone else happy but you.

What Our Tool Does

The EBS Volume Usage Exporter runs in your EKS cluster and collects detailed metrics about your EBS volumes, including:

• Actual usage patterns
• IOPS consumption
• Throughput utilization
• Available disk space
• Snapshot information

All this data gets exported into a simple CSV file that you can analyze however you want.

If you like convenience, we’ve also built a handy calculator (that runs entirely in your browser – no data leaves your machine!) to help you quickly understand potential cost savings. Here’s the link to our EBS Volume Usage Calculator. No need to use it, though. The data is easy enough to get basic insight. Our calculator just automates the pricing and potential saving calculation based on current AWS price lists.

Super Simple to Get Started

To get you started quickly, we packaged everything as a Helm chart to make deployment as smooth as possible. You’ll need:

• An EKS cluster with cluster-admin privileges
• An S3 bucket
• Basic AWS permissions

The setup takes just a few minutes – we’ve included all the commands you need in our GitHub repository.

After a successful run, you can simply delete the helm chart deployment and be done with it. The exported data are available in the provided S3 bucket for download.

We Want Your Feedback!

This is just the beginning, and we’d love to hear from you!

Do the calculated numbers match your actual costs?
What other features would you find useful?

We already heard people asking for a tool that can run outside of EKS, and we’re looking into it. We would also love to extend support to utilize existing orchestration infrastructure such as DataDog, Dynatrace, or others. Most of the data is already available and should be easy to extract.

For those storage pros out there who can answer the EBS utilization question off the top of your head – we’d love to hear your stories, too!

Share your experiences and help us make this tool even better.

Try It Out!

The AWS EBS Volume Usage Exporter is open source and available now on GitHub. Give it a spin, and let us know what you think!

And hey – if you’re wondering whether you really need this tool, ask yourself: “Do I know exactly how much of my provisioned EBS storage is actually being used right now?”

If there’s even a moment of hesitation, you should check this out!

At simplyblock, we’re passionate about helping organizations optimize their cloud storage. This tool represents our commitment to the open-source community and our mission to eliminate cloud storage waste.

The post We Built a Tool to Help You Understand Your Real EBS Usage! appeared first on simplyblock.

When data grows, storage needs to grow, too. That’s when remotely attached SAN (Storage Area Network) systems come in. So far, these were commonly connected through one of three protocols: Fibre Channel, Infiniband, or iSCSI. However, with the latter being on the “low end” side of things, without the need for special hardware to operate. NMVe over Fabrics (NVMe-oF), and specifically NVMe over TCP (NVMe/TCP) as the successor of iSCSI, is on the rise to replace these legacy protocols and bring immediate improvements in latency, throughput, and IOPS.

iSCSI: The Quick History Lesson

iSCSI is a protocol that connects remote storage solutions (commonly hardware storage appliances) to storage clients. The latter are typically servers without (or with minimal) local storage, as well as virtual machines. In recent years, we have also seen iSCSI being used as a backend for container storage.

iSCSI stands for Internet Small Computer Storage Interface and encapsulates the standard SCSI commands within TCP/IP packets. That means that iSCSI works over commodity Ethernet networks, removing the need for specialized hardware such as network cards (NICs) and switches.

The iSCSI standard was first released in early 2000. A world that was very different from today. Do you remember what a phone looked like in 2000?

That said, while there was access to the first flash-based systems, prices were still outrageous, and storage systems were designed with spinning disks in mind. Remember that. We’ll come back to it later.

What is SCSI?

SCSI, or, you guessed it, the Small Computer Storage Interface, is a set of standards for connecting and transferring data between computers and peripheral devices. Originally developed in the 1980s, SCSI has been a foundational technology for data storage interfaces, supporting various device types, primarily hard drives, optical drives, and scanners.

While SCSI kept improving and adding new commands for technologies like NVMe. The foundation is still rooted in the early 1980s, though. However, many standards still use the SCSI command set, SATA (home computers), SAS (servers), and iSCSI.

What is NVMe?

Non-Volatile Memory Express (NVMe) is a modern PCI Express-based (PCI-e) storage interface. With the original specification dating back to 2011, NVMe is engineered specifically for solid-state drives (SSDs) connected via the PCIe bus. Therefore, NVMe devices are directly connected to the CPU and other devices on the bus to increase throughput and latency. NVMe dramatically reduces latency and increases input/output operations per second (IOPS) compared to traditional storage interfaces.

As part of the NVMe standard, additional specifications are developed, such as the transport specification which defines how NVMe commands are transported (e.g., via the PCI Express Bus, but also networking protocols like TCP/IP).

The Fundamental Difference of Spinning Disks and NVMe

Traditional spinning hard disk drives (HDDs) rely on physical spinning platters and moveable read/write heads to write or access data. When data is requested, the mechanical component must be physically placed in the correct location of the platter stack, resulting in significant access latencies ranging from 10-14 milliseconds.

Flash storage, including NVMe devices, eliminates the mechanical parts, utilizing NAND flash chips instead. NAND stores data purely electronically and achieves access latencies as low as 20 microseconds (and even lower on super high-end gear). That makes them 100 times faster than their HDD counterparts.

For a long time, flash storage had the massive disadvantage of limited storage capacity. However, this disadvantage slowly fades away with companies introducing higher-capacity devices. For example, Toshiba just announced a 180TB flash storage device.

Cost, the second significant disadvantage, also keeps falling with improvements in development and production. Technologies like QLC NAND offer incredible storage density for an affordable price.

Anyhow, why am I bringing up the mechanical vs electrical storage principle? The reason is simple: the access latency. SCSI and iSCSI were never designed for super low access latency devices because they didn’t really exist at the time of their development. And, while some adjustments were made to the protocol over the years, their fundamental design is outdated and can’t be changed for backward compatibility reasons.

NVMe over Fabrics: Flash Storage on the Network

NVMe over Fabrics (also known as NVMe-oF) is an extension to the NVMe base specification. It allows NVMe storage to be accessed over a network fabric while maintaining the low-latency, high-performance characteristics of local NVMe devices.

NVMe over Fabrics itself is a collection of multiple sub-specifications, defining multiple transport layer protocols.

• NVMe over TCP: NVMe/TCP utilizes the common internet standard protocol TCP/IP. It deploys on commodity Ethernet networks and can run parallel to existing network traffic. That makes NVMe over TCP the modern successor to iSCSI, taking over where iSCSI left off. Therefore, NVMe over TCP is the perfect solution for public cloud-based storage solutions that typically only provide TCP/IP networking.
• NVME over Fibre Channel: NVMe/FC builds upon the existing Fibre Channel network fabric. It tunnels NVMe commands through Fibre Channel packets and enables reusing available Fibre Channel hardware. I wouldn’t recommend it for new deployments due to the high entry cost of Fibre Channel equipment.
• NVMe over Infiniband: Like NVMe over Fibre Channel, NVMe/IB utilizes existing Infiniband networks to tunnel the NVMe protocol. If you have existing Infiniband equipment, NVMe over Infiniband might be your way. For new deployments, the initial entry cost is too high.
• NVME over RoCE: NVME over Converged Ethernet is a transport layer that uses an Ethernet fabric for remote direct memory access (RDMA). To use NVMe over RoCE, you need RDMA-capable NICs. RoCE comes in two versions: RoCEv1, which is a layer-2 protocol and not routable, and RoCEv2, which uses UDP/IP and can be routed across complex networks. NVMe over RoCE doesn’t scale as easily as NVMe over TCP but provides even lower latencies.

NVMe over TCP vs iSCSI: The Comparison

When comparing NVMe over TCP vs iSCSI, we see considerable improvements in all three primary metrics: latency, throughput, and IOPS.

The folks over at Blockbridge ran an extensive comparison of the two technologies, which shows that NVMe over TCP outperformed iSCSI, regardless of the benchmark.

I’ll provide the most critical benchmarks here, but I recommend you read through the full benchmark article right after finishing here.

Anyhow, let’s dive a little deeper into the actual facts on the NVMe over TCP vs iSCSI benchmark.

Editor’s Note: Our Developer Advocate, Chris Engelbert, gave a talk recently at SREcon in Dublin, talking about the performance between NVMe over TCP and iSCSI, which led to this blog post. Find the full presentation NVMe/TCP makes iSCSI look like Fortran.

Benchmarking Network Storage

Evaluating storage performance involves comparing four major performance indicators.

1. IOPS: Number of input/output operations processed per second
2. Latency: Time required to complete a single input/output operation
3. Throughput: Total data transferred per unit of time
4. Protocol Overhead: Additional processing required by the communication protocol

Editor’s note: For latency, throughput, and IOPS, we have an exhaustive blog post talking deeper about the necessities, their relationships, and how to calculate them.

A comprehensive performance testing involves simulated workloads that mirror real-world scenarios. To simplify this process, benchmarks use tools like FIO (Flexible I/O Tester) to generate consistent, reproducible test data and results across different storage configurations and systems.

IOPS Improvements of NVMe over TCP vs iSCSI

Running IOPS-intensive applications, the number of available IOPS in a storage system is critical. IOPS-intensive application means systems such as databases, analytics platforms, asset servers, and similar solutions.

Improving IOPS by exchanging the storage-network protocol is an immediate win for the database and us.

Using NVMe over TCP instead of iSCSI shows a dramatic increase in IOPS, especially for smaller block sizes. At 512 bytes block size, Blockbridge found an average 35.4% increase in IOPS. At a more common 4KiB block size, the average increase was 34.8%.

That means the same hardware can provide over one-third more IOPS using NVMe over TCP vs iSCSI at no additional cost.

Latency Improvements of NVMe over TCP vs iSCSI

While IOPS-hungry use cases, such as compaction events in databases (Cassandra), benefit from the immense increase in IOPS, latency-sensitive applications love low access latencies. Latency is the primary factor that causes people to choose local NVMe storage over remotely-attached storage, knowing about many or all drawbacks.

Latency-sensitive applications range from high-frequency trading systems, where milliseconds are measured in hard money, over telecommunication systems, where latency can introduce issues with system synchronization, to cybersecurity and threat detection solutions that need to react as fast as possible.

Therefore, decreasing latency is a significant benefit for many industries and solutions. Apart from that, a lower access latency always speeds up data access, even if your system isn’t necessarily latency-sensitive. You will feel the difference.

Blockbridge found the most significant benefit in access latency reduction with a block size of 16KiB with a queue depth of 128 (which can easily be hit with I/O demanding solutions). The average latency for iSCSI was 5,871μs compared to NVMe over TCP with 5,089μs. A 782μs (~25%) decrease in access latency—just by exchanging the storage protocol.

Throughput Improvement of NVMe over TCP vs iSCSI

As the third primary metric of storage performance, throughput describes how much data is actually pumped from the disk into your workload.

Throughput is the major factor for applications such as video encoding or streaming platforms, large analytical systems, and game servers streaming massive worlds into memory. Furthermore, there are also time-series storage, data lakes, and historian databases.

Throughput-heavy systems benefit from higher throughput to get the “job done faster.” Oftentimes, increasing the throughput isn’t easy. You’re either bound by the throughput provided by the disk or, in the case of a network-attached system, the network bandwidth. To achieve high throughput and capacity, remote network storage utilizes high bandwidth networking or specialized networking systems such as Fibre Channel or Infiniband.

Blockbridge ran their tests on a dual-port 100Gbit/s network card, limited by the PCI Express x16 Gen3 bus to a maximum throughput of around 126Gbit/s. Newer PCIe standards achieve much higher throughput. Hence, NVMe devices and NICs aren’t bound by the “limiting” factor of the PCIe bus anymore.

With a 16KiB block size and a queue depth of 32, their benchmark saw a whopping 2.3GB/s increase in performance on NVMe over TCP vs iSCSI. The throughput increased from 10.387GBit/s on iSCSI to 12.665GBit/s, an easy 20% on top—again, using the same hardware. That’s how you save money.

The Compelling Case for NVMe over TCP

We’ve seen that NVMe over TCP has significant performance advantages over iSCSI in all three primary storage performance metrics. Nevertheless, there are more advantages to NVMe over TCP vs iSCSI.

• Standard Ethernet: NVMe over TCP’s most significant advantage is its ability to operate over standard Ethernet networks. Unlike specialized networking technologies (Infiniband, Fibre Channel), NVMe/TCP requires no additional hardware investments or complex configuration, making it remarkably accessible for organizations of all sizes.
• Performance Characteristics: NVMe over TCP delivers exceptional performance by minimizing protocol overhead and leveraging the efficiency of NVMe’s design. It can achieve latencies comparable to local storage while providing the flexibility of network-attached resources. Modern implementations can sustain throughput rates exceeding traditional storage protocols by significant margins.
• Ease of Deployment: NVMe over TCP integrates seamlessly with Linux and Windows (Server 2025 and later) since the necessary drivers are already part of the kernel. That makes NVMe/TCP straightforward to implement and manage. Seamless compatibility reduces the learning curve and integration challenges typically associated with new storage technologies.

Choosing Between NVMe over TCP and iSCSI

Deciding between two technologies isn’t always easy. It isn’t that hard in the case of NVMe over TCP vs iSCSI. The use cases for new iSCSI deployment are very sparse. From my perspective, the only valid use case is the integration of pre-existing legacy systems that don’t yet support NVMe over TCP

That’s why simplyblock, as an NVMe over TCP first solution, still provides iSCSI if you really need it. We offer it exactly for the reason that migrations don’t happen from today to tomorrow. Still, you want to leverage the benefits of newer technologies, such as NVMe over TCP, wherever possible. With simplyblock, logical volumes can easily be provisioned as NVMe over TCP or iSCSI devices. You can even switch over from iSCSI to NVMe over TCP later on.

In any case, you should go with NVMe over TCP when:

• You operate high-performance computing environments
• You have modern data centers with significant bandwidth
• You deploy workloads requiring low-latency, high IOPS, or throughput storage access
• You find yourself in scenarios that demand scalable, flexible storage solutions
• You are in any other situation where you need remotely attached storage

You should stay on iSCSI (or slowly migrate away) when:

• You have legacy infrastructure with limited upgrade paths

You see, there aren’t a lot of reasons. Given that, it’s just a matter of selecting your new storage solution. Personally, these days, I would always recommend software-defined storage solutions such as simplyblock, but I’m biased. Anyhow, an SDS provides the best of both worlds: commodity storage hardware (with the option to go all in with your 96-bay storage server) and performance.

Simplyblock: Embracing Versatility

Simplyblock demonstrates forward-thinking storage design by supporting both NVMe over TCP and iSCSI, providing customers with the best performance when available and the chance to migrate slowly in the case of existing legacy clients.

Furthermore, simplyblock offers features known from traditional SAN storage systems or “filesystems” such as ZFS. This includes a full copy-on-write backend with instant snapshots and clones. It includes synchronous and asynchronous replication between storage clusters. Finally, simplyblock is your modern storage solution, providing storage to dedicated hosts, virtual machines, and containers. Regardless of the client, simplyblock offers the most seamless integration with your existing and upcoming environments.

The Future of NVMe over TCP

As enterprise and cloud computing continue to evolve, NVMe over TCP stands as the technology of choice for remotely attached storage. Firstly, it combines simplicity, performance, and broad compatibility. Secondly, it provides a cost-efficient and scalable solution utilizing commodity network gear.

The protocol’s ongoing development (last specification update May 2024) and increasing adoption show continued improvements in efficiency, reduced latency, and enhanced scalability.

NVMe over TCP represents a significant step forward in storage networking technology. Furthermore, combining the raw performance of NVMe with the ubiquity of Ethernet networking offers a compelling solution for modern computing environments. While iSCSI remains relevant for specific use cases and during migration phases, NVME over TCP represents the future and should be adopted as soon as possible.

We, at simplyblock, are happy to be part of this important step in the history of storage.

Questions and Answers

The post NVMe over TCP vs iSCSI: Evolution of Network Storage appeared first on simplyblock.

Kubernetes storage is a sophisticated ecosystem designed to address the complex data management needs of containerized applications. At its core, Kubernetes storage provides a flexible mechanism to manage data across dynamic, distributed computing environments. It allows your containers to store, access, and persist data with unprecedented flexibility.

Storage Types in Kubernetes

Fundamentally, Kubernetes provides two types of storage: ephemeral volumes are bound to the container’s lifecycle, and persistent volumes survive a container restart or termination.

Ephemeral (Non-Persistent) Storage

Ephemeral storage represents the default storage mechanism in Kubernetes. It provides a temporary storage solution, existing only for the duration of a container’s lifecycle. Therefore, when a container is terminated or removed, all data stored in this temporary storage location is permanently deleted.

This type of storage is ideal for transient data that doesn’t require long-term preservation, such as temporary computation results or cache files. Most stateless workloads utilize ephemeral storage for these kinds of temporary data. That said, a “stateless workload” doesn’t necessarily mean no data is stored temporarily. It means there is no issue if this storage disappears from one second to the next.

Persistent Storage

Persistent storage is a critical concept in Kubernetes that addresses one of the fundamental challenges of containerized applications: maintaining data integrity and accessibility across dynamic and ephemeral computing environments.

Unlike ephemeral storage, which exists only for the lifetime of a container, persistent storage is not bound to the lifetime of a container. Hence, persistent storage provides a robust mechanism for storing and managing data that must survive container restarts, pod rescheduling, or even complete cluster redesigns. You enable persistent Kubernetes storage through the concepts of Persistent Volumes (PV) as well as Persistent Volume Claims (PVC).

Fundamental Kubernetes Storage Entities

Storage in Kubernetes is built up from multiple entities, depending on how storage is provided and if it is ephemeral or persistent.

Persistent Volumes (PV)

A Persistent Volume (PV) is a slice of storage in the Kubernetes cluster that has been provisioned by an administrator or dynamically created through a StorageClass. Think of a PV as a virtual storage resource that exists independently of any individual pod’s lifecycle. Consequently, this abstraction allows for several key capabilities:

• Lifecycle Independence: A PV exists even when the owner-pod is deleted, recreated, or moved to different nodes.
• Storage Type Abstraction: A PV is not bound to a specific storage technology. Hence, it can represent various storage types, including:
  • Network-attached storage (NFS)
  • Cloud provider storage solutions (AWS EBS, Azure Disk)
  • Local storage devices
  • Storage area networks (SAN)
  • Distributed storage systems

Persistent Volume Claims (PVC): Requesting Storage Resources

Persistent Volume Claims act as a user’s request for storage resources. Image your PVC as a demand for storage with specific requirements, similar to how a developer requests computing resources.

When a user creates a PVC, Kubernetes attempts to find and bind an appropriate Persistent Volume that meets the specified criteria. If no existing volume is found but a storage class is defined or a cluster-default one is available, the persistent volume will be dynamically allocated.

Key PersistentVolumeClaim Characteristics:

• Size Specification: Defines a user storage capacity request
• Access Modes: Defines how the volume can be accessed
  • ReadWriteOnce (RWO): Allows all pods on a single node to mount the volume in read-write mode.
  • ReadWriteOncePod: Allows a single pod to read-write mount the volume on a single node.
  • ReadOnlyMany (ROX): Allows multiple pods on multiple nodes to read the volume. Very practical for a shared configuration state.
  • ReadWriteMany (RWO): Allows multiple pods on multiple nodes to read and write to the volume. Remember, this could be dangerous for databases and other applications that don’t support a shared state.
• StorageClass: Allows requesting specific types of storage based on performance, redundancy, or other characteristics

The Container Storage Interface (CSI)

The Container Storage Interface (CSI) represents a pivotal advancement in Kubernetes storage architecture. Before CSI, integrating storage devices with Kubernetes was a complex and often challenging process that required a deep understanding of both storage systems and container orchestration.

The Container Storage Interface introduces a standardized approach to storage integration. Storage providers (commonly referred to as CSI drivers) are so-called out-of-process entities that communicate with Kubernetes via an API. The integration of CSI into the Kubernetes ecosystem provides three major benefits:

1. CSI provides a vendor-neutral, extensible plugin architecture
2. CSI simplifies the process of adding new storage systems to Kubernetes
3. CSI enables third-party storage providers to develop and maintain their own storage plugins without modifying Kubernetes core code

Volumes: The Basic Storage Units

In Kubernetes, volumes are fundamental storage entities that solve the problem of data persistence and sharing between containers. Unlike traditional storage solutions, Kubernetes volumes are not limited to a single type of storage medium. They can represent:

• Network file systems
• Local storage devices
• Cloud storage services
• Custom storage implementations such as simplyblock

Volumes provide a flexible abstraction layer that allows applications to interact with storage resources without being directly coupled to the underlying storage infrastructure.

StorageClasses: Dynamic Storage Provisioning

StorageClasses represent a powerful abstraction that enables dynamic and flexible storage provisioning because they allow cluster administrators to define different types of storage services with varying performance characteristics, such as:

• High-performance SSD storage
• Economical magnetic drive storage
• Geo-redundant cloud storage solutions

When a user requests storage through a PVC, Kubernetes tries to find an existing persistent volume. If none was found, the appropriate StorageClass defines how to automatically provision a suitable storage resource, significantly reducing administrative overhead.

Best Practices for Kubernetes Storage Management

1. Resource Limitation
   • Implement strict resource quotas
   • Control storage consumption across namespaces
   • Set clear boundaries for storage requests
2. Configuration Recommendations
   • Always use Persistent Volume Claims in container configurations
   • Maintain a default StorageClass
   • Use meaningful and descriptive names for storage classes
3. Performance and Security Considerations
   • Implement quality of service (QoS) controls
   • Create isolated storage environments
   • Enable multi-tenancy through namespace segregation

Practical Storage Provisioning Example

While specific implementations vary, here’s a conceptual example of storage provisioning using Helm:

helm install storage-solution storage-provider/csi-driver \
  --set storage.size=100Gi \
  --set storage.type=high-performance \
  --set access.mode=ReadWriteMany

Kubernetes Storage with Simplyblock CSI: Practical Implementation Guide

Simplyblock is a storage platform for stateful workloads such as databases, message queues, data warehouses, file storage, and similar. Therefore, simplyblock provides many features tailored to the use cases, simplifying deployments, improving performance, or enabling features such as instant database clones.

Basic Installation Example

When deploying storage in a Kubernetes environment, organizations need a reliable method to integrate storage solutions seamlessly. The Simplyblock CSI driver installation process begins by adding the Helm repository, which allows teams to easily access and deploy the storage infrastructure. By creating a dedicated namespace called simplyblock-csi, administrators ensure clean isolation of storage-related resources from other cluster components.

The installation command specifies critical configuration parameters that connect the Kubernetes cluster to the storage backend. The unique cluster UUID identifies the specific storage cluster, while the API endpoint provides the connection mechanism. The secret token ensures secure authentication, and the pool name defines the initial storage pool where volumes will be provisioned. This approach allows for a standardized, secure, and easily repeatable storage deployment process.

Here’s an example of installing the Simplyblock CSI driver:

helm repo add simplyblock-csi https://raw.githubusercontent.com/simplyblock-io/simplyblock-csi/master/charts

helm repo update

helm install -n simplyblock-csi --create-namespace \
  simplyblock-csi simplyblock-csi/simplyblock-csi \
  --set csiConfig.simplybk.uuid=[random-cluster-uuid] \
  --set csiConfig.simplybk.ip=[cluster-ip] \
  --set csiSecret.simplybk.secret=[random-cluster-secret] \
  --set logicalVolume.pool_name=[cluster-name]

Advanced Configuration Scenarios

1. Performance-Optimized Storage Configuration

Modern applications often require precise control over storage performance, making custom StorageClasses invaluable.

Firstly, by creating a high-performance storage class, organizations can define exact performance characteristics for different types of workloads. The configuration sets a specific IOPS (Input/Output Operations Per Second) limit of 5000, ensuring that applications receive consistent and predictable storage performance.

Secondly, bandwidth limitations of 500 MB/s prevent any single application from monopolizing storage resources, promoting fair resource allocation. The added encryption layer provides an additional security measure, protecting sensitive data at rest. This approach allows DevOps teams to create storage resources that precisely match application requirements, balancing performance, security, and resource management.

# Example StorageClass configuration
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: high-performance-storage
provisioner: csi.simplyblock.io
parameters:
  qos_rw_iops: "5000"    # High IOPS performance
  qos_rw_mbytes: "500"   # Bandwidth limit
  encryption: "True"      # Enable encryption

2. Multi-Tenant Storage Setup

As a large organization or cloud provider, you require a robust environment and workload separation mechanism. For that reason, teams organize workloads between development, staging, and production environments by creating a dedicated namespace for production applications.

Therefore, the custom storage class for production workloads ensures critical applications have access to dedicated storage resources with specific performance and distribution characteristics.

The distribution configuration with multiple network domain controllers (NDCs) provides enhanced reliability and performance. Indeed, this approach supports complex enterprise requirements by enabling granular control over storage resources, improving security, and ensuring that production workloads receive the highest quality of service.

# Namespace-based storage isolation
apiVersion: v1
kind: Namespace
metadata:
  name: production-apps

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: encrypted-volume
  annotations:
    simplybk/secret-name: encrypted-volume-keys
spec:
  storageClassName: encrypted-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi

Multipath Storage Configuration

Network resilience is a critical consideration in enterprise storage solutions. Hence, multipath storage configuration provides redundancy by allowing multiple network paths for storage communication. By enabling multipathing and specifying a default network interface, organizations can create more robust storage infrastructures that can withstand network interruptions.

The caching node creation further enhances performance by providing an intelligent caching layer that can improve read and write operations. Furthermore, this configuration supports load balancing and reduces potential single points of failure in the storage network.

cachingnode:
  create: true
  multipathing: true
  ifname: eth0  # Default network interface

Best Practices for Kubernetes Storage with Simplyblock

1. Always specify a unique pool name for each storage configuration
2. Implement encryption for sensitive workloads
3. Use QoS parameters to control storage performance
4. Leverage multi-tenancy features for environment isolation
5. Regularly monitor storage node capacities and performance

Deletion and Cleanup

# Uninstall the CSI driver
helm uninstall "simplyblock-csi" --namespace "simplyblock-csi"

# Remove the namespace
kubectl delete namespace simplyblock-csi

The examples demonstrate the flexibility of Kubernetes storage, showcasing how administrators can fine-tune storage resources to meet specific application requirements while maintaining performance, security, and scalability. Try simplyblock for the most flexible Kubernetes storage solution on the market today.

The post Kubernetes Storage 201: Concepts and Practical Examples appeared first on simplyblock.

Today, data is the “new gold” for companies. Data collection happens everywhere and at any time. The global amount of data collected is projected to reach 181 Zettabytes (that is 181 billion Terabytes) by the end of 2025. A whopping 23.13% increase over 2024.

That means data protection is becoming increasingly important. Hence, data security has become paramount for organizations of all sizes. One key aspect of data security protocols is data-at-rest encryption, which provides crucial protection for stored data.

Understanding the Data-in-Use, Data-in-Transit, and Data-at-Rest

Before we go deep into DARE, let’s quickly discuss the three states of data within a computing system and infrastructure.

To start, any type of data is created inside an application. While the application holds onto it, it is considered as data in use. That means data in use describes information actively being processed, read, or modified by applications or users.

For example, imagine you open a spreadsheet to edit its contents or a database to process a query. This data is considered “in use.” This state is often the most vulnerable as the data must be in an unencrypted form for processing. However, technologies like confidential computing enable encrypted memory to process even these pieces of data in an encrypted manner.

Next, data in transit describes any information moving between locations. Locations means across the internet, within a private network, or between memory and processors. Examples include email messages being sent, files being downloaded, or database query results traveling between the database server and applications—all examples of data in transit.

Last but not least, data at rest refers to any piece of information stored physically on a digital storage media such as flash storage or hard disk. It also considers storage solutions like cloud storage, offline backups, and file systems as valid digital storage. Hence, data stored in these services is also data at rest. Think of files saved on your laptop’s hard drive or photos stored in cloud storage such as Google Drive, Dropbox, or similar.

Criticality of Data Protection

For organizations, threats to their data are omnipresent. Starting from unfortunate human error, deleting important information, coordinated ransomware attacks, encryption of your data, and asking for a ransom to actual data leaks.

Especially with data leaks, most people think about external hackers copying data off of the organization’s network and making it public. However, this isn’t the only way data is leaked to the public. There are many examples of Amazon S3 buckets without proper authorization, databases being accessible from the outside world, or backup services being accessed.

Anyhow, organizations face increasing threats to their data security. Any data breach has consequences, which are categorized into four segments:

1. Financial losses from regulatory fines and legal penalties
2. Damage to brand reputation and loss of customer trust
3. Exposure of intellectual property to competitors
4. Compliance violations with regulations like GDPR, HIPAA, or PCI DSS

While data in transit is commonly protected through TLS (transport layer encryption), data at rest encryption (or DARE) is often an afterthought. This is typically because the setup isn’t as easy and straightforward as it should be. It’s this “I can still do this afterward” effect: “What could possibly go wrong?”

However, data at rest is the most vulnerable of all. Data in use is often unprotected but very transient. While there is a chance of a leak, it is small. Data at rest persists for more extended periods of time, giving hackers more time to plan and execute their attacks. Secondly, persistent data often contains the most valuable pieces of information, such as customer records, financial data, or intellectual property. And lastly, access to persistent storage enables access to large amounts of data within a single breach, making it so much more interesting to attackers.

Understanding Data At Rest Encryption (DARE)

Simply spoken, Data At Rest Encryption (DARE) transforms stored data into an unreadable format that can only be read with the appropriate encryption or decryption key. This ensures that the information isn’t readable even if unauthorized parties gain access to the storage medium.

That said, the strength of the encryption used is crucial. Many encryption algorithms we have considered secure have been broken over time. Encrypting data once and taking for granted that unauthorized parties can’t access it is just wrong. Data at rest encryption is an ongoing process, potentially involving re-encrypting information when more potent, robust encryption algorithms are available.

Available Encryption Types for Data At Rest Encryption

Two encryption methods are generally available today for large-scale setups. While we look forward to quantum-safe encryption algorithms, a widely adopted solution has yet to be developed. Quantum-safe means that the encryption will resist breaking attempts from quantum computers.

Anyhow, the first typical type of encryption is the Symmetric Encryption. It uses the same key for encryption and decryption. The most common symmetric encryption algorithm is AES, the Advanced Encryption Standard.

const cipherText = encrypt(plaintext, encryptionKey);
const plaintext = decrypt(cipherText, encryptionKey);

The second type of encryption is Asymmetric Encryption. Hereby, the encryption and decryption routines use different but related keys, generally referred to as Private Key and Public Key. According to their names, the public key can be publicly known, while the private key must be kept private. Both keys are mathematically connected and generally based on a hard-to-solve mathematical problem. Two standard algorithms are essential: RSA (the Rivest–Shamir–Adleman encryption) and ECDSA (the Elliptic Curve Digital Signature Algorithm). While RSA is based on the prime factorization problem, ECDSA is based on the discrete log problem. To go into detail about those problems is more than just one additional blog post, though.

const cipherText = encrypt(plaintext, publicKey);
const plaintext = decrypt(cipherText, privateKey);

Encryption in Storage

The use cases for symmetric and asymmetric encryption are different. While considered more secure, asymmetric encryption is slower and typically not used for large data volumes. Symmetric encryption, however, is fast but requires the sharing of the encryption key between the encrypting and decrypting parties, making it less secure.

To encrypt large amounts of data and get the best of both worlds, security and speed, you often see a combination of both approaches. The symmetric encryption key is encrypted with an asymmetric encryption algorithm. After decrypting the symmetric key, it is used to encrypt or decrypt the stored data.

Simplyblock provides data-at-rest encryption directly through its Kubernetes CSI integration or via CLI and API. Additionally, simplyblock can be configured to use a different encryption key per logical volume for the highest degree of security and muti-tenant isolation.

Key Management Solutions

Next to selecting the encryption algorithm, managing and distributing the necessary keys is crucial. This is where key management solutions (KMS) come in.

Generally, there are two basic types of key management solutions: hardware and software-based.

For hardware-based solutions, you’ll typically utilize an HSM (Hardware Security Module). These HSMs provide a dedicated hardware token (commonly a USB key) to store keys. They offer the highest level of security but need to be physically attached to systems and are more expensive.

Software-based solutions offer a flexible key management alternative. Almost all cloud providers offer their own KMS systems, such as Azure Key Vault, AWS KMS, or Google Cloud KMS. Additionally, third-party key management solutions are available when setting up an on-premises or private cloud environment.

When managing more than a few encrypted volumes, you should implement a key management solution. That’s why simplyblock supports KMS solutions by default.

Implementing Data At Rest Encryption (DARE)

Simply said, you should have a key management solution ready to implement data-at-rest encryption in your company. If you run in the cloud, I recommend using whatever the cloud provider offers. If you run on-premises or the cloud provider doesn’t provide one, select from the existing third-party solutions.

DARE with Linux

As the most popular server operating system, Linux has two options to choose from when you want to encrypt data. The first option works based on files, providing a filesystem-level encryption.

Typical solutions for filesystem-level based encryption are eCryptfs, a stacked filesystem, which stores encryption information in the header information of each encrypted file. The benefit of eCryptfs is the ability to copy encrypted files to other systems without the need to decrypt them first. As long as the target node has the necessary encryption key in its keyring, the file will be decrypted. The alternative is EncFS, a user-space filesystem that runs without special permissions. Both solutions haven’t seen updates in many years, and I’d generally recommend the second approach.

Block-level encryption transparently encrypts the whole content of a block device (meaning, hard disk, SSD, simplyblock logical volume, etc.). Data is automatically decrypted when read. While there is VeraCrypt, it is mainly a solution for home setups and laptops. For server setups, the most common way to implement block-level encryption is a combination of dm-crypt and LUKS, the Linux Unified Key Setup.

Linux Data At Rest Encryption with dm-crypt and LUKS

The fastest way to encrypt a volume with dm-crypt and LUKS is via the cryptsetup tools.

Debian / Ubuntu / Derivates:

sudo apt install cryptsetup 

RHEL / Rocky Linux / Oracle:

yum install cryptsetup-luks

Now, we need to enable encryption for our block device. In this example, we assume we already have a whole block device or a partition we want to encrypt.

Warning: The following command will delete all data from the given device or partition. Make sure you use the correct device file.

cryptsetup -y luksFormat /dev/xvda

I recommend always running cryptsetup with the -y parameter, which forces it to ask for the passphrase twice. If you misspelled it once, you’ll realize it now, not later.

Now open the encrypted device.

cryptsetup luksOpen /dev/xvda encrypted

This command will ask for the passphrase. The passphrase is not recoverable, so you better remember it.

Afterward, the device is ready to be used as /dev/mapper/encrypted. We can format and mount it.

mkfs.ext4 /dev/mapper/encrypted
mkdir /mnt/encrypted
mount /dev/mapper/encrypted /mnt/encrypted

Data At Rest Encryption with Kubernetes

Kubernetes offers ephemeral and persistent storage as volumes. Those volumes can be statically or dynamically provisioned.

For pre-created and statically provisioned volumes, you can follow the above guide on encrypting block devices on Linux and make the already encrypted device available to Kubernetes.

However, Kubernetes doesn’t offer out-of-the-box support for encrypted and dynamically provisioned volumes. Encrypting persistent volumes is not in Kubernetes’s domain. Instead, it delegates this responsibility to its container storage provider, connected through the CSI (Container Storage Interface).

Note that not all CSI drivers support the data-at-rest encryption, though! But simplyblock does!

Data At Rest Encryption with Simplyblock

Due to the importance of DARE, simplyblock enables you to secure your data immediately through data-at-rest encryption. Simplyblock goes above and beyond with its features and provides a fully multi-tenant DARE feature set. That said, in simplyblock, you can encrypt multiple logical volumes (virtual block devices) with the same key or one key per logical volume.

The use cases are different. One key per volume enables the highest level of security and complete isolation, even between volumes. You want this to encapsulate applications or teams fully.

When multiple volumes share the same encryption key, you want to provide one key per customer. This ensures that you isolate customers against each other and prevent data from being accessible by other customers on a shared infrastructure in the case of a configuration failure or similar incident.

To set up simplyblock, you can configure keys manually or utilize a key management solution. In this example, we’ll set up the key manual. We also assume that simplyblock has already been deployed as your distributed storage platform, and the simplyblock CSI driver is available in Kubernetes.

First, let’s create our Kubernetes StorageClass for simplyblock. Deploy the YAML file via kubectl, just as any other type of resource.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
    name: encrypted-volumes
provisioner: csi.simplyblock.io
parameters:
    encryption: "True"
    csi.storage.k8s.io/fstype: ext4
    ... other parameters
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true

Secondly, we generate the two keys. Note the results of the two commands down.

openssl rand -hex 32   # Key 1
openssl rand -hex 32   # Key 2

Now, we can create our secret and Persistent Volume Claim (PVC).

apiVersion: v1
kind: Secret
metadata:
    name: encrypted-volume-keys
data:
    crypto_key1: 
    crypto_key2: 
–--
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
    annotations:
        simplybk/secret-name: encrypted-volume-keys
    name: encrypted-volume-claim
spec:
    storageClassName: encrypted-volumes
    accessModes:
        - ReadWriteOnce
    resources:
        requests:
            storage: 200Gi

And we’re done. Whenever we use the persistent volume claim, Kubernetes will delegate to simplyblock and ask for the encrypted volume. If it doesn’t exist yet, simplyblock will create it automatically. Otherwise, it will just provide it to Kubernetes directly. All data written to the logical volume is fully encrypted.

Best Practices for Securing Data at Rest

Implementing robust data encryption is crucial. In the best case, data should never exist in a decrypted state.

That said, data-in-use encryption is still complicated as of writing. However, solutions such as Edgeless Systems’ Constellation exist and make it possible using hardware memory encryption.

Data-in-transit encryption is commonly used today via TLS. If you don’t use it yet, there is no time to waste. Low-hanging fruits first.

Data-at-rest encryption in Windows, Linux, or Kubernetes doesn’t have to be complicated. Solutions such as simplyblock enable you to secure your data with minimal effort.

However, there are a few more things to remember when implementing DARE effectively.

Data Classification

Organizations should classify their data based on sensitivity levels and regulatory requirements. This classification guides encryption strategies and key management policies. A robust data classification system includes three things:

• Sensitive data identification: Identify sensitive data through automated discovery tools and manual review processes. For example, personally identifiable information (PII) like social security numbers should be classified as highly sensitive.
• Classification levels: Establish clear classification levels such as Public, Internal, Confidential, and Restricted. Each level should have defined handling requirements and encryption standards.
• Automated classification: Implement automated classification tools to scan and categorize data based on content patterns and metadata.

Access Control and Key Management

Encryption is only as strong as the key management and permission control around it. If your keys are leaked, the strongest encryption is useless.

Therefore, it is crucial to implement strong access controls and key rotation policies. Additionally, regular key rotation helps minimize the impact of potential key compromises and, I hate to say it, employees leaving the company.

Monitoring and Auditing

Understanding potential risks early is essential. That’s why it must maintain comprehensive logs of all access to encrypted data and the encryption keys or key management solutions. Also, regular audits should be scheduled for suspicious activities.

In the best case, multiple teams run independent audits to prevent internal leaks through dubious employees. While it may sound harsh, there are situations in life where people take the wrong path. Not necessarily on purpose or because they want to.

Data Minimization

The most secure data is the data that isn’t stored. Hence, you should only store necessary data.

Apart from that, encrypt only what needs protection. While this sounds counterproductive, it reduces the attack surface and the performance impact of encryption.

Data At Rest Encryption: The Essential Component of Data Management

Data-at-rest encryption (DARE) has become essential for organizations handling sensitive information. The rise in data breaches and increasingly stringent regulatory requirements make it vital to protect stored information. Additionally, with the rise of cloud computing and distributed systems, implementing DARE is more critical than ever.

Simplyblock integrates natively with Kubernetes to provide a seamless approach to implementing data-at-rest encryption in modern containerized environments. With our support for transparent encryption, your organization can secure its data without any application changes. Furthermore, simplyblock utilizes the standard NVMe over TCP protocol which enables us to work natively with Linux. No additional drivers are required. Use a simplyblock logical volume straight from your dedicated or virtual machine, including all encryption features.

Anyhow, for organizations running Kubernetes, whether in public clouds, private clouds, or on-premises, DARE serves as a fundamental security control. By following best practices and using modern tools like Simplyblock, organizations can achieve robust data protection while maintaining system performance and usability.

But remember that DARE is just one component of a comprehensive data security strategy. It should be combined with other security controls, such as access management, network security, and security monitoring, to create a defense-in-depth approach to protecting sensitive information.

That all said, by following the guidelines and implementations detailed in this article, your organization can effectively protect its data at rest while maintaining system functionality and performance.

As threats continue to evolve, having a solid foundation in data encryption becomes increasingly crucial for maintaining data security and regulatory compliance.

The post Encryption At Rest: A Comprehensive Guide to DARE appeared first on simplyblock.

One of the most important questions to answer when designing an application or infrastructure is the architecture approach to system scalability. Traditionally, systems used the scale-up approach or vertical scalability. Many modern systems use a scale-out approach, especially in the cloud-native ecosystem. Also called horizontal scalability.

Understanding the Basics

Understanding the fundamental concepts is essential when discussing system architectures. Hence, let’s briefly overview the two approaches before exploring them in more depth.

• With Scale Up (Vertical Scalability), you increase resources (typically CPU, memory, and storage) in the existing system to improve performance and capacity.
• With Scale Out (Horizontal Scalability), you add additional nodes or machines to the existing workforce to distribute the workload across multiple systems.

Both architectural approaches have their respective advantages and disadvantages. While scale-up architectures are easier to implement, they are harder to scale at a certain point. On the other hand, scale-out architectures are more complex to implement but scale almost linearly if done right.

Vertical Scaling (Scale Up) Architectures: The Traditional Approach

Vertical scaling, commonly known as scaling up, involves adding more resources to an existing system to increase its power or capacity.

Think of it as upgrading your personal computer. Instead of buying a second computer, you add more RAM or install a faster processor or larger storage device. In enterprise storage systems, this typically means adding more CPU cores, memory, or storage drives to an existing server. Meanwhile, for virtual machines it usually involves increasing the host machine’s assigned resources.

To clarify, let’s use a real-world example from the storage industry. With a ZFS-based SAN (Storage Area Network) system, a scaling up system design is required. Or as Jason Lohrey wrote: «However, ZFS has a significant issue – it can’t scale out. ZFS’s biggest limitation is that it is “scale-up” only.» ZFS, as awesome as it is, is limited to a single machine. That said, increasing the storage capacity always means adding larger or more disks to the existing machine. This approach maintains the simplicity of the original architecture while increasing storage capacity and potentially improving performance.

Strengths of Vertical Scaling

Today, many people see the vertical scalability approach as outdated and superfluous. That is, however, not necessarily true. Vertical scaling shines in several scenarios.

First, implementing a scale-up system is generally more straightforward since it doesn’t require changes to your application architectures or complex data distribution logic. When you scale up a transactional database like PostgreSQL or MySQL, you essentially give it more operational resources while maintaining the same operational model.

Secondly, the management overhead is lower. Tasks such as backups, monitoring, and maintenance are straightforward. This simplicity often translates to lower operational costs despite the potentially higher hardware costs.

Here is a quick overview of all the advantages:

1. Simplicity: It’s straightforward to implement since you’re just adding resources to an existing system
2. Lower Complexity: Less architectural overhead since you’re working with a single system
3. Consistent Performance: Lower latency due to all resources being in one place
4. Software Compatibility: Most traditional software is designed to run on a single system
5. Lower Initial Costs: Often cheaper for smaller workloads due to simpler licensing and management

Weaknesses and Limitations of Scale-Up Architectures

Like anything in this world, vertical scaling architectures also have drawbacks. The most significant limitation is the so-called physical ceiling. A system is limited by its server chassis’s space capacity or the hardware architecture’s limitation. You can only add as much hardware as those limitations allow. Alternatively, you need to migrate to a bigger base system.

Traditional monolithic applications often face another challenge with vertical scaling: adding more resources doesn’t always translate to linear performance improvements. For example, doubling the CPU cores might yield only a 50% performance increase due to software architecture limitations, especially resource contention.

Here is a quick overview of all the disadvantages:

1. Hardware Limits: The physical ceiling limits how much you can scale up based on maximum hardware specifications
2. Downtime During Upgrades: Usually requires system shutdown for hardware upgrades
3. Cost Efficiency: High-end hardware becomes exponentially more expensive
4. Single Point of Failure: No built-in redundancy
5. Limited Flexibility: Cannot easily scale back down when demand decreases

When to Scale Up?

After all that, here is when you really want to go with a scale-up architecture:

• You have traditional monolithic applications
• You look for an easier way to optimize for performance, not capacity
• You’re dealing with applications that aren’t designed for distributed computing
• You need a quick solution for immediate performance issues

Horizontal Scaling (Scale Out) Architectures: The Distributed Approach

The fundamentally different approach is the horizontal scaling or scale-out architecture. Instead of increasing the available resources on the existing system, you add more systems to distribute the load across them. This is actually similar to adding additional workers to an assembly line rather than trying to make one worker more efficient.

Consider a distributed storage system like simplyblock or a distributed database like MongoDB. When you scale out these systems, you add more nodes to the cluster, and the workload gets distributed across all nodes. Each node handles a portion of the data and processing, allowing the system to grow almost limitlessly.

Advantages of Horizontal Scaling

Large-scale deployments and highly distributed systems are the forte of scale-out architectures. As a simple example, most modern web applications utilize load balancers. They distribute the traffic across multiple application servers. This allows us to handle millions of concurrent requests and users. Similarly, distributed storage systems like simplyblock scale to petabytes of data by adding additional storage nodes.

Secondly, another significant advantage is improved high availability and fault tolerance. In a properly designed scale-out system, if one node fails, the system continues operating. While it may degrade to a reduced service, it will not experience a complete system failure or outage.

To bring this all to a point:

1. Near-Infinite Scalability: Can continue adding nodes as needed
2. Better Fault Tolerance: Built-in redundancy through multiple nodes
3. Cost Effectiveness: Can use commodity hardware
4. Flexible Resource Allocation: Easy to scale up or down based on demand
5. High Availability: No single point of failure

The Cost of Distribution: Weakness and Limitations of Horizontal Scalability

The primary challenge when considering scale-out architectures is complexity. Distributed systems must maintain data consistency across system boundaries, handle network communications or latencies, and handle failure recovery. Multiple algorithms have been developed over the years. The most commonly used ones are Raft and Paxos, but that’s a different blog post. Anyhow, this complexity typically requires more sophisticated management tools and distributed systems expertise. Normally also for the team operating the system.

The second challenge is the overhead of system coordination. In a distributed system, nodes must synchronize their operations. If not careful, this can introduce latency and even reduce the performance of certain types of operations. Great distributed systems utilize sophisticated algorithms to prevent these issues from happening.

Here is a quick overview of the disadvantages of horizontal scaling:

1. Increased Complexity: More moving parts to manage
2. Data Consistency Challenges: Maintaining consistency across nodes can be complex
3. Higher Initial Setup Costs: Requires more infrastructure and planning
4. Software Requirements: Applications must be designed for distributed computing
5. Network Overhead: Communication between nodes adds latency

Kubernetes: A Modern Approach to Scaling

Kubernetes has become the de facto platform for container orchestration. It comes in multiple varieties, in its vanilla form or as the basis for systems like OpenShift or Rancher. Either way, it can be used for both vertical and horizontal scaling capabilities. However, Kubernetes has become a necessity when deploying scale-out services. Let’s look at how different workloads scale in a Kubernetes environment.

Scaling Stateless Workloads

Stateless applications, like web servers or API gateways, are natural candidates for horizontal scaling in Kubernetes. The Horizontal Pod Autoscaler (HPA) provided by Kubernetes automatically adjusts the number of pods based on metrics such as CPU or RAM utilization. Custom metrics as triggers are also possible.

Horizontally scaling stateless applications is easy. As the name suggests, stateless applications do not maintain persistent local or shared state data. Each instance or pod is entirely independent and interchangeable. Each request to the service contains all the required information needed for processing.

That said, automatically scaling up and down (in the meaning of starting new instances or shutting some down) is part of the typical lifecycle and can happen at any point in time.

Scaling Stateful Workloads

Stateful workloads, like databases, require more careful consideration.

A common approach for more traditional databases like PostgreSQL or MySQL is to use a primary-replica architecture. In this design, write operations always go to the primary instance, while read operations can be distributed across all replicas.

On the other hand, MongoDB, which uses a distributed database design, can scale out more naturally by adding more shards to the cluster. Their internal cluster design uses a technique called sharding. Data is assigned to horizontally scaling partitions distributed across the cluster nodes. Shard assignment happens either automatically (based on the data) or by providing a specific shard key, enabling data affinity. Adding a shard to the cluster will increase capacity when additional scale is necessary. Data rebalancing happens automatically.

Why we built Simplyblock on a Scale-Out Architecture

Stateful workloads, like Postgres or MySQL, can scale out by adding additional read-replicas to the cluster. However, every single instance needs storage to store its very own data. Hence, the need for scalable storage arrives.

Simplyblock is a cloud-native and distributed storage platform built to deliver scalable performance and virtually infinite capacity for logical devices through horizontal scalability. Unlike traditional storage systems, simplyblock distributes data across all cluster nodes, multiplying the performance and capacity.

Designed as an NVMe-first architecture, simplyblock using the NVMe over Fabrics protocol family. This extends the reach of the highly scalable NVMe protocol over network fabrics such as TCP, Fibre Channel, and others. Furthermore, it provides built-in support for multi-pathing, enabling seamless failover and load balancing.

The system uses a distributed data placement algorithm to spread data across all available cluster nodes, automatically rebalancing data when nodes are added or removed. When writing data, simplyblock splits the item into multiple, smaller chunks and distributes them. This allows for parallel access during read operations. The data distribution also provides redundancy, with parity information stored on other nodes in the cluster. This protects the data against individual disk and node failures.

Using this architecture, simplyblock provides linear capacity and performance scalability by pooling all available disks and parallelizing access. This enables simplyblock to scale from mere terabytes to multiple petabytes while maintaining performance, consistency, and durability characteristics throughout the cluster-growth process.

Building Future-Proof Infrastructure

To wrap up, when you build out a new system infrastructure or application, consider these facts:

1. Workload characteristics: CPU-intensive workloads might benefit more from vertical scaling. Distributing operations comes with its own overhead. If the operation itself doesn’t set off this overhead, you might see lower performance than with vertical scaling. On the other hand, I/O-heavy workloads might perform better with horizontal scaling. If the access patterns are highly parallelizable, a horizontal architecture will most likely out scale a vertical one.
   
2. Growth patterns: Predictable, steady growth might favor scaling up, while rapid growth patterns might necessitate the flexibility of scaling out. This isn’t a hard rule, though. A carefully designed scale-out system will provide a very predictable growth pattern and latency. However, the application isn’t the only element to take into account when designing the system, as there are other components, most prominently the network and network equipment.
   
3. Future-Proofing: Scaling out often requires little upfront investment in infrastructure but higher investment in development and expertise. It can, however, provide better long-term cost efficiency for large deployments. That said, buying a scale-out solution is a great idea. With a storage solution like simplyblock, for example, you can start small and add required resources whenever necessary. With traditional storage solutions, you have to go with a higher upfront cost and are limited by the physical ceiling.
   
4. Operational Complexity: Scale-up architectures are typically easier to manage, while a stronger DevOps or operations team is required to handle scale-out solutions. That’s why simplyblock’s design is carefully crafted to be fully autonomous and self-healing, with as few hands-on requirements as possible.

The Answer Depends

That means there is no universal answer to whether scaling up or out is better. A consultant would say, “It depends.” Seriously, it does. It depends on your specific requirements, constraints, and goals.

Many successful organizations use a hybrid approach, scaling up individual nodes while also scaling out their overall infrastructure. The key is understanding the trade-offs and choosing the best approach to your needs while keeping future growth in mind. Hence, simplyblock provides the general scale-out architecture for infinite scalability. It also provides a way to utilize storage located in Kubernetes worker nodes as part of the storage cluster to provide the highest possible performance. At the same time, it maintains the option to spill over when local capacity is reached and the high durability and fault tolerance of a fully distributed storage system.

Remember, the best scaling strategy aligns with your business objectives while maintaining performance, reliability, and cost-effectiveness. Whether you scale up, out, or both, ensure your choice supports your long-term infrastructure goals.

The post Scale Up vs Scale Out: System Scalability Strategies appeared first on simplyblock.