One of the most important questions to answer when designing an application or infrastructure is the architecture approach to system scalability. Traditionally, systems used the scale-up approach or vertical scalability. Many modern systems use a scale-out approach, especially in the cloud-native ecosystem. Also called horizontal scalability.

Understanding the Basics

Understanding the fundamental concepts is essential when discussing system architectures. Hence, let’s briefly overview the two approaches before exploring them in more depth.

• With Scale Up (Vertical Scalability), you increase resources (typically CPU, memory, and storage) in the existing system to improve performance and capacity.
• With Scale Out (Horizontal Scalability), you add additional nodes or machines to the existing workforce to distribute the workload across multiple systems.

Both architectural approaches have their respective advantages and disadvantages. While scale-up architectures are easier to implement, they are harder to scale at a certain point. On the other hand, scale-out architectures are more complex to implement but scale almost linearly if done right.

Vertical Scaling (Scale Up) Architectures: The Traditional Approach

Vertical scaling, commonly known as scaling up, involves adding more resources to an existing system to increase its power or capacity.

Think of it as upgrading your personal computer. Instead of buying a second computer, you add more RAM or install a faster processor or larger storage device. In enterprise storage systems, this typically means adding more CPU cores, memory, or storage drives to an existing server. Meanwhile, for virtual machines it usually involves increasing the host machine’s assigned resources.

To clarify, let’s use a real-world example from the storage industry. With a ZFS-based SAN (Storage Area Network) system, a scaling up system design is required. Or as Jason Lohrey wrote: «However, ZFS has a significant issue – it can’t scale out. ZFS’s biggest limitation is that it is “scale-up” only.» ZFS, as awesome as it is, is limited to a single machine. That said, increasing the storage capacity always means adding larger or more disks to the existing machine. This approach maintains the simplicity of the original architecture while increasing storage capacity and potentially improving performance.

Strengths of Vertical Scaling

Today, many people see the vertical scalability approach as outdated and superfluous. That is, however, not necessarily true. Vertical scaling shines in several scenarios.

First, implementing a scale-up system is generally more straightforward since it doesn’t require changes to your application architectures or complex data distribution logic. When you scale up a transactional database like PostgreSQL or MySQL, you essentially give it more operational resources while maintaining the same operational model.

Secondly, the management overhead is lower. Tasks such as backups, monitoring, and maintenance are straightforward. This simplicity often translates to lower operational costs despite the potentially higher hardware costs.

Here is a quick overview of all the advantages:

1. Simplicity: It’s straightforward to implement since you’re just adding resources to an existing system
2. Lower Complexity: Less architectural overhead since you’re working with a single system
3. Consistent Performance: Lower latency due to all resources being in one place
4. Software Compatibility: Most traditional software is designed to run on a single system
5. Lower Initial Costs: Often cheaper for smaller workloads due to simpler licensing and management

Weaknesses and Limitations of Scale-Up Architectures

Like anything in this world, vertical scaling architectures also have drawbacks. The most significant limitation is the so-called physical ceiling. A system is limited by its server chassis’s space capacity or the hardware architecture’s limitation. You can only add as much hardware as those limitations allow. Alternatively, you need to migrate to a bigger base system.

Traditional monolithic applications often face another challenge with vertical scaling: adding more resources doesn’t always translate to linear performance improvements. For example, doubling the CPU cores might yield only a 50% performance increase due to software architecture limitations, especially resource contention.

Here is a quick overview of all the disadvantages:

1. Hardware Limits: The physical ceiling limits how much you can scale up based on maximum hardware specifications
2. Downtime During Upgrades: Usually requires system shutdown for hardware upgrades
3. Cost Efficiency: High-end hardware becomes exponentially more expensive
4. Single Point of Failure: No built-in redundancy
5. Limited Flexibility: Cannot easily scale back down when demand decreases

When to Scale Up?

After all that, here is when you really want to go with a scale-up architecture:

• You have traditional monolithic applications
• You look for an easier way to optimize for performance, not capacity
• You’re dealing with applications that aren’t designed for distributed computing
• You need a quick solution for immediate performance issues

Horizontal Scaling (Scale Out) Architectures: The Distributed Approach

The fundamentally different approach is the horizontal scaling or scale-out architecture. Instead of increasing the available resources on the existing system, you add more systems to distribute the load across them. This is actually similar to adding additional workers to an assembly line rather than trying to make one worker more efficient.

Consider a distributed storage system like simplyblock or a distributed database like MongoDB. When you scale out these systems, you add more nodes to the cluster, and the workload gets distributed across all nodes. Each node handles a portion of the data and processing, allowing the system to grow almost limitlessly.

Advantages of Horizontal Scaling

Large-scale deployments and highly distributed systems are the forte of scale-out architectures. As a simple example, most modern web applications utilize load balancers. They distribute the traffic across multiple application servers. This allows us to handle millions of concurrent requests and users. Similarly, distributed storage systems like simplyblock scale to petabytes of data by adding additional storage nodes.

Secondly, another significant advantage is improved high availability and fault tolerance. In a properly designed scale-out system, if one node fails, the system continues operating. While it may degrade to a reduced service, it will not experience a complete system failure or outage.

To bring this all to a point:

1. Near-Infinite Scalability: Can continue adding nodes as needed
2. Better Fault Tolerance: Built-in redundancy through multiple nodes
3. Cost Effectiveness: Can use commodity hardware
4. Flexible Resource Allocation: Easy to scale up or down based on demand
5. High Availability: No single point of failure

The Cost of Distribution: Weakness and Limitations of Horizontal Scalability

The primary challenge when considering scale-out architectures is complexity. Distributed systems must maintain data consistency across system boundaries, handle network communications or latencies, and handle failure recovery. Multiple algorithms have been developed over the years. The most commonly used ones are Raft and Paxos, but that’s a different blog post. Anyhow, this complexity typically requires more sophisticated management tools and distributed systems expertise. Normally also for the team operating the system.

The second challenge is the overhead of system coordination. In a distributed system, nodes must synchronize their operations. If not careful, this can introduce latency and even reduce the performance of certain types of operations. Great distributed systems utilize sophisticated algorithms to prevent these issues from happening.

Here is a quick overview of the disadvantages of horizontal scaling:

1. Increased Complexity: More moving parts to manage
2. Data Consistency Challenges: Maintaining consistency across nodes can be complex
3. Higher Initial Setup Costs: Requires more infrastructure and planning
4. Software Requirements: Applications must be designed for distributed computing
5. Network Overhead: Communication between nodes adds latency

Kubernetes: A Modern Approach to Scaling

Kubernetes has become the de facto platform for container orchestration. It comes in multiple varieties, in its vanilla form or as the basis for systems like OpenShift or Rancher. Either way, it can be used for both vertical and horizontal scaling capabilities. However, Kubernetes has become a necessity when deploying scale-out services. Let’s look at how different workloads scale in a Kubernetes environment.

Scaling Stateless Workloads

Stateless applications, like web servers or API gateways, are natural candidates for horizontal scaling in Kubernetes. The Horizontal Pod Autoscaler (HPA) provided by Kubernetes automatically adjusts the number of pods based on metrics such as CPU or RAM utilization. Custom metrics as triggers are also possible.

Horizontally scaling stateless applications is easy. As the name suggests, stateless applications do not maintain persistent local or shared state data. Each instance or pod is entirely independent and interchangeable. Each request to the service contains all the required information needed for processing.

That said, automatically scaling up and down (in the meaning of starting new instances or shutting some down) is part of the typical lifecycle and can happen at any point in time.

Scaling Stateful Workloads

Stateful workloads, like databases, require more careful consideration.

A common approach for more traditional databases like PostgreSQL or MySQL is to use a primary-replica architecture. In this design, write operations always go to the primary instance, while read operations can be distributed across all replicas.

On the other hand, MongoDB, which uses a distributed database design, can scale out more naturally by adding more shards to the cluster. Their internal cluster design uses a technique called sharding. Data is assigned to horizontally scaling partitions distributed across the cluster nodes. Shard assignment happens either automatically (based on the data) or by providing a specific shard key, enabling data affinity. Adding a shard to the cluster will increase capacity when additional scale is necessary. Data rebalancing happens automatically.

Why we built Simplyblock on a Scale-Out Architecture

Stateful workloads, like Postgres or MySQL, can scale out by adding additional read-replicas to the cluster. However, every single instance needs storage to store its very own data. Hence, the need for scalable storage arrives.

Simplyblock is a cloud-native and distributed storage platform built to deliver scalable performance and virtually infinite capacity for logical devices through horizontal scalability. Unlike traditional storage systems, simplyblock distributes data across all cluster nodes, multiplying the performance and capacity.

Designed as an NVMe-first architecture, simplyblock using the NVMe over Fabrics protocol family. This extends the reach of the highly scalable NVMe protocol over network fabrics such as TCP, Fibre Channel, and others. Furthermore, it provides built-in support for multi-pathing, enabling seamless failover and load balancing.

The system uses a distributed data placement algorithm to spread data across all available cluster nodes, automatically rebalancing data when nodes are added or removed. When writing data, simplyblock splits the item into multiple, smaller chunks and distributes them. This allows for parallel access during read operations. The data distribution also provides redundancy, with parity information stored on other nodes in the cluster. This protects the data against individual disk and node failures.

Using this architecture, simplyblock provides linear capacity and performance scalability by pooling all available disks and parallelizing access. This enables simplyblock to scale from mere terabytes to multiple petabytes while maintaining performance, consistency, and durability characteristics throughout the cluster-growth process.

Building Future-Proof Infrastructure

To wrap up, when you build out a new system infrastructure or application, consider these facts:

1. Workload characteristics: CPU-intensive workloads might benefit more from vertical scaling. Distributing operations comes with its own overhead. If the operation itself doesn’t set off this overhead, you might see lower performance than with vertical scaling. On the other hand, I/O-heavy workloads might perform better with horizontal scaling. If the access patterns are highly parallelizable, a horizontal architecture will most likely out scale a vertical one.
   
2. Growth patterns: Predictable, steady growth might favor scaling up, while rapid growth patterns might necessitate the flexibility of scaling out. This isn’t a hard rule, though. A carefully designed scale-out system will provide a very predictable growth pattern and latency. However, the application isn’t the only element to take into account when designing the system, as there are other components, most prominently the network and network equipment.
   
3. Future-Proofing: Scaling out often requires little upfront investment in infrastructure but higher investment in development and expertise. It can, however, provide better long-term cost efficiency for large deployments. That said, buying a scale-out solution is a great idea. With a storage solution like simplyblock, for example, you can start small and add required resources whenever necessary. With traditional storage solutions, you have to go with a higher upfront cost and are limited by the physical ceiling.
   
4. Operational Complexity: Scale-up architectures are typically easier to manage, while a stronger DevOps or operations team is required to handle scale-out solutions. That’s why simplyblock’s design is carefully crafted to be fully autonomous and self-healing, with as few hands-on requirements as possible.

The Answer Depends

That means there is no universal answer to whether scaling up or out is better. A consultant would say, “It depends.” Seriously, it does. It depends on your specific requirements, constraints, and goals.

Many successful organizations use a hybrid approach, scaling up individual nodes while also scaling out their overall infrastructure. The key is understanding the trade-offs and choosing the best approach to your needs while keeping future growth in mind. Hence, simplyblock provides the general scale-out architecture for infinite scalability. It also provides a way to utilize storage located in Kubernetes worker nodes as part of the storage cluster to provide the highest possible performance. At the same time, it maintains the option to spill over when local capacity is reached and the high durability and fault tolerance of a fully distributed storage system.

Remember, the best scaling strategy aligns with your business objectives while maintaining performance, reliability, and cost-effectiveness. Whether you scale up, out, or both, ensure your choice supports your long-term infrastructure goals.

The post Scale Up vs Scale Out: System Scalability Strategies appeared first on simplyblock.

In this article, we explore the impact of unstructured data on data volumes. We’ll emphasize the shift from ML to AI. Finally, we underscore the significance of a forward-looking data architecture for businesses aiming to be data-first in the era of AI. Scale-out storage infrastructure plays a key role in this process. We will put that in the context of Intelligent Data Infrastructure (IDI).

Understanding the Scale-Out Approaches in AI Storage

Scale-up approaches keep adding more resources like CPU, RAM, and storage to existing servers. While traditional data centers rely heavily on such scale-up architecture, modern AI workloads demand vertical and horizontal scaling capabilities. Scale-out architecture involves adding more nodes to distribute the workload. It has become crucial for handling massive AI datasets. However, organizations need the flexibility to scale up individual nodes. Flexibility for performance-intensive workloads and to scale out their infrastructure to handle growing data volumes.

Comparing Scale Up vs Scale Out Approaches

Scale-up architecture focuses on increasing the capacity of existing nodes by adding more CPU, memory, and storage resources. This vertical scaling approach offers benefits like:

• Simpler management of fewer, more powerful nodes
• Lower network overhead and latency
• Better performance for single-threaded workloads

In contrast, scale-out architecture distributes workloads across multiple nodes, offering advantages such as:

• Linear performance scaling by adding nodes
• Better fault tolerance through redundancy
• More cost-effective growth using commodity hardware

For AI workloads, organizations often need both approaches—scaling up nodes for compute-intensive tasks like model training and scaling out storage and processing capacity for massive datasets. The key is finding the right balance based on specific workload requirements.

Why AI Storage for Unstructured Data?

One of the defining characteristics of the AI era is the exponential growth of unstructured data. It is estimated that even up to 95% of today’s data is unstructured. That means it is not considered “data” in the context of current data infrastructures. These are images, videos, text documents, social media feeds, and other types of “data” that aren’t used as a base for data-driven decision-making today. AI is changing that with its ability to convert unstructured data into structured data. AI models feed themselves with diverse data types that are invaluable for their training, yet they also pose a significant challenge in terms of storage, processing, and retrieval. All the data that nobody cared about in cold storage as of yesterday is now at the core of data infrastructure today.

Unstructured data, such as images and videos, tends to be larger in size compared to structured data. This exponential growth in data volumes strains traditional data infrastructure, necessitating more scalable solutions. It also comes in a myriad of formats and structures. Managing this complexity becomes critical for organizations. They aim to harness the insights buried within unstructured datasets. Data Infrastructure’s adaptability is indispensable in handling the variety and complexity inherent in unstructured data.

The Scale of Training Data

AI models that leverage unstructured data, especially in tasks like image recognition or natural language processing, require significant computational power. The demand for scalable compute resources becomes paramount, and the ability to dynamically allocate resources between storage and compute is key for efficiency at scale. Distinct from traditional Machine Learning (ML) datasets, these AI-scale datasets, in the realm of image recognition, natural language processing, and complex simulations, reach massive scales and often come with storage requirements in the hundreds of terabytes. Data infrastructure must be tailor-made for such workloads, enabling dynamic resource allocation and efficient management of these vast datasets.

Introducing Intelligent Data Infrastructure (IDI)

Intelligent Data Infrastructure (IDI) is a novel concept that reimagines how organizations handle and utilize their data. At its core, it involves decomposing traditional monolithic data systems into modular components that can be dynamically orchestrated to meet specific requirements. IDI can be built on public or private clouds, on-premises, or hybrid cloud scenarios. This modular, containerized, and fully portable approach enables organizations to build a data infrastructure that is not only scalable but also adaptable to the evolving needs of AI applications and businesses. The Intelligent Data Infrastructure (IDI) requires a few components to deliver its full potential.

Decoupled Storage and Compute

Intelligent Data Infrastructure (IDI) separates storage and compute resources, allowing organizations to scale each independently. This decoupling is particularly beneficial for AI workloads, where computational demands vary significantly. By allocating resources dynamically, organizations can optimize performance and cost-effectiveness.

Metadata-Driven Architecture

A metadata-driven architecture is crucial to Intelligent Data Infrastructures (IDI). Metadata provides essential information about the data. That makes it easier to discover, understand, and process. In the context of AI, where diverse datasets with varying structures are common, a metadata-driven approach enhances flexibility and facilitates efficient data handling. Storing and accessing large amounts of metadata might require the ability to scale IOPS without limitations to accommodate the unpredictability of the workloads. Today, IOPS limitations are a common problem faced by users of public clouds.

API-Based Connectivity

Intelligent Data Infrastructure (IDI) relies on APIs (Application Programming Interfaces) for seamless connectivity between different components. This API-centric approach enables interoperability and integration with various tools and platforms. This fosters a collaborative ecosystem for AI development.

Orchestration and Automation

Orchestration and automation are pivotal in Intelligent Data Infrastructure (IDI). By automating tasks such as data ingestion, processing, and model deployment, organizations can streamline their AI workflows and reduce the time-to-value for AI projects. Automation on the storage layer is key to cater to these requirements.

Portability, Tiering, and Containerization

Workload portability has never been better. But data portability (or data infrastructures) has yet to catch up. Kubernetes made it easy to orchestrate the movement of workloads. However, due to storage and data gravity, the most common use cases for Kubernetes are stateless workloads. The shift of stateful workloads into Kubernetes is consistent with the rise of Intelligent Data Infrastructure. Intelligent storage tiering further allows us to build data infrastructures in the most efficient and agnostic way.

Building Storage for the AI Era

Unlike traditional systems, Intelligent Data Infrastructure (IDI) is architected to handle the massive scale of AI datasets. The flexibility to both scale up individual nodes for performance-intensive workloads and scale out storage across multiple nodes. The ability to scale out storage horizontally and vertically, coupled with dynamic resource allocation, ensures optimal performance for AI workloads. Future-proofing data platforms is crucial in the fast-paced AI era. With its modular and adaptable design, Intelligent Data Infrastructure (IDI) enables organizations to stay ahead by easily integrating new technologies and methodologies as they emerge, ensuring longevity and relevance.

As AI becomes a driving force across industries, every business is poised to become a data and AI business. Intelligent Data Infrastructure (IDI) facilitates this transition by providing the flexibility and scalability needed by businesses to leverage data as a strategic asset. The modular nature of Intelligent Data Infrastructure (IDI) empowers organizations to adapt to evolving AI requirements. Whether integrating new data sources or accommodating changes in processing algorithms, a flexible infrastructure ensures agility in the face of dynamic AI landscapes.

Organizations can optimize infrastructure costs by decoupling storage and computing resources and dynamically allocating them as needed. This cost efficiency is particularly valuable in AI, where resource requirements can vary widely depending on the nature of the tasks at hand. While cloud services are becoming commoditized, the edge lies in how businesses build and optimize their data infrastructure. A unique approach to data management, storage, and processing can provide a competitive advantage, making businesses more agile, innovative, and responsive to the demands of the AI era.

How can Organizations adopt Intelligent Data Infrastructure as AI Storage?

In the era of AI, where unstructured data reigns supreme and businesses are transitioning to become data-first, the role of Intelligent Data Infrastructure (IDI) cannot be overstated. It addresses the challenges posed by the sheer volumes of unstructured data and provides a forward-looking foundation for businesses to thrive in AI. As businesses strive to differentiate themselves, a strategic focus on building a unique and scalable data infrastructure will undoubtedly be the key to gaining a competitive edge in the evolving world of artificial intelligence. The first step of adopting IDI in your organization should be identifying bottlenecks and challenges with the current data infrastructure. Some of the questions one should ask are:

1. Is your current data infrastructure horizontally scalable?
2. Do you face IOPS limits?
3. Are you resorting to the use of sub-optimal storage services to save costs? (e.g., using object storage because it’s “cheap”)
4. Can you scale out storage and compute resources without scaling storage or vice versa?
5. Based on workload demands, can your infrastructure scale up (add resources to existing nodes) and scale out (add new nodes)?
6. Can you easily and efficiently migrate data and workloads between clouds and environments?
7. What is the level of automation in your data infrastructure?
8. Do you use intelligent data services (such as deduplication and automatic resource balancing) to decrease your organization’s data storage requirements?

Organizations following the traditional approaches to data infrastructures would not be able to answer these questions easily, which by itself would be a warning sign that they are far from adopting IDI. As always, awareness of the problem needs to come first. At simplyblock, we help you adopt an intelligent data infrastructure without the burden of re-architecting everything, providing drop-in solutions to boost your data infrastructure with the sight of the AI era.

How can Simplyblock help to build a Storage System for IDI?

Simplyblock’s high-performance scale-out storage clusters are built upon EC2 instances with local NVMe disks. Our technology uses NVMe over TCP for minimal access latency, high IOPS/GB, and efficient CPU core utilization, surpassing local NVMe disks and Amazon EBS in cost/performance ratio at scale. Ideal for high-performance Kubernetes environments, simplyblock combines the benefits of local-like latency with the scalability and flexibility necessary for dynamic AWS EKS deployments, ensuring optimal performance for I/O-sensitive workloads like databases. Using erasure coding (a better RAID) instead of replicas helps to minimize storage overhead without sacrificing data safety and fault tolerance.

Additional features such as instant snapshots (full and incremental), copy-on-write clones, thin provisioning, compression, encryption, and many more, make simplyblock the perfect solution that meets your requirements before you set them. Get started using simplyblock right now, or learn more about our feature set.

The post AI Storage: How To Build Scalable Data Infrastructures for AI workloads? appeared first on simplyblock.