Discord dreamed of something like it a little while ago but couldn’t find a solution, so they started implementing their own. But not too fast. Let’s get the background out of the way first.

The Challenge: Balancing Performance and Cost

Running a high-velocity database in the cloud constantly struggles to balance high performance versus your cloud spending. It’s not only you and me, it’s everyone. Discord saw the same challenge with storing more than 4 billion messages by millions of users per day.

My immediate first reaction is local storage. It has low latency and high throughput. But it comes at the cost of, first and foremost, cost, as well as the problems of backup and scalability.

Persistent cloud disks, on the other hand, offer great scalability and are automatically replicated. However, due to the introduced network latency, they typically have lower throughput and higher access latency. This issue quickly becomes the main bottleneck in a system. You can get around this issue but at a hefty price.

Simplyblock’s Local SSD Caching Solution

At simplyblock, we love a good challenge. Hence, we took it upon ourselves to implement the best of both worlds: a super-scalable and low-latency software-defined storage engine with zero downtime resource scalability and the option to add local SSD near caching for the lowest latency possible.

This feature helps to significantly boost the read performance of simplyblock’s logical SSD volumes by caching data locally on directly attached SSD devices. But what does this mean in practice, and how does it benefit users?

How does Local SSD Caching Work?

At its core, simplyblock’s caching functionality leverages the blazing-fast speeds of local NVMe SSDs to create a high-performance and transparent caching layer. Here’s our simplified process:

1. When data is read from the logical volume, it’s stored in the local SSD cache (read-behind).
2. Subsequent read requests for the same data are served directly from the cache, bypassing access to the main storage volume.
3. This caching mechanism dramatically reduces latency and increases throughput for frequently accessed data.
4. When data is written to the logical volume, the information is passed through to the underlying backend storage and stored locally in the cache (write-through).

The beauty of this approach lies in its transparency, simplicity, and effectiveness. By caching data close to your workload, simplyblock eliminates the performance bottlenecks associated with network-attached storage, all while maintaining the benefits of centralized storage management.

Additionally, the caching layer is built into the solution and fully transparent to your application, meaning there is no need to change the application. As a developer, my heart could not jump higher.

What are the Benefits of Local SSD Caching?

In addition, simplyblock’s local SSD caching brings quite a few more benefits to the table:

1. Dramatically Improved Read Performance: By serving frequently accessed data from local SSDs, latency is significantly reduced, and throughput is increased.
2. Cost-Effective Performance Boost: Users can achieve high-performance storage without the need to invest in expensive, all-flash storage for their entire dataset.
3. Scalability: The caching functionality can be easily scaled to meet growing performance demands without overhauling the entire storage infrastructure.
4. Flexibility: Simplyblock’s solution provides logical (virtual) block storage devices, which means it works seamlessly with existing setups and provides a performance boost without disrupting current deployment processes.
5. Optimized Resource Utilization: By caching hot data locally, network traffic to the main storage volume is reduced, optimizing overall resource utilization.
6. Transparency: Making caching an integral part of simplyblock’s storage architecture, it is fully transparent to users. Hence, it works with any existing workload, such as databases, analytics tools, etc.

Real-World Validation: Discord’s Super-Disks

As mentioned before, in 2021 / 2022, Discord faced a similar challenge. They needed to scale persistent storage and get the benefits of internal storage data protection (in their case, GCP Persistent Disk with automatic replication). Still, they found that remotely attached disks were “too slow.”

That said, Discord, known for its popular communication platform, encountered performance issues using Google Cloud Platform’s Persistent Disks. Much like simplyblock’s caching functionality, their workaround involved using local SSDs to cache data and dramatically improve performance.

In his blog post from 2022, Glen Oakley, Senior Software Engineer at Discord, wrote about the reason to implement it in-house: “No such disk exists, at least not within the ecosystem of common cloud providers.” Glen noted in the blog post: “By using local SSDs as a cache, we can get the best of both worlds: the durability and ease-of-use of network block storage, with performance that rivals local disks.”

This is what simplyblock is. But it’s always good to have a real-world example from a major tech player to underscore the validity and importance of local caching solutions like simplyblock’s. Clearly, the challenge of balancing performance and cost-effectiveness in cloud storage is a widespread concern, and innovative and standardized caching solutions are emerging as a powerful answer. Who knows how many people built their own workaround already? Even during my time at Hazelcast, I kept telling people, “Don’t roll your own caching; it comes back to bite you,” as caching is commonly more complicated than just storing stuff.

Anyhow, I totally recommend you read Glen’s full blog post at Discord’s blog site.

Simplyblock’s Architecture: A Closer Look to Local SSD Caching

While the concept behind simplyblock’s local SSD caching is straightforward, the implementation is special in more than one regard and designed for the highest storage optimization. Let’s dive deeper into simplyblock.

SPDK-Based

Simplyblock’s persistent storage engine is built upon the Storage Performance Development Kit (SPDK) but extends it with data distribution, cluster management, erasure coding, and more. SPDK is a set of tools and libraries designed to write high-performance, scalable, user-mode storage applications. By integrating SPDK, simplyblock ensures that its storage solution is effective, highly efficient, and scalable.

Flexible Configuration

One of the standout features of simplyblock’s implementation is its flexibility. You can easily configure logical volumes, their performance profiles, and caching properties to suit your specific needs. This includes setting up custom cache sizes, choosing between different caching algorithms, and fine-tuning other parameters to optimize performance for your unique workloads.

Seamless Integration

Simplyblock has designed its caching functionality to be as seamless and transparent to you as possible. This means you can implement this performance-boosting feature without disrupting your current infrastructure or requiring significant changes to your workload.

Intelligent Cache Management

Behind the scenes, simplyblock’s caching solution employs algorithms to manage the data and caches effectively. This includes:

Intelligent data placement to ensure the most frequently accessed data is always readily available Efficient cache eviction policies to make room for new data when the cache fills up Consistency mechanisms to ensure data integrity between the cache and the main storage volume (write-through and read-behind)

The Bigger Picture: Transforming Storage Performance

Simplyblock’s local SSD caching functionality represents more than just a performance boost – it is a paradigm shift in how we approach storage in the cloud era. By bridging the gap between high-performance local storage and the flexibility of network-attached volumes, simplyblock is paving the way for a new generation of storage solutions.

This innovation has far-reaching implications across various industries and use cases:

• High-Performance Computing: Research institutions and scientific organizations can accelerate data-intensive computations without breaking the bank on all-flash arrays.
• Databases: High-velocity databases that benefit from low access latency and high throughput to quickly read data from disk for large reports or real-time data analytics.
• Observability: Append-only or append-mostly storage engines and analytics platforms, running large aggregations and anomaly detection algorithms.
• AI / Machine Learning: Running the latest LLMs (Large Language Models) or other AI workloads and training sets requires large amounts of data and high-performance storage, but also consistent latency for predictable runtime performance.
• Financial Services: Banks and fintech companies can speed up transaction processing and analytics workloads, improving customer experiences and decision-making capabilities.
• Media and Entertainment: Content delivery networks can cache frequently accessed media files locally, reducing latency and improving streaming quality.
• E-commerce: Online retailers can boost the performance of their product catalogs and recommendation engines, leading to faster page loads and improved customer satisfaction.
• Your High-Performance Workload: Whatever your data-intensive, low-latency sensitive, and high-performance use case looks like, simplyblock’s persistent storage solution can help you with a fully cloud-native solution.

Looking Ahead: The Future of Persistent Storage Performance

As data volumes continue to grow exponentially year-over-year and workloads, such as databases, become increasingly demanding, solutions like simplyblock’s local SSD caching will play a crucial role in shaping the future of storage performance. We can expect to see further innovations in this space, including:

More sophisticated caching algorithms leveraging machine learning to predict and preload data Tighter integration between caching solutions and emerging technologies like persistent memory Expansion of caching concepts to other areas of the data center, such as network and compute resources

Conclusion: Free the Full Performance Potential of your Data

In an era where data is the breath of our businesses, the ability to access and process information quickly can make or break an organization’s success. Simplyblock’s local SSD caching functionality represents a significant leap forward in our ability to harness the full performance potential of our data.

By offering a solution that combines the performance of local SSDs with the flexibility and scalability of network-attached persistent storage, simplyblock empowers businesses to achieve unprecedented storage performance without sacrificing cost-effectiveness or ease of use and management.

As we look to the future, innovations like simpleblock’s caching functionality will play a pivotal role in shaping the next generation of data storage and processing technologies. For organizations looking to stay ahead of the curve and unlock the full potential of their data, embracing these cutting-edge solutions is not just an option—it’s a necessity.

Learn more about simplyblock and its innovative storage solutions.

The post Say Goodbye to High Data Access-Latency Cloud Storage with Simplyblock’s Local NVMe SSD Caching appeared first on simplyblock.

This article deep dives into how we’ve improved upon traditional distributed data placement algorithms to create a high-performance I/O processing environment that meets modern enterprise storage requirements.

Design Of Simplyblock’s Storage Cluster

Simplyblock storage cluster technology is designed to utilize NVMe storage devices in public cloud environments for use cases that require predictable and ultra-low access latency (sub-millisecond) and the highest performance density (high IOPS per GiB).

To combine high performance with a high degree of data durability, high availability, and fault tolerance, as well as zero downtime scalability, the known distributed data placement algorithms had to be improved, re-combined, and implemented into a high-performance IO processing environment.

Our innovative approach combines:

• Predictable, ultra-low latency performance (<1ms)
• Maximum IOPS density optimization
• Enterprise-grade durability and availability
• Zero-downtime scalability
• Advanced failure domain management

Modern Storage Requirements

Use cases such as high-load databases, time-series databases with high-velocity data, Artificial Intelligence (AI), Machine Learning (ML), and many others require fast and predictable storage solutions.

Anyhow, performance isn’t everything. The fastest storage is writing to /dev/null, but only if you don’t need the data durability. That said, the main goals for a modern storage solution are:

• High Performance Density, meaning a high amount of IOPS per Gigabyte (at an affordable price).
• Predictable, low Latency, especially for use cases that require consistent response times.
• High degree of Data Durability, to distribute the data across failure domains, enabling it to survive multiple failure scenarios.
• High Availability and Fault Tolerance, for the data to remain accessible in case of node outage. Clusters are automatically re-balanced in the case of element failures.
• Zero Downtime Scalability, meaning that clusters can grow in real-time and online and are automatically re-balanced.

Distributed Data Placement

Data placement in storage clusters commonly uses pseudo-randomization. Additionally, features such as weighted distribution of storage across the cluster (based on the capacity and performance of available data buckets) are introduced to handle failure domains and cluster rebalancing – for scaling, downsizing, or removal of failed elements – at minimal cost. A prominent example of such an algorithm is CRUSH (Controlled, Scalable, Decentralized Placement of Replicated Data), which is used in Ceph, an open-source software-defined storage platform designed to provide object storage, block storage, and file storage in a unified system.

Simplyblock uses a different algorithm to achieve the following characteristics for its distributed data placement feature:

• High storage efficiency (raw to effective storage ratio) with minimal performance overhead. Instead of using three data replicas, which is the standard mechanism to protect data from storage device failure in software-defined storage clusters, simplyblock uses error coding algorithms with a raw-to-effective ratio of about 1.33 (instead of 3).
• Very low access latency below 100 microseconds for read and write. Possible write amplification below 2.
• Ultra-high IOPS density with more than 200.000 IOPS per CPU core.
• Performant re-distribution of storage in the cluster in the case of cluster scaling and removal of failed storage devices. Simplyblock’s algorithm will only re-distribute the amount of data that is close to the theoretical minimum to rebalance the cluster.
• Support for volume high-availability based on the NVMe industry standard. Support for simple failure domains as they are available in cloud environments (device, node, rack, availability zone).
• Performance efficiency aims to address the typical performance bottlenecks in cloud environments.

Implementing a storage solution to keep up with current trends required us to think out of the box. The technical design consists of several elements.

Low-level I/O Processing Pipeline

On the lower level, simplyblock uses a fixed-size page mapping algorithm implemented in a virtual block device (a virtual block device implements a filter or transformation step in the IO processing pipeline).

For that purpose, IO is organized into “pages” ( 2^m blocks, with m in the range of 8 to 12). Cross-page IO has to be split before processing. This is done on the mid-level processing pipeline. We’ll get to that in a second.

This algorithm can place data received via IO-write requests from multiple virtual block devices on a single physical block device. Each virtual block device has its own logical block address space though. The algorithm is designed to read, write, and unmap (deallocate) data with minimal write amplification for metadata updates (about 2%) and minimal increase in latency (on average in the sub-microseconds range). Furthermore, it is optimized for sudden power cuts (crash-consistent) by storing all metadata inline of the storage blocks on the underlying device.

Like all block device IO requests, each request contains an LBA (logical block address) and a length (in blocks). The 64-bit LBA is internally organized into a 24-bit VUID (a cluster-wide unique identifier of the logical volume) and a 39-bit virtual LBA. The starting LBA of the page on the physical device is identified by the key (VUID, LPA), where LPA is the logical page address (LBA / (2^m)), and the address offset within the page is determined by (LBA modulo 2^m).

The IO processing services of this virtual device work entirely asynchronously on CPU-pinned threads with entirely private data (no synchronization mechanisms between IO threads required).

They are placed on top of an entirely asynchronous NVMe driver, which receives IO and submits responses via IO-queue pairs sitting at the bottom of the IO processing stack.

Mid-level IO Processing Pipeline

On top of the low-level mapping device, a virtual block device, which implements distributed data placement, has access to the entire cluster topology. This topology is maintained by a centralized multi-tenant control plane, which knows about the state of each node and device in the attached clusters and manages changes to cluster topology (adding or removing devices and nodes).

It uses multiple mechanisms to determine the calculated and factual location of each data page and then issues asynchronous IO to this mapping device in the cluster locally (NVMe) or remotely using NVMe over Fabrics (NVMe-oF):

1. All IO is received and forwarded on private IO threads, with no inter-thread communication, and entirely asynchronously, both inbound and outbound.
2. First, IO is split at page boundaries so that single requests can be processed within a single page.
3. Data is then striped into n chunks, and (double) parity is calculated from the chunks. Double parity is calculated using the RDP algorithm. The data is, therefore, organized in 2-dimensional arrays. n equals 1, 2, 4, or 8. This way, 4 KiB blocks can be mapped into 512-byte device blocks, and expensive partial stripe writes can be avoided.
4. To determine a primary target for each combination of (VUID, page, chunk-index), a flat list of devices is fed into the “list bucket” algorithm (see …) with (VUID, page, chunk-index) being the key.
5. Each of the data and parity chunks in a stripe have to be placed on a different device. In addition, more failure domains, such as nodes and racks, can be considered for placement anti-affinity rules. In case of a collision, the algorithm repeats recursively with an adjusted chunk-index (chunk-index + i x p, where p is the next prime number larger than the maximum chunk index and i is the iteration).
6. In case a selected device is (currently) not available, the algorithm repeats recursively to find an alternative and also stores the temporary placement data for each chunk in the IO. This temporary placement data is now also journaled as metadata. Metadata journaling is an important and complex part of the algorithm. It is described separately below.
7. On read, the process is reversed: the chunks to read from a determined placement location are determined by the same algorithm.
8. In case of single or dual device failure at read, the missing data will be reconstructed on the fly from parity chunks.
9. The algorithm pushes any information on IO errors straight to the control plane, and the control plane may update the cluster map (status of nodes and devices) and push the updated cluster map back to all nodes and virtual devices.

Top-level IO Processing Pipeline

On top of the stack of virtual block devices, simplyblock includes multiple optional virtual block devices, including a snapshot device – the device can take instant snapshots of volumes, supports snapshot chains, and instant (copy-on-write) cloning of volumes. Additionally, there is a virtual block device layer, which supports synchronous and asynchronous replication of block storage volumes across availability zones.

The highest virtual block device in the stack is then published to the fabric as a separate NVMe-oF volume with its own unique NVMe identifier (NQN) via the control plane.

High-Availability Support

The algorithm supports highly available volumes based on NVMe multipathing and ANA (asynchronous namespace access). This means that a transparent fail-over of IO for a single volume in case of a node outage is realized without having to add any additional software to clients.

Due to the features of the high-, mid-, and low-level IO pipeline, this is easy to realize: identical stacks of virtual block devices with an identical VUID are created on multiple nodes and published to the fabric using “asynchronous namespace access” (which prefers one volume over others and essentially implements an active/passive/passive mechanism).

Metadata Journaling

Metadata Journaling persists in non-primary placement locations to locate data in the cluster. It has the following important features:

• It has to persist every change in location for the block range addressed in an IO request to be consistent in “sudden power-off” situations (node outages)
• It has to minimize write amplification – this is achieved by smartly “batching” multiple metadata write requests into single IO operations in the form of “high-priority” IO
• It has to be fast – not delaying data IO – this is achieved by high-priority NVMe queues
• It has to be reliable – this is achieved by replicating metadata writes to three nodes using NVMe over Fabrics remote connections
• Its storage footprint has to be small and remain constant over time; it cannot grow forever with new IO – this is achieved by introducing a regular compression mechanism, which replaces the transactional journal with a “snapshot” of placement metadata at a certain moment

Data Migrations

Data Migrations run as background processes, which take care of the movement of data in cases of failed devices (re-rebuild and re-distribution of data in the cluster), cluster scaling (to reduce the load on utilized elements and rebalance the cluster), and temporary element outage (to migrate data back to its primary locations). Running data migrations keeps a cluster in a state of transition and has to be coordinated to not conflict with any ongoing IO.

Conclusion

Building an architecture for a fast, scalable, fault-tolerant distributed storage solution isn’t easy. To be fair, I don’t think anyone expected that. Distributed systems are always complicated, and a lot of brain power goes into their design.

Simplyblock separates itself by rethinking the data placement in distributed storage environments. Part of it is the fundamentally different way of using erasure coding for parity information. We don’t just use them on a single node, between the local drives, simplyblock uses erasure coding throughout the cluster, distributing parity information from each disk on another disk on another node, hence increasing the fault tolerance.

To test simplyblock, get started right away. If you want to learn more about the features simplyblock offers you, see our feature overview.

The post How We Built Our Distributed Data Placement Algorithm appeared first on simplyblock.

In this installment, we’re talking to Abdel Sghiouar from Google, a company that needs no introduction. Abdel is Developer Advocate for GKE (Google Kubernetes Engine) and talks to us about Kubernetes, serverless platforms, and the future of containerization. See key learnings below on Kubernetes for AI, where to find the best Kubernetes tutorials for beginners and how simplyblock can speed up your AI/ML workload on Kubernetes. Also see interview transcript section at the end.

Key Learnings

Can Kubernetes be used for AI / ML?

Kubernetes can be used for AI/ML. There are several benefits for managing AI/ML workloads including: Scalability: Kubernetes can easily scale up or down to handle the varying computational demands of AI/ML tasks. Resource Management: It efficiently manages resources such as CPU, memory, and GPUs, ensuring optimal utilization. Portability: AI/ML models can be deployed across different environments without modification, thanks to Kubernetes’ container orchestration capabilities. Automation: Kubernetes automates deployment, scaling, and management of containerized applications, which is beneficial for continuous integration and continuous deployment (CI/CD) pipelines in AI/ML projects. Flexibility: It supports various AI/ML frameworks and tools, allowing for a versatile development and deployment ecosystem. Reproducibility: Containers ensure consistent environments for development, testing, and production, enhancing reproducibility of AI/ML experiments.

These features make Kubernetes a powerful platform for deploying, scaling, and managing AI/ML applications. Many AI companies, including those running ChatGPT, utilize Kubernetes because it allows for rapid scaling and performance management of large models and workloads.

Where to Find the best Kubernetes Tutorials for Beginners?

There are several excellent resources where beginners can find comprehensive Kubernetes tutorials, including various learning styles from video tutorials, in-depth articles, interactive labs to official documentation. Kubernetes.io Documentation: The official Kubernetes documentation provides a wealth of information, including beginner-friendly tutorials, concepts, and guides. The “Getting Started” section is particularly useful KubeAcademy by VMware: Offers free, high-quality video courses on Kubernetes basics, cluster operations, and application management. Udemy: Offers a variety of Kubernetes courses, often including hands-on labs and real-world examples. Popular courses include “Kubernetes for the Absolute Beginners” and “Kubernetes Mastery.” Coursera: Partnered with top universities and organizations to offer courses on Kubernetes. The “Architecting with Google Kubernetes Engine” specialization is a notable example. edX: Provides courses from institutions like the Linux Foundation and Red Hat. The “Introduction to Kubernetes” course by the Linux Foundation is a good starting point. YouTube: There are many YouTube channels that offer high-quality tutorials on Kubernetes. Channels like “TechWorld with Nana,” “Kunal Kushwaha,” and “DigitalOcean” provide beginner-friendly content. Play with Kubernetes: An interactive learning environment provided by Docker, offering hands-on tutorials to practice Kubernetes commands and concepts. Medium and Dev.to Articles: Both platforms have numerous articles and tutorials written by the community. Searching for “Kubernetes beginner tutorial” can yield many helpful results.

How can Simplyblock Speed up your AI / ML Workload on Kubernetes?

Simplyblock offers various features and tools that can significantly enhance the performance and efficiency of AI/ML workloads on Kubernetes. High performance, low latency: Simplyblock provides a fully cloud-native storage solution, designed for predictable low latency workloads, such as AI/ML tasks. Scalability: Kubernetes inherently supports scaling, and simplyblock enhances this capability by: Storage Scalability: Simplyblock is designed to seamlessly scale out with the growing amount of data in AI/ML use cases. Automatic Rebalancing: when nodes or disks are added to a simplyblock storage cluster, stored data is automatically rebalanced in the background for highest read and write performance, as well as lowest latency, as close to a local disk as possible. Improved Data Management: AI/ML workloads often involve large datasets. Simplyblock improves data handling by: Data Locality: Ensuring that data is processed close to where it is stored to reduce latency and improve performance. Persistent Storage Solutions: Providing robust and high-performance storage solutions that can handle the I/O demands of AI/ML workloads. Monitoring and Optimization: Effective monitoring and optimization tools provided by simplyblock help in maintaining performance and efficiency: Performance Monitoring: Offering real-time monitoring of storage usage and storage performance to identify and mitigate issues quickly Cost Optimization: Offering an easy way to reduce the cost of cloud storage without sacrificing performance, latency, or capacity, thereby reducing the overall cost of running AI/ML workloads on Kubernetes. Enhanced Security: Simplyblock ensures that AI/ML workloads on Kubernetes are secure by: Secure Data Handling: Implementing encryption and secure data transmission between the AI/ML workload and the storage cluster. Access Controls: Providing granular access controls to manage who can mount, access, and modify AI/ML workloads.

By leveraging the features provided by simplyblock, organizations can significantly speed up their AI/ML workloads on Kubernetes. These enhancements lead to improved performance, better resource utilization, scalability, and overall efficiency, ultimately resulting in faster model development and deployment cycles.

Transcript

Chris Engelbert: Hello everyone, today with me, another very good friend of mine. We have known each other for quite a few years. Abdel, and I don’t try to pronounce your last name. I’m really sorry [laughs]. Maybe you can introduce yourself real quick, like, who are you, what do you do, and where you come from.

Abdel Sghiouar: Sure. Abdel is fine. So yes, Abdel Sghiouar if you want. That’s how typically people pronounce my last name. I’m based out of Stockholm, and I work for Google. I’ve been at the company for 10 years. I do Kubernetes stuff. I do a Kubernetes podcast, and I talk about Kubernetes and containers. I arguably do more talks about why you probably do not need any of these technologies than why you actually need them. Originally from Morocco, that’s the back story. Have you made it to Devoxx Morocco yet?

Chris Engelbert: I was supposed to go this year, but I caught COVID the week before.

Abdel Sghiouar: Oh, yes, I remember.

Chris Engelbert: Yeah, that was really unfortunate. I was so looking forward to that. I was like, no.

Abdel Sghiouar: I keep promoting the conference everywhere I go, and then I forget who made it or who didn’t. Okay, well, maybe 2024 is what happened.

Chris Engelbert: Yes, next time. I promise. Like last time, I promised last time, and I still failed. All right, you said you work for Google. I think everyone knows Google, but your role is slightly different. You’re not working on a specific… Well, you work on a specific product, but as you said, you’re trying to tell people why not to use it.

Abdel Sghiouar: Which I don’t think my manager would be very happy if he knows that that’s what I do.

Chris Engelbert: All right, tell us about it so I can send it to him.

Abdel Sghiouar: So, yes, I work on GKE. GKE is Google Kubernetes Engine, which is the managed version of Kubernetes that we do on Google Cloud. That’s technically the product I work on. I think that we were having this conversation. I avoid being too salesy, so I don’t really talk about GKE itself unless there are specific circumstances. And of course, Kubernetes itself, as a platform, it’s more specific to an audience. You cannot just go to random front-end conferences and start talking about Kubernetes. No one would understand whatever you’re saying. But, yeah, GKE is my main thing. I do a lot of networking and security within GKE. I actually started this year looking a little bit into storage and provisioning because, surprise, surprise, AI is here. And people need it. And when you are doing wild machine learning workloads, really, on anything, you need storage because these models are super huge. And storage performance is important. So, yeah, so that’s what I do.

Chris Engelbert: I think predictability is also important, right? It’s important for the data model. It’s not like you want it sometimes fast and you want it sometimes slow. You want it very fast. You want it to have the same speed consistently.

Abdel Sghiouar: Yeah, that’s true. It’s actually interesting. I mean, besides the product itself, it’s interesting. A lot of these AI companies that came to the market in the last, I’d say, year, year and a half, they basically just use Kubernetes. I mean, ChatGPT is trained on Kubernetes, right? It sounds like, oh, yeah, I just need nodes with GPU, Kubernetes, boom. And to be honest, it totally makes sense, right? Because you need to have a lot of compute power. You have to scale up and down really fast, depending on how many users you currently have, right? So I think anything like that with Kubernetes makes perfect sense.

Chris Engelbert: Exactly. So you mentioned you also have a podcast. Tell me about it. It’s not like I have listeners that you don’t have, but who knows?

Abdel Sghiouar: Yeah, it’s the Kubernetes podcast by Google. It’s a show that’s been running for almost six years. I think we’re close to six years. We’ve been doing it for two years, me and my co-host. So me and Kaslin Fields from Seattle. It’s a twice a month show. We basically invite people from different parts of the cloud-native world, I would say. So we talk to maintainers. We talk to community members. We are looking into bringing some other interesting guests. One of my personal interests is basically the intersection of technology and science. And I don’t know if your audience would know this, but at KubeCon in Barcelona in 2019, one of the keynotes that was actually done was done by people from CERN who came on stage and showed how they could replicate the Higgs boson experiments on Kubernetes. So that was before I was a host. But we are exploring the idea now of finding organizations that their job is not really technology. They’re doing science, but they’re using Kubernetes to enable science. And we’re talking to them. So there’s quite a lot of interesting things happening. That’s how we are going to be releasing very soon. But yeah, so it’s a one-hour conversation. We try to do some news. We try to figure out who bought who, who acquired who, who removed which version, who changed licenses in the last two weeks since the last episode. And then we just interviewed the guests. So yeah, that’s the show essentially.

Chris Engelbert: Right. And correct me if I’m wrong, but I think you’re also writing the “What Happened in Kubernetes” newsletter this week.

Abdel Sghiouar: This week in GKE. Yes. Which is not only about GKE. So I have a newsletter on LinkedIn called “This Week in GKE”, which covers a couple of things that happened in GKE in this week, but also covers other cloud stuff.

Chris Engelbert: All right. Fair enough. So you’re allowed to talk about that.

Abdel Sghiouar: Yeah, yeah. It’s actually interesting. It started as a conversation on Twitter. Somebody put a tweet that said this other cloud provider without mentioning a name introduced a feature into their managed Kubernetes. And I replied saying, we had this on GKE for the last two years. It’s a feature called image streaming. It’s basically a feature to speed up the image pool. When you are trying to start a pod, the image has to be pulled very quickly. And then the person replied to me saying, well, you’re not doing a very good job talking about it. I was like, well, challenge accepted.

Chris Engelbert: Fair enough. So I see you’re not talking about other cloud providers. You’re not naming people or naming things. It’s like the person you’re not talking about. What is it? Not Voldemort. Is it Voldemort? I’m really bad at Harry Potter stuff.

Abdel Sghiouar: I’m really bad, too. Maybe I am as bad as you are. But yes.

Chris Engelbert: All right. Let’s dive a little bit deeper. I mean, it’s a cloud podcast. So when you build an application, how would you start? Where is the best start? Obviously, it’s GKE. But–

Abdel Sghiouar: Well, I mean, actually, arguably, no. So this is a very quick question. I mean, specifically, if you’re using any cloud provider, going straight into Kubernetes is probably going to be frustrating because you will have to learn a bunch of things. And what has been happening through the last couple of years is that a lot of these cloud providers are just offering you a CAS, a container as a service tool, Fargate on AWS. I think it’s called ACS, Azure Container Services on Azure, and Cloud Run on GCP. So you can just basically write code, put it inside the container, and then ship it to us. And then we will give you an URL and a certificate. And we will scale it up and down for you. If you are just writing an app that you need to answer a web request and scale up and down on demand, you don’t need Kubernetes for this. Where things start to be interesting or where people start looking into using Kubernetes, specifically– and of course, we’re talking here about Google Cloud, so GKE, is when you start having or start requiring things that this simple container service platform doesn’t give you. And since we’re in the age of AI, we can talk about GPUs. So if you need a GPU, the promise of a serverless platform is very fast scaling. Very fast scaling and GPUs don’t really go hand in hand. Just bringing up a Linux node and installing Nvidia drivers to have a GPU ready, that takes 15 minutes. I don’t think anybody will be able to sell a serverless platform that scales in 15 minutes. [laughs]

Chris Engelbert: It will be complicated, I guess.

Abdel Sghiouar: It will be complicated, yes. That’s where people go then into Kubernetes, when you need those really specific kinds of configuration, and more fine-tuning knobs that you can turn on and off and experiment to try things out. This is what I like to call the happy path. The happy path is you start with something simple. And as you do in this case, it gets more complicated. You move to something more complex. Of course, that’s not how it always works. And people usually just go head first, dive into GKE.

Chris Engelbert: I’m as big as Google. I need Kubernetes right now.

Abdel Sghiouar: Sure. Knock yourself down, please. Actually, managed Kubernetes makes more money than container service, technically. So whatever.

Chris Engelbert: So the container service, just for people that may not know, I think underneath is basically also something like Kubernetes. It’s just like you’ll never see that. It’s operated by the cloud provider, whatever it is. And you basically just give them an image, run it for me.

Abdel Sghiouar: Exactly. If people want to go dive into how these things are built, there is a project called Knative that was also released by the Kubernetes community a few years ago. And that’s typically what people use to be able to give you this serverless experience in a container format. So it’s Kubernetes with Knative, but everything is managed by the cloud provider. And as you said, we expose just the interface that allows you to say container in, container out.

Chris Engelbert: Fair. So about a decade ago people started really going into first VMs and doing all of that. Then we thought, oh, VMs. So it all started with physical servers are s**t. It’s bad. And it’s so heavy. So let’s use virtual machines. They’re more lightweight. And then we figure out, oh, virtual machines are still very heavy. So let’s do containers. And a couple of years ago, companies started really buying into this whole container thing. I think it basically was when Kubernetes got big. I don’t remember the original Google technology, Kubernetes was basically built after Bork was started.

Abdel Sghiouar: Bork, yes.

Chris Engelbert: Right. So Bork is the piece, it was probably borked. Anyway, we saw this big uptake in migrations to the cloud, and specifically container platforms. Do you think it has slowed down? Do you think it’s still on the rise? Is it pretty much steady? How do you see that right now?

Abdel Sghiouar: It’s a very good question. I think that there are multiple ways you can answer that question. I think that people moving to containers is something that is probably still happening. I think that what’s probably happening is that we don’t talk about it as much. Because– so the way I like to describe this is because Kubernetes as a technology is 10 years old, it’s going to be 10 years old in June, by the way. So June, I think, 5th or something. June 5, 2014 was the first pull request that was pushed, that pushed the first version of Kubernetes. It’s going to be a commodity. It’s becoming a commodity. It’s becoming something that people don’t even have to think about. And even cloud providers are making it such a way that they give you the experience of Kubernetes, which is essentially the API. And the API of Kubernetes itself is pretty cool. It’s a really nice way of expressing intent. As a developer, you just say, this is how my application looks like. Run it for me. I don’t care. And so on the other hand, also this is– also interesting is a lot of programming language frameworks started building into the framework ways of going from code to containers without Docker files. And you are a Java developer, so you know what I’m talking about. Like a Jib, you can just import the Jib plug-in in Maven. And then you just run your Maven, and then blah, you have a container. So you don’t have to think about Docker files. You don’t really have to worry about them too much. You don’t even have to learn them. And so I think that the conversation is more now about cost optimization, optimizing, bin packing, rather than the simple, oh, I want to move from a VM to a container. So the conversation shifts somewhere else because the technology itself is becoming more mainstream, I guess.

Chris Engelbert: So a couple of years ago, people asked me about Docker, just because you mentioned Docker. And they asked me what I think about Docker. And I said, well, it’s the best tool we have right now. I hope it’s not going to stick. That was probably a very mean way of saying, I think the technology, the idea of containerization and the container images is good. But I don’t think Docker is the best way. And you said a lot of tools actually started building their own kinds of interface. They all still use Docker or any of the other imaging tools underneath, but they’re trying to hide all of that from you.

Abdel Sghiouar: Yeah. And you don’t even need Docker in this case, right? I think just very quickly, I’m going to do a shameless plug here. We have an episode on the podcast where we interviewed somebody who is one of the core maintainers of ContainerD. And the episode was not about ContainerD. It was really about the history of containers. And I think it’s very important to go listen to the episode because we talked about the evolution from Docker initially to the Open Container Initiative, the OCI, which is actually a standardization part, to ContainerD, to all the container runtimes that exist on the market today. And I think through that history, you will be able to understand what you’re exactly talking about. We don’t need Docker anymore. You can build containers without even touching Docker. Because I don’t really like Docker personally.

Chris Engelbert: I think we’re not the only ones, to be honest. Anyway, what I wanted to direct my question to is that you also said you have the things like Fargate or all of those serverless technologies that underneath use Kubernetes. But it’s hidden from the user. Is that the same thing?

Abdel Sghiouar: I mean, yes, in the sense that yes, because Kubernetes is a sign, it’s becoming a commodity that people shouldn’t actually– people would probably be shocked to hear me say this. I think Kubernetes should probably have never gotten out like this. I mean, the fact that it became a super popular tool is a good thing, because it attracted a lot of interest and a lot of investments. I do not think it’s something that people should learn. But it’s a platform. It’s something you can build on top of. I mean, you need to run an application, go run it on top of a container as a service. Do not learn Kubernetes. You don’t have to, right? Like, I can see how we put it once in a tweet, which is basically, Kubernetes is a platform to build platforms. That’s what it is.

Chris Engelbert: That makes sense. You don’t even need to understand how it works. And I think that makes perfect sense. From a user’s perspective, the whole idea of Kubernetes was to abstract away whatever you run on. But now, there’s so many APIs in Kubernetes that abstract away Kubernetes that it’s basically possible to do whatever. And I think Microsoft did a really interesting implementation on top of Hyper-V, which uses micro VMs, whatever they call those things.

Abdel Sghiouar: Oh, yes, yes, yes, yes, yes, yes.

Chris Engelbert: The shared kernel kind of stuff, which is kind of the same idea as a container.

Abdel Sghiouar: Yeah, I think it’s based on Kata Containers. I know what you’re talking about.

Chris Engelbert: But it’s interesting, because they still put the Kubernetes APIs on top, or the Kubernetes implementation on top, making it look exactly like a Linux container, which is really cool. Anyway, what do you think is or will be the next big thing for the cloud? What is the trend that is upcoming? You already mentioned AI, so that is out.

Abdel Sghiouar: That’s a very big one, right? It’s actually funny, because I’m in Berlin this week. I am here for a conference, and we were chatting with some of our colleagues. And the joke I was making was next time I go on stage to a big conference, if there are people talking about AI at the same conference, I will go on stage and go, “OK, people talked about AI. Now let’s talk about the things that actually matter. Let’s talk about the thing that people are using and making money from. Let’s stop wishful thinking, right?” I think Kubernetes for AI is big. That’s going to be around. AI is not going to disappear. It’s going to be big. I think we’re in the phase where we’re discovering what people can do with it. So I think it’s a super exciting time to be alive, I think, in my opinion. There’s like a shift in our field that lots of people don’t get to experience. I think the last time such a shift happened in our field was people moving from mainframes to pizza box servers. So we’re living through some interesting times. So anyway, I think that that’s what’s going to happen. So security remains a big problem across the board for everything. Access, security, management, identity, software security. You’re a developer. You know what I’m talking about. People pulling random dependencies from the internet without knowing where they’re coming from. People pulling containers from Docker Hub without knowing who built them or how they were built. Zero ways of establishing trust, like all that stuff. So that’s going to remain a problem, I would say, but it’s going to remain a theme that we’re going to hear about over and over again. And we have to solve this eventually. I think the other thing would be just basically cost saving, because we live in an interesting world where everybody cares about cost saving. So cost optimization, bin packing, making sure you get the most out of your buck that you’re paying to your cloud provider. And I think that the cloud native ecosystem enabled a lot of people to go do some super niche solutions. I think we’re going to get to a stage now where all these companies doing super niche solutions will be filtered out in a way that only those that have really, really interesting things that solve real problems, not made up problems, will remain on the markets.

Chris Engelbert: That makes sense. Only the companies that really have a solution that solves something will stay. And I think that ethics will also play a big part in the whole AI idea. Well, in the next decade of AI, I think we need to be really careful what we do. The technology makes big steps, but we also see all of the downsides already. The main industry that is always on the forefront of every new technology already is there, and they’re misusing it for a lot of really stupid things. But it is what it is. Anyway, because we’re already running out of time. 20 Minutes is so super short. Your favorite stack right now?

Abdel Sghiouar: Yeah, you asked me that question before. Still unchanged. Go, as a backend technology, I don’t do a lot of front-end, so I don’t have a favorite stack in that space. Mac for development, Visual Studio Code, VSS for coding. I started doubling into IntelliJ, but recently I kind of like it, actually. Because I’m not a Java developer, I never had a need for it, but I’m just experimenting. And so Go for the backend. I think it’s just backend. I only do backend, so Go.

Chris Engelbert: That makes sense. Go and I have a love-hate relationship. I wouldn’t say Go is a perfect language, but it’s super efficient for microservices. When you write microservices, all of the integration of Let’s Encrypt or the ACME protocol, all that kind of stuff, it’s literally you just dump it down and it works. And that was a first for me, coming from the Java world. A lot of people claim that Java is very verbose. I don’t think so. I think Java was always meant to be more readable than writeable, which is, from my perspective, a good thing. And I sometimes think Go did some things, at least, wrong. But it’s much more complicated, because Java is coming from a very different direction. If you want to write something really small, and you hinted at the frameworks like Quarkus and stuff, Go all just has that. They built it with the idea that the standard library should be pretty much everything you need for a microservice.

Abdel Sghiouar: Exactly.

Chris Engelbert: All right. We’re recording that the week before KubeCon. KubeCon Europe, will I see you next week?

Abdel Sghiouar: Yes, I’m going to be at KubeCon. I’m going to speak at Cloud Data Rejects on the weekend, and I’ll be at KubeCon the whole week.

Chris Engelbert: All right. So if that comes out next Friday, I think, and you hear that, and you’re still at KubeCon, come see us.

Abdel Sghiouar: Yes. I’m going to be at the booth a lot of times, but you will be able to see me. I am a tall, brown person with curly hair. So I don’t know how many people like me will be there, but I speak very loud. So you’ll be able to both hear me and see me.

Chris Engelbert: That’s fair. All right. Thank you, Abdel. Thank you for being here. I really appreciated the session. It was a good chat. I love that. Thank you very much.

Abdel Sghiouar: Thanks for having me, Chris.

Key Takeaways

In this episode of simplyblock’s Cloud Commute Podcast, host Chris Engelbert welcomes Abdellfetah Sghiouar from Google, who talks about his community work for Kubernetes, his Kubernetes podcast, relevant conferences, as well as his role at Google and insights into future cloud trends with the advent of AI.

Abdel Sghiouar is a Google Kubernetes expert with 10 years of experience at the company. He also hosts a podcast where he talks about Kubernetes and containers.

Abdel focuses on networking, security, and now storage within GKE (Google Kubernetes Engine), especially for AI workloads. He emphasizes the importance of predictable latency and high-performance storage, such as simplyblock, for AI, rendering Kubernetes essential for AI as it provides scalable compute power.

His podcast has been running for almost 6 years, featuring guests from the cloud-native ecosystem, and covers cloud-native technologies and industry news. He also writes a LinkedIn newsletter, ‘This Week in GKE,’ covering GKE updates and other cloud topics.

Abdel highlights the shift from physical servers to VMs, and now to containers and discusses the simplification of container usage and the standardization through projects like Knative.

He is of the opinion that Kubernetes has become a commodity and explains that it should serve as a platform for building platforms. He also highlights the importance of user-friendly interfaces and managed services.

Abdel believes AI will continue to grow, with Kubernetes playing a significant role. Security and cost optimization will be critical focuses. He also emphasizes the need for real solutions to genuine problems, rather than niche solutions.

While asked about his favourite tech stack, Abdel includes Go for backend, Mac for development, and Visual Studio Code for coding. He also likes IntelliJ and is currently experimenting with it. Chris also appreciates Go’s efficiency for microservices despite some criticisms.

Chris and Abdel also touch upon conferences, particularly Devoxx Morocco and KubeCon.

The post Kubernetes for AI, GKE, and serverless with Abdellfetah Sghiouar from Google (interview) appeared first on simplyblock.