Autoscaling Generative AI Workloads

at KCD Praha 24

Short lightning talk about KEDA being used as autoscaler for AI/ML workload. Stable diffusion model was used as an example that generates images based on the text input. Demo application was scaling the worker pods based on the length of message queue. I also briefly talks about pitfalls of GPU intensive workloads on K8s.

( recording )

KEDA AI/ML KCD kubernetes 2024

Multi-Cloud Global Content Distribution at Cloud Native Speeds

at OpenSourceSummit EU 24 @ Vienna

If you’ve been globally distributing digital content for a while, you’ll understand that merely having numerous datacenters with advanced caching patterns isn’t sufficient. When your users need to retrieve an object that’s available in different locations worldwide, they should ideally be directed automatically to the location that’s nearest and fastest for the best experience. Cloud service providers typically offer services to handle this for you within their own clouds, but what if you are running a multi-cloud or hybrid environment? K8GB is a cloud-native solution that handles GeoDNS across heterogeneous environments and enables you to reach the same level of multiregion service resilience offered by cloud providers.

( recording , slides )

k8gb oss kubernetes 2024

k8gb meets Cluster API

at FOSDEM 24

In this talk we will be talking about an open-source way to fully automated K8s clusters that can host workloads that can survive any failure, using pure DNS as the underlying tool for switching the communication among available Kubernetes clusters. No single vendor lock-in. Workloads can be deployed in AWS, Azure, GCP, on-prem. The only common denominators are Kubernetes and Cluster-API.

( recording , slides )

k8gb FOSDEM Cluster-API kubernetes 2024

CRUDing Kubernetes Clusters with Cluster API @ KCD Bratislava 23

These days k8s namespaces don't provide enough isolation for our cloud native experiments. It's much easier to give a user the whole cluster to play with. Let them to break it; repeat. However, this assumes the cluster creation and deletion is an easy thing to do. Also there should be a nice API for that, not just some 5 years old web. Have you ever heard about clusterctl? If not, then come to this talk to learn how easy it is to start using it. If yes, then come to this talk to learn how hard it is to use it in production. Cluster API (CAPI) is a unique standardization effort among multiple cloud providers such as GCP, AWS, Azure but can also work with on-prem solutions such as OpenStack, KVM or vSphere. It allows you to dedicate one cluster in your infra as a control plane for creating the workload clusters. If you are into self-replicating robots, you are going to love this API!

Cluster-API KCD kubernetes 2023

A cloud native Kubernetes Global Balancer @ FOSDEM 22

k8gb is DNS based global service load balancer that can interconnect multiple Kubernetes clusters into one resilient system. Join this talk to learn how it can handle a failover scenario when pods in one cluster go down and second cluster in different location saves the situation. k8gb is an open-source Kubernetes operator that is deployed in each participating cluster. It is comprised of CoreDNS, ExternalDNS and the k8gb controller itself. Using ExternalDNS it can create a zone delegation on a common cloud DNS server like Route53 or Infoblox so that the embedded CoreDNS servers work as an authoritative DNS. K8gb controller makes sure these CoreDNS servers are updated accordingly based on the readiness probes of the application. In this sense this solution is unique, because it is using Kubernetes native tools with customisable probes and battle tested DNS protocol instead of HTTP pings or other similar approaches where single point of failure might be a problem. In k8gb architecture all k8s clusters are equal and there is no SPoF except the common edge DNS server.

k8gb FOSDEM kubernetes-operator 2022

log2rbac @ DevConf mini 22

If you have ever developed an operator pattern for Kubernetes, you have probably had to tweak your service account and assign it to a role. Setting up the RBAC correctly is not that hard, but it's not fun and it distracts you from the real problem the operator is about to solve. This often leads to assigning the cluster admin to the operator and neglecting the security altogether. Log2rbac is a tool (yet another operator) that aims to solve this issue. It assists you with setting up your RBAC roles that are tailored for your application's needs. Come to see this talk and learn more.

log2rbac DevConf kubernetes-operator kubernetes 2022

k8gb office hours @ Kubecon NA 21

Project office hours with community showcasing the k8gb in a live demo

k8gb Kubecon office-hours kubernetes-operator 2021

Polyglot Operators, Spark Operator @ DevConf.us 19

In this talk Jiri Kremser and Mike McCune will show a library for implementing the operator pattern for Kubernetes in JVM languages. The library has been used to develop an operator for deploying and managing Apache Spark clusters in Kubernetes. The talk will also feature a live-coding demo in which you will see how easy it is to create a new operator from scratch on your own.

spark DevConf openshift kubernetes-operator 2019

Spark Operator—Deploy, Manage and Monitor Spark Clusters on Kubernetes @ Spark Summit eu 19

Have you ever wondered how to implement your own operator pattern for you service X in Kubernetes? You can learn this in this session and see an example of open-source project that does spawn Apache Spark clusters on Kubernetes and OpenShift following the pattern. You will leave this talk with a better understanding of how spark-on-k8s native scheduling mechanism can be leveraged and how you can wrap your own service into operator pattern not only in Go lang but also in Java. The pod with spark operator and optionally the spark clusters expose the metrics for Prometheus so it makes it easy for monitoring and alerting.

spark SAIS openshift kubernetes-operator 2019

Spark Operator @ OpenShift ML Sig

Really short intro into operators and also comparison of two existing operators for Apache Spark.

spark ML Sig openshift kubernetes-operator 2018