spark

Spark Operator

Operator for managing the Spark clusters on Kubernetes and OpenShift.
spark kubernetes openshift kubernetes-operator controller

Bitcoin Insights

Analysis of Blockchain transaction captured in a project that uses Jupyter notebook with GraphFrames and NetworkX, spark-notebook with GrapX. Notebooks attaches to a Spark cluster deployed in a standalone mode, everything containerized and running in Kubernetes or OpenShift.
Spark Bitcoin data jupyter kubernetes

ansible-openshift-spark-operator

Ansible role for spark-operator publicly available in Ansible Galaxy
ansible spark kubernetes-operator

Polyglot Operators, Spark Operator @ DevConf.us 19

In this talk Jiri Kremser and Mike McCune will show a library for implementing the operator pattern for Kubernetes in JVM languages. The library has been used to develop an operator for deploying and managing Apache Spark clusters in Kubernetes. The talk will also feature a live-coding demo in which you will see how easy it is to create a new operator from scratch on your own.
spark DevConf openshift kubernetes-operator 2019

Spark Operator—Deploy, Manage and Monitor Spark Clusters on Kubernetes @ Spark Summit eu 19

Have you ever wondered how to implement your own operator pattern for you service X in Kubernetes? You can learn this in this session and see an example of open-source project that does spawn Apache Spark clusters on Kubernetes and OpenShift following the pattern. You will leave this talk with a better understanding of how spark-on-k8s native scheduling mechanism can be leveraged and how you can wrap your own service into operator pattern not only in Go lang but also in Java. The pod with spark operator and optionally the spark clusters expose the metrics for Prometheus so it makes it easy for monitoring and alerting.
spark SAIS openshift kubernetes-operator 2019

Spark Operator @ OpenShift ML Sig

Really short intro into operators and also comparison of two existing operators for Apache Spark.
spark ML Sig openshift kubernetes-operator 2018

Analzying Blockchain transactions in Apache Spark @ FOSDEM 18

I will show the Blockchain analysis in Jupyter interactive notebook using the external Spark cluster running in Kubernetes, everything dockerized. The talk will briefly describe how Blockchain transactions work, but most of the time would be the demo. In the demo I will show how we can run various queries on the publicly available Blockchain data, graph algorithms such as PageRank for identifying significant BTC addresses and more. Intended audience: intermediate, analysts, Bitcoin/Altcoin enthusiasts
spark FOSDEM openshift bitcoin 2018

Spark Operator @ CNCF Big Data Sig

Brief talk and demo about Spark Operator.
spark kubernetes CNCF kubernetes-operator 2018

Analyzing Blockchain Transactions in Apache Spark @ Spark Summit 18

In this presentation I showed a simple way of leveraging Spark's GraphX and GraphFrames for analyzing the transaction graph of Bitcoin transactions. Real data was used.
spark SAIS openshift bitcoin 2018

Spark on OpenSlava 2017

I will show how you can leverage the containers and run the Spark cluster in PaaS, namely OpenShift and Kubernetes. For demonstration purposes, I'll be demoing the Blockchain analysis in Jupyter notebook using the Spark cluster running in OpenShift, everything dockerized. I am out of buzzwords.
spark OpenSlava openshift kubernetes-operator 2017