Efficiency in tech operations is embodied in the modern Platform Engineering process, with its aim of streamlining infrastructure operations and providing a product-like experience for developers, QA professionals, and engineers. Core tasks such as launching environments, deploying services, and conducting tests are all seamlessly integrated into this approach.

At the core of this transformation lies a unified CI/CD system, where most interactions are executed through Git commits and job initiations. However, the conventional use of Git and task management via the command line interface lack intuitiveness for end-users.

So, what we are looking to do is create a UI that has a more modern feel to it and improves the user experience. For example, defining variables via drop-down menus, changing settings with sliders, and selecting multiple options with a simple click.

Example Task

Consider the scenario where we need to offer a user interface which will enable the product team to bootstrap an EKS Kubernetes cluster in AWS Cloud with preconfigured addons, domains, network policies, logging, monitoring, and more. The goal is to cover everything as code and allow users to modify only essential settings like cluster size, node scaling, domain, and project name through the UI. It should all be bootstrapped with just one click…!

An example interface we are building

Selecting the UI Framework — Streamlit

While there are plenty of options for building a UI with a framework, such as Backstage, I find them a little daunting. Instead, I’m inclined towards using a solution that’s easily scripted, tested, and launched independently. Among the available options, Streamlit stands out.

Streamlit, an open-source Python library, enables developers to create web applications for data science and machine learning effortlessly. Specifically, Streamlit allows Python developers and data scientists to convert data scripts into shareable web applications without needing any knowledge of web development.

Key Features:

• Python-centric UI Development: Streamlit is designed for Python developers, enabling the creation of full-fledged web applications using just Python, so there’s no need for HTML, CSS, or JavaScript.
• Quick Launch: A simple command (streamlit run your_script.py) gets your Streamlit app running in the browser.
• Hot-reloading: As you modify your script, Streamlit apps automatically update, meaning real-time changes without manual browser refresh.
• Quick Prototyping: Streamlit has intuitive API and interactive widgets that facilitate rapid prototyping and experimentation, enabling swift iterations on app design and functionality.

It appears to be the optimal candidate for creating a DevOps UI.

Selecting Infrastructure Framework — Cluster.dev

DevOps tasks for infrastructure development involve integrating various technologies into high-level abstractions. We work with a multitude of modules and libraries, crafting a cohesive experience for users who prefer abstraction over delving into underlying tech.

As everything has to be codified and to expedite efficiency, we’ll leverage established technologies like Terraform, Helm, Kubernetes, and the versatile tool of shell scripting. In addition, instead of crafting infrastructure code from scratch, we’ll use public open-source patterns for infrastructure (such as Terraform modules) and for addons and software deployment (like Helm charts)
I’m a developer of Cluster.dev, a framework that can seamlessly combine various high-level open-source technologies, like Terraform Modules and Helm Charts, into the StackTemplate.

Deco Agency

Sample StackTemplate with Terraform Modules, Helm Charts and Shell Scripts

The StackTemplate is a static object that can be maintained in a repository with its distinct lifecycle. This enables easy testing, component upgrades, and default value settings. For reference, you can refer to this StackTemplate relevant to our task: StackTemplate Example.

The only thing that we have to do to initialize the infrastructure is specify the variables essential for this StackTemplate, achievable with a straightforward YAML file (the Stack object in Cluster.dev). Its simplicity makes it an ideal base on which to build a user interface.

Sample for Cluster.dev Stack yaml

What’s also great is that by utilizing Streamlit, ChatGPT can effortlessly generate UI components using Python code, such as switches, sliders, multi-selects, etc. I just created an example YAML and it provides me with UI code that’s ready to go!

Selecting Execution Environment — GitHub Actions

With the UI and Infrastructure frameworks in place, an execution environment is needed for configurations and executing the resulting code. Following GitOps best practices, reliance on the CI/CD system is crucial. I’ve chosen GitHub Actions as its Workflow syntax is widely used and could be both easily generated and distributed.

User Scenario for Cluster Configuration and Deployment

Before initiating the project, creating a walkthrough of the process and necessary steps is essential. Here’s the list that I had:

• UI Selection: Users customize their cluster through the UI, selecting crucial parameters like node size, region, domain, addons, and other specifics.
• Repository & Credentials: Users specify their repository, provide an access token, and furnish AWS account credentials.
• Secure AWS Credentials: The system ensures user data security by storing AWS credentials in GitHub’s secret storage.
• Dynamic YAML Generation: Based on user input, the system dynamically generates necessary YAML files. In combination with GitHub workflow files, these are then pushed to the specified repository.
• Initiate Pull Request: An automatic Pull Request (PR) creation triggers a workflow initiating the Plan step on GitHub. This step involves creating an S3 bucket for Terraform (TF) and Cluster.dev states, testing, and providing a preview of the planned cluster and associated resources set to be created.
• Review & Merge: Users review the PR details. Once satisfied, merging triggers GitHub to execute cdev apply, activating Terraform, Helm, and other units, thereby bootstrapping the cluster.

Summary

Part one in this series has covered the task, technologies involved, and steps required to implement the necessary UI and functionality. Subsequent chapters will delve deeper into the technical implementation. Meanwhile, check out the ready-to-use EKS Cluster creation demo. For those interested in the UI code, it’s available in the template repository.