Setting up your personal cloud with Kubernetes on the Oracle always free tier

kubernetes cloud metal ci/cd

What are the tools to deploy a reliable private environment in Kubernetes, MetalLB, Nginx Ingress, Prometheus, Grafana, and more on Oracle’s Ampere instance

This guide shows you how to set up a strong personal cloud system on Oracle's Always Free Tier with Kubernetes. This setup uses open-source tools such as MetalLB, Nginx Ingress, Prometheus, Grafana, and others to build a flexible and scalable structure.

kubernetes

Overview

Creating your own cloud setup can make it much easier to get things up and running. When you use Kubernetes to handle apps in containers, you can roll out changes and new features without breaking a sweat. After you've got everything in place, putting a new app online is a breeze - you just need a Docker image, and it'll be ready to go in no time.

Tools Used

  • Kubernetes (K8s): Orchestrates containerized applications.
  • MetalLB: Provides load balancing for bare metal Kubernetes clusters.
  • Nginx Ingress: Manages external access to services within the cluster.
  • Cert-Manager: Automates SSL certificate management with Cloudflare DNS.
  • ArgoCD: Automates continuous deployment of applications.
  • Prometheus: Collects metrics from Kubernetes and applications.
  • Grafana: Visualizes Prometheus metrics with dashboards.
  • Velero: Backs up and restores Kubernetes resources.

Initial Setup

Start by setting up a Kubernetes cluster on Oracle's Ampere instance using kubeadm. Set up Calico to handle networking, which helps with network policies and keeps things secure.

sudo kubeadm init phase certs all --apiserver-advertise-address=0.0.0.0 --apiserver-cert-extra-sans=10.96.0.1,10.0.0.144,129.154.255.75

I've allowed multiple IPs address for which kube API server certificate would be valid for. This will let use same kubeconfig from multiple instances like Master node, my local machine.

MetalLB Configuration

MetalLB assigns the machine's public IP for load balancing across services. Ensure correct layer2 configuration for your setup.

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  namespace: metallb-system
  name: default
spec:
  autoAssign: true
  addresses:
    - 129.154.255.75/32
yaml for IP address pool which are available for metallb to provision any resource.

Nginx Ingress setup

This will deploy a service of LoadBalancer. This is where MetalLB comes into picture.

controller:
  hostNetwork: true
  metrics:
    enabled: true
    service:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "10254"
values.yaml content for Helm ingress-nginx/ingress-nginx repository

Nginx Ingress for Individual service

Install Nginx Ingress to handle requests from wildcard subdomains redirected through Cloudflare for CDN and SSL termination. Each application is deployed on its subdomain, such as https://typeahead-rs.0xamit.com/.

ingress:
  enabled: true
  className: "nginx"
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-issuer
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
  hosts:
    - host: typeahead-rs.0xamit.com
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls:
    - secretName: 0xamit-com-tls
      hosts:
        - typeahead-rs.0xamit.com
Example snippet from Helm values.yaml for typeahead-rs application

Continuous Deployment with ArgoCD

ArgoCD automates application deployments. Simply create a Helm application in your infrastructure repository and add it to ArgoCD. GitHub Actions handles CI, updating Helm repositories with new Docker images. ArgoCD application Overview

Monitoring and Observability

Monitor cluster health with Prometheus and visualize metrics in Grafana dashboards. This ensures proactive management of resources and application performance.

Grafana availability Currently cluster availability is determined by the uptime SLA of oracle instance.

Backups with Velero

Protect your Kubernetes setup with Velero. Regularly back up cluster resources to prevent data loss and streamline disaster recovery.

Future Steps

Stateful Services Deployment

Explore deploying stateful applications across multiple nodes other than Local PV.

High Availability (HA) Considerations

Plan for multi-master Kubernetes clusters to mitigate single-point-of-failure risks and improve overall system reliability. There are multiple cloud vendor that provides bare metal servers at reasonable price.

Conclusion

Setting up your personal cloud on Oracle's Always Free Tier empowers you to deploy and manage applications efficiently. With Kubernetes and robust tooling, you can scale your infrastructure as needed and ensure secure, reliable service delivery.

For further details, explore:

Feel free to construct and create with your own private cloud configuration!