All decks

Going full arctic circle:
Running ALASCA on open cloud infrastructure & standards.

TECH TALK #38 by ALASCA FOCIS

Image from sysadminday.com

Shows them that you truly appreciate their hard work and dedication

Agenda

1. Which dogfood do you need to eat to run ALASCA?

2. Running on open standards - what worked for us, and what did not

  1. Tofu-based IDM
  2. GitLab CI autoscaler
  3. Woodpecker CI autoscaler
  4. RacStack

3. Open build infrastructure for ALASCA

Von Caro Wedekind - http://foxitalic.de/2014/12/31c3/, CC BY 3.0, Link

Wimmelbild of 31st Chaos Communication Congress

ALASCA projects

The tip of the iceberg

yaook Arko IXpect Krake Seconlay Tarook Yake

ALASCA Friends

Underneath the surface

ALASCA Tools and Services

The Abyss

Do you want to have hedgedocs? Sure. — TSC, Projects & FOCIS meeting · That escalated quickly...
  • is Hedgedoc up? -> install status monitor (Uptime Kuma)
  • is Hedgedoc secure? -> install Traefik/Let's Encrypt
  • how to share admin password -> install password manager (VaultWarden)
  • how to login in password manager -> install identity provider (Keycloak)
  • are service up to date -> install renovate
  • what if service breaks -> configure S3 backup & restore
  • document everything -> configure MkDocs at docs.alasca.cloud -> store IaC repo

ALASCA Tools and Services

The Abyss II

CI Infrastructure

ALASCA needs to provide resources for:

US governance

GitHub

EU governance

Codeberg/Forgejo

GitLab Runner Manager

Scale „To infinity and beyond!”

  • similar to GitHub Actions but for Gitlab
  • Why? isolated, throw-away VMs/runners for ALASCA Community (more security)
  • GitLab Runner instance group autoscaler provides fleeting framework
  • fleeting framework: is a library that GitLab Runner uses to provide a plugin-based abstraction for a cloud provider’s instance groups.
  • (Sarcastic) question: Which cloud providers are supported officially?

    AWS, Google Cloud, Azure

    • (Even more sarcastic) question: Which community providers are documented officially?

      Yes, VMware.

GitLab Runner Manager

ALASCA FOCIS setup

  • we found another community project by Sardina Systems, fleeting-plugin-openstack
  •                 flowchart LR
                      gl[GitLab] -- pending jobs --> mgr["GitLab Runner Manager<br />(docker-autoscaler + fleeting)"]
                      mgr -- create / destroy VMs --> os[(SCS OpenStack project)]
                      os -- fresh, single-use VM per job --> job[CI job runs isolated]
                  
  • What does SCS/dd8a offer? Standardized set of OpenStack APIs, flavors and images
    • Image upload via Glance MUST be allowed based on a fair-use policy. (scs-0104)
    • Mandatory flavors (scs-0103)
    • Mandatory images: SCS is missing image for CI jobs (i.e. flatcar)

GitLab Runner Manager

ALASCA FOCIS setup

  • Issues with upstream project
    • project is ini file based and therefore hard to configure dynamically
    • resource referenced by UUID (change after updates)
    • manually provide and maintain build image
    • not properly maintained -> move to ALASCA (supported by Helmholtz and Syseleven? 🤞)
  • How to DoS your cloud:
    • configure runners to use ephemeral storage & start a bunch of runners 🏃‍♂️
    • 🔥 No valid host was found. There are not enough hosts available. 🔥
  • source code of our extension is found at ALASCA GitLab group
  • All you need is: GitLab Runner Token, OpenStack project & application credential, Host with docker -> docker compose up -d

Measure what you use

RacStack - Billing Dashboard for OpenStack

  • Why?
    • ALASCA provides community resources and needs to measure them
    • FOCIS looked for commercial use-cases : equipping existing heterogeneous cloud infrastructures with efficient monitoring
  • developed on top of racletteJS 🧀 by pacifico
  • source code (MIT), issues and documentation can be found at GitLab, discussions in matrix room
  • needs Keystone, Nova, Gnocchi, Cinder and Neutron
  • ideal to measure Woodpecker and GitLab CI runners

RacStack - Measure what you use

Features

  • Measurable OpenStack resources: Instances, Floating IPs, Volumes
  • Installable via docker compose
  • Configurable dashboard with charts and tables
  • Export data to XRechnung and ZUGFeRD
  • Project -> Customer -> System billing/pricing adjustment for OS resources

Configure your dashboard

Set prices for flavors, IPs and volumes

Get Breakdown of resources

View all resources of a project

Filter resources

Export custom formats

RacStack - Measure what you use

Issues

  • we have no admin priviliges in dd8a, so we are limited to single project view per RacStack instance -> application credential always project scoped
  • Metering only draft status within SCS (see scs-04xx) -> each CSP might have a different system

Thank you. Stay in touch.