Merge branch 'main' into pgi-add-workflows

Reviewed-on: #5

NETWORK.md

@@ -0,0 +1,65 @@
+# Infrastructure Overview – Wheatley
+
+This document describes the network and infrastructure architecture of the *Wheatley* environment. Its purpose is to clearly document **connectivity, routing, and responsibilities per site**, enabling predictable operations, troubleshooting, and future expansion.
+
+---
+
+## Overview
+
+The infrastructure consists of three primary sites:
+
+| Site | Role | Subnet |
+|------|------|--------|
+| **Hobbyrack** | Central hub, homelab, workloads | `10.13.37.0/24` |
+| **MAPPS** | Externally managed platform services | `10.7.65.192/26` |
+| **Home** | Client network | `192.168.1.0/24` |
+
+Connectivity between sites is provided via **IPSec** and **WireGuard**, with *Hobbyrack* acting as the central routing and transit hub.
+
+---
+
+## Network Diagram
+
+```mermaid
+flowchart LR
+    %% Hobbyrack
+    subgraph HR["Hobbyrack"]
+        HR_OPN["OPNsense Gateway
+10.13.37.1"]
+        HR_LAN["10.13.37.0/24"]
+        HR_CLUSTER["k8s-wheatley
+(Servarr stack)"]
+        HR_OPN --- HR_LAN
+        HR_LAN --- HR_CLUSTER
+    end
+
+    %% Managed Apps
+    subgraph MA["Forti – Managed"]
+        MA_GW["Gateway
+10.7.65.193"]
+        MA_LAN["10.7.65.192/26"]
+        MA_CLUSTER["MAPPS Cluster
+(Mimir, Loki, Vault)"]
+        MA_CLUSTER2["k8s-peterg
+(ArgoCD)"]
+        MA_GW --- MA_LAN
+        MA_LAN --- MA_CLUSTER
+        MA_LAN --- MA_CLUSTER2
+    end
+
+    %% Home
+    subgraph TH["Home"]
+        TH_GW["UniFi Gateway
+192.168.1.1"]
+        TH_LAN["192.168.1.0/24"]
+        TH_CLIENTS["Clients"]
+        TH_GW --- TH_LAN
+        TH_LAN --- TH_CLIENTS
+    end
+
+    %% Tunnels
+    HR_OPN <==>|"IPSec
+10.13.37.0/24 ↔ 10.7.65.192/26"| MA_GW
+    HR_OPN <==>|"WireGuard Hub
+10.167.84.0/24"| TH_GW
+

README.md

@@ -1,2 +1,47 @@
-# infrastructure
+# Hobbyrack Infrastructure
 
+This repository contains the infrastructure-as-code configuration for managing Wheatley, a homelab environment within Hobbyrack. The current focus is on deploying a Kubernetes clusters with Talos and Cilium on Proxmox VE using OpenTofu.
+
+## Contents
+
+- **modules/**: Contains reusable modules for cluster bootstrap and node provisioning.
+  - **talos-bootstrap/**: Bootstraps the Talos cluster, including templates for machine configuration and Cilium installation.
+  - **talos-node/**: Provisions individual Talos node VMs on Proxmox.
+- **pve01.wheatley.in/**: Configuration specific to the Proxmox VE host `pve01.wheatley.in`.
+  - **k8s-wheatley/**: Provisions the `k8s-wheatley` Kubernetes cluster.
+  - **templates/**: Provisions VM templates on the PVE host.
+
+## Requirements
+
+- [OpenTofu](https://opentofu.org/) >= 1.0
+- Access to a Proxmox VE environment
+- (Optional) talosctl for managing Talos nodes
+
+## Usage
+
+1. **Initialize OpenTofu**
+   ```sh
+   tofu init
+   ```
+2. **Review the execution plan**
+   ```sh
+   tofu plan
+   ```
+3. **Apply the infrastructure**
+   ```sh
+   tofu apply
+   ```
+
+> Adjust variables in `variables.tf` or provide a `terraform.tfvars` file for your environment.
+
+## Contributing
+
+Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
+
+## License
+
+This repository is licensed under the MIT License. See the `LICENSE` file for details.
+
+## Contact
+
+For questions or support, open an issue or contact via GitHub.

modules/talos-bootstrap/README.md

@@ -0,0 +1,44 @@
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_talos"></a> [talos](#requirement\_talos) | 0.9.0 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_talos"></a> [talos](#provider\_talos) | 0.9.0 |
+
+## Modules
+
+No modules.
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [talos_cluster_kubeconfig.kubeconfig](https://registry.terraform.io/providers/siderolabs/talos/0.9.0/docs/resources/cluster_kubeconfig) | resource |
+| [talos_machine_bootstrap.talos_machine_bootstrap](https://registry.terraform.io/providers/siderolabs/talos/0.9.0/docs/resources/machine_bootstrap) | resource |
+| [talos_machine_configuration_apply.config_apply_cp](https://registry.terraform.io/providers/siderolabs/talos/0.9.0/docs/resources/machine_configuration_apply) | resource |
+| [talos_machine_configuration_apply.config_apply_worker](https://registry.terraform.io/providers/siderolabs/talos/0.9.0/docs/resources/machine_configuration_apply) | resource |
+| [talos_machine_secrets.machine_secrets](https://registry.terraform.io/providers/siderolabs/talos/0.9.0/docs/resources/machine_secrets) | resource |
+| [talos_client_configuration.talosconfig](https://registry.terraform.io/providers/siderolabs/talos/0.9.0/docs/data-sources/client_configuration) | data source |
+| [talos_machine_configuration.machineconfig-cp](https://registry.terraform.io/providers/siderolabs/talos/0.9.0/docs/data-sources/machine_configuration) | data source |
+| [talos_machine_configuration.machineconfig-worker](https://registry.terraform.io/providers/siderolabs/talos/0.9.0/docs/data-sources/machine_configuration) | data source |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_node_config"></a> [node\_config](#input\_node\_config) | Talos node configuration | <pre>object({<br/>    ipv4_gateway           = string<br/>    talos_version          = string<br/>    gateway_api_version    = string<br/>    cilium_version         = string<br/>    cluster_name           = string<br/>    kubernetes_version     = string<br/>    cluster_endpoint       = string<br/>    proxmox_node           = string<br/>    controlplane_addresses = list(string)<br/>    worker_addresses       = list(string)<br/>  })</pre> | n/a | yes |
+| <a name="input_talos_nodes"></a> [talos\_nodes](#input\_talos\_nodes) | List of Talos nodes to bootstrap | <pre>list(object({<br/>    name            = string<br/>    type            = string<br/>    ipv4_address    = string<br/>    storage_address = optional(string)<br/>    pvc_disks       = optional(list(number))<br/>    gpu_enabled     = optional(bool, false)<br/>  }))</pre> | n/a | yes |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_kubeconfig"></a> [kubeconfig](#output\_kubeconfig) | n/a |
+| <a name="output_talosconfig"></a> [talosconfig](#output\_talosconfig) | n/a |
+<!-- END_TF_DOCS -->

modules/talos-bootstrap/main.tf

@@ -2,7 +2,7 @@ terraform {
   required_providers {
     talos = {
       source  = "siderolabs/talos"
-      version = "0.9.0"
+      version = "0.10.1"
     }
   }
 }
@@ -54,7 +54,6 @@ resource "talos_machine_configuration_apply" "config_apply_cp" {
       vip_address         = var.node_config.cluster_endpoint
       node_name           = format("%s.wheatley.in", var.node_config.proxmox_node)
       machine_type        = each.value.type
-      talos_version       = var.node_config.talos_version
       gateway-api_version = var.node_config.gateway_api_version
       cilium_version      = var.node_config.cilium_version
       cilium_install_file = file("${path.module}/templates/cilium-install.yaml.tmpl")
@@ -76,13 +75,15 @@ resource "talos_machine_configuration_apply" "config_apply_worker" {
     templatefile("${path.module}/templates/machineconfig-worker.yaml.tmpl", {
       hostname            = each.value.name
       cluster_name        = var.node_config.cluster_name
-      vip_address         = var.node_config.cluster_endpoint
       node_name           = format("%s.wheatley.in", var.node_config.proxmox_node)
+      storage_address     = each.value.storage_address
       machine_type        = each.value.type
       talos_version       = var.node_config.talos_version
       gateway-api_version = var.node_config.gateway_api_version
       cilium_version      = var.node_config.cilium_version
       cilium_install_file = file("${path.module}/templates/cilium-install.yaml.tmpl")
+      pvc_disks           = each.value.pvc_disks
+      gpu_enabled         = each.value.gpu_enabled
     })
   ]
 }

modules/talos-bootstrap/templates/machineconfig-cp.yaml.tmpl

@@ -3,7 +3,6 @@ machine:
   type: ${machine_type}
   install:
     disk: /dev/vda
-    image: factory.talos.dev/nocloud-installer-secureboot/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:v${talos_version}
   network:
     hostname: ${hostname}
     nameservers:
@@ -16,7 +15,6 @@ machine:
   kubelet:
     extraArgs:
       pod-max-pids: 1000
-      rotate-server-certificates: true
     extraConfig:
       imageGCHighThresholdPercent: 75
       imageGCLowThresholdPercent: 70
@@ -38,7 +36,6 @@ cluster:
     disabled: true
   extraManifests:
   - https://github.com/kubernetes-sigs/gateway-api/releases/download/v${gateway-api_version}/standard-install.yaml
-  - https://raw.githubusercontent.com/alex1989hu/kubelet-serving-cert-approver/refs/heads/main/deploy/standalone-install.yaml
   inlineManifests:
   - name: cilium-bootstrap
     contents: |

modules/talos-bootstrap/templates/machineconfig-worker.yaml.tmpl

@@ -3,18 +3,40 @@ machine:
   type: ${machine_type}
   install:
     disk: /dev/vda
-    image: factory.talos.dev/nocloud-installer-secureboot/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:v${talos_version}
+%{ if gpu_enabled != false ~}
+    image: factory.talos.dev/nocloud-installer-secureboot/29d29d87bf3b88fe13caf73c20c7a8b6a4355d8177d0d690d5c9f85d4ddb67b7:v${talos_version}
+%{ endif ~}
   network:
     hostname: ${hostname}
     nameservers:
       - 9.9.9.9
+%{ if storage_address != false ~}
+    interfaces:
+      - interface: eth1
+        mtu: 9000
+        dhcp: false
+        addresses:
+          - ${storage_address}/24
+%{ endif ~}
   kubelet:
     extraArgs:
       pod-max-pids: 1000
-      rotate-server-certificates: true
     extraConfig:
       imageGCHighThresholdPercent: 75
       imageGCLowThresholdPercent: 70
+    nodeIP:
+      validSubnets:
+        - 10.13.37.0/24
+  kernel:
+    modules:
+      - name: drbd
+        parameters:
+          - usermode_helper=disabled
+      - name: drbd_transport_tcp
+      - name: dm_thin_pool
+%{ if gpu_enabled != false ~}
+      - name: i915
+%{ endif ~}
   nodeLabels:
     topology.kubernetes.io/region: ${cluster_name}
     topology.kubernetes.io/zone: ${node_name}
@@ -33,7 +55,6 @@ cluster:
     disabled: true
   extraManifests:
   - https://github.com/kubernetes-sigs/gateway-api/releases/download/v${gateway-api_version}/standard-install.yaml
-  - https://raw.githubusercontent.com/alex1989hu/kubelet-serving-cert-approver/refs/heads/main/deploy/standalone-install.yaml
   inlineManifests:
   - name: cilium-bootstrap
     contents: |

modules/talos-bootstrap/variables.tf

@@ -20,5 +20,8 @@ variable "talos_nodes" {
     name            = string
     type            = string
     ipv4_address    = string
+    storage_address = optional(string)
+    pvc_disks       = optional(list(number))
+    gpu_enabled     = optional(bool, false)
   }))
 }

modules/talos-node/README.md

@@ -0,0 +1,34 @@
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_proxmox"></a> [proxmox](#requirement\_proxmox) | 0.86.0 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_proxmox"></a> [proxmox](#provider\_proxmox) | 0.86.0 |
+
+## Modules
+
+No modules.
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [proxmox_virtual_environment_vm.talos-node](https://registry.terraform.io/providers/bpg/proxmox/0.86.0/docs/resources/virtual_environment_vm) | resource |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_node"></a> [node](#input\_node) | Basic configuration for the Talos node | <pre>object({<br/>    id                 = number<br/>    type               = string<br/>    name               = string<br/>    ipv4_address       = string<br/>    ipv4_gateway       = string<br/>    cpu                = number<br/>    gpu                = optional(bool, false)<br/>    memory             = number<br/>    disk               = string<br/>    storagepool        = string<br/>    talos_version      = string<br/>    cluster_name       = string<br/>    kubernetes_version = string<br/>    cluster_endpoint   = string<br/>    proxmox_node       = string<br/>  })</pre> | n/a | yes |
+| <a name="input_pvc_disks"></a> [pvc\_disks](#input\_pvc\_disks) | List of extra disks to attach to the node | `list(number)` | `[]` | no |
+
+## Outputs
+
+No outputs.
+<!-- END_TF_DOCS -->

modules/talos-node/main.tf

@@ -2,7 +2,7 @@ terraform {
   required_providers {
     proxmox = {
       source  = "bpg/proxmox"
-      version = "0.86.0"
+      version = "0.95.0"
     }
   }
 }
@@ -12,6 +12,7 @@ resource "proxmox_virtual_environment_vm" "talos-node" {
   name      = var.node.name
   node_name = var.node.proxmox_node
   tags      = ["tofu"]
+  bios      = "ovmf"
 
   clone {
     vm_id   = 10000 + tonumber(replace(var.node.talos_version, ".", ""))
@@ -21,7 +22,7 @@ resource "proxmox_virtual_environment_vm" "talos-node" {
   cpu {
     cores   = var.node.cpu
     sockets = 1
-    type    = "x86-64-v2-AES"
+    type    = "host"
   }
 
   memory {
@@ -45,6 +46,31 @@ resource "proxmox_virtual_environment_vm" "talos-node" {
     }
   }
 
+  network_device {
+    bridge = "vmbr1"
+    model  = "virtio"
+    mtu    = 1500
+  }
+
+  dynamic "network_device" {
+    for_each = var.node.type == "worker" ? [1] : []
+    content {
+      bridge = "vmbr2"
+      model  = "virtio"
+      mtu    = 9000
+    }
+  }
+
+  dynamic "hostpci" {
+    for_each = var.node.gpu == true ? [1] : []
+    content {
+      device  = "hostpci0"
+      mapping = "A380_GPU"
+      pcie    = true
+      rombar  = true
+    }
+  }
+
   initialization {
     datastore_id = var.node.storagepool
     ip_config {
@@ -57,4 +83,10 @@ resource "proxmox_virtual_environment_vm" "talos-node" {
       servers = ["9.9.9.9"]
     }
   }
+
+  lifecycle {
+    ignore_changes = [
+      clone,
+    ]
+  }
 }

modules/talos-node/variables.tf

@@ -2,10 +2,12 @@ variable "node" {
   description = "Basic configuration for the Talos node"
   type = object({
     id                 = number
+    type               = string
     name               = string
     ipv4_address       = string
     ipv4_gateway       = string
     cpu                = number
+    gpu                = optional(bool, false)
     memory             = number
     disk               = string
     storagepool        = string

pve01.wheatley.in/k8s-wheatley/.terraform.lock.hcl

@@ -2,47 +2,73 @@
 # Manual edits may be lost in future updates.
 
 provider "registry.opentofu.org/bpg/proxmox" {
-  version     = "0.86.0"
+  version     = "0.95.0"
-  constraints = "0.86.0"
+  constraints = "0.95.0"
   hashes = [
-    "h1:brQxwkXGxNw5N2gNwwefF9XxzrudYgSyjHrRhdvAdjo=",
+    "h1:8gOZBH6hYhvvr9ycmz8AVdgnABQPEft6q/qj2uIdDR8=",
-    "zh:09b627b92a59848769fadfc3d8103eebf070a3800144bf03cb93f44472327f44",
+    "h1:APHNeh4Fayb+RnVzIUBGwD4pQZHEH6NBukvZKW9TETw=",
-    "zh:0e19eb7f1047d541e50b97d7ac440ea73685d0c28ed2dbe64217cbe2f0b353e0",
+    "h1:C/SnO5houq7BCNvpmH+QV+RbxcIyASlQm8dI51vxm34=",
-    "zh:20f1e70091ff3056876618c93afd79527c8995f955d153993e8fbb10fa42593b",
+    "h1:HDn4A1XRSbaKd4RIzHJM/U1TBLw1llq5sAjB1oPS2yM=",
-    "zh:3920315be565976f5a9da0803f8f1a108221282f1bc9e21160669d793af4e0c8",
+    "h1:NamnpRo4NLLqhjWFw0lPr4V0Vm/hHqNOXfCbiPsv9WY=",
-    "zh:5133b2a2027428d3926eaa3bcdc0ab65a75305d54f6cbc7c54cce746dfddbc8e",
+    "h1:P/DYln7s7I2YWrowkkvrAdNXCYCOcccR38mcZLzidbs=",
-    "zh:514c588b04738d55c9e6b1c5a4e3fb1ef4041dfb809d2268f14d29839ecfba59",
+    "h1:QmOVJQiSH80Jdmc8T4CLKhkkHHtCohYpgNXTGyfYNX0=",
-    "zh:55916034025b4833bd6a93bb5948dfb7d00830a772ef74fa70898c6f7de0da0b",
+    "h1:XqbY+SVrGaRp9T+nkw929kG3/4fp8SbFy7LUA2io8vU=",
-    "zh:58b485a4b0bde56ca7032fca0ac09cb4c6ff2579e06cf4f2a311bb695baa0df1",
+    "h1:aWrhONde+FDq3mgc3A2VJSvwt8R/gR+enf+GTww9TIU=",
-    "zh:75ebe44e6da4108af5fe02a9cd99ed0189985b486a2a56594952098d161ceb3d",
+    "h1:nJblnO+z23prp5I9vqlR0C984dJ53Vpzj+ad3iPdig4=",
-    "zh:a8c870bfb5958a3d49d639db3c2761cfb453c6a6f95e5e241890922b11c8a4d8",
+    "h1:rnReS6P0e2uXYVMhpPgHb1B+eQrjXCnP8Hp6THLPqPg=",
-    "zh:c2df2748b9be47a6c3e613667c64874d5cb1d3fbb5b985d6eb9c3af5af298454",
+    "h1:s5DW5MUXT5Yspd7Z4qTrAnnSismfI7gqL4+aiMUuDY8=",
-    "zh:c3059668f4f81e450e555a47310e7042044b335f131643262fd51f9ba96f2214",
+    "h1:zEUJdG/8tJnHkjq1DZBqpkGuabZ6Ev1ExY4Y/ITHTVA=",
-    "zh:ddbbb23910666f70cf4a9587ba57b45f5f58c53a1f8d7cee1d6f90a3d3ef38ef",
+    "h1:zxHxjwctc98DHZjOY4YOPkxYBk7cY28r0j9/6sI5o7I=",
-    "zh:e430138b897edcd3b64e4309db34ac872526187782626aa074d8d1647a0abfa8",
+    "zh:07439b6c10c48c25357cd574d9db024adb21419865c5b22019a2d0493314b2fb",
+    "zh:29b56acca3df3d1ad1c0d677165192d59ec850c26d013e19a18cc94b11789b39",
+    "zh:35f73b7eeef23867633ee2d687c1cf015eb15ff128fc9527f018aaf0c58ef1ce",
+    "zh:4a479884f6549e8f5b11659c4fe9e3af4fc3f95440d7d8b54335d608361c731f",
+    "zh:4c89f62fb4528f15ae71fc91f792fe5e747ce162e7bf6124f21989d8805a3971",
+    "zh:510ccaaddb1add81563417dcb8f25f364988e8797293d2889b0bdbf623abe01b",
+    "zh:5d791e444e671f6b41c371fddec9d3aba8c8c5c8996ec0864b5aa8678b70feee",
+    "zh:5e655ca1b5d10cfebbe649952d073699005015082de897f3424faa80a9c13667",
+    "zh:71d639eaa7aaa7a6fa68a8bd546d864842c163ff4f2abc48176e287642156d27",
+    "zh:beb1f2ea7ca4e1e42373749b3b1a3dc7bb2bf283001eb9181e162da4f1345b07",
+    "zh:cd8116cfb4988c7157ec2ef18a3fb2bfa1020342fd66f99aa66ef8db678fe09d",
+    "zh:d0e3eacce028554d641ea7bfb857888559f9f92f3dcc00bdd5d8a3bc70768580",
+    "zh:e0eabff8250b428de253f064324fc878a27fa258a79aa76a1b0dab158986cec6",
+    "zh:eef582a3fcc21a7c7c29719ec1f969089f9bba8376a6bb0c8f5aa6c8f89a8ee0",
     "zh:f26e0763dbe6a6b2195c94b44696f2110f7f55433dc142839be16b9697fa5597",
   ]
 }
 
 provider "registry.opentofu.org/siderolabs/talos" {
-  version     = "0.9.0"
+  version     = "0.10.1"
-  constraints = "0.9.0"
+  constraints = "0.10.1"
   hashes = [
-    "h1:50TaymX/KnjXcURmpKsVd7JUyLRrMDNXMHFgqzSe88U=",
+    "h1:1/HTp6cDJWQJzRj8preKQvw3x/qffivOJhZx27OmAig=",
+    "h1:5NH5P1sqJqFAhAQxfZ0CC1T5Aj6jZLe53lWgJHPhrk0=",
+    "h1:97x1aN9Ydi+yO9IOGFed+mbjh8myBMPKQY36vLS9Sk0=",
+    "h1:PVonzANqcaopoXwRFHWZL6Q4iPs9YFhaJ+Y2c9iYRTs=",
+    "h1:WaWU/5BK+X9s/Vt1f6Iz/6iyWxC1KBN+hMN0VxisILg=",
+    "h1:ZmyK5/k34tApxFFbqKe5W/Q9RZEMgUdaVqcF92AeUow=",
+    "h1:aZRJCGsYiSQK2GCwofbv6iFWLvVj+wwZrwyh8chYjVs=",
+    "h1:fUnidKAmrtS9rdzrH5QyPUH8LSHQCqU05UTSLPRDJos=",
+    "h1:fc7ekyeFDNNvScqgHgowGjM9jnKFyUOMGfnEKJwuf1c=",
+    "h1:gWMvxlc8zIyeA0snj+6rarGUuBGtpb5D/mRttkbQ2hQ=",
+    "h1:hcyPHdphfvQ9lH7yWwy8XTjfE2h2WJNydMMXbli9jGo=",
+    "h1:kNrX3umPDAktIxQWUtx5lN6TGcDTqZxR79mTRqzU7Hg=",
+    "h1:rXc4IengYQexTK0URU4MRGtrkO0iS3YFTUeozTigBHo=",
+    "h1:y+7wlj9usfM3aoSkU+J1VRatR4k1NGKMOW7oA5c8H/Y=",
     "zh:0fa82a384b25a58b65523e0ea4768fa1212b1f5cfc0c9379d31162454fedcc9d",
-    "zh:249ab08b810bb96ef7c35354a12622907df7cbda59c83ff314ee85e8b851128d",
+    "zh:349463cdd4cdb36e03276fdb855e687242237c7cf0bd5871aea995a83838c52e",
-    "zh:4f284c21f0cb4b7012fdb231ebba4803e5c389f7ea380fb5958ebfa6b822d055",
+    "zh:3885026ef7c1c7012d312fc37a35af70821650b10cef03b8ffd08d22145c117d",
-    "zh:6049f8d06bc92ab1a46d56c2b37799e6a1b7ce3c61905acf8f0c2009942be8ac",
+    "zh:3a5c4ddae27200663d3552daa8db6d6ad91f49746825e0556370f67e2d3ead5a",
-    "zh:7de5f0a6429c6856ade3c7e59d5041a7fe8e0010aea14a2074e91f8a526283ce",
+    "zh:456e233aec7d83558255da87ba0b1c7fec81bf71b1aa4e0f35a2787d3f104c2a",
-    "zh:9760420cf5feec0550556664962bc6f1af45115b9ea5bcbf47aa9bffd3dd4ac2",
+    "zh:6b781a16764c91259a6646f346cf47e99a31d7eefcdd188b1ea8b287b8fd0531",
-    "zh:9860272a9fbcea65393c478dc579ad2677a44338de24677702ce699a027801aa",
+    "zh:6d91f628812ec99a71964f00e83df93f6106b830af111b611b6a0f68ad6987ea",
-    "zh:a60311a8bf267b18dcf1c8250f45e371b51c6aef086e037a3f9496602f190187",
+    "zh:80430dea846ef82a4fb430abe4f0213ae79f9f38a7a61752bbeeec59c82c1387",
-    "zh:b74c119b2a964cae8d17b5b5f14c8eecf44708e3817c1f65ac89b813856aed98",
+    "zh:84717212f789a026492b0a4d121fcdf4a15f48916f9500cdb4ffa1f6c26479cc",
-    "zh:bc33e14650052c246ac2e6492a0a24755d4e8d27cb7259e3996f07b0537f2c77",
+    "zh:a0fcaa3fd515d78cf635884165e111fa37b6300cefb0a6412fb1ed584d88599a",
-    "zh:c4bc2c65b3dcef2508c14dbbfc18937c5ede11670c02a644cbc28c83a57de92b",
+    "zh:b0f8dbb8a667e00340ca9566d1bee933097b3b252a1df3737d2fb376964ab8cc",
-    "zh:c98bf4809220568d76ea2db232a88a60c29fd90d4c9e2f8c8a68cf91ceed066a",
+    "zh:b8191729e70d5e7e53730f7c499ab16faec23d45969424c1f304bab0f6fcfcb6",
-    "zh:de0179ee9c55bf36d3474049ceac3a105ed7bc1fb9c4787d4076c4b15d268b66",
+    "zh:be3b5cfc57abdbab689088b2677516703600a12eaefc02a2355ba9002f80f273",
-    "zh:e99e019d950131c2c4259c1cbca1d072cb813bdfdbbbf9359f5901d6b7542a0d",
+    "zh:d53afbcbeaeeed6c7e65477da4daca2cc4ab0c52ad833219c1141f9310325dc2",
-    "zh:feb941cc673692c6858f96590f3ca082683a62aa2d2842b9bfffcd327eb840d3",
+    "zh:f04f8a103f5136bdd6910d531192ad0cb10c8734b519b9e9dff4185d5cc7113e",
   ]
 }

pve01.wheatley.in/k8s-wheatley/README.md

@@ -0,0 +1,38 @@
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_proxmox"></a> [proxmox](#requirement\_proxmox) | 0.86.0 |
+| <a name="requirement_talos"></a> [talos](#requirement\_talos) | 0.9.0 |
+
+## Providers
+
+No providers.
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_controlplanes"></a> [controlplanes](#module\_controlplanes) | ../../modules/talos-node | n/a |
+| <a name="module_talos-bootstrap"></a> [talos-bootstrap](#module\_talos-bootstrap) | ../../modules/talos-bootstrap | n/a |
+| <a name="module_workers"></a> [workers](#module\_workers) | ../../modules/talos-node | n/a |
+
+## Resources
+
+No resources.
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_proxmox_api_token"></a> [proxmox\_api\_token](#input\_proxmox\_api\_token) | Proxmox API token | `string` | n/a | yes |
+| <a name="input_proxmox_endpoint"></a> [proxmox\_endpoint](#input\_proxmox\_endpoint) | Proxmox API endpoint | `string` | n/a | yes |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_kubeconfig"></a> [kubeconfig](#output\_kubeconfig) | n/a |
+| <a name="output_talosconfig"></a> [talosconfig](#output\_talosconfig) | n/a |
+<!-- END_TF_DOCS -->

pve01.wheatley.in/k8s-wheatley/main.tf

@@ -6,7 +6,7 @@ terraform {
     skip_credentials_validation = true
     skip_metadata_api_check     = true
     skip_region_validation      = true
-    force_path_style            = true
+    use_path_style              = true
   }
 }
 
@@ -64,20 +64,26 @@ locals {
     cluster_name       = local.cluster_name
     kubernetes_version = local.kubernetes_version
     ipv4_gateway       = local.ipv4_gateway
-    pvc_disks          = [100]
+    pvc_disks          = [150]
 
     nodes = [
       {
         name            = "worker01"
         ip_address      = "10.13.37.21"
+        storage_address = "10.0.69.21"
+        gpu_enabled     = false
       },
       {
         name            = "worker02"
         ip_address      = "10.13.37.22"
+        storage_address = "10.0.69.22"
+        gpu_enabled     = false
       },
       {
         name            = "worker03"
         ip_address      = "10.13.37.23"
+        storage_address = "10.0.69.23"
+        gpu_enabled     = false
       },
     ]
   }
@@ -88,11 +94,12 @@ locals {
 }
 
 module "controlplanes" {
-  source   = "./modules/talos-node"
+  source   = "../../modules/talos-node"
   for_each = { for node in local.controlplanes.nodes : node.name => node }
 
   node = {
     id                 = local.vm-id_prefix + 10 + tonumber(replace(each.key, "cp", ""))
+    type               = "controlplane"
     name               = format("k8s-wheatley-%s", each.value.name)
     ipv4_address       = each.value.ip_address
     cpu                = local.controlplanes.cpu
@@ -109,14 +116,16 @@ module "controlplanes" {
 }
 
 module "workers" {
-  source   = "./modules/talos-node"
+  source   = "../../modules/talos-node"
   for_each = { for node in local.workers.nodes : node.name => node }
 
   node = {
     id                 = local.vm-id_prefix + 20 + tonumber(replace(each.key, "worker", ""))
+    type               = "worker"
     name               = format("k8s-wheatley-%s", each.value.name)
     ipv4_address       = each.value.ip_address
     cpu                = local.workers.cpu
+    gpu                = each.value.gpu_enabled
     memory             = local.workers.memory
     disk               = local.workers.disk
     storagepool        = local.workers.storagepool
@@ -136,7 +145,7 @@ module "talos-bootstrap" {
     module.workers
   ]
 
-  source = "./modules/talos-bootstrap"
+  source = "../../modules/talos-bootstrap"
 
   node_config = {
     ipv4_gateway           = local.ipv4_gateway
@@ -164,6 +173,9 @@ module "talos-bootstrap" {
         name            = format("k8s-wheatley-%s", node.name)
         type            = "worker"
         ipv4_address    = node.ip_address
+        storage_address = node.storage_address
+        pvc_disks       = local.workers.pvc_disks
+        gpu_enabled     = node.gpu_enabled
       }
     ]
   )

pve01.wheatley.in/k8s-wheatley/providers.tf

@@ -2,11 +2,11 @@ terraform {
   required_providers {
     talos = {
       source  = "siderolabs/talos"
-      version = "0.9.0"
+      version = "0.10.1"
     }
     proxmox = {
       source  = "bpg/proxmox"
-      version = "0.86.0"
+      version = "0.95.0"
     }
   }
 }

pve01.wheatley.in/templates/.terraform.lock.hcl

@@ -0,0 +1,56 @@
+# This file is maintained automatically by "tofu init".
+# Manual edits may be lost in future updates.
+
+provider "registry.opentofu.org/bpg/proxmox" {
+  version     = "0.95.0"
+  constraints = "0.95.0"
+  hashes = [
+    "h1:8gOZBH6hYhvvr9ycmz8AVdgnABQPEft6q/qj2uIdDR8=",
+    "h1:APHNeh4Fayb+RnVzIUBGwD4pQZHEH6NBukvZKW9TETw=",
+    "h1:C/SnO5houq7BCNvpmH+QV+RbxcIyASlQm8dI51vxm34=",
+    "h1:HDn4A1XRSbaKd4RIzHJM/U1TBLw1llq5sAjB1oPS2yM=",
+    "h1:NamnpRo4NLLqhjWFw0lPr4V0Vm/hHqNOXfCbiPsv9WY=",
+    "h1:P/DYln7s7I2YWrowkkvrAdNXCYCOcccR38mcZLzidbs=",
+    "h1:QmOVJQiSH80Jdmc8T4CLKhkkHHtCohYpgNXTGyfYNX0=",
+    "h1:XqbY+SVrGaRp9T+nkw929kG3/4fp8SbFy7LUA2io8vU=",
+    "h1:aWrhONde+FDq3mgc3A2VJSvwt8R/gR+enf+GTww9TIU=",
+    "h1:nJblnO+z23prp5I9vqlR0C984dJ53Vpzj+ad3iPdig4=",
+    "h1:rnReS6P0e2uXYVMhpPgHb1B+eQrjXCnP8Hp6THLPqPg=",
+    "h1:s5DW5MUXT5Yspd7Z4qTrAnnSismfI7gqL4+aiMUuDY8=",
+    "h1:zEUJdG/8tJnHkjq1DZBqpkGuabZ6Ev1ExY4Y/ITHTVA=",
+    "h1:zxHxjwctc98DHZjOY4YOPkxYBk7cY28r0j9/6sI5o7I=",
+    "zh:07439b6c10c48c25357cd574d9db024adb21419865c5b22019a2d0493314b2fb",
+    "zh:29b56acca3df3d1ad1c0d677165192d59ec850c26d013e19a18cc94b11789b39",
+    "zh:35f73b7eeef23867633ee2d687c1cf015eb15ff128fc9527f018aaf0c58ef1ce",
+    "zh:4a479884f6549e8f5b11659c4fe9e3af4fc3f95440d7d8b54335d608361c731f",
+    "zh:4c89f62fb4528f15ae71fc91f792fe5e747ce162e7bf6124f21989d8805a3971",
+    "zh:510ccaaddb1add81563417dcb8f25f364988e8797293d2889b0bdbf623abe01b",
+    "zh:5d791e444e671f6b41c371fddec9d3aba8c8c5c8996ec0864b5aa8678b70feee",
+    "zh:5e655ca1b5d10cfebbe649952d073699005015082de897f3424faa80a9c13667",
+    "zh:71d639eaa7aaa7a6fa68a8bd546d864842c163ff4f2abc48176e287642156d27",
+    "zh:beb1f2ea7ca4e1e42373749b3b1a3dc7bb2bf283001eb9181e162da4f1345b07",
+    "zh:cd8116cfb4988c7157ec2ef18a3fb2bfa1020342fd66f99aa66ef8db678fe09d",
+    "zh:d0e3eacce028554d641ea7bfb857888559f9f92f3dcc00bdd5d8a3bc70768580",
+    "zh:e0eabff8250b428de253f064324fc878a27fa258a79aa76a1b0dab158986cec6",
+    "zh:eef582a3fcc21a7c7c29719ec1f969089f9bba8376a6bb0c8f5aa6c8f89a8ee0",
+    "zh:f26e0763dbe6a6b2195c94b44696f2110f7f55433dc142839be16b9697fa5597",
+  ]
+}
+
+provider "registry.opentofu.org/hashicorp/http" {
+  version     = "3.5.0"
+  constraints = "3.5.0"
+  hashes = [
+    "h1:eClUBisXme48lqiUl3U2+H2a2mzDawS9biqfkd9synw=",
+    "zh:0a2b33494eec6a91a183629cf217e073be063624c5d3f70870456ddb478308e9",
+    "zh:180f40124fa01b98b3d2f79128646b151818e09d6a1a9ca08e0b032a0b1e9cb1",
+    "zh:3e29e1de149dc10bf78620526c7cb8c62cd76087f5630dfaba0e93cda1f3aa7b",
+    "zh:4420950200cf86042ec940d0e2c9b7c89966bf556bf8038ba36217eae663bca5",
+    "zh:5d1f7d02109b2e2dca7ec626e5563ee765583792d0fd64081286f16f9433bd0d",
+    "zh:8500b138d338b1994c4206aa577b5c44e1d7260825babcf43245a7075bfa52a5",
+    "zh:b42165a6c4cfb22825938272d12b676e4a6946ac4e750f85df870c947685df2d",
+    "zh:b919bf3ee8e3b01051a0da3433b443a925e272893d3724ee8fc0f666ec7012c9",
+    "zh:d13b81ea6755cae785b3e11634936cdff2dc1ec009dc9610d8e3c7eb32f42e69",
+    "zh:f1c9d2eb1a6b618ae77ad86649679241bd8d6aacec06d0a68d86f748687f4eb3",
+  ]
+}

pve01.wheatley.in/templates/README.md

@@ -0,0 +1,33 @@
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_http"></a> [http](#requirement\_http) | 3.5.0 |
+| <a name="requirement_proxmox"></a> [proxmox](#requirement\_proxmox) | 0.86.0 |
+
+## Providers
+
+No providers.
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_talos"></a> [talos](#module\_talos) | ./talos | n/a |
+
+## Resources
+
+No resources.
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_proxmox_api_token"></a> [proxmox\_api\_token](#input\_proxmox\_api\_token) | Proxmox API token | `string` | n/a | yes |
+| <a name="input_proxmox_endpoint"></a> [proxmox\_endpoint](#input\_proxmox\_endpoint) | Proxmox API endpoint | `string` | n/a | yes |
+
+## Outputs
+
+No outputs.
+<!-- END_TF_DOCS -->

pve01.wheatley.in/templates/main.tf

@@ -0,0 +1,25 @@
+terraform {
+  backend "s3" {
+    bucket                      = "13225-wheatley-states"
+    key                         = "pve01-templates.tfstate"
+    region                      = "main"
+    skip_credentials_validation = true
+    skip_metadata_api_check     = true
+    skip_region_validation      = true
+    force_path_style            = true
+  }
+}
+
+provider "proxmox" {
+  endpoint  = var.proxmox_endpoint
+  api_token = var.proxmox_api_token
+  ssh {
+    agent    = true
+    username = "tofu"
+  }
+}
+
+module "talos" {
+  source         = "./talos"
+  talos_versions = ["1.12.2"]
+}

pve01.wheatley.in/templates/providers.tf

@@ -2,7 +2,7 @@ terraform {
   required_providers {
     proxmox = {
       source  = "bpg/proxmox"
-      version = "0.86.0"
+      version = "0.95.0"
     }
     http = {
       source  = "hashicorp/http"

pve01.wheatley.in/templates/talos/.terraform.lock.hcl

@@ -1,42 +0,0 @@
-# This file is maintained automatically by "tofu init".
-# Manual edits may be lost in future updates.
-
-provider "registry.opentofu.org/bpg/proxmox" {
-  version     = "0.86.0"
-  constraints = "0.86.0"
-  hashes = [
-    "h1:brQxwkXGxNw5N2gNwwefF9XxzrudYgSyjHrRhdvAdjo=",
-    "zh:09b627b92a59848769fadfc3d8103eebf070a3800144bf03cb93f44472327f44",
-    "zh:0e19eb7f1047d541e50b97d7ac440ea73685d0c28ed2dbe64217cbe2f0b353e0",
-    "zh:20f1e70091ff3056876618c93afd79527c8995f955d153993e8fbb10fa42593b",
-    "zh:3920315be565976f5a9da0803f8f1a108221282f1bc9e21160669d793af4e0c8",
-    "zh:5133b2a2027428d3926eaa3bcdc0ab65a75305d54f6cbc7c54cce746dfddbc8e",
-    "zh:514c588b04738d55c9e6b1c5a4e3fb1ef4041dfb809d2268f14d29839ecfba59",
-    "zh:55916034025b4833bd6a93bb5948dfb7d00830a772ef74fa70898c6f7de0da0b",
-    "zh:58b485a4b0bde56ca7032fca0ac09cb4c6ff2579e06cf4f2a311bb695baa0df1",
-    "zh:75ebe44e6da4108af5fe02a9cd99ed0189985b486a2a56594952098d161ceb3d",
-    "zh:a8c870bfb5958a3d49d639db3c2761cfb453c6a6f95e5e241890922b11c8a4d8",
-    "zh:c2df2748b9be47a6c3e613667c64874d5cb1d3fbb5b985d6eb9c3af5af298454",
-    "zh:c3059668f4f81e450e555a47310e7042044b335f131643262fd51f9ba96f2214",
-    "zh:ddbbb23910666f70cf4a9587ba57b45f5f58c53a1f8d7cee1d6f90a3d3ef38ef",
-    "zh:e430138b897edcd3b64e4309db34ac872526187782626aa074d8d1647a0abfa8",
-    "zh:f26e0763dbe6a6b2195c94b44696f2110f7f55433dc142839be16b9697fa5597",
-  ]
-}
-
-provider "registry.opentofu.org/hashicorp/http" {
-  version = "3.5.0"
-  hashes = [
-    "h1:eClUBisXme48lqiUl3U2+H2a2mzDawS9biqfkd9synw=",
-    "zh:0a2b33494eec6a91a183629cf217e073be063624c5d3f70870456ddb478308e9",
-    "zh:180f40124fa01b98b3d2f79128646b151818e09d6a1a9ca08e0b032a0b1e9cb1",
-    "zh:3e29e1de149dc10bf78620526c7cb8c62cd76087f5630dfaba0e93cda1f3aa7b",
-    "zh:4420950200cf86042ec940d0e2c9b7c89966bf556bf8038ba36217eae663bca5",
-    "zh:5d1f7d02109b2e2dca7ec626e5563ee765583792d0fd64081286f16f9433bd0d",
-    "zh:8500b138d338b1994c4206aa577b5c44e1d7260825babcf43245a7075bfa52a5",
-    "zh:b42165a6c4cfb22825938272d12b676e4a6946ac4e750f85df870c947685df2d",
-    "zh:b919bf3ee8e3b01051a0da3433b443a925e272893d3724ee8fc0f666ec7012c9",
-    "zh:d13b81ea6755cae785b3e11634936cdff2dc1ec009dc9610d8e3c7eb32f42e69",
-    "zh:f1c9d2eb1a6b618ae77ad86649679241bd8d6aacec06d0a68d86f748687f4eb3",
-  ]
-}

pve01.wheatley.in/templates/talos/README.md

@@ -0,0 +1,36 @@
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_proxmox"></a> [proxmox](#requirement\_proxmox) | 0.86.0 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_http"></a> [http](#provider\_http) | n/a |
+| <a name="provider_proxmox"></a> [proxmox](#provider\_proxmox) | 0.86.0 |
+
+## Modules
+
+No modules.
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [proxmox_virtual_environment_download_file.talos-image](https://registry.terraform.io/providers/bpg/proxmox/0.86.0/docs/resources/virtual_environment_download_file) | resource |
+| [proxmox_virtual_environment_vm.talos-template](https://registry.terraform.io/providers/bpg/proxmox/0.86.0/docs/resources/virtual_environment_vm) | resource |
+| [http_http.schematic](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) | data source |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_talos_versions"></a> [talos\_versions](#input\_talos\_versions) | List of Talos versions to create templates for | `list(string)` | `[]` | no |
+
+## Outputs
+
+No outputs.
+<!-- END_TF_DOCS -->

pve01.wheatley.in/templates/talos/main.tf

@@ -1,29 +1,12 @@
 terraform {
-  backend "s3" {
+  required_providers {
-    bucket                      = "13225-wheatley-states"
+    proxmox = {
-    key                         = "pve01-templates.tfstate"
+      source  = "bpg/proxmox"
-    region                      = "main"
+      version = "0.95.0"
-    skip_credentials_validation = true
-    skip_metadata_api_check     = true
-    skip_region_validation      = true
-    force_path_style            = true
     }
-}
-
-provider "proxmox" {
-  endpoint  = var.proxmox_endpoint
-  api_token = var.proxmox_api_token
-  ssh {
-    agent    = true
-    username = "root"
   }
 }
 
-
-locals {
-  talos_versions = ["1.11.5"]
-}
-
 data "http" "schematic" {
   url          = "https://factory.talos.dev/schematics"
   method       = "POST"
@@ -40,21 +23,20 @@ locals {
 }
 
 resource "proxmox_virtual_environment_download_file" "talos-image" {
-  for_each = toset(local.talos_versions)
+  for_each = toset(var.talos_versions)
 
   node_name    = "pve01"
   datastore_id = "local"
   content_type = "iso"
 
 
-  url                     = "https://factory.talos.dev/image/${local.schematic_id}/v${each.value}/nocloud-amd64-secureboot.raw.gz"
+  url       = "https://factory.talos.dev/image/${local.schematic_id}/v${each.value}/nocloud-amd64-secureboot.iso"
-  file_name               = "talos-${each.value}-nocloud-amd64-secureboot.img"
+  file_name = "talos-${each.value}-nocloud-amd64-secureboot.iso"
-  decompression_algorithm = "gz"
 }
 
 resource "proxmox_virtual_environment_vm" "talos-template" {
   depends_on = [proxmox_virtual_environment_download_file.talos-image]
-  for_each   = toset(local.talos_versions)
+  for_each   = toset(var.talos_versions)
 
   template  = true
   vm_id     = 10000 + tonumber(replace(each.value, ".", ""))
@@ -73,7 +55,7 @@ resource "proxmox_virtual_environment_vm" "talos-template" {
   cpu {
     cores   = 2
     sockets = 1
-    type    = "x86-64-v2-AES"
+    type    = "host"
   }
   memory {
     dedicated = 2048
@@ -93,7 +75,4 @@ resource "proxmox_virtual_environment_vm" "talos-template" {
     datastore_id = local.storagepool
     version      = "v2.0"
   }
-  network_device {
-    bridge = "vmbr1"
-  }
 }

pve01.wheatley.in/templates/talos/schematic.yaml

@@ -1,4 +1,5 @@
 customization:
   systemExtensions:
     officialExtensions:
+      - siderolabs/drbd
       - siderolabs/qemu-guest-agent

pve01.wheatley.in/templates/talos/variables.tf

@@ -1,8 +1,5 @@
-variable "proxmox_endpoint" {
+variable talos_versions {
-  description = "Proxmox API endpoint"
+  description = "List of Talos versions to create templates for"
-  type        = string
+  type        = list(string)
-}
+  default     = []
-variable "proxmox_api_token" {
-  description = "Proxmox API token"
-  type        = string
 }

pve01.wheatley.in/templates/variables.tf

@@ -0,0 +1,8 @@
+variable "proxmox_endpoint" {
+  description = "Proxmox API endpoint"
+  type        = string
+}
+variable "proxmox_api_token" {
+  description = "Proxmox API token"
+  type        = string
+}

renovate.json

@@ -0,0 +1,15 @@
+{
+  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
+  "extends": [
+    "config:best-practices"
+  ],
+  "dependencyDashboard": true,
+  "dependencyDashboardTitle": "Renovate Dashboard",
+  "labels": [
+    "renovate"
+  ],
+  "prHourlyLimit": 0,
+  "schedule": [
+    "before 6am on Tuesday"
+  ]
+}