fix goreleaser winget token field and temp disable docker workflow

- Updated GitHub Actions release workflow to set up .NET and build the LHM executable.
- Modified Makefile to include a conditional build step for the .NET executable on Windows.

.github/DISCUSSION_TEMPLATE/support.yml

@@ -0,0 +1,61 @@
+body:
+  - type: markdown
+    attributes:
+      value: |
+        ### Before opening a discussion:
+
+        - Check the [common issues guide](https://beszel.dev/guide/common-issues).
+        - Search existing [issues](https://github.com/henrygd/beszel/issues) and [discussions](https://github.com/henrygd/beszel/discussions) (including closed).
+
+  - type: textarea
+    id: description
+    attributes:
+      label: Description
+      description: A clear and concise description of the issue or question. If applicable, add screenshots to help explain your problem.
+    validations:
+      required: true
+  - type: input
+    id: system
+    attributes:
+      label: OS / Architecture
+      placeholder: linux/amd64 (agent), freebsd/arm64 (hub)
+    validations:
+      required: true
+  - type: input
+    id: version
+    attributes:
+      label: Beszel version
+      placeholder: 0.9.1
+    validations:
+      required: true
+  - type: dropdown
+    id: install-method
+    attributes:
+      label: Installation method
+      options:
+        - Docker
+        - Binary
+        - Nix
+        - Unraid
+        - Coolify
+        - Other (please describe above)
+    validations:
+      required: true
+  - type: textarea
+    id: config
+    attributes:
+      label: Configuration
+      description: Please provide any relevant service configuration
+      render: yaml
+  - type: textarea
+    id: hub-logs
+    attributes:
+      label: Hub Logs
+      description: Check the logs page in PocketBase (`/_/#/logs`) for relevant errors (copy JSON).
+      render: json
+  - type: textarea
+    id: agent-logs
+    attributes:
+      label: Agent Logs
+      description: Please provide any logs from the agent, if relevant. Use `LOG_LEVEL=debug` for more info.
+      render: shell

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,134 @@
+name: 🐛 Bug report
+description: Report a new bug or issue.
+title: '[Bug]: '
+labels: ['bug', "needs confirmation"]
+body:
+  - type: dropdown
+    id: component
+    attributes:
+      label: Component
+      description: Which part of Beszel is this about?
+      options:
+        - Hub
+        - Agent
+        - Hub & Agent
+    validations:
+      required: true
+  - type: markdown
+    attributes:
+      value: |
+        ### Thanks for taking the time to fill out this bug report!
+
+        - For more general support, please [start a support thread](https://github.com/henrygd/beszel/discussions/new?category=support).
+        - To request a change or feature, use the [feature request form](https://github.com/henrygd/beszel/issues/new?template=feature_request.yml).
+        - Please do not submit bugs that are specific to ZFS. We plan to add integration with ZFS utilities in the near future.
+
+        ### Before submitting a bug report:
+
+        - Check the [common issues guide](https://beszel.dev/guide/common-issues).
+        - Search existing [issues](https://github.com/henrygd/beszel/issues) and [discussions](https://github.com/henrygd/beszel/discussions) (including closed).
+  - type: textarea
+    id: description
+    attributes:
+      label: Description
+      description: Explain the issue you experienced clearly and concisely.
+      placeholder: I went to the coffee pot and it was empty.
+    validations:
+      required: true
+  - type: textarea
+    id: expected-behavior
+    attributes:
+      label: Expected Behavior
+      description: In a perfect world, what should have happened?
+      placeholder: When I got to the coffee pot, it should have been full.
+    validations:
+      required: true
+  - type: textarea
+    id: steps-to-reproduce
+    attributes:
+      label: Steps to Reproduce
+      description: Describe how to reproduce the issue in repeatable steps.
+      placeholder: |
+        1. Go to the coffee pot.
+        2. Make more coffee.
+        3. Pour it into a cup.
+    validations:
+      required: true
+  - type: dropdown
+    id: category
+    attributes:
+      label: Category
+      description: Which category does this relate to most?
+      options:
+        - Metrics
+        - Charts & Visualization
+        - Settings & Configuration
+        - Notifications & Alerts
+        - Authentication
+        - Installation
+        - Performance
+        - UI / UX
+        - Other
+    validations:
+      required: true
+  - type: dropdown
+    id: metrics
+    attributes:
+      label: Affected Metrics
+      description: If applicable, which specific metric does this relate to most?
+      options:
+        - CPU
+        - Memory
+        - Storage
+        - Network
+        - Containers
+        - GPU
+        - Sensors
+        - Other
+    validations:
+      required: true
+  - type: input
+    id: system
+    attributes:
+      label: OS / Architecture
+      placeholder: linux/amd64 (agent), freebsd/arm64 (hub)
+    validations:
+      required: true
+  - type: input
+    id: version
+    attributes:
+      label: Beszel version
+      placeholder: 0.9.1
+    validations:
+      required: true
+  - type: dropdown
+    id: install-method
+    attributes:
+      label: Installation method
+      options:
+        - Docker
+        - Binary
+        - Nix
+        - Unraid
+        - Coolify
+        - Other (please describe above)
+    validations:
+      required: true
+  - type: textarea
+    id: config
+    attributes:
+      label: Configuration
+      description: Please provide any relevant service configuration
+      render: yaml
+  - type: textarea
+    id: hub-logs
+    attributes:
+      label: Hub Logs
+      description: Check the logs page in PocketBase (`/_/#/logs`) for relevant errors (copy JSON).
+      render: json
+  - type: textarea
+    id: agent-logs
+    attributes:
+      label: Agent Logs
+      description: Please provide any logs from the agent, if relevant. Use `LOG_LEVEL=debug` for more info.
+      render: shell

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,8 @@
+blank_issues_enabled: false
+contact_links:
+  - name: 💬 Support and questions
+    url: https://github.com/henrygd/beszel/discussions
+    about: Ask and answer questions here.
+  - name: ℹ️ View the Common Issues page
+    url: https://beszel.dev/guide/common-issues
+    about: Find information about commonly encountered problems.

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,76 @@
+name: 🚀 Feature request
+description: Request a new feature or change.
+title: "[Feature]: "
+labels: ["enhancement", "needs review"]
+body:
+  - type: dropdown
+    id: component
+    attributes:
+      label: Component
+      description: Which part of Beszel is this about?
+      options:
+        - Hub
+        - Agent
+        - Hub & Agent
+    validations:
+      required: true
+  - type: markdown
+    attributes:
+      value: Before submitting, please search existing [issues](https://github.com/henrygd/beszel/issues) and [discussions](https://github.com/henrygd/beszel/discussions) (including closed).
+  - type: textarea
+    attributes:
+      label: Describe the feature you would like to see
+    validations:
+      required: true
+  - type: textarea
+    id: motivation
+    attributes:
+      label: Motivation / Use Case
+      description: Why do you want this feature? What problem does it solve?
+    validations:
+      required: true
+  - type: textarea
+    attributes:
+      label: Describe how you would like to see this feature implemented
+    validations:
+      required: true
+  - type: textarea
+    id: logs
+    attributes:
+      label: Screenshots
+      description: Please attach any relevant screenshots, such as images from your current solution or similar implementations.
+    validations:
+      required: false
+  - type: dropdown
+    id: category
+    attributes:
+      label: Category
+      description: Which category does this relate to most?
+      options:
+        - Metrics
+        - Charts & Visualization
+        - Settings & Configuration
+        - Notifications & Alerts
+        - Authentication
+        - Installation
+        - Performance
+        - UI / UX
+        - Other
+    validations:
+      required: true
+  - type: dropdown
+    id: metrics
+    attributes:
+      label: Affected Metrics
+      description: If applicable, which specific metric does this relate to most?
+      options:
+        - CPU
+        - Memory
+        - Storage
+        - Network
+        - Containers
+        - GPU
+        - Sensors
+        - Other
+    validations:
+      required: true

.github/funding.yml

@@ -0,0 +1 @@
+buy_me_a_coffee: henrygd

.github/pull_request_template.md

@@ -0,0 +1,33 @@
+## 📃 Description
+
+A short description of the pull request changes should go here and the sections below should list in detail all changes. You can remove the sections you don't need.
+
+## 📖 Documentation
+
+Add a link to the PR for [documentation](https://github.com/henrygd/beszel-docs) changes.
+
+## 🪵 Changelog
+
+### ➕ Added
+
+- one
+- two
+
+### ✏️ Changed
+
+- one
+- two
+
+### 🔧 Fixed
+
+- one
+- two
+
+### 🗑️ Removed
+
+- one
+- two
+
+## 📷 Screenshots
+
+If this PR has any UI/UX changes it's strongly suggested you add screenshots here.

.github/workflows/docker-images.yml

@@ -3,7 +3,7 @@ name: Make docker images
 on:
   push:
     tags:
-      - 'v*'
+      - "xv*"
 
 jobs:
   build:
@@ -15,9 +15,27 @@ jobs:
           - image: henrygd/beszel
             context: ./beszel
             dockerfile: ./beszel/dockerfile_Hub
+            registry: docker.io
+            username_secret: DOCKERHUB_USERNAME
+            password_secret: DOCKERHUB_TOKEN
           - image: henrygd/beszel-agent
             context: ./beszel
             dockerfile: ./beszel/dockerfile_Agent
+            registry: docker.io
+            username_secret: DOCKERHUB_USERNAME
+            password_secret: DOCKERHUB_TOKEN
+          - image: ghcr.io/${{ github.repository }}/beszel
+            context: ./beszel
+            dockerfile: ./beszel/dockerfile_Hub
+            registry: ghcr.io
+            username: ${{ github.actor }}
+            password_secret: GITHUB_TOKEN
+          - image: ghcr.io/${{ github.repository }}/beszel-agent
+            context: ./beszel
+            dockerfile: ./beszel/dockerfile_Agent
+            registry: ghcr.io
+            username: ${{ github.actor }}
+            password_secret: GITHUB_TOKEN
     permissions:
       contents: read
       packages: write
@@ -47,6 +65,7 @@ jobs:
         with:
           images: ${{ matrix.image }}
           tags: |
+            type=raw,value=edge
             type=semver,pattern={{version}}
             type=semver,pattern={{major}}.{{minor}}
             type=semver,pattern={{major}}
@@ -57,15 +76,16 @@ jobs:
         if: github.event_name != 'pull_request'
         uses: docker/login-action@v3
         with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
+          username: ${{ matrix.username || secrets[matrix.username_secret] }}
+          password: ${{ secrets[matrix.password_secret] }}
+          registry: ${{ matrix.registry }}
 
       # Build and push Docker image with Buildx (don't push on PR)
       # https://github.com/docker/build-push-action
       - name: Build and push Docker image
         uses: docker/build-push-action@v5
         with:
-          context: '${{ matrix.context }}'
+          context: "${{ matrix.context }}"
           file: ${{ matrix.dockerfile }}
           platforms: linux/amd64,linux/arm64,linux/arm/v7
           push: ${{ github.ref_type == 'tag' }}

.github/workflows/inactivity-actions.yml

@@ -0,0 +1,43 @@
+name: 'Issue and PR Maintenance'
+
+on:
+  schedule:
+    - cron: '0 0 * * *'   # runs at midnight UTC
+  workflow_dispatch:
+
+permissions:
+  issues: write
+  pull-requests: write
+
+jobs:
+  close-stale:
+    name: Close Stale Issues
+    runs-on: ubuntu-24.04
+    steps:
+      - name: Close Stale Issues
+        uses: actions/stale@v9
+        with:
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+
+          # Messaging
+          stale-issue-message: >
+            👋 This issue has been automatically marked as stale due to inactivity.
+            If this issue is still relevant, please comment to keep it open.
+            Without activity, it will be closed in 7 days.
+
+          close-issue-message: >
+            🔒 This issue has been automatically closed due to prolonged inactivity.
+            Feel free to open a new issue if you have further questions or concerns.
+
+          # Timing
+          days-before-issue-stale: 14
+          days-before-issue-close: 7
+
+          # Labels
+          stale-issue-label: 'stale'
+          remove-stale-when-updated: true
+          only-issue-labels: 'awaiting-requester'
+
+          # Exemptions
+          exempt-assignees: true
+          exempt-milestones: true

.github/workflows/label-from-dropdown.yml

@@ -0,0 +1,82 @@
+name: Label issues from dropdowns
+
+on:
+  issues:
+    types: [opened]
+
+jobs:
+  label_from_dropdown:
+    runs-on: ubuntu-latest
+    permissions:
+      issues: write
+    steps:
+      - name: Apply labels based on dropdown choices
+        uses: actions/github-script@v7
+        with:
+          script: |
+
+            const issueNumber = context.issue.number;
+            const owner = context.repo.owner;
+            const repo = context.repo.repo;
+
+            // Get the issue body
+            const body = context.payload.issue.body;
+
+            // Helper to find dropdown value in the body (assuming markdown format)
+            function extractSectionValue(heading) {
+              const regex = new RegExp(`### ${heading}\\s+([\\s\\S]*?)(?:\\n###|$)`, 'i');
+              const match = body.match(regex);
+              if (match) {
+                // Get the first non-empty line after the heading
+                const lines = match[1].split('\n').map(l => l.trim()).filter(Boolean);
+                return lines[0] || null;
+              }
+              return null;
+            }
+
+            // Extract dropdown selections
+            const category = extractSectionValue('Category');
+            const metrics = extractSectionValue('Affected Metrics');
+            const component = extractSectionValue('Component');
+
+            // Build labels to add
+            let labelsToAdd = [];
+            if (category) labelsToAdd.push(category);
+            if (metrics) labelsToAdd.push(metrics);
+            if (component) labelsToAdd.push(component);
+
+            // Get existing labels in the repo
+            const { data: existingLabels } = await github.rest.issues.listLabelsForRepo({
+              owner,
+              repo,
+              per_page: 100
+            });
+            const existingLabelNames = existingLabels.map(l => l.name);
+
+            // Find labels that need to be created
+            const labelsToCreate = labelsToAdd.filter(label => !existingLabelNames.includes(label));
+
+            // Create missing labels (with a default color)
+            for (const label of labelsToCreate) {
+              try {
+                await github.rest.issues.createLabel({
+                  owner,
+                  repo,
+                  name: label,
+                  color: 'ededed' // light gray, you can pick any hex color
+                });
+              } catch (e) {
+                // Ignore if label already exists (race condition), otherwise rethrow
+                if (!e || e.status !== 422) throw e;
+              }
+            }
+
+            // Now apply all labels (they all exist now)
+            if (labelsToAdd.length > 0) {
+              await github.rest.issues.addLabels({
+                owner,
+                repo,
+                issue_number: issueNumber,
+                labels: labelsToAdd
+              });
+            } 

.github/workflows/release.yml

@@ -3,7 +3,7 @@ name: Make release and binaries
 on:
   push:
     tags:
-      - '*'
+      - "v*"
 
 permissions:
   contents: write
@@ -29,7 +29,17 @@ jobs:
       - name: Set up Go
         uses: actions/setup-go@v5
         with:
-          go-version: '^1.22.1'
+          go-version: "^1.22.1"
+
+      - name: Set up .NET
+        uses: actions/setup-dotnet@v4
+        with:
+          dotnet-version: "9.0.x"
+
+      - name: Build .NET LHM executable for Windows sensors
+        run: |
+          dotnet build -c Release ./beszel/internal/agent/lhm/beszel_lhm.csproj
+        shell: bash
 
       - name: GoReleaser beszel
         uses: goreleaser/goreleaser-action@v6
@@ -39,4 +49,5 @@ jobs:
           version: latest
           args: release --clean
         env:
-          GITHUB_TOKEN: ${{ secrets.TOKEN }}
+          GITHUB_TOKEN: ${{ secrets.TOKEN || secrets.GITHUB_TOKEN }}
+          WINGET_TOKEN: ${{ secrets.WINGET_TOKEN }}

.github/workflows/vulncheck.yml

@@ -0,0 +1,33 @@
+# https://github.com/minio/minio/blob/master/.github/workflows/vulncheck.yml
+
+name: VulnCheck
+on:
+  pull_request:
+    branches:
+      - main
+
+  push:
+    branches:
+      - main
+
+permissions:
+  contents: read # to fetch code (actions/checkout)
+
+jobs:
+  vulncheck:
+    name: Analysis
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out code into the Go module directory
+        uses: actions/checkout@v4
+      - name: Set up Go
+        uses: actions/setup-go@v5
+        with:
+          go-version: 1.24.x
+          cached: false
+      - name: Get official govulncheck
+        run: go install golang.org/x/vuln/cmd/govulncheck@latest
+        shell: bash
+      - name: Run govulncheck
+        run: govulncheck -C ./beszel -show verbose ./...
+        shell: bash

.gitignore

@@ -11,3 +11,11 @@ dist
 beszel/cmd/hub/hub
 beszel/cmd/agent/agent
 node_modules
+beszel/build
+*timestamp*
+.swc
+beszel/site/src/locales/**/*.ts
+*.bak
+__debug_*
+beszel/internal/agent/lhm/obj
+beszel/internal/agent/lhm/bin

SECURITY.md

@@ -2,8 +2,6 @@
 
 ## Reporting a Vulnerability
 
-If you find a vulnerability in the latest version, please email me directly at hank@henrygd.me, or [submit a private advisory](https://github.com/henrygd/beszel/security/advisories/new).
+If you find a vulnerability in the latest version, please [submit a private advisory](https://github.com/henrygd/beszel/security/advisories/new).
 
-If you submit an advisory, open an empty issue as well to let me know that you did (or email me), as I'm not sure if I get notifications for that.
-
-If the issue is low severity (use best judgement) you may open an issue for it instead of contacting me directly.
+If it's low severity (use best judgement) you may open an issue instead of an advisory.

beszel/.goreleaser.yml

@@ -29,41 +29,195 @@ builds:
       - linux
       - darwin
       - freebsd
+      - openbsd
+      - windows
     goarch:
       - amd64
       - arm64
       - arm
       - mips64
+      - riscv64
+      - mipsle
+      - ppc64le
     ignore:
       - goos: freebsd
         goarch: arm
+      - goos: openbsd
+        goarch: arm
+      - goos: windows
+        goarch: arm
+      - goos: darwin
+        goarch: riscv64
+      - goos: windows
+        goarch: riscv64
 
 archives:
-  - id: beszel
-    format: tar.gz
+  - id: beszel-agent
+    formats: [tar.gz]
     builds:
       - beszel-agent
     name_template: >-
       {{ .Binary }}_
       {{- .Os }}_
       {{- .Arch }}
-  - id: beszel-agent
-    format: tar.gz
+    format_overrides:
+      - goos: windows
+        formats: [zip]
+
+  - id: beszel
+    formats: [tar.gz]
     builds:
       - beszel
     name_template: >-
       {{ .Binary }}_
       {{- .Os }}_
       {{- .Arch }}
-    # use zip for windows archives
-    # format_overrides:
-    #   - goos: windows
-    #     format: zip
+
+nfpms:
+  - id: beszel-agent
+    package_name: beszel-agent
+    description: |-
+      Agent for Beszel
+      Beszel is a lightweight server monitoring platform that includes Docker
+      statistics, historical data, and alert functions. It has a friendly web
+      interface, simple configuration, and is ready to use out of the box.
+      It supports automatic backup, multi-user, OAuth authentication, and
+      API access.
+    maintainer: henrygd <hank@henrygd.me>
+    section: net
+    builds:
+      - beszel-agent
+    formats:
+      - deb
+    contents:
+      - src: ../supplemental/debian/beszel-agent.service
+        dst: lib/systemd/system/beszel-agent.service
+        packager: deb
+      - src: ../supplemental/debian/copyright
+        dst: usr/share/doc/beszel-agent/copyright
+        packager: deb
+      - src: ../supplemental/debian/lintian-overrides
+        dst: usr/share/lintian/overrides/beszel-agent
+        packager: deb
+    scripts:
+      postinstall: ../supplemental/debian/postinstall.sh
+      preremove: ../supplemental/debian/prerm.sh
+      postremove: ../supplemental/debian/postrm.sh
+    deb:
+      predepends:
+        - adduser
+        - debconf
+      scripts:
+        templates: ../supplemental/debian/templates
+        # Currently broken due to a bug in goreleaser
+        # https://github.com/goreleaser/goreleaser/issues/5487
+        #config: ../supplemental/debian/config.sh
+
+scoops:
+  - ids: [beszel-agent]
+    name: beszel-agent
+    repository:
+      owner: henrygd
+      name: beszel-scoops
+    homepage: "https://beszel.dev"
+    description: "Agent for Beszel, a lightweight server monitoring platform."
+    license: MIT
+
+# # Needs choco installed, so doesn't build on linux / default gh workflow :(
+# chocolateys:
+#   - title: Beszel Agent
+#     ids: [beszel-agent]
+#     package_source_url: https://github.com/henrygd/beszel-chocolatey
+#     owners: henrygd
+#     authors: henrygd
+#     summary: 'Agent for Beszel, a lightweight server monitoring platform.'
+#     description: |
+#       Beszel is a lightweight server monitoring platform that includes Docker statistics, historical data, and alert functions.
+
+#       It has a friendly web interface, simple configuration, and is ready to use out of the box. It supports automatic backup, multi-user, OAuth authentication, and API access.
+#     license_url: https://github.com/henrygd/beszel/blob/main/LICENSE
+#     project_url: https://beszel.dev
+#     project_source_url: https://github.com/henrygd/beszel
+#     docs_url: https://beszel.dev/guide/getting-started
+#     icon_url: https://cdn.jsdelivr.net/gh/selfhst/icons/png/beszel.png
+#     bug_tracker_url: https://github.com/henrygd/beszel/issues
+#     copyright: 2025 henrygd
+#     tags: foss cross-platform admin monitoring
+#     require_license_acceptance: false
+#     release_notes: 'https://github.com/henrygd/beszel/releases/tag/v{{ .Version }}'
+
+brews:
+  - ids: [beszel-agent]
+    name: beszel-agent
+    repository:
+      owner: henrygd
+      name: homebrew-beszel
+    homepage: "https://beszel.dev"
+    description: "Agent for Beszel, a lightweight server monitoring platform."
+    license: MIT
+    skip_upload: auto
+    extra_install: |
+      (bin/"beszel-agent-launcher").write <<~EOS
+        #!/bin/bash
+        set -a
+        if [ -f "$HOME/.config/beszel/beszel-agent.env" ]; then
+          source "$HOME/.config/beszel/beszel-agent.env"
+        fi
+        set +a
+        exec #{bin}/beszel-agent "$@"
+      EOS
+      (bin/"beszel-agent-launcher").chmod 0755
+    service: |
+      run ["#{bin}/beszel-agent-launcher"]
+      log_path "#{Dir.home}/.cache/beszel/beszel-agent.log"
+      error_log_path "#{Dir.home}/.cache/beszel/beszel-agent.log"
+      keep_alive true
+      restart_delay 5
+      process_type :background
+
+winget:
+  - ids: [beszel-agent]
+    name: beszel-agent
+    package_identifier: henrygd.beszel-agent
+    publisher: henrygd
+    license: MIT
+    license_url: "https://github.com/henrygd/beszel/blob/main/LICENSE"
+    copyright: "2025 henrygd"
+    homepage: "https://beszel.dev"
+    release_notes_url: "https://github.com/henrygd/beszel/releases/tag/v{{ .Version }}"
+    publisher_support_url: "https://github.com/henrygd/beszel/issues"
+    short_description: "Agent for Beszel, a lightweight server monitoring platform."
+    skip_upload: auto
+    description: |
+      Beszel is a lightweight server monitoring platform that includes Docker
+      statistics, historical data, and alert functions. It has a friendly web
+      interface, simple configuration, and is ready to use out of the box.
+      It supports automatic backup, multi-user, OAuth authentication, and
+      API access.
+    tags:
+      - homelab
+      - monitoring
+      - self-hosted
+    repository:
+      owner: henrygd
+      name: beszel-winget
+      branch: henrygd.beszel-agent-{{ .Version }}
+      token: "{{ .Env.WINGET_TOKEN }}"
+      pull_request:
+        enabled: true
+        draft: false
+        base:
+          owner: microsoft
+          name: winget-pkgs
+          branch: master
+
+release:
+  draft: true
 
 changelog:
   disable: true
   sort: asc
   filters:
     exclude:
-      - '^docs:'
-      - '^test:'
+      - "^docs:"
+      - "^test:"

beszel/Makefile

@@ -0,0 +1,98 @@
+# Default OS/ARCH values
+OS ?= $(shell go env GOOS)
+ARCH ?= $(shell go env GOARCH)
+# Skip building the web UI if true
+SKIP_WEB ?= false
+
+# Set executable extension based on target OS
+EXE_EXT := $(if $(filter windows,$(OS)),.exe,)
+
+.PHONY: tidy build-agent build-hub build clean lint dev-server dev-agent dev-hub dev generate-locales
+.DEFAULT_GOAL := build
+
+clean:
+	go clean
+	rm -rf ./build
+
+lint:
+	golangci-lint run
+
+test: export GOEXPERIMENT=synctest
+test:
+	go test -tags=testing ./...
+
+tidy:
+	go mod tidy
+
+build-web-ui:
+	@if command -v bun >/dev/null 2>&1; then \
+		bun install --cwd ./site && \
+		bun run --cwd ./site build; \
+	else \
+		npm install --prefix ./site && \
+		npm run --prefix ./site build; \
+	fi
+
+# Conditional .NET build - only for Windows
+build-dotnet-conditional:
+	@if [ "$(OS)" = "windows" ]; then \
+		echo "Building .NET executable for Windows..."; \
+		if command -v dotnet >/dev/null 2>&1; then \
+			rm -rf ./internal/agent/lhm/bin; \
+			dotnet build -c Release ./internal/agent/lhm/beszel_lhm.csproj; \
+		else \
+			echo "Error: dotnet not found. Install .NET SDK to build Windows agent."; \
+			exit 1; \
+		fi; \
+	fi
+
+# Update build-agent to include conditional .NET build
+build-agent: tidy build-dotnet-conditional
+	GOOS=$(OS) GOARCH=$(ARCH) go build -o ./build/beszel-agent_$(OS)_$(ARCH)$(EXE_EXT) -ldflags "-w -s" beszel/cmd/agent
+
+build-hub: tidy $(if $(filter false,$(SKIP_WEB)),build-web-ui)
+	GOOS=$(OS) GOARCH=$(ARCH) go build -o ./build/beszel_$(OS)_$(ARCH)$(EXE_EXT) -ldflags "-w -s" beszel/cmd/hub
+
+build: build-agent build-hub
+
+generate-locales:
+	@if [ ! -f ./site/src/locales/en/en.ts ]; then \
+		echo "Generating locales..."; \
+		command -v bun >/dev/null 2>&1 && cd ./site && bun install && bun run sync || cd ./site && npm install && npm run sync; \
+	fi
+
+dev-server: generate-locales
+	cd ./site
+	@if command -v bun >/dev/null 2>&1; then \
+		cd ./site && bun run dev --host 0.0.0.0; \
+	else \
+		cd ./site && npm run dev --host 0.0.0.0; \
+	fi
+
+dev-hub: export ENV=dev
+dev-hub:
+	mkdir -p ./site/dist && touch ./site/dist/index.html
+	@if command -v entr >/dev/null 2>&1; then \
+		find ./cmd/hub/*.go ./internal/{alerts,hub,records,users}/*.go | entr -r -s "cd ./cmd/hub && go run . serve --http 0.0.0.0:8090"; \
+	else \
+		cd ./cmd/hub && go run . serve --http 0.0.0.0:8090; \
+	fi
+
+dev-agent:
+	@if command -v entr >/dev/null 2>&1; then \
+		find ./cmd/agent/*.go ./internal/agent/*.go | entr -r go run beszel/cmd/agent; \
+	else \
+		go run beszel/cmd/agent; \
+	fi
+	
+build-dotnet:
+	@if command -v dotnet >/dev/null 2>&1; then \
+		rm -rf ./internal/agent/lhm/bin; \
+		dotnet build -c Release ./internal/agent/lhm/beszel_lhm.csproj; \
+	else \
+		echo "dotnet not found"; \
+	fi
+
+
+# KEY="..." make -j dev
+dev: dev-server dev-hub dev-agent

beszel/cmd/agent/agent.go

@@ -3,39 +3,120 @@ package main
 import (
 	"beszel"
 	"beszel/internal/agent"
+	"beszel/internal/agent/health"
+	"flag"
 	"fmt"
 	"log"
 	"os"
-	"strings"
+
+	"golang.org/x/crypto/ssh"
 )
 
-func main() {
-	// handle flags / subcommands
-	if len(os.Args) > 1 {
-		switch os.Args[1] {
-		case "-v":
-			fmt.Println(beszel.AppName+"-agent", beszel.Version)
-		case "update":
-			agent.Update()
-		}
-		os.Exit(0)
-	}
-
-	var pubKey []byte
-	if pubKeyEnv, exists := os.LookupEnv("KEY"); exists {
-		pubKey = []byte(pubKeyEnv)
-	} else {
-		log.Fatal("KEY environment variable is not set")
-	}
-
-	addr := ":45876"
-	if portEnvVar, exists := os.LookupEnv("PORT"); exists {
-		// allow passing an address in the form of "127.0.0.1:45876"
-		if !strings.Contains(portEnvVar, ":") {
-			portEnvVar = ":" + portEnvVar
-		}
-		addr = portEnvVar
-	}
-
-	agent.NewAgent().Run(pubKey, addr)
+// cli options
+type cmdOptions struct {
+	key    string // key is the public key(s) for SSH authentication.
+	listen string // listen is the address or port to listen on.
+}
+
+// parse parses the command line flags and populates the config struct.
+// It returns true if a subcommand was handled and the program should exit.
+func (opts *cmdOptions) parse() bool {
+	flag.StringVar(&opts.key, "key", "", "Public key(s) for SSH authentication")
+	flag.StringVar(&opts.listen, "listen", "", "Address or port to listen on")
+
+	flag.Usage = func() {
+		fmt.Printf("Usage: %s [command] [flags]\n", os.Args[0])
+		fmt.Println("\nCommands:")
+		fmt.Println("  health    Check if the agent is running")
+		fmt.Println("  help      Display this help message")
+		fmt.Println("  update    Update to the latest version")
+		fmt.Println("  version   Display the version")
+		fmt.Println("\nFlags:")
+		flag.PrintDefaults()
+	}
+
+	subcommand := ""
+	if len(os.Args) > 1 {
+		subcommand = os.Args[1]
+	}
+
+	switch subcommand {
+	case "-v", "version":
+		fmt.Println(beszel.AppName+"-agent", beszel.Version)
+		return true
+	case "help":
+		flag.Usage()
+		return true
+	case "update":
+		agent.Update()
+		return true
+	case "health":
+		err := health.Check()
+		if err != nil {
+			log.Fatal(err)
+		}
+		fmt.Print("ok")
+		return true
+	}
+
+	flag.Parse()
+	return false
+}
+
+// loadPublicKeys loads the public keys from the command line flag, environment variable, or key file.
+func (opts *cmdOptions) loadPublicKeys() ([]ssh.PublicKey, error) {
+	// Try command line flag first
+	if opts.key != "" {
+		return agent.ParseKeys(opts.key)
+	}
+
+	// Try environment variable
+	if key, ok := agent.GetEnv("KEY"); ok && key != "" {
+		return agent.ParseKeys(key)
+	}
+
+	// Try key file
+	keyFile, ok := agent.GetEnv("KEY_FILE")
+	if !ok {
+		return nil, fmt.Errorf("no key provided: must set -key flag, KEY env var, or KEY_FILE env var. Use 'beszel-agent help' for usage")
+	}
+
+	pubKey, err := os.ReadFile(keyFile)
+	if err != nil {
+		return nil, fmt.Errorf("failed to read key file: %w", err)
+	}
+	return agent.ParseKeys(string(pubKey))
+}
+
+func (opts *cmdOptions) getAddress() string {
+	return agent.GetAddress(opts.listen)
+}
+
+func main() {
+	var opts cmdOptions
+	subcommandHandled := opts.parse()
+
+	if subcommandHandled {
+		return
+	}
+
+	var serverConfig agent.ServerOptions
+	var err error
+	serverConfig.Keys, err = opts.loadPublicKeys()
+	if err != nil {
+		log.Fatal("Failed to load public keys:", err)
+	}
+
+	addr := opts.getAddress()
+	serverConfig.Addr = addr
+	serverConfig.Network = agent.GetNetwork(addr)
+
+	agent, err := agent.NewAgent()
+	if err != nil {
+		log.Fatal("Failed to create agent: ", err)
+	}
+
+	if err := agent.Start(serverConfig); err != nil {
+		log.Fatal("Failed to start server: ", err)
+	}
 }

beszel/cmd/agent/agent_test.go

@@ -0,0 +1,303 @@
+package main
+
+import (
+	"beszel/internal/agent"
+	"crypto/ed25519"
+	"flag"
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"golang.org/x/crypto/ssh"
+)
+
+func TestGetAddress(t *testing.T) {
+	tests := []struct {
+		name     string
+		opts     cmdOptions
+		envVars  map[string]string
+		expected string
+	}{
+		{
+			name:     "default port when no config",
+			opts:     cmdOptions{},
+			expected: ":45876",
+		},
+		{
+			name: "use address from flag",
+			opts: cmdOptions{
+				listen: "8080",
+			},
+			expected: ":8080",
+		},
+		{
+			name: "use unix socket from flag",
+			opts: cmdOptions{
+				listen: "/tmp/beszel.sock",
+			},
+			expected: "/tmp/beszel.sock",
+		},
+		{
+			name: "use LISTEN env var",
+			opts: cmdOptions{},
+			envVars: map[string]string{
+				"LISTEN": "1.2.3.4:9090",
+			},
+			expected: "1.2.3.4:9090",
+		},
+		{
+			name: "use legacy PORT env var",
+			opts: cmdOptions{},
+			envVars: map[string]string{
+				"PORT": "7070",
+			},
+			expected: ":7070",
+		},
+		{
+			name: "use unix socket from env var",
+			opts: cmdOptions{
+				listen: "",
+			},
+			envVars: map[string]string{
+				"LISTEN": "/tmp/beszel.sock",
+			},
+			expected: "/tmp/beszel.sock",
+		},
+		{
+			name: "flag takes precedence over env vars",
+			opts: cmdOptions{
+				listen: ":8080",
+			},
+			envVars: map[string]string{
+				"LISTEN": ":9090",
+				"PORT":   "7070",
+			},
+			expected: ":8080",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			// Setup environment
+			for k, v := range tt.envVars {
+				t.Setenv(k, v)
+			}
+
+			addr := tt.opts.getAddress()
+			assert.Equal(t, tt.expected, addr)
+		})
+	}
+}
+
+func TestLoadPublicKeys(t *testing.T) {
+	// Generate a test key
+	_, priv, err := ed25519.GenerateKey(nil)
+	require.NoError(t, err)
+	signer, err := ssh.NewSignerFromKey(priv)
+	require.NoError(t, err)
+	pubKey := ssh.MarshalAuthorizedKey(signer.PublicKey())
+
+	tests := []struct {
+		name        string
+		opts        cmdOptions
+		envVars     map[string]string
+		setupFiles  map[string][]byte
+		wantErr     bool
+		errContains string
+	}{
+		{
+			name: "load key from flag",
+			opts: cmdOptions{
+				key: string(pubKey),
+			},
+		},
+		{
+			name: "load key from env var",
+			envVars: map[string]string{
+				"KEY": string(pubKey),
+			},
+		},
+		{
+			name: "load key from file",
+			envVars: map[string]string{
+				"KEY_FILE": "testkey.pub",
+			},
+			setupFiles: map[string][]byte{
+				"testkey.pub": pubKey,
+			},
+		},
+		{
+			name:        "error when no key provided",
+			wantErr:     true,
+			errContains: "no key provided",
+		},
+		{
+			name: "error on invalid key file",
+			envVars: map[string]string{
+				"KEY_FILE": "nonexistent.pub",
+			},
+			wantErr:     true,
+			errContains: "failed to read key file",
+		},
+		{
+			name: "error on invalid key data",
+			opts: cmdOptions{
+				key: "invalid-key-data",
+			},
+			wantErr: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			// Create a temporary directory for test files
+			if len(tt.setupFiles) > 0 {
+				tmpDir := t.TempDir()
+				for name, content := range tt.setupFiles {
+					path := filepath.Join(tmpDir, name)
+					err := os.WriteFile(path, content, 0600)
+					require.NoError(t, err)
+					if tt.envVars != nil {
+						tt.envVars["KEY_FILE"] = path
+					}
+				}
+			}
+
+			// Set up environment
+			for k, v := range tt.envVars {
+				t.Setenv(k, v)
+			}
+
+			keys, err := tt.opts.loadPublicKeys()
+			if tt.wantErr {
+				assert.Error(t, err)
+				if tt.errContains != "" {
+					assert.Contains(t, err.Error(), tt.errContains)
+				}
+				return
+			}
+
+			require.NoError(t, err)
+			assert.Len(t, keys, 1)
+			assert.Equal(t, signer.PublicKey().Type(), keys[0].Type())
+		})
+	}
+}
+
+func TestGetNetwork(t *testing.T) {
+	tests := []struct {
+		name     string
+		opts     cmdOptions
+		envVars  map[string]string
+		expected string
+	}{
+		{
+			name: "NETWORK env var",
+			envVars: map[string]string{
+				"NETWORK": "tcp4",
+			},
+			expected: "tcp4",
+		},
+		{
+			name:     "only port",
+			opts:     cmdOptions{listen: "8080"},
+			expected: "tcp",
+		},
+		{
+			name:     "ipv4 address",
+			opts:     cmdOptions{listen: "1.2.3.4:8080"},
+			expected: "tcp",
+		},
+		{
+			name:     "ipv6 address",
+			opts:     cmdOptions{listen: "[2001:db8::1]:8080"},
+			expected: "tcp",
+		},
+		{
+			name:     "unix network",
+			opts:     cmdOptions{listen: "/tmp/beszel.sock"},
+			expected: "unix",
+		},
+		{
+			name:     "env var network",
+			opts:     cmdOptions{listen: ":8080"},
+			envVars:  map[string]string{"NETWORK": "tcp4"},
+			expected: "tcp4",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			// Setup environment
+			for k, v := range tt.envVars {
+				t.Setenv(k, v)
+			}
+			network := agent.GetNetwork(tt.opts.listen)
+			assert.Equal(t, tt.expected, network)
+		})
+	}
+}
+
+func TestParseFlags(t *testing.T) {
+	// Save original command line arguments and restore after test
+	oldArgs := os.Args
+	defer func() {
+		os.Args = oldArgs
+		flag.CommandLine = flag.NewFlagSet(os.Args[0], flag.ExitOnError)
+	}()
+
+	tests := []struct {
+		name     string
+		args     []string
+		expected cmdOptions
+	}{
+		{
+			name: "no flags",
+			args: []string{"cmd"},
+			expected: cmdOptions{
+				key:    "",
+				listen: "",
+			},
+		},
+		{
+			name: "key flag only",
+			args: []string{"cmd", "-key", "testkey"},
+			expected: cmdOptions{
+				key:    "testkey",
+				listen: "",
+			},
+		},
+		{
+			name: "addr flag only",
+			args: []string{"cmd", "-listen", ":8080"},
+			expected: cmdOptions{
+				key:    "",
+				listen: ":8080",
+			},
+		},
+		{
+			name: "both flags",
+			args: []string{"cmd", "-key", "testkey", "-listen", ":8080"},
+			expected: cmdOptions{
+				key:    "testkey",
+				listen: ":8080",
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			// Reset flags for each test
+			flag.CommandLine = flag.NewFlagSet(tt.args[0], flag.ExitOnError)
+			os.Args = tt.args
+
+			var opts cmdOptions
+			opts.parse()
+			flag.Parse()
+
+			assert.Equal(t, tt.expected, opts)
+		})
+	}
+}

beszel/cmd/hub/hub.go

@@ -4,25 +4,96 @@ import (
 	"beszel"
 	"beszel/internal/hub"
 	_ "beszel/migrations"
+	"fmt"
+	"log"
+	"net/http"
+	"os"
+	"time"
 
 	"github.com/pocketbase/pocketbase"
+	"github.com/pocketbase/pocketbase/plugins/migratecmd"
 	"github.com/spf13/cobra"
 )
 
 func main() {
-	app := pocketbase.NewWithConfig(pocketbase.Config{
-		DefaultDataDir: beszel.AppName + "_data",
-	})
-	app.RootCmd.Version = beszel.Version
-	app.RootCmd.Use = beszel.AppName
-	app.RootCmd.Short = ""
+	// handle health check first to prevent unneeded execution
+	if len(os.Args) > 3 && os.Args[1] == "health" {
+		url := os.Args[3]
+		if err := checkHealth(url); err != nil {
+			log.Fatal(err)
+		}
+		fmt.Print("ok")
+		return
+	}
 
+	baseApp := getBaseApp()
+	h := hub.NewHub(baseApp)
+	if err := h.StartHub(); err != nil {
+		log.Fatal(err)
+	}
+}
+
+// getBaseApp creates a new PocketBase app with the default config
+func getBaseApp() *pocketbase.PocketBase {
+	isDev := os.Getenv("ENV") == "dev"
+
+	baseApp := pocketbase.NewWithConfig(pocketbase.Config{
+		DefaultDataDir: beszel.AppName + "_data",
+		DefaultDev:     isDev,
+	})
+	baseApp.RootCmd.Version = beszel.Version
+	baseApp.RootCmd.Use = beszel.AppName
+	baseApp.RootCmd.Short = ""
 	// add update command
-	app.RootCmd.AddCommand(&cobra.Command{
+	baseApp.RootCmd.AddCommand(&cobra.Command{
 		Use:   "update",
 		Short: "Update " + beszel.AppName + " to the latest version",
 		Run:   hub.Update,
 	})
+	// add health command
+	baseApp.RootCmd.AddCommand(newHealthCmd())
 
-	hub.NewHub(app).Run()
+	// enable auto creation of migration files when making collection changes in the Admin UI
+	migratecmd.MustRegister(baseApp, baseApp.RootCmd, migratecmd.Config{
+		Automigrate: isDev,
+		Dir:         "../../migrations",
+	})
+
+	return baseApp
+}
+
+func newHealthCmd() *cobra.Command {
+	var baseURL string
+
+	healthCmd := &cobra.Command{
+		Use:   "health",
+		Short: "Check health of running hub",
+		Run: func(cmd *cobra.Command, args []string) {
+			if err := checkHealth(baseURL); err != nil {
+				log.Fatal(err)
+			}
+			os.Exit(0)
+		},
+	}
+	healthCmd.Flags().StringVar(&baseURL, "url", "", "base URL")
+	healthCmd.MarkFlagRequired("url")
+	return healthCmd
+}
+
+// checkHealth checks the health of the hub.
+func checkHealth(baseURL string) error {
+	client := &http.Client{
+		Timeout: time.Second * 3,
+	}
+	healthURL := baseURL + "/api/health"
+	resp, err := client.Get(healthURL)
+	if err != nil {
+		return err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != 200 {
+		return fmt.Errorf("%s returned status %d", healthURL, resp.StatusCode)
+	}
+	return nil
 }

beszel/dockerfile_Agent

@@ -12,9 +12,15 @@ COPY internal ./internal
 ARG TARGETOS TARGETARCH
 RUN CGO_ENABLED=0 GOGC=75 GOOS=$TARGETOS GOARCH=$TARGETARCH go build -ldflags "-w -s" -o /agent ./cmd/agent
 
+RUN rm -rf /tmp/*
+
 # ? -------------------------
 FROM scratch
 
 COPY --from=builder /agent /agent
 
-ENTRYPOINT ["/agent"]
+# this is so we don't need to create the 
+# /tmp directory in the scratch container
+COPY --from=builder /tmp /tmp
+
+ENTRYPOINT ["/agent"]

beszel/go.mod

@@ -1,104 +1,72 @@
 module beszel
 
-go 1.22.4
+go 1.24.4
+
+// lock shoutrrr to specific version to allow review before updating
+replace github.com/nicholas-fedor/shoutrrr => github.com/nicholas-fedor/shoutrrr v0.8.8
 
 require (
 	github.com/blang/semver v3.5.1+incompatible
-	github.com/containrrr/shoutrrr v0.8.0
-	github.com/gliderlabs/ssh v0.3.7
-	github.com/goccy/go-json v0.10.3
-	github.com/labstack/echo/v5 v5.0.0-20230722203903-ec5b858dab61
-	github.com/pocketbase/dbx v1.10.1
-	github.com/pocketbase/pocketbase v0.22.22
+	github.com/fxamacker/cbor/v2 v2.9.0
+	github.com/gliderlabs/ssh v0.3.8
+	github.com/google/uuid v1.6.0
+	github.com/lxzan/gws v1.8.9
+	github.com/nicholas-fedor/shoutrrr v0.8.15
+	github.com/pocketbase/dbx v1.11.0
+	github.com/pocketbase/pocketbase v0.29.0
 	github.com/rhysd/go-github-selfupdate v1.2.3
-	github.com/shirou/gopsutil/v4 v4.24.9
-	github.com/spf13/cast v1.7.0
-	github.com/spf13/cobra v1.8.1
-	golang.org/x/crypto v0.28.0
+	github.com/shirou/gopsutil/v4 v4.25.6
+	github.com/spf13/cast v1.9.2
+	github.com/spf13/cobra v1.9.1
+	github.com/stretchr/testify v1.10.0
+	golang.org/x/crypto v0.40.0
+	golang.org/x/exp v0.0.0-20250718183923-645b1fa84792
 	gopkg.in/yaml.v3 v3.0.1
 )
 
 require (
-	github.com/AlecAivazis/survey/v2 v2.3.7 // indirect
 	github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be // indirect
 	github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
-	github.com/aws/aws-sdk-go-v2 v1.32.2 // indirect
-	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.6 // indirect
-	github.com/aws/aws-sdk-go-v2/config v1.28.0 // indirect
-	github.com/aws/aws-sdk-go-v2/credentials v1.17.41 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.17 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.33 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.21 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.21 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
-	github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.21 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.2 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.2 // indirect
-	github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.2 // indirect
-	github.com/aws/aws-sdk-go-v2/service/s3 v1.66.0 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sso v1.24.2 // indirect
-	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.2 // indirect
-	github.com/aws/aws-sdk-go-v2/service/sts v1.32.2 // indirect
-	github.com/aws/smithy-go v1.22.0 // indirect
 	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
 	github.com/disintegration/imaging v1.6.2 // indirect
+	github.com/dolthub/maphash v0.1.0 // indirect
 	github.com/domodwyer/mailyak/v3 v3.6.2 // indirect
 	github.com/dustin/go-humanize v1.0.1 // indirect
-	github.com/ebitengine/purego v0.8.0 // indirect
-	github.com/fatih/color v1.17.0 // indirect
-	github.com/gabriel-vasile/mimetype v1.4.6 // indirect
-	github.com/ganigeorgiev/fexpr v0.4.1 // indirect
+	github.com/ebitengine/purego v0.8.4 // indirect
+	github.com/fatih/color v1.18.0 // indirect
+	github.com/gabriel-vasile/mimetype v1.4.9 // indirect
+	github.com/ganigeorgiev/fexpr v0.5.0 // indirect
 	github.com/go-ole/go-ole v1.3.0 // indirect
 	github.com/go-ozzo/ozzo-validation/v4 v4.3.0 // indirect
-	github.com/golang-jwt/jwt/v4 v4.5.0 // indirect
-	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
+	github.com/go-sql-driver/mysql v1.9.1 // indirect
+	github.com/golang-jwt/jwt/v5 v5.2.3 // indirect
 	github.com/google/go-github/v30 v30.1.0 // indirect
 	github.com/google/go-querystring v1.1.0 // indirect
-	github.com/google/uuid v1.6.0 // indirect
-	github.com/googleapis/gax-go/v2 v2.13.0 // indirect
-	github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
 	github.com/inconshreveable/go-update v0.0.0-20160112193335-8152e7eb6ccf // indirect
 	github.com/inconshreveable/mousetrap v1.1.0 // indirect
-	github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
-	github.com/lufia/plan9stats v0.0.0-20240909124753-873cd0166683 // indirect
-	github.com/mattn/go-colorable v0.1.13 // indirect
+	github.com/klauspost/compress v1.18.0 // indirect
+	github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35 // indirect
+	github.com/mattn/go-colorable v0.1.14 // indirect
 	github.com/mattn/go-isatty v0.0.20 // indirect
-	github.com/mattn/go-sqlite3 v1.14.24 // indirect
-	github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
 	github.com/ncruces/go-strftime v0.1.9 // indirect
 	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
 	github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
 	github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
-	github.com/spf13/pflag v1.0.5 // indirect
+	github.com/spf13/pflag v1.0.7 // indirect
 	github.com/tcnksm/go-gitconfig v0.1.2 // indirect
-	github.com/tklauser/go-sysconf v0.3.14 // indirect
-	github.com/tklauser/numcpus v0.8.0 // indirect
+	github.com/tklauser/go-sysconf v0.3.15 // indirect
+	github.com/tklauser/numcpus v0.10.0 // indirect
 	github.com/ulikunitz/xz v0.5.12 // indirect
-	github.com/valyala/bytebufferpool v1.0.0 // indirect
-	github.com/valyala/fasttemplate v1.2.2 // indirect
+	github.com/x448/float16 v0.8.4 // indirect
 	github.com/yusufpapurcu/wmi v1.2.4 // indirect
-	go.opencensus.io v0.24.0 // indirect
-	gocloud.dev v0.40.0 // indirect
-	golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c // indirect
-	golang.org/x/image v0.21.0 // indirect
-	golang.org/x/net v0.30.0 // indirect
-	golang.org/x/oauth2 v0.23.0 // indirect
-	golang.org/x/sync v0.8.0 // indirect
-	golang.org/x/sys v0.26.0 // indirect
-	golang.org/x/term v0.25.0 // indirect
-	golang.org/x/text v0.19.0 // indirect
-	golang.org/x/time v0.7.0 // indirect
-	golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
-	google.golang.org/api v0.201.0 // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20241015192408-796eee8c2d53 // indirect
-	google.golang.org/grpc v1.67.1 // indirect
-	google.golang.org/protobuf v1.35.1 // indirect
-	modernc.org/gc/v3 v3.0.0-20241004144649-1aea3fae8852 // indirect
-	modernc.org/libc v1.61.0 // indirect
-	modernc.org/mathutil v1.6.0 // indirect
-	modernc.org/memory v1.8.0 // indirect
-	modernc.org/sqlite v1.33.1 // indirect
-	modernc.org/strutil v1.2.0 // indirect
-	modernc.org/token v1.1.0 // indirect
+	golang.org/x/image v0.29.0 // indirect
+	golang.org/x/net v0.42.0 // indirect
+	golang.org/x/oauth2 v0.30.0 // indirect
+	golang.org/x/sync v0.16.0 // indirect
+	golang.org/x/sys v0.34.0 // indirect
+	golang.org/x/text v0.27.0 // indirect
+	modernc.org/libc v1.65.10 // indirect
+	modernc.org/mathutil v1.7.1 // indirect
+	modernc.org/memory v1.11.0 // indirect
+	modernc.org/sqlite v1.38.0 // indirect
 )

beszel/go.sum

@@ -1,185 +1,77 @@
-cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
-cloud.google.com/go v0.116.0 h1:B3fRrSDkLRt5qSHWe40ERJvhvnQwdZiHu0bJOpldweE=
-cloud.google.com/go v0.116.0/go.mod h1:cEPSRWPzZEswwdr9BxE6ChEn01dWlTaF05LiC2Xs70U=
-cloud.google.com/go/auth v0.9.8 h1:+CSJ0Gw9iVeSENVCKJoLHhdUykDgXSc4Qn+gu2BRtR8=
-cloud.google.com/go/auth v0.9.8/go.mod h1:xxA5AqpDrvS+Gkmo9RqrGGRh6WSNKKOXhY3zNOr38tI=
-cloud.google.com/go/auth/oauth2adapt v0.2.4 h1:0GWE/FUsXhf6C+jAkWgYm7X9tK8cuEIfy19DBn6B6bY=
-cloud.google.com/go/auth/oauth2adapt v0.2.4/go.mod h1:jC/jOpwFP6JBxhB3P5Rr0a9HLMC/Pe3eaL4NmdvqPtc=
-cloud.google.com/go/compute v1.14.0 h1:hfm2+FfxVmnRlh6LpB7cg1ZNU+5edAHmW679JePztk0=
-cloud.google.com/go/compute/metadata v0.5.2 h1:UxK4uu/Tn+I3p2dYWTfiX4wva7aYlKixAHn3fyqngqo=
-cloud.google.com/go/compute/metadata v0.5.2/go.mod h1:C66sj2AluDcIqakBq/M8lw8/ybHgOZqin2obFxa/E5k=
-cloud.google.com/go/iam v1.1.13 h1:7zWBXG9ERbMLrzQBRhFliAV+kjcRToDTgQT3CTwYyv4=
-cloud.google.com/go/iam v1.1.13/go.mod h1:K8mY0uSXwEXS30KrnVb+j54LB/ntfZu1dr+4zFMNbus=
-cloud.google.com/go/storage v1.43.0 h1:CcxnSohZwizt4LCzQHWvBf1/kvtHUn7gk9QERXPyXFs=
-cloud.google.com/go/storage v1.43.0/go.mod h1:ajvxEa7WmZS1PxvKRq4bq0tFT3vMd502JwstCcYv0Q0=
 filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
 filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
-github.com/AlecAivazis/survey/v2 v2.3.7 h1:6I/u8FvytdGsgonrYsVn2t8t4QiRnh6QSTqkkhIiSjQ=
-github.com/AlecAivazis/survey/v2 v2.3.7/go.mod h1:xUTIdE4KCOIjsBAE1JYsUPoCqYdZ1reCfTwbto0Fduo=
-github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
-github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2 h1:+vx7roKuyA63nhn5WAunQHLTznkw5W8b1Xc0dNjp83s=
-github.com/Netflix/go-expect v0.0.0-20220104043353-73e0943537d2/go.mod h1:HBCaDeC1lPdgDeDbhX8XFpy1jqjK0IBG8W5K+xYqA0w=
 github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
 github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
 github.com/asaskevich/govalidator v0.0.0-20200108200545-475eaeb16496/go.mod h1:oGkLhpf+kjZl6xBf758TQhh5XrAeiJv/7FRz/2spLIg=
 github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
 github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
-github.com/aws/aws-sdk-go v1.55.5 h1:KKUZBfBoyqy5d3swXyiC7Q76ic40rYcbqH7qjh59kzU=
-github.com/aws/aws-sdk-go v1.55.5/go.mod h1:eRwEWoyTWFMVYVQzKMNHWP5/RV4xIUGMQfXQHfHkpNU=
-github.com/aws/aws-sdk-go-v2 v1.32.2 h1:AkNLZEyYMLnx/Q/mSKkcMqwNFXMAvFto9bNsHqcTduI=
-github.com/aws/aws-sdk-go-v2 v1.32.2/go.mod h1:2SK5n0a2karNTv5tbP1SjsX0uhttou00v/HpXKM1ZUo=
-github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.6 h1:pT3hpW0cOHRJx8Y0DfJUEQuqPild8jRGmSFmBgvydr0=
-github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.6/go.mod h1:j/I2++U0xX+cr44QjHay4Cvxj6FUbnxrgmqN3H1jTZA=
-github.com/aws/aws-sdk-go-v2/config v1.28.0 h1:FosVYWcqEtWNxHn8gB/Vs6jOlNwSoyOCA/g/sxyySOQ=
-github.com/aws/aws-sdk-go-v2/config v1.28.0/go.mod h1:pYhbtvg1siOOg8h5an77rXle9tVG8T+BWLWAo7cOukc=
-github.com/aws/aws-sdk-go-v2/credentials v1.17.41 h1:7gXo+Axmp+R4Z+AK8YFQO0ZV3L0gizGINCOWxSLY9W8=
-github.com/aws/aws-sdk-go-v2/credentials v1.17.41/go.mod h1:u4Eb8d3394YLubphT4jLEwN1rLNq2wFOlT6OuxFwPzU=
-github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.17 h1:TMH3f/SCAWdNtXXVPPu5D6wrr4G5hI1rAxbcocKfC7Q=
-github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.17/go.mod h1:1ZRXLdTpzdJb9fwTMXiLipENRxkGMTn1sfKexGllQCw=
-github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.33 h1:X+4YY5kZRI/cOoSMVMGTqFXHAMg1bvvay7IBcqHpybQ=
-github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.33/go.mod h1:DPynzu+cn92k5UQ6tZhX+wfTB4ah6QDU/NgdHqatmvk=
-github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.21 h1:UAsR3xA31QGf79WzpG/ixT9FZvQlh5HY1NRqSHBNOCk=
-github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.21/go.mod h1:JNr43NFf5L9YaG3eKTm7HQzls9J+A9YYcGI5Quh1r2Y=
-github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.21 h1:6jZVETqmYCadGFvrYEQfC5fAQmlo80CeL5psbno6r0s=
-github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.21/go.mod h1:1SR0GbLlnN3QUmYaflZNiH1ql+1qrSiB2vwcJ+4UM60=
-github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 h1:VaRN3TlFdd6KxX1x3ILT5ynH6HvKgqdiXoTxAF4HQcQ=
-github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1/go.mod h1:FbtygfRFze9usAadmnGJNc8KsP346kEe+y2/oyhGAGc=
-github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.21 h1:7edmS3VOBDhK00b/MwGtGglCm7hhwNYnjJs/PgFdMQE=
-github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.21/go.mod h1:Q9o5h4HoIWG8XfzxqiuK/CGUbepCJ8uTlaE3bAbxytQ=
-github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0 h1:TToQNkvGguu209puTojY/ozlqy2d/SFNcoLIqTFi42g=
-github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0/go.mod h1:0jp+ltwkf+SwG2fm/PKo8t4y8pJSgOCO4D8Lz3k0aHQ=
-github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.2 h1:4FMHqLfk0efmTqhXVRL5xYRqlEBNBiRI7N6w4jsEdd4=
-github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.2/go.mod h1:LWoqeWlK9OZeJxsROW2RqrSPvQHKTpp69r/iDjwsSaw=
-github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.2 h1:s7NA1SOw8q/5c0wr8477yOPp0z+uBaXBnLE0XYb0POA=
-github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.2/go.mod h1:fnjjWyAW/Pj5HYOxl9LJqWtEwS7W2qgcRLWP+uWbss0=
-github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.2 h1:t7iUP9+4wdc5lt3E41huP+GvQZJD38WLsgVp4iOtAjg=
-github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.2/go.mod h1:/niFCtmuQNxqx9v8WAPq5qh7EH25U4BF6tjoyq9bObM=
-github.com/aws/aws-sdk-go-v2/service/s3 v1.66.0 h1:xA6XhTF7PE89BCNHJbQi8VvPzcgMtmGC5dr8S8N7lHk=
-github.com/aws/aws-sdk-go-v2/service/s3 v1.66.0/go.mod h1:cB6oAuus7YXRZhWCc1wIwPywwZ1XwweNp2TVAEGYeB8=
-github.com/aws/aws-sdk-go-v2/service/sso v1.24.2 h1:bSYXVyUzoTHoKalBmwaZxs97HU9DWWI3ehHSAMa7xOk=
-github.com/aws/aws-sdk-go-v2/service/sso v1.24.2/go.mod h1:skMqY7JElusiOUjMJMOv1jJsP7YUg7DrhgqZZWuzu1U=
-github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.2 h1:AhmO1fHINP9vFYUE0LHzCWg/LfUWUF+zFPEcY9QXb7o=
-github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.2/go.mod h1:o8aQygT2+MVP0NaV6kbdE1YnnIM8RRVQzoeUH45GOdI=
-github.com/aws/aws-sdk-go-v2/service/sts v1.32.2 h1:CiS7i0+FUe+/YY1GvIBLLrR/XNGZ4CtM1Ll0XavNuVo=
-github.com/aws/aws-sdk-go-v2/service/sts v1.32.2/go.mod h1:HtaiBI8CjYoNVde8arShXb94UbQQi9L4EMr6D+xGBwo=
-github.com/aws/smithy-go v1.22.0 h1:uunKnWlcoL3zO7q+gG2Pk53joueEOsnNB28QdMsmiMM=
-github.com/aws/smithy-go v1.22.0/go.mod h1:irrKGvNn1InZwb2d7fkIRNucdfwR8R+Ts3wxYa/cJHg=
 github.com/blang/semver v3.5.1+incompatible h1:cQNTCjp13qL8KC3Nbxr/y2Bqb63oX6wdnnjpJbkM4JQ=
 github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
-github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
-github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
-github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
-github.com/containrrr/shoutrrr v0.8.0 h1:mfG2ATzIS7NR2Ec6XL+xyoHzN97H8WPjir8aYzJUSec=
-github.com/containrrr/shoutrrr v0.8.0/go.mod h1:ioyQAyu1LJY6sILuNyKaQaw+9Ttik5QePU8atnAdO2o=
-github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
-github.com/creack/pty v1.1.17 h1:QeVUsEDNrLBW4tMgZHvxy18sKtr6VI492kBhUfhDJNI=
-github.com/creack/pty v1.1.17/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
+github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
 github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1ei82L+c=
 github.com/disintegration/imaging v1.6.2/go.mod h1:44/5580QXChDfwIclfc/PCwrr44amcmDAg8hxG0Ewe4=
+github.com/dolthub/maphash v0.1.0 h1:bsQ7JsF4FkkWyrP3oCnFJgrCUAFbFf3kOl4L/QxPDyQ=
+github.com/dolthub/maphash v0.1.0/go.mod h1:gkg4Ch4CdCDu5h6PMriVLawB7koZ+5ijb9puGMV50a4=
 github.com/domodwyer/mailyak/v3 v3.6.2 h1:x3tGMsyFhTCaxp6ycgR0FE/bu5QiNp+hetUuCOBXMn8=
 github.com/domodwyer/mailyak/v3 v3.6.2/go.mod h1:lOm/u9CyCVWHeaAmHIdF4RiKVxKUT/H5XX10lIKAL6c=
 github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
 github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
-github.com/ebitengine/purego v0.8.0 h1:JbqvnEzRvPpxhCJzJJ2y0RbiZ8nyjccVUrSM3q+GvvE=
-github.com/ebitengine/purego v0.8.0/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
-github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
-github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
-github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
-github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
-github.com/fatih/color v1.17.0 h1:GlRw1BRJxkpqUCBKzKOw098ed57fEsKeNjpTe3cSjK4=
-github.com/fatih/color v1.17.0/go.mod h1:YZ7TlrGPkiz6ku9fK3TLD/pl3CpsiFyu8N92HLgmosI=
-github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
-github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
+github.com/ebitengine/purego v0.8.4 h1:CF7LEKg5FFOsASUj0+QwaXf8Ht6TlFxg09+S9wz0omw=
+github.com/ebitengine/purego v0.8.4/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
+github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
+github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
 github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
 github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
 github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
-github.com/gabriel-vasile/mimetype v1.4.6 h1:3+PzJTKLkvgjeTbts6msPJt4DixhT4YtFNf1gtGe3zc=
-github.com/gabriel-vasile/mimetype v1.4.6/go.mod h1:JX1qVKqZd40hUPpAfiNTe0Sne7hdfKSbOqqmkq8GCXc=
-github.com/ganigeorgiev/fexpr v0.4.1 h1:hpUgbUEEWIZhSDBtf4M9aUNfQQ0BZkGRaMePy7Gcx5k=
-github.com/ganigeorgiev/fexpr v0.4.1/go.mod h1:RyGiGqmeXhEQ6+mlGdnUleLHgtzzu/VGO2WtJkF5drE=
-github.com/gliderlabs/ssh v0.3.7 h1:iV3Bqi942d9huXnzEF2Mt+CY9gLu8DNM4Obd+8bODRE=
-github.com/gliderlabs/ssh v0.3.7/go.mod h1:zpHEXBstFnQYtGnB8k8kQLol82umzn/2/snG7alWVD8=
+github.com/fxamacker/cbor/v2 v2.9.0 h1:NpKPmjDBgUfBms6tr6JZkTHtfFGcMKsw3eGcmD/sapM=
+github.com/fxamacker/cbor/v2 v2.9.0/go.mod h1:vM4b+DJCtHn+zz7h3FFp/hDAI9WNWCsZj23V5ytsSxQ=
+github.com/gabriel-vasile/mimetype v1.4.9 h1:5k+WDwEsD9eTLL8Tz3L0VnmVh9QxGjRmjBvAG7U/oYY=
+github.com/gabriel-vasile/mimetype v1.4.9/go.mod h1:WnSQhFKJuBlRyLiKohA/2DtIlPFAbguNaG7QCHcyGok=
+github.com/ganigeorgiev/fexpr v0.5.0 h1:XA9JxtTE/Xm+g/JFI6RfZEHSiQlk+1glLvRK1Lpv/Tk=
+github.com/ganigeorgiev/fexpr v0.5.0/go.mod h1:RyGiGqmeXhEQ6+mlGdnUleLHgtzzu/VGO2WtJkF5drE=
+github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c=
+github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU=
 github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
 github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
-github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
-github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
 github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
 github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
 github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
 github.com/go-ozzo/ozzo-validation/v4 v4.3.0 h1:byhDUpfEwjsVQb1vBunvIjh2BHQ9ead57VkAEY4V+Es=
 github.com/go-ozzo/ozzo-validation/v4 v4.3.0/go.mod h1:2NKgrcHl3z6cJs+3Oo940FPRiTzuqKbvfrL2RxCj6Ew=
 github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
-github.com/go-sql-driver/mysql v1.8.1 h1:LedoTUt/eveggdHS9qUFC1EFSa8bU2+1pZjSRpvNJ1Y=
-github.com/go-sql-driver/mysql v1.8.1/go.mod h1:wEBSXgmK//2ZFJyE+qWnIsVGmvmEKlqwuVSjsCm7DZg=
-github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
-github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
-github.com/goccy/go-json v0.10.3 h1:KZ5WoDbxAIgm2HNbYckL0se1fHD6rz5j4ywS6ebzDqA=
-github.com/goccy/go-json v0.10.3/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
-github.com/golang-jwt/jwt/v4 v4.5.0 h1:7cYmW1XlMY7h7ii7UhUyChSgS5wUJEnm9uZVTGqOWzg=
-github.com/golang-jwt/jwt/v4 v4.5.0/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
-github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
-github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
-github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/go-sql-driver/mysql v1.9.1 h1:FrjNGn/BsJQjVRuSa8CBrM5BWA9BWoXXat3KrtSb/iI=
+github.com/go-sql-driver/mysql v1.9.1/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
+github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
+github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
+github.com/golang-jwt/jwt/v5 v5.2.3 h1:kkGXqQOBSDDWRhWNXTFpqGSCMyh/PLnqUvMGJPDJDs0=
+github.com/golang-jwt/jwt/v5 v5.2.3/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
 github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
 github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
 github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
-github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
-github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
-github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
-github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
-github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
-github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
-github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
-github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
-github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
-github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
-github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
-github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
-github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
-github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
+github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
 github.com/google/go-github/v30 v30.1.0 h1:VLDx+UolQICEOKu2m4uAoMti1SxuEBAl7RSEG16L+Oo=
 github.com/google/go-github/v30 v30.1.0/go.mod h1:n8jBpHl45a/rlBUtRJMOG4GhNADUQFEufcolZ95JfU8=
 github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
 github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
 github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
-github.com/google/pprof v0.0.0-20240827171923-fa2c70bbbfe5 h1:5iH8iuqE5apketRbSFBy+X1V0o+l+8NF1avt4HWl7cA=
-github.com/google/pprof v0.0.0-20240827171923-fa2c70bbbfe5/go.mod h1:vavhavw2zAxS5dIdcRluK6cSGGPlZynqzFM8NdvU144=
-github.com/google/s2a-go v0.1.8 h1:zZDs9gcbt9ZPLV0ndSyQk6Kacx2g/X+SKYovpnz3SMM=
-github.com/google/s2a-go v0.1.8/go.mod h1:6iNWHTpQ+nfNRN5E00MSdfDwVesa8hhS32PhPO8deJA=
-github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 h1:BHT72Gu3keYf3ZEu2J0b1vyeLSOYI8bm5wbJM/8yDe8=
+github.com/google/pprof v0.0.0-20250403155104-27863c87afa6/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
 github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
 github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
-github.com/google/wire v0.6.0 h1:HBkoIh4BdSxoyo9PveV8giw7ZsaBOvzWKfcg/6MrVwI=
-github.com/google/wire v0.6.0/go.mod h1:F4QhpQ9EDIdJ1Mbop/NZBRB+5yrR6qg3BnctaoUk6NA=
-github.com/googleapis/enterprise-certificate-proxy v0.3.4 h1:XYIDZApgAnrN1c855gTgghdIA6Stxb52D5RnLI1SLyw=
-github.com/googleapis/enterprise-certificate-proxy v0.3.4/go.mod h1:YKe7cfqYXjKGpGvmSg28/fFvhNzinZQm8DGnaburhGA=
-github.com/googleapis/gax-go/v2 v2.13.0 h1:yitjD5f7jQHhyDsnhKEBU52NdvvdSeGzlAnDPT0hH1s=
-github.com/googleapis/gax-go/v2 v2.13.0/go.mod h1:Z/fvTZXF8/uw7Xu5GuslPw+bplx6SS338j1Is2S+B7A=
-github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
-github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
-github.com/hinshun/vt10x v0.0.0-20220119200601-820417d04eec h1:qv2VnGeEQHchGaZ/u7lxST/RaJw+cv273q79D81Xbog=
-github.com/hinshun/vt10x v0.0.0-20220119200601-820417d04eec/go.mod h1:Q48J4R4DvxnHolD5P8pOtXigYlRuPLGl6moFx3ulM68=
 github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
 github.com/inconshreveable/go-update v0.0.0-20160112193335-8152e7eb6ccf h1:WfD7VjIE6z8dIvMsI4/s+1qr5EL+zoIGev1BQj1eoJ8=
 github.com/inconshreveable/go-update v0.0.0-20160112193335-8152e7eb6ccf/go.mod h1:hyb9oH7vZsitZCiBt0ZvifOrB+qc8PS5IiilCIb87rg=
 github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
 github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
-github.com/jarcoal/httpmock v1.3.0 h1:2RJ8GP0IIaWwcC9Fp2BmVi8Kog3v2Hn7VXM3fTd+nuc=
-github.com/jarcoal/httpmock v1.3.0/go.mod h1:3yb8rc4BI7TCBhFY8ng0gjuLKJNquuDNiPaZjnENuYg=
-github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
-github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
-github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
-github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
+github.com/jarcoal/httpmock v1.4.0 h1:BvhqnH0JAYbNudL2GMJKgOHe2CtKlzJ/5rWKyp+hc2k=
+github.com/jarcoal/httpmock v1.4.0/go.mod h1:ftW1xULwo+j0R0JJkJIIi7UKigZUXCLLanykgjwBXL0=
+github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
+github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
 github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
 github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
 github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
@@ -187,41 +79,34 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
-github.com/labstack/echo/v5 v5.0.0-20230722203903-ec5b858dab61 h1:FwuzbVh87iLiUQj1+uQUsuw9x5t9m5n5g7rG7o4svW4=
-github.com/labstack/echo/v5 v5.0.0-20230722203903-ec5b858dab61/go.mod h1:paQfF1YtHe+GrGg5fOgjsjoCX/UKDr9bc1DoWpZfns8=
-github.com/lufia/plan9stats v0.0.0-20240909124753-873cd0166683 h1:7UMa6KCCMjZEMDtTVdcGu0B1GmmC7QJKiCCjyTAWQy0=
-github.com/lufia/plan9stats v0.0.0-20240909124753-873cd0166683/go.mod h1:ilwx/Dta8jXAgpFYFvSWEMwxmbWXyiUHkd5FwyKhb5k=
-github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
-github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
-github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
-github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
-github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
+github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35 h1:PpXWgLPs+Fqr325bN2FD2ISlRRztXibcX6e8f5FR5Dc=
+github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg=
+github.com/lxzan/gws v1.8.9 h1:VU3SGUeWlQrEwfUSfokcZep8mdg/BrUF+y73YYshdBM=
+github.com/lxzan/gws v1.8.9/go.mod h1:d9yHaR1eDTBHagQC6KY7ycUOaz5KWeqQtP3xu7aMK8Y=
+github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
+github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
 github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
 github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
-github.com/mattn/go-sqlite3 v1.14.24 h1:tpSp2G2KyMnnQu99ngJ47EIkWVmliIizyZBfPrBWDRM=
-github.com/mattn/go-sqlite3 v1.14.24/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
-github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
-github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d h1:5PJl274Y63IEHC+7izoQE9x6ikvDFZS2mDVS3drnohI=
-github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
 github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
 github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
+github.com/nicholas-fedor/shoutrrr v0.8.8 h1:F/oyoatWK5cbHPPgkjRZrA0262TP7KWuUQz9KskRtR8=
+github.com/nicholas-fedor/shoutrrr v0.8.8/go.mod h1:T30Y+eoZFEjDk4HtOItcHQioZSOe3Z6a6aNfSz6jc5c=
 github.com/onsi/ginkgo v1.6.0 h1:Ix8l273rp3QzYgXSR+c8d1fTG7UPgYkOSELPhiY/YGw=
 github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
-github.com/onsi/ginkgo/v2 v2.9.2 h1:BA2GMJOtfGAfagzYtrAlufIP0lq6QERkFmHLMLPwFSU=
-github.com/onsi/ginkgo/v2 v2.9.2/go.mod h1:WHcJJG2dIlcCqVfBAwUCrJxSPFb6v4azBwgxeMeDuts=
+github.com/onsi/ginkgo/v2 v2.23.4 h1:ktYTpKJAVZnDT4VjxSbiBenUjmlL/5QkBEocaWXiQus=
+github.com/onsi/ginkgo/v2 v2.23.4/go.mod h1:Bt66ApGPBFzHyR+JO10Zbt0Gsp4uWxu5mIOTusL46e8=
 github.com/onsi/gomega v1.4.2/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
-github.com/onsi/gomega v1.27.6 h1:ENqfyGeS5AX/rlXDd/ETokDz93u0YufY1Pgxuy/PvWE=
-github.com/onsi/gomega v1.27.6/go.mod h1:PIQNjfQwkP3aQAH7lf7j87O/5FiNr+ZR8+ipb+qQlhg=
+github.com/onsi/gomega v1.37.0 h1:CdEG8g0S133B4OswTDC/5XPSzE1OeP29QOioj2PID2Y=
+github.com/onsi/gomega v1.37.0/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
 github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/pocketbase/dbx v1.10.1 h1:cw+vsyfCJD8YObOVeqb93YErnlxwYMkNZ4rwN0G0AaA=
-github.com/pocketbase/dbx v1.10.1/go.mod h1:xXRCIAKTHMgUCyCKZm55pUOdvFziJjQfXaWKhu2vhMs=
-github.com/pocketbase/pocketbase v0.22.22 h1:iA128U+cmM9euxPpuCN7blmQ2FZNzOix2aUUcnbbQu8=
-github.com/pocketbase/pocketbase v0.22.22/go.mod h1:u+l7T04g7eBXetoodXLch3WoV/QonRf1qYq+2vuTKuI=
+github.com/pocketbase/dbx v1.11.0 h1:LpZezioMfT3K4tLrqA55wWFw1EtH1pM4tzSVa7kgszU=
+github.com/pocketbase/dbx v1.11.0/go.mod h1:xXRCIAKTHMgUCyCKZm55pUOdvFziJjQfXaWKhu2vhMs=
+github.com/pocketbase/pocketbase v0.29.0 h1:oL6qvkU2QSybClVtQdaq9Z1F3Wk59iKYCfIaf1R8KUs=
+github.com/pocketbase/pocketbase v0.29.0/go.mod h1:SqyH7o/3e+/uLySATlJqxH4S8gyU6R0adG56ZSV1vuU=
 github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU=
 github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
-github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
 github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
 github.com/rhysd/go-github-selfupdate v1.2.3 h1:iaa+J202f+Nc+A8zi75uccC8Wg3omaM7HDeimXA22Ag=
@@ -229,174 +114,85 @@ github.com/rhysd/go-github-selfupdate v1.2.3/go.mod h1:mp/N8zj6jFfBQy/XMYoWsmfzx
 github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
 github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
-github.com/shirou/gopsutil/v4 v4.24.9 h1:KIV+/HaHD5ka5f570RZq+2SaeFsb/pq+fp2DGNWYoOI=
-github.com/shirou/gopsutil/v4 v4.24.9/go.mod h1:3fkaHNeYsUFCGZ8+9vZVWtbyM1k2eRnlL+bWO8Bxa/Q=
-github.com/spf13/cast v1.7.0 h1:ntdiHjuueXFgm5nzDRdOS4yfT43P5Fnud6DH50rz/7w=
-github.com/spf13/cast v1.7.0/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
-github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=
-github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y=
-github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
-github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/shirou/gopsutil/v4 v4.25.6 h1:kLysI2JsKorfaFPcYmcJqbzROzsBWEOAtw6A7dIfqXs=
+github.com/shirou/gopsutil/v4 v4.25.6/go.mod h1:PfybzyydfZcN+JMMjkF6Zb8Mq1A/VcogFFg7hj50W9c=
+github.com/spf13/cast v1.9.2 h1:SsGfm7M8QOFtEzumm7UZrZdLLquNdzFYfIbEXntcFbE=
+github.com/spf13/cast v1.9.2/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
+github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
+github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
+github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/spf13/pflag v1.0.7 h1:vN6T9TfwStFPFM5XzjsvmzZkLuaLX+HS+0SeFLRgU6M=
+github.com/spf13/pflag v1.0.7/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
-github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
-github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
 github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
-github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
-github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
-github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
-github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
-github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
-github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
+github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
 github.com/tcnksm/go-gitconfig v0.1.2 h1:iiDhRitByXAEyjgBqsKi9QU4o2TNtv9kPP3RgPgXBPw=
 github.com/tcnksm/go-gitconfig v0.1.2/go.mod h1:/8EhP4H7oJZdIPyT+/UIsG87kTzrzM4UsLGSItWYCpE=
-github.com/tklauser/go-sysconf v0.3.14 h1:g5vzr9iPFFz24v2KZXs/pvpvh8/V9Fw6vQK5ZZb78yU=
-github.com/tklauser/go-sysconf v0.3.14/go.mod h1:1ym4lWMLUOhuBOPGtRcJm7tEGX4SCYNEEEtghGG/8uY=
-github.com/tklauser/numcpus v0.8.0 h1:Mx4Wwe/FjZLeQsK/6kt2EOepwwSl7SmJrK5bV/dXYgY=
-github.com/tklauser/numcpus v0.8.0/go.mod h1:ZJZlAY+dmR4eut8epnzf0u/VwodKmryxR8txiloSqBE=
+github.com/tklauser/go-sysconf v0.3.15 h1:VE89k0criAymJ/Os65CSn1IXaol+1wrsFHEB8Ol49K4=
+github.com/tklauser/go-sysconf v0.3.15/go.mod h1:Dmjwr6tYFIseJw7a3dRLJfsHAMXZ3nEnL/aZY+0IuI4=
+github.com/tklauser/numcpus v0.10.0 h1:18njr6LDBk1zuna922MgdjQuJFjrdppsZG60sHGfjso=
+github.com/tklauser/numcpus v0.10.0/go.mod h1:BiTKazU708GQTYF4mB+cmlpT2Is1gLk7XVuEeem8LsQ=
 github.com/ulikunitz/xz v0.5.9/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
 github.com/ulikunitz/xz v0.5.12 h1:37Nm15o69RwBkXM0J6A5OlE67RZTfzUxTj8fB3dfcsc=
 github.com/ulikunitz/xz v0.5.12/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
-github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
-github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
-github.com/valyala/fasttemplate v1.2.2 h1:lxLXG0uE3Qnshl9QyaK6XJxMXlQZELvChBOCmQD0Loo=
-github.com/valyala/fasttemplate v1.2.2/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
-github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
+github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
+github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
 github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
 github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
-go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
-go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
-go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.54.0 h1:r6I7RJCN86bpD/FQwedZ0vSixDpwuWREjW9oRMsmqDc=
-go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.54.0/go.mod h1:B9yO6b04uB80CzjedvewuqDhxJxi11s7/GtiGa8bAjI=
-go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0 h1:TT4fX+nBOA/+LUkobKGW1ydGcn+G3vRw9+g5HwCphpk=
-go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0/go.mod h1:L7UH0GbB0p47T4Rri3uHjbpCFYrVrwc1I25QhNPiGK8=
-go.opentelemetry.io/otel v1.29.0 h1:PdomN/Al4q/lN6iBJEN3AwPvUiHPMlt93c8bqTG5Llw=
-go.opentelemetry.io/otel v1.29.0/go.mod h1:N/WtXPs1CNCUEx+Agz5uouwCba+i+bJGFicT8SR4NP8=
-go.opentelemetry.io/otel/metric v1.29.0 h1:vPf/HFWTNkPu1aYeIsc98l4ktOQaL6LeSoeV2g+8YLc=
-go.opentelemetry.io/otel/metric v1.29.0/go.mod h1:auu/QWieFVWx+DmQOUMgj0F8LHWdgalxXqvp7BII/W8=
-go.opentelemetry.io/otel/trace v1.29.0 h1:J/8ZNK4XgR7a21DZUAsbF8pZ5Jcw1VhACmnYt39JTi4=
-go.opentelemetry.io/otel/trace v1.29.0/go.mod h1:eHl3w0sp3paPkYstJOmAimxhiFXPg+MMTlEh3nsQgWQ=
-gocloud.dev v0.40.0 h1:f8LgP+4WDqOG/RXoUcyLpeIAGOcAbZrZbDQCUee10ng=
-gocloud.dev v0.40.0/go.mod h1:drz+VyYNBvrMTW0KZiBAYEdl8lbNZx+OQ7oQvdrFmSQ=
+go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
+go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
-golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
-golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
-golang.org/x/crypto v0.28.0 h1:GBDwsMXVQi34v5CCYUm2jkJvu4cbtru2U4TN2PSyQnw=
-golang.org/x/crypto v0.28.0/go.mod h1:rmgy+3RHxRZMyY0jjAJShp2zgEdOqj2AO7U0pYmeQ7U=
-golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
-golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c h1:7dEasQXItcW1xKJ2+gg5VOiBnqWrJc+rq0DPKyvvdbY=
-golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c/go.mod h1:NQtJDoLvd6faHhE7m4T/1IY708gDefGGjR/iUW8yQQ8=
+golang.org/x/crypto v0.40.0 h1:r4x+VvoG5Fm+eJcxMaY8CQM7Lb0l1lsmjGBQ6s8BfKM=
+golang.org/x/crypto v0.40.0/go.mod h1:Qr1vMER5WyS2dfPHAlsOj01wgLbsyWtFn/aY+5+ZdxY=
+golang.org/x/exp v0.0.0-20250718183923-645b1fa84792 h1:R9PFI6EUdfVKgwKjZef7QIwGcBKu86OEFpJ9nUEP2l4=
+golang.org/x/exp v0.0.0-20250718183923-645b1fa84792/go.mod h1:A+z0yzpGtvnG90cToK5n2tu8UJVP2XUATh+r+sfOOOc=
 golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
-golang.org/x/image v0.21.0 h1:c5qV36ajHpdj4Qi0GnE0jUc/yuo33OLFaa0d+crTD5s=
-golang.org/x/image v0.21.0/go.mod h1:vUbsLavqK/W303ZroQQVKQ+Af3Yl6Uz1Ppu5J/cLz78=
-golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
-golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
-golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
-golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
-golang.org/x/mod v0.21.0 h1:vvrHzRwRfVKSiLrG+d4FMl/Qi4ukBCE6kZlTUkDYRT0=
-golang.org/x/mod v0.21.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY=
+golang.org/x/image v0.29.0 h1:HcdsyR4Gsuys/Axh0rDEmlBmB68rW1U9BUdB3UVHsas=
+golang.org/x/image v0.29.0/go.mod h1:RVJROnf3SLK8d26OW91j4FrIHGbsJ8QnbEocVTOWQDA=
+golang.org/x/mod v0.26.0 h1:EGMPT//Ezu+ylkCijjPc+f4Aih7sZvaAr+O3EHBxvZg=
+golang.org/x/mod v0.26.0/go.mod h1:/j6NAhSk8iQ723BGAUyoAcn7SlD7s15Dp9Nd/SfeaFQ=
 golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
-golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
 golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
-golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
-golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
-golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
-golang.org/x/net v0.30.0 h1:AcW1SDZMkb8IpzCdQUaIq2sP4sZ4zw+55h6ynffypl4=
-golang.org/x/net v0.30.0/go.mod h1:2wGyMJ5iFasEhkwi13ChkO/t1ECNC4X4eBKkVFyYFlU=
+golang.org/x/net v0.42.0 h1:jzkYrhi3YQWD6MLBJcsklgQsoAcw89EcZbJw8Z614hs=
+golang.org/x/net v0.42.0/go.mod h1:FF1RA5d3u7nAYA4z2TkclSCKh68eSXtiFwcWQpPXdt8=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
-golang.org/x/oauth2 v0.23.0 h1:PbgcYx2W7i4LvjJWEbf0ngHV6qJYr86PkAV3bXdLEbs=
-golang.org/x/oauth2 v0.23.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI=
+golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
+golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ=
-golang.org/x/sync v0.8.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
-golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
+golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
 golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.26.0 h1:KHjCJyddX0LoSTb3J+vWpupP9p0oznkqVk/IfjymZbo=
-golang.org/x/sys v0.26.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
+golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
 golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
-golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
-golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
-golang.org/x/term v0.25.0 h1:WtHI/ltw4NvSUig5KARz9h521QvRC8RmF/cuYqifU24=
-golang.org/x/term v0.25.0/go.mod h1:RPyXicDX+6vLxogjjRxjgD2TKtmAO6NZBsBRfrOLu7M=
+golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
+golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
-golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
-golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
-golang.org/x/text v0.19.0 h1:kTxAhCbGbxhK0IwgSKiMO5awPoDQ0RpfiVYBfK860YM=
-golang.org/x/text v0.19.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY=
-golang.org/x/time v0.7.0 h1:ntUhktv3OPE6TgYxXWv9vKvUSJyIFJlyohwbkEwPrKQ=
-golang.org/x/time v0.7.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
+golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
+golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
-golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
-golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
-golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
-golang.org/x/tools v0.26.0 h1:v/60pFQmzmT9ExmjDv2gGIfi3OqfKoEP6I5+umXlbnQ=
-golang.org/x/tools v0.26.0/go.mod h1:TPVVj70c7JJ3WCazhD8OdXcZg/og+b9+tH/KxylGwH0=
-golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/tools v0.35.0 h1:mBffYraMEf7aa0sB+NuKnuCy8qI/9Bughn8dC2Gu5r0=
+golang.org/x/tools v0.35.0/go.mod h1:NKdj5HkL/73byiZSJjqJgKn3ep7KjFkBOkR/Hps3VPw=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
-golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da h1:noIWHXmPHxILtqtCOPIhSt0ABwskkZKjD3bXGnZGpNY=
-golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=
-google.golang.org/api v0.201.0 h1:+7AD9JNM3tREtawRMu8sOjSbb8VYcYXJG/2eEOmfDu0=
-google.golang.org/api v0.201.0/go.mod h1:HVY0FCHVs89xIW9fzf/pBvOEm+OolHa86G/txFezyq4=
 google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
 google.golang.org/appengine v1.3.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
-google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
 google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
-google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
-google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
-google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
-google.golang.org/genproto v0.0.0-20241007155032-5fefd90f89a9 h1:nFS3IivktIU5Mk6KQa+v6RKkHUpdQpphqGNLxqNnbEk=
-google.golang.org/genproto v0.0.0-20241007155032-5fefd90f89a9/go.mod h1:tEzYTYZxbmVNOu0OAFH9HzdJtLn6h4Aj89zzlBCdHms=
-google.golang.org/genproto/googleapis/api v0.0.0-20240814211410-ddb44dafa142 h1:wKguEg1hsxI2/L3hUYrpo1RVi48K+uTyzKqprwLXsb8=
-google.golang.org/genproto/googleapis/api v0.0.0-20240814211410-ddb44dafa142/go.mod h1:d6be+8HhtEtucleCbxpPW9PA9XwISACu8nvpPqF0BVo=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20241015192408-796eee8c2d53 h1:X58yt85/IXCx0Y3ZwN6sEIKZzQtDEYaBWrDvErdXrRE=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20241015192408-796eee8c2d53/go.mod h1:GX3210XPVPUjJbTUbvwI8f2IpZDMZuPJWDzDuebbviI=
-google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
-google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
-google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
-google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
-google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
-google.golang.org/grpc v1.67.1 h1:zWnc1Vrcno+lHZCOofnIMvycFcc0QRGIzm9dhnDX68E=
-google.golang.org/grpc v1.67.1/go.mod h1:1gLDyUQU7CTLJI90u3nXZ9ekeghjeM7pTDZlqFNg2AA=
-google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
-google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
-google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
-google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
-google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
-google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
-google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
-google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
-google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
-google.golang.org/protobuf v1.35.1 h1:m3LfL6/Ca+fqnjnlqQXNpFPABW1UD7mjh8KO2mKFytA=
-google.golang.org/protobuf v1.35.1/go.mod h1:9fA7Ob0pmnwhb644+1+CVWFRbNajQ6iRojtC/QF5bRE=
+google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
+google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -404,34 +200,33 @@ gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMy
 gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
 gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
-modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ=
-modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ=
-modernc.org/ccgo/v4 v4.21.0 h1:kKPI3dF7RIag8YcToh5ZwDcVMIv6VGa0ED5cvh0LMW4=
-modernc.org/ccgo/v4 v4.21.0/go.mod h1:h6kt6H/A2+ew/3MW/p6KEoQmrq/i3pr0J/SiwiaF/g0=
-modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE=
-modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ=
-modernc.org/gc/v2 v2.5.0 h1:bJ9ChznK1L1mUtAQtxi0wi5AtAs5jQuw4PrPHO5pb6M=
-modernc.org/gc/v2 v2.5.0/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU=
-modernc.org/gc/v3 v3.0.0-20241004144649-1aea3fae8852 h1:IYXPPTTjjoSHvUClZIYexDiO7g+4x+XveKT4gCIAwiY=
-modernc.org/gc/v3 v3.0.0-20241004144649-1aea3fae8852/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4=
-modernc.org/libc v1.61.0 h1:eGFcvWpqlnoGwzZeZe3PWJkkKbM/3SUGyk1DVZQ0TpE=
-modernc.org/libc v1.61.0/go.mod h1:DvxVX89wtGTu+r72MLGhygpfi3aUGgZRdAYGCAVVud0=
-modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
-modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
-modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E=
-modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU=
-modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
-modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
-modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc=
-modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss=
-modernc.org/sqlite v1.33.1 h1:trb6Z3YYoeM9eDL1O8do81kP+0ejv+YzgyFo+Gwy0nM=
-modernc.org/sqlite v1.33.1/go.mod h1:pXV2xHxhzXZsgT/RtTFAPY6JJDEvOTcTdwADQCCWD4k=
-modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA=
-modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0=
+modernc.org/cc/v4 v4.26.2 h1:991HMkLjJzYBIfha6ECZdjrIYz2/1ayr+FL8GN+CNzM=
+modernc.org/cc/v4 v4.26.2/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
+modernc.org/ccgo/v4 v4.28.0 h1:rjznn6WWehKq7dG4JtLRKxb52Ecv8OUGah8+Z/SfpNU=
+modernc.org/ccgo/v4 v4.28.0/go.mod h1:JygV3+9AV6SmPhDasu4JgquwU81XAKLd3OKTUDNOiKE=
+modernc.org/fileutil v1.3.8 h1:qtzNm7ED75pd1C7WgAGcK4edm4fvhtBsEiI/0NQ54YM=
+modernc.org/fileutil v1.3.8/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
+modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
+modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
+modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
+modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
+modernc.org/libc v1.65.10 h1:ZwEk8+jhW7qBjHIT+wd0d9VjitRyQef9BnzlzGwMODc=
+modernc.org/libc v1.65.10/go.mod h1:StFvYpx7i/mXtBAfVOjaU0PWZOvIRoZSgXhrwXzr8Po=
+modernc.org/libc v1.66.3 h1:cfCbjTUcdsKyyZZfEUKfoHcP3S0Wkvz3jgSzByEWVCQ=
+modernc.org/libc v1.66.3/go.mod h1:XD9zO8kt59cANKvHPXpx7yS2ELPheAey0vjIuZOhOU8=
+modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
+modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
+modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
+modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
+modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
+modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
+modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
+modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
+modernc.org/sqlite v1.38.0 h1:+4OrfPQ8pxHKuWG4md1JpR/EYAh3Md7TdejuuzE7EUI=
+modernc.org/sqlite v1.38.0/go.mod h1:1Bj+yES4SVvBZ4cBOpVZ6QgesMCKpJZDq0nxYzOpmNE=
+modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
+modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
 modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
 modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=

beszel/internal/agent/agent.go

@@ -4,42 +4,62 @@ package agent
 import (
 	"beszel"
 	"beszel/internal/entities/system"
-	"context"
+	"crypto/sha256"
+	"encoding/hex"
 	"log/slog"
 	"os"
+	"path/filepath"
 	"strings"
+	"sync"
+	"time"
 
-	"github.com/shirou/gopsutil/v4/common"
+	"github.com/gliderlabs/ssh"
+	"github.com/shirou/gopsutil/v4/host"
+	gossh "golang.org/x/crypto/ssh"
 )
 
 type Agent struct {
-	debug            bool                       // true if LOG_LEVEL is set to debug
-	zfs              bool                       // true if system has arcstats
-	memCalc          string                     // Memory calculation formula
-	fsNames          []string                   // List of filesystem device names being monitored
-	fsStats          map[string]*system.FsStats // Keeps track of disk stats for each filesystem
-	netInterfaces    map[string]struct{}        // Stores all valid network interfaces
-	netIoStats       system.NetIoStats          // Keeps track of bandwidth usage
-	dockerManager    *dockerManager             // Manages Docker API requests
-	sensorsContext   context.Context            // Sensors context to override sys location
-	sensorsWhitelist map[string]struct{}        // List of sensors to monitor
-	systemInfo       system.Info                // Host system info
+	sync.Mutex                                   // Used to lock agent while collecting data
+	debug             bool                       // true if LOG_LEVEL is set to debug
+	zfs               bool                       // true if system has arcstats
+	memCalc           string                     // Memory calculation formula
+	fsNames           []string                   // List of filesystem device names being monitored
+	fsStats           map[string]*system.FsStats // Keeps track of disk stats for each filesystem
+	netInterfaces     map[string]struct{}        // Stores all valid network interfaces
+	netIoStats        system.NetIoStats          // Keeps track of bandwidth usage
+	dockerManager     *dockerManager             // Manages Docker API requests
+	sensorConfig      *SensorConfig              // Sensors config
+	systemInfo        system.Info                // Host system info
+	gpuManager        *GPUManager                // Manages GPU data
+	cache             *SessionCache              // Cache for system stats based on primary session ID
+	connectionManager *ConnectionManager         // Channel to signal connection events
+	server            *ssh.Server                // SSH server
+	dataDir           string                     // Directory for persisting data
+	keys              []gossh.PublicKey          // SSH public keys
 }
 
-func NewAgent() *Agent {
-	return &Agent{
-		sensorsContext: context.Background(),
-		memCalc:        os.Getenv("MEM_CALC"),
-		fsStats:        make(map[string]*system.FsStats),
+// NewAgent creates a new agent with the given data directory for persisting data.
+// If the data directory is not set, it will attempt to find the optimal directory.
+func NewAgent(dataDir ...string) (agent *Agent, err error) {
+	agent = &Agent{
+		fsStats: make(map[string]*system.FsStats),
+		cache:   NewSessionCache(69 * time.Second),
 	}
-}
 
-func (a *Agent) Run(pubKey []byte, addr string) {
+	agent.dataDir, err = getDataDir(dataDir...)
+	if err != nil {
+		slog.Warn("Data directory not found")
+	} else {
+		slog.Info("Data directory", "path", agent.dataDir)
+	}
+
+	agent.memCalc, _ = GetEnv("MEM_CALC")
+	agent.sensorConfig = agent.newSensorConfig()
 	// Set up slog with a log level determined by the LOG_LEVEL env var
-	if logLevelStr, exists := os.LookupEnv("LOG_LEVEL"); exists {
+	if logLevelStr, exists := GetEnv("LOG_LEVEL"); exists {
 		switch strings.ToLower(logLevelStr) {
 		case "debug":
-			a.debug = true
+			agent.debug = true
 			slog.SetLogLoggerLevel(slog.LevelDebug)
 		case "warn":
 			slog.SetLogLoggerLevel(slog.LevelWarn)
@@ -50,57 +70,113 @@ func (a *Agent) Run(pubKey []byte, addr string) {
 
 	slog.Debug(beszel.Version)
 
-	// Set sensors context (allows overriding sys location for sensors)
-	if sysSensors, exists := os.LookupEnv("SYS_SENSORS"); exists {
-		slog.Info("SYS_SENSORS", "path", sysSensors)
-		a.sensorsContext = context.WithValue(a.sensorsContext,
-			common.EnvKey, common.EnvMap{common.HostSysEnvKey: sysSensors},
-		)
-	}
+	// initialize system info
+	agent.initializeSystemInfo()
 
-	// Set sensors whitelist
-	if sensors, exists := os.LookupEnv("SENSORS"); exists {
-		a.sensorsWhitelist = make(map[string]struct{})
-		for _, sensor := range strings.Split(sensors, ",") {
-			a.sensorsWhitelist[sensor] = struct{}{}
-		}
-	}
+	// initialize connection manager
+	agent.connectionManager = newConnectionManager(agent)
 
-	// initialize system info / docker manager
-	a.initializeSystemInfo()
-	a.initializeDiskInfo()
-	a.initializeNetIoStats()
-	a.dockerManager = newDockerManager()
+	// initialize disk info
+	agent.initializeDiskInfo()
+
+	// initialize net io stats
+	agent.initializeNetIoStats()
+
+	// initialize docker manager
+	agent.dockerManager = newDockerManager(agent)
+
+	// initialize GPU manager
+	if gm, err := NewGPUManager(); err != nil {
+		slog.Debug("GPU", "err", err)
+	} else {
+		agent.gpuManager = gm
+	}
 
 	// if debugging, print stats
-	if a.debug {
-		slog.Debug("Stats", "data", a.gatherStats())
+	if agent.debug {
+		slog.Debug("Stats", "data", agent.gatherStats(""))
 	}
 
-	a.startServer(pubKey, addr)
+	return agent, nil
 }
 
-func (a *Agent) gatherStats() system.CombinedData {
-	slog.Debug("Getting stats")
-	systemData := system.CombinedData{
+// GetEnv retrieves an environment variable with a "BESZEL_AGENT_" prefix, or falls back to the unprefixed key.
+func GetEnv(key string) (value string, exists bool) {
+	if value, exists = os.LookupEnv("BESZEL_AGENT_" + key); exists {
+		return value, exists
+	}
+	// Fallback to the old unprefixed key
+	return os.LookupEnv(key)
+}
+
+func (a *Agent) gatherStats(sessionID string) *system.CombinedData {
+	a.Lock()
+	defer a.Unlock()
+
+	cachedData, ok := a.cache.Get(sessionID)
+	if ok {
+		slog.Debug("Cached stats", "session", sessionID)
+		return cachedData
+	}
+
+	*cachedData = system.CombinedData{
 		Stats: a.getSystemStats(),
 		Info:  a.systemInfo,
 	}
-	slog.Debug("System stats", "data", systemData)
-	// add docker stats
-	if containerStats, err := a.dockerManager.getDockerStats(); err == nil {
-		systemData.Containers = containerStats
-		slog.Debug("Docker stats", "data", systemData.Containers)
-	} else {
-		slog.Debug("Error getting docker stats", "err", err)
-	}
-	// add extra filesystems
-	systemData.Stats.ExtraFs = make(map[string]*system.FsStats)
-	for name, stats := range a.fsStats {
-		if !stats.Root && stats.DiskTotal > 0 {
-			systemData.Stats.ExtraFs[name] = stats
+	slog.Debug("System stats", "data", cachedData)
+
+	if a.dockerManager != nil {
+		if containerStats, err := a.dockerManager.getDockerStats(); err == nil {
+			cachedData.Containers = containerStats
+			slog.Debug("Docker stats", "data", cachedData.Containers)
+		} else {
+			slog.Debug("Docker stats", "err", err)
 		}
 	}
-	slog.Debug("Extra filesystems", "data", systemData.Stats.ExtraFs)
-	return systemData
+
+	cachedData.Stats.ExtraFs = make(map[string]*system.FsStats)
+	for name, stats := range a.fsStats {
+		if !stats.Root && stats.DiskTotal > 0 {
+			cachedData.Stats.ExtraFs[name] = stats
+		}
+	}
+	slog.Debug("Extra filesystems", "data", cachedData.Stats.ExtraFs)
+
+	a.cache.Set(sessionID, cachedData)
+	return cachedData
+}
+
+// StartAgent initializes and starts the agent with optional WebSocket connection
+func (a *Agent) Start(serverOptions ServerOptions) error {
+	a.keys = serverOptions.Keys
+	return a.connectionManager.Start(serverOptions)
+}
+
+func (a *Agent) getFingerprint() string {
+	// first look for a fingerprint in the data directory
+	if a.dataDir != "" {
+		if fp, err := os.ReadFile(filepath.Join(a.dataDir, "fingerprint")); err == nil {
+			return string(fp)
+		}
+	}
+
+	// if no fingerprint is found, generate one
+	fingerprint, err := host.HostID()
+	if err != nil || fingerprint == "" {
+		fingerprint = a.systemInfo.Hostname + a.systemInfo.CpuModel
+	}
+
+	// hash fingerprint
+	sum := sha256.Sum256([]byte(fingerprint))
+	fingerprint = hex.EncodeToString(sum[:24])
+
+	// save fingerprint to data directory
+	if a.dataDir != "" {
+		err = os.WriteFile(filepath.Join(a.dataDir, "fingerprint"), []byte(fingerprint), 0644)
+		if err != nil {
+			slog.Warn("Failed to save fingerprint", "err", err)
+		}
+	}
+
+	return fingerprint
 }

beszel/internal/agent/agent_cache.go

@@ -0,0 +1,36 @@
+package agent
+
+import (
+	"beszel/internal/entities/system"
+	"time"
+)
+
+// Not thread safe since we only access from gatherStats which is already locked
+type SessionCache struct {
+	data           *system.CombinedData
+	lastUpdate     time.Time
+	primarySession string
+	leaseTime      time.Duration
+}
+
+func NewSessionCache(leaseTime time.Duration) *SessionCache {
+	return &SessionCache{
+		leaseTime: leaseTime,
+		data:      &system.CombinedData{},
+	}
+}
+
+func (c *SessionCache) Get(sessionID string) (stats *system.CombinedData, isCached bool) {
+	if sessionID != c.primarySession && time.Since(c.lastUpdate) < c.leaseTime {
+		return c.data, true
+	}
+	return c.data, false
+}
+
+func (c *SessionCache) Set(sessionID string, data *system.CombinedData) {
+	if data != nil {
+		*c.data = *data
+	}
+	c.primarySession = sessionID
+	c.lastUpdate = time.Now()
+}

beszel/internal/agent/agent_cache_test.go

@@ -0,0 +1,88 @@
+//go:build testing
+// +build testing
+
+package agent
+
+import (
+	"beszel/internal/entities/system"
+	"testing"
+	"testing/synctest"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestSessionCache_GetSet(t *testing.T) {
+	synctest.Run(func() {
+		cache := NewSessionCache(69 * time.Second)
+
+		testData := &system.CombinedData{
+			Info: system.Info{
+				Hostname: "test-host",
+				Cores:    4,
+			},
+			Stats: system.Stats{
+				Cpu:     50.0,
+				MemPct:  30.0,
+				DiskPct: 40.0,
+			},
+		}
+
+		// Test initial state - should not be cached
+		data, isCached := cache.Get("session1")
+		assert.False(t, isCached, "Expected no cached data initially")
+		assert.NotNil(t, data, "Expected data to be initialized")
+		// Set data for session1
+		cache.Set("session1", testData)
+
+		time.Sleep(15 * time.Second)
+
+		// Get data for a different session - should be cached
+		data, isCached = cache.Get("session2")
+		assert.True(t, isCached, "Expected data to be cached for non-primary session")
+		require.NotNil(t, data, "Expected cached data to be returned")
+		assert.Equal(t, "test-host", data.Info.Hostname, "Hostname should match test data")
+		assert.Equal(t, 4, data.Info.Cores, "Cores should match test data")
+		assert.Equal(t, 50.0, data.Stats.Cpu, "CPU should match test data")
+		assert.Equal(t, 30.0, data.Stats.MemPct, "Memory percentage should match test data")
+		assert.Equal(t, 40.0, data.Stats.DiskPct, "Disk percentage should match test data")
+
+		time.Sleep(10 * time.Second)
+
+		// Get data for the primary session - should not be cached
+		data, isCached = cache.Get("session1")
+		assert.False(t, isCached, "Expected data not to be cached for primary session")
+		require.NotNil(t, data, "Expected data to be returned even if not cached")
+		assert.Equal(t, "test-host", data.Info.Hostname, "Hostname should match test data")
+		// if not cached, agent will update the data
+		cache.Set("session1", testData)
+
+		time.Sleep(45 * time.Second)
+
+		// Get data for a different session - should still be cached
+		_, isCached = cache.Get("session2")
+		assert.True(t, isCached, "Expected data to be cached for non-primary session")
+
+		// Wait for the lease to expire
+		time.Sleep(30 * time.Second)
+
+		// Get data for session2 - should not be cached
+		_, isCached = cache.Get("session2")
+		assert.False(t, isCached, "Expected data not to be cached after lease expiration")
+	})
+}
+
+func TestSessionCache_NilData(t *testing.T) {
+	// Create a new SessionCache
+	cache := NewSessionCache(30 * time.Second)
+
+	// Test setting nil data (should not panic)
+	assert.NotPanics(t, func() {
+		cache.Set("session1", nil)
+	}, "Setting nil data should not panic")
+
+	// Get data - should not be nil even though we set nil
+	data, _ := cache.Get("session2")
+	assert.NotNil(t, data, "Expected data to not be nil after setting nil data")
+}

beszel/internal/agent/agent_test_helpers.go

@@ -0,0 +1,9 @@
+//go:build testing
+// +build testing
+
+package agent
+
+// TESTING ONLY: GetConnectionManager is a helper function to get the connection manager for testing.
+func (a *Agent) GetConnectionManager() *ConnectionManager {
+	return a.connectionManager
+}

beszel/internal/agent/client.go

@@ -0,0 +1,265 @@
+package agent
+
+import (
+	"beszel"
+	"beszel/internal/common"
+	"crypto/tls"
+	"errors"
+	"fmt"
+	"log/slog"
+	"net"
+	"net/http"
+	"net/url"
+	"os"
+	"path"
+	"strings"
+	"time"
+
+	"github.com/fxamacker/cbor/v2"
+	"github.com/lxzan/gws"
+	"golang.org/x/crypto/ssh"
+)
+
+const (
+	wsDeadline = 70 * time.Second
+)
+
+// WebSocketClient manages the WebSocket connection between the agent and hub.
+// It handles authentication, message routing, and connection lifecycle management.
+type WebSocketClient struct {
+	gws.BuiltinEventHandler
+	options            *gws.ClientOption                   // WebSocket client configuration options
+	agent              *Agent                              // Reference to the parent agent
+	Conn               *gws.Conn                           // Active WebSocket connection
+	hubURL             *url.URL                            // Parsed hub URL for connection
+	token              string                              // Authentication token for hub registration
+	fingerprint        string                              // System fingerprint for identification
+	hubRequest         *common.HubRequest[cbor.RawMessage] // Reusable request structure for message parsing
+	lastConnectAttempt time.Time                           // Timestamp of last connection attempt
+	hubVerified        bool                                // Whether the hub has been cryptographically verified
+}
+
+// newWebSocketClient creates a new WebSocket client for the given agent.
+// It reads configuration from environment variables and validates the hub URL.
+func newWebSocketClient(agent *Agent) (client *WebSocketClient, err error) {
+	hubURLStr, exists := GetEnv("HUB_URL")
+	if !exists {
+		return nil, errors.New("HUB_URL environment variable not set")
+	}
+
+	client = &WebSocketClient{}
+
+	client.hubURL, err = url.Parse(hubURLStr)
+	if err != nil {
+		return nil, errors.New("invalid hub URL")
+	}
+	// get registration token
+	client.token, err = getToken()
+	if err != nil {
+		return nil, err
+	}
+
+	client.agent = agent
+	client.hubRequest = &common.HubRequest[cbor.RawMessage]{}
+	client.fingerprint = agent.getFingerprint()
+
+	return client, nil
+}
+
+// getToken returns the token for the WebSocket client.
+// It first checks the TOKEN environment variable, then the TOKEN_FILE environment variable.
+// If neither is set, it returns an error.
+func getToken() (string, error) {
+	// get token from env var
+	token, _ := GetEnv("TOKEN")
+	if token != "" {
+		return token, nil
+	}
+	// get token from file
+	tokenFile, _ := GetEnv("TOKEN_FILE")
+	if tokenFile == "" {
+		return "", errors.New("must set TOKEN or TOKEN_FILE")
+	}
+	tokenBytes, err := os.ReadFile(tokenFile)
+	if err != nil {
+		return "", err
+	}
+	return string(tokenBytes), nil
+}
+
+// getOptions returns the WebSocket client options, creating them if necessary.
+// It configures the connection URL, TLS settings, and authentication headers.
+func (client *WebSocketClient) getOptions() *gws.ClientOption {
+	if client.options != nil {
+		return client.options
+	}
+
+	// update the hub url to use websocket scheme and api path
+	if client.hubURL.Scheme == "https" {
+		client.hubURL.Scheme = "wss"
+	} else {
+		client.hubURL.Scheme = "ws"
+	}
+	client.hubURL.Path = path.Join(client.hubURL.Path, "api/beszel/agent-connect")
+
+	client.options = &gws.ClientOption{
+		Addr:      client.hubURL.String(),
+		TlsConfig: &tls.Config{InsecureSkipVerify: true},
+		RequestHeader: http.Header{
+			"User-Agent": []string{getUserAgent()},
+			"X-Token":    []string{client.token},
+			"X-Beszel":   []string{beszel.Version},
+		},
+	}
+	return client.options
+}
+
+// Connect establishes a WebSocket connection to the hub.
+// It closes any existing connection before attempting to reconnect.
+func (client *WebSocketClient) Connect() (err error) {
+	client.lastConnectAttempt = time.Now()
+
+	// make sure previous connection is closed
+	client.Close()
+
+	client.Conn, _, err = gws.NewClient(client, client.getOptions())
+	if err != nil {
+		return err
+	}
+
+	go client.Conn.ReadLoop()
+
+	return nil
+}
+
+// OnOpen handles WebSocket connection establishment.
+// It sets a deadline for the connection to prevent hanging.
+func (client *WebSocketClient) OnOpen(conn *gws.Conn) {
+	conn.SetDeadline(time.Now().Add(wsDeadline))
+}
+
+// OnClose handles WebSocket connection closure.
+// It logs the closure reason and notifies the connection manager.
+func (client *WebSocketClient) OnClose(conn *gws.Conn, err error) {
+	slog.Warn("Connection closed", "err", strings.TrimPrefix(err.Error(), "gws: "))
+	client.agent.connectionManager.eventChan <- WebSocketDisconnect
+}
+
+// OnMessage handles incoming WebSocket messages from the hub.
+// It decodes CBOR messages and routes them to appropriate handlers.
+func (client *WebSocketClient) OnMessage(conn *gws.Conn, message *gws.Message) {
+	defer message.Close()
+	conn.SetDeadline(time.Now().Add(wsDeadline))
+
+	if message.Opcode != gws.OpcodeBinary {
+		return
+	}
+
+	if err := cbor.NewDecoder(message.Data).Decode(client.hubRequest); err != nil {
+		slog.Error("Error parsing message", "err", err)
+		return
+	}
+	if err := client.handleHubRequest(client.hubRequest); err != nil {
+		slog.Error("Error handling message", "err", err)
+	}
+}
+
+// OnPing handles WebSocket ping frames.
+// It responds with a pong and updates the connection deadline.
+func (client *WebSocketClient) OnPing(conn *gws.Conn, message []byte) {
+	conn.SetDeadline(time.Now().Add(wsDeadline))
+	conn.WritePong(message)
+}
+
+// handleAuthChallenge verifies the authenticity of the hub and returns the system's fingerprint.
+func (client *WebSocketClient) handleAuthChallenge(msg *common.HubRequest[cbor.RawMessage]) (err error) {
+	var authRequest common.FingerprintRequest
+	if err := cbor.Unmarshal(msg.Data, &authRequest); err != nil {
+		return err
+	}
+
+	if err := client.verifySignature(authRequest.Signature); err != nil {
+		return err
+	}
+
+	client.hubVerified = true
+	client.agent.connectionManager.eventChan <- WebSocketConnect
+
+	response := &common.FingerprintResponse{
+		Fingerprint: client.fingerprint,
+	}
+
+	if authRequest.NeedSysInfo {
+		response.Hostname = client.agent.systemInfo.Hostname
+		serverAddr := client.agent.connectionManager.serverOptions.Addr
+		_, response.Port, _ = net.SplitHostPort(serverAddr)
+	}
+
+	return client.sendMessage(response)
+}
+
+// verifySignature verifies the signature of the token using the public keys.
+func (client *WebSocketClient) verifySignature(signature []byte) (err error) {
+	for _, pubKey := range client.agent.keys {
+		sig := ssh.Signature{
+			Format: pubKey.Type(),
+			Blob:   signature,
+		}
+		if err = pubKey.Verify([]byte(client.token), &sig); err == nil {
+			return nil
+		}
+	}
+	return errors.New("invalid signature - check KEY value")
+}
+
+// Close closes the WebSocket connection gracefully.
+// This method is safe to call multiple times.
+func (client *WebSocketClient) Close() {
+	if client.Conn != nil {
+		_ = client.Conn.WriteClose(1000, nil)
+	}
+}
+
+// handleHubRequest routes the request to the appropriate handler.
+// It ensures the hub is verified before processing most requests.
+func (client *WebSocketClient) handleHubRequest(msg *common.HubRequest[cbor.RawMessage]) error {
+	if !client.hubVerified && msg.Action != common.CheckFingerprint {
+		return errors.New("hub not verified")
+	}
+	switch msg.Action {
+	case common.GetData:
+		return client.sendSystemData()
+	case common.CheckFingerprint:
+		return client.handleAuthChallenge(msg)
+	}
+	return nil
+}
+
+// sendSystemData gathers and sends current system statistics to the hub.
+func (client *WebSocketClient) sendSystemData() error {
+	sysStats := client.agent.gatherStats(client.token)
+	return client.sendMessage(sysStats)
+}
+
+// sendMessage encodes the given data to CBOR and sends it as a binary message over the WebSocket connection to the hub.
+func (client *WebSocketClient) sendMessage(data any) error {
+	bytes, err := cbor.Marshal(data)
+	if err != nil {
+		return err
+	}
+	return client.Conn.WriteMessage(gws.OpcodeBinary, bytes)
+}
+
+// getUserAgent returns one of two User-Agent strings based on current time.
+// This is used to avoid being blocked by Cloudflare or other anti-bot measures.
+func getUserAgent() string {
+	const (
+		uaBase    = "Mozilla/5.0 (%s) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
+		uaWindows = "Windows NT 11.0; Win64; x64"
+		uaMac     = "Macintosh; Intel Mac OS X 14_0_0"
+	)
+	if time.Now().UnixNano()%2 == 0 {
+		return fmt.Sprintf(uaBase, uaWindows)
+	}
+	return fmt.Sprintf(uaBase, uaMac)
+}

beszel/internal/agent/client_test.go

@@ -0,0 +1,538 @@
+//go:build testing
+// +build testing
+
+package agent
+
+import (
+	"beszel"
+	"beszel/internal/common"
+	"crypto/ed25519"
+	"net/url"
+	"os"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/fxamacker/cbor/v2"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"golang.org/x/crypto/ssh"
+)
+
+// TestNewWebSocketClient tests WebSocket client creation
+func TestNewWebSocketClient(t *testing.T) {
+	agent := createTestAgent(t)
+
+	testCases := []struct {
+		name        string
+		hubURL      string
+		token       string
+		expectError bool
+		errorMsg    string
+	}{
+		{
+			name:        "valid configuration",
+			hubURL:      "http://localhost:8080",
+			token:       "test-token-123",
+			expectError: false,
+		},
+		{
+			name:        "valid https URL",
+			hubURL:      "https://hub.example.com",
+			token:       "secure-token",
+			expectError: false,
+		},
+		{
+			name:        "missing hub URL",
+			hubURL:      "",
+			token:       "test-token",
+			expectError: true,
+			errorMsg:    "HUB_URL environment variable not set",
+		},
+		{
+			name:        "invalid URL",
+			hubURL:      "ht\ttp://invalid",
+			token:       "test-token",
+			expectError: true,
+			errorMsg:    "invalid hub URL",
+		},
+		{
+			name:        "missing token",
+			hubURL:      "http://localhost:8080",
+			token:       "",
+			expectError: true,
+			errorMsg:    "must set TOKEN or TOKEN_FILE",
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Set up environment
+			if tc.hubURL != "" {
+				os.Setenv("BESZEL_AGENT_HUB_URL", tc.hubURL)
+			} else {
+				os.Unsetenv("BESZEL_AGENT_HUB_URL")
+			}
+			if tc.token != "" {
+				os.Setenv("BESZEL_AGENT_TOKEN", tc.token)
+			} else {
+				os.Unsetenv("BESZEL_AGENT_TOKEN")
+			}
+			defer func() {
+				os.Unsetenv("BESZEL_AGENT_HUB_URL")
+				os.Unsetenv("BESZEL_AGENT_TOKEN")
+			}()
+
+			client, err := newWebSocketClient(agent)
+
+			if tc.expectError {
+				assert.Error(t, err)
+				if err != nil && tc.errorMsg != "" {
+					assert.Contains(t, err.Error(), tc.errorMsg)
+				}
+				assert.Nil(t, client)
+			} else {
+				require.NoError(t, err)
+				assert.NotNil(t, client)
+				assert.Equal(t, agent, client.agent)
+				assert.Equal(t, tc.token, client.token)
+				assert.Equal(t, tc.hubURL, client.hubURL.String())
+				assert.NotEmpty(t, client.fingerprint)
+				assert.NotNil(t, client.hubRequest)
+			}
+		})
+	}
+}
+
+// TestWebSocketClient_GetOptions tests WebSocket client options configuration
+func TestWebSocketClient_GetOptions(t *testing.T) {
+	agent := createTestAgent(t)
+
+	testCases := []struct {
+		name           string
+		inputURL       string
+		expectedScheme string
+		expectedPath   string
+	}{
+		{
+			name:           "http to ws conversion",
+			inputURL:       "http://localhost:8080",
+			expectedScheme: "ws",
+			expectedPath:   "/api/beszel/agent-connect",
+		},
+		{
+			name:           "https to wss conversion",
+			inputURL:       "https://hub.example.com",
+			expectedScheme: "wss",
+			expectedPath:   "/api/beszel/agent-connect",
+		},
+		{
+			name:           "existing path preservation",
+			inputURL:       "http://localhost:8080/custom/path",
+			expectedScheme: "ws",
+			expectedPath:   "/custom/path/api/beszel/agent-connect",
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Set up environment
+			os.Setenv("BESZEL_AGENT_HUB_URL", tc.inputURL)
+			os.Setenv("BESZEL_AGENT_TOKEN", "test-token")
+			defer func() {
+				os.Unsetenv("BESZEL_AGENT_HUB_URL")
+				os.Unsetenv("BESZEL_AGENT_TOKEN")
+			}()
+
+			client, err := newWebSocketClient(agent)
+			require.NoError(t, err)
+
+			options := client.getOptions()
+
+			// Parse the WebSocket URL
+			wsURL, err := url.Parse(options.Addr)
+			require.NoError(t, err)
+
+			assert.Equal(t, tc.expectedScheme, wsURL.Scheme)
+			assert.Equal(t, tc.expectedPath, wsURL.Path)
+
+			// Check headers
+			assert.Equal(t, "test-token", options.RequestHeader.Get("X-Token"))
+			assert.Equal(t, beszel.Version, options.RequestHeader.Get("X-Beszel"))
+			assert.Contains(t, options.RequestHeader.Get("User-Agent"), "Mozilla/5.0")
+
+			// Test options caching
+			options2 := client.getOptions()
+			assert.Same(t, options, options2, "Options should be cached")
+		})
+	}
+}
+
+// TestWebSocketClient_VerifySignature tests signature verification
+func TestWebSocketClient_VerifySignature(t *testing.T) {
+	agent := createTestAgent(t)
+
+	// Generate test key pairs
+	_, goodPrivKey, err := ed25519.GenerateKey(nil)
+	require.NoError(t, err)
+	goodPubKey, err := ssh.NewPublicKey(goodPrivKey.Public().(ed25519.PublicKey))
+	require.NoError(t, err)
+
+	_, badPrivKey, err := ed25519.GenerateKey(nil)
+	require.NoError(t, err)
+	badPubKey, err := ssh.NewPublicKey(badPrivKey.Public().(ed25519.PublicKey))
+	require.NoError(t, err)
+
+	// Set up environment
+	os.Setenv("BESZEL_AGENT_HUB_URL", "http://localhost:8080")
+	os.Setenv("BESZEL_AGENT_TOKEN", "test-token")
+	defer func() {
+		os.Unsetenv("BESZEL_AGENT_HUB_URL")
+		os.Unsetenv("BESZEL_AGENT_TOKEN")
+	}()
+
+	client, err := newWebSocketClient(agent)
+	require.NoError(t, err)
+
+	testCases := []struct {
+		name        string
+		keys        []ssh.PublicKey
+		token       string
+		signWith    ed25519.PrivateKey
+		expectError bool
+	}{
+		{
+			name:        "valid signature with correct key",
+			keys:        []ssh.PublicKey{goodPubKey},
+			token:       "test-token",
+			signWith:    goodPrivKey,
+			expectError: false,
+		},
+		{
+			name:        "invalid signature with wrong key",
+			keys:        []ssh.PublicKey{goodPubKey},
+			token:       "test-token",
+			signWith:    badPrivKey,
+			expectError: true,
+		},
+		{
+			name:        "valid signature with multiple keys",
+			keys:        []ssh.PublicKey{badPubKey, goodPubKey},
+			token:       "test-token",
+			signWith:    goodPrivKey,
+			expectError: false,
+		},
+		{
+			name:        "no valid keys",
+			keys:        []ssh.PublicKey{badPubKey},
+			token:       "test-token",
+			signWith:    goodPrivKey,
+			expectError: true,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Set up agent with test keys
+			agent.keys = tc.keys
+			client.token = tc.token
+
+			// Create signature
+			signature := ed25519.Sign(tc.signWith, []byte(tc.token))
+
+			err := client.verifySignature(signature)
+
+			if tc.expectError {
+				assert.Error(t, err)
+				assert.Contains(t, err.Error(), "invalid signature")
+			} else {
+				assert.NoError(t, err)
+			}
+		})
+	}
+}
+
+// TestWebSocketClient_HandleHubRequest tests hub request routing (basic verification logic)
+func TestWebSocketClient_HandleHubRequest(t *testing.T) {
+	agent := createTestAgent(t)
+
+	// Set up environment
+	os.Setenv("BESZEL_AGENT_HUB_URL", "http://localhost:8080")
+	os.Setenv("BESZEL_AGENT_TOKEN", "test-token")
+	defer func() {
+		os.Unsetenv("BESZEL_AGENT_HUB_URL")
+		os.Unsetenv("BESZEL_AGENT_TOKEN")
+	}()
+
+	client, err := newWebSocketClient(agent)
+	require.NoError(t, err)
+
+	testCases := []struct {
+		name        string
+		action      common.WebSocketAction
+		hubVerified bool
+		expectError bool
+		errorMsg    string
+	}{
+		{
+			name:        "CheckFingerprint without verification",
+			action:      common.CheckFingerprint,
+			hubVerified: false,
+			expectError: false, // CheckFingerprint is allowed without verification
+		},
+		{
+			name:        "GetData without verification",
+			action:      common.GetData,
+			hubVerified: false,
+			expectError: true,
+			errorMsg:    "hub not verified",
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			client.hubVerified = tc.hubVerified
+
+			// Create minimal request
+			hubRequest := &common.HubRequest[cbor.RawMessage]{
+				Action: tc.action,
+				Data:   cbor.RawMessage{},
+			}
+
+			err := client.handleHubRequest(hubRequest)
+
+			if tc.expectError {
+				assert.Error(t, err)
+				if tc.errorMsg != "" {
+					assert.Contains(t, err.Error(), tc.errorMsg)
+				}
+			} else {
+				// For CheckFingerprint, we expect a decode error since we're not providing valid data,
+				// but it shouldn't be the "hub not verified" error
+				if err != nil && tc.errorMsg != "" {
+					assert.NotContains(t, err.Error(), tc.errorMsg)
+				}
+			}
+		})
+	}
+}
+
+// TestWebSocketClient_GetUserAgent tests user agent generation
+func TestGetUserAgent(t *testing.T) {
+	// Run multiple times to check both variants
+	userAgents := make(map[string]bool)
+
+	for range 20 {
+		ua := getUserAgent()
+		userAgents[ua] = true
+
+		// Check that it's a valid Mozilla user agent
+		assert.Contains(t, ua, "Mozilla/5.0")
+		assert.Contains(t, ua, "AppleWebKit/537.36")
+		assert.Contains(t, ua, "Chrome/124.0.0.0")
+		assert.Contains(t, ua, "Safari/537.36")
+
+		// Should contain either Windows or Mac
+		isWindows := strings.Contains(ua, "Windows NT 11.0")
+		isMac := strings.Contains(ua, "Macintosh; Intel Mac OS X 14_0_0")
+		assert.True(t, isWindows || isMac, "User agent should contain either Windows or Mac identifier")
+	}
+
+	// With enough iterations, we should see both variants
+	// though this might occasionally fail
+	if len(userAgents) == 1 {
+		t.Log("Note: Only one user agent variant was generated in this test run")
+	}
+}
+
+// TestWebSocketClient_Close tests connection closing
+func TestWebSocketClient_Close(t *testing.T) {
+	agent := createTestAgent(t)
+
+	// Set up environment
+	os.Setenv("BESZEL_AGENT_HUB_URL", "http://localhost:8080")
+	os.Setenv("BESZEL_AGENT_TOKEN", "test-token")
+	defer func() {
+		os.Unsetenv("BESZEL_AGENT_HUB_URL")
+		os.Unsetenv("BESZEL_AGENT_TOKEN")
+	}()
+
+	client, err := newWebSocketClient(agent)
+	require.NoError(t, err)
+
+	// Test closing with nil connection (should not panic)
+	assert.NotPanics(t, func() {
+		client.Close()
+	})
+}
+
+// TestWebSocketClient_ConnectRateLimit tests connection rate limiting
+func TestWebSocketClient_ConnectRateLimit(t *testing.T) {
+	agent := createTestAgent(t)
+
+	// Set up environment
+	os.Setenv("BESZEL_AGENT_HUB_URL", "http://localhost:8080")
+	os.Setenv("BESZEL_AGENT_TOKEN", "test-token")
+	defer func() {
+		os.Unsetenv("BESZEL_AGENT_HUB_URL")
+		os.Unsetenv("BESZEL_AGENT_TOKEN")
+	}()
+
+	client, err := newWebSocketClient(agent)
+	require.NoError(t, err)
+
+	// Set recent connection attempt
+	client.lastConnectAttempt = time.Now()
+
+	// Test that connection fails quickly due to rate limiting
+	// This won't actually connect but should fail fast
+	err = client.Connect()
+	assert.Error(t, err, "Connection should fail but not hang")
+}
+
+// TestGetToken tests the getToken function with various scenarios
+func TestGetToken(t *testing.T) {
+	unsetEnvVars := func() {
+		os.Unsetenv("BESZEL_AGENT_TOKEN")
+		os.Unsetenv("TOKEN")
+		os.Unsetenv("BESZEL_AGENT_TOKEN_FILE")
+		os.Unsetenv("TOKEN_FILE")
+	}
+
+	t.Run("token from TOKEN environment variable", func(t *testing.T) {
+		unsetEnvVars()
+
+		// Set TOKEN env var
+		expectedToken := "test-token-from-env"
+		os.Setenv("TOKEN", expectedToken)
+		defer os.Unsetenv("TOKEN")
+
+		token, err := getToken()
+		assert.NoError(t, err)
+		assert.Equal(t, expectedToken, token)
+	})
+
+	t.Run("token from BESZEL_AGENT_TOKEN environment variable", func(t *testing.T) {
+		unsetEnvVars()
+
+		// Set BESZEL_AGENT_TOKEN env var (should take precedence)
+		expectedToken := "test-token-from-beszel-env"
+		os.Setenv("BESZEL_AGENT_TOKEN", expectedToken)
+		defer os.Unsetenv("BESZEL_AGENT_TOKEN")
+
+		token, err := getToken()
+		assert.NoError(t, err)
+		assert.Equal(t, expectedToken, token)
+	})
+
+	t.Run("token from TOKEN_FILE", func(t *testing.T) {
+		unsetEnvVars()
+
+		// Create a temporary token file
+		expectedToken := "test-token-from-file"
+		tokenFile, err := os.CreateTemp("", "token-test-*.txt")
+		require.NoError(t, err)
+		defer os.Remove(tokenFile.Name())
+
+		_, err = tokenFile.WriteString(expectedToken)
+		require.NoError(t, err)
+		tokenFile.Close()
+
+		// Set TOKEN_FILE env var
+		os.Setenv("TOKEN_FILE", tokenFile.Name())
+		defer os.Unsetenv("TOKEN_FILE")
+
+		token, err := getToken()
+		assert.NoError(t, err)
+		assert.Equal(t, expectedToken, token)
+	})
+
+	t.Run("token from BESZEL_AGENT_TOKEN_FILE", func(t *testing.T) {
+		unsetEnvVars()
+
+		// Create a temporary token file
+		expectedToken := "test-token-from-beszel-file"
+		tokenFile, err := os.CreateTemp("", "token-test-*.txt")
+		require.NoError(t, err)
+		defer os.Remove(tokenFile.Name())
+
+		_, err = tokenFile.WriteString(expectedToken)
+		require.NoError(t, err)
+		tokenFile.Close()
+
+		// Set BESZEL_AGENT_TOKEN_FILE env var (should take precedence)
+		os.Setenv("BESZEL_AGENT_TOKEN_FILE", tokenFile.Name())
+		defer os.Unsetenv("BESZEL_AGENT_TOKEN_FILE")
+
+		token, err := getToken()
+		assert.NoError(t, err)
+		assert.Equal(t, expectedToken, token)
+	})
+
+	t.Run("TOKEN takes precedence over TOKEN_FILE", func(t *testing.T) {
+		unsetEnvVars()
+
+		// Create a temporary token file
+		fileToken := "token-from-file"
+		tokenFile, err := os.CreateTemp("", "token-test-*.txt")
+		require.NoError(t, err)
+		defer os.Remove(tokenFile.Name())
+
+		_, err = tokenFile.WriteString(fileToken)
+		require.NoError(t, err)
+		tokenFile.Close()
+
+		// Set both TOKEN and TOKEN_FILE
+		envToken := "token-from-env"
+		os.Setenv("TOKEN", envToken)
+		os.Setenv("TOKEN_FILE", tokenFile.Name())
+		defer func() {
+			os.Unsetenv("TOKEN")
+			os.Unsetenv("TOKEN_FILE")
+		}()
+
+		token, err := getToken()
+		assert.NoError(t, err)
+		assert.Equal(t, envToken, token, "TOKEN should take precedence over TOKEN_FILE")
+	})
+
+	t.Run("error when neither TOKEN nor TOKEN_FILE is set", func(t *testing.T) {
+		unsetEnvVars()
+
+		token, err := getToken()
+		assert.Error(t, err)
+		assert.Equal(t, "", token)
+		assert.Contains(t, err.Error(), "must set TOKEN or TOKEN_FILE")
+	})
+
+	t.Run("error when TOKEN_FILE points to non-existent file", func(t *testing.T) {
+		unsetEnvVars()
+
+		// Set TOKEN_FILE to a non-existent file
+		os.Setenv("TOKEN_FILE", "/non/existent/file.txt")
+		defer os.Unsetenv("TOKEN_FILE")
+
+		token, err := getToken()
+		assert.Error(t, err)
+		assert.Equal(t, "", token)
+		assert.Contains(t, err.Error(), "no such file or directory")
+	})
+
+	t.Run("handles empty token file", func(t *testing.T) {
+		unsetEnvVars()
+
+		// Create an empty token file
+		tokenFile, err := os.CreateTemp("", "token-test-*.txt")
+		require.NoError(t, err)
+		defer os.Remove(tokenFile.Name())
+		tokenFile.Close()
+
+		// Set TOKEN_FILE env var
+		os.Setenv("TOKEN_FILE", tokenFile.Name())
+		defer os.Unsetenv("TOKEN_FILE")
+
+		token, err := getToken()
+		assert.NoError(t, err)
+		assert.Equal(t, "", token, "Empty file should return empty string")
+	})
+}

beszel/internal/agent/connection_manager.go

@@ -0,0 +1,220 @@
+package agent
+
+import (
+	"beszel/internal/agent/health"
+	"errors"
+	"log/slog"
+	"os"
+	"os/signal"
+	"syscall"
+	"time"
+)
+
+// ConnectionManager manages the connection state and events for the agent.
+// It handles both WebSocket and SSH connections, automatically switching between
+// them based on availability and managing reconnection attempts.
+type ConnectionManager struct {
+	agent         *Agent               // Reference to the parent agent
+	State         ConnectionState      // Current connection state
+	eventChan     chan ConnectionEvent // Channel for connection events
+	wsClient      *WebSocketClient     // WebSocket client for hub communication
+	serverOptions ServerOptions        // Configuration for SSH server
+	wsTicker      *time.Ticker         // Ticker for WebSocket connection attempts
+	isConnecting  bool                 // Prevents multiple simultaneous reconnection attempts
+}
+
+// ConnectionState represents the current connection state of the agent.
+type ConnectionState uint8
+
+// ConnectionEvent represents connection-related events that can occur.
+type ConnectionEvent uint8
+
+// Connection states
+const (
+	Disconnected       ConnectionState = iota // No active connection
+	WebSocketConnected                        // Connected via WebSocket
+	SSHConnected                              // Connected via SSH
+)
+
+// Connection events
+const (
+	WebSocketConnect    ConnectionEvent = iota // WebSocket connection established
+	WebSocketDisconnect                        // WebSocket connection lost
+	SSHConnect                                 // SSH connection established
+	SSHDisconnect                              // SSH connection lost
+)
+
+const wsTickerInterval = 10 * time.Second
+
+// newConnectionManager creates a new connection manager for the given agent.
+func newConnectionManager(agent *Agent) *ConnectionManager {
+	cm := &ConnectionManager{
+		agent: agent,
+		State: Disconnected,
+	}
+	return cm
+}
+
+// startWsTicker starts or resets the WebSocket connection attempt ticker.
+func (c *ConnectionManager) startWsTicker() {
+	if c.wsTicker == nil {
+		c.wsTicker = time.NewTicker(wsTickerInterval)
+	} else {
+		c.wsTicker.Reset(wsTickerInterval)
+	}
+}
+
+// stopWsTicker stops the WebSocket connection attempt ticker.
+func (c *ConnectionManager) stopWsTicker() {
+	if c.wsTicker != nil {
+		c.wsTicker.Stop()
+	}
+}
+
+// Start begins connection attempts and enters the main event loop.
+// It handles connection events, periodic health updates, and graceful shutdown.
+func (c *ConnectionManager) Start(serverOptions ServerOptions) error {
+	if c.eventChan != nil {
+		return errors.New("already started")
+	}
+
+	wsClient, err := newWebSocketClient(c.agent)
+	if err != nil {
+		slog.Warn("Error creating WebSocket client", "err", err)
+	}
+	c.wsClient = wsClient
+
+	c.serverOptions = serverOptions
+	c.eventChan = make(chan ConnectionEvent, 1)
+
+	// signal handling for shutdown
+	sigChan := make(chan os.Signal, 1)
+	signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
+
+	c.startWsTicker()
+	c.connect()
+
+	// update health status immediately and every 90 seconds
+	_ = health.Update()
+	healthTicker := time.Tick(90 * time.Second)
+
+	for {
+		select {
+		case connectionEvent := <-c.eventChan:
+			c.handleEvent(connectionEvent)
+		case <-c.wsTicker.C:
+			_ = c.startWebSocketConnection()
+		case <-healthTicker:
+			_ = health.Update()
+		case <-sigChan:
+			slog.Info("Shutting down")
+			_ = c.agent.StopServer()
+			c.closeWebSocket()
+			return health.CleanUp()
+		}
+	}
+}
+
+// handleEvent processes connection events and updates the connection state accordingly.
+func (c *ConnectionManager) handleEvent(event ConnectionEvent) {
+	switch event {
+	case WebSocketConnect:
+		c.handleStateChange(WebSocketConnected)
+	case SSHConnect:
+		c.handleStateChange(SSHConnected)
+	case WebSocketDisconnect:
+		if c.State == WebSocketConnected {
+			c.handleStateChange(Disconnected)
+		}
+	case SSHDisconnect:
+		if c.State == SSHConnected {
+			c.handleStateChange(Disconnected)
+		}
+	}
+}
+
+// handleStateChange updates the connection state and performs necessary actions
+// based on the new state, including stopping services and initiating reconnections.
+func (c *ConnectionManager) handleStateChange(newState ConnectionState) {
+	if c.State == newState {
+		return
+	}
+	c.State = newState
+	switch newState {
+	case WebSocketConnected:
+		slog.Info("WebSocket connected", "host", c.wsClient.hubURL.Host)
+		c.stopWsTicker()
+		_ = c.agent.StopServer()
+		c.isConnecting = false
+	case SSHConnected:
+		// stop new ws connection attempts
+		slog.Info("SSH connection established")
+		c.stopWsTicker()
+		c.isConnecting = false
+	case Disconnected:
+		if c.isConnecting {
+			// Already handling reconnection, avoid duplicate attempts
+			return
+		}
+		c.isConnecting = true
+		slog.Warn("Disconnected from hub")
+		// make sure old ws connection is closed
+		c.closeWebSocket()
+		// reconnect
+		go c.connect()
+	}
+}
+
+// connect handles the connection logic with proper delays and priority.
+// It attempts WebSocket connection first, falling back to SSH server if needed.
+func (c *ConnectionManager) connect() {
+	c.isConnecting = true
+	defer func() {
+		c.isConnecting = false
+	}()
+
+	if c.wsClient != nil && time.Since(c.wsClient.lastConnectAttempt) < 5*time.Second {
+		time.Sleep(5 * time.Second)
+	}
+
+	// Try WebSocket first, if it fails, start SSH server
+	err := c.startWebSocketConnection()
+	if err != nil && c.State == Disconnected {
+		c.startSSHServer()
+		c.startWsTicker()
+	}
+}
+
+// startWebSocketConnection attempts to establish a WebSocket connection to the hub.
+func (c *ConnectionManager) startWebSocketConnection() error {
+	if c.State != Disconnected {
+		return errors.New("already connected")
+	}
+	if c.wsClient == nil {
+		return errors.New("WebSocket client not initialized")
+	}
+	if time.Since(c.wsClient.lastConnectAttempt) < 5*time.Second {
+		return errors.New("already connecting")
+	}
+
+	err := c.wsClient.Connect()
+	if err != nil {
+		slog.Warn("WebSocket connection failed", "err", err)
+		c.closeWebSocket()
+	}
+	return err
+}
+
+// startSSHServer starts the SSH server if the agent is currently disconnected.
+func (c *ConnectionManager) startSSHServer() {
+	if c.State == Disconnected {
+		go c.agent.StartServer(c.serverOptions)
+	}
+}
+
+// closeWebSocket closes the WebSocket connection if it exists.
+func (c *ConnectionManager) closeWebSocket() {
+	if c.wsClient != nil {
+		c.wsClient.Close()
+	}
+}

beszel/internal/agent/connection_manager_test.go

@@ -0,0 +1,315 @@
+//go:build testing
+// +build testing
+
+package agent
+
+import (
+	"crypto/ed25519"
+	"fmt"
+	"net"
+	"net/url"
+	"os"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"golang.org/x/crypto/ssh"
+)
+
+func createTestAgent(t *testing.T) *Agent {
+	dataDir := t.TempDir()
+	agent, err := NewAgent(dataDir)
+	require.NoError(t, err)
+	return agent
+}
+
+func createTestServerOptions(t *testing.T) ServerOptions {
+	// Generate test key pair
+	_, privKey, err := ed25519.GenerateKey(nil)
+	require.NoError(t, err)
+	sshPubKey, err := ssh.NewPublicKey(privKey.Public().(ed25519.PublicKey))
+	require.NoError(t, err)
+
+	// Find available port
+	listener, err := net.Listen("tcp", "127.0.0.1:0")
+	require.NoError(t, err)
+	port := listener.Addr().(*net.TCPAddr).Port
+	listener.Close()
+
+	return ServerOptions{
+		Network: "tcp",
+		Addr:    fmt.Sprintf("127.0.0.1:%d", port),
+		Keys:    []ssh.PublicKey{sshPubKey},
+	}
+}
+
+// TestConnectionManager_NewConnectionManager tests connection manager creation
+func TestConnectionManager_NewConnectionManager(t *testing.T) {
+	agent := createTestAgent(t)
+	cm := newConnectionManager(agent)
+
+	assert.NotNil(t, cm, "Connection manager should not be nil")
+	assert.Equal(t, agent, cm.agent, "Agent reference should be set")
+	assert.Equal(t, Disconnected, cm.State, "Initial state should be Disconnected")
+	assert.Nil(t, cm.eventChan, "Event channel should be nil initially")
+	assert.Nil(t, cm.wsClient, "WebSocket client should be nil initially")
+	assert.Nil(t, cm.wsTicker, "WebSocket ticker should be nil initially")
+	assert.False(t, cm.isConnecting, "isConnecting should be false initially")
+}
+
+// TestConnectionManager_StateTransitions tests basic state transitions
+func TestConnectionManager_StateTransitions(t *testing.T) {
+	agent := createTestAgent(t)
+	cm := agent.connectionManager
+	initialState := cm.State
+	cm.wsClient = &WebSocketClient{
+		hubURL: &url.URL{
+			Host: "localhost:8080",
+		},
+	}
+	assert.NotNil(t, cm, "Connection manager should not be nil")
+	assert.Equal(t, Disconnected, initialState, "Initial state should be Disconnected")
+
+	// Test state transitions
+	cm.handleStateChange(WebSocketConnected)
+	assert.Equal(t, WebSocketConnected, cm.State, "State should change to WebSocketConnected")
+
+	cm.handleStateChange(SSHConnected)
+	assert.Equal(t, SSHConnected, cm.State, "State should change to SSHConnected")
+
+	cm.handleStateChange(Disconnected)
+	assert.Equal(t, Disconnected, cm.State, "State should change to Disconnected")
+
+	// Test that same state doesn't trigger changes
+	cm.State = WebSocketConnected
+	cm.handleStateChange(WebSocketConnected)
+	assert.Equal(t, WebSocketConnected, cm.State, "Same state should not trigger change")
+}
+
+// TestConnectionManager_EventHandling tests event handling logic
+func TestConnectionManager_EventHandling(t *testing.T) {
+	agent := createTestAgent(t)
+	cm := agent.connectionManager
+	cm.wsClient = &WebSocketClient{
+		hubURL: &url.URL{
+			Host: "localhost:8080",
+		},
+	}
+
+	testCases := []struct {
+		name          string
+		initialState  ConnectionState
+		event         ConnectionEvent
+		expectedState ConnectionState
+	}{
+		{
+			name:          "WebSocket connect from disconnected",
+			initialState:  Disconnected,
+			event:         WebSocketConnect,
+			expectedState: WebSocketConnected,
+		},
+		{
+			name:          "SSH connect from disconnected",
+			initialState:  Disconnected,
+			event:         SSHConnect,
+			expectedState: SSHConnected,
+		},
+		{
+			name:          "WebSocket disconnect from connected",
+			initialState:  WebSocketConnected,
+			event:         WebSocketDisconnect,
+			expectedState: Disconnected,
+		},
+		{
+			name:          "SSH disconnect from connected",
+			initialState:  SSHConnected,
+			event:         SSHDisconnect,
+			expectedState: Disconnected,
+		},
+		{
+			name:          "WebSocket disconnect from SSH connected (no change)",
+			initialState:  SSHConnected,
+			event:         WebSocketDisconnect,
+			expectedState: SSHConnected,
+		},
+		{
+			name:          "SSH disconnect from WebSocket connected (no change)",
+			initialState:  WebSocketConnected,
+			event:         SSHDisconnect,
+			expectedState: WebSocketConnected,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			cm.State = tc.initialState
+			cm.handleEvent(tc.event)
+			assert.Equal(t, tc.expectedState, cm.State, "State should match expected after event")
+		})
+	}
+}
+
+// TestConnectionManager_TickerManagement tests WebSocket ticker management
+func TestConnectionManager_TickerManagement(t *testing.T) {
+	agent := createTestAgent(t)
+	cm := agent.connectionManager
+
+	// Test starting ticker
+	cm.startWsTicker()
+	assert.NotNil(t, cm.wsTicker, "Ticker should be created")
+
+	// Test stopping ticker (should not panic)
+	assert.NotPanics(t, func() {
+		cm.stopWsTicker()
+	}, "Stopping ticker should not panic")
+
+	// Test stopping nil ticker (should not panic)
+	cm.wsTicker = nil
+	assert.NotPanics(t, func() {
+		cm.stopWsTicker()
+	}, "Stopping nil ticker should not panic")
+
+	// Test restarting ticker
+	cm.startWsTicker()
+	assert.NotNil(t, cm.wsTicker, "Ticker should be recreated")
+
+	// Test resetting existing ticker
+	firstTicker := cm.wsTicker
+	cm.startWsTicker()
+	assert.Equal(t, firstTicker, cm.wsTicker, "Same ticker instance should be reused")
+
+	cm.stopWsTicker()
+}
+
+// TestConnectionManager_WebSocketConnectionFlow tests WebSocket connection logic
+func TestConnectionManager_WebSocketConnectionFlow(t *testing.T) {
+	if testing.Short() {
+		t.Skip("Skipping WebSocket connection test in short mode")
+	}
+
+	agent := createTestAgent(t)
+	cm := agent.connectionManager
+
+	// Test WebSocket connection without proper environment
+	err := cm.startWebSocketConnection()
+	assert.Error(t, err, "WebSocket connection should fail without proper environment")
+	assert.Equal(t, Disconnected, cm.State, "State should remain Disconnected after failed connection")
+
+	// Test with invalid URL
+	os.Setenv("BESZEL_AGENT_HUB_URL", "invalid-url")
+	os.Setenv("BESZEL_AGENT_TOKEN", "test-token")
+	defer func() {
+		os.Unsetenv("BESZEL_AGENT_HUB_URL")
+		os.Unsetenv("BESZEL_AGENT_TOKEN")
+	}()
+
+	// Test with missing token
+	os.Setenv("BESZEL_AGENT_HUB_URL", "http://localhost:8080")
+	os.Unsetenv("BESZEL_AGENT_TOKEN")
+
+	_, err2 := newWebSocketClient(agent)
+	assert.Error(t, err2, "WebSocket client creation should fail without token")
+}
+
+// TestConnectionManager_ReconnectionLogic tests reconnection prevention logic
+func TestConnectionManager_ReconnectionLogic(t *testing.T) {
+	agent := createTestAgent(t)
+	cm := agent.connectionManager
+	cm.eventChan = make(chan ConnectionEvent, 1)
+
+	// Test that isConnecting flag prevents duplicate reconnection attempts
+	// Start from connected state, then simulate disconnect
+	cm.State = WebSocketConnected
+	cm.isConnecting = false
+
+	// First disconnect should trigger reconnection logic
+	cm.handleStateChange(Disconnected)
+	assert.Equal(t, Disconnected, cm.State, "Should change to disconnected")
+	assert.True(t, cm.isConnecting, "Should set isConnecting flag")
+}
+
+// TestConnectionManager_ConnectWithRateLimit tests connection rate limiting
+func TestConnectionManager_ConnectWithRateLimit(t *testing.T) {
+	agent := createTestAgent(t)
+	cm := agent.connectionManager
+
+	// Set up environment for WebSocket client creation
+	os.Setenv("BESZEL_AGENT_HUB_URL", "ws://localhost:8080")
+	os.Setenv("BESZEL_AGENT_TOKEN", "test-token")
+	defer func() {
+		os.Unsetenv("BESZEL_AGENT_HUB_URL")
+		os.Unsetenv("BESZEL_AGENT_TOKEN")
+	}()
+
+	// Create WebSocket client
+	wsClient, err := newWebSocketClient(agent)
+	require.NoError(t, err)
+	cm.wsClient = wsClient
+
+	// Set recent connection attempt
+	cm.wsClient.lastConnectAttempt = time.Now()
+
+	// Test that connection is rate limited
+	err = cm.startWebSocketConnection()
+	assert.Error(t, err, "Should error due to rate limiting")
+	assert.Contains(t, err.Error(), "already connecting", "Error should indicate rate limiting")
+
+	// Test connection after rate limit expires
+	cm.wsClient.lastConnectAttempt = time.Now().Add(-10 * time.Second)
+	err = cm.startWebSocketConnection()
+	// This will fail due to no actual server, but should not be rate limited
+	assert.Error(t, err, "Connection should fail but not due to rate limiting")
+	assert.NotContains(t, err.Error(), "already connecting", "Error should not indicate rate limiting")
+}
+
+// TestConnectionManager_StartWithInvalidConfig tests starting with invalid configuration
+func TestConnectionManager_StartWithInvalidConfig(t *testing.T) {
+	agent := createTestAgent(t)
+	cm := agent.connectionManager
+	serverOptions := createTestServerOptions(t)
+
+	// Test starting when already started
+	cm.eventChan = make(chan ConnectionEvent, 5)
+	err := cm.Start(serverOptions)
+	assert.Error(t, err, "Should error when starting already started connection manager")
+}
+
+// TestConnectionManager_CloseWebSocket tests WebSocket closing
+func TestConnectionManager_CloseWebSocket(t *testing.T) {
+	agent := createTestAgent(t)
+	cm := agent.connectionManager
+
+	// Test closing when no WebSocket client exists
+	assert.NotPanics(t, func() {
+		cm.closeWebSocket()
+	}, "Should not panic when closing nil WebSocket client")
+
+	// Set up environment and create WebSocket client
+	os.Setenv("BESZEL_AGENT_HUB_URL", "ws://localhost:8080")
+	os.Setenv("BESZEL_AGENT_TOKEN", "test-token")
+	defer func() {
+		os.Unsetenv("BESZEL_AGENT_HUB_URL")
+		os.Unsetenv("BESZEL_AGENT_TOKEN")
+	}()
+
+	wsClient, err := newWebSocketClient(agent)
+	require.NoError(t, err)
+	cm.wsClient = wsClient
+
+	// Test closing when WebSocket client exists
+	assert.NotPanics(t, func() {
+		cm.closeWebSocket()
+	}, "Should not panic when closing WebSocket client")
+}
+
+// TestConnectionManager_ConnectFlow tests the connect method
+func TestConnectionManager_ConnectFlow(t *testing.T) {
+	agent := createTestAgent(t)
+	cm := agent.connectionManager
+
+	// Test connect without WebSocket client
+	assert.NotPanics(t, func() {
+		cm.connect()
+	}, "Connect should not panic without WebSocket client")
+}

beszel/internal/agent/data_dir.go

@@ -0,0 +1,117 @@
+package agent
+
+import (
+	"errors"
+	"fmt"
+	"os"
+	"path/filepath"
+	"runtime"
+)
+
+// getDataDir returns the path to the data directory for the agent and an error
+// if the directory is not valid. Attempts to find the optimal data directory if
+// no data directories are provided.
+func getDataDir(dataDirs ...string) (string, error) {
+	if len(dataDirs) > 0 {
+		return testDataDirs(dataDirs)
+	}
+
+	dataDir, _ := GetEnv("DATA_DIR")
+	if dataDir != "" {
+		dataDirs = append(dataDirs, dataDir)
+	}
+
+	if runtime.GOOS == "windows" {
+		dataDirs = append(dataDirs,
+			filepath.Join(os.Getenv("APPDATA"), "beszel-agent"),
+			filepath.Join(os.Getenv("LOCALAPPDATA"), "beszel-agent"),
+		)
+	} else {
+		dataDirs = append(dataDirs, "/var/lib/beszel-agent")
+		if homeDir, err := os.UserHomeDir(); err == nil {
+			dataDirs = append(dataDirs, filepath.Join(homeDir, ".config", "beszel"))
+		}
+	}
+	return testDataDirs(dataDirs)
+}
+
+func testDataDirs(paths []string) (string, error) {
+	// first check if the directory exists and is writable
+	for _, path := range paths {
+		if valid, _ := isValidDataDir(path, false); valid {
+			return path, nil
+		}
+	}
+	// if the directory doesn't exist, try to create it
+	for _, path := range paths {
+		exists, _ := directoryExists(path)
+		if exists {
+			continue
+		}
+
+		if err := os.MkdirAll(path, 0755); err != nil {
+			continue
+		}
+
+		// Verify the created directory is actually writable
+		writable, _ := directoryIsWritable(path)
+		if !writable {
+			continue
+		}
+
+		return path, nil
+	}
+
+	return "", errors.New("data directory not found")
+}
+
+func isValidDataDir(path string, createIfNotExists bool) (bool, error) {
+	exists, err := directoryExists(path)
+	if err != nil {
+		return false, err
+	}
+
+	if !exists {
+		if !createIfNotExists {
+			return false, nil
+		}
+		if err = os.MkdirAll(path, 0755); err != nil {
+			return false, err
+		}
+	}
+
+	// Always check if the directory is writable
+	writable, err := directoryIsWritable(path)
+	if err != nil {
+		return false, err
+	}
+	return writable, nil
+}
+
+// directoryExists checks if a directory exists
+func directoryExists(path string) (bool, error) {
+	// Check if directory exists
+	stat, err := os.Stat(path)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return false, nil
+		}
+		return false, err
+	}
+	if !stat.IsDir() {
+		return false, fmt.Errorf("%s is not a directory", path)
+	}
+	return true, nil
+}
+
+// directoryIsWritable tests if a directory is writable by creating and removing a temporary file
+func directoryIsWritable(path string) (bool, error) {
+	testFile := filepath.Join(path, ".write-test")
+	file, err := os.Create(testFile)
+	if err != nil {
+		return false, err
+	}
+	defer file.Close()
+	defer os.Remove(testFile)
+	return true, nil
+}

beszel/internal/agent/data_dir_test.go

@@ -0,0 +1,263 @@
+//go:build testing
+// +build testing
+
+package agent
+
+import (
+	"os"
+	"path/filepath"
+	"runtime"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestGetDataDir(t *testing.T) {
+	// Test with explicit dataDir parameter
+	t.Run("explicit data dir", func(t *testing.T) {
+		tempDir := t.TempDir()
+		result, err := getDataDir(tempDir)
+		require.NoError(t, err)
+		assert.Equal(t, tempDir, result)
+	})
+
+	// Test with explicit non-existent dataDir that can be created
+	t.Run("explicit data dir - create new", func(t *testing.T) {
+		tempDir := t.TempDir()
+		newDir := filepath.Join(tempDir, "new-data-dir")
+		result, err := getDataDir(newDir)
+		require.NoError(t, err)
+		assert.Equal(t, newDir, result)
+
+		// Verify directory was created
+		stat, err := os.Stat(newDir)
+		require.NoError(t, err)
+		assert.True(t, stat.IsDir())
+	})
+
+	// Test with DATA_DIR environment variable
+	t.Run("DATA_DIR environment variable", func(t *testing.T) {
+		tempDir := t.TempDir()
+
+		// Set environment variable
+		oldValue := os.Getenv("DATA_DIR")
+		defer func() {
+			if oldValue == "" {
+				os.Unsetenv("BESZEL_AGENT_DATA_DIR")
+			} else {
+				os.Setenv("BESZEL_AGENT_DATA_DIR", oldValue)
+			}
+		}()
+
+		os.Setenv("BESZEL_AGENT_DATA_DIR", tempDir)
+
+		result, err := getDataDir()
+		require.NoError(t, err)
+		assert.Equal(t, tempDir, result)
+	})
+
+	// Test with invalid explicit dataDir
+	t.Run("invalid explicit data dir", func(t *testing.T) {
+		invalidPath := "/invalid/path/that/cannot/be/created"
+		_, err := getDataDir(invalidPath)
+		assert.Error(t, err)
+	})
+
+	// Test fallback behavior (empty dataDir, no env var)
+	t.Run("fallback to default directories", func(t *testing.T) {
+		// Clear DATA_DIR environment variable
+		oldValue := os.Getenv("DATA_DIR")
+		defer func() {
+			if oldValue == "" {
+				os.Unsetenv("DATA_DIR")
+			} else {
+				os.Setenv("DATA_DIR", oldValue)
+			}
+		}()
+		os.Unsetenv("DATA_DIR")
+
+		// This will try platform-specific defaults, which may or may not work
+		// We're mainly testing that it doesn't panic and returns some result
+		result, err := getDataDir()
+		// We don't assert success/failure here since it depends on system permissions
+		// Just verify we get a string result if no error
+		if err == nil {
+			assert.NotEmpty(t, result)
+		}
+	})
+}
+
+func TestTestDataDirs(t *testing.T) {
+	// Test with existing valid directory
+	t.Run("existing valid directory", func(t *testing.T) {
+		tempDir := t.TempDir()
+		result, err := testDataDirs([]string{tempDir})
+		require.NoError(t, err)
+		assert.Equal(t, tempDir, result)
+	})
+
+	// Test with multiple directories, first one valid
+	t.Run("multiple dirs - first valid", func(t *testing.T) {
+		tempDir := t.TempDir()
+		invalidDir := "/invalid/path"
+		result, err := testDataDirs([]string{tempDir, invalidDir})
+		require.NoError(t, err)
+		assert.Equal(t, tempDir, result)
+	})
+
+	// Test with multiple directories, second one valid
+	t.Run("multiple dirs - second valid", func(t *testing.T) {
+		tempDir := t.TempDir()
+		invalidDir := "/invalid/path"
+		result, err := testDataDirs([]string{invalidDir, tempDir})
+		require.NoError(t, err)
+		assert.Equal(t, tempDir, result)
+	})
+
+	// Test with non-existing directory that can be created
+	t.Run("create new directory", func(t *testing.T) {
+		tempDir := t.TempDir()
+		newDir := filepath.Join(tempDir, "new-dir")
+		result, err := testDataDirs([]string{newDir})
+		require.NoError(t, err)
+		assert.Equal(t, newDir, result)
+
+		// Verify directory was created
+		stat, err := os.Stat(newDir)
+		require.NoError(t, err)
+		assert.True(t, stat.IsDir())
+	})
+
+	// Test with no valid directories
+	t.Run("no valid directories", func(t *testing.T) {
+		invalidPaths := []string{"/invalid/path1", "/invalid/path2"}
+		_, err := testDataDirs(invalidPaths)
+		assert.Error(t, err)
+		assert.Contains(t, err.Error(), "data directory not found")
+	})
+}
+
+func TestIsValidDataDir(t *testing.T) {
+	// Test with existing directory
+	t.Run("existing directory", func(t *testing.T) {
+		tempDir := t.TempDir()
+		valid, err := isValidDataDir(tempDir, false)
+		require.NoError(t, err)
+		assert.True(t, valid)
+	})
+
+	// Test with non-existing directory, createIfNotExists=false
+	t.Run("non-existing dir - no create", func(t *testing.T) {
+		tempDir := t.TempDir()
+		nonExistentDir := filepath.Join(tempDir, "does-not-exist")
+		valid, err := isValidDataDir(nonExistentDir, false)
+		require.NoError(t, err)
+		assert.False(t, valid)
+	})
+
+	// Test with non-existing directory, createIfNotExists=true
+	t.Run("non-existing dir - create", func(t *testing.T) {
+		tempDir := t.TempDir()
+		newDir := filepath.Join(tempDir, "new-dir")
+		valid, err := isValidDataDir(newDir, true)
+		require.NoError(t, err)
+		assert.True(t, valid)
+
+		// Verify directory was created
+		stat, err := os.Stat(newDir)
+		require.NoError(t, err)
+		assert.True(t, stat.IsDir())
+	})
+
+	// Test with file instead of directory
+	t.Run("file instead of directory", func(t *testing.T) {
+		tempDir := t.TempDir()
+		tempFile := filepath.Join(tempDir, "testfile")
+		err := os.WriteFile(tempFile, []byte("test"), 0644)
+		require.NoError(t, err)
+
+		valid, err := isValidDataDir(tempFile, false)
+		assert.Error(t, err)
+		assert.False(t, valid)
+		assert.Contains(t, err.Error(), "is not a directory")
+	})
+}
+
+func TestDirectoryExists(t *testing.T) {
+	// Test with existing directory
+	t.Run("existing directory", func(t *testing.T) {
+		tempDir := t.TempDir()
+		exists, err := directoryExists(tempDir)
+		require.NoError(t, err)
+		assert.True(t, exists)
+	})
+
+	// Test with non-existing directory
+	t.Run("non-existing directory", func(t *testing.T) {
+		tempDir := t.TempDir()
+		nonExistentDir := filepath.Join(tempDir, "does-not-exist")
+		exists, err := directoryExists(nonExistentDir)
+		require.NoError(t, err)
+		assert.False(t, exists)
+	})
+
+	// Test with file instead of directory
+	t.Run("file instead of directory", func(t *testing.T) {
+		tempDir := t.TempDir()
+		tempFile := filepath.Join(tempDir, "testfile")
+		err := os.WriteFile(tempFile, []byte("test"), 0644)
+		require.NoError(t, err)
+
+		exists, err := directoryExists(tempFile)
+		assert.Error(t, err)
+		assert.False(t, exists)
+		assert.Contains(t, err.Error(), "is not a directory")
+	})
+}
+
+func TestDirectoryIsWritable(t *testing.T) {
+	// Test with writable directory
+	t.Run("writable directory", func(t *testing.T) {
+		tempDir := t.TempDir()
+		writable, err := directoryIsWritable(tempDir)
+		require.NoError(t, err)
+		assert.True(t, writable)
+	})
+
+	// Test with non-existing directory
+	t.Run("non-existing directory", func(t *testing.T) {
+		tempDir := t.TempDir()
+		nonExistentDir := filepath.Join(tempDir, "does-not-exist")
+		writable, err := directoryIsWritable(nonExistentDir)
+		assert.Error(t, err)
+		assert.False(t, writable)
+	})
+
+	// Test with non-writable directory (Unix-like systems only)
+	t.Run("non-writable directory", func(t *testing.T) {
+		if runtime.GOOS != "linux" && runtime.GOOS != "darwin" {
+			t.Skip("Skipping non-writable directory test on", runtime.GOOS)
+		}
+
+		tempDir := t.TempDir()
+		readOnlyDir := filepath.Join(tempDir, "readonly")
+
+		// Create the directory
+		err := os.Mkdir(readOnlyDir, 0755)
+		require.NoError(t, err)
+
+		// Make it read-only
+		err = os.Chmod(readOnlyDir, 0444)
+		require.NoError(t, err)
+
+		// Restore permissions after test for cleanup
+		defer func() {
+			os.Chmod(readOnlyDir, 0755)
+		}()
+
+		writable, err := directoryIsWritable(readOnlyDir)
+		assert.Error(t, err)
+		assert.False(t, writable)
+	})
+}

beszel/internal/agent/disk.go

@@ -3,18 +3,18 @@ package agent
 import (
 	"beszel/internal/entities/system"
 	"log/slog"
-	"time"
-
 	"os"
 	"path/filepath"
+	"runtime"
 	"strings"
+	"time"
 
 	"github.com/shirou/gopsutil/v4/disk"
 )
 
 // Sets up the filesystems to monitor for disk usage and I/O.
 func (a *Agent) initializeDiskInfo() {
-	filesystem := os.Getenv("FILESYSTEM")
+	filesystem, _ := GetEnv("FILESYSTEM")
 	efPath := "/extra-filesystems"
 	hasRoot := false
 
@@ -37,15 +37,32 @@ func (a *Agent) initializeDiskInfo() {
 
 	// Helper function to add a filesystem to fsStats if it doesn't exist
 	addFsStat := func(device, mountpoint string, root bool) {
-		key := filepath.Base(device)
+		var key string
+		if runtime.GOOS == "windows" {
+			key = device
+		} else {
+			key = filepath.Base(device)
+		}
+		var ioMatch bool
 		if _, exists := a.fsStats[key]; !exists {
 			if root {
 				slog.Info("Detected root device", "name", key)
-				// check if root device is in /proc/diskstats, use fallback if not
-				if _, exists := diskIoCounters[key]; !exists {
-					slog.Warn("Device not found in diskstats", "name", key)
-					key = findFallbackIoDevice(filesystem, diskIoCounters, a.fsStats)
-					slog.Info("Using I/O fallback", "name", key)
+				// Check if root device is in /proc/diskstats, use fallback if not
+				if _, ioMatch = diskIoCounters[key]; !ioMatch {
+					key, ioMatch = findIoDevice(filesystem, diskIoCounters, a.fsStats)
+					if !ioMatch {
+						slog.Info("Using I/O fallback", "device", device, "mountpoint", mountpoint, "fallback", key)
+					}
+				}
+			} else {
+				// Check if non-root has diskstats and fall back to folder name if not
+				// Scenario: device is encrypted and named luks-2bcb02be-999d-4417-8d18-5c61e660fb6e - not in /proc/diskstats.
+				// However, the device can be specified by mounting folder from luks device at /extra-filesystems/sda1
+				if _, ioMatch = diskIoCounters[key]; !ioMatch {
+					efBase := filepath.Base(mountpoint)
+					if _, ioMatch = diskIoCounters[efBase]; ioMatch {
+						key = efBase
+					}
 				}
 			}
 			a.fsStats[key] = &system.FsStats{Root: root, Mountpoint: mountpoint}
@@ -67,7 +84,7 @@ func (a *Agent) initializeDiskInfo() {
 	}
 
 	// Add EXTRA_FILESYSTEMS env var values to fsStats
-	if extraFilesystems, exists := os.LookupEnv("EXTRA_FILESYSTEMS"); exists {
+	if extraFilesystems, exists := GetEnv("EXTRA_FILESYSTEMS"); exists {
 		for _, fs := range strings.Split(extraFilesystems, ",") {
 			found := false
 			for _, p := range partitions {
@@ -92,9 +109,12 @@ func (a *Agent) initializeDiskInfo() {
 	for _, p := range partitions {
 		// fmt.Println(p.Device, p.Mountpoint)
 		// Binary root fallback or docker root fallback
-		if !hasRoot && (p.Mountpoint == "/" || (p.Mountpoint == "/etc/hosts" && strings.HasPrefix(p.Device, "/dev") && !strings.Contains(p.Device, "mapper"))) {
-			addFsStat(p.Device, "/", true)
-			hasRoot = true
+		if !hasRoot && (p.Mountpoint == "/" || (p.Mountpoint == "/etc/hosts" && strings.HasPrefix(p.Device, "/dev"))) {
+			fs, match := findIoDevice(filepath.Base(p.Device), diskIoCounters, a.fsStats)
+			if match {
+				addFsStat(fs, p.Mountpoint, true)
+				hasRoot = true
+			}
 		}
 
 		// Check if device is in /extra-filesystems
@@ -114,7 +134,7 @@ func (a *Agent) initializeDiskInfo() {
 				mountpoint := filepath.Join(efPath, folder.Name())
 				slog.Debug("/extra-filesystems", "mountpoint", mountpoint)
 				if !existingMountpoints[mountpoint] {
-					a.fsStats[folder.Name()] = &system.FsStats{Mountpoint: mountpoint}
+					addFsStat(folder.Name(), mountpoint, false)
 				}
 			}
 		}
@@ -122,7 +142,7 @@ func (a *Agent) initializeDiskInfo() {
 
 	// If no root filesystem set, use fallback
 	if !hasRoot {
-		rootDevice := findFallbackIoDevice(filepath.Base(filesystem), diskIoCounters, a.fsStats)
+		rootDevice, _ := findIoDevice(filepath.Base(filesystem), diskIoCounters, a.fsStats)
 		slog.Info("Root disk", "mountpoint", "/", "io", rootDevice)
 		a.fsStats[rootDevice] = &system.FsStats{Root: true, Mountpoint: "/"}
 	}
@@ -130,14 +150,15 @@ func (a *Agent) initializeDiskInfo() {
 	a.initializeDiskIoStats(diskIoCounters)
 }
 
-// Returns the device with the most reads in /proc/diskstats,
-// or the device specified by the filesystem argument if it exists
-func findFallbackIoDevice(filesystem string, diskIoCounters map[string]disk.IOCountersStat, fsStats map[string]*system.FsStats) string {
+// Returns matching device from /proc/diskstats,
+// or the device with the most reads if no match is found.
+// bool is true if a match was found.
+func findIoDevice(filesystem string, diskIoCounters map[string]disk.IOCountersStat, fsStats map[string]*system.FsStats) (string, bool) {
 	var maxReadBytes uint64
 	maxReadDevice := "/"
 	for _, d := range diskIoCounters {
-		if d.Name == filesystem {
-			return d.Name
+		if d.Name == filesystem || (d.Label != "" && d.Label == filesystem) {
+			return d.Name, true
 		}
 		if d.ReadBytes > maxReadBytes {
 			// don't use if device already exists in fsStats
@@ -147,7 +168,7 @@ func findFallbackIoDevice(filesystem string, diskIoCounters map[string]disk.IOCo
 			}
 		}
 	}
-	return maxReadDevice
+	return maxReadDevice, false
 }
 
 // Sets start values for disk I/O stats.

beszel/internal/agent/docker.go

@@ -2,6 +2,7 @@ package agent
 
 import (
 	"beszel/internal/entities/container"
+	"bytes"
 	"context"
 	"encoding/json"
 	"fmt"
@@ -22,21 +23,42 @@ type dockerManager struct {
 	wg                  sync.WaitGroup              // WaitGroup to wait for all goroutines to finish
 	sem                 chan struct{}               // Semaphore to limit concurrent container requests
 	containerStatsMutex sync.RWMutex                // Mutex to prevent concurrent access to containerStatsMap
-	apiContainerList    *[]container.ApiInfo        // List of containers from Docker API
+	apiContainerList    []*container.ApiInfo        // List of containers from Docker API (no pointer)
 	containerStatsMap   map[string]*container.Stats // Keeps track of container stats
 	validIds            map[string]struct{}         // Map of valid container ids, used to prune invalid containers from containerStatsMap
+	goodDockerVersion   bool                        // Whether docker version is at least 25.0.0 (one-shot works correctly)
+	isWindows           bool                        // Whether the Docker Engine API is running on Windows
+	buf                 *bytes.Buffer               // Buffer to store and read response bodies
+	decoder             *json.Decoder               // Reusable JSON decoder that reads from buf
+	apiStats            *container.ApiStats         // Reusable API stats object
+}
+
+// userAgentRoundTripper is a custom http.RoundTripper that adds a User-Agent header to all requests
+type userAgentRoundTripper struct {
+	rt        http.RoundTripper
+	userAgent string
+}
+
+// RoundTrip implements the http.RoundTripper interface
+func (u *userAgentRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
+	req.Header.Set("User-Agent", u.userAgent)
+	return u.rt.RoundTrip(req)
 }
 
 // Add goroutine to the queue
 func (d *dockerManager) queue() {
-	d.sem <- struct{}{}
 	d.wg.Add(1)
+	if d.goodDockerVersion {
+		d.sem <- struct{}{}
+	}
 }
 
 // Remove goroutine from the queue
 func (d *dockerManager) dequeue() {
-	<-d.sem
 	d.wg.Done()
+	if d.goodDockerVersion {
+		<-d.sem
+	}
 }
 
 // Returns stats for all running containers
@@ -45,13 +67,15 @@ func (dm *dockerManager) getDockerStats() ([]*container.Stats, error) {
 	if err != nil {
 		return nil, err
 	}
-	defer resp.Body.Close()
 
-	if err := json.NewDecoder(resp.Body).Decode(&dm.apiContainerList); err != nil {
+	dm.apiContainerList = dm.apiContainerList[:0]
+	if err := dm.decode(resp, &dm.apiContainerList); err != nil {
 		return nil, err
 	}
 
-	containersLength := len(*dm.apiContainerList)
+	dm.isWindows = strings.Contains(resp.Header.Get("Server"), "windows")
+
+	containersLength := len(dm.apiContainerList)
 
 	// store valid ids to clean up old container ids from map
 	if dm.validIds == nil {
@@ -60,9 +84,10 @@ func (dm *dockerManager) getDockerStats() ([]*container.Stats, error) {
 		clear(dm.validIds)
 	}
 
-	var failedContainters []container.ApiInfo
+	var failedContainers []*container.ApiInfo
 
-	for _, ctr := range *dm.apiContainerList {
+	for i := range dm.apiContainerList {
+		ctr := dm.apiContainerList[i]
 		ctr.IdShort = ctr.Id[:12]
 		dm.validIds[ctr.IdShort] = struct{}{}
 		// check if container is less than 1 minute old (possible restart)
@@ -75,10 +100,11 @@ func (dm *dockerManager) getDockerStats() ([]*container.Stats, error) {
 		go func() {
 			defer dm.dequeue()
 			err := dm.updateContainerStats(ctr)
+			// if error, delete from map and add to failed list to retry
 			if err != nil {
 				dm.containerStatsMutex.Lock()
 				delete(dm.containerStatsMap, ctr.IdShort)
-				failedContainters = append(failedContainters, ctr)
+				failedContainers = append(failedContainers, ctr)
 				dm.containerStatsMutex.Unlock()
 			}
 		}()
@@ -87,13 +113,13 @@ func (dm *dockerManager) getDockerStats() ([]*container.Stats, error) {
 	dm.wg.Wait()
 
 	// retry failed containers separately so we can run them in parallel (docker 24 bug)
-	if len(failedContainters) > 0 {
-		slog.Debug("Retrying failed containers", "count", len(failedContainters))
-		// time.Sleep(time.Millisecond * 1100)
-		for _, ctr := range failedContainters {
-			dm.wg.Add(1)
+	if len(failedContainers) > 0 {
+		slog.Debug("Retrying failed containers", "count", len(failedContainers))
+		for i := range failedContainers {
+			ctr := failedContainers[i]
+			dm.queue()
 			go func() {
-				defer dm.wg.Done()
+				defer dm.dequeue()
 				err = dm.updateContainerStats(ctr)
 				if err != nil {
 					slog.Error("Error getting container stats", "err", err)
@@ -117,7 +143,7 @@ func (dm *dockerManager) getDockerStats() ([]*container.Stats, error) {
 }
 
 // Updates stats for individual container
-func (dm *dockerManager) updateContainerStats(ctr container.ApiInfo) error {
+func (dm *dockerManager) updateContainerStats(ctr *container.ApiInfo) error {
 	name := ctr.Names[0][1:]
 
 	resp, err := dm.client.Get("http://localhost/containers/" + ctr.IdShort + "/stats?stream=0&one-shot=1")
@@ -143,31 +169,45 @@ func (dm *dockerManager) updateContainerStats(ctr container.ApiInfo) error {
 	stats.NetworkRecv = 0
 
 	// docker host container stats response
-	var res container.ApiStats
-	if err := json.NewDecoder(resp.Body).Decode(&res); err != nil {
+	// res := dm.getApiStats()
+	// defer dm.putApiStats(res)
+	//
+
+	res := dm.apiStats
+	res.Networks = nil
+	if err := dm.decode(resp, res); err != nil {
 		return err
 	}
 
-	// check if container has valid data, otherwise may be in restart loop (#103)
-	if res.MemoryStats.Usage == 0 {
-		return fmt.Errorf("%s - no memory stats - see https://github.com/henrygd/beszel/issues/144", name)
+	// calculate cpu and memory stats
+	var usedMemory uint64
+	var cpuPct float64
+
+	// store current cpu stats
+	prevCpuContainer, prevCpuSystem := stats.CpuContainer, stats.CpuSystem
+	stats.CpuContainer = res.CPUStats.CPUUsage.TotalUsage
+	stats.CpuSystem = res.CPUStats.SystemUsage
+
+	if dm.isWindows {
+		usedMemory = res.MemoryStats.PrivateWorkingSet
+		cpuPct = res.CalculateCpuPercentWindows(prevCpuContainer, stats.PrevReadTime)
+	} else {
+		// check if container has valid data, otherwise may be in restart loop (#103)
+		if res.MemoryStats.Usage == 0 {
+			return fmt.Errorf("%s - no memory stats - see https://github.com/henrygd/beszel/issues/144", name)
+		}
+		memCache := res.MemoryStats.Stats.InactiveFile
+		if memCache == 0 {
+			memCache = res.MemoryStats.Stats.Cache
+		}
+		usedMemory = res.MemoryStats.Usage - memCache
+
+		cpuPct = res.CalculateCpuPercentLinux(prevCpuContainer, prevCpuSystem)
 	}
 
-	// memory (https://docs.docker.com/reference/cli/docker/container/stats/)
-	memCache := res.MemoryStats.Stats.InactiveFile
-	if memCache == 0 {
-		memCache = res.MemoryStats.Stats.Cache
-	}
-	usedMemory := res.MemoryStats.Usage - memCache
-
-	// cpu
-	cpuDelta := res.CPUStats.CPUUsage.TotalUsage - stats.PrevCpu[0]
-	systemDelta := res.CPUStats.SystemUsage - stats.PrevCpu[1]
-	cpuPct := float64(cpuDelta) / float64(systemDelta) * 100
 	if cpuPct > 100 {
 		return fmt.Errorf("%s cpu pct greater than 100: %+v", name, cpuPct)
 	}
-	stats.PrevCpu = [2]uint64{res.CPUStats.CPUUsage.TotalUsage, res.CPUStats.SystemUsage}
 
 	// network
 	var total_sent, total_recv uint64
@@ -175,21 +215,25 @@ func (dm *dockerManager) updateContainerStats(ctr container.ApiInfo) error {
 		total_sent += v.TxBytes
 		total_recv += v.RxBytes
 	}
-	var sent_delta, recv_delta float64
-	// prevent first run from sending all prev sent/recv bytes
-	if initialized {
-		secondsElapsed := time.Since(stats.PrevNet.Time).Seconds()
-		sent_delta = float64(total_sent-stats.PrevNet.Sent) / secondsElapsed
-		recv_delta = float64(total_recv-stats.PrevNet.Recv) / secondsElapsed
+	var sent_delta, recv_delta uint64
+	millisecondsElapsed := uint64(time.Since(stats.PrevReadTime).Milliseconds())
+	if initialized && millisecondsElapsed > 0 {
+		// get bytes per second
+		sent_delta = (total_sent - stats.PrevNet.Sent) * 1000 / millisecondsElapsed
+		recv_delta = (total_recv - stats.PrevNet.Recv) * 1000 / millisecondsElapsed
+		// check for unrealistic network values (> 5GB/s)
+		if sent_delta > 5e9 || recv_delta > 5e9 {
+			slog.Warn("Bad network delta", "container", name)
+			sent_delta, recv_delta = 0, 0
+		}
 	}
-	stats.PrevNet.Sent = total_sent
-	stats.PrevNet.Recv = total_recv
-	stats.PrevNet.Time = time.Now()
+	stats.PrevNet.Sent, stats.PrevNet.Recv = total_sent, total_recv
 
 	stats.Cpu = twoDecimals(cpuPct)
 	stats.Mem = bytesToMegabytes(float64(usedMemory))
-	stats.NetworkSent = bytesToMegabytes(sent_delta)
-	stats.NetworkRecv = bytesToMegabytes(recv_delta)
+	stats.NetworkSent = bytesToMegabytes(float64(sent_delta))
+	stats.NetworkRecv = bytesToMegabytes(float64(recv_delta))
+	stats.PrevReadTime = res.Read
 
 	return nil
 }
@@ -201,17 +245,20 @@ func (dm *dockerManager) deleteContainerStatsSync(id string) {
 	delete(dm.containerStatsMap, id)
 }
 
-// Creates a new http client for Docker API
-func newDockerManager() *dockerManager {
-	dockerHost := "unix:///var/run/docker.sock"
-	if dockerHostEnv, exists := os.LookupEnv("DOCKER_HOST"); exists {
-		slog.Info("DOCKER_HOST", "host", dockerHostEnv)
-		dockerHost = dockerHostEnv
+// Creates a new http client for Docker or Podman API
+func newDockerManager(a *Agent) *dockerManager {
+	dockerHost, exists := GetEnv("DOCKER_HOST")
+	if exists {
+		// return nil if set to empty string
+		if dockerHost == "" {
+			return nil
+		}
+	} else {
+		dockerHost = getDockerHost()
 	}
 
 	parsedURL, err := url.Parse(dockerHost)
 	if err != nil {
-		slog.Error("Error parsing DOCKER_HOST", "err", err)
 		os.Exit(1)
 	}
 
@@ -236,7 +283,7 @@ func newDockerManager() *dockerManager {
 
 	// configurable timeout
 	timeout := time.Millisecond * 2100
-	if t, set := os.LookupEnv("DOCKER_TIMEOUT"); set {
+	if t, set := GetEnv("DOCKER_TIMEOUT"); set {
 		timeout, err = time.ParseDuration(t)
 		if err != nil {
 			slog.Error(err.Error())
@@ -245,37 +292,78 @@ func newDockerManager() *dockerManager {
 		slog.Info("DOCKER_TIMEOUT", "timeout", timeout)
 	}
 
-	dockerClient := &dockerManager{
-		client: &http.Client{
-			Timeout:   timeout,
-			Transport: transport,
-		},
-		containerStatsMap: make(map[string]*container.Stats),
+	// Custom user-agent to avoid docker bug: https://github.com/docker/for-mac/issues/7575
+	userAgentTransport := &userAgentRoundTripper{
+		rt:        transport,
+		userAgent: "Docker-Client/",
 	}
 
-	// Make sure sem is initialized
-	concurrency := 200
-	defer func() { dockerClient.sem = make(chan struct{}, concurrency) }()
+	manager := &dockerManager{
+		client: &http.Client{
+			Timeout:   timeout,
+			Transport: userAgentTransport,
+		},
+		containerStatsMap: make(map[string]*container.Stats),
+		sem:               make(chan struct{}, 5),
+		apiContainerList:  []*container.ApiInfo{},
+		apiStats:          &container.ApiStats{},
+	}
+
+	// If using podman, return client
+	if strings.Contains(dockerHost, "podman") {
+		a.systemInfo.Podman = true
+		manager.goodDockerVersion = true
+		return manager
+	}
 
 	// Check docker version
 	// (versions before 25.0.0 have a bug with one-shot which requires all requests to be made in one batch)
 	var versionInfo struct {
 		Version string `json:"Version"`
 	}
-	resp, err := dockerClient.client.Get("http://localhost/version")
+	resp, err := manager.client.Get("http://localhost/version")
 	if err != nil {
-		return dockerClient
+		return manager
 	}
 
-	if err := json.NewDecoder(resp.Body).Decode(&versionInfo); err != nil {
-		return dockerClient
+	if err := manager.decode(resp, &versionInfo); err != nil {
+		return manager
 	}
 
 	// if version > 24, one-shot works correctly and we can limit concurrent operations
 	if dockerVersion, err := semver.Parse(versionInfo.Version); err == nil && dockerVersion.Major > 24 {
-		concurrency = 5
+		manager.goodDockerVersion = true
+	} else {
+		slog.Info(fmt.Sprintf("Docker %s is outdated. Upgrade if possible. See https://github.com/henrygd/beszel/issues/58", versionInfo.Version))
 	}
-	slog.Debug("Docker", "version", versionInfo.Version, "concurrency", concurrency)
 
-	return dockerClient
+	return manager
+}
+
+// Decodes Docker API JSON response using a reusable buffer and decoder. Not thread safe.
+func (dm *dockerManager) decode(resp *http.Response, d any) error {
+	if dm.buf == nil {
+		// initialize buffer with 256kb starting size
+		dm.buf = bytes.NewBuffer(make([]byte, 0, 1024*256))
+		dm.decoder = json.NewDecoder(dm.buf)
+	}
+	defer resp.Body.Close()
+	defer dm.buf.Reset()
+	_, err := dm.buf.ReadFrom(resp.Body)
+	if err != nil {
+		return err
+	}
+	return dm.decoder.Decode(d)
+}
+
+// Test docker / podman sockets and return if one exists
+func getDockerHost() string {
+	scheme := "unix://"
+	socks := []string{"/var/run/docker.sock", fmt.Sprintf("/run/user/%v/podman/podman.sock", os.Getuid())}
+	for _, sock := range socks {
+		if _, err := os.Stat(sock); err == nil {
+			return scheme + sock
+		}
+	}
+	return scheme + socks[0]
 }

beszel/internal/agent/gpu.go

@@ -0,0 +1,348 @@
+package agent
+
+import (
+	"beszel/internal/entities/system"
+	"bufio"
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"os/exec"
+	"regexp"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+
+	"golang.org/x/exp/slog"
+)
+
+const (
+	// Commands
+	nvidiaSmiCmd  string = "nvidia-smi"
+	rocmSmiCmd    string = "rocm-smi"
+	tegraStatsCmd string = "tegrastats"
+
+	// Polling intervals
+	nvidiaSmiInterval  string        = "4"    // in seconds
+	tegraStatsInterval string        = "3700" // in milliseconds
+	rocmSmiInterval    time.Duration = 4300 * time.Millisecond
+
+	// Command retry and timeout constants
+	retryWaitTime     time.Duration = 5 * time.Second
+	maxFailureRetries int           = 5
+
+	cmdBufferSize uint16 = 10 * 1024
+
+	// Unit Conversions
+	mebibytesInAMegabyte float64 = 1.024  // nvidia-smi reports memory in MiB
+	milliwattsInAWatt    float64 = 1000.0 // tegrastats reports power in mW
+)
+
+// GPUManager manages data collection for GPUs (either Nvidia or AMD)
+type GPUManager struct {
+	sync.Mutex
+	nvidiaSmi  bool
+	rocmSmi    bool
+	tegrastats bool
+	GpuDataMap map[string]*system.GPUData
+}
+
+// RocmSmiJson represents the JSON structure of rocm-smi output
+type RocmSmiJson struct {
+	ID           string `json:"GUID"`
+	Name         string `json:"Card series"`
+	Temperature  string `json:"Temperature (Sensor edge) (C)"`
+	MemoryUsed   string `json:"VRAM Total Used Memory (B)"`
+	MemoryTotal  string `json:"VRAM Total Memory (B)"`
+	Usage        string `json:"GPU use (%)"`
+	PowerPackage string `json:"Average Graphics Package Power (W)"`
+	PowerSocket  string `json:"Current Socket Graphics Package Power (W)"`
+}
+
+// gpuCollector defines a collector for a specific GPU management utility (nvidia-smi or rocm-smi)
+type gpuCollector struct {
+	name    string
+	cmdArgs []string
+	parse   func([]byte) bool // returns true if valid data was found
+	buf     []byte
+}
+
+var errNoValidData = fmt.Errorf("no valid GPU data found") // Error for missing data
+
+// starts and manages the ongoing collection of GPU data for the specified GPU management utility
+func (c *gpuCollector) start() {
+	for {
+		err := c.collect()
+		if err != nil {
+			if err == errNoValidData {
+				slog.Warn(c.name + " found no valid GPU data, stopping")
+				break
+			}
+			slog.Warn(c.name+" failed, restarting", "err", err)
+			time.Sleep(retryWaitTime)
+			continue
+		}
+	}
+}
+
+// collect executes the command, parses output with the assigned parser function
+func (c *gpuCollector) collect() error {
+	cmd := exec.Command(c.name, c.cmdArgs...)
+	stdout, err := cmd.StdoutPipe()
+	if err != nil {
+		return err
+	}
+	if err := cmd.Start(); err != nil {
+		return err
+	}
+
+	scanner := bufio.NewScanner(stdout)
+	if c.buf == nil {
+		c.buf = make([]byte, 0, cmdBufferSize)
+	}
+	scanner.Buffer(c.buf, bufio.MaxScanTokenSize)
+
+	for scanner.Scan() {
+		hasValidData := c.parse(scanner.Bytes())
+		if !hasValidData {
+			return errNoValidData
+		}
+	}
+
+	if err := scanner.Err(); err != nil {
+		return fmt.Errorf("scanner error: %w", err)
+	}
+	return cmd.Wait()
+}
+
+// getJetsonParser returns a function to parse the output of tegrastats and update the GPUData map
+func (gm *GPUManager) getJetsonParser() func(output []byte) bool {
+	// use closure to avoid recompiling the regex
+	ramPattern := regexp.MustCompile(`RAM (\d+)/(\d+)MB`)
+	gr3dPattern := regexp.MustCompile(`GR3D_FREQ (\d+)%`)
+	tempPattern := regexp.MustCompile(`tj@(\d+\.?\d*)C`)
+	// Orin Nano / NX do not have GPU specific power monitor
+	// TODO: Maybe use VDD_IN for Nano / NX and add a total system power chart
+	powerPattern := regexp.MustCompile(`(GPU_SOC|CPU_GPU_CV) (\d+)mW`)
+
+	// jetson devices have only one gpu so we'll just initialize here
+	gpuData := &system.GPUData{Name: "GPU"}
+	gm.GpuDataMap["0"] = gpuData
+
+	return func(output []byte) bool {
+		gm.Lock()
+		defer gm.Unlock()
+		// Parse RAM usage
+		ramMatches := ramPattern.FindSubmatch(output)
+		if ramMatches != nil {
+			gpuData.MemoryUsed, _ = strconv.ParseFloat(string(ramMatches[1]), 64)
+			gpuData.MemoryTotal, _ = strconv.ParseFloat(string(ramMatches[2]), 64)
+		}
+		// Parse GR3D (GPU) usage
+		gr3dMatches := gr3dPattern.FindSubmatch(output)
+		if gr3dMatches != nil {
+			gr3dUsage, _ := strconv.ParseFloat(string(gr3dMatches[1]), 64)
+			gpuData.Usage += gr3dUsage
+		}
+		// Parse temperature
+		tempMatches := tempPattern.FindSubmatch(output)
+		if tempMatches != nil {
+			gpuData.Temperature, _ = strconv.ParseFloat(string(tempMatches[1]), 64)
+		}
+		// Parse power usage
+		powerMatches := powerPattern.FindSubmatch(output)
+		if powerMatches != nil {
+			power, _ := strconv.ParseFloat(string(powerMatches[2]), 64)
+			gpuData.Power += power / milliwattsInAWatt
+		}
+		gpuData.Count++
+		return true
+	}
+}
+
+// parseNvidiaData parses the output of nvidia-smi and updates the GPUData map
+func (gm *GPUManager) parseNvidiaData(output []byte) bool {
+	gm.Lock()
+	defer gm.Unlock()
+	scanner := bufio.NewScanner(bytes.NewReader(output))
+	var valid bool
+	for scanner.Scan() {
+		line := scanner.Text() // Or use scanner.Bytes() for []byte
+		fields := strings.Split(strings.TrimSpace(line), ", ")
+		if len(fields) < 7 {
+			continue
+		}
+		valid = true
+		id := fields[0]
+		temp, _ := strconv.ParseFloat(fields[2], 64)
+		memoryUsage, _ := strconv.ParseFloat(fields[3], 64)
+		totalMemory, _ := strconv.ParseFloat(fields[4], 64)
+		usage, _ := strconv.ParseFloat(fields[5], 64)
+		power, _ := strconv.ParseFloat(fields[6], 64)
+		// add gpu if not exists
+		if _, ok := gm.GpuDataMap[id]; !ok {
+			name := strings.TrimPrefix(fields[1], "NVIDIA ")
+			gm.GpuDataMap[id] = &system.GPUData{Name: strings.TrimSuffix(name, " Laptop GPU")}
+		}
+		// update gpu data
+		gpu := gm.GpuDataMap[id]
+		gpu.Temperature = temp
+		gpu.MemoryUsed = memoryUsage / mebibytesInAMegabyte
+		gpu.MemoryTotal = totalMemory / mebibytesInAMegabyte
+		gpu.Usage += usage
+		gpu.Power += power
+		gpu.Count++
+	}
+	return valid
+}
+
+// parseAmdData parses the output of rocm-smi and updates the GPUData map
+func (gm *GPUManager) parseAmdData(output []byte) bool {
+	var rocmSmiInfo map[string]RocmSmiJson
+	if err := json.Unmarshal(output, &rocmSmiInfo); err != nil || len(rocmSmiInfo) == 0 {
+		return false
+	}
+	gm.Lock()
+	defer gm.Unlock()
+	for _, v := range rocmSmiInfo {
+		var power float64
+		if v.PowerPackage != "" {
+			power, _ = strconv.ParseFloat(v.PowerPackage, 64)
+		} else {
+			power, _ = strconv.ParseFloat(v.PowerSocket, 64)
+		}
+		memoryUsage, _ := strconv.ParseFloat(v.MemoryUsed, 64)
+		totalMemory, _ := strconv.ParseFloat(v.MemoryTotal, 64)
+		usage, _ := strconv.ParseFloat(v.Usage, 64)
+
+		if _, ok := gm.GpuDataMap[v.ID]; !ok {
+			gm.GpuDataMap[v.ID] = &system.GPUData{Name: v.Name}
+		}
+		gpu := gm.GpuDataMap[v.ID]
+		gpu.Temperature, _ = strconv.ParseFloat(v.Temperature, 64)
+		gpu.MemoryUsed = bytesToMegabytes(memoryUsage)
+		gpu.MemoryTotal = bytesToMegabytes(totalMemory)
+		gpu.Usage += usage
+		gpu.Power += power
+		gpu.Count++
+	}
+	return true
+}
+
+// sums and resets the current GPU utilization data since the last update
+func (gm *GPUManager) GetCurrentData() map[string]system.GPUData {
+	gm.Lock()
+	defer gm.Unlock()
+
+	// check for GPUs with the same name
+	nameCounts := make(map[string]int)
+	for _, gpu := range gm.GpuDataMap {
+		nameCounts[gpu.Name]++
+	}
+
+	// copy / reset the data
+	gpuData := make(map[string]system.GPUData, len(gm.GpuDataMap))
+	for id, gpu := range gm.GpuDataMap {
+		gpuAvg := *gpu
+
+		gpuAvg.Temperature = twoDecimals(gpu.Temperature)
+		gpuAvg.MemoryUsed = twoDecimals(gpu.MemoryUsed)
+		gpuAvg.MemoryTotal = twoDecimals(gpu.MemoryTotal)
+
+		// avoid division by zero
+		if gpu.Count > 0 {
+			gpuAvg.Usage = twoDecimals(gpu.Usage / gpu.Count)
+			gpuAvg.Power = twoDecimals(gpu.Power / gpu.Count)
+		}
+
+		// reset accumulators in the original
+		gpu.Usage, gpu.Power, gpu.Count = 0, 0, 0
+
+		// append id to the name if there are multiple GPUs with the same name
+		if nameCounts[gpu.Name] > 1 {
+			gpuAvg.Name = fmt.Sprintf("%s %s", gpu.Name, id)
+		}
+		gpuData[id] = gpuAvg
+	}
+	slog.Debug("GPU", "data", gpuData)
+	return gpuData
+}
+
+// detectGPUs checks for the presence of GPU management tools (nvidia-smi, rocm-smi, tegrastats)
+// in the system path. It sets the corresponding flags in the GPUManager struct if any of these
+// tools are found. If none of the tools are found, it returns an error indicating that no GPU
+// management tools are available.
+func (gm *GPUManager) detectGPUs() error {
+	if _, err := exec.LookPath(nvidiaSmiCmd); err == nil {
+		gm.nvidiaSmi = true
+	}
+	if _, err := exec.LookPath(rocmSmiCmd); err == nil {
+		gm.rocmSmi = true
+	}
+	if _, err := exec.LookPath(tegraStatsCmd); err == nil {
+		gm.tegrastats = true
+		gm.nvidiaSmi = false
+	}
+	if gm.nvidiaSmi || gm.rocmSmi || gm.tegrastats {
+		return nil
+	}
+	return fmt.Errorf("no GPU found - install nvidia-smi, rocm-smi, or tegrastats")
+}
+
+// startCollector starts the appropriate GPU data collector based on the command
+func (gm *GPUManager) startCollector(command string) {
+	collector := gpuCollector{
+		name: command,
+	}
+	switch command {
+	case nvidiaSmiCmd:
+		collector.cmdArgs = []string{
+			"-l", nvidiaSmiInterval,
+			"--query-gpu=index,name,temperature.gpu,memory.used,memory.total,utilization.gpu,power.draw",
+			"--format=csv,noheader,nounits",
+		}
+		collector.parse = gm.parseNvidiaData
+		go collector.start()
+	case tegraStatsCmd:
+		collector.cmdArgs = []string{"--interval", tegraStatsInterval}
+		collector.parse = gm.getJetsonParser()
+		go collector.start()
+	case rocmSmiCmd:
+		collector.cmdArgs = []string{"--showid", "--showtemp", "--showuse", "--showpower", "--showproductname", "--showmeminfo", "vram", "--json"}
+		collector.parse = gm.parseAmdData
+		go func() {
+			failures := 0
+			for {
+				if err := collector.collect(); err != nil {
+					failures++
+					if failures > maxFailureRetries {
+						break
+					}
+					slog.Warn("Error collecting AMD GPU data", "err", err)
+				}
+				time.Sleep(rocmSmiInterval)
+			}
+		}()
+	}
+}
+
+// NewGPUManager creates and initializes a new GPUManager
+func NewGPUManager() (*GPUManager, error) {
+	var gm GPUManager
+	if err := gm.detectGPUs(); err != nil {
+		return nil, err
+	}
+	gm.GpuDataMap = make(map[string]*system.GPUData)
+
+	if gm.nvidiaSmi {
+		gm.startCollector(nvidiaSmiCmd)
+	}
+	if gm.rocmSmi {
+		gm.startCollector(rocmSmiCmd)
+	}
+	if gm.tegrastats {
+		gm.startCollector(tegraStatsCmd)
+	}
+
+	return &gm, nil
+}

beszel/internal/agent/gpu_test.go

@@ -0,0 +1,793 @@
+//go:build testing
+// +build testing
+
+package agent
+
+import (
+	"beszel/internal/entities/system"
+	"os"
+	"path/filepath"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestParseNvidiaData(t *testing.T) {
+	tests := []struct {
+		name      string
+		input     string
+		wantData  map[string]system.GPUData
+		wantValid bool
+	}{
+		{
+			name:  "valid multi-gpu data",
+			input: "0, NVIDIA GeForce RTX 3050 Ti Laptop GPU, 48, 12, 4096, 26.3, 12.73\n1, NVIDIA A100-PCIE-40GB, 38, 74, 40960, [N/A], 36.79",
+			wantData: map[string]system.GPUData{
+				"0": {
+					Name:        "GeForce RTX 3050 Ti",
+					Temperature: 48.0,
+					MemoryUsed:  12.0 / 1.024,
+					MemoryTotal: 4096.0 / 1.024,
+					Usage:       26.3,
+					Power:       12.73,
+					Count:       1,
+				},
+				"1": {
+					Name:        "A100-PCIE-40GB",
+					Temperature: 38.0,
+					MemoryUsed:  74.0 / 1.024,
+					MemoryTotal: 40960.0 / 1.024,
+					Usage:       0.0,
+					Power:       36.79,
+					Count:       1,
+				},
+			},
+			wantValid: true,
+		},
+		{
+			name: "more valid multi-gpu data",
+			input: `0, NVIDIA A10, 45, 19676, 23028, 0, 58.98
+1, NVIDIA A10, 45, 19638, 23028, 0, 62.35
+2, NVIDIA A10, 44, 21700, 23028, 0, 59.57
+3, NVIDIA A10, 45, 18222, 23028, 0, 61.76`,
+			wantData: map[string]system.GPUData{
+				"0": {
+					Name:        "A10",
+					Temperature: 45.0,
+					MemoryUsed:  19676.0 / 1.024,
+					MemoryTotal: 23028.0 / 1.024,
+					Usage:       0.0,
+					Power:       58.98,
+					Count:       1,
+				},
+				"1": {
+					Name:        "A10",
+					Temperature: 45.0,
+					MemoryUsed:  19638.0 / 1.024,
+					MemoryTotal: 23028.0 / 1.024,
+					Usage:       0.0,
+					Power:       62.35,
+					Count:       1,
+				},
+				"2": {
+					Name:        "A10",
+					Temperature: 44.0,
+					MemoryUsed:  21700.0 / 1.024,
+					MemoryTotal: 23028.0 / 1.024,
+					Usage:       0.0,
+					Power:       59.57,
+					Count:       1,
+				},
+				"3": {
+					Name:        "A10",
+					Temperature: 45.0,
+					MemoryUsed:  18222.0 / 1.024,
+					MemoryTotal: 23028.0 / 1.024,
+					Usage:       0.0,
+					Power:       61.76,
+					Count:       1,
+				},
+			},
+			wantValid: true,
+		},
+		{
+			name:      "empty input",
+			input:     "",
+			wantData:  map[string]system.GPUData{},
+			wantValid: false,
+		},
+		{
+			name:      "malformed data",
+			input:     "bad, data, here",
+			wantData:  map[string]system.GPUData{},
+			wantValid: false,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			gm := &GPUManager{
+				GpuDataMap: make(map[string]*system.GPUData),
+			}
+			valid := gm.parseNvidiaData([]byte(tt.input))
+			assert.Equal(t, tt.wantValid, valid)
+
+			if tt.wantValid {
+				for id, want := range tt.wantData {
+					got := gm.GpuDataMap[id]
+					require.NotNil(t, got)
+					assert.Equal(t, want.Name, got.Name)
+					assert.InDelta(t, want.Temperature, got.Temperature, 0.01)
+					assert.InDelta(t, want.MemoryUsed, got.MemoryUsed, 0.01)
+					assert.InDelta(t, want.MemoryTotal, got.MemoryTotal, 0.01)
+					assert.InDelta(t, want.Usage, got.Usage, 0.01)
+					assert.InDelta(t, want.Power, got.Power, 0.01)
+					assert.Equal(t, want.Count, got.Count)
+				}
+			}
+		})
+	}
+}
+
+func TestParseAmdData(t *testing.T) {
+	tests := []struct {
+		name      string
+		input     string
+		wantData  map[string]system.GPUData
+		wantValid bool
+	}{
+		{
+			name: "valid single gpu data",
+			input: `{
+				"card0": {
+					"GUID": "34756",
+					"Temperature (Sensor edge) (C)": "47.0",
+					"Current Socket Graphics Package Power (W)": "9.215",
+					"GPU use (%)": "0",
+					"VRAM Total Memory (B)": "536870912",
+					"VRAM Total Used Memory (B)": "482263040",
+					"Card Series": "Rembrandt [Radeon 680M]"
+				}
+			}`,
+			wantData: map[string]system.GPUData{
+				"34756": {
+					Name:        "Rembrandt [Radeon 680M]",
+					Temperature: 47.0,
+					MemoryUsed:  482263040.0 / (1024 * 1024),
+					MemoryTotal: 536870912.0 / (1024 * 1024),
+					Usage:       0.0,
+					Power:       9.215,
+					Count:       1,
+				},
+			},
+			wantValid: true,
+		},
+		{
+			name: "valid multi gpu data",
+			input: `{
+				"card0": {
+					"GUID": "34756",
+					"Temperature (Sensor edge) (C)": "47.0",
+					"Current Socket Graphics Package Power (W)": "9.215",
+					"GPU use (%)": "0",
+					"VRAM Total Memory (B)": "536870912",
+					"VRAM Total Used Memory (B)": "482263040",
+					"Card Series": "Rembrandt [Radeon 680M]"
+				},
+				"card1": {
+					"GUID": "38294",
+					"Temperature (Sensor edge) (C)": "49.0",
+					"Temperature (Sensor junction) (C)": "49.0",
+					"Temperature (Sensor memory) (C)": "62.0",
+					"Average Graphics Package Power (W)": "19.0",
+					"GPU use (%)": "20.3",
+					"VRAM Total Memory (B)": "25753026560",
+					"VRAM Total Used Memory (B)": "794341376",
+					"Card Series": "Navi 31 [Radeon RX 7900 XT]"
+				}
+			}`,
+			wantData: map[string]system.GPUData{
+				"34756": {
+					Name:        "Rembrandt [Radeon 680M]",
+					Temperature: 47.0,
+					MemoryUsed:  482263040.0 / (1024 * 1024),
+					MemoryTotal: 536870912.0 / (1024 * 1024),
+					Usage:       0.0,
+					Power:       9.215,
+					Count:       1,
+				},
+				"38294": {
+					Name:        "Navi 31 [Radeon RX 7900 XT]",
+					Temperature: 49.0,
+					MemoryUsed:  794341376.0 / (1024 * 1024),
+					MemoryTotal: 25753026560.0 / (1024 * 1024),
+					Usage:       20.3,
+					Power:       19.0,
+					Count:       1,
+				},
+			},
+			wantValid: true,
+		},
+		{
+			name:  "invalid json",
+			input: "{bad json",
+		},
+		{
+			name:      "invalid json",
+			input:     "{bad json",
+			wantData:  map[string]system.GPUData{},
+			wantValid: false,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			gm := &GPUManager{
+				GpuDataMap: make(map[string]*system.GPUData),
+			}
+			valid := gm.parseAmdData([]byte(tt.input))
+			assert.Equal(t, tt.wantValid, valid)
+
+			if tt.wantValid {
+				for id, want := range tt.wantData {
+					got := gm.GpuDataMap[id]
+					require.NotNil(t, got)
+					assert.Equal(t, want.Name, got.Name)
+					assert.InDelta(t, want.Temperature, got.Temperature, 0.01)
+					assert.InDelta(t, want.MemoryUsed, got.MemoryUsed, 0.01)
+					assert.InDelta(t, want.MemoryTotal, got.MemoryTotal, 0.01)
+					assert.InDelta(t, want.Usage, got.Usage, 0.01)
+					assert.InDelta(t, want.Power, got.Power, 0.01)
+					assert.Equal(t, want.Count, got.Count)
+				}
+			}
+		})
+	}
+}
+
+func TestParseJetsonData(t *testing.T) {
+	tests := []struct {
+		name        string
+		input       string
+		wantMetrics *system.GPUData
+	}{
+		{
+			name:  "valid data",
+			input: "11-14-2024 22:54:33 RAM 4300/30698MB GR3D_FREQ 45% tj@52.468C VDD_GPU_SOC 2171mW",
+			wantMetrics: &system.GPUData{
+				Name:        "GPU",
+				MemoryUsed:  4300.0,
+				MemoryTotal: 30698.0,
+				Usage:       45.0,
+				Temperature: 52.468,
+				Power:       2.171,
+				Count:       1,
+			},
+		},
+		{
+			name:  "more valid data",
+			input: "11-15-2024 08:38:09 RAM 6185/7620MB (lfb 8x2MB) SWAP 851/3810MB (cached 1MB) CPU [15%@729,11%@729,14%@729,13%@729,11%@729,8%@729] EMC_FREQ 43%@2133 GR3D_FREQ 63%@[621] NVDEC off NVJPG off NVJPG1 off VIC off OFA off APE 200 cpu@53.968C soc2@52.437C soc0@50.75C gpu@53.343C tj@53.968C soc1@51.656C VDD_IN 12479mW/12479mW VDD_CPU_GPU_CV 4667mW/4667mW VDD_SOC 2817mW/2817mW",
+			wantMetrics: &system.GPUData{
+				Name:        "GPU",
+				MemoryUsed:  6185.0,
+				MemoryTotal: 7620.0,
+				Usage:       63.0,
+				Temperature: 53.968,
+				Power:       4.667,
+				Count:       1,
+			},
+		},
+		{
+			name:  "orin nano",
+			input: "06-18-2025 11:25:24 RAM 3452/7620MB (lfb 25x4MB) SWAP 1518/16384MB (cached 174MB) CPU [1%@1420,2%@1420,0%@1420,2%@1420,2%@729,1%@729] GR3D_FREQ 0% cpu@50.031C soc2@49.031C soc0@50C gpu@49.031C tj@50.25C soc1@50.25C VDD_IN 4824mW/4824mW VDD_CPU_GPU_CV 518mW/518mW VDD_SOC 1475mW/1475mW",
+			wantMetrics: &system.GPUData{
+				Name:        "GPU",
+				MemoryUsed:  3452.0,
+				MemoryTotal: 7620.0,
+				Usage:       0.0,
+				Temperature: 50.25,
+				Power:       0.518,
+				Count:       1,
+			},
+		},
+		{
+			name:  "missing temperature",
+			input: "11-14-2024 22:54:33 RAM 4300/30698MB GR3D_FREQ 45% VDD_GPU_SOC 2171mW",
+			wantMetrics: &system.GPUData{
+				Name:        "GPU",
+				MemoryUsed:  4300.0,
+				MemoryTotal: 30698.0,
+				Usage:       45.0,
+				Power:       2.171,
+				Count:       1,
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			gm := &GPUManager{
+				GpuDataMap: make(map[string]*system.GPUData),
+			}
+			parser := gm.getJetsonParser()
+			valid := parser([]byte(tt.input))
+			assert.Equal(t, true, valid)
+
+			got := gm.GpuDataMap["0"]
+			require.NotNil(t, got)
+			assert.Equal(t, tt.wantMetrics.Name, got.Name)
+			assert.InDelta(t, tt.wantMetrics.MemoryUsed, got.MemoryUsed, 0.01)
+			assert.InDelta(t, tt.wantMetrics.MemoryTotal, got.MemoryTotal, 0.01)
+			assert.InDelta(t, tt.wantMetrics.Usage, got.Usage, 0.01)
+			if tt.wantMetrics.Temperature > 0 {
+				assert.InDelta(t, tt.wantMetrics.Temperature, got.Temperature, 0.01)
+			}
+			assert.InDelta(t, tt.wantMetrics.Power, got.Power, 0.01)
+			assert.Equal(t, tt.wantMetrics.Count, got.Count)
+		})
+	}
+}
+
+func TestGetCurrentData(t *testing.T) {
+	t.Run("calculates averages and resets accumulators", func(t *testing.T) {
+		gm := &GPUManager{
+			GpuDataMap: map[string]*system.GPUData{
+				"0": {
+					Name:        "GPU1",
+					Temperature: 50,
+					MemoryUsed:  2048,
+					MemoryTotal: 4096,
+					Usage:       100, // 100 over 2 counts = 50 avg
+					Power:       200, // 200 over 2 counts = 100 avg
+					Count:       2,
+				},
+				"1": {
+					Name:        "GPU1",
+					Temperature: 60,
+					MemoryUsed:  3072,
+					MemoryTotal: 8192,
+					Usage:       30,
+					Power:       60,
+					Count:       1,
+				},
+				"2": {
+					Name:        "GPU 2",
+					Temperature: 70,
+					MemoryUsed:  4096,
+					MemoryTotal: 8192,
+					Usage:       200,
+					Power:       400,
+					Count:       1,
+				},
+			},
+		}
+
+		result := gm.GetCurrentData()
+
+		// Verify name disambiguation
+		assert.Equal(t, "GPU1 0", result["0"].Name)
+		assert.Equal(t, "GPU1 1", result["1"].Name)
+		assert.Equal(t, "GPU 2", result["2"].Name)
+
+		// Check averaged values in the result
+		assert.InDelta(t, 50.0, result["0"].Usage, 0.01)
+		assert.InDelta(t, 100.0, result["0"].Power, 0.01)
+		assert.InDelta(t, 30.0, result["1"].Usage, 0.01)
+		assert.InDelta(t, 60.0, result["1"].Power, 0.01)
+
+		// Verify that accumulators in the original map are reset
+		assert.Equal(t, float64(0), gm.GpuDataMap["0"].Count, "GPU 0 Count should be reset")
+		assert.Equal(t, float64(0), gm.GpuDataMap["0"].Usage, "GPU 0 Usage should be reset")
+		assert.Equal(t, float64(0), gm.GpuDataMap["0"].Power, "GPU 0 Power should be reset")
+		assert.Equal(t, float64(0), gm.GpuDataMap["1"].Count, "GPU 1 Count should be reset")
+		assert.Equal(t, float64(0), gm.GpuDataMap["1"].Usage, "GPU 1 Usage should be reset")
+		assert.Equal(t, float64(0), gm.GpuDataMap["1"].Power, "GPU 1 Power should be reset")
+	})
+
+	t.Run("handles zero count without panicking", func(t *testing.T) {
+		gm := &GPUManager{
+			GpuDataMap: map[string]*system.GPUData{
+				"0": {
+					Name:  "TestGPU",
+					Count: 0,
+					Usage: 0,
+					Power: 0,
+				},
+			},
+		}
+
+		var result map[string]system.GPUData
+		assert.NotPanics(t, func() {
+			result = gm.GetCurrentData()
+		})
+
+		// Check that usage and power are 0
+		assert.Equal(t, 0.0, result["0"].Usage)
+		assert.Equal(t, 0.0, result["0"].Power)
+
+		// Verify reset count
+		assert.Equal(t, float64(0), gm.GpuDataMap["0"].Count)
+	})
+}
+
+func TestDetectGPUs(t *testing.T) {
+	// Save original PATH
+	origPath := os.Getenv("PATH")
+	defer os.Setenv("PATH", origPath)
+
+	// Set up temp dir with the commands
+	tempDir := t.TempDir()
+	os.Setenv("PATH", tempDir)
+
+	tests := []struct {
+		name           string
+		setupCommands  func() error
+		wantNvidiaSmi  bool
+		wantRocmSmi    bool
+		wantTegrastats bool
+		wantErr        bool
+	}{
+		{
+			name: "nvidia-smi not available",
+			setupCommands: func() error {
+				return nil
+			},
+			wantNvidiaSmi:  false,
+			wantRocmSmi:    false,
+			wantTegrastats: false,
+			wantErr:        true,
+		},
+		{
+			name: "nvidia-smi available",
+			setupCommands: func() error {
+				path := filepath.Join(tempDir, "nvidia-smi")
+				script := `#!/bin/sh
+echo "test"`
+				if err := os.WriteFile(path, []byte(script), 0755); err != nil {
+					return err
+				}
+				return nil
+			},
+			wantNvidiaSmi:  true,
+			wantTegrastats: false,
+			wantRocmSmi:    false,
+			wantErr:        false,
+		},
+		{
+			name: "rocm-smi available",
+			setupCommands: func() error {
+				path := filepath.Join(tempDir, "rocm-smi")
+				script := `#!/bin/sh
+echo "test"`
+				if err := os.WriteFile(path, []byte(script), 0755); err != nil {
+					return err
+				}
+				return nil
+			},
+			wantNvidiaSmi:  true,
+			wantRocmSmi:    true,
+			wantTegrastats: false,
+			wantErr:        false,
+		},
+		{
+			name: "tegrastats available",
+			setupCommands: func() error {
+				path := filepath.Join(tempDir, "tegrastats")
+				script := `#!/bin/sh
+echo "test"`
+				if err := os.WriteFile(path, []byte(script), 0755); err != nil {
+					return err
+				}
+				return nil
+			},
+			wantNvidiaSmi:  false,
+			wantRocmSmi:    true,
+			wantTegrastats: true,
+			wantErr:        false,
+		},
+		{
+			name: "no gpu tools available",
+			setupCommands: func() error {
+				os.Setenv("PATH", "")
+				return nil
+			},
+			wantErr: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if err := tt.setupCommands(); err != nil {
+				t.Fatal(err)
+			}
+
+			gm := &GPUManager{}
+			err := gm.detectGPUs()
+
+			t.Logf("nvidiaSmi: %v, rocmSmi: %v, tegrastats: %v", gm.nvidiaSmi, gm.rocmSmi, gm.tegrastats)
+
+			if tt.wantErr {
+				assert.Error(t, err)
+				return
+			}
+
+			assert.NoError(t, err)
+			assert.Equal(t, tt.wantNvidiaSmi, gm.nvidiaSmi)
+			assert.Equal(t, tt.wantRocmSmi, gm.rocmSmi)
+			assert.Equal(t, tt.wantTegrastats, gm.tegrastats)
+		})
+	}
+}
+
+func TestStartCollector(t *testing.T) {
+	// Save original PATH
+	origPath := os.Getenv("PATH")
+	defer os.Setenv("PATH", origPath)
+
+	// Set up temp dir with the commands
+	dir := t.TempDir()
+	os.Setenv("PATH", dir)
+
+	tests := []struct {
+		name     string
+		command  string
+		setup    func(t *testing.T) error
+		validate func(t *testing.T, gm *GPUManager)
+		gm       *GPUManager
+	}{
+		{
+			name:    "nvidia-smi collector",
+			command: "nvidia-smi",
+			setup: func(t *testing.T) error {
+				path := filepath.Join(dir, "nvidia-smi")
+				script := `#!/bin/sh
+echo "0, NVIDIA Test GPU, 50, 1024, 4096, 25, 100"`
+				if err := os.WriteFile(path, []byte(script), 0755); err != nil {
+					return err
+				}
+				return nil
+			},
+			validate: func(t *testing.T, gm *GPUManager) {
+				gpu, exists := gm.GpuDataMap["0"]
+				assert.True(t, exists)
+				if exists {
+					assert.Equal(t, "Test GPU", gpu.Name)
+					assert.Equal(t, 50.0, gpu.Temperature)
+
+				}
+			},
+		},
+		{
+			name:    "rocm-smi collector",
+			command: "rocm-smi",
+			setup: func(t *testing.T) error {
+				path := filepath.Join(dir, "rocm-smi")
+				script := `#!/bin/sh
+echo '{"card0": {"Temperature (Sensor edge) (C)": "49.0", "Current Socket Graphics Package Power (W)": "28.159", "GPU use (%)": "0", "VRAM Total Memory (B)": "536870912", "VRAM Total Used Memory (B)": "445550592", "Card Series": "Rembrandt [Radeon 680M]", "Card Model": "0x1681", "Card Vendor": "Advanced Micro Devices, Inc. [AMD/ATI]", "Card SKU": "REMBRANDT", "Subsystem ID": "0x8a22", "Device Rev": "0xc8", "Node ID": "1", "GUID": "34756", "GFX Version": "gfx1035"}}'`
+				if err := os.WriteFile(path, []byte(script), 0755); err != nil {
+					return err
+				}
+				return nil
+			},
+			validate: func(t *testing.T, gm *GPUManager) {
+				gpu, exists := gm.GpuDataMap["34756"]
+				assert.True(t, exists)
+				if exists {
+					assert.Equal(t, "Rembrandt [Radeon 680M]", gpu.Name)
+					assert.InDelta(t, 49.0, gpu.Temperature, 0.01)
+					assert.InDelta(t, 28.159, gpu.Power, 0.01)
+				}
+			},
+		},
+		{
+			name:    "tegrastats collector",
+			command: "tegrastats",
+			setup: func(t *testing.T) error {
+				path := filepath.Join(dir, "tegrastats")
+				script := `#!/bin/sh
+echo "11-14-2024 22:54:33 RAM 1024/4096MB GR3D_FREQ 80% tj@70C VDD_GPU_SOC 1000mW"`
+				if err := os.WriteFile(path, []byte(script), 0755); err != nil {
+					return err
+				}
+				return nil
+			},
+			validate: func(t *testing.T, gm *GPUManager) {
+				gpu, exists := gm.GpuDataMap["0"]
+				assert.True(t, exists)
+				if exists {
+					assert.InDelta(t, 70.0, gpu.Temperature, 0.1)
+				}
+			},
+			gm: &GPUManager{
+				GpuDataMap: map[string]*system.GPUData{
+					"0": {},
+				},
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if err := tt.setup(t); err != nil {
+				t.Fatal(err)
+			}
+			if tt.gm == nil {
+				tt.gm = &GPUManager{
+					GpuDataMap: make(map[string]*system.GPUData),
+				}
+			}
+			tt.gm.startCollector(tt.command)
+			time.Sleep(50 * time.Millisecond) // Give collector time to run
+			tt.validate(t, tt.gm)
+		})
+	}
+}
+
+// TestAccumulationTableDriven tests the accumulation behavior for all three GPU types
+func TestAccumulation(t *testing.T) {
+	type expectedGPUValues struct {
+		temperature float64
+		memoryUsed  float64
+		memoryTotal float64
+		usage       float64
+		power       float64
+		count       float64
+		avgUsage    float64
+		avgPower    float64
+	}
+
+	tests := []struct {
+		name           string
+		initialGPUData map[string]*system.GPUData
+		dataSamples    [][]byte
+		parser         func(*GPUManager) func([]byte) bool
+		expectedValues map[string]expectedGPUValues
+	}{
+		{
+			name: "Jetson GPU accumulation",
+			initialGPUData: map[string]*system.GPUData{
+				"0": {
+					Name:        "Jetson",
+					Temperature: 0,
+					Usage:       0,
+					Power:       0,
+					Count:       0,
+				},
+			},
+			dataSamples: [][]byte{
+				[]byte("11-14-2024 22:54:33 RAM 1024/4096MB GR3D_FREQ 30% tj@50.5C VDD_GPU_SOC 1000mW"),
+				[]byte("11-14-2024 22:54:33 RAM 1024/4096MB GR3D_FREQ 40% tj@60.5C VDD_GPU_SOC 1200mW"),
+				[]byte("11-14-2024 22:54:33 RAM 1024/4096MB GR3D_FREQ 50% tj@70.5C VDD_GPU_SOC 1400mW"),
+			},
+			parser: func(gm *GPUManager) func([]byte) bool {
+				return gm.getJetsonParser()
+			},
+			expectedValues: map[string]expectedGPUValues{
+				"0": {
+					temperature: 70.5,  // Last value
+					memoryUsed:  1024,  // Last value
+					memoryTotal: 4096,  // Last value
+					usage:       120.0, // Accumulated: 30 + 40 + 50
+					power:       3.6,   // Accumulated: 1.0 + 1.2 + 1.4
+					count:       3,
+					avgUsage:    40.0, // 120 / 3
+					avgPower:    1.2,  // 3.6 / 3
+				},
+			},
+		},
+		{
+			name:           "NVIDIA GPU accumulation",
+			initialGPUData: map[string]*system.GPUData{
+				// NVIDIA parser will create the GPU data entries
+			},
+			dataSamples: [][]byte{
+				[]byte("0, NVIDIA GeForce RTX 3080, 50, 5000, 10000, 30, 200"),
+				[]byte("0, NVIDIA GeForce RTX 3080, 60, 6000, 10000, 40, 250"),
+				[]byte("0, NVIDIA GeForce RTX 3080, 70, 7000, 10000, 50, 300"),
+			},
+			parser: func(gm *GPUManager) func([]byte) bool {
+				return gm.parseNvidiaData
+			},
+			expectedValues: map[string]expectedGPUValues{
+				"0": {
+					temperature: 70.0,            // Last value
+					memoryUsed:  7000.0 / 1.024,  // Last value
+					memoryTotal: 10000.0 / 1.024, // Last value
+					usage:       120.0,           // Accumulated: 30 + 40 + 50
+					power:       750.0,           // Accumulated: 200 + 250 + 300
+					count:       3,
+					avgUsage:    40.0,  // 120 / 3
+					avgPower:    250.0, // 750 / 3
+				},
+			},
+		},
+		{
+			name:           "AMD GPU accumulation",
+			initialGPUData: map[string]*system.GPUData{
+				// AMD parser will create the GPU data entries
+			},
+			dataSamples: [][]byte{
+				[]byte(`{"card0": {"GUID": "34756", "Temperature (Sensor edge) (C)": "50.0", "Current Socket Graphics Package Power (W)": "100.0", "GPU use (%)": "30", "VRAM Total Memory (B)": "10737418240", "VRAM Total Used Memory (B)": "1073741824", "Card Series": "Radeon RX 6800"}}`),
+				[]byte(`{"card0": {"GUID": "34756", "Temperature (Sensor edge) (C)": "60.0", "Current Socket Graphics Package Power (W)": "150.0", "GPU use (%)": "40", "VRAM Total Memory (B)": "10737418240", "VRAM Total Used Memory (B)": "2147483648", "Card Series": "Radeon RX 6800"}}`),
+				[]byte(`{"card0": {"GUID": "34756", "Temperature (Sensor edge) (C)": "70.0", "Current Socket Graphics Package Power (W)": "200.0", "GPU use (%)": "50", "VRAM Total Memory (B)": "10737418240", "VRAM Total Used Memory (B)": "3221225472", "Card Series": "Radeon RX 6800"}}`),
+			},
+			parser: func(gm *GPUManager) func([]byte) bool {
+				return gm.parseAmdData
+			},
+			expectedValues: map[string]expectedGPUValues{
+				"34756": {
+					temperature: 70.0,                          // Last value
+					memoryUsed:  3221225472.0 / (1024 * 1024),  // Last value
+					memoryTotal: 10737418240.0 / (1024 * 1024), // Last value
+					usage:       120.0,                         // Accumulated: 30 + 40 + 50
+					power:       450.0,                         // Accumulated: 100 + 150 + 200
+					count:       3,
+					avgUsage:    40.0,  // 120 / 3
+					avgPower:    150.0, // 450 / 3
+				},
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			// Create a new GPUManager for each test
+			gm := &GPUManager{
+				GpuDataMap: tt.initialGPUData,
+			}
+
+			// Get the parser function
+			parser := tt.parser(gm)
+
+			// Process each data sample
+			for i, sample := range tt.dataSamples {
+				valid := parser(sample)
+				assert.True(t, valid, "Sample %d should be valid", i)
+			}
+
+			// Check accumulated values
+			for id, expected := range tt.expectedValues {
+				gpu, exists := gm.GpuDataMap[id]
+				assert.True(t, exists, "GPU with ID %s should exist", id)
+				if !exists {
+					continue
+				}
+
+				assert.InDelta(t, expected.temperature, gpu.Temperature, 0.01, "Temperature should match")
+				assert.InDelta(t, expected.memoryUsed, gpu.MemoryUsed, 0.01, "Memory used should match")
+				assert.InDelta(t, expected.memoryTotal, gpu.MemoryTotal, 0.01, "Memory total should match")
+				assert.InDelta(t, expected.usage, gpu.Usage, 0.01, "Usage should match")
+				assert.InDelta(t, expected.power, gpu.Power, 0.01, "Power should match")
+				assert.Equal(t, expected.count, gpu.Count, "Count should match")
+			}
+
+			// Verify average calculation in GetCurrentData
+			result := gm.GetCurrentData()
+			for id, expected := range tt.expectedValues {
+				gpu, exists := result[id]
+				assert.True(t, exists, "GPU with ID %s should exist in GetCurrentData result", id)
+				if !exists {
+					continue
+				}
+
+				assert.InDelta(t, expected.temperature, gpu.Temperature, 0.01, "Temperature in GetCurrentData should match")
+				assert.InDelta(t, expected.avgUsage, gpu.Usage, 0.01, "Average usage in GetCurrentData should match")
+				assert.InDelta(t, expected.avgPower, gpu.Power, 0.01, "Average power in GetCurrentData should match")
+			}
+
+			// Verify that accumulators in the original map are reset
+			for id := range tt.expectedValues {
+				gpu, exists := gm.GpuDataMap[id]
+				assert.True(t, exists, "GPU with ID %s should still exist after GetCurrentData", id)
+				if !exists {
+					continue
+				}
+				assert.Equal(t, float64(0), gpu.Count, "Count should be reset for GPU ID %s", id)
+				assert.Equal(t, float64(0), gpu.Usage, "Usage should be reset for GPU ID %s", id)
+				assert.Equal(t, float64(0), gpu.Power, "Power should be reset for GPU ID %s", id)
+			}
+		})
+	}
+}

beszel/internal/agent/health/health.go

@@ -0,0 +1,43 @@
+// Package health provides functions to check and update the health of the agent.
+// It uses a file in the temp directory to store the timestamp of the last connection attempt.
+// If the timestamp is older than 90 seconds, the agent is considered unhealthy.
+// NB: The agent must be started with the Start() method to be considered healthy.
+package health
+
+import (
+	"errors"
+	"log"
+	"os"
+	"path/filepath"
+	"time"
+)
+
+// healthFile is the path to the health file
+var healthFile = filepath.Join(os.TempDir(), "beszel_health")
+
+// Check checks if the agent is connected by checking the modification time of the health file
+func Check() error {
+	fileInfo, err := os.Stat(healthFile)
+	if err != nil {
+		return err
+	}
+	if time.Since(fileInfo.ModTime()) > 91*time.Second {
+		log.Println("over 90 seconds since last connection")
+		return errors.New("unhealthy")
+	}
+	return nil
+}
+
+// Update updates the modification time of the health file
+func Update() error {
+	file, err := os.Create(healthFile)
+	if err != nil {
+		return err
+	}
+	return file.Close()
+}
+
+// CleanUp removes the health file
+func CleanUp() error {
+	return os.Remove(healthFile)
+}

beszel/internal/agent/health/health_test.go

@@ -0,0 +1,67 @@
+//go:build testing
+// +build testing
+
+package health
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+	"time"
+
+	"testing/synctest"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestHealth(t *testing.T) {
+	// Override healthFile to use a temporary directory for this test.
+	originalHealthFile := healthFile
+	tmpDir := t.TempDir()
+	healthFile = filepath.Join(tmpDir, "beszel_health_test")
+	defer func() { healthFile = originalHealthFile }()
+
+	t.Run("check with no health file", func(t *testing.T) {
+		err := Check()
+		require.Error(t, err)
+		assert.True(t, os.IsNotExist(err), "expected a file-not-exist error, but got: %v", err)
+	})
+
+	t.Run("update and check", func(t *testing.T) {
+		err := Update()
+		require.NoError(t, err, "Update() failed")
+
+		err = Check()
+		assert.NoError(t, err, "Check() failed immediately after Update()")
+	})
+
+	// This test uses synctest to simulate time passing.
+	// NOTE: This test requires GOEXPERIMENT=synctest to run.
+	t.Run("check with simulated time", func(t *testing.T) {
+		synctest.Run(func() {
+			// Update the file to set the initial timestamp.
+			require.NoError(t, Update(), "Update() failed inside synctest")
+
+			// Set the mtime to the current fake time to align the file's timestamp with the simulated clock.
+			now := time.Now()
+			require.NoError(t, os.Chtimes(healthFile, now, now), "Chtimes failed")
+
+			// Wait a duration less than the threshold.
+			time.Sleep(89 * time.Second)
+			synctest.Wait()
+
+			// The check should still pass.
+			assert.NoError(t, Check(), "Check() failed after 89s")
+
+			// Wait for the total duration to exceed the threshold.
+			time.Sleep(5 * time.Second)
+			synctest.Wait()
+
+			// The check should now fail as unhealthy.
+			err := Check()
+			require.Error(t, err, "Check() should have failed after 91s")
+			assert.Equal(t, "unhealthy", err.Error(), "Check() returned wrong error")
+		})
+	})
+}

beszel/internal/agent/lhm/beszel_lhm.cs

@@ -0,0 +1,80 @@
+using System;
+using System.Globalization;
+using LibreHardwareMonitor.Hardware;
+
+class Program
+{
+  static void Main()
+  {
+    var computer = new Computer
+    {
+      IsCpuEnabled = true,
+      IsGpuEnabled = true,
+      IsMemoryEnabled = true,
+      IsMotherboardEnabled = true,
+      IsStorageEnabled = true,
+      // IsPsuEnabled = true,
+      // IsNetworkEnabled = true,
+    };
+    computer.Open();
+
+    var reader = Console.In;
+    var writer = Console.Out;
+
+    string line;
+    while ((line = reader.ReadLine()) != null)
+    {
+      if (line.Trim().Equals("getTemps", StringComparison.OrdinalIgnoreCase))
+      {
+        foreach (var hw in computer.Hardware)
+        {
+          // process main hardware sensors
+          ProcessSensors(hw, writer);
+
+          // process subhardware sensors
+          foreach (var subhardware in hw.SubHardware)
+          {
+            ProcessSensors(subhardware, writer);
+          }
+        }
+        // send empty line to signal end of sensor data
+        writer.WriteLine();
+        writer.Flush();
+      }
+    }
+
+    computer.Close();
+  }
+
+  static void ProcessSensors(IHardware hardware, System.IO.TextWriter writer)
+  {
+    var updated = false;
+    foreach (var sensor in hardware.Sensors)
+    {
+      var validTemp = sensor.SensorType == SensorType.Temperature && sensor.Value.HasValue;
+      if (!validTemp || sensor.Name.Contains("Distance"))
+      {
+        continue;
+      }
+
+      if (!updated)
+      {
+        hardware.Update();
+        updated = true;
+      }
+
+      var name = sensor.Name;
+      // if sensor.Name starts with "Temperature" replace with hardware.Identifier but retain the rest of the name.
+      // usually this is a number like Temperature 3
+      if (sensor.Name.StartsWith("Temperature"))
+      {
+        name = hardware.Identifier.ToString().Replace("/", "_").TrimStart('_') + sensor.Name.Substring(11);
+      }
+
+      // invariant culture assures the value is parsable as a float
+      var value = sensor.Value.Value.ToString("0.##", CultureInfo.InvariantCulture);
+      // write the name and value to the writer
+      writer.WriteLine($"{name}|{value}");
+    }
+  }
+}

beszel/internal/agent/lhm/beszel_lhm.csproj

@@ -0,0 +1,11 @@
+<Project Sdk="Microsoft.NET.Sdk">
+  <PropertyGroup>
+    <OutputType>Exe</OutputType>
+    <TargetFramework>net48</TargetFramework>
+    <Platforms>x64</Platforms>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <PackageReference Include="LibreHardwareMonitorLib" Version="0.9.4" />
+  </ItemGroup>
+</Project>

beszel/internal/agent/network.go

@@ -2,7 +2,6 @@ package agent
 
 import (
 	"log/slog"
-	"os"
 	"strings"
 	"time"
 
@@ -15,10 +14,10 @@ func (a *Agent) initializeNetIoStats() {
 
 	// map of network interface names passed in via NICS env var
 	var nicsMap map[string]struct{}
-	nics, nicsEnvExists := os.LookupEnv("NICS")
+	nics, nicsEnvExists := GetEnv("NICS")
 	if nicsEnvExists {
 		nicsMap = make(map[string]struct{}, 0)
-		for _, nic := range strings.Split(nics, ",") {
+		for nic := range strings.SplitSeq(nics, ",") {
 			nicsMap[nic] = struct{}{}
 		}
 	}
@@ -58,6 +57,7 @@ func (a *Agent) skipNetworkInterface(v psutilNet.IOCountersStat) bool {
 		strings.HasPrefix(v.Name, "docker"),
 		strings.HasPrefix(v.Name, "br-"),
 		strings.HasPrefix(v.Name, "veth"),
+		strings.HasPrefix(v.Name, "bond"),
 		v.BytesRecv == 0,
 		v.BytesSent == 0:
 		return true

beszel/internal/agent/sensors.go

@@ -0,0 +1,197 @@
+package agent
+
+import (
+	"beszel/internal/entities/system"
+	"context"
+	"fmt"
+	"log/slog"
+	"path"
+	"runtime"
+	"strconv"
+	"strings"
+	"unicode/utf8"
+
+	"github.com/shirou/gopsutil/v4/common"
+	"github.com/shirou/gopsutil/v4/sensors"
+)
+
+type SensorConfig struct {
+	context        context.Context
+	sensors        map[string]struct{}
+	primarySensor  string
+	isBlacklist    bool
+	hasWildcards   bool
+	skipCollection bool
+}
+
+func (a *Agent) newSensorConfig() *SensorConfig {
+	primarySensor, _ := GetEnv("PRIMARY_SENSOR")
+	sysSensors, _ := GetEnv("SYS_SENSORS")
+	sensorsEnvVal, sensorsSet := GetEnv("SENSORS")
+	skipCollection := sensorsSet && sensorsEnvVal == ""
+
+	return a.newSensorConfigWithEnv(primarySensor, sysSensors, sensorsEnvVal, skipCollection)
+}
+
+// Matches sensors.TemperaturesWithContext to allow for panic recovery (gopsutil/issues/1832)
+type getTempsFn func(ctx context.Context) ([]sensors.TemperatureStat, error)
+
+// newSensorConfigWithEnv creates a SensorConfig with the provided environment variables
+// sensorsSet indicates if the SENSORS environment variable was explicitly set (even to empty string)
+func (a *Agent) newSensorConfigWithEnv(primarySensor, sysSensors, sensorsEnvVal string, skipCollection bool) *SensorConfig {
+	config := &SensorConfig{
+		context:        context.Background(),
+		primarySensor:  primarySensor,
+		skipCollection: skipCollection,
+		sensors:        make(map[string]struct{}),
+	}
+
+	// Set sensors context (allows overriding sys location for sensors)
+	if sysSensors != "" {
+		slog.Info("SYS_SENSORS", "path", sysSensors)
+		config.context = context.WithValue(config.context,
+			common.EnvKey, common.EnvMap{common.HostSysEnvKey: sysSensors},
+		)
+	}
+
+	// handle blacklist
+	if strings.HasPrefix(sensorsEnvVal, "-") {
+		config.isBlacklist = true
+		sensorsEnvVal = sensorsEnvVal[1:]
+	}
+
+	for sensor := range strings.SplitSeq(sensorsEnvVal, ",") {
+		sensor = strings.TrimSpace(sensor)
+		if sensor != "" {
+			config.sensors[sensor] = struct{}{}
+			if strings.Contains(sensor, "*") {
+				config.hasWildcards = true
+			}
+		}
+	}
+
+	return config
+}
+
+// updateTemperatures updates the agent with the latest sensor temperatures
+func (a *Agent) updateTemperatures(systemStats *system.Stats) {
+	// skip if sensors whitelist is set to empty string
+	if a.sensorConfig.skipCollection {
+		slog.Debug("Skipping temperature collection")
+		return
+	}
+
+	// reset high temp
+	a.systemInfo.DashboardTemp = 0
+
+	temps, err := a.getTempsWithPanicRecovery(getSensorTemps)
+	if err != nil {
+		// retry once on panic (gopsutil/issues/1832)
+		temps, err = a.getTempsWithPanicRecovery(getSensorTemps)
+		if err != nil {
+			slog.Warn("Error updating temperatures", "err", err)
+			if len(systemStats.Temperatures) > 0 {
+				systemStats.Temperatures = make(map[string]float64)
+			}
+			return
+		}
+	}
+	slog.Debug("Temperature", "sensors", temps)
+
+	// return if no sensors
+	if len(temps) == 0 {
+		return
+	}
+
+	systemStats.Temperatures = make(map[string]float64, len(temps))
+	for i, sensor := range temps {
+		// check for malformed strings on darwin (gopsutil/issues/1832)
+		if runtime.GOOS == "darwin" && !utf8.ValidString(sensor.SensorKey) {
+			continue
+		}
+
+		// scale temperature
+		if sensor.Temperature != 0 && sensor.Temperature < 1 {
+			sensor.Temperature = scaleTemperature(sensor.Temperature)
+		}
+		// skip if temperature is unreasonable
+		if sensor.Temperature <= 0 || sensor.Temperature >= 200 {
+			continue
+		}
+		sensorName := sensor.SensorKey
+		if _, ok := systemStats.Temperatures[sensorName]; ok {
+			// if key already exists, append int to key
+			sensorName = sensorName + "_" + strconv.Itoa(i)
+		}
+		// skip if not in whitelist or blacklist
+		if !isValidSensor(sensorName, a.sensorConfig) {
+			continue
+		}
+		// set dashboard temperature
+		switch a.sensorConfig.primarySensor {
+		case "":
+			a.systemInfo.DashboardTemp = max(a.systemInfo.DashboardTemp, sensor.Temperature)
+		case sensorName:
+			a.systemInfo.DashboardTemp = sensor.Temperature
+		}
+		systemStats.Temperatures[sensorName] = twoDecimals(sensor.Temperature)
+	}
+}
+
+// getTempsWithPanicRecovery wraps sensors.TemperaturesWithContext to recover from panics (gopsutil/issues/1832)
+func (a *Agent) getTempsWithPanicRecovery(getTemps getTempsFn) (temps []sensors.TemperatureStat, err error) {
+	defer func() {
+		if r := recover(); r != nil {
+			err = fmt.Errorf("panic: %v", r)
+		}
+	}()
+	// get sensor data (error ignored intentionally as it may be only with one sensor)
+	temps, _ = getTemps(a.sensorConfig.context)
+	return
+}
+
+// isValidSensor checks if a sensor is valid based on the sensor name and the sensor config
+func isValidSensor(sensorName string, config *SensorConfig) bool {
+	// if no sensors configured, everything is valid
+	if len(config.sensors) == 0 {
+		return true
+	}
+
+	// Exact match - return true if whitelist, false if blacklist
+	if _, exactMatch := config.sensors[sensorName]; exactMatch {
+		return !config.isBlacklist
+	}
+
+	// If no wildcards, return true if blacklist, false if whitelist
+	if !config.hasWildcards {
+		return config.isBlacklist
+	}
+
+	// Check for wildcard patterns
+	for pattern := range config.sensors {
+		if !strings.Contains(pattern, "*") {
+			continue
+		}
+		if match, _ := path.Match(pattern, sensorName); match {
+			return !config.isBlacklist
+		}
+	}
+
+	return config.isBlacklist
+}
+
+// scaleTemperature scales temperatures in fractional values to reasonable Celsius values
+func scaleTemperature(temp float64) float64 {
+	if temp > 1 {
+		return temp
+	}
+	scaled100 := temp * 100
+	scaled1000 := temp * 1000
+
+	if scaled100 >= 15 && scaled100 <= 95 {
+		return scaled100
+	} else if scaled1000 >= 15 && scaled1000 <= 95 {
+		return scaled1000
+	}
+	return scaled100
+}

beszel/internal/agent/sensors_default.go

@@ -0,0 +1,9 @@
+//go:build !windows
+
+package agent
+
+import (
+	"github.com/shirou/gopsutil/v4/sensors"
+)
+
+var getSensorTemps = sensors.TemperaturesWithContext

beszel/internal/agent/sensors_test.go

@@ -0,0 +1,553 @@
+//go:build testing
+// +build testing
+
+package agent
+
+import (
+	"beszel/internal/entities/system"
+	"context"
+	"fmt"
+	"os"
+	"testing"
+
+	"github.com/shirou/gopsutil/v4/common"
+	"github.com/shirou/gopsutil/v4/sensors"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestIsValidSensor(t *testing.T) {
+	tests := []struct {
+		name          string
+		sensorName    string
+		config        *SensorConfig
+		expectedValid bool
+	}{
+		{
+			name:       "Whitelist - sensor in list",
+			sensorName: "cpu_temp",
+			config: &SensorConfig{
+				sensors:     map[string]struct{}{"cpu_temp": {}},
+				isBlacklist: false,
+			},
+			expectedValid: true,
+		},
+		{
+			name:       "Whitelist - sensor not in list",
+			sensorName: "gpu_temp",
+			config: &SensorConfig{
+				sensors:     map[string]struct{}{"cpu_temp": {}},
+				isBlacklist: false,
+			},
+			expectedValid: false,
+		},
+		{
+			name:       "Blacklist - sensor in list",
+			sensorName: "cpu_temp",
+			config: &SensorConfig{
+				sensors:     map[string]struct{}{"cpu_temp": {}},
+				isBlacklist: true,
+			},
+			expectedValid: false,
+		},
+		{
+			name:       "Blacklist - sensor not in list",
+			sensorName: "gpu_temp",
+			config: &SensorConfig{
+				sensors:     map[string]struct{}{"cpu_temp": {}},
+				isBlacklist: true,
+			},
+			expectedValid: true,
+		},
+		{
+			name:       "Whitelist with wildcard - matching pattern",
+			sensorName: "core_0_temp",
+			config: &SensorConfig{
+				sensors:      map[string]struct{}{"core_*_temp": {}},
+				isBlacklist:  false,
+				hasWildcards: true,
+			},
+			expectedValid: true,
+		},
+		{
+			name:       "Whitelist with wildcard - non-matching pattern",
+			sensorName: "gpu_temp",
+			config: &SensorConfig{
+				sensors:      map[string]struct{}{"core_*_temp": {}},
+				isBlacklist:  false,
+				hasWildcards: true,
+			},
+			expectedValid: false,
+		},
+		{
+			name:       "Blacklist with wildcard - matching pattern",
+			sensorName: "core_0_temp",
+			config: &SensorConfig{
+				sensors:      map[string]struct{}{"core_*_temp": {}},
+				isBlacklist:  true,
+				hasWildcards: true,
+			},
+			expectedValid: false,
+		},
+		{
+			name:       "Blacklist with wildcard - non-matching pattern",
+			sensorName: "gpu_temp",
+			config: &SensorConfig{
+				sensors:      map[string]struct{}{"core_*_temp": {}},
+				isBlacklist:  true,
+				hasWildcards: true,
+			},
+			expectedValid: true,
+		},
+		{
+			name:       "No sensors configured",
+			sensorName: "any_temp",
+			config: &SensorConfig{
+				sensors:        map[string]struct{}{},
+				isBlacklist:    false,
+				hasWildcards:   false,
+				skipCollection: false,
+			},
+			expectedValid: true,
+		},
+		{
+			name:       "Mixed patterns in whitelist - exact match",
+			sensorName: "cpu_temp",
+			config: &SensorConfig{
+				sensors:      map[string]struct{}{"cpu_temp": {}, "core_*_temp": {}},
+				isBlacklist:  false,
+				hasWildcards: true,
+			},
+			expectedValid: true,
+		},
+		{
+			name:       "Mixed patterns in whitelist - wildcard match",
+			sensorName: "core_1_temp",
+			config: &SensorConfig{
+				sensors:      map[string]struct{}{"cpu_temp": {}, "core_*_temp": {}},
+				isBlacklist:  false,
+				hasWildcards: true,
+			},
+			expectedValid: true,
+		},
+		{
+			name:       "Mixed patterns in blacklist - exact match",
+			sensorName: "cpu_temp",
+			config: &SensorConfig{
+				sensors:      map[string]struct{}{"cpu_temp": {}, "core_*_temp": {}},
+				isBlacklist:  true,
+				hasWildcards: true,
+			},
+			expectedValid: false,
+		},
+		{
+			name:       "Mixed patterns in blacklist - wildcard match",
+			sensorName: "core_1_temp",
+			config: &SensorConfig{
+				sensors:      map[string]struct{}{"cpu_temp": {}, "core_*_temp": {}},
+				isBlacklist:  true,
+				hasWildcards: true,
+			},
+			expectedValid: false,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := isValidSensor(tt.sensorName, tt.config)
+			assert.Equal(t, tt.expectedValid, result, "isValidSensor(%q, config) returned unexpected result", tt.sensorName)
+		})
+	}
+}
+
+func TestNewSensorConfigWithEnv(t *testing.T) {
+	agent := &Agent{}
+
+	tests := []struct {
+		name           string
+		primarySensor  string
+		sysSensors     string
+		sensors        string
+		skipCollection bool
+		expectedConfig *SensorConfig
+	}{
+		{
+			name:          "Empty configuration",
+			primarySensor: "",
+			sysSensors:    "",
+			sensors:       "",
+			expectedConfig: &SensorConfig{
+				context:        context.Background(),
+				primarySensor:  "",
+				sensors:        map[string]struct{}{},
+				isBlacklist:    false,
+				hasWildcards:   false,
+				skipCollection: false,
+			},
+		},
+		{
+			name:           "Explicitly set to empty string",
+			primarySensor:  "",
+			sysSensors:     "",
+			sensors:        "",
+			skipCollection: true,
+			expectedConfig: &SensorConfig{
+				context:        context.Background(),
+				primarySensor:  "",
+				sensors:        map[string]struct{}{},
+				isBlacklist:    false,
+				hasWildcards:   false,
+				skipCollection: true,
+			},
+		},
+		{
+			name:          "Primary sensor only - should create sensor map",
+			primarySensor: "cpu_temp",
+			sysSensors:    "",
+			sensors:       "",
+			expectedConfig: &SensorConfig{
+				context:       context.Background(),
+				primarySensor: "cpu_temp",
+				sensors:       map[string]struct{}{},
+				isBlacklist:   false,
+				hasWildcards:  false,
+			},
+		},
+		{
+			name:          "Whitelist sensors",
+			primarySensor: "cpu_temp",
+			sysSensors:    "",
+			sensors:       "cpu_temp,gpu_temp",
+			expectedConfig: &SensorConfig{
+				context:       context.Background(),
+				primarySensor: "cpu_temp",
+				sensors: map[string]struct{}{
+					"cpu_temp": {},
+					"gpu_temp": {},
+				},
+				isBlacklist:  false,
+				hasWildcards: false,
+			},
+		},
+		{
+			name:          "Blacklist sensors",
+			primarySensor: "cpu_temp",
+			sysSensors:    "",
+			sensors:       "-cpu_temp,gpu_temp",
+			expectedConfig: &SensorConfig{
+				context:       context.Background(),
+				primarySensor: "cpu_temp",
+				sensors: map[string]struct{}{
+					"cpu_temp": {},
+					"gpu_temp": {},
+				},
+				isBlacklist:  true,
+				hasWildcards: false,
+			},
+		},
+		{
+			name:          "Sensors with wildcard",
+			primarySensor: "cpu_temp",
+			sysSensors:    "",
+			sensors:       "cpu_*,gpu_temp",
+			expectedConfig: &SensorConfig{
+				context:       context.Background(),
+				primarySensor: "cpu_temp",
+				sensors: map[string]struct{}{
+					"cpu_*":    {},
+					"gpu_temp": {},
+				},
+				isBlacklist:  false,
+				hasWildcards: true,
+			},
+		},
+		{
+			name:          "Sensors with whitespace",
+			primarySensor: "cpu_temp",
+			sysSensors:    "",
+			sensors:       "cpu_*, gpu_temp",
+			expectedConfig: &SensorConfig{
+				context:       context.Background(),
+				primarySensor: "cpu_temp",
+				sensors: map[string]struct{}{
+					"cpu_*":    {},
+					"gpu_temp": {},
+				},
+				isBlacklist:  false,
+				hasWildcards: true,
+			},
+		},
+		{
+			name:          "With SYS_SENSORS path",
+			primarySensor: "cpu_temp",
+			sysSensors:    "/custom/path",
+			sensors:       "cpu_temp",
+			expectedConfig: &SensorConfig{
+				primarySensor: "cpu_temp",
+				sensors: map[string]struct{}{
+					"cpu_temp": {},
+				},
+				isBlacklist:  false,
+				hasWildcards: false,
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := agent.newSensorConfigWithEnv(tt.primarySensor, tt.sysSensors, tt.sensors, tt.skipCollection)
+
+			// Check primary sensor
+			assert.Equal(t, tt.expectedConfig.primarySensor, result.primarySensor)
+
+			// Check sensor map
+			if tt.expectedConfig.sensors == nil {
+				assert.Nil(t, result.sensors)
+			} else {
+				assert.Equal(t, len(tt.expectedConfig.sensors), len(result.sensors))
+				for sensor := range tt.expectedConfig.sensors {
+					_, exists := result.sensors[sensor]
+					assert.True(t, exists, "Sensor %s should exist in the result", sensor)
+				}
+			}
+
+			// Check flags
+			assert.Equal(t, tt.expectedConfig.isBlacklist, result.isBlacklist)
+			assert.Equal(t, tt.expectedConfig.hasWildcards, result.hasWildcards)
+
+			// Check context
+			if tt.sysSensors != "" {
+				// Verify context contains correct values
+				envMap, ok := result.context.Value(common.EnvKey).(common.EnvMap)
+				require.True(t, ok, "Context should contain EnvMap")
+				sysPath, ok := envMap[common.HostSysEnvKey]
+				require.True(t, ok, "EnvMap should contain HostSysEnvKey")
+				assert.Equal(t, tt.sysSensors, sysPath)
+			}
+		})
+	}
+}
+
+func TestNewSensorConfig(t *testing.T) {
+	// Save original environment variables
+	originalPrimary, hasPrimary := os.LookupEnv("BESZEL_AGENT_PRIMARY_SENSOR")
+	originalSys, hasSys := os.LookupEnv("BESZEL_AGENT_SYS_SENSORS")
+	originalSensors, hasSensors := os.LookupEnv("BESZEL_AGENT_SENSORS")
+
+	// Restore environment variables after the test
+	defer func() {
+		// Clean up test environment variables
+		os.Unsetenv("BESZEL_AGENT_PRIMARY_SENSOR")
+		os.Unsetenv("BESZEL_AGENT_SYS_SENSORS")
+		os.Unsetenv("BESZEL_AGENT_SENSORS")
+
+		// Restore original values if they existed
+		if hasPrimary {
+			os.Setenv("BESZEL_AGENT_PRIMARY_SENSOR", originalPrimary)
+		}
+		if hasSys {
+			os.Setenv("BESZEL_AGENT_SYS_SENSORS", originalSys)
+		}
+		if hasSensors {
+			os.Setenv("BESZEL_AGENT_SENSORS", originalSensors)
+		}
+	}()
+
+	// Set test environment variables
+	os.Setenv("BESZEL_AGENT_PRIMARY_SENSOR", "test_primary")
+	os.Setenv("BESZEL_AGENT_SYS_SENSORS", "/test/path")
+	os.Setenv("BESZEL_AGENT_SENSORS", "test_sensor1,test_*,test_sensor3")
+
+	agent := &Agent{}
+	result := agent.newSensorConfig()
+
+	// Verify results
+	assert.Equal(t, "test_primary", result.primarySensor)
+	assert.NotNil(t, result.sensors)
+	assert.Equal(t, 3, len(result.sensors))
+	assert.True(t, result.hasWildcards)
+	assert.False(t, result.isBlacklist)
+
+	// Check that sys sensors path is in context
+	envMap, ok := result.context.Value(common.EnvKey).(common.EnvMap)
+	require.True(t, ok, "Context should contain EnvMap")
+	sysPath, ok := envMap[common.HostSysEnvKey]
+	require.True(t, ok, "EnvMap should contain HostSysEnvKey")
+	assert.Equal(t, "/test/path", sysPath)
+}
+
+func TestScaleTemperature(t *testing.T) {
+	tests := []struct {
+		name     string
+		input    float64
+		expected float64
+		desc     string
+	}{
+		// Normal temperatures (no scaling needed)
+		{"normal_cpu_temp", 45.0, 45.0, "Normal CPU temperature"},
+		{"normal_room_temp", 25.0, 25.0, "Normal room temperature"},
+		{"high_cpu_temp", 85.0, 85.0, "High CPU temperature"},
+		// Zero temperature
+		{"zero_temp", 0.0, 0.0, "Zero temperature"},
+		// Fractional values that should use 100x scaling
+		{"fractional_45c", 0.45, 45.0, "0.45 should become 45°C (100x)"},
+		{"fractional_25c", 0.25, 25.0, "0.25 should become 25°C (100x)"},
+		{"fractional_60c", 0.60, 60.0, "0.60 should become 60°C (100x)"},
+		{"fractional_75c", 0.75, 75.0, "0.75 should become 75°C (100x)"},
+		{"fractional_30c", 0.30, 30.0, "0.30 should become 30°C (100x)"},
+		// Fractional values that should use 1000x scaling
+		{"millifractional_45c", 0.045, 45.0, "0.045 should become 45°C (1000x)"},
+		{"millifractional_25c", 0.025, 25.0, "0.025 should become 25°C (1000x)"},
+		{"millifractional_60c", 0.060, 60.0, "0.060 should become 60°C (1000x)"},
+		{"millifractional_75c", 0.075, 75.0, "0.075 should become 75°C (1000x)"},
+		{"millifractional_35c", 0.035, 35.0, "0.035 should become 35°C (1000x)"},
+		// Edge cases - values outside reasonable range
+		{"very_low_fractional", 0.01, 1.0, "0.01 should default to 100x scaling (1°C)"},
+		{"very_high_fractional", 0.99, 99.0, "0.99 should default to 100x scaling (99°C)"},
+		{"extremely_low", 0.001, 0.1, "0.001 should default to 100x scaling (0.1°C)"},
+		// Boundary cases around the reasonable range (15-95°C)
+		{"boundary_low_100x", 0.15, 15.0, "0.15 should use 100x scaling (15°C)"},
+		{"boundary_high_100x", 0.95, 95.0, "0.95 should use 100x scaling (95°C)"},
+		{"boundary_low_1000x", 0.015, 15.0, "0.015 should use 1000x scaling (15°C)"},
+		{"boundary_high_1000x", 0.095, 95.0, "0.095 should use 1000x scaling (95°C)"},
+		// Values just outside reasonable range
+		{"just_below_range_100x", 0.14, 14.0, "0.14 should default to 100x (14°C)"},
+		{"just_above_range_100x", 0.96, 96.0, "0.96 should default to 100x (96°C)"},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := scaleTemperature(tt.input)
+			assert.InDelta(t, tt.expected, result, 0.001,
+				"scaleTemperature(%v) = %v, expected %v (%s)",
+				tt.input, result, tt.expected, tt.desc)
+		})
+	}
+}
+
+func TestScaleTemperatureLogic(t *testing.T) {
+	// Test the logic flow for ambiguous cases
+	t.Run("prefers_100x_when_both_valid", func(t *testing.T) {
+		// 0.5 could be 50°C (100x) or 500°C (1000x)
+		// Should prefer 100x since it's tried first and is in range
+		result := scaleTemperature(0.5)
+		expected := 50.0
+		assert.InDelta(t, expected, result, 0.001,
+			"scaleTemperature(0.5) = %v, expected %v (should prefer 100x scaling)",
+			result, expected)
+	})
+
+	t.Run("uses_1000x_when_100x_too_low", func(t *testing.T) {
+		// 0.05 -> 5°C (100x, too low) or 50°C (1000x, in range)
+		// Should use 1000x since 100x is below reasonable range
+		result := scaleTemperature(0.05)
+		expected := 50.0
+		assert.InDelta(t, expected, result, 0.001,
+			"scaleTemperature(0.05) = %v, expected %v (should use 1000x scaling)",
+			result, expected)
+	})
+
+	t.Run("defaults_to_100x_when_both_invalid", func(t *testing.T) {
+		// 0.005 -> 0.5°C (100x, too low) or 5°C (1000x, too low)
+		// Should default to 100x scaling
+		result := scaleTemperature(0.005)
+		expected := 0.5
+		assert.InDelta(t, expected, result, 0.001,
+			"scaleTemperature(0.005) = %v, expected %v (should default to 100x)",
+			result, expected)
+	})
+}
+
+func TestGetTempsWithPanicRecovery(t *testing.T) {
+	agent := &Agent{
+		systemInfo: system.Info{},
+		sensorConfig: &SensorConfig{
+			context: context.Background(),
+		},
+	}
+
+	tests := []struct {
+		name        string
+		getTempsFn  getTempsFn
+		expectError bool
+		errorMsg    string
+	}{
+		{
+			name: "successful_function_call",
+			getTempsFn: func(ctx context.Context) ([]sensors.TemperatureStat, error) {
+				return []sensors.TemperatureStat{
+					{SensorKey: "test_sensor", Temperature: 45.0},
+				}, nil
+			},
+			expectError: false,
+		},
+		{
+			name: "function_returns_error",
+			getTempsFn: func(ctx context.Context) ([]sensors.TemperatureStat, error) {
+				return []sensors.TemperatureStat{
+					{SensorKey: "test_sensor", Temperature: 45.0},
+				}, fmt.Errorf("sensor error")
+			},
+			expectError: false, // getTempsWithPanicRecovery ignores errors from the function
+		},
+		{
+			name: "function_panics_with_string",
+			getTempsFn: func(ctx context.Context) ([]sensors.TemperatureStat, error) {
+				panic("test panic")
+			},
+			expectError: true,
+			errorMsg:    "panic: test panic",
+		},
+		{
+			name: "function_panics_with_error",
+			getTempsFn: func(ctx context.Context) ([]sensors.TemperatureStat, error) {
+				panic(fmt.Errorf("panic error"))
+			},
+			expectError: true,
+			errorMsg:    "panic:",
+		},
+		{
+			name: "function_panics_with_index_out_of_bounds",
+			getTempsFn: func(ctx context.Context) ([]sensors.TemperatureStat, error) {
+				slice := []int{1, 2, 3}
+				_ = slice[10] // out of bounds panic
+				return nil, nil
+			},
+			expectError: true,
+			errorMsg:    "panic:",
+		},
+		{
+			name: "function_panics_with_any_conversion",
+			getTempsFn: func(ctx context.Context) ([]sensors.TemperatureStat, error) {
+				var i any = "string"
+				_ = i.(int) // type assertion panic
+				return nil, nil
+			},
+			expectError: true,
+			errorMsg:    "panic:",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			var temps []sensors.TemperatureStat
+			var err error
+
+			// The function should not panic, regardless of what the injected function does
+			assert.NotPanics(t, func() {
+				temps, err = agent.getTempsWithPanicRecovery(tt.getTempsFn)
+			}, "getTempsWithPanicRecovery should not panic")
+
+			if tt.expectError {
+				assert.Error(t, err, "Expected an error to be returned")
+				if tt.errorMsg != "" {
+					assert.Contains(t, err.Error(), tt.errorMsg,
+						"Error message should contain expected text")
+				}
+				assert.Nil(t, temps, "Temps should be nil when panic occurs")
+			} else {
+				assert.NoError(t, err, "Should not return error for successful calls")
+			}
+		})
+	}
+}

beszel/internal/agent/sensors_windows.go

@@ -0,0 +1,281 @@
+//go:build windows
+
+//go:generate dotnet build -c Release lhm/beszel_lhm.csproj
+
+package agent
+
+import (
+	"bufio"
+	"context"
+	"embed"
+	"errors"
+	"fmt"
+	"io"
+	"log/slog"
+	"os"
+	"os/exec"
+	"path"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/shirou/gopsutil/v4/sensors"
+)
+
+// Note: This is always called from Agent.gatherStats() which holds Agent.Lock(),
+// so no internal concurrency protection is needed.
+
+// lhmProcess is a wrapper around the LHM .NET process.
+type lhmProcess struct {
+	cmd                  *exec.Cmd
+	stdin                io.WriteCloser
+	stdout               io.ReadCloser
+	scanner              *bufio.Scanner
+	isRunning            bool
+	stoppedNoSensors     bool
+	consecutiveNoSensors uint8
+	execPath             string
+	tempDir              string
+}
+
+//go:embed all:lhm/bin/Release/net48
+var lhmFs embed.FS
+
+var (
+	beszelLhm     *lhmProcess
+	beszelLhmOnce sync.Once
+)
+
+var errNoSensors = errors.New("no sensors found (try running as admin)")
+
+// newlhmProcess copies the embedded LHM executable to a temporary directory and starts it.
+func newlhmProcess() (*lhmProcess, error) {
+	destDir := filepath.Join(os.TempDir(), "beszel")
+	execPath := filepath.Join(destDir, "beszel_lhm.exe")
+
+	if err := os.MkdirAll(destDir, 0755); err != nil {
+		return nil, fmt.Errorf("failed to create temp directory: %w", err)
+	}
+
+	// Only copy if executable doesn't exist
+	if _, err := os.Stat(execPath); os.IsNotExist(err) {
+		if err := copyEmbeddedDir(lhmFs, "lhm/bin/Release/net48", destDir); err != nil {
+			return nil, fmt.Errorf("failed to copy embedded directory: %w", err)
+		}
+	}
+
+	lhm := &lhmProcess{
+		execPath: execPath,
+		tempDir:  destDir,
+	}
+
+	if err := lhm.startProcess(); err != nil {
+		return nil, fmt.Errorf("failed to start process: %w", err)
+	}
+
+	return lhm, nil
+}
+
+// startProcess starts the external LHM process
+func (lhm *lhmProcess) startProcess() error {
+	// Clean up any existing process
+	lhm.cleanupProcess()
+
+	cmd := exec.Command(lhm.execPath)
+	stdin, err := cmd.StdinPipe()
+	if err != nil {
+		return err
+	}
+
+	stdout, err := cmd.StdoutPipe()
+	if err != nil {
+		stdin.Close()
+		return err
+	}
+
+	if err := cmd.Start(); err != nil {
+		stdin.Close()
+		stdout.Close()
+		return err
+	}
+
+	// Update process state
+	lhm.cmd = cmd
+	lhm.stdin = stdin
+	lhm.stdout = stdout
+	lhm.scanner = bufio.NewScanner(stdout)
+	lhm.isRunning = true
+
+	// Give process a moment to initialize
+	time.Sleep(100 * time.Millisecond)
+
+	return nil
+}
+
+// cleanupProcess terminates the process and closes resources but preserves files
+func (lhm *lhmProcess) cleanupProcess() {
+	lhm.isRunning = false
+
+	if lhm.cmd != nil && lhm.cmd.Process != nil {
+		lhm.cmd.Process.Kill()
+		lhm.cmd.Wait()
+	}
+
+	if lhm.stdin != nil {
+		lhm.stdin.Close()
+		lhm.stdin = nil
+	}
+	if lhm.stdout != nil {
+		lhm.stdout.Close()
+		lhm.stdout = nil
+	}
+
+	lhm.cmd = nil
+	lhm.scanner = nil
+	lhm.stoppedNoSensors = false
+	lhm.consecutiveNoSensors = 0
+}
+
+func (lhm *lhmProcess) getTemps(ctx context.Context) (temps []sensors.TemperatureStat, err error) {
+	if lhm.stoppedNoSensors {
+		// Fall back to gopsutil if we can't get sensors from LHM
+		return sensors.TemperaturesWithContext(ctx)
+	}
+
+	// Start process if it's not running
+	if !lhm.isRunning || lhm.stdin == nil || lhm.scanner == nil {
+		err := lhm.startProcess()
+		if err != nil {
+			return temps, err
+		}
+	}
+
+	// Send command to process
+	_, err = fmt.Fprintln(lhm.stdin, "getTemps")
+	if err != nil {
+		lhm.isRunning = false
+		return temps, fmt.Errorf("failed to send command: %w", err)
+	}
+
+	// Read all sensor lines until we hit an empty line or EOF
+	for lhm.scanner.Scan() {
+		line := strings.TrimSpace(lhm.scanner.Text())
+		if line == "" {
+			break
+		}
+
+		parts := strings.Split(line, "|")
+		if len(parts) != 2 {
+			slog.Debug("Invalid sensor format", "line", line)
+			continue
+		}
+
+		name := strings.TrimSpace(parts[0])
+		valueStr := strings.TrimSpace(parts[1])
+
+		value, err := strconv.ParseFloat(valueStr, 64)
+		if err != nil {
+			slog.Debug("Failed to parse sensor", "err", err, "line", line)
+			continue
+		}
+
+		if name == "" || value <= 0 || value > 150 {
+			slog.Debug("Invalid sensor", "name", name, "val", value, "line", line)
+			continue
+		}
+
+		temps = append(temps, sensors.TemperatureStat{
+			SensorKey:   name,
+			Temperature: value,
+		})
+	}
+
+	if err := lhm.scanner.Err(); err != nil {
+		lhm.isRunning = false
+		return temps, err
+	}
+
+	// Handle no sensors case
+	if len(temps) == 0 {
+		lhm.consecutiveNoSensors++
+		if lhm.consecutiveNoSensors >= 3 {
+			lhm.stoppedNoSensors = true
+			slog.Warn(errNoSensors.Error())
+			lhm.cleanup()
+		}
+		return sensors.TemperaturesWithContext(ctx)
+	}
+
+	lhm.consecutiveNoSensors = 0
+
+	return temps, nil
+}
+
+// getSensorTemps attempts to pull sensor temperatures from the embedded LHM process.
+// NB: LibreHardwareMonitorLib requires admin privileges to access all available sensors.
+func getSensorTemps(ctx context.Context) (temps []sensors.TemperatureStat, err error) {
+	defer func() {
+		if err != nil {
+			slog.Debug("Error reading sensors", "err", err)
+		}
+	}()
+
+	// Initialize process once
+	beszelLhmOnce.Do(func() {
+		beszelLhm, err = newlhmProcess()
+	})
+
+	if err != nil {
+		return temps, fmt.Errorf("failed to initialize lhm: %w", err)
+	}
+
+	if beszelLhm == nil {
+		return temps, fmt.Errorf("lhm not available")
+	}
+
+	return beszelLhm.getTemps(ctx)
+}
+
+// cleanup terminates the process and closes resources
+func (lhm *lhmProcess) cleanup() {
+	lhm.cleanupProcess()
+	if lhm.tempDir != "" {
+		os.RemoveAll(lhm.tempDir)
+	}
+}
+
+// copyEmbeddedDir copies the embedded directory to the destination path
+func copyEmbeddedDir(fs embed.FS, srcPath, destPath string) error {
+	entries, err := fs.ReadDir(srcPath)
+	if err != nil {
+		return err
+	}
+	if err := os.MkdirAll(destPath, 0755); err != nil {
+		return err
+	}
+
+	for _, entry := range entries {
+		srcEntryPath := path.Join(srcPath, entry.Name())
+		destEntryPath := filepath.Join(destPath, entry.Name())
+
+		if entry.IsDir() {
+			if err := copyEmbeddedDir(fs, srcEntryPath, destEntryPath); err != nil {
+				return err
+			}
+			continue
+		}
+
+		data, err := fs.ReadFile(srcEntryPath)
+		if err != nil {
+			return err
+		}
+
+		if err := os.WriteFile(destEntryPath, data, 0755); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}

beszel/internal/agent/server.go

@@ -1,34 +1,223 @@
 package agent
 
 import (
+	"beszel"
+	"beszel/internal/common"
+	"beszel/internal/entities/system"
 	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
 	"log/slog"
+	"net"
 	"os"
+	"strings"
+	"time"
 
-	sshServer "github.com/gliderlabs/ssh"
+	"github.com/blang/semver"
+	"github.com/fxamacker/cbor/v2"
+	"github.com/gliderlabs/ssh"
+	gossh "golang.org/x/crypto/ssh"
 )
 
-func (a *Agent) startServer(pubKey []byte, addr string) {
-	sshServer.Handle(a.handleSession)
-
-	slog.Info("Starting SSH server", "address", addr)
-	if err := sshServer.ListenAndServe(addr, nil, sshServer.NoPty(),
-		sshServer.PublicKeyAuth(func(ctx sshServer.Context, key sshServer.PublicKey) bool {
-			allowed, _, _, _, _ := sshServer.ParseAuthorizedKey(pubKey)
-			return sshServer.KeysEqual(key, allowed)
-		}),
-	); err != nil {
-		slog.Error("Error starting SSH server", "err", err)
-		os.Exit(1)
-	}
+// ServerOptions contains configuration options for starting the SSH server.
+type ServerOptions struct {
+	Addr    string            // Network address to listen on (e.g., ":45876" or "/path/to/socket")
+	Network string            // Network type ("tcp" or "unix")
+	Keys    []gossh.PublicKey // SSH public keys for authentication
 }
 
-func (a *Agent) handleSession(s sshServer.Session) {
-	stats := a.gatherStats()
-	if err := json.NewEncoder(s).Encode(stats); err != nil {
-		slog.Error("Error encoding stats", "err", err)
+// hubVersions caches hub versions by session ID to avoid repeated parsing.
+var hubVersions map[string]semver.Version
+
+// StartServer starts the SSH server with the provided options.
+// It configures the server with secure defaults, sets up authentication,
+// and begins listening for connections. Returns an error if the server
+// is already running or if there's an issue starting the server.
+func (a *Agent) StartServer(opts ServerOptions) error {
+	if a.server != nil {
+		return errors.New("server already started")
+	}
+
+	slog.Info("Starting SSH server", "addr", opts.Addr, "network", opts.Network)
+
+	if opts.Network == "unix" {
+		// remove existing socket file if it exists
+		if err := os.Remove(opts.Addr); err != nil && !os.IsNotExist(err) {
+			return err
+		}
+	}
+
+	// start listening on the address
+	ln, err := net.Listen(opts.Network, opts.Addr)
+	if err != nil {
+		return err
+	}
+	defer ln.Close()
+
+	// base config (limit to allowed algorithms)
+	config := &gossh.ServerConfig{
+		ServerVersion: fmt.Sprintf("SSH-2.0-%s_%s", beszel.AppName, beszel.Version),
+	}
+	config.KeyExchanges = common.DefaultKeyExchanges
+	config.MACs = common.DefaultMACs
+	config.Ciphers = common.DefaultCiphers
+
+	// set default handler
+	ssh.Handle(a.handleSession)
+
+	a.server = &ssh.Server{
+		ServerConfigCallback: func(ctx ssh.Context) *gossh.ServerConfig {
+			return config
+		},
+		// check public key(s)
+		PublicKeyHandler: func(ctx ssh.Context, key ssh.PublicKey) bool {
+			remoteAddr := ctx.RemoteAddr()
+			for _, pubKey := range opts.Keys {
+				if ssh.KeysEqual(key, pubKey) {
+					slog.Info("SSH connected", "addr", remoteAddr)
+					return true
+				}
+			}
+			slog.Warn("Invalid SSH key", "addr", remoteAddr)
+			return false
+		},
+		// disable pty
+		PtyCallback: func(ctx ssh.Context, pty ssh.Pty) bool {
+			return false
+		},
+		// close idle connections after 70 seconds
+		IdleTimeout: 70 * time.Second,
+	}
+
+	// Start SSH server on the listener
+	return a.server.Serve(ln)
+}
+
+// getHubVersion retrieves and caches the hub version for a given session.
+// It extracts the version from the SSH client version string and caches
+// it to avoid repeated parsing. Returns a zero version if parsing fails.
+func (a *Agent) getHubVersion(sessionId string, sessionCtx ssh.Context) semver.Version {
+	if hubVersions == nil {
+		hubVersions = make(map[string]semver.Version, 1)
+	}
+	hubVersion, ok := hubVersions[sessionId]
+	if ok {
+		return hubVersion
+	}
+	// Extract hub version from SSH client version
+	clientVersion := sessionCtx.Value(ssh.ContextKeyClientVersion)
+	if versionStr, ok := clientVersion.(string); ok {
+		hubVersion, _ = extractHubVersion(versionStr)
+	}
+	hubVersions[sessionId] = hubVersion
+	return hubVersion
+}
+
+// handleSession handles an incoming SSH session by gathering system statistics
+// and sending them to the hub. It signals connection events, determines the
+// appropriate encoding format based on hub version, and exits with appropriate
+// status codes.
+func (a *Agent) handleSession(s ssh.Session) {
+	a.connectionManager.eventChan <- SSHConnect
+
+	sessionCtx := s.Context()
+	sessionID := sessionCtx.SessionID()
+
+	hubVersion := a.getHubVersion(sessionID, sessionCtx)
+
+	stats := a.gatherStats(sessionID)
+
+	err := a.writeToSession(s, stats, hubVersion)
+	if err != nil {
+		slog.Error("Error encoding stats", "err", err, "stats", stats)
 		s.Exit(1)
-		return
+	} else {
+		s.Exit(0)
 	}
-	s.Exit(0)
+}
+
+// writeToSession encodes and writes system statistics to the session.
+// It chooses between CBOR and JSON encoding based on the hub version,
+// using CBOR for newer versions and JSON for legacy compatibility.
+func (a *Agent) writeToSession(w io.Writer, stats *system.CombinedData, hubVersion semver.Version) error {
+	if hubVersion.GTE(beszel.MinVersionCbor) {
+		return cbor.NewEncoder(w).Encode(stats)
+	}
+	return json.NewEncoder(w).Encode(stats)
+}
+
+// extractHubVersion extracts the beszel version from SSH client version string.
+// Expected format: "SSH-2.0-beszel_X.Y.Z" or "beszel_X.Y.Z"
+func extractHubVersion(versionString string) (semver.Version, error) {
+	_, after, _ := strings.Cut(versionString, "_")
+	return semver.Parse(after)
+}
+
+// ParseKeys parses a string containing SSH public keys in authorized_keys format.
+// It returns a slice of ssh.PublicKey and an error if any key fails to parse.
+func ParseKeys(input string) ([]gossh.PublicKey, error) {
+	var parsedKeys []gossh.PublicKey
+	for line := range strings.Lines(input) {
+		line = strings.TrimSpace(line)
+		// Skip empty lines or comments
+		if len(line) == 0 || strings.HasPrefix(line, "#") {
+			continue
+		}
+		// Parse the key
+		parsedKey, _, _, _, err := gossh.ParseAuthorizedKey([]byte(line))
+		if err != nil {
+			return nil, fmt.Errorf("failed to parse key: %s, error: %w", line, err)
+		}
+		parsedKeys = append(parsedKeys, parsedKey)
+	}
+	return parsedKeys, nil
+}
+
+// GetAddress determines the network address to listen on from various sources.
+// It checks the provided address, then environment variables (LISTEN, PORT),
+// and finally defaults to ":45876".
+func GetAddress(addr string) string {
+	if addr == "" {
+		addr, _ = GetEnv("LISTEN")
+	}
+	if addr == "" {
+		// Legacy PORT environment variable support
+		addr, _ = GetEnv("PORT")
+	}
+	if addr == "" {
+		return ":45876"
+	}
+	// prefix with : if only port was provided
+	if GetNetwork(addr) != "unix" && !strings.Contains(addr, ":") {
+		addr = ":" + addr
+	}
+	return addr
+}
+
+// GetNetwork determines the network type based on the address format.
+// It checks the NETWORK environment variable first, then infers from
+// the address format: addresses starting with "/" are "unix", others are "tcp".
+func GetNetwork(addr string) string {
+	if network, ok := GetEnv("NETWORK"); ok && network != "" {
+		return network
+	}
+	if strings.HasPrefix(addr, "/") {
+		return "unix"
+	}
+	return "tcp"
+}
+
+// StopServer stops the SSH server if it's running.
+// It returns an error if the server is not running or if there's an error stopping it.
+func (a *Agent) StopServer() error {
+	if a.server == nil {
+		return errors.New("SSH server not running")
+	}
+
+	slog.Info("Stopping SSH server")
+	_ = a.server.Close()
+	a.server = nil
+	a.connectionManager.eventChan <- SSHDisconnect
+	return nil
 }

beszel/internal/agent/server_test.go

@@ -0,0 +1,605 @@
+package agent
+
+import (
+	"beszel/internal/entities/container"
+	"beszel/internal/entities/system"
+	"context"
+	"crypto/ed25519"
+	"encoding/json"
+	"fmt"
+	"net"
+	"os"
+	"path/filepath"
+	"strings"
+	"sync"
+	"testing"
+	"time"
+
+	"github.com/blang/semver"
+	"github.com/fxamacker/cbor/v2"
+	"github.com/gliderlabs/ssh"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	gossh "golang.org/x/crypto/ssh"
+)
+
+func TestStartServer(t *testing.T) {
+	// Generate a test key pair
+	pubKey, privKey, err := ed25519.GenerateKey(nil)
+	require.NoError(t, err)
+	signer, err := gossh.NewSignerFromKey(privKey)
+	require.NoError(t, err)
+	sshPubKey, err := gossh.NewPublicKey(pubKey)
+	require.NoError(t, err)
+
+	// Generate a different key pair for bad key test
+	badPubKey, badPrivKey, err := ed25519.GenerateKey(nil)
+	require.NoError(t, err)
+	badSigner, err := gossh.NewSignerFromKey(badPrivKey)
+	require.NoError(t, err)
+	sshBadPubKey, err := gossh.NewPublicKey(badPubKey)
+	require.NoError(t, err)
+
+	socketFile := filepath.Join(t.TempDir(), "beszel-test.sock")
+
+	tests := []struct {
+		name        string
+		config      ServerOptions
+		wantErr     bool
+		errContains string
+		setup       func() error
+		cleanup     func() error
+	}{
+		{
+			name: "tcp port only",
+			config: ServerOptions{
+				Network: "tcp",
+				Addr:    ":45987",
+				Keys:    []gossh.PublicKey{sshPubKey},
+			},
+		},
+		{
+			name: "tcp with ipv4",
+			config: ServerOptions{
+				Network: "tcp4",
+				Addr:    "127.0.0.1:45988",
+				Keys:    []gossh.PublicKey{sshPubKey},
+			},
+		},
+		{
+			name: "tcp with ipv6",
+			config: ServerOptions{
+				Network: "tcp6",
+				Addr:    "[::1]:45989",
+				Keys:    []gossh.PublicKey{sshPubKey},
+			},
+		},
+		{
+			name: "unix socket",
+			config: ServerOptions{
+				Network: "unix",
+				Addr:    socketFile,
+				Keys:    []gossh.PublicKey{sshPubKey},
+			},
+			setup: func() error {
+				// Create a socket file that should be removed
+				f, err := os.Create(socketFile)
+				if err != nil {
+					return err
+				}
+				return f.Close()
+			},
+			cleanup: func() error {
+				return os.Remove(socketFile)
+			},
+		},
+		{
+			name: "bad key should fail",
+			config: ServerOptions{
+				Network: "tcp",
+				Addr:    ":45987",
+				Keys:    []gossh.PublicKey{sshBadPubKey},
+			},
+			wantErr:     true,
+			errContains: "ssh: handshake failed",
+		},
+		{
+			name: "good key still good",
+			config: ServerOptions{
+				Network: "tcp",
+				Addr:    ":45987",
+				Keys:    []gossh.PublicKey{sshPubKey},
+			},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if tt.setup != nil {
+				err := tt.setup()
+				require.NoError(t, err)
+			}
+
+			if tt.cleanup != nil {
+				defer tt.cleanup()
+			}
+
+			agent, err := NewAgent("")
+			require.NoError(t, err)
+
+			// Start server in a goroutine since it blocks
+			errChan := make(chan error, 1)
+			go func() {
+				errChan <- agent.StartServer(tt.config)
+			}()
+
+			// Add a short delay to allow the server to start
+			time.Sleep(100 * time.Millisecond)
+
+			// Try to connect to verify server is running
+			var client *gossh.Client
+
+			// Choose the appropriate signer based on the test case
+			testSigner := signer
+			if tt.name == "bad key should fail" {
+				testSigner = badSigner
+			}
+
+			sshClientConfig := &gossh.ClientConfig{
+				User: "a",
+				Auth: []gossh.AuthMethod{
+					gossh.PublicKeys(testSigner),
+				},
+				HostKeyCallback: gossh.InsecureIgnoreHostKey(),
+				Timeout:         4 * time.Second,
+			}
+
+			switch tt.config.Network {
+			case "unix":
+				client, err = gossh.Dial("unix", tt.config.Addr, sshClientConfig)
+			default:
+				if !strings.Contains(tt.config.Addr, ":") {
+					tt.config.Addr = ":" + tt.config.Addr
+				}
+				client, err = gossh.Dial("tcp", tt.config.Addr, sshClientConfig)
+			}
+
+			if tt.wantErr {
+				assert.Error(t, err)
+				if tt.errContains != "" {
+					assert.Contains(t, err.Error(), tt.errContains)
+				}
+				return
+			}
+
+			require.NoError(t, err)
+			require.NotNil(t, client)
+			client.Close()
+		})
+	}
+}
+
+/////////////////////////////////////////////////////////////////
+//////////////////// ParseKeys Tests ////////////////////////////
+/////////////////////////////////////////////////////////////////
+
+// Helper function to generate a temporary file with content
+func createTempFile(content string) (string, error) {
+	tmpFile, err := os.CreateTemp("", "ssh_keys_*.txt")
+	if err != nil {
+		return "", fmt.Errorf("failed to create temp file: %w", err)
+	}
+	defer tmpFile.Close()
+
+	if _, err := tmpFile.WriteString(content); err != nil {
+		return "", fmt.Errorf("failed to write to temp file: %w", err)
+	}
+
+	return tmpFile.Name(), nil
+}
+
+// Test case 1: String with a single SSH key
+func TestParseSingleKeyFromString(t *testing.T) {
+	input := "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKCBM91kukN7hbvFKtbpEeo2JXjCcNxXcdBH7V7ADMBo"
+	keys, err := ParseKeys(input)
+	if err != nil {
+		t.Fatalf("Expected no error, got: %v", err)
+	}
+	if len(keys) != 1 {
+		t.Fatalf("Expected 1 key, got %d keys", len(keys))
+	}
+	if keys[0].Type() != "ssh-ed25519" {
+		t.Fatalf("Expected key type 'ssh-ed25519', got '%s'", keys[0].Type())
+	}
+}
+
+// Test case 2: String with multiple SSH keys
+func TestParseMultipleKeysFromString(t *testing.T) {
+	input := "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKCBM91kukN7hbvFKtbpEeo2JXjCcNxXcdBH7V7ADMBo\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJDMtAOQfxDlCxe+A5lVbUY/DHxK1LAF2Z3AV0FYv36D \n #comment\n ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJDMtAOQfxDlCxe+A5lVbUY/DHxK1LAF2Z3AV0FYv36D"
+	keys, err := ParseKeys(input)
+	if err != nil {
+		t.Fatalf("Expected no error, got: %v", err)
+	}
+	if len(keys) != 3 {
+		t.Fatalf("Expected 3 keys, got %d keys", len(keys))
+	}
+	if keys[0].Type() != "ssh-ed25519" || keys[1].Type() != "ssh-ed25519" || keys[2].Type() != "ssh-ed25519" {
+		t.Fatalf("Unexpected key types: %s, %s, %s", keys[0].Type(), keys[1].Type(), keys[2].Type())
+	}
+}
+
+// Test case 3: File with a single SSH key
+func TestParseSingleKeyFromFile(t *testing.T) {
+	content := "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKCBM91kukN7hbvFKtbpEeo2JXjCcNxXcdBH7V7ADMBo"
+	filePath, err := createTempFile(content)
+	if err != nil {
+		t.Fatalf("Failed to create temp file: %v", err)
+	}
+	defer os.Remove(filePath) // Clean up the file after the test
+
+	// Read the file content
+	fileContent, err := os.ReadFile(filePath)
+	if err != nil {
+		t.Fatalf("Failed to read temp file: %v", err)
+	}
+
+	// Parse the keys
+	keys, err := ParseKeys(string(fileContent))
+	if err != nil {
+		t.Fatalf("Expected no error, got: %v", err)
+	}
+	if len(keys) != 1 {
+		t.Fatalf("Expected 1 key, got %d keys", len(keys))
+	}
+	if keys[0].Type() != "ssh-ed25519" {
+		t.Fatalf("Expected key type 'ssh-ed25519', got '%s'", keys[0].Type())
+	}
+}
+
+// Test case 4: File with multiple SSH keys
+func TestParseMultipleKeysFromFile(t *testing.T) {
+	content := "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKCBM91kukN7hbvFKtbpEeo2JXjCcNxXcdBH7V7ADMBo\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJDMtAOQfxDlCxe+A5lVbUY/DHxK1LAF2Z3AV0FYv36D \n #comment\n ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJDMtAOQfxDlCxe+A5lVbUY/DHxK1LAF2Z3AV0FYv36D"
+	filePath, err := createTempFile(content)
+	if err != nil {
+		t.Fatalf("Failed to create temp file: %v", err)
+	}
+	// defer os.Remove(filePath) // Clean up the file after the test
+
+	// Read the file content
+	fileContent, err := os.ReadFile(filePath)
+	if err != nil {
+		t.Fatalf("Failed to read temp file: %v", err)
+	}
+
+	// Parse the keys
+	keys, err := ParseKeys(string(fileContent))
+	if err != nil {
+		t.Fatalf("Expected no error, got: %v", err)
+	}
+	if len(keys) != 3 {
+		t.Fatalf("Expected 3 keys, got %d keys", len(keys))
+	}
+	if keys[0].Type() != "ssh-ed25519" || keys[1].Type() != "ssh-ed25519" || keys[2].Type() != "ssh-ed25519" {
+		t.Fatalf("Unexpected key types: %s, %s, %s", keys[0].Type(), keys[1].Type(), keys[2].Type())
+	}
+}
+
+// Test case 5: Invalid SSH key input
+func TestParseInvalidKey(t *testing.T) {
+	input := "invalid-key-data"
+	_, err := ParseKeys(input)
+	if err == nil {
+		t.Fatalf("Expected an error for invalid key, got nil")
+	}
+	expectedErrMsg := "failed to parse key"
+	if !strings.Contains(err.Error(), expectedErrMsg) {
+		t.Fatalf("Expected error message to contain '%s', got: %v", expectedErrMsg, err)
+	}
+}
+
+/////////////////////////////////////////////////////////////////
+//////////////////// Hub Version Tests //////////////////////////
+/////////////////////////////////////////////////////////////////
+
+func TestExtractHubVersion(t *testing.T) {
+	tests := []struct {
+		name            string
+		clientVersion   string
+		expectedVersion string
+		expectError     bool
+	}{
+		{
+			name:            "valid beszel client version with underscore",
+			clientVersion:   "SSH-2.0-beszel_0.11.1",
+			expectedVersion: "0.11.1",
+			expectError:     false,
+		},
+		{
+			name:            "valid beszel client version with beta",
+			clientVersion:   "SSH-2.0-beszel_1.0.0-beta",
+			expectedVersion: "1.0.0-beta",
+			expectError:     false,
+		},
+		{
+			name:            "valid beszel client version with rc",
+			clientVersion:   "SSH-2.0-beszel_0.12.0-rc1",
+			expectedVersion: "0.12.0-rc1",
+			expectError:     false,
+		},
+		{
+			name:            "different SSH client",
+			clientVersion:   "SSH-2.0-OpenSSH_8.0",
+			expectedVersion: "8.0",
+			expectError:     true,
+		},
+		{
+			name:          "malformed version string without underscore",
+			clientVersion: "SSH-2.0-beszel",
+			expectError:   true,
+		},
+		{
+			name:          "empty version string",
+			clientVersion: "",
+			expectError:   true,
+		},
+		{
+			name:            "version string with underscore but no version",
+			clientVersion:   "beszel_",
+			expectedVersion: "",
+			expectError:     true,
+		},
+		{
+			name:            "version with patch and build metadata",
+			clientVersion:   "SSH-2.0-beszel_1.2.3+build.123",
+			expectedVersion: "1.2.3+build.123",
+			expectError:     false,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result, err := extractHubVersion(tt.clientVersion)
+
+			if tt.expectError {
+				assert.Error(t, err)
+				return
+			}
+
+			require.NoError(t, err)
+			assert.Equal(t, tt.expectedVersion, result.String())
+		})
+	}
+}
+
+/////////////////////////////////////////////////////////////////
+/////////////// Hub Version Detection Tests ////////////////////
+/////////////////////////////////////////////////////////////////
+
+func TestGetHubVersion(t *testing.T) {
+	agent, err := NewAgent("")
+	require.NoError(t, err)
+
+	// Mock SSH context that implements the ssh.Context interface
+	mockCtx := &mockSSHContext{
+		sessionID:     "test-session-123",
+		clientVersion: "SSH-2.0-beszel_0.12.0",
+	}
+
+	// Test first call - should extract and cache version
+	version := agent.getHubVersion("test-session-123", mockCtx)
+	assert.Equal(t, "0.12.0", version.String())
+
+	// Test second call - should return cached version
+	mockCtx.clientVersion = "SSH-2.0-beszel_0.11.0" // Change version but should still return cached
+	version = agent.getHubVersion("test-session-123", mockCtx)
+	assert.Equal(t, "0.12.0", version.String()) // Should still be cached version
+
+	// Test different session - should extract new version
+	version = agent.getHubVersion("different-session", mockCtx)
+	assert.Equal(t, "0.11.0", version.String())
+
+	// Test with invalid version string (non-beszel client)
+	mockCtx.clientVersion = "SSH-2.0-OpenSSH_8.0"
+	version = agent.getHubVersion("invalid-session", mockCtx)
+	assert.Equal(t, "0.0.0", version.String()) // Should be empty version for non-beszel clients
+
+	// Test with no client version
+	mockCtx.clientVersion = ""
+	version = agent.getHubVersion("no-version-session", mockCtx)
+	assert.True(t, version.EQ(semver.Version{})) // Should be empty version
+}
+
+// mockSSHContext implements ssh.Context for testing
+type mockSSHContext struct {
+	context.Context
+	sync.Mutex
+	sessionID     string
+	clientVersion string
+}
+
+func (m *mockSSHContext) SessionID() string {
+	return m.sessionID
+}
+
+func (m *mockSSHContext) ClientVersion() string {
+	return m.clientVersion
+}
+
+func (m *mockSSHContext) ServerVersion() string {
+	return "SSH-2.0-beszel_test"
+}
+
+func (m *mockSSHContext) Value(key interface{}) interface{} {
+	if key == ssh.ContextKeyClientVersion {
+		return m.clientVersion
+	}
+	return nil
+}
+
+func (m *mockSSHContext) User() string                    { return "test-user" }
+func (m *mockSSHContext) RemoteAddr() net.Addr            { return nil }
+func (m *mockSSHContext) LocalAddr() net.Addr             { return nil }
+func (m *mockSSHContext) Permissions() *ssh.Permissions   { return nil }
+func (m *mockSSHContext) SetValue(key, value interface{}) {}
+
+/////////////////////////////////////////////////////////////////
+/////////////// CBOR vs JSON Encoding Tests ////////////////////
+/////////////////////////////////////////////////////////////////
+
+// TestWriteToSessionEncoding tests that writeToSession actually encodes data in the correct format
+func TestWriteToSessionEncoding(t *testing.T) {
+	tests := []struct {
+		name             string
+		hubVersion       string
+		expectedUsesCbor bool
+	}{
+		{
+			name:             "old hub version should use JSON",
+			hubVersion:       "0.11.1",
+			expectedUsesCbor: false,
+		},
+		{
+			name:             "non-beta release should use CBOR",
+			hubVersion:       "0.12.0",
+			expectedUsesCbor: true,
+		},
+		{
+			name:             "even newer hub version should use CBOR",
+			hubVersion:       "0.16.4",
+			expectedUsesCbor: true,
+		},
+		{
+			name:             "beta version below release threshold should use JSON",
+			hubVersion:       "0.12.0-beta0",
+			expectedUsesCbor: false,
+		},
+		// {
+		// 	name:             "matching beta version should use CBOR",
+		// 	hubVersion:       "0.12.0-beta2",
+		// 	expectedUsesCbor: true,
+		// },
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			// Reset the global hubVersions map to ensure clean state for each test
+			hubVersions = nil
+
+			agent, err := NewAgent("")
+			require.NoError(t, err)
+
+			// Parse the test version
+			version, err := semver.Parse(tt.hubVersion)
+			require.NoError(t, err)
+
+			// Create test data to encode
+			testData := createTestCombinedData()
+
+			var buf strings.Builder
+			err = agent.writeToSession(&buf, testData, version)
+			require.NoError(t, err)
+
+			encodedData := buf.String()
+			require.NotEmpty(t, encodedData)
+
+			// Verify the encoding format by attempting to decode
+			if tt.expectedUsesCbor {
+				var decodedCbor system.CombinedData
+				err = cbor.Unmarshal([]byte(encodedData), &decodedCbor)
+				assert.NoError(t, err, "Should be valid CBOR data")
+
+				var decodedJson system.CombinedData
+				err = json.Unmarshal([]byte(encodedData), &decodedJson)
+				assert.Error(t, err, "Should not be valid JSON data")
+
+				assert.Equal(t, testData.Info.Hostname, decodedCbor.Info.Hostname)
+				assert.Equal(t, testData.Stats.Cpu, decodedCbor.Stats.Cpu)
+			} else {
+				// Should be JSON - try to decode as JSON
+				var decodedJson system.CombinedData
+				err = json.Unmarshal([]byte(encodedData), &decodedJson)
+				assert.NoError(t, err, "Should be valid JSON data")
+
+				var decodedCbor system.CombinedData
+				err = cbor.Unmarshal([]byte(encodedData), &decodedCbor)
+				assert.Error(t, err, "Should not be valid CBOR data")
+
+				// Verify the decoded JSON data matches our test data
+				assert.Equal(t, testData.Info.Hostname, decodedJson.Info.Hostname)
+				assert.Equal(t, testData.Stats.Cpu, decodedJson.Stats.Cpu)
+
+				// Verify it looks like JSON (starts with '{' and contains readable field names)
+				assert.True(t, strings.HasPrefix(encodedData, "{"), "JSON should start with '{'")
+				assert.Contains(t, encodedData, `"info"`, "JSON should contain readable field names")
+				assert.Contains(t, encodedData, `"stats"`, "JSON should contain readable field names")
+			}
+		})
+	}
+}
+
+// Helper function to create test data for encoding tests
+func createTestCombinedData() *system.CombinedData {
+	return &system.CombinedData{
+		Stats: system.Stats{
+			Cpu:       25.5,
+			Mem:       8589934592, // 8GB
+			MemUsed:   4294967296, // 4GB
+			MemPct:    50.0,
+			DiskTotal: 1099511627776, // 1TB
+			DiskUsed:  549755813888,  // 512GB
+			DiskPct:   50.0,
+		},
+		Info: system.Info{
+			Hostname:     "test-host",
+			Cores:        8,
+			CpuModel:     "Test CPU Model",
+			Uptime:       3600,
+			AgentVersion: "0.12.0",
+			Os:           system.Linux,
+		},
+		Containers: []*container.Stats{
+			{
+				Name: "test-container",
+				Cpu:  10.5,
+				Mem:  1073741824, // 1GB
+			},
+		},
+	}
+}
+
+func TestHubVersionCaching(t *testing.T) {
+	// Reset the global hubVersions map to ensure clean state
+	hubVersions = nil
+
+	agent, err := NewAgent("")
+	require.NoError(t, err)
+
+	ctx1 := &mockSSHContext{
+		sessionID:     "session1",
+		clientVersion: "SSH-2.0-beszel_0.12.0",
+	}
+	ctx2 := &mockSSHContext{
+		sessionID:     "session2",
+		clientVersion: "SSH-2.0-beszel_0.11.0",
+	}
+
+	// First calls should cache the versions
+	v1 := agent.getHubVersion("session1", ctx1)
+	v2 := agent.getHubVersion("session2", ctx2)
+
+	assert.Equal(t, "0.12.0", v1.String())
+	assert.Equal(t, "0.11.0", v2.String())
+
+	// Verify caching by changing context but keeping same session ID
+	ctx1.clientVersion = "SSH-2.0-beszel_0.10.0"
+	v1Cached := agent.getHubVersion("session1", ctx1)
+	assert.Equal(t, "0.12.0", v1Cached.String()) // Should still be cached version
+
+	// New session should get new version
+	ctx3 := &mockSSHContext{
+		sessionID:     "session3",
+		clientVersion: "SSH-2.0-beszel_0.13.0",
+	}
+	v3 := agent.getHubVersion("session3", ctx3)
+	assert.Equal(t, "0.13.0", v3.String())
+}

beszel/internal/agent/system.go

@@ -14,16 +14,34 @@ import (
 	"github.com/shirou/gopsutil/v4/cpu"
 	"github.com/shirou/gopsutil/v4/disk"
 	"github.com/shirou/gopsutil/v4/host"
+	"github.com/shirou/gopsutil/v4/load"
 	"github.com/shirou/gopsutil/v4/mem"
 	psutilNet "github.com/shirou/gopsutil/v4/net"
-	"github.com/shirou/gopsutil/v4/sensors"
 )
 
 // Sets initial / non-changing values about the host system
 func (a *Agent) initializeSystemInfo() {
 	a.systemInfo.AgentVersion = beszel.Version
 	a.systemInfo.Hostname, _ = os.Hostname()
-	a.systemInfo.KernelVersion, _ = host.KernelVersion()
+
+	platform, _, version, _ := host.PlatformInformation()
+
+	if platform == "darwin" {
+		a.systemInfo.KernelVersion = version
+		a.systemInfo.Os = system.Darwin
+	} else if strings.Contains(platform, "indows") {
+		a.systemInfo.KernelVersion = strings.Replace(platform, "Microsoft ", "", 1) + " " + version
+		a.systemInfo.Os = system.Windows
+	} else if platform == "freebsd" {
+		a.systemInfo.Os = system.Freebsd
+		a.systemInfo.KernelVersion = version
+	} else {
+		a.systemInfo.Os = system.Linux
+	}
+
+	if a.systemInfo.KernelVersion == "" {
+		a.systemInfo.KernelVersion, _ = host.KernelVersion()
+	}
 
 	// cpu model
 	if info, err := cpu.Info(); err == nil && len(info) > 0 {
@@ -60,6 +78,17 @@ func (a *Agent) getSystemStats() system.Stats {
 		systemStats.Cpu = twoDecimals(cpuPct[0])
 	}
 
+	// load average
+	if avgstat, err := load.Avg(); err == nil {
+		// TODO: remove these in future release in favor of load avg array
+		systemStats.LoadAvg[0] = avgstat.Load1
+		systemStats.LoadAvg[1] = avgstat.Load5
+		systemStats.LoadAvg[2] = avgstat.Load15
+		slog.Debug("Load average", "5m", avgstat.Load5, "15m", avgstat.Load15)
+	} else {
+		slog.Error("Error getting load average", "err", err)
+	}
+
 	// memory
 	if v, err := mem.VirtualMemory(); err == nil {
 		// swap
@@ -116,11 +145,17 @@ func (a *Agent) getSystemStats() system.Stats {
 				continue
 			}
 			secondsElapsed := time.Since(stats.Time).Seconds()
-			readPerSecond := float64(d.ReadBytes-stats.TotalRead) / secondsElapsed
-			writePerSecond := float64(d.WriteBytes-stats.TotalWrite) / secondsElapsed
+			readPerSecond := bytesToMegabytes(float64(d.ReadBytes-stats.TotalRead) / secondsElapsed)
+			writePerSecond := bytesToMegabytes(float64(d.WriteBytes-stats.TotalWrite) / secondsElapsed)
+			// check for invalid values and reset stats if so
+			if readPerSecond < 0 || writePerSecond < 0 || readPerSecond > 50_000 || writePerSecond > 50_000 {
+				slog.Warn("Invalid disk I/O. Resetting.", "name", d.Name, "read", readPerSecond, "write", writePerSecond)
+				a.initializeDiskIoStats(ioCounters)
+				break
+			}
 			stats.Time = time.Now()
-			stats.DiskReadPs = bytesToMegabytes(readPerSecond)
-			stats.DiskWritePs = bytesToMegabytes(writePerSecond)
+			stats.DiskReadPs = readPerSecond
+			stats.DiskWritePs = writePerSecond
 			stats.TotalRead = d.ReadBytes
 			stats.TotalWrite = d.WriteBytes
 			// if root filesystem, update system stats
@@ -132,28 +167,38 @@ func (a *Agent) getSystemStats() system.Stats {
 	}
 
 	// network stats
+	if len(a.netInterfaces) == 0 {
+		// if no network interfaces, initialize again
+		// this is a fix if agent started before network is online (#466)
+		// maybe refactor this in the future to not cache interface names at all so we
+		// don't miss an interface that's been added after agent started in any circumstance
+		a.initializeNetIoStats()
+	}
 	if netIO, err := psutilNet.IOCounters(true); err == nil {
-		secondsElapsed := time.Since(a.netIoStats.Time).Seconds()
+		msElapsed := uint64(time.Since(a.netIoStats.Time).Milliseconds())
 		a.netIoStats.Time = time.Now()
-		bytesSent := uint64(0)
-		bytesRecv := uint64(0)
+		totalBytesSent := uint64(0)
+		totalBytesRecv := uint64(0)
 		// sum all bytes sent and received
 		for _, v := range netIO {
 			// skip if not in valid network interfaces list
 			if _, exists := a.netInterfaces[v.Name]; !exists {
 				continue
 			}
-			bytesSent += v.BytesSent
-			bytesRecv += v.BytesRecv
+			totalBytesSent += v.BytesSent
+			totalBytesRecv += v.BytesRecv
 		}
 		// add to systemStats
-		sentPerSecond := float64(bytesSent-a.netIoStats.BytesSent) / secondsElapsed
-		recvPerSecond := float64(bytesRecv-a.netIoStats.BytesRecv) / secondsElapsed
-		networkSentPs := bytesToMegabytes(sentPerSecond)
-		networkRecvPs := bytesToMegabytes(recvPerSecond)
+		var bytesSentPerSecond, bytesRecvPerSecond uint64
+		if msElapsed > 0 {
+			bytesSentPerSecond = (totalBytesSent - a.netIoStats.BytesSent) * 1000 / msElapsed
+			bytesRecvPerSecond = (totalBytesRecv - a.netIoStats.BytesRecv) * 1000 / msElapsed
+		}
+		networkSentPs := bytesToMegabytes(float64(bytesSentPerSecond))
+		networkRecvPs := bytesToMegabytes(float64(bytesRecvPerSecond))
 		// add check for issue (#150) where sent is a massive number
 		if networkSentPs > 10_000 || networkRecvPs > 10_000 {
-			slog.Warn("Invalid network stats. Resetting.", "sent", networkSentPs, "recv", networkRecvPs)
+			slog.Warn("Invalid net stats. Resetting.", "sent", networkSentPs, "recv", networkRecvPs)
 			for _, v := range netIO {
 				if _, exists := a.netInterfaces[v.Name]; !exists {
 					continue
@@ -165,52 +210,63 @@ func (a *Agent) getSystemStats() system.Stats {
 		} else {
 			systemStats.NetworkSent = networkSentPs
 			systemStats.NetworkRecv = networkRecvPs
+			systemStats.Bandwidth[0], systemStats.Bandwidth[1] = bytesSentPerSecond, bytesRecvPerSecond
 			// update netIoStats
-			a.netIoStats.BytesSent = bytesSent
-			a.netIoStats.BytesRecv = bytesRecv
+			a.netIoStats.BytesSent = totalBytesSent
+			a.netIoStats.BytesRecv = totalBytesRecv
 		}
 	}
 
-	// temperatures (skip if sensors whitelist is set to empty string)
-	if a.sensorsWhitelist == nil || len(a.sensorsWhitelist) > 0 {
-		temps, err := sensors.TemperaturesWithContext(a.sensorsContext)
-		if err != nil {
-			// err.(*sensors.Warnings).Verbose = true
-			slog.Debug("Sensor error", "err", err)
-		}
-		slog.Debug("Temperature", "sensors", temps)
-		if len(temps) > 0 {
-			systemStats.Temperatures = make(map[string]float64, len(temps))
-			for i, sensor := range temps {
-				// skip if temperature is 0
-				if sensor.Temperature <= 0 || sensor.Temperature >= 200 {
-					continue
-				}
-				if _, ok := systemStats.Temperatures[sensor.SensorKey]; ok {
-					// if key already exists, append int to key
-					systemStats.Temperatures[sensor.SensorKey+"_"+strconv.Itoa(i)] = twoDecimals(sensor.Temperature)
-				} else {
-					systemStats.Temperatures[sensor.SensorKey] = twoDecimals(sensor.Temperature)
-				}
+	// temperatures
+	// TODO: maybe refactor to methods on systemStats
+	a.updateTemperatures(&systemStats)
+
+	// GPU data
+	if a.gpuManager != nil {
+		// reset high gpu percent
+		a.systemInfo.GpuPct = 0
+		// get current GPU data
+		if gpuData := a.gpuManager.GetCurrentData(); len(gpuData) > 0 {
+			systemStats.GPUData = gpuData
+
+			// add temperatures
+			if systemStats.Temperatures == nil {
+				systemStats.Temperatures = make(map[string]float64, len(gpuData))
 			}
-			// remove sensors from systemStats if whitelist exists and sensor is not in whitelist
-			// (do this here instead of in initial loop so we have correct keys if int was appended)
-			if a.sensorsWhitelist != nil {
-				for key := range systemStats.Temperatures {
-					if _, nameInWhitelist := a.sensorsWhitelist[key]; !nameInWhitelist {
-						delete(systemStats.Temperatures, key)
+			highestTemp := 0.0
+			for _, gpu := range gpuData {
+				if gpu.Temperature > 0 {
+					systemStats.Temperatures[gpu.Name] = gpu.Temperature
+					if a.sensorConfig.primarySensor == gpu.Name {
+						a.systemInfo.DashboardTemp = gpu.Temperature
+					}
+					if gpu.Temperature > highestTemp {
+						highestTemp = gpu.Temperature
 					}
 				}
+				// update high gpu percent for dashboard
+				a.systemInfo.GpuPct = max(a.systemInfo.GpuPct, gpu.Usage)
+			}
+			// use highest temp for dashboard temp if dashboard temp is unset
+			if a.systemInfo.DashboardTemp == 0 {
+				a.systemInfo.DashboardTemp = highestTemp
 			}
 		}
 	}
 
 	// update base system info
 	a.systemInfo.Cpu = systemStats.Cpu
+	a.systemInfo.LoadAvg = systemStats.LoadAvg
+	// TODO: remove these in future release in favor of load avg array
+	a.systemInfo.LoadAvg1 = systemStats.LoadAvg[0]
+	a.systemInfo.LoadAvg5 = systemStats.LoadAvg[1]
+	a.systemInfo.LoadAvg15 = systemStats.LoadAvg[2]
 	a.systemInfo.MemPct = systemStats.MemPct
 	a.systemInfo.DiskPct = systemStats.DiskPct
 	a.systemInfo.Uptime, _ = host.Uptime()
+	// TODO: in future release, remove MB bandwidth values in favor of bytes
 	a.systemInfo.Bandwidth = twoDecimals(systemStats.NetworkSent + systemStats.NetworkRecv)
+	a.systemInfo.BandwidthBytes = systemStats.Bandwidth[0] + systemStats.Bandwidth[1]
 	slog.Debug("sysinfo", "data", a.systemInfo)
 
 	return systemStats

beszel/internal/alerts/alerts.go

@@ -2,27 +2,29 @@
 package alerts
 
 import (
-	"beszel/internal/entities/system"
 	"fmt"
 	"net/mail"
 	"net/url"
-	"strings"
+	"sync"
 	"time"
 
-	"github.com/containrrr/shoutrrr"
-	"github.com/goccy/go-json"
-	"github.com/labstack/echo/v5"
+	"github.com/nicholas-fedor/shoutrrr"
 	"github.com/pocketbase/dbx"
-	"github.com/pocketbase/pocketbase"
 	"github.com/pocketbase/pocketbase/apis"
-	"github.com/pocketbase/pocketbase/models"
+	"github.com/pocketbase/pocketbase/core"
 	"github.com/pocketbase/pocketbase/tools/mailer"
-	"github.com/pocketbase/pocketbase/tools/types"
-	"github.com/spf13/cast"
 )
 
+type hubLike interface {
+	core.App
+	MakeLink(parts ...string) string
+}
+
 type AlertManager struct {
-	app *pocketbase.PocketBase
+	hub           hubLike
+	alertQueue    chan alertTask
+	stopChan      chan struct{}
+	pendingAlerts sync.Map
 }
 
 type AlertMessageData struct {
@@ -45,11 +47,12 @@ type SystemAlertStats struct {
 	NetSent      float64            `json:"ns"`
 	NetRecv      float64            `json:"nr"`
 	Temperatures map[string]float32 `json:"t"`
+	LoadAvg      [3]float64         `json:"la"`
 }
 
 type SystemAlertData struct {
-	systemRecord *models.Record
-	alertRecord  *models.Record
+	systemRecord *core.Record
+	alertRecord  *core.Record
 	name         string
 	unit         string
 	val          float64
@@ -62,350 +65,52 @@ type SystemAlertData struct {
 	descriptor   string // override descriptor in notification body (for temp sensor, disk partition, etc)
 }
 
-func NewAlertManager(app *pocketbase.PocketBase) *AlertManager {
-	return &AlertManager{
-		app: app,
-	}
+// notification services that support title param
+var supportsTitle = map[string]struct{}{
+	"bark":       {},
+	"discord":    {},
+	"gotify":     {},
+	"ifttt":      {},
+	"join":       {},
+	"lark":       {},
+	"matrix":     {},
+	"ntfy":       {},
+	"opsgenie":   {},
+	"pushbullet": {},
+	"pushover":   {},
+	"slack":      {},
+	"teams":      {},
+	"telegram":   {},
+	"zulip":      {},
 }
 
-func (am *AlertManager) HandleSystemAlerts(systemRecord *models.Record, systemInfo system.Info, temperatures map[string]float64, extraFs map[string]*system.FsStats) error {
-	// start := time.Now()
-	// defer func() {
-	// 	log.Println("alert stats took", time.Since(start))
-	// }()
-	alertRecords, err := am.app.Dao().FindRecordsByExpr("alerts",
-		dbx.NewExp("system={:system}", dbx.Params{"system": systemRecord.Id}),
-	)
-	if err != nil || len(alertRecords) == 0 {
-		// log.Println("no alerts found for system")
-		return nil
+// NewAlertManager creates a new AlertManager instance.
+func NewAlertManager(app hubLike) *AlertManager {
+	am := &AlertManager{
+		hub:        app,
+		alertQueue: make(chan alertTask),
+		stopChan:   make(chan struct{}),
 	}
-
-	var validAlerts []SystemAlertData
-	now := systemRecord.Updated.Time().UTC()
-	oldestTime := now
-
-	for _, alertRecord := range alertRecords {
-		name := alertRecord.GetString("name")
-		var val float64
-		unit := "%"
-
-		switch name {
-		case "CPU":
-			val = systemInfo.Cpu
-		case "Memory":
-			val = systemInfo.MemPct
-		case "Bandwidth":
-			val = systemInfo.Bandwidth
-			unit = " MB/s"
-		case "Disk":
-			maxUsedPct := systemInfo.DiskPct
-			for _, fs := range extraFs {
-				usedPct := fs.DiskUsed / fs.DiskTotal * 100
-				if usedPct > maxUsedPct {
-					maxUsedPct = usedPct
-				}
-			}
-			val = maxUsedPct
-		case "Temperature":
-			if temperatures == nil {
-				continue
-			}
-			for _, temp := range temperatures {
-				if temp > val {
-					val = temp
-				}
-			}
-			unit = "°C"
-		}
-
-		triggered := alertRecord.GetBool("triggered")
-		threshold := alertRecord.GetFloat("value")
-
-		// CONTINUE
-		// IF alert is not triggered and curValue is less than threshold
-		// OR alert is triggered and curValue is greater than threshold
-		if (!triggered && val <= threshold) || (triggered && val > threshold) {
-			// log.Printf("Skipping alert %s: val %f | threshold %f | triggered %v\n", name, val, threshold, triggered)
-			continue
-		}
-
-		min := max(1, cast.ToUint8(alertRecord.Get("min")))
-		// add time to alert time to make sure it's slighty after record creation
-		time := now.Add(-time.Duration(min) * time.Minute)
-		if time.Before(oldestTime) {
-			oldestTime = time
-		}
-
-		validAlerts = append(validAlerts, SystemAlertData{
-			systemRecord: systemRecord,
-			alertRecord:  alertRecord,
-			name:         name,
-			unit:         unit,
-			val:          val,
-			threshold:    threshold,
-			triggered:    triggered,
-			time:         time,
-			min:          min,
-		})
-	}
-
-	systemStats := []struct {
-		Stats   []byte         `db:"stats"`
-		Created types.DateTime `db:"created"`
-	}{}
-
-	err = am.app.Dao().DB().
-		Select("stats", "created").
-		From("system_stats").
-		Where(dbx.NewExp(
-			"system={:system} AND type='1m' AND created > {:created}",
-			dbx.Params{
-				"system": systemRecord.Id,
-				// subtract some time to give us a bit of buffer
-				"created": oldestTime.Add(-time.Second * 90),
-			},
-		)).
-		OrderBy("created").
-		All(&systemStats)
-
-	if err != nil {
-		return err
-	}
-
-	// get oldest record creation time from first record in the slice
-	oldestRecordTime := systemStats[0].Created.Time()
-	// log.Println("oldestRecordTime", oldestRecordTime.String())
-
-	// delete from validAlerts if time is older than oldestRecord
-	for i := 0; i < len(validAlerts); i++ {
-		if validAlerts[i].time.Before(oldestRecordTime) {
-			// log.Println("deleting alert - time is older than oldestRecord", validAlerts[i].name, oldestRecordTime, validAlerts[i].time)
-			validAlerts = append(validAlerts[:i], validAlerts[i+1:]...)
-		}
-	}
-
-	if len(validAlerts) == 0 {
-		// log.Println("no valid alerts found")
-		return nil
-	}
-
-	var stats SystemAlertStats
-
-	// we can skip the latest systemStats record since it's the current value
-	for i := 0; i < len(systemStats); i++ {
-		stat := systemStats[i]
-		// subtract 10 seconds to give a small time buffer
-		systemStatsCreation := stat.Created.Time().Add(-time.Second * 10)
-		if err := json.Unmarshal(stat.Stats, &stats); err != nil {
-			return err
-		}
-		// log.Println("stats", stats)
-		for j := range validAlerts {
-			alert := &validAlerts[j]
-			// reset alert val on first iteration
-			if i == 0 {
-				alert.val = 0
-			}
-			// continue if system_stats is older than alert time range
-			if systemStatsCreation.Before(alert.time) {
-				continue
-			}
-			// add to alert value
-			switch alert.name {
-			case "CPU":
-				alert.val += stats.Cpu
-			case "Memory":
-				alert.val += stats.Mem
-			case "Bandwidth":
-				alert.val += stats.NetSent + stats.NetRecv
-			case "Disk":
-				if alert.mapSums == nil {
-					alert.mapSums = make(map[string]float32, len(extraFs)+1)
-				}
-				// add root disk
-				if _, ok := alert.mapSums["root"]; !ok {
-					alert.mapSums["root"] = 0.0
-				}
-				alert.mapSums["root"] += float32(stats.Disk)
-				// add extra disks
-				for key, fs := range extraFs {
-					if _, ok := alert.mapSums[key]; !ok {
-						alert.mapSums[key] = 0.0
-					}
-					alert.mapSums[key] += float32(fs.DiskUsed / fs.DiskTotal * 100)
-				}
-			case "Temperature":
-				if alert.mapSums == nil {
-					alert.mapSums = make(map[string]float32, len(stats.Temperatures))
-				}
-				for key, temp := range stats.Temperatures {
-					if _, ok := alert.mapSums[key]; !ok {
-						alert.mapSums[key] = float32(0)
-					}
-					alert.mapSums[key] += temp
-				}
-			default:
-				continue
-			}
-			alert.count++
-		}
-	}
-	// sum up vals for each alert
-	for _, alert := range validAlerts {
-		switch alert.name {
-		case "Disk":
-			maxPct := float32(0)
-			for key, value := range alert.mapSums {
-				sumPct := float32(value)
-				if sumPct > maxPct {
-					maxPct = sumPct
-					alert.descriptor = fmt.Sprintf("Usage of %s", key)
-				}
-			}
-			alert.val = float64(maxPct / float32(alert.count))
-		case "Temperature":
-			maxTemp := float32(0)
-			for key, value := range alert.mapSums {
-				sumTemp := float32(value) / float32(alert.count)
-				if sumTemp > maxTemp {
-					maxTemp = sumTemp
-					alert.descriptor = fmt.Sprintf("Highest sensor %s", key)
-				}
-			}
-			alert.val = float64(maxTemp)
-		default:
-			alert.val = alert.val / float64(alert.count)
-		}
-		minCount := float32(alert.min) / 1.2
-		// log.Println("alert", alert.name, "val", alert.val, "threshold", alert.threshold, "triggered", alert.triggered)
-		// log.Printf("%s: val %f | count %d | min-count %f | threshold %f\n", alert.name, alert.val, alert.count, minCount, alert.threshold)
-		// pass through alert if count is greater than or equal to minCount
-		if float32(alert.count) >= minCount {
-			if !alert.triggered && alert.val > alert.threshold {
-				alert.triggered = true
-				go am.sendSystemAlert(alert)
-			} else if alert.triggered && alert.val <= alert.threshold {
-				alert.triggered = false
-				go am.sendSystemAlert(alert)
-			}
-		}
-	}
-	return nil
+	am.bindEvents()
+	go am.startWorker()
+	return am
 }
 
-func (am *AlertManager) sendSystemAlert(alert SystemAlertData) {
-	// log.Printf("Sending alert %s: val %f | count %d | threshold %f\n", alert.name, alert.val, alert.count, alert.threshold)
-	systemName := alert.systemRecord.GetString("name")
-
-	// change Disk to Disk usage
-	if alert.name == "Disk" {
-		alert.name += " usage"
-	}
-
-	// make title alert name lowercase if not CPU
-	titleAlertName := alert.name
-	if titleAlertName != "CPU" {
-		titleAlertName = strings.ToLower(titleAlertName)
-	}
-
-	var subject string
-	if alert.triggered {
-		subject = fmt.Sprintf("%s %s above threshold", systemName, titleAlertName)
-	} else {
-		subject = fmt.Sprintf("%s %s below threshold", systemName, titleAlertName)
-	}
-	minutesLabel := "minute"
-	if alert.min > 1 {
-		minutesLabel += "s"
-	}
-	if alert.descriptor == "" {
-		alert.descriptor = alert.name
-	}
-	body := fmt.Sprintf("%s averaged %.2f%s for the previous %v %s.", alert.descriptor, alert.val, alert.unit, alert.min, minutesLabel)
-
-	alert.alertRecord.Set("triggered", alert.triggered)
-	if err := am.app.Dao().SaveRecord(alert.alertRecord); err != nil {
-		// app.Logger().Error("failed to save alert record", "err", err.Error())
-		return
-	}
-	// expand the user relation and send the alert
-	if errs := am.app.Dao().ExpandRecord(alert.alertRecord, []string{"user"}, nil); len(errs) > 0 {
-		// app.Logger().Error("failed to expand user relation", "errs", errs)
-		return
-	}
-	if user := alert.alertRecord.ExpandedOne("user"); user != nil {
-		am.sendAlert(AlertMessageData{
-			UserID:   user.GetId(),
-			Title:    subject,
-			Message:  body,
-			Link:     am.app.Settings().Meta.AppUrl + "/system/" + url.PathEscape(systemName),
-			LinkText: "View " + systemName,
-		})
-	}
+// Bind events to the alerts collection lifecycle
+func (am *AlertManager) bindEvents() {
+	am.hub.OnRecordAfterUpdateSuccess("alerts").BindFunc(updateHistoryOnAlertUpdate)
+	am.hub.OnRecordAfterDeleteSuccess("alerts").BindFunc(resolveHistoryOnAlertDelete)
 }
 
-func (am *AlertManager) HandleStatusAlerts(newStatus string, oldSystemRecord *models.Record) error {
-	var alertStatus string
-	switch newStatus {
-	case "up":
-		if oldSystemRecord.GetString("status") == "down" {
-			alertStatus = "up"
-		}
-	case "down":
-		if oldSystemRecord.GetString("status") == "up" {
-			alertStatus = "down"
-		}
-	}
-	if alertStatus == "" {
-		return nil
-	}
-	// check if use
-	alertRecords, err := am.app.Dao().FindRecordsByExpr("alerts",
-		dbx.HashExp{
-			"system": oldSystemRecord.GetId(),
-			"name":   "Status",
-		},
-	)
-	if err != nil || len(alertRecords) == 0 {
-		// log.Println("no alerts found for system")
-		return nil
-	}
-	for _, alertRecord := range alertRecords {
-		// expand the user relation
-		if errs := am.app.Dao().ExpandRecord(alertRecord, []string{"user"}, nil); len(errs) > 0 {
-			return fmt.Errorf("failed to expand: %v", errs)
-		}
-		user := alertRecord.ExpandedOne("user")
-		if user == nil {
-			return nil
-		}
-		emoji := "\U0001F534"
-		if alertStatus == "up" {
-			emoji = "\u2705"
-		}
-		// send alert
-		systemName := oldSystemRecord.GetString("name")
-		am.sendAlert(AlertMessageData{
-			UserID:   user.GetId(),
-			Title:    fmt.Sprintf("Connection to %s is %s %v", systemName, alertStatus, emoji),
-			Message:  fmt.Sprintf("Connection to %s is %s", systemName, alertStatus),
-			Link:     am.app.Settings().Meta.AppUrl + "/system/" + url.PathEscape(systemName),
-			LinkText: "View " + systemName,
-		})
-	}
-	return nil
-}
-
-func (am *AlertManager) sendAlert(data AlertMessageData) {
+// SendAlert sends an alert to the user
+func (am *AlertManager) SendAlert(data AlertMessageData) error {
 	// get user settings
-	record, err := am.app.Dao().FindFirstRecordByFilter(
+	record, err := am.hub.FindFirstRecordByFilter(
 		"user_settings", "user={:user}",
 		dbx.Params{"user": data.UserID},
 	)
 	if err != nil {
-		am.app.Logger().Error("Failed to get user settings", "err", err.Error())
-		return
+		return err
 	}
 	// unmarshal user settings
 	userAlertSettings := UserNotificationSettings{
@@ -413,18 +118,17 @@ func (am *AlertManager) sendAlert(data AlertMessageData) {
 		Webhooks: []string{},
 	}
 	if err := record.UnmarshalJSONField("settings", &userAlertSettings); err != nil {
-		am.app.Logger().Error("Failed to unmarshal user settings", "err", err.Error())
+		am.hub.Logger().Error("Failed to unmarshal user settings", "err", err)
 	}
 	// send alerts via webhooks
 	for _, webhook := range userAlertSettings.Webhooks {
 		if err := am.SendShoutrrrAlert(webhook, data.Title, data.Message, data.Link, data.LinkText); err != nil {
-			am.app.Logger().Error("Failed to send shoutrrr alert", "err", err.Error())
+			am.hub.Logger().Error("Failed to send shoutrrr alert", "err", err)
 		}
 	}
 	// send alerts via email
 	if len(userAlertSettings.Emails) == 0 {
-		// log.Println("No email addresses found")
-		return
+		return nil
 	}
 	addresses := []mail.Address{}
 	for _, email := range userAlertSettings.Emails {
@@ -435,22 +139,20 @@ func (am *AlertManager) sendAlert(data AlertMessageData) {
 		Subject: data.Title,
 		Text:    data.Message + fmt.Sprintf("\n\n%s", data.Link),
 		From: mail.Address{
-			Address: am.app.Settings().Meta.SenderAddress,
-			Name:    am.app.Settings().Meta.SenderName,
+			Address: am.hub.Settings().Meta.SenderAddress,
+			Name:    am.hub.Settings().Meta.SenderName,
 		},
 	}
-	if err := am.app.NewMailClient().Send(&message); err != nil {
-		am.app.Logger().Error("Failed to send alert: ", "err", err.Error())
-	} else {
-		am.app.Logger().Info("Sent email alert", "to", message.To, "subj", message.Subject)
+	err = am.hub.NewMailClient().Send(&message)
+	if err != nil {
+		return err
 	}
+	am.hub.Logger().Info("Sent email alert", "to", message.To, "subj", message.Subject)
+	return nil
 }
 
 // SendShoutrrrAlert sends an alert via a Shoutrrr URL
 func (am *AlertManager) SendShoutrrrAlert(notificationUrl, title, message, link, linkText string) error {
-	// services that support title param
-	supportsTitle := []string{"bark", "discord", "gotify", "ifttt", "join", "matrix", "ntfy", "opsgenie", "pushbullet", "pushover", "slack", "teams", "telegram", "zulip"}
-
 	// Parse the URL
 	parsedURL, err := url.Parse(notificationUrl)
 	if err != nil {
@@ -460,7 +162,7 @@ func (am *AlertManager) SendShoutrrrAlert(notificationUrl, title, message, link,
 	queryParams := parsedURL.Query()
 
 	// Add title
-	if sliceContains(supportsTitle, scheme) {
+	if _, ok := supportsTitle[scheme]; ok {
 		queryParams.Add("title", title)
 	} else if scheme == "mattermost" {
 		// use markdown title for mattermost
@@ -479,10 +181,12 @@ func (am *AlertManager) SendShoutrrrAlert(notificationUrl, title, message, link,
 
 	// Add link
 	if scheme == "ntfy" {
-		// if ntfy, add link to actions
 		queryParams.Add("Actions", fmt.Sprintf("view, %s, %s", linkText, link))
+	} else if scheme == "lark" {
+		queryParams.Add("link", link)
+	} else if scheme == "bark" {
+		queryParams.Add("url", link)
 	} else {
-		// else add link directly to the message
 		message += "\n\n" + link
 	}
 
@@ -493,37 +197,27 @@ func (am *AlertManager) SendShoutrrrAlert(notificationUrl, title, message, link,
 	err = shoutrrr.Send(parsedURL.String(), message)
 
 	if err == nil {
-		am.app.Logger().Info("Sent shoutrrr alert", "title", title)
+		am.hub.Logger().Info("Sent shoutrrr alert", "title", title)
 	} else {
-		am.app.Logger().Error("Error sending shoutrrr alert", "err", err.Error())
+		am.hub.Logger().Error("Error sending shoutrrr alert", "err", err)
 		return err
 	}
 	return nil
 }
 
-// Contains checks if a string is present in a slice of strings
-func sliceContains(slice []string, item string) bool {
-	for _, v := range slice {
-		if v == item {
-			return true
-		}
-	}
-	return false
-}
-
-func (am *AlertManager) SendTestNotification(c echo.Context) error {
-	requestData := apis.RequestInfo(c)
-	if requestData.AuthRecord == nil {
+func (am *AlertManager) SendTestNotification(e *core.RequestEvent) error {
+	info, _ := e.RequestInfo()
+	if info.Auth == nil {
 		return apis.NewForbiddenError("Forbidden", nil)
 	}
-	url := c.QueryParam("url")
+	url := e.Request.URL.Query().Get("url")
 	// log.Println("url", url)
 	if url == "" {
-		return c.JSON(200, map[string]string{"err": "URL is required"})
+		return e.JSON(200, map[string]string{"err": "URL is required"})
 	}
-	err := am.SendShoutrrrAlert(url, "Test Alert", "This is a notification from Beszel.", am.app.Settings().Meta.AppUrl, "View Beszel")
+	err := am.SendShoutrrrAlert(url, "Test Alert", "This is a notification from Beszel.", am.hub.Settings().Meta.AppURL, "View Beszel")
 	if err != nil {
-		return c.JSON(200, map[string]string{"err": err.Error()})
+		return e.JSON(200, map[string]string{"err": err.Error()})
 	}
-	return c.JSON(200, map[string]bool{"err": false})
+	return e.JSON(200, map[string]bool{"err": false})
 }

beszel/internal/alerts/alerts_history.go

@@ -0,0 +1,85 @@
+package alerts
+
+import (
+	"time"
+
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/core"
+)
+
+// On triggered alert record delete, set matching alert history record to resolved
+func resolveHistoryOnAlertDelete(e *core.RecordEvent) error {
+	if !e.Record.GetBool("triggered") {
+		return e.Next()
+	}
+	_ = resolveAlertHistoryRecord(e.App, e.Record)
+	return e.Next()
+}
+
+// On alert record update, update alert history record
+func updateHistoryOnAlertUpdate(e *core.RecordEvent) error {
+	original := e.Record.Original()
+	new := e.Record
+
+	originalTriggered := original.GetBool("triggered")
+	newTriggered := new.GetBool("triggered")
+
+	// no need to update alert history if triggered state has not changed
+	if originalTriggered == newTriggered {
+		return e.Next()
+	}
+
+	// if new state is triggered, create new alert history record
+	if newTriggered {
+		_, _ = createAlertHistoryRecord(e.App, new)
+		return e.Next()
+	}
+
+	// if new state is not triggered, check for matching alert history record and set it to resolved
+	_ = resolveAlertHistoryRecord(e.App, new)
+	return e.Next()
+}
+
+// resolveAlertHistoryRecord sets the resolved field to the current time
+func resolveAlertHistoryRecord(app core.App, alertRecord *core.Record) error {
+	alertHistoryRecords, err := app.FindRecordsByFilter(
+		"alerts_history",
+		"alert_id={:alert_id} && resolved=null",
+		"-created",
+		1,
+		0,
+		dbx.Params{"alert_id": alertRecord.Id},
+	)
+	if err != nil {
+		return err
+	}
+	if len(alertHistoryRecords) == 0 {
+		return nil
+	}
+	alertHistoryRecord := alertHistoryRecords[0] // there should be only one record
+	alertHistoryRecord.Set("resolved", time.Now().UTC())
+	err = app.Save(alertHistoryRecord)
+	if err != nil {
+		app.Logger().Error("Failed to resolve alert history", "err", err)
+	}
+	return err
+}
+
+// createAlertHistoryRecord creates a new alert history record
+func createAlertHistoryRecord(app core.App, alertRecord *core.Record) (alertHistoryRecord *core.Record, err error) {
+	alertHistoryCollection, err := app.FindCachedCollectionByNameOrId("alerts_history")
+	if err != nil {
+		return nil, err
+	}
+	alertHistoryRecord = core.NewRecord(alertHistoryCollection)
+	alertHistoryRecord.Set("alert_id", alertRecord.Id)
+	alertHistoryRecord.Set("user", alertRecord.GetString("user"))
+	alertHistoryRecord.Set("system", alertRecord.GetString("system"))
+	alertHistoryRecord.Set("name", alertRecord.GetString("name"))
+	alertHistoryRecord.Set("value", alertRecord.GetFloat("value"))
+	err = app.Save(alertHistoryRecord)
+	if err != nil {
+		app.Logger().Error("Failed to save alert history", "err", err)
+	}
+	return alertHistoryRecord, err
+}

beszel/internal/alerts/alerts_status.go

@@ -0,0 +1,172 @@
+package alerts
+
+import (
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/core"
+)
+
+type alertTask struct {
+	action      string // "schedule" or "cancel"
+	systemName  string
+	alertRecord *core.Record
+	delay       time.Duration
+}
+
+type alertInfo struct {
+	systemName  string
+	alertRecord *core.Record
+	expireTime  time.Time
+}
+
+// startWorker is a long-running goroutine that processes alert tasks
+// every x seconds. It must be running to process status alerts.
+func (am *AlertManager) startWorker() {
+	tick := time.Tick(15 * time.Second)
+	for {
+		select {
+		case <-am.stopChan:
+			return
+		case task := <-am.alertQueue:
+			switch task.action {
+			case "schedule":
+				am.pendingAlerts.Store(task.alertRecord.Id, &alertInfo{
+					systemName:  task.systemName,
+					alertRecord: task.alertRecord,
+					expireTime:  time.Now().Add(task.delay),
+				})
+			case "cancel":
+				am.pendingAlerts.Delete(task.alertRecord.Id)
+			}
+		case <-tick:
+			// Check for expired alerts every tick
+			now := time.Now()
+			for key, value := range am.pendingAlerts.Range {
+				info := value.(*alertInfo)
+				if now.After(info.expireTime) {
+					// Downtime delay has passed, process alert
+					am.sendStatusAlert("down", info.systemName, info.alertRecord)
+					am.pendingAlerts.Delete(key)
+				}
+			}
+		}
+	}
+}
+
+// StopWorker shuts down the AlertManager.worker goroutine
+func (am *AlertManager) StopWorker() {
+	close(am.stopChan)
+}
+
+// HandleStatusAlerts manages the logic when system status changes.
+func (am *AlertManager) HandleStatusAlerts(newStatus string, systemRecord *core.Record) error {
+	if newStatus != "up" && newStatus != "down" {
+		return nil
+	}
+
+	alertRecords, err := am.getSystemStatusAlerts(systemRecord.Id)
+	if err != nil {
+		return err
+	}
+	if len(alertRecords) == 0 {
+		return nil
+	}
+
+	systemName := systemRecord.GetString("name")
+	if newStatus == "down" {
+		am.handleSystemDown(systemName, alertRecords)
+	} else {
+		am.handleSystemUp(systemName, alertRecords)
+	}
+	return nil
+}
+
+// getSystemStatusAlerts retrieves all "Status" alert records for a given system ID.
+func (am *AlertManager) getSystemStatusAlerts(systemID string) ([]*core.Record, error) {
+	alertRecords, err := am.hub.FindAllRecords("alerts", dbx.HashExp{
+		"system": systemID,
+		"name":   "Status",
+	})
+	if err != nil {
+		return nil, err
+	}
+	return alertRecords, nil
+}
+
+// Schedules delayed "down" alerts for each alert record.
+func (am *AlertManager) handleSystemDown(systemName string, alertRecords []*core.Record) {
+	for _, alertRecord := range alertRecords {
+		// Continue if alert is already scheduled
+		if _, exists := am.pendingAlerts.Load(alertRecord.Id); exists {
+			continue
+		}
+		// Schedule by adding to queue
+		min := max(1, alertRecord.GetInt("min"))
+		am.alertQueue <- alertTask{
+			action:      "schedule",
+			systemName:  systemName,
+			alertRecord: alertRecord,
+			delay:       time.Duration(min) * time.Minute,
+		}
+	}
+}
+
+// handleSystemUp manages the logic when a system status changes to "up".
+// It cancels any pending alerts and sends "up" alerts.
+func (am *AlertManager) handleSystemUp(systemName string, alertRecords []*core.Record) {
+	for _, alertRecord := range alertRecords {
+		alertRecordID := alertRecord.Id
+		// If alert exists for record, delete and continue (down alert not sent)
+		if _, exists := am.pendingAlerts.Load(alertRecordID); exists {
+			am.alertQueue <- alertTask{
+				action:      "cancel",
+				alertRecord: alertRecord,
+			}
+			continue
+		}
+		// No alert scheduled for this record, send "up" alert
+		if err := am.sendStatusAlert("up", systemName, alertRecord); err != nil {
+			am.hub.Logger().Error("Failed to send alert", "err", err)
+		}
+	}
+}
+
+// sendStatusAlert sends a status alert ("up" or "down") to the users associated with the alert records.
+func (am *AlertManager) sendStatusAlert(alertStatus string, systemName string, alertRecord *core.Record) error {
+	switch alertStatus {
+	case "up":
+		alertRecord.Set("triggered", false)
+	case "down":
+		alertRecord.Set("triggered", true)
+	}
+	am.hub.Save(alertRecord)
+
+	var emoji string
+	if alertStatus == "up" {
+		emoji = "\u2705" // Green checkmark emoji
+	} else {
+		emoji = "\U0001F534" // Red alert emoji
+	}
+
+	title := fmt.Sprintf("Connection to %s is %s %v", systemName, alertStatus, emoji)
+	message := strings.TrimSuffix(title, emoji)
+
+	// if errs := am.hub.ExpandRecord(alertRecord, []string{"user"}, nil); len(errs) > 0 {
+	// 	return errs["user"]
+	// }
+	// user := alertRecord.ExpandedOne("user")
+	// if user == nil {
+	// 	return nil
+	// }
+
+	return am.SendAlert(AlertMessageData{
+		UserID:   alertRecord.GetString("user"),
+		Title:    title,
+		Message:  message,
+		Link:     am.hub.MakeLink("system", systemName),
+		LinkText: "View " + systemName,
+	})
+}

beszel/internal/alerts/alerts_system.go

@@ -0,0 +1,310 @@
+package alerts
+
+import (
+	"beszel/internal/entities/system"
+	"encoding/json"
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/types"
+	"github.com/spf13/cast"
+)
+
+func (am *AlertManager) HandleSystemAlerts(systemRecord *core.Record, data *system.CombinedData) error {
+	alertRecords, err := am.hub.FindAllRecords("alerts",
+		dbx.NewExp("system={:system} AND name!='Status'", dbx.Params{"system": systemRecord.Id}),
+	)
+	if err != nil || len(alertRecords) == 0 {
+		// log.Println("no alerts found for system")
+		return nil
+	}
+
+	var validAlerts []SystemAlertData
+	now := systemRecord.GetDateTime("updated").Time().UTC()
+	oldestTime := now
+
+	for _, alertRecord := range alertRecords {
+		name := alertRecord.GetString("name")
+		var val float64
+		unit := "%"
+
+		switch name {
+		case "CPU":
+			val = data.Info.Cpu
+		case "Memory":
+			val = data.Info.MemPct
+		case "Bandwidth":
+			val = data.Info.Bandwidth
+			unit = " MB/s"
+		case "Disk":
+			maxUsedPct := data.Info.DiskPct
+			for _, fs := range data.Stats.ExtraFs {
+				usedPct := fs.DiskUsed / fs.DiskTotal * 100
+				if usedPct > maxUsedPct {
+					maxUsedPct = usedPct
+				}
+			}
+			val = maxUsedPct
+		case "Temperature":
+			if data.Info.DashboardTemp < 1 {
+				continue
+			}
+			val = data.Info.DashboardTemp
+			unit = "°C"
+		case "LoadAvg1":
+			val = data.Info.LoadAvg[0]
+			unit = ""
+		case "LoadAvg5":
+			val = data.Info.LoadAvg[1]
+			unit = ""
+		case "LoadAvg15":
+			val = data.Info.LoadAvg[2]
+			unit = ""
+		}
+
+		triggered := alertRecord.GetBool("triggered")
+		threshold := alertRecord.GetFloat("value")
+
+		// CONTINUE
+		// IF alert is not triggered and curValue is less than threshold
+		// OR alert is triggered and curValue is greater than threshold
+		if (!triggered && val <= threshold) || (triggered && val > threshold) {
+			// log.Printf("Skipping alert %s: val %f | threshold %f | triggered %v\n", name, val, threshold, triggered)
+			continue
+		}
+
+		min := max(1, cast.ToUint8(alertRecord.Get("min")))
+
+		alert := SystemAlertData{
+			systemRecord: systemRecord,
+			alertRecord:  alertRecord,
+			name:         name,
+			unit:         unit,
+			val:          val,
+			threshold:    threshold,
+			triggered:    triggered,
+			min:          min,
+		}
+
+		// send alert immediately if min is 1 - no need to sum up values.
+		if min == 1 {
+			alert.triggered = val > threshold
+			go am.sendSystemAlert(alert)
+			continue
+		}
+
+		alert.time = now.Add(-time.Duration(min) * time.Minute)
+		if alert.time.Before(oldestTime) {
+			oldestTime = alert.time
+		}
+
+		validAlerts = append(validAlerts, alert)
+	}
+
+	systemStats := []struct {
+		Stats   []byte         `db:"stats"`
+		Created types.DateTime `db:"created"`
+	}{}
+
+	err = am.hub.DB().
+		Select("stats", "created").
+		From("system_stats").
+		Where(dbx.NewExp(
+			"system={:system} AND type='1m' AND created > {:created}",
+			dbx.Params{
+				"system": systemRecord.Id,
+				// subtract some time to give us a bit of buffer
+				"created": oldestTime.Add(-time.Second * 90),
+			},
+		)).
+		OrderBy("created").
+		All(&systemStats)
+	if err != nil || len(systemStats) == 0 {
+		return err
+	}
+
+	// get oldest record creation time from first record in the slice
+	oldestRecordTime := systemStats[0].Created.Time()
+	// log.Println("oldestRecordTime", oldestRecordTime.String())
+
+	// Filter validAlerts to keep only those with time newer than oldestRecord
+	filteredAlerts := make([]SystemAlertData, 0, len(validAlerts))
+	for _, alert := range validAlerts {
+		if alert.time.After(oldestRecordTime) {
+			filteredAlerts = append(filteredAlerts, alert)
+		}
+	}
+	validAlerts = filteredAlerts
+
+	if len(validAlerts) == 0 {
+		// log.Println("no valid alerts found")
+		return nil
+	}
+
+	var stats SystemAlertStats
+
+	// we can skip the latest systemStats record since it's the current value
+	for i := range systemStats {
+		stat := systemStats[i]
+		// subtract 10 seconds to give a small time buffer
+		systemStatsCreation := stat.Created.Time().Add(-time.Second * 10)
+		if err := json.Unmarshal(stat.Stats, &stats); err != nil {
+			return err
+		}
+		// log.Println("stats", stats)
+		for j := range validAlerts {
+			alert := &validAlerts[j]
+			// reset alert val on first iteration
+			if i == 0 {
+				alert.val = 0
+			}
+			// continue if system_stats is older than alert time range
+			if systemStatsCreation.Before(alert.time) {
+				continue
+			}
+			// add to alert value
+			switch alert.name {
+			case "CPU":
+				alert.val += stats.Cpu
+			case "Memory":
+				alert.val += stats.Mem
+			case "Bandwidth":
+				alert.val += stats.NetSent + stats.NetRecv
+			case "Disk":
+				if alert.mapSums == nil {
+					alert.mapSums = make(map[string]float32, len(data.Stats.ExtraFs)+1)
+				}
+				// add root disk
+				if _, ok := alert.mapSums["root"]; !ok {
+					alert.mapSums["root"] = 0.0
+				}
+				alert.mapSums["root"] += float32(stats.Disk)
+				// add extra disks
+				for key, fs := range data.Stats.ExtraFs {
+					if _, ok := alert.mapSums[key]; !ok {
+						alert.mapSums[key] = 0.0
+					}
+					alert.mapSums[key] += float32(fs.DiskUsed / fs.DiskTotal * 100)
+				}
+			case "Temperature":
+				if alert.mapSums == nil {
+					alert.mapSums = make(map[string]float32, len(stats.Temperatures))
+				}
+				for key, temp := range stats.Temperatures {
+					if _, ok := alert.mapSums[key]; !ok {
+						alert.mapSums[key] = float32(0)
+					}
+					alert.mapSums[key] += temp
+				}
+			case "LoadAvg1":
+				alert.val += stats.LoadAvg[0]
+			case "LoadAvg5":
+				alert.val += stats.LoadAvg[1]
+			case "LoadAvg15":
+				alert.val += stats.LoadAvg[2]
+			default:
+				continue
+			}
+			alert.count++
+		}
+	}
+	// sum up vals for each alert
+	for _, alert := range validAlerts {
+		switch alert.name {
+		case "Disk":
+			maxPct := float32(0)
+			for key, value := range alert.mapSums {
+				sumPct := float32(value)
+				if sumPct > maxPct {
+					maxPct = sumPct
+					alert.descriptor = fmt.Sprintf("Usage of %s", key)
+				}
+			}
+			alert.val = float64(maxPct / float32(alert.count))
+		case "Temperature":
+			maxTemp := float32(0)
+			for key, value := range alert.mapSums {
+				sumTemp := float32(value) / float32(alert.count)
+				if sumTemp > maxTemp {
+					maxTemp = sumTemp
+					alert.descriptor = fmt.Sprintf("Highest sensor %s", key)
+				}
+			}
+			alert.val = float64(maxTemp)
+		default:
+			alert.val = alert.val / float64(alert.count)
+		}
+		minCount := float32(alert.min) / 1.2
+		// log.Println("alert", alert.name, "val", alert.val, "threshold", alert.threshold, "triggered", alert.triggered)
+		// log.Printf("%s: val %f | count %d | min-count %f | threshold %f\n", alert.name, alert.val, alert.count, minCount, alert.threshold)
+		// pass through alert if count is greater than or equal to minCount
+		if float32(alert.count) >= minCount {
+			if !alert.triggered && alert.val > alert.threshold {
+				alert.triggered = true
+				go am.sendSystemAlert(alert)
+			} else if alert.triggered && alert.val <= alert.threshold {
+				alert.triggered = false
+				go am.sendSystemAlert(alert)
+			}
+		}
+	}
+	return nil
+}
+
+func (am *AlertManager) sendSystemAlert(alert SystemAlertData) {
+	// log.Printf("Sending alert %s: val %f | count %d | threshold %f\n", alert.name, alert.val, alert.count, alert.threshold)
+	systemName := alert.systemRecord.GetString("name")
+
+	// change Disk to Disk usage
+	if alert.name == "Disk" {
+		alert.name += " usage"
+	}
+	// format LoadAvg5 and LoadAvg15
+	if after, ok := strings.CutPrefix(alert.name, "LoadAvg"); ok {
+		alert.name = after + "m Load"
+	}
+
+	// make title alert name lowercase if not CPU
+	titleAlertName := alert.name
+	if titleAlertName != "CPU" {
+		titleAlertName = strings.ToLower(titleAlertName)
+	}
+
+	var subject string
+	if alert.triggered {
+		subject = fmt.Sprintf("%s %s above threshold", systemName, titleAlertName)
+	} else {
+		subject = fmt.Sprintf("%s %s below threshold", systemName, titleAlertName)
+	}
+	minutesLabel := "minute"
+	if alert.min > 1 {
+		minutesLabel += "s"
+	}
+	if alert.descriptor == "" {
+		alert.descriptor = alert.name
+	}
+	body := fmt.Sprintf("%s averaged %.2f%s for the previous %v %s.", alert.descriptor, alert.val, alert.unit, alert.min, minutesLabel)
+
+	alert.alertRecord.Set("triggered", alert.triggered)
+	if err := am.hub.Save(alert.alertRecord); err != nil {
+		// app.Logger().Error("failed to save alert record", "err", err)
+		return
+	}
+	// expand the user relation and send the alert
+	if errs := am.hub.ExpandRecord(alert.alertRecord, []string{"user"}, nil); len(errs) > 0 {
+		// app.Logger().Error("failed to expand user relation", "errs", errs)
+		return
+	}
+	if user := alert.alertRecord.ExpandedOne("user"); user != nil {
+		am.SendAlert(AlertMessageData{
+			UserID:   user.Id,
+			Title:    subject,
+			Message:  body,
+			Link:     am.hub.MakeLink("system", systemName),
+			LinkText: "View " + systemName,
+		})
+	}
+}

beszel/internal/common/common-ssh.go

@@ -0,0 +1,10 @@
+package common
+
+var (
+	// Allowed ssh key exchanges
+	DefaultKeyExchanges = []string{"curve25519-sha256"}
+	// Allowed ssh macs
+	DefaultMACs = []string{"hmac-sha2-256-etm@openssh.com"}
+	// Allowed ssh ciphers
+	DefaultCiphers = []string{"chacha20-poly1305@openssh.com"}
+)

beszel/internal/common/common-ws.go

@@ -0,0 +1,32 @@
+package common
+
+type WebSocketAction = uint8
+
+// Not implemented yet
+// type AgentError = uint8
+
+const (
+	// Request system data from agent
+	GetData WebSocketAction = iota
+	// Check the fingerprint of the agent
+	CheckFingerprint
+)
+
+// HubRequest defines the structure for requests sent from hub to agent.
+type HubRequest[T any] struct {
+	Action WebSocketAction `cbor:"0,keyasint"`
+	Data   T               `cbor:"1,keyasint,omitempty,omitzero"`
+	// Error  AgentError      `cbor:"error,omitempty,omitzero"`
+}
+
+type FingerprintRequest struct {
+	Signature   []byte `cbor:"0,keyasint"`
+	NeedSysInfo bool   `cbor:"1,keyasint"` // For universal token system creation
+}
+
+type FingerprintResponse struct {
+	Fingerprint string `cbor:"0,keyasint"`
+	// Optional system info for universal token system creation
+	Hostname string `cbor:"1,keyasint,omitempty,omitzero"`
+	Port     string `cbor:"2,keyasint,omitempty,omitzero"`
+}

beszel/internal/entities/container/container.go

@@ -27,38 +27,47 @@ type ApiInfo struct {
 
 // Docker container resources from /containers/{id}/stats
 type ApiStats struct {
-	// Common stats
-	// Read    time.Time `json:"read"`
-	// PreRead time.Time `json:"preread"`
+	Read        time.Time `json:"read"`               // Time of stats generation
+	NumProcs    uint32    `json:"num_procs,omitzero"` // Windows specific, not populated on Linux.
+	Networks    map[string]NetworkStats
+	CPUStats    CPUStats    `json:"cpu_stats"`
+	MemoryStats MemoryStats `json:"memory_stats"`
+}
 
-	// Linux specific stats, not populated on Windows.
-	// PidsStats  PidsStats  `json:"pids_stats,omitempty"`
-	// BlkioStats BlkioStats `json:"blkio_stats,omitempty"`
+func (s *ApiStats) CalculateCpuPercentLinux(prevCpuContainer uint64, prevCpuSystem uint64) float64 {
+	cpuDelta := s.CPUStats.CPUUsage.TotalUsage - prevCpuContainer
+	systemDelta := s.CPUStats.SystemUsage - prevCpuSystem
 
-	// Windows specific stats, not populated on Linux.
-	// NumProcs uint32 `json:"num_procs"`
-	// StorageStats StorageStats `json:"storage_stats,omitempty"`
-	// Networks request version >=1.21
-	Networks map[string]NetworkStats
+	// Avoid division by zero and handle first run case
+	if systemDelta == 0 || prevCpuContainer == 0 {
+		return 0.0
+	}
 
-	// Shared stats
-	CPUStats CPUStats `json:"cpu_stats,omitempty"`
-	// PreCPUStats CPUStats    `json:"precpu_stats,omitempty"` // "Pre"="Previous"
-	MemoryStats MemoryStats `json:"memory_stats,omitempty"`
+	return float64(cpuDelta) / float64(systemDelta) * 100.0
+}
+
+// from: https://github.com/docker/cli/blob/master/cli/command/container/stats_helpers.go#L185
+func (s *ApiStats) CalculateCpuPercentWindows(prevCpuUsage uint64, prevRead time.Time) float64 {
+	// Max number of 100ns intervals between the previous time read and now
+	possIntervals := uint64(s.Read.Sub(prevRead).Nanoseconds())
+	possIntervals /= 100                // Convert to number of 100ns intervals
+	possIntervals *= uint64(s.NumProcs) // Multiple by the number of processors
+
+	// Intervals used
+	intervalsUsed := s.CPUStats.CPUUsage.TotalUsage - prevCpuUsage
+
+	// Percentage avoiding divide-by-zero
+	if possIntervals > 0 {
+		return float64(intervalsUsed) / float64(possIntervals) * 100.0
+	}
+	return 0.00
 }
 
 type CPUStats struct {
 	// CPU Usage. Linux and Windows.
 	CPUUsage CPUUsage `json:"cpu_usage"`
-
 	// System Usage. Linux only.
 	SystemUsage uint64 `json:"system_cpu_usage,omitempty"`
-
-	// Online CPUs. Linux only.
-	// OnlineCPUs uint32 `json:"online_cpus,omitempty"`
-
-	// Throttling Data. Linux only.
-	// ThrottlingData ThrottlingData `json:"throttling_data,omitempty"`
 }
 
 type CPUUsage struct {
@@ -66,42 +75,15 @@ type CPUUsage struct {
 	// Units: nanoseconds (Linux)
 	// Units: 100's of nanoseconds (Windows)
 	TotalUsage uint64 `json:"total_usage"`
-
-	// Total CPU time consumed per core (Linux). Not used on Windows.
-	// Units: nanoseconds.
-	// PercpuUsage []uint64 `json:"percpu_usage,omitempty"`
-
-	// Time spent by tasks of the cgroup in kernel mode (Linux).
-	// Time spent by all container processes in kernel mode (Windows).
-	// Units: nanoseconds (Linux).
-	// Units: 100's of nanoseconds (Windows). Not populated for Hyper-V Containers.
-	// UsageInKernelmode uint64 `json:"usage_in_kernelmode"`
-
-	// Time spent by tasks of the cgroup in user mode (Linux).
-	// Time spent by all container processes in user mode (Windows).
-	// Units: nanoseconds (Linux).
-	// Units: 100's of nanoseconds (Windows). Not populated for Hyper-V Containers
-	// UsageInUsermode uint64 `json:"usage_in_usermode"`
 }
 
 type MemoryStats struct {
 	// current res_counter usage for memory
 	Usage uint64 `json:"usage,omitempty"`
 	// all the stats exported via memory.stat.
-	Stats MemoryStatsStats `json:"stats,omitempty"`
-	// maximum usage ever recorded.
-	// MaxUsage uint64 `json:"max_usage,omitempty"`
-	// TODO(vishh): Export these as stronger types.
-	// number of times memory usage hits limits.
-	// Failcnt uint64 `json:"failcnt,omitempty"`
-	// Limit   uint64 `json:"limit,omitempty"`
-
-	// // committed bytes
-	// Commit uint64 `json:"commitbytes,omitempty"`
-	// // peak committed bytes
-	// CommitPeak uint64 `json:"commitpeakbytes,omitempty"`
-	// // private working set
-	// PrivateWorkingSet uint64 `json:"privateworkingset,omitempty"`
+	Stats MemoryStatsStats `json:"stats"`
+	// private working set (Windows only)
+	PrivateWorkingSet uint64 `json:"privateworkingset,omitempty"`
 }
 
 type MemoryStatsStats struct {
@@ -119,16 +101,18 @@ type NetworkStats struct {
 type prevNetStats struct {
 	Sent uint64
 	Recv uint64
-	Time time.Time
 }
 
 // Docker container stats
 type Stats struct {
-	Name        string       `json:"n"`
-	Cpu         float64      `json:"c"`
-	Mem         float64      `json:"m"`
-	NetworkSent float64      `json:"ns"`
-	NetworkRecv float64      `json:"nr"`
-	PrevCpu     [2]uint64    `json:"-"`
-	PrevNet     prevNetStats `json:"-"`
+	Name        string  `json:"n" cbor:"0,keyasint"`
+	Cpu         float64 `json:"c" cbor:"1,keyasint"`
+	Mem         float64 `json:"m" cbor:"2,keyasint"`
+	NetworkSent float64 `json:"ns" cbor:"3,keyasint"`
+	NetworkRecv float64 `json:"nr" cbor:"4,keyasint"`
+	// PrevCpu     [2]uint64    `json:"-"`
+	CpuSystem    uint64       `json:"-"`
+	CpuContainer uint64       `json:"-"`
+	PrevNet      prevNetStats `json:"-"`
+	PrevReadTime time.Time    `json:"-"`
 }

beszel/internal/entities/system/system.go

@@ -1,47 +1,67 @@
 package system
 
+// TODO: this is confusing, make common package with common/types common/helpers etc
+
 import (
 	"beszel/internal/entities/container"
 	"time"
 )
 
 type Stats struct {
-	Cpu            float64             `json:"cpu"`
-	MaxCpu         float64             `json:"cpum,omitempty"`
-	Mem            float64             `json:"m"`
-	MemUsed        float64             `json:"mu"`
-	MemPct         float64             `json:"mp"`
-	MemBuffCache   float64             `json:"mb"`
-	MemZfsArc      float64             `json:"mz,omitempty"` // ZFS ARC memory
-	Swap           float64             `json:"s,omitempty"`
-	SwapUsed       float64             `json:"su,omitempty"`
-	DiskTotal      float64             `json:"d"`
-	DiskUsed       float64             `json:"du"`
-	DiskPct        float64             `json:"dp"`
-	DiskReadPs     float64             `json:"dr"`
-	DiskWritePs    float64             `json:"dw"`
-	MaxDiskReadPs  float64             `json:"drm,omitempty"`
-	MaxDiskWritePs float64             `json:"dwm,omitempty"`
-	NetworkSent    float64             `json:"ns"`
-	NetworkRecv    float64             `json:"nr"`
-	MaxNetworkSent float64             `json:"nsm,omitempty"`
-	MaxNetworkRecv float64             `json:"nrm,omitempty"`
-	Temperatures   map[string]float64  `json:"t,omitempty"`
-	ExtraFs        map[string]*FsStats `json:"efs,omitempty"`
+	Cpu            float64             `json:"cpu" cbor:"0,keyasint"`
+	MaxCpu         float64             `json:"cpum,omitempty" cbor:"1,keyasint,omitempty"`
+	Mem            float64             `json:"m" cbor:"2,keyasint"`
+	MemUsed        float64             `json:"mu" cbor:"3,keyasint"`
+	MemPct         float64             `json:"mp" cbor:"4,keyasint"`
+	MemBuffCache   float64             `json:"mb" cbor:"5,keyasint"`
+	MemZfsArc      float64             `json:"mz,omitempty" cbor:"6,keyasint,omitempty"` // ZFS ARC memory
+	Swap           float64             `json:"s,omitempty" cbor:"7,keyasint,omitempty"`
+	SwapUsed       float64             `json:"su,omitempty" cbor:"8,keyasint,omitempty"`
+	DiskTotal      float64             `json:"d" cbor:"9,keyasint"`
+	DiskUsed       float64             `json:"du" cbor:"10,keyasint"`
+	DiskPct        float64             `json:"dp" cbor:"11,keyasint"`
+	DiskReadPs     float64             `json:"dr" cbor:"12,keyasint"`
+	DiskWritePs    float64             `json:"dw" cbor:"13,keyasint"`
+	MaxDiskReadPs  float64             `json:"drm,omitempty" cbor:"14,keyasint,omitempty"`
+	MaxDiskWritePs float64             `json:"dwm,omitempty" cbor:"15,keyasint,omitempty"`
+	NetworkSent    float64             `json:"ns" cbor:"16,keyasint"`
+	NetworkRecv    float64             `json:"nr" cbor:"17,keyasint"`
+	MaxNetworkSent float64             `json:"nsm,omitempty" cbor:"18,keyasint,omitempty"`
+	MaxNetworkRecv float64             `json:"nrm,omitempty" cbor:"19,keyasint,omitempty"`
+	Temperatures   map[string]float64  `json:"t,omitempty" cbor:"20,keyasint,omitempty"`
+	ExtraFs        map[string]*FsStats `json:"efs,omitempty" cbor:"21,keyasint,omitempty"`
+	GPUData        map[string]GPUData  `json:"g,omitempty" cbor:"22,keyasint,omitempty"`
+	LoadAvg1       float64             `json:"l1,omitempty" cbor:"23,keyasint,omitempty"`
+	LoadAvg5       float64             `json:"l5,omitempty" cbor:"24,keyasint,omitempty"`
+	LoadAvg15      float64             `json:"l15,omitempty" cbor:"25,keyasint,omitempty"`
+	Bandwidth      [2]uint64           `json:"b,omitzero" cbor:"26,keyasint,omitzero"`  // [sent bytes, recv bytes]
+	MaxBandwidth   [2]uint64           `json:"bm,omitzero" cbor:"27,keyasint,omitzero"` // [sent bytes, recv bytes]
+	LoadAvg        [3]float64          `json:"la,omitempty" cbor:"28,keyasint"`
+	// TODO: remove other load fields in future release in favor of load avg array
+}
+
+type GPUData struct {
+	Name        string  `json:"n" cbor:"0,keyasint"`
+	Temperature float64 `json:"-"`
+	MemoryUsed  float64 `json:"mu,omitempty" cbor:"1,keyasint,omitempty"`
+	MemoryTotal float64 `json:"mt,omitempty" cbor:"2,keyasint,omitempty"`
+	Usage       float64 `json:"u" cbor:"3,keyasint"`
+	Power       float64 `json:"p,omitempty" cbor:"4,keyasint,omitempty"`
+	Count       float64 `json:"-"`
 }
 
 type FsStats struct {
 	Time           time.Time `json:"-"`
 	Root           bool      `json:"-"`
 	Mountpoint     string    `json:"-"`
-	DiskTotal      float64   `json:"d"`
-	DiskUsed       float64   `json:"du"`
+	DiskTotal      float64   `json:"d" cbor:"0,keyasint"`
+	DiskUsed       float64   `json:"du" cbor:"1,keyasint"`
 	TotalRead      uint64    `json:"-"`
 	TotalWrite     uint64    `json:"-"`
-	DiskReadPs     float64   `json:"r"`
-	DiskWritePs    float64   `json:"w"`
-	MaxDiskReadPS  float64   `json:"rm,omitempty"`
-	MaxDiskWritePS float64   `json:"wm,omitempty"`
+	DiskReadPs     float64   `json:"r" cbor:"2,keyasint"`
+	DiskWritePs    float64   `json:"w" cbor:"3,keyasint"`
+	MaxDiskReadPS  float64   `json:"rm,omitempty" cbor:"4,keyasint,omitempty"`
+	MaxDiskWritePS float64   `json:"wm,omitempty" cbor:"5,keyasint,omitempty"`
 }
 
 type NetIoStats struct {
@@ -51,23 +71,42 @@ type NetIoStats struct {
 	Name      string
 }
 
+type Os = uint8
+
+const (
+	Linux Os = iota
+	Darwin
+	Windows
+	Freebsd
+)
+
 type Info struct {
-	Hostname      string  `json:"h"`
-	KernelVersion string  `json:"k,omitempty"`
-	Cores         int     `json:"c"`
-	Threads       int     `json:"t,omitempty"`
-	CpuModel      string  `json:"m"`
-	Uptime        uint64  `json:"u"`
-	Cpu           float64 `json:"cpu"`
-	MemPct        float64 `json:"mp"`
-	DiskPct       float64 `json:"dp"`
-	Bandwidth     float64 `json:"b"`
-	AgentVersion  string  `json:"v"`
+	Hostname       string     `json:"h" cbor:"0,keyasint"`
+	KernelVersion  string     `json:"k,omitempty" cbor:"1,keyasint,omitempty"`
+	Cores          int        `json:"c" cbor:"2,keyasint"`
+	Threads        int        `json:"t,omitempty" cbor:"3,keyasint,omitempty"`
+	CpuModel       string     `json:"m" cbor:"4,keyasint"`
+	Uptime         uint64     `json:"u" cbor:"5,keyasint"`
+	Cpu            float64    `json:"cpu" cbor:"6,keyasint"`
+	MemPct         float64    `json:"mp" cbor:"7,keyasint"`
+	DiskPct        float64    `json:"dp" cbor:"8,keyasint"`
+	Bandwidth      float64    `json:"b" cbor:"9,keyasint"`
+	AgentVersion   string     `json:"v" cbor:"10,keyasint"`
+	Podman         bool       `json:"p,omitempty" cbor:"11,keyasint,omitempty"`
+	GpuPct         float64    `json:"g,omitempty" cbor:"12,keyasint,omitempty"`
+	DashboardTemp  float64    `json:"dt,omitempty" cbor:"13,keyasint,omitempty"`
+	Os             Os         `json:"os" cbor:"14,keyasint"`
+	LoadAvg1       float64    `json:"l1,omitempty" cbor:"15,keyasint,omitempty"`
+	LoadAvg5       float64    `json:"l5,omitempty" cbor:"16,keyasint,omitempty"`
+	LoadAvg15      float64    `json:"l15,omitempty" cbor:"17,keyasint,omitempty"`
+	BandwidthBytes uint64     `json:"bb" cbor:"18,keyasint"`
+	LoadAvg        [3]float64 `json:"la,omitempty" cbor:"19,keyasint"`
+	// TODO: remove load fields in future release in favor of load avg array
 }
 
 // Final data structure to return to the hub
 type CombinedData struct {
-	Stats      Stats              `json:"stats"`
-	Info       Info               `json:"info"`
-	Containers []*container.Stats `json:"container"`
+	Stats      Stats              `json:"stats" cbor:"0,keyasint"`
+	Info       Info               `json:"info" cbor:"1,keyasint"`
+	Containers []*container.Stats `json:"container" cbor:"2,keyasint"`
 }

beszel/internal/hub/agent_connect.go

@@ -0,0 +1,320 @@
+package hub
+
+import (
+	"beszel/internal/common"
+	"beszel/internal/hub/expirymap"
+	"beszel/internal/hub/ws"
+	"errors"
+	"net"
+	"net/http"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/blang/semver"
+	"github.com/lxzan/gws"
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/core"
+)
+
+// agentConnectRequest holds information related to an agent's connection attempt.
+type agentConnectRequest struct {
+	hub         *Hub
+	req         *http.Request
+	res         http.ResponseWriter
+	token       string
+	agentSemVer semver.Version
+	// isUniversalToken is true if the token is a universal token.
+	isUniversalToken bool
+	// userId is the user ID associated with the universal token.
+	userId string
+}
+
+// universalTokenMap stores active universal tokens and their associated user IDs.
+var universalTokenMap tokenMap
+
+type tokenMap struct {
+	store *expirymap.ExpiryMap[string]
+	once  sync.Once
+}
+
+// getMap returns the expirymap, creating it if necessary.
+func (tm *tokenMap) GetMap() *expirymap.ExpiryMap[string] {
+	tm.once.Do(func() {
+		tm.store = expirymap.New[string](time.Hour)
+	})
+	return tm.store
+}
+
+// handleAgentConnect is the HTTP handler for an agent's connection request.
+func (h *Hub) handleAgentConnect(e *core.RequestEvent) error {
+	agentRequest := agentConnectRequest{req: e.Request, res: e.Response, hub: h}
+	_ = agentRequest.agentConnect()
+	return nil
+}
+
+// agentConnect validates agent credentials and upgrades the connection to a WebSocket.
+func (acr *agentConnectRequest) agentConnect() (err error) {
+	var agentVersion string
+
+	acr.token, agentVersion, err = acr.validateAgentHeaders(acr.req.Header)
+	if err != nil {
+		return acr.sendResponseError(acr.res, http.StatusBadRequest, "")
+	}
+
+	// Check if token is an active universal token
+	acr.userId, acr.isUniversalToken = universalTokenMap.GetMap().GetOk(acr.token)
+
+	// Find matching fingerprint records for this token
+	fpRecords := getFingerprintRecordsByToken(acr.token, acr.hub)
+	if len(fpRecords) == 0 && !acr.isUniversalToken {
+		// Invalid token - no records found and not a universal token
+		return acr.sendResponseError(acr.res, http.StatusUnauthorized, "Invalid token")
+	}
+
+	// Validate agent version
+	acr.agentSemVer, err = semver.Parse(agentVersion)
+	if err != nil {
+		return acr.sendResponseError(acr.res, http.StatusUnauthorized, "Invalid agent version")
+	}
+
+	// Upgrade connection to WebSocket
+	conn, err := ws.GetUpgrader().Upgrade(acr.res, acr.req)
+	if err != nil {
+		return acr.sendResponseError(acr.res, http.StatusInternalServerError, "WebSocket upgrade failed")
+	}
+
+	go acr.verifyWsConn(conn, fpRecords)
+
+	return nil
+}
+
+// verifyWsConn verifies the WebSocket connection using the agent's fingerprint and
+// SSH key signature, then adds the system to the system manager.
+func (acr *agentConnectRequest) verifyWsConn(conn *gws.Conn, fpRecords []ws.FingerprintRecord) (err error) {
+	wsConn := ws.NewWsConnection(conn)
+
+	// must set wsConn in connection store before the read loop
+	conn.Session().Store("wsConn", wsConn)
+
+	// make sure connection is closed if there is an error
+	defer func() {
+		if err != nil {
+			wsConn.Close([]byte(err.Error()))
+		}
+	}()
+
+	go conn.ReadLoop()
+
+	signer, err := acr.hub.GetSSHKey("")
+	if err != nil {
+		return err
+	}
+
+	agentFingerprint, err := wsConn.GetFingerprint(acr.token, signer, acr.isUniversalToken)
+	if err != nil {
+		return err
+	}
+
+	// Find or create the appropriate system for this token and fingerprint
+	fpRecord, err := acr.findOrCreateSystemForToken(fpRecords, agentFingerprint)
+	if err != nil {
+		return err
+	}
+
+	return acr.hub.sm.AddWebSocketSystem(fpRecord.SystemId, acr.agentSemVer, wsConn)
+}
+
+// validateAgentHeaders extracts and validates the token and agent version from HTTP headers.
+func (acr *agentConnectRequest) validateAgentHeaders(headers http.Header) (string, string, error) {
+	token := headers.Get("X-Token")
+	agentVersion := headers.Get("X-Beszel")
+
+	if agentVersion == "" || token == "" || len(token) > 64 {
+		return "", "", errors.New("")
+	}
+	return token, agentVersion, nil
+}
+
+// sendResponseError writes an HTTP error response.
+func (acr *agentConnectRequest) sendResponseError(res http.ResponseWriter, code int, message string) error {
+	res.WriteHeader(code)
+	if message != "" {
+		res.Write([]byte(message))
+	}
+	return nil
+}
+
+// getFingerprintRecordsByToken retrieves all fingerprint records associated with a given token.
+func getFingerprintRecordsByToken(token string, h *Hub) []ws.FingerprintRecord {
+	var records []ws.FingerprintRecord
+	// All will populate empty slice even on error
+	_ = h.DB().NewQuery("SELECT id, system, fingerprint, token FROM fingerprints WHERE token = {:token}").
+		Bind(dbx.Params{
+			"token": token,
+		}).
+		All(&records)
+	return records
+}
+
+// findOrCreateSystemForToken finds an existing system matching the token and fingerprint,
+// or creates a new one for a universal token.
+func (acr *agentConnectRequest) findOrCreateSystemForToken(fpRecords []ws.FingerprintRecord, agentFingerprint common.FingerprintResponse) (ws.FingerprintRecord, error) {
+	// No records - only valid for active universal tokens
+	if len(fpRecords) == 0 {
+		return acr.handleNoRecords(agentFingerprint)
+	}
+
+	// Single record - handle as regular token
+	if len(fpRecords) == 1 && !acr.isUniversalToken {
+		return acr.handleSingleRecord(fpRecords[0], agentFingerprint)
+	}
+
+	// Multiple records or universal token - look for matching fingerprint
+	return acr.handleMultipleRecordsOrUniversalToken(fpRecords, agentFingerprint)
+}
+
+// handleNoRecords handles the case where no fingerprint records are found for a token.
+// A new system is created if the token is a valid universal token.
+func (acr *agentConnectRequest) handleNoRecords(agentFingerprint common.FingerprintResponse) (ws.FingerprintRecord, error) {
+	var fpRecord ws.FingerprintRecord
+
+	if !acr.isUniversalToken || acr.userId == "" {
+		return fpRecord, errors.New("no matching fingerprints")
+	}
+
+	return acr.createNewSystemForUniversalToken(agentFingerprint)
+}
+
+// handleSingleRecord handles the case with a single fingerprint record. It validates
+// the agent's fingerprint against the stored one, or sets it on first connect.
+func (acr *agentConnectRequest) handleSingleRecord(fpRecord ws.FingerprintRecord, agentFingerprint common.FingerprintResponse) (ws.FingerprintRecord, error) {
+	// If no current fingerprint, update with new fingerprint (first time connecting)
+	if fpRecord.Fingerprint == "" {
+		if err := acr.hub.SetFingerprint(&fpRecord, agentFingerprint.Fingerprint); err != nil {
+			return fpRecord, err
+		}
+		// Update the record with the fingerprint that was set
+		fpRecord.Fingerprint = agentFingerprint.Fingerprint
+		return fpRecord, nil
+	}
+
+	// Abort if fingerprint exists but doesn't match (different machine)
+	if fpRecord.Fingerprint != agentFingerprint.Fingerprint {
+		return fpRecord, errors.New("fingerprint mismatch")
+	}
+
+	return fpRecord, nil
+}
+
+// handleMultipleRecordsOrUniversalToken finds a matching fingerprint from multiple records.
+// If no match is found and the token is a universal token, a new system is created.
+func (acr *agentConnectRequest) handleMultipleRecordsOrUniversalToken(fpRecords []ws.FingerprintRecord, agentFingerprint common.FingerprintResponse) (ws.FingerprintRecord, error) {
+	// Return existing record with matching fingerprint if found
+	for i := range fpRecords {
+		if fpRecords[i].Fingerprint == agentFingerprint.Fingerprint {
+			return fpRecords[i], nil
+		}
+	}
+
+	// No matching fingerprint record found, but it's
+	// an active universal token so create a new system
+	if acr.isUniversalToken {
+		return acr.createNewSystemForUniversalToken(agentFingerprint)
+	}
+
+	return ws.FingerprintRecord{}, errors.New("fingerprint mismatch")
+}
+
+// createNewSystemForUniversalToken creates a new system and fingerprint record for a universal token.
+func (acr *agentConnectRequest) createNewSystemForUniversalToken(agentFingerprint common.FingerprintResponse) (ws.FingerprintRecord, error) {
+	var fpRecord ws.FingerprintRecord
+	if !acr.isUniversalToken || acr.userId == "" {
+		return fpRecord, errors.New("invalid token")
+	}
+
+	fpRecord.Token = acr.token
+
+	systemId, err := acr.createSystem(agentFingerprint)
+	if err != nil {
+		return fpRecord, err
+	}
+	fpRecord.SystemId = systemId
+
+	// Set the fingerprint for the new system
+	if err := acr.hub.SetFingerprint(&fpRecord, agentFingerprint.Fingerprint); err != nil {
+		return fpRecord, err
+	}
+
+	// Update the record with the fingerprint that was set
+	fpRecord.Fingerprint = agentFingerprint.Fingerprint
+
+	return fpRecord, nil
+}
+
+// createSystem creates a new system record in the database using details from the agent.
+func (acr *agentConnectRequest) createSystem(agentFingerprint common.FingerprintResponse) (recordId string, err error) {
+	systemsCollection, err := acr.hub.FindCachedCollectionByNameOrId("systems")
+	if err != nil {
+		return "", err
+	}
+	remoteAddr := getRealIP(acr.req)
+	// separate port from address
+	if agentFingerprint.Hostname == "" {
+		agentFingerprint.Hostname = remoteAddr
+	}
+	if agentFingerprint.Port == "" {
+		agentFingerprint.Port = "45876"
+	}
+	// create new record
+	systemRecord := core.NewRecord(systemsCollection)
+	systemRecord.Set("name", agentFingerprint.Hostname)
+	systemRecord.Set("host", remoteAddr)
+	systemRecord.Set("port", agentFingerprint.Port)
+	systemRecord.Set("users", []string{acr.userId})
+
+	return systemRecord.Id, acr.hub.Save(systemRecord)
+}
+
+// SetFingerprint creates or updates a fingerprint record in the database.
+func (h *Hub) SetFingerprint(fpRecord *ws.FingerprintRecord, fingerprint string) (err error) {
+	// // can't use raw query here because it doesn't trigger SSE
+	var record *core.Record
+	switch fpRecord.Id {
+	case "":
+		// create new record for universal token
+		collection, _ := h.FindCachedCollectionByNameOrId("fingerprints")
+		record = core.NewRecord(collection)
+		record.Set("system", fpRecord.SystemId)
+	default:
+		record, err = h.FindRecordById("fingerprints", fpRecord.Id)
+	}
+	if err != nil {
+		return err
+	}
+	record.Set("token", fpRecord.Token)
+	record.Set("fingerprint", fingerprint)
+	return h.SaveNoValidate(record)
+}
+
+// getRealIP extracts the client's real IP address from request headers,
+// checking common proxy headers before falling back to the remote address.
+func getRealIP(r *http.Request) string {
+	if ip := r.Header.Get("CF-Connecting-IP"); ip != "" {
+		return ip
+	}
+	if ip := r.Header.Get("X-Forwarded-For"); ip != "" {
+		// X-Forwarded-For can contain a comma-separated list: "client_ip, proxy1, proxy2"
+		// Take the first one
+		ips := strings.Split(ip, ",")
+		if len(ips) > 0 {
+			return strings.TrimSpace(ips[0])
+		}
+	}
+	// Fallback to RemoteAddr
+	ip, _, err := net.SplitHostPort(r.RemoteAddr)
+	if err != nil {
+		return r.RemoteAddr
+	}
+	return ip
+}

beszel/internal/hub/config.go

@@ -1,222 +0,0 @@
-package hub
-
-import (
-	"beszel/internal/entities/system"
-	"fmt"
-	"log"
-	"os"
-	"path/filepath"
-	"strconv"
-
-	"github.com/labstack/echo/v5"
-	"github.com/pocketbase/dbx"
-	"github.com/pocketbase/pocketbase/apis"
-	"github.com/pocketbase/pocketbase/models"
-	"github.com/spf13/cast"
-	"gopkg.in/yaml.v3"
-)
-
-type Config struct {
-	Systems []SystemConfig `yaml:"systems"`
-}
-
-type SystemConfig struct {
-	Name  string   `yaml:"name"`
-	Host  string   `yaml:"host"`
-	Port  uint16   `yaml:"port"`
-	Users []string `yaml:"users"`
-}
-
-// Syncs systems with the config.yml file
-func (h *Hub) syncSystemsWithConfig() error {
-	configPath := filepath.Join(h.app.DataDir(), "config.yml")
-	configData, err := os.ReadFile(configPath)
-	if err != nil {
-		return nil
-	}
-
-	var config Config
-	err = yaml.Unmarshal(configData, &config)
-	if err != nil {
-		return fmt.Errorf("failed to parse config.yml: %v", err)
-	}
-
-	if len(config.Systems) == 0 {
-		log.Println("No systems defined in config.yml.")
-		return nil
-	}
-
-	var firstUser *models.Record
-
-	// Create a map of email to user ID
-	userEmailToID := make(map[string]string)
-	users, err := h.app.Dao().FindRecordsByExpr("users", dbx.NewExp("id != ''"))
-	if err != nil {
-		return err
-	}
-	if len(users) > 0 {
-		firstUser = users[0]
-		for _, user := range users {
-			userEmailToID[user.GetString("email")] = user.Id
-		}
-	}
-
-	// add default settings for systems if not defined in config
-	for i := range config.Systems {
-		system := &config.Systems[i]
-		if system.Port == 0 {
-			system.Port = 45876
-		}
-		if len(users) > 0 && len(system.Users) == 0 {
-			// default to first user if none are defined
-			system.Users = []string{firstUser.Id}
-		} else {
-			// Convert email addresses to user IDs
-			userIDs := make([]string, 0, len(system.Users))
-			for _, email := range system.Users {
-				if id, ok := userEmailToID[email]; ok {
-					userIDs = append(userIDs, id)
-				} else {
-					log.Printf("User %s not found", email)
-				}
-			}
-			system.Users = userIDs
-		}
-	}
-
-	// Get existing systems
-	existingSystems, err := h.app.Dao().FindRecordsByExpr("systems", dbx.NewExp("id != ''"))
-	if err != nil {
-		return err
-	}
-
-	// Create a map of existing systems for easy lookup
-	existingSystemsMap := make(map[string]*models.Record)
-	for _, system := range existingSystems {
-		key := system.GetString("host") + ":" + system.GetString("port")
-		existingSystemsMap[key] = system
-	}
-
-	// Process systems from config
-	for _, sysConfig := range config.Systems {
-		key := sysConfig.Host + ":" + strconv.Itoa(int(sysConfig.Port))
-		if existingSystem, ok := existingSystemsMap[key]; ok {
-			// Update existing system
-			existingSystem.Set("name", sysConfig.Name)
-			existingSystem.Set("users", sysConfig.Users)
-			existingSystem.Set("port", sysConfig.Port)
-			if err := h.app.Dao().SaveRecord(existingSystem); err != nil {
-				return err
-			}
-			delete(existingSystemsMap, key)
-		} else {
-			// Create new system
-			systemsCollection, err := h.app.Dao().FindCollectionByNameOrId("systems")
-			if err != nil {
-				return fmt.Errorf("failed to find systems collection: %v", err)
-			}
-			newSystem := models.NewRecord(systemsCollection)
-			newSystem.Set("name", sysConfig.Name)
-			newSystem.Set("host", sysConfig.Host)
-			newSystem.Set("port", sysConfig.Port)
-			newSystem.Set("users", sysConfig.Users)
-			newSystem.Set("info", system.Info{})
-			newSystem.Set("status", "pending")
-			if err := h.app.Dao().SaveRecord(newSystem); err != nil {
-				return fmt.Errorf("failed to create new system: %v", err)
-			}
-		}
-	}
-
-	// Delete systems not in config
-	for _, system := range existingSystemsMap {
-		if err := h.app.Dao().DeleteRecord(system); err != nil {
-			return err
-		}
-	}
-
-	log.Println("Systems synced with config.yml")
-	return nil
-}
-
-// Generates content for the config.yml file as a YAML string
-func (h *Hub) generateConfigYAML() (string, error) {
-	// Fetch all systems from the database
-	systems, err := h.app.Dao().FindRecordsByFilter("systems", "id != ''", "name", -1, 0)
-	if err != nil {
-		return "", err
-	}
-
-	// Create a Config struct to hold the data
-	config := Config{
-		Systems: make([]SystemConfig, 0, len(systems)),
-	}
-
-	// Fetch all users at once
-	allUserIDs := make([]string, 0)
-	for _, system := range systems {
-		allUserIDs = append(allUserIDs, system.GetStringSlice("users")...)
-	}
-	userEmailMap, err := h.getUserEmailMap(allUserIDs)
-	if err != nil {
-		return "", err
-	}
-
-	// Populate the Config struct with system data
-	for _, system := range systems {
-		userIDs := system.GetStringSlice("users")
-		userEmails := make([]string, 0, len(userIDs))
-		for _, userID := range userIDs {
-			if email, ok := userEmailMap[userID]; ok {
-				userEmails = append(userEmails, email)
-			}
-		}
-
-		sysConfig := SystemConfig{
-			Name:  system.GetString("name"),
-			Host:  system.GetString("host"),
-			Port:  cast.ToUint16(system.Get("port")),
-			Users: userEmails,
-		}
-		config.Systems = append(config.Systems, sysConfig)
-	}
-
-	// Marshal the Config struct to YAML
-	yamlData, err := yaml.Marshal(&config)
-	if err != nil {
-		return "", err
-	}
-
-	// Add a header to the YAML
-	yamlData = append([]byte("# Values for port and users are optional.\n# Defaults are port 45876 and the first created user.\n\n"), yamlData...)
-
-	return string(yamlData), nil
-}
-
-// New helper function to get a map of user IDs to emails
-func (h *Hub) getUserEmailMap(userIDs []string) (map[string]string, error) {
-	users, err := h.app.Dao().FindRecordsByIds("users", userIDs)
-	if err != nil {
-		return nil, err
-	}
-
-	userEmailMap := make(map[string]string, len(users))
-	for _, user := range users {
-		userEmailMap[user.Id] = user.GetString("email")
-	}
-
-	return userEmailMap, nil
-}
-
-// Returns the current config.yml file as a JSON object
-func (h *Hub) getYamlConfig(c echo.Context) error {
-	requestData := apis.RequestInfo(c)
-	if requestData.AuthRecord == nil || requestData.AuthRecord.GetString("role") != "admin" {
-		return apis.NewForbiddenError("Forbidden", nil)
-	}
-	configContent, err := h.generateConfigYAML()
-	if err != nil {
-		return err
-	}
-	return c.JSON(200, map[string]string{"config": configContent})
-}

beszel/internal/hub/config/config.go

@@ -0,0 +1,291 @@
+// Package config provides functions for syncing systems with the config.yml file
+package config
+
+import (
+	"beszel/internal/entities/system"
+	"fmt"
+	"log"
+	"os"
+	"path/filepath"
+
+	"github.com/google/uuid"
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/apis"
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/spf13/cast"
+	"gopkg.in/yaml.v3"
+)
+
+type config struct {
+	Systems []systemConfig `yaml:"systems"`
+}
+
+type systemConfig struct {
+	Name  string   `yaml:"name"`
+	Host  string   `yaml:"host"`
+	Port  uint16   `yaml:"port,omitempty"`
+	Token string   `yaml:"token,omitempty"`
+	Users []string `yaml:"users"`
+}
+
+// Syncs systems with the config.yml file
+func SyncSystems(e *core.ServeEvent) error {
+	h := e.App
+	configPath := filepath.Join(h.DataDir(), "config.yml")
+	configData, err := os.ReadFile(configPath)
+	if err != nil {
+		return nil
+	}
+
+	var config config
+	err = yaml.Unmarshal(configData, &config)
+	if err != nil {
+		return fmt.Errorf("failed to parse config.yml: %v", err)
+	}
+
+	if len(config.Systems) == 0 {
+		log.Println("No systems defined in config.yml.")
+		return nil
+	}
+
+	var firstUser *core.Record
+
+	// Create a map of email to user ID
+	userEmailToID := make(map[string]string)
+	users, err := h.FindAllRecords("users", dbx.NewExp("id != ''"))
+	if err != nil {
+		return err
+	}
+	if len(users) > 0 {
+		firstUser = users[0]
+		for _, user := range users {
+			userEmailToID[user.GetString("email")] = user.Id
+		}
+	}
+
+	// add default settings for systems if not defined in config
+	for i := range config.Systems {
+		system := &config.Systems[i]
+		if system.Port == 0 {
+			system.Port = 45876
+		}
+		if len(users) > 0 && len(system.Users) == 0 {
+			// default to first user if none are defined
+			system.Users = []string{firstUser.Id}
+		} else {
+			// Convert email addresses to user IDs
+			userIDs := make([]string, 0, len(system.Users))
+			for _, email := range system.Users {
+				if id, ok := userEmailToID[email]; ok {
+					userIDs = append(userIDs, id)
+				} else {
+					log.Printf("User %s not found", email)
+				}
+			}
+			system.Users = userIDs
+		}
+	}
+
+	// Get existing systems
+	existingSystems, err := h.FindAllRecords("systems", dbx.NewExp("id != ''"))
+	if err != nil {
+		return err
+	}
+
+	// Create a map of existing systems
+	existingSystemsMap := make(map[string]*core.Record)
+	for _, system := range existingSystems {
+		key := system.GetString("name") + system.GetString("host") + system.GetString("port")
+		existingSystemsMap[key] = system
+	}
+
+	// Process systems from config
+	for _, sysConfig := range config.Systems {
+		key := sysConfig.Name + sysConfig.Host + cast.ToString(sysConfig.Port)
+		if existingSystem, ok := existingSystemsMap[key]; ok {
+			// Update existing system
+			existingSystem.Set("name", sysConfig.Name)
+			existingSystem.Set("users", sysConfig.Users)
+			existingSystem.Set("port", sysConfig.Port)
+			if err := h.Save(existingSystem); err != nil {
+				return err
+			}
+
+			// Only update token if one is specified in config, otherwise preserve existing token
+			if sysConfig.Token != "" {
+				if err := updateFingerprintToken(h, existingSystem.Id, sysConfig.Token); err != nil {
+					return err
+				}
+			}
+
+			delete(existingSystemsMap, key)
+		} else {
+			// Create new system
+			systemsCollection, err := h.FindCollectionByNameOrId("systems")
+			if err != nil {
+				return fmt.Errorf("failed to find systems collection: %v", err)
+			}
+			newSystem := core.NewRecord(systemsCollection)
+			newSystem.Set("name", sysConfig.Name)
+			newSystem.Set("host", sysConfig.Host)
+			newSystem.Set("port", sysConfig.Port)
+			newSystem.Set("users", sysConfig.Users)
+			newSystem.Set("info", system.Info{})
+			newSystem.Set("status", "pending")
+			if err := h.Save(newSystem); err != nil {
+				return fmt.Errorf("failed to create new system: %v", err)
+			}
+
+			// For new systems, generate token if not provided
+			token := sysConfig.Token
+			if token == "" {
+				token = uuid.New().String()
+			}
+
+			// Create fingerprint record for new system
+			if err := createFingerprintRecord(h, newSystem.Id, token); err != nil {
+				return err
+			}
+		}
+	}
+
+	// Delete systems not in config (and their fingerprint records will cascade delete)
+	for _, system := range existingSystemsMap {
+		if err := h.Delete(system); err != nil {
+			return err
+		}
+	}
+
+	log.Println("Systems synced with config.yml")
+	return nil
+}
+
+// Generates content for the config.yml file as a YAML string
+func generateYAML(h core.App) (string, error) {
+	// Fetch all systems from the database
+	systems, err := h.FindRecordsByFilter("systems", "id != ''", "name", -1, 0)
+	if err != nil {
+		return "", err
+	}
+
+	// Create a Config struct to hold the data
+	config := config{
+		Systems: make([]systemConfig, 0, len(systems)),
+	}
+
+	// Fetch all users at once
+	allUserIDs := make([]string, 0)
+	for _, system := range systems {
+		allUserIDs = append(allUserIDs, system.GetStringSlice("users")...)
+	}
+	userEmailMap, err := getUserEmailMap(h, allUserIDs)
+	if err != nil {
+		return "", err
+	}
+
+	// Fetch all fingerprint records to get tokens
+	type fingerprintData struct {
+		ID     string `db:"id"`
+		System string `db:"system"`
+		Token  string `db:"token"`
+	}
+	var fingerprints []fingerprintData
+	err = h.DB().NewQuery("SELECT id, system, token FROM fingerprints").All(&fingerprints)
+	if err != nil {
+		return "", err
+	}
+
+	// Create a map of system ID to token
+	systemTokenMap := make(map[string]string)
+	for _, fingerprint := range fingerprints {
+		systemTokenMap[fingerprint.System] = fingerprint.Token
+	}
+
+	// Populate the Config struct with system data
+	for _, system := range systems {
+		userIDs := system.GetStringSlice("users")
+		userEmails := make([]string, 0, len(userIDs))
+		for _, userID := range userIDs {
+			if email, ok := userEmailMap[userID]; ok {
+				userEmails = append(userEmails, email)
+			}
+		}
+
+		sysConfig := systemConfig{
+			Name:  system.GetString("name"),
+			Host:  system.GetString("host"),
+			Port:  cast.ToUint16(system.Get("port")),
+			Users: userEmails,
+			Token: systemTokenMap[system.Id],
+		}
+		config.Systems = append(config.Systems, sysConfig)
+	}
+
+	// Marshal the Config struct to YAML
+	yamlData, err := yaml.Marshal(&config)
+	if err != nil {
+		return "", err
+	}
+
+	// Add a header to the YAML
+	yamlData = append([]byte("# Values for port, users, and token are optional.\n# Defaults are port 45876, the first created user, and a generated UUID token.\n\n"), yamlData...)
+
+	return string(yamlData), nil
+}
+
+// New helper function to get a map of user IDs to emails
+func getUserEmailMap(h core.App, userIDs []string) (map[string]string, error) {
+	users, err := h.FindRecordsByIds("users", userIDs)
+	if err != nil {
+		return nil, err
+	}
+
+	userEmailMap := make(map[string]string, len(users))
+	for _, user := range users {
+		userEmailMap[user.Id] = user.GetString("email")
+	}
+
+	return userEmailMap, nil
+}
+
+// Helper function to update or create fingerprint token for an existing system
+func updateFingerprintToken(app core.App, systemID, token string) error {
+	// Try to find existing fingerprint record
+	fingerprint, err := app.FindFirstRecordByFilter("fingerprints", "system = {:system}", dbx.Params{"system": systemID})
+	if err != nil {
+		// If no fingerprint record exists, create one
+		return createFingerprintRecord(app, systemID, token)
+	}
+
+	// Update existing fingerprint record with new token (keep existing fingerprint)
+	fingerprint.Set("token", token)
+	return app.Save(fingerprint)
+}
+
+// Helper function to create a new fingerprint record for a system
+func createFingerprintRecord(app core.App, systemID, token string) error {
+	fingerprintsCollection, err := app.FindCollectionByNameOrId("fingerprints")
+	if err != nil {
+		return fmt.Errorf("failed to find fingerprints collection: %v", err)
+	}
+
+	newFingerprint := core.NewRecord(fingerprintsCollection)
+	newFingerprint.Set("system", systemID)
+	newFingerprint.Set("token", token)
+	newFingerprint.Set("fingerprint", "") // Empty fingerprint, will be set on first connection
+
+	return app.Save(newFingerprint)
+}
+
+// Returns the current config.yml file as a JSON object
+func GetYamlConfig(e *core.RequestEvent) error {
+	info, _ := e.RequestInfo()
+	if info.Auth == nil || info.Auth.GetString("role") != "admin" {
+		return apis.NewForbiddenError("Forbidden", nil)
+	}
+	configContent, err := generateYAML(e.App)
+	if err != nil {
+		return err
+	}
+	return e.JSON(200, map[string]string{"config": configContent})
+}

beszel/internal/hub/config/config_test.go

@@ -0,0 +1,245 @@
+//go:build testing
+// +build testing
+
+package config_test
+
+import (
+	"beszel/internal/hub/config"
+	"beszel/internal/tests"
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"gopkg.in/yaml.v3"
+)
+
+// Config struct for testing (copied from config package since it's not exported)
+type testConfig struct {
+	Systems []testSystemConfig `yaml:"systems"`
+}
+
+type testSystemConfig struct {
+	Name  string   `yaml:"name"`
+	Host  string   `yaml:"host"`
+	Port  uint16   `yaml:"port,omitempty"`
+	Users []string `yaml:"users"`
+	Token string   `yaml:"token,omitempty"`
+}
+
+// Helper function to create a test system for config tests
+// func createConfigTestSystem(app core.App, name, host string, port uint16, userIDs []string) (*core.Record, error) {
+// 	systemCollection, err := app.FindCollectionByNameOrId("systems")
+// 	if err != nil {
+// 		return nil, err
+// 	}
+
+// 	system := core.NewRecord(systemCollection)
+// 	system.Set("name", name)
+// 	system.Set("host", host)
+// 	system.Set("port", port)
+// 	system.Set("users", userIDs)
+// 	system.Set("status", "pending")
+
+// 	return system, app.Save(system)
+// }
+
+// Helper function to create a fingerprint record
+func createConfigTestFingerprint(app core.App, systemID, token, fingerprint string) (*core.Record, error) {
+	fingerprintCollection, err := app.FindCollectionByNameOrId("fingerprints")
+	if err != nil {
+		return nil, err
+	}
+
+	fp := core.NewRecord(fingerprintCollection)
+	fp.Set("system", systemID)
+	fp.Set("token", token)
+	fp.Set("fingerprint", fingerprint)
+
+	return fp, app.Save(fp)
+}
+
+// TestConfigSyncWithTokens tests the config.SyncSystems function with various token scenarios
+func TestConfigSyncWithTokens(t *testing.T) {
+	testHub, err := tests.NewTestHub()
+	require.NoError(t, err)
+	defer testHub.Cleanup()
+
+	// Create test user
+	user, err := tests.CreateUser(testHub.App, "admin@example.com", "testtesttest")
+	require.NoError(t, err)
+
+	testCases := []struct {
+		name        string
+		setupFunc   func() (string, *core.Record, *core.Record) // Returns: existing token, system record, fingerprint record
+		configYAML  string
+		expectToken string // Expected token after sync
+		description string
+	}{
+		{
+			name: "new system with token in config",
+			setupFunc: func() (string, *core.Record, *core.Record) {
+				return "", nil, nil // No existing system
+			},
+			configYAML: `systems:
+  - name: "new-server"
+    host: "new.example.com"
+    port: 45876
+    users:
+      - "admin@example.com"
+    token: "explicit-token-123"`,
+			expectToken: "explicit-token-123",
+			description: "New system should use token from config",
+		},
+		{
+			name: "existing system without token in config (preserve existing)",
+			setupFunc: func() (string, *core.Record, *core.Record) {
+				// Create existing system and fingerprint
+				system, err := tests.CreateRecord(testHub.App, "systems", map[string]any{
+					"name":  "preserve-server",
+					"host":  "preserve.example.com",
+					"port":  45876,
+					"users": []string{user.Id},
+				})
+				require.NoError(t, err)
+
+				fingerprint, err := createConfigTestFingerprint(testHub.App, system.Id, "preserve-token-999", "preserve-fingerprint")
+				require.NoError(t, err)
+
+				return "preserve-token-999", system, fingerprint
+			},
+			configYAML: `systems:
+  - name: "preserve-server"
+    host: "preserve.example.com"
+    port: 45876
+    users:
+      - "admin@example.com"`,
+			expectToken: "preserve-token-999",
+			description: "Existing system should preserve original token when config doesn't specify one",
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Setup test data
+			_, existingSystem, existingFingerprint := tc.setupFunc()
+
+			// Write config file
+			configPath := filepath.Join(testHub.DataDir(), "config.yml")
+			err := os.WriteFile(configPath, []byte(tc.configYAML), 0644)
+			require.NoError(t, err)
+
+			// Create serve event and sync
+			event := &core.ServeEvent{App: testHub.App}
+			err = config.SyncSystems(event)
+			require.NoError(t, err)
+
+			// Parse the config to get the system name for verification
+			var configData testConfig
+			err = yaml.Unmarshal([]byte(tc.configYAML), &configData)
+			require.NoError(t, err)
+			require.Len(t, configData.Systems, 1)
+			systemName := configData.Systems[0].Name
+
+			// Find the system after sync
+			systems, err := testHub.FindRecordsByFilter("systems", "name = {:name}", "", -1, 0, map[string]any{"name": systemName})
+			require.NoError(t, err)
+			require.Len(t, systems, 1)
+			system := systems[0]
+
+			// Find the fingerprint record
+			fingerprints, err := testHub.FindRecordsByFilter("fingerprints", "system = {:system}", "", -1, 0, map[string]any{"system": system.Id})
+			require.NoError(t, err)
+			require.Len(t, fingerprints, 1)
+			fingerprint := fingerprints[0]
+
+			// Verify token
+			actualToken := fingerprint.GetString("token")
+			if tc.expectToken == "" {
+				// For generated tokens, just verify it's not empty and is a valid UUID format
+				assert.NotEmpty(t, actualToken, tc.description)
+				assert.Len(t, actualToken, 36, "Generated token should be UUID format") // UUID length
+			} else {
+				assert.Equal(t, tc.expectToken, actualToken, tc.description)
+			}
+
+			// For existing systems, verify fingerprint is preserved
+			if existingFingerprint != nil {
+				actualFingerprint := fingerprint.GetString("fingerprint")
+				expectedFingerprint := existingFingerprint.GetString("fingerprint")
+				assert.Equal(t, expectedFingerprint, actualFingerprint, "Fingerprint should be preserved")
+			}
+
+			// Cleanup for next test
+			if existingSystem != nil {
+				testHub.Delete(existingSystem)
+			}
+			if existingFingerprint != nil {
+				testHub.Delete(existingFingerprint)
+			}
+			// Clean up the new records
+			testHub.Delete(system)
+			testHub.Delete(fingerprint)
+		})
+	}
+}
+
+// TestConfigMigrationScenario tests the specific migration scenario mentioned in the discussion
+func TestConfigMigrationScenario(t *testing.T) {
+	testHub, err := tests.NewTestHub(t.TempDir())
+	require.NoError(t, err)
+	defer testHub.Cleanup()
+
+	// Create test user
+	user, err := tests.CreateUser(testHub.App, "admin@example.com", "testtesttest")
+	require.NoError(t, err)
+
+	// Simulate migration scenario: system exists with token from migration
+	existingSystem, err := tests.CreateRecord(testHub.App, "systems", map[string]any{
+		"name":  "migrated-server",
+		"host":  "migrated.example.com",
+		"port":  45876,
+		"users": []string{user.Id},
+	})
+	require.NoError(t, err)
+
+	migrationToken := "migration-generated-token-123"
+	existingFingerprint, err := createConfigTestFingerprint(testHub.App, existingSystem.Id, migrationToken, "existing-fingerprint-from-agent")
+	require.NoError(t, err)
+
+	// User exports config BEFORE this update (so no token field in YAML)
+	oldConfigYAML := `systems:
+  - name: "migrated-server"
+    host: "migrated.example.com"
+    port: 45876
+    users:
+      - "admin@example.com"`
+
+	// Write old config file and import
+	configPath := filepath.Join(testHub.DataDir(), "config.yml")
+	err = os.WriteFile(configPath, []byte(oldConfigYAML), 0644)
+	require.NoError(t, err)
+
+	event := &core.ServeEvent{App: testHub.App}
+	err = config.SyncSystems(event)
+	require.NoError(t, err)
+
+	// Verify the original token is preserved
+	updatedFingerprint, err := testHub.FindRecordById("fingerprints", existingFingerprint.Id)
+	require.NoError(t, err)
+
+	actualToken := updatedFingerprint.GetString("token")
+	assert.Equal(t, migrationToken, actualToken, "Migration token should be preserved when config doesn't specify a token")
+
+	// Verify fingerprint is also preserved
+	actualFingerprint := updatedFingerprint.GetString("fingerprint")
+	assert.Equal(t, "existing-fingerprint-from-agent", actualFingerprint, "Existing fingerprint should be preserved")
+
+	// Verify system still exists and is updated correctly
+	updatedSystem, err := testHub.FindRecordById("systems", existingSystem.Id)
+	require.NoError(t, err)
+	assert.Equal(t, "migrated-server", updatedSystem.GetString("name"))
+	assert.Equal(t, "migrated.example.com", updatedSystem.GetString("host"))
+}

beszel/internal/hub/expirymap/expirymap.go

@@ -0,0 +1,104 @@
+package expirymap
+
+import (
+	"reflect"
+	"time"
+
+	"github.com/pocketbase/pocketbase/tools/store"
+)
+
+type val[T any] struct {
+	value   T
+	expires time.Time
+}
+
+type ExpiryMap[T any] struct {
+	store           *store.Store[string, *val[T]]
+	cleanupInterval time.Duration
+}
+
+// New creates a new expiry map with custom cleanup interval
+func New[T any](cleanupInterval time.Duration) *ExpiryMap[T] {
+	m := &ExpiryMap[T]{
+		store:           store.New(map[string]*val[T]{}),
+		cleanupInterval: cleanupInterval,
+	}
+	m.startCleaner()
+	return m
+}
+
+// Set stores a value with the given TTL
+func (m *ExpiryMap[T]) Set(key string, value T, ttl time.Duration) {
+	m.store.Set(key, &val[T]{
+		value:   value,
+		expires: time.Now().Add(ttl),
+	})
+}
+
+// GetOk retrieves a value and checks if it exists and hasn't expired
+// Performs lazy cleanup of expired entries on access
+func (m *ExpiryMap[T]) GetOk(key string) (T, bool) {
+	value, ok := m.store.GetOk(key)
+	if !ok {
+		return *new(T), false
+	}
+
+	// Check if expired and perform lazy cleanup
+	if value.expires.Before(time.Now()) {
+		m.store.Remove(key)
+		return *new(T), false
+	}
+
+	return value.value, true
+}
+
+// GetByValue retrieves a value by value
+func (m *ExpiryMap[T]) GetByValue(val T) (key string, value T, ok bool) {
+	for key, v := range m.store.GetAll() {
+		if reflect.DeepEqual(v.value, val) {
+			// check if expired
+			if v.expires.Before(time.Now()) {
+				m.store.Remove(key)
+				break
+			}
+			return key, v.value, true
+		}
+	}
+	return "", *new(T), false
+}
+
+// Remove explicitly removes a key
+func (m *ExpiryMap[T]) Remove(key string) {
+	m.store.Remove(key)
+}
+
+// RemovebyValue removes a value by value
+func (m *ExpiryMap[T]) RemovebyValue(value T) (T, bool) {
+	for key, val := range m.store.GetAll() {
+		if reflect.DeepEqual(val.value, value) {
+			m.store.Remove(key)
+			return val.value, true
+		}
+	}
+	return *new(T), false
+}
+
+// startCleaner runs the background cleanup process
+func (m *ExpiryMap[T]) startCleaner() {
+	go func() {
+		tick := time.Tick(m.cleanupInterval)
+		for range tick {
+			m.cleanup()
+		}
+	}()
+}
+
+// cleanup removes all expired entries
+func (m *ExpiryMap[T]) cleanup() {
+	now := time.Now()
+	for key, val := range m.store.GetAll() {
+		if val.expires.Before(now) {
+			m.store.Remove(key)
+		}
+	}
+}

beszel/internal/hub/expirymap/expirymap_test.go

@@ -0,0 +1,477 @@
+//go:build testing
+// +build testing
+
+package expirymap
+
+import (
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+// Not using the following methods but are useful for testing
+
+// TESTING: Has checks if a key exists and hasn't expired
+func (m *ExpiryMap[T]) Has(key string) bool {
+	_, ok := m.GetOk(key)
+	return ok
+}
+
+// TESTING: Get retrieves a value, returns zero value if not found or expired
+func (m *ExpiryMap[T]) Get(key string) T {
+	value, _ := m.GetOk(key)
+	return value
+}
+
+// TESTING: Len returns the number of non-expired entries
+func (m *ExpiryMap[T]) Len() int {
+	count := 0
+	now := time.Now()
+	for _, val := range m.store.Values() {
+		if val.expires.After(now) {
+			count++
+		}
+	}
+	return count
+}
+
+func TestExpiryMap_BasicOperations(t *testing.T) {
+	em := New[string](time.Hour)
+
+	// Test Set and GetOk
+	em.Set("key1", "value1", time.Hour)
+	value, ok := em.GetOk("key1")
+	assert.True(t, ok)
+	assert.Equal(t, "value1", value)
+
+	// Test Get
+	value = em.Get("key1")
+	assert.Equal(t, "value1", value)
+
+	// Test Has
+	assert.True(t, em.Has("key1"))
+	assert.False(t, em.Has("nonexistent"))
+
+	// Test Remove
+	em.Remove("key1")
+	assert.False(t, em.Has("key1"))
+}
+
+func TestExpiryMap_Expiration(t *testing.T) {
+	em := New[string](time.Hour)
+
+	// Set a value with very short TTL
+	em.Set("shortlived", "value", time.Millisecond*10)
+
+	// Should exist immediately
+	assert.True(t, em.Has("shortlived"))
+
+	// Wait for expiration
+	time.Sleep(time.Millisecond * 20)
+
+	// Should be expired and automatically cleaned up on access
+	assert.False(t, em.Has("shortlived"))
+	value, ok := em.GetOk("shortlived")
+	assert.False(t, ok)
+	assert.Equal(t, "", value) // zero value for string
+}
+
+func TestExpiryMap_LazyCleanup(t *testing.T) {
+	em := New[int](time.Hour)
+
+	// Set multiple values with short TTL
+	em.Set("key1", 1, time.Millisecond*10)
+	em.Set("key2", 2, time.Millisecond*10)
+	em.Set("key3", 3, time.Hour) // This one won't expire
+
+	// Wait for expiration
+	time.Sleep(time.Millisecond * 20)
+
+	// Access expired keys should trigger lazy cleanup
+	_, ok := em.GetOk("key1")
+	assert.False(t, ok)
+
+	// Non-expired key should still exist
+	value, ok := em.GetOk("key3")
+	assert.True(t, ok)
+	assert.Equal(t, 3, value)
+}
+
+func TestExpiryMap_Len(t *testing.T) {
+	em := New[string](time.Hour)
+
+	// Initially empty
+	assert.Equal(t, 0, em.Len())
+
+	// Add some values
+	em.Set("key1", "value1", time.Hour)
+	em.Set("key2", "value2", time.Hour)
+	em.Set("key3", "value3", time.Millisecond*10) // Will expire soon
+
+	// Should count all initially
+	assert.Equal(t, 3, em.Len())
+
+	// Wait for one to expire
+	time.Sleep(time.Millisecond * 20)
+
+	// Len should reflect only non-expired entries
+	assert.Equal(t, 2, em.Len())
+}
+
+func TestExpiryMap_CustomInterval(t *testing.T) {
+	// Create with very short cleanup interval for testing
+	em := New[string](time.Millisecond * 50)
+
+	// Set a value that expires quickly
+	em.Set("test", "value", time.Millisecond*10)
+
+	// Should exist initially
+	assert.True(t, em.Has("test"))
+
+	// Wait for expiration + cleanup cycle
+	time.Sleep(time.Millisecond * 100)
+
+	// Should be cleaned up by background process
+	// Note: This test might be flaky due to timing, but demonstrates the concept
+	assert.False(t, em.Has("test"))
+}
+
+func TestExpiryMap_GenericTypes(t *testing.T) {
+	// Test with different types
+	t.Run("Int", func(t *testing.T) {
+		em := New[int](time.Hour)
+
+		em.Set("num", 42, time.Hour)
+		value, ok := em.GetOk("num")
+		assert.True(t, ok)
+		assert.Equal(t, 42, value)
+	})
+
+	t.Run("Struct", func(t *testing.T) {
+		type TestStruct struct {
+			Name string
+			Age  int
+		}
+
+		em := New[TestStruct](time.Hour)
+
+		expected := TestStruct{Name: "John", Age: 30}
+		em.Set("person", expected, time.Hour)
+
+		value, ok := em.GetOk("person")
+		assert.True(t, ok)
+		assert.Equal(t, expected, value)
+	})
+
+	t.Run("Pointer", func(t *testing.T) {
+		em := New[*string](time.Hour)
+
+		str := "hello"
+		em.Set("ptr", &str, time.Hour)
+
+		value, ok := em.GetOk("ptr")
+		assert.True(t, ok)
+		require.NotNil(t, value)
+		assert.Equal(t, "hello", *value)
+	})
+}
+
+func TestExpiryMap_ZeroValues(t *testing.T) {
+	em := New[string](time.Hour)
+
+	// Test getting non-existent key returns zero value
+	value := em.Get("nonexistent")
+	assert.Equal(t, "", value)
+
+	// Test getting expired key returns zero value
+	em.Set("expired", "value", time.Millisecond*10)
+	time.Sleep(time.Millisecond * 20)
+
+	value = em.Get("expired")
+	assert.Equal(t, "", value)
+}
+
+func TestExpiryMap_Concurrent(t *testing.T) {
+	em := New[int](time.Hour)
+
+	// Simple concurrent access test
+	done := make(chan bool, 2)
+
+	// Writer goroutine
+	go func() {
+		for i := 0; i < 100; i++ {
+			em.Set("key", i, time.Hour)
+			time.Sleep(time.Microsecond)
+		}
+		done <- true
+	}()
+
+	// Reader goroutine
+	go func() {
+		for i := 0; i < 100; i++ {
+			_ = em.Get("key")
+			time.Sleep(time.Microsecond)
+		}
+		done <- true
+	}()
+
+	// Wait for both to complete
+	<-done
+	<-done
+
+	// Should not panic and should have some value
+	assert.True(t, em.Has("key"))
+}
+
+func TestExpiryMap_GetByValue(t *testing.T) {
+	em := New[string](time.Hour)
+
+	// Test getting by value when value exists
+	em.Set("key1", "value1", time.Hour)
+	em.Set("key2", "value2", time.Hour)
+	em.Set("key3", "value1", time.Hour) // Duplicate value - should return first match
+
+	// Test successful retrieval
+	key, value, ok := em.GetByValue("value1")
+	assert.True(t, ok)
+	assert.Equal(t, "value1", value)
+	assert.Contains(t, []string{"key1", "key3"}, key) // Should be one of the keys with this value
+
+	// Test retrieval of unique value
+	key, value, ok = em.GetByValue("value2")
+	assert.True(t, ok)
+	assert.Equal(t, "value2", value)
+	assert.Equal(t, "key2", key)
+
+	// Test getting non-existent value
+	key, value, ok = em.GetByValue("nonexistent")
+	assert.False(t, ok)
+	assert.Equal(t, "", value) // zero value for string
+	assert.Equal(t, "", key)   // zero value for string
+}
+
+func TestExpiryMap_GetByValue_Expiration(t *testing.T) {
+	em := New[string](time.Hour)
+
+	// Set a value with short TTL
+	em.Set("shortkey", "shortvalue", time.Millisecond*10)
+	em.Set("longkey", "longvalue", time.Hour)
+
+	// Should find the short-lived value initially
+	key, value, ok := em.GetByValue("shortvalue")
+	assert.True(t, ok)
+	assert.Equal(t, "shortvalue", value)
+	assert.Equal(t, "shortkey", key)
+
+	// Wait for expiration
+	time.Sleep(time.Millisecond * 20)
+
+	// Should not find expired value and should trigger lazy cleanup
+	key, value, ok = em.GetByValue("shortvalue")
+	assert.False(t, ok)
+	assert.Equal(t, "", value)
+	assert.Equal(t, "", key)
+
+	// Should still find non-expired value
+	key, value, ok = em.GetByValue("longvalue")
+	assert.True(t, ok)
+	assert.Equal(t, "longvalue", value)
+	assert.Equal(t, "longkey", key)
+}
+
+func TestExpiryMap_GetByValue_GenericTypes(t *testing.T) {
+	t.Run("Int", func(t *testing.T) {
+		em := New[int](time.Hour)
+
+		em.Set("num1", 42, time.Hour)
+		em.Set("num2", 84, time.Hour)
+
+		key, value, ok := em.GetByValue(42)
+		assert.True(t, ok)
+		assert.Equal(t, 42, value)
+		assert.Equal(t, "num1", key)
+
+		key, value, ok = em.GetByValue(99)
+		assert.False(t, ok)
+		assert.Equal(t, 0, value)
+		assert.Equal(t, "", key)
+	})
+
+	t.Run("Struct", func(t *testing.T) {
+		type TestStruct struct {
+			Name string
+			Age  int
+		}
+
+		em := New[TestStruct](time.Hour)
+
+		person1 := TestStruct{Name: "John", Age: 30}
+		person2 := TestStruct{Name: "Jane", Age: 25}
+
+		em.Set("person1", person1, time.Hour)
+		em.Set("person2", person2, time.Hour)
+
+		key, value, ok := em.GetByValue(person1)
+		assert.True(t, ok)
+		assert.Equal(t, person1, value)
+		assert.Equal(t, "person1", key)
+
+		nonexistent := TestStruct{Name: "Bob", Age: 40}
+		key, value, ok = em.GetByValue(nonexistent)
+		assert.False(t, ok)
+		assert.Equal(t, TestStruct{}, value)
+		assert.Equal(t, "", key)
+	})
+}
+
+func TestExpiryMap_RemoveValue(t *testing.T) {
+	em := New[string](time.Hour)
+
+	// Test removing existing value
+	em.Set("key1", "value1", time.Hour)
+	em.Set("key2", "value2", time.Hour)
+	em.Set("key3", "value1", time.Hour) // Duplicate value
+
+	// Remove by value should remove one instance
+	removedValue, ok := em.RemovebyValue("value1")
+	assert.True(t, ok)
+	assert.Equal(t, "value1", removedValue)
+
+	// Should still have the other instance or value2
+	assert.True(t, em.Has("key2")) // value2 should still exist
+
+	// Check if one of the duplicate values was removed
+	// At least one key with "value1" should be gone
+	key1Exists := em.Has("key1")
+	key3Exists := em.Has("key3")
+	assert.False(t, key1Exists && key3Exists) // Both shouldn't exist
+	assert.True(t, key1Exists || key3Exists)  // At least one should be gone
+
+	// Test removing non-existent value
+	removedValue, ok = em.RemovebyValue("nonexistent")
+	assert.False(t, ok)
+	assert.Equal(t, "", removedValue) // zero value for string
+}
+
+func TestExpiryMap_RemoveValue_GenericTypes(t *testing.T) {
+	t.Run("Int", func(t *testing.T) {
+		em := New[int](time.Hour)
+
+		em.Set("num1", 42, time.Hour)
+		em.Set("num2", 84, time.Hour)
+
+		// Remove existing value
+		removedValue, ok := em.RemovebyValue(42)
+		assert.True(t, ok)
+		assert.Equal(t, 42, removedValue)
+		assert.False(t, em.Has("num1"))
+		assert.True(t, em.Has("num2"))
+
+		// Remove non-existent value
+		removedValue, ok = em.RemovebyValue(99)
+		assert.False(t, ok)
+		assert.Equal(t, 0, removedValue)
+	})
+
+	t.Run("Struct", func(t *testing.T) {
+		type TestStruct struct {
+			Name string
+			Age  int
+		}
+
+		em := New[TestStruct](time.Hour)
+
+		person1 := TestStruct{Name: "John", Age: 30}
+		person2 := TestStruct{Name: "Jane", Age: 25}
+
+		em.Set("person1", person1, time.Hour)
+		em.Set("person2", person2, time.Hour)
+
+		// Remove existing struct
+		removedValue, ok := em.RemovebyValue(person1)
+		assert.True(t, ok)
+		assert.Equal(t, person1, removedValue)
+		assert.False(t, em.Has("person1"))
+		assert.True(t, em.Has("person2"))
+
+		// Remove non-existent struct
+		nonexistent := TestStruct{Name: "Bob", Age: 40}
+		removedValue, ok = em.RemovebyValue(nonexistent)
+		assert.False(t, ok)
+		assert.Equal(t, TestStruct{}, removedValue)
+	})
+}
+
+func TestExpiryMap_RemoveValue_WithExpiration(t *testing.T) {
+	em := New[string](time.Hour)
+
+	// Set values with different TTLs
+	em.Set("key1", "value1", time.Millisecond*10) // Will expire
+	em.Set("key2", "value2", time.Hour)           // Won't expire
+	em.Set("key3", "value1", time.Hour)           // Won't expire, duplicate value
+
+	// Wait for first value to expire
+	time.Sleep(time.Millisecond * 20)
+
+	// Try to remove the expired value - should remove one of the "value1" entries
+	removedValue, ok := em.RemovebyValue("value1")
+	assert.True(t, ok)
+	assert.Equal(t, "value1", removedValue)
+
+	// Should still have key2 (different value)
+	assert.True(t, em.Has("key2"))
+
+	// Should have removed one of the "value1" entries (either key1 or key3)
+	// But we can't predict which one due to map iteration order
+	key1Exists := em.Has("key1")
+	key3Exists := em.Has("key3")
+
+	// Exactly one of key1 or key3 should be gone
+	assert.False(t, key1Exists && key3Exists) // Both shouldn't exist
+	assert.True(t, key1Exists || key3Exists)  // At least one should still exist
+}
+
+func TestExpiryMap_ValueOperations_Integration(t *testing.T) {
+	em := New[string](time.Hour)
+
+	// Test integration of GetByValue and RemoveValue
+	em.Set("key1", "shared", time.Hour)
+	em.Set("key2", "unique", time.Hour)
+	em.Set("key3", "shared", time.Hour)
+
+	// Find shared value
+	key, value, ok := em.GetByValue("shared")
+	assert.True(t, ok)
+	assert.Equal(t, "shared", value)
+	assert.Contains(t, []string{"key1", "key3"}, key)
+
+	// Remove shared value
+	removedValue, ok := em.RemovebyValue("shared")
+	assert.True(t, ok)
+	assert.Equal(t, "shared", removedValue)
+
+	// Should still be able to find the other shared value
+	key, value, ok = em.GetByValue("shared")
+	assert.True(t, ok)
+	assert.Equal(t, "shared", value)
+	assert.Contains(t, []string{"key1", "key3"}, key)
+
+	// Remove the other shared value
+	removedValue, ok = em.RemovebyValue("shared")
+	assert.True(t, ok)
+	assert.Equal(t, "shared", removedValue)
+
+	// Should not find shared value anymore
+	key, value, ok = em.GetByValue("shared")
+	assert.False(t, ok)
+	assert.Equal(t, "", value)
+	assert.Equal(t, "", key)
+
+	// Unique value should still exist
+	key, value, ok = em.GetByValue("unique")
+	assert.True(t, ok)
+	assert.Equal(t, "unique", value)
+	assert.Equal(t, "key2", key)
+}

beszel/internal/hub/hub.go

@@ -4,525 +4,348 @@ package hub
 import (
 	"beszel"
 	"beszel/internal/alerts"
-	"beszel/internal/entities/system"
+	"beszel/internal/hub/config"
+	"beszel/internal/hub/systems"
 	"beszel/internal/records"
 	"beszel/internal/users"
 	"beszel/site"
-
-	"context"
 	"crypto/ed25519"
 	"encoding/pem"
 	"fmt"
-	"log"
+	"io/fs"
 	"net/http"
 	"net/http/httputil"
 	"net/url"
 	"os"
+	"path"
 	"strings"
-	"sync"
 	"time"
 
-	"github.com/goccy/go-json"
-	"github.com/labstack/echo/v5"
+	"github.com/google/uuid"
 	"github.com/pocketbase/pocketbase"
 	"github.com/pocketbase/pocketbase/apis"
 	"github.com/pocketbase/pocketbase/core"
-	"github.com/pocketbase/pocketbase/models"
-	"github.com/pocketbase/pocketbase/plugins/migratecmd"
-	"github.com/pocketbase/pocketbase/tools/cron"
 	"golang.org/x/crypto/ssh"
 )
 
 type Hub struct {
-	app               *pocketbase.PocketBase
-	connectionLock    *sync.Mutex
-	systemConnections map[string]*ssh.Client
-	sshClientConfig   *ssh.ClientConfig
-	pubKey            string
-	am                *alerts.AlertManager
-	um                *users.UserManager
-	rm                *records.RecordManager
-	systemStats       *models.Collection
-	containerStats    *models.Collection
+	core.App
+	*alerts.AlertManager
+	um     *users.UserManager
+	rm     *records.RecordManager
+	sm     *systems.SystemManager
+	pubKey string
+	signer ssh.Signer
+	appURL string
 }
 
-func NewHub(app *pocketbase.PocketBase) *Hub {
-	return &Hub{
-		app:               app,
-		connectionLock:    &sync.Mutex{},
-		systemConnections: make(map[string]*ssh.Client),
-		am:                alerts.NewAlertManager(app),
-		um:                users.NewUserManager(app),
-		rm:                records.NewRecordManager(app),
+// NewHub creates a new Hub instance with default configuration
+func NewHub(app core.App) *Hub {
+	hub := &Hub{}
+	hub.App = app
+
+	hub.AlertManager = alerts.NewAlertManager(hub)
+	hub.um = users.NewUserManager(hub)
+	hub.rm = records.NewRecordManager(hub)
+	hub.sm = systems.NewSystemManager(hub)
+	hub.appURL, _ = GetEnv("APP_URL")
+	return hub
+}
+
+// GetEnv retrieves an environment variable with a "BESZEL_HUB_" prefix, or falls back to the unprefixed key.
+func GetEnv(key string) (value string, exists bool) {
+	if value, exists = os.LookupEnv("BESZEL_HUB_" + key); exists {
+		return value, exists
 	}
+	// Fallback to the old unprefixed key
+	return os.LookupEnv(key)
 }
 
-func (h *Hub) Run() {
-	// loosely check if it was executed using "go run"
-	isGoRun := strings.HasPrefix(os.Args[0], os.TempDir())
-
-	// enable auto creation of migration files when making collection changes in the Admin UI
-	migratecmd.MustRegister(h.app, h.app.RootCmd, migratecmd.Config{
-		// (the isGoRun check is to enable it only during development)
-		Automigrate: isGoRun,
-		Dir:         "../../migrations",
-	})
-
-	// initial setup
-	h.app.OnBeforeServe().Add(func(e *core.ServeEvent) error {
-		// create ssh client config
-		err := h.createSSHClientConfig()
-		if err != nil {
-			log.Fatal(err)
-		}
-		// set auth settings
-		usersCollection, err := h.app.Dao().FindCollectionByNameOrId("users")
-		if err != nil {
-			return err
-		}
-		usersAuthOptions := usersCollection.AuthOptions()
-		usersAuthOptions.AllowUsernameAuth = false
-		if os.Getenv("DISABLE_PASSWORD_AUTH") == "true" {
-			usersAuthOptions.AllowEmailAuth = false
-		} else {
-			usersAuthOptions.AllowEmailAuth = true
-		}
-		usersCollection.SetOptions(usersAuthOptions)
-		if err := h.app.Dao().SaveCollection(usersCollection); err != nil {
+func (h *Hub) StartHub() error {
+	h.App.OnServe().BindFunc(func(e *core.ServeEvent) error {
+		// initialize settings / collections
+		if err := h.initialize(e); err != nil {
 			return err
 		}
 		// sync systems with config
-		return h.syncSystemsWithConfig()
-	})
-
-	// serve web ui
-	h.app.OnBeforeServe().Add(func(e *core.ServeEvent) error {
-		switch isGoRun {
-		case true:
-			proxy := httputil.NewSingleHostReverseProxy(&url.URL{
-				Scheme: "http",
-				Host:   "localhost:5173",
-			})
-			e.Router.Any("/*", echo.WrapHandler(proxy))
-		default:
-			csp, cspExists := os.LookupEnv("CSP")
-			e.Router.Any("/*", func(c echo.Context) error {
-				if cspExists {
-					c.Response().Header().Del("X-Frame-Options")
-					c.Response().Header().Set("Content-Security-Policy", csp)
-				}
-				indexFallback := !strings.HasPrefix(c.Request().URL.Path, "/static/")
-				return apis.StaticDirectoryHandler(site.Dist, indexFallback)(c)
-			})
+		if err := config.SyncSystems(e); err != nil {
+			return err
 		}
-		return nil
-	})
-
-	// set up scheduled jobs / ticker for system updates
-	h.app.OnBeforeServe().Add(func(e *core.ServeEvent) error {
-		// 15 second ticker for system updates
-		go h.startSystemUpdateTicker()
-		// set up cron jobs
-		scheduler := cron.New()
-		// delete old records once every hour
-		scheduler.MustAdd("delete old records", "8 * * * *", h.rm.DeleteOldRecords)
-		// create longer records every 10 minutes
-		scheduler.MustAdd("create longer records", "*/10 * * * *", func() {
-			if systemStats, containerStats, err := h.getCollections(); err == nil {
-				h.rm.CreateLongerRecords([]*models.Collection{systemStats, containerStats})
-			}
-		})
-		scheduler.Start()
-		return nil
-	})
-
-	// custom api routes
-	h.app.OnBeforeServe().Add(func(e *core.ServeEvent) error {
-		// returns public key
-		e.Router.GET("/api/beszel/getkey", func(c echo.Context) error {
-			requestData := apis.RequestInfo(c)
-			if requestData.AuthRecord == nil {
-				return apis.NewForbiddenError("Forbidden", nil)
-			}
-			return c.JSON(http.StatusOK, map[string]string{"key": h.pubKey, "v": beszel.Version})
-		})
-		// check if first time setup on login page
-		e.Router.GET("/api/beszel/first-run", func(c echo.Context) error {
-			adminNum, err := h.app.Dao().TotalAdmins()
-			if err != nil {
-				return err
-			}
-			return c.JSON(http.StatusOK, map[string]bool{"firstRun": adminNum == 0})
-		})
-		// send test notification
-		e.Router.GET("/api/beszel/send-test-notification", h.am.SendTestNotification)
-		// API endpoint to get config.yml content
-		e.Router.GET("/api/beszel/config-yaml", h.getYamlConfig)
-		return nil
-	})
-
-	// system creation defaults
-	h.app.OnModelBeforeCreate("systems").Add(func(e *core.ModelEvent) error {
-		record := e.Model.(*models.Record)
-		record.Set("info", system.Info{})
-		record.Set("status", "pending")
-		return nil
-	})
-
-	// immediately create connection for new systems
-	h.app.OnModelAfterCreate("systems").Add(func(e *core.ModelEvent) error {
-		go h.updateSystem(e.Model.(*models.Record))
-		return nil
+		// register api routes
+		if err := h.registerApiRoutes(e); err != nil {
+			return err
+		}
+		// register cron jobs
+		if err := h.registerCronJobs(e); err != nil {
+			return err
+		}
+		// start server
+		if err := h.startServer(e); err != nil {
+			return err
+		}
+		// start system updates
+		if err := h.sm.Initialize(); err != nil {
+			return err
+		}
+		return e.Next()
 	})
 
+	// TODO: move to users package
 	// handle default values for user / user_settings creation
-	h.app.OnModelBeforeCreate("users").Add(h.um.InitializeUserRole)
-	h.app.OnModelBeforeCreate("user_settings").Add(h.um.InitializeUserSettings)
+	h.App.OnRecordCreate("users").BindFunc(h.um.InitializeUserRole)
+	h.App.OnRecordCreate("user_settings").BindFunc(h.um.InitializeUserSettings)
 
-	// empty info for systems that are paused
-	h.app.OnModelBeforeUpdate("systems").Add(func(e *core.ModelEvent) error {
-		if e.Model.(*models.Record).GetString("status") == "paused" {
-			e.Model.(*models.Record).Set("info", system.Info{})
-		}
-		return nil
-	})
-
-	// do things after a systems record is updated
-	h.app.OnModelAfterUpdate("systems").Add(func(e *core.ModelEvent) error {
-		newRecord := e.Model.(*models.Record)
-		oldRecord := newRecord.OriginalCopy()
-		newStatus := newRecord.GetString("status")
-
-		// if system is disconnected and connection exists, remove it
-		if newStatus == "down" || newStatus == "paused" {
-			h.deleteSystemConnection(newRecord)
-		}
-
-		// if system is set to pending (unpause), try to connect immediately
-		if newStatus == "pending" {
-			go h.updateSystem(newRecord)
-		} else {
-			h.am.HandleStatusAlerts(newStatus, oldRecord)
-
-		}
-
-		return nil
-	})
-
-	// do things after a systems record is deleted
-	h.app.OnModelAfterDelete("systems").Add(func(e *core.ModelEvent) error {
-		// if system connection exists, close it
-		h.deleteSystemConnection(e.Model.(*models.Record))
-		return nil
-	})
-
-	if err := h.app.Start(); err != nil {
-		log.Fatal(err)
-	}
-}
-
-func (h *Hub) startSystemUpdateTicker() {
-	ticker := time.NewTicker(15 * time.Second)
-	for range ticker.C {
-		h.updateSystems()
-	}
-}
-
-func (h *Hub) updateSystems() {
-	records, err := h.app.Dao().FindRecordsByFilter(
-		"2hz5ncl8tizk5nx",    // systems collection
-		"status != 'paused'", // filter
-		"updated",            // sort
-		-1,                   // limit
-		0,                    // offset
-	)
-	// log.Println("records", len(records))
-	if err != nil || len(records) == 0 {
-		// h.app.Logger().Error("Failed to query systems")
-		return
-	}
-	fiftySecondsAgo := time.Now().UTC().Add(-50 * time.Second)
-	batchSize := len(records)/4 + 1
-	done := 0
-	for _, record := range records {
-		// break if batch size reached or if the system was updated less than 50 seconds ago
-		if done >= batchSize || record.GetDateTime("updated").Time().After(fiftySecondsAgo) {
-			break
-		}
-		// don't increment for down systems to avoid them jamming the queue
-		// because they're always first when sorted by least recently updated
-		if record.GetString("status") != "down" {
-			done++
-		}
-		go h.updateSystem(record)
-	}
-}
-
-func (h *Hub) updateSystem(record *models.Record) {
-	var client *ssh.Client
-	var err error
-
-	// check if system connection data exists
-	if _, ok := h.systemConnections[record.Id]; ok {
-		client = h.systemConnections[record.Id]
-	} else {
-		// create system connection
-		client, err = h.createSystemConnection(record)
+	if pb, ok := h.App.(*pocketbase.PocketBase); ok {
+		// log.Println("Starting pocketbase")
+		err := pb.Start()
 		if err != nil {
-			if record.GetString("status") != "down" {
-				h.app.Logger().Error("Failed to connect:", "err", err.Error(), "system", record.GetString("host"), "port", record.GetString("port"))
-				h.updateSystemStatus(record, "down")
-			}
-			return
+			return err
 		}
-		h.connectionLock.Lock()
-		h.systemConnections[record.Id] = client
-		h.connectionLock.Unlock()
-	}
-	// get system stats from agent
-	var systemData system.CombinedData
-	if err := h.requestJsonFromAgent(client, &systemData); err != nil {
-		if err.Error() == "bad client" {
-			// if previous connection was closed, try again
-			h.app.Logger().Error("Existing SSH connection closed. Retrying...", "host", record.GetString("host"), "port", record.GetString("port"))
-			h.deleteSystemConnection(record)
-			h.updateSystem(record)
-			return
-		}
-		h.app.Logger().Error("Failed to get system stats: ", "err", err.Error())
-		h.updateSystemStatus(record, "down")
-		return
-	}
-	// update system record
-	dao := h.app.Dao()
-	record.Set("status", "up")
-	record.Set("info", systemData.Info)
-	if err := dao.SaveRecord(record); err != nil {
-		h.app.Logger().Error("Failed to update record: ", "err", err.Error())
-	}
-	// add system_stats and container_stats records
-	if systemStats, containerStats, err := h.getCollections(); err != nil {
-		h.app.Logger().Error("Failed to get collections: ", "err", err.Error())
-	} else {
-		// add new system_stats record
-		systemStatsRecord := models.NewRecord(systemStats)
-		systemStatsRecord.Set("system", record.Id)
-		systemStatsRecord.Set("stats", systemData.Stats)
-		systemStatsRecord.Set("type", "1m")
-		if err := dao.SaveRecord(systemStatsRecord); err != nil {
-			h.app.Logger().Error("Failed to save record: ", "err", err.Error())
-		}
-		// add new container_stats record
-		if len(systemData.Containers) > 0 {
-			containerStatsRecord := models.NewRecord(containerStats)
-			containerStatsRecord.Set("system", record.Id)
-			containerStatsRecord.Set("stats", systemData.Containers)
-			containerStatsRecord.Set("type", "1m")
-			if err := dao.SaveRecord(containerStatsRecord); err != nil {
-				h.app.Logger().Error("Failed to save record: ", "err", err.Error())
-			}
-		}
-	}
-
-	// system info alerts (todo: extra fs alerts)
-	if err := h.am.HandleSystemAlerts(record, systemData.Info, systemData.Stats.Temperatures, systemData.Stats.ExtraFs); err != nil {
-		h.app.Logger().Error("System alerts error", "err", err.Error())
-	}
-}
-
-// return system_stats and container_stats collections
-func (h *Hub) getCollections() (*models.Collection, *models.Collection, error) {
-	if h.systemStats == nil {
-		systemStats, err := h.app.Dao().FindCollectionByNameOrId("system_stats")
-		if err != nil {
-			return nil, nil, err
-		}
-		h.systemStats = systemStats
-	}
-	if h.containerStats == nil {
-		containerStats, err := h.app.Dao().FindCollectionByNameOrId("container_stats")
-		if err != nil {
-			return nil, nil, err
-		}
-		h.containerStats = containerStats
-	}
-	return h.systemStats, h.containerStats, nil
-}
-
-// set system to specified status and save record
-func (h *Hub) updateSystemStatus(record *models.Record, status string) {
-	if record.GetString("status") != status {
-		record.Set("status", status)
-		if err := h.app.Dao().SaveRecord(record); err != nil {
-			h.app.Logger().Error("Failed to update record: ", "err", err.Error())
-		}
-	}
-}
-
-func (h *Hub) deleteSystemConnection(record *models.Record) {
-	if _, ok := h.systemConnections[record.Id]; ok {
-		if h.systemConnections[record.Id] != nil {
-			h.systemConnections[record.Id].Close()
-		}
-		h.connectionLock.Lock()
-		defer h.connectionLock.Unlock()
-		delete(h.systemConnections, record.Id)
-	}
-}
-
-func (h *Hub) createSystemConnection(record *models.Record) (*ssh.Client, error) {
-	client, err := ssh.Dial("tcp", fmt.Sprintf("%s:%s", record.GetString("host"), record.GetString("port")), h.sshClientConfig)
-	if err != nil {
-		return nil, err
-	}
-	return client, nil
-}
-
-func (h *Hub) createSSHClientConfig() error {
-	key, err := h.getSSHKey()
-	if err != nil {
-		h.app.Logger().Error("Failed to get SSH key: ", "err", err.Error())
-		return err
-	}
-
-	// Create the Signer for this private key.
-	signer, err := ssh.ParsePrivateKey(key)
-	if err != nil {
-		return err
-	}
-
-	h.sshClientConfig = &ssh.ClientConfig{
-		User: "u",
-		Auth: []ssh.AuthMethod{
-			ssh.PublicKeys(signer),
-		},
-		HostKeyCallback: ssh.InsecureIgnoreHostKey(),
-		Timeout:         5 * time.Second,
-	}
-	return nil
-}
-
-// Fetches system stats from the agent and decodes the json data into the provided struct
-func (h *Hub) requestJsonFromAgent(client *ssh.Client, systemData *system.CombinedData) error {
-	session, err := newSessionWithTimeout(client, 5*time.Second)
-	if err != nil {
-		return fmt.Errorf("bad client")
-	}
-	defer session.Close()
-
-	stdout, err := session.StdoutPipe()
-	if err != nil {
-		return err
-	}
-
-	if err := session.Shell(); err != nil {
-		return err
-	}
-
-	if err := json.NewDecoder(stdout).Decode(systemData); err != nil {
-		return err
-	}
-
-	// wait for the session to complete
-	if err := session.Wait(); err != nil {
-		return err
 	}
 
 	return nil
 }
 
-// Adds timeout to SSH session creation to avoid hanging in case of network issues
-func newSessionWithTimeout(client *ssh.Client, timeout time.Duration) (*ssh.Session, error) {
-	ctx, cancel := context.WithTimeout(context.Background(), timeout)
-	defer cancel()
-
-	// use goroutine to create the session
-	sessionChan := make(chan *ssh.Session, 1)
-	errChan := make(chan error, 1)
-	go func() {
-		if session, err := client.NewSession(); err != nil {
-			errChan <- err
-		} else {
-			sessionChan <- session
-		}
-	}()
-
-	select {
-	case session := <-sessionChan:
-		return session, nil
-	case err := <-errChan:
-		return nil, err
-	case <-ctx.Done():
-		return nil, fmt.Errorf("session creation timed out")
+// initialize sets up initial configuration (collections, settings, etc.)
+func (h *Hub) initialize(e *core.ServeEvent) error {
+	// set general settings
+	settings := e.App.Settings()
+	// batch requests (for global alerts)
+	settings.Batch.Enabled = true
+	// set URL if BASE_URL env is set
+	if h.appURL != "" {
+		settings.Meta.AppURL = h.appURL
 	}
+	if err := e.App.Save(settings); err != nil {
+		return err
+	}
+	// set auth settings
+	usersCollection, err := e.App.FindCollectionByNameOrId("users")
+	if err != nil {
+		return err
+	}
+	// disable email auth if DISABLE_PASSWORD_AUTH env var is set
+	disablePasswordAuth, _ := GetEnv("DISABLE_PASSWORD_AUTH")
+	usersCollection.PasswordAuth.Enabled = disablePasswordAuth != "true"
+	usersCollection.PasswordAuth.IdentityFields = []string{"email"}
+	// disable oauth if no providers are configured (todo: remove this in post 0.9.0 release)
+	if usersCollection.OAuth2.Enabled {
+		usersCollection.OAuth2.Enabled = len(usersCollection.OAuth2.Providers) > 0
+	}
+	// allow oauth user creation if USER_CREATION is set
+	if userCreation, _ := GetEnv("USER_CREATION"); userCreation == "true" {
+		cr := "@request.context = 'oauth2'"
+		usersCollection.CreateRule = &cr
+	} else {
+		usersCollection.CreateRule = nil
+	}
+	if err := e.App.Save(usersCollection); err != nil {
+		return err
+	}
+	// allow all users to access systems if SHARE_ALL_SYSTEMS is set
+	systemsCollection, err := e.App.FindCachedCollectionByNameOrId("systems")
+	if err != nil {
+		return err
+	}
+	shareAllSystems, _ := GetEnv("SHARE_ALL_SYSTEMS")
+	systemsReadRule := "@request.auth.id != \"\""
+	if shareAllSystems != "true" {
+		// default is to only show systems that the user id is assigned to
+		systemsReadRule += " && users.id ?= @request.auth.id"
+	}
+	updateDeleteRule := systemsReadRule + " && @request.auth.role != \"readonly\""
+	systemsCollection.ListRule = &systemsReadRule
+	systemsCollection.ViewRule = &systemsReadRule
+	systemsCollection.UpdateRule = &updateDeleteRule
+	systemsCollection.DeleteRule = &updateDeleteRule
+	if err := e.App.Save(systemsCollection); err != nil {
+		return err
+	}
+	return nil
 }
 
-func (h *Hub) getSSHKey() ([]byte, error) {
-	dataDir := h.app.DataDir()
+// startServer sets up the server for Beszel
+func (h *Hub) startServer(se *core.ServeEvent) error {
+	// TODO: exclude dev server from production binary
+	switch h.IsDev() {
+	case true:
+		proxy := httputil.NewSingleHostReverseProxy(&url.URL{
+			Scheme: "http",
+			Host:   "localhost:5173",
+		})
+		se.Router.GET("/{path...}", func(e *core.RequestEvent) error {
+			proxy.ServeHTTP(e.Response, e.Request)
+			return nil
+		})
+	default:
+		// parse app url
+		parsedURL, err := url.Parse(h.appURL)
+		if err != nil {
+			return err
+		}
+		// fix base paths in html if using subpath
+		basePath := strings.TrimSuffix(parsedURL.Path, "/") + "/"
+		indexFile, _ := fs.ReadFile(site.DistDirFS, "index.html")
+		indexContent := strings.ReplaceAll(string(indexFile), "./", basePath)
+		indexContent = strings.Replace(indexContent, "{{V}}", beszel.Version, 1)
+		indexContent = strings.Replace(indexContent, "{{HUB_URL}}", h.appURL, 1)
+		// set up static asset serving
+		staticPaths := [2]string{"/static/", "/assets/"}
+		serveStatic := apis.Static(site.DistDirFS, false)
+		// get CSP configuration
+		csp, cspExists := GetEnv("CSP")
+		// add route
+		se.Router.GET("/{path...}", func(e *core.RequestEvent) error {
+			// serve static assets if path is in staticPaths
+			for i := range staticPaths {
+				if strings.Contains(e.Request.URL.Path, staticPaths[i]) {
+					e.Response.Header().Set("Cache-Control", "public, max-age=2592000")
+					return serveStatic(e)
+				}
+			}
+			if cspExists {
+				e.Response.Header().Del("X-Frame-Options")
+				e.Response.Header().Set("Content-Security-Policy", csp)
+			}
+			return e.HTML(http.StatusOK, indexContent)
+		})
+	}
+	return nil
+}
+
+// registerCronJobs sets up scheduled tasks
+func (h *Hub) registerCronJobs(_ *core.ServeEvent) error {
+	// delete old system_stats and alerts_history records once every hour
+	h.Cron().MustAdd("delete old records", "8 * * * *", h.rm.DeleteOldRecords)
+	// create longer records every 10 minutes
+	h.Cron().MustAdd("create longer records", "*/10 * * * *", h.rm.CreateLongerRecords)
+	return nil
+}
+
+// custom api routes
+func (h *Hub) registerApiRoutes(se *core.ServeEvent) error {
+	// returns public key and version
+	se.Router.GET("/api/beszel/getkey", func(e *core.RequestEvent) error {
+		info, _ := e.RequestInfo()
+		if info.Auth == nil {
+			return apis.NewForbiddenError("Forbidden", nil)
+		}
+		return e.JSON(http.StatusOK, map[string]string{"key": h.pubKey, "v": beszel.Version})
+	})
+	// check if first time setup on login page
+	se.Router.GET("/api/beszel/first-run", func(e *core.RequestEvent) error {
+		total, err := h.CountRecords("users")
+		return e.JSON(http.StatusOK, map[string]bool{"firstRun": err == nil && total == 0})
+	})
+	// send test notification
+	se.Router.GET("/api/beszel/send-test-notification", h.SendTestNotification)
+	// API endpoint to get config.yml content
+	se.Router.GET("/api/beszel/config-yaml", config.GetYamlConfig)
+	// handle agent websocket connection
+	se.Router.GET("/api/beszel/agent-connect", h.handleAgentConnect)
+	// get or create universal tokens
+	se.Router.GET("/api/beszel/universal-token", h.getUniversalToken)
+	// create first user endpoint only needed if no users exist
+	if totalUsers, _ := h.CountRecords("users"); totalUsers == 0 {
+		se.Router.POST("/api/beszel/create-user", h.um.CreateFirstUser)
+	}
+	return nil
+}
+
+// Handler for universal token API endpoint (create, read, delete)
+func (h *Hub) getUniversalToken(e *core.RequestEvent) error {
+	info, err := e.RequestInfo()
+	if err != nil || info.Auth == nil {
+		return apis.NewForbiddenError("Forbidden", nil)
+	}
+
+	tokenMap := universalTokenMap.GetMap()
+	userID := info.Auth.Id
+	query := e.Request.URL.Query()
+	token := query.Get("token")
+	tokenSet := token != ""
+
+	if !tokenSet {
+		// return existing token if it exists
+		if token, _, ok := tokenMap.GetByValue(userID); ok {
+			return e.JSON(http.StatusOK, map[string]any{"token": token, "active": true})
+		}
+		// if no token is provided, generate a new one
+		token = uuid.New().String()
+	}
+	response := map[string]any{"token": token}
+
+	switch query.Get("enable") {
+	case "1":
+		tokenMap.Set(token, userID, time.Hour)
+	case "0":
+		tokenMap.RemovebyValue(userID)
+	}
+	_, response["active"] = tokenMap.GetOk(token)
+	return e.JSON(http.StatusOK, response)
+}
+
+// generates key pair if it doesn't exist and returns signer
+func (h *Hub) GetSSHKey(dataDir string) (ssh.Signer, error) {
+	if h.signer != nil {
+		return h.signer, nil
+	}
+
+	if dataDir == "" {
+		dataDir = h.DataDir()
+	}
+
+	privateKeyPath := path.Join(dataDir, "id_ed25519")
+
 	// check if the key pair already exists
-	existingKey, err := os.ReadFile(dataDir + "/id_ed25519")
+	existingKey, err := os.ReadFile(privateKeyPath)
 	if err == nil {
-		if pubKey, err := os.ReadFile(h.app.DataDir() + "/id_ed25519.pub"); err == nil {
-			h.pubKey = strings.TrimSuffix(string(pubKey), "\n")
+		private, err := ssh.ParsePrivateKey(existingKey)
+		if err != nil {
+			return nil, fmt.Errorf("failed to parse private key: %s", err)
 		}
-		// return existing private key
-		return existingKey, nil
+		pubKeyBytes := ssh.MarshalAuthorizedKey(private.PublicKey())
+		h.pubKey = strings.TrimSuffix(string(pubKeyBytes), "\n")
+		return private, nil
+	} else if !os.IsNotExist(err) {
+		// File exists but couldn't be read for some other reason
+		return nil, fmt.Errorf("failed to read %s: %w", privateKeyPath, err)
 	}
 
 	// Generate the Ed25519 key pair
-	pubKey, privKey, err := ed25519.GenerateKey(nil)
+	_, privKey, err := ed25519.GenerateKey(nil)
 	if err != nil {
-		// h.app.Logger().Error("Error generating key pair:", "err", err.Error())
 		return nil, err
 	}
-
-	// Get the private key in OpenSSH format
-	privKeyBytes, err := ssh.MarshalPrivateKey(privKey, "")
-	if err != nil {
-		// h.app.Logger().Error("Error marshaling private key:", "err", err.Error())
-		return nil, err
-	}
-
-	// Save the private key to a file
-	privateFile, err := os.Create(dataDir + "/id_ed25519")
-	if err != nil {
-		// h.app.Logger().Error("Error creating private key file:", "err", err.Error())
-		return nil, err
-	}
-	defer privateFile.Close()
-
-	if err := pem.Encode(privateFile, privKeyBytes); err != nil {
-		// h.app.Logger().Error("Error writing private key to file:", "err", err.Error())
-		return nil, err
-	}
-
-	// Generate the public key in OpenSSH format
-	publicKey, err := ssh.NewPublicKey(pubKey)
+	privKeyPem, err := ssh.MarshalPrivateKey(privKey, "")
 	if err != nil {
 		return nil, err
 	}
 
-	pubKeyBytes := ssh.MarshalAuthorizedKey(publicKey)
+	if err := os.WriteFile(privateKeyPath, pem.EncodeToMemory(privKeyPem), 0600); err != nil {
+		return nil, fmt.Errorf("failed to write private key to %q: err: %w", privateKeyPath, err)
+	}
+
+	// These are fine to ignore the errors on, as we've literally just created a crypto.PublicKey | crypto.Signer
+	sshPrivate, _ := ssh.NewSignerFromSigner(privKey)
+	pubKeyBytes := ssh.MarshalAuthorizedKey(sshPrivate.PublicKey())
 	h.pubKey = strings.TrimSuffix(string(pubKeyBytes), "\n")
 
-	// Save the public key to a file
-	publicFile, err := os.Create(dataDir + "/id_ed25519.pub")
-	if err != nil {
-		return nil, err
-	}
-	defer publicFile.Close()
+	h.Logger().Info("ed25519 key pair generated successfully.")
+	h.Logger().Info("Saved to: " + privateKeyPath)
 
-	if _, err := publicFile.Write(pubKeyBytes); err != nil {
-		return nil, err
-	}
-
-	h.app.Logger().Info("ed25519 SSH key pair generated successfully.")
-	h.app.Logger().Info("Private key saved to: " + dataDir + "/id_ed25519")
-	h.app.Logger().Info("Public key saved to: " + dataDir + "/id_ed25519.pub")
-
-	existingKey, err = os.ReadFile(dataDir + "/id_ed25519")
-	if err == nil {
-		return existingKey, nil
-	}
-	return nil, err
+	return sshPrivate, err
+}
+
+// MakeLink formats a link with the app URL and path segments.
+// Only path segments should be provided.
+func (h *Hub) MakeLink(parts ...string) string {
+	base := strings.TrimSuffix(h.Settings().Meta.AppURL, "/")
+	for _, part := range parts {
+		if part == "" {
+			continue
+		}
+		base = fmt.Sprintf("%s/%s", base, url.PathEscape(part))
+	}
+	return base
 }

beszel/internal/hub/hub_test.go

@@ -0,0 +1,256 @@
+//go:build testing
+// +build testing
+
+package hub_test
+
+import (
+	"beszel/internal/tests"
+	"testing"
+
+	"crypto/ed25519"
+	"encoding/pem"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"golang.org/x/crypto/ssh"
+)
+
+func getTestHub(t testing.TB) *tests.TestHub {
+	hub, _ := tests.NewTestHub(t.TempDir())
+	return hub
+}
+
+func TestMakeLink(t *testing.T) {
+	hub := getTestHub(t)
+
+	tests := []struct {
+		name     string
+		appURL   string
+		parts    []string
+		expected string
+	}{
+		{
+			name:     "no parts, no trailing slash in AppURL",
+			appURL:   "http://localhost:8090",
+			parts:    []string{},
+			expected: "http://localhost:8090",
+		},
+		{
+			name:     "no parts, with trailing slash in AppURL",
+			appURL:   "http://localhost:8090/",
+			parts:    []string{},
+			expected: "http://localhost:8090", // TrimSuffix should handle the trailing slash
+		},
+		{
+			name:     "one part",
+			appURL:   "http://example.com",
+			parts:    []string{"one"},
+			expected: "http://example.com/one",
+		},
+		{
+			name:     "multiple parts",
+			appURL:   "http://example.com",
+			parts:    []string{"alpha", "beta", "gamma"},
+			expected: "http://example.com/alpha/beta/gamma",
+		},
+		{
+			name:     "parts with spaces needing escaping",
+			appURL:   "http://example.com",
+			parts:    []string{"path with spaces", "another part"},
+			expected: "http://example.com/path%20with%20spaces/another%20part",
+		},
+		{
+			name:     "parts with slashes needing escaping",
+			appURL:   "http://example.com",
+			parts:    []string{"a/b", "c"},
+			expected: "http://example.com/a%2Fb/c", // url.PathEscape escapes '/'
+		},
+		{
+			name:     "AppURL with subpath, no trailing slash",
+			appURL:   "http://localhost/sub",
+			parts:    []string{"resource"},
+			expected: "http://localhost/sub/resource",
+		},
+		{
+			name:     "AppURL with subpath, with trailing slash",
+			appURL:   "http://localhost/sub/",
+			parts:    []string{"item"},
+			expected: "http://localhost/sub/item",
+		},
+		{
+			name:     "empty parts in the middle",
+			appURL:   "http://localhost",
+			parts:    []string{"first", "", "third"},
+			expected: "http://localhost/first/third",
+		},
+		{
+			name:     "leading and trailing empty parts",
+			appURL:   "http://localhost",
+			parts:    []string{"", "path", ""},
+			expected: "http://localhost/path",
+		},
+		{
+			name:     "parts with various special characters",
+			appURL:   "https://test.dev/",
+			parts:    []string{"p@th?", "key=value&"},
+			expected: "https://test.dev/p@th%3F/key=value&",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			// Store original app URL and restore it after the test
+			originalAppURL := hub.Settings().Meta.AppURL
+			hub.Settings().Meta.AppURL = tt.appURL
+			defer func() { hub.Settings().Meta.AppURL = originalAppURL }()
+
+			got := hub.MakeLink(tt.parts...)
+			assert.Equal(t, tt.expected, got, "MakeLink generated URL does not match expected")
+		})
+	}
+}
+
+func TestGetSSHKey(t *testing.T) {
+	hub := getTestHub(t)
+
+	// Test Case 1: Key generation (no existing key)
+	t.Run("KeyGeneration", func(t *testing.T) {
+		tempDir := t.TempDir()
+
+		// Ensure pubKey is initially empty or different to ensure GetSSHKey sets it
+		hub.SetPubkey("")
+
+		signer, err := hub.GetSSHKey(tempDir)
+		assert.NoError(t, err, "GetSSHKey should not error when generating a new key")
+		assert.NotNil(t, signer, "GetSSHKey should return a non-nil signer")
+
+		// Check if private key file was created
+		privateKeyPath := filepath.Join(tempDir, "id_ed25519")
+		info, err := os.Stat(privateKeyPath)
+		assert.NoError(t, err, "Private key file should be created")
+		assert.False(t, info.IsDir(), "Private key path should be a file, not a directory")
+
+		// Check if h.pubKey was set
+		assert.NotEmpty(t, hub.GetPubkey(), "h.pubKey should be set after key generation")
+		assert.True(t, strings.HasPrefix(hub.GetPubkey(), "ssh-ed25519 "), "h.pubKey should start with 'ssh-ed25519 '")
+
+		// Verify the generated private key is parsable
+		keyData, err := os.ReadFile(privateKeyPath)
+		require.NoError(t, err)
+		_, err = ssh.ParsePrivateKey(keyData)
+		assert.NoError(t, err, "Generated private key should be parsable by ssh.ParsePrivateKey")
+	})
+
+	// Test Case 2: Existing key
+	t.Run("ExistingKey", func(t *testing.T) {
+		tempDir := t.TempDir()
+
+		// Manually create a valid key pair for the test
+		rawPubKey, rawPrivKey, err := ed25519.GenerateKey(nil)
+		require.NoError(t, err, "Failed to generate raw ed25519 key pair for pre-existing key test")
+
+		// Marshal the private key into OpenSSH PEM format
+		pemBlock, err := ssh.MarshalPrivateKey(rawPrivKey, "")
+		require.NoError(t, err, "Failed to marshal private key to PEM block for pre-existing key test")
+
+		privateKeyBytes := pem.EncodeToMemory(pemBlock)
+		require.NotNil(t, privateKeyBytes, "PEM encoded private key bytes should not be nil")
+
+		privateKeyPath := filepath.Join(tempDir, "id_ed25519")
+		err = os.WriteFile(privateKeyPath, privateKeyBytes, 0600)
+		require.NoError(t, err, "Failed to write pre-existing private key")
+
+		// Determine the expected public key string
+		sshPubKey, err := ssh.NewPublicKey(rawPubKey)
+		require.NoError(t, err)
+		expectedPubKeyStr := strings.TrimSpace(string(ssh.MarshalAuthorizedKey(sshPubKey)))
+
+		// Reset h.pubKey to ensure it's set by GetSSHKey from the file
+		hub.SetPubkey("")
+
+		signer, err := hub.GetSSHKey(tempDir)
+		assert.NoError(t, err, "GetSSHKey should not error when reading an existing key")
+		assert.NotNil(t, signer, "GetSSHKey should return a non-nil signer for an existing key")
+
+		// Check if h.pubKey was set correctly to the public key from the file
+		assert.Equal(t, expectedPubKeyStr, hub.GetPubkey(), "h.pubKey should match the existing public key")
+
+		// Verify the signer's public key matches the original public key
+		signerPubKey := signer.PublicKey()
+		marshaledSignerPubKey := strings.TrimSpace(string(ssh.MarshalAuthorizedKey(signerPubKey)))
+		assert.Equal(t, expectedPubKeyStr, marshaledSignerPubKey, "Signer's public key should match the existing public key")
+	})
+
+	// Test Case 3: Error cases
+	t.Run("ErrorCases", func(t *testing.T) {
+		tests := []struct {
+			name       string
+			setupFunc  func(dir string) error
+			errorCheck func(t *testing.T, err error)
+		}{
+			{
+				name: "CorruptedKey",
+				setupFunc: func(dir string) error {
+					return os.WriteFile(filepath.Join(dir, "id_ed25519"), []byte("this is not a valid SSH key"), 0600)
+				},
+				errorCheck: func(t *testing.T, err error) {
+					assert.Error(t, err)
+					assert.Contains(t, err.Error(), "ssh: no key found")
+				},
+			},
+			{
+				name: "PermissionDenied",
+				setupFunc: func(dir string) error {
+					// Create the key file
+					keyPath := filepath.Join(dir, "id_ed25519")
+					if err := os.WriteFile(keyPath, []byte("dummy content"), 0600); err != nil {
+						return err
+					}
+					// Make it read-only (can't be opened for writing in case a new key needs to be written)
+					return os.Chmod(keyPath, 0400)
+				},
+				errorCheck: func(t *testing.T, err error) {
+					// On read-only key, the parser will attempt to parse it and fail with "ssh: no key found"
+					assert.Error(t, err)
+				},
+			},
+			{
+				name: "EmptyFile",
+				setupFunc: func(dir string) error {
+					// Create an empty file
+					return os.WriteFile(filepath.Join(dir, "id_ed25519"), []byte{}, 0600)
+				},
+				errorCheck: func(t *testing.T, err error) {
+					assert.Error(t, err)
+					// The error from attempting to parse an empty file
+					assert.Contains(t, err.Error(), "ssh: no key found")
+				},
+			},
+		}
+
+		for _, tc := range tests {
+			t.Run(tc.name, func(t *testing.T) {
+				tempDir := t.TempDir()
+
+				// Setup the test case
+				err := tc.setupFunc(tempDir)
+				require.NoError(t, err, "Setup failed")
+
+				// Reset h.pubKey before each test case
+				hub.SetPubkey("")
+
+				// Attempt to get SSH key
+				_, err = hub.GetSSHKey(tempDir)
+
+				// Verify the error
+				tc.errorCheck(t, err)
+
+				// Check that pubKey was not set in error cases
+				assert.Empty(t, hub.GetPubkey(), "h.pubKey should not be set if there was an error")
+			})
+		}
+	})
+}

beszel/internal/hub/hub_test_helpers.go

@@ -0,0 +1,21 @@
+//go:build testing
+// +build testing
+
+package hub
+
+import "beszel/internal/hub/systems"
+
+// TESTING ONLY: GetSystemManager returns the system manager
+func (h *Hub) GetSystemManager() *systems.SystemManager {
+	return h.sm
+}
+
+// TESTING ONLY: GetPubkey returns the public key
+func (h *Hub) GetPubkey() string {
+	return h.pubKey
+}
+
+// TESTING ONLY: SetPubkey sets the public key
+func (h *Hub) SetPubkey(pubkey string) {
+	h.pubKey = pubkey
+}

beszel/internal/hub/systems/system.go

@@ -0,0 +1,387 @@
+package systems
+
+import (
+	"beszel"
+	"beszel/internal/entities/system"
+	"beszel/internal/hub/ws"
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"math/rand"
+	"net"
+	"strings"
+	"time"
+
+	"github.com/blang/semver"
+	"github.com/fxamacker/cbor/v2"
+	"github.com/pocketbase/pocketbase/core"
+	"golang.org/x/crypto/ssh"
+)
+
+type System struct {
+	Id           string               `db:"id"`
+	Host         string               `db:"host"`
+	Port         string               `db:"port"`
+	Status       string               `db:"status"`
+	manager      *SystemManager       // Manager that this system belongs to
+	client       *ssh.Client          // SSH client for fetching data
+	data         *system.CombinedData // system data from agent
+	ctx          context.Context      // Context for stopping the updater
+	cancel       context.CancelFunc   // Stops and removes system from updater
+	WsConn       *ws.WsConn           // Handler for agent WebSocket connection
+	agentVersion semver.Version       // Agent version
+	updateTicker *time.Ticker         // Ticker for updating the system
+}
+
+func (sm *SystemManager) NewSystem(systemId string) *System {
+	system := &System{
+		Id:   systemId,
+		data: &system.CombinedData{},
+	}
+	system.ctx, system.cancel = system.getContext()
+	return system
+}
+
+// StartUpdater starts the system updater.
+// It first fetches the data from the agent then updates the records.
+// If the data is not found or the system is down, it sets the system down.
+func (sys *System) StartUpdater() {
+	// Channel that can be used to set the system down. Currently only used to
+	// allow a short delay for reconnection after websocket connection is closed.
+	var downChan chan struct{}
+
+	// Add random jitter to first WebSocket connection to prevent
+	// clustering if all agents are started at the same time.
+	// SSH connections during hub startup are already staggered.
+	var jitter <-chan time.Time
+	if sys.WsConn != nil {
+		jitter = getJitter()
+		// use the websocket connection's down channel to set the system down
+		downChan = sys.WsConn.DownChan
+	} else {
+		// if the system does not have a websocket connection, wait before updating
+		// to allow the agent to connect via websocket (makes sure fingerprint is set).
+		time.Sleep(11 * time.Second)
+	}
+
+	// update immediately if system is not paused (only for ws connections)
+	// we'll wait a minute before connecting via SSH to prioritize ws connections
+	if sys.Status != paused && sys.ctx.Err() == nil {
+		if err := sys.update(); err != nil {
+			_ = sys.setDown(err)
+		}
+	}
+
+	sys.updateTicker = time.NewTicker(time.Duration(interval) * time.Millisecond)
+	// Go 1.23+ will automatically stop the ticker when the system is garbage collected, however we seem to need this or testing/synctest will block even if calling runtime.GC()
+	defer sys.updateTicker.Stop()
+
+	for {
+		select {
+		case <-sys.ctx.Done():
+			return
+		case <-sys.updateTicker.C:
+			if err := sys.update(); err != nil {
+				_ = sys.setDown(err)
+			}
+		case <-downChan:
+			sys.WsConn = nil
+			downChan = nil
+			_ = sys.setDown(nil)
+		case <-jitter:
+			sys.updateTicker.Reset(time.Duration(interval) * time.Millisecond)
+			if err := sys.update(); err != nil {
+				_ = sys.setDown(err)
+			}
+		}
+	}
+}
+
+// update updates the system data and records.
+func (sys *System) update() error {
+	if sys.Status == paused {
+		sys.handlePaused()
+		return nil
+	}
+	data, err := sys.fetchDataFromAgent()
+	if err == nil {
+		_, err = sys.createRecords(data)
+	}
+	return err
+}
+
+func (sys *System) handlePaused() {
+	if sys.WsConn == nil {
+		// if the system is paused and there's no websocket connection, remove the system
+		_ = sys.manager.RemoveSystem(sys.Id)
+	} else {
+		// Send a ping to the agent to keep the connection alive if the system is paused
+		if err := sys.WsConn.Ping(); err != nil {
+			sys.manager.hub.Logger().Warn("Failed to ping agent", "system", sys.Id, "err", err)
+			_ = sys.manager.RemoveSystem(sys.Id)
+		}
+	}
+}
+
+// createRecords updates the system record and adds system_stats and container_stats records
+func (sys *System) createRecords(data *system.CombinedData) (*core.Record, error) {
+	systemRecord, err := sys.getRecord()
+	if err != nil {
+		return nil, err
+	}
+	hub := sys.manager.hub
+	// add system_stats and container_stats records
+	systemStatsCollection, err := hub.FindCachedCollectionByNameOrId("system_stats")
+	if err != nil {
+		return nil, err
+	}
+
+	systemStatsRecord := core.NewRecord(systemStatsCollection)
+	systemStatsRecord.Set("system", systemRecord.Id)
+	systemStatsRecord.Set("stats", data.Stats)
+	systemStatsRecord.Set("type", "1m")
+	if err := hub.SaveNoValidate(systemStatsRecord); err != nil {
+		return nil, err
+	}
+	// add new container_stats record
+	if len(data.Containers) > 0 {
+		containerStatsCollection, err := hub.FindCachedCollectionByNameOrId("container_stats")
+		if err != nil {
+			return nil, err
+		}
+		containerStatsRecord := core.NewRecord(containerStatsCollection)
+		containerStatsRecord.Set("system", systemRecord.Id)
+		containerStatsRecord.Set("stats", data.Containers)
+		containerStatsRecord.Set("type", "1m")
+		if err := hub.SaveNoValidate(containerStatsRecord); err != nil {
+			return nil, err
+		}
+	}
+	// update system record (do this last because it triggers alerts and we need above records to be inserted first)
+	systemRecord.Set("status", up)
+
+	systemRecord.Set("info", data.Info)
+	if err := hub.SaveNoValidate(systemRecord); err != nil {
+		return nil, err
+	}
+	return systemRecord, nil
+}
+
+// getRecord retrieves the system record from the database.
+// If the record is not found, it removes the system from the manager.
+func (sys *System) getRecord() (*core.Record, error) {
+	record, err := sys.manager.hub.FindRecordById("systems", sys.Id)
+	if err != nil || record == nil {
+		_ = sys.manager.RemoveSystem(sys.Id)
+		return nil, err
+	}
+	return record, nil
+}
+
+// setDown marks a system as down in the database.
+// It takes the original error that caused the system to go down and returns any error
+// encountered during the process of updating the system status.
+func (sys *System) setDown(originalError error) error {
+	if sys.Status == down || sys.Status == paused {
+		return nil
+	}
+	record, err := sys.getRecord()
+	if err != nil {
+		return err
+	}
+	if originalError != nil {
+		sys.manager.hub.Logger().Error("System down", "system", record.GetString("name"), "err", originalError)
+	}
+	record.Set("status", down)
+	return sys.manager.hub.SaveNoValidate(record)
+}
+
+func (sys *System) getContext() (context.Context, context.CancelFunc) {
+	if sys.ctx == nil {
+		sys.ctx, sys.cancel = context.WithCancel(context.Background())
+	}
+	return sys.ctx, sys.cancel
+}
+
+// fetchDataFromAgent attempts to fetch data from the agent,
+// prioritizing WebSocket if available.
+func (sys *System) fetchDataFromAgent() (*system.CombinedData, error) {
+	if sys.data == nil {
+		sys.data = &system.CombinedData{}
+	}
+
+	if sys.WsConn != nil && sys.WsConn.IsConnected() {
+		wsData, err := sys.fetchDataViaWebSocket()
+		if err == nil {
+			return wsData, nil
+		}
+		// close the WebSocket connection if error and try SSH
+		sys.closeWebSocketConnection()
+	}
+
+	sshData, err := sys.fetchDataViaSSH()
+	if err != nil {
+		return nil, err
+	}
+	return sshData, nil
+}
+
+func (sys *System) fetchDataViaWebSocket() (*system.CombinedData, error) {
+	if sys.WsConn == nil || !sys.WsConn.IsConnected() {
+		return nil, errors.New("no websocket connection")
+	}
+	err := sys.WsConn.RequestSystemData(sys.data)
+	if err != nil {
+		return nil, err
+	}
+	return sys.data, nil
+}
+
+// fetchDataViaSSH handles fetching data using SSH.
+// This function encapsulates the original SSH logic.
+// It updates sys.data directly upon successful fetch.
+func (sys *System) fetchDataViaSSH() (*system.CombinedData, error) {
+	maxRetries := 1
+	for attempt := 0; attempt <= maxRetries; attempt++ {
+		if sys.client == nil || sys.Status == down {
+			if err := sys.createSSHClient(); err != nil {
+				return nil, err
+			}
+		}
+
+		session, err := sys.createSessionWithTimeout(4 * time.Second)
+		if err != nil {
+			if attempt >= maxRetries {
+				return nil, err
+			}
+			sys.manager.hub.Logger().Warn("Session closed. Retrying...", "host", sys.Host, "port", sys.Port, "err", err)
+			sys.closeSSHConnection()
+			// Reset format detection on connection failure - agent might have been upgraded
+			continue
+		}
+		defer session.Close()
+
+		stdout, err := session.StdoutPipe()
+		if err != nil {
+			return nil, err
+		}
+		if err := session.Shell(); err != nil {
+			return nil, err
+		}
+
+		*sys.data = system.CombinedData{}
+
+		if sys.agentVersion.GTE(beszel.MinVersionCbor) {
+			err = cbor.NewDecoder(stdout).Decode(sys.data)
+		} else {
+			err = json.NewDecoder(stdout).Decode(sys.data)
+		}
+
+		if err != nil {
+			sys.closeSSHConnection()
+			if attempt < maxRetries {
+				continue
+			}
+			return nil, err
+		}
+
+		// wait for the session to complete
+		if err := session.Wait(); err != nil {
+			return nil, err
+		}
+
+		return sys.data, nil
+	}
+
+	// this should never be reached due to the return in the loop
+	return nil, fmt.Errorf("failed to fetch data")
+}
+
+// createSSHClient creates a new SSH client for the system
+func (s *System) createSSHClient() error {
+	if s.manager.sshConfig == nil {
+		if err := s.manager.createSSHClientConfig(); err != nil {
+			return err
+		}
+	}
+	network := "tcp"
+	host := s.Host
+	if strings.HasPrefix(host, "/") {
+		network = "unix"
+	} else {
+		host = net.JoinHostPort(host, s.Port)
+	}
+	var err error
+	s.client, err = ssh.Dial(network, host, s.manager.sshConfig)
+	if err != nil {
+		return err
+	}
+	s.agentVersion, _ = extractAgentVersion(string(s.client.Conn.ServerVersion()))
+	return nil
+}
+
+// createSessionWithTimeout creates a new SSH session with a timeout to avoid hanging
+// in case of network issues
+func (sys *System) createSessionWithTimeout(timeout time.Duration) (*ssh.Session, error) {
+	if sys.client == nil {
+		return nil, fmt.Errorf("client not initialized")
+	}
+
+	ctx, cancel := context.WithTimeout(sys.ctx, timeout)
+	defer cancel()
+
+	sessionChan := make(chan *ssh.Session, 1)
+	errChan := make(chan error, 1)
+
+	go func() {
+		if session, err := sys.client.NewSession(); err != nil {
+			errChan <- err
+		} else {
+			sessionChan <- session
+		}
+	}()
+
+	select {
+	case session := <-sessionChan:
+		return session, nil
+	case err := <-errChan:
+		return nil, err
+	case <-ctx.Done():
+		return nil, fmt.Errorf("timeout")
+	}
+}
+
+// closeSSHConnection closes the SSH connection but keeps the system in the manager
+func (sys *System) closeSSHConnection() {
+	if sys.client != nil {
+		sys.client.Close()
+		sys.client = nil
+	}
+}
+
+// closeWebSocketConnection closes the WebSocket connection but keeps the system in the manager
+// to allow updating via SSH. It will be removed if the WS connection is re-established.
+// The system will be set as down a few seconds later if the connection is not re-established.
+func (sys *System) closeWebSocketConnection() {
+	if sys.WsConn != nil {
+		sys.WsConn.Close(nil)
+	}
+}
+
+// extractAgentVersion extracts the beszel version from SSH server version string
+func extractAgentVersion(versionString string) (semver.Version, error) {
+	_, after, _ := strings.Cut(versionString, "_")
+	return semver.Parse(after)
+}
+
+// getJitter returns a channel that will be triggered after a random delay
+// between 40% and 90% of the interval.
+// This is used to stagger the initial WebSocket connections to prevent clustering.
+func getJitter() <-chan time.Time {
+	minPercent := 40
+	maxPercent := 90
+	jitterRange := maxPercent - minPercent
+	msDelay := (interval * minPercent / 100) + rand.Intn(interval*jitterRange/100)
+	return time.After(time.Duration(msDelay) * time.Millisecond)
+}

beszel/internal/hub/systems/system_manager.go

@@ -0,0 +1,346 @@
+package systems
+
+import (
+	"beszel"
+	"beszel/internal/common"
+	"beszel/internal/entities/system"
+	"beszel/internal/hub/ws"
+	"errors"
+	"fmt"
+	"time"
+
+	"github.com/blang/semver"
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/store"
+	"golang.org/x/crypto/ssh"
+)
+
+// System status constants
+const (
+	up      string = "up"      // System is online and responding
+	down    string = "down"    // System is offline or not responding
+	paused  string = "paused"  // System monitoring is paused
+	pending string = "pending" // System is waiting on initial connection result
+
+	// interval is the default update interval in milliseconds (60 seconds)
+	interval int = 60_000
+	// interval int = 10_000 // Debug interval for faster updates
+
+	// sessionTimeout is the maximum time to wait for SSH connections
+	sessionTimeout = 4 * time.Second
+)
+
+var (
+	// errSystemExists is returned when attempting to add a system that already exists
+	errSystemExists = errors.New("system exists")
+)
+
+// SystemManager manages a collection of monitored systems and their connections.
+// It handles system lifecycle, status updates, and maintains both SSH and WebSocket connections.
+type SystemManager struct {
+	hub       hubLike                       // Hub interface for database and alert operations
+	systems   *store.Store[string, *System] // Thread-safe store of active systems
+	sshConfig *ssh.ClientConfig             // SSH client configuration for system connections
+}
+
+// hubLike defines the interface requirements for the hub dependency.
+// It extends core.App with system-specific functionality.
+type hubLike interface {
+	core.App
+	GetSSHKey(dataDir string) (ssh.Signer, error)
+	HandleSystemAlerts(systemRecord *core.Record, data *system.CombinedData) error
+	HandleStatusAlerts(status string, systemRecord *core.Record) error
+}
+
+// NewSystemManager creates a new SystemManager instance with the provided hub.
+// The hub must implement the hubLike interface to provide database and alert functionality.
+func NewSystemManager(hub hubLike) *SystemManager {
+	return &SystemManager{
+		systems: store.New(map[string]*System{}),
+		hub:     hub,
+	}
+}
+
+// Initialize sets up the system manager by binding event hooks and starting existing systems.
+// It configures SSH client settings and begins monitoring all non-paused systems from the database.
+// Systems are started with staggered delays to prevent overwhelming the hub during startup.
+func (sm *SystemManager) Initialize() error {
+	sm.bindEventHooks()
+
+	// Initialize SSH client configuration
+	err := sm.createSSHClientConfig()
+	if err != nil {
+		return err
+	}
+
+	// Load existing systems from database (excluding paused ones)
+	var systems []*System
+	err = sm.hub.DB().NewQuery("SELECT id, host, port, status FROM systems WHERE status != 'paused'").All(&systems)
+	if err != nil || len(systems) == 0 {
+		return err
+	}
+
+	// Start systems in background with staggered timing
+	go func() {
+		// Calculate staggered delay between system starts (max 2 seconds per system)
+		delta := interval / max(1, len(systems))
+		delta = min(delta, 2_000)
+		sleepTime := time.Duration(delta) * time.Millisecond
+
+		for _, system := range systems {
+			time.Sleep(sleepTime)
+			_ = sm.AddSystem(system)
+		}
+	}()
+	return nil
+}
+
+// bindEventHooks registers event handlers for system and fingerprint record changes.
+// These hooks ensure the system manager stays synchronized with database changes.
+func (sm *SystemManager) bindEventHooks() {
+	sm.hub.OnRecordCreate("systems").BindFunc(sm.onRecordCreate)
+	sm.hub.OnRecordAfterCreateSuccess("systems").BindFunc(sm.onRecordAfterCreateSuccess)
+	sm.hub.OnRecordUpdate("systems").BindFunc(sm.onRecordUpdate)
+	sm.hub.OnRecordAfterUpdateSuccess("systems").BindFunc(sm.onRecordAfterUpdateSuccess)
+	sm.hub.OnRecordAfterDeleteSuccess("systems").BindFunc(sm.onRecordAfterDeleteSuccess)
+	sm.hub.OnRecordAfterUpdateSuccess("fingerprints").BindFunc(sm.onTokenRotated)
+}
+
+// onTokenRotated handles fingerprint token rotation events.
+// When a system's authentication token is rotated, any existing WebSocket connection
+// must be closed to force re-authentication with the new token.
+func (sm *SystemManager) onTokenRotated(e *core.RecordEvent) error {
+	systemID := e.Record.GetString("system")
+	system, ok := sm.systems.GetOk(systemID)
+	if !ok {
+		return e.Next()
+	}
+	// No need to close connection if not connected via websocket
+	if system.WsConn == nil {
+		return e.Next()
+	}
+	system.setDown(nil)
+	sm.RemoveSystem(systemID)
+	return e.Next()
+}
+
+// onRecordCreate is called before a new system record is committed to the database.
+// It initializes the record with default values: empty info and pending status.
+func (sm *SystemManager) onRecordCreate(e *core.RecordEvent) error {
+	e.Record.Set("info", system.Info{})
+	e.Record.Set("status", pending)
+	return e.Next()
+}
+
+// onRecordAfterCreateSuccess is called after a new system record is successfully created.
+// It adds the new system to the manager to begin monitoring.
+func (sm *SystemManager) onRecordAfterCreateSuccess(e *core.RecordEvent) error {
+	if err := sm.AddRecord(e.Record, nil); err != nil {
+		e.App.Logger().Error("Error adding record", "err", err)
+	}
+	return e.Next()
+}
+
+// onRecordUpdate is called before a system record is updated in the database.
+// It clears system info when the status is changed to paused.
+func (sm *SystemManager) onRecordUpdate(e *core.RecordEvent) error {
+	if e.Record.GetString("status") == paused {
+		e.Record.Set("info", system.Info{})
+	}
+	return e.Next()
+}
+
+// onRecordAfterUpdateSuccess handles system record updates after they're committed to the database.
+// It manages system lifecycle based on status changes and triggers appropriate alerts.
+// Status transitions are handled as follows:
+// - paused: Closes SSH connection and deactivates alerts
+// - pending: Starts monitoring (reuses WebSocket if available)
+// - up: Triggers system alerts
+// - down: Triggers status change alerts
+func (sm *SystemManager) onRecordAfterUpdateSuccess(e *core.RecordEvent) error {
+	newStatus := e.Record.GetString("status")
+	prevStatus := pending
+	system, ok := sm.systems.GetOk(e.Record.Id)
+	if ok {
+		prevStatus = system.Status
+		system.Status = newStatus
+	}
+
+	switch newStatus {
+	case paused:
+		if ok {
+			// Pause monitoring but keep system in manager for potential resume
+			system.closeSSHConnection()
+		}
+		_ = deactivateAlerts(e.App, e.Record.Id)
+		return e.Next()
+	case pending:
+		// Resume monitoring, preferring existing WebSocket connection
+		if ok && system.WsConn != nil {
+			go system.update()
+			return e.Next()
+		}
+		// Start new monitoring session
+		if err := sm.AddRecord(e.Record, nil); err != nil {
+			e.App.Logger().Error("Error adding record", "err", err)
+		}
+		_ = deactivateAlerts(e.App, e.Record.Id)
+		return e.Next()
+	}
+
+	// Handle systems not in manager
+	if !ok {
+		return sm.AddRecord(e.Record, nil)
+	}
+
+	// Trigger system alerts when system comes online
+	if newStatus == up {
+		if err := sm.hub.HandleSystemAlerts(e.Record, system.data); err != nil {
+			e.App.Logger().Error("Error handling system alerts", "err", err)
+		}
+	}
+
+	// Trigger status change alerts for up/down transitions
+	if (newStatus == down && prevStatus == up) || (newStatus == up && prevStatus == down) {
+		if err := sm.hub.HandleStatusAlerts(newStatus, e.Record); err != nil {
+			e.App.Logger().Error("Error handling status alerts", "err", err)
+		}
+	}
+	return e.Next()
+}
+
+// onRecordAfterDeleteSuccess is called after a system record is successfully deleted.
+// It removes the system from the manager and cleans up all associated resources.
+func (sm *SystemManager) onRecordAfterDeleteSuccess(e *core.RecordEvent) error {
+	sm.RemoveSystem(e.Record.Id)
+	return e.Next()
+}
+
+// AddSystem adds a system to the manager and starts monitoring it.
+// It validates required fields, initializes the system context, and starts the update goroutine.
+// Returns error if a system with the same ID already exists.
+func (sm *SystemManager) AddSystem(sys *System) error {
+	if sm.systems.Has(sys.Id) {
+		return errSystemExists
+	}
+	if sys.Id == "" || sys.Host == "" {
+		return errors.New("system missing required fields")
+	}
+
+	// Initialize system for monitoring
+	sys.manager = sm
+	sys.ctx, sys.cancel = sys.getContext()
+	sys.data = &system.CombinedData{}
+	sm.systems.Set(sys.Id, sys)
+
+	// Start monitoring in background
+	go sys.StartUpdater()
+	return nil
+}
+
+// RemoveSystem removes a system from the manager and cleans up all associated resources.
+// It cancels the system's context, closes all connections, and removes it from the store.
+// Returns an error if the system is not found.
+func (sm *SystemManager) RemoveSystem(systemID string) error {
+	system, ok := sm.systems.GetOk(systemID)
+	if !ok {
+		return errors.New("system not found")
+	}
+
+	// Stop the update goroutine
+	if system.cancel != nil {
+		system.cancel()
+	}
+
+	// Clean up all connections
+	system.closeSSHConnection()
+	system.closeWebSocketConnection()
+	sm.systems.Remove(systemID)
+	return nil
+}
+
+// AddRecord creates a System instance from a database record and adds it to the manager.
+// If a system with the same ID already exists, it's removed first to ensure clean state.
+// If no system instance is provided, a new one is created.
+// This method is typically called when systems are created or their status changes to pending.
+func (sm *SystemManager) AddRecord(record *core.Record, system *System) (err error) {
+	// Remove existing system to ensure clean state
+	if sm.systems.Has(record.Id) {
+		_ = sm.RemoveSystem(record.Id)
+	}
+
+	// Create new system if none provided
+	if system == nil {
+		system = sm.NewSystem(record.Id)
+	}
+
+	// Populate system from record
+	system.Status = record.GetString("status")
+	system.Host = record.GetString("host")
+	system.Port = record.GetString("port")
+
+	return sm.AddSystem(system)
+}
+
+// AddWebSocketSystem creates and adds a system with an established WebSocket connection.
+// This method is called when an agent connects via WebSocket with valid authentication.
+// The system is immediately added to monitoring with the provided connection and version info.
+func (sm *SystemManager) AddWebSocketSystem(systemId string, agentVersion semver.Version, wsConn *ws.WsConn) error {
+	systemRecord, err := sm.hub.FindRecordById("systems", systemId)
+	if err != nil {
+		return err
+	}
+
+	system := sm.NewSystem(systemId)
+	system.WsConn = wsConn
+	system.agentVersion = agentVersion
+
+	if err := sm.AddRecord(systemRecord, system); err != nil {
+		return err
+	}
+	return nil
+}
+
+// createSSHClientConfig initializes the SSH client configuration for connecting to an agent's server
+func (sm *SystemManager) createSSHClientConfig() error {
+	privateKey, err := sm.hub.GetSSHKey("")
+	if err != nil {
+		return err
+	}
+
+	sm.sshConfig = &ssh.ClientConfig{
+		User: "u",
+		Auth: []ssh.AuthMethod{
+			ssh.PublicKeys(privateKey),
+		},
+		Config: ssh.Config{
+			Ciphers:      common.DefaultCiphers,
+			KeyExchanges: common.DefaultKeyExchanges,
+			MACs:         common.DefaultMACs,
+		},
+		HostKeyCallback: ssh.InsecureIgnoreHostKey(),
+		ClientVersion:   fmt.Sprintf("SSH-2.0-%s_%s", beszel.AppName, beszel.Version),
+		Timeout:         sessionTimeout,
+	}
+	return nil
+}
+
+// deactivateAlerts finds all triggered alerts for a system and sets them to inactive.
+// This is called when a system is paused or goes offline to prevent continued alerts.
+func deactivateAlerts(app core.App, systemID string) error {
+	// Note: Direct SQL updates don't trigger SSE, so we use the PocketBase API
+	// _, err := app.DB().NewQuery(fmt.Sprintf("UPDATE alerts SET triggered = false WHERE system = '%s'", systemID)).Execute()
+
+	alerts, err := app.FindRecordsByFilter("alerts", fmt.Sprintf("system = '%s' && triggered = 1", systemID), "", -1, 0)
+	if err != nil {
+		return err
+	}
+
+	for _, alert := range alerts {
+		alert.Set("triggered", false)
+		if err := app.SaveNoValidate(alert); err != nil {
+			return err
+		}
+	}
+	return nil
+}

beszel/internal/hub/systems/systems_test.go

@@ -0,0 +1,419 @@
+//go:build testing
+// +build testing
+
+package systems_test
+
+import (
+	"beszel/internal/entities/container"
+	"beszel/internal/entities/system"
+	"beszel/internal/hub/systems"
+	"beszel/internal/tests"
+	"fmt"
+	"sync"
+	"testing"
+	"testing/synctest"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+func TestSystemManagerNew(t *testing.T) {
+	hub, err := tests.NewTestHub(t.TempDir())
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer hub.Cleanup()
+	sm := hub.GetSystemManager()
+
+	user, err := tests.CreateUser(hub, "test@test.com", "testtesttest")
+	require.NoError(t, err)
+
+	synctest.Run(func() {
+		sm.Initialize()
+
+		record, err := tests.CreateRecord(hub, "systems", map[string]any{
+			"name":  "it-was-coney-island",
+			"host":  "the-playground-of-the-world",
+			"port":  "33914",
+			"users": []string{user.Id},
+		})
+		require.NoError(t, err)
+
+		assert.Equal(t, "pending", record.GetString("status"), "System status should be 'pending'")
+		assert.Equal(t, "pending", sm.GetSystemStatusFromStore(record.Id), "System status should be 'pending'")
+
+		// Verify the system host and port
+		host, port := sm.GetSystemHostPort(record.Id)
+		assert.Equal(t, record.GetString("host"), host, "System host should match")
+		assert.Equal(t, record.GetString("port"), port, "System port should match")
+
+		time.Sleep(13 * time.Second)
+		synctest.Wait()
+
+		assert.Equal(t, "pending", record.Fresh().GetString("status"), "System status should be 'pending'")
+		// Verify the system was added by checking if it exists
+		assert.True(t, sm.HasSystem(record.Id), "System should exist in the store")
+
+		time.Sleep(10 * time.Second)
+		synctest.Wait()
+
+		// system should be set to down after 15 seconds (no websocket connection)
+		assert.Equal(t, "down", sm.GetSystemStatusFromStore(record.Id), "System status should be 'down'")
+		// make sure the system is down in the db
+		record, err = hub.FindRecordById("systems", record.Id)
+		require.NoError(t, err)
+		assert.Equal(t, "down", record.GetString("status"), "System status should be 'down'")
+
+		assert.Equal(t, 1, sm.GetSystemCount(), "System count should be 1")
+
+		err = sm.RemoveSystem(record.Id)
+		assert.NoError(t, err)
+
+		assert.Equal(t, 0, sm.GetSystemCount(), "System count should be 0")
+		assert.False(t, sm.HasSystem(record.Id), "System should not exist in the store after removal")
+
+		// let's also make sure a system is removed from the store when the record is deleted
+		record, err = tests.CreateRecord(hub, "systems", map[string]any{
+			"name":  "there-was-no-place-like-it",
+			"host":  "in-the-whole-world",
+			"port":  "33914",
+			"users": []string{user.Id},
+		})
+		require.NoError(t, err)
+
+		assert.True(t, sm.HasSystem(record.Id), "System should exist in the store after creation")
+
+		time.Sleep(8 * time.Second)
+		synctest.Wait()
+		assert.Equal(t, "pending", sm.GetSystemStatusFromStore(record.Id), "System status should be 'pending'")
+
+		sm.SetSystemStatusInDB(record.Id, "up")
+		time.Sleep(time.Second)
+		synctest.Wait()
+		assert.Equal(t, "up", sm.GetSystemStatusFromStore(record.Id), "System status should be 'up'")
+
+		// make sure the system switches to down after 11 seconds
+		sm.RemoveSystem(record.Id)
+		sm.AddRecord(record, nil)
+		assert.Equal(t, "pending", sm.GetSystemStatusFromStore(record.Id), "System status should be 'pending'")
+		time.Sleep(12 * time.Second)
+		synctest.Wait()
+		assert.Equal(t, "down", sm.GetSystemStatusFromStore(record.Id), "System status should be 'down'")
+
+		// sm.SetSystemStatusInDB(record.Id, "paused")
+		// time.Sleep(time.Second)
+		// synctest.Wait()
+		// assert.Equal(t, "paused", sm.GetSystemStatusFromStore(record.Id), "System status should be 'paused'")
+
+		// delete the record
+		err = hub.Delete(record)
+		require.NoError(t, err)
+		assert.False(t, sm.HasSystem(record.Id), "System should not exist in the store after deletion")
+
+		testOld(t, hub)
+
+		time.Sleep(time.Second)
+		synctest.Wait()
+
+		for _, systemId := range sm.GetAllSystemIDs() {
+			err = sm.RemoveSystem(systemId)
+			require.NoError(t, err)
+			assert.False(t, sm.HasSystem(systemId), "System should not exist in the store after deletion")
+		}
+
+		assert.Equal(t, 0, sm.GetSystemCount(), "System count should be 0")
+
+		// TODO: test with websocket client
+	})
+}
+
+func testOld(t *testing.T, hub *tests.TestHub) {
+	user, err := tests.CreateUser(hub, "test@testy.com", "testtesttest")
+	require.NoError(t, err)
+
+	sm := hub.GetSystemManager()
+	assert.NotNil(t, sm)
+
+	// error expected when creating a user with a duplicate email
+	_, err = tests.CreateUser(hub, "test@test.com", "testtesttest")
+	require.Error(t, err)
+
+	// Test collection existence. todo: move to hub package tests
+	t.Run("CollectionExistence", func(t *testing.T) {
+		// Verify that required collections exist
+		systems, err := hub.FindCachedCollectionByNameOrId("systems")
+		require.NoError(t, err)
+		assert.NotNil(t, systems)
+
+		systemStats, err := hub.FindCachedCollectionByNameOrId("system_stats")
+		require.NoError(t, err)
+		assert.NotNil(t, systemStats)
+
+		containerStats, err := hub.FindCachedCollectionByNameOrId("container_stats")
+		require.NoError(t, err)
+		assert.NotNil(t, containerStats)
+	})
+
+	t.Run("RemoveSystem", func(t *testing.T) {
+		// Get the count before adding the system
+		countBefore := sm.GetSystemCount()
+
+		// Create a test system record
+		record, err := tests.CreateRecord(hub, "systems", map[string]any{
+			"name":  "i-even-got-lost-at-coney-island",
+			"host":  "but-they-found-me",
+			"port":  "33914",
+			"users": []string{user.Id},
+		})
+		require.NoError(t, err)
+
+		// Verify the system count increased
+		countAfterAdd := sm.GetSystemCount()
+		assert.Equal(t, countBefore+1, countAfterAdd, "System count should increase after adding a system via event hook")
+
+		// Verify the system exists
+		assert.True(t, sm.HasSystem(record.Id), "System should exist in the store")
+
+		// Remove the system
+		err = sm.RemoveSystem(record.Id)
+		assert.NoError(t, err)
+
+		// Check that the system count decreased
+		countAfterRemove := sm.GetSystemCount()
+		assert.Equal(t, countAfterAdd-1, countAfterRemove, "System count should decrease after removing a system")
+
+		// Verify the system no longer exists
+		assert.False(t, sm.HasSystem(record.Id), "System should not exist in the store after removal")
+
+		// Verify the system is not in the list of all system IDs
+		ids := sm.GetAllSystemIDs()
+		assert.NotContains(t, ids, record.Id, "System ID should not be in the list of all system IDs after removal")
+
+		// Verify the system status is empty
+		status := sm.GetSystemStatusFromStore(record.Id)
+		assert.Equal(t, "", status, "System status should be empty after removal")
+
+		// Try to remove it again - should return an error since it's already removed
+		err = sm.RemoveSystem(record.Id)
+		assert.Error(t, err)
+	})
+
+	t.Run("NewRecordPending", func(t *testing.T) {
+		// Create a test system
+		record, err := tests.CreateRecord(hub, "systems", map[string]any{
+			"name":  "and-you-know",
+			"host":  "i-feel-very-bad",
+			"port":  "33914",
+			"users": []string{user.Id},
+		})
+		require.NoError(t, err)
+
+		// Add the record to the system manager
+		err = sm.AddRecord(record, nil)
+		require.NoError(t, err)
+
+		// Test filtering records by status - should be "pending" now
+		filter := "status = 'pending'"
+		pendingSystems, err := hub.FindRecordsByFilter("systems", filter, "-created", 0, 0, nil)
+		require.NoError(t, err)
+		assert.GreaterOrEqual(t, len(pendingSystems), 1)
+	})
+
+	t.Run("SystemStatusUpdate", func(t *testing.T) {
+		// Create a test system record
+		record, err := tests.CreateRecord(hub, "systems", map[string]any{
+			"name":  "we-used-to-sleep-on-the-beach",
+			"host":  "sleep-overnight-here",
+			"port":  "33914",
+			"users": []string{user.Id},
+		})
+		require.NoError(t, err)
+
+		// Add the record to the system manager
+		err = sm.AddRecord(record, nil)
+		require.NoError(t, err)
+
+		// Test status changes
+		initialStatus := sm.GetSystemStatusFromStore(record.Id)
+
+		// Set a new status
+		sm.SetSystemStatusInDB(record.Id, "up")
+
+		// Verify status was updated
+		newStatus := sm.GetSystemStatusFromStore(record.Id)
+		assert.Equal(t, "up", newStatus, "System status should be updated to 'up'")
+		assert.NotEqual(t, initialStatus, newStatus, "Status should have changed")
+
+		// Verify the database was updated
+		updatedRecord, err := hub.FindRecordById("systems", record.Id)
+		require.NoError(t, err)
+		assert.Equal(t, "up", updatedRecord.Get("status"), "Database status should match")
+	})
+
+	t.Run("HandleSystemData", func(t *testing.T) {
+		// Create a test system record
+		record, err := tests.CreateRecord(hub, "systems", map[string]any{
+			"name":  "things-changed-you-know",
+			"host":  "they-dont-sleep-anymore-on-the-beach",
+			"port":  "33914",
+			"users": []string{user.Id},
+		})
+		require.NoError(t, err)
+
+		// Create test system data
+		testData := &system.CombinedData{
+			Info: system.Info{
+				Hostname:      "data-test.example.com",
+				KernelVersion: "5.15.0-generic",
+				Cores:         4,
+				Threads:       8,
+				CpuModel:      "Test CPU",
+				Uptime:        3600,
+				Cpu:           25.5,
+				MemPct:        40.2,
+				DiskPct:       60.0,
+				Bandwidth:     100.0,
+				AgentVersion:  "1.0.0",
+			},
+			Stats: system.Stats{
+				Cpu:         25.5,
+				Mem:         16384.0,
+				MemUsed:     6553.6,
+				MemPct:      40.0,
+				DiskTotal:   1024000.0,
+				DiskUsed:    614400.0,
+				DiskPct:     60.0,
+				NetworkSent: 1024.0,
+				NetworkRecv: 2048.0,
+			},
+			Containers: []*container.Stats{},
+		}
+
+		// Test handling system data. todo: move to hub/alerts package tests
+		err = hub.HandleSystemAlerts(record, testData)
+		assert.NoError(t, err)
+	})
+
+	t.Run("ErrorHandling", func(t *testing.T) {
+		// Try to add a non-existent record
+		nonExistentId := "non_existent_id"
+		err := sm.RemoveSystem(nonExistentId)
+		assert.Error(t, err)
+
+		// Try to add a system with invalid host
+		system := &systems.System{
+			Host: "",
+		}
+		err = sm.AddSystem(system)
+		assert.Error(t, err)
+	})
+
+	t.Run("ConcurrentOperations", func(t *testing.T) {
+		// Create a test system
+		record, err := tests.CreateRecord(hub, "systems", map[string]any{
+			"name":  "jfkjahkfajs",
+			"host":  "localhost",
+			"port":  "33914",
+			"users": []string{user.Id},
+		})
+		require.NoError(t, err)
+
+		// Run concurrent operations
+		const goroutines = 5
+		var wg sync.WaitGroup
+		wg.Add(goroutines)
+
+		for i := range goroutines {
+			go func(i int) {
+				defer wg.Done()
+
+				// Alternate between different operations
+				switch i % 3 {
+				case 0:
+					status := fmt.Sprintf("status-%d", i)
+					sm.SetSystemStatusInDB(record.Id, status)
+				case 1:
+					_ = sm.GetSystemStatusFromStore(record.Id)
+				case 2:
+					_, _ = sm.GetSystemHostPort(record.Id)
+				}
+			}(i)
+		}
+
+		wg.Wait()
+
+		// Verify system still exists and is in a valid state
+		assert.True(t, sm.HasSystem(record.Id), "System should still exist after concurrent operations")
+		status := sm.GetSystemStatusFromStore(record.Id)
+		assert.NotEmpty(t, status, "System should have a status after concurrent operations")
+	})
+
+	t.Run("ContextCancellation", func(t *testing.T) {
+		// Create a test system record
+		record, err := tests.CreateRecord(hub, "systems", map[string]any{
+			"name":  "lkhsdfsjf",
+			"host":  "localhost",
+			"port":  "33914",
+			"users": []string{user.Id},
+		})
+		require.NoError(t, err)
+
+		// Verify the system exists in the store
+		assert.True(t, sm.HasSystem(record.Id), "System should exist in the store")
+
+		// Store the original context and cancel function
+		originalCtx, originalCancel, err := sm.GetSystemContextFromStore(record.Id)
+		assert.NoError(t, err)
+
+		// Ensure the context is not nil
+		assert.NotNil(t, originalCtx, "System context should not be nil")
+		assert.NotNil(t, originalCancel, "System cancel function should not be nil")
+
+		// Cancel the context
+		originalCancel()
+
+		// Wait a short time for cancellation to propagate
+		time.Sleep(10 * time.Millisecond)
+
+		// Verify the context is done
+		select {
+		case <-originalCtx.Done():
+			// Context was properly cancelled
+		default:
+			t.Fatal("Context was not cancelled")
+		}
+
+		// Verify the system is still in the store (cancellation shouldn't remove it)
+		assert.True(t, sm.HasSystem(record.Id), "System should still exist after context cancellation")
+
+		// Explicitly remove the system
+		err = sm.RemoveSystem(record.Id)
+		assert.NoError(t, err, "RemoveSystem should succeed")
+
+		// Verify the system is removed
+		assert.False(t, sm.HasSystem(record.Id), "System should be removed after RemoveSystem")
+
+		// Try to remove it again - should return an error
+		err = sm.RemoveSystem(record.Id)
+		assert.Error(t, err, "RemoveSystem should fail for non-existent system")
+
+		// Add the system back
+		err = sm.AddRecord(record, nil)
+		require.NoError(t, err, "AddRecord should succeed")
+
+		// Verify the system is back in the store
+		assert.True(t, sm.HasSystem(record.Id), "System should exist after re-adding")
+
+		// Verify a new context was created
+		newCtx, newCancel, err := sm.GetSystemContextFromStore(record.Id)
+		assert.NoError(t, err)
+		assert.NotNil(t, newCtx, "New system context should not be nil")
+		assert.NotNil(t, newCancel, "New system cancel function should not be nil")
+		assert.NotEqual(t, originalCtx, newCtx, "New context should be different from original")
+
+		// Clean up
+		err = sm.RemoveSystem(record.Id)
+		assert.NoError(t, err)
+	})
+}

beszel/internal/hub/systems/systems_test_helpers.go

@@ -0,0 +1,102 @@
+//go:build testing
+// +build testing
+
+package systems
+
+import (
+	entities "beszel/internal/entities/system"
+	"context"
+	"fmt"
+)
+
+// TESTING ONLY: GetSystemCount returns the number of systems in the store
+func (sm *SystemManager) GetSystemCount() int {
+	return sm.systems.Length()
+}
+
+// TESTING ONLY: HasSystem checks if a system with the given ID exists in the store
+func (sm *SystemManager) HasSystem(systemID string) bool {
+	return sm.systems.Has(systemID)
+}
+
+// TESTING ONLY: GetSystemStatusFromStore returns the status of a system with the given ID
+// Returns an empty string if the system doesn't exist
+func (sm *SystemManager) GetSystemStatusFromStore(systemID string) string {
+	sys, ok := sm.systems.GetOk(systemID)
+	if !ok {
+		return ""
+	}
+	return sys.Status
+}
+
+// TESTING ONLY: GetSystemContextFromStore returns the context and cancel function for a system
+func (sm *SystemManager) GetSystemContextFromStore(systemID string) (context.Context, context.CancelFunc, error) {
+	sys, ok := sm.systems.GetOk(systemID)
+	if !ok {
+		return nil, nil, fmt.Errorf("no system")
+	}
+	return sys.ctx, sys.cancel, nil
+}
+
+// TESTING ONLY: GetSystemFromStore returns a store from the system
+func (sm *SystemManager) GetSystemFromStore(systemID string) (*System, error) {
+	sys, ok := sm.systems.GetOk(systemID)
+	if !ok {
+		return nil, fmt.Errorf("no system")
+	}
+	return sys, nil
+}
+
+// TESTING ONLY: GetAllSystemIDs returns a slice of all system IDs in the store
+func (sm *SystemManager) GetAllSystemIDs() []string {
+	data := sm.systems.GetAll()
+	ids := make([]string, 0, len(data))
+	for id := range data {
+		ids = append(ids, id)
+	}
+	return ids
+}
+
+// TESTING ONLY: GetSystemData returns the combined data for a system with the given ID
+// Returns nil if the system doesn't exist
+// This method is intended for testing
+func (sm *SystemManager) GetSystemData(systemID string) *entities.CombinedData {
+	sys, ok := sm.systems.GetOk(systemID)
+	if !ok {
+		return nil
+	}
+	return sys.data
+}
+
+// TESTING ONLY: GetSystemHostPort returns the host and port for a system with the given ID
+// Returns empty strings if the system doesn't exist
+func (sm *SystemManager) GetSystemHostPort(systemID string) (string, string) {
+	sys, ok := sm.systems.GetOk(systemID)
+	if !ok {
+		return "", ""
+	}
+	return sys.Host, sys.Port
+}
+
+// TESTING ONLY: SetSystemStatusInDB sets the status of a system directly and updates the database record
+// This is intended for testing
+// Returns false if the system doesn't exist
+func (sm *SystemManager) SetSystemStatusInDB(systemID string, status string) bool {
+	if !sm.HasSystem(systemID) {
+		return false
+	}
+
+	// Update the database record
+	record, err := sm.hub.FindRecordById("systems", systemID)
+	if err != nil {
+		return false
+	}
+
+	record.Set("status", status)
+	err = sm.hub.Save(record)
+	if err != nil {
+		return false
+	}
+
+	return true
+}

beszel/internal/hub/ws/ws.go

@@ -0,0 +1,180 @@
+package ws
+
+import (
+	"beszel/internal/common"
+	"beszel/internal/entities/system"
+	"errors"
+	"time"
+	"weak"
+
+	"github.com/fxamacker/cbor/v2"
+	"github.com/lxzan/gws"
+	"golang.org/x/crypto/ssh"
+)
+
+const (
+	deadline = 70 * time.Second
+)
+
+// Handler implements the WebSocket event handler for agent connections.
+type Handler struct {
+	gws.BuiltinEventHandler
+}
+
+// WsConn represents a WebSocket connection to an agent.
+type WsConn struct {
+	conn         *gws.Conn
+	responseChan chan *gws.Message
+	DownChan     chan struct{}
+}
+
+// FingerprintRecord is fingerprints collection record data in the hub
+type FingerprintRecord struct {
+	Id          string `db:"id"`
+	SystemId    string `db:"system"`
+	Fingerprint string `db:"fingerprint"`
+	Token       string `db:"token"`
+}
+
+var upgrader *gws.Upgrader
+
+// GetUpgrader returns a singleton WebSocket upgrader instance.
+func GetUpgrader() *gws.Upgrader {
+	if upgrader != nil {
+		return upgrader
+	}
+	handler := &Handler{}
+	upgrader = gws.NewUpgrader(handler, &gws.ServerOption{})
+	return upgrader
+}
+
+// NewWsConnection creates a new WebSocket connection wrapper.
+func NewWsConnection(conn *gws.Conn) *WsConn {
+	return &WsConn{
+		conn:         conn,
+		responseChan: make(chan *gws.Message, 1),
+		DownChan:     make(chan struct{}, 1),
+	}
+}
+
+// OnOpen sets a deadline for the WebSocket connection.
+func (h *Handler) OnOpen(conn *gws.Conn) {
+	conn.SetDeadline(time.Now().Add(deadline))
+}
+
+// OnMessage routes incoming WebSocket messages to the response channel.
+func (h *Handler) OnMessage(conn *gws.Conn, message *gws.Message) {
+	conn.SetDeadline(time.Now().Add(deadline))
+	if message.Opcode != gws.OpcodeBinary || message.Data.Len() == 0 {
+		return
+	}
+	wsConn, ok := conn.Session().Load("wsConn")
+	if !ok {
+		_ = conn.WriteClose(1000, nil)
+		return
+	}
+	select {
+	case wsConn.(*WsConn).responseChan <- message:
+	default:
+		// close if the connection is not expecting a response
+		wsConn.(*WsConn).Close(nil)
+	}
+}
+
+// OnClose handles WebSocket connection closures and triggers system down status after delay.
+func (h *Handler) OnClose(conn *gws.Conn, err error) {
+	wsConn, ok := conn.Session().Load("wsConn")
+	if !ok {
+		return
+	}
+	wsConn.(*WsConn).conn = nil
+	// wait 5 seconds to allow reconnection before setting system down
+	// use a weak pointer to avoid keeping references if the system is removed
+	go func(downChan weak.Pointer[chan struct{}]) {
+		time.Sleep(5 * time.Second)
+		downChanValue := downChan.Value()
+		if downChanValue != nil {
+			*downChanValue <- struct{}{}
+		}
+	}(weak.Make(&wsConn.(*WsConn).DownChan))
+}
+
+// Close terminates the WebSocket connection gracefully.
+func (ws *WsConn) Close(msg []byte) {
+	if ws.IsConnected() {
+		ws.conn.WriteClose(1000, msg)
+	}
+}
+
+// Ping sends a ping frame to keep the connection alive.
+func (ws *WsConn) Ping() error {
+	ws.conn.SetDeadline(time.Now().Add(deadline))
+	return ws.conn.WritePing(nil)
+}
+
+// sendMessage encodes data to CBOR and sends it as a binary message to the agent.
+func (ws *WsConn) sendMessage(data common.HubRequest[any]) error {
+	if ws.conn == nil {
+		return gws.ErrConnClosed
+	}
+	bytes, err := cbor.Marshal(data)
+	if err != nil {
+		return err
+	}
+	return ws.conn.WriteMessage(gws.OpcodeBinary, bytes)
+}
+
+// RequestSystemData requests system metrics from the agent and unmarshals the response.
+func (ws *WsConn) RequestSystemData(data *system.CombinedData) error {
+	var message *gws.Message
+
+	ws.sendMessage(common.HubRequest[any]{
+		Action: common.GetData,
+	})
+	select {
+	case <-time.After(10 * time.Second):
+		ws.Close(nil)
+		return gws.ErrConnClosed
+	case message = <-ws.responseChan:
+	}
+	defer message.Close()
+	return cbor.Unmarshal(message.Data.Bytes(), data)
+}
+
+// GetFingerprint authenticates with the agent using SSH signature and returns the agent's fingerprint.
+func (ws *WsConn) GetFingerprint(token string, signer ssh.Signer, needSysInfo bool) (common.FingerprintResponse, error) {
+	var clientFingerprint common.FingerprintResponse
+	challenge := []byte(token)
+
+	signature, err := signer.Sign(nil, challenge)
+	if err != nil {
+		return clientFingerprint, err
+	}
+
+	err = ws.sendMessage(common.HubRequest[any]{
+		Action: common.CheckFingerprint,
+		Data: common.FingerprintRequest{
+			Signature:   signature.Blob,
+			NeedSysInfo: needSysInfo,
+		},
+	})
+	if err != nil {
+		return clientFingerprint, err
+	}
+
+	var message *gws.Message
+	select {
+	case message = <-ws.responseChan:
+	case <-time.After(10 * time.Second):
+		return clientFingerprint, errors.New("request expired")
+	}
+	defer message.Close()
+
+	err = cbor.Unmarshal(message.Data.Bytes(), &clientFingerprint)
+	return clientFingerprint, err
+}
+
+// IsConnected returns true if the WebSocket connection is active.
+func (ws *WsConn) IsConnected() bool {
+	return ws.conn != nil
+}

beszel/internal/hub/ws/ws_test.go

@@ -0,0 +1,221 @@
+//go:build testing
+// +build testing
+
+package ws
+
+import (
+	"beszel/internal/common"
+	"crypto/ed25519"
+	"testing"
+	"time"
+
+	"github.com/fxamacker/cbor/v2"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"golang.org/x/crypto/ssh"
+)
+
+// TestGetUpgrader tests the singleton upgrader
+func TestGetUpgrader(t *testing.T) {
+	// Reset the global upgrader to test singleton behavior
+	upgrader = nil
+
+	// First call should create the upgrader
+	upgrader1 := GetUpgrader()
+	assert.NotNil(t, upgrader1, "Upgrader should not be nil")
+
+	// Second call should return the same instance
+	upgrader2 := GetUpgrader()
+	assert.Same(t, upgrader1, upgrader2, "Should return the same upgrader instance")
+
+	// Verify it's properly configured
+	assert.NotNil(t, upgrader1, "Upgrader should be configured")
+}
+
+// TestNewWsConnection tests WebSocket connection creation
+func TestNewWsConnection(t *testing.T) {
+	// We can't easily mock gws.Conn, so we'll pass nil and test the structure
+	wsConn := NewWsConnection(nil)
+
+	assert.NotNil(t, wsConn, "WebSocket connection should not be nil")
+	assert.Nil(t, wsConn.conn, "Connection should be nil as passed")
+	assert.NotNil(t, wsConn.responseChan, "Response channel should be initialized")
+	assert.NotNil(t, wsConn.DownChan, "Down channel should be initialized")
+	assert.Equal(t, 1, cap(wsConn.responseChan), "Response channel should have capacity of 1")
+	assert.Equal(t, 1, cap(wsConn.DownChan), "Down channel should have capacity of 1")
+}
+
+// TestWsConn_IsConnected tests the connection status check
+func TestWsConn_IsConnected(t *testing.T) {
+	// Test with nil connection
+	wsConn := NewWsConnection(nil)
+	assert.False(t, wsConn.IsConnected(), "Should not be connected when conn is nil")
+}
+
+// TestWsConn_Close tests the connection closing with nil connection
+func TestWsConn_Close(t *testing.T) {
+	wsConn := NewWsConnection(nil)
+
+	// Should handle nil connection gracefully
+	assert.NotPanics(t, func() {
+		wsConn.Close([]byte("test message"))
+	}, "Should not panic when closing nil connection")
+}
+
+// TestWsConn_SendMessage_CBOR tests CBOR encoding in sendMessage
+func TestWsConn_SendMessage_CBOR(t *testing.T) {
+	wsConn := NewWsConnection(nil)
+
+	testData := common.HubRequest[any]{
+		Action: common.GetData,
+		Data:   "test data",
+	}
+
+	// This will fail because conn is nil, but we can test the CBOR encoding logic
+	// by checking that the function properly encodes to CBOR before failing
+	err := wsConn.sendMessage(testData)
+	assert.Error(t, err, "Should error with nil connection")
+
+	// Test CBOR encoding separately
+	bytes, err := cbor.Marshal(testData)
+	assert.NoError(t, err, "Should encode to CBOR successfully")
+
+	// Verify we can decode it back
+	var decodedData common.HubRequest[any]
+	err = cbor.Unmarshal(bytes, &decodedData)
+	assert.NoError(t, err, "Should decode from CBOR successfully")
+	assert.Equal(t, testData.Action, decodedData.Action, "Action should match")
+}
+
+// TestWsConn_GetFingerprint_SignatureGeneration tests signature creation logic
+func TestWsConn_GetFingerprint_SignatureGeneration(t *testing.T) {
+	// Generate test key pair
+	_, privKey, err := ed25519.GenerateKey(nil)
+	require.NoError(t, err)
+
+	signer, err := ssh.NewSignerFromKey(privKey)
+	require.NoError(t, err)
+
+	token := "test-token"
+
+	// This will timeout since conn is nil, but we can verify the signature logic
+	// We can't test the full flow, but we can test that the signature is created properly
+	challenge := []byte(token)
+	signature, err := signer.Sign(nil, challenge)
+	assert.NoError(t, err, "Should create signature successfully")
+	assert.NotEmpty(t, signature.Blob, "Signature blob should not be empty")
+	assert.Equal(t, signer.PublicKey().Type(), signature.Format, "Signature format should match key type")
+
+	// Test the fingerprint request structure
+	fpRequest := common.FingerprintRequest{
+		Signature:   signature.Blob,
+		NeedSysInfo: true,
+	}
+
+	// Test CBOR encoding of fingerprint request
+	fpData, err := cbor.Marshal(fpRequest)
+	assert.NoError(t, err, "Should encode fingerprint request to CBOR")
+
+	var decodedFpRequest common.FingerprintRequest
+	err = cbor.Unmarshal(fpData, &decodedFpRequest)
+	assert.NoError(t, err, "Should decode fingerprint request from CBOR")
+	assert.Equal(t, fpRequest.Signature, decodedFpRequest.Signature, "Signature should match")
+	assert.Equal(t, fpRequest.NeedSysInfo, decodedFpRequest.NeedSysInfo, "NeedSysInfo should match")
+
+	// Test the full hub request structure
+	hubRequest := common.HubRequest[any]{
+		Action: common.CheckFingerprint,
+		Data:   fpRequest,
+	}
+
+	hubData, err := cbor.Marshal(hubRequest)
+	assert.NoError(t, err, "Should encode hub request to CBOR")
+
+	var decodedHubRequest common.HubRequest[cbor.RawMessage]
+	err = cbor.Unmarshal(hubData, &decodedHubRequest)
+	assert.NoError(t, err, "Should decode hub request from CBOR")
+	assert.Equal(t, common.CheckFingerprint, decodedHubRequest.Action, "Action should be CheckFingerprint")
+}
+
+// TestWsConn_RequestSystemData_RequestFormat tests system data request format
+func TestWsConn_RequestSystemData_RequestFormat(t *testing.T) {
+	// Test the request format that would be sent
+	request := common.HubRequest[any]{
+		Action: common.GetData,
+	}
+
+	// Test CBOR encoding
+	data, err := cbor.Marshal(request)
+	assert.NoError(t, err, "Should encode request to CBOR")
+
+	// Test decoding
+	var decodedRequest common.HubRequest[any]
+	err = cbor.Unmarshal(data, &decodedRequest)
+	assert.NoError(t, err, "Should decode request from CBOR")
+	assert.Equal(t, common.GetData, decodedRequest.Action, "Should have GetData action")
+}
+
+// TestFingerprintRecord tests the FingerprintRecord struct
+func TestFingerprintRecord(t *testing.T) {
+	record := FingerprintRecord{
+		Id:          "test-id",
+		SystemId:    "system-123",
+		Fingerprint: "test-fingerprint",
+		Token:       "test-token",
+	}
+
+	assert.Equal(t, "test-id", record.Id)
+	assert.Equal(t, "system-123", record.SystemId)
+	assert.Equal(t, "test-fingerprint", record.Fingerprint)
+	assert.Equal(t, "test-token", record.Token)
+}
+
+// TestDeadlineConstant tests that the deadline constant is reasonable
+func TestDeadlineConstant(t *testing.T) {
+	assert.Equal(t, 70*time.Second, deadline, "Deadline should be 70 seconds")
+}
+
+// TestCommonActions tests that the common actions are properly defined
+func TestCommonActions(t *testing.T) {
+	// Test that the actions we use exist and have expected values
+	assert.Equal(t, common.WebSocketAction(0), common.GetData, "GetData should be action 0")
+	assert.Equal(t, common.WebSocketAction(1), common.CheckFingerprint, "CheckFingerprint should be action 1")
+}
+
+// TestHandler tests that we can create a Handler
+func TestHandler(t *testing.T) {
+	handler := &Handler{}
+	assert.NotNil(t, handler, "Handler should be created successfully")
+
+	// The Handler embeds gws.BuiltinEventHandler, so it should have the embedded type
+	assert.NotNil(t, handler.BuiltinEventHandler, "Should have embedded BuiltinEventHandler")
+}
+
+// TestWsConnChannelBehavior tests channel behavior without WebSocket connections
+func TestWsConnChannelBehavior(t *testing.T) {
+	wsConn := NewWsConnection(nil)
+
+	// Test that channels are properly initialized and can be used
+	select {
+	case wsConn.DownChan <- struct{}{}:
+		// Should be able to write to channel
+	default:
+		t.Error("Should be able to write to DownChan")
+	}
+
+	// Test reading from DownChan
+	select {
+	case <-wsConn.DownChan:
+		// Should be able to read from channel
+	case <-time.After(10 * time.Millisecond):
+		t.Error("Should be able to read from DownChan")
+	}
+
+	// Response channel should be empty initially
+	select {
+	case <-wsConn.responseChan:
+		t.Error("Response channel should be empty initially")
+	default:
+		// Expected - channel should be empty
+	}
+}

beszel/internal/records/records.go

@@ -4,20 +4,19 @@ package records
 import (
 	"beszel/internal/entities/container"
 	"beszel/internal/entities/system"
+	"encoding/json"
+	"fmt"
 	"log"
 	"math"
+	"strings"
 	"time"
 
-	"github.com/goccy/go-json"
 	"github.com/pocketbase/dbx"
-	"github.com/pocketbase/pocketbase"
-	"github.com/pocketbase/pocketbase/daos"
-	"github.com/pocketbase/pocketbase/models"
-	"github.com/pocketbase/pocketbase/tools/types"
+	"github.com/pocketbase/pocketbase/core"
 )
 
 type RecordManager struct {
-	app *pocketbase.PocketBase
+	app core.App
 }
 
 type LongerRecordData struct {
@@ -27,21 +26,28 @@ type LongerRecordData struct {
 	minShorterRecords  int
 }
 
-type RecordDeletionData struct {
-	recordType string
-	retention  time.Duration
+type RecordIds []struct {
+	Id string `db:"id"`
 }
 
-type RecordStats []struct {
-	Stats []byte `db:"stats"`
-}
-
-func NewRecordManager(app *pocketbase.PocketBase) *RecordManager {
+func NewRecordManager(app core.App) *RecordManager {
 	return &RecordManager{app}
 }
 
+type StatsRecord struct {
+	Stats []byte `db:"stats"`
+}
+
+// global variables for reusing allocations
+var statsRecord StatsRecord
+var containerStats []container.Stats
+var sumStats system.Stats
+var tempStats system.Stats
+var queryParams = make(dbx.Params, 1)
+var containerSums = make(map[string]*container.Stats)
+
 // Create longer records by averaging shorter records
-func (rm *RecordManager) CreateLongerRecords(collections []*models.Collection) {
+func (rm *RecordManager) CreateLongerRecords() {
 	// start := time.Now()
 	longerRecordData := []LongerRecordData{
 		{
@@ -71,15 +77,24 @@ func (rm *RecordManager) CreateLongerRecords(collections []*models.Collection) {
 		},
 	}
 	// wrap the operations in a transaction
-	rm.app.Dao().RunInTransaction(func(txDao *daos.Dao) error {
-		activeSystems, err := txDao.FindRecordsByExpr("systems", dbx.NewExp("status = 'up'"))
+	rm.app.RunInTransaction(func(txApp core.App) error {
+		var err error
+		collections := [2]*core.Collection{}
+		collections[0], err = txApp.FindCachedCollectionByNameOrId("system_stats")
 		if err != nil {
-			log.Println("failed to get active systems", "err", err.Error())
 			return err
 		}
+		collections[1], err = txApp.FindCachedCollectionByNameOrId("container_stats")
+		if err != nil {
+			return err
+		}
+		var systems RecordIds
+		db := txApp.DB()
+
+		db.NewQuery("SELECT id FROM systems WHERE status='up'").All(&systems)
 
 		// loop through all active systems, time periods, and collections
-		for _, system := range activeSystems {
+		for _, system := range systems {
 			// log.Println("processing system", system.GetString("name"))
 			for i := range longerRecordData {
 				recordData := longerRecordData[i]
@@ -92,50 +107,51 @@ func (rm *RecordManager) CreateLongerRecords(collections []*models.Collection) {
 				for _, collection := range collections {
 					// check creation time of last longer record if not 10m, since 10m is created every run
 					if recordData.longerType != "10m" {
-						lastLongerRecord, err := txDao.FindFirstRecordByFilter(
+						count, err := txApp.CountRecords(
 							collection.Id,
-							"type = {:type} && system = {:system} && created > {:created}",
-							dbx.Params{"type": recordData.longerType, "system": system.Id, "created": longerRecordPeriod},
+							dbx.NewExp(
+								"system = {:system} AND type = {:type} AND created > {:created}",
+								dbx.Params{"type": recordData.longerType, "system": system.Id, "created": longerRecordPeriod},
+							),
 						)
 						// continue if longer record exists
-						if err == nil || lastLongerRecord != nil {
-							// log.Println("longer record found. continuing")
+						if err != nil || count > 0 {
 							continue
 						}
 					}
 					// get shorter records from the past x minutes
-					var stats RecordStats
+					var recordIds RecordIds
 
-					err := txDao.DB().
-						Select("stats").
+					err := txApp.DB().
+						Select("id").
 						From(collection.Name).
 						AndWhere(dbx.NewExp(
-							"type={:type} AND system={:system} AND created > {:created}",
+							"system={:system} AND type={:type} AND created > {:created}",
 							dbx.Params{
 								"type":    recordData.shorterType,
 								"system":  system.Id,
 								"created": shorterRecordPeriod,
 							},
 						)).
-						All(&stats)
+						All(&recordIds)
 
 					// continue if not enough shorter records
-					if err != nil || len(stats) < recordData.minShorterRecords {
-						// log.Println("not enough shorter records. continue.", len(allShorterRecords), recordData.expectedShorterRecords)
+					if err != nil || len(recordIds) < recordData.minShorterRecords {
 						continue
 					}
 					// average the shorter records and create longer record
-					longerRecord := models.NewRecord(collection)
+					longerRecord := core.NewRecord(collection)
 					longerRecord.Set("system", system.Id)
 					longerRecord.Set("type", recordData.longerType)
 					switch collection.Name {
 					case "system_stats":
-						longerRecord.Set("stats", rm.AverageSystemStats(stats))
+						longerRecord.Set("stats", rm.AverageSystemStats(db, recordIds))
 					case "container_stats":
-						longerRecord.Set("stats", rm.AverageContainerStats(stats))
+
+						longerRecord.Set("stats", rm.AverageContainerStats(db, recordIds))
 					}
-					if err := txDao.SaveRecord(longerRecord); err != nil {
-						log.Println("failed to save longer record", "err", err.Error())
+					if err := txApp.SaveNoValidate(longerRecord); err != nil {
+						log.Println("failed to save longer record", "err", err)
 					}
 				}
 			}
@@ -144,23 +160,34 @@ func (rm *RecordManager) CreateLongerRecords(collections []*models.Collection) {
 		return nil
 	})
 
+	statsRecord.Stats = statsRecord.Stats[:0]
+
 	// log.Println("finished creating longer records", "time (ms)", time.Since(start).Milliseconds())
 }
 
 // Calculate the average stats of a list of system_stats records without reflect
-func (rm *RecordManager) AverageSystemStats(records RecordStats) system.Stats {
-	sum := system.Stats{
-		Temperatures: make(map[string]float64),
-		ExtraFs:      make(map[string]*system.FsStats),
-	}
+func (rm *RecordManager) AverageSystemStats(db dbx.Builder, records RecordIds) *system.Stats {
+	// Clear/reset global structs for reuse
+	sumStats = system.Stats{}
+	tempStats = system.Stats{}
+	sum := &sumStats
+	stats := &tempStats
 
 	count := float64(len(records))
-	// use different counter for temps in case some records don't have them
 	tempCount := float64(0)
 
-	var stats system.Stats
-	for i := range records {
-		json.Unmarshal(records[i].Stats, &stats)
+	// Accumulate totals
+	for _, record := range records {
+		id := record.Id
+		// clear global statsRecord for reuse
+		statsRecord.Stats = statsRecord.Stats[:0]
+
+		queryParams["id"] = id
+		db.NewQuery("SELECT stats FROM system_stats WHERE id = {:id}").Bind(queryParams).One(&statsRecord)
+		if err := json.Unmarshal(statsRecord.Stats, stats); err != nil {
+			continue
+		}
+
 		sum.Cpu += stats.Cpu
 		sum.Mem += stats.Mem
 		sum.MemUsed += stats.MemUsed
@@ -176,96 +203,148 @@ func (rm *RecordManager) AverageSystemStats(records RecordStats) system.Stats {
 		sum.DiskWritePs += stats.DiskWritePs
 		sum.NetworkSent += stats.NetworkSent
 		sum.NetworkRecv += stats.NetworkRecv
-		// set peak values
+		sum.LoadAvg[0] += stats.LoadAvg[0]
+		sum.LoadAvg[1] += stats.LoadAvg[1]
+		sum.LoadAvg[2] += stats.LoadAvg[2]
+		sum.Bandwidth[0] += stats.Bandwidth[0]
+		sum.Bandwidth[1] += stats.Bandwidth[1]
+		// Set peak values
 		sum.MaxCpu = max(sum.MaxCpu, stats.MaxCpu, stats.Cpu)
 		sum.MaxNetworkSent = max(sum.MaxNetworkSent, stats.MaxNetworkSent, stats.NetworkSent)
 		sum.MaxNetworkRecv = max(sum.MaxNetworkRecv, stats.MaxNetworkRecv, stats.NetworkRecv)
 		sum.MaxDiskReadPs = max(sum.MaxDiskReadPs, stats.MaxDiskReadPs, stats.DiskReadPs)
 		sum.MaxDiskWritePs = max(sum.MaxDiskWritePs, stats.MaxDiskWritePs, stats.DiskWritePs)
-		// add temps to sum
+		sum.MaxBandwidth[0] = max(sum.MaxBandwidth[0], stats.MaxBandwidth[0], stats.Bandwidth[0])
+		sum.MaxBandwidth[1] = max(sum.MaxBandwidth[1], stats.MaxBandwidth[1], stats.Bandwidth[1])
+
+		// Accumulate temperatures
 		if stats.Temperatures != nil {
+			if sum.Temperatures == nil {
+				sum.Temperatures = make(map[string]float64, len(stats.Temperatures))
+			}
 			tempCount++
 			for key, value := range stats.Temperatures {
-				if _, ok := sum.Temperatures[key]; !ok {
-					sum.Temperatures[key] = 0
-				}
 				sum.Temperatures[key] += value
 			}
 		}
-		// add extra fs to sum
+
+		// Accumulate extra filesystem stats
 		if stats.ExtraFs != nil {
+			if sum.ExtraFs == nil {
+				sum.ExtraFs = make(map[string]*system.FsStats, len(stats.ExtraFs))
+			}
 			for key, value := range stats.ExtraFs {
 				if _, ok := sum.ExtraFs[key]; !ok {
 					sum.ExtraFs[key] = &system.FsStats{}
 				}
-				sum.ExtraFs[key].DiskTotal += value.DiskTotal
-				sum.ExtraFs[key].DiskUsed += value.DiskUsed
-				sum.ExtraFs[key].DiskWritePs += value.DiskWritePs
-				sum.ExtraFs[key].DiskReadPs += value.DiskReadPs
-				// peak values
-				sum.ExtraFs[key].MaxDiskReadPS = max(sum.ExtraFs[key].MaxDiskReadPS, value.MaxDiskReadPS, value.DiskReadPs)
-				sum.ExtraFs[key].MaxDiskWritePS = max(sum.ExtraFs[key].MaxDiskWritePS, value.MaxDiskWritePS, value.DiskWritePs)
+				fs := sum.ExtraFs[key]
+				fs.DiskTotal += value.DiskTotal
+				fs.DiskUsed += value.DiskUsed
+				fs.DiskWritePs += value.DiskWritePs
+				fs.DiskReadPs += value.DiskReadPs
+				fs.MaxDiskReadPS = max(fs.MaxDiskReadPS, value.MaxDiskReadPS, value.DiskReadPs)
+				fs.MaxDiskWritePS = max(fs.MaxDiskWritePS, value.MaxDiskWritePS, value.DiskWritePs)
+			}
+		}
+
+		// Accumulate GPU data
+		if stats.GPUData != nil {
+			if sum.GPUData == nil {
+				sum.GPUData = make(map[string]system.GPUData, len(stats.GPUData))
+			}
+			for id, value := range stats.GPUData {
+				gpu, ok := sum.GPUData[id]
+				if !ok {
+					gpu = system.GPUData{Name: value.Name}
+				}
+				gpu.Temperature += value.Temperature
+				gpu.MemoryUsed += value.MemoryUsed
+				gpu.MemoryTotal += value.MemoryTotal
+				gpu.Usage += value.Usage
+				gpu.Power += value.Power
+				gpu.Count += value.Count
+				sum.GPUData[id] = gpu
 			}
 		}
 	}
 
-	stats = system.Stats{
-		Cpu:            twoDecimals(sum.Cpu / count),
-		Mem:            twoDecimals(sum.Mem / count),
-		MemUsed:        twoDecimals(sum.MemUsed / count),
-		MemPct:         twoDecimals(sum.MemPct / count),
-		MemBuffCache:   twoDecimals(sum.MemBuffCache / count),
-		MemZfsArc:      twoDecimals(sum.MemZfsArc / count),
-		Swap:           twoDecimals(sum.Swap / count),
-		SwapUsed:       twoDecimals(sum.SwapUsed / count),
-		DiskTotal:      twoDecimals(sum.DiskTotal / count),
-		DiskUsed:       twoDecimals(sum.DiskUsed / count),
-		DiskPct:        twoDecimals(sum.DiskPct / count),
-		DiskReadPs:     twoDecimals(sum.DiskReadPs / count),
-		DiskWritePs:    twoDecimals(sum.DiskWritePs / count),
-		NetworkSent:    twoDecimals(sum.NetworkSent / count),
-		NetworkRecv:    twoDecimals(sum.NetworkRecv / count),
-		MaxCpu:         sum.MaxCpu,
-		MaxDiskReadPs:  sum.MaxDiskReadPs,
-		MaxDiskWritePs: sum.MaxDiskWritePs,
-		MaxNetworkSent: sum.MaxNetworkSent,
-		MaxNetworkRecv: sum.MaxNetworkRecv,
-	}
-
-	if len(sum.Temperatures) != 0 {
-		stats.Temperatures = make(map[string]float64, len(sum.Temperatures))
-		for key, value := range sum.Temperatures {
-			stats.Temperatures[key] = twoDecimals(value / tempCount)
+	// Compute averages in place
+	if count > 0 {
+		sum.Cpu = twoDecimals(sum.Cpu / count)
+		sum.Mem = twoDecimals(sum.Mem / count)
+		sum.MemUsed = twoDecimals(sum.MemUsed / count)
+		sum.MemPct = twoDecimals(sum.MemPct / count)
+		sum.MemBuffCache = twoDecimals(sum.MemBuffCache / count)
+		sum.MemZfsArc = twoDecimals(sum.MemZfsArc / count)
+		sum.Swap = twoDecimals(sum.Swap / count)
+		sum.SwapUsed = twoDecimals(sum.SwapUsed / count)
+		sum.DiskTotal = twoDecimals(sum.DiskTotal / count)
+		sum.DiskUsed = twoDecimals(sum.DiskUsed / count)
+		sum.DiskPct = twoDecimals(sum.DiskPct / count)
+		sum.DiskReadPs = twoDecimals(sum.DiskReadPs / count)
+		sum.DiskWritePs = twoDecimals(sum.DiskWritePs / count)
+		sum.NetworkSent = twoDecimals(sum.NetworkSent / count)
+		sum.NetworkRecv = twoDecimals(sum.NetworkRecv / count)
+		sum.LoadAvg[0] = twoDecimals(sum.LoadAvg[0] / count)
+		sum.LoadAvg[1] = twoDecimals(sum.LoadAvg[1] / count)
+		sum.LoadAvg[2] = twoDecimals(sum.LoadAvg[2] / count)
+		sum.Bandwidth[0] = sum.Bandwidth[0] / uint64(count)
+		sum.Bandwidth[1] = sum.Bandwidth[1] / uint64(count)
+		// Average temperatures
+		if sum.Temperatures != nil && tempCount > 0 {
+			for key := range sum.Temperatures {
+				sum.Temperatures[key] = twoDecimals(sum.Temperatures[key] / tempCount)
+			}
 		}
-	}
 
-	if len(sum.ExtraFs) != 0 {
-		stats.ExtraFs = make(map[string]*system.FsStats, len(sum.ExtraFs))
-		for key, value := range sum.ExtraFs {
-			stats.ExtraFs[key] = &system.FsStats{
-				DiskTotal:      twoDecimals(value.DiskTotal / count),
-				DiskUsed:       twoDecimals(value.DiskUsed / count),
-				DiskWritePs:    twoDecimals(value.DiskWritePs / count),
-				DiskReadPs:     twoDecimals(value.DiskReadPs / count),
-				MaxDiskReadPS:  value.MaxDiskReadPS,
-				MaxDiskWritePS: value.MaxDiskWritePS,
+		// Average extra filesystem stats
+		if sum.ExtraFs != nil {
+			for key := range sum.ExtraFs {
+				fs := sum.ExtraFs[key]
+				fs.DiskTotal = twoDecimals(fs.DiskTotal / count)
+				fs.DiskUsed = twoDecimals(fs.DiskUsed / count)
+				fs.DiskWritePs = twoDecimals(fs.DiskWritePs / count)
+				fs.DiskReadPs = twoDecimals(fs.DiskReadPs / count)
+			}
+		}
+
+		// Average GPU data
+		if sum.GPUData != nil {
+			for id := range sum.GPUData {
+				gpu := sum.GPUData[id]
+				gpu.Temperature = twoDecimals(gpu.Temperature / count)
+				gpu.MemoryUsed = twoDecimals(gpu.MemoryUsed / count)
+				gpu.MemoryTotal = twoDecimals(gpu.MemoryTotal / count)
+				gpu.Usage = twoDecimals(gpu.Usage / count)
+				gpu.Power = twoDecimals(gpu.Power / count)
+				gpu.Count = twoDecimals(gpu.Count / count)
+				sum.GPUData[id] = gpu
 			}
 		}
 	}
 
-	return stats
+	return sum
 }
 
 // Calculate the average stats of a list of container_stats records
-func (rm *RecordManager) AverageContainerStats(records RecordStats) []container.Stats {
-	sums := make(map[string]*container.Stats)
+func (rm *RecordManager) AverageContainerStats(db dbx.Builder, records RecordIds) []container.Stats {
+	// Clear global map for reuse
+	for k := range containerSums {
+		delete(containerSums, k)
+	}
+	sums := containerSums
 	count := float64(len(records))
 
-	var containerStats []container.Stats
 	for i := range records {
-		// Reset the slice length to 0, but keep the capacity
+		id := records[i].Id
+		// clear global statsRecord and containerStats for reuse
+		statsRecord.Stats = statsRecord.Stats[:0]
 		containerStats = containerStats[:0]
-		if err := json.Unmarshal(records[i].Stats, &containerStats); err != nil {
+
+		queryParams["id"] = id
+		db.NewQuery("SELECT stats FROM container_stats WHERE id = {:id}").Bind(queryParams).One(&statsRecord)
+
+		if err := json.Unmarshal(statsRecord.Stats, &containerStats); err != nil {
 			return []container.Stats{}
 		}
 		for i := range containerStats {
@@ -293,42 +372,80 @@ func (rm *RecordManager) AverageContainerStats(records RecordStats) []container.
 	return result
 }
 
-// Deletes records older than what is displayed in the UI
+// Delete old records
 func (rm *RecordManager) DeleteOldRecords() {
-	collections := []string{"system_stats", "container_stats"}
-	recordData := []RecordDeletionData{
-		{
-			recordType: "1m",
-			retention:  time.Hour,
-		},
-		{
-			recordType: "10m",
-			retention:  12 * time.Hour,
-		},
-		{
-			recordType: "20m",
-			retention:  24 * time.Hour,
-		},
-		{
-			recordType: "120m",
-			retention:  7 * 24 * time.Hour,
-		},
-		{
-			recordType: "480m",
-			retention:  30 * 24 * time.Hour,
-		},
+	rm.app.RunInTransaction(func(txApp core.App) error {
+		err := deleteOldSystemStats(txApp)
+		if err != nil {
+			return err
+		}
+		err = deleteOldAlertsHistory(txApp, 200, 250)
+		if err != nil {
+			return err
+		}
+		return nil
+	})
+}
+
+// Delete old alerts history records
+func deleteOldAlertsHistory(app core.App, countToKeep, countBeforeDeletion int) error {
+	db := app.DB()
+	var users []struct {
+		Id string `db:"user"`
 	}
-	db := rm.app.Dao().NonconcurrentDB()
-	for _, recordData := range recordData {
-		for _, collectionSlug := range collections {
-			formattedDate := time.Now().UTC().Add(-recordData.retention).Format(types.DefaultDateLayout)
-			expr := dbx.NewExp("[[created]] < {:date} AND [[type]] = {:type}", dbx.Params{"date": formattedDate, "type": recordData.recordType})
-			_, err := db.Delete(collectionSlug, expr).Execute()
-			if err != nil {
-				rm.app.Logger().Error("Failed to delete records", "err", err.Error())
-			}
+	err := db.NewQuery("SELECT user, COUNT(*) as count FROM alerts_history GROUP BY user HAVING count > {:countBeforeDeletion}").Bind(dbx.Params{"countBeforeDeletion": countBeforeDeletion}).All(&users)
+	if err != nil {
+		return err
+	}
+	for _, user := range users {
+		_, err = db.NewQuery("DELETE FROM alerts_history WHERE user = {:user} AND id NOT IN (SELECT id FROM alerts_history WHERE user = {:user} ORDER BY created DESC LIMIT {:countToKeep})").Bind(dbx.Params{"user": user.Id, "countToKeep": countToKeep}).Execute()
+		if err != nil {
+			return err
 		}
 	}
+	return nil
+}
+
+// Deletes system_stats records older than what is displayed in the UI
+func deleteOldSystemStats(app core.App) error {
+	// Collections to process
+	collections := [2]string{"system_stats", "container_stats"}
+
+	// Record types and their retention periods
+	type RecordDeletionData struct {
+		recordType string
+		retention  time.Duration
+	}
+	recordData := []RecordDeletionData{
+		{recordType: "1m", retention: time.Hour},             // 1 hour
+		{recordType: "10m", retention: 12 * time.Hour},       // 12 hours
+		{recordType: "20m", retention: 24 * time.Hour},       // 1 day
+		{recordType: "120m", retention: 7 * 24 * time.Hour},  // 7 days
+		{recordType: "480m", retention: 30 * 24 * time.Hour}, // 30 days
+	}
+
+	now := time.Now().UTC()
+
+	for _, collection := range collections {
+		// Build the WHERE clause
+		var conditionParts []string
+		var params dbx.Params = make(map[string]any)
+		for i := range recordData {
+			rd := recordData[i]
+			// Create parameterized condition for this record type
+			dateParam := fmt.Sprintf("date%d", i)
+			conditionParts = append(conditionParts, fmt.Sprintf("(type = '%s' AND created < {:%s})", rd.recordType, dateParam))
+			params[dateParam] = now.Add(-rd.retention)
+		}
+		// Combine conditions with OR
+		conditionStr := strings.Join(conditionParts, " OR ")
+		// Construct and execute the full raw query
+		rawQuery := fmt.Sprintf("DELETE FROM %s WHERE %s", collection, conditionStr)
+		if _, err := app.DB().NewQuery(rawQuery).Bind(params).Execute(); err != nil {
+			return fmt.Errorf("failed to delete from %s: %v", collection, err)
+		}
+	}
+	return nil
 }
 
 /* Round float to two decimals */

beszel/internal/records/records_test.go

@@ -0,0 +1,381 @@
+//go:build testing
+// +build testing
+
+package records_test
+
+import (
+	"beszel/internal/records"
+	"beszel/internal/tests"
+	"fmt"
+	"testing"
+	"time"
+
+	"github.com/pocketbase/dbx"
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tools/types"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+// TestDeleteOldRecords tests the main DeleteOldRecords function
+func TestDeleteOldRecords(t *testing.T) {
+	hub, err := tests.NewTestHub(t.TempDir())
+	require.NoError(t, err)
+	defer hub.Cleanup()
+
+	rm := records.NewRecordManager(hub)
+
+	// Create test user for alerts history
+	user, err := tests.CreateUser(hub, "test@example.com", "testtesttest")
+	require.NoError(t, err)
+
+	// Create test system
+	system, err := tests.CreateRecord(hub, "systems", map[string]any{
+		"name":   "test-system",
+		"host":   "localhost",
+		"port":   "45876",
+		"status": "up",
+		"users":  []string{user.Id},
+	})
+	require.NoError(t, err)
+
+	now := time.Now()
+
+	// Create old system_stats records that should be deleted
+	var record *core.Record
+	record, err = tests.CreateRecord(hub, "system_stats", map[string]any{
+		"system": system.Id,
+		"type":   "1m",
+		"stats":  `{"cpu": 50.0, "mem": 1024}`,
+	})
+	require.NoError(t, err)
+	// created is autodate field, so we need to set it manually
+	record.SetRaw("created", now.UTC().Add(-2*time.Hour).Format(types.DefaultDateLayout))
+	err = hub.SaveNoValidate(record)
+	require.NoError(t, err)
+	require.NotNil(t, record)
+	require.InDelta(t, record.GetDateTime("created").Time().UTC().Unix(), now.UTC().Add(-2*time.Hour).Unix(), 1)
+	require.Equal(t, record.Get("system"), system.Id)
+	require.Equal(t, record.Get("type"), "1m")
+
+	// Create recent system_stats record that should be kept
+	_, err = tests.CreateRecord(hub, "system_stats", map[string]any{
+		"system":  system.Id,
+		"type":    "1m",
+		"stats":   `{"cpu": 30.0, "mem": 512}`,
+		"created": now.Add(-30 * time.Minute), // 30 minutes old, should be kept
+	})
+	require.NoError(t, err)
+
+	// Create many alerts history records to trigger deletion
+	for i := range 260 { // More than countBeforeDeletion (250)
+		_, err = tests.CreateRecord(hub, "alerts_history", map[string]any{
+			"user":    user.Id,
+			"name":    "CPU",
+			"value":   i + 1,
+			"system":  system.Id,
+			"created": now.Add(-time.Duration(i) * time.Minute),
+		})
+		require.NoError(t, err)
+	}
+
+	// Count records before deletion
+	systemStatsCountBefore, err := hub.CountRecords("system_stats")
+	require.NoError(t, err)
+	alertsCountBefore, err := hub.CountRecords("alerts_history")
+	require.NoError(t, err)
+
+	// Run deletion
+	rm.DeleteOldRecords()
+
+	// Count records after deletion
+	systemStatsCountAfter, err := hub.CountRecords("system_stats")
+	require.NoError(t, err)
+	alertsCountAfter, err := hub.CountRecords("alerts_history")
+	require.NoError(t, err)
+
+	// Verify old system stats were deleted
+	assert.Less(t, systemStatsCountAfter, systemStatsCountBefore, "Old system stats should be deleted")
+
+	// Verify alerts history was trimmed
+	assert.Less(t, alertsCountAfter, alertsCountBefore, "Excessive alerts history should be deleted")
+	assert.Equal(t, alertsCountAfter, int64(200), "Alerts count should be equal to countToKeep (200)")
+}
+
+// TestDeleteOldSystemStats tests the deleteOldSystemStats function
+func TestDeleteOldSystemStats(t *testing.T) {
+	hub, err := tests.NewTestHub(t.TempDir())
+	require.NoError(t, err)
+	defer hub.Cleanup()
+
+	// Create test system
+	user, err := tests.CreateUser(hub, "test@example.com", "testtesttest")
+	require.NoError(t, err)
+
+	system, err := tests.CreateRecord(hub, "systems", map[string]any{
+		"name":   "test-system",
+		"host":   "localhost",
+		"port":   "45876",
+		"status": "up",
+		"users":  []string{user.Id},
+	})
+	require.NoError(t, err)
+
+	now := time.Now().UTC()
+
+	// Test data for different record types and their retention periods
+	testCases := []struct {
+		recordType   string
+		retention    time.Duration
+		shouldBeKept bool
+		ageFromNow   time.Duration
+		description  string
+	}{
+		{"1m", time.Hour, true, 30 * time.Minute, "1m record within 1 hour should be kept"},
+		{"1m", time.Hour, false, 2 * time.Hour, "1m record older than 1 hour should be deleted"},
+		{"10m", 12 * time.Hour, true, 6 * time.Hour, "10m record within 12 hours should be kept"},
+		{"10m", 12 * time.Hour, false, 24 * time.Hour, "10m record older than 12 hours should be deleted"},
+		{"20m", 24 * time.Hour, true, 12 * time.Hour, "20m record within 24 hours should be kept"},
+		{"20m", 24 * time.Hour, false, 48 * time.Hour, "20m record older than 24 hours should be deleted"},
+		{"120m", 7 * 24 * time.Hour, true, 3 * 24 * time.Hour, "120m record within 7 days should be kept"},
+		{"120m", 7 * 24 * time.Hour, false, 10 * 24 * time.Hour, "120m record older than 7 days should be deleted"},
+		{"480m", 30 * 24 * time.Hour, true, 15 * 24 * time.Hour, "480m record within 30 days should be kept"},
+		{"480m", 30 * 24 * time.Hour, false, 45 * 24 * time.Hour, "480m record older than 30 days should be deleted"},
+	}
+
+	// Create test records for both system_stats and container_stats
+	collections := []string{"system_stats", "container_stats"}
+	recordIds := make(map[string][]string)
+
+	for _, collection := range collections {
+		recordIds[collection] = make([]string, 0)
+
+		for i, tc := range testCases {
+			recordTime := now.Add(-tc.ageFromNow)
+
+			var stats string
+			if collection == "system_stats" {
+				stats = fmt.Sprintf(`{"cpu": %d.0, "mem": %d}`, i*10, i*100)
+			} else {
+				stats = fmt.Sprintf(`[{"name": "container%d", "cpu": %d.0, "mem": %d}]`, i, i*5, i*50)
+			}
+
+			record, err := tests.CreateRecord(hub, collection, map[string]any{
+				"system": system.Id,
+				"type":   tc.recordType,
+				"stats":  stats,
+			})
+			require.NoError(t, err)
+			record.SetRaw("created", recordTime.Format(types.DefaultDateLayout))
+			err = hub.SaveNoValidate(record)
+			require.NoError(t, err)
+			recordIds[collection] = append(recordIds[collection], record.Id)
+		}
+	}
+
+	// Run deletion
+	err = records.TestDeleteOldSystemStats(hub)
+	require.NoError(t, err)
+
+	// Verify results
+	for _, collection := range collections {
+		for i, tc := range testCases {
+			recordId := recordIds[collection][i]
+
+			// Try to find the record
+			_, err := hub.FindRecordById(collection, recordId)
+
+			if tc.shouldBeKept {
+				assert.NoError(t, err, "Record should exist: %s", tc.description)
+			} else {
+				assert.Error(t, err, "Record should be deleted: %s", tc.description)
+			}
+		}
+	}
+}
+
+// TestDeleteOldAlertsHistory tests the deleteOldAlertsHistory function
+func TestDeleteOldAlertsHistory(t *testing.T) {
+	hub, err := tests.NewTestHub(t.TempDir())
+	require.NoError(t, err)
+	defer hub.Cleanup()
+
+	// Create test users
+	user1, err := tests.CreateUser(hub, "user1@example.com", "testtesttest")
+	require.NoError(t, err)
+
+	user2, err := tests.CreateUser(hub, "user2@example.com", "testtesttest")
+	require.NoError(t, err)
+
+	system, err := tests.CreateRecord(hub, "systems", map[string]any{
+		"name":   "test-system",
+		"host":   "localhost",
+		"port":   "45876",
+		"status": "up",
+		"users":  []string{user1.Id, user2.Id},
+	})
+	require.NoError(t, err)
+	now := time.Now().UTC()
+
+	testCases := []struct {
+		name                  string
+		user                  *core.Record
+		alertCount            int
+		countToKeep           int
+		countBeforeDeletion   int
+		expectedAfterDeletion int
+		description           string
+	}{
+		{
+			name:                  "User with few alerts (below threshold)",
+			user:                  user1,
+			alertCount:            100,
+			countToKeep:           50,
+			countBeforeDeletion:   150,
+			expectedAfterDeletion: 100, // No deletion because below threshold
+			description:           "User with alerts below countBeforeDeletion should not have any deleted",
+		},
+		{
+			name:                  "User with many alerts (above threshold)",
+			user:                  user2,
+			alertCount:            300,
+			countToKeep:           100,
+			countBeforeDeletion:   200,
+			expectedAfterDeletion: 100, // Should be trimmed to countToKeep
+			description:           "User with alerts above countBeforeDeletion should be trimmed to countToKeep",
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			// Create alerts for this user
+			for i := 0; i < tc.alertCount; i++ {
+				_, err := tests.CreateRecord(hub, "alerts_history", map[string]any{
+					"user":    tc.user.Id,
+					"name":    "CPU",
+					"value":   i + 1,
+					"system":  system.Id,
+					"created": now.Add(-time.Duration(i) * time.Minute),
+				})
+				require.NoError(t, err)
+			}
+
+			// Count before deletion
+			countBefore, err := hub.CountRecords("alerts_history",
+				dbx.NewExp("user = {:user}", dbx.Params{"user": tc.user.Id}))
+			require.NoError(t, err)
+			assert.Equal(t, int64(tc.alertCount), countBefore, "Initial count should match")
+
+			// Run deletion
+			err = records.TestDeleteOldAlertsHistory(hub, tc.countToKeep, tc.countBeforeDeletion)
+			require.NoError(t, err)
+
+			// Count after deletion
+			countAfter, err := hub.CountRecords("alerts_history",
+				dbx.NewExp("user = {:user}", dbx.Params{"user": tc.user.Id}))
+			require.NoError(t, err)
+
+			assert.Equal(t, int64(tc.expectedAfterDeletion), countAfter, tc.description)
+
+			// If deletion occurred, verify the most recent records were kept
+			if tc.expectedAfterDeletion < tc.alertCount {
+				records, err := hub.FindRecordsByFilter("alerts_history",
+					"user = {:user}",
+					"-created", // Order by created DESC
+					tc.countToKeep,
+					0,
+					map[string]any{"user": tc.user.Id})
+				require.NoError(t, err)
+				assert.Len(t, records, tc.expectedAfterDeletion, "Should have exactly countToKeep records")
+
+				// Verify records are in descending order by created time
+				for i := 1; i < len(records); i++ {
+					prev := records[i-1].GetDateTime("created").Time()
+					curr := records[i].GetDateTime("created").Time()
+					assert.True(t, prev.After(curr) || prev.Equal(curr),
+						"Records should be ordered by created time (newest first)")
+				}
+			}
+		})
+	}
+}
+
+// TestDeleteOldAlertsHistoryEdgeCases tests edge cases for alerts history deletion
+func TestDeleteOldAlertsHistoryEdgeCases(t *testing.T) {
+	hub, err := tests.NewTestHub(t.TempDir())
+	require.NoError(t, err)
+	defer hub.Cleanup()
+
+	t.Run("No users with excessive alerts", func(t *testing.T) {
+		// Create user with few alerts
+		user, err := tests.CreateUser(hub, "few@example.com", "testtesttest")
+		require.NoError(t, err)
+
+		system, err := tests.CreateRecord(hub, "systems", map[string]any{
+			"name":   "test-system",
+			"host":   "localhost",
+			"port":   "45876",
+			"status": "up",
+			"users":  []string{user.Id},
+		})
+
+		// Create only 5 alerts (well below threshold)
+		for i := range 5 {
+			_, err := tests.CreateRecord(hub, "alerts_history", map[string]any{
+				"user":   user.Id,
+				"name":   "CPU",
+				"value":  i + 1,
+				"system": system.Id,
+			})
+			require.NoError(t, err)
+		}
+
+		// Should not error and should not delete anything
+		err = records.TestDeleteOldAlertsHistory(hub, 10, 20)
+		require.NoError(t, err)
+
+		count, err := hub.CountRecords("alerts_history")
+		require.NoError(t, err)
+		assert.Equal(t, int64(5), count, "All alerts should remain")
+	})
+
+	t.Run("Empty alerts_history table", func(t *testing.T) {
+		// Clear any existing alerts
+		_, err := hub.DB().NewQuery("DELETE FROM alerts_history").Execute()
+		require.NoError(t, err)
+
+		// Should not error with empty table
+		err = records.TestDeleteOldAlertsHistory(hub, 10, 20)
+		require.NoError(t, err)
+	})
+}
+
+// TestRecordManagerCreation tests RecordManager creation
+func TestRecordManagerCreation(t *testing.T) {
+	hub, err := tests.NewTestHub(t.TempDir())
+	require.NoError(t, err)
+	defer hub.Cleanup()
+
+	rm := records.NewRecordManager(hub)
+	assert.NotNil(t, rm, "RecordManager should not be nil")
+}
+
+// TestTwoDecimals tests the twoDecimals helper function
+func TestTwoDecimals(t *testing.T) {
+	testCases := []struct {
+		input    float64
+		expected float64
+	}{
+		{1.234567, 1.23},
+		{1.235, 1.24}, // Should round up
+		{1.0, 1.0},
+		{0.0, 0.0},
+		{-1.234567, -1.23},
+		{-1.235, -1.23}, // Negative rounding
+	}
+
+	for _, tc := range testCases {
+		result := records.TestTwoDecimals(tc.input)
+		assert.InDelta(t, tc.expected, result, 0.02, "twoDecimals(%f) should equal %f", tc.input, tc.expected)
+	}
+}

beszel/internal/records/records_test_helpers.go

@@ -0,0 +1,23 @@
+//go:build testing
+// +build testing
+
+package records
+
+import (
+	"github.com/pocketbase/pocketbase/core"
+)
+
+// TestDeleteOldSystemStats exposes deleteOldSystemStats for testing
+func TestDeleteOldSystemStats(app core.App) error {
+	return deleteOldSystemStats(app)
+}
+
+// TestDeleteOldAlertsHistory exposes deleteOldAlertsHistory for testing
+func TestDeleteOldAlertsHistory(app core.App, countToKeep, countBeforeDeletion int) error {
+	return deleteOldAlertsHistory(app, countToKeep, countBeforeDeletion)
+}
+
+// TestTwoDecimals exposes twoDecimals for testing
+func TestTwoDecimals(value float64) float64 {
+	return twoDecimals(value)
+}

beszel/internal/tests/hub.go

@@ -0,0 +1,88 @@
+//go:build testing
+// +build testing
+
+// Package tests provides helpers for testing the application.
+package tests
+
+import (
+	"beszel/internal/hub"
+
+	"github.com/pocketbase/pocketbase/core"
+	"github.com/pocketbase/pocketbase/tests"
+
+	_ "github.com/pocketbase/pocketbase/migrations"
+)
+
+// TestHub is a wrapper hub instance used for testing.
+type TestHub struct {
+	core.App
+	*tests.TestApp
+	*hub.Hub
+}
+
+// NewTestHub creates and initializes a test application instance.
+//
+// It is the caller's responsibility to call app.Cleanup() when the app is no longer needed.
+func NewTestHub(optTestDataDir ...string) (*TestHub, error) {
+	var testDataDir string
+	if len(optTestDataDir) > 0 {
+		testDataDir = optTestDataDir[0]
+	}
+
+	return NewTestHubWithConfig(core.BaseAppConfig{
+		DataDir:       testDataDir,
+		EncryptionEnv: "pb_test_env",
+	})
+}
+
+// NewTestHubWithConfig creates and initializes a test application instance
+// from the provided config.
+//
+// If config.DataDir is not set it fallbacks to the default internal test data directory.
+//
+// config.DataDir is cloned for each new test application instance.
+//
+// It is the caller's responsibility to call app.Cleanup() when the app is no longer needed.
+func NewTestHubWithConfig(config core.BaseAppConfig) (*TestHub, error) {
+	testApp, err := tests.NewTestAppWithConfig(config)
+	if err != nil {
+		return nil, err
+	}
+
+	hub := hub.NewHub(testApp)
+
+	t := &TestHub{
+		App:     testApp,
+		TestApp: testApp,
+		Hub:     hub,
+	}
+
+	return t, nil
+}
+
+// Helper function to create a test user for config tests
+func CreateUser(app core.App, email string, password string) (*core.Record, error) {
+	userCollection, err := app.FindCachedCollectionByNameOrId("users")
+	if err != nil {
+		return nil, err
+	}
+
+	user := core.NewRecord(userCollection)
+	user.Set("email", email)
+	user.Set("password", password)
+
+	return user, app.Save(user)
+}
+
+// Helper function to create a test record
+func CreateRecord(app core.App, collectionName string, fields map[string]any) (*core.Record, error) {
+	collection, err := app.FindCachedCollectionByNameOrId(collectionName)
+	if err != nil {
+		return nil, err
+	}
+
+	record := core.NewRecord(collection)
+	record.Load(fields)
+
+	return record, app.Save(record)
+}

beszel/internal/users/users.go

@@ -2,43 +2,48 @@
 package users
 
 import (
+	"beszel/migrations"
 	"log"
+	"net/http"
 
-	"github.com/pocketbase/pocketbase"
 	"github.com/pocketbase/pocketbase/core"
-	"github.com/pocketbase/pocketbase/models"
 )
 
 type UserManager struct {
-	app *pocketbase.PocketBase
+	app core.App
 }
 
 type UserSettings struct {
 	ChartTime            string   `json:"chartTime"`
 	NotificationEmails   []string `json:"emails"`
 	NotificationWebhooks []string `json:"webhooks"`
-	// Language             string   `json:"lang"`
+	// UnitTemp             uint8    `json:"unitTemp"` // 0 for Celsius, 1 for Fahrenheit
+	// UnitNet              uint8    `json:"unitNet"`  // 0 for bytes, 1 for bits
+	// UnitDisk             uint8    `json:"unitDisk"` // 0 for bytes, 1 for bits
+
+	// New field for alert history retention (e.g., "1m", "3m", "6m", "1y")
+	AlertHistoryRetention string `json:"alertHistoryRetention,omitempty"`
 }
 
-func NewUserManager(app *pocketbase.PocketBase) *UserManager {
+func NewUserManager(app core.App) *UserManager {
 	return &UserManager{
 		app: app,
 	}
 }
 
-func (um *UserManager) InitializeUserRole(e *core.ModelEvent) error {
-	user := e.Model.(*models.Record)
-	if user.GetString("role") == "" {
-		user.Set("role", "user")
+// Initialize user role if not set
+func (um *UserManager) InitializeUserRole(e *core.RecordEvent) error {
+	if e.Record.GetString("role") == "" {
+		e.Record.Set("role", "user")
 	}
-	return nil
+	return e.Next()
 }
 
-func (um *UserManager) InitializeUserSettings(e *core.ModelEvent) error {
-	record := e.Model.(*models.Record)
+// Initialize user settings with defaults if not set
+func (um *UserManager) InitializeUserSettings(e *core.RecordEvent) error {
+	record := e.Record
 	// intialize settings with defaults
 	settings := UserSettings{
-		// Language:             "en",
 		ChartTime:            "1h",
 		NotificationEmails:   []string{},
 		NotificationWebhooks: []string{},
@@ -46,7 +51,7 @@ func (um *UserManager) InitializeUserSettings(e *core.ModelEvent) error {
 	record.UnmarshalJSONField("settings", &settings)
 	if len(settings.NotificationEmails) == 0 {
 		// get user email from auth record
-		if errs := um.app.Dao().ExpandRecord(record, []string{"user"}, nil); len(errs) == 0 {
+		if errs := um.app.ExpandRecord(record, []string{"user"}, nil); len(errs) == 0 {
 			// app.Logger().Error("failed to expand user relation", "errs", errs)
 			if user := record.ExpandedOne("user"); user != nil {
 				settings.NotificationEmails = []string{user.GetString("email")}
@@ -61,5 +66,54 @@ func (um *UserManager) InitializeUserSettings(e *core.ModelEvent) error {
 	// 	settings.NotificationWebhooks = []string{""}
 	// }
 	record.Set("settings", settings)
-	return nil
+	return e.Next()
+}
+
+// Custom API endpoint to create the first user.
+// Mimics previous default behavior in PocketBase < 0.23.0 allowing user to be created through the Beszel UI.
+func (um *UserManager) CreateFirstUser(e *core.RequestEvent) error {
+	// check that there are no users
+	totalUsers, err := um.app.CountRecords("users")
+	if err != nil || totalUsers > 0 {
+		return e.JSON(http.StatusForbidden, map[string]string{"err": "Forbidden"})
+	}
+	// check that there is only one superuser and the email matches the email of the superuser we set up in initial-settings.go
+	adminUsers, err := um.app.FindAllRecords(core.CollectionNameSuperusers)
+	if err != nil || len(adminUsers) != 1 || adminUsers[0].GetString("email") != migrations.TempAdminEmail {
+		return e.JSON(http.StatusForbidden, map[string]string{"err": "Forbidden"})
+	}
+	// create first user using supplied email and password in request body
+	data := struct {
+		Email    string `json:"email"`
+		Password string `json:"password"`
+	}{}
+	if err := e.BindBody(&data); err != nil {
+		return e.JSON(http.StatusBadRequest, map[string]string{"err": err.Error()})
+	}
+	if data.Email == "" || data.Password == "" {
+		return e.JSON(http.StatusBadRequest, map[string]string{"err": "Bad request"})
+	}
+
+	collection, _ := um.app.FindCollectionByNameOrId("users")
+	user := core.NewRecord(collection)
+	user.SetEmail(data.Email)
+	user.SetPassword(data.Password)
+	user.Set("role", "admin")
+	user.Set("verified", true)
+	if err := um.app.Save(user); err != nil {
+		return e.JSON(http.StatusInternalServerError, map[string]string{"err": err.Error()})
+	}
+	// create superuser using the email of the first user
+	collection, _ = um.app.FindCollectionByNameOrId(core.CollectionNameSuperusers)
+	adminUser := core.NewRecord(collection)
+	adminUser.SetEmail(data.Email)
+	adminUser.SetPassword(data.Password)
+	if err := um.app.Save(adminUser); err != nil {
+		return e.JSON(http.StatusInternalServerError, map[string]string{"err": err.Error()})
+	}
+	// delete the intial superuser
+	if err := um.app.Delete(adminUsers[0]); err != nil {
+		return e.JSON(http.StatusInternalServerError, map[string]string{"err": err.Error()})
+	}
+	return e.JSON(http.StatusOK, map[string]string{"msg": "User created"})
 }

beszel/migrations/0_collections_snapshot_0_12_0_7.go

@@ -0,0 +1,900 @@
+package migrations
+
+import (
+	"github.com/google/uuid"
+	"github.com/pocketbase/pocketbase/core"
+	m "github.com/pocketbase/pocketbase/migrations"
+)
+
+func init() {
+	m.Register(func(app core.App) error {
+		// update collections
+		jsonData := `[
+	{
+		"id": "elngm8x1l60zi2v",
+		"listRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"viewRule": "",
+		"createRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"updateRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"deleteRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"name": "alerts",
+		"type": "base",
+		"fields": [
+			{
+				"autogeneratePattern": "[a-z0-9]{15}",
+				"hidden": false,
+				"id": "text3208210256",
+				"max": 15,
+				"min": 15,
+				"name": "id",
+				"pattern": "^[a-z0-9]+$",
+				"presentable": false,
+				"primaryKey": true,
+				"required": true,
+				"system": true,
+				"type": "text"
+			},
+			{
+				"cascadeDelete": true,
+				"collectionId": "_pb_users_auth_",
+				"hidden": false,
+				"id": "hn5ly3vi",
+				"maxSelect": 1,
+				"minSelect": 0,
+				"name": "user",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "relation"
+			},
+			{
+				"cascadeDelete": true,
+				"collectionId": "2hz5ncl8tizk5nx",
+				"hidden": false,
+				"id": "g5sl3jdg",
+				"maxSelect": 1,
+				"minSelect": 0,
+				"name": "system",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "relation"
+			},
+			{
+				"hidden": false,
+				"id": "zj3ingrv",
+				"maxSelect": 1,
+				"name": "name",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "select",
+				"values": [
+					"Status",
+					"CPU",
+					"Memory",
+					"Disk",
+					"Temperature",
+					"Bandwidth",
+					"LoadAvg1",
+					"LoadAvg5",
+					"LoadAvg15"
+				]
+			},
+			{
+				"hidden": false,
+				"id": "o2ablxvn",
+				"max": null,
+				"min": null,
+				"name": "value",
+				"onlyInt": false,
+				"presentable": false,
+				"required": false,
+				"system": false,
+				"type": "number"
+			},
+			{
+				"hidden": false,
+				"id": "fstdehcq",
+				"max": 60,
+				"min": null,
+				"name": "min",
+				"onlyInt": true,
+				"presentable": false,
+				"required": false,
+				"system": false,
+				"type": "number"
+			},
+			{
+				"hidden": false,
+				"id": "6hgdf6hs",
+				"name": "triggered",
+				"presentable": false,
+				"required": false,
+				"system": false,
+				"type": "bool"
+			},
+			{
+				"hidden": false,
+				"id": "autodate2990389176",
+				"name": "created",
+				"onCreate": true,
+				"onUpdate": false,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			},
+			{
+				"hidden": false,
+				"id": "autodate3332085495",
+				"name": "updated",
+				"onCreate": true,
+				"onUpdate": true,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			}
+		],
+		"indexes": [
+			"CREATE UNIQUE INDEX ` + "`" + `idx_MnhEt21L5r` + "`" + ` ON ` + "`" + `alerts` + "`" + ` (\n  ` + "`" + `user` + "`" + `,\n  ` + "`" + `system` + "`" + `,\n  ` + "`" + `name` + "`" + `\n)"
+		],
+		"system": false
+	},
+	{
+		"id": "pbc_1697146157",
+		"listRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"viewRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"createRule": null,
+		"updateRule": null,
+		"deleteRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"name": "alerts_history",
+		"type": "base",
+		"fields": [
+			{
+					"autogeneratePattern": "[a-z0-9]{15}",
+					"hidden": false,
+					"id": "text3208210256",
+					"max": 15,
+					"min": 15,
+					"name": "id",
+					"pattern": "^[a-z0-9]+$",
+					"presentable": false,
+					"primaryKey": true,
+					"required": true,
+					"system": true,
+					"type": "text"
+				},
+				{
+					"cascadeDelete": true,
+					"collectionId": "_pb_users_auth_",
+					"hidden": false,
+					"id": "relation2375276105",
+					"maxSelect": 1,
+					"minSelect": 0,
+					"name": "user",
+					"presentable": false,
+					"required": true,
+					"system": false,
+					"type": "relation"
+				},
+				{
+					"cascadeDelete": true,
+					"collectionId": "2hz5ncl8tizk5nx",
+					"hidden": false,
+					"id": "relation3377271179",
+					"maxSelect": 1,
+					"minSelect": 0,
+					"name": "system",
+					"presentable": false,
+					"required": true,
+					"system": false,
+					"type": "relation"
+				},
+				{
+					"autogeneratePattern": "",
+					"hidden": false,
+					"id": "text2466471794",
+					"max": 0,
+					"min": 0,
+					"name": "alert_id",
+					"pattern": "",
+					"presentable": false,
+					"primaryKey": false,
+					"required": false,
+					"system": false,
+					"type": "text"
+				},
+				{
+					"autogeneratePattern": "",
+					"hidden": false,
+					"id": "text1579384326",
+					"max": 0,
+					"min": 0,
+					"name": "name",
+					"pattern": "",
+					"presentable": false,
+					"primaryKey": false,
+					"required": true,
+					"system": false,
+					"type": "text"
+				},
+				{
+					"hidden": false,
+					"id": "number494360628",
+					"max": null,
+					"min": null,
+					"name": "value",
+					"onlyInt": false,
+					"presentable": false,
+					"required": false,
+					"system": false,
+					"type": "number"
+				},
+				{
+					"hidden": false,
+					"id": "autodate2990389176",
+					"name": "created",
+					"onCreate": true,
+					"onUpdate": false,
+					"presentable": false,
+					"system": false,
+					"type": "autodate"
+				},
+				{
+					"hidden": false,
+					"id": "date2276568630",
+					"max": "",
+					"min": "",
+					"name": "resolved",
+					"presentable": false,
+					"required": false,
+					"system": false,
+					"type": "date"
+				}
+		],
+		"indexes": [
+			"CREATE INDEX ` + "`" + `idx_YdGnup5aqB` + "`" + ` ON ` + "`" + `alerts_history` + "`" + ` (` + "`" + `user` + "`" + `)",
+			"CREATE INDEX ` + "`" + `idx_taLet9VdME` + "`" + ` ON ` + "`" + `alerts_history` + "`" + ` (` + "`" + `created` + "`" + `)"
+		],
+		"system": false
+	},
+	{
+		"id": "juohu4jipgc13v7",
+		"listRule": "@request.auth.id != \"\"",
+		"viewRule": null,
+		"createRule": null,
+		"updateRule": null,
+		"deleteRule": null,
+		"name": "container_stats",
+		"type": "base",
+		"fields": [
+			{
+				"autogeneratePattern": "[a-z0-9]{15}",
+				"hidden": false,
+				"id": "text3208210256",
+				"max": 15,
+				"min": 15,
+				"name": "id",
+				"pattern": "^[a-z0-9]+$",
+				"presentable": false,
+				"primaryKey": true,
+				"required": true,
+				"system": true,
+				"type": "text"
+			},
+			{
+				"cascadeDelete": true,
+				"collectionId": "2hz5ncl8tizk5nx",
+				"hidden": false,
+				"id": "hutcu6ps",
+				"maxSelect": 1,
+				"minSelect": 0,
+				"name": "system",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "relation"
+			},
+			{
+				"hidden": false,
+				"id": "r39hhnil",
+				"maxSize": 2000000,
+				"name": "stats",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "json"
+			},
+			{
+				"hidden": false,
+				"id": "vo7iuj96",
+				"maxSelect": 1,
+				"name": "type",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "select",
+				"values": [
+					"1m",
+					"10m",
+					"20m",
+					"120m",
+					"480m"
+				]
+			},
+			{
+				"hidden": false,
+				"id": "autodate2990389176",
+				"name": "created",
+				"onCreate": true,
+				"onUpdate": false,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			},
+			{
+				"hidden": false,
+				"id": "autodate3332085495",
+				"name": "updated",
+				"onCreate": true,
+				"onUpdate": true,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			}
+		],
+		"indexes": [
+			"CREATE INDEX ` + "`" + `idx_d87OiXGZD8` + "`" + ` ON ` + "`" + `container_stats` + "`" + ` (\n  ` + "`" + `system` + "`" + `,\n  ` + "`" + `type` + "`" + `,\n  ` + "`" + `created` + "`" + `\n)"
+		],
+		"system": false
+	},
+	{
+		"id": "pbc_3663931638",
+		"listRule": "@request.auth.id != \"\" && system.users.id ?= @request.auth.id",
+		"viewRule": "@request.auth.id != \"\" && system.users.id ?= @request.auth.id",
+		"createRule": "@request.auth.id != \"\" && system.users.id ?= @request.auth.id && @request.auth.role != \"readonly\"",
+		"updateRule": "@request.auth.id != \"\" && system.users.id ?= @request.auth.id && @request.auth.role != \"readonly\"",
+		"deleteRule": null,
+		"name": "fingerprints",
+		"type": "base",
+		"fields": [
+			{
+				"autogeneratePattern": "[a-z0-9]{9}",
+				"hidden": false,
+				"id": "text3208210256",
+				"max": 15,
+				"min": 9,
+				"name": "id",
+				"pattern": "^[a-z0-9]+$",
+				"presentable": false,
+				"primaryKey": true,
+				"required": true,
+				"system": true,
+				"type": "text"
+			},
+			{
+				"cascadeDelete": true,
+				"collectionId": "2hz5ncl8tizk5nx",
+				"hidden": false,
+				"id": "relation3377271179",
+				"maxSelect": 1,
+				"minSelect": 0,
+				"name": "system",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "relation"
+			},
+			{
+				"autogeneratePattern": "[a-zA-Z9-9]{20}",
+				"hidden": false,
+				"id": "text1597481275",
+				"max": 255,
+				"min": 9,
+				"name": "token",
+				"pattern": "",
+				"presentable": false,
+				"primaryKey": false,
+				"required": true,
+				"system": false,
+				"type": "text"
+			},
+			{
+				"autogeneratePattern": "",
+				"hidden": false,
+				"id": "text4228609354",
+				"max": 255,
+				"min": 9,
+				"name": "fingerprint",
+				"pattern": "",
+				"presentable": false,
+				"primaryKey": false,
+				"required": false,
+				"system": false,
+				"type": "text"
+			},
+			{
+				"hidden": false,
+				"id": "autodate3332085495",
+				"name": "updated",
+				"onCreate": true,
+				"onUpdate": true,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			}
+		],
+		"indexes": [
+			"CREATE INDEX ` + "`" + `idx_p9qZlu26po` + "`" + ` ON ` + "`" + `fingerprints` + "`" + ` (` + "`" + `token` + "`" + `)",
+			"CREATE UNIQUE INDEX ` + "`" + `idx_ngboulGMYw` + "`" + ` ON ` + "`" + `fingerprints` + "`" + ` (` + "`" + `system` + "`" + `)"
+		],
+		"system": false
+	},
+	{
+		"id": "ej9oowivz8b2mht",
+		"listRule": "@request.auth.id != \"\"",
+		"viewRule": null,
+		"createRule": null,
+		"updateRule": null,
+		"deleteRule": null,
+		"name": "system_stats",
+		"type": "base",
+		"fields": [
+			{
+				"autogeneratePattern": "[a-z0-9]{15}",
+				"hidden": false,
+				"id": "text3208210256",
+				"max": 15,
+				"min": 15,
+				"name": "id",
+				"pattern": "^[a-z0-9]+$",
+				"presentable": false,
+				"primaryKey": true,
+				"required": true,
+				"system": true,
+				"type": "text"
+			},
+			{
+				"cascadeDelete": true,
+				"collectionId": "2hz5ncl8tizk5nx",
+				"hidden": false,
+				"id": "h9sg148r",
+				"maxSelect": 1,
+				"minSelect": 0,
+				"name": "system",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "relation"
+			},
+			{
+				"hidden": false,
+				"id": "azftn0be",
+				"maxSize": 2000000,
+				"name": "stats",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "json"
+			},
+			{
+				"hidden": false,
+				"id": "m1ekhli3",
+				"maxSelect": 1,
+				"name": "type",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "select",
+				"values": [
+					"1m",
+					"10m",
+					"20m",
+					"120m",
+					"480m"
+				]
+			},
+			{
+				"hidden": false,
+				"id": "autodate2990389176",
+				"name": "created",
+				"onCreate": true,
+				"onUpdate": false,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			},
+			{
+				"hidden": false,
+				"id": "autodate3332085495",
+				"name": "updated",
+				"onCreate": true,
+				"onUpdate": true,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			}
+		],
+		"indexes": [
+			"CREATE INDEX ` + "`" + `idx_GxIee0j` + "`" + ` ON ` + "`" + `system_stats` + "`" + ` (\n  ` + "`" + `system` + "`" + `,\n  ` + "`" + `type` + "`" + `,\n  ` + "`" + `created` + "`" + `\n)"
+		],
+		"system": false
+	},
+	{
+		"id": "4afacsdnlu8q8r2",
+		"listRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"viewRule": null,
+		"createRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"updateRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
+		"deleteRule": null,
+		"name": "user_settings",
+		"type": "base",
+		"fields": [
+			{
+				"autogeneratePattern": "[a-z0-9]{15}",
+				"hidden": false,
+				"id": "text3208210256",
+				"max": 15,
+				"min": 15,
+				"name": "id",
+				"pattern": "^[a-z0-9]+$",
+				"presentable": false,
+				"primaryKey": true,
+				"required": true,
+				"system": true,
+				"type": "text"
+			},
+			{
+				"cascadeDelete": true,
+				"collectionId": "_pb_users_auth_",
+				"hidden": false,
+				"id": "d5vztyxa",
+				"maxSelect": 1,
+				"minSelect": 0,
+				"name": "user",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "relation"
+			},
+			{
+				"hidden": false,
+				"id": "xcx4qgqq",
+				"maxSize": 2000000,
+				"name": "settings",
+				"presentable": false,
+				"required": false,
+				"system": false,
+				"type": "json"
+			},
+			{
+				"hidden": false,
+				"id": "autodate2990389176",
+				"name": "created",
+				"onCreate": true,
+				"onUpdate": false,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			},
+			{
+				"hidden": false,
+				"id": "autodate3332085495",
+				"name": "updated",
+				"onCreate": true,
+				"onUpdate": true,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			}
+		],
+		"indexes": [
+			"CREATE UNIQUE INDEX ` + "`" + `idx_30Lwgf2` + "`" + ` ON ` + "`" + `user_settings` + "`" + ` (` + "`" + `user` + "`" + `)"
+		],
+		"system": false
+	},
+	{
+		"id": "2hz5ncl8tizk5nx",
+		"listRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id",
+		"viewRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id",
+		"createRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id && @request.auth.role != \"readonly\"",
+		"updateRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id && @request.auth.role != \"readonly\"",
+		"deleteRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id && @request.auth.role != \"readonly\"",
+		"name": "systems",
+		"type": "base",
+		"fields": [
+			{
+				"autogeneratePattern": "[a-z0-9]{15}",
+				"hidden": false,
+				"id": "text3208210256",
+				"max": 15,
+				"min": 15,
+				"name": "id",
+				"pattern": "^[a-z0-9]+$",
+				"presentable": false,
+				"primaryKey": true,
+				"required": true,
+				"system": true,
+				"type": "text"
+			},
+			{
+				"autogeneratePattern": "",
+				"hidden": false,
+				"id": "7xloxkwk",
+				"max": 0,
+				"min": 0,
+				"name": "name",
+				"pattern": "",
+				"presentable": false,
+				"primaryKey": false,
+				"required": true,
+				"system": false,
+				"type": "text"
+			},
+			{
+				"hidden": false,
+				"id": "waj7seaf",
+				"maxSelect": 1,
+				"name": "status",
+				"presentable": false,
+				"required": false,
+				"system": false,
+				"type": "select",
+				"values": [
+					"up",
+					"down",
+					"paused",
+					"pending"
+				]
+			},
+			{
+				"autogeneratePattern": "",
+				"hidden": false,
+				"id": "ve781smf",
+				"max": 0,
+				"min": 0,
+				"name": "host",
+				"pattern": "",
+				"presentable": false,
+				"primaryKey": false,
+				"required": true,
+				"system": false,
+				"type": "text"
+			},
+			{
+				"autogeneratePattern": "",
+				"hidden": false,
+				"id": "pij0k2jk",
+				"max": 0,
+				"min": 0,
+				"name": "port",
+				"pattern": "",
+				"presentable": false,
+				"primaryKey": false,
+				"required": false,
+				"system": false,
+				"type": "text"
+			},
+			{
+				"hidden": false,
+				"id": "qoq64ntl",
+				"maxSize": 2000000,
+				"name": "info",
+				"presentable": false,
+				"required": false,
+				"system": false,
+				"type": "json"
+			},
+			{
+				"cascadeDelete": true,
+				"collectionId": "_pb_users_auth_",
+				"hidden": false,
+				"id": "jcarjnjj",
+				"maxSelect": 2147483647,
+				"minSelect": 0,
+				"name": "users",
+				"presentable": false,
+				"required": true,
+				"system": false,
+				"type": "relation"
+			},
+			{
+				"hidden": false,
+				"id": "autodate2990389176",
+				"name": "created",
+				"onCreate": true,
+				"onUpdate": false,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			},
+			{
+				"hidden": false,
+				"id": "autodate3332085495",
+				"name": "updated",
+				"onCreate": true,
+				"onUpdate": true,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			}
+		],
+		"indexes": [],
+		"system": false
+	},
+	{
+		"id": "_pb_users_auth_",
+		"listRule": "id = @request.auth.id",
+		"viewRule": "id = @request.auth.id",
+		"createRule": null,
+		"updateRule": null,
+		"deleteRule": null,
+		"name": "users",
+		"type": "auth",
+		"fields": [
+			{
+				"autogeneratePattern": "[a-z0-9]{15}",
+				"hidden": false,
+				"id": "text3208210256",
+				"max": 15,
+				"min": 15,
+				"name": "id",
+				"pattern": "^[a-z0-9]+$",
+				"presentable": false,
+				"primaryKey": true,
+				"required": true,
+				"system": true,
+				"type": "text"
+			},
+			{
+				"cost": 10,
+				"hidden": true,
+				"id": "password901924565",
+				"max": 0,
+				"min": 8,
+				"name": "password",
+				"pattern": "",
+				"presentable": false,
+				"required": true,
+				"system": true,
+				"type": "password"
+			},
+			{
+				"autogeneratePattern": "[a-zA-Z0-9_]{50}",
+				"hidden": true,
+				"id": "text2504183744",
+				"max": 60,
+				"min": 30,
+				"name": "tokenKey",
+				"pattern": "",
+				"presentable": false,
+				"primaryKey": false,
+				"required": true,
+				"system": true,
+				"type": "text"
+			},
+			{
+				"exceptDomains": null,
+				"hidden": false,
+				"id": "email3885137012",
+				"name": "email",
+				"onlyDomains": null,
+				"presentable": false,
+				"required": true,
+				"system": true,
+				"type": "email"
+			},
+			{
+				"hidden": false,
+				"id": "bool1547992806",
+				"name": "emailVisibility",
+				"presentable": false,
+				"required": false,
+				"system": true,
+				"type": "bool"
+			},
+			{
+				"hidden": false,
+				"id": "bool256245529",
+				"name": "verified",
+				"presentable": false,
+				"required": false,
+				"system": true,
+				"type": "bool"
+			},
+			{
+				"autogeneratePattern": "users[0-9]{6}",
+				"hidden": false,
+				"id": "text4166911607",
+				"max": 150,
+				"min": 3,
+				"name": "username",
+				"pattern": "^[\\w][\\w\\.\\-]*$",
+				"presentable": false,
+				"primaryKey": false,
+				"required": false,
+				"system": false,
+				"type": "text"
+			},
+			{
+				"hidden": false,
+				"id": "qkbp58ae",
+				"maxSelect": 1,
+				"name": "role",
+				"presentable": false,
+				"required": false,
+				"system": false,
+				"type": "select",
+				"values": [
+					"user",
+					"admin",
+					"readonly"
+				]
+			},
+			{
+				"hidden": false,
+				"id": "autodate2990389176",
+				"name": "created",
+				"onCreate": true,
+				"onUpdate": false,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			},
+			{
+				"hidden": false,
+				"id": "autodate3332085495",
+				"name": "updated",
+				"onCreate": true,
+				"onUpdate": true,
+				"presentable": false,
+				"system": false,
+				"type": "autodate"
+			}
+		],
+		"indexes": [
+			"CREATE UNIQUE INDEX ` + "`" + `__pb_users_auth__username_idx` + "`" + ` ON ` + "`" + `users` + "`" + ` (username COLLATE NOCASE)",
+			"CREATE UNIQUE INDEX ` + "`" + `__pb_users_auth__email_idx` + "`" + ` ON ` + "`" + `users` + "`" + ` (` + "`" + `email` + "`" + `) WHERE ` + "`" + `email` + "`" + ` != ''",
+			"CREATE UNIQUE INDEX ` + "`" + `__pb_users_auth__tokenKey_idx` + "`" + ` ON ` + "`" + `users` + "`" + ` (` + "`" + `tokenKey` + "`" + `)"
+		],
+		"system": false,
+		"authRule": "verified=true",
+		"manageRule": null
+	}
+]`
+
+		err := app.ImportCollectionsByMarshaledJSON([]byte(jsonData), false)
+		if err != nil {
+			return err
+		}
+
+		// Get all systems that don't have fingerprint records
+		var systemIds []string
+		err = app.DB().NewQuery(`
+			SELECT s.id FROM systems s
+			LEFT JOIN fingerprints f ON s.id = f.system
+			WHERE f.system IS NULL
+		`).Column(&systemIds)
+		if err != nil {
+			return err
+		}
+		// Create fingerprint records with unique UUID tokens for each system
+		for _, systemId := range systemIds {
+			token := uuid.New().String()
+			_, err = app.DB().NewQuery(`
+				INSERT INTO fingerprints (system, token)
+				VALUES ({:system}, {:token})
+			`).Bind(map[string]any{
+				"system": systemId,
+				"token":  token,
+			}).Execute()
+			if err != nil {
+				return err
+			}
+		}
+
+		return nil
+	}, func(app core.App) error {
+		return nil
+	})
+}

beszel/migrations/1728772062_collections_snapshot.go

@@ -1,481 +0,0 @@
-package migrations
-
-import (
-	"encoding/json"
-
-	"github.com/pocketbase/dbx"
-	"github.com/pocketbase/pocketbase/daos"
-	m "github.com/pocketbase/pocketbase/migrations"
-	"github.com/pocketbase/pocketbase/models"
-)
-
-func init() {
-	m.Register(func(db dbx.Builder) error {
-		jsonData := `[
-			{
-				"id": "2hz5ncl8tizk5nx",
-				"created": "2024-07-07 16:08:20.979Z",
-				"updated": "2024-10-12 18:55:51.623Z",
-				"name": "systems",
-				"type": "base",
-				"system": false,
-				"schema": [
-					{
-						"system": false,
-						"id": "7xloxkwk",
-						"name": "name",
-						"type": "text",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"min": null,
-							"max": null,
-							"pattern": ""
-						}
-					},
-					{
-						"system": false,
-						"id": "waj7seaf",
-						"name": "status",
-						"type": "select",
-						"required": false,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"maxSelect": 1,
-							"values": [
-								"up",
-								"down",
-								"paused",
-								"pending"
-							]
-						}
-					},
-					{
-						"system": false,
-						"id": "ve781smf",
-						"name": "host",
-						"type": "text",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"min": null,
-							"max": null,
-							"pattern": ""
-						}
-					},
-					{
-						"system": false,
-						"id": "pij0k2jk",
-						"name": "port",
-						"type": "text",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"min": null,
-							"max": null,
-							"pattern": ""
-						}
-					},
-					{
-						"system": false,
-						"id": "qoq64ntl",
-						"name": "info",
-						"type": "json",
-						"required": false,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"maxSize": 2000000
-						}
-					},
-					{
-						"system": false,
-						"id": "jcarjnjj",
-						"name": "users",
-						"type": "relation",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"collectionId": "_pb_users_auth_",
-							"cascadeDelete": true,
-							"minSelect": null,
-							"maxSelect": null,
-							"displayFields": null
-						}
-					}
-				],
-				"indexes": [],
-				"listRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id",
-				"viewRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id",
-				"createRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id && @request.auth.role != \"readonly\"",
-				"updateRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id && @request.auth.role != \"readonly\"",
-				"deleteRule": "@request.auth.id != \"\" && users.id ?= @request.auth.id && @request.auth.role != \"readonly\"",
-				"options": {}
-			},
-			{
-				"id": "ej9oowivz8b2mht",
-				"created": "2024-07-07 16:09:09.179Z",
-				"updated": "2024-10-12 18:55:51.623Z",
-				"name": "system_stats",
-				"type": "base",
-				"system": false,
-				"schema": [
-					{
-						"system": false,
-						"id": "h9sg148r",
-						"name": "system",
-						"type": "relation",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"collectionId": "2hz5ncl8tizk5nx",
-							"cascadeDelete": true,
-							"minSelect": null,
-							"maxSelect": 1,
-							"displayFields": null
-						}
-					},
-					{
-						"system": false,
-						"id": "azftn0be",
-						"name": "stats",
-						"type": "json",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"maxSize": 2000000
-						}
-					},
-					{
-						"system": false,
-						"id": "m1ekhli3",
-						"name": "type",
-						"type": "select",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"maxSelect": 1,
-							"values": [
-								"1m",
-								"10m",
-								"20m",
-								"120m",
-								"480m"
-							]
-						}
-					}
-				],
-				"indexes": [
-					"CREATE INDEX ` + "`" + `idx_GxIee0j` + "`" + ` ON ` + "`" + `system_stats` + "`" + ` (` + "`" + `system` + "`" + `)"
-				],
-				"listRule": "@request.auth.id != \"\"",
-				"viewRule": null,
-				"createRule": null,
-				"updateRule": null,
-				"deleteRule": null,
-				"options": {}
-			},
-			{
-				"id": "juohu4jipgc13v7",
-				"created": "2024-07-07 16:09:57.976Z",
-				"updated": "2024-10-12 18:55:51.623Z",
-				"name": "container_stats",
-				"type": "base",
-				"system": false,
-				"schema": [
-					{
-						"system": false,
-						"id": "hutcu6ps",
-						"name": "system",
-						"type": "relation",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"collectionId": "2hz5ncl8tizk5nx",
-							"cascadeDelete": true,
-							"minSelect": null,
-							"maxSelect": 1,
-							"displayFields": null
-						}
-					},
-					{
-						"system": false,
-						"id": "r39hhnil",
-						"name": "stats",
-						"type": "json",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"maxSize": 2000000
-						}
-					},
-					{
-						"system": false,
-						"id": "vo7iuj96",
-						"name": "type",
-						"type": "select",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"maxSelect": 1,
-							"values": [
-								"1m",
-								"10m",
-								"20m",
-								"120m",
-								"480m"
-							]
-						}
-					}
-				],
-				"indexes": [],
-				"listRule": "@request.auth.id != \"\"",
-				"viewRule": null,
-				"createRule": null,
-				"updateRule": null,
-				"deleteRule": null,
-				"options": {}
-			},
-			{
-				"id": "_pb_users_auth_",
-				"created": "2024-07-14 16:25:18.226Z",
-				"updated": "2024-10-12 22:27:19.081Z",
-				"name": "users",
-				"type": "auth",
-				"system": false,
-				"schema": [
-					{
-						"system": false,
-						"id": "qkbp58ae",
-						"name": "role",
-						"type": "select",
-						"required": false,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"maxSelect": 1,
-							"values": [
-								"user",
-								"admin",
-								"readonly"
-							]
-						}
-					},
-					{
-						"system": false,
-						"id": "users_avatar",
-						"name": "avatar",
-						"type": "file",
-						"required": false,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"mimeTypes": [
-								"image/jpeg",
-								"image/png",
-								"image/svg+xml",
-								"image/gif",
-								"image/webp"
-							],
-							"thumbs": null,
-							"maxSelect": 1,
-							"maxSize": 5242880,
-							"protected": false
-						}
-					}
-				],
-				"indexes": [],
-				"listRule": "id = @request.auth.id",
-				"viewRule": "id = @request.auth.id",
-				"createRule": null,
-				"updateRule": null,
-				"deleteRule": null,
-				"options": {
-					"allowEmailAuth": true,
-					"allowOAuth2Auth": true,
-					"allowUsernameAuth": false,
-					"exceptEmailDomains": null,
-					"manageRule": null,
-					"minPasswordLength": 8,
-					"onlyEmailDomains": null,
-					"onlyVerified": true,
-					"requireEmail": false
-				}
-			},
-			{
-				"id": "elngm8x1l60zi2v",
-				"created": "2024-07-15 01:16:04.044Z",
-				"updated": "2024-10-12 22:27:29.128Z",
-				"name": "alerts",
-				"type": "base",
-				"system": false,
-				"schema": [
-					{
-						"system": false,
-						"id": "hn5ly3vi",
-						"name": "user",
-						"type": "relation",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"collectionId": "_pb_users_auth_",
-							"cascadeDelete": true,
-							"minSelect": null,
-							"maxSelect": 1,
-							"displayFields": null
-						}
-					},
-					{
-						"system": false,
-						"id": "g5sl3jdg",
-						"name": "system",
-						"type": "relation",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"collectionId": "2hz5ncl8tizk5nx",
-							"cascadeDelete": true,
-							"minSelect": null,
-							"maxSelect": 1,
-							"displayFields": null
-						}
-					},
-					{
-						"system": false,
-						"id": "zj3ingrv",
-						"name": "name",
-						"type": "select",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"maxSelect": 1,
-							"values": [
-								"Status",
-								"CPU",
-								"Memory",
-								"Disk",
-								"Temperature",
-								"Bandwidth"
-							]
-						}
-					},
-					{
-						"system": false,
-						"id": "o2ablxvn",
-						"name": "value",
-						"type": "number",
-						"required": false,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"min": null,
-							"max": null,
-							"noDecimal": false
-						}
-					},
-					{
-						"system": false,
-						"id": "fstdehcq",
-						"name": "min",
-						"type": "number",
-						"required": false,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"min": null,
-							"max": 60,
-							"noDecimal": true
-						}
-					},
-					{
-						"system": false,
-						"id": "6hgdf6hs",
-						"name": "triggered",
-						"type": "bool",
-						"required": false,
-						"presentable": false,
-						"unique": false,
-						"options": {}
-					}
-				],
-				"indexes": [],
-				"listRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
-				"viewRule": "",
-				"createRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
-				"updateRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
-				"deleteRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
-				"options": {}
-			},
-			{
-				"id": "4afacsdnlu8q8r2",
-				"created": "2024-09-12 17:42:55.324Z",
-				"updated": "2024-10-12 18:55:51.624Z",
-				"name": "user_settings",
-				"type": "base",
-				"system": false,
-				"schema": [
-					{
-						"system": false,
-						"id": "d5vztyxa",
-						"name": "user",
-						"type": "relation",
-						"required": true,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"collectionId": "_pb_users_auth_",
-							"cascadeDelete": false,
-							"minSelect": null,
-							"maxSelect": 1,
-							"displayFields": null
-						}
-					},
-					{
-						"system": false,
-						"id": "xcx4qgqq",
-						"name": "settings",
-						"type": "json",
-						"required": false,
-						"presentable": false,
-						"unique": false,
-						"options": {
-							"maxSize": 2000000
-						}
-					}
-				],
-				"indexes": [
-					"CREATE UNIQUE INDEX ` + "`" + `idx_30Lwgf2` + "`" + ` ON ` + "`" + `user_settings` + "`" + ` (` + "`" + `user` + "`" + `)"
-				],
-				"listRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
-				"viewRule": null,
-				"createRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
-				"updateRule": "@request.auth.id != \"\" && user.id = @request.auth.id",
-				"deleteRule": null,
-				"options": {}
-			}
-		]`
-
-		collections := []*models.Collection{}
-		if err := json.Unmarshal([]byte(jsonData), &collections); err != nil {
-			return err
-		}
-
-		return daos.New(db).ImportCollections(collections, true, nil)
-	}, func(db dbx.Builder) error {
-		return nil
-	})
-}

beszel/migrations/initial-settings.go

@@ -1,19 +1,29 @@
 package migrations
 
 import (
-	"github.com/pocketbase/dbx"
-	"github.com/pocketbase/pocketbase/daos"
+	"github.com/pocketbase/pocketbase/core"
 	m "github.com/pocketbase/pocketbase/migrations"
 )
 
-func init() {
-	m.Register(func(db dbx.Builder) error {
-		dao := daos.New(db)
+const (
+	TempAdminEmail = "_@b.b"
+)
 
-		settings, _ := dao.FindSettings()
+func init() {
+	m.Register(func(app core.App) error {
+		// initial settings
+		settings := app.Settings()
 		settings.Meta.AppName = "Beszel"
 		settings.Meta.HideControls = true
-
-		return dao.SaveSettings(settings)
+		settings.Logs.MinLevel = 4
+		if err := app.Save(settings); err != nil {
+			return err
+		}
+		// create superuser
+		collection, _ := app.FindCollectionByNameOrId(core.CollectionNameSuperusers)
+		user := core.NewRecord(collection)
+		user.SetEmail(TempAdminEmail)
+		user.SetRandomPassword()
+		return app.Save(user)
 	}, nil)
 }

beszel/site/.prettierrc

@@ -0,0 +1,8 @@
+{
+	"trailingComma": "es5",
+	"useTabs": true,
+	"tabWidth": 2,
+	"semi": false,
+	"singleQuote": false,
+	"printWidth": 120
+}

beszel/site/components.json

@@ -1,17 +1,17 @@
 {
-  "$schema": "https://ui.shadcn.com/schema.json",
-  "style": "default",
-  "rsc": false,
-  "tsx": true,
-  "tailwind": {
-    "config": "tailwind.config.js",
-    "css": "src/index.css",
-    "baseColor": "gray",
-    "cssVariables": true,
-    "prefix": ""
-  },
-  "aliases": {
-    "components": "@/components",
-    "utils": "@/lib/utils"
-  }
-}
+	"$schema": "https://ui.shadcn.com/schema.json",
+	"style": "default",
+	"rsc": false,
+	"tsx": true,
+	"tailwind": {
+		"config": "tailwind.config.js",
+		"css": "src/index.css",
+		"baseColor": "gray",
+		"cssVariables": true,
+		"prefix": ""
+	},
+	"aliases": {
+		"components": "@/components",
+		"utils": "@/lib/utils"
+	}
+}

beszel/site/embed.go

@@ -3,11 +3,11 @@ package site
 
 import (
 	"embed"
-
-	"github.com/labstack/echo/v5"
+	"io/fs"
 )
 
 //go:embed all:dist
-var assets embed.FS
+var distDir embed.FS
 
-var Dist = echo.MustSubFS(assets, "dist")
+// DistDirFS contains the embedded dist directory files (without the "dist" prefix)
+var DistDirFS, _ = fs.Sub(distDir, "dist")

beszel/site/index.html

@@ -1,10 +1,18 @@
-<!DOCTYPE html>
-<html lang="en">
+<!doctype html>
+<html lang="en" dir="ltr">
 	<head>
 		<meta charset="UTF-8" />
-		<link rel="icon" type="image/svg+xml" href="/static/favicon.svg" />
-		<meta name="viewport" content="width=device-width, initial-scale=1.0" />
+		<link rel="manifest" href="./static/manifest.json" />
+		<link rel="icon" type="image/svg+xml" href="./static/favicon.svg" />
+		<meta name="viewport" content="width=device-width, initial-scale=1.0,maximum-scale=1.0, user-scalable=no, viewport-fit=cover" />
 		<title>Beszel</title>
+		<script>
+			globalThis.BESZEL = {
+				BASE_PATH: "%BASE_URL%",
+				HUB_VERSION: "{{V}}",
+				HUB_URL: "{{HUB_URL}}"
+			}
+		</script>
 	</head>
 	<body>
 		<div id="app"></div>

beszel/site/lingui.config.ts

@@ -0,0 +1,45 @@
+import { defineConfig } from "@lingui/cli"
+
+export default defineConfig({
+	locales: [
+		"en",
+		"ar",
+		"bg",
+		"cs",
+		"da",
+		"de",
+		"es",
+		"fa",
+		"fr",
+		"hr",
+		"hu",
+		"it",
+		"is",
+		"ja",
+		"ko",
+		"nl",
+		"no",
+		"pl",
+		"pt",
+		"tr",
+		"ru",
+		"sl",
+		"sv",
+		"uk",
+		"vi",
+		"zh",
+		"zh-CN",
+		"zh-HK",
+	],
+	sourceLocale: "en",
+	compileNamespace: "ts",
+	formatOptions: {
+		lineNumbers: false,
+	},
+	catalogs: [
+		{
+			path: "<rootDir>/src/locales/{locale}/{locale}",
+			include: ["src"],
+		},
+	],
+})

beszel/site/package.json

@@ -1,59 +1,74 @@
 {
 	"name": "beszel",
 	"private": true,
-	"version": "0.0.0",
+	"version": "0.12.2",
 	"type": "module",
 	"scripts": {
 		"dev": "vite",
-		"build": "vite build",
-		"preview": "vite preview"
+		"build": "lingui extract --overwrite && lingui compile && vite build",
+		"preview": "vite preview",
+		"sync": "lingui extract --overwrite && lingui compile",
+		"sync_and_purge": "lingui extract --overwrite --clean && lingui compile"
 	},
 	"dependencies": {
 		"@henrygd/queue": "^1.0.7",
+		"@henrygd/semaphore": "^0.0.2",
+		"@lingui/detect-locale": "^5.3.3",
+		"@lingui/macro": "^5.3.3",
+		"@lingui/react": "^5.3.3",
 		"@nanostores/react": "^0.7.3",
 		"@nanostores/router": "^0.11.0",
-		"@radix-ui/react-alert-dialog": "^1.1.2",
-		"@radix-ui/react-checkbox": "^1.1.2",
-		"@radix-ui/react-dialog": "^1.1.2",
-		"@radix-ui/react-dropdown-menu": "^2.1.2",
-		"@radix-ui/react-label": "^2.1.0",
-		"@radix-ui/react-select": "^2.1.2",
-		"@radix-ui/react-separator": "^1.1.0",
-		"@radix-ui/react-slider": "^1.2.1",
-		"@radix-ui/react-slot": "^1.1.0",
-		"@radix-ui/react-switch": "^1.1.1",
-		"@radix-ui/react-tabs": "^1.1.1",
-		"@radix-ui/react-toast": "^1.2.2",
-		"@radix-ui/react-tooltip": "^1.1.3",
-		"@tanstack/react-table": "^8.20.5",
-		"@vitejs/plugin-react": "^4.3.2",
-		"class-variance-authority": "^0.7.0",
+		"@radix-ui/react-alert-dialog": "^1.1.14",
+		"@radix-ui/react-checkbox": "^1.3.2",
+		"@radix-ui/react-dialog": "^1.1.14",
+		"@radix-ui/react-direction": "^1.1.1",
+		"@radix-ui/react-dropdown-menu": "^2.1.15",
+		"@radix-ui/react-label": "^2.1.7",
+		"@radix-ui/react-select": "^2.2.5",
+		"@radix-ui/react-separator": "^1.1.7",
+		"@radix-ui/react-slider": "^1.3.5",
+		"@radix-ui/react-slot": "^1.2.3",
+		"@radix-ui/react-switch": "^1.2.5",
+		"@radix-ui/react-tabs": "^1.1.12",
+		"@radix-ui/react-toast": "^1.2.14",
+		"@radix-ui/react-tooltip": "^1.2.7",
+		"@tanstack/react-table": "^8.21.3",
+		"class-variance-authority": "^0.7.1",
 		"clsx": "^2.1.1",
-		"cmdk": "^1.0.0",
+		"cmdk": "^1.1.1",
 		"d3-time": "^3.1.0",
 		"lucide-react": "^0.452.0",
-		"nanostores": "^0.11.3",
-		"pocketbase": "^0.21.5",
+		"nanostores": "^0.11.4",
+		"pocketbase": "^0.26.2",
 		"react": "^18.3.1",
 		"react-dom": "^18.3.1",
-		"recharts": "^2.13.0",
-		"tailwind-merge": "^2.5.4",
+		"recharts": "^2.15.4",
+		"tailwind-merge": "^2.6.0",
 		"tailwindcss-animate": "^1.0.7",
-		"valibot": "^0.36.0"
+		"valibot": "^0.42.1"
 	},
 	"devDependencies": {
-		"@types/bun": "^1.1.11",
-		"@types/react": "^18.3.11",
-		"@types/react-dom": "^18.3.1",
-		"autoprefixer": "^10.4.20",
-		"postcss": "^8.4.47",
-		"tailwindcss": "^3.4.14",
-		"typescript": "^5.6.3",
-		"vite": "^5.4.9"
+		"@lingui/cli": "^5.3.3",
+		"@lingui/swc-plugin": "^5.5.2",
+		"@lingui/vite-plugin": "^5.3.3",
+		"@tailwindcss/container-queries": "^0.1.1",
+		"@types/bun": "^1.2.19",
+		"@types/react": "^18.3.23",
+		"@types/react-dom": "^18.3.7",
+		"@vitejs/plugin-react-swc": "^3.11.0",
+		"autoprefixer": "^10.4.21",
+		"postcss": "^8.5.6",
+		"tailwindcss": "^3.4.17",
+		"tailwindcss-rtl": "^0.9.0",
+		"typescript": "^5.8.3",
+		"vite": "^6.3.5"
 	},
 	"overrides": {
 		"@nanostores/router": {
 			"nanostores": "^0.11.3"
 		}
+	},
+	"optionalDependencies": {
+		"@esbuild/linux-arm64": "^0.21.5"
 	}
-}
+}

beszel/site/postcss.config.js

@@ -1,6 +1,6 @@
 export default {
-  plugins: {
-    tailwindcss: {},
-    autoprefixer: {},
-  },
+	plugins: {
+		tailwindcss: {},
+		autoprefixer: {},
+	},
 }

beszel/site/public/static/apple.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12.2 6.9c-1 0-2.5-1-4-1-2 0-4 1.1-5 3-2 3.6-.5 9 1.5 12 1 1.5 2.3 3.2 3.8 3.1 1.6 0 2.1-1 4-1 1.8 0 2.3 1 4 1 1.6 0 2.6-1.5 3.6-3a13 13 0 0 0 1.7-3.4 5.3 5.3 0 0 1-.6-9.4 5.6 5.6 0 0 0-4.4-2.4C14.8 5.6 13 7 12.2 7zm3.3-3c.9-1 1.4-2.5 1.3-3.9-1.2 0-2.7.8-3.6 1.8A5 5 0 0 0 12 5.5c1.3.1 2.7-.7 3.5-1.7"/></svg>

beszel/site/public/static/bitbucket.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M.8 1.2a.8.8 0 0 0-.8 1l3.3 19.7c0 .5.5.9 1 .9h15.6a.8.8 0 0 0 .8-.7l3.3-20a.8.8 0 0 0-.8-.9zm13.7 14.3h-5l-1.3-7h7.5z"/></svg>

beszel/site/public/static/discord.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20.3 4.4a19.8 19.8 0 0 0-4.9-1.5L14.7 4C13 4 11.1 4 9.3 4.1L8.6 3a19.7 19.7 0 0 0-5 1.5C.6 9-.4 13.6.1 18.1c2 1.5 4 2.4 6 3h.1c.5-.6.9-1.3 1.2-2l-1.9-1V18l.4-.3c4 1.8 8.2 1.8 12.1 0h.1l.4.3v.1a12.3 12.3 0 0 1-2 1l1.3 2c2-.6 4-1.5 6-3h.1c.5-5.2-.8-9.7-3.6-13.7zM8 15.4c-1.2 0-2.1-1.2-2.1-2.5s1-2.4 2.1-2.4c1.2 0 2.2 1 2.2 2.4 0 1.3-1 2.4-2.2 2.4zm8 0c-1.2 0-2.2-1.2-2.2-2.5s1-2.4 2.2-2.4c1.2 0 2.2 1 2.2 2.4 0 1.3-1 2.4-2.2 2.4Z"/></svg>

beszel/site/public/static/facebook.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.1 23.7v-8H6.6V12h2.5v-1.5c0-4.1 1.8-6 5.9-6h1.4a8.7 8.7 0 0 1 1.2.3V8a8.6 8.6 0 0 0-.7 0 26.8 26.8 0 0 0-.7 0c-.7 0-1.3 0-1.7.3a1.7 1.7 0 0 0-.7.6c-.2.4-.3 1-.3 1.7V12h3.9l-.4 2.1-.3 1.6h-3.2V24a12 12 0 1 0-4.4-.3Z"/></svg>

beszel/site/public/static/gitea.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M4.2 4.6a4.2 4.2 0 0 0-2.9 1.1C-.4 7.3 0 9.7.1 10.1c0 .4.3 1.6 1.2 2.7C3 15 6.8 15 6.8 15S7.3 16 8 17c1 1.3 2 2.3 2.9 2.4H18s.4 0 1-.4c.6-.3 1-.9 1-.9s.6-.5 1.3-1.7l.5-1s2.1-4.6 2.1-9c0-1.2-.4-1.5-.4-1.5l-.4-.2s-4.5.3-6.8.3h-1.5v4.5l-.6-.3V5h-3.5l-6-.4h-.6zm.4 1.8s.3 2.3.7 3.6c.2 1.1 1 3 1 3l-1.7-.3c-1-.4-1.4-.8-1.4-.8s-.8-.5-1.1-1.5c-.7-1.7 0-2.7 0-2.7s.2-.9 1.4-1.1c.4-.2.9-.2 1-.2zM12.9 9l.5.1.9.4-.6 1.1a.7.7 0 0 0-.6.4.7.7 0 0 0 .1.7l-1 2a.7.7 0 0 0-.6.5.7.7 0 0 0 .3.7.7.7 0 0 0 1-.2.7.7 0 0 0-.2-.8l1-2a.7.7 0 0 0 .2 0 .7.7 0 0 0 .3 0 8.8 8.8 0 0 1 1 .4.8.8 0 0 1 .3.3l-.1.6c0 .3-.7 1.5-.7 1.5a.7.7 0 0 0-.7.5.7.7 0 1 0 1.2-.2l.2-.5.5-1.1c0-.1.2-.4.1-.8a1 1 0 0 0-.5-.7l-1-.6-.1-.2a.7.7 0 0 0-.2-.3l.5-1 3 1.4s.4.2.5.6v.6L16 16.8s-.2.5-.7.5a1 1 0 0 1-.4 0h-.2L10.4 15s-.4-.2-.5-.6l.1-.7 2-4.2s.3-.4.5-.5A.9.9 0 0 1 13 9z"/></svg>

beszel/site/public/static/gitee.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 0A12 12 0 0 0 0 12a12 12 0 0 0 12 12 12 12 0 0 0 12-12A12 12 0 0 0 12 0zm6 5.3c.4 0 .7.3.7.6v1.5a.6.6 0 0 1-.6.6H9.8C8.8 8 8 8.8 8 9.8v5.6c0 .3.3.6.6.6h5.6c1 0 1.8-.8 1.8-1.8V14a.6.6 0 0 0-.6-.6h-4.1a.6.6 0 0 1-.6-.6v-1.4a.6.6 0 0 1 .6-.6H18c.3 0 .6.2.6.6v3.4a4 4 0 0 1-4 4H5.9a.6.6 0 0 1-.6-.6V9.8a4.4 4.4 0 0 1 4.5-4.5H18Z"/></svg>

beszel/site/public/static/github.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 .3a12 12 0 0 0-3.8 23.4c.6.1.8-.3.8-.6v-2c-3.3.7-4-1.6-4-1.6-.6-1.4-1.4-1.8-1.4-1.8-1-.7.1-.7.1-.7 1.2 0 1.9 1.2 1.9 1.2 1 1.8 2.8 1.3 3.5 1 0-.8.4-1.3.7-1.6-2.7-.3-5.5-1.3-5.5-6 0-1.2.5-2.3 1.3-3.1-.2-.4-.6-1.6 0-3.2 0 0 1-.3 3.4 1.2a11.5 11.5 0 0 1 6 0c2.3-1.5 3.3-1.2 3.3-1.2.6 1.6.2 2.8 0 3.2.9.8 1.3 1.9 1.3 3.2 0 4.6-2.8 5.6-5.5 5.9.5.4.9 1 .9 2.2v3.3c0 .3.1.7.8.6A12 12 0 0 0 12 .3"/></svg>

beszel/site/public/static/gitlab.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M23.6 9.6 20.3 1a.9.9 0 0 0-.3-.4.9.9 0 0 0-1 0 .9.9 0 0 0-.3.5l-2.2 6.7h-9L5.3 1.1A.9.9 0 0 0 5 .6a.9.9 0 0 0-1 0 .9.9 0 0 0-.3.4L.4 9.5a6 6 0 0 0 2 7.1l5 3.8 2.5 1.8 1.5 1.1a1 1 0 0 0 1.2 0l1.5-1 2.5-2 5-3.7a6 6 0 0 0 2-7z"/></svg>

beszel/site/public/static/google.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12.5 11v3.2h7.8a7 7 0 0 1-1.8 4.1 8 8 0 0 1-6 2.4c-4.8 0-8.6-3.9-8.6-8.7a8.6 8.6 0 0 1 14.5-6.4l2.3-2.3C18.7 1.4 16 0 12.5 0 5.9 0 .3 5.4.3 12S6 24 12.5 24a11 11 0 0 0 8.4-3.4c2.1-2.1 2.8-5.2 2.8-7.6 0-.8 0-1.5-.2-2h-11z"/></svg>

beszel/site/public/static/instagram.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M7 0C5.8.2 5 .4 4.1.7 3.3 1 2.7 1.4 2 2c-.7.7-1 1.4-1.4 2.2C.3 4.9.1 5.8.1 7a84.6 84.6 0 0 0 .5 12.8c.4.8.8 1.4 1.4 2.1.7.7 1.4 1 2.2 1.4.7.3 1.6.5 2.9.5a85 85 0 0 0 12.8-.5c.8-.4 1.4-.8 2.1-1.4.7-.7 1-1.4 1.4-2.2.3-.7.5-1.6.5-2.9a85 85 0 0 0-.5-12.8C23 3.3 22.6 2.7 22 2c-.7-.7-1.4-1-2.2-1.4-.7-.3-1.6-.5-2.9-.5A85.5 85.5 0 0 0 7 0m.2 21.7c-1.2 0-1.8-.3-2.3-.4-.5-.2-1-.5-1.3-1-.5-.3-.7-.7-1-1.3-.1-.4-.3-1-.4-2.2a84.8 84.8 0 0 1 .4-12c.2-.5.5-1 1-1.3.3-.5.7-.7 1.3-1 .4-.1 1-.3 2.2-.4a84.4 84.4 0 0 1 12 .4c.5.3 1 .5 1.3 1 .5.3.7.7 1 1.3.1.4.3 1 .4 2.2a82.7 82.7 0 0 1-.4 12c-.2.5-.5 1-1 1.3-.3.5-.7.7-1.3 1-.4.1-1 .3-2.2.4a84.9 84.9 0 0 1-9.7 0M17 5.6A1.4 1.4 0 1 0 18.4 4 1.4 1.4 0 0 0 17 5.6M5.8 12a6.2 6.2 0 1 0 12.4 0 6.2 6.2 0 0 0-12.4 0M8 12a4 4 0 1 1 4 4 4 4 0 0 1-4-4"/></svg>

beszel/site/public/static/lock.svg

@@ -1 +0,0 @@
-<svg xmlns="http://www.w3.org/2000/svg" fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" viewBox="0 0 24 24"><rect width="18" height="11" x="3" y="11" rx="2" ry="2"/><path d="M7 11V7a5 5 0 0 1 10 0v4"/></svg>