Hey everyone, here's a quick update on the current issue with memory pressure / the instance not being online The new version 0.6.16 is causing UMH to continuously re-deploy. Each deployment creates a new Helm release secret that isn’t cleaned up. Over time, these secrets accumulate—sometimes hundreds of them—and each one is quite large. When the system (and our companion, which polls every ~10 seconds) tries to list all these secrets, it ends up processing hundreds of items (in some cases 289 MB of data) on every API call. This heavy load is causing high memory pressure and slowing k3s down, eventually taking instances offline. Rolling back will not solve the issue as the buildup is already done in k3s. Enteprise users are not affected as they are running on an older version. Instant Relief: You can temporarily alleviate the memory pressure by deleting all the old Helm secrets except for the latest one. Running the following commands will remove the extra revisions for a couple of hours until we push a permanent fix:

bash
# Get the latest secret name (highest version)
LATEST=$(kubectl -n united-manufacturing-hub get secret --kubeconfig /etc/rancher/k3s/k3s.yaml -o name \
  | grep 'sh\.helm\.release\.v1\.united-manufacturing-hub\.v' \
  | sed 's|^secret/||' \
  | sort -V \
  | tail -n1)

# Delete all secrets except the latest
kubectl -n united-manufacturing-hub get secret --kubeconfig /etc/rancher/k3s/k3s.yaml -o name \
  | grep 'sh\.helm\.release\.v1\.united-manufacturing-hub\.v' \
  | sed 's|^secret/||' \
  | grep -v "^${LATEST}$" \
  | xargs -r kubectl -n united-manufacturing-hub delete secret --kubeconfig /etc/rancher/k3s/k3s.yaml

What’s Next: We’re actively investigating and working on a permanent solution to prevent this excessive Helm redeployment and secret buildup. Thanks for your patience and please keep sharing any logs or feedback if you experience further issues. Feel free to reach out if you have any questions or need further assistance!