Hi, we just released a new article on learn.umh.app. Go check it out, and discuss it here! Title: New Bridges and Automatic Helm Upgrade in UMH Excerpt: UMH introduces Benthos-based data bridges and automated Helm upgrades, enhancing reliability, maintainability and user experience. Explore how these updates simplify maintenance, optimize data flows, and enable custom data contracts. Link: https://learn.umh.app/course/new-bridges-and-automatic-helm-upgrade-in-umh/

Must-Read for everyone upgrading the UMH, @Diederik , @trentc @Brian Pribe @DanielH

i was actually starting this morning with a bash script for helm upgrades, to automate it more with comparison of user changed values, current values against new chart version etc, to always have a clean chart.. How is this handled in the automatic upgrade?

we decreased the amount of options in the chart, and will over time extend it again. the user values will be taken to the next version automatically from this version on

Currently we prefer user values over our values (basically --reuse-values).

Did the upgrade, is there a reason why the RedPanda version "flickers"? It shows this message about the version, then it goes green to latest ver

I am having the same on my instance, @Ferdinand @Athavan kanapuli :/

Issue [ENG-1833](https://linear.app/united-manufacturing-hub/issue/ENG-1833/redpanda-flickers-with-060) created.

is this coming in the new version? can you do a top / htop on your instance to show us what is taking the most CPU?

Yes in a few....i have a call

i'll be leaving for sports in a couple of minutes, i dont know about @Ferdinand

Ill stay for a while

Thanks for the info! I will analyse this decently before updating, seems important on first sight!

I know what is happening! The new bridges have new consumer groups. So they will start at the oldest message available in Kafka and go through all messages, e.g. send all messages again to MQTT and try to insert it into the database. Until it catches up, there is some high CPU usage

Does that mean we will have duplicates in the tag table?

no, tag table automatically filters out duplicates before insertio

It depending on your MQTT consumer logic in Node-RED, it might recreate a lot of messages causing duplicated messages. Maybe what we should do is hot fix the current helm version to disable the “from oldest” and provide a new version with that. So current users on the helm chart can upgrade, but people upgrading with larger instances don’t have this CPU issue

i will update my testsetup tomorrow around noon, feedback by tomorrow evening! If i see no weird things i will update my production setup

maybe dont update yet, we need to add a couple of more fixes for people extensively using MQTT

ok, let me know

ok. so this will still mean that if the chart changes you will have old keys that does not exist in the new chart right? i have spent quite some time now to create a bash script that let you choose namespace, chart and version to upgrade and then it downloads the chart temporary to get the default values, compare with the current values from the chart and extracts them to a new yaml file. this way it is easy to filter out what values you have changed. Next i will compare these values against the new chart version to see if the keys exists in the new chart.. It was not so easy to do with only bash scripting but i want it to be simple so it can run direct in the cli on flatcar.. i was also thinking about creating a simple pod running npm and have a small dashboard to handle this with the kubernetes api..

i think this behaviour might be overwritten by our current logic, where you can specify the helm chart version in the mgmtcompanion-config, and the mgmtcompanion updates it then.

ok. well, i am not worried about the update, currently my new setup is empty and i have vacation this week since the kids have Autumn leave.. What i mean is if you use the --reuse in helm it will put in all values from the current setup regardless of what is in the new chart. From my understanding if reading both the manual and forum threads it is better to reset the values and ensure you only apply valid values for the new chart. This is why i making a quite large bash script that reads out the changed values in the current chart, saves them to a file and check against the new chart version if the keys exists. This way you will see what values you have changed and also if some setting has been removed.