The SecsorConnect Pod logs shows that it has Found...
# umh-supportl
lmullane
03/21/2023, 12:33 PMThe SecsorConnect Pod logs shows that it has Found the IFM Gateway
j
Jermuk
03/21/2023, 2:02 PMJermuk
03/21/2023, 2:02 PMapparently there is a bug in sensorconnect, which causes it to continue finding the device (it did not crash), but it keeps running and sending nothing to kafka. we will receive the logs in a couple of hours. a pod restart fixes the problem (but also deletes the logs)
Jermuk
03/21/2023, 2:03 PM@lmullane that that power cycling the gateway does not fix the bug, and apparently only sensorconnect restart. @Ferdinand @DaruZero can one of you take a look there what could have gone wrong here?
Jermuk
03/21/2023, 2:03 PM@lmullane just to confirm: you turned down the power of the ifm gaetway for a couple of seconds, turned it on again and the error was still like this right?
Jermuk
03/21/2023, 2:04 PMwhat happens if you open the device in the browser?
Jermuk
03/21/2023, 2:05 PMl
lmullane
03/21/2023, 2:05 PMI reset power before our initial call at 12:30 , on this failure 14:05 I have not reset IFM yet
lmullane
03/21/2023, 2:06 PMWill I power cycle the IFM Gateway ?
j
Jermuk
03/21/2023, 2:06 PMcan you first try to connect to the gateway with your browser?
Jermuk
03/21/2023, 2:06 PMl
lmullane
03/21/2023, 2:07 PMThe IFM Gateway is currently not responding through the Browser
lmullane
03/21/2023, 2:08 PMI can ping its IP
lmullane
03/21/2023, 2:13 PMIFM Gateway bug, note I still have not power cycled it in case you want me to do anything while in that state
j
Jermuk
03/21/2023, 4:46 PMok, if you cannot reach the gateway its likely an ifm bug. but it should be fixed with a restart
Jermuk
03/21/2023, 4:46 PMlets just flash the new firmware to be on the safe side here
Jermuk
03/21/2023, 4:46 PMl
lmullane
03/21/2023, 5:27 PMI agree the IFM Gateway has a bug, it should not go unresponsive. However restarting it does not resolve the issue, However Removing the SensorConnect Pod, and waiting for the UMH stack to Start another SensorConnect POD does get the stack working again, Before I flash the IFM Gateway would it be better to fix the SensorConnect POD, ie once the Gateway is restarted the SensorConnect should detect and work as normal ?
a
alexkrueger
03/21/2023, 7:34 PMI think is also the Same bug that we had at 🥩
j
Jermuk
03/21/2023, 10:15 PMHypothesis: maybe sensorconnect is doing too many requests, stuck in an endless loop, etc and overloads the ifm gateway, so that is stays unresponsive. What you can try is lowering the poll rate in the environment variables.
Can you try setting LOWER_POLLING_TIME_MS in the sensorconnect pod or deployment from 20 to 500? And then check whether the gateway is unresponsive?
Also, let's try flashing the new firmware
l
lmullane
03/22/2023, 9:42 AMOk, I will try that and report back, thanks for the help again Jermuk
lmullane
03/22/2023, 11:01 AMJust trying to figure how to change it
lmullane
03/22/2023, 11:01 AMk:{"name":"LOWER_POLLING_TIME_MS"}:
.: {}
f:name: {}
f:value: {}
lmullane
03/22/2023, 11:01 AMor is it
lmullane
03/22/2023, 11:01 AM- name: LOWER_POLLING_TIME_MS
value: '20'
lmullane
03/22/2023, 11:02 AMIf I try to change second option UMH lens generates a ERROR when saving
f
Ferdinand
03/22/2023, 11:10 AMthe 2nd one
Ferdinand
03/22/2023, 11:10 AMCan you show me which error you get ?
l
lmullane
03/22/2023, 11:17 AMThere is a recording of the process I'm using
f
Ferdinand
03/22/2023, 11:20 AMOk, ill check if this is incorrectly documented, but you are supposed to change it in the StatefulSet
l
lmullane
03/22/2023, 11:20 AMah ok I'll try there so
lmullane
03/22/2023, 11:21 AMYes I was able to change the setting in StatefulSet, thanks for your help Flinnenberg. I restart the simulation and leave run for a few hours
lmullane
03/22/2023, 12:44 PMIts still working, but I need to go to office, but I will leave it run overnight to see how it gets on
lmullane
03/23/2023, 10:47 AMI left the Stack run overnight on my laptop and its still working (so that's over 12 hours) . Only change I made so far wash the "LOWER_POLLING_TIME_MS" change from 20 to 500. (so IFM Gateway not flashed yet) .
lmullane
03/23/2023, 10:54 AMI'm still a bit concerned that the SensorConnect Pod fails after a few hours at the 20ms setting. Ok the IFM Gatway bug could be the source, but I think the SensorConnect POD needs to be robust to handle it, ie on a customer site all sorts of things may go wrong, network and power outages etc
j
Jermuk
03/23/2023, 11:05 AMi understand your concern, we could set the lower polling time to 500 by default. there is already a complicated logic inside of of sensorconnect trying not to overload the ifm gateways with old firmware (like for example LOWER_POLLING_TIME_MS), but the point where the ifm IO-link master fails is also dependent on the amount of sensors that are connected, which makes it hard for us to establish a logic there. we even talked with the ifm people about that and they already said that its a bug on their side and flashing the new firmware will fix it.
l
lmullane
03/23/2023, 11:38 AMok thanks for the Feedback Jermuk, I'll flash the IFM Gateway with the pre-release version of the firmware and do some further tests
lmullane
03/24/2023, 3:57 PMOk Just a update, I flashed the IFM Gateway yesterday. I left it run overnight again and its still working I guess over 30 hours now
2 Views