Issue: tagname repeating....
# umh-support
t
Issue: tagname repeating....
MQTT Topic "40hero/delaware/office/iiotdemo/devices/S8VKX" payload
Copy code
{
  "timestamp": 1729185434378,
  "schema": "s8vkx",
  "Status": {
    "MemoryErr": false,
    "OverheatAbn": false,
    "CurrentMeasErr": false,
    "VoltageMeasErr": false,
    "OverheatingAlm": false,
    "YearsToReplaceFUL": true,
    "YearsToReplaceHLF": false
  },
  "OutputVoltage": 24.2,
  "OutputCurrent": 2.2,
  "PeakHoldCurrent": 5,
  "YearsToReplace": 14.6,
  "PercentToReplace": 97.4,
  "TotalRunTime": 3438,
  "ContinuousRunTime": 24323
}
This processing code
Copy code
# these variables are autogenerated from your instance location
let enterprise = "40Hero"
let site = "Delaware"
let area = "office"
let line = "iiotdemo"
let schema = "_historian"

let tagname = "mytag"

let value = this.without("timestamp")  # Remove the timestamp field
value.timestamp_ms = this.timestamp  # Add new field timestamp_ms
produces this in the historian
the tagname is repeated.....
tag browser
b
Interesting
t
More information.....refer to my other thread about the produced and consumed variables in the processor and UNS....and how they relate. https://discord.com/channels/700613971941785680/984083100520808478/1296527097593987208
I discovered that my "repeated" tag name is related to the variable "payloadTagName" now if I use this code
Copy code
# these variables are autogenerated from your instance location
let enterprise = "40Hero"
let site = "Delaware"
let area = "office"
let line = "iiotdemo"
let schema = "_historian"

let tagname = "mytag"
let payloadTagName = "myPayloadTagName"
# Directly modify the root object to remove timestamp and add timestamp_ms
let value = this.without("timestamp")  # Remove the timestamp field
value.timestamp_ms = this.timestamp  # Add new field timestamp_ms
I get this in the historian....
and now the new tag in the tag browser
more info....I deleted everything in the tag table and tag_string table.... weirdly I don't have any mytag$mytag names....but the tag browser shows what looks like there is?
j
Because the tag browser fetches the data from Kafka and not from the database. So that it can also show all the tags that not land up in the database. The issue with the repeated tag name stems from this: - one wants to see the tag name also in the topic so that one can filter / find it easily in MQTT (_historian.tagGroup.tagName) - but there is also a tag name “payload tag name” in the payload Any suggestions?
t
If i don’t include a tag name, it still repeats the topic structure after _historian
I like the implicit idea of using the topic structure as long as it doesn’t repeat. But it should also be an option to explicitly define the tag and grouping in the processor. It’s just unclear to me how they relate. The conditions of when and how it uses the topic implicitly or how it works explicitly. This case OR this case
Use topic structure if tag name and tag group are undefined in the processor.
If it fetches from kafka....why did deleting the tags in the DB delete everything else except this.... I used to have lots of tags....because i've been testing and it was a mess.
j
technically its "all new messages in kafka since the last companion restart". I am currently creating a PoC to fix it
but the PoC is technically complicated so it will not be easy :/ it involves adjusting kafka to allow to be again one MQTT topic = one Kafka Topic (without Kafka keys)
then we can just query all topics and we are ready to go
t
Ok now I understand
for reference, this is how I ended up getting what I desired.... "tagname" used as the "group" and "payloadTagName" used as my tag name, which is just the last part of the topic. Then each key from the payload shows below that.
Copy code
# these variables are autogenerated from your instance location
let enterprise = "40Hero"
let site = "Delaware"
let area = "office"
let line = "iiotdemo"
let schema = "_historian"

let tagname = "devices"
let mqtt_topic_appendix = meta("mqtt_topic").replace_all("40hero/delaware/office/iiotdemo/devices/", "")
let mqtt_topic_appendix_sanitized = $mqtt_topic_appendix.replace_all("/", ".")
let payloadTagName = $mqtt_topic_appendix_sanitized

let value = this.without("timestamp")  # Remove the timestamp field
value.timestamp_ms = this.timestamp  # Add new field timestamp_ms
b
The topic and message hierarchy should definitely represent current state, while the details should display the various methods to interact with the data. I think this could become a neat way to add third party plugins to the ui. So example would be in that topic it could represent all the Kafka topics and their state (depreciated, latest, etc), while another represents a graphql plugin to query the namespace (that being UMH is the only source for the api).
t
This…. https://learn.umh.app/lesson/data-modeling-in-the-unified-namespace-mqtt-kafka/ needs to be in the docs as a primary resource. It has a great explanation of the topic and payload structure. I found it in the docs as a link…. But it took me waaaay toooo long to stumble upon it
j
I am currently revising it all and putting it into an overall article on data modeling 😄 but interesting that this would have helped you at this point of time. What were you missing? Probably all the background information that not only tells you what to do, but also how to do it right?
Our idea was to: 1. have the tag browser show all current and previous tags (even across restarts) in kafka 2. when you click on a tag, be also able to see th real-time data (done) 3. when you click on a tag, be also able to query the historical data besides just showing the sql command 4. extend the filter function to allow also fitlering by "metadata" (so static information that might be appended to some payloads with the key "metadata") and allow more complex queries. Then you would be able to search for your tags like "select all tags where "type=energy-meter" and "site=cologne" 5. maybe provide this also as an API is this what you mean?
t
Good morning from the US! In this moment, step 2: topic hierarchy would have been very helpful. All of it is a good article and really helps to explain the "why" of your design choices, it's the "how do I work with it" that I was looking for.
b
1. I am torn on showing all pervious tags in the dropdown, maybe a different color to represent the topic's state? white v grey? 2. Love it 3. Love it more 4. cyberchef? Been trying to play with it but I think I need more data 5. This. I think that when you drill down to the topic, the API autopopulates the queries based on the topic you are in.