What are your thoughts on this? At UMH, we are "thinking through" features before we implement them. We are now thinking about how to progress for data modeling, and we have identified two types of data modeling: First: Device/Machine specific modeling. Where you model it into a physical location or sensor, e.g., _pump_v1 or something like this. You could also say modeling from "Bronze" to "Silver". The current pattern is using a bridge for this. And we ould improve our product here by allowing better integration fo this in the bridgecreation process, so that e.g., the conditions are precreated based on the model selected. So one selects a model, and then it autoamtically creates ffor you the "If tag_name == xxx, then data_contract=_pump_v1 and virtual_path="pump.pressure" and tagname="current". This is usual time-series. Second: Use-Case specific modeling Where you then take "Silver" data, so device/machine specific data, from the UNS, and apply a use-case specific transformation on it to convert it to "orders" or "maintenance requests". We do this at the moment using stream processors. We feel we are already strong for this on the time-series front, but need to develop somethign to convert time-series into relational. This is usually relational data. What are your thoughts on this?

There is in my opinion 2 different use cases for this. One is live data for triggering events like alarm, service etc. and the other is historical data for analyze. I am using both atm. Some data i cache in nodered and trigger events based on conditions. This is done only with live data and not from querying the DB. I also use historical data both with large queries for analyze and with different views for simpler queries (will be changed later to continuous aggregates).. It would be really nice to have inbuilt caching with conditions for live data. One way i use it is when some conditions are met, i push data to the DB. This reduces the DB quite significantly since i don't store a lot of useless data.

we are tracking this internally under: 1. add cache.set and cache.get to nodered_js/tag_processor 2. converting time-series data to relational data (e.g., if certain conditions are met, emit a maintenance order or an alert) that is what you mean right?

yes. type of. What is missing in my case atm is that even though i push data only at certain conditions it is still time series. This would benefit in being relational/wide format when doing later analyze of the data. Not sure on what the best approach would be to handle this. In our case we are joining other data from erp and other sources to get certain reports etc. Storing the data only in relational could be bad if wanting to have dashboards with graphs etc, but storing it both in timeseries and in relational creates duplicates which could cause other problems.

@DanielH are you looking for a feature to have conditions identified on when to historize the real-time data? Quick example, by default, don’t historize the pump power value, but when the valve is on, store the pump power value at some frequency?

yes. something like that. I am doing that for some machines today. We are polling opca-ua data constantly, but we only need a small amount of data like cycle done, cycle time, some settings data etc. so based on some conditions i only save data when cycle is done. i also poll some data from the erp that is combined with this data. This is to have easily readable dashboards and to ensure data is synced. For another machine we have several digital signals that needs to be cached and combined in order to calculate when the cycle is started, stopped and the cycletime. it makes no sense to push these digital signals to the historian since they are basically an extension of the plc.. This is today solved using nodered due to the complexity of caching and using contidioning in benthos. I have tested it in benthos and it works fine there as well but the increased complexity can be a problem when someone else has to manage the system. Nodred is easier in that way but gets messy quick when scaling.

Yeah, we thought about like this: 1. you basically want to transform time-series data to relational data based on a condition. 2. To decide whether to store or not to store a value you can use data contracts. 3. We need to implement proper caching cache.get and cache.set in benthos to make it not super cursed to do caching there 4. We will add a sub-feature of the stream processor to allow sending out conditional events in triggers. 5. So you would subscribe to all the cycle data and load it into the local cache upon startup (so every time you restart you continue where you left off) 6. When a new message comes in, and a condition is true, you can then fetch all the old values from the cache and send out a new payload 7. That payload can be enforced in a data contract to ensure its validity (which is btw already possible right now!)

That seems quite similar to my approach. On thing that can be a bit tricky/important is for example if you have for example 10 values from a machine using subscription and 1-2 values rarely change. It could be extraHeating that you use occasionally. If that is turned on week 12 it pushes a data point. Every cycle you read the cache and save data to db on cycleDone, including extraHeating since you want to know the status on that for every cycle. If the extra heating is turned on for 20w it newer sends a new data point. This makes the cache important and you need to ensure the cache data is still valid and up to date. Using a traditional historian with pub/sub could make it cumbersome to visualize the data if the latest extraHeating is 20w old. For push/pull data this is easier since you can fetch the latest data constantly to ensure the validity of the cache. For a pub/sub setup it might be an idea to check the timestamps of the data and if older than a certain time you switch to pulling and fetch the latest data and switch back to subscription.