My question about connecting directly to Kafka and about how best to connect to the data layer came from knowing that there are native protocols for Siemens and Beckhoff, and that from looking at the system diagrams, it looked to me like the MQTT broker might not be providing any value if I could instead send some of the data directly to Kafka.
There is no native Codesys protocol like S7 or ADS, but with the ctrlX data layer, it would be very possible to write something similar.
My understanding of MQTT is that it sends a single message per updated value, so if I want to update 250 IO on a 250ms cycle, that's 1000 messages per second.
It seems to me like if I could instead write a protocol that would send all 250 values as one message - or some other non-arbitrary number of messages - that it may require less communication overhead and processing.