Starting a new thread about Benthos
# umh-support
d
Starting a new thread about Benthos development with some questions about best practices in developing custom Data Flow Components
I'm editing a copy of the kafka_to_timescale_mes_work_order_bridge. To make bloblang easier, I want to edit the code in Benthos Studio. I've copied the entire YAML code into a file to get the flow as shown in the picture. Now, because Benthos Studio can't directly connect to Repanda inside the UMH, I want to mock the output from the input node (in the left column "Input" in the image). This way, the subsequent node can process this output and show their intermediate results, making development easier. The problem is that I'm not sure what the output from that Input node looks like. I know that the input is essentially a Kafka message that I can download from Redpanda. According to ChatGPT, the output message will be:
Copy code
{
  "actual_quantity": 0,
  "asset_id": 1,
  "created_at": "2024-12-11T10:22:14.441Z",
  "end_time": "2024-12-11T10:22:14.441Z",
  "operation_id": 6660,
  "order_id": 5550,
  "scrap_quantity": 0,
  "sequence_number": 1,
  "start_time": "2024-12-11T10:22:14.441Z",
  "target_quantity": 100,
  "metadata": {
    "kafka_topic": "umh.v1.Belgium",
    "kafka_partition": 2,
    "kafka_offset": 10,
    "kafka_key": "Antwerp.Kalmthout.Heide.Begonia._mes.operation",
    "bridged_by": "[\"mqtt_to_kafka_bridge_mes_default\"]",
    "parsing_failure": "false"
  }
}
However, in the subsequent node, the
kafka_topic
cannot be extracted:
let kafka_topic = meta("kafka_topic")
returns
null
, crashing the pipeline. Essentially, I'm looking for a way to log I/O and variables to the console for debugging. Has anyone found a way to do this? https://cdn.discordapp.com/attachments/1316366172907638835/1316368299596189727/image.png?ex=675acb03&is=67597983&hm=0bab6aba35b84c33beea931386cae17d20a6d1bc81716d6a7c00e09bf7260aa9&
In the meantime, I'll just continue editing the YAML in the Management Console directly
d
output: stdout: {} logger: level: DEBUG format: logfmt add_timestamp: true does that not work ?
d
No, it doesn't produce output Ah, I didn't have the
stdout: {}
before. Let me try that
Where do you add this in the YAML though?
d
in the output section
d
This works, thanks
This is the preceding bridge that transfers data from MQTT to kafka:
2 Views