UMH parquet
# generald
Diederik
12/16/2024, 10:13 AMhow would you guys go from "UMH" to parquet files?
costumer wants to do analysis in azure and i have to deliver data in parquet format on a blob storage
j
Jermuk
12/16/2024, 10:27 AMSome thoughts:
- the paid version of timescaledb with tiered storage used parquet: https://docs.timescale.com/use-timescale/latest/data-tiering/about-data-tiering/
- there is a benthos parquet processor https://docs.redpanda.com/redpanda-connect/components/processors/parquet_encode/
d
Diederik
12/16/2024, 10:33 AMwould you deploy this as DFC? Kafka_to_parquet_bridge ish?
Diederik
12/16/2024, 10:37 AMcombining this as cold and hot path to azure
would you subscribe to certain topics on the kafka/mqtt stream and publish them to parquet for cold path and iot/event hub for hot path?
Diederik
12/16/2024, 10:38 AMboth using DFC?
Diederik
12/16/2024, 10:39 AMthis is in prep for an UMH demo for the costumer i've contacted @Jermuk and @Alex about! 😉
j
Jermuk
12/16/2024, 3:18 PMyes, exactly!
Jermuk
12/16/2024, 3:19 PM