I've just started exploring adding OpenTelemetry support to the Comet subproject of DataFusion. I'm excited to see the integration with Apache Arrow (Rust) and potentially DataFusion in the future.
Integrations with OTLP are critical to driving adoption and probably one of the biggest pain points we've encountered when adopting it ourselves (and encouraging others to the same).
Adopting OTLP without third-party support is pretty time consuming, especially is your tech stack is large and/or varied.
Re runtimes: curious about this too. Feels like the right direction if you’re optimizing a telemetry pipeline.
Wow, anyone able to provide a ELI5? OTel sounds amazing but this is flying over my head
Man Ive dipped my toes into this too, and yeah, the way everyone wants different things always shakes things up fast. Kinda love seeing where it all ends up tbh.
> We are interested in making OTAP pipelines safely embeddable, through strict controls on memory and through support for thread-per-core runtimes.
I'm curious about the thread-per-core runtimes, are there even any mature thread-per-core runtimes in Rust around?
[dead]
Damn that's some scope creep if I ever saw it: 'try sending Arrow frames end to end' => 'rewrite the otel pipeline in rust'. Seems like the goals of the contributors don't exactly align with the goals of the project.
Kind of a bummer - one thing i was hoping to come out of this was better Arrow ecosystem support for golang.