OTLP - was working not now - more testers needed

[Southside]

Anybody had a nosey? First spotted in the Update 21 July, 2022

and a brief intro at safe_network/sn_node at main · maidsafe/safe_network · GitHub
I am building at the moment with
cargo build --release --features statemap,otlp
so I’ll be all set when I get it configured.
https://opentelemetry.io/(upload://i2e1RTw4tiTr1IYDn8n6RQvlmJp.png)
safe_network/sn_node at main · maidsafe/safe_network · GitHub

It installs very easily with docker but my first attempt at simply dropping one of the genesis node logs onto the GUI failed. A tad hopeful I know Time for some quality RTFM time …

Anything that makes analysing logs easier has to be worthwhile.

EDIT: hit this snag Starting GRPC server 16685 connection refused · Issue #3852 · jaegertracing/jaeger · GitHub
seems to affect all versions of the all-in-one install

[Southside]

It works if you add another port mapping for 16685

docker run --name jaeger \
  -e COLLECTOR_OTLP_ENABLED=true \
  -p 16686:16686 \
  -p 16685:16685 \
  -p 4317:4317 \
  -p 4318:4318 \
  jaegertracing/all-in-one:1.36

Lots to explore with this tool so have a go at it yourself.

Just follow safe_network/sn_node at main · maidsafe/safe_network · GitHub and add the extra port mapping

[bzee]

Seems like the documentation from Jaeger has also been updated (v1.38):

Getting Started

docker run -d --name jaeger \
  -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
  -e COLLECTOR_OTLP_ENABLED=true \
  -p 6831:6831/udp \
  -p 6832:6832/udp \
  -p 5778:5778 \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 14250:14250 \
  -p 14268:14268 \
  -p 14269:14269 \
  -p 9411:9411 \
  jaegertracing/all-in-one:1.38

[Southside]

@bzee

jaegertracing/all-in-one:1.38

I’m only seeing the jaeger-query service with that command string. I f I add the 16685 port it still fails…

Each time I check, I stop the SAFE network and clear the relevant dirs in ~/.safe/node, run the jaeger docker command, restart SAFE and check at localhost:16686.

Really need a few folk testing this - Am I doing something wrong here or is this problem replicable?

[Shu]

I was trying to get sn_node → OTLP → Prometheus → Grafana today.

I ended up seeing the traces come into OTLP (separate process) from sn_node, but could not get them to export to Prometheus, which was initially configured as the data source backend for Grafana.

Then I ended up realizing the data type coming in were traces, and not metrics, and I believe OTLP’s prometheus export does not support traces at this time.

I then ended up going back to:
sn_node → jaeger + otlp (stand alone binary), which I was able to successfully setup at home on a local testnet (alpine container), as seen below:

Versions:
jaeger:1.42.0, sn_node: 0.77.10, safe: 0.72.2

Very cool to see this working at home, thank you MaidSafe team!