400 Bad Request - There Was An Error Returned Querying The Prometheus Api -

curl -G http://localhost:9090/api/v1/query \ --data-urlencode 'query=upjob="node"' (This is the safest way to use curl with Prometheus.) Sometimes, it genuinely isn't your fault. A bug in a third-party exporter or a service discovery crash can inject NaN or Inf values into the label set. If your query tries to filter on a label that contains a newline character ( \n ) or a control character, the JSON marshaller fails.

Prometheus expects a Content-Type header for POST queries ( application/x-www-form-urlencoded ). If the proxy manipulates this, Prometheus throws a 400. Prometheus expects a Content-Type header for POST queries

Check your up metric. If you see weird label values, you have found your problem. The "400 Bad Request" error is Prometheus's way of saying, "I don't understand what you want, and I refuse to guess." It is frustrating, but it is safe . It protects your Prometheus instance from crashing due to bad input. If you see weird label values, you have found your problem