Root cause analysis / Investigation

api.x.ai

Reference RCA-213CDFCECAA0E728

xAI API endpoint

Measurement point: Stealth NYC, 32 Avenue of the Americas.

Published investigation. Updates run automatically.

01 / Network path

Network path.

Endpoint measurements / ICMP IPv4

Latency over time.

Successful endpoint RTT recordings from up to one hour before and after the event.

Enter time range

Times use the selected timezone; choices include recorded samples and intervals across the available chart.

Use the recordings table below to view latency values.

Successful RTT recording Event interval

29 Oct 2026 18:10 – 29 Oct 2026 20:59 UTC. Hover or tap to inspect; drag across the chart to zoom. On touchscreens, first choose Select time range. Escape or Reset zoom restores the full range. Missing recordings are gaps; this is not an application availability or packet-loss chart.

Select a recording to explore nearby path observations.

View recorded values
Successful ICMP IPv4 RTT recordings.
Recorded at (UTC)RTT
1.370 ms
1.423 ms
1.475 ms
1.405 ms
1.410 ms
1.352 ms
1.405 ms
1.378 ms
1.362 ms
1.453 ms
1.389 ms
1.521 ms
1.330 ms
1.448 ms
1.377 ms
1.388 ms
2.512 ms
1.576 ms
1.563 ms
1.370 ms
1.454 ms
1.400 ms
1.411 ms
1.379 ms
1.420 ms
1.370 ms
1.385 ms
1.372 ms
1.511 ms
1.433 ms
1.484 ms
1.386 ms
1.488 ms
1.471 ms
1.578 ms
1.373 ms
1.405 ms
2.669 ms
1.391 ms
1.378 ms
1.370 ms
1.385 ms
1.355 ms
1.355 ms
1.362 ms
1.385 ms
1.430 ms
1.396 ms
1.386 ms
1.388 ms
1.305 ms
1.389 ms
1.356 ms
1.373 ms
1.424 ms
1.461 ms
1.356 ms
1.411 ms
1.323 ms
1.428 ms
1.376 ms
1.379 ms
1.329 ms
1.354 ms
1.352 ms
1.344 ms
1.362 ms
1.412 ms
1.372 ms
1.369 ms
1.587 ms
1.398 ms
1.379 ms
1.339 ms
1.462 ms
1.351 ms
1.568 ms
1.394 ms
1.440 ms
1.385 ms
2.799 ms
1.540 ms
1.439 ms
1.398 ms
1.322 ms
1.356 ms
2.120 ms
1.546 ms
1.363 ms
1.375 ms
1.400 ms
1.364 ms
1.384 ms
1.382 ms
1.338 ms
1.415 ms
1.366 ms
1.347 ms
1.367 ms
1.406 ms
1.375 ms
1.485 ms
1.372 ms
1.376 ms
1.361 ms
1.362 ms
1.375 ms
1.335 ms
1.325 ms
1.398 ms
3.911 ms
1.386 ms
1.453 ms
1.381 ms
1.422 ms
1.395 ms
1.541 ms
1.341 ms
1.396 ms
1.421 ms
1.466 ms
1.388 ms
1.413 ms
1.444 ms
18.337 ms
16.503 ms
1.424 ms
1.358 ms
1.455 ms
7.452 ms
19.780 ms
10.485 ms
4.887 ms
1.404 ms
2.035 ms
9.893 ms
1.347 ms
11.175 ms
1.422 ms
1.359 ms
5.418 ms
1.436 ms
12.203 ms
1.476 ms
3.038 ms
15.225 ms
1.367 ms
1.342 ms
1.417 ms
1.402 ms
1.385 ms
1.380 ms
3.194 ms
2.008 ms
1.454 ms
1.392 ms
1.438 ms
1.377 ms
1.382 ms
1.415 ms
1.337 ms
1.452 ms
6.079 ms
1.380 ms
1.384 ms
1.342 ms
1.335 ms
1.378 ms
1.384 ms
1.396 ms
1.439 ms
1.433 ms
1.448 ms
1.365 ms
1.515 ms
1.363 ms
1.382 ms
1.381 ms
1.357 ms
1.439 ms
1.614 ms
1.958 ms
1.312 ms
1.373 ms
1.403 ms
1.393 ms
1.350 ms
1.340 ms
5.685 ms
1.345 ms
1.367 ms
1.436 ms
1.372 ms
1.374 ms
1.358 ms
1.365 ms
1.369 ms
1.374 ms
1.383 ms
1.355 ms
1.465 ms
1.495 ms
1.368 ms
1.376 ms
1.409 ms
1.391 ms
1.335 ms
1.432 ms
1.545 ms
1.455 ms
1.371 ms
1.357 ms
1.375 ms
1.428 ms
1.582 ms
1.435 ms
1.338 ms
1.387 ms
1.442 ms
1.492 ms
1.346 ms
1.512 ms
1.429 ms
1.362 ms
1.392 ms
1.350 ms
1.362 ms
1.463 ms
1.319 ms
1.427 ms
1.423 ms
1.355 ms
1.391 ms
1.345 ms
1.372 ms
1.407 ms
1.338 ms
2.053 ms
1.533 ms
1.372 ms
4.782 ms
1.436 ms
1.353 ms
1.386 ms
1.510 ms
1.436 ms
1.325 ms
1.377 ms
1.393 ms
1.446 ms
1.375 ms
1.437 ms
1.375 ms
1.497 ms
1.395 ms
1.377 ms
1.340 ms
1.387 ms
1.366 ms
1.350 ms
1.347 ms
1.415 ms
1.391 ms
1.362 ms
1.397 ms
1.479 ms
1.438 ms
1.345 ms
1.415 ms
1.486 ms
1.379 ms
1.364 ms
1.371 ms
1.326 ms
1.445 ms
1.348 ms
1.397 ms
1.430 ms
1.375 ms
1.324 ms
1.327 ms
1.596 ms
1.333 ms
1.408 ms
1.358 ms
1.385 ms
1.432 ms
1.365 ms
1.388 ms
1.388 ms
1.313 ms
1.421 ms
1.490 ms
1.399 ms
1.463 ms
1.347 ms
1.383 ms
1.421 ms
1.389 ms
1.386 ms
1.400 ms
1.384 ms
1.340 ms
1.387 ms
1.410 ms
1.331 ms
1.396 ms
1.406 ms
1.342 ms
1.427 ms
1.464 ms
1.335 ms
1.344 ms
1.360 ms
1.454 ms
1.363 ms
1.433 ms
1.464 ms
1.484 ms
1.473 ms
1.347 ms
1.365 ms
1.414 ms
1.347 ms
1.370 ms
1.343 ms
1.375 ms
1.401 ms
1.480 ms
1.367 ms
1.392 ms
1.386 ms
1.448 ms
1.342 ms
1.352 ms
1.356 ms
1.435 ms
1.395 ms
1.438 ms
Network path / recorded observations

Network path observations.

17 observations

Choose a measurement type to see the closest of the available sampled observations to the selected chart point. Each probe is separate and may follow a different path.

Before

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.123 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.189 msReply RTT AS8002 · Stealth
Before

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.148 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.177 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop1.705 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.477 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.526 msReply RTT AS13335 · Cloudflare
Before

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.127 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.181 msReply RTT AS8002 · Stealth
Before

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.176 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.256 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop2.234 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.348 msReply RTT AS13335 · Cloudflare
Before

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.138 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.182 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop2.557 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.492 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.137 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.174 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop22.304 msReply RTT AS13335 · Cloudflare
During

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.149 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.183 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop1.255 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.606 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Probe hopNo replyNo comparable RTT ASN unavailable
  7. TTL 7 Destination19.030 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.127 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.172 msReply RTT AS8002 · Stealth
During

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.228 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.203 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop14.476 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.235 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.138 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.236 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop22.889 msReply RTT AS13335 · Cloudflare
During

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.151 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.189 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop16.305 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop2.006 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.645 msReply RTT AS13335 · Cloudflare
After

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.197 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.330 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop0.871 msReply RTT AS13335 · Cloudflare
After

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.159 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.182 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop5.322 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop4.378 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.383 msReply RTT AS13335 · Cloudflare
After

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.132 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.179 msReply RTT AS8002 · Stealth
After

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.162 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.278 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop1.583 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.722 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination8.861 msReply RTT AS13335 · Cloudflare
After

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.123 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.182 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop3.119 msReply RTT AS13335 · Cloudflare
After

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.266 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.270 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop0.980 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop4.626 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.468 msReply RTT AS13335 · Cloudflare

RTT is the round trip to each responding hop, not the delay of the preceding link. TTL order may omit routers and does not prove direct links. A missing reply is not proof of packet loss. ASN labels are dated address associations; they do not establish the forwarding path or the operator at the event time. Up to three recorded traces per method and period are selected across the archive. Where no individual trace is available, a comparison may summarize a window; MTR observations aggregate multiple probes. Address aliases are independent across measurement types.

Same method / same recorded address

Compare the traces.

Compare one selected trace from before the event with one during it. Values are individual replies, not a statistical comparison.

ICMP traceroute · Recorded address 2

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.127 msAS8002 · Stealth0.138 msAS8002 · StealthSame responder
20.181 msAS8002 · Stealth0.236 msAS8002 · StealthSame responder
3Not recorded22.889 msAS13335 · CloudflareNot established
TCP traceroute / 443 · Recorded address 1

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.138 msAS8002 · Stealth0.149 msAS8002 · StealthSame responder
20.182 msAS8002 · Stealth0.183 msAS8002 · StealthSame responder
3No reply1.255 msAS13335 · CloudflareNot established
4No replyNo replyNot established
52.557 msAS13335 · Cloudflare1.606 msAS13335 · CloudflareDifferent responders
61.492 msAS13335 · CloudflareNo replyNot established
7Not recorded19.030 msAS13335 · CloudflareNot established
TCP traceroute / 443 · Recorded address 2

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.148 msAS8002 · Stealth0.151 msAS8002 · StealthSame responder
20.177 msAS8002 · Stealth0.189 msAS8002 · StealthSame responder
31.705 msAS13335 · Cloudflare16.305 msAS13335 · CloudflareSame responder
4No replyNo replyNot established
51.477 msAS13335 · Cloudflare2.006 msAS13335 · CloudflareDifferent responders
61.526 msAS13335 · Cloudflare1.645 msAS13335 · CloudflareSame responder

RTTs include the return path. Different responders, missing replies and independent probe flows limit what a change can localize. MTR snapshots remain separate in the replay.

Event timeline.

Possible SLA event. Times shown are UTC.

Event opened
UTC
Last event observation
UTC
Recovery observed
UTC

Recorded recovery describes this event, not current reachability.

02 / Assessment

Assessment.

Investigation stateRecovered
Path assessment

Elevated latency observed.

Endpoint round-trip time was elevated during part of the event. The available path observations do not identify where the increase occurred.

What we observed

  • A sustained increase in measured round trip time was observed.

Recording counts describe collected measurements, not application availability or packet loss. Exact packet paths and the return path remain unverified.

Measurements at a glance.

Baseline RTT1.390 msUsual round trip before the event
Last recorded event RTT1.492 msLast successful event recording
Event p95 RTT11.226 ms95th percentile of successful recordings
Change from baseline+0.101 msLast event RTT compared with the baseline
Successful / expected recordings100 / 100Recorded endpoint samples during the event

RTT is round trip time; p95 describes the upper range of successful recordings. Endpoint measurements differ from individual trace replies and are not an uptime percentage.

Record sources and timestamps
Analysis updated
UTC
Public snapshot published
UTC
Public observation window
– UTC

Event and source timestamps are UTC. The latency chart uses its selected timezone. Page refresh times are separate from measurement timestamps.

03 / Method and limits

Method and limits.

How to interpret the sampled path and its limits

Corroborating the path.

Comparisons use matching destinations before and during the event. ICMP traces use echo probes. TCP/443 traces send SYN probes to port 443 only; they do not complete a TCP handshake or perform a TLS or HTTP exchange.

The Stealth gNMI interface verifies the active route-table choice and maps its primary next hop to the Stealth logical interface, configured attachment and, where applicable, the known LAG member set at each snapshot. Stable Stealth replies, together with mapped interface evidence showing no corresponding errors or sustained capacity pressure, reduce the likelihood of a Stealth forwarding issue. An 80% load warning warrants investigation; it does not prove congestion. Lower interval averages can conceal bursts or queue pressure. Optical readings assess the physical link, not congestion.

What remains unseen.

Probe reply handling can differ from packet forwarding. A slow intermediate reply that disappears at later hops does not locate a forwarding problem. A destination that does not respond to a trace is not thereby shown to be unavailable.

Stealth does not use ECMP in this monitored scope. The route snapshot does not identify which physical LAG member carried a particular independent probe, attest line-card forwarding at that packet instant, prove the exact application packet flow or observe the return path. RTT includes the return path, which may differ. No remote reverse traceroute is available. Tunnels and silent routers may hide parts of the path. Address registration alone does not establish the forwarding operator or responsibility.

References: ICMP reply handling / RFC 1812 · Round trip measurements / RFC 2681 · Full methodology