Root cause analysis / Investigation

api.x.ai

Reference RCA-C85EB73DFB04C7E7

xAI API endpoint

Measurement point: Stealth NYC, 32 Avenue of the Americas.

Published investigation. Updates run automatically.

01 / Network path

Network path.

Endpoint measurements / ICMP IPv4

Latency over time.

Successful endpoint RTT recordings from up to one hour before and after the event.

Enter time range

Times use the selected timezone; choices include recorded samples and intervals across the available chart.

Use the recordings table below to view latency values.

Successful RTT recording Event interval

05 Nov 2026 16:55 – 05 Nov 2026 19:24 UTC. Hover or tap to inspect; drag across the chart to zoom. On touchscreens, first choose Select time range. Escape or Reset zoom restores the full range. Missing recordings are gaps; this is not an application availability or packet-loss chart.

Select a recording to explore nearby path observations.

View recorded values
Successful ICMP IPv4 RTT recordings.
Recorded at (UTC)RTT
1.359 ms
1.451 ms
1.349 ms
1.340 ms
1.542 ms
1.525 ms
1.336 ms
1.434 ms
1.460 ms
1.354 ms
1.376 ms
1.521 ms
1.632 ms
1.339 ms
1.449 ms
1.527 ms
1.372 ms
1.393 ms
1.568 ms
1.618 ms
1.356 ms
1.360 ms
1.525 ms
1.411 ms
1.381 ms
1.505 ms
1.300 ms
1.411 ms
1.437 ms
1.402 ms
1.330 ms
1.389 ms
1.578 ms
1.403 ms
1.487 ms
1.387 ms
1.311 ms
1.592 ms
1.340 ms
1.919 ms
1.327 ms
1.329 ms
1.440 ms
1.323 ms
1.312 ms
4.509 ms
1.382 ms
1.344 ms
1.351 ms
1.348 ms
1.385 ms
1.314 ms
1.346 ms
1.369 ms
1.319 ms
1.357 ms
1.420 ms
1.377 ms
1.378 ms
1.433 ms
1.365 ms
1.393 ms
1.397 ms
1.570 ms
1.320 ms
1.512 ms
2.130 ms
1.394 ms
1.317 ms
1.407 ms
1.343 ms
1.334 ms
1.358 ms
1.356 ms
1.591 ms
1.454 ms
12.263 ms
1.387 ms
1.447 ms
1.488 ms
1.425 ms
1.585 ms
1.372 ms
2.005 ms
5.924 ms
2.241 ms
1.358 ms
1.412 ms
1.412 ms
1.385 ms
1.446 ms
1.393 ms
1.327 ms
1.365 ms
1.439 ms
1.481 ms
1.296 ms
1.445 ms
2.075 ms
1.366 ms
19.034 ms
1.300 ms
6.573 ms
1.379 ms
1.316 ms
1.321 ms
1.357 ms
1.442 ms
1.377 ms
1.334 ms
1.879 ms
1.358 ms
2.335 ms
1.528 ms
1.357 ms
1.479 ms
1.327 ms
1.429 ms
2.456 ms
1.339 ms
1.361 ms
1.716 ms
1.563 ms
1.400 ms
1.479 ms
1.596 ms
1.357 ms
1.328 ms
7.526 ms
1.332 ms
1.301 ms
1.303 ms
1.359 ms
1.588 ms
1.546 ms
1.397 ms
1.349 ms
1.338 ms
9.836 ms
1.406 ms
2.833 ms
1.458 ms
1.399 ms
2.157 ms
8.705 ms
1.350 ms
1.336 ms
1.365 ms
1.385 ms
2.874 ms
1.376 ms
1.488 ms
1.416 ms
1.319 ms
1.355 ms
4.009 ms
1.480 ms
1.396 ms
1.349 ms
1.337 ms
1.436 ms
1.373 ms
1.438 ms
1.404 ms
1.336 ms
1.313 ms
1.351 ms
1.480 ms
1.423 ms
1.316 ms
1.328 ms
1.342 ms
1.367 ms
1.408 ms
1.400 ms
1.437 ms
1.316 ms
1.412 ms
1.330 ms
1.443 ms
1.539 ms
1.418 ms
1.415 ms
1.324 ms
1.322 ms
1.369 ms
2.895 ms
1.368 ms
1.397 ms
1.337 ms
1.364 ms
1.414 ms
1.407 ms
1.344 ms
1.384 ms
1.427 ms
1.449 ms
1.343 ms
1.525 ms
1.434 ms
1.377 ms
1.383 ms
1.682 ms
1.442 ms
1.329 ms
1.432 ms
1.370 ms
1.350 ms
1.353 ms
1.352 ms
1.367 ms
1.383 ms
1.402 ms
1.400 ms
1.388 ms
1.676 ms
1.393 ms
7.130 ms
1.388 ms
1.327 ms
1.375 ms
1.375 ms
1.457 ms
1.401 ms
1.365 ms
1.530 ms
1.600 ms
1.479 ms
1.476 ms
3.718 ms
1.404 ms
1.416 ms
1.435 ms
1.365 ms
1.342 ms
1.689 ms
1.343 ms
1.473 ms
1.445 ms
6.395 ms
1.358 ms
1.644 ms
1.367 ms
1.444 ms
1.332 ms
5.963 ms
1.363 ms
1.360 ms
1.418 ms
1.424 ms
1.381 ms
1.469 ms
1.330 ms
1.403 ms
1.311 ms
1.362 ms
1.360 ms
1.348 ms
1.318 ms
1.312 ms
1.435 ms
1.512 ms
1.393 ms
1.332 ms
1.381 ms
1.544 ms
1.545 ms
1.350 ms
1.364 ms
1.334 ms
1.362 ms
1.564 ms
7.911 ms
1.403 ms
1.406 ms
1.371 ms
1.438 ms
4.067 ms
1.521 ms
1.385 ms
1.364 ms
1.343 ms
1.340 ms
1.411 ms
1.359 ms
1.512 ms
1.323 ms
1.480 ms
1.595 ms
1.682 ms
1.405 ms
1.429 ms
1.335 ms
1.346 ms
1.347 ms
1.348 ms
1.579 ms
1.346 ms
1.341 ms
Network path / recorded observations

Network path observations.

17 observations

Choose a measurement type to see the closest of the available sampled observations to the selected chart point. Each probe is separate and may follow a different path.

Before

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.140 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.176 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop18.442 msReply RTT AS13335 · Cloudflare
Before

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.145 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.191 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop17.837 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.952 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Probe hopNo replyNo comparable RTT ASN unavailable
  7. TTL 7 Destination1.963 msReply RTT AS13335 · Cloudflare
Before

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.113 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.172 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop17.447 msReply RTT ASN unavailable
Before

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.151 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.193 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop16.732 msReply RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop4.616 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Probe hopNo replyNo comparable RTT ASN unavailable
  7. TTL 7 Destination1.294 msReply RTT AS13335 · Cloudflare
Before

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.188 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.297 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop48.303 msReply RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.841 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination12.820 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.125 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.293 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop13.971 msReply RTT ASN unavailable
During

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.201 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.421 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop11.306 msReply RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.319 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.458 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.138 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.186 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop15.899 msReply RTT ASN unavailable
During

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.196 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.339 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop53.000 msReply RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop17.389 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.429 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.116 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.172 msReply RTT AS8002 · Stealth
During

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.150 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.190 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop10.267 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.420 msReply RTT AS13335 · Cloudflare
After

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.112 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.178 msReply RTT AS8002 · Stealth
After

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.176 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.181 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hop1.816 msReply RTT AS13335 · Cloudflare
  5. TTL 5 Probe hop9.756 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.493 msReply RTT AS13335 · Cloudflare
After

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.123 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.182 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop4.460 msReply RTT ASN unavailable
After

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.242 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.193 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop1.774 msReply RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop2.161 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.515 msReply RTT AS13335 · Cloudflare
After

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.123 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.180 msReply RTT AS8002 · Stealth
After

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.184 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.203 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop16.469 msReply RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop14.174 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination12.674 msReply RTT AS13335 · Cloudflare

RTT is the round trip to each responding hop, not the delay of the preceding link. TTL order may omit routers and does not prove direct links. A missing reply is not proof of packet loss. ASN labels are dated address associations; they do not establish the forwarding path or the operator at the event time. Up to three recorded traces per method and period are selected across the archive. Where no individual trace is available, a comparison may summarize a window; MTR observations aggregate multiple probes. Address aliases are independent across measurement types.

Same method / same recorded address

Compare the traces.

Compare one selected trace from before the event with one during it. Values are individual replies, not a statistical comparison.

ICMP traceroute · Recorded address 1

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.113 msAS8002 · Stealth0.116 msAS8002 · StealthSame responder
20.172 msAS8002 · Stealth0.172 msAS8002 · StealthSame responder
317.447 msNot recordedNot established
ICMP traceroute · Recorded address 2

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.140 msAS8002 · Stealth0.125 msAS8002 · StealthSame responder
20.176 msAS8002 · Stealth0.293 msAS8002 · StealthSame responder
318.442 msAS13335 · Cloudflare13.971 msSame responder
TCP traceroute / 443 · Recorded address 1

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.188 msAS8002 · Stealth0.196 msAS8002 · StealthSame responder
20.297 msAS8002 · Stealth0.339 msAS8002 · StealthSame responder
348.303 ms53.000 msSame responder
4No replyNo replyNot established
51.841 msAS13335 · Cloudflare17.389 msAS13335 · CloudflareDifferent responders
612.820 msAS13335 · Cloudflare1.429 msAS13335 · CloudflareSame responder

RTTs include the return path. Different responders, missing replies and independent probe flows limit what a change can localize. MTR snapshots remain separate in the replay.

Event timeline.

Latency anomaly. Times shown are UTC.

Event opened
UTC
Last event observation
UTC
Recovery observed
UTC

Recorded recovery describes this event, not current reachability.

02 / Assessment

Assessment.

Investigation stateRecovered
Path assessment

Elevated latency observed.

Endpoint round-trip time was elevated during part of the event. The available path observations do not identify where the increase occurred.

What we observed

  • A sustained increase in measured round trip time was observed.
  • Stored traces were compared using matching source and destination addresses.

Recording counts describe collected measurements, not application availability or packet loss. Exact packet paths and the return path remain unverified.

Measurements at a glance.

Baseline RTT1.354 msUsual round trip before the event
Last recorded event RTT1.443 msLast successful event recording
Event p95 RTT4.185 ms95th percentile of successful recordings
Change from baseline+0.089 msLast event RTT compared with the baseline
Successful / expected recordings60 / 60Recorded endpoint samples during the event

RTT is round trip time; p95 describes the upper range of successful recordings. Endpoint measurements differ from individual trace replies and are not an uptime percentage.

Record sources and timestamps
Analysis updated
UTC
Public snapshot published
UTC
Public observation window
– UTC

Event and source timestamps are UTC. The latency chart uses its selected timezone. Page refresh times are separate from measurement timestamps.

03 / Method and limits

Method and limits.

How to interpret the sampled path and its limits

Corroborating the path.

Comparisons use matching destinations before and during the event. ICMP traces use echo probes. TCP/443 traces send SYN probes to port 443 only; they do not complete a TCP handshake or perform a TLS or HTTP exchange.

The Stealth gNMI interface verifies the active route-table choice and maps its primary next hop to the Stealth logical interface, configured attachment and, where applicable, the known LAG member set at each snapshot. Stable Stealth replies, together with mapped interface evidence showing no corresponding errors or sustained capacity pressure, reduce the likelihood of a Stealth forwarding issue. An 80% load warning warrants investigation; it does not prove congestion. Lower interval averages can conceal bursts or queue pressure. Optical readings assess the physical link, not congestion.

What remains unseen.

Probe reply handling can differ from packet forwarding. A slow intermediate reply that disappears at later hops does not locate a forwarding problem. A destination that does not respond to a trace is not thereby shown to be unavailable.

Stealth does not use ECMP in this monitored scope. The route snapshot does not identify which physical LAG member carried a particular independent probe, attest line-card forwarding at that packet instant, prove the exact application packet flow or observe the return path. RTT includes the return path, which may differ. No remote reverse traceroute is available. Tunnels and silent routers may hide parts of the path. Address registration alone does not establish the forwarding operator or responsibility.

References: ICMP reply handling / RFC 1812 · Round trip measurements / RFC 2681 · Full methodology