Root cause analysis / Investigation

copilot.microsoft.com

Reference RCA-65F3246F94E644CA

Microsoft Chat website

Measurement point: Stealth NYC, 32 Avenue of the Americas.

Published investigation. Updates run automatically.

01 / Network path

Network path.

Endpoint measurements / ICMP IPv4

Latency over time.

Successful endpoint RTT recordings from up to one hour before the event.

Enter time range

Times use the selected timezone; choices include recorded samples and intervals across the available chart.

Use the recordings table below to view latency values.

Successful RTT recording Event interval

09 Nov 2026 20:00 – 09 Nov 2026 21:34 UTC. Hover or tap to inspect; drag across the chart to zoom. On touchscreens, first choose Select time range. Escape or Reset zoom restores the full range. Missing recordings are gaps; this is not an application availability or packet-loss chart.

Select a recording to explore nearby path observations.

View recorded values
Successful ICMP IPv4 RTT recordings.
Recorded at (UTC)RTT
1.021 ms
0.972 ms
1.941 ms
1.529 ms
1.348 ms
1.021 ms
1.359 ms
1.523 ms
1.429 ms
1.016 ms
1.501 ms
16.604 ms
5.194 ms
1.000 ms
0.961 ms
17.401 ms
1.903 ms
0.963 ms
1.021 ms
0.961 ms
1.382 ms
1.446 ms
0.962 ms
0.975 ms
1.429 ms
0.952 ms
1.416 ms
1.381 ms
19.659 ms
1.080 ms
1.343 ms
0.962 ms
1.325 ms
1.104 ms
1.596 ms
1.104 ms
0.990 ms
1.365 ms
1.371 ms
1.354 ms
1.473 ms
1.091 ms
1.380 ms
1.435 ms
0.950 ms
1.405 ms
1.376 ms
1.064 ms
1.369 ms
1.059 ms
1.355 ms
1.011 ms
1.098 ms
1.440 ms
1.472 ms
2.263 ms
1.046 ms
1.030 ms
1.032 ms
1.405 ms
0.950 ms
1.418 ms
0.995 ms
1.506 ms
1.032 ms
1.425 ms
1.069 ms
1.372 ms
1.394 ms
0.995 ms
1.403 ms
0.958 ms
1.497 ms
0.967 ms
1.123 ms
0.975 ms
1.391 ms
1.000 ms
1.455 ms
1.412 ms
1.006 ms
0.939 ms
0.979 ms
1.421 ms
0.961 ms
0.973 ms
1.366 ms
1.371 ms
1.392 ms
1.432 ms
1.435 ms
1.569 ms
1.006 ms
1.366 ms
0.962 ms
1.372 ms
1.394 ms
0.969 ms
1.357 ms
1.071 ms
1.333 ms
1.044 ms
1.338 ms
1.410 ms
1.031 ms
1.010 ms
1.622 ms
1.369 ms
0.961 ms
1.410 ms
1.368 ms
1.512 ms
1.426 ms
1.404 ms
1.389 ms
1.359 ms
0.978 ms
1.377 ms
1.391 ms
1.375 ms
0.980 ms
4.317 ms
1.343 ms
10.809 ms
0.990 ms
1.513 ms
1.079 ms
1.048 ms
1.033 ms
3.216 ms
1.060 ms
10.345 ms
1.034 ms
0.973 ms
1.009 ms
0.959 ms
7.932 ms
1.363 ms
0.975 ms
1.033 ms
1.384 ms
1.427 ms
0.972 ms
1.388 ms
1.034 ms
0.967 ms
0.962 ms
1.425 ms
5.661 ms
1.028 ms
18.320 ms
1.390 ms
1.368 ms
1.421 ms
1.385 ms
0.990 ms
1.013 ms
1.010 ms
1.068 ms
1.364 ms
1.376 ms
1.356 ms
1.005 ms
1.397 ms
1.357 ms
1.406 ms
19.030 ms
0.966 ms
1.381 ms
1.032 ms
0.984 ms
1.395 ms
1.042 ms
1.380 ms
1.366 ms
1.550 ms
0.941 ms
8.038 ms
0.965 ms
1.383 ms
0.997 ms
0.995 ms
1.360 ms
0.924 ms
1.423 ms
1.200 ms
1.398 ms
1.025 ms
0.982 ms
1.369 ms
Network path / recorded observations

Network path observations.

13 observations

Choose a measurement type to see the closest of the available sampled observations to the selected chart point. Each probe is separate and may follow a different path.

Before

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.125 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.191 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop6.228 msReply RTT AS13335 · Cloudflare
Before

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.190 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.201 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop7.035 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop2.046 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.384 msReply RTT AS13335 · Cloudflare
Before

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.127 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.184 msReply RTT AS8002 · Stealth
Before

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.162 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.180 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.316 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.124 msReply RTT AS13335 · Cloudflare
Before

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.415 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.177 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop1.841 msReply RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.435 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.320 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.123 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.189 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop11.644 msReply RTT ASN unavailable
During

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.216 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.225 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop1.293 msReply RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.621 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Probe hopNo replyNo comparable RTT ASN unavailable
  7. TTL 7 Destination2.238 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.132 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.182 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop1.365 msReply RTT AS13335 · Cloudflare
During

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.140 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.195 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop0.723 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop3.494 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.467 msReply RTT AS13335 · Cloudflare
During

MTR TCP/443 · IPv4 · M1

Recorded snapshot · 10 probe cycles


to UTC

  1. TTL 1 Responding hop0.200 msAverage reply RTT · 10/10 replies AS8002 · Stealth
  2. TTL 2 Responding hop0.300 msAverage reply RTT · 10/10 replies AS8002 · Stealth
  3. TTL 3 Responding hop4.600 msAverage reply RTT · 10/10 replies AS13335 · Cloudflare
  4. TTL 4 Responding hop1,418.000 msAverage reply RTT · 8/10 replies AS13335 · Cloudflare
  5. TTL 5 Responding hop5.200 msAverage reply RTT · 10/10 replies AS13335 · Cloudflare
  6. TTL 6 Responding hop1.300 msAverage reply RTT · 8/10 replies AS13335 · Cloudflare
During

MTR TCP/443 · IPv6 · M2

Recorded snapshot · 10 probe cycles


to UTC

  1. TTL 1 Responding hop0.400 msAverage reply RTT · 10/10 replies AS8002 · Stealth
  2. TTL 2 Responding hop0.400 msAverage reply RTT · 10/10 replies AS8002 · Stealth
  3. TTL 3 Responding hop403.700 msAverage reply RTT · 8/10 replies AS13335 · Cloudflare
  4. TTL 4 Responding hop4,774.000 msAverage reply RTT · 3/10 replies AS13335 · Cloudflare
  5. TTL 5 Responding hop7.500 msAverage reply RTT · 10/10 replies AS13335 · Cloudflare
  6. TTL 6 Responding hop1.600 msAverage reply RTT · 7/10 replies AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.118 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.174 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop15.757 msReply RTT AS13335 · Cloudflare
During

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.163 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.184 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop14.074 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop2.543 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination18.691 msReply RTT AS13335 · Cloudflare

RTT is the round trip to each responding hop, not the delay of the preceding link. TTL order may omit routers and does not prove direct links. A missing reply is not proof of packet loss. ASN labels are dated address associations; they do not establish the forwarding path or the operator at the event time. Up to three recorded traces per method and period are selected across the archive. Where no individual trace is available, a comparison may summarize a window; MTR observations aggregate multiple probes. Address aliases are independent across measurement types.

Same method / same recorded address

Compare the traces.

Compare one selected trace from before the event with one during it. Values are individual replies, not a statistical comparison.

ICMP traceroute · Recorded address 1

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.127 msAS8002 · Stealth0.132 msAS8002 · StealthSame responder
20.184 msAS8002 · Stealth0.182 msAS8002 · StealthSame responder
3Not recorded1.365 msAS13335 · CloudflareNot established
ICMP traceroute · Recorded address 2

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.125 msAS8002 · Stealth0.123 msAS8002 · StealthSame responder
20.191 msAS8002 · Stealth0.189 msAS8002 · StealthSame responder
36.228 msAS13335 · Cloudflare11.644 msSame responder
TCP traceroute / 443 · Recorded address 1

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.415 msAS8002 · Stealth0.216 msAS8002 · StealthSame responder
20.177 msAS8002 · Stealth0.225 msAS8002 · StealthSame responder
31.841 ms1.293 msSame responder
4No replyNo replyNot established
51.435 msAS13335 · Cloudflare1.621 msAS13335 · CloudflareDifferent responders
61.320 msAS13335 · CloudflareNo replyNot established
7Not recorded2.238 msAS13335 · CloudflareNot established
TCP traceroute / 443 · Recorded address 2

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.190 msAS8002 · Stealth0.140 msAS8002 · StealthSame responder
20.201 msAS8002 · Stealth0.195 msAS8002 · StealthSame responder
37.035 msAS13335 · Cloudflare0.723 msAS13335 · CloudflareSame responder
4No replyNo replyNot established
52.046 msAS13335 · Cloudflare3.494 msAS13335 · CloudflareDifferent responders
61.384 msAS13335 · Cloudflare1.467 msAS13335 · CloudflareSame responder

RTTs include the return path. Different responders, missing replies and independent probe flows limit what a change can localize. MTR snapshots remain separate in the replay.

Event timeline.

Latency anomaly. Times shown are UTC.

Event opened
UTC
Last event observation
UTC
02 / Assessment

Assessment.

Investigation stateInvestigating
Path assessment

Elevated latency observed.

Endpoint round-trip time was elevated during part of the event. The available path observations do not identify where the increase occurred.

What we observed

  • A sustained increase in measured round trip time was observed.

Recording counts describe collected measurements, not application availability or packet loss. Exact packet paths and the return path remain unverified.

Measurements at a glance.

Baseline RTT1.089 msUsual round trip before the event
Last recorded event RTT1.369 msLast successful event recording
Event p95 RTT9.307 ms95th percentile of successful recordings
Change from baseline+0.279 msLast event RTT compared with the baseline
Successful / expected recordings70 / 70Recorded endpoint samples during the event

RTT is round trip time; p95 describes the upper range of successful recordings. Endpoint measurements differ from individual trace replies and are not an uptime percentage.

Record sources and timestamps
Analysis updated
Unavailable
Public snapshot published
UTC
Public observation window
– UTC

Event and source timestamps are UTC. The latency chart uses its selected timezone. Page refresh times are separate from measurement timestamps.

03 / Method and limits

Method and limits.

How to interpret the sampled path and its limits

Corroborating the path.

Comparisons use matching destinations before and during the event. ICMP traces use echo probes. TCP/443 traces send SYN probes to port 443 only; they do not complete a TCP handshake or perform a TLS or HTTP exchange.

The Stealth gNMI interface verifies the active route-table choice and maps its primary next hop to the Stealth logical interface, configured attachment and, where applicable, the known LAG member set at each snapshot. Stable Stealth replies, together with mapped interface evidence showing no corresponding errors or sustained capacity pressure, reduce the likelihood of a Stealth forwarding issue. An 80% load warning warrants investigation; it does not prove congestion. Lower interval averages can conceal bursts or queue pressure. Optical readings assess the physical link, not congestion.

What remains unseen.

Probe reply handling can differ from packet forwarding. A slow intermediate reply that disappears at later hops does not locate a forwarding problem. A destination that does not respond to a trace is not thereby shown to be unavailable.

Stealth does not use ECMP in this monitored scope. The route snapshot does not identify which physical LAG member carried a particular independent probe, attest line-card forwarding at that packet instant, prove the exact application packet flow or observe the return path. RTT includes the return path, which may differ. No remote reverse traceroute is available. Tunnels and silent routers may hide parts of the path. Address registration alone does not establish the forwarding operator or responsibility.

References: ICMP reply handling / RFC 1812 · Round trip measurements / RFC 2681 · Full methodology