Diagnosing blocks: logs, metrics, tools

"Won't connect" is the most common and most useless message in support. Don't guess. I'll show how I corner the problem by layers: three questions for the client, then server-side checks top to bottom until a link is found where the chain breaks. The commands work.

This material is about engineering your own infrastructure and is educational in nature. Complying with the laws of your own jurisdiction is on you.

Don't guess — walk the chain

A connection travels a long path: client → subscription → node → port → protocol → routing. "Doesn't work" is a break somewhere along that path. The job of diagnostics isn't to guess but to quickly localize the layer. You start at the top (with the client), descend to the server, and at each step you eliminate half the options.

Three questions for the client

Before you climb onto the server — three questions that immediately narrow the search by half:

1. Carrier and city. Mobile or home? Mobile DPI is stricter — if it works on Wi-Fi but not on cellular, that's almost certainly about "whitelists" or transport, not a dead node.
2. App and which server is selected. Happ / Hiddify / v2rayNG and the specific tile.
3. What exactly doesn't work: no connectivity at all / only some sites load / slow / a Russian site won't open.

Symptom → cause

A quick map so you don't start from scratch every time:

• Won't connect for anyone — node offline, port closed, or the certificate expired.
• Won't connect for one person — subscription not refreshed, hit the device limit (hwidDeviceLimit), or picked the wrong server.
• "Doesn't see the server" in the app — the inbound isn't added to the user's squad.
• Works on Wi-Fi, not on mobile — carrier DPI/"whitelists," needs a WL entry or a cascade.
• Connects but sites won't load — routing or client DNS (set 8.8.8.8 DoH).
• A Russian site/bank won't open — no RU-direct routing.
• Downloads go, but YouTube hangs — stuck QUIC, need to block UDP/443 on the node.
• Captchas, "VPN detected" — dirty node IP, replace from the spare pool.

Server-side checks, by layer

Layer 1 — node is alive and the port is open. From any machine we check the port; on the node — the container and Xray logs:

# from any machine:
nc -vz SERVER_IP 443
# on the node:
docker ps | grep remnanode
docker logs --tail 50 remnanode

Xray logs are the first place to look: they show both a startup crash (a broken config) and certificate errors.

Layer 2 — certificate (for TLS inbounds). We check the validity and issuer:

echo | openssl s_client -connect SERVER_IP:443 -servername your-donor.de 2>/dev/null \
  | openssl x509 -noout -dates -issuer

Expired or the wrong domain — reissue (certbot renew).

Layer 3 — Reality, parameter match. Nine out of ten "won't connect on Reality" cases are a mismatch between the Host in the panel and the node config. We check point by point:

• Public Key in Host = REALITY_PUBLIC_KEY from the pair (the config holds PRIVATE_KEY_X25519).
• Short ID in Host = one of the config's shortIds (or both empty).
• SNI in Host = the config's serverNames.
• Fingerprint is set (firefox/qq).

Check whether https://your-donor.de opens in a browser — for self-steal the decoy should open.

Layer 4 — squad. Inbound on the node is enabled, Host is created, and the user still doesn't see the server? Look into Internal Squads: the needed inbound must be checked in that user's squad. The access formula: (inbounds in the user's squads) ∩ (inbounds on the node) ∩ (visible hosts).

Layer 5 — subscription responds. We check that the subscription page is alive:

curl -s -o /dev/null -w "%{http_code}\n" https://sub.your-domain.com/SHORT_UUID

Not 200 — the problem is on the subscription page or the reverse proxy, not in the node itself.

What the client should do if the server is healthy

When all server-side layers are clean — the problem is on the client's side. The memo you hand them:

1. Refresh the subscription in the app.
2. Switch servers (another from the list or "Auto").
3. Set DNS to https://8.8.8.8/dns-query (not Cloudflare — it lags in Russia).
4. Toggle "bypass for Russia" on/off.
5. Reinstall the profile from the link.

If it went down en masse

One client is debugging. Everyone at once is a block, and the algorithm is different:

1. Confirm the block from a mobile carrier, not only from home — mobile DPI is harsher.
2. Fail over to the spare (a cascade/WL/CDN usually do this themselves).
3. Swap the node's IP from the spare pool, put the burned one in the blocklist.
4. Change the SNI/donor/fingerprint.
5. Post a status in the channel. Silence in support during a mass block is the #1 cause of churn and negativity.

Diagnostics is the discipline of eliminating layers one at a time, not grabbing the first hypothesis. Once you find the layer where it breaks — that's where you fix it.