Node didn't import blocks until restart

Support
,
1

Issue Report

Node didn’t import blocks until restart.

Environment

  • Operating System:
  • Pulsar/Advanced CLI/Docker:

Problem

2024-06-20T04:32:19.972604714+08:00  INFO Consensus: substrate: 💤 Idle (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 10.5kiB/s ⬆ 3.9kiB/s
2024-06-20T04:32:24.973335817+08:00  INFO Consensus: substrate: 💤 Idle (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 5.9kiB/s ⬆ 5.5kiB/s
2024-06-20T04:32:29.974002695+08:00  INFO Consensus: substrate: 💤 Idle (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 2.1kiB/s ⬆ 2.4kiB/s
2024-06-20T04:32:34.974889691+08:00  INFO Consensus: substrate: 💤 Idle (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 3.6kiB/s ⬆ 1.9kiB/s
2024-06-20T04:32:39.975891761+08:00  INFO Consensus: substrate: 💤 Idle (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 4.0kiB/s ⬆ 2.4kiB/s
2024-06-20T04:32:44.976659291+08:00  INFO Consensus: substrate: 💤 Idle (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 3.1kiB/s ⬆ 1.6kiB/s
2024-06-20T04:32:49.977240969+08:00  INFO Consensus: substrate: 💤 Idle (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 2.5kiB/s ⬆ 1.5kiB/s
2024-06-20T04:32:54.977908132+08:00  INFO Consensus: substrate: 💤 Idle (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 2.7kiB/s ⬆ 1.6kiB/s
2024-06-20T04:32:59.978624949+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 2.4kiB/s ⬆ 2.2kiB/s
2024-06-20T04:33:04.97929438+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.3kiB/s ⬆ 1.0kiB/s
2024-06-20T04:33:09.979738187+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.5kiB/s ⬆ 1.3kiB/s
2024-06-20T04:33:14.980350568+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032251 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.0kiB/s ⬆ 0.8kiB/s
2024-06-20T04:33:19.981245705+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032251 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 4.1kiB/s ⬆ 1.2kiB/s
2024-06-20T04:33:24.981971441+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032251 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 0.9kiB/s ⬆ 0.8kiB/s
2024-06-20T04:33:29.982515236+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 20.8kiB/s ⬆ 1.2kiB/s
2024-06-20T04:33:34.983139223+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.9kiB/s ⬆ 1.5kiB/s
2024-06-20T04:33:39.983523432+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.9kiB/s ⬆ 1.5kiB/s
2024-06-20T04:33:44.984198763+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.6kiB/s ⬆ 1.5kiB/s
2024-06-20T04:33:49.984871887+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 2.1kiB/s ⬆ 1.9kiB/s
2024-06-20T04:33:54.985527602+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032253 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.5kiB/s ⬆ 1.0kiB/s
2024-06-20T04:33:59.98626436+08:00  INFO Consensus: substrate: 💤 Idle (0 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.6kiB/s ⬆ 1.6kiB/s
2024-06-20T04:34:04.987125419+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032254 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 11.3kiB/s ⬆ 1.1kiB/s
2024-06-20T04:34:09.98816718+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032254 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 0.5kiB/s ⬆ 0.5kiB/s
2024-06-20T04:34:14.989107748+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032254 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 0.9kiB/s ⬆ 0.9kiB/s
2024-06-20T04:34:19.989803779+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032254 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 10.3kiB/s ⬆ 0.7kiB/s
2024-06-20T04:34:24.990462681+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032256 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 3.0kiB/s ⬆ 0.9kiB/s
2024-06-20T04:34:29.991151355+08:00  INFO Consensus: substrate: 💤 Idle (0 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 9.7kiB/s ⬆ 1.7kiB/s
2024-06-20T04:34:34.991903343+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032259 (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 2.2kiB/s ⬆ 1.7kiB/s
2024-06-20T04:34:39.992540182+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032259 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.4kiB/s ⬆ 1.0kiB/s
2024-06-20T04:34:44.993343883+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032259 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.5kiB/s ⬆ 1.6kiB/s
2024-06-20T04:34:49.994406917+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032259 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 10.9kiB/s ⬆ 1.1kiB/s
2024-06-20T04:34:54.995036631+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 13.7kiB/s ⬆ 0.9kiB/s
2024-06-20T04:34:59.995684244+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 2.8kiB/s ⬆ 1.2kiB/s
2024-06-20T04:35:04.996359471+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.7kiB/s ⬆ 1.5kiB/s
2024-06-20T04:35:09.996993548+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.0kiB/s ⬆ 1.0kiB/s
2024-06-20T04:35:14.997733903+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.0kiB/s ⬆ 1.0kiB/s
2024-06-20T04:35:19.998498915+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.8kiB/s ⬆ 1.3kiB/s
2024-06-20T04:35:24.999175498+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.6kiB/s ⬆ 1.4kiB/s
2024-06-20T04:35:29.999548694+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.6kiB/s ⬆ 1.5kiB/s
2024-06-20T04:35:35.000305616+08:00  INFO Consensus: substrate: ⏳ Pending (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.2kiB/s ⬆ 1.1kiB/s
2024-06-20T04:35:40.000745689+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032272 (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.8kiB/s ⬆ 1.0kiB/s
2024-06-20T04:35:45.001324248+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032273 (1 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 2.6kiB/s ⬆ 1.0kiB/s
2024-06-20T04:35:50.002413125+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032273 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 13.2kiB/s ⬆ 1.3kiB/s
2024-06-20T04:35:55.003082337+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032274 (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 3.3kiB/s ⬆ 1.1kiB/s
2024-06-20T04:36:00.004062213+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032275 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 5.6kiB/s ⬆ 1.3kiB/s
2024-06-20T04:36:05.004773504+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032275 (2 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 7.1kiB/s ⬆ 1.1kiB/s
2024-06-20T04:36:10.005528717+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032276 (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 1.6kiB/s ⬆ 1.5kiB/s
2024-06-20T04:36:15.006866941+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032276 (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 5.5kiB/s ⬆ 1.8kiB/s
2024-06-20T04:36:20.007729478+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032276 (4 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 14.3kiB/s ⬆ 1.1kiB/s
2024-06-20T04:36:25.008198593+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032278 (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 7.7kiB/s ⬆ 1.3kiB/s
2024-06-20T04:36:30.008850021+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032278 (4 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 14.2kiB/s ⬆ 1.6kiB/s
2024-06-20T04:36:35.009465676+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032278 (4 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 14.0kiB/s ⬆ 1.6kiB/s
2024-06-20T04:36:40.01018518+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032278 (4 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 23.0kiB/s ⬆ 0.9kiB/s
2024-06-20T04:36:45.010978653+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032282 (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 5.7kiB/s ⬆ 0.8kiB/s
2024-06-20T04:36:50.011700598+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032283 (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 5.5kiB/s ⬆ 1.5kiB/s
2024-06-20T04:36:55.012584683+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032283 (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 12.2kiB/s ⬆ 1.3kiB/s
2024-06-20T04:37:00.013350405+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032283 (4 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 6.4kiB/s ⬆ 1.7kiB/s
2024-06-20T04:37:05.014045061+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032285 (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 7.7kiB/s ⬆ 1.2kiB/s
2024-06-20T04:37:10.014797693+08:00  INFO Consensus: substrate: ⚙️  Syncing  0.0 bps, target=#2032286 (3 peers), best: #2032244 (0x0d87…60aa), finalized #1963002 (0x9dbc…4da8), ⬇ 4.1kiB/s ⬆ 1.4kiB/s
2024-06-20T04:37:15.015565185+08:00  INFO Consensus: substrate: ⚙️
2

The log provided is not sufficient, can you upload the full log since node started somewhere? Also without any information what version and how you’re running there is even less chance we can help.

3

the full log in https://we.tl/t-0MvymuHPkJ;
version: gemini-3h-2024-jun-18

4

It looks like networking issues. It sometimes drops number of peers to 0, can’t reach telemetry server repeatedly, etc.

What is your Internet connection speed and where are you located in the world?

5

Public network bandwidth 500Mbps, located in China

6

I think Chinese firewall might be an issue here, it is simply very slow when reaching out to the rest of the network. I don’t have specific suggestions on how to deal with this, but you can try to ask in #chinese channel on Discord and see what others suggest.

7

Thank you for taking the time to answer my question.

8

If you find a solution, please comment here such that others can find it

9

Will deploying a few bootstrap nodes in China solve this problem? I would be happy to deploy a few stable nodes in China.

10

Bootstrap nodes are only helping to join the network, most nodes typically do not maintain an active connections to bootstrap nodes. Looks like you were able to reach bootstrap nodes, but having a hard time maintaining connection with the rest of the network and boostrap nodes in China will not fix that I’m afraid.