FetchOfflineLookups Timeout


#1

Hello, guys!

Trying to setup the Zil node, doing everything as written in the manual but when I try to start it, I get -

FetchOfflineLookups Timeout

Attaching a screenshot.

Any thoughts?

Thanks!


#2

Same here, not sure what the deal is


#3

It happens when your port forwarding is broken. Make sure your 33333 or whatever you choose port is open to the world. Also make sure its both TCP/UDP.


#4

Wait, what has the port forwarding to do with all that? I am receiving this on my node log, so basically it tries to connect to some hosts that are not reachable. I think port forwarding got to do for the incomming connections.
Actually, if I try to ping some of the ip addresses, I receive no answer.

For example:

Do you think the problem may lay there? Are those IPs correct?

Thanks!


#5

verify what Noncoder000 said.


#6

Ok, I will do that. Should I expect some responce when pinging the IP addresses? This looks very strange that I can’t reach them, on any PC.


#7

I don’t think pinging matters, it’s your listening port that isn’t open. Set a static IP for your PC in your router and forward the 3**** port (whatever you chose) and point it at your pc IP. 192.169.. etc.


#8

I had the same problem and it turns out I had forgotten to this step from the GPU guide:

change the following parameters for the constants.xml file in the join folder to true :

<FULL_DATASET_MINE>true</FULL_DATASET_MINE>
<CUDA_GPU_MINE>true</CUDA_GPU_MINE>

The join folder is where you downloaded the Zilliqa files from a previous step.


#9

I have the same problem. Port forwarding to pc with port 33133 or 33333 does not work. But the port is accessible from the outside. I checked this with the Open Port Check Tool. Could it be possible that Ubuntu is still causing problems here? I tried the Docker version. On a VM and on a new local Ubuntu system. The IP is fixed to the MAC address. Any idea?


#10

I don’t think you can port forward from a VM like this.


#11

Thanks for your answer. I am back at the Local Ubuntu. Here it works now. The question is, does it work correctly?

[INFO][   21][19-02-02T17:52:04.522][okup/Lookup.cpp:3158][Execute             ] BEG
[INFO][   21][19-02-02T17:52:04.524][ookup/Lookup.cpp:378][GetLookupNodes      ] BEG
[INFO][   21][19-02-02T17:52:04.524][ookup/Lookup.cpp:378][GetLookupNodes      ] END
[WARN][   21][19-02-02T17:52:04.524][okup/Lookup.cpp:3047][ProcessSetDirectoryB] No Directory blocks sent/ I have the latest blocks
[INFO][   21][19-02-02T17:52:04.524][okup/Lookup.cpp:3158][Execute             ] END
[INFO][    1][19-02-02T17:52:04.545][work/P2PComm.cpp:601][EventCallback       ] Incoming normal <34.220.11.234:58468> (Len=121): 01110000007304050A0608D10C10D00C12230A2103A940303B869120D21A...
[INFO][   23][19-02-02T17:52:04.545][okup/Lookup.cpp:3158][Execute             ] BEG
[INFO][   23][19-02-02T17:52:04.545][okup/Lookup.cpp:1689][ProcessSetTxBlockFro] BEG
[INFO][   23][19-02-02T17:52:04.545][e/Messenger.cpp:5230][GetLookupSetTxBlockF] BEG
[INFO][   23][19-02-02T17:52:04.547][e/Messenger.cpp:5230][GetLookupSetTxBlockF] END
[INFO][   23][19-02-02T17:52:04.547][okup/Lookup.cpp:1720][ProcessSetTxBlockFro] [Epoch 1617] ProcessSetTxBlockFromSeed sent by <34.220.11.234:58468> for blocks 1617 to 1616
[WARN][   23][19-02-02T17:52:04.547][okup/Lookup.cpp:1740][ProcessSetTxBlockFro] The lowBlockNum is higher the highblocknum, maybe DS epoch ongoing
[INFO][   23][19-02-02T17:52:04.547][okup/Lookup.cpp:1689][ProcessSetTxBlockFro] END
[INFO][   23][19-02-02T17:52:04.547][okup/Lookup.cpp:3158][Execute             ] END
[INFO][ 1812][19-02-02T17:52:48.982][okup/Lookup.cpp:3138][ComposeAndSendGetDir] BEG
[INFO][ 1812][19-02-02T17:52:48.982][ookup/Lookup.cpp:881][SendMessageToRandomS] BEG
[INFO][ 1812][19-02-02T17:52:49.006][ookup/Lookup.cpp:881][SendMessageToRandomS] END
[INFO][ 1812][19-02-02T17:52:49.006][okup/Lookup.cpp:3138][ComposeAndSendGetDir] END
[INFO][ 1812][19-02-02T17:52:49.006][ookup/Lookup.cpp:672][GetTxBlockFromSeedNo] BEG
[INFO][ 1812][19-02-02T17:52:49.006][ookup/Lookup.cpp:619][ComposeGetTxBlockMes] BEG
[INFO][ 1812][19-02-02T17:52:49.006][ookup/Lookup.cpp:626][ComposeGetTxBlockMes] [Epoch 1617] ComposeGetTxBlockMessage for blocks 1617 to 0
[INFO][ 1812][19-02-02T17:52:49.006][e/Messenger.cpp:5144][SetLookupGetTxBlockF] BEG
[INFO][ 1812][19-02-02T17:52:49.006][e/Messenger.cpp:5144][SetLookupGetTxBlockF] END
[INFO][ 1812][19-02-02T17:52:49.006][ookup/Lookup.cpp:619][ComposeGetTxBlockMes] END
[INFO][ 1812][19-02-02T17:52:49.006][ookup/Lookup.cpp:881][SendMessageToRandomS] BEG
[INFO][ 1812][19-02-02T17:52:49.025][ookup/Lookup.cpp:881][SendMessageToRandomS] END
[INFO][ 1812][19-02-02T17:52:49.025][ookup/Lookup.cpp:672][GetTxBlockFromSeedNo] END
[INFO][    1][19-02-02T17:52:49.569][work/P2PComm.cpp:601][EventCallback       ] Incoming normal <54.149.110.46:63087> (Len=117): 01110000006F04160A02085D12230A2103A137D5730A17B57215219B3E58...
[INFO][   24][19-02-02T17:52:49.569][okup/Lookup.cpp:3158][Execute             ] BEG
[INFO][   24][19-02-02T17:52:49.570][ookup/Lookup.cpp:378][GetLookupNodes      ] BEG
[INFO][   24][19-02-02T17:52:49.570][ookup/Lookup.cpp:378][GetLookupNodes      ] END
[WARN][   24][19-02-02T17:52:49.570][okup/Lookup.cpp:3047][ProcessSetDirectoryB] No Directory blocks sent/ I have the latest blocks
[INFO][   24][19-02-02T17:52:49.570][okup/Lookup.cpp:3158][Execute             ] END
[INFO][    1][19-02-02T17:52:49.586][work/P2PComm.cpp:601][EventCallback       ] Incoming normal <34.210.214.209:35570> (Len=121): 01110000007304050A0608D10C10D00C12230A2103038A400644F300CAFD...
[INFO][   25][19-02-02T17:52:49.586][okup/Lookup.cpp:3158][Execute             ] BEG
[INFO][   25][19-02-02T17:52:49.586][okup/Lookup.cpp:1689][ProcessSetTxBlockFro] BEG
[INFO][   25][19-02-02T17:52:49.586][e/Messenger.cpp:5230][GetLookupSetTxBlockF] BEG
[INFO][   25][19-02-02T17:52:49.587][e/Messenger.cpp:5230][GetLookupSetTxBlockF] END
[INFO][   25][19-02-02T17:52:49.587][okup/Lookup.cpp:1720][ProcessSetTxBlockFro] [Epoch 1617] ProcessSetTxBlockFromSeed sent by <34.210.214.209:35570> for blocks 1617 to 1616
[WARN][   25][19-02-02T17:52:49.587][okup/Lookup.cpp:1740][ProcessSetTxBlockFro] The lowBlockNum is higher the highblocknum, maybe DS epoch ongoing
[INFO][   25][19-02-02T17:52:49.587][okup/Lookup.cpp:1689][ProcessSetTxBlockFro] END
[INFO][   25][19-02-02T17:52:49.587][okup/Lookup.cpp:3158][Execute             ] END

#12

The port on mine is definitely open and allowing tcp+udp. I can telnet to it fine from outside.
I am trying to run as single ‘CPU’ node with other GPU miners hitting it. (Miners connect fine.)
I compiled using the mainnet-4.0.1 tag on github. (git checkout mainnet-4.0.1)
I used the mainnet configuration download as the start folder and edited constants.xml as per guide:

<GETWORK_SERVER_MINE>true</GETWORK_SERVER_MINE>
<CUDA_GPU_MINE>false</CUDA_GPU_MINE>
<FULL_DATASET_MINE>false</FULL_DATASET_MINE>
<OPENCL_GPU_MINE>false</OPENCL_GPU_MINE>
<REMOTE_MINE>false</REMOTE_MINE>

I added Scilla per https://github.com/Zilliqa/Zilliqa/wiki/Mining#mining-with-native-build-locally

<SCILLA_ROOT>/home/builds/Scilla</SCILLA_ROOT>

I am using 0.0.0.0 as the --address
I see no errors other than this warning. (As per OP) Zilliqa then stalls and sits there.

[WARN][18333][19-02-02T21:01:44.304][libNode/Node.cpp:908][GetOfflineLookups   ] FetchOfflineLookups Timeout... tried 2/3 times
[INFO][18333][19-02-02T21:01:44.304][okup/Lookup.cpp:2923][GetOfflineLookupNode] BEG
[INFO][18333][19-02-02T21:01:44.304][ookup/Lookup.cpp:125][SetLookupNodes      ] BEG
[INFO][18333][19-02-02T21:01:44.309][ookup/Lookup.cpp:125][SetLookupNodes      ] END
[INFO][18333][19-02-02T21:01:44.309][okup/Lookup.cpp:2899][ComposeGetOfflineLoo] BEG
[INFO][18333][19-02-02T21:01:44.309][e/Messenger.cpp:5626][SetLookupGetOfflineL] BEG
[INFO][18333][19-02-02T21:01:44.309][e/Messenger.cpp:5626][SetLookupGetOfflineL] END
[INFO][18333][19-02-02T21:01:44.310][okup/Lookup.cpp:2899][ComposeGetOfflineLoo] END
[INFO][18333][19-02-02T21:01:44.310][ookup/Lookup.cpp:440][SendMessageToLookupN] BEG
[INFO][18333][19-02-02T21:01:44.529][ookup/Lookup.cpp:460][SendMessageToLookupN] Sending to lookup <54.184.239.252:33133>
[INFO][18333][19-02-02T21:01:44.607][ookup/Lookup.cpp:460][SendMessageToLookupN] Sending to lookup <34.210.214.209:33133>
[INFO][18333][19-02-02T21:01:44.659][ookup/Lookup.cpp:460][SendMessageToLookupN] Sending to lookup <34.220.13.140:33133>
[INFO][18333][19-02-02T21:01:44.695][ookup/Lookup.cpp:460][SendMessageToLookupN] Sending to lookup <54.149.110.46:33133>
[INFO][18333][19-02-02T21:01:44.736][ookup/Lookup.cpp:460][SendMessageToLookupN] Sending to lookup <34.220.11.234:33133>
[INFO][18333][19-02-02T21:01:44.736][ookup/Lookup.cpp:440][SendMessageToLookupN] END
[INFO][18333][19-02-02T21:01:44.736][okup/Lookup.cpp:2923][GetOfflineLookupNode] END
[WARN][18333][19-02-02T21:02:29.736][libNode/Node.cpp:908][GetOfflineLookups   ] FetchOfflineLookups Timeout... tried 3/3 times
[WARN][18333][19-02-02T21:02:29.736][libNode/Node.cpp:917][GetOfflineLookups   ] Fetch offline lookup nodes failed
[WARN][18333][19-02-02T21:02:29.736][libNode/Node.cpp:936][operator()          ] Cannot rejoin currently

PS shows this running as root:
zilliqa --privk {REMOVED} --pubk {REMOVED} --address 0.0.0.0 --port 33133 --synctype 1


#13

Likewise my port is definitely open as OpenPortChecker was able to verify the existence of the open port. Not using a VM. Constants file have those 2 flags set to true. Is there something else that needs to be done? Using the docker image here.


#14

Right now the network is pooched, and it looks like the seed nodes are not accepting any new peers.


#15

@noncoder000 - Must be - because for life of me, I cannot find anything wrong on my end.


#16

I cant connect myself. And was a ds node.
Oh well.


#18

Just an update to my previous post:

I am now connecting correctly.
My last post is above and no settings have changed.

If this was a blacklist problem, the amount of ‘deny’ time needs to be fixed. It’s too long. Blacklisting appears to be too sensitive as well.

I had never been blacklisted on any chain in past (afaik) - even during 1st setup/start. I appreciate good blacklisting code, but it’s also a fine line between blocking idiots and blocking people just trying to connect first time with maybe one ‘bad’ setting.


#19

“I don’t think you can port forward from a VM like this.”

Why wouldn’t you? I am using a vbox VM right now.
You need to set port forwards inside the settings/network for VM as well as your router’s NAT.

Hint: You do NOT need a bridged adapter unless you have a odd setup. Just use default “NAT” adapter under VBOX: https://i.imgur.com/QrZ16Z8.png <-that works fine (don’t forget UDP too). As long as your host isn’t using the port, no need to change it.


#21

So it looks like blacklist lasts 24 hours and not just one epoch?


#22

Not sure. I started this zilliqa setup early yesterday. Tried first thing this morning and is talking.

ps
Now I am having a getWork issue with zilminer or server. My guess is it’s the server side/CPU node.
Seems to connect fine, but no work sent. >Posted in zilminer thread.<
Sigh. ‘Pain in the ass’ comes to mind a lot lately…

Edit:
Well mining fixed - I guess. Never knew the gpu miner sits idle for 99% of time. Yep - good to know. No reply to my post (^link) - thanks.
Oddly enough, this isn’t CLEARLY mentioned in Mining Wiki. Seems kinda important.