Node app (sockss process) not stable

[mrhodlr]

This is a tough one to report. I know best practice is to include information on how to replicate an error, and include log files when possible. I do not know how to replicate this issue, and I cannot figure out if or where the sockss process logs to, but I know there is an issue with stability and uptime.

We have a lot of users from the community reporting that they have all their nodes correctly installed and running perfectly, without errors. But after some minutes or hours or days, they come back and check on their nodes to find that the Node app has disappeared from the Manager UI. Behind the scenes, I have verified that when this happens the sockss process is no longer running. Rebooting the node typically resolves this. I have also played with starting just the sockss process manually with

cd $GOPATH/bin
nohup ./sockss -node-address :5000 &

Anecdotally, it seems like this sockss process crashing is triggered at least sometimes by a remote node connecting. In other words, Node A connects to remote Node B, and subsequently the Node app (sockss process) crashes on Node B, either while Node A is connected or after Node A terminates the connection. This is hard to replicate, because it doesn't always happen. Most of the time the Node app (sockss process) is crashing while the Manager and Nodes are unattended.

This is a very serious problem for people who are trying to maintain 75% uptime. Users have been told that if that Node app (sockss process) is not running, their node will not qualify for rewards during testnet. It has people in the community trying to figure out how to monitor their nodes for uptime 24 hours a day, from work and while on vacation. Users are trying to figure out how to remotely reboot their nodes while away from home. Some users are setting up cron jobs to reboot their nodes every few hours just in case.

I don't know what causes the Node app (sockss process) to be so unstable, but it is something that needs addressed.