These systems have never been able to connect to Graylog so they were configured to send all logs to 127.0.0.1, and that is great because we can setup a listening tunnel on localhost to forward the connection and we can do all this without making changes to the target systems.
The Graylog server has SSH access to the target systems so we use that to create a reverse tunnel. SSH works over TCP but Graylog uses UDP so we use NC (netcat) to convert the UDP to TCP for tunneling and then convert it back again on the server side.
The first command forks a local listener to send TDP packets to our Graylog on UDP.
The second command opens a reverse tunnel and starts listening on UDP to send packets back via the tunnel over TCP.
nc -l 127.0.0.1 12203 | nc -u 127.0.0.1 12201 &
ssh target2 -R 12201:localhost:12203 "nc -kluw 0 127.0.0.1 12201|nc 127.0.0.1 12201"
Some important points.
- Each connection needs its own port, Graylog is using 12201, so target1 would listen on 12202 and target2 is using 12203
- Netcat wants to disconnect when the logger on target finishes a message to that nc uses 'k' to keep the connection alive and is uses 'w 0' to have no wait time out.
Update
After a day or so running netcat likes to die so this little change will automatically relaunch nc instantly by running this on the target side.
while nc -kluw 0 127.0.0.1 12201; do true; done | while cat - | nc 127.0.0.1 12201; do true; done
