compiling syncthing protocol buffers for python

how to interact with syncthing - from python

turns out, hacking something in python to play with syncthing is not too hard, since they used protocol buffers to define their protocols for local discovery and block exchange!

that means, we can just generate code to parse the packets! :)

install protobuf package

pacman -S protobuf  # depending on your distro

clone syncthing & requirements

git clone
cd syncthing
git clone repos/protobuf

generate protobuf

# in a new folder, somewhere else
protoc --proto_path PATH/TO/SYNCTHING --python_out=. repos/protobuf/gogoproto/gogo.proto lib/protocol/bep.proto lib/discover/local.proto

this generates generates a lib/ folder, containing our code.

example: reading local discovery announcement (or: the hello world of the syncthing protocol)

the exact specifications for the local discover are here

in short, this says:

  • packets will be broadcasted to port 21027, UDP.
  • a packet starts with a 32bit magic field, containing 0x2EA7D90B in big endian. that means, if a packet starts with this value, its one of our announcement packets!

the code to create a socket, wait for a packet and read it would look like this:

import socket

from import local_pb2  # the stuff that was generated for us! :)

HOST = ''  # listen on all interfaces
PORT = 21027

# open a datagram socket (==udp)
# since python3.2, sockets support contexts, so we use a context manager here
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:  
    s.bind((HOST, PORT))
    while True:
        data, addr = s.recvfrom(1024)
        magic = int.from_bytes(data[:4], byteorder="big")  # read the magic value
        if magic == DISCOVERY_MAGIC:
            packet_data = data[4:]  # cut off the magic
            announce = local_pb2.Announce()
            announce.ParseFromString(packet_data)  # parse the packet data
            print(announce)  # aaand were done!

if there is a local instance of syncthing running in your network, you should get a packet every 30-60 seconds when you run this.

the packet that you receive will have

  • a random instance id (generated at startup),
  • a list of addresses where the instance can be found,
  • and the syncthing id, which is the SHA-256 of the devices X.509 certificate.


i guess just reading the local announcements is pretty useless, so if you want to continue on this, you want to read the protocol specification.

also, you need to generate (or read) rsa keys (->), make the whole thing non-blocking (maybe by using threads or coroutines), need to understand the way more complex block exchange protocol, communicate between the socket threads you set up and (most likely) a lot more things.

i still find it interesting, so there are probably more posts coming on this. :)

Last modified 2019.08.03