Constant regular network access

[davew_uk]

I've been using Bitorrent Sync v1.0.116 this afternoon and have already got most of my folders set up and syncing fine. I was a heavy Windows Live Mesh user till the service shut down, whereupon I've been limping along with just Synctoy.

After setting things up and letting it do its thing I figured the network activity would pretty much die down to nothing once the syncs completed, but even several hours after finishing this was not the case.

Instead, I'm seeing regular spikes of network activity up to 100 Kbps-ish between the various devices on the network I've just synced. The spikes seem to occur at roughly 8-10 second intervals.

After fiddling with the folder preferences I have come to the conclusion that its down to peer discovery and it seems unusually "chatty" - I don't see why it should need to use quite so much bandwidth so regularly just to "ping" other peers, is there a way that this could be reduced?

Kind regards

Dave

[GreatMarko]

Always nice to see a fellow ex-Mesher come to BitTorrent Sync!

Yes, you are correct - this activity is due to BitTorrent Sync connecting to the tracker every few seconds to get a list of peers.

To my knowledge there isn't currently a setting to change this interval, although, you could try unticking the "Use tracker server" option on each of your shared folders.

If all your devices are on the same LAN, you could also untick the "Use relay server when required" option to further reduce network traffic.

If all your devices are have fixed IP addresses, you could further untick the "Search LAN" option, tick the "Use predefined hosts" option, and then list the corresponding IP addresses of your devices. Again, this should further reduce network traffic.

[davew_uk]

Always nice to see a fellow ex-Mesher come to BitTorrent Sync!

Yes, you are correct - this activity is due to BitTorrent Sync connecting to the tracker every few seconds to get a list of peers.

To my knowledge there isn't currently a setting to change this interval, although, you could try unticking the "Use tracker server" option on each of your shared folders

Thanks :-)

Actually I went through all the options individually and in combination - with "use tracker server" enabled on its own, all traffic goes out over the internet to some AWS server. With "Search LAN" enabled on its own, all the traffic remains within the local network, seemingly using Multicast. Now I don't mind if it has to ping every few seconds - but it is averaging 50 Kbps and peaking over 100 Kbps. That is *way* too much for "are you still there?" messages

[kos13]

Instead, I'm seeing regular spikes of network activity up to 100 Kbps-ish between the various devices on the network I've just synced. The spikes seem to occur at roughly 8-10 second intervals.

After fiddling with the folder preferences I have come to the conclusion that its down to peer discovery and it seems unusually "chatty" - I don't see why it should need to use quite so much bandwidth so regularly just to "ping" other peers, is there a way that this could be reduced?

Kind regards

Dave

Dave,

It talks with the peers to see if something was changed in the folders, so peers could exchange with changes they might have. Sync doesn't have bunch options to configure this, actually has none. But we plan to bring all such things to advanced configuration pane, so you could configure Sync to better fit your needs.

[davew_uk]

Here's a quick screencap of what I'm seeing in Performance Monitor:-

[davew_uk]

It talks with the peers to see if something was changed in the folders, so peers could exchange with changes they might have.

Right, but this is long-term idle performance - I've been monitoring it for several hours, no files are being changed. Do you think that the bandwidth usage as shown in the screenshot elsewhere in the thread is reasonable for idle usage? I think it should be as near to zero as humanly possible myself.

[kos13]

Sync needs to verify that there were no changes in directory tree. To do this, Sync will exchange with directory structure and file names they have, so both nodes will decide if they need to transfer something.

[GreatMarko]

Right, but this is long-term idle performance - I've been monitoring it for several hours, no files are being changed.

As kos said, "It talks with the peers to see if something was changed in the folders, so peers could exchange with changes they might have" i.e. this communication happens continually regardless of whether any of your files have changed - so even if no files have actually changed on the device you've been focusing on, BitTorrent Sync will still have to check for changes to files on other devices you're syncing with, etc.

[davew_uk]

Sync needs to verify that there were no changes in directory tree. To do this, Sync will exchange with directory structure and file names they have, so both nodes will decide if they need to transfer something.

Two questions:-

1. Why continually send data that hasn't changed? that's inefficient - you should only send the directory tree data when something has changed, and you should only send the part that has changed.

2. Why does it still multicast just as much data even when no other peers have been detected?

Sorry guys, but anything that uses that much bandwidth constantly AT IDLE on my network isn't fit for purpose.

[kos13]

I'll get back to this topic with more detailed information soon

[davew_uk]

OK - but I'd like to also pose a third question:-

it seems to continue to use large amounts of bandwidth even when syncing is paused and no other peers are on the network. This is when only "Search LAN" is checked in the folder preferences, though the behaviour isn't significantly different with the default options.

[tjasko]

I also just have to say that I noticed that when "Search LAN" is enabled, it is searching the network vigorously every two seconds. I'd personally make it search every 30s-1min (or even higher), as it's not like the devices on your network are going to change that often...

My server logs were filling up all of the sudden, and I quickly found the cause:

Apr 23 20:43:50 server kernel: Firewall: *UDP_OUT Blocked* IN= OUT=eth0 SRC=192.168.1.101 DST=239.192.0.0
Apr 23 20:43:52 server kernel: Firewall: *UDP_OUT Blocked* IN= OUT=eth0 SRC=192.168.1.101 DST=239.192.0.0
Apr 23 20:43:54 server kernel: Firewall: *UDP_OUT Blocked* IN= OUT=eth0 SRC=192.168.1.101 DST=239.192.0.0
Apr 23 20:43:56 server kernel: Firewall: *UDP_OUT Blocked* IN= OUT=eth0 SRC=192.168.1.101 DST=239.192.0.0
Apr 23 20:43:58 server kernel: Firewall: *UDP_OUT Blocked* IN= OUT=eth0 SRC=192.168.1.101 DST=239.192.0.0
Apr 23 20:44:00 server kernel: Firewall: *UDP_OUT Blocked* IN= OUT=eth0 SRC=192.168.1.101 DST=239.192.0.0
Apr 23 20:44:02 server kernel: Firewall: *UDP_OUT Blocked* IN= OUT=eth0 SRC=192.168.1.101 DST=239.192.0.0

[kos13]

Details:

We do want to establish connection with peer as fast as possible, so time-outs are very short

We do send multicast every second and every 10 seconds requests peers from tracker. And every minute check addresses of relay servers. It comes 16KB for one secret in minute

Multicast - 140 bytes/second;

Peers on relay - 140 /bytes/minute

Getting peers from tracker - 74/bytes every 10 seconds

[chiefsucker]

It comes 16KB for one secret in minute

That may not sound very much, but it’s 16 × 60 × 24 × 30 / 1024 = ~675 MB/month. I assume that due to the nature of how you send the multicast the amount of nodes does not change the traffic volume. But now assume that an average user has 3 shares. That’s about 2 GB/month just for control data. Even if we divide it by 2 (I assume that most people turn their computers off when they sleep), we’ll have about 1 GB of control data just from BT Sync and we don’t even have to transfer one file in that time period. That’s kind of P2P’s nature, but you don’t have to make the communication that aggressive in BT Sync.

[davew_uk]

We do send multicast every second and every 10 seconds requests peers from tracker. And every minute check addresses of relay servers. It comes 16KB for one secret in minute

Multicast - 140 bytes/second;

Peers on relay - 140 /bytes/minute

Getting peers from tracker - 74/bytes every 10 seconds

That doesn't sound like a lot compared to what was observed on my computers. If it is working as designed, its using way too much bandwidth for an idle state. If it isn't working as designed, then fair do's we'll wait for the fix.

FWIW on the first point, I believe that it is possible to design a protocol that doesn't require constantly polling the network. For example, you could cache peers and only broadcast "are you there" pings if they are found to be unreachable at their previous IP address when the app actually needs to contact them to send them a file. Files change very little, certainly not on a second-by-second basis!

My expectation, as a user of this type of application, is that when it has nothing to do, it should sit quietly and do *nothing*.

[GreatMarko]

My expectation, as a user of this type of application, is that when it has nothing to do, it should sit quietly and do *nothing*.

It still has to "monitor" for changes to your folders/files though - so it's can never just sit there doing "nothing" and still be expected to work!

I do think the scanning intervals should be configurable though, and I do believe this is an advanced setting which will be added in a future update

[davew_uk]

In general there are two broad approaches to detecting changes. What you describe is brute-force polling where you have to go check the resource you want to see if it has changed on a regular schedule. The alternative is that the resource offers up an API where your app can register an interest in certain changes, and then go to sleep. Then the system will "call back" to your app and wake it up when the resource has changed in a way that you said you were interested in.

On windows at least there are various file APIs that you can use to automatically monitor a folder for changes in the second, callback-like manner. If all apps had to wake up, traverse the file structure and check for changes and go back to sleep again on a regular schedule then your antivirus, backup software like Acronis etc. would be a nightmare hell of disk thrashing.

[Hacker]

Sync needs to verify that there were no changes in directory tree. To do this, Sync will exchange with directory structure and file names they have, so both nodes will decide if they need to transfer something.

Why don't you just exchange a hash of all the data that is needed for comparison (file names, dir structure, time stamps, etc.)?

[Automatic Coding]

That may not sound very much, but it’s 16 × 60 × 24 × 30 / 1024 = ~675 MB/month. I assume that due to the nature of how you send the multicast the amount of nodes does not change the traffic volume. But now assume that an average user has 3 shares. That’s about 2 GB/month just for control data. Even if we divide it by 2 (I assume that most people turn their computers off when they sleep), we’ll have about 1 GB of control data just from BT Sync and we don’t even have to transfer one file in that time period. That’s kind of P2P’s nature, but you don’t have to make the communication that aggressive in BT Sync.

Since when is 2GB a month a lot? Hell, I probably pull ~ 7TB/month. If you're on such a data cap that you can't even afford 2GB/month, then, you probably shouldn't be using a file sharer.

[Muther]

For me it is waking up to hear three systems with fans in high gear. I check to see what's going on and its BT Sync that seems to be very busy, although nothing is being transferred. I'm new to BT Sync, so I select "pause syncing," which you all know does nothing to the network activity.

I am by no means adept at this kind of program, but it seems like any folder that updates could send out "pings" to clients, announcing updates. Any BT Sync that starts could "ping" all its connection to check for changes. The rest of the time the program could just lay around waiting for something to happen. What else is needed?

I'm so stoked with this program in so many ways. I look forward to the improvements the developers will make as they move towards Beta.

[Xanza]

Two questions:-

1. Why continually send data that hasn't changed? that's inefficient - you should only send the directory tree data when something has changed, and you should only send the part that has changed.

2. Why does it still multicast just as much data even when no other peers have been detected?

Sorry guys, but anything that uses that much bandwidth constantly AT IDLE on my network isn't fit for purpose.

To expand upon that idea, why not just create another .SyncSums file or something that calculates the total MD5 (or SHA1 whatever) sum of all files located within the directory tree that are set to sync. Then simply compare this single MD5/SHA1 hash between clients, if it's different, then something has changed, if it's the same, then things haven't.

Would probably work a bit better than polling every XX seconds with a real time analysis of whatever is in the directories. (which will probably get VERY slow as you accumulate a ridiculous amount of files (100,000+).

Any thoughts?

[Muther]

@Xanza: I think that sounds very efficient, yet you did not mention how an MD5 would effect when/how often BTSync compares sums. Or is it that using an MD5 would make checking sooo effiecient that it would matter less how often two folders were compared?

Forgive me for being a bit dense.

[compdave7681]

Just to mention this is also having an effect on battery life with android phones. I searched desperately to figure out why my download was constantly lit on my phone. Come to find out after some wireshark investigation it was btsync sending out this "chatter". My question is this, how do I set the config file (running linux version) to only check certain ip's on my subnet? How do I keep it internal and not even try to look external?

[rdebath]

Scheesh, OKay ...

What a NAT punching UDP Transport system needs to do.

The connection tracking on a firewall needs to keep a connection open to receive replies to responses to our outgoing packets; but it also needs to remove the connection from it's tables once it finishes. The only way it can know when to remove the connection is to use a timeout. Most firewalls seem to use a timeout between 1 and 10 minutes for UDP, the default for Linux is 3 minutes.

Assuming there are some bad firewalls out there we need one 100byte packet every 15 seconds to keep the connection open, which is about 18Mbyte per peer per month. Plus two more peers, one to keep the connection to the relay open and one for the tracker.

If a TCP connection can be arranged it's timeouts are normally of the order of days BUT simultaneously opening a TCP connection from both ends to achieve a TCP NAT punching connection is rather difficult.

The relay can be be used to help in the NAT punching by sending a 'please call me' message to a peer with whom the direct NAT hole has lapsed. If this method is used the relay can be used for all idle chatter further reducing the minimum data rate.

What a multicast search needs to do ...

Nothing.

Well, almost. If the local network is always connected or if we can always sense when it's connection state changes we only need to send out multicast packets (a) On startup, (b ) when the connection state changes. ( c ) In response to a novel multicast message.

In practice it's probably best to assume that we cannot detect all the connection state changes and ping it like a normal peer (15 seconds) plus responses to novel messages.

What a sync (mirror) tools needs to do...

BTSync uses inotify*() so it doesn't need to scan the filesystem ... in theory. In practice inotify*() can miss things, if changes are too complex BTSync may get confused and BTSync may be restarted.

Assuming it needs to scan the filesystem it must check the files against it's own database of path+size+mtime+ctime+hash it is usual to assume that the hash will match if the others do. ONLY if it finds differences does it need to post these changes to the other peers in the form of "IHAVE" style messages. (There will normally be an option to do a full hash scan to check that all the hashes still match too)

The mirror manager needs NO idle network traffic, everything can be done locally.

If a new peer comes online the two mirror managers can agree that they have the same filesystem by exchanging hashes of their databases; total data transfer required 20 to 64 bytes.

(NB: This 'database hash' will have to be computed only on parts of the filesystem that are mirrored; ie not the ctime fields )

---------------------------

This does not add up to Gigabytes/Month of network traffic.

[davew_uk]

No need for NAT punching or multicast searching in a LAN-only scenario with fixed IP addresses. That is my scenario - and I'm not bothered about gb/month or traffic caps either, the point is if the app has nothing to do it should do NOTHING.

If you are accurately tracking file system changes (via "instantaneous" callbacks or via regular timed scanning or a combination of both, I don't care) there's surely no need to generate any network traffic unless a file has actually changed? what can you possibly gain from sending 50-150kbs of data constantly?

I look forward to seeing how this app grows and gets better in the future, but right now I'm not using it.