JimKnopf

Theory Of Operation: How Does The Upload Balancing Work?

Recommended Posts

Hi,

 

can some of you explain to me how a btsync client decides what upload bandwidth will be used to the other members sharing the same secret?

 

Background: When my team (consisting of 5 clients) shares data over the internet we can see at the seeder client, the system with the lowest upload bandwidth gets the most bandwidth from the seeder. "iftop" is a nice tool to check this on a Linux client.

 

We have here the effect while transferring large data files that the clients with the smallest upload capability get ready first and the systems with largest possible upload bandwidth finish at last and this in a strict order!

This is obviously the wrong order since it would be of course optimal to fill the client with the highest upload capability first so this resource can be fully used while distributing the data instead of waiting for the clients with the smallest capability to upload data to the others.

 

But perhaps this simply bases on the understanding of the algorithm used and can be influenced somehow, so that's why I'm asking this question, here.

 

Greetings from Germany

Share this post


Link to post
Share on other sites

Perhaps I'm alone with my question and the most people don't care how this works.

 

In case you're also interested in the answer to the questions I placed, please, express you curiosity by making a simple post to this thread.

 

Thx in advance,

Share this post


Link to post
Share on other sites

@JimKnopf

 

BTSync does not do any load-balancing. Maybe in future we'll do some load-balancing adjustments, but now the traffic goes as fast as it can

 

In your post you correlate the upload bandwidth to the fact how quickly the peer gets data. Does it correlate in some way with download bandwidth? It actually should have more influence.

Share this post


Link to post
Share on other sites

Hi RomanZ:

 

Many thanks for your answer. No the download bandwidth cannot be a limiting factor since even the slowest internet connection has 14 Mbit download bandwidth capacity whereas the fastest internet connection only has 10 Mbit upload bandwidth.

 

But your answer let me check the latency between my internet connection and the others. And really:

 

The slowest internet connection shows the best ping round-trip times (used average of 100 pings). The member with the fastest internet connection showed the slowest ping response times.

 

Comparing with the behaviour seen, the client order of getting finish with a very large file synchronization exactly matches with the order of ping latencies between the seeder and the clients.

 

Sounds logic and is clearly a correlation but the effect is quite dramatic in real. Would be very cool and helpful if one could influence the upload bandwidth of the own btsync client to other clients.

Best would be if the protocol would automatically determine the upload capacity of the other clients and trying to balance the local upload to them so the data distribution gets done as fast as possible.

Share this post


Link to post
Share on other sites

+1

 

I wondered this myself because I have 5 clients and often I'm creating large files on one client for sharing out.  I see the same sort of thing in the upload distribution.

 

I was also wondering how a file was uploaded.  For example, do the other devices ever demand the same chunks of the new file so causing the seed device to send out duplicate data?  Or do they somehow sort out different chunks?

 

RTRPM.

Share this post


Link to post
Share on other sites

@rtrpm-bittorrentsync

 

If BTSync client has multiple known peers, he'll request different chunks of data from them. The only case when it can ask for the same piece of data from another peer is if it was unable to receive a piece (damaged, peer closed connection, etc.).

Share this post


Link to post
Share on other sites

@RomanZ:

 

I understood rtrpm-bittorrentsync that way that we he wants to avoid the situation that the seeder (the client with original file) has to send the same data chunks to more than one client in the network.

 

I totally agree with this since this would be an optimum process with highest efficiency. But if the current logic of btsync just base on a pure polling mechanism where a client without the data dictates what chunks other clients have to send than this cannot be guaranteed.

 

This in fact is the behavior I can see. In a perfect network the seeder would have to send the amount of data of a file only once by distributing the pieces in a disjunctive way to the clients in the network and these clients exchange the missing data among them without the seeder.

With current implementation the seeder runs much longer than the required minimum of a single transfer time of the file.

My experience is that if you compare the time a seeder would need to distribute a file in a pure sequential way (so any client would get the data by FTP i.e.) with the runtime btsync currently offers then the time saving is around 30% - 50% ... but this very depends on the file size and number of files to distribute.

Edited by JimKnopf

Share this post


Link to post
Share on other sites

@JimKnopf I agree 100% that would be the ideal situation.

 

@RomanZ Would it not be possible for the seeder to service only those requests that would meet the goal of no duplication of chucks to its peers until a full distribution of the file is achieved. Then service the repeat requests? because in theory the client peers are should be requesting all of the chunks anyway, particularly when it is an initial seed.

 

On to my question: I would like to know if during an initial seed the leaches do not share chunks. In the tests I have done chunks from multiple sources do not appear until they have finished downloading the entire file. 

 

It would be great if we could get a representation in the clients of what chunks are being moved much like standard bittorrent clients as it would vastly improve debugging.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.