BitTorrent Sync FAQ *unofficial*


GreatMarko
 Share

Recommended Posts

If I have one instance of Sync running on a machine with a publicly routable IP address or domain name, and, I have entered this machine into the "use predefined hosts" entry on the other peers, will this machine operate as a "rendevous" server and allow the other peers to connect directly between themselves without relaying content through the publc server. Even if all other public rendevous mechanisms ( ie: tracker server, relay server, and external BHT network ) are disabled ?

If not this would be very desirable to have.

PK

Link to comment
Share on other sites

If I have one instance of Sync running on a machine with a publicly routable IP address or domain name, and, I have entered this machine into the "use predefined hosts" entry on the other peers, will this machine operate as a "rendevous" server and allow the other peers to connect directly between themselves without relaying content through the publc server. Even if all other public rendevous mechanisms ( ie: tracker server, relay server, and external BHT network ) are disabled ?

Hmmm... if I understand you correctly, what you're after is to have "Device A", for example, act as a "super node" or "private tracker" through which all other devices would be able to establish connections to each other.

This is not how Sync currently works, as it is decentralized in nature.

If you have 3 devices, A, B, and C, and you configure A in the manner you've described above, this will not automatically mean that B can communicate directly with C via A

Link to comment
Share on other sites

What is the format of .SyncIgnore ? I ask because I want to sync part of a directory ( a set of sub folders and files ) and the logic I see is that I would first exclude everything, and then include (recursively) only the folders I want to sync. This requires both "include" and "exclude" logic as well as recursion and wildcards.

My use case is I want to sync 25 folders in "home/peterk" but nothing else out of the dynamic 100 or so folders and files in there, and it is impractical to have a separate "secret" for each one.

ie: does "folder/**" mean recursive as in .gitignore ?

is there an inversion descriptor meaning include an item or items instead of exclude ?

does a .SyncIgnore in a subfolder to a "secret" act on all it's children ?

if so does it overwrite (add includeds and remove excludeds from the .SyncIgnore state in the parent folder ?)

( there seems to be some evolution toward a common-standard idea of thes types of descriptors - ie: ruby filelist, .gitignore and the like )

The fuller logic would be extremely desirable, and could be added to the GUI ( sub folder selection ) for a secret later.

Whatever the format of course, it needs to be fully documented in the user guide.

Thanks.

Link to comment
Share on other sites

What is the format of .SyncIgnore ?

.SyncIgnore includes exclusion rules (one per file) for excluding certain files/folders from being synced, and may include wildcards. - Please see the Unofficial FAQ for examples

There is presently no way to specify a separate set "inclusion" rules (Please search the forum for "syncinclude" for more discussion on this)

Whatever the format of course, it needs to be fully documented in the user guide.

There's a wealth of topics/examples on the format of .SyncIgnore - just search for this forum for "syncignore" - it's pretty extensively covered!

Link to comment
Share on other sites

Hmmm... if I understand you correctly, what you're after is to have "Device A", for example, act as a "super node" or "private tracker" through which all other devices would be able to establish connections to each other.

This is not how Sync currently works, as it is decentralized in nature.

If you have 3 devices, A, B, and C, and you configure A in the manner you've described above, this will not automatically mean that B can communicate directly with C via A

I some ways yes. I believe I understand the de-centralized nature of peer-to-peer which is why I like it but not the specific intracacies of the torrent protocol - I *want* the peer to peer architecture, that is assigning direct connections between peers for bandwidth optimization and distributed reliability. I would prefer if we all had publicly accessable addresses and peer trackers were not necessary but this ( alas ) is how the internet has evolved.

- I desire not to be *required* to be dependent on the existence of peers/trackers outside of peers configured for my "peer group"

that is the ability to use one ( or more ) of "our" peers as "private" trackers. Hopefully if any peer in the graph can be reached, it can inform the contacting peer of the existence and reachability of any of the other peers it has contact with. If that is true then if any peer has a public IP address then it can effectively *reliably* allow all the other NATed peers to reach each other if part of the distributed reachability graph database is lost. The more peers that have pieces of the reachability graph the better.

- enabling a *reliable* always contactable tracker either within a private firewalled off LAN or on a network segment that for some reason ( natural disaster, ISP port blocking, etc etc ) might be disconnected from the greater cloud. Or enabling a private tracker cloud that uses other ports that are not part of the "torrent cloud" for one reason or another such as preventing malicious DDOS attacks on the trackers.

- In no way ( unless the ouside tracker cloud contact is disabled ) preventing peers from using the greater torrent tracker cloud.

Link to comment
Share on other sites

On the speed of synchronization startup:

I have set up two machines in a fast LAN, DNS and everything, and data transfer between

both machines is very fast (MacMini server and Macbook Pro).

But I recognized that the syncing seems to take 10-15 seconds until I see the transfer starting.

This seems not to change, with Ethernet, Wifi or even a server on the internet.

Does that have something to do with the time difference setting, maybe?

Is there to get that down to, say, 1-2 seconds in such tightly coupled scenarios?

thanks in advance!

Link to comment
Share on other sites

I have set up two machines in a fast LAN, DNS and everything, and data transfer between

both machines is very fast (MacMini server and Macbook Pro).

But I recognized that the syncing seems to take 10-15 seconds until I see the transfer starting.

Does that have something to do with the time difference setting, maybe?

I think this is more to do with the limitations of the Mac environment - i.e. file changes are not reported to/detected by Sync as quickly as they are under a Windows/Linux environment for example.

You could potentially lower the advanced "folder_rescan_interval" setting - which specifies how often Sync should also manually rescan your sync'd folders - to just a few seconds. Although if set that low, this may adversely affect overall system performance and lead to increased hard disc thrashing, etc.

Personally, I don't think a delay of 10-15 seconds for file changes to be detected by Sync is all that unreasonable

Link to comment
Share on other sites

I think this is more to do with the limitations of the Mac environment - i.e. file changes are not reported to/detected by Sync as quickly as they are under a Windows/Linux environment for example.

You could potentially lower the advanced "folder_rescan_interval" setting - which specifies how often Sync should also manually rescan your sync'd folders - to just a few seconds. Although if set that low, this may adversely affect overall system performance and lead to increased hard disc thrashing, etc.

Personally, I don't think a delay of 10-15 seconds for file changes to be detected by Sync is all that unreasonable

I agree that this delay does normally not matter, but I have special purposes in a mesh of machines.

To confirm the better notification time of windows, I created two virtual Windows machines in Parallels 7.

Then I installed the same test scenario and switched BTsync off on the Mac machines.

BTsync was now running in the two virtuals which could see each other through local networking.

I checked again by syncing a file and renaming it on one machine.

As with OS X, the name change took 10-20 seconds to propagate.

When I lowered the scan interval stepwise down to 3 seconds, the sync reacted a bit earlier, but took

still up to 12 seconds.

I'm just a bit wondering, because on my python programs, I use a watchdog which has optimized

system events for OS X, Linux and Windows, and notifications work in a fraction of a second.

https://pypi.python.org/pypi/watchdog

The way BTSync operates feels pretty much like a polling implementation, or maybe there are other reasons

to throttle this?

Fast notification of changes was actually my hope to get rid of a central server in my application.

cheers - chris

Link to comment
Share on other sites

Actually, this leads me to a feature request:

If the sync is not really reactive (which is fine for average use cases),

then I would like to see an API to enable some broadcasting of information

that does not have to be file based, but where machines can be notified of events.

(will submit something to the wishlist, but I fear this is already a black hole ;-) )

Link to comment
Share on other sites

  • 3 weeks later...

Hello, here's a cautionary tale..

I have a directory \Hardware which contains subdirectories for each of my toys.  I set up a BTSync task so the whole of \Hardware syncs between my main W7 PC and a new W8 netbook. However I also want to sync \Hardware\Phone with my phone but when I set up a new task I got the error message "This folder cannot be added to BitTorrent Sync as it is part of a folder that is already syncing.".

So I made a new directory "\Hardware\Phone\_BTSync to Phone" and set up a little utility called Synkron to copy relevant files into it.  Then, after deleting the PC sync task, I excluded the new directory using \Hardware\.SyncIgnore.  Or at least, I thought I did, although the syntax isn't entirely spelt out in the faq it looked obvious enough, just add a line

D:\Hardware\Phone\_BTSync to phone

 

Then I recreated the PC sync task and er, and then I had a nested directory \Hardware\Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone\_BTSync to Phone recurring forever.

 

Happily this was replicated on both PCs  :)

 

Then ensued a long scouring of the web for how to delete folders whose names are too long for Windows to cope and many, many fruitless attempts, including downloading dodgy utilities <avoid http://longpathtool dot com/ it's a scam>, following MS Technet threads, making batchfiles I barely understood, using Robocopy and Total Commander and even booting into Linux and using rm -fr *  In every case the names were too long or the directory tree was in use, or both.  On the main machine I've eventually had to admit defeat, make sure the backups were all up to date and reformat the drive, which has worked. And, praise be to Acronis, the restore worked.

 

Easy with a data drive, not so convenient when the system & data are both on the same drive.  I may have to factory reset the other machine, which is only a few days old so there's nothing significant to lose, except my time.

 

So don't do this at home kids, it's not hugely enjoyable :)

 

For the time being I've disabled BTSync until all is back to square 1.  I don't think what I was trying to achieve was unduly ambitious, so if anyone has any (obviously idiot proof) ideas as to the best way I'd be most grateful.

Link to comment
Share on other sites

@palpable Firstly, BitTorrent Sync isn't capable of "nested shares" (Please search for the forum for more on this, as well as potential work-arounds)

 

You're really over-complicating things by running BitTorent Sync and Synkron - i.e. running two different Sync apps in parallel - this is never going to be a good solution and inevitably leads to issues, like the race condition you're experiencing!

 

My advice:

1) Search the forums for better workarounds to the "nested share" issue - they do exist! (i.e. set up a separate sync for each sub-folder rather than a single sync on parent folder - this then allows you to selectively sync different folders to different devices, or sync the parent folder, but then investigate using .SyncIgnore to "exclude" certain sub-folders from certain devices, or if you're familiar with them, use symlinks/folder junctions, etc)

2) Most importantly - Don't run multiple Syncronization apps at the same time - that's never a good idea and WILL cause issues!

Link to comment
Share on other sites

Synkron is entirely innocent in this, it never actually got to run and in any case was only intended to perform a scheduled copy of *.txt once an hour.  Whatever happened was beteween me and BTSync, and tbh I'm a bit baffled about what transpired. I've done my best to explain what I did, but I'm at a loss to see how thousands of nested directories could be created and replicated.

 

I'm sure I could set up an individual sync for each of the 20 or so subdirectories of \Hardware, but that's hardly a convenient way of working.  I thought I had done "investigate using .SyncIgnore", before embarking on this-  I read both the offical and unofficial FAQs and looked through these froums.  I didn't find anything detailing the exact syntax I should use, nor any discussion of how to achieve what I was trying to do.  If you know where that info is, please point it out.

Link to comment
Share on other sites

Is there any way to use BtSync Via a proxy or tunneling. How can i set this up. I have tried setting up the proxy on the internet explorer but it appears that BtSync does not use the same path to connect to the internet.

 

Unfortunately, my network prvider blocks p2p and thus i must have use a proxy for various activities. as of now, i am unable to use BtSync on my machine

 

I look forward to your responses :-)

-Thanks

 

PS: I have a Virtual proxy server of my own.

Link to comment
Share on other sites

Can you answer to the questions from the attached picture and to add this necessery information to the new version of Bittorrent Sync to prevent these questions in future?

 

Yes, I've previously raised this with the development team so hopefully units will be indicated in a future build, but for reference, the units for the various numeric advanced settings are as follows:

folder_rescan_interval = seconds (default 600)

max_file_size_diff_for_patching = Megabytes (default 1000)

max_file_size_for_versioning = Megabytes (default 1000)

sync_max_time_diff = seconds (default 600)

sync_trash_ttl = days (default 30)

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share