File names too long

[ifoo]

I'm using btsync (1.0.134) on 3 linux machines (32 and 64 bit Ubuntu and 32 bit Debian wheezy) and am experiencing sync issues with long file names.

In order to provide enough security to store my data on untrusted hosts, I use encfs (https://en.wikipedia.org/wiki/EncFS) to encrypt my files. Unfortunately, encfs creates very long file names, for example

/home/ifoo/.encrypted/Documents/0ZmxrOATx4YgPC5jBr4gRkKU/-VljqWC1rAVUjB9GnB,iaBRY/3NeHN3GTSu5BSe9k9kKE1,T-/hYMaULYtXDVXkWcjRU1Y954D/x,u0qWitYS9Xl1E-mjxjzKMpi,XdoikblNSE0kJVDMKEkDvShNXVvgWmJTBUl6IzSzq1JmAPbBmcHc6Tx39Ao9dYvaatj890Q-HFooH,u,aE-wX-UZmWQlJKPdBZyRJJjvwlNErjSjPoE-Ppj9RQsLOz

is a single file name. It looks like btsync can not handle such long file names. Here is an entry from the log file:

[20130426 15:43:08] Error opening "/home/ifoo/.encrypted/Documents/0ZmxrOATx4YgPC5jBr4gRkKU/-VljqWC1rAVUjB9GnB,iaBRY/3NeHN3GTSu5BSe9k9kKE1,T-/hYMaULYtXDVXkWcjRU1Y954D/x,u0qWitYS9Xl1E-mjxjzKMpi,XdoikblNSE0kJVDMKEkDvShNXVvgWmJTBUl6IzSzq1JmAPbBmcHc6Tx39Ao9dYvaatj890Q-HFooH,u,aE-wX-UZmWQlJKPdBZyRJJjvwlNErjSjPoE-Ppj9RQsLOz.!sync":

[20130426 15:43:08] Error: x,u0qWitYS9Xl1E-mjxjzKMpi,XdoikblNSE0kJVDMKEkDvShNXVvgWmJTBUl6IzSzq1JmAPbBmcHc6Tx39Ao9dYvaatj890Q-HFooH,u,aE-wX-UZmWQlJKPdBZyRJJjvwlNErjSjPoE-Ppj9RQsLOz - WriteToDisk: File name too long

The web-ui is downloading everything except for the last 4.2 megabytes (which I assume is the blocksize of the torrent entries) and shows constant network traffic (221.3 kB/s up, 0.2 kB/s down).

@devs: My log file is 100mb+ so I provided just a single line. If you need more information, I am happy to upload the file somewhere. (see next post)

Edited May 26, 2013 by ifoo

[ifoo]

Here is a compressed version of the log file (uncompressed size: 135 MB) and a screenshot of the web interface.

Log file (xz compressed): https://s3.amazonaws...sync.log.tar.xz

[rdebath]

Okay kos13, that's it. I want to be able to store the partial files in the .Sync directory

(eg: .Sync/partial/7a/b8dc8456c25f132551f157c77a1888ef918fac.part )

NB: While they're at it clean up the clutter; move everything into the .Sync directory ... .Sync/Ignore, .Sync/Trash .Sync/ID .Sync/settings.dat .Sync/config.dat .Sync/metacache.db etc etc.

[rdebath]

ifoo, sorry I skipped a couple of steps.

Strictly speaking your issue isn't a BTSync one the problem is that a filename on ext3/4 can only be upto 255 bytes long; your filename is long enough that the extra extension the BTSync adds takes it to 258 characters. combine that with UTF-8 and the base64 encoding that encfs uses and you have very short filenames ...

this is too long "ｦｧｨｩｪｫｬｭｮｯｱｲｳｴｵｶｷｸｹｺｻｼｽｾｿﾀﾁﾂﾃﾄﾅﾆﾇﾈﾉﾊﾋﾌﾍﾎﾏﾐﾑﾒﾓﾔﾕﾖﾗﾘﾙﾚﾛﾜﾝ"

Add to that the fact that I really don't like the clutter that BTSync scatters around (eg the .!sync files) the obvious solution would be for BTSync to stop cluttering up the share with the *.!sync files and put them somewhere hidden, like a .Sync directory.

[ifoo]

@rdebath, thanks for the explanation. Makes perfect sense now. Are you aware of any temporary workaround (different filesystem that supports longer filenames or some way to tell encfs to produce shorter filenames)? Also, is there any indication from the devs that they would change the sync mechanisms to what you proposed in the post above?

[rebeccazoly]

I have download Long Path Tool and it is the best solution of your problem.

[bradmurray]

I just ran into this problem in Windows/OSX as well. I am syncing a file where the full path is 250 chars. If you add on the .!sync extension for the temp file you get to the magic 256 length and the file just syncs over and over again. It is only a 1MB file. I tried closing sync and manually syncing the file in both locations and then starting up, but it seems to puke on that length of file name. When I shorten the name everything works fine.

[Disappointed Cat]

Check out this thread.

I don't know about Mac, and Linux doesn't has this problem.

[ifoo]

@rebeccazoly, Long Path Tool seems to be Windows only and probably wouldn't help my case anyways, but thanks for the info.

@Lightning, thx, a dev just announced that this problem (the problem with UNC paths on Windows) will be fixed in the "next versions" (although, I wonder how):

Vadimt: We aware of this issue and it will be fixed in the next versions.

Thanks!

Edited June 9, 2013 by ifoo

[rdebath]

,

The problem to be solved in that thread was with UNC paths on windows.

The problem in this thread is for ALL platforms and is a simple result of making the temp files 6 characters longer than the existing file they refer to. When the existing file name is the maximum length it can be (or nearly so) the file cannot be transferred because the temp file name will be too long for the filesystem.

I propose the temp file name is in the (per share) .Sync directory with a name based on the info hash.

[Bootalito]

I'm trying to run resilio in a docker on UNRAID and the encrypted file names are +255 characters in length which is causing problems for XFS and BTRFS file system.  Is this something that resilio can limit?