Filename Length Restriction When Syncing Encrypted Files

[dddpaul]

 

Hi.

 

I'm trying to sync encrypted filesystem between these nodes:

- local - home desktop with Ubuntu 12.04 x86_64, btsync 1.3.106;

- remote - Digital Ocean droplet with Ubuntu 14.04 x86_64, btsync 1.3.106.

 

There are  93348 files totally, most of them encrypted with ecryptfs.

 

All these files is indexed by btsync on the local node successfully:

6.16 GB in 93348 files

 

And remote site shows this after syncing:

4.41 GB in 82223 files

 

Remote site log have bunch of entries like this:

 

[20140725 08:15:12.202] TorrentFile: Failed to create empty suffix for file "/home/paul/.Private/Dropbox/ECRYPTFS_FNEK_ENCRYPTED.FWbjThHi7A-2EESY6saysdXRllqFB1HW7SMG5B6XtmxAhG7i5ufKLFe9tk--/ECRYPTFS_FNEK_ENCRYPTED.FXbjThHi7A-2EESY6saysdXRllqFB1HW7SMGmSrAzHw50.jjxikqnnbHsOh1NXTPR5hBLzy8RwPBFro-/ECRYPTFS_FNEK_ENCRYPTED.FWbjThHi7A-2EESY6saysdXRllqFB1HW7SMGZE4jmDMGFSk3-ZWAVR3a-k--/ECRYPTFS_FNEK_ENCRYPTED.FXbjThHi7A-2EESY6saysdXRllqFB1HW7SMGKTsmmN8svnFWsl8DFf4syVi7fA7gAdfaqeIf1lejI.A-/ECRYPTFS_FNEK_ENCRYPTED.FebjThHi7A-2EESY6saysdXRllqFB1HW7SMGqM9FDsE8E9L4QugSy76rF17SRBiMKue21o4-4eY4DnWdV3Hxm-ZaW.8UgClFCRubQCoFqiCiK4490NQSa9UQtjNZeilWqyobZLvCjCTZUpm7Z-h6FiEWlYO5XBOVkVevjZ25ZMGuMPMFoQ5RvCqN1jZgDFZnC3.HJechP.XwZ-MhyrLgMbMDKgFKAmfgtyNX" - 36

 

[20140725 08:15:12.202] SyncFilesController: failed to load torrent for file "/home/paul/.Private/Dropbox/ECRYPTFS_FNEK_ENCRYPTED.FWbjThHi7A-2EESY6saysdXRllqFB1HW7SMG5B6XtmxAhG7i5ufKLFe9tk--/ECRYPTFS_FNEK_ENCRYPTED.FXbjThHi7A-2EESY6saysdXRllqFB1HW7SMGmSrAzHw50.jjxikqnnbHsOh1NXTPR5hBLzy8RwPBFro-/ECRYPTFS_FNEK_ENCRYPTED.FWbjThHi7A-2EESY6saysdXRllqFB1HW7SMGZE4jmDMGFSk3-ZWAVR3a-k--/ECRYPTFS_FNEK_ENCRYPTED.FXbjThHi7A-2EESY6saysdXRllqFB1HW7SMGKTsmmN8svnFWsl8DFf4syVi7fA7gAdfaqeIf1lejI.A-/ECRYPTFS_FNEK_ENCRYPTED.FebjThHi7A-2EESY6saysdXRllqFB1HW7SMGqM9FDsE8E9L4QugSy76rF17SRBiMKue21o4-4eY4DnWdV3Hxm-ZaW.8UgClFCRubQCoFqiCiK4490NQSa9UQtjNZeilWqyobZLvCjCTZUpm7Z-h6FiEWlYO5XBOVkVevjZ25ZMGuMPMFoQ5RvCqN1jZgDFZnC3.HJechP.XwZ-MhyrLgMbMDKgFKAmfgtyNX"

 

Looks like it'a very long filename Full pathname is 660 characters long and filename is 252 chars long.

 

Another example:

 

[20140725 08:41:16.157] TorrentFile: Failed to create empty suffix for file "/home/paul/.Private/Dropbox/ECRYPTFS_FNEK_ENCRYPTED.FWbjThHi7A-2EESY6saysdXRllqFB1HW7SMG5B6XtmxAhG7i5ufKLFe9tk--/ECRYPTFS_FNEK_ENCRYPTED.FXbjThHi7A-2EESY6saysdXRllqFB1HW7SMG-fiq9d8IRiJbV-12CNjd-9WDR3lQlX4pOOBTHjxNjHE-/ECRYPTFS_FNEK_ENCRYPTED.FYbjThHi7A-2EESY6saysdXRllqFB1HW7SMGikXbsA3GPZ1B54jPn3UBD0TV93dXKWqUgizqc.uCT-hXk9At5ke.TggDfRxT-.5N/ECRYPTFS_FNEK_ENCRYPTED.FebjThHi7A-2EESY6saysdXRllqFB1HW7SMG-fiq9d8IRiJbV-12CNjd-5zsJpo7bVBHMfyWsUSmfSOw92uqpwE9jSii6oaOe-CH4Yj9Ls0nxM6I8-K9pecrDl8KD6T4SN8hM6ReSf31.YTkwlTVWHibImCCj0g5QDP7-nTQMr76AwAYl.pjbp9iq5AWD-nPHepqWeQ3UkMo67bfLFIAInWCcQvA2HUZOoxm" - 36

 

[20140725 08:41:16.157] SyncFilesController: failed to load torrent for file "/home/paul/.Private/Dropbox/ECRYPTFS_FNEK_ENCRYPTED.FWbjThHi7A-2EESY6saysdXRllqFB1HW7SMG5B6XtmxAhG7i5ufKLFe9tk--/ECRYPTFS_FNEK_ENCRYPTED.FXbjThHi7A-2EESY6saysdXRllqFB1HW7SMG-fiq9d8IRiJbV-12CNjd-9WDR3lQlX4pOOBTHjxNjHE-/ECRYPTFS_FNEK_ENCRYPTED.FYbjThHi7A-2EESY6saysdXRllqFB1HW7SMGikXbsA3GPZ1B54jPn3UBD0TV93dXKWqUgizqc.uCT-hXk9At5ke.TggDfRxT-.5N/ECRYPTFS_FNEK_ENCRYPTED.FebjThHi7A-2EESY6saysdXRllqFB1HW7SMG-fiq9d8IRiJbV-12CNjd-5zsJpo7bVBHMfyWsUSmfSOw92uqpwE9jSii6oaOe-CH4Yj9Ls0nxM6I8-K9pecrDl8KD6T4SN8hM6ReSf31.YTkwlTVWHibImCCj0g5QDP7-nTQMr76AwAYl.pjbp9iq5AWD-nPHepqWeQ3UkMo67bfLFIAInWCcQvA2HUZOoxm"

 

Full pathname is 595 characters long and filename is 252 chars long.

 

And I repeat - btsync have no problem with indexing these files on my local node. So why these long filenames isn't indexed on the remote node?

[dddpaul]

I've made some tests with different filename lengths. In each test filename consists of given number of '1' characters and file content itself consist of the same number of '1' plus ending CRLF.

 

1. Filename length is 248 characters.

 

Local node:

=11:20=paul@paulhome[~/.Private]$ cat > 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111=11:20=paul@paulhome[~/.Private]$ ll-rw-rw-r--   1 paul   paul    249 Jul 26 11:20 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

Remote node:

=08:21=root@btsync[/home/paul/.Private]# ll-rw-rw-r-- 1 paul btsync  249 Jul 26 08:20 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111sync.log:[20140726 08:21:14.331] Got metadata size 68, pieces 1[20140726 08:21:14.494] Finished receiving metadata for '/home/paul/.Private/11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111'[20140726 08:21:14.520] 5.35.3.162:64021: did not pick any blocks. blocking peer temporarily[20140726 08:21:14.729] /home/paul/.Private/11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111: Piece 0 complete

So, everything is ok.

 

2. Filename length is 249 characters.

 

Local node:

=11:25=paul@paulhome[~/.Private]$ cat > 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111=11:25=paul@paulhome[~/.Private]$ ll-rw-rw-r--   1 paul   paul    250 Jul 26 11:25 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

Remote node:

=08:26=root@btsync[/home/paul/.Private]# ll-rw-rw-r-- 1 paul btsync    0 Jul 26 08:25 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111.!syncsync.log:[20140726 08:25:55.697] Blocked downloading file 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 due Connection closed

btsync have created !sync file but then failed to sync content.

 

3. Filename length is 250 characters.

 

Local node:

=11:02=paul@paulhome[~/.Private]$ cat > 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111=11:02=paul@paulhome[~/.Private]$ ll-rw-rw-r--   1 paul   paul    251 Jul 26 11:02 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111

Remote node:

=08:04=root@btsync[/home/paul/.Private]# ll# No such file on remote nodesync.log:[20140726 08:03:32.400] TorrentFile: Failed to create empty suffix for file "/home/paul/.Private/1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111" - 36[20140726 08:03:32.401] SyncFilesController: failed to load torrent for file "/home/paul/.Private/1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111"[20140726 08:03:33.455] TorrentFile: Failed to create empty suffix for file "/home/paul/.Private/1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111" - 36[20140726 08:03:33.455] SyncFilesController: failed to load torrent for file "/home/paul/.Private/1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111"...

No files at remote node at all.

 

Environment.

 

Local node:

=11:38=paul@paulhome[~/.Private]$ cat /etc/lsb-release DISTRIB_ID=UbuntuDISTRIB_RELEASE=12.04DISTRIB_CODENAME=preciseDISTRIB_DESCRIPTION="Ubuntu 12.04.4 LTS"=11:38=paul@paulhome[~/.Private]$ uname -aLinux paulhome 3.11.0-26-generic #45~precise1-Ubuntu SMP Tue Jul 15 04:02:35 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux=11:39=paul@paulhome[~/.Private]$ /usr/lib/btsync-common/btsync-core --helpBitTorrent Sync 1.3.106

Remote node:

=08:40=root@btsync[/home/paul/.Private]# cat /etc/lsb-release DISTRIB_ID=UbuntuDISTRIB_RELEASE=14.04DISTRIB_CODENAME=trustyDISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"=08:40=root@btsync[/home/paul/.Private]# uname -aLinux btsync 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux=08:40=root@btsync[/home/paul/.Private]# /usr/lib/btsync-common/btsync-core --helpBitTorrent Sync 1.3.106

Edited July 26, 2014 by dddpaul