upper- lowercase issue in heterogenous environments


raimokr

Recommended Posts

Hi

Some friends an I have set up a bittorrent sync share that consists of Windows 7, OS X and Linux clients.

The underlying file systems are:

- Windows: NTFS, exFAT (case insensitive)

- OS X: HFS+ (I think) (case sensitive)

- Linux: ext4, xfs (case sensitive)

We have the following problem:

- client A (Windows) creates a file "Test.txt"

- client B (linux) creates a file TEST.TXT"

- now client A has "Test.txt" / client B has "Test.txt" and "TEST.TXT" -> inconsistent

- Client B deletes "TEST.TXT"

- now client A deletes "Test.txt" and after that client B deletes "Test.txt" also -> all gone

There are other similar problems!

Hot to prevent btsync from doing this? Is it possible to force btsync to use case insensitiv paths?

Do we have to force that on the unerlying file systems?

Thanks!

Link to comment
Share on other sites

Hey there.

All your linux and OS-X clients could simply stop using file names that just differ by case. Even if it's possible and even though I use Linux all day long, I consider it a bad habit having no proper naming convention. If it's human readable stuff like word files, I name my files exactly the way they would be spelled in a real sentence. My native language is german, so we have upper cases for all nouns, for example. If it's not human readable, I usually create file names that don't contain whitespaces but use lowerCamelCase. But that has to do with coding guidelines I like to stick to, I guess.

HFS+ can be case sensitive, or can be case insensitive. Depends on mount options and/or partition settings, I think. I know that there are are both kinds of.

I know that there are other programs, e.g. some (not so old) Photoshop versions that just can't handel those. So it's not only a problem of btsync but seems to be kind of common in the iWorld.

You could simply switch to all case insensitive file systems on all your computers. Think about e.g. mobile phones where FAT is very common even if the phone itselfe (Android, in my case) is aware of case sensitivity.

Have a look at this one, there's a little more then just case sensitivity when it comes to different file sytems:

Regards,

Stephan.

Link to comment
Share on other sites

Hi Stephan

Thanks for you answer.

You talk more generally about how one should name files and paths. That's ok but filenames often are generated automatically or you haven't the possibility to specify this because of other dependencies...

btsync's sync algorithm has to handle these scenarios imo. At the moment we have problems like data loss, duplicates and sync loops because of this problem.

I'm not a developer and I'm not a sync guru and I don't know btsyncs sync algorithm but I would suggest not to use file names as identifiers for files, instead we could use e.g. MD5 hashs and metadata for file names, attributes and so on... Just an idea, maybe I'm wrong.

Great work anyway guys!

Regards,

Raimo

Link to comment
Share on other sites

  • 1 month later...

Hi!

 

I've bumped into this issue as well.

 

The way it showed up for me is that my desktop (ubuntu, case sensitive) had duped files  (RED-WOLF.JPG and red-wolf.jpg).  It would continually list itself as "syncing 96.5kB" but would never transfer anything.  The other computer connected is case insensitive. 

 

Nowhere in the log was this problem mentioned, it seemingly just spun its wheels with no indication what was broken.  After repeatedly upgrading to the newest version and posting the logs to the support site, I got a list of all the files from the folder on both computers, and diff'ed the lists. 

 

I understand that sync can't make these descrepancies go away, but it should at least notify the user when something is (possibly) wrong. 

 

This issue aside, btsync is great - thanks for ya'lls work on it. 

Link to comment
Share on other sites

It's a really tricky issue to address, and sadly more of a limitation of operating systems than of Sync.

 

For example, Windows would see "TEST.TXT", "test.txt", "Test.txt" (or any other variation) as being one and the same (try creating these variations in the same Windows folder - you can't!), where as Linux would see each of these as unique, individual files.

 

Because the Windows Operating System see "TEST.TXT", "test.txt", "Test.txt" as identical, Sync on Windows would also see these as identical.

 

It's a really difficult problem to address, and really, the best advice at the moment would be to avoid multiple files with the same name, but various cases, in the same Sync folder.

Link to comment
Share on other sites

  • 3 months later...

Case insensitiveness is so wrong. Especially if it's mixed with case sensitiveness within the same application.

 

And apparently btsync uses both approaches. If it tries to access files it uses whatever the OS thinks is right but then internally it uses case sensitive comparison.

 

For example when you rename the .SyncArchive folder to .SYNCARCHIVE on a windows machine. Then trashed files still land in this folder. But at the same time the folder gets synced to other clients because the name is no longer on the ignore list.

 

Someone I have a bittorrent sync share with did this. And whenever I tried to delete this folder his client moved his copy of the .SYNCARCHIVE folder into the .SYNCARCHIVE folder itself and synced it back to me. So now I had a .SYNCARCHIVE/.SYNCARCHIVE folder. :huh:

Edited by rob204
Link to comment
Share on other sites

  • 3 years later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.