2.3.1 Ghost Folders, .Conflict Directories, 100% CPU and poor support


sc

Recommended Posts

We're running 200 Gb of data in 300.000 files divided in about 20 folders (each folder is never more than 40 Gb).

We've been experiencing issues with "ghost" folder happening after renames of big folders.
We have a ticket open since November and after months of sending log files, a Teamviewer session showing the EXACT problem on two peers (that were not having any file permission issue) we have been told to update to latest to fix our problems.

What we've got is:

- All Macs are running at 100% CPU all the time while before they where fine (I still need to check the windows machines).
- Some folders now have the majority of the subfolders that have become .conflict folder, leaving the original one empty

The support is having hard time debugging the problem because they need the right logs from the right peers at the right time. The logs themselves are overwriting at incredible speed and makes this always very hard.

Why don't you provide a "Collect all logs from peers from Date to Date" functionality if you desperately need these logs every time?
 

Link to comment
Share on other sites

2 hours ago, sc said:

The logs themselves are overwriting at incredible speed and makes this always very hard.

Please see Increasing Debug Log size

A larger log size will mean the log gets rotated less frequently

2 hours ago, sc said:

Why don't you provide a "Collect all logs from peers from Date to Date" functionality if you desperately need these logs every time?

...because that would mean that Sync had to retain logs for an unlimited number of days/weeks/years, etc - not very practical!

Link to comment
Share on other sites

Another suggestion that I have is to provide a BitTorrent Sync Folder where users can upload log files and share then with you, because in your support page it's hard to upload log files that are several gigabytes big.

After all BitTorrent Sync should be the best tool to share large amount of large files with someone, right?

Why not use it to share logs?

Link to comment
Share on other sites

Hi

I just noticed I'm also having this problem. Not sure when it started happening but I'd say not too long ago, definitely sometime after upgrading to 2.3 around a week ago.

Worth mentioning, it's only happening for the moment in one particular shared folder, and only folders with accents on the filename (e.g. Títulos, Energético, Begoña, etc.) are getting these .Conflict, .Conflict1... .ConflictN siblings. No .Conflict in any folder without such characters. 

I'll try to get some logs during the following copule of days, and try to drop the shared folder and re-sync from scratch. For the record my setup includes one Synology DS415+ (Avoton), two Macs running El Capitán, one Android phone and one iPad, all updated to the latest versions (2.3.1 for the Synology and the Macs, latest version on the app stores for phone and iPad).

Regards

Link to comment
Share on other sites

Same problem here,

Hundreds of *.Conflict* files. Most of them symbolic links to folders with accents in filename. 

Mac OSX 10.11.3, BTsync 2.3.1

Even if one of the macs is set to stop syncing, the other generates the conflict files.

Edited by VMoreno
Link to comment
Share on other sites

I will write a detailed story some time in the future, but I would like to share with everyone that the latest BitTorrent Sync 2.3.1 update has been causing multiple .Conflict directories on regular subfolders and has been randomly deleting many files in some directories for some other obscure reason. Oh don't tell me that they've been moved to .archive directory please.

We have been forced to restore a backup from our constantly running Time Machine as the alternative would have been moving hundreds of deleted files one by one in their proper places, dividing what's in the archive because it has been explicitly deleted by some user from what is in the archive because latest BT update has gone nuts.

Losing files for a product that as of a version 2.x.x should be considered mature and stable is unacceptable.

We're sharing our gigabytes of log files with the PRO level support but we get useless and pointless answers.

As we cannot get to talk with a human being after months of malfunctioning behaviors, that I was able to _show_ to the support via Teamviewer, we're just closing the issue open since november 2015. This thing has been wasting already too much time in debugging BTSync software logs.

That being said, I would suggest any user to think twice before thinking this product could be more useful than personal backup.

BitTorrent Sync is not ready for company usage, even small company like us, unless you have few files and a strong backup policy.

Link to comment
Share on other sites

  • 2 weeks later...

@sc The 2.3 release had issues, many were addressed in 2.3.1 and even more fixes to come in 2.3.2. Unfortunately, the Support failed to find the root cause of your initial issue with renaming huge folders. The issue is unique and we don't have any other sources of debugging information other than yours setup (it does not reproduce in lab, neither for other users). I apologize for the situation. When we find the root cause and fix it - I'll inform you.

@VMoreno Indeed, we get some reports of appearing of .Conflicts folder, although we are still working to get it pinned. If you want to assist - please send us your logs so we can analyze the root cause.

 

Link to comment
Share on other sites

  • 2 weeks later...

@RomanZ I am ok in helping providing data to fix issues and I have made myself available multiple times for a team viewer session to show exactly the problem and gather all the logs together. I've lost patience when applying the update 2.3.1 that was supposed to fix some issues has disrupted our folders, forcing us to restore backup of several days before losing some files updated in the meantime. If a problem stays unsolved for months like in our case I think should allow us at least to talk to your support instead of just sending emails and log files, to explain some context that may help you finding your bugs.

During the only team viewer session that I was able to make with the support I have explicitly tried to talk to the support person but I have been replied that only chat is allowed. This is not helpful if you want to find and fix huge bugs like this one, that has been leading us to losing files during the update.

Link to comment
Share on other sites

Symptom: Multiple .conflict files/folders happening on foreign characters

Since: v2.3.3 (prior to 2.3.x did not had this issue)

Similar to accented characters, Korean characters also seems to confuse BTSync between OSX & Windows.

There had been some quirks between their foreign character compositions since decades ago regardless of BTSync. OSX saves "안녕하세요 (5 letters)" as "ㅇㅏㄴㄴㅕㅇㅎㅏㅅㅔㅇㅛ (12 separated consonants)" in a nutshell and then display it as final composition: "안녕하세요(back to 5 letters)", while Windows simply saves "안녕하세요" as "안녕하세요(5 characters)". :-S

So in theory, a folder named "한글" may exist multiple times, while seemingly identical. "한글" / "ㅎㅏㄴㄱㅡㄹ" / "한ㄱㅡㄹ( in theory ;p )"...

Hope this would help you support folks pin down the issue. 

Link to comment
Share on other sites

@zenyr We are aware of this issue. Mac stores UTF-16 symbols in decomposed format, while Windows and Linux does not care at all and may have both composed and decomposed UTF charactars within same filename. So at the end of day, it may happen (and often does) that Sync stumbles upon 2 files that have the same filename from human POV, while it is differently encoded in UTF. We don't have a solution for this issue yet.

Link to comment
Share on other sites

Hi

Ok @RomanZ, so this is a known issue with accents and other "combined" characters. Good to know.

For the record, I've been quite busy and couldn't take the time to do tests and get logs, but what I did is deleting the shared folders with this .Conflict issue (completely, from all systems), recreate them after removing all the .Conflict folders/files on one Mac and then share/resync in all the other systems. No issues since then, but after reading @RomanZ I guess it could be just a matter of time until this happens again. Maybe it has to do with where a file is created, then depending on how the "origin" OS treats the compressed/decompressed UTF filename it's shared that way to all the others leading to .Conflicts if the "receiving" OS doesn't use the same compressed/decompressed representation? Could this UTF compression/decompression be "normalized" in transit or upon receiving the file, depending on the OS?

Regards

Link to comment
Share on other sites

7 hours ago, RomanZ said:

@zenyr We are aware of this issue. Mac stores UTF-16 symbols in decomposed format, while Windows and Linux does not care at all and may have both composed and decomposed UTF charactars within same filename. So at the end of day, it may happen (and often does) that Sync stumbles upon 2 files that have the same filename from human POV, while it is differently encoded in UTF. We don't have a solution for this issue yet.

Thanks for clear statement. Then "UTF-composition-issue" is kinda "wontfix" labeled then? (pretty sure this includes accented characters as well)  While I understand your post, one weird thing is that I had only found this .conflict issue after half a year of BTSync usage. I've been developing a cordova app(hybrid app platform supporting multiple OS devices) and synced between W7 & OSX with no problem. Hmm. 

Anyways I think I could live with that, (maybe I could explicitly build up ios platform files on Windows machine then.) I'll find a workaround myself. 

Thanks!

Edited by zenyr
Link to comment
Share on other sites

@zenyr as GreatMarko said - we'll get it fixed as soon as we get some acceptable fix. Yes, this issue is also applicable to characters with diacritic signs. The most straightforward workaround is to use Sync to transfer files. As soon as you transfer it via, say. flash drive - you'll get the bunch of conflicts.

Although, it is also not an absolute solution: if any app you use on Windows / Linux will not normalize the filename to NFC (composed) canonical form - you'll get a conflict when syncing to OS X.

Link to comment
Share on other sites

Yes, BTSync itself alone does NOT cause .conflict issue indeed. I cannot think of a rational "fix" either in this case! My cordova-cli need to re-prep each .html & assets on each build, causing this issue. (seemingly it cleans the directory and rebuild from ground up)

Thanks for all your help and corrections. I do stand corrected thanks to you.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.