Illegal characters replaced - ???.bit:com.bittorrent.sync was renamed using supported symbols

[jammin]

I have a rather annoying problem.  Sync usually creates .bts files on the new device as placeholders for the downloading file.  But in my case it's creating files with a ":com.bittorrent.sync" extension which it then attempts to sync to other devices, resulting in the message:

Illegal characters replaced
???.bts:com.bittorrent.sync was renamed using supported symbols

The file then gets removed (from all devices) once the actual file has completed syncing.  Ultimately it seems to sort itself out in the end but creates a lot of annoying error messages and unneccessary chatter in the process.  I also end up with a lot of deleted placeholders in my "Archive" folder (see sync2.jpg).

The problem has occurred with several different folders.   All devices are running Sync Pro 2.3.7, two of them installed as a service and one running as a regular app.  There's also an android device and an iOS device, both with the folder connected in "selective" mode.

Any ideas what could be causing it?  I am mostly loving sync and this is really my only major complaint.

edit: originally I stated that it only happened on newly added folders, but this is not the case.  It's also happening on syncing folders that were already connected.

edit2: originally I was using Sync Pro 2.3.6 but I have since updated to 2.3.7 and am still experiencing the issue.

Edited May 10, 2016 by jammin

[GreatMarko]

The issue is due to the fact that a colon character (:) is not a valid character in file names under Windows operating systems. Therefore, if you're syncing a file from another OS where : characters are allowed in filenames, to a Windows system, the : character will be substituted for an underscore.

To avoid this, ensure that the filenames of the files your syncing contain characters which are legal on all the operating systems you're syncing between.

[jammin]

1 hour ago, GreatMarko said:

The issue is due to the fact that a colon character (:) is not a valid character in file names under Windows operating systems. Therefore, if you're syncing a file from another OS where : characters are allowed in filenames, to a Windows system, the : character will be substituted for an underscore.

To avoid this, ensure that the filenames of the files your syncing contain characters which are legal on all the operating systems you're syncing between.

Thanks for the reply.  I understand that colon is an illegal character on windows, and my files do not have a colon character in their names.  In the above example, my file is DSC06609.ARW and Sync has added the ";bts:com.bittorrent.sync" part itself when creating some kind of temporary file, which it then tries to sync.

 

 

[Helen]

:com.bittorrent.sync is not extension, it's a file stream. this is how Windows keeps them filename:<stream_name>, the streamname being invisible . BTW this is why Windows doesn't allow colon in filename - by colon it recognizes stream. 

I suspect that your sync share is actually physically located on a Linux (which does not support streams) and mapped to your Windows over smb, or this windows is a Virtual machine on a linux host sharing a host's mapped folder? ....or a similar setup? but the key point here is that when a file arrives at Windows through Sync, NTFS saves it correctly (filename:<stream>), then this file is passed over to the linux for which it looks just like a regular filename: file:stream, which returned back to PC. 

This was worked around longs ago, so perhaps your case is something yet undiscovered? any details about your setup? 

[jammin]

11 hours ago, Helen said:

:com.bittorrent.sync is not extension, it's a file stream. this is how Windows keeps them filename:<stream_name>, the streamname being invisible . BTW this is why Windows doesn't allow colon in filename - by colon it recognizes stream. 

I suspect that your sync share is actually physically located on a Linux (which does not support streams) and mapped to your Windows over smb, or this windows is a Virtual machine on a linux host sharing a host's mapped folder? ....or a similar setup? but the key point here is that when a file arrives at Windows through Sync, NTFS saves it correctly (filename:<stream>), then this file is passed over to the linux for which it looks just like a regular filename: file:stream, which returned back to PC. 

This was worked around longs ago, so perhaps your case is something yet undiscovered? any details about your setup? 

Thanks Helen.  There are no linux boxes involved, only Windows 10 machines and a couple of mobile devices (which I have disconnected for now).

However one of the windows machines has it's folder on a FlexRaid storage array (a software RAID solution) which creates a virtual drive out of a pool of hard drives. I wonder if that has something to do with it, since it's essentially emulating the NTFS file system - perhaps it doesn't have support for streams.  I'll post on their forums and see if I can find out anything.

I must admit I was curious why searching didn't come up with a single relevant result, as I guess I'm the only person using these two particular pieces of software together.

 

Edited May 11, 2016 by jammin

[Helen]

@jammin

Thanks for the details. Managed to reproduce it with FlexRaid. Actually you're the second person ever with this problem and the first one with it on FLexRaid Frankly speaking I don't know details of how that software works, need to read documentation, so adding this case to the fix may take time.

So far adding *com.bittorrent.sync to IgnoreList on this window (and other Windows with same problem) helps. Sorry, you'll need to re-add the folders, cause the streams already synced there won't un-sync. And with the stream in Ignore you will not see process bar on a bts in Windows Explorer. 

[jammin]

8 hours ago, Helen said:

Thanks for the details. Managed to reproduce it with FlexRaid. Actually you're the second person ever with this problem and the first one with it on FLexRaid Frankly speaking I don't know details of how that software works, need to read documentation, so adding this case to the fix may take time.

Awesome, I didn't expect you to go to that much trouble, and greatly appreciate it!  What was the other person who experienced this problem running?

I've been in touch with FlexRaid support and they are puzzled by it.  Please let me know if you think it's something they need to address with as much info as possible.

8 hours ago, Helen said:

So far adding *com.bittorrent.sync to IgnoreList on this window (and other Windows with same problem) helps. Sorry, you'll need to re-add the folders, cause the streams already synced there won't un-sync. And with the stream in Ignore you will not see process bar on a bts in Windows Explorer. 

Thanks for the workaround.  The problem seems to sort itself out without the IgnoreList entry - the errant file gets synced around and then deleted - so I think I'll leave it and wait hopefully for a fix.  

 

Edited May 11, 2016 by jammin

[Helen]

@jammin

This is all about how RAID processes notifications on stream: from it Sync receives  FS notification about stream itself, while with native Windows system Sync receives a notification about the file to which that steam belongs. 

We have worked this around in Sync, but fixing the way FlexRaid works with stream is not under our control. I've PM'ed the build with fix to you. 

[jammin]

Thanks Helen, as I mentioned in the PM the new build seems to have fixed the problem.  Thanks!

So in your opinion is this a bug with Flexraid, or is this method of notification using the stream common to other RAID systems too?  

 

[jammin]

For the record, the new build fixes the error, but has a side-effect of not picking up changes on the Flexraid volume until a scheduled reindex.

I've advised Helen by PM and will post any update here.

[Helen]

We've sent the fix to @jammin some time ago, but haven't heard anything back. So I assume it's working OK? 

[jammin]

On 09/06/2016 at 4:42 PM, Helen said:

We've sent the fix to @jammin some time ago, but haven't heard anything back. So I assume it's working OK? 

I am still having this issue occasionally but have not been able to pin down the exact cause.  If I can I will repost here.