Locked files only when copied automatically

[jbrugger]

Scenario: 1 PC with a shared folder, Resilio sync to sync that folder with another PC.

If I use an "automated" (windows scheduled task) process to copy some files to the shared folder, files remain "locked" and Resilio dont syncronize them.

If I copy manually same files, all goes fine.

Scheduled task runs as the same user I use.

Log says:
 

Quote

[2017-02-02 13:17:15] Entry "\\?\C:\compartido\Estadisticas\2017-02-02\Perfumeria y Accesorios.xls" was updated, current state: pvi[type:1 s:4322578 mt:1486045327 h:0000000000000000000000000000000000000000], expected: pvi[NULL]
[2017-02-02 13:17:15] FC[0A3E]: fs event for entry "\\?\C:\compartido\Estadisticas\2017-02-02\Perfumeria y Accesorios.xls" t:1486045327 s:4322578 id:655360:521019 type:1
[2017-02-02 13:17:15] SyncTorrentCreator: failed to obtain opp lock for file "\\?\C:\compartido\Estadisticas\2017-02-02\Perfumeria y Accesorios.xls" - 3221225698
[2017-02-02 13:17:15] JOURNAL[0A3E]: cancelling new entry job for "\\?\C:\compartido\Estadisticas\2017-02-02\Perfumeria y Accesorios.xls" - failed to init tf
[2017-02-02 13:17:15] API: callback id=12, value="{ "value": "Can't sync 'Perfumeria y Accesorios.xls', because it is locked by another application." }", can_deferred=1, _delegate=0x5a4830
[2017-02-02 13:17:15] WebUISession call callback 12, ListeningSyncEvents = 1
[2017-02-02 13:17:15] JOURNAL[0A3E]: repost fs event for path "\\?\C:\compartido\Estadisticas\2017-02-02\Perfumeria y Accesorios.xls"  - torrent creation aborted
[2017-02-02 13:17:15] [OnNotifyFileChange] "\\?\C:\compartido\Estadisticas\2017-02-02\Perfumeria y Accesorios.xls"

What could be happening and how can I fix this?

Thanks!

[Helen]

 failed to obtain opp lock for file ".....xls" - 3221225698

3221225698 is STATUS_OPLOCK_NOT_GRANTED. To put it simply - when Sync tried to read the file, system kernel informed that oplock is not available, i.e. some other process is holding handle for file. So Sync can do nothing here but to skip the file and try again later. 

[sbhouchen]

Helen, do you know how soon the "try again later" happens?

 

[Helen]

actually there are a lot of "ifs: if it's local file uploading or downloading file from remote (updating), if it's new file or file update, etc. It can be either 10 secs or up to 5 minutes. 
restarting Sync may help 

[jbrugger]

I can confirm 2.4.4 has a ¿bug? not recognizing when a locked file is unlocked. Restarting Resilio service works to fix the issue (but you have to do manually every time you see locked files).

I hope dev team can find and fix this issue.

[Helen]

Can you try and wait for 10 minutes (or your rescan interval if you've changed that) after you close the app. 
We've been testing the case in the lab, and once the file is indeed released, Sync is notified about it and rechecks the files. 
In your case restarting may be required, for example,  if there was no system notification. 

[jbrugger]

Thanks Helen.

I waited several hours and even days, with no luck. I usually realize this when I look for a file and I can not find it on the PC where it should have been synchronized.

I'm pretty sure this did not happen in versions prior to 2.4.4 (not 100% anyway)

[Helen]

ok, then can you please collect the logs from this PC, and send them to support? Here's full instruction, and please mention the filename in the logs. Thanks. 

[flabbamann]

Hi, I disabled Filesystem Notifications because of this problem:

So, when a file is open in another Program, Resilio can't sync it. But when the lock is released the file is still not transferred. Not after waiting 10 minutes (default rescan interval) and not even after a restart of Resilio.

[Timbo]

On 2/15/2017 at 11:56 PM, Helen said:

Can you try and wait for 10 minutes (or your rescan interval if you've changed that) after you close the app. 
We've been testing the case in the lab, and once the file is indeed released, Sync is notified about it and rechecks the files. 
In your case restarting may be required, for example,  if there was no system notification. 

Helen,

Use case:

uTorrent running on Windows Home Server 2011. Upon torrent completion, file is copied from a temp dir to a network share on the WHS that is also a sync folder with my main Win10 PC. File may or may not continue to be available for seeding.

Whenever I've had this problem, I've used "Unlocker" to check and see what file had a file lock, and it always comes back saying there isn't a lock.  However, I know that if I remove the file from uTorrent and restart Resilio, the non-stop notifications about the locked file go away (they don't go away on their own until restarted, so bug exists here).  But, the file still never syncs (it's FN annoying to discover this after days of non-syncing).  If I close Resilio, copy the file manually to the destination sync folder that wasn't getting the file copied automatically (which ends up changing the timestamp I believe), then I end up with HUNDREDS OF KILOBITS PER SECOND of constant traffic trying to copy the file BACK to the WHS folder.  It's trying to overwrite the original file with a copy of itself over and over, due to this locked permissions thing.  It's the same dang file!  Some really bad error handling logic from what I can see.  Glad this is over a local LAN and not using up my bandwidth.  Actually, now I wonder if this could explain some unexplained traffic spikes as I also have a Resilio sync across the Internet.  I was gone for 3/4 weeks in December and yet my Internet bandwidth was higher than average months. (I didn't investigate because I wasn't near by monthly limit to care enough).  Probably wasn't good on the SSD for all the constant log writes.

Same thing happened on my PlayOn PC.  Video's would record to a folder and be constantly written to for the duration of the stream which may be minutes to an hour or more. This starts endless popup notifications about not being able to get lock (WHS would show endless, but I just tried and got a notification at start of video recording but wasn't continuous on a Win10 PC). Then when the recording finished, the file did not get synced, even if Resilio was restarted.

This is broken behaviour that started after 2.3.8.  I've reverted to 2.3.8 on the WHS and problem went away.  It also automatically fixed all the not-synced files that I didn't manually copy over that Resilio never synced (reverting to 2.3.8 lost all my sync, even when I restored the database file as per a post here, but whatever).

The file locking changes added in 2.4.x are buggy and break previous functionality.  Please test the very typical use case where a program outputs a file to a sync folder and watch for Resilio to start spamming with the notifications.

Edited March 1, 2017 by Timbo
Win10 wasn't showing endless notifications, just WHS

[Helen]

Timbo,

thanks for giving another use case. The team is working on this. 

[Flivver]

Anyone have a solution to this? I have the exact same issue:

- Resilio Synch  2.5.2 (966) Home running on Windows 7 Home Premium, 32Bit Sp1

• Files are downloaded by µTorrent. Once the download is complete, the file is copied by a script (using XCOPY) to another location, which is synched with Resilio.
• Reslio reports the file is locked, and will not sync it. Lock is shown as never released
• If you stop/start Reslio it then no longer shows the file as locked, but does not sync it (even though it indicates all files are synced). File never makes it to the other machines.
• If the exact same task is performed using a manual copy (in Windows Explorer) the file will sync OK.
• If the destination is a Dropbox folder, sync works perfectly (been using the script to do this for a couple of years)

This issue is a show stopper; if no solution is available in the current release, then I'd like to try the older version (I see a comment this worked ok in 2.3.8). Is this available as download from Reslio's web site? 

Hoping someone has a solution....

-Steve

 

 

 

[Timbo]

1 hour ago, Flivver said:

Anyone have a solution to this? I have the exact same issue:

- Resilio Synch  2.5.2 (966) Home running on Windows 7 Home Premium, 32Bit Sp1

• Files are downloaded by µTorrent. Once the download is complete, the file is copied by a script (using XCOPY) to another location, which is synched with Resilio.
• Reslio reports the file is locked, and will not sync it. Lock is shown as never released
• If you stop/start Reslio it then no longer shows the file as locked, but does not sync it (even though it indicates all files are synced). File never makes it to the other machines.
• If the exact same task is performed using a manual copy (in Windows Explorer) the file will sync OK.
• If the destination is a Dropbox folder, sync works perfectly (been using the script to do this for a couple of years)

This issue is a show stopper; if no solution is available in the current release, then I'd like to try the older version (I see a comment this worked ok in 2.3.8). Is this available as download from Reslio's web site? 

Hoping someone has a solution....

-Steve

 

 

 

Steve,

That sucks. I thought 2.5.2 was going to be the fix.  On my Playon media PC running version Resilio 2.4.4, it would behave similar to uTorrent in that it would start saving a continuous stream of video to a folder within Resilio sync and get a message about the file being locked. When I ran into this the other day, I updated it from 2.4.4 to 2.5.2 and there was an observed difference in the file locking notifications and the file did get transferred when it was finished streaming. I didn't update my uTorrent PC to 2.5.2 from 2.3.8 mostly because I didn't have time, but now that you confirmed this bug isn't truly solved, I'll hold off on upgrading as well.

If you do go back to 2.3.8, you will have to setup your sync configuration again.  I believe there were instructions to restore a backup database from 2.3.8 or something, but it didn't work for me.

"If the exact same task is performed using a manual copy (in Windows Explorer) the file will sync OK."

You may not notice unless you are running a program that graphs your transmit/receive bandwidth, but when I would manually copy the file, there would be hundreds of kbps of data trying to resend the file over and over and over... (despite the file existing in both folders, the timestamp is messed on one of them so it thinks it needs to resend) you'd have to view the logs to see it if you didn't notice the constant background traffic.  

Or did you mean this only happens when using the xcopy script? Not that it matters, but any reason you don't just have utorrent copy the file when its completed? I guess if you copy it to multiple places, but ain't that what Resilio is for?

[Helen]

Steve, once lock is 'resleased', try to touch a file, for example, manually move it out of share and put it back in a while. The file shall get uploaded now. 

Unfortunately, the issue with syncing locked files is not yet fixed in v2.5.2. 

 

[Flivver]

Thanks Helen; although that's useful to know it doesn't really help in my situation, as the machine needs to operate unattended.

Good to know that your team is aware of the problem - do you have any idea when it may be resolved?

In the interim, would you be able to post a link to the older version (2.3.8) which others report works? (I really want to be be able to evaluate before purchasing the 'Pro' version and migrating away from DropBox, but need this to work. I'd prefer not to download from one of the 3rd party sites which might have 'dodgy' versions.....

-Steve

 

 

[Flivver]

On 6/1/2017 at 0:49 AM, Timbo said:

Steve,

That sucks. I thought 2.5.2 was going to be the fix.  On my Playon media PC running version Resilio 2.4.4, it would behave similar to uTorrent in that it would start saving a continuous stream of video to a folder within Resilio sync and get a message about the file being locked. When I ran into this the other day, I updated it from 2.4.4 to 2.5.2 and there was an observed difference in the file locking notifications and the file did get transferred when it was finished streaming. I didn't update my uTorrent PC to 2.5.2 from 2.3.8 mostly because I didn't have time, but now that you confirmed this bug isn't truly solved, I'll hold off on upgrading as well.

If you do go back to 2.3.8, you will have to setup your sync configuration again.  I believe there were instructions to restore a backup database from 2.3.8 or something, but it didn't work for me.

"If the exact same task is performed using a manual copy (in Windows Explorer) the file will sync OK."

You may not notice unless you are running a program that graphs your transmit/receive bandwidth, but when I would manually copy the file, there would be hundreds of kbps of data trying to resend the file over and over and over... (despite the file existing in both folders, the timestamp is messed on one of them so it thinks it needs to resend) you'd have to view the logs to see it if you didn't notice the constant background traffic.  

Or did you mean this only happens when using the xcopy script? Not that it matters, but any reason you don't just have utorrent copy the file when its completed? I guess if you copy it to multiple places, but ain't that what Resilio is for?

 

Investigated this some more. Discovered that if you rename the file or folder after the file has been copied, then reslio doesn't have the lock issue. Also (strangely) it seems that if multiple files were copied (or perhaps because I was using XCOPY /S) then the problem doesn't occur.

So, I have modifed my uTorrent script that copies the files so that if it is a 'single' copy it will add a "+" to the folder name (then remove after the copy is complete). If it is a "multi" copy (i.e. a set of multiple files and/or folders) that is not necessary. Just in case it is useful to someone, here's the code from the script that is triggered by uTorrent each time the status of a download changes. Once the download is complete, the files are copied to a reslio folder. I have confirmed this works with Reslio 2.5.2 running on Win 7 SP1

REM %4 is status, value 5 is seeding (i.e. we have the entire file)
REM %3 is the name; either of a single file or a folder with multiple files)

SET sDestination=D:\SynchedMedia\Video

IF %4 == "12" echo %DATE% %TIME%: %3 is queued for download      >> "%sDestination%\uTorrentStatus.txt"
IF %4 == "6"  echo %DATE% %TIME%: %3 is downloading              >> "%sDestination%\uTorrentStatus.txt"

IF %5 == "multi" goto foldercopy

 

REM We append the "+" to the start of the file name so we can then change the filename back later...
IF %4 == "5" ( echo D |xcopy %2\%3 "%sDestination%\"+%3  )

REM Renaming the file works around a problem Resilio Synch has with the file lock; below we are removing the "+" we added before

ren "%sDestination%\"+%3 %3

goto donecopy

:foldercopy
IF %4 == "5" ( echo D |xcopy %2 "%sDestination%\"%3 /S )

:donecopy

IF %4 == "5"  echo %DATE% %TIME%: %3 has been copied to %sDestination%  >> "%sDestination%\uTorrentStatus.txt"

REM The End

 

 

 

 

 

[TheBestPessimist]

Hello,

Just wanted to say that i also have a problem with locked files.

My usecase: 

- sync v252 (966)

- windows 10 pro x64 vm on vmware pro 12.

- vm location: parity storage spaces, with parity (RAID 1 equivalent)

- i am copying a 3 GB 3000 photos (jpeg, tiff, etc.) to the same folder, which is synced. (ctrl+a -> ctrl +v  10 times ).

- this means that i have  10 windows explorer copy dialogs open.

- while resilio is syncing with my other vm (identical to this one, on the same virtual storage pool), some files appear as locked, and even after the file copy is finished and the VMs restarted, and resilio restarted as well, it sill does not sync the locked files. 

- even more: now it simply doesnt' show that there are any unsynced files.

- in order to see that there are unsynced files, i use windows explorer-> show properties, and see how many files are on the disk of the target vm.

 

As you can see, in my usecase there is nothing weird happening, just me copying some files. Note that since i am doing so many different copies, the copy speed is very slow, maybe even less than 10 MBPS.