RomanZ
-
Posts
3,588 -
Joined
-
Last visited
Posts posted by RomanZ
-
-
Guys,
Anyone else with "saw-shaped-traffic" issue managed to do the test with 1-2megs numerous file sync between 2 peers?
-
s24joe,
I would also appreciate a console output for couple of commands, please perform second command in the same folder where your problematic sync is stored:
ps aux | grep btsync
ls -l
Thanks!
-
Isn't testing with a bunch of small files a different scenario since BTsync probably isn't breaking those files up into separate segments/blocks/pieces since they're so small? I know that when I have a folder with 75 zip files, each around 50MB, the slow/fast pattern occurs.
Could this be an ISP throttling issue? I'm pretty sure I noticed the same pattern when I setup a private bittorrent tracker to share a file between my two windows computers. If so, how do I get around it? I need to do some more testing and report back.
Also from what I noticed, BTSync will not download more than one file per folder. If I add one new folder with 50 files it will only sync one file at a time. If I add two or more folders then BTSync will download multiple files at the same time but still only one file per folder.
When you are sending files below ~12 megs, BTSync is going to open several connections instead of only one and transfer several files simultaneously. Standard piece being send is around 1 meg, so these files would consist of at least 2 pieces.
I suspect it could be ISP throttling. For now it looks like someone (router? ISP? some relay server) intentionally drops a packet or tho every time period and limiting the traffic by doing so. If this is true, then opening several connections is going to occupy full bandwidth.
From other side, if there is a bottleneck on, say, your router because of amount of UDP packets - splitting connections is not going to help as amount of packets should stay the same.
I did some testing and found out that if I have a folder inside the sync folder that has multiple files of different sizes between 3-15MB each, whenever btsync syncs sub-10MB files it send 2 files concurrently, resulting in more constant speed. Whenever it was time to sync a 10MB+ file it only did this on a single thread, causing the bandwidth to throttle after every 10 seconds or so and then pick up again until it throttled it again. On a network utilisation graphs this looks like a 'saw' pattern that was on the first post(s).
[...]
It would be nice to have a setting where we can set how many concurrent connections, threads or blocks to try to handle at the same time, going into tens of blocks per file and multiple files at the same time should be configurable.
sty,
Thanks for the information. In your case it looks like ISP is limiting the speed. I'll check how (and if) this can be workarounded. Also, thanks for the proposal - it's worth of posting in wishlist forum
-
Hi unlimitedbacon,
BTSync access to file system only using OS API. It is not a kernel-space application, so it is very unlikely that it damages filesystem unless OS does that.
I see that F2FS is relatively fresh file system and it keeps update by it's creator. According to Wiki the latest unstable release was in Feb 2013 and stable is yet to be done. It also depends on which kernel version do you use in your Raspbmc, as there are still commits related to F2FS.
The only role I see BTSync can do here is to read and write data extensively.
I can advise you to set up your Raspbmc using traditional ext file system instead of F2FS and see if issue persists.
-
proactiveservices,
As it fits the topic name, join the thread. Could you please share more details? Devices, OSes, versions of BTSync, type of data synced (at least - avg size and amount of files)?
Only one suggestion I can make without having info above - try turning on "lan_use_tcp" advanced setting to force using TCP instead of UDP in your LAN.
-
sebknzl,
I'll get back to you as soon as I'm done with log analysis.
-
Johan,
This limitation is more a bug, than limitation. We want to fix it as technically there should be no limit for file size (well, unless there is some filesystem which supports it).
The product is in Beta phase now, so I cannot claim for any ETA now, sorry.
-
Hi Watchout,
Actually it hardly depends how iTunes work. I would avoid syncing iTunes data by two reasons:
a) iTunes might load earlier than your user (and BTSync does). In this case if it modifies it's files - they would be fresher than ones you are trying to sync
Some of iTunes files might be bound to hardware (HDD id, custom paths, etc.) - so I'm really not sure that iTunes files are going to work if they are moved either manually or by BTSync.hernangeles,
I tried syncing music to "Auto add to iTunes" directory and it worked perfectly. However, there is one little peculiarity: when iTunes can't add something (due to unsupported format), it moves file to "Not Added" folder instead. I can guess that iTunes tries to add file before BTSync complete the download with .synctemp extension - and moves to "Not Added" folder.
Can you please check the "Not Added"? If this is not your case - can you describe precisely how it is not working?
Thanks.
-
Hi cfrancesca,
Indeed, the complexity of working with documents and sharing them between apps grows from the iOS architecture, which forces every app to run in their private environment and restricts access to other apps environments.
In general, there is an ability to "Open in..." document in some text editor, edit and then save it back to BTSync. To do this, your text editor must support "Open In..." functionality. For example, Safari, default browser, can easily send document to any app installed in iOS which accepts documents - including BTSync. So, it depends on your text editor if it has an ability to send documents to other apps, or not.
-
Hi stlBilek,
Is the app which edits these frequently-changed files still running? I guess it can keep them open exclusively and restricts other apps (BTSync) from reading.
The fact that they are getting in .SyncArchive is really strange, it is like someone is deleting these files on other peers... Anyway, full debug logs can shed some light on this mystery.
-
Hi pruppert,
No, mobile app does not allow same settings as desktop apps. It has default settings (pretty much the same default settings as on PC) which can't be changed. They are not synced together with data.
-
Hi lynnard,
We never encountered such problems on Android. I can suggest that issue covers within Android runtime for Blackberry.
-
Hi s24joe,
Most likely you have issues with write permissions on your RPI device. I suggest checking the user which runs btsync process and sync folder permissions.
If it does not help - Can I ask you to collect debug logs on both computers and drop to me for analysis?
-
Hi AndyIbanez,
If you delete your files on a remote server, it will be deleted on other peers as well, including read-only peers. If you want to make sure that Sync is not locking any of your files - you can shut down the Sync. When running - it has to open your files and re-check them for changes periodically. Also, you can reduce this period of checking on your read-only peer by adjusting folder_rescan_interval parameter in advanced settings.
-
Hi Thorvald,
- BTSync is designed to sync your existing files without copying them to some special folder. You can add the folder and all the content will be synced with conforming directory tree.
- BTSync does not support users and user groups, however it supports 2 level of access to data you sharing: read-write and read-only. So, some of peers you are syncing data with can get a read-only key from you and will not be able to affect data on other peers.
- BTSync is free.
- BTSync uses external Bittorrent tracker server for peers visibility and relay server for data exchange in cases where direct connection is not possible. If direct connection is possible, the data flows directly between peers.
- You need to install client on all peers that have folders to be synced.
-
Hi hawibtsync,
We are working hard to make BTSync better, more stable, user and hardware friendly. Thank you for your feedback.
-
Hi Tony,
We want to add control over interfaces to BTSync in future. I suggest putting your request in Wishlist as well.
-
elephanthunter,
There is no way to get the secret in mobile UI. The secret which can be displayed as QR code is a one-time secret for transferring particular files, not for permanent sync folder.
Thanks for posting your feature request, we'll consider it in future releases. I also suggest putting it into wishlist topic.
-
vasaka,
This is a known issue which we would like to resolve in 1.3 upcoming release. For now recommendation would be - don't rename files changing only case.
-
Automatic Coding,
I suggest putting your idea into wishlist, which can be used as a new features pool.
-
antol111,
I could agree on calculation issue if we are talking about low-performance CPUs, but if you are using desktop PCs with x86/x64 architecture - calculating hashes and signing wouldn't be an issue, the bottleneck would be the communication channel.
There is another way to make multiple connections between peers rather than adding more peers. Could you please try to make a folder containing large amount of files (enough to make a speed measurement and a graph) sized from 1 to 2Mb? In this case both peers would transfer files simultaneously using several connections.
Thanks!
-
deoren,
If you are logged in as a different user and no processes are running under user account - then you are right, no big amount of handles should be open to user's profile. And even if you do log in under user - 30k files seems too much to be locked, only in the case if some folder in user profile is completely unaccessible for the user... which is unlikely (unless done intentionally).
I would appreciate if you drop a short note with ticket ID here or to my PM when you send logs to sync app@bittorrent.
Thanks!
-
Dear Johan,
BTSync could not sync such a huge files. Current limitation for a single file size is around 1Tb.
-
Hi jwinter,
Thanks a lot for logs. I'll come up with results analysis when i'm done.
Some of iTunes files might be bound to hardware (HDD id, custom paths, etc.) - so I'm really not sure that iTunes files are going to work if they are moved either manually or by BTSync.
Latest 1.2.82 Build
in Sync General Discussion
Posted
Dear forum,
As @vadimt mentioned, 1.2.87 contains no functional changes from 1.2.82, only minor UI changes during install time. Please treat it as if it is 1.2.82.
We are publishing the change log for a new version when it contains functional changes - and we'll do in future.