goli

Hey there. Btsync is no file system driver. Never was, and most likely never will be. It just works ontop of a given file system and deals with local files. So, what's the process of keeping a local change history if you are not a file system yourselfe? I have a local file. Then I open this file with, say, excel, and hit the "save" button. Excel persists local changes, the file gets changed. Now, how should btsync know about the previous version of the file as soon as the current one is in place? Excel doesn't tell btsync "there's something going on with the file, I'll write in 500ms!" which would enable btsync to copy the last version to its history. There are two options: Btsync keeps a static copy of the current versions of each and every file. This is the only way to have a standalone application be aware of the history. And that's pretty much what e.g. GIT does: Keeping its own ".git" directory that, dealing with binary files, usually takes the amount of storage space necessary for the source files in addition. This can be seen as a local backup. So this would double the necessary disk space instantly even if not a single file has ever been modified. Just twice the disk space needed for keeping one untouched copy of each file. Btsync could ask other nodes what's their current version. That's what currently is going to happen just to upload the modified file. Btsync could, prior to copying the modified file to other nodes, fetch the last version from the remote nodes. To sum this up: Btsync could synchronize the history, too. The first thig is clearly awfull. There's 20GB in my music folder. Most of the stuff never changes. I activate history just to make sure that no wrong timed node trashes the library. This currently creates ~30MB of history files because of modified ID3 tags. Using a local history based on a untouched backup folder here would instantly increase the disk space from 20GB to 40GB. The second thing, on the first sight, is awfull too. It creates unnecessary traffic. But on the second side, this would only double the traffic. Based on the fact that btsync uses block based synchronization and usually doesn't copy the whole file, I guess I could live with that, if I would want local history. Compared to Dropbox: There's no local history of dropbox files here, is it? The only history you have is the remote one. The reason btsync uses a local history isn't to have a local history. It's because in a serverless environment the only storage you have is the local one. But: To be honest, if I would want local history stuff, I would use a dedicated tool for that. There's no reason for btsync to do this, too. The remote history is a nice thing because you never know which of your participants manipulates your files. So the btsync remote history just keeps track of anwanted changes of remote nodes that are not under your control but under the control of other people. What about using GIT? I think there are GUI or GUI-less tools that do local versioning. I'm thinking of Sparkleshare, currently. I don't know how far they have come recently, but that would be my best bet. Regards, Stephan.

I guess it's because the .sync folder contains kind of the "self identity" of a btsync instance. If you clone a computer that contains a configured btsync, both machines might think they are the same. As soon as one computer connects to the other one, both believe they're talking to wired port forwarding of themselves. I don't know why btsync restricted this self-synchronization thing, but I guess it's the source of all evil if those become looping recusively. But I don't know what's going to happen if two not-empty folders start merging for the very first time. The thing is: What's the parameter both determine that both are sources? Could be that both one is a source and the other one is a target, based on the filemtime of the source folder. Then one could start to delete files the other node cannot provide. That's something you should try a couple of times in test environments before starting to sync life data. Kind regards, Stephan.

Well, you're becoming off topic. But to give a quick response ... I don't want to repeate myselfe, especially because this thread is going hell off topic. You might want to participate here? Regards, Stephan.

Hey there. Since you didn't resond with "yes, i turned if off" I would assume you have DHT and/or Tracker Server on. By default, Tracker Server is on and DHT is off. You ask why your local computers would need such a thing in your private network? Well, what in your opinion should have told your computers that there are no others than your private ones? Defaults are: There is another world besides your local network. Btsync isn't directly created as LAN synchronization tool but for a more general purpose. From this perspective it makes much more sense to have the Tracker Server thing on by default, don't you think? If you don't want that: Turn it off. Nothing easier than that. It's a per-share configuration. You can (or must) decide per share, if your computer should use Tracker Server and DHT. Just like you can (or must) decide per share if your computer should use "Search on LAN" (broadcasts) or known hosts. So rightclick on your share, Preferences, here you are. And remember: You don't configure the share in a global perspective, you configure the behavior of your very personal node dealing with this share. So you nee to do it on all of your computers. If you still discover unknown network traffic: This could be the auto update notification feature. You need to turn this off, too, in order to have no external connections going on. That's a per-device setting, not a per-share setting. But you're not the first one asking those questions, using search would have helped you, too. Regards, Stephan.

Hey there. What are "bad time stamps"? Are they 0 (1.1.1970) or in another way "bad"? If you have one master machine that chould be used as an ultimate source, you could simply "touch" them to give them the current time. That isn't really valid, semanticaly, but it's working. But I don't understand what's the deal with those files. It shouldn't matter which exact time stamp is in place. Btsync should handle those files propperly and create them with the exact same modification time on the targets. Then they should stay in sync as long as a single machine changes its local copy. Then the modification time gets adjusted and the transfer starts. So: What's the exact sitaution you're trying to fix? Regards, Stephan.

Hey there. To ChrisH: I was just about to answer the exact same thing. But I would expect btsync to show all 8 files and have the open ones just neer be finished. So this might be a different problem. To the problem: Maybe permissions? I really don't know which permission an exchange instance sets on its files, but I hope it completely refuses read access from everybody that isn't an "Administrator" role. Btsync has this debugging feature. It's something like "FFFF" in an "debug.txt" file inside of the .Sync folder, I think. See this sticky thread: Regards, Stephan.

Hey there. I'm a little split minded. On one side I agree, and on the other I really dislike this. You're suggesting one thing: Btsync could have a deep knowledge about different file formats, could know different types of common changes on those and therefore could be able to use more effitient diffing algorithms based on the given file type. But there is one thing I am in fear of: This is like reprogramming storage engines of other programms inside of btsync, which directly leads to some "magic". It's not that btsync couldn't only change a view bits on a file if necessary. But there are file formats where kind of randomnes is part of the algorithm of persisting data. On those formats, if you save a single file twice, the resulting files do have different hashes and so are different. This situation has clearly to be avoided. And since this can easily become an over and over discussion about which file formats are important and easily to track, I would suggest to not implement a single of them but make kind of a plugin concept which allows individual differs/resamplers for different file types. So whoever wants his very own file type synchronized in a very individual and performant way is free to publis a diffing pluign for that. I have no resource here to quote, but I remember a thread where Bittorrent staff talked about blockwise synchronization somewhere. I don't know the exact block size, but I guess it's like 4MB or something. So whatever you do in a file, as long as your concurrent changes in the bitstream of a file do not exceed 4MB, you should never synchronize more than 8MB per file (which is two blocks). Usually you only need to synchronize 4MB. Might be a nice thing to adjust this block size, but currently I only know for "max_file_size_diff_for_patching" parameter, which is the total file size a file should not exceed to be subject to partial synchronization. You should move your comment to the wishlist thread, btw. That's the place for feature requests. Regards, Stephan.

Hey there. You're talking about things that are way more than just a couple of more options. One clearly can discuss this over and over and will always find something else to be added that's file system related. Just to give some notes: Currently there is now privilege support. Not chown, not chmod. You want to introduce that? Sure, do su. Think about ACLs. That's a little more than just this and clearly is a valid use case. So add this, too. Switch to Windows, where you have way different access priviligation. Would be just fair to have those, too. Now think about Windows in AD environments. At the first sight, that's just like usual Windows priviliges. But it shows that you should not rely on local computer or account names (human readable stuff) but on UUIDs in that case instead. So it's different from synchronizing those privileges to ADless environments, unfortunately. You talked about Resource Forks. Ok, got that. NTFS has "Alternative Data Streams" here, which can be compared to solve a similar problem, but it's cleary implemented differently. And luckily it isn't widely used. Now think about FAT, which happens to be the default file system on e.g. androids. There's non of the mentioned previliges available, access rights are enforced by mount parameters only. And shares being placed on SMB or NFS. The privilege configuration doesn't necessarily match the underlying file systems possibilities, does it? I find it quite likely to have a cheap 2TB NAS mounted through NFS wich doesn't support btsync itselfe. So I would run the btsync on my computer and do the synchronization remote. I woud suggest to not share those kind of data. Not a single one. No Resource Fork, no Alternative Data Stream, no Privilege. Just plain file name and binary content. On the second step, one could define "comparable classes". Those could be no privilege support simple privilege support (chmod/chown) extended privilege support (Linux ACLs) NTFS HFS+ And this list is to be continued, of course. This should be a basic configuration parameter for each share, and as soon as a node has one of those options (exclusively selectable, so no combination of those possible) selected, it should simply refuse to share data with nodes that are configured in a different way. The thing is: As soon as you do other magic, it's likely to connect a non-capable node and corrupt your data. So I really would like to see none of those features being activated by default. Kind regards, Stephan.

Hi Roza. The way to create vhosts differs a little between different linux distributions, and is even another thing on windows. On debian based distributions, there is a /etc/apache2/sites-avialble folder where each vhost should have its very own vhost file. Then for each file there is a symlink from /etc/apache2/sites-available/* to /etc/apache2/sites-enabled/* which allows you to enable and disable vhosts on demand without requiring to completely delete them. The vhost itself has a "ServerName", which is the primary domain name the vhost responds to, as well as many ServerAlias values that are other domain names the vhost responds to, too. You need to also enable the enable "NameVirtualHost" for both, port 80 and port 443. Debian based systems use to have it only enabled on *:80 but not on *:443. So you need to adjust this. I just googled "NameVirtualHost" and found this very link containing the basics: https://www.linode.com/wiki/index.php/Configure_apache_to_use_virtual_hosts_on_ubuntu_server But again: It very much depends on the distribution you use. Regards, Stephan.

Assuming that btsync uses the number of repeated requests of the same idea to make an ordrered list of new feautures, you should go to the wishlist thread and give your +1 there.

Hey there. Without any profound information and investigation, I would guess that it's broadcasting your're upset on. The btsync "Search LAN" feature completely relies on default broadcast. For a simple explanation (even though it's not that simple in all cases), it's like this: If your local network has 192.168.0.0/24, that means that 192.168.0.1 is the very first available IP in this range, 192.168.0.253 the very last. The number 192.168.0.0 is your network name and not available as IP address. The number 192.168.0.254 isn't available as IP address since that's the broadcast address. Broadcast addresses are defined as "everybody can address this, and everybody is allowed to respond". And ontop of this, boradcasts don't get routed, by default. Broadcast requests are bound to the very local wire. I guess you have to investigate how to allow OpenVPN to route broadcasts, then you'll be fine. Or you should use the known hosts feature and assign the VPN IPs here. This way is ost likely the easier one. Regards, Stephan.

Hey there. What about adding those files starting with "~" to the .SyncIgnore file? Should do the trick, imho. Kind regards, Stephan.

Hey there. While browsing the new "Sync Hacks" forum, an idea came to my mind. If it can be done, would be pretty awesome. The thing is: Provide a binary which runs on VMware ESXi 5.x environments. I currently run a single VMware ESXi (free edition) hosting a couple of VMs. Using a NAS here is way to expensive in terms of providing enough IOPS, so I stick to an internal RAID controller wich does RAID10 with 4 SATA-HDDs. I know, it's kind of a playground environment and far away from supported configuration, but that's not as unusual for geeky guys . It would be pretty awesome if I could make the ESX do regular backups automatically to any cheap 2TB NAS for $100 just by btsync. My current backup setup is driven by ESXs interl exporting feature "copy virtual hdd to attached NFS storage". But that's one of to major disadvantages. I can either convert the virtual HDDs from regular to "2tb split", which is nice in terms of backup space usage but completely eliminates any offsite synchronization due to internal block shifts. Or I could backup the files "as they are", which would allow for rsync-like offsite mirrors but needs an awefull lot of storage space because the NFS way blows the spares away. So the thing is: btsync here would be just great for both, backupping to locally attached storage and offsite backups. Currently the linux builds die teling me "terminate called after throwing an instance of 'std::length_error' what(): basic_string::assign". Kind regards, Stephan.

Hey roza. If you already run a webserver doing HTTPS, you could use a reverse proxy configuration. Just in case you have an apache up and running, this should do the trick as a fresh new vhost: ProxyPass / http://127.0.0.1:8888/ ProxyPassReverse / http://127.0.0.1:8888/ <Proxy *> Allow from all </Proxy> Ontop of this, you can do almost everything mod_access provides. So feel free to add little more restrictive policy than "allow from all" . I use such a thing to almost every feature any of my servers provides by both, HTTP and HTTPs. There are two reasons for this: With this configuration, I can use almost every webinterface bound to default port 80 and 443 and just destinguish them by domain names. Very convenient, because I'm more likely to remember domain names than port numbers. The webserver doesn't expose any product specific signatures unless you found the matching vhost. Noone can just enter ":8888" and guess, based on the responses content, which service might be running and maybe open for attacks. It's realy easy to have widely proved HTTPS implementation protecting the stream, instead of having a service that uses a HTTPS implementation of its own and I don't know its codebase. Having apache here in place for both, authentication and HTTPS makes me sleep slightly better. Regards, Stephan.

As far as I know, btsync currently is completely unaware of what type of connection is between two nodes. So if you limit the upload or download to a certain bandwidth, that should be the case for all connectivity, no matter if it's your 1GBit LAN link or your 512kBit WAN link.

Hey there. I guess that's not possible because the fileAccessTime isn't stored, at least when using one of the ROMs I tested. The only parameters you have are fileModificationTme and fileCreationTime. But those don't help you. If you feel like using fileModificationTime is for you, then you can use any "cron" app of your desire. Search the google play store, there are several of them. Simply write a short "find" which deletes old files and run it every couple of hours. find /data/sdcard/btsync/share -type f -mtime +3 -exec rm {} \; But since write actions of sd-cards is said do be limited (to be honest, I haven't seen a single dying ever, but ...), I would suggest to not go this way. Ontop of that: This might be something for the Mobile Wishlist thread? Maybe you can figure out kind of a process/workflow which makes btsync aware of active/used files and internally handles storage management? I tried to explain such a thing in my last post in the Mobile Wishlist thread, it's related to mp3 files, or media in general. This might be achievable becaus the Android media library is the very central API more or less every media app (images, sounds, videos) uses. But you're talking about file types that aren't covered by the Android media library, things might become difficult. I know that my post isn't exactly what you're trying to do, but if you have a better idea, add some comments there please . Regards, Stephan.

Hey there. There is this thread: The cheapest one I know is something like $35 for an empty box (zyxel, I think) which can easily hold a 2TB hdd for another $50. You can go cheaper if you use smaller HDDs, but then the price per GB raises. Kind regards, Stephan.

Hey there. On Windows 7 it doesn't. Do you see btsync to stop indexing and then restart? If not, if it's indexing constantly and starts as soon as you start btsync, then this might be the necessary initial indexing run. Think about USB drives hosting btsync storage or dualboot setups. If btsync starts, there's no way to ensure the storage folder is exactly the same since it has been indexed. Regards, Stephan.

Hmm. That's some very special and detailed thing. So I guess we need some staff here that answers protocol questions. Or refuses to answer those in the current state of the software. But this most likely isn't publicly available information. I don't think that nodes do provide other nodes with node lists. I haven't seen a single node doing so. A small setup that would show this if it would be implemented: completely disabled tracker servers and LAN discovery, only known hosts network A contains an Ubuntu node having a private IP and static port-forwarding and a Windows 7 node without static port-forwarding but wth enabled UPNP. network B contains two Windows 7 nodes with disabled UPNP but with enabled LAN discovery All three Windows 7 nodes know the Ubuntu node as "known host" (the public IP/port which is forwareded) All three WIndows 7 nodes see the Ubuntu node The two Windows 7 nodes on network B know each other, most likely through LAN discovery The Windows 7 node on network A is only connected to the Ubuntu node If they would exchange node lists, all three Windows 7 nodes would see eachother. But they don't. So they most likely don't exchange node lists. At least the network A Windows 7 should be able to connect to the network B Windows 7 nodes, because that direction has the very same public IP route. Maybe you had added them tho tracker servers for a very short time? Even if you disable tracker server, that only prevents your nodes from renewing the tracker server data. It doesn't delete any information that already are there. And back to the "read-only servers don't act as seeds" thing: I really think this isn't implemented and possible currently. And if you did see this in previouse versions, I guess that was kind of a bug. Not the current state. Regards, Stephan.

Hey there. I suspect that this will be ever implemented. Think about syncing from windows to linux. Or think about using android, which is linux based, but must likely has FAT32 sd-card storage where no permissions can be applied to. And think about running btsync in a restricted environment where the btsync user cannot change the permissions and ownership of files and folders. And there's another one. What about synchronizing two hosts that don't share a common user/group list? In my office, all users of all of my computers are based on Active Directory. If I connect my private notebook to my companies network, which is the case when I did huge music library cleanups at home and don't want to upload each and everything to the very place I do visit personally every day, there's no single common user name on those systems. And a fourth thing: Huuu, security. Even if it's unlikely and requires additional interaction afterwards: I really don't want one of my computers to create files having execution flag set on another computer. And since I don't want one of my computers to create files owned by root on another computer, all of my nodes do run btsync as a dedicated btsync user. Synchronizing permissions and ownership always is a critical thing. The most likely situation on linux is additional config paramters that allow to select user/group names and flags. Regards, Stephan.

Hey there. Are you sure this has worked before? Ok, having 165 setups tells me you might have noticed this earlier. But when I used 1.0.x I tried to setup kind of a backup node which only gets a read-only secret. As its task as backup only node this did work, but the backup only node wasn't able to provie the other nodes with changed data, even if the changes weren't introduced by the backup node but by another node which did have a write key. So pretty much the same thing you discovered: Read-only nodes can not distribute any files, even not for relay purpose. If I'm wrong, I would be glad to know because that currently is the only thing which is not pretty nice. But to discover what happens: Maybe the read-only nodes don't connect to any tracker as providers and do not send out any broadcasts? What if you add one read-only node as known host to another read-only node? Still the same? That's the very sport where I stopped thinking about syncing with read-only nodes and just provide write secrets, back then . Regards, Stephan.

Hey there. My current one is: Word and Excel Simultaneous Editing The most information google provides you with is marketing stuff for Microsoft SkyDrive. But I know for sure that this feature also works for Word and Excel documents being accessed on a common SMB share. So there needs to be some other synchronization protocol which isn't based on SkyDrive. I guess it's based on meta information in the original files or in additional "shared files", but that's just a wild guess. The thing is: Maybe there's a plugin API in Word an Excel which allows btsync to hook in and use btsync here instead of SkyDrive. If two computers are connected through btsync (read: Two computers are up and running at the same time and share the same secret), Word and Excel could use btsync as transport channel for the Co-Authoring feature. Using the regular btsync file synchronization mechanism would not suffice as soon as there are a certain number of co-authors editing files. Imho the Co-Authoring synchronization needs to be done a little faster than usual btsync file exchange. And ontop of that, there might be no need to reindex those changed word files as BLOBs when the required information is presented as structured information by a nice API. Being protocol aware here, you know? Yea, that's my very first real business use case Regards, Stephan.

You might go to the wishlist thread and as for a bug tracker. Both wishlist threads, the regular one and the mobile one also allow for things that are only related to the apps. There's some "please provide .apk files for download for those who do not have Google Play Store" post that got replied to positively. So asking for a bug tracker does have a real chance to at least get an answer, whatever the answer may look like. What does Dropbox do in this case? Is if fast enough or does it behave just the same way? I would suggest not to work that way around. I would drop a little NAS right in the middle of the office that does both, exposing its storage as SMB and providing your network with a LAN/broadcast btsync for that particular folder. Then your works can just mount the SMB mount and work right in there. If you workers need to have a local copy for offsite work, then they can add the btsync folder to their local sync, too. But when being at the office, they should go for plain SMB. This scenario would allow you to use the "simultaneous editing" functionality of Word and Excel. And another thing for the wishlist thread: Maybe there is an API for that wich makes btsync aware of the "simultaneouse editing" thing of Word and can interact with it. That would be quite aweson. Since I'm pretty excited abut this feature, I will go to the wishlist thread and post it there. Kind regards, Stephan.

Is this related to btsync or related to androids file system? I'm running Cyanogenmod 10.1 on a Samsung Galaxy S2. It has 8GB of internal storage and I added 64GB as additional storage. Both cards are formated as fat32, which limits the file size. I've been there when I tried to add and mount truecrypt files, completely unrelated to synchronization. This, after all, was one of the reasons for me to use EncFS instead of Truecrypt on my phone: The file size limit. But of course that needs to be fixed in btsync if synchronization kees the phone bussy although there's no way to finish the file. Regards, Stephan.

Hey there. Let me extend your use case a little bit. While thinking about it, I gues that your appoach is based on the currently available mechanisms of btsync. Let's think about having completely different synchronization mechanisms for music. Using music as regular share can be done, too. But I really like the idea of having music handled separately, in a way that isn't driven by requiring the user to do file system operations. Here's what I'm thinking about: My first idea is to have some kind of music streaming support. MP3 files usually have something between 128kBit/s and 256kBit/s, so it's quite feasible for regular household network connections to provide the required bandwith. If there are multiple uploaders, that should be no problem at all. Just like "music on demand". That of course requires at least a single up and running host which provides you with music. But that's doable, I think. And this requires a constant internet connection. But since there are several music streaming platforms out there, this should be no problem, too. My second idea is to deeply couple the btsync mobile app to the phones music player in terms of scanning the current playlist and interact with it. Maybe it's possible to provide the phones music player with a cusotm playlist, or maybe provide an integrated music player. Whatever is going to be realized, I'm thinking about having a playlist and knowning the exact current pointer of the song that is currently played. The btsync app could make sure the last x played songs are keppt, songs that appear earlier in the playlist are deleted automatically. Songs that are going to be played in the future according to the playlist are autoatically downloaded. Let's say automatically download the next 100 songs. This can be conigured by a second "use WiFi only" switch and adjustable maxAge and lookAhead parameters. Both parameters can be used as both, a number of music files or an amount of file sizes. I realy don't know if one of my ideas can be implemented, since this is both, a huge amount of work and complicated to interact with system API stuff. But both ideas would be cool stuff on my mobile phone. Kind regards, Stephan.