Problem with syncing between Amazon EC2 servers

[Pij Mleko]

Hi

I've been using btsync for a while with mac,linux and pc clients. Today I notices an issue with my two ec2 servers. They don't sync between each other. When I connect the third, external btsync client (mac), files are synced via this client, but never directly between ec2 instances (which are running in the same region/subnetwork).

I run ver 1.1.70, both on port 30002 which has been opened for UDP. Both instances have elastic IP assigned. I can successfully telnet from one to the other using elastic IP and this port.

Obviously I tried "predefined hosts" feature, disabled tracker/relay/upnp.

In debug log I see that server with predefined host specified tries to "ping X.X.X.X:30002 directly" but after that nothing happens, no error message or anything else. There's no sign of incoming connection in the target host log either.

as I mention, this is happening only between ec2 instances. they both sync without any problems with any other btsync client.

any ideas?

thanks

[Pij Mleko]

solved

I created second instance based on image of the first instance. this included also ".sync" folder with all shite inside. After removing this dir and adding back all folders to sync, two nodes see each other and sync perfectly.

There's absolutely no clue in logs, but I suspect that ".sync" folder contains some sort of fingerprint of btsync node and because both servers used exactly the same fingerprints, it was causing some issues.

so, the lesson is - after cloning an instance, always remove .sync!

[stiffarm]

I just want to understand what happened. You pre-seeded a second EC2 server with the contents of the original server? That is, the second server contained everything in the BTS shared folder(s) that the original did.

This might be a use case that I will run across. Is there any reason to believe that removing the .sync folder while BTS is up and running might get the sync done? Or is it the case that the .sync folder needs to be removed prior to launching BTS?

I hope I understood your issue.

[goli]

I guess it's because the .sync folder contains kind of the "self identity" of a btsync instance. If you clone a computer that contains a configured btsync, both machines might think they are the same. As soon as one computer connects to the other one, both believe they're talking to wired port forwarding of themselves.

I don't know why btsync restricted this self-synchronization thing, but I guess it's the source of all evil if those become looping recusively.

But I don't know what's going to happen if two not-empty folders start merging for the very first time. The thing is: What's the parameter both determine that both are sources? Could be that both one is a source and the other one is a target, based on the filemtime of the source folder. Then one could start to delete files the other node cannot provide. That's something you should try a couple of times in test environments before starting to sync life data.

Kind regards,

Stephan.

[Pij Mleko]

what I did initially:

1) created an ec2 instance, installed btsync, configured listening port, opened it in security policy etc. I validated that it works with btsync installed on my laptop. All files were synced to ec2 (and back)

2) I created an image of this ec2 instance with btsync already installed

3) used this image to create second ec2 instance (btsync was cloned with all files from 1st instance, including initialized .sync folder)

after that, both instances were syncing properly with my local btsync, but they never talked directly to each other, even when I set 'preferred hosts'. files were transferred always via my laptop.

when I switched off local btsync, no syncing was happening. no errors in logs either. I was removing/adding folders back, changing ports, nothing worked.

however, when I stopped btsync on the second instance, removed .sync folder, started btsync and added back all folders, suddenly everything works fine.

I guess this issue will be very easy to reproduce. I've done it couple of times today, always with the same result (luckily the last time I created a fresh instance and that gave me a hint that something smelly is going on with pre-installed btsync)

thanks

[stiffarm]

But I don't know what's going to happen if two not-empty folders start merging for the very first time.

Good point. I think it's something to avoid.