rdebath
-
Posts
289 -
Joined
-
Last visited
-
Days Won
6
Posts posted by rdebath
-
-
I just checked the packet sent to the tracker; it's in bencode and the "la" is a local address which matches the internal IP and port for my client.
la = $(6,"c0a88b0c98d0")
m = 4
peer = $(20,"91aaffb2463df40c6ca21a281470ec915e31b2f7")
share = <<<
: 00 87 10 4f 34 0d da 84 95 8c a9 01 4f 49 ad d9 ...O4.......OI..
: 60 c2 e7 69 f2 c4 b4 e8 f8 c1 67 f9 81 3c 81 a2 `..i......g..<..
>>>So you're right, the tracker does know the internal address and so can aid in your setup. It seems the reason it didn't work for me was that it only has one internal address, the NAT address, but not the internal VPN address.
-
How could it be a firewall issue if computer inside the same domain controller can see each other?
The firewalls are routing traffic between the sites, presumably over a VPN. But, as I said in the linked message, peers on a local lan find each other by using multicast packets. It's very unlikely that you've got the multicast routing right for the peers to see each other between sites over your VPN. Therefore they will have to communicate over the internet by hole punching through your firewalls. As this isn't working it's a firewall issue. (Probably the firewall is blocking all UDP traffic)
If you add the peers as known hosts through the VPN tunnel they'll probably be able to sync with each other that way.
Note: BTSync does not register the internal IP addresses with the tracker so even if they can both communicate with the tracker the data will not be routed through the VPN.
Note: it could also be the Windows firewall on the PC as this does know the difference between local lan and everywhere else. This is unlikely though because the BTSync installer adds the proper rules to the windows firewall on the PC it's installed on.
PS: Windows domain controllers have absolutely nothing to do with it.
-
It creates two million files THEN DELETES THEM, once the script completes there are no extra files in the share. But the extra memory usage will stay, even after a restart of BTSync.If I read the script correctly it creates 1M files. Sync can consume 2Gb for 1M files, but not for 300 files.
But, if you don't like that one, how about this more sneaky version; it only creates a thousand files so you'll have to leave it running for a few hours (lets say a day).
#!/bin/bash -
B=/home/btsync/zero/memuse
P=$B/XXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXX
mkdir -p "$P"
touch "$P"/0
rm -rf "$P"/*
f=`date +%Y%m%d%H%M%S`
mkdir -p "$P/$f"
for j in \
00 01 02 03 04 05 06 07 08 09 \
; do
for i in \
00 01 02 03 04 05 06 07 08 09 \
10 11 12 13 14 15 16 17 18 19 \
20 21 22 23 24 25 26 27 28 29 \
30 31 32 33 34 35 36 37 38 39 \
40 41 42 43 44 45 46 47 48 49 \
50 51 52 53 54 55 56 57 58 59 \
60 61 62 63 64 65 66 67 68 69 \
70 71 72 73 74 75 76 77 78 79 \
80 81 82 83 84 85 86 87 88 89 \
90 91 92 93 94 95 96 97 98 99 \
; do :> "$P/$f/$j$i"
done
done
while :
do f=`date +%Y%m%d%H%M%S`
mv "$P"/* "$P/$f"
sleep 120
done -
Did I misunderstand this last, is there a 32 and 64-bit version of the SyncApp?
On Linux there are. The Windows one is currently 32bit only and really there shouldn't be any need for a 64bit version on any platform. (From the amount of memory that is.)
-
Can you please give us more details? Sync can't allocate that amount of memory for 300 files.
Startup BT, run the script below on a share it's watching.
The share can be empty, that's not what matters.
I don't think this one will get you to 5Gb, BTsync gets too slow, but it's the right idea.
#!/bin/bash -
B=/home/btsync/zero/memuse
P=$B/XXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXX
mkdir -p "$P"
for x in A B ; do
f=`date +%Y%m%d%H%M%S`
mkdir -p "$P/$f"
echo -ne "\r \rFile: $f :"
for k in \
00 01 02 03 04 05 06 07 08 09 \
10 11 12 13 14 15 16 17 18 19 \
20 21 22 23 24 25 26 27 28 29 \
30 31 32 33 34 35 36 37 38 39 \
40 41 42 43 44 45 46 47 48 49 \
50 51 52 53 54 55 56 57 58 59 \
60 61 62 63 64 65 66 67 68 69 \
70 71 72 73 74 75 76 77 78 79 \
80 81 82 83 84 85 86 87 88 89 \
90 91 92 93 94 95 96 97 98 99 \
; do
for j in \
00 01 02 03 04 05 06 07 08 09 \
10 11 12 13 14 15 16 17 18 19 \
20 21 22 23 24 25 26 27 28 29 \
30 31 32 33 34 35 36 37 38 39 \
40 41 42 43 44 45 46 47 48 49 \
50 51 52 53 54 55 56 57 58 59 \
60 61 62 63 64 65 66 67 68 69 \
70 71 72 73 74 75 76 77 78 79 \
80 81 82 83 84 85 86 87 88 89 \
90 91 92 93 94 95 96 97 98 99 \
; do
for i in \
00 01 02 03 04 05 06 07 08 09 \
10 11 12 13 14 15 16 17 18 19 \
20 21 22 23 24 25 26 27 28 29 \
30 31 32 33 34 35 36 37 38 39 \
40 41 42 43 44 45 46 47 48 49 \
50 51 52 53 54 55 56 57 58 59 \
60 61 62 63 64 65 66 67 68 69 \
70 71 72 73 74 75 76 77 78 79 \
80 81 82 83 84 85 86 87 88 89 \
90 91 92 93 94 95 96 97 98 99 \
; do :> "$P/$f/.$k$j$i"
done
echo -ne "\rFile: $f $k$j-- :"
done
done
done # x
echo -e "\r File: $f : Done"
echo -ne Waiting 15 minutes
sleep 900
echo -ne Removing
rm -rf "$B" -
You're being a little unclear; but I'd guess your firewall isn't correctly configured. Best guess is you've opened the TCP port on the BTsync listening port.
It still works because of the NAT punching. Ie: it didn't actually need you to open the port, it managed it for itself.
-
I think 255 bytes per path level - at least on Debian based distros.
So you can't have longer filenames than that, but each directory can also have a long name.
You're right, It's each component of the path and it depends on the filesystem, 255 for ext3/4, around 140 for encfs on ext3 etc.
Have you tried syncing a linux client and a windows client which has these "short" links or just windows to windows?
Because they do get synced between linux and windows but maybe another windows client would just say "Got it, chill".
Hmm, most of the time I've only got one Windows machine and the rest are Linux ... Though I haven't tried a UNC path to the share ∴ your
to raise. -
The windows API is limited to 255 UCS2 characters, or with Windows 7 it's limited to 255 (two byte) words encoded using UTF16.
Many Linux filesystems are limited to 255 Bytes normally encoded with UTF8 (BTSync seems to assume UTF8).
I haven't seen BTSync taking any notice of the "short" filenames in Windows; I suspect it's a bug if it does.
On windows relative paths only work in relation to the current directory; the current directory must begin with a mapped drive letter.
BTW: UNC means Universal/Uniform Naming Convention, not anything to do with unicode.
But in answer to your primary question; IMO UNC paths should work properly, if they don't raise a bug report.
-
Oh yes, Of course if there's a Windows client connected to the share it dies.
It takes a few minutes to create a million zero length files in a deep directory (maybe 5-10min), leave them for, say, 15 minutes, then delete them.
If you left them long enough for BTSync to notice a 32bit version will die; a 64bit version will kill any 32bit version that subsequently connects to the share. (Or maybe kill the machine that BTsync is running on.)
Like we've all said BTsync uses huge amounts of memory.
-
The NAS is in communication with the outside world.
The MAC is not.
Please confirm that your OpenVPN connection is NOT configured to use TCP as it appears that your company firewall is blocking all UDP.
It also appears that you have some sort of NAT on the phone connection despite the IP address.
Many small company IT departments don't seem to be able to cope with UDP, basically, it gets left at default and for many Cisco routers UDP defaults to blocked. Also this IT department appear to be blocking ICMP time exceeded messages; this is a BAD idea and frequently means the firewall admin doesn't understand what they're doing.
-
Highest I've seen is about 5.3GBytes. On a share with about 300 files.
-
You need the logs from both ends.
To date for me, BTSync connects just like OpenVPN when you turn off everything except the "predefined hosts".
It copes with port remapping firewalls (on one end) or double NAT punching when the port numbers are preserved but can't work though both double NATs and port remapping; just like OpenVPN. But the relay helps in the last case.
BTW: UDP is more difficult to scan for, a closed port will respond with an ICMP error packet but both an open and a filtered port will give nmap no response whatsoever (unless it's the echo service). That's why you have to do a traceroute, so you can get ICMP time exceeded packets from everything except the last hop.
Also BTSync does seem to make distinctions between "lan" and "other" so, what range of IPs does your work use for their internal network? 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, or something else ?
Oh, except for one last thing; OpenVPN will give up after a little while, snooze and try something different. BTSync tends to be rather aggressive about trying to punch through firewalls and NATs. This means that sometimes it will get a broken connection into a firewall's table and keep it alive even though it does nothing. So try turning both ends off for 10 minutes at the same time to clear this.
-
Actually, thinking about it, with the current version, if you want to modify settings from a windows program you are probably best connecting to the Linux webgui and sending it commands that way. That would obviously mean having all the shares on a Linux NAS machine but it's likely a lot easier than "carefully stopping" the windows app and mangling the sync.dat file. After all you have some example code for calling the webgui.
-
Not much really. The windows version doesn't respond to any command line arguments.
To actually do it you'd have to stop the BTSync client (wait for it to stop cleanly) then manually edit the sync.dat file.
That file is very complicated so you'll probably break it. I wouldn't want to try more than flipping the odd flag myself.
Something for you to add to the wishlist thread.
-
The linux binary uses "inotify" to trigger for file changes you get messages like this in the log:
[20130416 07:48:46] [OnNotifyFileChange] /home/btsync/zero/testfile.bin
It then waits maybe 15 seconds to see if there are any other changes to the same file. Once it's been quiet for a while it starts the sync on the file.
I haven't checked but I understand this is turned off in the glibc2.3 version.
I'm using "Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-40-generic i686)" on one of my machines and "it works for me".
-
You can change the webui.zip, but you must do so after BTSync has started up and before the first web client connects.
The certificate is stored in the settings.dat file. As this is a bencode format file you can probably change it (delete the ".fileguard" tag) using a normal bencode editor when btsync isn't running. The format of the key looks standard to me.
Actually, I didn't know it could even do SSL, It doesn't seem to be in the user guide, I'm using "stunnel" to encrypt the connection.
-
-
But what happens if UDP port 3000 is blocked on my company's firewall?
It doesn't work.
You could see how far a UDP traceroute gets ... http://forum.bittorr...dpost&pid=45652
NB: The "Got 2 relay IPs" is a DNS lookup, not being able to contact them.
Assuming UDP is blocked (and you can't a port or two opened) you'll probably have to wrap the UDP in a TCP/IP connection. Perhaps even in a port 443 TCP/IP connection.
But first you might try putting your NAS on port 123 or port 53. Some company firewalls let those ports through even if they block others. You'll have to add the NAS as a known peer though because you still won't be able to connect to the tracker. NB: http://www.dnsdynamic.org/ will help you find the peer if it's on a dynamic IP.
If you need to wrap the connection tools you can use include Zebedee, OpenVPN and udptunnel.
PS: That traceroute is windows. You should be able to run a traditional traceroute on a Mac.
-
Yup, absolutely right. Maths is maths.
Chance of winning the lottery ...
14000000 : 1
Chance of guessing a BTSync (160bit) key.
1461501637330902918203684832716283019655932542976 : 1
The lottery gets won every week, the BTSync key won't be hit till after the heat death of the universe ...
(or the big crunch, whichever comes first)
It ain't gonna happen man.
-
You need to have this port open for normal operation.
If you only open port 3000 all your traffic will go via the relay, this is likely a lot slower. Especially if the relay is half way round the world. Having the tracker on another continent isn't really a problem.
Remember this port is open for connections initiated from INSIDE, not for connections initiated from outside. So a blacklist is normally considered good enough control.
-
2. periodically restart daemon to release memory if number of files was decreased
It's correct?
Unfortunately no, currently the zombies breed.
Items are stored in the sync.dat file that record every file that has been deleted. Currently, these records are not cleaned out of the sync.dat file. The only way to remove them is to remove the share and start from clean. Furthermore, this must be done on ALL the peers at the same time because the deletion records in the sync.dat file are synchronised between peers. So if you reset one peer all the deletion records it had will be recreated from the other peers.
-
Okay, about minimum firewall requirements...
The relay and the tracker are found using DNS, these are the current settings for relay and tracker.
Note the TTL less than 5 minutes in both cases, this is the warning they need to give to change these.
;; ANSWER SECTION:
r.usyncapp.com. 203 IN A 67.215.231.242
r.usyncapp.com. 203 IN A 67.215.229.106
;; ANSWER SECTION:
t.usyncapp.com. 258 IN A 54.225.100.8
t.usyncapp.com. 258 IN A 54.225.196.38
t.usyncapp.com. 258 IN A 54.225.92.50
Normal operation is using the tracker to find peers and using direct connections between the peers to transfer data. All data is transferred using UDP packets.
Your BTSync has a port configured, say 20001.
The Peer has a port configured, say 20002.
The tracker has port 3000 configured.
The relay has port 3000 configured.
Requirements are:
- Unsolicited packets must be able to travel from your port 20001 to the tracker on port 3000.
- Solicited replies from the tracker on port 3000 to your port 20001 are required.
- Unsolicited packets must be able to traverse your firewall from your port 20001 to Peer's port 20002
- Solicited replies from Peer on port 20002 to your port 20001 are required.
- The public port that the firewall presents must be the same as the BTSync configured port. If your firewall renumbers ports unpredictably only the relay server can be used.
This assumes your firewall uses the normal 'timeout' method of noticing solicited responses. The problem is that the firewall will not see the request for the first response as it travels via the tracker. It must not do anything "unfortunate" when it sees this "response".
If your firewall is broken in this way then unsolicited packets must be accepted both ways between UDP ports 20001 and 20002.
If your firewall cannot be fixed connections to the relay must be opened for all peers that need to communicate with you.
If you wish to use DHT you must accept unsolicited packets on your port 20001 from any address.
If you configure known peers you can turn off access to the tracker; no packets then need to go to the tracker (See http://forum.bittorrent.com/index.php?app=forums&module=forums§ion=findpost&pid=49550 ).
- Unsolicited packets must be able to travel from your port 20001 to the tracker on port 3000.
-
Ignoring the *.!sync files won't help (I expect they're already ignored anyway like the .SyncID file)
You'll only run into trouble with them if the file transfers overlap. So you have two remote peers trying to send data to what the local BT sees as two independent *.!sync files but are physically the same file. You won't end up with the earlier or the later. If you're lucky BTSync will do a final hash check on the *.!sync file, notice it doesn't match and try again. If you're unlucky it'll assume the *.!sync file is correct because it only ever wrote correct data to the file ... except it didn't.
This will only happen with large valuable files
-
Actually, it seems more related to the total number of files you have ever sync'ed.
Once a file's name gets into the sync.dat file (and so into memory) it never leaves, so as an ultimate endpoint the share may have no files but sync can still run out of memory.
not seeing computer on different domain controllers
in Sync General Discussion
Posted
Absolute minimum is opening to the tracker and relay on UDP port 3000, but best would be a rule something like "All UDP outbound with port numbers over 1024, with solicited replies inbound." This would in fact make it the same as normal TCP configurations. Though, you can configure the range of ports if you have control over all the peers.