Sync API wishlist


kos13

Recommended Posts

I wish to share picture (and/or) video with my family and friends. But I don't want to full fill their hard drive. So I thing BTsync would be nice to do the share work. But I need a way to custumize right (group of friends). I will create a low resolution copy of the share material and share it with only the people I want. In the end they would be able to get the full size picture if they want (to print it or to play with it). Before i was using pixvillage which was very nice to share picture only with friend but the service is down so I working on a new solution. I working in python most of the time. I thinking to make a mix with BTsync for the share/sync work and Gajim for the people identification work.

Link to comment
Share on other sites

I would like to use Bit Torrent Sync to create shared storage pools used for multi-site synchronization of backup data. I would like to be able to write a management engine which queries BTSync to check on synch status at each remote storage pool and be presented in my own management interface so we can keep tabs on the status of backup transfer and, infact, whether new backup files were added to the storage pool.

I would also like the BT Syncapp to run as a service such that it does not need to run in the user context.

Thank you for your work on this, I am excited about what it can do for us!

Link to comment
Share on other sites

Search for a share by key
  • List contents of share (without synching the entire share locally)
  • Upload/Download a specific file (again without a full sync)
  • Sync a share (probably on demand, to avoid killing the device's battery and bandwidth)

I second gorpo's wish list. My needs are simple, I want basic functionality available via API for mobile apps. I primarily intend to use BTSync shares as external storage / export for app data.

Link to comment
Share on other sites

Ability to connect (mount) a remotely shared folder:

127.0.0.1:8888?action=mount&secret=<secret>&locfolder=<local folder>

Ability to create a new share folder:

127.0.0.1:8888?action=newsync&secret=<secret>&locfolder=<local folder>

Code should be able to create folders on the fly if needed, perhaps a true/flase parametr

Link to comment
Share on other sites

Ability to connect (mount) a remotely shared folder:

127.0.0.1:8888?action=mount&secret=<secret>&locfolder=<local folder>

Ability to create a new share folder:

127.0.0.1:8888?action=newsync&secret=<secret>&locfolder=<local folder>

Code should be able to create folders on the fly if needed, perhaps a true/flase parametr

I don't really think HTTP would be the best protocol for this kind of API (Being P2P and all).

Also, you can't really mount a folder using HTTP unless you mean you'd be running a executable on the program requesting the API, which, would kind of ruin the point.

Link to comment
Share on other sites

I'm just going to post this because I'm still wondering how this is going to work, but, here's my idea once again, but, more thought out:-

Connection to tracker to find IPs, application is marked with an A, bitSync (On the clients) is marked with a B and the tracker is marked with a T. This is English, but, obviously could be turned into more network I/O effective terms:-

//Application opens a connection with the tracker

A:- Hi, I'm looking for all users who have this share:- sha512/256/whatever($share)

T:- Here you go:- {["127.0.0.1", 80], ["192.168.0.1", 54434], ["10.0.0.1", 48929]}

//Tracker closes connection, assuming it's TCP, which I recommend

//Application runs
for(int i = 0; i < hosts.length; i++) {
//hosts in the above array returned from the tracker, the object would obviously be turned into a keypair.
Socket mySocket = new Socket(hosts.key, hosts.value);
try {
mySocket.connect();
} catch (Exception e) {
//Host doesn't like me
continue; //hop onto the next client
}
//At this point we're connected
talkToHost();
}

Then, the talkToHost function would run something like (Using the same intentifiers as above):-

A:- Hi, this is my handshake. I'm now going to encrypt everything with some kind of encryption (SSL/etc) that's come from the secret, so, assuming we are both using the same secret, we can talk
//Encryption
A:- Ping //If the btsync client doesn't understand this, it'll disconnect here
B:- Pong //If the application doesn't understand this, it'll disconnect here
//At this point both BTSync and the application agree they're using the same secret, however, for security sake they'll continue to run under this handshake

//BTSync at this point will just idle, with a timeout of ~ an hour. Each thirty minutes the BTSync client will submit a random 10 character string like following:-
PING:- TenCharsxx
//Then the application will reply back with a 24 bit proof of work using the hashcash method which much match the random 10 characters that the server just sent. If the application doesn't reply with ~ 5 minutes, the BTSync client should DC
PONG:- 1:24:130503:tencharsxx::uRDE5YjYAgOyl0lr:00000003e7B
//In case you don't know what hashcash is, if you sha1 the above statement it'll start with at-least 24 bits that are 0 (or 6 (24/8(bits per byte)*2(Characters required to represent a byte in hexadecimal) hexadecimal 0s):-
//000000e0cd0c26f841b975cd6808803ad2916a17
//You'll also notice that the random 10 characters are located inside the text, along with the date, and some more random characters
//The above 24 bit proof of work took under a second to generate on my laptop, [b]any[/b] PC will be able to generate it extremely quickly.

//The application can also issue requests from here, each request must follow the following syntax:-
//COMMAND($nulLCharacter (0x00))arguments($nullCharacter (0x00))split($nullCharacter (0x00))by($nullCharacter (0x00))by($nullCharacter (0x00))0x00($nullCharacter(0x00))COMMANDIDENTIFIER
//The command will then run, and, any returns/errors/status/callbacks will use the command identifier as the last object (split by 0x00) so that an application can run multiple commands at a time and be able to identify which returns are coming from which commands. For example:-

0x01|0x00|sha512(THISISMYSECRET)|0x00|APPLESAUCE

//Obviously it wouldn't have the pipe characters, but, this would me:-
//0x01 (Command identifier, in this case "Folder status request")
//0x00 (Split this with the next argument)
//The argument for the command, in this case the secret
//0x00 (Split this wiuth the next argument)
//No more arguments, so, the last one will be the command identifier "APPLESAUCE"

The BTSync application will then reply:-
0x01|0x00|APPLESAUCE
//0x01 being the status (or 0x01, for "All nodes in sync", 0x02 for "misisng folder", 0x03 for "downloading", 0x04 for "Uploading", etc...
//0x00 splits it
//No more reports, so, the command identifier, "APPLESAUCE"

Anyway, in a rush, I can't proof read at the moment and my spelling without spell check is terrible, and, I typed this out in gedit (E.G. no spell check). Sorry, gotta go, bye.

Link to comment
Share on other sites

Hi team, thanks for the hard work.

As some of the other commenters have mentioned, we'd love to be able to manage the adding/removing of folders and the aspects of key generation that go along with it, particularly returning both the read-only and read/write key for a folder.

One thing I think others haven't mentioned is that we'd like to switch off 'File downloaded' notifications for folders we create, and the user might not want notifications off for all their folders, so the ability to do this on a per-folder basis would be good.

Link to comment
Share on other sites

Personally i would like to see an API call that returns the files (or simply the amount) scheduled for upload. An use case for this would be a third party program that uses Bit Torrent Sync to distribute files. When the user wishes to close the third party program, the API would allow the programmer to check if everything was synced before closing.

Link to comment
Share on other sites

* A way to exclude certain files from being downloaded. Let's say you have a read-only key to a folder, which you want to keep up to date, but you want to exclude several large files from being saved. You can't delete them on the source, so perhaps just placing a zero-size file or having an ignore list that prevents them from downloading?

Link to comment
Share on other sites

Hooks/callbacks would be helpful. (Like sync starting, sync finishing, maybe conflicts).

I'm not sure what what the Bittorrent folks think of this, but it would be really nice if the protocol was specified so people could implement it themselves if they wanted to, without reverse engineering the protocol. (I think this technology could be used quite a bit in the open source world, but they tend to be more accepting of things if there is an open source implementation. I could see cases where people would join a linux distro share, and on release day automatically get the latest images.)

Link to comment
Share on other sites

I would like to sync my application-specific data using BTSync. This data could be as simple as basic application settings, or something more complicated (e.g. a photo library with metadata).

Ideally my application would tell BTSync it supports syncing of <insert text description> (e.g. "FooLibrary photos for Bob" or "AcmeWidget settings"). The user would then go to BTSync and "Add Application" just like they would normally add a Shared Folder. This would show a list of applications which report some sync ability. It would then act just like a Shared Folder acts currently, including keys and such.

On the API side, I'd like my application to be queried just like a filesystem would: what "folders" are there in the root folder, and what "files" (binary blobs) does each "folder" have. Perhaps BTSync could also ask the application for the file hashes, just in case it has them (but if not, could compute them by itself).

In case you're wondering why I might want my photo library application to offer sync instead of just syncing the photo files. Well, suppose I have carefully tagged each photo as "show this to Jane" or "show this to Bob". There is no *folder* that contains all the photos to be shown to Jane. Likewise for Bob. Moreover, the folder structure is sensible in another sense; it organizes the photos by event. So Jane might get "C:\Photos\Event A" and *some of* "C:\Photos\Event B", while Bob would get *all of* "C:\Photos\Event B". BTSync wouldn't need to know any of these details; it would just query my application for a list of folders and files, and the application would figure out what to say.

My application would, of course, have to report that it supports two separate syncs, a "Photos for Bob" and a "Photos for Jane".

These are some grand plans I've outlined, but it would also be unbelievably awesome if this were actually possible. So many use cases and possibilities are currently prevented by the inherent difficulty of actually implementing that sync feature for my Super Cool Photo Library Manager... Please be the ones to fix this!

Link to comment
Share on other sites

I could see this as a replacement for dropbox and the dropbox API in applications. Also iCloud on iOS.

So as an iOS developer I would expect a library I could include in my application and an API to interact with it.

In the first instance it may just be a low-level C based API. Followed later by an OO Objective-C API for iOS. In this case the developer is completely responsible for managing the user experience and making the process of syncing simple and secure for the user, in whatever way suits their application:

For instance in a hypothetical photo sharing app. I develop a mobile version, a desktop version, and perhaps an online service, and sync the users photos between them. I would be expected to implement some kind of user account system myself and make the process of setting up and linking each device simple for the user. In this case BitTorrent sync would just be plumbing.

A next step / alternative might be an API into a BitTorrent Sync library again, however in this case the concept of having apps use BitTorrent Sync for syncing is built into BitTorrent Sync. Perhaps with an alternative BitTorrent Sync desktop client that is specifically tailored to this use and easily understood by the average user. There might be an abstraction built over the top of the existing system that managed a single folder per application on the users behalf. The process of adding an App to your BitTorrent Sync "Cloud" would be standardised and simplified for the user. The library for use by the application would then have a common user interface for linking apps. The user would become familiar with a standard "login" / "authorisation" page that would be common across many apps.

In this case the user installs the hypothetical photo sharing app. The user has already installed BitTorrent Sync on their desktop in the past and authorised other applications. In order to sync devices I present a standard BitTorrent Sync screen where the user "authorises", "links", or "logs in", to their BitTorrent Sync "Cloud". As an app developer I then simply have a single folder where i read and write and manage my apps data. The BitTorrent Sync desktop client is then a place where the user can go to see what "Apps" are linked to their "Cloud" perhaps de-authorising them, limiting their bandwidth, disk usage etc.

Link to comment
Share on other sites

As a game developer that builds a modifiable game platform (rFactor 2). I would love a couple of things.

Limit bandwidth upload in real time so that transfer do not effect data that has a real time requirement.

Notification of a sync start/complete/error.

Use in house tracker (or manageable tracker) for usages stats and to limit copyright infringing material. Think of a "whats hot" feature.

I can see us using this for everything we do. I would love an opportunity to be a early tester/developer.

Link to comment
Share on other sites

It will be very useful to have a standard API that is supported across all these types of services (Dropbox, GoogleDrive, BTS, etc) so that adding support to a new ones such BTS could be invisible to the user, it becomes just like a new drive. (Just wishing here)

I really hoppe the API could be use across all the devices just like dropbox. But here are a few request that will be great:

1. The ability to collect information about the nodes, (IP,Ports,On/OFF,etc).

2. Having one node be able to have exclusive lock into one file or folder. (So in other nodes local users can no overwrite their local copies. ) Good use for this could be for collaborating with others.

3. Been able to send network messages to other nodes and have the nodes act like a proxy to a local script. (Or the ability to make this happen.) Basically to leverage the connectivity that the nodes have without having to open additional ports for our own scripts.

4. a call that makes the local node update itself or update a file/folder.

5. Make your share folders act like a cache rather than a full sync folder. (This could be useful for mobile devices)

6. Change the frequency it detects/sync changes in certain folders. (Will help with collaboration)

7. Better File/Folder history. (May be with some integration with Git/SVN?)

Link to comment
Share on other sites

Idk if this is what you all want but I would like to make some software in Java that could interact with BT Sync in the following ways

-Get all the names of folders, locations, keys and preferences.

-be able to get those folders and delete them (from BTSync)

-be able to modify the preferences of folders

-be able to add folders and set its preferences in btsync

Link to comment
Share on other sites

I want to have an API so that I can check what files are contained in a secret, for example:

Endpoint: api.bittorrentsync/preview?secret=RDF44TA5ELGVEPFGYWPACCAKOEGCDMUMC

Return:

[

"file1",

"file2",

"folder1":[

"subfile1",

"subfile2"

],

]

If the secrete is invalide, return:

{error:invalid}

This would be very helpful for developer who want to validate a shared secret, and for user who want to have a preview before put the secret to sync.

Link to comment
Share on other sites

Hey there.

I would like to have two CLI arguments:

"btsync --last-completed-synchronization $folder"

I've written some startup scripts that trigger some tools and wait for mounts to appear, e.g. TrueCrypt, Samba and so on. Would be nice to have a tool in place which tells me when a specific sync folder was completely in sync. Usually having folders in sync for 4h or more I must wait because synchronization will start in a couple of seconds.

"btsync --block-until-next-synchronization-finishes $folder"

In addition to the use case above, I would call this command and expect it to wait just as long as the next synchronization job for the given folder is completed. If there is no synchronization task running currently, it sould still wait since the next synchronization run will start in a couple of seconds.

The thing I want to achieve as a CLI task (to run as bash on linux or batch on windows) like "sleep until the given folder was synchronized less then 4h ago".

Having a simple "execute task something as soon as synchronization is completed" would require me to implement a polling loop in bash ans batch, so having an in-place wait would be really nice here.

Regards,

Stephan.

Link to comment
Share on other sites

Just being able to read, write and serve files from a directory would be pretty awesome - I envision apps like instagram or vine that sit on top the bittorent sync storage.

This way all the media always belongs to the user; and we have none of the pesky T&C's where the company owns the user's content.

Taking it a step further and being able to "serve" said files directly from the storage would be pretty damn amazing - being able to a build an app when you don't have to worry about scaling / bandwidth; and let the p2p protocol handle it.

Link to comment
Share on other sites

  • 4 weeks later...

I started toying with using BTSync as a messaging system, where users create outboxes for each other and exchange read-only secrets. The setup is a little cumbersome, and an API that allows for creating and managing shares would help automate nearly all the steps. (edit: Actually, even just a command-line version on non-Linux computers would solve this.)

Also, not API-specific, but a write-only share would be interesting, to use as a public inbox for the above usecase, and also more generally. This kind of thing should have the ability to whitelist files based on parameters, like size, extension (regex pattern for the name, perhaps). Being able to customize the write-only secret, like the master secrets can be, would also be great, so they could be made more friendly and suitable for a public "address" of sorts.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.