Bittorrent Sync Memory / Ram Millions Of Files


smartinmedia

Recommended Posts

Hi all!

 

Our company has websites, which host millions of files. These are 10-50 KB each. One Windows server carries around 100 million jpeg-images with a total of around 1 TB disk space.

 

To have a fail-over (or a round robin), we would like to sync files between Windows servers (WAN, not LAN). HOwever, we saw that BitTorrent sync requires around 300 bytes / file. This would mean a memory requirement of 30 GB for BitTorrent alone.

 

Is there a solution to this?

 

Thanks so much!

Link to comment
Share on other sites

@smartinmedia,

 

Currently there is no way for BTSync to reduce amount of memory consumed per file. However, I can suggest a trick which you can use to sync that amount of files. 

 

1. Most likely, your data can be split in some smaller logical pieces (like, sites, or customers, or some other division).

2a. Use API to add folder, sync it, remove.

2b. OR - use config files, one per logical piece. Run BTSync with the config file, shut it down once sync is done, run next.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.