Archives For January 2013

BitTorrent Tech Talks: DHT

Arvid Norberg —  January 22, 2013 — 6 Comments

 

Every Wednesday, we meet in San Francisco, in a conference room creatively named San Francisco, for something we like to call Tech Talks. It’s one hour devoted to sharing the stuff that keeps us busy/keeps us up at night/keeps us coding.

In this week’s Tech Talks, we break down Distributed Hash Tables. Curious about how they work? Catch the video (above), or grab the slides over here.

 

I started working on an HTML5 torrent client a few months ago when I had the realization that if enough BitTorrent clients would accept WebSocket connections, it would be possible to download pieces directly from a desktop torrent client to your web browser. There would be several advantages to being able to participate it BitTorrent swarms in the “naked” web. By naked, I mean without any special plugins or software. A whole new set of users could participate in the BitTorrent ecosystem. From those who are unable and who are unwilling to install custom desktop software, to those who are stuck in walled gardens, but especially for those who don’t know or don’t care what a torrent is.

There were a number of challenges I encountered when developing the prototype. The most difficult problem is the lack of wide browser support for the FileSystem API. Google Chrome provides this very convenient and stable and usable interface. The only complaint I have is that it does not support sparse files (i.e. seeking far into a file and writing without having to manually fill in a lot of null bytes). I was very disappointed when I read Mozilla’s arguments against the implementation. There are polyfills that make use of IndexedDB, but they have severe limitations for my use case, which needs to support files up to multiple gigabytes in size.

I went ahead anyway implementing the protocol and saving files to the browser sandbox filesystem, intent on supporting only the Chrome browser for the time being. You can see the app hosted at jstorrent.com. It also can be loaded as a Chrome packaged app and will make use of the chrome experimental socket API.

BitTorrent maintains both the BitTorrent and the popular µTorrent client. Updates to these clients are slow and methodical, so most clients as of this date are still unable to accept Websocket connections (this is in our version 3.3). If you have been following the latest developments with WebRTC, you’d know that Firefox and Chrome are rolling out the DataChannel API, which will also allow for browsers to connect directly to each other. You can bet µTorrent will learn to speak WebRTC, too.

For purposes of demonstration, I decided to write a small embeddable html page that attempts to stream a video from a BitTorrent swarm directly in your browser. Since it is still not possible to connect directly to most swarms until µTorrent 3.3 usage is widespread, these make use of a custom proxy server that behaves much like websockify.

Original post (with working demo)