Update on BitTorrent Chat

Abe —  December 19, 2013 — 90 Comments

Inside BitTorrent’s approach to building serverless messaging apps.


First, a few words on Chat’s origins. Here at BitTorrent, we value privacy. With the news this year reminding us all of the susceptibility of the communications platforms we rely on to snooping, we found ourselves wanting something new, something secure, something private. We ultimately realized that we were uniquely qualified to build this platform.

The primary weakness that we see in the available communications platforms is that they all rely on some central server to route and store all of your communication. Even if your provider can deliver industry-standard security, they cannot provide you with any kind of assurance that your communication is private. All it takes is the right (or wrong) person gaining access to your provider’s central servers, and your privacy evaporates.

Enter BitTorrent Chat. We’re building a product that allows you to talk to your friends using peer-to-peer. No central authority required.

Continue Reading…

DHT Bootstrap Update

Arvid Norberg —  December 19, 2013 — 5 Comments

Arvid Norberg, chief architect for BitTorrent, Inc, introduces a new DHT bootstrap server. This latest version introduces Node ID enforcement as an important step in our development for BitTorrent Chat. It’s also now open source so that anyone can run their own bootstrap node.

The BitTorrent Distributed Hash Table (DHT) has a fundamental dependency on being introduced to some nodes that are already in the network. There are many sources of these nodes. For instance, your client is likely to save nodes on disk to retry them when you start back up again. Any BitTorrent peers are likely to be on the DHT as well, so those are also tried. However, if you just installed a BitTorrent client, and you don’t have any BitTorrent peers, you must rely on a bootstrap server.

BitTorrent Inc. runs ``router.bittorrent.com`` on port 8991 for this purpose.

We are now providing our DHT bootstrap server open source on github. You can now run your own DHT bootstrap node! Please play with it and contribute fixes, features, and performance improvements.

The DHT bootstrapper has some interesting properties. Up until 5 years ago or so, ``router.bittorrent.com`` was running just another DHT node, just like the one in µTorrent. This had some obvious problems. Since the default routing table size is 8 nodes per bucket, half of all requests to the bootstrap would get the same 8 nodes handed back to it. At several thousand requests per second, this would effectively DDoS any poor node that happened to end up in its routing table.

We rewrote the bootstrap server to have a flat array of nodes instead and to have two cursors, one for reading and one for writing new nodes into it. Every node that pings the bootstrap server is put in a queue and pulled out 15 minutes later to be pinged. If it is still alive, it is added to the node list.

This is still the case with the latest rewrite, with one addition: Node ID enforcement. We have been looking at securing the DHT, making it harder to attack (especially with sybils). One thing we’re implementing to support this is requiring DHT nodes to calculate their node ID based on their external IP, with some flexibility to support NATs and such. More info on Node ID enforcement can be found here.

The idea is that with Node ID enforcement sybil attacks, where one machine pretends to be thousands of nodes, will become impossible.

The new bootstrap server will still serve nodes with invalid node IDs (in fact, legitimate nodes just joining are not likely to know their external IP yet). However, it will not ping nor add these nodes to the node list for handing out.

This is one step in the preparations we’re making for BitTorrent Chat, which will rely on the DHT and benefits from having a DHT that’s harder to eavesdrop and scrape.

An Nguyen, developer at BitTorrent

An Nguyen, developer at BitTorrent.
(An wants you to know he wears a tank top most days, and not just for blog photo shoots.)

In Tech Talk, we share the stuff that keeps us busy, keeps us up at night, and keeps us coding.

Often, we have a bunch of data that we want to view or edit over and over again. Here’s how to get started with a graphical user interface (GUI) tool to simplify your workflow, help you work more efficiently, and prevent errors along the way.


If you’re on Windows, native on a PC or via VM/virtual machine, you can leverage Windows Presentation Foundation (WPF), a free Microsoft GUI framework with many widgets and controls out of the box. Others are freely available here.

Continue Reading…

uT Linux Server
Good news!

We’ve just updated the µTorrent Server for Linux. It’s a pretty major refresh from the last version. What’s new? Here’s the rundown.

New features:

Touch based input option for Web UI: use your tablet or mobile phone to control your torrent client in the home network
Remote.utorrent.com integration: use your tablet, mobile phone or computer to control your torrent client from anywhere on the Internet. Either visit remote.utorrent.com or use the Android µTorrent Remote application.

Big improvements:

Performance enhancements to the Torrent engine: the faster just got faster.
Improved web UI: a modern rewrite of the UI for controlling the torrent client in the home network
□ Support for signed torrents
□ Updated RSS feed parser
□ Multiple bug fixes and enhancements

Give the new version a shot, and let us know what you think. Our forums are open. All questions, ideas, and feedback are welcome.


–Team BitTorrent

Screen Shot 2013-04-23 at 8.06.24 PM

Brave inventors, intrepid tinkerers, awesome doers, and startups everywhere: we’ve got a job for you.

Over the course of the past few months, we’ve been busy in the Lab. We’ve been hard at work on a solution for personal file management; opening up an Alpha version of BitTorrent Sync. We’ve been looking at ways to streamline creative collaboration; using a simple file delivery service we call SoShare. We’ve been exploring search prioritization; helping artists find visibility via BitTorrent Surf. And we’ve been experimenting with the storytelling and monetization potential of BitTorrent publishing; via content releases with media innovators: authors like Tim Ferriss, studios like Cinedigm.

Our goal is to create a more sustainable distribution model for the Internet’s creators and fans.

We’re in this together.

And we want your help.

We’re opening up our innovation lab to new media startups. If you have a publicly available product in sound, film, live streaming, media sharing, or peer-to-peer technologies, we’ll work with you to accelerate your idea’s success. Continue Reading…

There are a couple of fairly nasty bugs in the LSP sample code from Microsoft in the function WSPSelect, and as far as I can tell, this one hasn’t been reported anywhere before. Since we’re working on an LSP here, I’ve had to fix this bug. 🙂

The bug affected iTunes and safari, which I have noticed some traffic about.

The actual bug is in the three clauses that copy the activations from the provider-socket fd_sets into the client-supplied fd_sets that contain client-visible socket objects

They look like this:
(note that ReadFds is internal, and contains provider sockets, readfds is the user’s parameter, and contains client facing socket descriptors)

And there are three copies of this code for the three fd_sets passed into select.

So what happens if we pass two fd_sets to this code with the same address? Note that in each clause (seperately), the client’s fd_set is zeroed!

So WSASelect(0, &my_sockets, &my_sockets, &my_sockets, NULL) will report the correct count, but only sockets that have exceptions will remain activated when we return to the caller! Moving the FD_ZERO macro expansions and the count captures all above these passes through the fd_sets will fix this bug.

The second one is less severe but still might be harmful; note that the return from the lower provider’s select is a count of activated sockets. This count is not the number of fd_set members that achieved activation, but the number of sockets, so if a socket is repeated for some reason in the incoming fd_set, it will only be counted once on input, but it will be represented twice in our internal array. Microsoft’s sample code will terminate early in this case (note that HandleCount– will be executed twice for one socket), with two activations of one socket, but leaving any remaining events uncopied, My suggestion is to also check whether the we’d be setting the same socket twice into the client’s fd_set, so we can skip double counting as the underlying provider apparently does.

In any case, you should definitely check the code you’re importing. Even OS authors make mistakes, as much as their frequent machismo might suggest otherwise.


Now that we have begun to send out invites to BitTorrent Live broadcaster applicants, it seemed like a great time to begin a series of technical articles discussing the product. We’re taking slow steps toward completely opening the floodgates, trying to make sure that we have a usable, rock solid product once we release it to the world. Right now, you can enjoy the channels of our first broadcasters without an invite.

Meanwhile, expect to see explanations of how Live operates internally, as well as tutorials on how to put it to use, here on the BitTorrent Engineering Blog.

The development of BitTorrent Live has consisted of work in essentially two camps. First is the Python-based ‘core’, which handles the actual functioning of the protocol, as well as the underlying cryptography used when transmitting and receiving data. Second is the ‘app’, which consists of a tight layer around the core, providing a command line interface, as well as a layer beyond that which provides a Windows, Mac, or Linux GUI.

As the core has become more stable over time, and major changes to the protocol less frequent, providing features like Local Peer Discovery has become a possibility. It is present in all major BitTorrent clients, and provides great performance and efficiency gains. Live is an entirely new protocol, written from the ground up, so adding support required a bit of planning.
Continue Reading…

BitTorrent Tech Talks: DHT

Arvid Norberg —  January 22, 2013 — 6 Comments


Every Wednesday, we meet in San Francisco, in a conference room creatively named San Francisco, for something we like to call Tech Talks. It’s one hour devoted to sharing the stuff that keeps us busy/keeps us up at night/keeps us coding.

In this week’s Tech Talks, we break down Distributed Hash Tables. Curious about how they work? Catch the video (above), or grab the slides over here.

[slideshare id=16047050&doc=bittorrenttechtalksdht-130117164434-phpapp01]


I started working on an HTML5 torrent client a few months ago when I had the realization that if enough BitTorrent clients would accept WebSocket connections, it would be possible to download pieces directly from a desktop torrent client to your web browser. There would be several advantages to being able to participate it BitTorrent swarms in the “naked” web. By naked, I mean without any special plugins or software. A whole new set of users could participate in the BitTorrent ecosystem. From those who are unable and who are unwilling to install custom desktop software, to those who are stuck in walled gardens, but especially for those who don’t know or don’t care what a torrent is.

There were a number of challenges I encountered when developing the prototype. The most difficult problem is the lack of wide browser support for the FileSystem API. Google Chrome provides this very convenient and stable and usable interface. The only complaint I have is that it does not support sparse files (i.e. seeking far into a file and writing without having to manually fill in a lot of null bytes). I was very disappointed when I read Mozilla’s arguments against the implementation. There are polyfills that make use of IndexedDB, but they have severe limitations for my use case, which needs to support files up to multiple gigabytes in size.

I went ahead anyway implementing the protocol and saving files to the browser sandbox filesystem, intent on supporting only the Chrome browser for the time being. You can see the app hosted at jstorrent.com. It also can be loaded as a Chrome packaged app and will make use of the chrome experimental socket API.

BitTorrent maintains both the BitTorrent and the popular µTorrent client. Updates to these clients are slow and methodical, so most clients as of this date are still unable to accept Websocket connections (this is in our version 3.3). If you have been following the latest developments with WebRTC, you’d know that Firefox and Chrome are rolling out the DataChannel API, which will also allow for browsers to connect directly to each other. You can bet µTorrent will learn to speak WebRTC, too.

For purposes of demonstration, I decided to write a small embeddable html page that attempts to stream a video from a BitTorrent swarm directly in your browser. Since it is still not possible to connect directly to most swarms until µTorrent 3.3 usage is widespread, these make use of a custom proxy server that behaves much like websockify.

Original post (with working demo)

Introducing BeamItOver

patrick —  December 21, 2012 — Leave a comment


Today, I’m happy to bring you BeamItOver, the latest project from BitTorrent Torque Labs.

Continue Reading…