How Does Bleep Work?

Farid Fadaie —  September 17, 2014 — 11 Comments

We unveiled Bleep a few weeks ago, and we have received nothing but love from our users. We’ve also had many great questions that we will be gradually answering in blog posts, our forums and other social channels. And today we’ve taken a step further, bringing Bleep to Open Alpha and adding clients for Mac and Android. There’s more to Bleep than we can fit in a single blog post, but here’s an overview of the big picture, including some high level technical details and answers to some of the questions around how Bleep works.

Continue Reading…

Installing Mavericks inside a virtual machine is fairly easy, but there are a few tricks to be aware of if you’re on a newer Mac. Credit to Natsuki’s post for sharing how to get the Mavericks installer to run on Apple computers with Intel Haswell CPU’s. Natsuki also notes a workaround for Apple computers with ECC RAM that requires the removal of a kernel module from the install image using iesd.

Download and install the latest VirtualBox for OS X hosts from here. We’ll be using VirtualBox so that anybody can follow these steps.

Download the Mavericks Installer App through the App Store.The Mavericks installer is provided for free by Apple for users to upgrade their computers to the latest version of OS X. We’ll be making use of the install image provided by this application to install Mavericks within a virtual machine.

While we’re waiting for the Mavericks Installer to download, lets get started on configuring our new Virtual Machine.

Create a New Virtual Machine

2014-06-13_1303
Continue Reading…

uTorrent

Our flagship torrent clients, uTorrent and BitTorrent, do many different things in the course of downloading content onto your computer faster than what’s physically possible with other protocols. There is a lot of functionality going on under the hood in the course of a download, which fall into three broad categories: networking, disk IO and user interface. We have written before about enhancements in the first two categories (uTP, DHT and multithreaded disk IO, for instance) and are now pleased to announce new improvements in how the user interface interacts with the rest of the system.

In addition to requesting and sharing pieces of files on the network and assembling them on disk, the client needs to convey these activities to the user by showing them on the screen. The logic which displays this information often interacted with the rest of the program in a coarse and intrusive way, requiring networking and/or disk IO to briefly halt while the information was gathered. For example, while viewing the list of files contained in a torrent and their relative download progress, the networking subsystem would be halted several times a second. However brief these pauses were, when multiplied by several UI elements and possibly large lists of torrents and their files, it was enough to slow overall performance.

As an analogy, consider a car’s speedometer as the user interface and the engine, transmission and wheels as the core subsystem doing the work. If the engine had to pause several times a second to update the state of the speedometer, performance would suffer.

To avoid these pauses, the layer between the UI and the rest of the system was made richer and less synchronous. Instead of pausing the subsystem to interrogate values, the UI first conveys broadly what it is interested in (which tab is visible, which torrent is selected), and the subsystem then asynchronously presents only the updated information relative to that state. The user interface layer is then free to present this information without holding up the other layers.

This is an exciting update for us to issue, as the entire engineering team is constantly refining products to try and provide the best user experience.

There’s always more to come on this front, so do stay tuned!

BitTorrent Tech Talks are one-hour sessions dedicated to the stuff that keeps us busy / keeps us up at night / keeps us coding. From time to time, we post them here. Because sharing.

In this edition of Tech Talks: an overview of some C++ gems. I threw this talk together because my team was about to start a new project in C++11. Since it’s fairly new, I figured some of it might not be as well-known as it should. Fundamentally, I’m pretty excited about all the new possibilities in C++11. Even higher-level abstractions, at even lower cost than C++98.

In the video below, we go over for-loops, automatic type deduction, lambda functions and more.

Correction: I say that lambdas with an empty capture statement defaults to by-value, which is incorrect. It defaults to not capturing anything.

Follow along with the C++ in the 21st Century slides:


 

Looking for more on BitTorrent engineering? We’ve got Tech Talks aplenty. Check out Distributed Hash Tables, DHT Bootstrap Update, and Writing High Performance Software.

client-hero-torrents

3.4 is an exciting release for the uTorrent team.

3.4 is the first version to include a major change in the way that uTorrent chooses peers in a swarm. Designed by our own Arvid Norberg, Canonical Peer Priority is a way to help peers connect to the swarm faster, as well as reduce the average hop length from you to any other peer in the swarm.

When a bittorrent client joins a swarm, it needs a way to select which peers it connects to. If it chooses poorly, or if there are malicious actors in the swarm, the connections between clients are not well distributed through the swarm, leading to a large number of hops from node to node. That slows down the ability to each client to pass data on to the next.

You can read a more detailed technical discussion of the issues here, along with graphs and figures that drive home how bad the worst case can be. You can read more about graph connectivity here.

Perhaps one of the biggest changes, though, is one you cannot see. Our engineering team has been growing rapidly, and we have been busy changing our development and release processes. uTorrent 3.4 will mark the first release using improved processes that should allow us to release much more often, while keeping stability at the levels you have come to expect from the world’s fastest and lightest torrent client.

Our previous release cycle was slow. We followed the traditional alpha -> beta -> stable model that a lot of software development follows, for example large video games or operating systems. One of the problems with this style of development is as stabilization work continues on the features you just developed, new features are requested, or requirements change, and now you have to balance two lines of development in the same tree.

Also, with more developers, more changes can be made simultaneously … in theory. In reality, changes in unrelated modules (e.g. the installer) would impact when we could ship new code in other areas (e.g. the disk code), and of course, vice versa. This creates a vicious cycle, where each small problem creates a knock-on effect that impacts other features.

In a situation like this, instead of asking the business to “pick one thing and stick with it” the correct response is for the engineering team to change how they operate.

* On a small scale, picking one thing and sticking with it.
* On a larger scale Multiplexing the work into separate branches.

We needed a way to release changes fast and reliably. This implied quite a few things:
* Don’t mix changes
* Release fast, review results fast

This required us to build a few systems. Some of the larger ones:
* Our release system (code-named “Cherry”)
* Or automatic update system (code-named “The automatic update system”)

It also required programming policies into the smaller parts of the system that already existed
* The build server
* The version control system
* New test servers

These systems, working together, can now answer the question: Is this feature ready for release?
Will deploying this feature likely increase or decrease the crash rate?

We now build individual features in separate branches, which are automatically tested for stability before being integrated into the mainline. That gives us confidence that we won’t slow other engineers down, and that we won’t release a low-quality build to customers.

This effort would not have been possible without the support of the excellent engineering team at Bittorrent.

I look forward to covering these in detail in later posts.

From the uTorrent engineering team, and the rest of Bittorrent as a whole, Happy torrenting!

BitTorrent Tech Talks are one-hour sessions dedicated to the stuff that keeps us busy / keeps us up at night / keeps us coding. From time to time, we post them here. Because sharing.

In this edition of Tech Talks: BitTorrent’s Chief Architect Arvid Norberg describes how modern computers work, and the key challenges to software performance. He then breaks down how to modify data structures to better take advantage of your memory cache. Want to write your own high performance software? Watch the full video, or follow along with the slides below.


Writing High-Performance Software by Arvid Norberg from bittorrentinc

For more talks by Arvid, check out distributed hashtables and his follow-up, distributed hashtables updated.

Update on BitTorrent Chat

Abe —  December 19, 2013 — 88 Comments

Inside BitTorrent’s approach to building serverless messaging apps.

121713-bt-chat-blog-2

First, a few words on Chat’s origins. Here at BitTorrent, we value privacy. With the news this year reminding us all of the susceptibility of the communications platforms we rely on to snooping, we found ourselves wanting something new, something secure, something private. We ultimately realized that we were uniquely qualified to build this platform.

The primary weakness that we see in the available communications platforms is that they all rely on some central server to route and store all of your communication. Even if your provider can deliver industry-standard security, they cannot provide you with any kind of assurance that your communication is private. All it takes is the right (or wrong) person gaining access to your provider’s central servers, and your privacy evaporates.

Enter BitTorrent Chat. We’re building a product that allows you to talk to your friends using peer-to-peer. No central authority required.

Continue Reading…

DHT Bootstrap Update

arvid —  December 19, 2013 — 4 Comments

Arvid Norberg, chief architect for BitTorrent, Inc, introduces a new DHT bootstrap server. This latest version introduces Node ID enforcement as an important step in our development for BitTorrent Chat. It’s also now open source so that anyone can run their own bootstrap node.

The BitTorrent Distributed Hash Table (DHT) has a fundamental dependency on being introduced to some nodes that are already in the network. There are many sources of these nodes. For instance, your client is likely to save nodes on disk to retry them when you start back up again. Any BitTorrent peers are likely to be on the DHT as well, so those are also tried. However, if you just installed a BitTorrent client, and you don’t have any BitTorrent peers, you must rely on a bootstrap server.

BitTorrent Inc. runs ``router.bittorrent.com`` on port 8991 for this purpose.

We are now providing our DHT bootstrap server open source on github. You can now run your own DHT bootstrap node! Please play with it and contribute fixes, features, and performance improvements.

The DHT bootstrapper has some interesting properties. Up until 5 years ago or so, ``router.bittorrent.com`` was running just another DHT node, just like the one in µTorrent. This had some obvious problems. Since the default routing table size is 8 nodes per bucket, half of all requests to the bootstrap would get the same 8 nodes handed back to it. At several thousand requests per second, this would effectively DDoS any poor node that happened to end up in its routing table.

We rewrote the bootstrap server to have a flat array of nodes instead and to have two cursors, one for reading and one for writing new nodes into it. Every node that pings the bootstrap server is put in a queue and pulled out 15 minutes later to be pinged. If it is still alive, it is added to the node list.

This is still the case with the latest rewrite, with one addition: Node ID enforcement. We have been looking at securing the DHT, making it harder to attack (especially with sybils). One thing we’re implementing to support this is requiring DHT nodes to calculate their node ID based on their external IP, with some flexibility to support NATs and such. More info on Node ID enforcement can be found here.

The idea is that with Node ID enforcement sybil attacks, where one machine pretends to be thousands of nodes, will become impossible.

The new bootstrap server will still serve nodes with invalid node IDs (in fact, legitimate nodes just joining are not likely to know their external IP yet). However, it will not ping nor add these nodes to the node list for handing out.

This is one step in the preparations we’re making for BitTorrent Chat, which will rely on the DHT and benefits from having a DHT that’s harder to eavesdrop and scrape.

An Nguyen, developer at BitTorrent

An Nguyen, developer at BitTorrent.
(An wants you to know he wears a tank top most days, and not just for blog photo shoots.)

In Tech Talk, we share the stuff that keeps us busy, keeps us up at night, and keeps us coding.

Often, we have a bunch of data that we want to view or edit over and over again. Here’s how to get started with a graphical user interface (GUI) tool to simplify your workflow, help you work more efficiently, and prevent errors along the way.

mvvm_blog

If you’re on Windows, native on a PC or via VM/virtual machine, you can leverage Windows Presentation Foundation (WPF), a free Microsoft GUI framework with many widgets and controls out of the box. Others are freely available here.

Continue Reading…

uT Linux Server
Good news!

We’ve just updated the µTorrent Server for Linux. It’s a pretty major refresh from the last version. What’s new? Here’s the rundown.

New features:

Touch based input option for Web UI: use your tablet or mobile phone to control your torrent client in the home network
Remote.utorrent.com integration: use your tablet, mobile phone or computer to control your torrent client from anywhere on the Internet. Either visit remote.utorrent.com or use the Android µTorrent Remote application.

Big improvements:

Performance enhancements to the Torrent engine: the faster just got faster.
Improved web UI: a modern rewrite of the UI for controlling the torrent client in the home network
□ Support for signed torrents
□ Updated RSS feed parser
□ Multiple bug fixes and enhancements

Give the new version a shot, and let us know what you think. Our forums are open. All questions, ideas, and feedback are welcome.

button-free-download

–Team BitTorrent