libtorrent blog

bittorrent development from the trenches

link compatibility

A major source of errors among users of libtorrent has traditionally been caused by binary incompatibility between the (built) libtorrent library and the client linking against it. Binary- or link compatibility is having two sides of a shared library (or translation unit) boundary have different understandings about the layout of objects or calling conventions. Consider the following library interface: . . .

DHT bootstrap node

With the release of libtorrent-1.1.1, libtorrent finally got its very own default DHT bootstrap node, dht.libtorrent.org:25401. This post gives some background on the work that went into setting it up.

unsigned integers

In this post I will talk about the use of unsigned (integral) types in C++, or perhaps more specifically the rationale for using them. In my experience, it is common to use unsigned types for any variable holding a value that cannot be negative. Say, the number of bytes in a buffer.

slow start

This post is a result of looking into a slow-start performance issue in uTP. Slow-start is a mechanism in TCP employed to discover the capacity of a link, before transitioning into the steady state regime of additive increase and multiplicative decrease. Slow start is employed on new connections and after time-outs (where the congestion window . . .

libtorrent alert queue

The main mechanism libtorrent uses to report events and errors to the client is via alerts. Alerts are messages as c++ objects with additional information depending on the type of message. Periodically clients poll for new alerts from a session object. In the next major release of libtorrent detailed peer logging will be available as . . .

bdecode parsers

I have recently revisited the bdecoder in libtorrent, and ended up implementing a new bdecoder that is two orders of magnitude faster than the original (naive) parser. This is the 3rd decoder in libtorrent’s history, and I would like to cover its evolution of parsing bencoded data.

a bittorrent filesystem

One of the main bottlenecks when downloading and seeding content over bittorrent is accessing the disk. This post explores the option to bypass traditional filesystems and use a block device as storage for torrents, in order to improve download performance. bittorrent protocol BitTorrent downloads conceptually divide up content into pieces, which are downloaded in rarest-first order. . . .

filenames

A .torrent file is very flexible in what it allows a path or filename to contain. Each directory name in a path is a length-prefixed utf-8 string. It can be an empty string, and it can have any character imaginable in it. The way a .torrent file represents filenames and paths is also agnostic to . . .

VirtualAlloc pitfall

When allocating blocks in the disk cache, libtorrent uses valloc(), to allocate page-aligned 16kiB blocks. On windows, the natural couterpart to valloc() is VirtualAlloc(). Having these blocks page aligned may provide performance improvements when reading and writing files that are aligned to the block boundaries. The 16kiB allocation size is derived from the bittorrent protocol . . .

DHT routing table maintenance

I’ve been working on performance improvements of the DHT recently that I would like to cover in this post. NICE routing table One of the proposed improvements from the sub-seconds lookups paper is referred to as NICE. It proposes replacing the method of maintaining the routing table buckets (see kademlia paper) with directly pinging the nodes, the most . . .