libtorrent blog

bittorrent development from the trenches

VirtualAlloc pitfall

When allocating blocks in the disk cache, libtorrent uses valloc(), to allocate page-aligned 16kiB blocks. On windows, the natural couterpart to valloc() is VirtualAlloc(). Having these blocks page aligned may provide performance improvements when reading and writing files that are aligned to the block boundaries. The 16kiB allocation size is derived from the bittorrent protocol . . .

DHT routing table maintenance

I’ve been working on performance improvements of the DHT recently that I would like to cover in this post. NICE routing table One of the proposed improvements from the sub-seconds lookups paper is referred to as NICE. It proposes replacing the method of maintaining the routing table buckets (see kademlia paper) with directly pinging the nodes, the most . . .

running averages

Many aspects of bittorrent requires maintaining an estimate of some kinds of samples. uTP keeps a running estimate of round-trip times for each connection. When streaming torrents, it is useful to keep an estimate of download time per piece (to know what a reasonable timeout is). This post takes a closer look at how to . . .

memory cache optimizations

When optimizing memory access, and memory cache misses in particular, there are surprisingly few tools to help you. valgrind’s cachegrind tool is the closest one I’ve found. It gives you a lot of information on cache misses, but not necessarily in the form you need it. About a week ago I started looking into lowering . . .

swarm connectivity

In bittorrent it is important to keep the swarm as evenly connected as possible. Clustering of peers may create bottlenecks for piece distribution and create a skewed market for trading pieces. Keep in mind that local piece availability is used as an approximation for global piece availability in the rarest-first piece picking algorithm. This post . . .

DHT security

One of the vulnerabilities of typical DHTs, in particular the bittorrent DHT, is the fact that participants can choose their own node ID. This enables an attacker to deliberately place themselves at a locaton in the DHT where they know they will be responsible for storing some specific data. At that point, there are a few naughty things . . .

principles of high performance programs

This article is an attempt to sum up a small number of generic rules that appear to be useful rules of thumb when creating high performing programs. It is structured by first establishing some fundamental causes of performance hits followed by their extensions. memory latency A significant source of performance degradation on modern computers is the . . .

asynchronous disk I/O

Since 2010, I’ve been working, on and off, on a branch off of libtorrent which use asynchronous disk I/O, instead of the synchronous disk calls in the disk thread in 0.16.x versions. The aio branch has several performance improvements apart from allowing multiple disk operations outstanding at any given time. For instance: 1. the disk . . .

windows’ disk cache

A long standing problem with bittorrent clients on windows is that if you’re seeding large files or downloading large files, windows may decide to essentially use all your physical RAM for disk cache. The disk cache grows to the point where running processes start having their working set swapped out, significantly slowing down the system . . .

bittorrent over SSL

Running bittorrent over SSL could make sense for several applications. Anything you want distributed to a closed group, but large enough to warrant bittorrent would do well being distributed over bittorrent/SSL. Currently closed group distributions either don’t use any peer-to-peer distribution at all, or they use poor-man’s privacy/security. I’m referring to the “private” flag of . . .