id	summary	reporter	owner	description	type	status	priority	milestone	component	version	resolution	keywords	cc	launchpad_bug
1545	add readv() API to immutable-share storage-server protocol, use in downloader	warner	davidsarah	"One of the most obvious fixes for the immutable-download performance problems tracked in #1264 (and on the [[Performance/Sep2011]] results) is to implement a scatter/gather {{{readv()}}} method for immutable shares. The graphs show MDMF downloads running just as fast with k=60 as with k=3, whereas for immutable files there is a drastic slowdown (10x) between k=3 and k=60. We're still investigating, but I suspect that Foolscap's message-serialization performance is to blame, and an easy way to mitigate that is to send fewer messages.

The interface should probably be just like the mutable-share's {{{remote_readv()}}} API: a vector of {{{(offset,length)}}} tuples, and the return value is a vector of data strings. (A future HTTP-based interface will probably pack these vectors into a single string, but we might experiment with doing that here too (basically do the marshalling before handing anything to foolscap, trading off generality for performance).

David-Sarah mentioned that some of their new storage-backend code (for LAE) provides this interface, so we're likely to have the back half of this feature fairly soon. The rest of the work is to change immutable/downloader/share.py to turn a Request span into a read vector, instead of looping over all pieces of the span and sending separate {{{read()}}} requests for each.
"	enhancement	assigned	major	soon	code-storage	1.9.0a2		performance