start downloading as soon as you know where to get K shares

[zooko]

The current code (2a2cc5c4b8a490ae) performs immutable file download in two phases. In phase 1 it sends a get_buckets() remote invocation message to every storage server that it knows about (CiphertextDownloader._get_all_shareholders()). In phase 2 it requests the CEB (a.k.a. UEB) from each server in turn until it gets a valid CEB (CiphertextDownloader._obtain_uri_extension()). In phase 3 it requests the crypttext hash tree from each server in turn until it gets a valid hash tree (CiphertextDownloader._get_crypttext_hash_tree()). In phase 4 is actually downloads the blocks of data from servers in parallel. Out of the shares which it learned about in phase 1, which shares does it choose? It chooses "primary shares" first (their sharenum is < K) because those can be erasure-decoded at no computational cost, then it chooses randomly from among secondary shares until it has K.

Now currently phase 1 does not end, and therefore phase 2 does not begin, until all servers have answered the get_buckets()! To close this ticket, make it so phase 2 ends as soon as at least K buckets have been found.

The nice non-invasive way to do this is to replace the DeferredList? in CiphertextDownloader._get_all_shareholders() with an object that acts like a DeferredList? but fires once at least K shares (or possibly once at least K servers) have been found.