= [wiki:Summit 2nd Summit] Day 1 = Tuesday 08-Nov-2011. Mozilla SF. [http://people.mozilla.com/~bwarner/tahoe/Summit2Day1.flv Video] (1.7GB flash .flv) == Attendees (with IRC nicks) == * Brian Warner (warner) * Zooko (zooko) * David-Sarah Hopwood (davidsarah) * Zancas (zancas) * Shawn Willden (divegeek) * Zack Weinberg (zwol) * Zack Kansler (?) * Online: amiller, Dcoder == !Agent/Gateway split == - Shawn Willden observed that "tahoe backup" is usually run on a laptop (frequently sleeping/offline), whereas he's got some other machine (a desktop or home server) with limited CPU which *is* online 24/7, so he wants a backup program that quickly dumps the laptop's contents to the server, then (slowly/lazily) uploads that data from the server into Tahoe - extra points for only putting ciphertext on that server - Brian wants a long-running process (specifically a twisted.application.service.Service object) to manage backup jobs, storing state in a sqlite db (which directories have been visited, time since last backup, ETA). Likewise for renew/repair/rebalance jobs. - maybe web interface to control/monitor these jobs - vague consensus was to introduce an "Agent" service, distinct from the current "Client/Gateway" service. The normal client/gateway process will include both (and the co-resident Agent will have local access to the IClient object for upload/download). But it will also be possible to create one in a separate process (with no client/gateway), in which case it speaks WAPI over HTTP (and must be configured with a node.url). - backup and renew/repair/rebalance jobs run in an Agent - not sure about where the WAPI/WUI lives. One idea was to have the Agent provide the WUI, and the C/G provide the WAPI. Another is to have the C/G provide most webapi but add new webapi to the Agent for managing backup/renew/repair/rebalance jobs - [attachment:day1-agent.jpg whiteboard pix] == server-selection UI == expected vs required vs known, k-of-N, H, points, understandability - Brian talked about an old #467 explicit-server-selection message and his proposed UI to list all known servers in a table, with "use this server?" and "require this server?" buttons - David-Sarah (and Zooko) pointed out that "require?" is a bit harsh given our current H= ("Servers Of Happiness") share-placement code - tradeoffs between clear-and-restrictive vs confusing-but-fails-less - challenge of identifying reliability of nodes - Brian says client/gateway should expect user to teach it what sort of grid they expect, so it can bail when expectations aren't met - Shawn would prefer scheme where client measures node reliability itself (though in the near term allows the user to specify node reliabilities), sets N to the number of servers currently accepting shares, sets H slightly below N (to improve upload availability) and computes and uses a k that maximizes performance (or minimizes cost) while achieving a target reliability. - whiteboard: [attachment:day1-grid1.jpg grid1], [attachment:day1-grid2.jpg grid2] == encrypted git, or revision control on tahoe == - Zack(?) is thinking about revision control on top of Tahoe, will present "big crazy idea" later when everyone is there - Brian mentioned his signed-git-revisionid project (not yet released), and how git fetch/push is fast because both sides know full revision graph and can compute missing objects in one RTT. To get this in Tahoe, we must add deep-(verify)-caps and let servers see shape of directory tree. - Lunch conversation about Monotone's mutable metadata and the problem of transferring it efficiently == grid management == non-transitive one-at-a-time invitations, transitive clique invitations, Grid Admin star config - Brian is thinking about grid setup and Accounting, and pondering a startup mode where servers issue Invitations to clients - pasting Invitation code into a web form is sufficient to get connected to grid (combines Introducer functionality with Account authorization) - probably set up bidirectional connection: when Alice accepts Invitation from Bob, both Alice and Bob can use each other's storage - three modes: - issue/accept one Invitation per link - each node needs one Invitation to join clique, then they get access to all storage servers in the clique (and offer service to all clients in the clique): grid grows one node at a time - issue: can two grids merge? or can you only accept an invitation when you aren't already in a grid? - managed grid: a central Grid Admin is the only one who can issue Invitations. When accepted, Alice can use storage of all members. - Shawn thinks a Request model is more natural: Server admin (or Grid Admin) sends ambient URL to new user, they paste it into a field that says "Request Access", this sends a Request to the server (probably containing a pubkey), the server records it, then later the server admin Accepts or Rejects the request. - Invite and Request are duals, modulo some channel and workflow variations (confidential vs authentic, who-sends-first-message) - Brian will explore how hard/feasible it is to run one workflow on top of the other: can a Request be expressed with a note saying "please send an Invitation to this public encryption key" sent to the server? == #466 new-introducer review == - Brian walked through most of the #466 new-introducer code (https://github.com/warner/tahoe-lafs/tree/466-take7) with David-Sarah and Zooko - David-Sarah found one critical security bug (signature checking failure), lots of good cleanups to recommend, tests to add - overall it looks good - Brian will make suggested cleanups and prepare for landing == Beer! == == signature consensus! == - over drinks, Brian and David-Sarah and Zooko discussed signature options (needed for #466, Accounting, non-Foolscap storage protocol, new mutable file formats) - choices: - [https://github.com/warner/python-ed25519 python-ed25519] (standalone C extension module) - [https://github.com/warner/python-ecdsa python-ecdsa] (standalone pure-Python module) - ECDSA from Crypto++ via pycryptopp - non-EC DSA (eww) - get Ed25519 into Crypto++, then expose in pycryptopp - add Ed25519 into pycryptopp (making it more than just a python binding to Crypto++, hence nicknamed "pycryptoppp") - get ECDSA from pyOpenSSL (we think it isn't exposed) - evaluation: - security: David-Sarah prefers Ed25519, Zooko slightly prefers ECDSA (older, more exposure), Brian (who currently has a crush on everything 25519) slightly prefers Ed25519. Crypto++'s entropy-using signature code includes nonce-safety (entropy is hashed with message to mitigate VM-rollback failure). Ed25519 has deterministic signatures and nonce-safety. - speed: requirement is <10s startup with 100 servers (specifically, make sure the Announcement sign/verify is small compared to connection establishment time). That's sign+verify<100ms . This rules out python-ecdsa (sign+verify=330ms). Both a non-pure-python ECDSA and Ed25519 will do. A really fast primitive (optimized Ed25519 is like 20us) might enable new applications in the future (key-per-lease, key-per-write-request). - pure-python: slight preference for something that could be pure-python in the future if !PyPy could make it fast enough. Seems unlikely in the near-term for any of the options. - patents: murky, of course. !Redhat/Fedora core currently eschew all ECC, might change, might not, too bad. Not a clear differentiator between ECDSA and Ed25519. Nobody was willing to tolerate non-EC DSA (would need 4kbit keys to feel safe, not confident of hitting speed requirements). We can always back it out if it proves to be a problem (at the cost of regenerating all serverids). Hopefully the scene will settle down before we want to use it for data (which would be harder to back out). - packaging: biggest differentiator - python-ed25519: must build eggs as we did for pycrypto, need to get into debian (which has other benefits, but delays tahoe), increases build pain marginally. - python-ecdsa (too slow, ruled out): pure-python, so no need for eggs, but still need to get into debian and increases build pain - ECDSA-via-pycryptopp: easy, code is mostly done, needs final review and polish, no new dependencies. - ed25519-in-Crypto++: probably good idea in long term, but will take a while (must convince Crypto++ to change, wait for a release, then add bindings to pycryptopp). Must also wait for distributions to pick up new Crypto++. Technically no new dependencies, but increases the version requirements on an external module with a historically slow (1/yr) release cycle. - ed25519-in-pycryptopp: a bit weird (pycryptoppp), fairly fast (we control pycryptopp), no external delays. No new dependencies. - **winner: ed25519-in-pycryptopp** (aka pycryptoppp). Ed25519 wins over ECDSA with potential better security and future-coolness-enabling speed. Delivering in pycryptopp means no new dependency and no external parties to block. - future goal is to get python-ed25519 into debian, then switch Tahoe to depend on it instead. And/or once Ed25519 gets into Crypto++, remove the separate implementation from pycryptopp (i.e. remove one "p" from pycryptoppp) and have pycryptopp rely on Crypto++'s version. - also, get pycryptopp's ECDSA finished off in the ed25519-bearing release, just to have it available. - dev plan: - brian makes new python-ed25519 release with API tweaks - brian makes patch for pycryptopp with most of python-ed25519 - zooko makes new pycryptopp release - get folks to build eggs of new release - tahoe starts depending on new release - land #466 accounting code that uses ed25519