devchat notes 12-Sep-2017

[Brian Warner]

Tahoe-LAFS devchat 12-Sep-2017

Attendees: warner, exarkun, liz, simpson, meejah

* exarkun's performance investigations
  * foolscap vs eliot
* bug in publish
  * SDMF publish respects node's k/N for half of the logic, and the
    sharemap's k/N for the other half
  * so you can't modify a mutable file if the default encoding
    parameters have changed
    *
https://github.com/LeastAuthority/tahoe-lafs/blob/publish-error-extra-debugging/src/allmydata/mutable/publish.py#L736
      - respects node configuration via self.required_shares
    *
https://github.com/LeastAuthority/tahoe-lafs/blob/publish-error-extra-debugging/src/allmydata/mutable/publish.py#L878
      - respects servermap state via self.writers (initialised based on
      self.goal initialised via self._servermap.get_known_shares()
* LeastAuthority made a comic explaining tahoe:
  https://leastauthority.com/blog/least-authority-presents-a-new-comic/
  * warner wants something similar for magic-wormhole
* LAE spake2 grant: warner will connect with Watson Ladd to see how the
  RFC process works, loop him into the email thread
* meejah PRs 417 443: no-daemonize, pull config out of Node, refactor
  create-client/create-introducer
  * AppVeyor is sad because of long paths in the test suite
  * Possibly caused by tests changing working directory and not changing
    it back resulting in ever growing temp path names
  * Length limit is 260 on Windows for old-style paths (\\?\ prefix
    increases length limit to 32k)
  * daemonize does os.chdir(), expects that it won't ever return
  * meejah will look into wrapping all these tests with something that
    restores CWD
* basedir in Node should be a twisted FilePath
  * then all file access should be through a child of that FilePath,
    instead of manual os.path.join()
* adding underscore prefixes to private things
  * should _file.py contain _function too? or is the internal privacy
    implied by the file-level privacy?
  * tahoe as embedded library: we're changing the API now anyway
  * duplicity: wraps?
  * matador: simpson likes the idea of having a new namespace
    * the storage-client (daemon) approach lets him have an ambient
      tahoe client running in a kubernetes cluster, for use by all nodes
      * but that implies something about multiple grids vs one-true-grid
* "long-distance" filecaps
  * would it leak which grid you're a part of?
  * maybe model access as rights-amplification: you need two pieces to
    access a file
    * 1: access to a grid with a particular "area code" (grid-id)
    * 2: knowledge of a filecap with that grid-id prefix
  * then grids could choose other access control for the first part
  * or participate in a federation/directory-service system to allow
    anybody to access their grid
  *
*  Performance concerns
  * Profiling is hard
  * Client takes timing measurements
  * Could add an opt-in feature for sending timing information to a
    timing observer
  * Upload/download results object has a timing area in it that's
    probably foolscap copyable
  * If a timing server furl is provided to the client then it could send
    timing information for transfers up to that server
  * Maybe add a checkbox to GridSync for opting into this
  * With client-side-measured transfer timing information it might be
    more clear what complaints of "slow" mean.
  * Brian has lots of ideas about areas where there are possible
    improvements to be made
    * Various ideas about making better use of resources (eg transfer
      pipelining, better CPU/network alternation, etc)
* longer-term directions?
  * shim that speaks memcached as a frontend
  * tahoe-over-IPFS
  * software-as-a-service: should be storing data encrypted, usually
    arent
  * owncloud / RemoteStorage
*