#283 new enhancement

implement mutable-file upload/download helpers

Reported by: warner Owned by:
Priority: major Milestone: undecided
Component: code-mutable Version: 0.7.0
Keywords: mutable upload-helper upload download Cc:
Launchpad Bug:

Description (last modified by zooko)

In addition to the CHK-upload helper (#116) and the CHK-download helper (#268), we could use a helper for mutable files. This helper would allow clients to read and write directories without talking to all of the storage servers.

If all four helpers are implemented, then we can change the introducer configuration to isolate the clients from the storage servers. If any of these helpers are not implemented, the clients must continue to talk to all the storage servers.

This ticket is specifically about the mutable-file upload/download helper.

Change History (7)

comment:1 Changed at 2008-04-24T23:46:43Z by warner

  • Component changed from code-encoding to code-mutable

comment:2 Changed at 2008-06-01T20:43:21Z by warner

  • Milestone changed from eventually to undecided

comment:3 Changed at 2009-12-20T23:14:57Z by zooko

Perhaps this could be part of the solution to #868 (use helper to bypass NAT/firewall).

comment:4 Changed at 2010-01-07T00:20:41Z by davidsarah

  • Keywords upload download added

comment:5 Changed at 2010-07-21T16:35:12Z by zooko

But despite its potential use for #868, I'm -1 on this ticket. I think of the existence of the current immutable file upload erasure-coding helper as doubling the engineering costs of improving immutable upload. Inventing helpers for mutable files and for download would multiply this problem. On the other hand, I'm pretty enthusiastic about tickets like #993 (refactor download interfaces to treat immutable files and mutable versions more uniformly), which offer to reduce the number of codebases that we're maintaining and improving.

I recognize that immutable upload helper offers a performance benefit that the current immutable file upload code doesn't. I'm not proposing to kill it, which would be a regression for people who use that performance benefit. But I hope that we'll improve the normal upload, repair, and rebalancing features to such a point that it becomes better for users to rely on those features and they stop needing the upload helper.

Ways that we can improve those things: better and more customizable share placement, #610 (upload should take better advantage of existing shares), #809 (Measure how segment size affects upload/download speed.), #873 (upload: tolerate lost or missing servers), #1130 (Failure to achieve happiness in upload).

repair-and-rebalance behavior: #232 (Peer selection doesn't rebalance shares on overwrite of mutable file.), #483 (repairer service), #450 (checker / repair agent), #543 ('rebalancing manager'), #643 (Automatically schedule repair process), #699 (rebalance during repair or upload), #711 (repair to different levels of M), #1004 (how to fix 'multiple versions are recoverable'?).

See also the discussion on tahoe-dev that prompted me to write this comment.

comment:6 Changed at 2015-08-16T15:17:09Z by zooko

  • Description modified (diff)
  • Keywords upload-helper added; helper removed

comment:7 Changed at 2015-10-07T23:29:37Z by lpirl

An implied limitation seems to be #2403 (daira pointed that out): when the helper is the only node that sees all other nodes (star topology), clients of that helper fail on a lot of commands since they can neither read nor write (enough shares of) directories.

Note: See TracTickets for help on using tickets.