Opened at 2008-12-05T07:18:13Z
Last modified at 2013-11-01T17:17:53Z
#549 assigned defect
MODE_WRITE mapupdate: maybe increase epsilon to handle large batches of new servers better
| Reported by: | warner | Owned by: | bazuka |
|---|---|---|---|
| Priority: | major | Milestone: | soon |
| Component: | code-mutable | Version: | 1.2.0 |
| Keywords: | availability upload easy mutable ucwe test-needed | Cc: | |
| Launchpad Bug: |
Description (last modified by bazuka)
Part of the problem described in #546 is due to adding a large number of new servers. This inserts large gaps in the permuted server list, and the mapupdate code that handles MODE_WRITE may not look far enough to get past the gaps.
It might be a good idea to increase the "epsilon" used for MODE_WRITE from its current value of "k" (usually 3) to "N" (usually 10). As noted at the bottom of #546, if all the servers are writeable, this should be enough to make sure we don't need to send any shares to unqueried servers.
Change History (14)
comment:1 Changed at 2010-03-25T01:14:59Z by davidsarah
- Keywords availability upload easy added
- Milestone changed from undecided to 1.7.0
comment:2 Changed at 2010-05-27T22:09:07Z by zooko
- Milestone changed from 1.7.0 to 1.8.0
comment:3 Changed at 2010-08-10T04:10:09Z by davidsarah
- Milestone changed from 1.8.0 to soon
comment:4 Changed at 2011-08-17T04:56:23Z by davidsarah
- Keywords mutable ucwe added
comment:5 Changed at 2013-11-01T08:24:53Z by bazuka
- Description modified (diff)
- Owner set to bazuka
comment:6 Changed at 2013-11-01T08:50:59Z by bazuka
- Status changed from new to assigned
comment:7 Changed at 2013-11-01T08:53:36Z by bazuka
review-needed
comment:8 Changed at 2013-11-01T08:56:29Z by bazuka
comment:9 Changed at 2013-11-01T08:58:41Z by bazuka
Keywords review-needed
comment:10 Changed at 2013-11-01T12:08:51Z by daira
- Keywords test-needed added
The name EPSILON implies that it's a constant. If it's no longer a constant then I think the attribute is no longer needed.
Also a test would be needed. If you're not sure how to write one, please ask us on IRC.
comment:11 follow-up: ↓ 12 Changed at 2013-11-01T12:21:15Z by daira
Hmm, it appears that self.EPSILON is already not a constant (and there's some dubious code around src/allmydata/mutable/servermap.py#L491 that sets k and N based on assuming encoding defaults of 3/10, which I think should never be reached).
comment:12 in reply to: ↑ 11 Changed at 2013-11-01T17:02:55Z by zooko
Replying to daira:
Hmm, it appears that self.EPSILON is already not a constant (and there's some dubious code around src/allmydata/mutable/servermap.py#L491 that sets k and N based on assuming encoding defaults of 3/10, which I think should never be reached).
Yes, that code should be removed (to remove it is the goal of ticket #1847). That its absence would make it easier for us to evaluate this patch is a good example of why #1847 would be useful.
comment:13 follow-up: ↓ 14 Changed at 2013-11-01T17:04:27Z by zooko
Hey bazuka, how would you like to fix #1847 now? It doesn't require writing new unit tests! Remove the fallback code and make sure all the current unit tests stay green, and you're done.
It's really bothering me that mutable file upload and download behavior is so finicky, buggy, inefficient, hard to understand, different from immutable file upload and download behavior, etc. So I'm putting a bunch of tickets into the "1.8" Milestone. I am not, however, at this time, volunteering to work on these tickets, so it might be a mistake to put them into the 1.8 Milestone, but I really hope that someone else will volunteer or that I will decide to do it myself. :-)