[tahoe-dev] [tahoe-lafs] #983: high CPU load on storage servers when uploading large mutable file

[tahoe-lafs]

#983: high CPU load on storage servers when uploading large mutable file
------------------------------+---------------------------------------------
     Reporter:  zooko         |       Owner:  somebody                  
         Type:  defect        |      Status:  new                       
     Priority:  major         |   Milestone:  undecided                 
    Component:  code-storage  |     Version:  1.6.0                     
   Resolution:                |    Keywords:  performance mutable upload
Launchpad_bug:                |  
------------------------------+---------------------------------------------

Comment(by zooko):

 !ChosenOne reproduced this problem on his Ubuntu Lucid (alpha) at my
 request. It really seems like on his system there is a superlinear cost
 (whether CPU cost or some other time-sink I'm not sure) to uploading
 mutable files.
 http://pastie.org/914448
 {{{
 400 MB

 real    23m3.963s
 user    0m0.030s
 sys     0m0.930s

 # Pushing: 16 minutes (412.1kBps)
 }}}
 http://pastie.org/private/c8zkyjea2ogn5so6dlxq
 {{{
 ## 400MB
    real 15m55.535s
    user 0m0.070s
    sys  0m0.510s
    # Pushing: 7 minutes (923.1kBps)

 ## 200MB
 1
    real 1m58.554s
    user 0m0.020s
    sys  0m0.400s
    # Pushing: 37 seconds (5.57MBps)

 2
    real 1m22.431s
    user 0m0.030s
    sys  0m0.400s
    # Pushing: 26 seconds (7.80MBps)
 3:
    real 1m2.264s
    user 0m0.020s
    sys  0m0.390s
    # Pushing: 15 seconds (13.65MBps)
 4:
    real 0m59.669s
    user 0m0.050s
    sys  0m0.320s
    # Pushing: 16 seconds (12.44MBps)
 5:
    real 1m3.264s
    user 0m0.020s
    sys  0m0.320s
    # Pushing: 18 seconds (11.15MBps)
 6:
    real 0m56.371s
    user 0m0.020s
    sys  0m0.340s
    # Pushing: 17 seconds (12.07MBps)
 7:
    real 1m4.337s
    user 0m0.060s
    sys  0m0.310s
    # Pushing: 21 seconds (9.85MBps)
 8:
    real 0m52.630s
    user 0m0.020s
    sys  0m0.350s
    # Pushing: 18 seconds (11.60MBps)
 9:
    real 0m49.644s
    user 0m0.000s
    sys  0m0.330s
    # Pushing: 17 seconds (11.88MBps)
 10:
    real 0m47.338s
    user 0m0.000s
    sys  0m0.310s
    # Pushing: 13 seconds (16.04MBps)

 }}}

-- 
Ticket URL: <http://allmydata.org/trac/tahoe/ticket/983#comment:10>
tahoe-lafs <http://allmydata.org>
secure decentralized file storage grid