ImportError: No module named setuptools_darcs.setuptools_darcs

[zooko]

On the test-clean builder: ​http://tahoe-lafs.org/buildbot/builders/clean/builds/2303/steps/test-desert-island/logs/stdio

Traceback (most recent call last):
  File "setup.py", line 370, in <module>
    **setup_args
  File "/usr/local/lib/python2.6/distutils/core.py", line 113, in setup
    _setup_distribution = dist = klass(attrs)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/dist.py", line 260, in __init__
    self.fetch_build_eggs(attrs.pop('setup_requires'))
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/dist.py", line 284, in fetch_build_eggs
    parse_requirements(requires), installer=self.fetch_build_egg
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/pkg_resources.py", line 563, in resolve
    dist = best[req.key] = env.best_match(req, self, installer)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/pkg_resources.py", line 799, in best_match
    return self.obtain(req, installer) # try and download/install
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/pkg_resources.py", line 811, in obtain
    return installer(requirement)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/dist.py", line 327, in fetch_build_egg
    return cmd.easy_install(req)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 452, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 482, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 661, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 936, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 925, in run_setup
    run_setup(setup_script, args)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/sandbox.py", line 62, in run_setup
    lambda: execfile(
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/sandbox.py", line 105, in run
    return func()
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/sandbox.py", line 64, in <lambda>
    {'__file__':setup_script, '__name__':'__main__'}
  File "setup.py", line 79, in <module>
    "Topic :: Utilities",
  File "/usr/local/lib/python2.6/distutils/core.py", line 113, in setup
    _setup_distribution = dist = klass(attrs)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/dist.py", line 260, in __init__
    self.fetch_build_eggs(attrs.pop('setup_requires'))
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/dist.py", line 284, in fetch_build_eggs
    parse_requirements(requires), installer=self.fetch_build_egg
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/pkg_resources.py", line 563, in resolve
    dist = best[req.key] = env.best_match(req, self, installer)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/pkg_resources.py", line 799, in best_match
    return self.obtain(req, installer) # try and download/install
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/pkg_resources.py", line 811, in obtain
    return installer(requirement)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/dist.py", line 327, in fetch_build_egg
    return cmd.easy_install(req)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 452, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 482, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 661, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 936, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/easy_install.py", line 925, in run_setup
    run_setup(setup_script, args)
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/sandbox.py", line 62, in run_setup
    lambda: execfile(
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/sandbox.py", line 105, in run
    return func()
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/sandbox.py", line 64, in <lambda>
    {'__file__':setup_script, '__name__':'__main__'}
  File "setup.py", line 78, in <module>
    "Programming Language :: Python :: 2.6",
  File "/usr/local/lib/python2.6/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/lib/python2.6/distutils/dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/bdist_egg.py", line 167, in run
    self.run_command("egg_info")
  File "/usr/local/lib/python2.6/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/egg_info.py", line 177, in run
    self.find_sources()
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/egg_info.py", line 252, in find_sources
    mm.run()
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/egg_info.py", line 306, in run
    self.add_defaults()
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/egg_info.py", line 333, in add_defaults
    rcfiles = list(walk_revctrl())
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/setuptools/command/sdist.py", line 45, in walk_revctrl
    for item in ep.load()(dirname):
  File "/home/buildbot/tahoe-lafs/clean/build/setuptools-0.6c15dev.egg/pkg_resources.py", line 1954, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
ImportError: No module named setuptools_darcs.setuptools_darcs

The attached patch works-around this by commenting-out the requirement to have setuptools_darcs at build-time. The drawback of this approach is that if someone builds a distribution, e.g. a source distribution by running python setup.py sdist, then they have to manually install setuptools_darcs first. If they don't, then the resulting distribution will be incomplete -- it will be missing all files that aren't part of the allmydata Python package-directory, such as docs and .xhtml files for the WUI.

[zooko]

I get this error if I try to install Tahoe-LAFS using distribute-0.6.14, like so:

$ sudo easy_install allmydata-tahoe

…

install_dir /Library/Python/2.6/site-packages/
Processing 
Running setup.py -q bdist_egg --dist-dir /Users/zooko/playground/tahoe-lafs/pristine/egg-dist-tmp-ZD_wTc
install_dir .
Checking .pth file support in .
/usr/bin/python -E -c pass
Not found: tahoe-deps
Not found: ../tahoe-deps
Searching for setuptools-darcs>=1.1.0
Reading http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-dep-sdists/
Reading http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/
Best match: setuptools-darcs 1.2.11
Downloading http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/setuptools_darcs-1.2.11.tar.bz2
Processing setuptools_darcs-1.2.11.tar.bz2
Running setuptools_darcs-1.2.11/setup.py -q bdist_egg --dist-dir /Users/zooko/playground/tahoe-lafs/pristine/temp/easy_install-LQp44p/setuptools_darcs-1.2.11/egg-dist-tmp-XW6U6z
install_dir .
Checking .pth file support in .
/usr/bin/python -E -c pass
Searching for darcsver>=1.2.0
Best match: darcsver 1.7.1
Processing darcsver-1.7.1.egg

Using /Users/zooko/playground/tahoe-lafs/pristine/darcsver-1.7.1.egg
setup.py darcsver: I'm unable to find a tag in the darcs history matching "^TAG setuptools_darcs-((\d+)(\.(\d+)(\.(\d+))?)?((a|b|c)(\d+))?(\.dev(\d+))?)$", so I'm leaving ['setuptools_darcs/_version.py'
] alone.
Removing allmydata-tahoe 1.8.1-r4928 from easy-install.pth file
Adding allmydata-tahoe 1.8.1-r4914 to easy-install.pth file
Installing tahoe script to /usr/local/bin

Installed /Library/Python/2.6/site-packages/allmydata_tahoe-1.8.1_r4914-py2.6.egg
Processing dependencies for allmydata-tahoe==1.8.1-r4914
Searching for pycryptopp>=0.5.20
Reading http://pypi.python.org/simple/pycryptopp/
Reading http://allmydata.org/trac/pycryptopp
Reading http://allmydata.org/source/pycryptopp
Reading http://allmydata.org/source/zfec
Best match: pycryptopp 0.5.28
Downloading http://pypi.python.org/packages/source/p/pycryptopp/pycryptopp-0.5.28.tar.gz#md5=829a211dd1b54eb26f9879f17f162a3e
Processing pycryptopp-0.5.28.tar.gz
Running pycryptopp-0.5.28/setup.py -q bdist_egg --dist-dir /tmp/easy_install-VOMDTv/pycryptopp-0.5.28/egg-dist-tmp-o7UYqM
Traceback (most recent call last):
  File "/usr/local/bin/easy_install", line 9, in <module>
    load_entry_point('distribute==0.6.14', 'console_scripts', 'easy_install')()
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 1855, in main
    with_ei_usage(lambda:
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 1836, in with_ei_usage
    return f()
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 1859, in <lambda>
    distclass=DistributionWithoutHelpCommands, **kw
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 975, in run_commands
    cmd_obj.run()
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 342, in run
    self.easy_install(spec, not self.no_deps)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 562, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 614, in install_item
    self.process_distribution(spec, dist, deps)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 666, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/pkg_resources.py", line 544, in resolve
    dist = best[req.key] = env.best_match(req, self, installer)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/pkg_resources.py", line 786, in best_match
    return self.obtain(req, installer) # try and download/install
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/pkg_resources.py", line 798, in obtain
    return installer(requirement)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 582, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 612, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 802, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 1079, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/easy_install.py", line 1068, in run_setup
    run_setup(setup_script, args)
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/sandbox.py", line 29, in run_setup
    lambda: execfile(
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/sandbox.py", line 70, in run
    return func()
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/sandbox.py", line 31, in <lambda>
    {'__file__':setup_script, '__name__':'__main__'}
  File "setup.py", line 322, in <module>
    
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/bdist_egg.py", line 172, in run
    self.run_command("egg_info")
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 995, in run_command
    cmd_obj.run()
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/egg_info.py", line 179, in run
    self.find_sources()
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/egg_info.py", line 254, in find_sources
    mm.run()
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/egg_info.py", line 308, in run
    self.add_defaults()
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/egg_info.py", line 335, in add_defaults
    rcfiles = list(walk_revctrl())
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/setuptools/command/sdist.py", line 46, in walk_revctrl
    for item in ep.load()(dirname):
  File "/Library/Python/2.6/site-packages/distribute-0.6.14-py2.6.egg/pkg_resources.py", line 1954, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
ImportError: No module named setuptools_darcs.setuptools_darcs

This is caused by a bug in Distribute: ​https://bitbucket.org/tarek/distribute/issue/55/revision-control-plugin-automatically-installed-as-a-build-dependency-is-not-present-when-another-build-dependency-is-being

[zooko]

This patch is needed to fix #668 (easy_install allmydata-tahoe is broken).

[davidsarah]

Will review tomorrow.

[zooko]

Here is an alternative which avoids the failure mode of people building broken distributions at the cost of bundling a copy of setuptools_darcs with Tahoe-LAFS.

[davidsarah]

Looks good. I'll combine the first and third patches when pushing this to trunk.

[zooko@…]

In 0fb49375c0365e0a:

setup: bundle a copy of setuptools_darcs-1.2.12
This is to work-around ​https://bitbucket.org/tarek/distribute/issue/55/revision-control-plugin-automatically-installed-as-a-build-dependency-is-not-present-when-another-build-dependency-is-being . refs #1054.

[david-sarah@…]

In 4cba75e000ba1b16:

(The changeset message doesn't reference this ticket)

[davidsarah]

This error is still happening in this install-to-egg step:

​http://tahoe-lafs.org/buildbot/builders/Josh%20ubuntu-amd64%20laptop/builds/26/steps/install-to-egg/logs/stdio

which is a build of r4941, after the above two patches.

It appears from the line

Using /tmp/easy_install-5VhQya/zfec-1.4.15/temp/easy_install-UdokMi/setuptools_trial-0.5.10/setuptools_darcs-1.2.11.egg

that this may have something to do with setuptools_trial. But setuptools_trial is going away if #1296 is fixed, so maybe this error will too.

[zooko]

The error on the Josh ubuntu-amd64 has gone away now that I uploaded a new version of setuptools_trial ​setuptools_trial-0.5.12, which doesn't require setuptools_darcs.