Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vendored Bundler causes LoadError when running bundle exec from ruby script after bundler/setup #2055

Closed
tf opened this issue Nov 3, 2017 · 10 comments

Comments

@tf
Copy link

tf commented Nov 3, 2017

See rubygems/bundler#6144

Problem

Trying to run the following script when gems have been installed with bundle install --deployment

# script.rb
require 'bundler/setup'
system('bundle exec whatever')

fails with

$ ruby ./script.rb
/home/travis/.rvm/gems/ruby-2.4.1/bin/bundle:23:in `load': cannot load such file -- /home/travis/.rvm/rubies/ruby-2.4.1/lib/ruby/site_ruby/exe/bundle (LoadError)

Minimal example:

https://github.com/tf/bundler-executable-not-found
https://travis-ci.org/tf/bundler-executable-not-found/builds/296324791

If Gem.activate_bin_path('bundler', 'bundle', '>= 0.a') (which is what is loaded in the bundle executable) is called for the first time after bundler/setup, it reports the missing path .rvm/rubies/ruby-2.1.7/lib/ruby/site_ruby/exe/bundle. If it is called before bundler/setup, the correct value is already cached. The above Travis build contains examples of both cases for different Ruby versions.

In my case this causes all invocations of the form system('bundler exec ...') to fail that come from a binstub since binstubs include the bundler/setup line.

Cause

I looked at the code and here's what is happening:

When bundler/setup is loaded, it patches Rubygems and overrides activate_bin_path to return the contents of ENV["BUNDLE_BIN_PATH"] when we are looking for the bin path of the bundler executable:

https://github.com/bundler/bundler/blob/d17f2e43236ba53f05b70e7317d218ba94e293c8/lib/bundler/rubygems_integration.rb#L472-L483

This environment variable is set by first trying to look up the bin path via rubygems, but falls back to a fixed relative path to Bundler's exe directory:

https://github.com/bundler/bundler/blob/d17f2e43236ba53f05b70e7317d218ba94e293c8/lib/bundler/shared_helpers.rb#L294-L300

When gems have been installed with bundle install --deployment, this rescue case kicks in, because apparently Bundler does not include its own sources in the vendor/bundle directory.

In the presence of Rubygems 2.7, shared_helpers.rb is loaded from .rvm/rubies/ruby-2.3.1/lib/ruby/site_ruby/2.3.0/bundler/shared_helpers.rb. But there simply is no ../../../exe/bundle directory in the vendored version of Bundler.

System Info

This issue is related to:

  • The vendored version of Bundler
  • The command line gem
  • Other

Here are my current environment details:

$ gem env version
RubyGems Environment:
  - RUBYGEMS VERSION: 2.7.0
  - RUBY VERSION: 2.3.1 (2016-04-26 patchlevel 112) [x86_64-linux]
  - INSTALLATION DIRECTORY: /home/tfischbach/.rvm/gems/ruby-2.3.1
  - USER INSTALLATION DIRECTORY: /home/tfischbach/.gem/ruby/2.3.0
  - RUBY EXECUTABLE: /home/tfischbach/.rvm/rubies/ruby-2.3.1/bin/ruby
  - EXECUTABLE DIRECTORY: /home/tfischbach/.rvm/gems/ruby-2.3.1/bin
  - SPEC CACHE DIRECTORY: /home/tfischbach/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /etc
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86_64-linux
  - GEM PATHS:
     - /home/tfischbach/.rvm/gems/ruby-2.3.1
     - /home/tfischbach/.rvm/gems/ruby-2.3.1@global
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
     - :sources => ["http:https://rubygems.org/"]
     - "benchmark" => false
     - "gem" => "--no-ri --no-rdoc"
  - REMOTE SOURCES:
     - http:https://rubygems.org/
  - SHELL PATH:
     - /home/tfischbach/opt/apache-maven/bin
     - /media/tfischbach/c8eb1b66-54b4-48f2-8248-7b14aa37c9cc/data/adt-bundle-linux-x86_64-20131030/sdk/tools
     - /home/tfischbach/.nvm/versions/node/v6.9.5/bin
     - /home/tfischbach/bin
     - /home/tfischbach/.local/bin
     - /usr/lib/jvm/java-8-openjdk-amd64/bin
     - /home/tfischbach/.rvm/gems/ruby-2.3.1/bin
     - /home/tfischbach/.rvm/gems/ruby-2.3.1@global/bin
     - /home/tfischbach/.rvm/rubies/ruby-2.3.1/bin/
     - /home/tfischbach/opt/apache-maven/bin
     - /media/tfischbach/c8eb1b66-54b4-48f2-8248-7b14aa37c9cc/data/adt-bundle-linux-x86_64-20131030/sdk/tools
     - /home/tfischbach/bin
     - /home/tfischbach/.local/bin
     - /usr/local/sbin
     - /usr/local/bin
     - /usr/sbin
     - /usr/bin
     - /sbin
     - /bin
     - /usr/games
     - /usr/local/games
     - /snap/bin
     - /home/tfischbach/.rvm/bin
     - /home/tfischbach/bin
     - /snap/bin
     - /home/tfischbach/.rvm/bin
     - /home/tfischbach/.rvm/bin
     - /home/tfischbach/bin

I will abide by the code of conduct.

tf added a commit to tf/pageflow that referenced this issue Nov 3, 2017
Calling `bundle exec` from a ruby script after Bundler has been set up
fails for vendored Bundler version that comes with Rubygems 2.7.

  rubygems/rubygems#2055

`rake pageflow:dummy` invokes `bundle exec rails new`. Calling `rake`
via `bundle exec` can be used as a temporary workaround.
tf added a commit to tf/pageflow that referenced this issue Nov 3, 2017
Calling `bundle exec` from a ruby script after Bundler has been set up
fails for vendored Bundler version that comes with Rubygems 2.7.

  rubygems/rubygems#2055

`rake pageflow:dummy` invokes `bundle exec rails new`. Calling `rake`
via `bundle exec` can be used as a temporary workaround.
@eric-hu
Copy link

eric-hu commented Nov 8, 2017

I was able to reproduce this on my system with Ruby 2.3.3, Rubygems 2.7.0 and Bundler 1.16.0

Updated to the latest stable Rubygems (2.7.1) and the issue went away.

Edit: With Rubygems 2.7.1, I no longer see an error for the script.rb example above. However, I still see an error with bundle install.

@hsbt
Copy link
Member

hsbt commented Nov 30, 2017

@tf Can you try with RubyGems 2.7.3? I couldn't reproduce RG 2.7.3 with your reproducible repository.

see. https://travis-ci.org/hsbt/bundler-executable-not-found/builds/309343716

@tf
Copy link
Author

tf commented Dec 5, 2017

RubyGems 2.7.3 appears so have fixed the problem. I can also no longer reproduce the problem in the project where I originally encountered the issue.

@tf tf closed this as completed Dec 5, 2017
@hsbt
Copy link
Member

hsbt commented Dec 6, 2017

@tf Thanks for your confirmation and reporting this issue.

@sahilbathla
Copy link

The issue exists with RubyGems 2.7.3 & Bundler 1.16.1. Fixed if using Bundler 1.16.0

deanwilson added a commit to alphagov/govuk-puppet that referenced this issue Dec 22, 2017
There are currently issues with certain upstream version combinations:

  rubygems/bundler#6144
  rubygems/rubygems#2055

So don't do a manual update of the bundler gem until they are resolved
so we can continue to run travis builds in the meantime.
hugocorbucci added a commit to agile-alliance-brazil/identity that referenced this issue Dec 23, 2017
Unfortunately, rubygems/rubygems#2055 is
a bug that forces us to use bundler 1.16.0 instead of 1.16.1.
hugocorbucci added a commit to agile-alliance-brazil/certificates that referenced this issue Dec 28, 2017
This should help address rubygems/rubygems#2055
temporarily
hugocorbucci added a commit to agile-alliance-brazil/identity that referenced this issue Dec 29, 2017
hugocorbucci added a commit to agile-alliance-brazil/event_registrations that referenced this issue Dec 29, 2017
hugocorbucci added a commit to agile-alliance-brazil/submissions that referenced this issue Dec 29, 2017
Because of rubygems/rubygems#2055.
The rubocop new rules fixes for inverse_of everywhere.
Also preparing for a ruby 2.4.3 upgrade
mshibuya added a commit to carrierwaveuploader/carrierwave that referenced this issue Jan 2, 2018
rafa-3l pushed a commit to rafa-3l/pageflow that referenced this issue Jan 8, 2018
Calling `bundle exec` from a ruby script after Bundler has been set up
fails for vendored Bundler version that comes with Rubygems 2.7.

  rubygems/rubygems#2055

`rake pageflow:dummy` invokes `bundle exec rails new`. Calling `rake`
via `bundle exec` can be used as a temporary workaround.
@ms-ati
Copy link
Contributor

ms-ati commented Jan 12, 2018

@tf Based on the fact that this problem still exists in Travis CI, and therefore many projects are pinning Bundler to 1.16.0 in CI, is there a disconnect between the way you are failing to reproduce, and what all these projects are seeing? Should this issue be re-opened until all the projects see it as resolved?

@hsbt
Copy link
Member

hsbt commented Jan 12, 2018

It's the environment of Travis issue. not rubygems maybe.

https://travis-ci.org/colszowka/simplecov/jobs/326822785

I investigate above log. Can you try to use gem install bundler --force?

It's complex problem with rubygems, bundler, rvm, and travis on https://github.com/travis-ci/travis-rubies

@PragTob
Copy link

PragTob commented Jan 13, 2018

👋

The travis cache might be to blame. gem install bundler --force did not fix it for us, but I upgraded a few rubies on the side and that seems to have fixed it. Unless all those rubies magically included fixes for it. That might also be, not entirely sure - I will know when I disable cache for funzies.

Build overview: https://travis-ci.org/colszowka/simplecov/builds/328432586?utm_source=github_status&utm_medium=notification

Ruby versions 2.2+ were updated.

edit: disabled cache, works now - one of the failures seems to be flaky but unrelated https://travis-ci.org/colszowka/simplecov/builds/328434182

@hsbt
Copy link
Member

hsbt commented Jan 13, 2018

@PragTob Great. Thanks for your investigation. I try to fix this issue with Travis together.

@PragTob
Copy link

PragTob commented Jan 13, 2018

@hsbt FYI I opened travis-ci/travis-ci#9062 with travis but maybe you got an insight. Thanks for your work! :)

cupakromer added a commit to RadiusNetworks/omniauth-radius-rails that referenced this issue May 11, 2018
The builds are failing due to a strange error where rake cannot find
other dependencies.

    bundler: failed to load command: rake
    LoadError: cannot load such file -- bundler/dep_proxy

It appears this is a known issue (travis-ci/travis-ci#9062) which links
to a Ruby Gem issue (rubygems/rubygems#2055).

This implements a hopefully simple fix which ensures bundler is updated
on the system, and that the installed version of Rails has the bundler
gem available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants