Skip to content

/ octocatalog-diff

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

preserve-environments using r10k deployed environments not working #191

Open
walkleyn opened this issue Jun 13, 2018 · 3 comments
Open

preserve-environments using r10k deployed environments not working #191

walkleyn opened this issue Jun 13, 2018 · 3 comments

Comments

@walkleyn
Copy link

@walkleyn walkleyn commented Jun 13, 2018 •
edited

Description of problem

  • What did you do?
    I am trying to run octocatalog-diff on a puppetserver running puppet 4.10.12 with puppet environments managed by r10k in the following structure:
    `/etc/puppetlabs/puppet
    • environments
      • production
        • environment.conf
        • manifests
          • site.pp
        • modules
        • site
      • staging
        • environment.conf
        • manifests
          • site.pp
        • modules
        • site
      • testing_branch
        • environment.conf
        • manifests
          • site.pp
        • modules
        • site`
          I am trying to run octocatalog-diff against the checked out code using the --preserve-environments option as follows:
  • What happened?
    octocatalog-diff exits with an error code 1, expecting a basedir configuration specified, which isn't required as per documentation on advanced environments
  • What did you expect to happen?
    I expected octocatalog-diff to create and compare catalogues created from the two r10k deployed environments.
    Puppet agent runs against both environments work as expected.
  • How can someone reproduce the problem?
    Create two different working puppet environments and run octocatalog-diff as explained above

Command used and debugging output

octocatalog-diff -n [nodename] --bootstrapped-from-dir /etc/puppetlabs/puppet --bootstrapped-to-dir /etc/puppetlabs/puppet --from-environment production --to-environment testing_branch --preserve-environments
D, [2018-06-13T17:07:41.649263 #24328] DEBUG -- : Running octocatalog-diff 1.5.3 with ruby 2.1.5 D, [2018-06-13T17:07:41.649465 #24328] DEBUG -- : Command line arguments: ["-n", "node_name", "--bootstrapped-from-dir", "/etc/puppetlabs/puppet", "--bootstrapped-to-dir", "/etc/puppetlabs/puppet", "--from-environment", "production", "--to-environment", "testing_branch", "--preserve-environments", "-d"] D, [2018-06-13T17:07:41.649552 #24328] DEBUG -- : Running on host puppetserver_name (x86_64-linux-gnu) /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/git.rb:63:in fetch': key not found: :basedir (KeyError)
from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/git.rb:63:in branch_sha' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/cached_master_directory.rb:112:in git_repo_checkout_bootstrap'
from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/cached_master_directory.rb:49:in run' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/cli.rb:106:in cli'
from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/bin/octocatalog-diff:34:in <top (required)>' from /usr/local/bin/octocatalog-diff:23:in load'
from /usr/local/bin/octocatalog-diff:23:in <main>'

Platform and version information

  • Your OS: Debian Linux 8 (Jessie)
  • Your Ruby version: ruby 2.1.5p273 (2014-11-13) [x86_64-linux-gnu]
  • Your version of Puppet: puppet 4.10.12 OSE
  • Your version of octocatalog-diff: octocatalog-diff 1.5.3

Do the tests pass from a clean checkout?
yes

Anything else to add that you think will be helpful?
When I add the -f and -t options, the catalog compiles, but then I get errors about custom providers not found:
command:
octocatalog-diff -n node_name --bootstrapped-from-dir /etc/puppetlabs/puppet/environments/production --bootstrapped-to-dir /etc/puppetlabs/puppet/environments/testing_branch --from-environment production --to-environment testing_branch -f production -t testing_branch -d
catalog error:
Error: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper Error: Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper at /tmp/ocd-ipc-20180613-28056-1iximb4/ocd-builddir-20180613-28061-30pfww/environments/testing_branch/modules/monolith/quotaman/manifests/config.pp:80:3 on node quotaman-staging3.cpt3.host-h.net Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper at /tmp/ocd-ipc-20180613-28056-1iximb4/ocd-builddir-20180613-28061-30pfww/environments/testing_branch/modules/monolith/quotaman/manifests/config.pp:80:3 on node quotaman-staging3.cpt3.host-h.net Error: Failed to compile catalog for node quotaman-staging3.cpt3.host-h.net: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/network_config: Could not autoload puppet/provider/network_config/interfaces: cannot load such file -- puppetx/filemapper at /tmp/ocd-ipc-20180613-28056-1iximb4/ocd-builddir-20180613-28061-30pfww/environments/testing_branch/modules/monolith/quotaman/manifests/config.pp:80:3 on node node_name
@vinzent
Copy link
Contributor

@vinzent vinzent commented Jun 13, 2018

@walkleyn can you paste the complete commandline you ouse and the complete output of octocatalog-diff?
@walkleyn
Copy link
Author

@walkleyn walkleyn commented Jun 13, 2018

@vinzent here you go:
octocatalog-diff -n [nodename] --bootstrapped-from-dir /etc/puppetlabs/puppet --bootstrapped-to-dir /etc/puppetlabs/puppet --from-environment production --to-environment testing_branch --preserve-environments
D, [2018-06-13T17:07:41.649263 #24328] DEBUG -- : Running octocatalog-diff 1.5.3 with ruby 2.1.5 D, [2018-06-13T17:07:41.649465 #24328] DEBUG -- : Command line arguments: ["-n", "node_name", "--bootstrapped-from-dir", "/etc/puppetlabs/puppet", "--bootstrapped-to-dir", "/etc/puppetlabs/puppet", "--from-environment", "production", "--to-environment", "testing_branch", "--preserve-environments", "-d"] D, [2018-06-13T17:07:41.649552 #24328] DEBUG -- : Running on host puppetserver_name (x86_64-linux-gnu) /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/git.rb:63:in fetch': key not found: :basedir (KeyError)
from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/git.rb:63:in branch_sha' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/cached_master_directory.rb:112:in git_repo_checkout_bootstrap'
from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/catalog-util/cached_master_directory.rb:49:in run' from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/lib/octocatalog-diff/cli.rb:106:in cli'
from /var/lib/gems/2.1.0/gems/octocatalog-diff-1.5.3/bin/octocatalog-diff:34:in <top (required)>' from /usr/local/bin/octocatalog-diff:23:in load'
from /usr/local/bin/octocatalog-diff:23:in <main>'
@xakraz
Copy link
Contributor

@xakraz xakraz commented Sep 10, 2018

Hello !

We are facing the same issue with the elastic_search module (https://github.com/elastic/puppet-elasticsearch) v6.3.0 ..

Error: Failed to compile catalog for node xxx9-8.domain.io: 
Evaluation Error: Error while evaluating a Virtual Query, Could not autoload puppet/type/elasticsearch_user_file: 
Could not autoload puppet/provider/elasticsearch_user_file/xpack: 
cannot load such file -- puppet/provider/elastic_parsedfile at /tmp/ocd-ipc-20180910-186-kdxjnr/ocd-builddir-20180910-191-g6wb0/environments/production/modules/elasticsearch/manifests/init.pp:545:6 on node xxx9-8.domain.io

CMD

cmd: octocatalog-diff \
  --retry-failed-catalog 1 -o /dev/null \
  --hostnamexxx9-8.domain.io \
  --command_line --certname\=xxxx \
  --command_line --confdir\=/home/xxxx/.puppetlabs/etc/puppet \
  --catalog-only --debug

Platform and version information

Your OS: Debian Linux 8 (Jessie)
Your Ruby version: ruby 2.4.4
Your version of Puppet: puppet 4.10.12 OSE
Your version of octocatalog-diff: octocatalog-diff 1.5.3

I had to do ugly things to work around:

$ git diff modules/elasticsearch/
diff --git a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/elasticsearch_users.rb b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/elasticsearch_users.rb
index 2acf877369..af6af98375 100644
--- a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/elasticsearch_users.rb
+++ b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/elasticsearch_users.rb
@@ -1,4 +1,5 @@
-require 'puppet/provider/elastic_user_command'
+#require 'puppet/provider/elastic_user_command'
+require File.join(File.dirname(__FILE__), '..','..','..', '..', 'lib/puppet/provider/elastic_user_command')
 
 Puppet::Type.type(:elasticsearch_user).provide(
   :elasticsearch_users,
diff --git a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/esusers.rb b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/esusers.rb
index f900f51cfd..983629f402 100644
--- a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/esusers.rb
+++ b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/esusers.rb
@@ -1,4 +1,5 @@
-require 'puppet/provider/elastic_user_command'
+#require 'puppet/provider/elastic_user_command'
+require File.join(File.dirname(__FILE__), '..','..','..', '..', 'lib/puppet/provider/elastic_user_command')
 
 Puppet::Type.type(:elasticsearch_user).provide(
   :esusers,
diff --git a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/users.rb b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/users.rb
index d078a48cee..e6122eae76 100644
--- a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/users.rb
+++ b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user/users.rb
@@ -1,4 +1,5 @@
-require 'puppet/provider/elastic_user_command'
+#require 'puppet/provider/elastic_user_command'
+require File.join(File.dirname(__FILE__), '..','..','..', '..', 'lib/puppet/provider/elastic_user_command')
 
 Puppet::Type.type(:elasticsearch_user).provide(
   :users,
diff --git a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user_file/oss_xpack.rb b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user_file/oss_xpack.rb
index a160bea7de..23fb0f30f7 100644
--- a/modules/elasticsearch/lib/puppet/provider/elasticsearch_user_file/oss_xpack.rb
+++ b/modules/elasticsearch/lib/puppet/provider/elasticsearch_user_file/oss_xpack.rb
@@ -1,4 +1,5 @@
-require 'puppet/provider/elastic_parsedfile'
+#require 'puppet/provider/elastic_parsedfile'
+require File.join(File.dirname(__FILE__), '..','..','..', '..', 'lib/puppet/provider/elastic_parsedfile')
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
@vinzent @xakraz @walkleyn
You can’t perform that action at this time.