由于某些ssl错误,新服务器无法从Puppetmaster获取configuration

生产环境中的三台机器出现了一些硬件问题,已经退役。 基础架构团队已经重新安装了它们,并给了它们相同的主机名和IP地址。 目标是在这些系统上运行Puppet,这样可以再次debugging。


尝试

1)通过发布以下命令将旧木偶证书从Puppetmaster中删除:

puppet cert revoke grb16.company.com puppet cert clean grb16.company.com 

2)删除旧证书后,通过从其中一个重新安装的节点发出以下命令,创build新的证书请求:

 [root@grb16 ~]# puppet agent -t Info: csr_attributes file loading from /etc/puppet/csr_attributes.yaml Info: Creating a new SSL certificate request for grb16.company.com Info: Certificate Request fingerprint (SHA256): 6F:2D:1D:71:67:18:99:86:2C:22:A1:14:80:55:34:35:FD:20:88:1F:36:ED:A7:7B:2A:12:09:4D:F8:EC:BF:6D Exiting; no certificate found and waitforcert is disabled [root@grb16 ~]# 

3)一旦证书请求在Puppetmaster上可见,就会发出以下命令来签署证书请求:

 [root@foreman ~]# puppet cert sign grb16.company.com Notice: Signed certificate request for grb16.company.com Notice: Removing file Puppet::SSL::CertificateRequest grb16.company.com at '/var/lib/puppet/ssl/ca/requests/grb16.company.com.pem' [root@foreman ~]# 

问题

一旦证书请求已经被签名并且一个Puppet运行已经被启动,就会引发下面的错误:

 [root@grb16 ~]# puppet agent -t Info: Caching certificate for grb16.company.com Error: Could not request certificate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] Exiting; failed to retrieve certificate and waitforcert is disabled [root@grb16 ~]# 

第二次运行Puppet会导致:

 [root@grb16 ~]# puppet agent -t Warning: Unable to fetch my node definition, but the agent run will continue: Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] Info: Retrieving pluginfacts Error: /File[/var/lib/puppet/facts.d]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] Error: /File[/var/lib/puppet/facts.d]: Could not evaluate: Could not retrieve file metadata for puppet://foreman.company.com/pluginfacts: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] Wrapped exception: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] Info: Retrieving plugin Error: /File[/var/lib/puppet/lib]: Failed to generate additional resources using 'eval_generate': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] Error: /File[/var/lib/puppet/lib]: Could not evaluate: Could not retrieve file metadata for puppet://foreman.company.com/plugins: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] Wrapped exception: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] Error: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] Warning: Not using cache on failed catalog Error: Could not retrieve catalog; skipping run Error: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [CRL is not yet valid for /CN=Puppet CA: foreman.company.com] [root@grb16 ~]# 

分析

为了解决这个问题,错误消息被调查,看起来问题是SSL或Puppet相关。 也许其中一个软件包安装不正确,或者在重新安装的节点上安装了错误的版本。

木偶

 [root@grb16 ~]# yum list installed |grep puppet facter.x86_64 1:2.3.0-1.el6 @puppetlabs_6_products hiera.noarch 1.3.4-1.el6 @puppetlabs_6_products puppet.noarch 3.7.3-1.el6 @puppetlabs_6_products puppetlabs-release.noarch 6-11 @puppetlabs_6_products ruby-augeas.x86_64 0.4.1-3.el6 @puppetlabs_6_deps ruby-shadow.x86_64 1:2.2.0-2.el6 @puppetlabs_6_deps rubygem-json.x86_64 1.5.5-3.el6 @puppetlabs_6_deps 

SSL

 [root@grb16 ~]# yum list installed |grep ssl nss_compat_ossl.x86_64 0.9.6-1.el6 @anaconda-CentOS-201410241409.x86_64/6.6 openssl.x86_64 1.0.1e-30.el6_6.4 openssl-devel.x86_64 1.0.1e-30.el6_6.4 [root@grb16 ~]# 

在各种服务器上安装的SSL和Puppet软件包之间没有发现差异。 尚未退役或重新安装的系统仍然可以运行Puppet。 该问题仅限于重新安装的服务器。 请注意,Puppet还没有在其他两个重新安装的服务器上运行。 什么是造成这个问题,以及如何解决这个问题?

简洁的回答

CRL is not yet valid for的问题CRL is not yet valid for表明Puppet-agent和Puppetmaster之间的时间不同步 。 同步时间(NTP)。 从Puppet-agent和Puppetmaster中删除证书,然后在代理上运行Puppet。


综合答案

CRL is not yet valid for在以下代码段中CRL is not yet valid for

以下testing代码片段描述了导致此问题的原因:

 it 'includes the CRL issuer in the verify error message' do crl = OpenSSL::X509::CRL.new crl.issuer = OpenSSL::X509::Name.new([['CN','Puppet CA: puppetmaster.example.com']]) crl.last_update = Time.now + 24 * 60 * 60 ssl_context.stubs(:current_crl).returns(crl) subject.call(false, ssl_context) expect(subject.verify_errors).to eq(["CRL is not yet valid for /CN=Puppet CA: puppetmaster.example.com"]) end 

ssl_context

 let(:ssl_context) do mock('OpenSSL::X509::StoreContext') end 

学科

 subject do described_class.new(ssl_configuration, ssl_host) end 

代码包含来自OpenSSL :: X509 :: CRL类的片段。

发行者=(P1)

  static VALUE ossl_x509crl_set_issuer(VALUE self, VALUE issuer) { X509_CRL *crl; GetX509CRL(self, crl); if (!X509_CRL_set_issuer_name(crl, GetX509NamePtr(issuer))) { /* DUPs name */ ossl_raise(eX509CRLError, NULL); } return issuer; } 

LAST_UPDATE =(P1)

  static VALUE ossl_x509crl_set_last_update(VALUE self, VALUE time) { X509_CRL *crl; time_t sec; sec = time_to_time_t(time); GetX509CRL(self, crl); if (!X509_time_adj(crl->crl->lastUpdate, 0, &sec)) { ossl_raise(eX509CRLError, NULL); } return time; } 

last_updated时间将是当前时间加上一个额外的一天,并将传递给调用驻留在default_validator类中的调用函数的主题函数。

 class Puppet::SSL::Validator::DefaultValidator #< class Puppet::SSL::Validator attr_reader :peer_certs attr_reader :verify_errors attr_reader :ssl_configuration FIVE_MINUTES_AS_SECONDS = 5 * 60 def initialize( ssl_configuration = Puppet::SSL::Configuration.new( Puppet[:localcacert], { :ca_auth_file => Puppet[:ssl_client_ca_auth] }), ssl_host = Puppet::SSL::Host.localhost) reset! @ssl_configuration = ssl_configuration @ssl_host = ssl_host end def call(preverify_ok, store_context) if preverify_ok ... else ... crl = store_context.current_crl if crl if crl.last_update && crl.last_update < Time.now + FIVE_MINUTES_AS_SECONDS ... else @verify_errors << "#{error_string} for #{crl.issuer}" end ... end end end 

如果preverify_ok为false,则else子句适用。 就if crl.last_update && crl.last_update < Time.now + FIVE_MINUTES_AS_SECONDS结果是错误的,因为时间已被删除了一个额外的日子else语句将适用。 @verify_errors << "#{error_string} for #{crl.issuer}"@verify_errors << "#{error_string} for #{crl.issuer}"的评估结果在CRL is not yet valid for /CN=Puppet CA: puppetmaster.example.com

为了解决这个问题:

  1. 同步Puppet-agent和Puppetmaster之间的时间。 NTP服务器在两个节点上运行(良好)吗?
  2. 从代理中删除或重命名完整的ssl文件夹( / var / lib / puppet / ssl )。
  3. 通过发行sudo puppet cert clean <fqdn-puppet-agent>
  4. 如果自动签名被禁用,请签署证书
  5. 在代理上运行木偶

总之,木偶代理和木偶大师的时间应该一直同步。 超过最大允许偏差5分钟将导致问题。

跑到同一个问题。

我们的木偶设置是使用GitHub进行版本控制的,所以每次我们提供一个新的木偶大师,我们都会遇到证书问题。 通常puppet ca --clean --all工作,但我们发现以下更可靠:

 rm -rf $(puppet master --configprint ssldir)