rsyslog – 模板 – 正则expression式数据插入到数据库

在过去几天里,我一直在searchGoogle,寻找一个如何将所需数据的日志条目正则expression式然后插入数据库的可靠示例,但显然我的google-fu缺乏。

我试图做的是跟踪何时发送电子邮件,然后跟踪远程mta响应,特别是dsn代码。 在这一点上,我有两个模板为每种情况设置:

# /etc/rsyslog.conf ... $Template tpl_custom_header, "MPurcell: CUSTOM HEADER Template: %msg%\n" $Template tpl_response_dsn, "MPurcell: RESPONSE DSN Template: %msg%\n" # /etc/rsyslog.d/mail if $programname == 'mail-myapp' then /var/log/mail/myapp.log if ($programname == 'mail-myapp') and ($msg contains 'X-custom_header') then /var/log/mail/test.log;tpl_custom_header if ($programname == 'mail-myapp') and ($msg contains 'dsn=') then /var/log/mail/test.log;tpl_response_dsn & ~ 

示例日志条目:

 MPurcell: CUSTOM HEADER Template: D921940A1A: prepend: header X-custom_header: 101 from localhost[127.0.0.1]; from=<[email protected]> to=<[email protected]> proto=ESMTP helo=<localhost>: headername: message-id MPurcell: RESPONSE DSN Template: D921940A1A: to=<[email protected]>, relay=gmail-smtp-in.l.google.com[2607:f8b0:400e:c02::1a]:25, delay=2, delays=0.12/0.01/0.82/1.1, dsn=2.0.0, status=sent (250 2.0.0 OK 1372378600 o4si2828280pac.279 - gsmtp) 

从CUSTOM HEADER模板中,我想提取:D921940A1A和X-custom_header值; 101

从RESPONSE DSN模板我想提取:D921940A1A和“dsn = 2.0.0”

如果有人面临同样的情况,这就是我最终做的事情:

 # /etc/rsyslog.conf # Not sure what R signifies but saw it in other examples # ERE = extended regex # 0 = The submatch we want # DFLT = How should a non match be returned? $Template tpl_custom_header, "%msg:R,ERE,0,DFLT:[^:]+--end% | %msg:R,ERE,2,DFLT:X-custom_header:( )([0-9]*)--end%\n" $Template tpl_response_dsn, "%msg:R,ERE,0,DFLT:[^:]+--end% | %msg:R,ERE,1,DFLT:dsn=([0-9][.][0-9][.][0-9])--end% \n" 

要testing你的正则expression式,你应该使用: http ://www.rsyslog.com/regex/,这有点hokey,但完成工作。

原始日志条目示例,从OP略有更改:

 Jun 29 05:40:28 service1 mail-myapp/cleanup[22200]: 6F67240A1A: prepend: header X-custom_header: 136 from localhost[127.0.0.1]; from=<[email protected]> to=<[email protected]> proto=ESMTP helo=<localhost>: headername: message-id Jun 29 05:40:30 service1 mail-myapp/smtp[22201]: 6F67240A1A: to=<[email protected]>, relay=gmail-smtp-in.l.google.com[2607:f8b0:400e:c01::1a]:25, delay=2, delays=0.09/0/0.82/1, dsn=2.0.0, status=sent (250 2.0.0 OK 1372485254 rs6si5760686pbc.32 - gsmtp) 

模板应用后他们看起来像什么:

  6F67240A1A | 136 6F67240A1A | 2.0.0 

当我插入到MySQL中,我将插入dsn作为int vsstring以获得更好的性能,所以想用这个:

  insert into response_log_dsn set mail_id = '6F67240A1A', dsn = (select cast(replace('2.0.0', '.', '') as unsigned));