Posted by the ManGeek

Friday, June 3rd 2011 6:52 pm CDT

Last night I found myself desperate to read the output of tail but looking for specific content. (Basically tailing a log file and needing to have the fields pop out at me.) There's no easy colorize command in linux so I did some digging, found something similar and modified it. (This is based on Kenny Moen's blog entry [www.functor.be] about baretail on unix.) Anyway, the following perl script will allow any content to be highlighted using a simple regex and a pipe.

For example, to highlight an entire line wherever named is listed:
tail -f /var/log/messages | colorize --highlight "blue:.*named.*"

Or to highlight the incoming interface and outgoing interface on a firewall log with red for the inside and green for the outside:
tail -f /var/log/messages | colorize --highlight "red:w+=eth0" --highlight "green:w+=eth1"

Enjoy!

#!/usr/bin/perl
use Getopt::Long qw(:config pass_through);

my %colors =
(
   'red'     => '41;37;1m',
   'green'   => '42;37;1m',
   'yellow'  => '43;30;1m',
   'blue'    => '44;37;1m',
   'magenta' => '45;37;1m',
   'cyan'    => '46;30;1m',
   'white'   => '47;30;1m'
);

my @highlights;
GetOptions ("highlight=s" => \@highlights);

my $tailargs;
for(my $c = 0; $c < scalar(@ARGV); $c++)
{
   $tailargs .= ' ' . $ARGV[$c];
}

my $highlightargs;
for(my $c = 0; $c < scalar(@highlights); $c++)
{
   my @data   = split(/:/, $highlights[$c], 2);
   my $color  = $colors{$data[0]};
   my $search = $data[1];

   $highlightargs .= "s/($search)/\033\[$color\$1\033\[0m/gi; ";

}

while (<>)
{
        eval $highlightargs;
        print $_;
}

Trackback URL: http://www.mangeek.com/blogc/59track.html

Comments (0)

Posted by the ManGeek

Thursday, April 7th 2011 6:38 am CDT

Ever try to setup iptables in a redundant configuration? That's exactly what I recently set out to accomplish and after a few bumps, figured out a nice clean way to accomplish it. Granted, I don't yet have a way to do state failover but for my environment that wasn't a concern. That being said, this was accomplished using iptables and keepalived. I did not use ipvs. I just built two iptables systems and installed keepalived. vrrp is done on the inside interface. Pretty easy other than a few gotchas. Here are some issues I ran into and how I got over them.

• Limited IP addresses on the outside interface
  To get over this issue, I installed both firewalls without an outside IP. I turned all used addresses into virtual interfaces in the config. This way the firewall is not addressable if it's not active. Until it claims an address it stays quiet.
• Keepalived uses gratuitous ARP
  This was unfortunate. I use comcast business for my connection and their router does not support gratuitous ARP. It would keep the old MAC address after a failover. There are ways to get around this using aliased interfaces (look up ip link add link on google) but my way to get around this was much simpler. My firewalls are virtual machines and VMware lets you adjust MAC addresses. Therefore I gave the same MAC address to both machine's external interfaces. To the internet it will look no different than business as usual.
• Routes are not configured on failover
  Because I do not have an IP bound upon failover, the result is that the IP fails over but the routing table is wrong. To get over this I used a notify script as part of the config. You just put a line like line "notify_master /opt/script/masterrouteupdate" in your vrrp_instance configuration directive for the internal interface and in that file, I put:
    #!/bin/sh
    /sbin/route del -net default gw 192.168.0.1
    /sbin/route add -net 75.149.154.200 netmask 255.255.255.248 eth2
    /sbin/route add -net default gw 75.149.154.206
  Of course you'll want to modify to fit your needs! (Some IPs modified to protect the innocent.) Note that you see I have it deleting a default rule - this is because I have two scripts, a master update and a backup update. The backup adds an internal route to the other firewall so when it reverts and pulls back inside it acts just like an inside host - mangeable, updatable, all that fun.
• Services bound externally don't pickup
  Some services running on the machine that bind to the external interface did not pickup on the IP change. For those, I had to add kill -HUP to the script to reread their configs and the machine setup. Problem solved.

And thus, my config is born:

global_defs {
   notification_email {
     oh@you.com
   }
   notification_email_from keepalived@silly.wabbit
   smtp_server 10.0.1.7
   smtp_connect_timeout 60
   router_id stigr
}

vrrp_instance VI_INT {
  interface eth0
  state MASTER
  virtual_router_id 151
  priority 101
  virtual_ipaddress {
    192.168.1.1 dev eth4
    75.149.154.201 dev eth5
    75.149.154.202 dev eth5
  }

  notify_master /path/to/master_active
  notify_backup /path/to/backup_active
  smtp_alert
}

The end configuration is a very fast failover that allows me to manage both systems internally but save on resources externally. It's not flawless as I said - no state failover - but the end result is admirable.

Note: Some may wonder why I didn't use the virtual_routes directive. Well honestly, because I found it to not work and it didn't give me the flexibility to have an alternate backup route - something I needed for system updates.

Trackback URL: http://www.mangeek.com/blogc/58track.html

Comments (0)

Posted by the ManGeek

Thursday, December 16th 2010 10:34 pm CST

Thought I'd share this little script. Basically what I needed to do was build a way to track what hard drives were connected to my linux raid arrays along with their serial numbers and temperatures. This script requires hdparm and hddtemp which are both readily available on most distros.

#!/bin/bash

# set LANG for encoding celcius symbol, UTF-8 screws it up
LANG=en_US
IFS=$'\n'
for file in /dev/md[0-9]*; do
  VAL=`mdadm -D $file`;
  for line in $VAL; do
    if [[ $line =~ \/dev\/(h|s)d[a-z][0-9] ]] ; then
      DISK=`echo $line | awk '{ print $7 }'`
      TEMP=`hddtemp $DISK | awk -F': ' '{ print $3 }'`
      SERIAL=`hdparm -I $DISK | grep "Serial Number" | awk '{ print $3 }'`
      SPECLINE="${DISK} ${SERIAL} ${TEMP}"
      echo $line | sed "s;$DISK;$SPECLINE;"
    else
      echo $line
    fi
  done

  for disk in `echo $VAL | grep '^[ ]*[0-9]' | awk '{ print $7 }'`; do
    echo $disk;
  done
done

 

--- sample output ---

/dev/md1:
        Version : 1.2
  Creation Time : Tue Nov 30 15:12:06 2010
     Raid Level : raid5
     Array Size : 5860552704 (5589.06 GiB 6001.21 GB)
  Used Dev Size : 976758784 (931.51 GiB 1000.20 GB)
   Raid Devices : 7
  Total Devices : 7
    Persistence : Superblock is persistent
    Update Time : Fri Dec 17 01:58:35 2010
          State : clean
 Active Devices : 7
Working Devices : 7
 Failed Devices : 0
  Spare Devices : 0
         Layout : left-symmetric
     Chunk Size : 128K
           Name : skyn:bigstore  (local to host skyn)
           UUID : 77df4227:2c4c30e8:93438ed9:49939563
         Events : 60568
    Number   Major   Minor   RaidDevice State
       0       8       97        0      active sync   /dev/sdg1 WD-WMATV8578493 36°C
       1       8      145        1      active sync   /dev/sdj1 WD-WMATV7948948 34°C
       2       8      209        2      active sync   /dev/sdn1 WD-WMATV7901255 36°C
       3       8       81        3      active sync   /dev/sdf1 WD-WMATV7875948 37°C
       4       8      161        4      active sync   /dev/sdk1 WD-WMATV8049056 37°C
       5       8       33        5      active sync   /dev/sdc1 WD-WMATV8516087 34°C
       7       8        1        6      active sync   /dev/sda1 WD-WMATV7450752 34°C

Trackback URL: http://www.mangeek.com/blogc/57track.html

Comments (0)