Saturday, 24 November 2012

OpenELEC Raspberry Pi - Build R12577

Time for another new image. The OpenELEC guys have added something pretty cool to this release. If you've got a new Pi with 512MB of RAM it can run OpenELEC entirely from RAM! I can't test this myself because I've got a first revision, but Matt @ RaspPiBeginners has tested it out and it works well.

OpenELEC R12577 - Raspberry Pi 1GB image

Tuesday, 13 November 2012

F5 and Websockets

UPDATE: Well even though the F5 engineer swore that websockets were unsupported - it appears a bug has appeared with relation to handling of websockets and according to ask.f5.com - they now work as of TMOS >= 11.4

If you're interested in getting some shiny new HTML5 websockets working through an F5 load balancer with HTTP profiles enabled, you're in for some disappointment.
Despite websockets being documented in the RFC as proxy compatible - they don't work through an F5 when you are using full-proxy mode (i.e. you have an HTTP profile assigned). This looks to be down to the websockets connection sending an 'upgrade' notification to the client which turns the HTTP connection into a standard TCP connection (still on port 80/443 however but not strictly HTTP).

Anyway, unless you know of some hardcore iRule magic - I had to have a bit of an architectural rethink. Our websockets application on the backend works off of a virtual host which has the websockets proxy enabled on it. It runs on the same server as the main web application so you can see where disabling the HTTP profile becomes a problem if I want to run any iRules relating to HTTP. The way we've got around it is to create a new public virtual server then redirect any connections going to the /websocket  to the new virtual server using an iRule. On the new virtual server - you still assign an HTTP profile and then create an iRule so that only connections to the /websocket path go to the virtual server and anything else is redirected back to the main VS. It's a bit of a crude solution, but it's simple enough to do the job. For what it's worth, websockets support is in the pipeline apparently.
So for the iRules...

iRule for VS_main_web_app


when HTTP::REQUEST {
if {[HTTP::URI] starts_with '/websocket'}
  {
  HTTP::redirect "https://websocket.domain.com"
  }
}


iRule for VS_websocket_only


when HTTP::REQUEST {
if {![HTTP::URI] starts_with "/websocket"}
  {
  HTTP::redirect "https://mainapp.domain.com"
  }
else
  {
  HTTP::disable
  }
}


Sunday, 11 November 2012

SSL Decryption with Wireshark (Private key and Pre-Master secret)

Troubleshooting communication problems with Wireshark can be difficult at the best of times, yet alone when the connection is encrypted with SSL/TLS.
There are a couple of ways you can approach decrypting the SSL/TLS traffic. One assumes you have root access to the server you are having problems with and you're able to obtain a copy of the public and private key. If you only have access to the server as a client with no special privileges, there are still a couple of ways to take look at the encrypted data.

Before you look at either method, it would be useful to do a little background reading on SSL/TLS as the encryption methods might not work quite as you think.
In a nutshell, when a client (in most cases, a web browser) makes a connection to a web server requiring SSL/TLS encryption - the encrypted channel is setup using a symmetric session key. This key is a random string generated by the client and then encrypted and transmitted using the servers public key, known as the Pre-master Secret. Once shared, the client and server use this shared key to encrypt and decrypt traffic. There's a more detailed version of this here, but knowing this you be able to see how you can decrypt the traffic using the SSL session key or the servers private key.

Method 1 : Decrypting the traffic with the server private key.



  1. You'll need the private key from the server first. This can be hidden in a variety of locations so you'll have to refer to your server documentation for where to find it. Usually it will be a single file with an extension of .CRT, .PEM, .DER or in some cases .PFX or .P12. The latter two are certificate containers which contain the private key, public key and the certificate of the issuing CA.
  2. Once you've found the private key, it will make things easier if they're in PEM format with an unencrypted key. The best way to do the conversion is to use OpenSSL. It can be a bit confusing at first but you'll soon get used to how it works. There is a good selection of commands here (they also offer an upload / convert tool. Not to say they're not trustworthy, but I wouldn't recommend giving your private key to anyone. Ever.)
  3. Now you have the private key, you'll need to load this into Wireshark and configure it to use this key for any traffic to or from your web server. Go to Edit > Preferences. Then Protocols > SSL. Next to the RSA keys list text, click the edit button. From here select the servers private key and enter the IP address of the web server that will be present in the capture. Select port 443 (or whichever port your application runs on) and the protocol which is inside the encrypted tunnel. In most cases, this will be HTTP but could be IMAP, SMTP etc. The password field can be left blank if you've saved the key in an unencrypted form, otherwise provide the password here. Click OK until you're back to the Wireshark main screen.
  4. Now start the capture and generate some encrypted traffic to your web server. Make sure you are using a totally new session. Clearing the cache then closing and reopening your browser would be ideal. Stop the capture when you've generated a few connections.
  5. Back in the main window, you should now find that the SSL wrapping has been removed and you're able to view the protocol details within.

Method 2: Decrypting with Pre-Master Secrets


This method is relatively new to Wireshark and allows you decrypt the encrypted traffic using the Pre-Master Secret which is generated by the client. By default, this key isn't logged anywhere for obvious reasons but with Chrome it's possible to set an environment variable and have these written to disk.
  1. (Windows 7) Right click on 'My Computer' and then go to properties.Then click Advanced System Settings > Environment Variables. Then under system variables - create a new variable named SSLKEYLOGFILE with the value being a text file. In this case I went with C:\premaster.txt. Click OK through all open dialogs. I've found this didn't take effect immediately and needed a reboot before it started logging.
  2. Back in Wireshark, head to Edit > Preferences > Protocols > SSL. Under the option for '(Pre)Master-Secret log file name' - select your log file you created above (so C:\premaster.txt).
  3. Start your capture in Wireshark and then generate a few SSL connections in Chrome. Stop the capture when you're done.
  4. In Wireshark, you should again see that your encrypted traffic is now unwrapped ready for some troubleshooting. If you don't, check the pre-master file you've created exists and has some contents. If not you should double check your environment variable.

These is of course a much easier method for looking into encrypted traffic if you're only looking at HTTP traffic (or traffic that supports a proxy setting) and that's to use Fiddler. Fiddler is an excellent local proxy application which can perform SSL termination and re-encryption (otherwise known as 'man-in-the-middle' / MITM). If you don't need a full packet capture and can make do with the HTTP requests and responses this might save you a lot of pain.

Saturday, 13 October 2012

Raspberry Pi - Hi-res Gertboard shots

Today's lunch break saw a desk in our office turned into a Blue Peter style photo studio to take some hi-res shots of Matt's Gertboard. (Blue Peter style = 2x sheets of A4 paper and a milk bottle!).
If anyone wants / needs them, I still have the massive TIFF files.

Top view

Rear view

Monday, 1 October 2012

Raspberry Pi - LCD scroller tutorial

After having my Raspberry Pi for months now - I've finally started to have a little tinker with the GPIO. After a good few hours talking with Matt @ RaspberryPi Beginners and blaming my lack of electronics knowledge on things like...
- Broken resistors (also, claiming the resistors he gave me were magical resistor-diodes that only worked the right way around)
- Faulty multimeter (actually not faulty - was on the wrong setting for the resistors I was testing)
- Broken breadboard (not broken, I just can't read schematics)

...I finally got a 2x16 LCD working and displaying some bits and pieces through a cobbling of Python scripts.

The first thing I wrote was a simple message scroller for the LCD (file available here). The GPIO skeleton script is from RaspberryPi-Spy.co.uk. If you're going to try this you'll need to have your LCD setup first (I would recommend the videos by RaspberryPi Beginners for a step-by-step guide).The scrolling action is handled by the code from lines 74 - 83.
If you're new to Python - I'll walk you through how the scrolling works.

1:   str_pad = " " * 16  
2:   my_long_string = "this is a string that needs to scroll"  
3:   my_long_string = str_pad + my_long_string  
4:   for i in range (0, len(my_long_string)):  
5:    lcd_byte(LCD_LINE_1, LCD_CMD)  
6:    lcd_text = my_long_string[i:(i+15)]  
7:    lcd_string(lcd_text,1)  
8:    time.sleep(0.4)  
9:   lcd_byte(LCD_LINE_1, LCD_CMD)  
10:   lcd_string(str_pad,1)  

1. str_pad is the variable we are going to use to help us with the scrolling effect (I say 'effect', because the text isn't technically scrolling. What we're doing is providing a string that is too long to display on the LCD screen and then displaying the rightmost part of the string left one character at a time. If it helps, think of it as looking at a long line of text through a window only 16 characters wide and then moving that string left while keeping the window still). We've gone with 16 blank characters for our padding string - this will make the text look like it's scrolling in from the right onto a blank screen. A nice little trick to save typing in 16 actual spaces is to type a single " " character in quotes and then multiply it by 16. Python and other modern languages can perform (what looks like) arithmetic on strings. For example, "print 'badger' * 100" would give you badger badger badger badger...

2. my_long_string, this is our actual message. This can be whatever you like and is the actual message that will be displayed.

3. Now we join (in programming terms, Concatenate) the two strings together so we end up with the 16 padding spaces (for the blank screen) plus our message at the tail end.

4. Here we move right into loops. This one-liner is doing a few things at once. We are creating 'i' as our iterator (I might write another tutorial on this later on, in the mean time - there is some background here). Our iterator is based on the range of characters in our my_long_string variable. We get this figure through the range command, which in this case takes two parameters; the start value and an end value. It can also take a third 'step' value if you want to move through the iterator in particular jumps. The starting value is 0 and the end is the length of our my_long_string variable which is calculated with the last inline command len(my_long_string). This is pretty simple - len() will return the length of the object you are testing between the brackets. Also note that the following few lines are indented. This is how Python knows which code is part of the loop.

5. This is part of the GPIO > LCD functionality - I haven't had a good poke around this code yet to see what's going on under the hood, but it tells the GPIO which memory location to use for outputting to line 1 or 2 of the LCD.

6. Here we create a new string variable using a slice of the variable my_long_string. A slice is similar to a substring(substr) in other languages in that it allows you to take a section of an existing string based on a start and end position. In our code, we are using the value of our iterator to move through the string from left to right. This is because when using a for loop, the iterator increases / moves on to the next object in the iterator, allowing us to work through each character in our string.

7. This is where we display the current 'window' of our string. The lcd_string function is what sends your string to the GPIO and to the LCD. The first parameter passed to it is the slice of my_long_string which is stored in lcd_string. The second value passed to the function is the number 1 - which if you look at the rest of the script means the text is left justified.

8. time.sleep(0.4) uses the Time module to apply a brief delay at the end of the loop. Without this, the text will fly past and either be unreadable or you'll think the script is broken. At the end of the 0.4 seconds the script will loop back to line 5 until the end of my_long_string is reached.

9. Make note of the subtle indentation here again - moving back out two spaces to show we're not in the loop any more. We're setting up the LCD for writing to line 1 again.

10. A simple finish - we're writing the blank padding string to the screen to clear the LCD.

Hope this is useful, I'll be writing some more Python / Pi tutorials in the coming weeks. If there's a particular element of Python that needs explaining, I'll try and break it up into easier terms (disclaimer, I'm not a Python expert by any means but I know enough to get by).





Monday, 3 September 2012

OpenELEC R11850 1GB image and binaries

New build up - based on a git clone as of 2nd September 2012.
NOTE: The image itself works fine but I had problems accessing any online videos (tested with youtube and wired.com add-ons. Maybe I was just unlucky?). Could well be the network I was testing on though so please drop a comment if your build works ok!

1GB SD Image or Binaries

Friday, 24 August 2012

RFC1918 IP address word-lists

Turns out no one on the internet has got any word lists of internal IP addresses! We had a security test to perform which required an IP address 'word list' to use with the Acunetix WVS fuzzing module (there was a website linking external feeds via an internal proxy so we tried using the word list to enumerate their internal networks). The word lists are attached below:

RFC1918 IP address lists

Tuesday, 24 July 2012

Creating a loopback file-system image with partitions

With building the OpenELEC images for Raspberry Pi - I ran into an odd problem with creating system images for people to download.
The first image I made was DD'd from an 8GB SD card that I had already created and tested. I formatted the SD card, tested in the Pi and then took an image with DD. Then, rewrote it to the card and tested again. All good. Then a few people commented that the image was slightly too large to fit on other 8GB cards which is when I checked and my card it was indeed a few sectors large than some others. So how do you create images which are useful for more people without going out and buying an army of various sized SD cards? The answer is creating loopback devices which are basically file-systems in a single file (which is what you create when you DD a working image from an SD card, for example).

1. Decide what size you would like your image to be. For this example and the OpenELEC image I posted previously, I used 1GB.
2. Create your empty container file using DD. You can replace the expression after seek with 1024000000 if you prefer - this is the size of your image in bytes.

 dd if=/dev/zero of=~/1gb_file_image.img bs=1024 count=0 seek=$[1000*1000]  

3. Mount this device as a loopback device

 sudo losetup /dev/loop0 ~/1gb_file_image.img  

4. This will give you a loopback block device under /dev/loop0. You can now create your partitions using fdisk / parted. I've used the examples from the OpenELEC wiki here.

 sudo parted -s /dev/loop0 mklabel msdos
 sudo parted -s /dev/loop0 unit cyl mkpart primary fat32 -- 0 16  
 sudo parted -s /dev/loop0 set 1 boot on  
 sudo parted -s /dev/loop0 unit cyl mkpart primary ext2 -- 16 -2  
 sudo mkfs.vfat -n System /dev/loop0p1  
 sudo mkfs.ext4 -L Storage /dev/loop0p2  

5. Now you have a filesystem in a file with two partitions. Things can get a little tricky here when you need to mount them individually. You need to umount the current loopback device so you can remount it with some sector offsets and size limits. If you search and go by the scraps of mailing lists that mention this, it can be a little daunting but it's not as bad as it seems. Lets look at an fdisk output of our new filesystem in a file.

 sudo fdisk -l ~/1gb_file_image  
 Disk 1gb_file_image.img: 1024 MB, 1024000000 bytes  
 255 heads, 63 sectors/track, 124 cylinders, total 2000000 sectors  
 Units = sectors of 1 * 512 = 512 bytes  
 Sector size (logical/physical): 512 bytes / 512 bytes  
 I/O size (minimum/optimal): 512 bytes / 512 bytes  
 Disk identifier: 0xd00251a6  
 Device              Boot Start    End     Blocks Id  System  
 1gb_file_image.img1 *    2048     264191  131072 c   W95 FAT32 (LBA)  
 1gb_file_image.img2      264192   1999999 867904 83  Linux  

What we need to note here are the start and end sectors as we will use these to calculate the offset and sizelimit parameters to pass to losetup when mounting the partitions in the file. Taking the first partition as an example (shown as an actual file with a different number at the end, that's just how fdisk rolls I guess). We have a starting sector of 2048 and and end sector of 264191. We need to multiply each of these by 512 to get the total amount of bytes instead of sectors (bytes per sector is in the output - line 4 above). So, our starting offset becomes 1048576, and the size being  (end - start, multiplied by 512 again) 134217216.
We'll add these to our next command.

6. Mount the loopback image file again, specifying the offset and sizelimit for the first partition.

 sudo losetup /dev/loop0 ~/1gb_image_file.img -o 1048576 --sizelimit 134217216  

7. Now, you can mount this loopback device as a block device with the standard mount command

 sudo mount /dev/loop0 /mnt/my_mount_point  

This should give you the first FAT partition (if you used the OpenELEC example anyway) mounted under /mnt/my_mount_point (make sure it exists first!). Then you can copy files into your loopback device to populate your image. To mount the second partition, follow the same procedure as in steps 5-7 but use a new loopback device (/dev/loop1 is a good starting point) and calculate the new offsets and sizelimits based on the start and end sectors of the second partition. When you're finished, make sure to unmount the loop devices, then detach the loopback devices with

 sudo losetup -d /dev/loop0   
 sudo losetup -d /dev/loop1  

It can be a finicky procedure, but once you've got it nailed it's pretty handy. Let me know if any parts don't work as advertised (especially seeing as it took several attempts to get mine right too!).

OpenELEC Build R11639 available

It's been a while since I compiled the latest OpenELEC GIT commit and created an image. So long ago apparently I forgot how to use anything Linux based (long story short - don't do 'apt-get purge ruby* on ubuntu, it removes grub :-/ ).
Built from the GIT source as of 23/07/2012. Binaries and 1GB SD image available below.

OpenELEC Binaries for manual setup

OpenELEC 1GB SD image for DD / Win32imager

Enjoy - any problems let me know in the comments (unless you're going to leave a comment like 'it not work' - if you're going to do that at least leave a description of the problem please).


Wednesday, 11 July 2012

Puppet - 'host name was not match with the server'

...or another way to put it 'you're going to lose several hours of your life to this stupid error message that you'll never ever get back'.

Puppet is a great tool for automated system builds. But as with anything that uses SSL, it can be an utter pig if you start doing things that stray from the standard install guide.

Our scenario looked fairly simple; a handful of puppet agents finding their master by looking up 'puppet.[domain]' - but with a small twist of puppet being a DNS CNAME to another host. My belief was that the agent would be clever enough to follow the CNAME reference and accept the correctly issued certificate from the alternative name. No. Not the case...

The agent (unless configured otherwise) will only lookup 'puppet' and 'puppet.[domain]'. So no matter how you configure your puppetmaster with certs for different DNS names, unless you've added those DNS names to the agent /etc/puppet/puppet.conf file under the 'server=' setting - it won't work. Ever.
There are articles about setting dns_alternative_names when you generate the puppetmaster certificates which is great - but nothing tells you that won't be any use until you configure the client to accept that new hostname.
The page here http://docs.puppetlabs.com/guides/troubleshooting.html points you in the right direction - but is a bit misleading when it doesn't mention that you have to edit the 'server' property on the AGENT too.
The simplest way to solve this that I could find was to configure your puppetmaster to use the setting in /etc/puppet/puppet.conf of 'certname=puppet'. OK, internally the cert name will be different to the actual hostname but it will please the puppet agents and you won't have to edit all of your agent settings in the long run. You can still have the host and DNS name as whatever you like because puppet doesn't ever change the hostname it's requesting after following DNS.
I hope other people getting caught up on this manage to find this and save at least a few minutes of angry typing / throwing heavy things across the room.

Wednesday, 30 May 2012

New OpenELEC RasPi build up - R11170

I've built the latest version available from the OpenELEC Git repo - unfortunately I can't easily see from their readme file what the changes are but looking through the open issues - it's most likely bug fixes. I haven't created an SD image yet - but it'll be up later today in a new and improved smaller image size for people with smaller cards. I'll add some instructions for how to resize the storage partition if you have a larger card too.

R11170 Bootloaders (only if you're creating manually)
R11170 System and Kernel binaries

1GB SD image (I haven't been able to test this yet so let me know if it works for you.)
I've made the SD image with the smallest card I had available because lots of people commented that 8GB was too large. The image should write to cards 1GB+, but you'll need to resize your Storage partition if you want to take advantage of any extra space on your card.

Friday, 25 May 2012

Raspberry Pi - Formatting an SD card for OpenELEC

Turns out the SD card I was using for my OpenELEC image is slightly larger then everyone elses - meaning the image won't write back correctly using Win32 imager / dd.
There is a good start to finish guide here, but for clarity and simplicity - I'll outline them here too.

First identify where your Linux OS has attached your SD card (mine was the same as this guide, /dev/sdb). Typing dmesg and examining the output will most often be the easiest way to find this. For these instructions, we'll assume yours is also attached to /dev/sdb


We need to ensure the device is not mounted anywhere first (Ubuntu, for example, will automatically try to mount removable media in /media/[device-name]


sudo umount /dev/sdb

Next, remove all existing partitions (and also, data - make sure this card is not important, backed up etc.)

sudo parted -s /dev/sdb mklabel msdos

Data removed, we'll need to create two partitions. The first being FAT32 which will become the system / boot partition for OpenELEC. The second, an EXT file system for OpenELEC using all remaining space on the card.

sudo parted -s /dev/sdb unit cyl mkpart primary fat32 -- 0 16
sudo parted -s /dev/sdb set 1 boot on
sudo parted -s /dev/sdb unit cyl mkpart primary ext2 -- 16 -2

This should give you your two required partitions. Next, you will need to format these new partitions.

sudo mkfs.vfat -n System /dev/sdb1
sudo mkfs.ext4 -L Storage /dev/sdb2

At this point, it's best to pop the card out and reinsert it to have the OS recognise the new partitions and if you're lucky - mount them for you. Otherwise, you'll need to create some mount points and reload the cards partitions manually.

sudo partprobe
sudo mkdir /mnt/System
sudo mkdir /mnt/Storage

sudo mount /dev/sdb1 /mnt/System

Now, it's as simple as copying the files from my zipfile into the right places. Extract everything to an easy to access directory, we'll use /tmp. Copy the files across (note - some of them need to be renamed which is taken care of in the code below). Then make sure you're not in a mounted directory and unmount the SD card. Fingers crossed, it should be ready for use!

sudo cp /tmp/OpenELEC-RPi.arm-devel-20120514154309-r10954.kernel /mnt/System/kernel.img
sudo cp /tmp/OpenELEC-RPi.arm-devel-20120514154309-r10954.system /mnt/System/SYSTEM

sudo cp /tmp/bootcode.bin /mnt/System/
sudo cp /tmp/loader.bin /mnt/System/
sudo cp /tmp/arm128_start.elf /mnt/System/start.elf
cd /
sudo umount /dev/sdb

Tuesday, 15 May 2012

Raspberry Pi - OpenELEC / XBMC

After several hours of compiling - I've got a working OpenELEC (XBMC) image for Raspberry Pi! I've given it a quick test and most things seem to work ok. I followed the build instructions from the OpenELEC site and the builds from the past couple of days seemed to fail for various reasons but yesterday mornings (10954) seemed to build perfectly! For anyone not wanting to bother with compiling and formatting their SD card - I've put this into an image you can write to your SD card with dd or win32imager.

(Update - 21/05/2012) - A few people have commented that the image is too large for some 8Gb SD cards. The card I used to create the image shows up as 16152576 512byte sectors so if your card is smaller than this, you might be out of luck. In this case, use the Binary only download and follow the 'preparing and formatting' instructions below.

OpenELEC for Raspberry Pi - Build 10954 (8GB Image)

Edit: I've added the binaries too for anyone using an SD card which is smaller than 8Gb. If you want to use these, you'll need to prepare your SD card according to the this page, but using the files from the zip instead of the target directory.

OpenELEC for Raspberry Pi - Binaries only

Monday, 14 May 2012

CP hitcount - A Python rule-hit counter for Check Point

This is something I was working on a while ago. I needed a way to chew through a large amount of Check Point logs and find out which rules were being hit in the rulebase according to the UUID of the rule and the UUID in the log entry. This has now been implemented to some degree in R75.40 with the hit counter in the dashboard but this might still be useful for anyone that wants to make some pretty graphs out of it.
It requires you to export your logs into text format, either with Smartview Tracker (which seems to take forever) or with 'fwm logexport' (there's a script included if you want to modify it). All code is provided as is, drop me a comment if you find any use for it!

I don't have any formal instructions but the comments should be enough to get you though. It's all Python 2.6 but should work in 2.7. Not tested in 3.x. When you've got the code downloaded, run analyse.py and point the script to your rulebases_5.0.fws and text log file. It will churn through the log file and should spit out an HTML file showing which logs are hit, how many times and which are never hit. It's not perfect, but it's a good starting point for clearing out a rulebase.


Analyse.py : Run this and follow prompts

fwlog2txt.sh : This will dump logs in a particular range to a big text file to use with the tool. To be run on your gateway / Smartcenter

logFunc.py : The modules and routines called by analyse.py

style.css : Not essential, but makes the html output a little prettier

Sample output from the CP hitcount tool

Wednesday, 15 February 2012

Useful tool: BinaryPlant ARP watch

If you've ever been in the situation where your network connectivity is flakier than croissant in a blender - you might be inclined to check someone/something hasn't stolen the IP address of your default gateway. This is probably an easy thing to monitor in Linux (watch "arp -an x.x.x.x" for example) but on Windows it's not that easy. BinaryPlant have a nice tool which provides live updates on any ARP requests and responses it can see on the wire, making any conflicting devices easy to identify. This will be taking place in my essential build toolkit!



Check it out here: http://binaryplant.com/

Using the Zabbix API with Perl

I've recently started looking into Zabbix as a monitoring solution for a project at work. First impressions are it's as ugly as a bag of smashed crabs garnished with ugly berries, but it's certainly functional. One of those functions is a pretty extensive JSON API. To test it out, I put together a basic Perl script because I couldn't find anyone elses to steal learn from. Maybe it will be of use to someone who needs to see the basic structure of sending requests to Zabbix via the API.

Source file: Here
use 5.010;
use strict;
use warnings;
use JSON::RPC::Legacy::Client;
use Data::Dumper;

# Step 1 - request the auth hash
#
#create new JSON_RPC object
my $client = new JSON::RPC::Legacy::Client; 
my $uri = 'http://x.x.x.x/zabbix/api_jsonrpc.php'; 
#replace x.x.x.x with Zabbix IP / Hostname

my $json_obj = {
 jsonrpc => "2.0",
 method => "user.login", #method for creating the authentication hash
 params => {
  user => "admin", #user must have Zabbix API access
  password => "zabbix"
 },
 id => 1 #Request tagging for sending multiple requests
};

#Send finished JSON_RPC request to Zabbix
my $response = $client->call($uri, $json_obj);  
my $authHash;
if ($response->content->{'result'}) { 
 #'result' value is only present in a successful request
 $authHash = $response->content->{'result'}; 
 print "Auth successful - ID hash: " . $authHash . "\n"; 
 #Print to screen, not necessary but good to know everything is working
 }
else { #handle invalid credentials
 print "sorry, incorrect auth\n";
 exit; #exit - we don't have an auth hash so can't proceed
 }

# Step 2 - if we have an auth hash, get host groups available.
# As the third step in our example is going to create a host
# that host needs to belong to a group so we need to know 
# which group IDs are available to put our new host in

$client = new JSON::RPC::Legacy::Client;
$json_obj = {
 jsonrpc=> '2.0',
 method => 'hostgroup.get',
 params => 
  {
  output => 'extended',
  sortfield => 'name',
  },
 id => 2,
 auth => "$authHash",
};
$response = $client->call($uri, $json_obj);
#Dump the response in a basic a format - this will give you a feel for 
#what data is returned by the calls
print Dumper($response);


# Step 3 - create a host using the information from the previous call
# In this example - I have used a host group ID 
# which existed in my demo install.
# This will almost certainly be different in your environment!
$client = new JSON::RPC::Legacy::Client;
$json_obj = {
 jsonrpc => '2.0',
 method => 'host.create',
 params => 
  {
  host => 'myAPI_Host',
  ip => '1.2.3.4',
  port => '10050',
  useip => 1,
  groups => 
   {
   groupid => 7
   }
  },
 id => 3,
 auth => "$authHash"
 };
$response = $client->call($uri, $json_obj);
print Dumper($response);

Friday, 10 February 2012

OSPF lab - with Check Point and Vyatta

Just for fun, I thought it would be useful to build a simple OSPF lab in VMware with the help of Vyatta's open-source router software. On top of this, I've added in three Check Point R75.20 gateways to also run seperate OSPF instances over VPN VTIs to check out route-based VPN (because for all these years of working with Check Point I've been far too lazy to stray far from the simple domain based VPN setup).

So - the ingredients for this lab are...

  • VMware ESXi 4.1 (not essential - you can probably get it working in any other flavour hypervisor, maybe even VirtualBox if you like pain and suffering).
  • 3 Desktop Clients (Windows XP - any OS will be fine but you will need a Windows machine to be able to use Smart Dashboard for firewall policy).
  • 3 Virtual Router VMs (Linux/Other 32bit, 512MB RAM, 4GB HDD, 3 virtual NICS)
  • 3 Firewall VMs (Linux / Redhat5 32bit, 2 CPUs if possible, 512MB RAM, 10GB HDD, 2 virtual NICs).
  • 9 Virtual Networks named Net_CoreX (1-3), Net_EdgeX(1-3), Net_IntX(1-3). These should be attached to the VMs as per the topology below.

Suggested lab topology



1. The first thing to set up is the Vyatta virtual routers in your hypervisor. You will need to attach the virtual networks to your VM according to the diagram. Usually, the ordering of the virtual NICs in VMware matches up nicely with the guest OS. So NIC 1,2 and 3 will present in the OS as eth0, eth1, eth2. This helps when you need to match up the networks to the virtual NICs.

2. Next, you should configure the Vyatta software on each router. This is a lot easier than the documentation might have you believe. Don't misread this as the Vyatta docs being bad - instead read as the software is so easy to use with the Tab and '?' combo you don't really need to refer to the docs. 
Give each Vyatta VM a name to match the topology diagram - something cunning like 'VRx' depending on which router it is. Then, configure the NICs according to the network they are on.
If you're too lazy to tab your way through the config - try this for configuring interfaces on VR1 as an example...



configure

set interface ethernet eth0 address 3.0.0.1/29

set interface ethernet eth1 address 1.0.0.1/29

set interface ethernet eth2 address 172.16.1.1/24

commit


3. Once you have your IPs configured, test your connectivity between the external NICs of the routers. Once happy, you can either test the connectivity fully by adding static routes to each for the 'Edge' networks - or just dive straight in and setup OSPF which, with hindsight, is actually a bit easier than faffing with static routes. I won't delve into the depths of OSPF because I'll probably get it wrong - so some nice links to Wikipedia and Cisco are at the end of the post. Essentially though, you will break your network up into areas. Area 0.0.0.0 is reserved for the backbone area (note: although it's in dotted quad format - it certainly isn't an IP address you're using). Whether it's correct or not, I've setup some extra areas here to break things down further. So, there is a backbone area 0.0.0.0 and edge areas of 0.0.0.1, 0.0.0.2 and 0.0.0.3.
I've added the OSPF config for VR1 below - you can figure out the config for the other routes yourself (you did want to learn something, right?).

protocols {
    ospf {
        area 0.0.0.0 {
            network 1.0.0.0/29
            network 3.0.0.0/29
        }
        area 0.0.0.1 {
            network 172.16.1.0/24
        }
        redistribute {
            connected {
            }
        }
    }
}


4. Repeat this on the other VRs but replace the networks above with the relevant directly connected networks. Once it's setup - you should be able to verify that OSPF is sending and receiving OSPF LSA's with

show ip route

(if that shows up as an invalid command, you're probably within configure mode, just add 'run' to the start to break out temporarily). Now you should be able to ping other routers edge network addresses.

5. You're now ready to start messing with the Check Point side of things. I'm going to assume you have already built your three Check Point gateways. In my setup, I opted for standalone gateways to save on resources and to keep things nice and simple. If you haven't already done it - you will need to run 'pro enable' and reboot to activate the advanced networking components of Secure Platform (in production - these are licensable features).

6. Once setup as per the topology again, it's easiest to install a policy of 'any, any, accept'; we can worry about playing with policies once everything is configured. Make sure the topologies are correct on each gateway object.

7. Create two objects to represent the peer gateways (We'll create this from the context of FW1). I chose to create these with Interoperable Devices but if they're externally managed Check Point gateways that should be fine too. Make sure to set the external interface correctly to the 172.16.x.2 address.

8. Create an empty group for each peer gateway to use as the encryption domain (we're setting up route-based VPNs here so no encryption domain as such). Then, select this group under peer object > topology > VPN Domain > Manually defined.


9. Create a new VPN community and add in your local gateway object plus the two peers. Give it a fitting name, such as OSPF-vpn then go to through the configurations for the community. The default encryption settings will be fine - just so long as you keep them consistent. The only changes to make are to add the shared secrets in for the peer gateways and to disable NAT within the community.

10. Next, before installing this we will need to configure the VTIs from the VLI using 'vpn shell'. This is a little odd in how it works and stacks command levels in a linux-ish structure ('/show/interface/all' for example) but thankfully there isn't much config to be had here.
We need to add a VTI, which is a point-to-point link used to tunnel VPN traffic between peers. In this instance, we need to add two VTI interfaces, one for FW2 and one for FW3. We'll call them vtiFW2 and vtiFW3. The syntax for this is '/interface/add/numbered <LocalIP> <RemoteIP> <Peer name as per policy> <meaningful interface name>'.
Local IP is the local address of the tunnel interface (this should be internal, unused and definitely not the external or other IP address. The RemoteIP is the remote VTI address, not the remote external address!

Adding a VTI through the VPN Shell

11. Once you've added these - you can get a summary with '/show/interface/summary/all' or '/show/interface/detailed/all' and then the interfaces will also show up with your chosen names when using ifconfig. Something else worth knowing - you have to be in the root of the vpn shell to quit - or use '/quit' from any other level.

12. Now, go back to the dashboard and find your local gateway (extFW1 in case you forgot) and update the interfaces with Topology > Get... > Interfaces with topology.

Topology showing new VTI interfaces

13. Accept these changes and install the policy. Fingers crossed, you'll get the green tick.

14. Back to the CLI and this time we'll be going into the routing shell to setup OSPF. To enter this mode, run 'cligated'. You'll be whisked to a Cisco'ish type shell. After 'enable' and 'configure terminal' you'll be ready to go.
15. Some more sample config here for the routing setup - again you can find your way through with Tab and '?'


router ospf 1
    advertise-subnet
    network 192.168.102.1 0.0.0.0 area 0.0.0.0
    network 192.168.103.1 0.0.0.0 area 0.0.0.0
    redistribute direct
    redistribute ospf
    enable
    exit
interface vtiFW2
    ip ospf 1 area 0.0.0.0
    enable
    exit
interface vtiFW3
    ip ospf 1 area 0.0.0.0
    enable
    exit
exit

16. Once this is in place, exit configuration mode and issue a 'show running-configuration' to see what you've setup and check it roughly matches the above. Now - repeat steps 5 to 16 for the other gateways, remembering to configure the relevant gateways as the peers. Then, after some more finger crossing, you should have a VPN mesh with OSPF running over VTI's! So test it out by disabling VTI links and seeing if the firewalls are clever enough to route around the fault :)