PROGRAMMING INTERVIEWS

In part 1 and part 2, we discussed about the problem and gathered knowledge about SimpleDB respectively. In this part we will discuss about used libraries and design and walk through the code.

I wrote a ruby script, which traversed through the live haproxy logs on our load balancer; got IP addresses from those logs; got country corresponding to those IPs and put that in SimpleDB.

Then we have another script running on an EC2 instance (Consumer), which read data from SimpleDB and generated corresponding graphs refreshing continuously. This instance also works as a web server so that one can see these graphs. In this manner, we were able to get the live analytics.

We chose ruby as scripting language for this purpose. Our requirements are few ruby gems:

file-tail: This is a small ruby library that allows it to “tail” files in Ruby, including following a file, that still is growing like the unix command ‘tail -f’ can.

The library can be installed via rubygems:

right_aws: The RightScale AWS gems have been designed to provide a robust, fast, and secure interface to Amazon EC2, EBS, S3, SQS, SDB, and CloudFront. These gems have been used in production by RightScale since late 2006.

The library can be installed via rubygems:

geoip: GeoIP searches a GeoIP database for a given host or IP address, and returns information about the country where the IP address is allocated, and the city, ISP and other information, if you have that database version.

The library can be installed via rubygems:

For using this gem you need to have at least the free GeoIP.dat. That can be downloaded from here.

Raphaël: Raphaël is a small JavaScript library for creating vector graphics on the web. If you want to create your own specific chart or image crop and rotate widget, for example, you can achieve it simply and easily with this library. Raphaël ['ræfeɪəl] uses the SVG W3C Recommendation and VML as a base for creating graphics.

Before inserting the data in SimpleDB, we needed to create domains to insert data. I wrote a simple ruby script to create those domains. Domain names were

Analytics – For all time analytics

Analytics-x – For analytics of xth hour

You can also create these domains suing SDB explorer or SDB tool. Also we did it only for 24 hours because of limited time. You can collect the data for weeks, months, years as well.

Note: You can also adjust all this data in a single domain in place of multiple domains. Since it was just for demo purpose, we created multiple domains.

In next parts, we will see the code for logging in SimpleDB and reading from SimpleDB.