Posts tagged Linux
Load balanced and High Availability cluster for your web site under USD 60 pm
Apr 6th
Update 2009-09-02: Now I’m using a single Linode and a Xen VPS from my very own hosting service. This means the VPSes have one more thing less in common, hosting company.
Until recently I used one Linode VPS for hosting all my sites. On 26th March, there was a DDoS attack on one of the Linode customers in the Fremont Datacenter (where my node was as well). This made my sites inaccessible for couple of hours. This got me thinking, what could be done to mitigate such downtime. Answer of course is having a load balanced and high availability cluster. However I couldn’t afford 2 dedicated servers to do this, but I of course can afford 2 Linodes 
. I’ll try to explain how I set up a load balanced, high availability and shared nothing cluster using Linodes (you can use any VPS or dedicated server). I used two Linode 540 s for the job.
All of my web sites are either using PHP, Python or Perl. All of them are using MySQL as the database. Problems I had to solve were;
- replicate files across the nodes
- replicate databases across the nodes
- replicate session (PHP session variables) across the nodes
All the replication needs to be done securely, so I went for a SSH tunnel between the nodes of the cluster. Over which I’ll;
- use
rsyncto replicate/synchronize the document root - use MySQL asynchronous replication (not a
NDBCLUSTER) to synchronize data across the nodes - use
session_mysqlPECL extension to store PHP session in MySQL database transparent to all applications
Check back next week when I’ll post with configuration examples on how I configured my server. If you are in a hurry above pointers are good enough to get you started.
Subversion to Git
Jan 20th
My new year resolution was to ditch Subversion and move to Git. I switched to Git as my SCM for all my development work on new year day it self. I did hit some issues because my ignorance about how things are done with Git, however everything was fixed within the day. Now, 20 days later; I’m really happy that I did make the switch. Git has reduced development time greatly. No longer do I think about whether I should commit, wait for commit to finish for many minutes, or worry about overwriting someone’s work. Now I spend more time doing actual development than thinking about planning merges and commits.
I’ll blog about the few gotchas that I faced when I have time, hopefully soon. Development is fun again, thanks to Git. 
Use KernelCheck to build the latest kernel for debian/ubuntu
Jan 19th
I recently found this awesome project called KernelCheck that allows you to build the latest Linux Kernel for your distribution. It requires very little interaction from the user and automatically optimizes the kernel to user’s needs. Currently it only supports Debian based distributions but support for RPM and Slackware based distributions is planned. KernelCheck is build around the AutoKernel idea by PinguinZ.
Building the Linux Kernel was never easier on Debian (and derivatives) before. I just compiled the 2.6.28.1, it wasn’t a pain at all.
v4l supports Avermedia PCI pure analog (M135A)
Jan 10th
I bought a Avermedia PCI pure analog (M135A) recently (26th December) and to much to my delight it was just plug and play on my home media center running Debian testing with custom built Linux kernel 2.6.28 (Released on 24th December). TV tuner was working with no issues. All local TV channels we accessible . Even the remote was working (not all but the most critical ones like volume control and channel selection are working). Since my sound card didn’t have a mixer I had to use sox to redirect the sound from the TV tuner to the sound card. Running the following at start up did the job.
sox -r 32000 -w -t alsa hw:1,0 -t alsa hw:0,0
Just in case not all required modules are loaded in your case, the required modules to use this radio tuner are:
- saa7134
- saa7134_alsa
- tda827x
- tda8290
I’m really happy that now most of the hardware I can find in local shop is just plug and play on GNU/Linux. My kudos to v4l (video4linux) and the Linux kernel developers .
CUPS spool in devices with limited space
Dec 24th
I was trying to print a large document, and it would never print. A small print job had no issues. To add to that I was printing a stupid PDF form that will only open with Acrobat Reader. The print job was passing through many places, VMWare guest, my notebook, and finally print server. I spent hours looking for what’s wrong.
Finally after many hours lost the issue was found to be lack of storage space in the print server; specifically print job spool was filling the disk.
I was unable to find a work around or a fix other than printing in smaller batches. I believe not many people come across this issue, IMHO this is not even worth fixing. I just blogged it for my own reference.
Going multi uplink
Nov 22nd
Last Friday I got a 2nd connection for my home-office. Now I have 1Mbit/s WiMAX uplink from Dialog Broadband and 512Kbit/s (Soon will be upgraded to 1Mbit/s) WiMAX uplink from Lanka Bell.
I have setup one of my old PCs as the router. I couldn’t find a single router with multi-uplink here in Sri Lanka, but PC router is more flexible, IMO. I’m running Debian on the router and using Shoreline Firewall aka Shorewall for firewalling and traffic shaping/control. It took a good few hours to setup mainly because I mixed up the ethernet interfaces 
. Shorewall documentation on multiple internet connections and traffic shaping/control by Tom Eastep helped me a lot in setting up my router.
Get Monit to repair your server!
Oct 31st
Monit is an open source utility for managing and monitoring, processes, files, directories and filesystems on a UNIX system. Monit is capable of automatic maintenance and repair and can execute meaningful causal actions in error situations. It takes less than 15 minutes to setup and run this wonderful tool on most Unix servers. It also comes with a buit in web based service manager.
I personally prefer Monit over Nagios or ZABBIX. They are pain to install and not as flexible as Monit. AFAIK, Nagios only notifies and records events. It is unable to take a casual maintainance action such as restarting the service.
You will find some useful Monit scripts here.
My Kudos to the Monit team. I’m one happy Monit user
Ubuntu 8.10 on Lenovo 3000 N200
Oct 31st
Few hours ago I upgraded my Ubuntu 8.04 to 8.10. Upgrade it self was a smooth one. Download took around 1.5 hours and the installation was around 45 minutes. Ubuntu 8.10 Human theme looks sexy. New wireless driver for Intel 3945ABG has support for the LED indicator as well.
Only issues were:
- ALSA was locked while it’s being used by any application.
- OpenVPN Client was not routing all traffic through the tunnel (There was no obvious option to do add the routes in the NetworkManager)
ALSA issue was fixed with almost no effort but the solution for the OpenVPN client issue was not so obvious (at least for me).
Adding the following line to /etc/modprobe.d/alsa-base fixed the ALSA locking issue.
options snd-hda-intel model=lenovo
In NetworkManager 0.7 all traffic will not be routed through the tunnel if the OpenVPN serve pushes any routes or all of the rules that are pushed through are ignored. You can make NetworkManager to route all traffic through the tunnel by pushing a route similar to 0.0.0.0 0.0.0.0 gw 172.16.1.5 by adding a line similar to bellow to /etc/openvpn/openvpn.conf in the OpenVPN server
push "route 0.0.0.0 0.0.0.0 gw 172.16.1.5"
or by making NetworkManager to ignore all routes pushed from the server. Check the “Ignore automatically obtained routes” checkbox in the Routes dialog in the VPN editing dialog (IPv4 Setting).
That’s it and my notebook is working better than it was before the upgrade.
References: http://bugzilla.gnome.org/show_bug.cgi?id=552594 | https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.22/+bug/136810
One more day for Ubuntu 8.10 release
Oct 29th
Ubuntu 8.10 named Intrepid Ibex will be released on 30th October 2008. I’m looking forward for the release tomorrow. I’ll be upgrading my machines to Ibex. New features in 8.10 are:
- GNOME 2.24
- X.Org 7.4
- Linux kernel 2.6.27
- Encrypted private directory
- Guest session
- Network Manager 0.7
- Samba 3.2
- PAM authentication framework
- Totem BBC plugin
- Server Virtualization
There is more, you can check out http://www.ubuntu.com/testing/810rc.
Duplicity chokes on OSError: [Errno 24] Too many open files
Feb 17th
It was little bit too scary. Duplicity backup scripts were failing on the EC2 instances again, this time around it was not about not able to reach S3, but having too many files open. That was weird because it didn’t give such a error in the past. However the work around was to increase the maximum number of file descripters allowed for the user that was running the backup script.
How ever finding this solution was tought, actually it was a FreeBSD forum that had the solution. I though I would just write it down for Linux.
Step 1: Find out the current limit
To find out the current file descripter limit for a given use, log in as the particular user and run the following command.
$ ulimit -n
By default on Debian it would be 1024.
Step 2: Increase the limit
You would have to edit /etc/security/limits.conf. You will find details on how to setup different limits in limits.conf itself. The record that you have to put in should look like the following.
username hard nofile 2048
Step 3: Log out and Log back in
You would have to log out and log back in as the user that we updated the file descripter limit. Then run the following command.
$ ulimit -n
You should see the updated file descripter limit.
Hope this helps someone like me in desperation to get the backups in track. I would be doing more investigation as to why there are so many files open. If I find anything interesting I would definitely blog about it. Also for everyone’s reference there is a bug filed at the Savanah bug tracker by someone else who ran into the same issue
