Tag Archives: Ubuntu

APT: How to prefer local packages over remote?

I recently had to test some of my locally generated MariaDB debian packages. So, created a local repository of generated packages and added it to sources.list file. However, since these packages required some other packages stored on a mirror, I had to add the mirror to the sources.list file as well. Since, this mirror also contained the packages that I intended to test, now when I try to install the packages, APT would always pick/prefer the ones stored on the mirror. How to solve this? How to make APT prefer the local packages instead? Lets start by taking a peek into the sources.list file:

$cat /etc/apt/sources.list
..
# remote repo
deb http://mirror.jmu.edu/pub/mariadb/repo/5.5/ubuntu precise main

# local repo
deb file:///home/nirbhay/project/repo/maria/testing/debian/5.5/debs binary/
..

$ sudo apt-cache policy mariadb-galera-server
mariadb-galera-server:
  Installed: (none)
  Candidate: 5.5.37+maria-1~precise
  Version table:
     5.5.37+maria-1~precise 0
        500 file:/home/nirbhay/project/repo/maria/testing/debian/5.5/debs/ binary/ Packages
        500 http://mirror.jmu.edu/pub/mariadb/repo/5.5/ubuntu/ precise/main i386 Packages

The following tips can help you fix this problem :

  1. If remote and local packages have the same version (as in my case), place one that you want to be preferred higher in sources.list file.

  2. APT prefers authenticated repository over unauthenticated. So, as against the above case, even if the local repository is placed over the remote, APT will prefer remote one if its authenticated and the local repository is not. In that case, –allow-unauthenticated can be used to make local packages take precedence.

  3. In case local and remote packages have different versions, APT would always prefer the package with higher version. However, this rule can be eased by apt-pinning, where a higher priority is assigned to a particular repository. For example, local repository can be pinned with higher priority (or precedence in APT’s context) by adding a file under /etc/apt/preferences.d with .pref extension and the following content :

    Package: *
    Pin: origin ""
    Pin-Priority: 1000
    

    This has been explained really well in this thread.

Lastly, do not forget to run “apt-get update” for changes to take effect.

Generating SSH key pair

SSH key pair is a set of private/public keys used in securing network communication. These keys are normally required for passwordless SSH login to a remote host running SSH daemon (sshd). Here is how you would generate a pair of RSA keys:

$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/nirbhay/.ssh/id_rsa):
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/nirbhay/.ssh/id_rsa.
Your public key has been saved in /home/nirbhay/.ssh/id_rsa.pub.
The key fingerprint is:
5f:1a:b5:50:a8:b6:d6:2b:48:1b:b6:df:4c:54:a2:28 nirbhay@nirbhay-VirtualBox
The key's randomart image is:
+--[ RSA 2048]----+
|           ..    |
|          ..     |
|         .o o    |
...
|       .o. = .   |
|       ...o      |
+-----------------+
$ ls ~/.ssh/
id_rsa  id_rsa.pub

Now that we have the private/public key files, all you need to do is copy/append the public key (id_rsa.pub) contents to the remote machine’s ~/.ssh/authorized_keys (600) file. DO NOT share the “private key”.

Note: On debian-based distributions, ssh-keygen is provided by openssh-client package.

Oracle VM Virtual Box : Installing guest additions

Environment:

  • Host : Windows 8
  • Guest : Ubuntu-12.04
  • Oracle VM Virtual Box & Extension pack v.4.3.4 (available here)

Instructions:

  1. Install Ubuntu as guest OS and boot into it.
  2. Install necessary packages (required for building VirtualBox Guest Additions kernel modules).

    • build-essential
    • dkms
    • linux-headers-generic
    $ sudo apt-get install build-essential dkms linux-headers-generic
    
  3. Install Guest Additions:
    Guest OS Menu -> Devices -> Install Guest Additions
  4. Open a terminal in the guest OS, cd into the mounted VBOXADDITIONS_4.3.4_XXXXX CD drive and run VBoxLinuxAdditions.run script as root.

    $ cd /media/VBOXADDITIONS_4.3.4_XXXXX
    $ sudo ./VBoxLinuxAdditions.run
    
  5. Reboot!
  6. Optionally, bidirectional copying & Drag’n’Drop between host and guest can be enabled by checking “Bidirectional” under Devices -> ‘Shared Clipboard’ and Devices -> Drag’n’Drop respectively.

References:
(i) https://forums.virtualbox.org/viewtopic.php?f=3&t=15679

What is Galera Arbitrator?

Galera Arbitrator (garbd) is a stateless daemon that can act like a node in a Galera cluster. It is normally used to avoid split-brain situation which mostly occurs because of hardware/link failure, as a result of which the cluster gets divided into two parts and each part remains operational thinking they are in majority (primary component). This may lead to inconsistent data sets. Garbd should be installed on a separate machine. However, it can share the machine running load-balancer. It is interesting to note that as garbd joins the cluster, it makes a request for SST.

Let us now try to add galera arbitrator (garbd) to an existing 2-node MariaDB Galera cluster. Check out this post for steps to setup a MariaDB Galera cluster. We start by connecting to one of the nodes to get the size and name of the cluster.

MariaDB [(none)]> show status like 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 2     |
+--------------------+-------+
1 row in set (0.00 sec)

MariaDB [(none)]> show variables like 'wsrep_cluster_name';
+--------------------+------------------+
| Variable_name      | Value            |
+--------------------+------------------+
| wsrep_cluster_name | my_wsrep_cluster |
+--------------------+------------------+
1 row in set (0.00 sec)

Now, lets start the garbd as a daemon:

$ cd galera-23.2.7-src/garb

$ ./garbd
    --address='gcomm://127.0.0.1:4567'
    --options='gmcast.listen_addr=tcp://127.0.0.1:4569'
    --group='my_wsrep_cluster'
    --log=garbd.err
    --daemon

That’s it, garbd should now be running and connected to the cluster. Let’s verify this by checking wsrep_cluster_size again.

MariaDB [test]> show status like 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 3     |
+--------------------+-------+
1 row in set (0.00 sec)

Great! So, the new garbd is now part of the cluster.

Setting up MariaDB Galera Cluster on Ubuntu

MariaDB Galera Cluster is a multi-master synchronous replication system. In this article, I would be setting up a 3-node cluster on a single machine running Ubuntu. However, in a production scenario it is advised to run each cluster node on a separate box in a WAN.

Requirements

  1. MariaDB Galera Cluster
    • Download it from the official site : https://downloads.mariadb.org/mariadb-galera/, OR
    • Install it using Advanced packaging tool (APT), steps can be found in erkules’ blog post. OR
    • Build it from source : lp:~maria-captains/maria/maria-5.5-galera
      Note : Build would require additional cmake options : WITH_WSREP=ON and WITH_INNODB_DISALLOW_WRITES=1
  2. Galera wsrep provider (libgalera_smm.so)
  3. Some extra Ubuntu packages (in case you choose to build Galera from source!)
    • scons (Build utility, replacement for make)
    • check (Unit test framework for C)
    • libboost-dev
    • libboost-program-options-dev
    • libboost-system-dev (for 23.2.7)
    • libssl-dev

Setup

Now that we have all requirements in place, lets bring up the cluster nodes.

  1. Node#1 : Start 1st node (at port 4001 for instance) with empty cluster address (–wsrep_cluster_address=’gcomm://’).
     $ mysqld
        --no-defaults 
        --basedir=.
        --datadir=./data
        --port=4001
        --socket=/tmp/mysql_4001.sock
        --binlog_format=ROW
        --wsrep_provider=/path-to-galera-provider/libgalera_smm.so
        --wsrep_cluster_address='gcomm://'

    There are certain points to note before we proceed :

    • wsrep_provider option points to the galera wsrep provider, i.e. libgalera_smm.so library.
    • wsrep_cluster_address contains the address of existing cluster members. But in this case, as it is the very first node, the address must be empty.

      Important! In case you are planning to put these options in a config file (my.cnf) – once the cluster is up and running, make sure to change this option to hold a valid address. Failure to do so would disable the node’s capability to auto-join the cluster in case it restarts after a shutdown or crash.

    • From the server’s error log make a note of the base_host & base_port (default 4567) of wsrep. This information would be required to start the subsequent nodes.
  2. Node#2 : Start 2nd node at a different port (4002).
     $ mysqld
        --no-defaults
        --basedir=.
        --datadir=./data
        --port=4002
        --socket=/tmp/mysql_4002.sock
        --binlog_format=ROW
        --wsrep_provider=/path-to-galera-provider/libgalera_smm.so
        --wsrep_cluster_address='gcomm://127.0.0.1:4567'
        --wsrep_provider_options='gmcast.listen_addr=tcp://127.0.0.1:4568'

    Here, we have to specify the address of 1st node via wsrep_cluser_address option. It consists of base_host & base_port of the 1st node that we noted earlier in step 1 (i.e. gcomm://127.0.0.1:4567). Also, as we are starting this 2nd node on the same machine, in order to avoid port conflict, we must provide a different port for wsrep provider to listen to via gmcast.listen_addr wsrep provider option (tcp://127.0.0.1:4568).

  3. Node#3 : As with 2nd node, 3rd (and all subsequent nodes) can be started by same set of options with appropriate selection of available ports.
     $ mysqld
        --no-defaults
        --basedir=.
        --datadir=./data
        --port=4003
        --socket=/tmp/mysql_4003.sock
        --binlog_format=ROW
        --wsrep_provider=/path-to-galera-provider/libgalera_smm.so
        --wsrep_cluster_address='gcomm://127.0.0.1:4567'
        --wsrep_provider_options='gmcast.listen_addr=tcp://127.0.0.1:4569'

The cluster should now be up and running with 3 nodes. This can easily be verified and monitored further by inspecting server’s status & system variables :

MariaDB [(none)]> show status like 'wsrep_cluster_size';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| wsrep_cluster_size | 3     |
+--------------------+-------+
1 row in set (0.06 sec)

MariaDB [(none)]> show variables like 'wsrep%';
...
MariaDB [(none)]> show status like 'wsrep%';
...

Before I close, let me list out some important options which were omitted from this article for brevity :

  • default_storage_engine=INNODB
  • innodb_autoinc_lock_mode=2
  • innodb_locks_unsafe_for_binlog=1
  • wsrep_sst_auth=”user:pass” : to be used by SST (Snapshot state transfer) script

That’s all for now!

Apache2: php file getting downloaded instead?

After having installed all the necessary packages on my fresh Ubuntu, while testing different configurations I came across this problem where the php file was being offered for download when tried to execute via browser! That is, whenever I tried to open a php script through browser, the file was getting downloaded, instead of being executed. If you are a LAMP-fan (like me), you might come across this problem in future or perhaps you are facing it now and have reached here.. 😉 The problem is straight forward :

“the apache server is not able to execute the requested php script (even though the core Apache & PHP packages are installed) and hence offering it for download”

Fortunately, in my case, the resoluton turned out to be simple. While installing php+apache, I missed out the php module for apache, which basically enables apache to handle php scripts. So, I installed it, restarted the apache server and the problem just got resolved!

$ sudo apt-get install libapache2-mod-php5

$ sudo /etc/init.d/apache2 restart

$ find /etc/apache2/ | grep php
/etc/apache2/mods-enabled/php5.load
/etc/apache2/mods-enabled/php5.conf
/etc/apache2/mods-available/php5.load
/etc/apache2/mods-available/php5.conf