Provisioning a MPICH cluster

As follows is shown how to deploy and test a cluster which uses MPICH as library for supporting message passing paradigm.

The process is divided in four steps (each step is accompanied by a video where a description of the each step is presented) and a final step where the up and running cluster is presented.

Steps

1. Creating/installing the operating system(OPTIONAL). http://showterm.io/d10fc998748cc57cf3eb6
1. Vagrant is the tool used to rapid provision of virtual machines. http://www.vagrantup.com
2. The Vagrantfile used to deploy the machines in this demo is located in this github repository, directory cluster.
3. Este archivo crea tres máquinas virtuales (un nodo maestro y dos nodos de procesamiento).
2. Preparing virtual machines for accepting chef recipes. http://showterm.io/b3591039c559e34f4798a
1. To run the chef recipes is necessary that each machine where the provision process will be run has a user having root privileges with no password when the sudo command is invoked.
2. knife-solo command needs to be installed in the machine where the provisioning process will be run (e.g. gem install knife-solo). 
3. Scripts used in this video can be downloaded from here.
3. Tailoring the configuration for your particular needs. http://showterm.io/3c9a596c2fa39ddf3a402
1. Two python scripts and one bash script were written at this time to tailor the configuration for this demo however the scripts can be used in your own test environment
2. The files can be obtained from this github repository
4. Provisioning the cluster. http://showterm.io/9fee9e1fc5f8e19192698
5. Using the cluster. http://showterm.io/f5cbb251b2c37cee19e8d

python, python everywhere...

Python is one of the most relevant programming languages in the computing landscape. It has been backed by great companies like Google in such a way that its flagship products are developed first in Python. 

Recently, I heard about Twisted Python and it looks pretty similar to Node.js. I read this post and certainly they are similar but the post writer switched from a Twisted to Node because the Twisted's programming model was not easy to follow and this misunderstanding created problems during the development process of their application.

For me, Python keeps doing a great job on different areas and different scenarios from small computing (e.g. Raspberry Pi) to big computing (e.g. Scientific Computing - ScyPy and IPython). 

Python is an attractive language because its syntax is pretty simple. Expert people coming from diverse areas of knowledge can easily express their ideas and turning into algorithms with important dividends.

How to setting a password for Transmission Web UI

Nowadays I am playing with my Raspberry Pi. I'm using it as media stream server (link), NFS server (link1, link2) and recently it is now a P2P file sharing "server" (link1, link2). 

I am using Transmission for enabling my Raspberry Pi as P2P server because it is a versatile program which has command line and web interfaces. 

Transmission's configuration file is located at /etc/transmission-daemon/settings.json and I read a bunch of web pages that helped me to configure it. However, for me was not clear how to set a password for the web interface then here is my recipe:

1. sudo service transmission-daemon stop
2. sudo vi /etc/transmission-daemon/settings.json
3. Find a line beginning with "rpc-password":. Change the string between double quotes for your [clear] password, e.g. "transmission"
4. sudo service transmission-daemon start

If you left unmodified most of the parameters, your transmission deployment can be accessed from any browser using this URL "192.168.1.4:9091". The IP address could change and reflect your network configuration but the 9091 port is the default TCP port for Transmission. 

Transmission web page will prompt you with for a user name ("rpc-username" : "pi",) and password which is the string that you set in the third item above.

Installing VirtualBox + Vagrant + Chef

This entry shows you how to install these three packages: VirtualBox, Vagrant and Chef. VirtualBox is a hypervisor that allows to execute virtual machines running different operating systems such as Windows, Linux and MacOS. Vagrant is used for for deploying virtual machines on VirtualBox and VMWare Fusion using a Ruby-based domain specific language (DSL). Chef is a provisioning tool, like Vagrant, written in Ruby and uses a DSL this time for describing procedures used for performing different Unix administration tasks such as installing and configuring services, modifying files and managing users.

These are the steps that you should run for deploying the aforementioned tools:

• sudo apt-get update
• sudo apt-get upgrade
  
• sudo apt-get install build-essential linux-headers-{uname -r} # es posible que aquí debas re-iniciar el equipo
• Install VirtualBox
• Download a 32- or 64-bits VirtualBox version
• http://download.virtualbox.org/virtualbox/4.1.22/VirtualBox-4.1.22-80657-Linux_amd64.run
• http://download.virtualbox.org/virtualbox/4.1.22/VirtualBox-4.1.22-80657-Linux_x86.run
• Change file permissions, chmod 544 VirtualBox-4.1.22-80657-Linux_${arch}.run, where ${arch} can be amd64 or x86.
• Run, sudo ./VirtualBox-4.1.22-80657-Linux_${arch}.run
• sudo apt-get install rubygems
• sudo gem install vagrant
• sudo gem install chef

That's all!

Remove local Ubuntu repositories

Ubuntu package manager suggests local repositories for those who live outside of United States in order to have low latency. For instance, I live in Colombia so the main repository where updates are checked is in a repository located in Colombia. Good in theory but not so good in practice because, in my particular case, that repository causes that the "apt-get update" command  execution be aborted. 

In order to change your default repositories go to "Software Sources" and select either "United State Servers" or "Main Servers". 

Installing Ubuntu packages [offline mode]

When repetitive deployment tasks are run, to download the required packages just once and installing them on different deployments is an excellent option for saving time.

When the following command is executed:

sudo apt-get -d -o=dir::cache=/tmp install {some-package-name}

It downloads the required package with its corresponding dependencies. 

Later when the downloaded package needs to be installed, the following command should be executed:

sudo apt-get -o=dir::cache=/tmp install {some-package-name}

It installs the required package using the files located at /tmp directory. 

Note these commands differ only by the presence/absence of the "-d" flag.

Current projects

Nowadays I'm actively thinking about four projects to be written in Pyhon

1. A DSL to allow the modification of network configuration files for Linux-based virtual and physical machines 
2. An ecosystem to support the migration of VirtualBox-based virtual machines to HPC data centers
3. An web-based application to run on-line polls
4. A script to detect the number of words written in a Google Drive doc in a range of time

More details in the near feature...