tag:blogger.com,1999:blog-70407061437062693662024-03-13T08:12:06.842-07:00Global Computing - 365This blog is about large scale computing, tricks of easy to learn programming languages, and Linux hacks.John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-7040706143706269366.post-72693889525847996722014-03-11T20:50:00.001-07:002014-03-11T20:50:20.435-07:00Provisioning a MPICH cluster<div style="text-align: justify;">
<span style="font-family: "Trebuchet MS",sans-serif;">As follows is shown how to deploy and test a cluster which uses MPICH as library for supporting message passing paradigm.</span></div>
<div style="text-align: justify;">
<span style="font-family: "Trebuchet MS",sans-serif;">The process is divided in four steps (each step is accompanied by a video where a description of the each step is presented) and a final step where the up and running cluster is presented.</span></div>
<h4 style="text-align: justify;">
<span style="font-family: "Trebuchet MS",sans-serif;">Steps</span></h4>
<ol style="text-align: justify;">
<li><span style="font-family: "Trebuchet MS",sans-serif;">Creating/installing the operating system(<b>OPTIONAL</b>). <a href="http://showterm.io/d10fc998748cc57cf3eb6">http://showterm.io/d10fc998748cc57cf3eb6</a></span></li>
<ol>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Vagrant is the tool used to rapid provision of virtual machines. <a href="http://www.vagrantup.com/">http://www.vagrantup.com</a> </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">The Vagrantfile used to deploy the machines in this demo is located in <a href="https://github.com/johnsanabria/vagrant-files.git">this github repository</a>, directory cluster. </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Este archivo crea tres máquinas virtuales (un nodo maestro y dos nodos de procesamiento).</span></li>
</ol>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Preparing virtual machines for accepting chef recipes. <a href="http://showterm.io/b3591039c559e34f4798a">http://showterm.io/b3591039c559e34f4798a</a></span></li>
<ol>
<li><span style="font-family: "Trebuchet MS",sans-serif;">To run the chef recipes is necessary that each machine where the provision process will be run has a user having root privileges with no password when the <span style="font-family: Times,"Times New Roman",serif;">sudo</span> command is invoked.</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">knife-solo command needs to be installed in the machine where the provisioning process will be run (e.g. <span style="font-family: Times,"Times New Roman",serif;">gem install knife-solo</span>). </span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Scripts used in this video can be downloaded from <a href="https://github.com/johnsanabria/installing-chef.git">here.</a></span></li>
</ol>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Tailoring the configuration for your particular needs. <a href="http://showterm.io/3c9a596c2fa39ddf3a402">http://showterm.io/3c9a596c2fa39ddf3a402</a></span></li>
<ol>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Two
python scripts and one bash script were written at this time to tailor
the configuration for this demo however the scripts can be used in your
own test environment</span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">The files can be obtained from <a href="https://github.com/johnsanabria/coccoa.git">this github repository</a> </span></li>
</ol>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Provisioning the cluster. <a href="http://showterm.io/9fee9e1fc5f8e19192698">http://showterm.io/9fee9e1fc5f8e19192698</a></span></li>
<li><span style="font-family: "Trebuchet MS",sans-serif;">Using the cluster. <a href="http://showterm.io/f5cbb251b2c37cee19e8d">http://showterm.io/f5cbb251b2c37cee19e8d</a></span></li>
</ol>
John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.com0tag:blogger.com,1999:blog-7040706143706269366.post-51198562458436929302013-07-03T09:40:00.002-07:002013-07-03T09:40:42.533-07:00python, python everywhere...<div style="text-align: justify;">
Python is one of the most relevant programming languages in the computing landscape. It has been backed by great companies like Google in such a way that its flagship products are developed first in Python. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Recently, I heard about Twisted Python and it looks pretty similar to Node.js. I read <a href="http://journal.paul.querna.org/articles/2011/12/18/the-switch-python-to-node-js/">this post</a> and certainly they are similar but the post writer switched from a Twisted to Node because the Twisted's programming model was not easy to follow and this misunderstanding created problems during the development process of their application.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
For me, Python keeps doing a great job on different areas and different scenarios from small computing (e.g. Raspberry Pi) to big computing (e.g. Scientific Computing - ScyPy and IPython). </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Python is an attractive language because its syntax is pretty simple. Expert people coming from diverse areas of knowledge can easily express their ideas and turning into algorithms with important dividends.</div>
John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.com0tag:blogger.com,1999:blog-7040706143706269366.post-76658096928926239782013-06-05T19:41:00.001-07:002013-06-05T19:41:02.714-07:00How to setting a password for Transmission Web UI<div style="text-align: justify;">
Nowadays I am playing with my Raspberry Pi. I'm using it as media stream server (<a href="http://naich.net/wordpress/?p=641">link</a>), NFS server (<a href="https://help.ubuntu.com/community/SettingUpNFSHowTo">link1</a>, <a href="http://tech.theswamp.in/post/2013/02/16/setting-up-a-raspberry-pi-powered-headless-media-server-and-nas/">link2</a>) and recently it is now a P2P file sharing "server" (<a href="http://www.robertsetiadi.net/installing-transmission-in-raspberry-pi/">link1</a>, <a href="http://www.computeristas.com/wp/?p=98">link2</a>). </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
I am using Transmission for enabling my Raspberry Pi as P2P server because it is a versatile program which has command line and web interfaces. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Transmission's configuration file is located at <span style="font-family: "Courier New",Courier,monospace;">/etc/transmission-daemon/settings.json</span> and I read a bunch of web pages that helped me to configure it. However, for me was not clear how to set a password for the web interface then here is my recipe:</div>
<ol>
<li><span style="font-family: "Courier New",Courier,monospace;">sudo service transmission-daemon stop</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">sudo vi /etc/transmission-daemon/settings.json</span></li>
<li>Find a line beginning with <span style="font-family: "Courier New",Courier,monospace;">"rpc-password":. </span>Change the string between double quotes for your [clear] password, e.g. <span style="font-family: "Courier New",Courier,monospace;">"transmission"</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">sudo service transmission-daemon start</span></li>
</ol>
<div style="text-align: justify;">
If you left unmodified most of the parameters, your transmission deployment can be accessed from any browser using this URL "<b>192.168.1.4:9091</b>". The IP address could change and reflect your network configuration but the <b>9091</b> port is the default TCP port for Transmission. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Transmission web page will prompt you with for a user name (<span style="font-family: "Courier New",Courier,monospace;">"rpc-username" : "pi",</span>) and password which is the string that you set in the third item above.</div>
John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.com0tag:blogger.com,1999:blog-7040706143706269366.post-38299653873546154742013-04-17T14:01:00.000-07:002013-04-17T14:03:13.216-07:00Installing VirtualBox + Vagrant + Chef<div style="text-align: justify;">
<span style="font-family: courier new,monospace;">This entry shows you how to install these three packages: VirtualBox, Vagrant and Chef. <a href="http://virtualbox.org/">VirtualBox</a> is a hypervisor that allows to execute virtual machines running different operating systems such as Windows, Linux and MacOS. <a href="http://www.vagrantup.com/">Vagrant</a> is used for for deploying virtual machines on VirtualBox and VMWare Fusion using a Ruby-based domain specific language (DSL). <a href="http://wiki.opscode.com/display/chef/Home">Chef</a> is a provisioning tool, like Vagrant, written in Ruby and uses a DSL this time for describing procedures used for performing different Unix administration tasks such as installing and configuring services, modifying files and managing users.</span></div>
<div style="text-align: justify;">
<span style="font-family: courier new,monospace;"><br /></span></div>
<div style="text-align: justify;">
<span style="font-family: courier new,monospace;">These are the steps that you should run for deploying the aforementioned tools:</span></div>
<ul>
<li><span style="font-family: courier new,monospace;">sudo <span class="il">apt</span>-<span class="il">get</span> update</span></li>
<li><span style="font-family: courier new,monospace;">sudo <span class="il">apt</span>-<span class="il">get</span> upgrade</span><br />
</li>
<li><span style="font-family: courier new,monospace;">sudo <span class="il">apt</span>-<span class="il">get</span> install build-essential linux-headers-{uname -r}</span> # es posible que aquí debas re-iniciar el equipo</li>
<li>Install VirtualBox </li>
<ul>
<li>Download a 32- or 64-bits VirtualBox version</li>
<ul>
<li><i><a href="http://download.virtualbox.org/virtualbox/4.1.22/VirtualBox-4.1.22-80657-Linux_amd64.run" target="_blank">http://download.virtualbox.<wbr></wbr>org/virtualbox/4.1.22/<wbr></wbr>VirtualBox-4.1.22-80657-Linux_<wbr></wbr>amd64.run</a></i></li>
<li><i><a href="http://download.virtualbox.org/virtualbox/4.1.22/VirtualBox-4.1.22-80657-Linux_x86.run" target="_blank">http://download.virtualbox.<wbr></wbr>org/virtualbox/4.1.22/<wbr></wbr>VirtualBox-4.1.22-80657-Linux_<wbr></wbr>x86.run</a></i></li>
</ul>
<li>Change file permissions, <span style="font-family: courier new,monospace;">chmod 544 VirtualBox-4.1.22-80657-Linux_<wbr></wbr>${arch}.run<span style="font-family: arial,helvetica,sans-serif;">, where <span style="font-family: courier new,monospace;">${arch}</span> can be amd64 or x86.</span></span></li>
<li><span style="font-family: courier new,monospace;"><span style="font-family: arial,helvetica,sans-serif;">Run, <span style="font-family: courier new,monospace;">sudo ./</span></span>VirtualBox-4.1.22-80657-<wbr></wbr>Linux_${arch}.run</span></li>
</ul>
<li><span style="font-family: courier new,monospace;">sudo <span class="il">apt</span>-<span class="il">get</span> install rubygems</span></li>
<li><span style="font-family: courier new,monospace;">sudo gem install vagrant</span></li>
<li><span style="font-family: courier new,monospace;">sudo gem install chef</span></li>
</ul>
<span style="font-family: courier new,monospace;">That's all! </span>John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.com0tag:blogger.com,1999:blog-7040706143706269366.post-40579121168433794412013-03-25T20:36:00.003-07:002013-03-25T20:36:48.270-07:00Remove local Ubuntu repositories<div style="text-align: justify;">
Ubuntu package manager suggests local repositories for those who live outside of United States in order to have low latency. For instance, I live in Colombia so the main repository where updates are checked is in a repository located in Colombia. Good in theory but not so good in practice because, in my particular case, that repository causes that the "<span style="font-family: Times,"Times New Roman",serif;">apt-get update</span>" command execution be aborted. </div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFbYGh9YUnhQmRd9ESOHSbLOtNOXTDMMN8tMnsHf5VUp4SuZNQDlfzeBIruuxcjmxvfJIPxZZn0Caq5N5hg9k6fz-IrjjbiBpfwaRd128jO6anvlFEsKNqhDSx2k3G_reyXNDQp7Vdt2Y/s1600/Screenshot+from+2013-03-25+22:27:37.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="193" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFbYGh9YUnhQmRd9ESOHSbLOtNOXTDMMN8tMnsHf5VUp4SuZNQDlfzeBIruuxcjmxvfJIPxZZn0Caq5N5hg9k6fz-IrjjbiBpfwaRd128jO6anvlFEsKNqhDSx2k3G_reyXNDQp7Vdt2Y/s320/Screenshot+from+2013-03-25+22:27:37.png" width="320" /></a></div>
<div style="text-align: justify;">
In order to change your default repositories go to "Software Sources" and select either "United State Servers" or "Main Servers". </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<br /></div>
John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.com0tag:blogger.com,1999:blog-7040706143706269366.post-49435127769726560582013-03-25T18:43:00.000-07:002013-03-25T18:43:12.027-07:00Installing Ubuntu packages [offline mode]<div style="text-align: justify;">
When repetitive deployment tasks are run, to download the required packages just once and installing them on different deployments is an excellent option for saving time. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
When the following command is executed:</div>
<div style="text-align: justify;">
<span style="font-family: Times,"Times New Roman",serif;">sudo apt-get -d -o=dir::cache=/tmp install {some-package-name}</span></div>
<div style="text-align: justify;">
It downloads the required package with its corresponding dependencies. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Later when the downloaded package needs to be installed, the following command should be executed:</div>
<div style="text-align: justify;">
<span style="font-family: Times,"Times New Roman",serif;">sudo apt-get -o=dir::cache=/tmp install {some-package-name}</span></div>
<div style="text-align: justify;">
It installs the required package using the files located at <span style="font-family: Times,"Times New Roman",serif;">/tmp</span> directory. </div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Note these commands differ only by the presence/absence of the "<span style="font-family: Times,"Times New Roman",serif;">-d</span>" flag.</div>
John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.com0tag:blogger.com,1999:blog-7040706143706269366.post-61096317626047425882012-11-30T08:46:00.000-08:002012-11-30T08:46:11.934-08:00Current projectsNowadays I'm actively thinking about four projects to be written in Pyhon<br />
<ol>
<li>A DSL to allow the modification of network configuration files for Linux-based virtual and physical machines </li>
<li>An ecosystem to support the migration of VirtualBox-based virtual machines to HPC data centers</li>
<li>An web-based application to run on-line polls</li>
<li>A script to detect the number of words written in a Google Drive doc in a range of time </li>
</ol>
More details in the near feature...John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.com0tag:blogger.com,1999:blog-7040706143706269366.post-40969464421815145172012-11-26T20:02:00.000-08:002012-11-26T20:02:27.003-08:00Google Drive SDKI'm a frequent user of Google applications not only Google mail but also Google's office suite.<br />
Time ago I read about the discipline to write at least 750 words per day and I wonder how I can count the number of words that I did write in a day on a particular file residing in Google Drive.<br />
Well, I searched and found <a href="https://developers.google.com/drive/quickstart-python#optional_view_a_quickstart_video">this video</a> to show how to create a python-based command line application to access files in Google Drive.<br />
I'm watching it and I will figure out then how to count the number of words written in a particular day. I have to say that Google Drive uses the revision concept for managing versions in its files. Lets see how hard is this.<br />
<br />
PS: My idea is write a short program to receive a file name and shows how many words were written during the last day. If additional parameters are passed to the program, such as a range of days, the program will show how many words have written each day during this particular range of days.<br />
<br />
Stay tuned!John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.com1tag:blogger.com,1999:blog-7040706143706269366.post-81108428647100581652012-11-26T10:56:00.001-08:002012-11-26T10:56:20.804-08:00750 words is not more a dream or jokeMost of my work is related with sharing knowledge and experience with others. My interactions are mostly via talks but for sure it is not the only mechanism to exist but also writing is an option.<br />
<br />
Writing, as many other activities in my life, shall have a high priority. I have assigned a high priority to read relevant info about [distributed] computing, do physical activity, share with my close people and strength my spiritual side.<br />
<br />
You should be worry about this entry because is not related with the title of this blog. Right, but it is part of my training, part of my warming up time because from this time and afterwards; I will be posting different links regarding with distributed computing (e.g. grid and cloud computing), developments around Infrastructure as a Code (e.g. DevOps), programming stuff (e.g. python) and my "own" Linux hacks (they are not really my hacks but recipes from others working for me).<br />
<br />
And what about 750 words? Well, this is my writing goal for business days. <br />
<br />
<br />
<br />
John Sanabriahttp://www.blogger.com/profile/16130331025668383846noreply@blogger.com0