Using docker for application development

I found Docker super useful, but going through a manual is quite time consuming. Here, very stripped manual to create your first image and push it online 🙂

# install docker
wget -qO- https://get.docker.com/ | sh
 
# add your user to docker group
sudo usermod -aG docker $USER
 
# check if it's working
docker run docker/whalesay cowsay "hello world!"
 
# create an account on https://hub.docker.com
# and login
docker login -u $USER --email=EMAIL
 
# run image
docker run -it ubuntu
 
# make some changes ie. create user, install needed software etc
 
# finally open new terminal & commit changes (SESSIONID=HOSTNAME)
docker commit SESSIONID $USER/image:version
 
# mount local directory `pwd`/test as /test in read/write mode
docker run -it -v `pwd`/test:/test:rw $USER/image:version some command with arguments
 
# push image
docker push $USER/image:version

From now, you can get your image from any other machine connected to Internet by executing:

docker run -it $USER/image:version
# ie. redundans image
docker run -it -w /root/src/redundans lpryszcz/redundans:v0.11b ./redundans.py -v -i test/{600,5000}_{1,2}.fq.gz -f test/contigs.fa -o test/run1
 
# you can create alias latest, then version can be skipped on running
docker tag lpryszcz/redundans:v0.11b lpryszcz/redundans:latest
docker push lpryszcz/redundans:latest
 
docker run -it lpryszcz/redundans

You can add info about your repository at https://hub.docker.com/r/$USER/image/

Conflicting config for htop on machines sharing same /home directory

My friend spotted a problem with htop configuration. Simply when htop was executed on two different Ubuntu distros (10.04 and 14.04) the config was reset.
After some interrogation, we have spotted that 10.04 stores htop config to ~/.htoprc, while 14.04 to ~/.config/htop/htoprc. It was enough to remove one of them and link another one as below:

rm .htoprc
ln -s .config/htop/htoprc .htoprc

Easy citation in LibreOffice / OpenOffice with Mendeley

Creating reference list is always a nightmare. Mendeley and its handy LibreOffice / OpenOffice plugin may be of great help to many. It was for me. Below, I’ll describe how to make it working.

# get & install mendeley from https://www.mendeley.com/download-mendeley-desktop/

# check version of your mendeley
#  Help > About Mendeley Desktop

# clone repo and build plugin
git clone git@github.com:Mendeley/openoffice-plugin.git
cd openoffice-plugin/
python build.py 1.15.2 false

# add to LibreOffice
#  Tools > Extension Manager > Add...
#   and look for `Mendeley-1.15.2.oxt`

After OpenOffice / LibreOffice restart, you should see new bar. Note, in order for the plugin to work, Mendeley has to be running.

What’s great about this plugin, you can adjust citation style by just a few clicks by clicking on `Choose Citation Style`. There is quite extensive database of predefined citation styles, so adjusting the reference style to your favourite journal will take just a few seconds 🙂
More info about the plugin on github.

Download youtube videos

More weekend oriented tip today;)
I have found downloading videos from youtube super easy with youtube-dl.

# install it
sudo apt-get install youtube-dl

# you can fetch many videos at once
youtube-dl LINK1 [LINK2 LINK3]

This tool provide many useful options, like changing the audio/video compression format and quality. You can read more here.

Installing Gene Cluster on Ubuntu

Gene Cluster is a program for clustering. I wanted to use it to analyse gene expression data. However, I had problems during installation under Ubuntu 14.04. This is how I solved it:

# install dependencies: Motif libraries
sudo apt-get install libxext-dev libmotif-dev

Get GeneCluster3.0 source code and unpack it.

# configure to install in local dir
./configure --prefix=`pwd` --program-prefix=gene_ && make && make install

# add install dir to ~/.bashrc

pysam installation error on Ubuntu 14.04

If you encounter error during pysam installation:

sudo easy_install -U pysam

... 

pysam/csamtools.c:8:22: fatal error: pyconfig.h: No such file or directory
 #include "pyconfig.h"
                      ^
compilation terminated.
error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

try installing python-dev first.

sudo apt-get install python-dev

Solution found on github.

Encrypted swapfile

Sometimes, it’s worth to encrypt swap space, especially if you process some privacy-sensitive data.

# install cryptsetup
sudo apt-get install cryptsetup
 
# generate swap
sudo truncate -s 1GB /cryptswap
sudo mkswap /cryptswap
 
# add to /etc/crypttab
cryptswap /cryptswap /dev/urandom swap
 
# add to /etc/fstab
/dev/mapper/cryptswap none swap sw 0 0
 
# activate encryption and swap
sudo /etc/init.d/cryptdisks reload && sudo swapon -a

Inspired by AskUbuntu.

Transfer WordPress to Amazon EC2

After rather successful year of using WordPress, I have decided to move my blog to AWS. I was considering the move for long time, motivated by Free Tier and finally I found some time to do it.

At first, I have created WordPress Stack using CloudFormation, but personally I prefer Ubuntu over Amazon Linux and I will focus on configuration of Ubuntu EC2 instance here.

  1. Export your existing blog
    WP-Admin > Tools > Export

  2. Login to AWS console and Create Key Pair
  3. Launch EC2 instance
    I use Ubuntu HVM. I recommend t2.micro, as it’s free for the first year. You should specify created/uploaded key.

  4. Login to your EC2 instance using Public DNS or IP and your key
    ssh -i .aws/your_key.pem ubuntu@ec2xxxxx.compute.amazonaws.com

    NOTE: you key should be readable only by you. To achieve that, you can do:

    chmod 600 .aws/your_key.pem
  5. Configure Ubuntu
    sudo apt-get update && sudo apt-get upgrade
    sudo apt-get install apache2 php5 php5-mysql libapache2-mod-php5 libapache2-mod-auth-mysql mysql-server
    
  6. Configure MySQL
    sudo mysql_secure_installation
    mysql -uroot -p
    
    CREATE DATABASE wordpress;
    CREATE USER 'wordpress' IDENTIFIED BY 'SOMEPASS';
    GRANT ALL ON wordpress.* TO 'wordpress';
    
  7. Configure wordpress
    sudo -i
    cd /var/www/html/
    wget https://wordpress.org/latest.tar.gz
    tar xpfz latest.tar.gz
    rm latest.tar.gz
    cd wordpress/
    mv wp-config-sample.php wp-config.php
    sudo chown -R www-data:www-data /var/www/html
    
    # edit wp-config.php
    define('DB_NAME', 'wordpress');
    define('DB_USER', 'wordpress');
    define('DB_PASSWORD', 'SOMEPASS');
    define('DB_HOST', 'localhost');
    
  8. Configure Apache
    # edit /etc/apache2/sites-available/wordpress.conf
    
    ServerName ec2xxxxx.compute.amazonaws.com
    ServerAlias YOURDOMAIN.COM
    DocumentRoot /var/www/html/wordpress
    DirectoryIndex index.php
    
    AllowOverride All
    Order Deny,Allow
    Allow from all
    
    # enable wordpress in apache2
    sudo a2ensite wordpress
    sudo service apache2 restart
    
  9. Enable HTTP access to your EC2 instance
    Go to EC2 console > Instances > Select you instance > Description >
    Click on your `Security group` > Select Inbound > Edit > Add rule > HTTP > Save

  10. Point your webrowser to your EC2 instance: http://ec2xxxxx.compute.amazonaws.com/
  11. Setup your wordpress account
  12. Upload dumped wordpress data
    WP-Admin > Tools > Import > WordPress > > Upload file import
    NOTES:
    You will need to install WordPress Importer plugin.

  13. Assign post to correct user.
    Don’t forget to Import Attachments!

  14. Install your favourite plugins and themes
    As for plugins, I strongly recommend: JetPack, SyntaxHighlighter Evolved, Google Analytics Dashboard for WP and BackUpWordPress or ajax-load-more.

  15. Add favicon
    Copy selected favicon.ico to /var/www/html/wordpress

Voilà!
BTW: You may want to increase security of your instance and setup swap just in case memory usage exceeds your EC2 instance size.

Sending emails from command-line using Gmail & SSMTP

I wanted to send email reports from my computers. For that, the easiest is to create additional Gmail account (so your private email is safe) and set-up SSMTP:

sudo apt-get install ssmtp mailutils

Configure SSMTP: comment all lines in /etc/ssmtp/ssmtp.conf and add at the end (replacing HOSTNAME/USER/PASSWORD with your info):

hostname=HOSTNAME 
mailhub=smtp.gmail.com:587
AuthUser=USER@gmail.com
AuthPass=PASSWORD
FromLineOverride=YES
UseTLS=YES

Then, allow access from less secure apps to created Gmail account and enable one-time sign-in for new app without captcha. It’s strongly recommended not to use your private account for that purpose!

And send emails:

echo "email body" | mail -s "Test email from "`hostname` SOMEUSER@gmail.com

Voilà!

Note, this works perfectly fine in Ubuntu, as well as in Raspberry Pi.