Mount USB drive in RPi2

By default, RPi2 limits power supply of USB ports to 0.6A, while most USB drives require more than that. This is why most USB drives cannot be used with RPi2 without external power supply to RPi2. Luckily, given that you have decent power supply (ie I have 2A charger), you can increase USB port max current, by editing `/boot/config.txt`.

IMPORTANT!!! RPi2 allows max 1.2A current through all 4 USB ports combined, so don’t even think about plugging two USB drives!!! This is because the maximum power allowed into the RPi2 is limited to 2A by the fuse (F1) so if one of your USB device draw 1A, then that leaves 1A for the RPi + GPIO + remaining USB devices.

First, check if your drive can be detected without tweaking

sudo blkid

If you see only mmcblk0 entries, this means that your drive is not recognised.

/dev/mmcblk0p1: ...
/dev/mmcblk0p2: ...
/dev/mmcblk0: PTUUID="d2fd971c" PTTYPE="dos"

Add max_usb_current=1 to /boot/config.txt:

Reboot and check if your USB drive is visible after restart:

sudo reboot
sudo blkid

Now, you should see new entry:

/dev/mmcblk0p1: ...
/dev/mmcblk0p2: ...
/dev/mmcblk0: PTUUID="d2fd971c" PTTYPE="dos"
/dev/sda1: LABEL="turo"...

And you should be able to mount it:

sudo mkdir -p /media/turo
sudo mount /dev/sda1 /media/turo

Inspired by www.htpcguides.com and www.raspberrypi.org/forums.

Raspberry Pi2 with Ubuntu Sever and Drupal?

I decided to celebrate 25th B-day of Linux by putting the latest Ubuntu 16.04 on my Raspberry Pi 2 and setting up a webserver.
This is how I did it:

  1. First, get Ubuntu armf image and prepare memory card
  2. # get image
    wget http://cdimage.ubuntu.com/ubuntu/releases/16.04/release/ubuntu-16.04-preinstalled-server-armhf+raspi2.img.xz
    
    # make sure your SD card is on sdb ie by df -h
    xzcat ubuntu-16.04-preinstalled-server-armhf+raspi2.img.xz | sudo dd of=/dev/sdb
    
  3. Configure new user & setup Drupal8 webserver
  4. # create new user & change hostname
    sudo adduser USERNAME && sudo usermod -a -G sudo USERNAME
    # edit /etc/hostname and add `127.0.1.1 newHostname` to /etc/hosts
    sudo reboot
    
    # generate locales
    sudo locale-gen en_US.UTF-8
    sudo dpkg-reconfigure locales
    
    # install software
    sudo apt install htop apache2 mysql-server libapache2-mod-php php-mysql php-sqlite3 php-curl php-xml php-gd git sqlite3 emacs-nox
    

My first impressions?
sudo apt is veeery slow. At first, I thought it’s due to old SD card I’ve been using, but it’s also true for newer SD card.
Some packages are missing (ie. git-lfs), but you can get them using some workarounds.

But everything just works!
You can check the mirror of https://ngschool.eu/ running on RPi2 here.
Maybe it’s not speed devil, but it stable and uses almost no energy 🙂

Cheers!

Inspired by Ubuntu’s Insights.

Installation of git-lfs on Ubuntu RPi2 (armf)

Unfortunately, the standard way to install git-lfs doesn’t work on RPi2. But there is simple workaround:

# install go
sudo apt-get install golang
# you may want to grab a coffee at this stage... 😉 

# get git-lfs from github
sudo -i
mkdir /root/gocode
export GOPATH=/root/gocode
go get github.com/github/git-lfs
cp gocode/bin/git-lfs /usr/bin

Reducing the size of large git repository

The github repository of #NGSchool website has grown to over 5GB. I wanted to reduce the size & simplify this repository, but this task turned out to quite complicated. Instead, I have decided to leave current repo as is (and probably removed it soon) and start new repo for existing version. I could do that, as I don’t care about version earlier than the one I’m currently using. This is short how-to:

  1. Push all changes and remove .git folder
  2. git push origin master
    rm -rI .git
    
  3. Rename existing repo
  4. Settings > Repository name > RENAME

  5. Start new repository using old repo name
  6. Don’t need to create any files as all already exists.

  7. Init your local repo and add new remote
  8. git init
    git remote add origin git@github.com:USER/REPO
    
  9. Commit changes and push
  10. git add --all . && git commit -m "fresh" && git push origin master
    

Doing so, my new repo size is below 1GB, which is much better compared to 5GB previously.

Convert xls table into abstract book PDF

I had to generate Abstract book for #NGSchool2016 (). I had spreadsheet generated by Google Forms with all necessary information. I could copy-paste all entries and format it later on, but I found LaTeX more robust for the task.
As I had already LaTeX template, the only missing part was conversion of .xls to .tex. Thus I have written simple script, xls2tex.py, that generate .tex file based on table from .xls file.
This script, among many other things, convert utf into LaTeX escape characters.

xls2tex.py depends on xlrd and utf8tolatex (from pylatexenc/latexencode, but this is given as single file)

# install dependencies
sudo apt-get install python-xlrd
# generate tex
xls2tex.py

# generate pdf
make

Output pdf.

Github push fails due to large files

Lately, I have had lots of problems with pushing large files to github. I am maintaining compilation of materials and software deposited by other people, so cannot control the size of files… and this makes push to fail often.

git push
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
remote: error: Trace: 6f0f7f66995a394598595375954732db
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File chip_seq/reads/sox2_chip.fastq.gz is 109.69 MB; this exceeds GitHub's file size limit of 100.00 MB

To remove large files from commit, execute

git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch chip_seq/reads/sox2_chip.fastq.gz'
git push

To add large files using git-lfs, execute

# tract by git lfs files larger than 50MB, skipping those in .git folder
find . -type f -size +50M ! -iwholename "*.git*" | rev | cut -f1 -d'/' | rev | xargs git lfs track
# 
git add --all . && git commit -m "final" && git push origin

Make sure that your file are smaller than 2GB, otherwise your push will fail again 😉

Then, to before pull in another machine, make sure to install git-lfs

git lfs install
git pull

Stream audio & video from webcam using VLC

Yesterday, I’ve posted about streaming webcam image to www using motion. This solution, although very simple, has many limitations, lack of sound, usage of high bandwidth and low image quality, just to mention a few. In a way, motion stream is just a set of jpeg files.
In order to solve all of these, I have spend quite some time playing with VLC, an open source cross-platform multimedia player, that is able to transcode and stream audio & video.
Streaming can be started from graphical interface, just go to:

Media >> Stream… >> Capture Device, select your devices, Add HTML destination (ie. :8080/webcam.ogg), select Video-Theora + Vorbis (OGG) profile & press Stream.

You stream will be available at: http://localhost:8081/webcam.ogg

But normally, using command line is preferred under Linux:

vlc v4l2:// :input-slave=alsa:// :v4l2-standard=1 :v4l2-dev=/dev/video0 :v4l2-width=1280 :v4l2-height=720 :sout="#transcode{vcodec=theo,vb=2000,acodec=vorb,ab=128,channels=2,samplerate=44100}:http{dst=:8081/webcam.ogg}" -I dummy

Initially, I had problem with streaming sound along with video. Adding, `:input-slave=alsa:// :v4l2-standard=1` solved this. You can try another values for `:v4l2-standard` ie. 0, 1 or 2, depending which microphone you want to use.

Above command will stream HD video (1280×720) in .ogg format (natively suported by most browsers) @ ~2Mbps (2000kbps). If you have slower connection, you can change `vb=2000` to `vb=1000` (~1Mbps) and play with lower resolutions. You can check available resolutions of your camera by:

lsusb -v | egrep -B10 'Width|Height'

This stream, however, is available to everyone. To limit it only to localhost, you can use iptables:

sudo iptables -A INPUT -p tcp -s localhost --dport 8081 -j ACCEPT && sudo iptables -A INPUT -p tcp --dport 8081 -j DROP && vlc v4l2:// :input-slave=alsa:// :v4l2-standard=1 :v4l2-dev=/dev/video0 :v4l2-width=1280 :v4l2-height=720 :sout="#transcode{vcodec=theo,vb=2000,acodec=vorb,ab=128,channels=2,samplerate=44100}:http{dst=:8081/webcam.ogg}" -I dummy

Now, you can create apache2 proxy, similarly to previous post:

# install apache2-utils
sudo apt install apache2-utils
 
# setup new user & passwd
sudo htpasswd -c /etc/apache2/.htpasswd webcam

# configure apache2 - add to your VirtualHost config
    # webcam
    <Location "/webcam.ogg">
        ProxyPass http://localhost:8081/webcam.ogg
        ProxyPassReverse http://localhost:8081/webcam.ogg
        # htpasswd
        AuthType Basic
        AuthName "Restricted Content"
        AuthUserFile /etc/apache2/.htpasswd
        Require valid-user
    </Location>

Streaming image from webcam through www

Willing to stream image from your webcam through Internet? Nothing easier with Ubuntu!

# install
sudo apt-get install motion

# create config file
mkdir ~/.motion && gedit ~/.motion/motion.conf

# define the port and motion settings
webcam_port 8081
webcam_localhost on
# increase maxrate & quality
webcam_maxrate 30
webcam_quality 90
# slow down the stream to 1 frame per second if no motion
webcam_motion on

# run motion
motion

You can find the stream at http://localhost:8081/.

If you wish to stream it publicly, I recommend at least basic HTTP based authentication.

# install apache2-utils
sudo apt install apache2-utils

# setup new user & passwd
sudo htpasswd -c /etc/apache2/.htpasswd webcam

# configure apache2 - add to your VirtualHost config
    # webcam
    <Location "/cam">
        # proxy
        ProxyPass http://localhost:8081/
        ProxyPassReverse http://localhost:8081/        
        # htpasswd
        AuthType Basic
        AuthName "Restricted Content"
        AuthUserFile /etc/apache2/.htpasswd
        Require valid-user
    </Location>

Now, image from your webcam will be accessible at http://YOURDOMAIN.COM/cam

Finally, you can configure motion to run only when you are away.

Inspired by gist.