Raspberry PI follow-up

raspberry_pi_logoSo, i did get a (actually 2) Raspberry PI and did get it up and running, it was time to do anything useful with it…

Time to setup a Samba server for network recording of security camera feeds. I went along with SWAT, a web based graphical interface to samba configuration. Like all Debian based software, the installation process is pretty straight forward,

apt-get update
apt-get install swat

And auto-magically it installs everything that you need, it evens adds the needed configuration line to /etc/inetd.conf 🙂

[global]
        netbios name = INTRANET
        server string = %h server
        map to guest = Bad Password
        syslog = 0
        log file = /var/log/samba/log.%m
        max log size = 1000
        dns proxy = No
        guest account = nobody
        usershare allow guests = Yes
        panic action = /usr/share/samba/panic-action %d
        idmap config * : backend = tdb

[public1]
        comment = Samba Public 1
        browsable = yes
        public = yes
        writable = yes
        read only = No
        guest ok = Yes
        path = /media/usb0/samba/cam1/

[public2]
        comment = Samba Public 2
        browsable = yes
        public = yes
        writable = yes
        read only = No
        guest ok = Yes

Then just point your browser to the PI ip at port 901. Curious enough, i found the SWAT tool too complex for the simple configuration that i wanted: i trust all users in the network, so my need was just two shares that anyone could read/write. So, i ditch SWAT and went on to good ol’style configuration file editing. The final /etc/samba/smb.conf that is working for me:

Fired up samba
# service samba restart

And the cams had no problem finding the samba shares and recording into them.

Next step was to get an easy way to navigate and download recordings. Of course you can also use the samba shares to navigate and read, but specially to outside access it would implied to configure a VPN access to the network (you don’t want your security camera feeds exposed in the Internets with read/write permissions to the world, right?). I went for HTTP with some kind of a file explorer software that allows users/permissions, file/directory browsing, and file download. For the server part i opted for lighttpd, a small footprint server, and for the voodoo PHP (all pretty familiar technology to me). Again the installation is for dummies:

apt-get install lighttpd
apt-get install php5-cgi

Then just a tiny adjustment at /etc/lighttpd/lighttpd.conf:

index-file.names            = ( "index.php", "index.html", "index.lighttpd.html" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

fastcgi.server = ( ".php" => ((
                     "bin-path" => "/usr/bin/php-cgi",
                     "socket" => "/tmp/php.socket"
                 )))

and restart it. For the software i went for the super nice, cool and powerful AjaXplorer. Just download it and untar to /var/www directory. Then point your browser to PI and log in with admin/admin (changed the password) and then it was just a matter of setting up a user account and a repository pointing to /media/usb0/samba/ (the parent directory of both samba shares).

Now, only one thing left, clean up and report. What to use? Of course PHP again. But this one in command line, so i installed the CLI version.

apt-get install php5-cli

And i did i script that cleans up old recordings and send me a daily report email using basic functions and the great PHPMailer class.

require('phpmailer/class.phpmailer.php');

function deleteDir($dir, $days) {
    $now      = time();
    $diff     = 60*60*24*$days;
    $treshold = $now - $diff;

    $d = dir($dir);
    while (false !== ($entry = $d->read())) {
        if ($entry != '.' && $entry != '..') {
            $year  = substr($entry, 0, 4);
            $month = substr($entry, 4, 2);
            $day   = substr($entry, 6, 2);

            if (mktime(0, 0, 0, $month, $day, $year) < $treshold)
                exec("/bin/rm -rf ".$dir.$entry);
        }
    }

    $d->close();
}

function getDirUsage($dir) {
    exec("/usr/bin/du -sh ".$dir, $output, $return);

    if ($return > 0)
        return 0;

    $output = $output[0];
    $output = explode("\t", $output);

    return $output[0];
}

/*
 * DELETE OLD FILES, +30d
 */

deleteDir('/media/usb0/samba/cam1/video/', 30);
deleteDir('/media/usb0/samba/cam2/video/', 30);

/*
 * GET USED/FREE SPACE
 */

exec ('df -h', $output);
foreach ($output as $line) {
    if (strpos($line, '/media/usb0')) {
        $disk_line = $line;
        break;
    }
}

$disk_line = explode(" ", $disk_line);
$disk_line = array_values(array_filter($disk_line));

$disk_used_space = $disk_line[2];
$disk_used_perc  = $disk_line[4];
$disk_free_space = $disk_line[3];

/*
 *  GET YESTERDAY RECORDINGS USAGE
 */

$yesterday  = date("Ymd", time() - 60 * 60 * 24);
$cam1_space = getDirUsage('/media/usb0/samba/cam1/video/'.$yesterday.'/');
$cam2_space = getDirUsage('/media/usb0/samba/cam2/video/'.$yesterday.'/');

/*
 *  GET YESTERDAY RECORDINGS USAGE
 */

$yesterday  = date("Ymd", time() - 60 * 60 * 24);
$cam1_space = getDirUsage('/media/usb0/samba/cam1/video/'.$yesterday.'/');
$cam2_space = getDirUsage('/media/usb0/samba/cam2/video/'.$yesterday.'/');

/*
 * SEND REPORT EMAIL
 */

$mail = new PHPMailer();
$mail->IsSMTP();                            // telling the class to use SMTP
$mail->SMTPAuth = true;                     // enable SMTP authentication
$mail->Port     = 25;                       // set the SMTP port
$mail->Host     = "mail.domain.com";        // SMTP server
$mail->Username = "username";               // SMTP account username
$mail->Password = "password";               // SMTP account password

$mail->From     = "email@domain.com";
$mail->FromName = "Descriptive email";
$mail->AddAddress("my_email@domain.com");

$mail->CharSet = "UTF-8";
$mail->Subject  = "Cam Report";
$mail->Body     = "YESTERDAY RECORDINGS\n".
                  "Cam 1: $cam1_space\n".
                  "Cam 2: $cam2_space\n".
                  "\n\n".
                  "HDD SPACE STATUS\n".
                  "Free: $disk_free_space\n".
                  "Used: $disk_used_space ($disk_used_perc)\n";
$mail->WordWrap = 50;

if(!$mail->Send())
        error_log($mail->ErrorInfo);

Then just run it daily with cron
30 3 * * * /usr/bin/php /path/to/script/cams.php > /dev/null

For now that’s all, but i guess there will be more updates on the Raspberry PI as i have still some ideas floating in my head.

Raspberry PI first steps

As every and each true geek, i get myself a Raspberry PI. First error, i ordered only the Raspberry, should have bought also a case (i will buy one soon, but with added shipment costs…).

First choice to make, a cheap decent SD card. I bought a 4GB SD card, 6 class (meaning 6 MB/sec – higher the better). The OS image (Raspbian – yet another debian clone) has a considerable size, almost 2GB, so anything smaller than 4GB is probably not a very good idea…

DSC00437Powered it up, and first problem, even though i had an HDMI connection to my TV screen no output was detected – shit! Besides no output, everything seemed normal and in my router it was registered a new device, so ssh to the assigned IP and could log easy with the default pi/raspberry credentials…. (yes i did tried with another cable and still no output), strangely enough if connected to a monitor with the same HDMI cable works fine and also the TV reads perfectly from other HDMI inputs (Laptop, PS3)….

Still about this, i have found that you can do a lot of tweaking in Raspberry HDMI settings, i will thinker with the various options and then report back:
http://elinux.org/RPi_config.txt#Video_mode_options

After boot you must (or at least should) re-size (or make another) partition to reclaim the entire card space. After that i wanted to connect an external HDD, as documented and expected, if connected directly to one of the Raspberry USB ports there is not enough power and the Raspberry crashes and reboots (so does the HDD).  So, i went out to get a self powered cheap USB hub…

DSC00435I connected the USB Hub, and plugged in the Raspberry to it, worked fine. And then the HDD, also powered up nice and no more crashes and reboots as expected. But the damn thing wasn’t recognized… why? Yet another rookie mistake… the micro USB input on the Raspberry is power input only, doesn’t support data, so i fixed it with another external power adapter providing power to the Pi. I think another cable running from the USB hub would also do the job.

Of course i did setup root ssh access (as all good security practices advise not to do, but what the heck i like to live on the edge). Boot, and minimum setup complete, time to move to the real fun stuff 🙂

Update:
Raspberry PI with Raspbian “Wheezy” connected to a LG 50PV350 trough HDMI. The TV set reports “no signal” and shows no image. To fix this, open the SD card boot partition edit config.txt and add

hdmi_force_hotplug=1
hdmi_group=1
hdmi_mode=16

this will set the PI output to HDMI always even if no device is detected, group 1 means TV (2 is monitor) and mode is set to 1080p.

My bite on Apple

Love it or hate it, it’s the kind of company that leaves no one indifferent. It showed the world (at least to the masses) the personal computer, the graphic interface, the decent portable music player, a real smartphone and what a tablet should be like. All very nice and disruptive. All products are incredible well made, beautiful design and taste, and they just work out of the box like no other in the market. I should love it. But the sad true is i don’t.

Company culture runs top down, and Steve Jobs was indeed a visionary and incredible smart person, but also a manipulative, control freak, indifferent to his own child’s, cruel kind of person (probably a big jerk). Unfortunately a lot of that spilled over to Apple.

Lot’s of (unnecessary) proprietary stuff.
Long story of conflicts with other companies and standards. Adobe, Google, just to name a couple.
Closing the market with all kind of patents and legal tricks.
Mac OS X has a huge portion of FreeBSD on it but i don’t see a cent donated by Apple (list of donors include Cisco, Google, Juniper, NetApp, McAfee, Dell, etc).
I see Google doing some incredible work at a social level, Google foundation, X-Prize. In counterpart is there an Apple foundation?
One buys an Iphone/Ipad and must pay a developer account or jail brake. The SDK is only available for Apple computers.
One must get his OWN files to his OWN device trough Itunes…

And the list goes on and on… does the profit, a vision, or whatever justify this kind of policies? Do the ends justify the means? For Steve Jobs sure, for Apple sure, but not for me. So me choices are rather obvious (check here, here and here).

FREEDOM, FREEDOM, FREEDOM bitch!

One more thing 🙂 … it’s kind of sad that all those geeks that were upset some ten years ago by Microsoft monopolistic actions and autism, most of all switched to Apple, giving their love and support to an even worse company in that matter. You can get in a meeting with IT staff and 90% of those that were running Windows laptops some years ago now proudly show their Apple gear… talk about brainwash… i can only recall the old Apple motto “Think different”… probably will glue this to my Linux laptop one of those days.

Nasty qmail-remote hangs forever bug

I am very happy with my current mail setup. But a nasty bug popped up out of nowhere and i can’t trace it…. Some qmail-remote processes, in some circunstances yet to determine, just hang-up forever eating all the available CPU. The qmail-remote is the piece of Qmail that takes care of message delivery to recipients at a remote host.

When this happens, the stuck process doesn’t conform to the timeoutremote control and stays active forever. The truss command (FreeBSD equivalent to strace) doesn’t show any activity and neither appears to be related network activity… it looks like some kind of race condition.

For now i couldn’t really address the issue, to both lack of time and deep understanding about C and debugging with GDB, so i just mitigate the problem with a cron running a bash script to detect and kill the offending processes.

#!/usr/local/bin/bash

# limit in seconds
LIMIT_TIME=120
# limit in cpu
LIMIT_CPU=35


IFS=$'\n'

for line in `ps -xao pid,etime,command,%cpu | grep qmail-remote`; do

  pid=`echo $line | awk '{split($0,a," "); print a[1]}'`
  time=`echo $line | awk '{split($0,a," "); print a[2]}'`
  cpu=`echo $line | awk '{split($0,a," "); print a[5]}'`
  cpu=`echo "($cpu)/1+1" | bc`

  IFS=$':'
  time_parts=($time)

  if [ ${#time_parts[@]} -lt 3 ]; then
    elapsed=`echo ${time_parts[0]}*60 + ${time_parts[1]} | bc`
  else
    elapsed=`echo ${time_parts[0]}*3600 + ${time_parts[1]}*60 + ${time_parts[2]} | bc`
  fi

  if [ $elapsed -gt $LIMIT_TIME -a $cpu -gt $LIMIT_CPU ]; then
    kill -s 9 $pid
   fi

IFS=$'\n'
done

But i’m not really happy with this “solution”, and will be pursuing a real understanding and solution for this proble.

Some interesting links about other people with the same problem:

http://permalink.gmane.org/gmane.os.freebsd.stable/82760
http://copilotco.com/mail-archives/qmail.2002/msg08733.html

UPDATE and SOLUTION

All credits, to where credits are due.
To replicate this, you should catch an hanging qmail-remote with top. Then filter the offending qmail-remote pid trough ps to get full arguments list:

ps -wwaux | grep pid_number

You should get something like ‘qmail-remote mailserver from@email to@email’. With this information, and with top and truss you can invoke qmail-remote from the command line and get a nice qmail-remote hang…

truss /var/qmail/bin/qmail-remote mailserver my@email nonexisten@email < ./test

test is just a file with some bogus input to serve as the sending email, as qmail-remote expects a message in stdin. Just for the note, most of the hangs happens when talking to the Symantec Email Gateway software.

Now, with an updated ports tree, just recompile qmail, you can follow my guide (all good there). But, just issue make, no need to make install. Then move qmail-remote to /var/qmail/bin/ and set the right permissions (711) and ownership (root:qmail).

And voilá, if you repeat the test procedure, you will find that qmail-remote is not hanging anymore 🙂

FreeBSD FTP mount

Mount a FTP share in your local filesystem is really easy. And it just makes a FTP client feel like dark age software, with a local mount you can freely use your commands over the files and folders in the share.

First install curlftpfs

cd /usr/ports/sysutils/fusefs-curlftpfs/
make install clean

edit /etc/rc.conf and add

fusefs_enable="YES"

and fire it up (actually no daemon here, just a kernel module load), you can see it with kldstat command.

/usr/local/etc/rc.d/fusefs start

Now we can happily mount FTP shares to local filesystem with the curlftpfs command.

curlftpfs ftp://user:pass@ftp.myserver.com /my/local/mount/

If you don’t want the user/pass typed in the command, and probably you don’t because it can be a bad thing ™ (out in the wild waiting for a ps done by other users…), just setup a .netrc file in your home dir with

machine ftp.myserver.com
login myuser
password mypass

then connect without the user/pass like:

curlftpfs ftp://ftp.myserver.com /my/local/mount/

UPDATE

I had some problems with this with a server that was running flawlessly for over a year and it stalled, so i don’t recommend it for production environments, must also test the asynchronous option