Howtos

Bash pattern matching

Add new comment

BASH offers a nice way to replace patterns. Until now I always used basename and a variable.
For example moving all .jpeg files in a folder to .jpg (hey, it's just an example..)

for moo in *.jpeg; do
    newname="`basename $moo .jpeg`.jpg"
    mv $moo $newname
done

But isn't that just plain ugly? That one is lot nicer:

for i in *.jpeg; do
    mv "$i" "${i%.*}.jpg"
done

1337

Tags:

Howto connect OpenOffice to MS SQL server

Just had to connect to a MS SQL server that holds collected SAR statistics from our production servers.

Until now I used Toad in a virtual XP machine, but there has to be some way to connect in a graphical way from Linux too. And indeed there is!

Tools needed:

OpenOffice Base
JTDS connector from http://jtds.sourceforge.net/
or Microsoft's JDBC driver: http://msdn.microsoft.com/en-us/data/aa937724.aspx

I chose JTDS, got the JAR file and copied it somehwere in the classpath. On OpenSUSE that is /usr/share/java. You can also add it in OpenOffice directly: Extras -> Options -> Java -> Classpath

Then fire up OpenOffice Base, and in the database wizard

Open existing Database
JDBC

jdbc:jtds:sqlserver://hostname:portnumber/databasename

The port number usually is 1433. The JDBC driver class is

net.sourceforge.jtds.jdbc.Driver

Then enter the db username, in the next step enter the password, and connect.

That's it.

Tags:

Howto import SAR data into a PostgreSQL database and create graphs in OpenOffice.org

This is a walkthrough on how to get SAR data into a database, and use OpenOffice.org to create nice graphs using a JDBC connection to that database, and SUN Report Builder to do the graphs.

In this example I will use SAR's CPU utilisation data.

Software used:

1. Setup the Database

I use a PostgreSQL database named sar here. The SAR data is on the same host as the database, and the PostgreSQL user has read permission on the SAR files.

First we need to create a table

CREATE TABLE sar_cpu 
(
"host" varchar, 
"interval" numeric, 
"time" timestamp with time zone, 
"offset" numeric(1), 
"user" numeric(5,2), 
"nice" numeric(5,2), 
"system" numeric(5,2), 
"iowait" numeric(5,2), 
"steal" numeric(5,2), 
"idle" numeric(5,2)
)

2. Convert SAR data to CSV

Then we need to output the SAR data in a format we can load into the database, using sadf.

LC_ALL=en_US sadf -d /var/log/sa/sa15 -- -u > /tmp/sa20090215.csv

Note: The LC_ALL=en_US ensures that decimal values are seperated with a dot, not a comma.

3. Load the CSV directly into the database

Then we can load that CSV directly into the database:

echo "COPY sar_cpu FROM '/tmp/sa20090215.csv' DELIMITERS ';' CSV;"|psql -d sar

Cool, isn't it? This can be scripted easily and controlled via a cronjob, like import "yesterday's sar data, using a simple wrapper script and some date function.

4. Create an OpenOffice.org Base DB and connect it to PostgreSQL

Now start OOo Base, create a new DB -> JDBC.

See http://jdbc.postgresql.org/documentation/83/connect.html for an overwiev about the connection parameters:

URL:

jdbc:postgresql:sar

Class:

org.postgresql.Driver

5. Install SUN Report Builder

Then download SRB from http://extensions.services.openoffice.org/project/reportdesign, open Extras -> Extension Manager, and install it.

In OOo Base you'll find it in Reports -> Report in Design View

There you have the possibility to create graphs and diagrams as usual in Calc.

Tags:

Howto use KDE4's device notifier to play a DVD in SMplayer

Just finally figured this one out! It's possible to add an entry to KDE4's device notifier, so you can just click & play a DVD in SMplayer. Awesome!

You need a very recent SMplayer and Mplayer, latter one compiled with dvdnav support.

In SMplayer's preferences, go to mouse & keyboard -> mouse, and for left click select "Activate option in DVD menus".
Create a file called smplayer-playdvd-predicate.desktop
in $KDEDIR/share/apps/solid/actions, here that means /usr/kde/svn/share/apps/solid/actions/smplayer-playdvd-predicate.desktop, with following in it:

[Desktop Entry]
X-KDE-Solid-Predicate=[ StorageVolume.ignored == false AND OpticalDisc.availableContent == 'Data|VideoDvd' ]
Type=Service
Actions=open;

[Desktop Action open]
Name=Play DVD with SMplayer
Exec=smplayer dvdnav:////%d
Icon=smplayer

Now restart KDE, insert a DVD in your drive, or load DVD ISO image in cdemu, wait for the device notifier to pop up, and select.

Device notifier play DVD in SMplayer

That's it!

Tags:

Howto: Convert avi/mov/any video to Flash/flv on Linux

Add new comment

Recently I wanted to convert a Quicktime .mov video to Flash .flv in order to post it on this page, for the fun of it.
First thought: uuuh how do I do this on Linux? But it's pretty simple, thanks to ffmpeg. Works fine with anything that ffmpeg can play, so avi, mpeg, wmv or whatever.

It's as simple as:

ffmpeg -i input.avi -s 320x240 -ar 44100 -o movie.flv

There is also a nice tool to add metadata info like duration etc to flv files, called flvtool2.

cat movie.flv | flvtool2 -U stdin movie.flv

Now your flv has duration info and is ready for the website.

Tags:

Multimedia

Howto: Log firewall from OpenWrt to a remote rsyslog

Add new comment

At Planethome I'm currently setting up a central logserver, using rsyslog. So to get familiar with it, I installed it on my Gentoo box at home, set it as default syslog daemon, and feed it with data from my router running OpenWrt Kamikaze 8.09 RC1.

On the server (Gentoo) side, I enabled remote logging over UDP. On the client (OpenWrt) side a little bit more had to be done

1) Enable remote logging

Edit /etc/config/system and enable remote logging by adding:

option 'log_ip' '192.168.1.2'

Now reboot the router and see if it logs correctly.

2) -j LOG ++

Then I had to get IPtables to produce some log output. With Kamikaze's new firewall config layout this was a bit tricky. I decided to just log SYN flood protection actions, and the dropping of INVALID packets on INPUT and FORWARD chains. Therefore we need to edit /lib/firewall/uci_firewall.sh and add 3 lines (those with -j LOG)

In function fw_defaults()

$IPTABLES -A INPUT -m state --state INVALID -j LOG --log-prefix "DROP INVALID (INPUT): "
$IPTABLES -A INPUT -m state --state INVALID -j DROP
...
$IPTABLES -A FORWARD -m state --state INVALID -j LOG --log-prefix "DROP INVALID (FORWARD): "
$IPTABLES -A FORWARD -m state --state INVALID -j DROP

and for the SYN flood stuff, in function load_synflood()

$IPTABLES -A syn_flood -j LOG --log-prefix "SYN FLOOD: "
$IPTABLES -A syn_flood -j DROP

Tags:

Networking

Howto: receive mail and save attachment with fetchmail, procmail and metamail

Add new comment

At work I recently had to set up a solution that periodically checks a POP3 account on our M$ Exchange wannabe mailserver, and saves the attachments to some folder for further processing. As I didn't find a ready-to-go-solution for this on the web, just snippets here and there, and of course hundrets of other people asking the same, here it is.

You will need

fetchmail
procmail
metamail (or uudeview, see first comment below)

In this example I'll use a POP3 account, the full mail will be backed up to ~/mail_backup, and attachments will be unpacked to ~/attachments. fetchmail also handles IMAP accounts just fine. Please refer to the fetchmail documentation.

Setting up fetchmail

First create a file $HOME/.fetchmailrc

poll my.pop3.server 
protocol pop3 
user 'myuser' 
password 'mypassword' 
mda '/usr/bin/procmail -d %T'

Setting up procmail and metamail

Then we configure procmail so it forwards the messages to metamail in $HOME/.procmailrc

:0
*^content-Type:
{
        # backup the complete mail first..
        # you can leave out this part if you don't want a backup of the complete mail
        :0c:
        $HOME/mail_backup

        # Now the actual unpacking part
        #
        # this is the place where the attachments will be unpacked to
        METAMAIL_TMPDIR=$HOME/attachments

        # forward to metamail
        :0fw
        | metamail -w -y -x
}

Regarding metamail, we tell it to ignore any mailcap file, so it doesn't use interpreters (-w), yanking the message and save the content raw (-y) and force it in non-interactive mode (-x).

That's it about it. We are ready for testing.

Test run

Now we simply fire up fetchmail, the rest should be magic.

fetchmail -kv

HTH

Tags:

Howto create a youtube video from mp3/ogg audio using a picture

If you want to create a youtube video from an audio file, here is how to do this.
All you need is the audio file, a single picture, and ffmpeg.

First find out the lenght of the audio file in seconds, you'll need it. Here is an example with a 420 seconds file:

ffmpeg -loop_input -i picture.jpg -vcodec mpeg4 -r 25.00 -qscale 2 -s 480x360 \
-i audiofile.mp3 -acodec libmp3lame -ab 128k -t 420 video.avi

This will create a Hi-Res MPEG-4 video with 128k audio. The trick here is to use that one picture and loop it for -t seconds.

Tags:

Multimedia