Howto: Compile Firefox PGO on Gentoo using an ebuild

Warning! All this doesn’t work at all anymore. Don’t even try it.

This Howto explains how to build Mozilla Firefox with PGO (profile guided optimization) on Gentoo using portage and the ebuild. It’s quite tricky, so make sure the prerequisites are correct on your system.

Prerequisites:

  • Your user has to be in the "portage" group!
  • /var/tmp/portage has to be 775 or 770 (chmod g+w /var/tmp/portage)
  • You need a customized ebuild

I use the mozilla-firefox-3.1_beta3 ebuild from mozilla overlay. The needed run-firefox.sh should be in the files/ subdir. You can svn checkout my modified one from here:

svn co http://gimpel.ath.cx/svn/www-client/mozilla-firefox/

Just put that in your overlay.

Make sure to enable the useflags "pgo -xulrunner" in your /etc/portage/package.use file!

Now as user we call the ebuild phases manually

cd /path/to/your/overlay/www-client/mozilla-firefox
ebuild mozilla-firefox-3.1_beta3-r1.ebuild compile

This will start the compile, after the first PGO run, the browser will open up, and you should do some usual browsing for some minutes. Maybe go to the spidermonkey JS speed test page. When done, just close the browser.

Now the second run will be done. When finished, we install it as root with

sudo ebuild mozilla-firefox-3.1_beta3-r1.ebuild qmerge

Howto control Tomcat using wget

I just had to restart a webapp in Tomcat without stopping a second app running in the same tomcat instance.

Usually this can be done easily via the Tomcat Manager, but in this case I was not able to access the Manager due to firewall rules. Though I was able to access the server using ssh, but there was no curl installed.

Luckily wget did the trick too!

wget \
--http-user=manager-user \
--http-password=manager-password \
-q -O - http://localhost:8080/manager/html/reload?path=/test \
| grep -A1 Message|awk -F'>' '{print $NF}'

OK - Started application at context path /test

Perfect!

Howto import SAR data into a PostgreSQL database and create graphs in OpenOffice.org

This is a walkthrough on how to get SAR data into a database, and use OpenOffice.org to create nice graphs using a JDBC connection to that database, and SUN Report Builder to do the graphs.

In this example I will use SAR’s CPU utilisation data.

Software used:

 

1. Setup the Database

I use a PostgreSQL database named sar here. The SAR data is on the same host as the database, and the PostgreSQL user has read permission on the SAR files.

First we need to create a table

CREATE TABLE sar_cpu 
(
"host" varchar, 
"interval" numeric, 
"time" timestamp with time zone, 
"offset" numeric(1), 
"user" numeric(5,2), 
"nice" numeric(5,2), 
"system" numeric(5,2), 
"iowait" numeric(5,2), 
"steal" numeric(5,2), 
"idle" numeric(5,2)
)

 

 

2. Convert SAR data to CSV

Then we need to output the SAR data in a format we can load into the database, using sadf.

LC_ALL=en_US sadf -d /var/log/sa/sa15 -- -u > /tmp/sa20090215.csv

Note: The LC_ALL=en_US ensures that decimal values are seperated with a dot, not a comma.

 

3. Load the CSV directly into the database

Then we can load that CSV directly into the database:

echo "COPY sar_cpu FROM '/tmp/sa20090215.csv' DELIMITERS ';' CSV;"|psql -d sar

Cool, isn’t it? This can be scripted easily and controlled via a cronjob, like import “yesterday’s sar data, using a simple wrapper script and some date function.

 

4. Create an OpenOffice.org Base DB and connect it to PostgreSQL

Now start OOo Base, create a new DB -> JDBC.

See http://jdbc.postgresql.org/documentation/83/connect.html for an overwiev about the connection parameters:

URL:

jdbc:postgresql:sar

Class:

org.postgresql.Driver

 

5. Install SUN Report Builder

Then download SRB from http://extensions.services.openoffice.org/project/reportdesign, open Extras -> Extension Manager, and install it.

In OOo Base you’ll find it in Reports -> Report in Design View

There you have the possibility to create graphs and diagrams as usual in Calc.

Howto connect OpenOffice to MS SQL server

Just had to connect to a MS SQL server that holds collected SAR statistics from our production servers.

Until now I used Toad in a virtual XP machine, but there has to be some way to connect in a graphical way from Linux too. And indeed there is!

Tools needed:

I chose JTDS, got the JAR file and copied it somehwere in the classpath. On OpenSUSE that is /usr/share/java. You can also add it in OpenOffice directly: Extras -> Options -> Java -> Classpath

Then fire up OpenOffice Base, and in the database wizard

  1. Open existing Database
  2. JDBC
jdbc:jtds:sqlserver://hostname:portnumber/databasename

The port number usually is 1433. The JDBC driver class is

net.sourceforge.jtds.jdbc.Driver

Then enter the db username, in the next step enter the password, and connect.

That’s it.

 

BASH arrays and filenames with spaces

I’m sure I can use this one quite often.

1) Arrays are nice to avaoid temporaray files

2) Spaces in file names are a PITA. But there are nice ways to work around this

Array[0]="/some/path/to/mooo foo.txt"
Array[1]="/some/path/to/mooo foo bar 2.txt"
Array[2]="/some/path/to/mooo foo bar 3.txt"

There we have some example array.

for ((i=0;i<${#Array[@]};i++)); do echo ${Array[$i]}; done
/some/path/to/mooo foo.txt
/some/path/to/mooo foo bar 2.txt
/some/path/to/mooo foo bar 3.txt

Ok, now what if want to pass that to ls, we could quote it somehow. Or use some pattern expansion to replace the space with backslash space, like

for ((i=0;i<${#Array[@]};i++)); do echo ${Array[$i]// /\\ }; done
/some/path/to/mooo\ foo.txt
/some/path/to/mooo\ foo\ bar\ 2.txt
/some/path/to/mooo\ foo\ bar\ 3.txt

Great! Another way would be to temporarily change the special variable $IFS that determines which character is used for word splitting. Let’s set this to “newline”

OLDIFS="$IFS"
IFS=$'\n'

for ((i=0;i<${#Array[$i]};i+=1)); do
   ls ${Array[$i]}
done

IFS="$OLDIFS"

That $IFS change can also be used in all sorts of other loops. Hell, I’m going to need that really often. Thanks to the guys on http://forums.gentoo.org/viewtopic-t-714971.html

Bash pattern matching

BASH offers a nice way to replace patterns. Until now I always used basename and a variable.
For example moving all .jpeg files in a folder to .jpg (hey, it’s just an example..)

for moo in *.jpeg; do
    newname="`basename $moo .jpeg`.jpg"
    mv $moo $newname
done

But isn’t that just plain ugly? That one is lot nicer:

for i in *.jpeg; do
    mv "$i" "${i%.*}.jpg"
done

1337