Project T5220, try 1: How NOT to configure a T5220 as a complete Oracle/Weblogic development environment

Currently I have to configure a Sun Enterprise T5220 as our new “developemnt environment”, replacing our V440 (Oracle) and the T1000 (Weblogic). We chose a Sun CMT machine as we wanted to stay close to production in regards of architecture (processor type/OS etc).

The machine will have to run an Oracle instance, and some containers for the Weblogic environments (we have an integration env, daily build env + some development environments for projects). The new T5220 ist packed with a 1.4GHz T2 Niagara, 64GB of memory, 2x146GB for the OSes, and 6x300GB for the database and NFS.

Given that, the ideal setup sounds like:

  • 2 Guest LDOMs, where:
  • 1 Oracle-LDOM
  • 1 Weblogic LDOM, using Solaris zones to seperate the environments
  • The setup is 100% ZFS
  • The control domain runs on one slice of the 146GB drives in ZFS raid1
  • The guest domain roots run on the other slice of that disk, seperate zpool, exported as ZVOL
  • The 6 300GB disks are exported as raw disk slices (EFI labeled), and formed into a ZFS raidz inside the Oracle LDOM
  • Inside that raidz zpool, there is a ZFS with recordsize=8k for the Oracle datafiles, and a ZFS with 128k blocksize for the redo logs

Regarding the database, this is pretty similar to our current setup, a SunFire V440 with a ST2540 FC-Storage attached, running ZFS on top of the hardware RAID5. Similar in regards of the filesystem, at least. ZFS runs very well on the V440.

Okay, now, after setting this up, it turned out that the DB performance is unacceptable. Absolutely horrible to be honest. We’re doing large imports of our production database on a regular basis, and on the V440 it takes about 50 minutes, now we’re up to 150 minutes on the T5220.

Here are some numbers. Simple dd testing: creating a 15GB file.

V440 with St2540 RAID5 volume, exported via 2x2Gbit FC, configured as a simple ZFS

$ time dd if=/dev/zero of=ddtest.file bs=8k count=2000000
2000000+0 records in
2000000+0 records out

real    2m32.380s
user    0m3.021s
sys     1m27.533s
$ echo "((16384000000/152)/1024)/1024"|bc -l

T5220 with 10k/rpm local SAS, exported as raw disk slice into guest LDOM and configured as ZFS RAIDz:

$ echo "((16384000000/336)/1024)/1024"|bc -l

Now things get strange. Inside the control domain, onto the ldompool:

$ echo "((1638400000/35)/1024)/1024"|bc -l


That is worse than I get with a Linux guest in VirtualBox onto a virtual drive. Not acceptable for 300€ 10k rpm/s SAS drives.

Me asking for help and further infos at Sun forums

Anyways, I’m going to attach the documented step-by-step guide on how to setup all this from scratch (M$-Word .doc, in German, sorry)

Solaris and DNS

Just that I’ll never have to google this one up again….

To get Solaris to make use of your ususal DNS

cp /etc/nsswitch.conf /etc/nsswitch.conf.orig
cp /etc/nsswitch.dns /etc/nsswitch.conf
echo "nameserver xxx.yyy.zzz.bla" >> /etc/resolv.conf
svcadm restart nscd

There we go.