lsmcli with netapp on ubuntu

Skimming through the RHEL 7.1 Release Notes on Storage I noticed libStorageMgmt, and got curious. Netapp provides an oncommand system manager rpm, which when extracted works on ubuntu too, but it starts a web application on localhost and then opens a browser, and that's how you manage the storage. Yuck! So let's try this libStorageMgmt stuff:

$ suod apt-get install libstoragemgmtd libstoragemgmt-tools

$ lsmcli
Traceback (most recent call last):
  File "/usr/bin/lsmcli", line 19, in 
    import lsm.cmdline
  File "/usr/lib/python2.7/dist-packages/lsm/__init__.py", line 23, in 
    from smis import Smis
  File "/usr/lib/python2.7/dist-packages/lsm/smis.py", line 20, in 
    import pywbem
ImportError: No module named pywbem

$ sudo apt-get install python-pywbem

$ sudo service libstoragemgmtd start
$ sudo gpasswd -a paul.tobias libstoragemgmt

$ lsmcli --list=SYSTEMS
ID         | Name     | Status
-----------+----------+-------
1918101635 | netapp01 | OK    

$ lsmcli --list POOLS
ID                                   | Name  | Total space    | Free space    | System ID 
-------------------------------------+-------+----------------+---------------+-----------
c6b01446-7146-11e3-a01e-123478563412 | aggr0 | 11322462519296 | 3456236142592 | 1918101635

$ lsmcli --list POOLS -H
ID                                   | Name  | Total space | Free space | System ID 
-------------------------------------+-------+-------------+------------+-----------
c6b01446-7146-11e3-a01e-123478563412 | aggr0 | 10.30 TiB   | 3.14 TiB   | 1918101635

$ lsmcli --list FS -H
ID                                   | Name                      | Total space | Free space | Pool ID                             
-------------------------------------+---------------------------+-------------+------------+-------------------------------------
c7b367ea-7146-11e3-a01e-123478563412 | vol0                      | 171.00 GiB  | 164.84 GiB | c6b01446-7146-11e3-a01e-123478563412
7ff4e8db-bb3b-11e3-821a-123478563412 | vms                       | 4.75 TiB    | 1.63 TiB   | c6b01446-7146-11e3-a01e-123478563412
04d28a88-12f9-425a-8149-5b70d1d9951d | brush_backup_restore_test | 2.50 TiB    | 2.46 TiB   | c6b01446-7146-11e3-a01e-123478563412
2f228ee9-4b09-4ce2-9e05-ca5bcdbbcc7c | brush_extra               | 1.00 TiB    | 63.63 GiB  | c6b01446-7146-11e3-a01e-123478563412
5548a33d-d0f7-4dd1-ac49-3ca07f5d76a9 | test                      | 1.24 GiB    | 1.23 GiB   | c6b01446-7146-11e3-a01e-123478563412

$ lsmcli --list EXPORTS
Key            | Value                                                  
---------------+--------------------------------------------------------
ID             | 998e7bde67701ad96ebb14cbb5927c90                       
File system ID | 7ff4e8db-bb3b-11e3-821a-123478563412                   
Export Path    | /vol/vms                                               
Authentication | sys                                                    
Root           | 172.28.50.100, 172.28.50.63, 172.28.50.62, 172.28.50.61
Read/Write     | *                                                      
ReadOnly       | None                                                   
Anon UID       | None                                                   
Anon GID       | None                                                   
Options        | None                                                   
ID             | f962aa0dc595c52f676bc3d52b8e9c92                       
File system ID | c7b367ea-7146-11e3-a01e-123478563412                   
Export Path    | /vol/vol0                                              
Authentication | sys                                                    
Root           | 172.28.50.21                                           
Read/Write     | 172.28.50.21                                           
ReadOnly       | None                                                   
Anon UID       | None                                                   
Anon GID       | None                                                   
Options        | None                                                   
ID             | f962aa0dc595c52f676bc3d52b8e9c92                       
File system ID | c7b367ea-7146-11e3-a01e-123478563412                   
Export Path    | /vol/vol0/home                                         
Authentication | sys                                                    
Root           | 172.28.50.21                                           
Read/Write     | *                                                      
ReadOnly       | None                                                   
Anon UID       | None                                                   
Anon GID       | None                                                   
Options        | None                                                   
ID             | 8aa21057ed71d0c1b579db8ea1f3e5ae                       
File system ID | 2f228ee9-4b09-4ce2-9e05-ca5bcdbbcc7c                   
Export Path    | /vol/brush_extra                                       
Authentication | sys                                                    
Root           | 172.28.5000.21, 172.28.50.100                          
Read/Write     | *                                                      
ReadOnly       | None                                                   
Anon UID       | None                                                   
Anon GID       | None                                                   
Options        | None                                                   
ID             | ca24797f40f9b55a2c6aa8174f1f0fee                       
File system ID | 04d28a88-12f9-425a-8149-5b70d1d9951d                   
Export Path    | /vol/brush_backup_restore_test                         
Authentication | sys                                                    
Root           | 172.28.50.100                                          
Read/Write     | 172.28.50.100                                          
ReadOnly       | None                                                   
Anon UID       | None                                                   
Anon GID       | None                                                   
Options        | None                                                   

$ lsmcli --create-fs=test --size 1G --pool c6b01446-7146-11e3-a01e-123478563412
ID                                   | Name | Total space | Free space | Pool ID                             
-------------------------------------+------+-------------+------------+-------------------------------------
5548a33d-d0f7-4dd1-ac49-3ca07f5d76a9 | test |  1326075904 | 1325924352 | c6b01446-7146-11e3-a01e-123478563412

Sweet! I can finally forget that oncommand system manager and use the command line to manage netapp.

How to install sun java 5 on Ubuntu 10.04 Lucid

We sometimes have requests to install the sun java 5 JDK on recent Ubuntu distributions. The problem is that Ubuntu 9.04 Jauty was the last release of Ubuntu to include sun java5 packages. So here is how I do it:

First, download the required packages:

wget http://de.archive.ubuntu.com/ubuntu/pool/multiverse/s/sun-java5/{ia32-sun-java5-bin_1.5.0-22-0ubuntu0.8.04_amd64.deb,sun-java5-bin_1.5.0-22-0ubuntu0.8.04_amd64.deb,sun-java5-demo_1.5.0-22-0ubuntu0.8.04_amd64.deb,sun-java5-jdk_1.5.0-22-0ubuntu0.8.04_amd64.deb,sun-java5-jre_1.5.0-22-0ubuntu0.8.04_all.deb}


Now, install them with dpkg -i:

dpkg -i ia32-sun-java5-bin_1.5.0-22-0ubuntu0.8.04_amd64.deb sun-java5-bin_1.5.0-22-0ubuntu0.8.04_amd64.deb sun-java5-demo_1.5.0-22-0ubuntu0.8.04_amd64.deb sun-java5-jdk_1.5.0-22-0ubuntu0.8.04_amd64.deb sun-java5-jre_1.5.0-22-0ubuntu0.8.04_all.deb


You will get unresolved dependency problems, but we are going to fix them:

aptitude full-upgrade


Keep pressing n until aptitude doesn't want to remove or downgrade any packages, only install. The solution will look like this:

The following actions will resolve these dependencies:

Install the following packages:
ia32-libs [2.7ubuntu25 (lucid)]
lib32asound2 [1.0.22-0ubuntu7 (lucid)]
lib32bz2-1.0 [1.0.5-4 (lucid)]
lib32gcc1 [1:4.4.3-4ubuntu5 (lucid)]
lib32ncurses5 [5.7+20090803-2ubuntu3 (lucid)]
lib32stdc++6 [4.4.3-4ubuntu5 (lucid)]
lib32v4l-0 [0.6.4-1ubuntu1 (lucid)]
lib32z1 [1:1.2.3.3.dfsg-15ubuntu1 (lucid)]
libc6-i386 [2.11.1-0ubuntu7.1 (lucid-updates, lucid-security)]
odbcinst [2.2.11-21 (lucid)]
odbcinst1debian1 [2.2.11-21 (lucid)]
unixodbc [2.2.11-21 (lucid)]

Score is -58


After accepting the solution and installing the missing packages, you will have both 32 and 64bit sun java5 installed:

# update-alternatives --list java
/usr/lib/jvm/ia32-java-1.5.0-sun/jre/bin/java
/usr/lib/jvm/java-1.5.0-sun/jre/bin/java
/usr/lib/jvm/java-6-openjdk/jre/bin/java


And you can use one of these values, as JAVA_HOME:

# java 1.5 32bit:
JAVA_HOME="/usr/lib/jvm/ia32-java-1.5.0-sun"
# java 1.5 64bit:
JAVA_HOME="/usr/lib/jvm/java-1.5.0-sun"
# java 1.6 64bit:
JAVA_HOME="/usr/lib/jvm/java-6-sun"


How to install open source vmware tools on Ubuntu lucid

I did not find a simple and straightforward solution anywhere, so here it is:

apt-get install --no-install-recommends open-vm-source module-assistant
module-assistant prepare open-vm
module-assistant auto-install open-vm
apt-get install --no-install-recommends open-vm-tools

This approach has a nice side effect: when you update your kernel, the modules gets recompiled too.

Accessing services behind NAT on public address

We have a gateway with haproxy+tproxy. In the haproxy.cfg we have transparent proxying:

source 0.0.0.0 usesrc clientip


Behind the gateway there are several NAT-ed hosts, acting as backends for haproxy. Everythong works fine, but when we want to access a website served by haproxy from the NAT-ed hosts, we get an 503 error, after a timeout.

The problem is the following:
Let there be host A, with ip address 172.21.0.2. This hosts wants to connect to the site hosted on B (172.21.0.3). This website has an address of B.example.com, which resolves to the public IP address on the gateway. The gateway receives a packet from A, and forwards it to the backend server B, but because of the tproxy setup, haproxy keeps the source IP address of A. Now B receives a packet from A and sends a response to A's IP address. But because they are on the same subnet, the packet will reach A directly and haproxy won't see it. A now received a packet from B (172.21.0.3), but he is expecting a packet from the public interface ip address, so he discards it and keep trying to connect. The gateway cannot see a reply from B either, so after a timeout haproxy sends back an 503 error reply to A.

After some trial and error I remembered a presentation from Kadlecsik Jozsef (in hungarian), where he also talks about how to address this problem. He talks about several solutions, and the NETMAP one sounds good for our setup.

So on the gateway we issue the following command:

iptables -t nat -I POSTROUTING -o eth1 -s 172.21.0.0/16 -j NETMAP --to 172.31.0.0/16

And voila! Everything works.

Some explanation about the command: The eth1 interface is the LAN interface, the 172.21.0.0/16 is the LAN subnet, and the 172.31.0.0/16 is an imaginary unused subnet. Packets originating from the LAN and going back to the same LAN will now have a source IP address from the 172.31.x.x range. And because LAN host don't have a route for this subnet, they will route their answer back through the default gateway, which will map the imaginary addresses back to their original.

But the same iptables command with the NETMAP target can be used on any other NAT-ed LAN. This way port forwarding from the public address can be used inside from the LAN too.

Update: There is a slight issue with the above command. Packets originating from the gateway machine will have their IP addresses NETMAP-ed too, which is not pretty. But we can fix that too:

iptables -t nat -I POSTROUTING -m addrtype ! --src-type LOCAL -o eth1 -s 172.21.0.0/16 -j NETMAP --to 172.31.0.0/16