Weblogic 12c WLST compatibility with Windows Server 2016

During the course of my work I've had to support the upgrade of some legacy applications running on Weblogic 12c (WebLogic Server Version: 12.1.2.0.0) from Windows Server 2008 to Windows Server 2016. These particular domains use a WLST script for deployment, essentially tearing down and re-creating the domains when deployed. This started failing after the servers were upgraded to Windows 2016, with the following error:

Failed to get environment, environ will be empty: (0, ‘Failed to execute command ([\’sh\’, \’-c\’, \’env\’]): java.io.IOException: Cannot run program “sh”: CreateProcess error=2, The system cannot find the file specified’)

This happens because the built-in WLST Jython interpreter is unable to recognise the OS version (Windows 2016) that it is running on. Upgrading the Weblogic installation itself was not an available option in this case, sow e updated the WLST to accept the newer operating system:

1. Open the following jar file: \wlserver\common\wlst\modules\jython-modules.jar
2. In that jython-modules.jar file, edit the file javashell.py file and search for the _getOsType function on line 136:

   def _getOsType( os=None ):
       """Select the OS behavior based on os argument, 'python.os' registry
       setting and 'os.name' Java property.
       os: explicitly select desired OS. os=None to autodetect, os='None' to
       disable
       """
       os = str(os or sys.registry.getProperty( "python.os" ) or \
                  System.getProperty( "os.name" ))
            
       _osTypeMap = (
           ( "nt", ( 'nt', 'Windows NT', 'Windows NT 4.0', 'WindowsNT',
                     'Windows 2000', 'Windows 2003', 'Windows XP', 'Windows CE',
                     'Windows Vista', 'Windows Server 2008', 'Windows 7', 'Windows 8' )),
           ( "dos", ( 'dos', 'Windows 95', 'Windows 98', 'Windows ME' )),
           ( "mac", ( 'mac', 'MacOS', 'Darwin' )),
           ( "None", ( 'None', )),
           )
       foundType = None
       for osType, patterns in _osTypeMap:
           for pattern in patterns:
               if os.startswith( pattern ):
                   foundType = osType
                   break
           if foundType:
               break
       if not foundType:
           foundType = "posix" # default - posix seems to vary most widely
    
       return foundType
   

3. Edit the _getOsType function to add the missing OS version, and for safety, set it to default to "nt" instead of "posix" if the OS is not recognised (we've seen some variations of what Jython detects as the OS):

   def _getOsType( os=None ):
       """Select the OS behavior based on os argument, 'python.os' registry
       setting and 'os.name' Java property.
       os: explicitly select desired OS. os=None to autodetect, os='None' to
       disable
       """
       os = str(os or sys.registry.getProperty( "python.os" ) or \
                  System.getProperty( "os.name" ))
            
       _osTypeMap = (
           ( "nt", ( 'nt', 'Windows NT', 'Windows NT 4.0', 'WindowsNT',
                     'Windows 2000', 'Windows 2003', 'Windows XP', 'Windows CE',
                     'Windows Vista', 'Windows Server 2008', 'Windows Server 2012 R2', 'Windows 7', 'Windows 8', 'Windows 10', 'Windows NT (unknown)' )),
           ( "dos", ( 'dos', 'Windows 95', 'Windows 98', 'Windows ME' )),
           ( "mac", ( 'mac', 'MacOS', 'Darwin' )),
           ( "None", ( 'None', )),
           )
       foundType = None
       for osType, patterns in _osTypeMap:
           for pattern in patterns:
               if os.startswith( pattern ):
                   foundType = osType
                   break
           if foundType:
               break
       if not foundType:
           foundType = "nt" # default to NT
    
       return foundType
   

WLST will now be able to detect the OS, or default to NT if it doesn't; either way, it will now again be able to work correctly on Windows.

Mission Failed: Disabling UART completely on a Raspberry Pi

Pins 8 and 10 on the Raspberry Pi is usually configured to act as the transmit and receive pins for an RS-232 serial port (the Pi's mini UART). However, if you want to use these as plain GPIO pins, you can do so by simply configuring them as GPIO inputs or -outputs using the python-GPIO library... until your Raspberry Pi reboots, that is.

Upon reboot, you will find that one or both of these pins are set to high due to them being booted into ALT0 (read: UART) mode. This obviously causes problems if you are using these pins for GPIO in an application (unless you can ensure that your Pi is *never* rebooted). I have tried in vain to find a solution for this, but it seems to be a firmware issue (or at least baked deeply into the kernel). The following post simply documents what I tried, but it provides insight on two things: how to deactivate the default serial console running on /dev/ttyAMA0, and how to compile a custom kernel for your Raspberry Pi.

tl;dr: Tried to completely disable UART (pins 8 and 10) on a Raspberry Pi by forcing it not to boot into ALT0 mode. You can't, it seems - or at least I have not found a method. Please leave comment if you know of a way!

My attempt: disable serial console, then remove UART drivers from kernel

Note: I started down this path because pins 8 and 10 on the Pi are not driven high when the Pi is turned on without an SD card (i.e. without an operating system). This led me to suspect that the OS is setting the pins into ALT0 mode in software. As far as I can tell, I was wrong. Now I suspect it's the firmware, but that's sadly not solved as easily.

The serial console device /dev/ttyAMA0 is created automatically by the default Raspbian kernel (since it uses it for diagnostic- and boot messages, and as a login console) and because of this, the UART driver is compiled directly into to the kernel and not as an unloadable module. The following steps changes that, first by deactivating most serial messages and the serial login prompt. Next, the kernel is recompiled to build the UART driver as a separate unloadable module. Finally, the UART driver is disabled.

1. Disable boot- and debug message logging to the serial console (this is configured to be active by default on Raspbian):
   
   Edit /boot/cmdline.txt:

   sudo vim /boot/cmdline.txt

   Remove the options referring to the serial console ("/dev/ttyAMA0"); in other words, change it from this:
   
   
   dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait
   
   to this:
   
   
   dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait
   
   
2. Next you must disable the serial console login prompt. Edit /etc/inittab:
   

   sudo vim /etc/inittab

   At the end of the file you will find the following line:

   T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100

   Disable it by commenting it out (put a # character at the start of the line), or delete it.


3. Longest step: Kernel recompilation
   Before you start, make sure your system is up to date:

   sudo apt-get update
   sudo apt-get dist-upgrade
   

   Next, install the compiler and some packages needed for recompiling the kernel:

   sudo apt-get install gcc make git libncurses5-dev

   Make and change into a directory in which you can build the kernel:

   sudo mkdir /opt/build
   sudo chmod 777 /opt/build
   cd /opt/build
   

   Download a copy of the kernel source from the github repository:

   git clone --depth 1 git://github.com/raspberrypi/linux.git
   cd linux

   Now you are ready to configure and rebuild the kernel. First, make sure your build directory is clean:

   make mrproper

   Next, save the configuration of your currently-running kernel (this allows your new kernel to be configured based on your current kernel settings, which saves a lot of time):

   zcat /proc/config.gz > .config

   To ensure that your current kernel configuration is up-to-date with the kernel you downloaded, do:

   make oldconfig

   Now we are going to modify the kernel to modularize the UART driver and remove bootup dependencies on it:

   make menuconfig

   You will be presented with the Linux kernel configuration menu. Since the configuration is based on your currently-running kernel's configuration, we only need to change two things:
   
   
   1. Disable all /dev/ttyAMA0 kernel boot messages: Highlight "Boot options" and press [ENTER]. Now highlight the "Default kernel command string"; it will look like the following:
      
      
      (dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait)
      
      Press [ENTER] to select it. Edit the string and remove the "console" and "kgdboc" options; i.e. change it to:
      
      
      dwc_otg.lpm_enable=0 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait
      
      Once edited, press [ESC] twice to go back to the previous level. Continue pressing [ESC] until you get back to the main menu.
      
      
   2. Modularize UART driver (so that it can be disabled): From the main kernel configuration menu, go to Device Drivers -> Character devices -> Serial drivers, then highlight "ARM AMBA PL011 serial port support" and press M to modularize it.
      
      Now press [ESC] until you get back to the main menu. Press [ESC] twice again, and save the configuration when prompted.
   You are now ready to start the build. Build the kernel and its modules by entering:

   make && make modules

4. Install the custom kernel
   Your newly-built kernel image will be /opt/build/linux/arch/arm/boot/Image. Copy the new kernel image to the Raspberry Pi's boot partition, using a different name in case you need to roll back later:

   sudo cp -v arch/arm/boot/Image /boot/kernel_mod_uart.img

   Now edit the file /boot/config.txt:

   sudo vim /boot/config.txt

   In this file, change the "kernel=" line to the following (or add it if it does not exist):

   kernel=kernel_mod_uart.img

   Finally, install the kernel modules that you compiled earlier, by issuing (in your build directory):

   sudo make modules_install

   You have now replaced the default kernel with your "UART-as-a-module" version. Reboot the Raspberry Pi to load the new kernel and ensure everything is OK.


5. Final step: Disable UART driver module
   The new kernel still loads and sets up the UART device, but now you can disable it. Edit the file /etc/modprobe.d/raspi-blacklist.conf:

   sudo vim /etc/modprobe.d/raspi-blacklist.conf

   Add the following line to the file, and save it:

   blacklist amba_pl011

   You may now reboot the Pi for the changes to take affect, or simply remove the already-loaded driver directly by entering:

   sudo rmmod amba_pl011

Conclusion

Sadly, pins 8 and 10 of the Raspberry Pi are still set to ALT0 mode upon bootup after applying the changes documented above. I have not looked into the kernel source for other clues (for my needs it is far simpler to use two different pins). If you know of way to force them to be set to GPIO mode (and driven LOW), please let me know.

ffmpeg with libfaac support on QNAP TS-419P II NAS

Having recently upgraded to a QNAP TS-419P II NAS, I opted to replace the default ffmpeg installation with one that is compiled against libfaac for proper AAC encoding/decoding support.  On "normal" Linux distributions this is an easy task, and in all fairness it isn't particularly difficult on the QNAP either... but it did take me a couple of hours to sort the quirks out. And since I could only find a few random forum posts about the topic, I decided to create a simple step-by-step tutorial.

The TS-419P II has a 2 GHz Marvell Feroceon "Kirkwood" CPU, which is an armv5tel architecture, according to "uname -m" - we have to keep this in mind when compiling some of the packages, since we will be tuning builds for this architecture and will have to disable platform-specific assembly optimizations for the most part. Now, onto the steps involved:

1. Install "Optware IPKG" via the QNAP Qpkg centre, if you haven't already.
2. Log into the NAS via ssh.
3. Install the some packages required for building ffmpeg and its dependencies via ipkg, by typing:

   ipkg update
   ipkg install gcc make pkgconfig libstdc++ git coreutils
   

4. Ensure that /etc/ld.so.conf is set up so that /opt/lib is a location for shared libraries:

   if ! grep -q /opt/lib /etc/ld.so.conf; then echo /opt/lib >> /etc/ld.so.conf; else echo no; fi

   Alternatively, you can manually edit /etc/ld.so.conf and make sure it looks like this:

   /lib
   /usr/lib
   /usr/local/lib
   /opt/lib
   

5. Install some codec libraries for ffmpeg via ipkg:

   ipkg install lame faad2 libogg libtheora libvorbis libmatroska libmpeg2

6. Next, create a directory in which you can download and compile some required packages:

   mkdir /opt/build

7. Build & install faac:

   cd /opt/build/
   wget http://downloads.sourceforge.net/project/faac/faac-src/faac-1.28/faac-1.28.tar.gz
   tar -xzvf faac-1.28.tar.gz
   cd faac-1.28/
   CFLAGS="-march=armv5te -mtune=marvell-f -Wall -O2 -fstrength-reduce -finline-functions -ffast-math -fomit-frame-pointer" ./configure --prefix=/opt --enable-static=no
   make
   make install
   

8. Build & install x264 (the version shipped by ipkg is too old for ffmpeg):

   cd /opt/build/
   git clone git://git.videolan.org/x264.git
   cd x264/
   CFLAGS="-march=armv5te -Wall -O2 -fstrength-reduce -finline-functions -ffast-math -fomit-frame-pointer" ./configure --prefix=/opt --enable-shared --system-libx264 --disable-asm
   make
   make install
   

9. Build & install xvidcore:
   Note: the configure script for the latest xvidcore isn't compatible with the awk tool available on the QNAP NAS. The workaround is to download an earlier version of xvidcore as well and replace the current version's configure script with the older one; this is done in the example below:

   cd /opt/build/
   wget http://downloads.xvid.org/downloads/xvidcore-1.3.2.tar.gz
   tar -xzvf xvidcore-1.3.2.tar.gz
   cd xvidcore/build/generic/
   wget http://downloads.xvid.org/downloads/xvidcore-1.3.0.tar.gz
   tar -xzvf vidcore-1.3.0.tar.gz
   cp -v xvidcore/build/generic/configure .
   CFLAGS="-march=armv5te -mtune=marvell-f -Wall -O2 -fstrength-reduce -finline-functions -ffast-math -fomit-frame-pointer" ./configure --prefix=/opt
   make
   make install
   

10. Finally, build & install ffmpeg:
    

    cd /opt/build/
    wget http://ffmpeg.org/releases/ffmpeg-1.2.tar.gz
    tar -xzvf ffmpeg-1.2.tar.gz
    cd ffmpeg-1.2/
    CFLAGS="-march=armv5te -Wall -O2 -fstrength-reduce -finline-functions -ffast-math -fomit-frame-pointer" ./configure --prefix=/opt --enable-gpl --enable-nonfree --enable-bzlib --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --disable-asm --disable-debug
    make
    make install
    

11. Run ldconfig to make sure all shared library additions are picked up:

    ldconfig

...and that's it. You now have a libfaac-enabled version of ffmpeg at /opt/bin/ffmpeg.

If you would like to override the system's ffmpeg with your new custom version, simply edit your PATH and make sure /opt/bin comes before /usr/bin.

GlassFish 3.1: Deactivation of MDB does not call endpointDeactivation() of custom resource adapter

I recently developed a custom JCA resource adapter for a sockets-based protocol used in one of our projects where I work. The resource adapter is deployed standalone to GlassFish 3.1 as a .rar file, and is then used by two other projects deployed to the same application server: an EAR-based project consisting of several EJB- and WAR-modules, and another standalone web (WAR-only) project containing some embedded EJBs (as is allowed with Java EE 6). Both these "client" projects contain message-driven beans (MDBs) that use the different message listener interfaces defined by the resource adapter.

However, during testing it became apparent that the endpointDeaction() method of the resource adapter was not being called when a client MDB is deactivated or undeployed - obviously resulting in resource constraint problems. Since endpointActivation() was being called just fine, I started thinking that this might be a bug in GlassFish, and sure enough, someone had reported it already:

http://java.net/jira/browse/GLASSFISH-16056

Anyhow, the solution (or workaround, rather) was provided in the first comment of that bug report by leonline. I am repeating it here in the hope that someone searching for a solution to this in the future will find the answer a bit quicker.

Add a glassfish-ejb-jar.xml file (GlassFish deployment descriptor; formerly known as sun-ejb-jar.xml) to the project module containing your MDB. If your MDB is in a Java EE 6-style WAR module, add it to the module's WEB-INF folder.

The contents of the glassfish-ejb-jar.xml file should be:

<glassfish-ejb-jar>
    <enterprise-beans>
        <ejb>
           <ejb-name>YOUR MDB-NAME HERE</ejb-name>
        <mdb-resource-adapter>
                <resource-adapter-mid>YOUR ADAPTER NAME HERE (WITHOUT .RAR)</resource-adapter-mid>
            </mdb-resource-adapter>
        </ejb>
    </enterprise-beans>
</glassfish-ejb-jar>

Here is the actual contents of one of our glassfish-ejb-jar.xml files:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE glassfish-ejb-jar PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1 EJB 3.1//EN" "http://glassfish.org/dtds/glassfish-ejb-jar_3_1-1.dtd">
<glassfish-ejb-jar>
    <enterprise-beans>
        <ejb>
           <ejb-name>SsmiServerRequestHandler</ejb-name>
        <mdb-resource-adapter>
                <resource-adapter-mid>SSMIConnector</resource-adapter-mid>
            </mdb-resource-adapter>
        </ejb>
    </enterprise-beans>
</glassfish-ejb-jar>

Note that in the above example, the relevant MDB's @MessageDriven annotation has its "name" parameter set to "SsmiServerRequestHandler". The name of the standalone resource adapter/connector resource is SSMIConnector.

With the glassfish-ejb-jar.xml deployment descriptor in place, endpointDeactivation() should now be called as expected.

IE problems with jQuery.Form AJAX POSTs

jQuery.Form is an excellent plugin for jQuery that makes it really easy to post HTML forms via AJAX and parse the resulting response in a JavaScript handler (similar to jQuery's standard AJAX methods).

However, when submitting a "multipart/form-data"-form with a JSON response (content type set to "application/json"), Internet Explorer tries to download the response instead of calling your JavaScript reponse handler...  very irritating indeed. Of course, Firefox and Chrome handle these responses without any issues (I haven't tried IE10 yet).

A workaround is to set the response type to "text/html" on the server, and parse the resulting text string via jQuery.parse(). Not ideal, but at least works on all browsers that I tested.

On a side note: if you set the response content type to "text/plain", IE (and some versions of Firefox) will add <pre> tags to the response string - thus using "text/html" instead is quicker.

Adding a prefix to Bootstrap CSS classes

Update 2013-03-15: Thanks to everyone for the bug reports and support. I apologise for the long silence - I had been busy with a lot of other things and didn't get round to updating the script. Should be fine from now on again, though. :-)

I recently started integrating Twitter's excellent Bootstrap framework into a hobby project of mine, but hit a snag: Bootstrap uses a lot of fairly common CSS classnames (e.g. "row", "container", etc)... and many of these clashed with classes that I had already defined in my own code, resulting in a fine mess.

The obvious solution was to modify the Bootstrap CSS so that everything in there uses a common prefix... however, the creator of Bootstrap made it clear that no prefixing of Bootstrap CSS classes would be implemented. Fair enough, it's their library, their call - and I suppose writing "row" instead of something like "tb-row" makes more sense if you just start out. But it does make implementing Bootstrap into an existing project harder than it needs to be, and I really did not want to rename my own CSS classes.

An alternative: Namespacing

One way to avoid the CSS name conflicts would be to wrap the Bootstrap CSS classes into a new top-level namespace using the Bootstrap Less source code and then wrapping your Bootstrap-sensitve page elements into a suitable container (as suggested in this stackoverflow answer). Not really what I was aiming for - having an option to merely define a "$prefix" variable in Bootstrap source and recompiling would've been far nicer. But, alas, things are not that simple: the CSS classes are referenced heavily in the Bootstrap Javascript source files, negating the usefulness of such a "prefix" Less variable.

Solution: Prefixing via post-processing script

To address this issue I whipped up a quick Python script that edits the Bootstrap CSS and Javascript files and adds a prefix to any CSS classnames it finds. The prefix defaults to "tb-" (for "Twitter Bootstrap"), but can be easily changed (see the docs below).

Using it is simple: just supply it with the path to your Bootstrap installation (it expects the default Bootstrap directory structure to be in place). The script will then output modified versions of the CSS and Javascript files it finds, with a ".prefixed" tag added to the filename.

Finally, include these "prefixed" files in your project instead of the original Bootstrap CSS and Javascript files - now you can (and must) refer to all Bootstrap CSS classes by adding the "tb-" prefix to the name, e.g. "row" becomes "tb-row", etc.

The "bootstrap_namespace_prefixer.py" script is available from GitHub, or you can download it by clicking here.

Please note: this script works for me. YMMV, but let me know if something goes wrong and I'll take a look at it.

Documentation

The script expects a Bootstrap directory structure like the following:
/bootstrap - top-level bootstrap dir; this is the path that must be supplied to the script
/bootstrap/css - directory containing CSS files for Boostrap
/bootstrap/js - directory containing Bootstrap  Javascript files

Call the script as follows:

$ python bootstrap_namespace_prefixer.py /path/to/boostrap/dir

The script will output the following files:
/bootstrap/css/bootstrap.prefixed.css - CSS with the added prefix
/bootstrap/css/bootstrap.min.prefixed.css - Minified CSS with the added prefix
/bootstrap/js/bootstrap.prefixed.js - Javascript using the prefixed classes
/bootstrap/js/bootstrap.min.prefixed.js - Minified Javascript using the prefixed classes

To change the classname prefix to something other than "tb-", edit the bootstrap_namespace_prefixer.py file and set the CSS_CLASS_PREFIX constant to whatever you want to use as a prefix.