JarScan is a handy utility (available as a client and online tool) which can help locate a java class or package within jars and zip files (libraries). This is especially useful when debugging java.lang.ClassNotFoundException and java.lang.NoClassDefFoundError exceptions.

For more details about JarScan (developed by Geoff Yaworski), click here. In case Geoff’s JarScan-hosting website (inetfeedback.com) isn’t available, you may download JarScan version 2.1 from here.

A screenshot of JarScan usage is given below:

Refer this example which shows how JarScan was used to find the location of the MulticastTest class.

A while ago, when I had a requirement to locate a Java class file, I simply used jar –tvf and grep. Although that worked, I would henceforth prefer to use JarScan, as it provides a good output, it is fast and it is platform independent.

In Oracle WebLogic (formerly BEA WebLogic) versions prior to version 10.0, WebLogic Servers relied on IP multicast to ensure cluster membership (versions 10.0 and later provide the alternative of Unicast which is preferred over Multicast). This article will pertain to IP multicast used by WebLogic.

What is IP multicast?

IP multicast is a technology used to broadcast data (or datagrams) across a network using IP. For IP multicasting, certain special IP addresses called multicast addresses are defined. According to the Internet Assigned Numbers Authority (IANA), its RFC3171 guidelines specify that addresses 224.0.0.0 to 239.255.255.255 are designated as multicast addresses. A multicast address is associated with a group of receivers. When a sender wishes to send a datagram to a group of receivers using IP multicast, it will send the datagram to the multicast address/port associated with that group of receivers. When routers or switches on the network receive the datagram, they know which servers (receivers) are associated with the multicast address (using IGMP) and so they make copies of the datagram and send a copy to every registered receiver. This is illustrated in the figure below:

Why does WebLogic use IP multicast?

A WebLogic Cluster is a group of WebLogic servers which provides similar services, with resilience (if a server crashes and exits the cluster, it can rejoin the cluster later), high availability (if a server in the cluster crashes, other servers in the cluster can continue the provision of services) and load balancing (the load on all servers in a cluster can be uniformly distributed) for an application deployed on a WebLogic cluster. WebLogic makes these beneficial clustering features possible by using IP Multicast for the following:

(1) Cluster heartbeats: All servers in a WebLogic cluster must always know which servers are part of the cluster. To make this possible, each server in the cluster uses IP multicast to broadcast regular "heartbeat" messages that advertise its availability.

(2) Cluster-wide JNDI updates: Each WebLogic Server in a cluster uses IP multicast to announce the availability of clustered objects that are deployed or removed locally.

How does WebLogic use IP multicast?

All servers in a WebLogic Cluster send out multicast fragments (heartbeat messages) from their interface addresses to the multicast IP address and port configured for the WebLogic Cluster. All servers in the cluster are registered with the multicast address and port and so every server in the cluster receives fragments from all other servers in the cluster as well as the fragment it sent out. So, since every server in the cluster sends out fragments every 10 seconds, based on the fragments it receives, it can determine which servers are still part of the cluster. If a server (say Server A) does not receive a fragment from another server in the cluster within 30 seconds (3 multicast heartbeat failures), then it will remove that server from its cluster membership list. When fragments from the removed server start arriving at Server A, then Server A will add the removed server back to its cluster membership list. This way, every server in a WebLogic cluster maintains its own cluster membership list. Regarding cluster-wide JNDI updates, each server instance in the cluster monitors these announcements and updates its local JNDI tree to reflect current deployments of clustered objects.

Note: Clustered server instances also monitor IP sockets as a more immediate method of determining when a server instance has failed.

The figure below illustrates how a WebLogic cluster uses IP multicast.

How do you configure and test multicast for a WebLogic Cluster?

Configuring IP Multicast for a WebLogic Cluster is simple. The steps required are given below:

STEP 1: If your WebLogic cluster is part of a network containing other clusters, obtain a multicast address and port for it, from your Network Admins. Understandably, a multicast address and port combination is unique for every WebLogic cluster. Several WebLogic clusters may share the same multicast address if and only if they use different multicast ports. Typically, in organizations, network admins allocate multicast addresses and ports for various projects to ensure there are no conflicts across the network. By default, WebLogic uses a multicast address of 237.0.0.1 and the listen port of the Administration server as the multicast port.

STEP 2: Having obtained a multicast address and port for your WebLogic cluster, you must test them before starting your WebLogic cluster to ensure that there are no network glitches and conflicts with other WebLogic clusters. You may do so with the MulticastTest utility provided with the WebLogic installation (part of weblogic.jar). An example test for a cluster containing 2 WebLogic servers on UNIX hosts and using multicast address/port of 237.0.0.1/30000 is given below:

# Command to run on both server hosts (any one of the following within the WebLogic domain directory)
# to set the WebLogic domain environment
. ./setDomainEnv.sh
. ./setEnv.sh
 
# Command to run on server 1 (within any directory)
${JAVA_HOME}/bin/java utils.MulticastTest -N server1 -A 237.0.0.1 -P 30000
 
# Command to run on server 2 (within any directory)
${JAVA_HOME}/bin/java utils.MulticastTest -N server2 -A 237.0.0.1 -P 30000
 
# NOTE: Both java commands must be run on both WebLogic server hosts concurrently.

View screenshots of the tests executed (on Windows Vista) when the WebLogic cluster was running (conflicts between test and running cluster outlined in red) and when the WebLogic cluster was stopped, by clicking on the images below:

Note: On Vinny Carpenter’s blog, he mentions a problem when using the utils.MulticastTest utility bundled with WebLogic Server 8.1 SP4. Well, I have never faced any issues with the utils.MulticastTest utility, but I am not sure if I’ve used it with WLS 8.1 SP4.

STEP 3: After successfully testing the multicast address and port, you may use the WebLogic Administration Console to configure multicast for the cluster. Descriptions of various multicast parameters are available on the console itself. The three most important parameters are (1) Multicast Address, (2) Multicast Port and (3) Interface Address. The Interface Address may be left blank if the WebLogic servers use their hosts’ default interface. On multi-NIC machines or in WebLogic clusters with Network channels, you may have to configure an Interface Address. Given below is a screenshot from a WLS 8.1 SP6 Administration Console indicating the various multicast parameters that may be configured for a cluster. Note that the interface address is on a different screen as it is associated with each server in the cluster, rather than the cluster itself.

After configuring Multicast for a WebLogic cluster, you can monitor the health of the cluster and exchange of multicast fragments among the servers in the cluster by using the WebLogic Administration console. A screenshot of such a monitoring screen with WLS 8.1 SP6 is given below:

Note that the screenshot above indicates that:

(1) All servers are participating in the cluster ("Servers" column).

(2) Every server in the cluster is aware of every other server in the cluster. The "Known Servers" column is especially useful for large clusters to know exactly which servers are not participating in the cluster.

(3) The total number of fragments received by each server (34) is equal to the sum of all the fragments sent by all the servers in the cluster (17 + 17). Note that the "Fragments Sent" and "Fragments Received" columns on the console need not always indicate a correct relationship even if multicast works fine. That’s because these stats on the console are reset to 0 when servers are restarted.

Troubleshooting WebLogic’s Multicast configuration

When you encounter a problem with WebLogic multicast (or any problem for that matter), it is important to confirm the problem by executing as many tests as possible and gather as much data as possible when the problem occurs. For WebLogic multicast, you may confirm the problem by using the MulticastTest utility or checking the Administration console as described above. To troubleshoot WebLogic multicast, refer to the Oracle documentation. Also, check the section below to determine if the problem you’ve encountered is similar to one of the problems described, to provide you with a quick resolution.

WebLogic Multicast eureka!

Given below are WebLogic multicast problems which I’ve encountered and investigated, along with solutions that worked:

PROBLEM 1:

SYMPTOMS: All WebLogic servers could not see any other server in the cluster. Tests using the MulticastTest utility failed indicating that all servers could only receive the multicast fragments which they sent out.

ANALYSIS: The MulticastTest utility was tried with the correct multicast address, multicast port and interface address. No conflict with any other cluster was observed, but no messages were received from other servers. Assuming that all servers in the cluster are not hung, the symptoms indicate a problem with the underlying network or the multicast configuration on the network.

SOLUTIONS:

Solution 1: The Network Admin just gave us another multicast address/port pair and multicast tests worked. The multicast address/port pair which failed was not registered correctly on the network.

Solution 2: The Network Admin informed us that more than one switch was used on the cluster network and this configuration did not ensure that multicast fragments sent by one server in the cluster were copied and transmitted to other servers in the cluster. Refer to this CISCO document for details regarding this problem and its solutions. As a tactical solution, the Network Admin configured static multicast MAC entries on the switches (Solution 4 in the CISCO document). This tactical solution requires the Network Admin to maintain those static entries, but since there weren’t too many WebLogic clusters using multicast on the network, this solution was chosen.

Solution 3: The two managed servers in a cluster were in geographically separated data centres and several hops across the network were required for the servers to receive each other’s multicast fragments. Increasing the multicast TTL solved this problem and both the MulticastTest utility and the WebLogic servers successfully multicasted.

PROBLEM 2:

SYMPTOMS: The following errors were seen in the WebLogic managed server logs and the managed servers did not start.

Exception:weblogic.server.ServerLifecycleException: Failed to listen on multicast address
weblogic.server.ServerLifecycleException: Failed to listen on multicast address
        at weblogic.cluster.ClusterCommunicationService.initialize()V(ClusterCommunicationService.java:48)
        at weblogic.t3.srvr.T3Srvr.initializeHere()V(T3Srvr.java:923)
        at weblogic.t3.srvr.T3Srvr.initialize()V(T3Srvr.java:669)
        at weblogic.t3.srvr.T3Srvr.run([Ljava/lang/String;)I(T3Srvr.java:343)
        at weblogic.Server.main([Ljava/lang/String;)V(Server.java:32)
Caused by: java.net.BindException: Cannot assign requested address
        at jrockit.net.SocketNativeIO.setMulticastAddress(ILjava/net/InetAddress;)V(Unknown Source)
        at jrockit.net.SocketNativeIO.setMulticastAddress(Ljava/io/FileDescriptor;Ljava/net/InetAddress;)V(Unknown Source)
        .
        .
        .

ANALYSIS: The errors occured irrespective of whichever multicast address/pair was used. The error indicates that the WebLogic server could not bind to an address to send datagrams to the multicast address. i.e. it could not bind to its Interface Address

SOLUTION: The WebLogic server host was a multi-NIC machine and another interface had to be specified for communication with the multicast address/port. Specifying the correct interface address solved the problem.

PROBLEM 3:

SYMPTOMS: The following errors were seen in the WebLogic managed server logs. The managed servers were running, but clustering features (like JNDI replication) were not working.

####<May 20, 2008 4:00:58 AM BST> <Error> <Cluster> <kips1host> <kips1_managed1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1211252458100> <BEA-000170> <Server kips1_managed1 did not receive the multicast packets that were sent by itself>
####<May 20, 2008 4:00:58 AM BST> <Critical> <Health> <kips1host> <kips1_managed1> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1211252458100> <BEA-310006> <Critical Subsystem Cluster has failed. Setting server state to FAILED.
Reason: Unable to receive self generated multicast messages>

ANALYSIS: The errors above indicate that the WebLogic server kips_managed1 could not receive its own multicast fragments from the multicast address/port. Probably, the server’s multicast fragments did not reach the multicast address/port in the first place and this points to an issue with the configuration of the interface address or route from interface address to multicast address/port or multicast address/port (most likely the interface address or route as if the multicast address/port were wrong, the server would not have received multicast fragments from any other server, as in PROBLEM 1).

SOLUTION: The WebLogic server kips_managed1 used -Dhttps.nonProxyHosts and -Dhttp.nonProxyHosts JVM parameters in its server startup command and these parameter values did not contain the name of the host which hosted kips_managed1. After including the relevant hostname in these parameter values, the errors stopped occurring. I am not sure how these HTTP proxy parameters affected self-generated multicast messages (will try to investigate this).

PROBLEM 4:

SYMPTOMS: All WebLogic servers were often (not always) part of a cluster and intermittently, servers in the cluster were removed and added and the LostMulticastMessageCount was increasing for some servers in the cluster. However, tests using the MulticastTest utility (when the cluster was stopped) were successful.

ANALYSIS: The problem occurred intermittently when the WebLogic servers were running, but never occurred when using the MulticastTest utility. This indicates that the underlying IP multicast works fine and something is preventing the servers in the cluster from IP multicasting properly. Further analysis revealed that the servers had issues with JVM Garbage Collection with long stop-the-world pauses (> 30 secs) during which the JVM did absolutely nothing else other than garbage collection. Also, the times of occurrences of these long GC pauses correlated with the times of increases in LostMulticastMessageCount and removal of servers from the cluster.

SOLUTION: The JVMs hosting the WebLogic servers were tuned to minimize stop-the-world GC pauses to allow the servers to multicast properly. For the specific GC problem I encountered, you may refer to the tuning details here.

Reference: Oracle documentation

NOTE:

Your rating of this post will be much appreciated. Also, feel free to leave comments (especially if you have constructive negative feedback).

When using the Concurrent Low Pause or Concurrent Mark Sweep Garbage collector with a Sun Hotspot JVM, you may observe the following "concurrent mode failures" errors in your GC logs or stderr:

(concurrent mode failure): 1404669K->602298K(1482752K), 35.9660286 secs] 1988728K->602298K(2096576K), 46.2083962 secs]

A concurrent mode failure indicates that a concurrent garbage collection of the tenured generation did not complete before the tenured generation filled up.

Impact of a concurrent mode failure: When a 1.4 JVM encounters a concurrent mode failure, it will trigger the default stop-the-world garbage collector in the tenured generation, resulting in a relatively long pause. In the example error above, note that the concurrent mode failure results in a GC (default stop-the-world) of around 46 seconds.

When does it occur?: A concurrent mode failure typically occurs with the following scenarios:

(1) Heavy load: Heavy load on the application running in the JVM, causing a huge promotion from the young to the tenured generation is typically what causes concurrent mode failures.

(2) Young generation guarantee failure: With JVM 1.4, a requirement for successful promotions from young to tenured generation is to have a contiguous block of free space equal to the size of the young generation, in the tenured generation to cater to the worst case requirement of having to promote all objects. Even if there is required space in the tenured generation, but it is not contiguous (i.e. it is fragmented), promotion failures and concurrent mode failures could occur. Fragmentation problems typically manifest themselves after a long period of continuous use. So, your application and JVM may be running fine for a while and suddenly (when the tenured generation is too fragmented) exhibit these problems.

How to fix it?: To fix these concurrent mode failures with JVM 1.4, typical solutions are:

(1) Increase the size of the JVM heap and consequently the size of the old generation.

(2) Tune the JVM depending on your application profile.

(3) Scale your platform to distribute load and subject each JVM to a smaller load.

JVM Tuning: My colleagues and I observed that our Hotspot 1.4.2_11 JVMs running WebLogic 8.1 SP6 threw several concurrent mode failures during peak load, at specific times of the day. The JVM heap for each WebLogic managed server was 2 GB which was already quite big (cannot increase much more anyway, as were using a 32-bit 1.4.2 JVM on Solaris 9). So, we decided to tune the JVM. Here are details of our JVM tuning to significantly reduce the number of concurrent mode failures:

(1) JVM Options before tuning: Given below, are the JVM parameters used when the problem occurred (WLjvm output):

**************************************************************************************************
JVM DETAILS FOR WEBLOGIC SERVER m1 (PID = 9999)
**************************************************************************************************
 
VERSION & BUILD
===============
 
java version "1.4.2_11"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_11-b06)
Java HotSpot(TM) Client VM (build 1.4.2_11-b06, mixed mode)
 
OPTIONS
=======
 
-Dbt.weblogic.RootDirectory=/software/weblogic
-Dbt.weblogic.RootDirectory=/software/weblogic
-Dweblogic.Name=m1
-Dweblogic.ProductionModeEnabled=true
-Dweblogic.management.server=http://localhost:50000
-Dweblogic.system.BootIdentityFile=/software/weblogic/boot.properties
-XX:+CMSParallelRemarkEnabled
-XX:+JavaMonitorsInStackTrace
-XX:+PrintGCDetails
-XX:+PrintGCTimeStamps
-XX:+UseConcMarkSweepGC
-XX:+UseLWPSynchronization
-XX:+UseParNewGC
-XX:+UsePerfData
-XX:MaxNewSize=600m
-XX:MaxPermSize=128m
-XX:NewSize=600m
-Xloggc:/software/weblogic/logs/GC/m1_200909012300.log
-Xms2048m
-Xmx2048m
-Xoss2m
-Xss2m
-server
 
**************************************************************************************************

(2) GC Analysis: I used PrintGCStats to analyze GC logs. It is also important to determine the CMSInitiatingOccupancyFraction (the % occupancy of the tenured generation at which a CMS GC is triggered). As the JVM options above do not set this parameter, the JVM calculates its own CMSInitiatingOccupancyFraction  at runtime. The default for JVM 1.4.2 is 68%, but do NOT assume that the JVM always uses the default if you do not specify this parameter. Instead, analyze the GC logs to understand when the JVM triggers a CMS GC. The script CMSGCStats.ksh (formerly CMSGCTrigger.ksh) may be used for this purpose, along with the provision of other metrics. By using PrintGCStats and CMSGCTrigger.ksh, we determined that a lot of objects were promoted to the tenured generation and CMS GC was mostly triggered at 50% occupancy of the tenured generation with some occurrences between 60% to 75% also.

(3) Requirements Analysis: Our application mostly created short-lived objects and very low latency was not critical due to the mostly asynchronous services provided by the application. Hence, our goal was to retain most of the objects for as long as possible in the young generation, to facilitate their garbage collection in the Young generation, thereby promoting fewer objects to the tenured generation and reducing the probability of filling up the tenured generation during a CMS GC (causing concurrent mode failures).

(4) Implementation:

In order to garbage collect objects in the Young Generation and decrease the number of objects being promoted to the tenured generation, we tuned the following parameters:

In order to override the JVM’s choice of when to trigger a CMS GC in the tenured generation, we tuned the following parameters:

(5) Results: Based on 5 iterations of testing with different sets of the parameters mentioned above, we obtained best results with the following JVM parameters:

**************************************************************************************************
JVM DETAILS FOR WEBLOGIC SERVER m1 (PID = 8888)
**************************************************************************************************
 
VERSION & BUILD
===============
 
java version "1.4.2_11"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_11-b06)
Java HotSpot(TM) Client VM (build 1.4.2_11-b06, mixed mode)
 
OPTIONS
=======
 
-Dbt.weblogic.RootDirectory=/software/weblogic
-Dbt.weblogic.RootDirectory=/software/weblogic
-Dweblogic.Name=m1
-Dweblogic.ProductionModeEnabled=true
-Dweblogic.management.server=http://localhost:50000
-Dweblogic.system.BootIdentityFile=/software/weblogic/boot.properties
-XX:+CMSParallelRemarkEnabled
-XX:+JavaMonitorsInStackTrace
-XX:+PrintGCDetails
-XX:+PrintGCTimeStamps
-XX:+UseConcMarkSweepGC
-XX:+UseLWPSynchronization
-XX:+UseParNewGC
-XX:+UsePerfData
-XX:MaxPermSize=128m
-XX:MaxNewSize=768m
-XX:NewSize=768m
-XX:MaxTenuringThreshold=8
-XX:TargetSurvivorRatio=90
-XX:SurvivorRatio=4
-XX:CMSInitiatingOccupancyFraction=55
-XX:+UseCMSInitiatingOccupancyOnly
-Xloggc:/software/weblogic/logs/GC/m1_200909022300.log
-Xms2048m
-Xmx2048m
-Xoss2m
-Xss2m
-server
 
**************************************************************************************************

Some graphs of parameters before (baseline) and after JVM tuning for 4 WebLogic managed servers are given below:

The above graphs indicate the following:

(1) The number of occurrences of "concurrent mode failures" significantly decreased after JVM tuning, although these errors weren’t eliminated.

(2) The amount of objects (MB) promoted from the young to the tenured generation significantly decreased after JVM Tuning, thereby indication more garbage collection in the young generation.

(3) The GC Sequential load (% of total time spent in GC with all application threads suspended) increased after JVM Tuning. Although this is not desired, given that our application did not require very low latency, we were willing to make a compromise and accept the increase in GC Sequential load. As a matter of fact, when our JVM changes were rolled out onto Production, the GC Sequential load was only around 1%.  We were just being cautious and testing our changes on a test environment with very high loads.

NOTE: The tuning exercise described above worked well for us. However, depending on your scenario, you may have to do further tuning. Also, there’s only so much you can benefit by tuning a JVM and you must give due consideration to scaling the platform and/or application design. In the example above, although JVM tuning helped, the load on our platform was too high and we had to scale our platform by adding more WebLogic managed servers (JVMs). Problems such as "concurrent mode failures" are typically caused by high loads when the CMS GC isn’t able to keep up with the rate of allocation of objects in the tenured generation.

REFERENCES:

(1) When the Sum of the parts – Jon Masamitsu

(2) What the Heck’s a Concurrent Mode? – Jon Masamitsu

Problem:

When upgrading my blog to Wordpress 2.8.4, the upgrade failed with the following error:

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 2357046 bytes) in xxx.php on line yyy

Background:

I wanted to upgrade my blog to the latest Wordpress version (2.8.4). Also, I was using 10 plugins on my Wordpress blog, the latest addition being GD Star Rating 1.6.4.

Solution:

I upgraded GD Star Rating to version 1.6.5 and this upgrade fixed the problem, thereby permitting me to upgrade Wordpress to version 2.8.4. However, after both upgrades, my blog’s dashboard displayed the fatal error in 2 locations as shown in the screenshot below:

Such fatal errors occur when a PHP script hits the threshold for the maximum amount of memory it may consume. Some Wordpress forums indicate that Wordpress 2.8 is more memory-intensive than earlier wordpress versions. And my hosting provider defined a memory_limit of 32MB in php.ini.

In order to override my PHP system memory_limit of 32 MB and allow the Wordpress application to use more memory, I edited the wp-config.php file (in server docroot) and added the following:

define('WP_MEMORY_LIMIT', '128M');

After I modified and saved wp-config.php, the fatal errors disappeared from my Wordpress Dashboard.

Some more investigation revealed how Wordpress sets its memory limit in wp-settings.php via the following code:

if ( !defined('WP_MEMORY_LIMIT') )
	define('WP_MEMORY_LIMIT', '32M');
 
if ( function_exists('memory_get_usage') && ( (int) @ini_get('memory_limit') < abs(intval(WP_MEMORY_LIMIT)) ) )
	@ini_set('memory_limit', WP_MEMORY_LIMIT);

So, that’s why setting the WP_MEMORY_LIMIT variable in wp-config.php (you could set it in wp-settings.php also, but it’s recommended to consolidate all config in one file) increases the memory limit for the Wordpress application.

Root Cause:

The PHP memory limit of 32MB was too low for the Wordpress 2.8.4 application.

NOTE:

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated. Also, feel free to leave comments.

WebLogic health checker is a basic CLI monitoring software that provides a quick and easy option to check the health of a WebLogic domain. It’s just a bunch of korn shell scripts which I developed, to wrap around the popular wlshell.

Download WLhc v1.2      Download WLhc v1.2 User Guide         

Earlier Versions:

Download WLhc v1.1      Download WLhc v1.1 User Guide         

Salient Features:

• Monitors WebLogic Server versions 7.0, 8.1, 9.x and 10.x
• Interactive and non-interactive modes of execution
• Unintrusive, external monitoring: No application is required to be deployed on the WebLogic Server being monitored and no server restart required.
• Interactive execution provides a well-formatted dashboard of current runtime values of monitored parameters
• Non-interactive execution (e.g. cron) generates data files in csv format, thereby allowing graphing programs to generate graphs from the data. The data files are retained for 30 days by default and can be used for troubleshooting and trend analysis.
• Quick and easy Re-configuration: An auxiliary script controlled by a template, facilitates rebuilding the configuration file for a monitored server. This script will come in handy when changes are made to a WebLogic Server and/or when you wish to monitor new parameters.
• Auxiliary scripts obtain data on all runtime MBeans and their attributes for a specified WebLogic Server.
• Traps can be easily configured using simple rules in configuration files
• Traps may be enabled/disabled globally for one or more WebLogic Servers or for individual parameters
• Clear and structured logging for alarms and errors, thereby facilitating the use of log scanners

System Requirements:

• Korn Shell (/bin/ksh)
• Perl
• nawk/gawk or equivalent
• /usr/ucb/ps (Solaris) or ps (Linux)

Screenshots:

(1) Installation

(2) Interactive Execution

YouTube is arguably the most popular video streaming/sharing website in the world today. You can find pretty much any video you want, be it videos of the current music chart topper or those classic videos of yore. You may wish to download your favourite videos from video streaming websites and perhaps even burn the videos onto a DVD for playback on your DVD player (DVD Authoring). YouTube and other streaming video websites don’t provide options to download the streaming videos. However, there is a myriad of software tools out there (both free and paid) that make the download and burn tasks very straightforward. Given below are the steps I used to create a DVD-Video containing 30 of my favourite YouTube videos:

STEP 1: Install a Download Manager

In order to download streaming videos, you must install a download manager. I’ve tried the following download managers:

(1) Orbit Downloader: You may download and install Orbit Downloader from here. After installation of Orbit downloader, if you visit a streaming video website and hover your mouse over  the streaming video, you will see a drop-down as shown in the image below:

Clicking on the GetIt button or using drop-down options like “With Grab++” will start downloading the streaming video.

(2) RealPlayer SP: You may download and install RealPlayer SP from here. After installation of RealPlayer SP, if you visit a streaming video website and hover your mouse over a streaming video, you will see a "Download This Video" drop-down on the upper right corner of the video, as shown in the image below:

Click on the “Download This Video” button to start downloading the streaming video.

Which download manager do I use?: I used Orbit Downloader to download the 30 YouTube videos. However, having been introduced to RealPlayer SP recently, I recommend you use RealPlayer SP, because as per my observations, it has advantages over Orbit downloader in the following areas:

(1) Speed: RealPlayer SP downloads streaming videos faster than Orbit downloader, though not a significant difference.

(2) Compatibility: RealPlayer SP downloads streaming videos played in the popular browsers (Mozilla, IE, Chrome) whereas Orbit Downloader does not work with Chrome.

STEP 2: Install a DVD Authoring tool

In order to burn videos onto a DVD in DVD-Video format, you must install a DVD Authoring software. I used a free, easy-to-use DVD Authoring software called DVD Flick, which you may download from here and install.

STEP 3: Download streaming videos

Having installed the required software, you may now proceed with downloading your favourite streaming videos. To do so, play the streaming video and use the buttons (Orbit Downloader or RealPlayer SP) to download the streaming video. When saving the video, ensure you save it in an appropriate format. Visit here to check the video formats supported by DVD Flick for creating a DVD-Video. For example, on YouTube, videos are typically available in flv (flash video) and mov (QuickTime movie) formats, both of which are supported by DVD Flick. I downloaded standard quality (SQ) YouTube videos in flv format and high quality (HQ) YouTube videos in mov format (renaming .mov to .mp4, as Quicktime and MP4 are interchangeable).

STEP 4: Create a DVD-Video

After downloading your favourite videos, you can burn them onto a DVD for playback on both hardware and software DVD players, with features such as menus and subtitles. Here’s how you do it:

(1) Insert a blank recordable/rewritable DVD in your computer’s DVD drive and launch the DVD Flick software.

(2) Click on “Add Title” in the right-menu, select all the videos you wish to burn and then click the “Open” button. This will load all the selected videos into the project window. You may choose to save this project for later burning or just proceeding with the next step.

(3) Click on “Menu Settings”, choose a Menu and click “Accept”

(4) Click on “Project Settings”, set various options including “Burning” (which tells DVD Flick to burn the DVD after converting the video files into DVD-Video format).

(4) Click on “Create DVD” and select a project destination folder (if prompted).

After completing the steps above, DVD Flick should commence the process of encoding and burning your DVD-Video. A successful completion should resemble the image below:

It took 1 hour and 38 minutes to encode and burn my 30 favourite YouTube videos onto a 4.7 GB DVD. The resulting DVD-Video occupied 4.3 GB. The DVD played fine in my software DVD players such as Windows Media Player and VLC Media Player, but unfortunately exhibited sync issues (audio out of sync with video) when played on my hardware DVD player. However, until I test the DVD on another DVD  player, I will assume that the sync issues are caused by my cheap Alba DVD player.

DVD Flick is an easy-to-use free DVD Authoring software that can burn videos in several formats onto a DVD for playback on your hardware and software DVD players.

I used DVD Flick to create a DVD video of some of my favourite music videos (mp4 and flv formats). My first attempt failed because the audio and video were out of sync when I played the DVD video using Windows Media Player or VLC Media Player. I presumed this sync problem occurred because the video files I burned onto the DVD were being read from a USB 2.0 memory stick. So, when I moved these video files to my HDD and burned them onto a DVD using DVD Flick, the DVD video played just fine in the software media players without any sync issues. When I played the burned DVD video on my DVD player, the sync problems returned to haunt me. However, until I test my DVD video with another DVD player, I will assume this issue is caused by my cheap DVD player (an Alba player which I bought for £20 a half a dozen years ago).

Overall, DVD Flick is very simple to use and makes DVD authoring a very straightforward task. It took 1.5 hours to burn 4.3 GB of music videos onto a 4.7 GB DVD at a speed of 4x (~5.28 MB/s).

Orbit Downloader is a free download manager software aimed at managing streaming media downloads from social networking websites such as MySpace, YouTube, etc. In fact, you can use it as a download manager for downloading any resource from the web.

I use Orbit downloader to download YouTube videos and it works like a charm!

Problem:

When executing a korn shell script, execution fails with the following error:

/bin/ksh^M: bad interpreter: No such file or directory

Background:

I transferred the executable korn shell script from my backup on my Windows server to a RHEL 5.1 host. On Linux, opened the file using the vi editor and did not find any ^M characters. Also, checked file with "set list" in vi.

Solution:

I used dos2unix on the file as follows:

dos2unix myscript.ksh

Note: Usually, improper file transfer modes in ftp can cause this problem. However, I used ascii mode and checked the file using the vi editor, but didn’t find anything abnormal. So, dos2unix is your best bet!

Root Cause:

Invalid file format after transferring it from a Windows machine to a UNIX host.

NOTE:

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated. Also, feel free to leave comments.

Problem:

When performing an svn commit using TortoiseSVN, the commit operation failed with the following error:

Server sent unexpected return value (400 Bad Request) in response to MKACTIVITY

Background:

I accessed my SVN repository (http://) using TortoiseSVN via a proxy server and tried to commit some changes.

Solution:

I disabled the proxy server in Tortoise SVN’s Settings. i.e. I did NOT use a proxy server to access my SVN repository.

Note: If you must use a proxy server, you may fix this problem by installing certificates and accessing your SVN repository via HTTPS.

Root Cause:

TortoiseSVN uses WebDAV to access an SVN repository. Proxy servers which are not compliant with WebDAV will strip off HTTP headers containing WebDAV methods such as MKACTIVITY. However, if you access the SVN repository via HTTPS and a proxy, the proxy server will not inspect the request and will forward it as is.

NOTE:

(1) The solution above describes a successful problem-solving experience and may not be applicable to other problems with similar symptoms.

(2) Your rating of this post will be much appreciated. Also, feel free to leave comments.

Recently, I used a Reverse Proxy server to work around a constraint. I wanted to host a monitoring dashboard (website) at port 80 on a corporate intranet host, but running Apache (or any software) to listen at port 80 (or any port from 1 to 1024) on UNIX requires root privilege. Since the use of root privilege has been significantly restricted by my System Administrators, it would have been very inefficient and cumbersome to administer the website because my team would have had to raise a formal request to the Sys Admins via Change Management procedures whenever it was required to restart/administer Apache. So, I used a standard user to run Apache (hosting the website) to listen the a non-privileged port 7777 and configured a reverse proxy Apache server instance to listen at port 80 and forward requests to the Apache server instances which hosted the website at port 7777. Now, we can do whatever administration work we require on the Apache web server instance hosting the website without being dependent on the Sys Admins. If the reverse proxy apache web instance goes down, we will need to seek assistance from the Sys Admins to start it, but there’s very low probability of the reverse proxy apache web instance going down. My use of a reverse proxy server, as described above, is illustrated in the figure below:

Given below, are the steps I followed to set up a reverse proxy server using Apache 2.2.12 on Solaris 9:

Note: As a prerequisite, Apache 2.2.12 must be compiled with the ––enable-so option to enable Apache to load modules dynamically at runtime.

STEP 1: Build and load the Apache proxy modules

Apache requires the mod_proxy and mod_proxy_http modules to serve as a reverse proxy server for HTTP requests. The source code (.c files) for these modules are available in the apache source code repository. Build and install the required proxy modules using the APache eXtenSion (apxs) tool as follows:

# Note that the apache ServerRoot is /apache-2.2.12 in the examples below. Run the following commands in the source repository directory containing the relevant “.c” files (httpd-2.2.12/modules/proxy/).
#
# Build and install mod_proxy module. 
/apache-2.2.12/bin/apxs -i -a -o mod_proxy.so -c mod_proxy.c proxy_util.c
 
# Build and install mod_proxy_http module. 
/apache-2.2.12/bin/apxs -i -a -o mod_proxy_http.so -c mod_proxy_http.c proxy_util.c

STEP 2: Configure the required reverse proxy directives

Having extended Apache with "proxy" functionality in STEP 1, you will now need to tell Apache how you wish to use this new functionality and to do this, you must use the Proxypass and ProxyPassReverse directives in httpd.conf. Given below are directives which I used in httpd.conf to reverse proxy all HTTP requests coming in at port 80 to a website hosted on apache at port 7777:

ProxyPass / http://www.mydomain.com:7777/
ProxyPassReverse / http://www.mydomain.com:7777/

STEP 3: Restart Apache for the changes to take effect

mrkips@kipsserver: /apache-2.2.12/bin/apachectl -k restart

Sun VirtualBox is a popular, easy-to-use, free Virtualization software. I use it on 64-bit Windows Vista Ultimate (host OS) to virtualize Solaris10 and a few other Linux distributions. A frequent requirement is to share files and directories between the host OS and guest OSes. Sun VirtualBox makes this sharing easy with the "Shared Folders" feature. Given below are steps I followed to permanently share a directory on my host OS (Windows Vista) with Ubuntu 9.04 (guest OS) on Sun VirtualBox 3.04:

STEP 1: Configure the Shared Folder on the Sun VirtualBox GUI.

Select the Virtual Machine (Ubuntu 9.04) and click "Shared Folders" in the "Details" tab on the right pane. You will see the following window pop-up:

Click the “Add shared folder” icon, select a name for your shared folder, the path to the actual folder on your host OS and the access level for the folder.

I configured the following:

For Ubuntu 9.04 Guest OS:

Name = Ubuntu9.04

Path = E:\VirtualMachines\Ubuntu9.04

Access = Full

For Solaris 10 Guest OS:

Name = Solaris10

Path = E:\VirtualMachines\Solaris10

Access = Full

STEP 2: Mount the Shared Folder

For Ubuntu 9.04 Guest OS:

Temporary Mountpoint(disappears after rebooting Guest OS)

Start the Ubuntu 9.04 Virtual Machine, open a terminal window and enter the following command:

sudo mount -t vboxsf Ubuntu9.04 /whatever-you-want

Permanent Mountpoint (remains after rebooting Guest OS)

Start the Ubuntu 9.04 Virtual Machine, open a terminal window and add the following line to the /etc/fstab file:

Ubuntu9.04      /whatever-you-want  vboxsf    rw 0       0

Restart the Ubuntu 9.04 Guest OS.

For Solaris 10 Guest OS:

Temporary Mountpoint(disappears after rebooting Guest OS)

Start the Solaris 10 Virtual Machine, open a terminal window and enter the following command as the root user:

pfexec mount -F vboxsf Solaris10 /whatever-you-want

Permanent Mountpoint (remains after rebooting Guest OS)

Start the Solaris 10 Virtual Machine, open a terminal window and add the following line to the /etc/vfstab file:

Solaris10       - /mrkips_vista   vboxfs - yes -

Restart the Solaris 10 Guest OS.

References:

UNIX manual (man mount, man fstab)

Subversion is one of the most popular open source version control tools currently available. Many organizations are moving from clunky, proprietary version control or configuration management software to Subversion. Recently, I compiled and built Subversion 1.6.4 on Solaris 9. While compiling and building software from source is usually straightforward with the 3-step “configure-make-make install” process, doing so on Solaris(unlike Linux) can be a bit frustrating at times, primarily due to the lack of packages/tools that are commonly available on Linux. So, here are the steps I followed to build subversion 1.6.4 from source on Solaris 9:

Note: By default, Solaris 9 does not have the gcc C compiler and some other packages generally available on Linux platforms. I had installed packages libiconv, gcc and libxml2 to the default build installed by my Sys Admins. You do not require root privileges to install subversion. I installed subversion using a non-root user. Refer this UPDATE for instructions specific to Red Hat Enterprise Linux 5.1

STEP 1: Download, unzip and extract the source code:

#
# Download subversion. Use your browser or the wget command below:
#
wget http://subversion.tigris.org/downloads/subversion-1.6.4.tar.gz
#
# Unzip and extract source (can be done in many ways)
#
gunzip subversion-1.6.4.tar.gz
tar xvf subversion-1.6.4.tar

The above commands create a directory called subversion-1.6.4.

STEP 2: Download, unzip and extract the dependencies (source):

#
# Download subversion dependencies. Use your browser or the wget command below:
#
wget http://subversion.tigris.org/downloads/subversion-deps-1.6.4.tar.gz
#
# Unzip and extract source (can be done in many ways)
#
gunzip subversion-deps-1.6.4.tar.gz
tar xvf subversion-deps-1.6.4.tar

The above commands will add the contents of the subversion-deps-1.6.4.tar archive into the subversion-1.6.4 directory created in STEP 1.

STEP 3: Remove the serf directory:

Subversion 1.6.4 uses two HTTP client libraries called neon and serf to enable users access the subversion repository using a browser (via HTTP/WebDAV). You don’t need both libraries and since the serf library made it mandatory to have SSL as a pre-requisite, I removed the serf library and used only neon. I’ve provided some feedback regarding this requirement to the serf team. This meant I could access the subversion repository via HTTP, but not via HTTPS (SSL) which met my requirements. However, if you do need HTTPS, then ensure you have an SSL installation (e.g. OpenSSL) and pass the appropriate options (–with-ssl) to subversion during the configure process. Refer Subversion INSTALL file.

#
# Remove serf directory within subversion-1.6.4
#
cd subversion-1.6.4
rm -rf serf

STEP 4: Set your PATH:

Set the PATH variable correctly, so that binaries like gcc (typically in /usr/local/bin) and make (typically in /usr/ccs/bin) can be accessed. For example, my PATH is /bin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/ccs/bin

STEP 5: Build Subversion:

Compile and build Subversion as follows:

#
# Change directory to subversion-1.6.4 and execute the following commands:
# Note: The --with-apxs option is required to build and load the mod_dav_svn to access subversion repositories via Apache using the WebDAV protocol. It creates the svn modules (mod_dav_svn.so and mod_authz_svn.so) in the Apache modules directory.
#
./configure --prefix=/wherever-you-want --with-apxs=/apache-ServerRoot/bin/apxs
make
make install

STEP 6: Test Subversion:

Test Subversion as follows:

#
# Change directory to where subversion-1.6.4 has been installed (cd /wherever-you-want)
#
bin/svn --version

If subversion is built properly, the above command will display subversion’s version and allowed methods of accessing the repository. For my installation, the output is given below:

svn, version 1.6.4 (r38063)
  compiled Sep  4 2009, 20:42:03
 
Copyright  2000-2009 CollabNet.
Subversion is open source software, see _http://subversion.tigris.org/
This product includes software developed by CollabNet (_http://www.Collab.Net/).
 
The following repository access (RA) modules are available:
 
* ra_neon : Module for accessing a repository via WebDAV protocol using Neon.
 - handles 'http' scheme
* ra_svn : Module for accessing a repository using the svn network protocol.
 - handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
 - handles 'file' scheme

—– T H E     E N D —–

Note: Installing subversion on Solaris 10 is as straightforward as doing so on Linux systems as Solaris 10 includes all the GNU software goodies in /usr/sfw/bin , where sfw => Sun Freeware

UPDATE (27th October 2009): I just installed Subversion 1.6.5 and 1.6.6 on Red Hat Enterprise Linux 5.1. The steps involved are the same as listed above for Solaris 9 with the exception of STEP 5 (Build Subversion) which was done as follows:

#
# Change directory to subversion-1.6.5/zlib (or subversion-1.6.6/zlib) and execute the following commands:
#
./configure --prefix=/mysvn-location --shared
make
make install
#
# Change directory to subversion-1.6.5 (or subversion-1.6.6)  and execute the following commands:
# Note: The --with-apxs option is required to build and load the mod_dav_svn to access subversion repositories via Apache using the WebDAV protocol. It creates the svn modules (mod_dav_svn.so and mod_authz_svn.so) in the Apache modules directory.
#
./configure --prefix=/mysvn-location --with-zlib=/mysvn-location --with-apxs=/apache-ServerRoot/bin/apxs
make
make install