Debug/profile heap/gc in Java

HPROF

Profiler agent.
Examples:
java -agentlib:hprof=help
java -agentlib:hprof=heap=sites
java -agentlib:hprof=heap=dump
java -agentlib:hprof=cpu=samples

" By default, heap profiling information (sites and dump) is written out to java.hprof.txt (in ASCII) in the current working directory.

The output is normally generated when the VM exits, although this can be disabled by setting the “dump on exit” option to “n” ( doe=n). In addition, a profile is generated when Ctrl-\ or Ctrl-Break (depending on platform) is pressed. On Solaris OS and Linux a profile is also generated when a QUIT signal is received ( kill -QUIT pid). If Ctrl-\ or Ctrl-Break is pressed multiple times, multiple profiles are generated to the one file.  "

jmap

The jmap command-line utility prints memory related statistics for a running VM or core file.

Commands:

jmap -histo <pid>                          #show histogram of objects
jmap -dump:format=b,file=<file>    #dump heap in HPROF format (can be processed by jhat)

jstat

"The jstat utility uses the built-in instrumentation in the HotSpot VM to provide information on performance and resource consumption of running applications. "

show garbage collection info, class loading info, compilation info, etc.

visualgc

GUI to show results of jstat.

Java VisualVM

http://download.oracle.com/javase/6/docs/technotes/guides/visualvm/index.html
command:  jvisualvm

"Java VisualVM is a tool that provides a visual interface for viewing detailed information about Java applications while they are running on a Java Virtual Machine (JVM), and for troubleshooting and profiling these applications."

JConsole

"This tool is compliant with Java Management Extensions (JMX). The tool uses the built-in JMX instrumentation in the Java Virtual Machine to provide information on the performance and resource consumption of running applications."

jhat (java heap analysis tool)

"The jhat tool provides a convenient means to browse the object topology in a heap snapshot. This tool was introduced in the Java SE 6 release to replace the Heap Analysis Tool (HAT). "

Command:

jhat <hprof_file_name>

Eclipse MAT

 

jdb

Misc.

"As of Java SE 5.0 update 7, the -XX:+HeapDumpOnOutOfMemoryError command-line option
tells the HotSpot VM to generate a heap dump when an OutOfMemoryError occurs (see
section 1.9).
As of Java SE 5.0 update 14, the -XX:+HeapDumpOnCtrlBreak command-line option tells the
HotSpot VM to generate a heap dump when a Ctrl-Break or SIGQUIT signal is received (see
section 1.10). "

Resources

http://www.oracle.com/technetwork/java/javase/index-137495.html

JAASRealm + SSL

http://tomcat.apache.org/tomcat-6.0-doc/realm-howto.html#JAASRealm

This is a good article on the topic: http://blog.frankel.ch/custom-loginmodule-in-tomcat. Some of following code is borrowed from the article.

Following steps are listed on the official tomcat doc. I will elaborate each of them.

1. "Write your own LoginModule, User and Role classes based on JAAS (see the JAAS Authentication Tutorial and the JAAS Login Module Developer's Guide) to be managed by the JAAS Login Context (javax.security.auth.login.LoginContext) When developing your LoginModule, note that JAASRealm's built-in CallbackHandler only recognizes the NameCallback and PasswordCallback at present. "

   package test;
   
   import java.io.IOException;
   import java.util.Map;
   
   import javax.security.auth.Subject;
   import javax.security.auth.callback.Callback;
   import javax.security.auth.callback.CallbackHandler;
   import javax.security.auth.callback.NameCallback;
   import javax.security.auth.callback.PasswordCallback;
   import javax.security.auth.callback.UnsupportedCallbackException;
   import javax.security.auth.login.LoginException;
   import javax.security.auth.spi.LoginModule;
   
   /**
    * Login module that simply matches name and password to perform authentication.
    * If successful, set principal to name and credential to "AuthorizedUser".
    *
    * @author Nicolas Fränkel. Modified by Gerald Guo.
    * @since 2 avr. 2009
    */
   public class PlainLoginModule implements LoginModule {
   
       /** Callback handler to store between initialization and authentication. */
       private CallbackHandler handler;
   
       /** Subject to store. */
       private Subject subject;
   
       /** Login name. */
       private String login;
   
       /**
        * This implementation always return false.
        *
        * @see javax.security.auth.spi.LoginModule#abort()
        */
       @Override
       public boolean abort() throws LoginException {
   
           return false;
       }
   
       /**
        * This is where, should the entire authentication process succeeds,
        * principal would be set.
        *
        * @see javax.security.auth.spi.LoginModule#commit()
        */
       @Override
       public boolean commit() throws LoginException {
   
           try {
   
               PlainUserPrincipal user = new PlainUserPrincipal(login);
               PlainRolePrincipal role = new PlainRolePrincipal("AuthorizedUser");
   
               subject.getPrincipals().add(user);
               subject.getPrincipals().add(role);
   
               return true;
   
           } catch (Exception e) {
   
               throw new LoginException(e.getMessage());
           }
       }
   
       /**
        * This implementation ignores both state and options.
        *
        * @see javax.security.auth.spi.LoginModule#initialize(javax.security.auth.Subject,
        *      javax.security.auth.callback.CallbackHandler, java.util.Map,
        *      java.util.Map)
        */
       @Override
       public void initialize(Subject aSubject, CallbackHandler aCallbackHandler, Map aSharedState, Map aOptions) {
   
           handler = aCallbackHandler;
           subject = aSubject;
       }
   
       /**
        * This method checks whether the name and the password are the same.
        *
        * @see javax.security.auth.spi.LoginModule#login()
        */
       @Override
       public boolean login() throws LoginException {
   
           Callback[] callbacks = new Callback[2];
           callbacks[0] = new NameCallback("login");
           callbacks[1] = new PasswordCallback("password", true);
   
           try {
   
               handler.handle(callbacks);
   
               String name = ((NameCallback) callbacks[0]).getName();
               String password = String.valueOf(((PasswordCallback) callbacks[1]).getPassword());
   
               if (!name.equals(password)) {
   
                   throw new LoginException("Authentication failed");
               }
   
               login = name;
   
               return true;
   
           } catch (IOException e) {
   
               throw new LoginException(e.getMessage());
   
           } catch (UnsupportedCallbackException e) {
   
               throw new LoginException(e.getMessage());
           }
       }
   
       /**
        * Clears subject from principal and credentials.
        *
        * @see javax.security.auth.spi.LoginModule#logout()
        */
       @Override
       public boolean logout() throws LoginException {
   
           try {
   
               PlainUserPrincipal user = new PlainUserPrincipal(login);
               PlainRolePrincipal role = new PlainRolePrincipal("admin");
   
               subject.getPrincipals().remove(user);
               subject.getPrincipals().remove(role);
   
               return true;
   
           } catch (Exception e) {
   
               throw new LoginException(e.getMessage());
           }
       }
   }

2. "Although not specified in JAAS, you should create seperate classes to distinguish between users and roles, extending javax.security.Principal, so that Tomcat can tell which Principals returned from your login module are users and which are roles (see org.apache.catalina.realm.JAASRealm). Regardless, the first Principal returned is always treated as the user Principal. "
   Also read the API doc http://tomcat.apache.org/tomcat-5.5-doc/catalina/docs/api/org/apache/catalina/realm/JAASRealm.html. If authentication succeeds, your LoginModule must attach at least a user principal and a user role to subject.

   package test;
   
   import java.security.Principal;
   
   public class PlainRolePrincipal implements Principal {
   
       String roleName;
       
       public PlainRolePrincipal(String name) {
           roleName = name;
       }
       public String getName() {
           return roleName;
       }
       
       public String toString() {
           return ("RolePrincipal: " + roleName);
       }   
   
       public boolean equals(Object obj) {
           if (this == obj) {
               return true;
           }   
           if (obj instanceof PlainRolePrincipal) {
               PlainRolePrincipal other = (PlainRolePrincipal) obj;
               return roleName.equals(other.roleName);
           }   
           return false;
       }   
   
       public int hashCode() {
           return roleName.hashCode();
       }   
   }

   You get the idea, you can implement class PlainUserPrincipal in a similar way.
   
3. "Place the compiled classes on Tomcat's classpath "
   
4. "Set up a login.config file for Java (see JAAS LoginConfig file) and tell Tomcat where to find it by specifying its location to the JVM, for instance by setting the environment variable: JAVA_OPTS=$JAVA_OPTS -Djava.security.auth.login.config==$CATALINA_BASE/conf/jaas.config "
   Create a JAAS config file:
   -----------------------------
   

   CertBasedCustomLogin {
       test.CertBasedLoginModule
       sufficient;
   };

   -----------------------------
   When you launch tomcat, use  -Djava.security.auth.login.config= to specify where the config file is stored.
   
5. "Configure your security-constraints in your web.xml for the resources you want to protect"
   The goal of the whole process is to protect some resources. This step specifies which resources should be protected.

   <security-constraint>
       <web-resource-collection>
           <web-resource-name>Secure Content</web-resource-name>
           <url-pattern>/cert-protected-users/*</url-pattern>
       </web-resource-collection>
       <auth-constraint>
           <role-name>AuthorizedUser</role-name>
       </auth-constraint>
       <user-data-constraint>
           <transport-guarantee>NONE</transport-guarantee>
       </user-data-constraint>
   </security-constraint>
   <!-- ... -->
   <login-config>
       <auth-method>CLIENT-CERT</auth-method>
       <realm-name>The Restricted Zone</realm-name>
   </login-config>
   <!-- ... -->
   <security-role>
       <description>The role required to access restricted content </description>
       <role-name>AuthorizedUser</role-name>
   </security-role>

   Basically, it says only users with role "AuthorizedUser" can access the resources cert-protected-users/*.
   Note:role-name must match the role attached to subject in step 1) ("AuthorizedUser" in our case) for successful access.
6. "Configure the JAASRealm module in your server.xml"
   Actually, to put web app specific context config into server.xml is not recommended. Instead, I put a file named context.xml under directory META-INF.

   <Context>
       <Realm className="org.apache.catalina.realm.JAASRealm" appName="CertBasedCustomLogin"
           userClassNames="test.PlainUserPrincipal"
           roleClassNames="test.PlainRolePrincipal">
       </Realm>
   </Context>

   The value of appName must match the name specified in step 4.
   
7. Add "-Dsun.security.ssl.allowUnsafeRenegotiation=true" for renegotiation support. (Read http://java.sun.com/javase/javaseforbusiness/docs/TLSReadme.html for more information)

Note

Some versions of Tomcat have problems to support JAASRealm + SSL mutual auth (https://issues.apache.org/bugzilla/show_bug.cgi?id=45576).  I tried 6.0.18, 6.0.20 and 6.0.29. Only 6.0.20 works for me. 6.0.29 gave errors when I tried.

More resources:

http://tomcat.apache.org/tomcat-6.0-doc/realm-howto.html#JAASRealm

http://tomcat.apache.org/tomcat-5.5-doc/catalina/docs/api/org/apache/catalina/realm/JAASRealm.html

http://wiki.metawerx.net/wiki/Web.xml.AuthConstraint

https://issues.apache.org/bugzilla/show_bug.cgi?id=45576

http://java.sun.com/javase/javaseforbusiness/docs/TLSReadme.html

JEE 6 has been approved!

See the ballot here: http://jcp.org/en/jsr/results?id=5025

ASF voted against the proposal. The comment ASF gave is:

"The Apache Software Foundation's vote is based on the point of view that this spec lead - Sun - is in violation of the JSPA

http://www.apache.org/jcp/sunopenletter.html

and therefore shouldn't be allowed to lead other JSRs until the above matter is resolved.

This vote is not a comment on the technical merits of the JSR.  If not for the issue of the spec lead, the ASF would have otherwise voted "yes".

I read http://www.apache.org/jcp/sunopenletter.html. It seems the dispute originates from license of JCK (Java Compatibility Kit) which is needed to demonstrate compatibility of Java SE spec. Also IBM voted for the proposal but stated that it wants an open licensing model, etc. Interesting relationships among those vendors and foundations.

The problem has been there for several years, and it still has not been solved.

Version matching of JSP, JSTL and Servlet

Working with JSP, JSTL, Servlet and Container is not an easy job if you download and configure every part by yourself. The most important problem is version matching. Also web.xml must be correctly configured to make use of the right version of Servlet/JSP.

Web app descriptor (web.xml)
To make use of correct version of JSP/JSTL/EL, you should read following posts which give detailed information you need to know to write web.xml:
http://faq.javaranch.com/java/ServletsWebXml

Some useful resources I found:
http://faq.javaranch.com/java/ElOrJstlNotWorkingAsExpected
http://forum.springframework.org/archive/index.php/t-19866.html
http://blog.csdn.net/eviliw/archive/2007/12/17/1944270.aspx
http://faq.javaranch.com/java/JstlTagLibDefinitions

On Sun's website, it is so hard to find a download link for JSTL 1.2. Usually you will be directed to the JSTL spec site.
You can download JSTL 1.2 from here: https://maven-repository.dev.java.net/repository/jstl/jars/.
Note for JSTL 1.2 there is just one jar file instead of two.
In JSTL 1.1/1.0, there are two jars : jstl.jar and standard.jar.
You can download JSTL 1.1 from here http://jakarta.apache.org/site/downloads/downloads_taglibs-standard.cgi and JSTL 1.0 from http://jakarta.apache.org/site/downloads/downloads_taglibs-standard.cgi.

Version match
JSTL 1.0 : Servlet 2.3 : JSP 1,2 (tomcat 4)
JSTL 1.1 : Servlet 2.4 : JSP 2.0 (tomcat 5)
JSTL 1.2 : Servlet 2.5 : JSP 2.1 (tomcat 6)
In old versions of JSP, EL is not enabled by default!! You can enable EL manually by using JSP page directory.
JSTL  jars are not included in tomcat distributions so far. You need to download and deploy JSTL by yourself.

JSTL 1.0 specifies a set of custom tag libraries based on the JSP 1.2 API. There are four separate tag libraries, each containing custom actions targeting a specific functional area. This table lists each library with its recommended tag prefix and default URI:

Description	Prefix	Default URI
Core	c	http://java.sun.com/jstl/core
XML Processing	x	http://java.sun.com/jstl/xml
I18N & Formatting	fmt	http://java.sun.com/jstl/fmt
Database Access	sql	http://java.sun.com/jstl/sql

JSTL 1.1 specifies a set of custom tag libraries based on the JSP 2.0 API. There are five separate tag libraries, each containing custom actions targeting a specific functional area. This table lists each library with its recommended tag prefix and default URI:

Description	Prefix	Default URI
Core	c	http://java.sun.com/jsp/jstl/core
XML Processing	x	http://java.sun.com/jsp/jstl/xml
I18N & Formatting	fmt	http://java.sun.com/jsp/jstl/fmt
Database Access	sql	http://java.sun.com/jsp/jstl/sql

One missed tag in the table is "functions".
It seems that you also can use tag URI like this: http://java.sum.com/jstl/core_rt. And it works in my application. However, I have not investigated what's going on behind the scene. So this may or may not work generally. Instead, you should always use the new tag URI.

Prefixes and URL of JSTL 1.2 Tag Libraries are the same as that of JSTL 1.1.

Troubleshooting
(1) If you get following error "According  to TLD or attribute directive in tag file, attribute test does not accept any expressions", it is possible that you don't specify tag prefix and URI correctly. You may be using JSTL 1.2 while you specify tag URI of JSTL 1.0 wrongly. Take Core as an example, tag URI in JSTL 1.1/1.2 is http://java.sun.com/jsp/jstl/core while tag URI in JSTL 1.0 is http://java.sun.com/jstl/core.
(2) If your EL/JSP is displayed directly without evaluation. it is highly possible that your web.xml file specifies wrong JSP version. See this post http://faq.javaranch.com/java/ServletsWebXml.

How to use Maven2

Download and install maven: http://maven.apache.org/download.html.

Running Maven
http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html
http://maven.apache.org/guides/getting-started/index.html
Generally the local repository is provided in USER_HOME/.m2/repository.

Configuration
http://maven.apache.org/guides/mini/guide-configuring-maven.html
Three levels:

• Project - most static configuration occurs in pom.xml.
  There should be a project configuration for every project. The configuration in pom.xml overrides the same configuration in other two levels. A user does not need to specify every aspect of the configuration. Those unspecified aspects are inherited from Maven's default Super POM.
  Introduction to POM: http://maven.apache.org/guides/introduction/introduction-to-the-pom.html.
  POM reference: http://maven.apache.org/pom.html
• Installation - this is configuration added once for a Maven installation
  This configuration file is generally located in M2_HOME/conf/settings.xml.
  NOTE: This location can be overridden with the system property:
  -Dorg.apache.maven.global-settings=/path/to/global/settings.xml
  Guide: http://maven.apache.org/guides/mini/guide-configuring-maven.html
  Reference: http://maven.apache.org/ref/2.0.8/maven-settings/settings.html
  Detailed reference: http://maven.apache.org/settings.html
• User - this is configuration specific to a particular user
  This configuration file is generally located in USER_HOME/.m2/settings.xml.
  NOTE: This location can be overridden with the system property:
  -Dorg.apache.maven.user-settings=/path/to/user/settings.xml

Build your own private/internal repository:
This article introduces how to create a repository using Artifactory: http://www.theserverside.com/tt/articles/article.tss?l=SettingUpMavenRepository. In addition, the author also compares some mainstream maven remote repository managers including Standard maven proxy, Dead simple Maven Proxy, Proximity and Artifactory.
In my case, I also use Artifactory and deploy it to tomcat. It has a nice web-based interface. Artifactory uses database(derby I think) to store various repository data so a user can not know the repository content by directly looking at the directory.

Deploy your artifacts to remote repository by using maven-deploy plugin:
http://maven.apache.org/plugins/maven-deploy-plugin/usage.html
(1) If the artifacts are built by using Maven, you should use deploy:deploy Mojo.
In your pom.xml, element <distributionManagement/> should be inserted to tell Maven how to deploy current package. If your repository is secured, you may also want to configure your settings.xml file to define corresponding <server/> entries which provides authentication information.
Command: mvn deploy.
(2) If the artifacts are NOT built by using Maven, you should use deploy:deploy-file Mojo.
Sample command:
mvn deploy:deploy-file -Dpackaging=jar -Durl=file:/grids/c2/www/htdocs/maven2 
-Dfile=./junit.jar -DgroupId=gridshib -DartifactId=junit -Dversion=GTLAB

FAQ:
(1) What does maven standard directory layout look like?
http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html
(1) How to specify parent artifact in pom.xml?
Read http://maven.apache.org/guides/introduction/introduction-to-the-pom.html.
(2) If a dependent package can not be download from central Maven repository, three methods can be used to deal with it:

"

1. Install the dependency locally using the install plugin. The method is the simplest recommended method. For example:

   mvn install:install-file -Dfile=non-maven-proj.jar -DgroupId=some.group -DartifactId=non-maven-proj -Dversion=1

   Notice that an address is still required, only this time you use the command line and the install plugin will create a POM for you with the given address.

2. Create your own repository and deploy it there. This is a favorite method for companies with an intranet and need to be able to keep everyone in synch. There is a Maven goal called deploy:deploy-file which is similar to the install:install-file goal (read the plugin's goal page for more information).
3. Set the dependency scope to system and define a systemPath. This is not recommended, however, but leads us to explaining the following elements:

"

(2) How to add new repository?
Put following code snippet into pom.xml or settings.xml.

<repository>
  <id>your-new-repository-id</id>
  <name>New Maven Repository </name>
  <layout>default</layout>
  <url>Address of the new repository</url>
  <snapshots>
    <enabled>enable-it?</enabled>
  </snapshots>
  <releases>
    <enabled>enable-it?</enabled>
  </releases>
</repository>

(3) How to disable default central maven repository?
Put following snippet into your pom.xml.

<repository>
  <id>central</id>
  <name>Maven Repository Switchboard</name>
  <layout>default</layout>
  <url>http://repo1.maven.org/maven2</url>
  <snapshots>
    <enabled>false</enabled>
  </snapshots>
  <releases>
    <enabled>false</enabled>
  </releases>
</repository>

(4) How can I package source code without run test?
Feed parameter -Dmaven.test.skip=true into the command line.
Note this property is defined by maven plugin surefire.
(5) Why does "mvn clean" delete my source code?
In your pom.xml, if content of element <directory> nested in element <build> is "./", "mvn clean" will delete all content in current directory including the src directory.
There are two more elements which can be used to specify locations of compiled classes.
outputDirectory:  The directory where compiled application classes are placed.
testOutputDirectory:  The directory where compiled test classes are placed.
(6) How to add resources into built package?
http://maven.apache.org/guides/getting-started/index.html#How_do_I_add_resources_to_my_JAR.
http://maven.apache.org/guides/getting-started/index.html#How_do_I_filter_resource_files
(7) Sometime, you want to use some libraries at compilation time and you don't want Maven to add them into your package(jar or war). How to do that?
Use dependency scope "provided" instead of default "compile". Read this post for details:
http://maven.apache.org/general.html#scope-provided. And this post elaborates Maven's dependency mechanism: http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
(8) How to build a war instead of jar?
Use Maven WAR Plugin: http://maven.apache.org/plugins/maven-war-plugin/usage.html.
First you should set entry packaging in pom.xml to war.
<packaging>war</packaging>
Then you can use one of the following commands to build your war:
mvn package
mvn compile war:war
mvn compile war:exploded
mvn compile war:inplace
Also you can filter resources of your web app. See this post http://maven.apache.org/plugins/maven-war-plugin/examples/adding-filtering-webresources.html.
(9) How to make war and jar at the same time?
By default your source code is compiled into class files and placed into directory WEB-INF/classes. Sometimes you may want to build a jar and then put it into directory WEB-INF/lib.
http://communitygrids.blogspot.com/2007/11/maven-making-war-and-jar-at-same-time.html
http://maven.apache.org/plugins/maven-war-plugin/war-mojo.html
http://maven.apache.org/plugins/maven-war-plugin/faq.html#attached
(10) help plugin can be used get information about a project, plugin or the system.
http://maven.apache.org/plugins/maven-help-plugin/
mvn help:describe -DgroupId=org.apache.maven.plugins -DartifactId=maven-compiler-plugin -Dfull=true
mvn help:describe -DgroupId=org.apache.maven.plugins -DartifactId=maven-compiler-plugin
mvn help:system
mvn help:all-profiles
mvn help:active-profiles


Properties reference:
http://docs.codehaus.org/display/MAVENUSER/MavenPropertiesGuide

During my using of Maven2, I encountered several bugs:
(1) targetPath support on the webResources war plugin parameter:
http://jira.codehaus.org/browse/MWAR-54
(2) maven-war-plugin webResources -- relative path:
http://www.mail-archive.com/users@maven.apache.org/msg77274.html
http://jira.codehaus.org/browse/MNG-2382
http://jira.codehaus.org/browse/MWAR-79
http://jira.codehaus.org/browse/MWAR-77

JAXP + Java Endorsed Packages

In this post I summarized XML related stuff. After all JAXP was constructed with the hope to unify interface.
Here is a very good FAQ about JAXP.

Pluggability
Pluggability is necessary so that programmers can choose the desired implementations of JAXP.
From the JAXP specification, I found the lookup procedure. For XSLT, the procedure is:

• Use the javax.xml.transform.TransformerFactory system property
• Use the properties file "lib/jaxp.properties" in the JRE directory. This configuration file is in standard java.util.Properties format and contains the fully qualified name of the implementation class with the key being the system property defined above. The jaxp.properties file is read only once by the JSR-000206 Java™API for XML Processing (“Specification”) implementation and its values are then cached for future use. If the file does not exist when the first attempt is made to read from it, no further attempts are made to check for its existence. It is not possible to change the value of any property in jaxp.properties after it has been read for the first time.
• Use the Services API (as detailed in the JAR specification), if available, to determine the classname.
The Services API will look for the classname in the file META-INF/services/javax.xml.transform.TransformerFactory
in jars available to the runtime.
• Platform default TransformerFactory instance.

It seems that the #3 option is used mostly. If you download an implementation jar (xalan.jar or xerces-api.jar...), you should be able to find the corresponding file META-INF/services/javax.xml.xxx.xxx.

Problem with Java SE 1.4
However, there is a problem for Java 1.4. It bundled in an implementation of JAXP 1.1 from Apache. Unfortunately, the developers did not change the package names. Even if you downloads and uses a newer version of package from Apache, none of the plug-in options described above works because the class loader always uses the built-in implementation.

Solution
Then two solutions are provided:
(1) Change the package names to other internal package names (e.g. com.sun.org.apache.*)
This is what Sun does in later Java SE releases.
(2) For old Java SE, Endorsed override mechanism can be used.
Java 5 Endorsed Standard: http://java.sun.com/j2se/1.5.0/docs/guide/standards/index.html.
Users can specify the endorsed package path by setting system property - java.endorsed.dirs.
The default path is: <jre-home>/lib/endorsed
On my machine, the path is /usr/java/jdk1.5.0_09/jre/lib/endorsed.
Information about system properties can be accessed here http://java.sun.com/docs/books/tutorial/essential/environment/sysprop.html.
Also I found a very simple program on the web to list all system properties:

public class DisplaySystemProps {
    public static void main(String[] args) {
         System.getProperties().list(System.out);
    }
}

Then you can check the properties you are interested in, such as user home directory, java home, vm version, java class version, path separator, file separator...

Tomcat
For old tomcat releases which rely on old versions of Java, users can put XML parser  into CATALINA_HOME/common/lib.
But for latest versions, it does not work. Read http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html for more information.

"Classes which are part of the JRE base classes cannot be overriden. For some classes (such as the XML parser components in J2SE 1.4+), the J2SE 1.4 endorsed feature can be used.
In previous versions of Tomcat, you could simply replace the XML parser in the $CATALINA_HOME/common/lib directory to change the parser used by all web applications. However, this technique will not be effective when you are running on JSE 5, because the usual class loader delegation process will always choose the implementation inside the JDK in preference to this one.
Tomcat utilizes this mechanism by including the system property setting -Djava.endorsed.dirs=$JAVA_ENDORSED_DIRS in the command line that starts the container."

In other words, users MUST use Endorsed override mechanism to use another XML parser/XSLT in Tomcat. See this post for more information.

Java XML related concepts and implementations

Revolving around XML processing, there are many concepts and corresponding Java libraries/packages. For a beginner, he/she may get confused with bunch of different concepts and libraries. I aim to record what I have learnt about xml processing.

(1) The first aspect is how to parse and process xml documents. Following is a list of current popular xml processing interface.

(1.1) DOM (Document Object Model)

This is a model standardized by W3C (http://www.w3.org/DOM/DOMTR). It is a suite which contains three levels. DOM is platform- and language-neutral interface by which users can manipulate xml document. For example, users can retrieve the root element of a xml document and get its child nodes and attributes.
Generally, implementation of DOM is to build a tree structure in memory. It means that only when the whole document is gained by the parser can the parser build dom tree. The obvious drawback is its performance considering a large xml document. What's more, sometimes the user just wants to get a small part of a large xml document. In this case, to build a in-memory tree is not a good solution.

(1.2)SAX (Simple API for XML)

    This interface does not come from formal organization. At first, it was a Java implementation of XML parser with some different concepts from DOM. After it was published, it gradually was accepted by industry. Finally, SAX becomes a set of interfaces which define a new way to parse XML document. It is language independent. Currently, SAX are implemented in many programming languages.
    " SAX is a streaming interface — applications receive information from XML documents in a continuous stream, with no backtracking or navigation allowed. This approach makes SAX extremely efficient, handing XML documents of nearly any size in linear time and near-constant memory, but it also places greater demands on the software developer's skills."
    SAX is the best known example of Event-based APIs. "An event-based API, on the other hand, reports parsing events (such as the start and end of elements) directly to the application through callbacks, and does not usually build an internal tree. The application implements handlers to deal with the different events, much like handling events in a graphical user interface."
    This mechanism requires programmers to understand concept similar to state-machine. Programmers should maintain a state machine conversion of different states is based on events SAX generates.

(1.3)XPP (XML Pull Parser)

    SAX is push-based interface. It means SAX reads XML document from a stream and invokes corresponding event handlers you have registered. However, this is still inefficient considering sometimes users just want to handle a small part of xml document. In this case, XPP is better. XPP manipulates corresponding elements/attributes based on users' requests. As a result, only parts of xml document are processed with other parts untouched.

(1.4)StAX (Streaming API for XMl)

    Excerpt from http://en.wikipedia.org/wiki/StAX:
    "These two access metaphors can be thought of as polar opposites. A tree based API allows unlimited, random, access and manipulation, while an event based API is a 'one shot' pass through the source document.

StAX was designed as a median between these two opposites. In the StAX metaphor, the programmatic entry point is a cursor that represents a point within the document. The application moves the cursor forward - 'pulling' the information from the parser as it needs. This is different from an event based API - such as SAX - which 'pushes' data to the application - requiring the application to maintain state between events as necessary to keep track of location within the document."   

    Here(http://www.xml.com/pub/a/2001/11/14/dom-sax.html?page=1) is a good discussion about DOM and SAX in high level other than detailed processing details.

(2) Validation of XML document.

(2.1)DOM

    Validation of XML document based on W3C Schema or DTD is straightforward by using DOM. DOM maintains a global structure of processed xml document so that all information of the document can be retrieved easily without much effort.

(2.2)SAX

    Validation in SAX requires more effort. The reason is that SAX does not maintain information about full document. Certain kinds of validation require to access information of document in full. For example, DTD IDREF attribute is used to refer to other elements defined in document. It requires there exists an element in the document that uses that string as an ID attribute. So, to work around this issue, one needs to maintain every encountered ID attribute and IDREF attribute. In addition, some kinds of XML document processing require to access whole document (i.e. XPath). In this case, to build a document tree is better, which actually is included in DOM.

(2.3)XPP

    The same kind of thought applies here as well. To validate a XML document, XPP needs more workaround than DOM and SAX.

(3) Java-specific XML processing interface

     To make Java programmers access various xml processing interfaces in easier way, various Java-specific interfaces have been defined. This can make XML processing code parser independent. In other words, user can switch between different parsers if these parsers comply with the same set of interface definition. This is a good thing because the code is not bound to a specific parser.

(2.1) JAXP

    This is constructed by JCP. It provides a set of interfaces by which Java programmers are able to process xml documents. It contains DOM, SAX, XSLT, XInclude, XPath and XML validation. Actually, by putting JAXP into JDK, it increases its chance of being accepted more and more. Users don't need to install additional packages to make use of JAXP. This makes it convenient to use and makes configuration of run time environment more easily. To process XML documents, JDK is all you need. Of course, if you are not satisfied with performance of built-in parser shipped with JDK, you can download and install other parsers. This increases flexibility for sure.

(2.1) dom4j

    I think this interface is an extension of JAXP. It means dom4j is compliant with JAXP but it provides more extra functionalities. It seems that dom4j is more convenient for Java programmers to work with.

(2.1) JDOM

    This also shares the same goal with other interfaces described above. If I don't remember something wrong, it came out before JAXP. By using many Java language specific features, it can ease processing of XML document in Java. However, I don't think currently it is still popular considering standard JAXP and JAXP extension dom4j.

(4) Parsers

    Now it is time to introduce some XML parsers which actually process XML documents. Two popular open source parses I know are Xerces and Crimson. You can google and find comparison between these two parsers. Basically, both of them support parsing by DOM and SAX.

(5) Relationship between these terms.

    To eliminate possible confusion, some clarification of relationship among these terms may be necessary.
    For APIs/Interfaces, they define implementation independent specification of XMl manipulation. They are just sets of interfaces and not implementations. Implementation of these interfaces are generally based on parsers which do the actual work. The implementation consists of a slim layer sitting on top of functionalities of parsers. It wraps functionalities of underlying parsers (may be Xerces, Crimson) to provide a unified interface which is defined by corresponding specification (JAXP or dom4j...). So generally after you download a library from website of dom4j or JDOM or JAXP, the library contains Xerces/dom4j lib. You can figure it out if you browse directory layout of the dom4j/JDOM/JAXP library.

(6) Further work

    Although JAXP/dom4j/JDOM eases processing of XML document, it is far from what Java programmers expect. To work on DOM level is still a clumsy job. You have to retrieve an element or attribute so that you can manipulate its content. Generally, programmers need to know details of the XML documents.
    One advanced idea of XML processing is to build correspondence between Java classes and XML Schema(may be DTD) and correspondence between Java objects and XML document. Then programmers just need to manipulate those Java objects instead of elements/attributes in XML documents.
    There are many libraries which implement binding between XMl and Java, e.g. JIBX, XMLBeans, ADB. JCP constructed a standard called JAXB (Java API for XML Binding?). Glassfish provides a reference implementation. JAXME is also an implementation of JAXB. However JAXME have not published a new version since 2006. So I don't know whether it is still developed actively.
    However, because correspondence between these two parts sometimes is not so natural, it may increase burden of programmers. Sometimes, the correspondence does not comply with what programmers expect. In this case, some human intervention is necessary. Then programmers must understand details of rules used during conversion. There may be many rules so that it is not a trivial task to grasp them all. Sometimes you don't need to understand them all. However, to figure out which rule you should customize is still not an easy task. As a result, some programmers prefer to manipulate XML using DOM/SAX instead of XML-Java binding.

(7) Related area

    To illustrate usage of XML related technologies, one inevitable area is Web Service.
    Some useful projects which ease of development of web service in Java are created. Here(http://wiki.apache.org/ws/StackComparison) is a list of frameworks and some stack comparison is presented as well. One additional lib which is not mentioned in that article is XINS(http://xins.sourceforge.net/). For web service client, WSIF is a client framework which can make web service client be composed easily. It is donated by IBM to Apache foundation. However, it seems not to be actively developed now. I am not sure.
    Every time variation of Java technologies exist, JCP is willing to standardize it. The web service area is not exceptional. Firstly, it proposed JAX-RPC specification. It standardized web service development based on WSDL/SOAP. One part inside the specification is binding between Java classes/objects and WSDL. In JAX-RPC specification, it contains detailed binding rules. After some time, JAX-RPC evolved into 2.0 and it is renamed to JAX-WS 2.0. The reason may be WS is a buzzword in industry and it may better capture the intent of the specification. In JAX-WS 2.0, WSDL-Java binding is delegated to JAXB. I remember JAX-WS implementation is included in JDK 6. JAX-RPC/JAX-WS specification describes interface by which programmers can easily build web service client and server programs. Besides, programmers can customized handling of transmissioned messages(may be SOAP) by plugging in handlers.

Java Bytecode

When javac is used to compile a Java program, bytecode(.class files) is generated which can be run on JVM.
Command javap can be used to display content of .class files in a meaningful way.
    javap ClassName        //print out field/method definition.
    javap -c ClassName    //print out all disassembled code.

A simple introduction:
http://www.ibm.com/developerworks/ibm/library/it-haggar_bytecode/

JVM specification:
http://java.sun.com/docs/books/jvms/

Java Reflection and bytecode ...

I found a series of very useful articles about Java. Here is the address:http://www.ibm.com/developerworks/java/library/j-dyn0429/.
In this series, the author introduces classloading, bytecode, reflection, Class-transformation on-the-fly...

Javassist --- manipulation of bytecode
Introduction from official site:
"Javassist (Java programming assistant) is a load-time reflective system for Java. It is a class library for editing bytecodes in Java; it enables Java programs to define a new class at runtime and to modify a class file before the JVM loads it. Unlike other similar systems, Javassist provides source-level abstraction; programmers can modify a class file without detailed knowledge of the Java bytecode. They do not have to even write an inserted bytecode sequence; Javassist instead can compile a fragment of source text on line (for example, just a single statement). This ease of use is a unique feature of Javassit against other tools."
http://labs.jboss.com/javassist/
http://www.csg.is.titech.ac.jp/~chiba/javassist/