Friday, October 29, 2010

Kindle 3rd generation

 

I got problems when I tried to read some books in Chinese.

I will basically investigate following questions

  1. What encodings are supported by Kindle
  2. What fonts are supported
  3. If the fonts used in the PDF cannot be found, how Kindle handles it.
  4. What formats are supported?

Wednesday, October 27, 2010

GNOME notes

GDM

/etc/init.d/gdm, /etc/init/gdm.conf: these two files are used to start gdm (actually gdm-binary).

Default display manager is stored here: /etc/X11/default-display-manager

GDM configuration: http://library.gnome.org/admin/gdm/2.32/gdm.html#configuration

Tools: gdmsetup

/etc/gdm

/user/share/gdm/

After the user logins in, gnome-session is invoked.

http://library.gnome.org/admin/system-admin-guide/stable/sessions-1.html.en

man gnome-session

Default session is specified at /desktop/gnome/session/default_session.

Use gconf-editor to view value of /desktop/gnome/session/default_session. In my case the value is "gnome-settings-daemon".
/usr/lib/gnome-settings-daemon/gnome-settings-daemon --gconf-prefix=/apps/gdm/simple-greeter/settings-manager-plugins. The description of option "--gconf-prefix" is "GConf prefix from which to load plugin settings"

 

Some resources:

http://live.gnome.org/SessionManagement/GnomeSession

http://standards.freedesktop.org/desktop-entry-spec/latest/

Autostart-spec:  http://standards.freedesktop.org/autostart-spec/autostart-spec-latest.html

http://lists.freedesktop.org/archives/xdg/2007-January/007436.html

Autostart apps are specified in directory /etc/xdg/autostart/, ~/.config/autostart/

 

Use /usr/bin/gnome-session-properties to set session properties (e.g. autostart programs). Those changes are specific to a user.

Tuesday, October 26, 2010

JAASRealm + SSL

http://tomcat.apache.org/tomcat-6.0-doc/realm-howto.html#JAASRealm

This is a good article on the topic: http://blog.frankel.ch/custom-loginmodule-in-tomcat. Some of following code is borrowed from the article.

Following steps are listed on the official tomcat doc. I will elaborate each of them.

  1. "Write your own LoginModule, User and Role classes based on JAAS (see the JAAS Authentication Tutorial and the JAAS Login Module Developer's Guide) to be managed by the JAAS Login Context (javax.security.auth.login.LoginContext) When developing your LoginModule, note that JAASRealm's built-in CallbackHandler only recognizes the NameCallback and PasswordCallback at present. "
    package test;
    
    import java.io.IOException;
    import java.util.Map;
    
    import javax.security.auth.Subject;
    import javax.security.auth.callback.Callback;
    import javax.security.auth.callback.CallbackHandler;
    import javax.security.auth.callback.NameCallback;
    import javax.security.auth.callback.PasswordCallback;
    import javax.security.auth.callback.UnsupportedCallbackException;
    import javax.security.auth.login.LoginException;
    import javax.security.auth.spi.LoginModule;
    
    /**
     * Login module that simply matches name and password to perform authentication.
     * If successful, set principal to name and credential to "AuthorizedUser".
     *
     * @author Nicolas Fränkel. Modified by Gerald Guo.
     * @since 2 avr. 2009
     */
    public class PlainLoginModule implements LoginModule {
    
        /** Callback handler to store between initialization and authentication. */
        private CallbackHandler handler;
    
        /** Subject to store. */
        private Subject subject;
    
        /** Login name. */
        private String login;
    
        /**
         * This implementation always return false.
         *
         * @see javax.security.auth.spi.LoginModule#abort()
         */
        @Override
        public boolean abort() throws LoginException {
    
            return false;
        }
    
        /**
         * This is where, should the entire authentication process succeeds,
         * principal would be set.
         *
         * @see javax.security.auth.spi.LoginModule#commit()
         */
        @Override
        public boolean commit() throws LoginException {
    
            try {
    
                PlainUserPrincipal user = new PlainUserPrincipal(login);
                PlainRolePrincipal role = new PlainRolePrincipal("AuthorizedUser");
    
                subject.getPrincipals().add(user);
                subject.getPrincipals().add(role);
    
                return true;
    
            } catch (Exception e) {
    
                throw new LoginException(e.getMessage());
            }
        }
    
        /**
         * This implementation ignores both state and options.
         *
         * @see javax.security.auth.spi.LoginModule#initialize(javax.security.auth.Subject,
         *      javax.security.auth.callback.CallbackHandler, java.util.Map,
         *      java.util.Map)
         */
        @Override
        public void initialize(Subject aSubject, CallbackHandler aCallbackHandler, Map aSharedState, Map aOptions) {
    
            handler = aCallbackHandler;
            subject = aSubject;
        }
    
        /**
         * This method checks whether the name and the password are the same.
         *
         * @see javax.security.auth.spi.LoginModule#login()
         */
        @Override
        public boolean login() throws LoginException {
    
            Callback[] callbacks = new Callback[2];
            callbacks[0] = new NameCallback("login");
            callbacks[1] = new PasswordCallback("password", true);
    
            try {
    
                handler.handle(callbacks);
    
                String name = ((NameCallback) callbacks[0]).getName();
                String password = String.valueOf(((PasswordCallback) callbacks[1]).getPassword());
    
                if (!name.equals(password)) {
    
                    throw new LoginException("Authentication failed");
                }
    
                login = name;
    
                return true;
    
            } catch (IOException e) {
    
                throw new LoginException(e.getMessage());
    
            } catch (UnsupportedCallbackException e) {
    
                throw new LoginException(e.getMessage());
            }
        }
    
        /**
         * Clears subject from principal and credentials.
         *
         * @see javax.security.auth.spi.LoginModule#logout()
         */
        @Override
        public boolean logout() throws LoginException {
    
            try {
    
                PlainUserPrincipal user = new PlainUserPrincipal(login);
                PlainRolePrincipal role = new PlainRolePrincipal("admin");
    
                subject.getPrincipals().remove(user);
                subject.getPrincipals().remove(role);
    
                return true;
    
            } catch (Exception e) {
    
                throw new LoginException(e.getMessage());
            }
        }
    }
  2. "Although not specified in JAAS, you should create seperate classes to distinguish between users and roles, extending javax.security.Principal, so that Tomcat can tell which Principals returned from your login module are users and which are roles (see org.apache.catalina.realm.JAASRealm). Regardless, the first Principal returned is always treated as the user Principal. "
    Also read the API doc http://tomcat.apache.org/tomcat-5.5-doc/catalina/docs/api/org/apache/catalina/realm/JAASRealm.html. If authentication succeeds, your LoginModule must attach at least a user principal and a user role to subject.
    package test;
    
    import java.security.Principal;
    
    public class PlainRolePrincipal implements Principal {
    
        String roleName;
        
        public PlainRolePrincipal(String name) {
            roleName = name;
        }
        public String getName() {
            return roleName;
        }
        
        public String toString() {
            return ("RolePrincipal: " + roleName);
        }   
    
        public boolean equals(Object obj) {
            if (this == obj) {
                return true;
            }   
            if (obj instanceof PlainRolePrincipal) {
                PlainRolePrincipal other = (PlainRolePrincipal) obj;
                return roleName.equals(other.roleName);
            }   
            return false;
        }   
    
        public int hashCode() {
            return roleName.hashCode();
        }   
    }
    You get the idea, you can implement class PlainUserPrincipal in a similar way.
  3. "Place the compiled classes on Tomcat's classpath "
  4. "Set up a login.config file for Java (see JAAS LoginConfig file) and tell Tomcat where to find it by specifying its location to the JVM, for instance by setting the environment variable: JAVA_OPTS=$JAVA_OPTS -Djava.security.auth.login.config==$CATALINA_BASE/conf/jaas.config "
    Create a JAAS config file:
    -----------------------------
    CertBasedCustomLogin {
        test.CertBasedLoginModule
        sufficient;
    };
    -----------------------------
    When you launch tomcat, use  -Djava.security.auth.login.config= to specify where the config file is stored.
  5. "Configure your security-constraints in your web.xml for the resources you want to protect"
    The goal of the whole process is to protect some resources. This step specifies which resources should be protected.
    <security-constraint>
        <web-resource-collection>
            <web-resource-name>Secure Content</web-resource-name>
            <url-pattern>/cert-protected-users/*</url-pattern>
        </web-resource-collection>
        <auth-constraint>
            <role-name>AuthorizedUser</role-name>
        </auth-constraint>
        <user-data-constraint>
            <transport-guarantee>NONE</transport-guarantee>
        </user-data-constraint>
    </security-constraint>
    <!-- ... -->
    <login-config>
        <auth-method>CLIENT-CERT</auth-method>
        <realm-name>The Restricted Zone</realm-name>
    </login-config>
    <!-- ... -->
    <security-role>
        <description>The role required to access restricted content </description>
        <role-name>AuthorizedUser</role-name>
    </security-role>
    Basically, it says only users with role "AuthorizedUser" can access the resources cert-protected-users/*.
    Note:role-name must match the role attached to subject in step 1) ("AuthorizedUser" in our case) for successful access.
  6. "Configure the JAASRealm module in your server.xml"
    Actually, to put web app specific context config into server.xml is not recommended. Instead, I put a file named context.xml under directory META-INF.
    <Context>
        <Realm className="org.apache.catalina.realm.JAASRealm" appName="CertBasedCustomLogin"
            userClassNames="test.PlainUserPrincipal"
            roleClassNames="test.PlainRolePrincipal">
        </Realm>
    </Context>
    The value of appName must match the name specified in step 4.
  7. Add "-Dsun.security.ssl.allowUnsafeRenegotiation=true" for renegotiation support. (Read http://java.sun.com/javase/javaseforbusiness/docs/TLSReadme.html for more information)

Note

Some versions of Tomcat have problems to support JAASRealm + SSL mutual auth (https://issues.apache.org/bugzilla/show_bug.cgi?id=45576).  I tried 6.0.18, 6.0.20 and 6.0.29. Only 6.0.20 works for me. 6.0.29 gave errors when I tried.

More resources:

http://tomcat.apache.org/tomcat-6.0-doc/realm-howto.html#JAASRealm

http://tomcat.apache.org/tomcat-5.5-doc/catalina/docs/api/org/apache/catalina/realm/JAASRealm.html

http://wiki.metawerx.net/wiki/Web.xml.AuthConstraint

https://issues.apache.org/bugzilla/show_bug.cgi?id=45576

http://java.sun.com/javase/javaseforbusiness/docs/TLSReadme.html

Monday, October 25, 2010

web.xml schema (simplified)

  <xsd:complexType name="web-appType">

    <xsd:choice minOccurs="0" maxOccurs="unbounded">
      <xsd:group ref="j2ee:descriptionGroup"/>
      <xsd:element name="distributable" type="j2ee:emptyType"/>
      <xsd:element name="context-param" type="j2ee:param-valueType">
        <xsd:annotation>
          <xsd:documentation> 
            The context-param element contains the declaration
            of a web application's servlet context
            initialization parameters.  
          </xsd:documentation>
        </xsd:annotation>
      </xsd:element>

      <xsd:element name="filter" type="j2ee:filterType"/>
      <xsd:element name="filter-mapping" type="j2ee:filter-mappingType"/>
      <xsd:element name="listener" type="j2ee:listenerType"/>
      <xsd:element name="servlet" type="j2ee:servletType"/>
      <xsd:element name="servlet-mapping" type="j2ee:servlet-mappingType"/>
      <xsd:element name="session-config" type="j2ee:session-configType"/>
      <xsd:element name="mime-mapping" type="j2ee:mime-mappingType"/>
      <xsd:element name="welcome-file-list" type="j2ee:welcome-file-listType"/>
      <xsd:element name="error-page" type="j2ee:error-pageType"/>
      <xsd:element name="jsp-config" type="j2ee:jsp-configType"/>
      <xsd:element name="security-constraint" type="j2ee:security-constraintType"/>
      <xsd:element name="login-config" type="j2ee:login-configType"/>
      <xsd:element name="security-role" type="j2ee:security-roleType"/>
      <xsd:group ref="j2ee:jndiEnvironmentRefsGroup"/>
      <xsd:element name="message-destination" type="j2ee:message-destinationType"/>
      <xsd:element name="locale-encoding-mapping-list" type="j2ee:locale-encoding-mapping-listType"/>
    </xsd:choice>

    <xsd:attribute name="version" type="j2ee:web-app-versionType" use="required"/>
    <xsd:attribute name="id" type="xsd:ID"/>
  </xsd:complexType>

Friday, October 22, 2010

Apache SSL + SVN notes

Recently I started to use Apache http server again. I am trying to build a SVN repository which can be accessed through HTTPS.

Environment

Ubuntu
Apache 2.2.17 source

Doc: http://httpd.apache.org/docs/2.2/

Build

  ./configure --prefix=/home/gerald/servers/httpd-2.2.17 --enable-ssl --enable-dav --enable-so
  make
  make install

add bin directory to your PATH

add man pages:
function addManPath() { 
    if (($# != 1)); then return 0; fi

    path="$1" 
    if [ "x$MANPATH" == "x" ]; then 
        export MANPATH="$(manpath):$path" 
    else 
        export MANPATH="${MANPATH}:$path" 
    fi  
}

addManPath "~/servers/httpd-2.2.17/bin/man"

start up apache server: apachectl start

benchmarking: ab -n 10000 -c 100 http://localhost:80/

Show modules:
  httpd -M  //show all loaded modules
  httpd -S      // show parsed virtual host settings
  httpd -l    //listed compiled in modules
  httpd -L   //list available configuration directives
  httpd -V  //show compile settings (not settings for compiling the whole package, the settings for compiling the server - httpd).

Configure SSL

Prepare your certificate and private key.
Uncomment line "Include conf/extra/httpd-ssl.conf" in httpd.conf.
Change file "conf/extra/httpd-ssl.conf". The most important directives are SSLCertificateFile and SSLCertificateKeyFile.
Test whether you can access your website through HTTPS.

SSL + SVN

Get modules dav_svn and authz_svn

wget http://altruistic.lbl.gov/mirrors/ubuntu/pool/universe/s/subversion/libapache2-svn_1.6.5dfsg-1ubuntu1_i386.deb

dpkg-deb -x libapache2-svn_1.6.5dfsg-1ubuntu1_i386.deb

copy two module (.so files) to apache modules directory.

Configure modules

Edit file <Apache>/conf/extra/dav_svn.load

    LoadModule dav_svn_module modules/mod_dav_svn.so
    LoadModule authz_svn_module modules/mod_authz_svn.so

Edit file <Apache>/conf/extra/httpd.conf, add following two lines

    Include conf/extra/dav_svn.load
    Include conf/extra/dav_svn.conf

Edit file <Apache>/conf/extra/dav_svn.conf

<Location /svn/> <!-- trailing / is necessary!! -->
  DAV svn
  
SSLRequireSSL # enforce use of HTTPS #SVNPath /var/lib/svn SVNParentPath /home/svn/projects SVNListParentPath on AuthType Basic AuthName "Subversion Repository" AuthUserFile Apache_Dir/conf/dav_svn.passwd # To enable authorization via mod_authz_svn AuthzSVNAccessFile Apache_Dir/conf/dav_svn.authz Require valid-user </Location>

http://stackoverflow.com/questions/488778/how-do-i-list-all-repositories-with-the-svnparentpath-directive-on-apachesvn

Create authentication and authorization files

Create password file: htpassword -cm <Apache>/conf/dav_svn.passwd gerald

Edit file <Apache>/conf/dav_svn.authz

[groups]
admin=gerald
guests=guest

[/]
@admin=rw

[repository_name:/directory]
@admin=rw

Test

Restart Apache httpd server.
Go to https://your_ip/svn/ (note: the trailing / is necessary!)

 

Permission Problem

If you see following error when you try to commit some code:

svn: Commit failed (details follow):
svn: Can't open file '/path/to/your/repo/db/txn-current-lock': Permission denied

follow these steps:
  1. Execute command: ps -wwf $(pgrep httpd)
    You should say one of the processes is run as root. All other processes are run as daemon (in my case).
  2. To make httpd able to access(read/write) your svn repository, you should set the file permissions of svn repository correctly.
    chown -R gerald:daemon /path/to/svn/repo
    chmod -R 770 /path/to/svn/repo

Friday, October 15, 2010

Replace token using ant in Maven 2

<plugin>
    <artifactid>maven-antrun-plugin</artifactid>
    <executions>
      <execution>
        <id>Copy and filter af file</id>
        <goals><goal>run</goal></goals>
        <phase>prepare-package</phase>
        <configuration>
          <tasks> 
            <copy file="source_file" filtering="true" failonerror="true" overwrite="true" tofile="dest_file">
              <filterset>
                <filter value="${variable_name}" token="token_to_be_replaced" />
                <filter value="value" token="token_to_be_replaced" />
               </filterset>
            </copy>
          </tasks>
        </configuration>
      </execution>
    </executions>
</plugin>

Wednesday, September 15, 2010

XAuth!

Meebo and some other supporters just released XAuth. The video on page http://xauth.org/info/ is really informative. XAuth provides front-end solution for registration of various web service sessions.

  • If you are authenticated to a service, the service puts a registry entry into XAuth local storage.
  • Other mashup apps/publisher websites can ask XAuth for a list of web services that the user has been authenticated to. Then the app can adjust UI according to the retrieved data.

In current reference implementation, it requires HTML5 features – postMessage and local storage.

Javascript code: http://github.com/xauth/xauth
Official web site: http://xauth.org/

Sunday, September 12, 2010

Numbers Everyone Should Know

LADIS2009 keynote slides: http://www.cs.cornell.edu/projects/ladis2009/talks/dean-keynote-ladis2009.pdf

Following numbers are excerpted from the above slides.

L1 cache reference  0.5 ns 
Branch mispredict  5 ns 
L2 cache reference  7 ns 
Mutex lock/unlock  25 ns 
Main memory reference  100 ns 
Compress 1K bytes with Zippy  3,000 ns 
Send 2K bytes over 1 Gbps network  20,000 ns 
Read 1 MB sequentially from memory  250,000 ns 
Round trip within same datacenter  500,000 ns 
Disk seek  10,000,000 ns 
Read 1 MB sequentially from disk  20,000,000 ns 

Saturday, June 26, 2010

Clear memory cache in Linux

Read this article

http://linux-mm.org/Drop_Caches

Basically, run commands
    sync
    sudo echo 3 > /proc/sys/vm/drop_caches

If you want to control "swappiness" (to what extent apps are swapped), read following posts:
http://kerneltrap.org/node/3000
http://lwn.net/Articles/92892/

Basic command: sudo echo 0-100 > /proc/sys/vm/swappiness

Sunday, June 06, 2010

Steve Huffman talks about lessons learned at Reddit

Very interesting talk. Helpful to those who want to build scalable distributed systems.

Video: http://vimeo.com/10506751
Transcript is here: http://carsonified.com/blog/dev/steve-huffman-on-lessons-learned-at-reddit/

One interesting lesson is lesson 3: open schema.
They combined relational data model and key-value store. Relational data model is powerful (in some sense) and can naturally represent real-world data models. However, as data set size grows, it seems that relational databases cannot scale up easily. As a result, many large companies develop their own storage systems - Amazon's Dynamo, Google's BigTable, Cassandra, etc. Some use key-value model which seems to be able to scale better than relational model. To change schema does NOT incur much overhead. Different rows/entities can have different number of columns/attributes.

Another post on caching in web app: http://www.mysqlperformanceblog.com/2010/05/19/beyond-great-cache-hit-ratio/