Friday, January 30, 2009

Vim advanced topics

As I am using vim frequently, gradually I find it necessary to grasp some advanced vim topics. Sometimes, search engines don't return the useful posts/resources in the first page.

Vim built-in functions, internal variables, data structures, etc. http://vimdoc.sourceforge.net/htmldoc/eval.html
Filetype http://www.vim.org/htmldoc/filetype.html
Command line mode http://vimdoc.sourceforge.net/htmldoc/cmdline.html
Options and commands:
http://vimdoc.sourceforge.net/htmldoc/options.html
http://vimdoc.sourceforge.net/htmldoc/various.html
Configuration: http://peox.net/articles/vimconfig.html
Other topics (FAQs): http://www.cs.utah.edu/~kad/vim/vimnotes.shtml#17
Vim Buffer Resources
http://www.vim.org/htmldoc/windows.html
http://vim.wikia.com/wiki/Vim_buffer_FAQ

File manipulation commands:
Information getting
Ctrl+g, :f, :file print some info of current file in command line
:ls, :buffers, :files list
{count}Ctrl+g 1Ctrl+g: print current file name with full path
2Ctrl+g: currentbuffer number is printed also
g Ctrl+g

print cursor position in 5 ways:     Column, Line, Word, Character and Byte.

{Visual} g Ctrl+g

print Word, Character, Line, and Byte counts of selected text

Editing switch
:view {file} same as :edit, except the file is opened in readonly mode.
:file name set current file name to name. The old name becomes alternate file name.
:0file Remove the name of the current buffer.
:edit[!] Re-load current file. Useful when the file is modified outside of vim.
:edit[!] {file} Edit a new file.
Ctrl + ^ Edit alternate file(equivalent to ":e #").By default, the alternate file is the previously edited file
:{count} Ctrl + ^ go to a specific buffer
:edit #[count] Edit the [count]th buffer
:b[uffer] [count] Edit the [count]th buffer
:enew[!] edit a new unnamed buffer
Find and Edit
:find[!] {file} Find {file} in path and then edit
:{count}find[!] {file} Find {count}-th {file} in path and then edit
[count] gf Edit the [count]-th file in path whose name is under the cursor. If [count] is omitted, just get the first match. The file in opened in current window.
[count] Ctrl-w gf Edit the [count]-th file in path whose name is under the cursor. It is opened in a new tab.
[count] Ctrl-w f Edit the [count]-th file in path whose name is under the cursor. It is opened in a newly split window in current tab.
{Visual}[count]gf selected text is used as filename to search
[count]gF Similar to the commands above. If a number follows the filename, after opening the file, cursor would be moved to the specified position.
{Visual}[count]gF  
Writing
:w[rite][!] write current buffer to current file.
:{range}w[rite][!] write specified lines to current file
:{range}w[rite][!] {file} write specified lines (or whole buffer) to {file}
:{range}w[rite][!] >> append specified lines to current file
:{range}w[rite][!] >> {file} append specified lines to {file}
:[range]w[rite] !{cmd} feed specified lines into {cmd} as standard input
Insert a file
:r[ead] [file] insert file {file} after the cursor
:{range}r[ead] {name} insert file {file} below the specified line
:[range]r[ead] !{cmd} Execute {cmd} and insert its standard output after the cursor or below the specified line. E.g. :read !ls

Window manipulation

Command Explanation
Ctrl-w s, Ctrl-w Ctrl-s
:[N]sp[lit] [file]
Split current window into two horizontally.Option splitbelow controls where to put newly created window.
Ctrl-w v, Ctrl-w Ctrl-v
:[N]vsp[lit] [file]
Split current window into two vertically. Option splitright controls where to put newly created window.
Ctrl-w n, Ctrl-w Ctrl-n
:[N]new [file]
Create a new window with an empty file in it or with the specified file. split horizontally
:[N]vnew [file] Create a new window with an empty file in it or with the specified file. split vertically.
:[N]sv[iew] {file} same as :split, but set 'readonly' option
:[N]sf[find] {file} same as :split, search for {file} in path.
Ctrl-w Ctrl-^, Ctrl-w ^ Does ":split #". Split window into two and edit alternate file.
When a count is given, it is equivalent to ":split #N". Split and edit buffer N
Explictly set position of window
:vertical {cmd} Explicitly set position of a newly created window which is generated by invoking {cmd}
:leftabove {cmd}
:aboveleft {cmd}
:rightbelow {cmd}
:belowright {cmd}
:topleft
:botright
Cursor movement and window movement
Ctrl-w k
Ctrl-w j
Ctrl-w h
Ctrl-w l
Move cursor to Nth window above current one.
Move cursor to Nth window below current one
Move cursor to Nth window left of current one
Move cursor to Nth window right of current one
Ctrl-w w
Ctrl-w W
Ctrl-w t
Ctrl-w b
Ctrl-w p
move cursor to window below/right of current one.
move cursor to window above/left of current one.
move cursor to top-left window
move cursor to bottom-right window
move cursor to previous window
Ctrl-w K
Ctrl-w J
Ctrl-w H
Ctrl-w L
move window to be at the very top, with full width of screen
at the very bottom, with full width of screen
at the far left, with full height of screen
at the far right, with full height of screen
Ctrl-w r
Ctrl-w R
rotate windows downwards/rightwards
upwards, leftwards
Ctrl-w x exchange current window with next one or with Nth window if count is given. The two exchanged windows must be located in the same row or column.
Ctrl-w T Move current window to a new tab

Buffer
A buffer is a file loaded into memory for editing.
Unlisted buffer: It means it exists, but it is not listed in buffer list.
active buffer:  (1) displayed in a window (2) loaded
hidden buffer:  (1) is not displayed in a window, (2) loaded into memory
inactive buffer:(1) is not displayed (2) not loaded (3) Options are remembered.
unlisted buffer: some buffers are not listed in buffer list. when you apply :bdelete or :bwipeout to a buffer, it would be deleted from buffer list.
hidden buffer -> unhidden buffer: start to edit it with any command or delete it with :bdelete command.

:q[uit] [!]  quit current window. The buffer becomes inactive.
For following three commands, modified buffers are never abandoned (never becomes inactive).
:close [!]   close current window. (1) if 'hidden' is set, the buffer becomes hidden  (2) else abandon it (becomes inactive).
:hid[e]      Quite current window and state of the buffer depends on option bufhidden.
:only[!]     Make the current window the only one in current tab. All other windows are closed.

Direct buffer editing command

# Remove association between a buffer and a file. The buffer is now just a buffer without any content.
:bd[elete][!] [N] Unload buffer [N] (default: current buffer) and delete it from buffer list. :bdelete[!] {bufname} Unload and delete buffer corresponding to {bufname} :bdelete[!] N1 N2 ... Unload and delete buffer N1, N2, ... :N,Mbdelete[!] Unload and delete all buffers in range N to M
# The buffer is removed thoroughly which means the buffer does not exist any more.

:[N]bwipout[!] Like :bdelete, but really delete the buffer. :bw[ipeout][!] {bufname} Everything releated to the buffer is lost. :N,Mbw[ipeout][!] :bw[ipeout][!] N1 N2 ...
# memory is freed. When you switch to that buffer later, the content would be read from the file.
:[N]bun[load][!] The memory allocated for those specified :bun[load][!] [N] buffers is freed. But the buffers remain in the :bunload[!]{bufname} buffer list. :N,Mbunload[!] :bunload[!] N1 N2 ...

Important options:
hidden: off - a buffer is unloaded when it is abandoned.
                   When (1) the buffer is modified and
                            (2) 'autowrite' is off or writing is not possible
                            (3) the '!' flag was used.
                   a buffer becomes hidden even if hidden option is off.
            on  - a buffer becomes hidden when it is abandoned.
bufhidden: what happens when a buffer becomes hidden.
    <empty> - follow hidden option
    hide  - the buffer becomes hidden (not unloaded)
    unload  - unload the buffer
    delete -  delete the buffer from buffer list (like using :bdelete)
    wipe - wipe out buffer from buffer list (like using :bwipeout)
When "unload", "delete" or "wipe" is used, changes in a buffer are load without any warning.

Usually, if option hidden is on, the operations of buffer manipulation commands depend on option bufhidden.
If option hidden is off, buffer is unloaded


A list of buffer/arg switch command from vim help doc:

args list		       buffer list	   meaning ~
1. :[N]argument [N]	11. :[N]buffer [N]	to arg/buf N
2. :[N]next [file ..]	12. :[N]bnext [N]	to Nth next arg/buf
3. :[N]Next [N]		13. :[N]bNext [N]	to Nth previous arg/buf
4. :[N]previous	[N]	14. :[N]bprevious [N]	to Nth previous arg/buf
5. :rewind / :first	15. :brewind / :bfirst	to first arg/buf
6. :last		16. :blast		to last arg/buf
7. :all			17. :ball		edit all args/buffers
			18. :unhide		edit all loaded buffers
			19. :[N]bmod [N]	to Nth modified buf

  split & args list	  split & buffer list	   meaning ~
21. :[N]sargument [N]   31. :[N]sbuffer [N]	split + to arg/buf N
22. :[N]snext [file ..] 32. :[N]sbnext [N]      split + to Nth next arg/buf
23. :[N]sNext [N]       33. :[N]sbNext [N]      split + to Nth previous arg/buf
24. :[N]sprevious [N]   34. :[N]sbprevious [N]  split + to Nth previous arg/buf
25. :srewind / :sfirst	35. :sbrewind / :sbfirst split + to first arg/buf
26. :slast		36. :sblast		split + to last arg/buf
27. :sall		37. :sball		edit all args/buffers
			38. :sunhide		edit all loaded buffers
			39. :[N]sbmod [N]	split + to Nth modified buf

40. :args		list of arguments
41. :buffers		list of buffers

More Misc. commands
:scriptnames    a list of sourced script files in the order they were first sourced.
:verbose set [option]
:verbose map
:echo g:colors_name
:Ctrl-w T
:read !{cmd}
:tab {cmd}    execute the external command and open the result file in a new tab. E.g. :tab split
:tab split filename
:tabedit filename
:tab ball    open all buffers, each in a separate tab page.
:tabnew |r !dir    execute command dir and redirect its output to a file opened in a new tab page.
:tabnew |:E
:echo functionname(argument)  call an internal function and returns the output
:call functionname(argument)    call an internal function and returned value is discarded.
:exe string_expr    execute the commands contained in the string in Ex mode.

Windows file association
For example, if you want to associate .txt files with vim, use following commands:

	ftype VIM="e:\program files\vim\vim72\gvim.exe" --remote-tab-silent "%1"
	assoc .txt=VIM
Replace the path with your own vim installation directory..
If you want to maximize the window, use
               ftype VIM="e:\program files\vim\vim72\gvim.exe" -c "simalt ~x" --remote-tab-silent "%1"

Monday, January 26, 2009

More VIM text editing command

The more I use VIM, the more it surprises me.
Here, I list some useful commands in everyday text editing. I have known some of them for long time. But others were learnt recently.

Command Explanation Note
Delete
cc delete current lines and change to insert mode  
dd delete current lines  
c{motion} delete until the character {motion} moves to and change to insert mode.  
d{motion} delete until the character {motion} moves to  
C delete characters until the end of the line. Then change to insert mode  
D delete characters until the end of the line.  
s delete [count] characters and change to insert mode  
x delete [count] characters under and after the cursor  
:[range]d delete those lines in the range  
S delete [count] lines and change to insert mode  
Replace
R Enter Replace mode  
gR    
Line joining
[range]J join [count] lines

Steps of execution of these commands
(1) delete leading white space on the next line,
(2) if there is trailing white space for current line the next line starts with a ')',
        delete <EOL>
    else if the 'joinspaces' option is on and current line ends up with a '.', '!' or '?'
        replace <EOL> with two spaces
    else
        replace <EOL> with one space

{Visual}J join the selected lines
:[range]j[!] join range lines. With !, the join doesn't insert/remove any spaces.
[range]gJ join [count] lines Don't insert or delete any spaces.
{VIsual}gJ join the selected lines

Some commands useful in source code editing

Commands Explanation
<{motion} shift covered lines leftwards by 1 shiftwidth
>{motion} shift covered lines rightwards by 1 shiftwidth
{Visual}[count]< shift selected lines leftwards by 1 shiftwidth
{Visual}[count]> shift selected lines rightwards by 1 shiftwidth
[range]< shift selected lines leftwards by 1 shiftwidth
[range]> shift selected lines rightwards by 1 shiftwidth
<< shit current line leftwards by 1 shiftwidth
>> shit current line rightwards by 1 shiftwidth

Resource
http://www.vim.org/htmldoc/change.html

Text formatting in VIM

How to set options for VIM?

using set command.
Examples:
Setting Explanation
set paste set options for commands that don't need parameters.
set filetype=java Directly set options for commands that accept parameters.
set guioptions-=option Remove a specific option from the configuration of a command.
set guioptions+=option Add a specific option to the configuration of a command.

Basic concept
Sentence/Paragraph/Section: http://www.vim.org/htmldoc/motion.html#sentence

Text format
http://www.vim.org/htmldoc/change.html#formatting
Alignment

Setting Explanation Examples
:[range]ce[nter] [width] center lines :.,+3 center 80
:[range]ri[ght] [width] right alignment :% right
:[range]le[ft] [indent] left alignment.(Unit of identation is space) :left 4

Format

Command

Explanation

Examples

Note

gq{motion} Format lines that {motion} moves over.
Cursor is put where {motion} moves to
gqap
format a paragraph
Motion:
http://www.vim.org/
htmldoc/motion.html
gw{motion} Format lines that {motion} moves over.
Put cursor back to the original position.
gwj
gqgq
gqq
Format current line    
{Visual}gq Format highlighted text    

How to control formatting setting of gq command?
Formatting does not change empty lines, but it changes lines only containing white spaces.

(1) textwidth
This option controls length of every formatted line.
If option textwidth is 0, the formatted line length is the screen width (max value is 79).

(2) autoindent
If this option is on, Vim uses indent of the first line for the following lines.

(3) joinspaces (boolean)
Used when lines are joined together. By default, it is on. Use set nojoinspaces to turn it off.

(4) formatprg
Set an external program to format text. In this case, the textwidth and other options have no effect on text formatting.

(5) formatoptions
Most useful options:
t: Auto-wrap text using textwidth. (does not apply to comments)
c: Auto-wrap commens using textwidth.
q: Allow formatting of comments with gq command. When using gq command in comments, blank lines and lines only with comment leaders and white spaces are considered as paragraph delimiters.
r: auto insert comment leader after hitting 'Enter' in Insert mode.
o: auto insert comment leader after hitting 'o' or 'O'.
a: auto format paragraphs when text is inserted or deleted.
w: A trailing non white space ends a paragraph.
Note: Formatting would be applied anyway no matter whether the paragraphs are delimited correctly.
You can use set formatoptions-=a to disable automatic formatting.
More options: http://www.vim.org/htmldoc/change.html#fo-table

(6) comments
This option controls how to format comments (usually in source code).
Default value is 

"s1:/*,mb:*,ex:*/,://,b:#,:%,:XCOMM,n:>,fb:-"
s: start of a three-piece comment
s{digit}: add extra indent to middle part. Unit is space (not tab).
s-{digit}: remove extra indent to middle part. Unit is space (not tab).
m: middle part of a three-piece comment. Middle part is left adjusted with start part of the comment by default. This is controlled by option l.
b: blanks required after the specified string.
e: end of a three-piece comment
x: In C++ comment, just type / to end a comment when middle-comment string is inserted. The space between middle-part and / would be removed automatically.
n: recognize numbered list(see below for details)
More options: http://www.vim.org/htmldoc/change.html#format-comments
My setting is
set textwidth=79
set formatoptions=tcqron

How to format numbered lists?
This sometimes is really tricky.
See this post: http://objectmix.com/editors/332035-vim-how-create-bullet-list-numbered-list.html
Usually, textwidth must be set and formatoptions must include nwt.
Also see option formatlistpat(flp) for how to set list header. That is used by vim to recognize numbered lists.

Thursday, January 22, 2009

MyProxy and proxy certificate

All of following discussion is based on PKI (public key infrastructure).

What's important in terms of security?
integrity of the user's identity credential(certificate + public/private key pair).
Note: What is important is not just a single part of the credential, but also integrity of different parts. If a certain component is compromised, the whole credential is not secure any more.

how to protect private key?
(1) Store it in desktop
May not be a good idea because usaually desktop is not protected professionally. It is vulnerable to keystrok loggers, trojan ...
(2) Smart card
A specific piece of hardware provides improved security for key storage.
Pros: allows keys to be used for signing and encryption without allowing the keys to be exported from the card's protected memory.
Cons: cost and support
(3) Virtual smart card
Similar to the physical smart card approach. It is a software system which protects private keys on secure servers without allowing the keys to be exported.

Grid computing
Usually, the user has a certificate issued by a CA and it has long life time (maybe several years). Before successul running of a task, the certificate must be authenticated by grid infrastructure. Tasks may be submitted and launched from different hosts.
(*) So it is not convenient to ask users to copy their credentials manually across different machines.
(*) It is not secure to put long-term credentials to many hosts in the grid.
It would be great if users can obtain a short-time(hours or days) credential which can be used for authentication during execution of a task.
There are two solutions:
(1) Online CA
Every time the user needs a short-time credential, he/she just sends certificate request to CA and CA returns the newly created certificate.
Drawback: cost. Adding a new CA needs a lot of extra work in terms of time, money, negotiating policies...
(2) Online credential repository
This solution makes use of X.509 proxy certificate described below. 
MyProxy uses the third approach.

Proxy Certificate (RFC 3820)
Key idea: An entity delegates its rights or a subset of its rights to another entity which can behave on behalf of the original entity.
Security
Restrictions can be placed on the PC be means of policies.
Terminology
EEC(End Entity Certificate): an X.509 Public Key Certificate issued to an end entity by a CA.
PC(Proxy Certificate): can be issued by EEC or another PC.
PI(Proxy Issuer): an entity with an EEC or PC that issues a PC. The issued PC is signed using the corresponding private key.
Properties of PC
(*) Signed by EEC or PC
(*) It can sign another PC. Cannot sign EEC
(*) Its public/private key pair is different from that of the proxy issuer.
(*) Its identity is derived from identity of the original ECC that signed the PC.
Note: its identity is unique so that it can be used as an independent identity.
(*) It contains a new X.509 extension to indicate that it is a PC and to place restrictions on use of the PC.
Procedure
If an entity A wants to delegate its rights to entity B, the procedure is
(1) Entity B creates a private/public key pair
(2) Entity B creates a request for a PC and sends it to entity A
(3) Entity A creates a PC, singed by the private key of A's certificate (EEC or another PC).
In this process, restrictions can be placed on the newly created PC. For example, validity period...
(4) Entity A sends the certificate back to entity B.
Note: the private key of entity A is stored on the local file system.
Significant advantages
Private keys are never transferred over the internet.

How MyProxy works?
Proxy storing
This is similar to the procedure described above in section <Proxy Certificate> / <Procedure>. Entity A is the end user and entity B is MyProxy server. In step (4), end user sends both the newly created Proxy Certificate and the entire certificate chain to the server. Also access control can be imposed by means described below.
Proxy retrieval
Similar to proxy storing, except in reverse. Access control policies must be checked to see whether the client is allowed to retrieve the certificate.
Access Control
Server administrator can control who can store/retrieve credentials by:
    (1) Requiring client-side TLS authentication
    (2) Configuring a regular expression that must match the client certificate's subject for storage requests
    (3) Configuring a regular expression that must match the client certificate's subject for retrieval requests
Credential owner can control access to their certificates by :
    (1) setting a password when storing the certificates
     The password is not stored in the server. It is used to encrypt server's private key which would be used to sign the proxy certificates.
    (2) settting regular expression on the client certificate's subject that would retrieve the certificate.
Renewal
    to be filled soon

To be investigated in the future
MyProxy and OGSI (Open Grid Services Infrastructure):
    MyProxy is encapsulated as web services.
MyProxy and WS-Resource Framework

Wednesday, January 21, 2009

Uniform Resource Identifier(URI) and Uniform Resource Locators (URL) - (RFC 1738 and RFC 3986)

Character Escape
URI consists of a set of characters.
uric = reserved | unreserved | escaped

Reserved characters
Principle: a character is reserved if the semantics of the URI changes if the character is replaced with its escaped escaped encoding.
    gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"
    sub-delims  = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "="


Their usage within URI component is limited to their reserved purpose. If the data would conflict with the reserved purpose, it must be escaped.
Characters in the "reserved" set are NOT reserved in all contexts. The reserved characters in a URI component is defined by that specific component.

Unreserved characters
a-z A-z 0-9 "-" "_" "." "~"
These characters can be escaped WITHOUT changing the semantics of the URI.
But this should NOT be done UNLESS the escape is necessary.

Disallowed characters
Some characters are disallowed for various reasons. To use those characters, they MUST be escaped.
Disallowed US-ASCII Characters:
control:      <US-ASCII coded characters 0x00-0x1F and 0x7F>
space:        <US-ASCII coded character 0x20>
delimiters:  < > # % "
unwise:       { } | \ ^ [ ] `

When to escape?
When a character does not have a representation using an unreserved character, it must be escaped. It includes:
(1) data that does not correspond to printable characters (ANSII coding)
(2) disallowed characters
Note: here, whether a character is unreserved is context-specific.

Escape sequences:
A "%" followed by hex representation of the character.
escaped = "%" hex hex
E.g. %20 %35
Uppercase hexadecimal digits should be used in percent-encoding!

Syntax
Generic URI syntax:
 
    <scheme>:<scheme-specific-part>
Interpretation of scheme-specific-part depends on the scheme.
    <scheme>://<authority><path>?<query>

scheme
    alpha *( alpha | digit | "+" | "-" | "." )

authority
URI component authority can be internet-based server or a scheme-specific registry.
authority (server based) = username@host:port
userinfo = *( unreserved | escaped |";" | ":" | "&" | "=" | "+" | "$" | "," )
About domain label:

"The rightmost domain label of a fully qualified domain name will never start with a digit, thus syntactically distinguishing domain names from IPv4 addresses, and may be followed by a single "." if it is necessary to distinguish between the complete domain name and any local domain."

Query
query = *uric
Within a query component, the characters ";", "/", "?", ":", "@", "&", "=", "+", ",", and "$" are reserved.

Fragment
Fragment is not part of a URI, but is often used in conjunction with a URI.
URI-reference = [ absoluteURI | relativeURI ] [ "#" fragment ]

from RFC 2396:
"The semantics of a fragment identifier is a property of the data resulting from a retrieval action, regardless of the type of URI used in the reference.
   A fragment identifier is only meaningful when a URI reference is intended for retrieval and the result of that retrieval is a document for which the identified fragment is consistently defined."

Relative URI reference
to be continued in the future.

Specific schemes

scheme syntax Explanation Note
file file://<host>/<path> Access a file on a specific host.
<host> can be "localhost" or empty to indicate local host. E.g. file:///usr/home
Unlike http and ftp, It does not specify an internet protocol to access the files.
ftp ftp://<host>:<port>/
<cwd1>/<cwd2>/.../<cwdN>/
<name>;type=<typecode>
<cwd1> through <cwdN> are strings and <typecode> can be "a", "i" or "d". If <typecode> is "d", <name> is used as the argument of NLIST command. Within the <name> or a CWD component, / and ; must be escaped. E.g. ftp://test.com/%2Froot/a.txt
mailto mailto:<mail-address> RFC 2822 specifies the format of internet messages. Usually, "%" must be escaped.
http http://<host>:<port>/
<path>?<query>
   

Resources
URI working group: http://labs.apache.org/webarch/uri/

Wednesday, January 14, 2009

Some resources of symbolic learning and connectionist

This post http://www.cs.indiana.edu/classes/c661/sym-subsym.html gives some breif introduction and comparison of these two methods. It is pretty good.

Also I found this paper - Comparing connectionist and symbolic learning method which compares these two methods based on some experimental results. It is an old paper but I think maybe it is worth reading.

After searching on web, I found this book: Perspectives of Neural-Symbolic Integration(http://www.neural-symbolic.org/). Its name seems to imply that this book will introduction the combination of these two methods. I have not read it yet.

Wednesday, January 07, 2009

MySQL error 1045(28000): Access denied for user ...

Problem
I have a database called hibernate in my MySQL server and I grant privileges to a user using this command
    > grant all on hibernate.* to 'testuser'@'%' identified by 'testuser';
Wildchar % means no matter where the user logs in mysql server from, the access should be allowed for further checking/verification.
When I tried to log in as the newly created user from local host, I got this error:
    ERROR 1045 (28000): Access denied for user 'testuser'@'localhost' (using password: YES)
Also I tried some suggested solutions:
   > FLUSH PRIVILEGES:
   > SET PASSWORD FOR 'testuser'@'%'='testuser';
None of them works.

My solution:
Using following two commands:
   > grant all on hibernate.* to 'testuser'@'%' identified by 'testuser';
   > grant all on hibernate.* to 'testuser'@'localhost' identified by 'testuser';

Theoretically, wildchar % should have included all hosts. But it may not cover localhost. Who knows.

Install and Configure MySQL as a non-root user on Linux

Recently, I want to install MySQL on a server in my lab. It should not be difficult considering the wide usage of MySQL. I must install MySQL as a non-root user. This results in difficulties of installation.
My target OS is Red Hat Enterprise Linux Server release 5.2.
My username is gerald and my home directory is /home/gerald.

MySQL website provides a binary package(rpm) for RedHat. I downloaded package MySQL-server-community-5.0.67-0.rhel5.i386.rpm.
After executing command
> rpm -q -p MySQL-server-community-5.0.67-0.rhel5.i386.rpm -i
I got following output:

Name        : MySQL-server-community       Relocations: (not relocatable)
Version     : 5.0.67                       Vendor: MySQL AB
Release     : 0.rhel5                      Build Date: Mon 04 Aug 2008 03:31:42 PM EDT
Install Date: (not installed)              Build Host: blade11.mysql.com
Group       : Applications/Databases       Source RPM: MySQL-community-5.0.67-0.rhel5.src.rpm
Size        : 42294654                     License: GPL
Signature   : DSA/SHA1, Wed 06 Aug 2008 05:36:22 AM EDT, Key ID 8c718d3b5072e1f5
Packager    : MySQL Product Engineering Team <build@mysql.com>
URL         : http://www.mysql.com/
Summary     : @COMMENT@ for Red Hat Enterprise Linux 5
See the red text above?
So It did not work because the rpm package should be installed as root.

As a result, I must install MySQL from source. Steps:

Installation
(1) Download and untar MySQL-5.1.30. Then Change location into the top-level directory of the unpacked distribution.
(2) Configure
CFLAGS="-O3" CXX=gcc CXXFLAGS="-O3 -felide-constructors -fno-exceptions -fno-rtti" ; \
./configure --prefix=/home/gerald/share/mysql \
           
--enable-assembler \
            --with-mysqld-ldflags=-all-static \
            --with-client-ldflags=-all-static \
            --with-mysqld-user=gerald \
            --with-unix-socket-path=/home/gerald/share/mysql/tmp/mysql.sock \
            --localstatedir=/home/gerald/share/mysql/data

--localstatedir: overrides the default location for database directories (normally /usr/local/var for installation from source)
--with-unix-socket-path: where unix socket file would be stored. (must be accessible)
--with-mysqld-ldflags=-all-static: compile statically linked daemon programs
--with-client-ldflags=-all-static: compile statically linked mysql client programs
--enable-assembler: Use assembler versions of some string functions if available
--prefix: Install architecture-independent files in the specified directory.

Values of some of options above should be replaced with your own paths.
Also this option --enable-thread-safe-client may should be applied in your case if your client programs use multi-thread.
Description of all available configuration options is here http://dev.mysql.com/doc/refman/5.1/en/configure-options.html.
(3) make
(4) make install

Post-installation
http://dev.mysql.com/doc/refman/5.1/en/unix-post-installation.html
(5) Create and edit configuration file (also called option file) my.cnf.
http://dev.mysql.com/doc/refman/5.1/en/option-files.html
My my.cnf file looks like this:

[mysqld]
user=gerald
basedir=/home/gerald/share/mysql
datadir=/home/gerald/share/mysql/data
port=3307
socket=/home/gerald/share/mysql/tmp/mysql.sock

[mysqld_safe]
log-error=/home/gerald/share/mysql/log/mysqld.log
pid-file=/home/gerald/share/mysql/mysqld.pid

[client]
port=3307
user=gerald
socket=/home/gerald/share/mysql/tmp/mysql.sock

[mysqladmin]
user=root
port=3307
socket=/home/gerald/share/mysql/tmp/mysql.sock

[mysql]
port=3307
socket=/home/gerald/share/mysql/tmp/mysql.sock

[mysql_install_db]
user=gerald
port=3307
basedir=/home/gerald/share/mysql
datadir=/home/gerald/share/mysql/data
socket=/home/gerald/share/mysql/tmp/mysql.sock

On Unix, MySQL programs read startup option file from the following files:

Filename Purpose
/etc/my.cnf Global options
/etc/mysql/my.cnf Global options (as of MySQL 5.1.15)
SYSCONFDIR/my.cnf Global options
$MYSQL_HOME/my.cnf Server-specific options
defaults-extra-file The file specified with --defaults-extra-file=path, if any
~/.my.cnf User-specific options

I use the fourth option. So I set environment variable MYSQL_HOME using this command:
    > export MYSQL_HOME=/home/gerald/share/mysql
This environment variable contains the directory where mysql is installed.
If you don't want to manually set the environment variable every time you start up the mysql daemon, you can add it to file ~/.bash_profile(if you are using bash).
Then copy the option file to the directory specified by MYSQL_HOME.
Note: options specified in my.cnf can also be supplied on the command line when you execute a mysql program.
(6) Change location to the directory where mysql is installed (in my case, it is /home/gerald/share/mysql).
(7) Initialize system tables using command
    >bin/mysql_install_db
Options of this command are specified in our option file $MYSQL_HOME/my.cnf. Of course, you can specify the options in command line. But I think using the option file is more convenient.
I got a warning: [Warning] Ignoring user change to 'gerald' because the user was set to 'mysql' earlier on the command line. I don't know how to fix it now. But mysql seems to work well despite the warning.
Note: If you install MySQL using a binary package, usually this step is automatically done when you install the package.
(8) mkdir log
create log directory under which log files are saved. This must match the configuration of option log-error in my.cnf.
In my case, log-error is set to /home/gerald/share/mysql/log/mysqld.log
(9) Start up mysql using command
    > bin/mysqld_safe &
(10) Verify that mysql daemon runs well. Use any of following commands:

> bin/mysqladmin version
> bin/mysqladmin ping
> bin/mysqladmin status
> bin/mysqladmin variables
Show the created databases
> bin/mysqlshow
Output:
+--------------------+
|      Databases     |
+--------------------+
| information_schema |
| test               |
+--------------------+

> bin/mysqlshow-u root
Output:
+--------------------+
|      Databases     |
+--------------------+
| information_schema |
| mysql              |
| test               |
+--------------------+

(11) shut down mysql daemon
   > bin/mysqladmin -u root shutdown

Initial Account Security
http://dev.mysql.com/doc/refman/5.1/en/default-privileges.html
By default, password of root user is empty. In other words, anyone who can access the host can also access information of your mysql database. You can set password of root user using
    > mysqladmin -u root password newpwd
or
    > mysqladmin -u root -h host_name password newpwd

Account management statements
http://dev.mysql.com/doc/refman/5.1/en/account-management-sql.html

Privilege system
http://dev.mysql.com/doc/refman/5.1/en/privilege-system.html
Usually, you need to check table mysql.user:
    > select * from mysql.user;
How can you know the indentity used by mysql?
After running mysql successfully, use command
    > select CURRENT_USER();
Set password of a user:
    > SET PASSWORD FOR 'someuser'@'somehost' = PASSWORD('newpass');
Grant privileges:
Usually, commands are like this:

    > GRANT ALL ON mydb.* TO 'someuser'@'somehost';
    > GRANT ALL ON mydb.mytbl TO 'someuser'@'somehost';
    > GRANT SELECT (col1), INSERT (col1,col2) ON mydb.mytbl TO 'someuser'@'somehost'; 
Revoke privileges:
    > revoke all on mydb.* from 'someuser'@'somehost';
Remove user:
    > drop user 'someuser'@'somehost'

Resources
Related environment variables:
http://dev.mysql.com/doc/refman/5.1/en/environment-variables.html
List of programs/tools in the mysql package:
http://dev.mysql.com/doc/refman/5.1/en/programs.html

Monday, January 05, 2009

Ubuntu error "I've detected a panel already running"

Prerequisite: I use GNOME, not KDE.
I want to install Nvidia driver to my old Ubuntu machine. However, it prompts that a X server is running and the installation can not proceed. Obviously, X server must be killed to install the driver. To press Ctrl+Alt + Backspace does NOT do the work because it RESTARTS the X server instead of kills it.
After searching online, I found that following command worked for me:

sudo /etc/init.d/gdm stop

After I installed the driver, I used command startx to start GNOME. At that time the error "I've detected a panel already running ..." came out. I found this command worked:

killall gnome-panel
killall gdm
startx

In other words, the processing gnome-panel is not terminated even if you press Ctrl+Alt+Backspace to kill the X server.

Saturday, January 03, 2009

Version matching of JSP, JSTL and Servlet

Working with JSP, JSTL, Servlet and Container is not an easy job if you download and configure every part by yourself. The most important problem is version matching. Also web.xml must be correctly configured to make use of the right version of Servlet/JSP.

Web app descriptor (web.xml)
To make use of correct version of JSP/JSTL/EL, you should read following posts which give detailed information you need to know to write web.xml:
http://faq.javaranch.com/java/ServletsWebXml

Some useful resources I found:
http://faq.javaranch.com/java/ElOrJstlNotWorkingAsExpected
http://forum.springframework.org/archive/index.php/t-19866.html
http://blog.csdn.net/eviliw/archive/2007/12/17/1944270.aspx
http://faq.javaranch.com/java/JstlTagLibDefinitions

On Sun's website, it is so hard to find a download link for JSTL 1.2. Usually you will be directed to the JSTL spec site.
You can download JSTL 1.2 from here: https://maven-repository.dev.java.net/repository/jstl/jars/.
Note for JSTL 1.2 there is just one jar file instead of two.
In JSTL 1.1/1.0, there are two jars : jstl.jar and standard.jar.
You can download JSTL 1.1 from here http://jakarta.apache.org/site/downloads/downloads_taglibs-standard.cgi and JSTL 1.0 from http://jakarta.apache.org/site/downloads/downloads_taglibs-standard.cgi.

Version match
JSTL 1.0 : Servlet 2.3 : JSP 1,2 (tomcat 4)
JSTL 1.1 : Servlet 2.4 : JSP 2.0 (tomcat 5)
JSTL 1.2 : Servlet 2.5 : JSP 2.1 (tomcat 6)
In old versions of JSP, EL is not enabled by default!! You can enable EL manually by using JSP page directory.
JSTL  jars are not included in tomcat distributions so far. You need to download and deploy JSTL by yourself.

JSTL 1.0 specifies a set of custom tag libraries based on the JSP 1.2 API. There are four separate tag libraries, each containing custom actions targeting a specific functional area. This table lists each library with its recommended tag prefix and default URI:

Description Prefix Default URI
Core c http://java.sun.com/jstl/core
XML Processing x http://java.sun.com/jstl/xml
I18N & Formatting fmt http://java.sun.com/jstl/fmt
Database Access sql http://java.sun.com/jstl/sql

JSTL 1.1 specifies a set of custom tag libraries based on the JSP 2.0 API. There are five separate tag libraries, each containing custom actions targeting a specific functional area. This table lists each library with its recommended tag prefix and default URI:
Description Prefix Default URI
Core c http://java.sun.com/jsp/jstl/core
XML Processing x http://java.sun.com/jsp/jstl/xml
I18N & Formatting fmt http://java.sun.com/jsp/jstl/fmt
Database Access sql http://java.sun.com/jsp/jstl/sql

One missed tag in the table is "functions".
It seems that you also can use tag URI like this: http://java.sum.com/jstl/core_rt. And it works in my application. However, I have not investigated what's going on behind the scene. So this may or may not work generally. Instead, you should always use the new tag URI.

Prefixes and URL of JSTL 1.2 Tag Libraries are the same as that of JSTL 1.1.

Troubleshooting
(1) If you get following error "According  to TLD or attribute directive in tag file, attribute test does not accept any expressions", it is possible that you don't specify tag prefix and URI correctly. You may be using JSTL 1.2 while you specify tag URI of JSTL 1.0 wrongly. Take Core as an example, tag URI in JSTL 1.1/1.2 is http://java.sun.com/jsp/jstl/core while tag URI in JSTL 1.0 is http://java.sun.com/jstl/core.
(2) If your EL/JSP is displayed directly without evaluation. it is highly possible that your web.xml file specifies wrong JSP version. See this post http://faq.javaranch.com/java/ServletsWebXml.