@@@@@@@=))) @=))) @=))) @=))) @=))) @=))) @=))) @=))) @@@@@=)@@@@@@=))) @@=) @=))) @=))F o r a l l)) h a c k e r s.))) @=@=))@=))) @=))) @@@@@@=@=)))@=))) @=)@@@@@=)))@= @=))@=))) @=))) @=))@=)))@=))) @=)@=)))@=)@@@@@=))@=))) @=))) @=))@=)))@=))) @=)@=)))@=@=)))@=))@=))) @@@@@@@@=)))@@@@=)@@@@@=)@@@@@@=@@@=)@@@@=)@@@@@@=))) Shigio Yamaguchi 10-Dec-98 Copyright 1996, 1997, 1998 Shigio Yamaguchi All right resereved. GLOBAL is a source code tag system that works the same way across diverse environments. It supports C, Yacc and Java source code. It brings benefits to all hackers. Enjoy! Contents -------------------------------- 0. Introduction 1. Global 1.1. Features 1.2. Preparation 1.3. Basic usage 1.4. Applied usage 2. Extended nvi using global 2.1. Features 2.2. Preparation 2.3. Basic usage 2.4. Applied usage 3. Extended emacs using global 3.1. Features 3.2. Preparation 3.3. Basic usage 3.4. Applied usage 4. Hypertext generator 4.1. Features 4.2. Preparation 4.3. Usage 4.4. Making hypertext of the kernel 4.5. Gozilla 5. Elvis using global 5.1. Features 5.2. Preparation 5.3. Basic usage 5.4. Applied usage A. Other topics A.1. How to plug in a parser A.2. How to use compact format A.3. Incremental updating A.4. Plans for the furture -------------------------------- 0. Introduction GLOBAL is a source code tag system that works the same way across diverse environments. Currently, it supports the following: o Shell command line(see '1. Global') o nvi editor(see '2. Extended nvi using global') o emacs editor(see '3. Emacs using global') o Web browser(see '4. Hypertext generator') o Elvis editor(see '5. Elvis using global') Supported languages are C/Yacc and Java. You can locate a specified function in the source files and move there easily. It is useful for hacking a large project containing many subdirectories, many '#ifdef' and many main() functions, like MH, X or BSD kernel. GLOBAL consists of global(1), gtags(1), btreeop(1), gctags(1), htags(1), extended nvi(1), gtags.el and gozilla(1). * 'extended' means being entended for GLOBAL. * Btreeop and gctags are used internally, so you do not need to understand them. ------------------------------------------------------------------------------ 1. Global 1.1. Features o Global can find the locations of a specified object quickly. o Global can locate not only object definitions but also object references. o Global allows duplicate objects. o Global can treat a source tree containing subdirectories as a logical scope. You can get the relative path of objects from anywhere within that scope. o Global allows duplicate objects. o Global can understand POSIX 1003.2 regular expressions. o Global can search not only in a source tree but also in library paths. o Tag files are indepent of machine architecture. o Global can use a plugged in parser with the global.conf (or $HOME/.globalrc). o Global can use a tag file in a compact format to save disk space. I think these features are useful for a large project containing many subdirectories, many '#ifdef' and many main() functions like MH, mozilla, X and kernels. 1.2. Preparation First of all, you must execute gtags(1) at the root of source tree. For example, if you want to browse vi's source code: % cd /usr/src/usr.bin/vi % gtags Gtags traverse subdirectories and makes three databases at the root of the source tree. % ls G*TAGS GRTAGS GTAGS GSYMS GTAGS - database of function definitions GRTAGS - database of function references GSYMS - database of other symbols 1.3. Basic usage Consider the following source tree: ROOT/ <- the root of source tree (GTAGS,GRTAGS) | |- DIR1/ | | | |- fileA.c ..... +---------------+ | | |main(){ | | | | func1();| | | | func2();| | | |} | | | +---------------+ | | | |- fileB.c ..... +---------------+ | |func1(){ ... } | | +---------------+ |- DIR2/ | |- fileC.c ..... +---------------+ |#ifdef X | |func2(){ i++; }| |#else | |func2(){ i--; }| |#endif | |func3(){ | | func1();| |} | +---------------+ You can get the relative path of your object from anywhere in the source tree. You need not specify where the tag file is. Global will locate the tag file by itself. % cd ROOT % global func1 DIR1/fileB.c <- func1() is defined in fileB.c % cd DIR1 % global func1 fileB.c <- relative path from DIR1 % cd ../DIR2 % global func1 ../DIR1/fileB.c <- relative path from DIR2 The -r option locates function references. % global -r func2 ../DIR1/fileA.c <- func2() is referred from fileA.c You can use POSIX regular expressions. % cd ROOT % global 'func[1-3]' DIR1/fileB.c <- func1, func2 and func3 are matched DIR2/fileC.c The -x option shows the details. It is similar to the -x option in ctags(1). % global func2 DIR2/fileC.c % global -x func2 func2 2 DIR2/fileC.c func2(){ i++; } func2 4 DIR2/fileC.c func2(){ i--; } The -a option produces the absolute path name. % global -a func1 /home/user/ROOT/DIR1/fileB.c The -s command locates any symbols other than functions. % global -xs X X 1 DIR2/fileC.c #ifdef X The -g command locates any patterns including symbols. It is similar to grep(1). % global -xg '#ifdef' #ifdef 1 DIR2/fileC.c #ifdef X You can edit all files that include a specified function by typing one command, for example: % vi `global func1` <- edit fileB.c 1.4. Applied usage You can make multiple tag files. For example, you can execute gtags at ROOT/, version1.0/ and version2.0/. ROOT/ <- the root of source tree (GTAGS,GRTAGS) | |- version1.0/ <- the root of version1.0 (GTAGS,GRTAGS) | | | |- file.c ..... +---------------+ | |func1(){ i++; }| | +---------------+ | |- version2.0/ <- the root of version2.0 (GTAGS,GRTAGS) | |- file.c ..... +---------------+ |func1(){ i--; }| +---------------+ When you are in the version1.0 directory, global will only locate functions that are in version1.0. % cd ROOT/version1.0 % global -x func1 func1 1 file.c func1(){ i++; } When you are in the version2.0, global will only locate functions that are in version2.0. % cd ROOT/version2.0 % global -x func1 func1 1 file.c func1(){ i--; } If you are at ROOT/, or you set the GTAGSROOT environment variable to ROOT, then global will locate functions in both directories. % cd ROOT % global -x func1 func1 1 version1.0/file.c func1(){ i++; } func1 1 version2.0/file.c func1(){ i--; } =-=-=-= There is another usage of GTAGSROOT. If your source files are on a read-only device, such as CDROM, then you cannot make databases at the root of the source tree. In such cases, please do the following: % mkdir /var/dbpath % cd /cdrom/src <- the root of source tree % gtags /var/dbpath <- make tag file in /var/dbpath % setenv GTAGSROOT `pwd` % setenv GTAGSDBPATH /var/dbpath % global func =-=-=-= If you want all references to a function that is not defined in the source tree to be treated as calls to library functions or system calls, then you can specify library directories with the GTAGSLIBPATH environment variable. You should execute gtags at each directory of the path. If GTAGS is not found in a directory, global ignores that directory. % pwd /develop/src/mh <- this is the source tree % gtags % ls G*TAGS GRTAGS GTAGS % global mhl uip/mhlsbr.c <- mhl() is found % global strlen <- strlen() is not found % (cd /usr/src/lib; gtags) <- library source % (cd /usr/src/sys; gtags) <- kernel source % setenv GTAGSLIBPATH /usr/src/lib:/usr/src/sys % global strlen ../../../usr/src/lib/libc/string/strlen.c <- strlen() is found in library % global access ../../../usr/src/sys/kern/vfs_syscalls.c <- access() is found in kernel Of course, the user program does not call kernel functions directly, but at least it is useful. =-=-=-= If you forget a function name, you can use the -c (complete) command. % global -c kmem <- maybe k..k.. kmem.. kmem_alloc kmem_alloc_pageable kmem_alloc_wait kmem_free kmem_free_wakeup kmem_init kmem_malloc kmem_suballoc <- This is what I need! % global kmem_suballoc ../vm/vm_kern.c You can use the -c command with tcsh's complete command. % set funcs=(`global -c`) % complete global 'n/*/$funcs/' % global kmem_ kmem_alloc kmem_alloc_wait kmem_free_wakeup kmem_malloc kmem_alloc_pageable kmem_free kmem_init kmem_suballoc % global kmem_s % global kmem_suballoc ../vm/vm_kern.c * means the tab key or Ctrl-I. =-=-=-= If you want to browse many files in order, do the following: % global -xr fork | awk '{printf "view +%s %s\n",$2,$3}' | tee /tmp/list view +650 ../dev/aic7xxx/aic7xxx_asm.c view +250 ibcs2/ibcs2_misc.c view +401 linux/linux_misc.c view +310 ../kern/init_main.c view +318 ../kern/init_main.c view +336 ../kern/init_main.c view +351 ../kern/init_main.c % sh !$ <- from now on, go to next tag with 'ZZ'. 2. Extended nvi using global 2.1. Features o The tag function of extended vi can locate not only function definitions but also function references. o Extended nvi allows duplicate tag entries. o Extended nvi can understand POSIX regular expressions as a tag name for the search. o Extended nvi is completely backward-compatible with the original nvi. The above functions are available only in 'gtags mode'. 2.2. Preparation First, do the preparation of global. (Please see "1.2. Preparation"). Second, to use global from vi, you need to get into 'gtagsmode'. There are several ways to do this: (a) Start vi with -G option % vi -G file.c (b) Start vi and execute "set gtagsmode" % vi file.c ~ ~ ~ :set gtagsmode (c) Write the above set command to the .exrc or .nexrc file and start vi $HOME/.exrc +---------------------------- |set gtagsmode You must start vi under the source tree described in "1.2. Preparation". 2.3. Basic usage o To go to func1, you can say :tag func1 It seemes the same as original vi, but extended vi use GTAGS instead of tags. o To go to the referenced point of func1, add the prefix 'r' :rtag func1 Extended nvi use GRTAGS. o If a number of functions are located, extended vi goes to the first tag. You can go to next tag by typing ':tagnext' and back by typing ':tagprev'. Suggested .nexrc: set gtagsmode map ^N :tagnext^M map ^P :tagprev^M o command is available. In gtags mode, if you are in the first column of a line, it is equivalent to ":rtag ", otherwise it is equivalent to ":tag ". o Other tag commands are also available: - Return to the most recent tag context. ":tagpop" - Go to the specified tag in the tags stack. ":tagtop" - Go to the top tag in the tags stack. ":display tags" - Display the tags stack. Please read the online manual. 2.4. Applied usage o In large projects that include many main() function like MH, you can start vi like this: % vi -G -t main You can browse all commands sequentially. o When you want to check functions the name of which start with "set" or "get", use: % vi -G -t '^[sg]et' Of course, the following command is also available: :tag ^[sg]et o If your source files are on a read only device like a CD-ROM, please do the following: % mkdir /var/dbpath <- directory for the tag file % cd /cdrom/src <- the root of the source tree % gtags /var/dbpath <- make tag files in /var/dbpath % setenv GTAGSROOT `pwd` % setenv GTAGSDBPATH /var/dbpath % vi -G -t main o If you want all references to function that are not defined in the source tree to be treated as references to library functions or as system calls, do the following: % cd /usr/src/lib % gtags <- probably as a root % cd /usr/src/sys % gtags % setenv GTAGSLIBPATH /usr/src/lib:/usr/src/sys If you examine vi's source, % cd /usr/src/usr.bin/vi % gtags % vi -G -t main You can start from vi and browse the whole unix world as if you were using hypertext. 3. Extended emacs using global 3.1. Features Addition to the extended vi, o More intelligent recognition of the current token and its type. o Tag completion is available for input tag name. o Symbol search and pattern search are available. o Mouse events are supported. 3.2. Preparation First, do the preparation of global. (Please see "1.2. Preparation"). Second, to use global from emacs, you need to load the gtags.el file and execute gtags-mode function in it. There are several ways to do this: (a) Start emacs, load gtags.el and execute gtags-mode function. % emacs | |J_:-----Mule: *scratch* (Lisp Interaction)--L16--All---- |M-x load-library[RET] +------------------------------------------------------ | |J_:-----Mule: *scratch* (Lisp Interaction)--L16--All---- |Load library: gtags[RET] +------------------------------------------------------ | |J_:-----Mule: *scratch* (Lisp Interaction)--L16--All---- |M-x gtags-mode[RET] +------------------------------------------------------ (b) Write the autoload function to the $HOME/.emacs file, start emacs and execute the gtags-mode function. $HOME/.emacs +------------------------------------------------------ |(autoload 'gtags-mode "gtags" "" t) % emacs | |J_:-----Mule: *scratch* (Lisp Interaction)--L16--All---- |M-x gtags-mode[RET] +------------------------------------------------------ (c) Write the autoload function to the $HOME/.emacs file and start emacs with the -f option. $HOME/.emacs +------------------------------------------------------ |(autoload 'gtags-mode "gtags" "" t) % emacs -f gtags-mode You must start emacs under the source tree described in "1.2. Preparation". 3.3. Basic usage o To go to func1, press 'ESC-t' and you can see a prompt in mini-buffer. Then input the tag name. :tag func1 # ':tag ' is a prompt. ~~~~~ o To go to a point that references func1, press 'ESC-r'. :rtag func1 # 'rtag ' is a prompt ~~~~~ o Tag name completion is available. :tag fu :tag func1 # 'nc1' is appended by emacs o If a number of functions are located, emacs goes into 'GTAGS SELECT MODE' like this: +------------------------------------------------------------- |main 347 i386/isa/ultra14f.c main() |main 128 kern/init_main.c main(framep) |main 104 netiso/clnp_debug.c main() |main 164 netiso/xebec/main.c main(argc, argv) | | | | | |J_:--%*-Mule: *scratch* (Gtags Select)--L1--All---- |[GTAGS SELECT MODE] 4 lines +------------------------------------------------------------- You can select a tag line by using any emacs command and pressing [RETURN], and you can go to the tag's point. When you want to go to the next or the previous tag, you can return to 'GTAGS SELECT MODE' with and reselect. o command is available. If current token is a definition, it is equivalent to ":rtag [RETURN]", otherwise it is equivalent to ":tag [RETURN]". (GLOBAL decides this intelligentlly, but may sometimes misunderstand.) o To go to any symbols other than function, try 'ESC-s'. Find symbol: lbolt ~~~~~ o To go to any strings other than symbols, try 'ESC-g'. Find pattern: Copyright ~~~~~~~~~ 3.4. Applied usage o You can use POSIX regular expressions. :tag ^put_ # locate tags start with 'put_'. o If your source files are on a read-only device like a CDROM, please do the following: % mkdir /var/dbpath <- directory for the tag file % cd /cdrom/src <- the root of the source tree % gtags /var/dbpath <- make tag files in /var/dbpath % setenv GTAGSROOT `pwd` % setenv GTAGSDBPATH /var/dbpath % emacs -f gtags-mode o If you want all references to functions that are not defined in the source tree to be treated as references to library functions or as system calls, do the following: % cd /usr/src/lib % gtags <- probably as a root % cd /usr/src/sys % gtags % setenv GTAGSLIBPATH /usr/src/lib:/usr/src/sys % emacs -f gtags-mode o Mouse command is avalable. If you use X version emacs, try the following (but it doesn't work well in xemacs; I don't know why). Move the mouse cursor to a function name and click the middle button. You will then go to the function's definition, or to its references, depending on the context. In 'GTAGS SELECT MODE', move the mouse cursor to a line and click the center button. To return to the previous position, click the right button. 4. Hypertext generator 4.1. Features o Htags makes hypertext from C, Yacc and Java source files. o Once the hypertext is generated, you need nothing other than a WWW browser. o You can move the hypertext to anywhere. It is independent of the source code. o You can use all of your browser's functions, such as search, history, bookmark, save, frames, windows. 4.2. Preparation At first, you must ensure that you have a lot of disk space. Hypertext needs a great amount of disk space. For example, the source code of FreeBSD kernel needs: source code(/usr/src/sys) 14.0MB GTAGS 1.5MB GRTAGS 8.0MB GSYMS 12.0MB HTML/ 55MB(!!!) ------------------------------------------------- total 77MB Please do the following: (at your source directory) % gtags <- make the tag database(GTAGS,GRTAGS,GSYMS) % htags <- make the hypertext(HTML/) Then you will find an 'HTML' subdirectory in the current directory. 4.3. Usage Please start a web browser like this: % lynx HTML/index.html You will understand the usage by looking at the examples. You can move the HTML directory to anywhere. It is independent of the source code. 4.4. Making hypertext of the kernel If you would like to make hypertext of the FreeBSD kernel source, it is convenient to use the systags script in this package. % cd /usr/src/sys % systags then % netscape HTML/index.html You can use following functions with systags: o An input form for a dynamic index. (this reqires you to set up an httpd server for the CGI program.) o Unlimited nested index. Of course, you can use the above functions without systags. Please see gtags(1) and htags(1). 4.5. Gozilla If you use Netscape Navigator, you can control the browser from the command line by using gozilla(1). % gtags % htags % global -x fork fork 60 kern/kern_fork.c fork(p, uap, retval) % gozilla +60 kern/kern_fork.c % Then you can see the hypertext specified around the fork function directly on Netscape Navigator. If the browser has not been loaded, gozilla loads it. If you use another browser like lynx, try this: % gozilla -b lynx +60 kern/kern_fork.c or % setenv BROWSER lynx % gozilla +60 kern/kern_fork.c You can send a remote command to Netscape Navigator too. Try this: % gozilla -C pagedown % gozilla -C pageup The remote commands are undocumented, but you can see the hint in the resource file (Netscape.ad). NOTES: Netscape Navigator is a registered trademark of Netscape Communications Corporation in the United States and other countries. 5. Elvis using global Elvis 2.1 has new "tagprg" and "tagprgonce" variables for running an external tag search program. You can use them with GLOBAL. 5.1. Features o The tag function of elvis+GLOBAL can locate not only function definitions but also function references. o Elvis+GLOBAL allows duplicate tag entries. o Elvis+GLOBAL can understand POSIX regular expressions as a tag name for the search. o Symbol search and pattern search are available. o Mouse events are supported. 5.2. Preparation First, do the preparation of global. (Please see "1.2. Preparation"). Second, start elvis and execute 'set tagprg="global -t $1"' like this. % elvis ~ ~ ~ ~ ~ ~ :set tagprg="global -t $1" 5.3. Basic usage o To go to func1, you can say :tag func1 It seemes the same as original elvis, but elvis execute 'global -t func1' internally and read it instead of tags file. o To go to the referenced point of func1, add -r option. :tag -r func1 Elvis executes command like 'global -t -r func1' internally. o To go to any symbols other than function, try this. :tag -s lbolt o To go to any strings other than symbols, try this. :tag -g Copyright o When using -r, -s or -g, you had better to use browse command. :browse -r fork It brings a following selection list. You can select tag and go to the point. Browse -r fork (2 matches) +----------------+----------------+------------------------------- | TAG NAME | SOURCE FILE | SOURCE LINE +----------------+----------------|------------------------------- |fork |ux/linux_misc.c | (line 565) |fork |ern/init_main.c | (line 191) +----------------+----------------+------------------------------- o To get list of functions in a file, use -f command. :browse -f main.c # locate functions in main.c. o Other tag commands are also available: - go to the definition of current token. - return to the most recent tag context. ":tag" - without argment, go to the next tag. ":pop" - return to the most recent tag context. ":stack" - display the tags stack. ":stag" - creates a new window and moves its cursor to the tag's definition point. ":sbrowse" - same with 'browse' but show in a new window. 5.4. Applied usage o You can use POSIX regular expressions. :tag ^put_ # locate functions start with 'put_'. :browse -g 'fseek(.*L_SET)' # locate fseek() using L_SET argment. o You can browse functions list of many files. :browse -f *.c # locate functions in *.c. o You can browse project files which includs specified pattern. :browse -P ^vm/ # under vm/ directory. :browse -P \.h$ # all include files. :browse -P init # files including 'init' o You can use mouse to select tag. +----------------+----------------+------------------------------- | TAG NAME | SOURCE FILE | SOURCE LINE +----------------+----------------|------------------------------- |fork |ux/linux_misc.c | (line 565) |fork |ern/init_main.c | (line 191) +----------------+----------------+------------------------------- Please select tag name with mouse cursor and double click on the left button and you go to the tag's point. In source screen, also select function name and double click on the left button and you can go to the point that the function is defined. To come back, double click on the right button. A. Other topics A.1. How to plug in a parser You can use a plugged-in parser. Copy global.conf to /etc/global.conf or $HOME/.globalrc. For example, if you would like to use ctags based on etags (included by Emacs), % cd //lib-src % make ctags # cp ctags /usr/local/bin/ctags-emacs % setenv GTAGSLABEL ctags-emacs <- see global.conf % gtags % ls G* GPATH GTAGS Or if you would like to use exuberant ctags (included by Vim editor), % cd //src/ctags % cp Makefile.unix Makefile % make # cp ctags /usr/local/bin/ctags-exuberant % setenv GTAGSLABEL ctags-exuberant <- see global.conf % gtags % ls G* GPATH GTAGS GRTAGS and GSYMS don't exist, simply because these parsers don't support the -r option and -s option like gctags(1) does. All plugged-in parsers must print tag information to standard output in the same style as 'ctags -x', ie.: [1] [2] [3] [4] ---------------------------------------------------------------- main 20 ./main.c main(argc, argv) /* xxx */ [1] tag name [2] line number the tag appeared [3] path name. It must be equal to argment path name. [4] line image Otherwise, you can make a suitable wrapper for the pug-in parser. A.2. Compact format. You can save disk space with the compact format. The compact format needs source files. o To specify the use of the compact format on the command line, add the -c option: % gtags -c o To specify the use of the compact format in the configuration file: +---------------------------- |... |default:\ | format=compact:... o If you will publish hypertext generated by htags then use the -c option of htags too: % htags -c With the -c option, htags makes gzipped hypertext. You need to set up an HTTP server so that gzipped files can be read (see 'HTML/.htaccess.skel'). Example: Standard Compact Compressed rate ------------------------------------------------------- GTAGS 1744896 bytes 720896 bytes -59% GRTAGS 10133504 bytes 1409024 bytes -86% GSYMS 11911168 bytes 9306112 bytes -22% Standard Compact Compressed rate ------------------------------------------------------- HTML/ 56618 kbytes 15219 kbytes -73% A.3. Incremental updating Modifying some source files, you need not remake whole tag files. Instead, you can use incremental updating facility (-i option). % gtags % cd kern % vi tty.c # modify tty.c. ... :wq % global -vi # -v means verbose. [Sun Dec 6 16:27:47 JST 1998] Gtags started Tag found in '/usr/src/sys'. Incremental update. Updating tags of 'kern/tty.c' ...GTAGS..GRTAGS..GSYMS.. Done. Global databases have been modified. [Sun Dec 6 16:28:30 JST 1998] Done. % global -vi # try again. [Sun Dec 6 16:28:48 JST 1998] Gtags started Tag found in '/usr/src/sys'. Incremental update. Global databases are up to date. # do nothing. [Sun Dec 6 16:28:52 JST 1998] Done. A.4. Plans for the furture Please see GLOBAL home page. Thank you for your reading. And of course, I'm also grateful to all excellent tools (vi, ctags, emacs, perl, C, db, mozilla ...) and its authors. ---------------------------------------------------------------------------- E-Mail: WWW: (You can find the latest version here.) ----------------------------------------------------------------------------