malacology
/
gde_linux
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
gde_linux/CORE/xylem/fetch.doc

321 lines
14 KiB
Plaintext
Raw Blame History

FETCH.DOC update 24 Feb 96
NAME
fetch - retrieves database entries by name or accession number
SYNOPSIS
fetch {interactive mode}
fetch [options] namefile [output file] {batch mode}
DESCRIPTION
fetch retrieves one or more entries from a database.
Interactive mode: fetch prompts the user to set search parameters,
using an interactive menu:
___________________________________________________________________
FETCH - Version 7 Feb 94
Please cite: Fristensky (1993) Nucl. Acids Res. 21:5997-6003
___________________________________________________________________
Namefile:
Outfile:
Database:
-------------------------------------------------------------------
Parameter Description Value
1) Name/Acc Name or Accession sequence to get
2) Namefile Get list of sequences from Namefile
3) WhatToGet a:annotation s:sequence b:both b
4) Database g:GenBank p:PIR v:VecBase l:LiMB g
G:GenBank dataset P:PIR dataset
5) Outfile Send all output to a single file (Outfile)
6) Files f:Send each entry to a separate file f
-------------------------------------------------------------
Type number of your choice or 0 to continue:
After all parameters have been set, type 0 to commence the search.
Messages regarding the progress of the search will be printed.
(1,2) Which entries to get?
If you want to get a single entry, option 1 lets you type in the
name of that entry, without having to create a namefile. To get
more than one entry, choose option 2, and specify the name of a
file containing sequence names or accession numbers.
namefile is a file containing one or more sequence names or
accession numbers, each on a separate line. Names and accession
numbers can even be interspersed, in upper or lowercase, and in
any order. For example, the namefile prp.nam might contain
; plant pathogenesis related proteins
; (these are sample comment lines)
; note that any line containing a semicolon is ignored
x06362
x05454
TOBPR1A1
; comments can be interspersed with names.
PUMPR13
tobpr1ar
Options 1 & 2 are mutually exclusive. Setting one will negate the
other. If option 2 is chosen, the name of the namefile will appear
at the top of the menu.
(3) WhatToGet
Use this option to specify whether to get annotation, sequence,
or both (default=both).
(4) Database
Use this option to select the database. (default=GenBank).
G and P select user-created database subsets containing GenBank
or PIR entries, respectively. It is assumed that the database
has been split into .ano, .wrp and .ind files using splitdb.
For example, if you had created a database subset called PR1.pir,
splitdb would create PR1.ano, PR1.wrp and PR1.ind. These are
the files actually read by FETCH. When prompted for the name
of the database, simply type "PR1", without a file extension.
(If you do type a file extension, it will be ignored).
(5, 6) Where to send output
By default, option 6 is set to f, and each entry will be written to
a separate file, where the name of the file is the name of the
entry, followed by a file extension. If a complete entry is
retrieved, the file extension will indicate the type of database
(GenBank: .gen; PIR: .pir, Vecbase: .vec; LiMB: .LiMB). If only
annotation or sequence are retrieved, the file extensions will be
.ano or .wrp, respectively. Using the default, the namefile above
would create the following files:
PUMPR13.gen
TOBPR1A1.gen
TOBPR1AR.gen
TOBPR1CR.gen
TOBPR1PS.gen
By choosing option 5, you can specify the name of an output file
for all entries to go to. This filename will appear at the top
of the menu. Obviously, options 5 & 6 are mutually exclusive.
Note entries retrieved are writen in alphabetical order (sorting by
ASCII values), not the order in which they appeared in namefile.
(Note for remote users only: -f will only work for a single
name/accession supplied in 1). -f IS NOT ENABLED FOR NAMEFILES
specified in 2).)
Batch mode:
Although it is transparent to the user, all fetch really does
is call getloc, saving the user the trouble of knowing which
database files to retrieve sequences from, or of having to
execute getloc multiple times to retrieve sequences from
different database files. Thus, the options are identical to those
for getloc:
-a Write annotation portions of entries only, terminated by '//'.
-s Write sequence data only, in Pearson (.wrp) format.
-f Write each entry to a separate file.
-g GenBank (default)
-e EMBL {not implemented}
-p PIR (NBRF)
-v Vecbase
-l LiMB
-G GenBank_dataset
-P PIR_dataset
If -f is not specified, outfile must be specified.
-L force execution of findkey on local host even if
$XYLEM_RHOST is set. See "REMOTE EXECUTION" below
PIR_dataset
GenBank_dataset
This can be either a file of PIR entries, a file of GenBank entries,
or a XYLEM dataset created by splitdb. A file of PIR entries must
have the file extension ".pir". A file of GenBank entries must have
the file extension ".gen". A XYLEM dataset contains PIR entries split
among three files by splitdb: annotation (.ano), sequence (.wrp)
and index (.ind). These file extensions must be used!
When specifying a split dataset, only the base name needs to be
used. For example given a XYLEM dataset consisting of the files
myset.ano, myset.wrp and myset.ind, the following two commands
are equivalent:
fetch -P myset something.nam something.pir
fetch -P myset.ano something.nam something.pir
If the original .pir file had been used, the command would have
been
fetch -P myset.pir something.nam something.pir
The ability to work directly with .gen or .pir files is quite
convenient. However, since FETCH needs to work with a split
FETCH automatically splits .pir or .gen files into .ano, .wrp
and .ind files, which are removed when finished. This requires
extra disk space and execution time, which could be significant
for large datasets.
EXAMPLES
Batch example:
fetch -f chitinase.nam
will retrieve annotation and sequence for sequences listed in
chitinase.nam from GenBank, writing each entry to a separate file
with the extension .gen.
fetch -s -v pbr.nam pbr.wrp
will retrieve sequence data only for the entries listed in pbr.nam,
from VecBase, and write all sequences to a Pearson format file
(ie. readable by fasta) with the name pbr.wrp.
fetch -G sample sample.nam new.gen
fetch -G sample.ano sample.nam new.gen
Assumes that a set of GenBank entries has been split by splitdb
into sample.ano sample.wrp and sample.ind. The entries listed in
sample.nam are written to new.gen.
FILES
Database files:
The directories for database files are specified by the environment
variables $GB (GenBank) $PIR (PIR/NBRF) $VEC(Vecbase) and $LIMB
(LiMB).
Index files are $GB/gbacc.idx for GenBank (this file is supplied
with each GenBank release), while the other databases
use .ind files generated by splitdb. Split database files MUST
have the following file extensions: .ano {annotation}, .wrp
{sequence} and .ind {index}. Thus, when creating database files
for pir1.dat with splitdb, the output files should be pir1.ano,
pir1.wrp and pir1.ind.
Temporary files:
NAMEFILE.fetch
PRELIMINARY.fetch
TMP.fetch
FOUND.fetch
FETCHDIR {temporary directory}
REMOTE EXECUTION
Where the databases can not be stored locally, FETCH can call
FETCH on another system and retrieve the results. To run
FETCH remotely, your .cshrc file should contain the following
lines:
setenv XYLEM_RHOST remotehostname
setenv XYLEM_USERID remoteuserid
where remotehostname is the name of the host on which the
databases reside (in XYLEM split format) and remoteuserid
is your userid on the remote system. When run remotely,
your local copy of FETCH will generate the following
commands:
rcp filename $XYLEM_USERID@$XYLEM_HOST:filename
rsh $XYLEM_RHOST -l $XYLEM_USERID fetch ...
rcp $XYLEM_USERID@$XYLEM_HOST:outputfilename outputfilename
rsh $XYLEM_RHOST -l $XYLEM_USERID $RM temporary_files
Because FETCH uses rsh and rcp, your home directory on both
the local and remote systems must have a world-readable
file called .rhosts, containing the names of trusted remote
hosts and your userid on each host. Before trying to get
FETCH to work remotely, make sure that you can rcp and
rsh to the remote host.
Obviously, remote execution of FETCH implies that FETCH
must already be installed on the remote host. When FETCH
runs another copy of FETCH remotely, it uses the -L option
(findkey -L) to insure that the remote FETCH job executes,
rather than calling yet another FETCH on another host.
---------- Remote execution on more than 1 host -----------
If more than 1 remote host is available for running FINDKEY
(say, in a clustered environment where many servers mount
a common filesystem) the choice of a host can be determined
by the csh script choosehost, such that execution of
choosehost returns the name of a remote server. To use this
approach, the following script, called 'choosehost' should
be in your bin directory:
#!/bin/csh
# choosehost - choose a host to use for a remote job.
# This script rotates among servers listed in .rexhosts,
# by choosing the host at the top of the list and moving
# it to the bottom.
#Rotate the list, putting the current host to the bottom.
set HOST = `head -1 $home/.rexhosts`
set JOBID = $$
tail +2 $home/.rexhosts > /tmp/.rexhosts.$JOBID
echo $HOST >> /tmp/.rexhosts.$JOBID
/usr/bin/mv /tmp/.rexhosts.$JOBID $home/.rexhosts
# Write out the current host name
echo $HOST
You must also have a file in your home directory called
.rexhosts, listing remote hosts, such as
graucho.cc.umanitoba.ca
harpo.cc.umanitoba.ca
chico.cc.umanitoba.ca
zeppo.cc.umanitoba.ca
Each time choosehost is called, choosehost will rotate the
names in the file. For example, starting with the .rexhosts
as shown, it will move graucho.cc.umanitoba.ca to the bottom
of the file, and write the line 'graucho.cc.umanitoba.ca'
to the standard output. The next time choosehosts is
run, it would write 'harpo.cc.umanitoba.ca', and so on.
Depending on your local configuration, you may wish to
rewrite choosehosts. All that is really necessary is that
echo `choosehost` should return the name of a valid host.
Once you have installed choosehost and tested it, you can
get FINDKEY to use choosehost simply by setting
setenv XYLEM_RHOST choosehost
in your .cshrc file.
--------------- Remote filesystems -----------------------
Finally, an alternative to remote execution is to remotely mount
the file system containing the databases across the network.
This has the advantage of simplicity, and means that the
databases are available for ALL programs on your local
workstation. However, it may still be advantageous to run
FETCH remotely, since that will shift much of the computational
load to another host.
BUGS
When retrieving entries directly from GenBank, FETCH uses the
Accession Number index file gbacc.idx. In this case, FETCH
can retrieve all entries containing a given accession number.
This capability makes it possible to retrieve an entry using a
secondary accession number. However if more than one entry
share a secondary accession number, all of those entries will
be retrieved. While this behavior might be a bit of an
annoyance at times, it can also be useful because it alerts
the user to the presence of other, related entries that might
be of interest.
SEE ALSO
getloc features
AUTHOR
Dr. Brian Fristensky
Dept. of Plant Science
University of Manitoba
Winnipeg, MB Canada R3T 2N2
Phone: 204-474-6085
FAX: 204-261-5732
frist@cc.umanitoba.ca
REFERENCE
Fristensky, B. (1993) Feature expressions: creating and manipulating
sequence datasets. Nucleic Acids Research 21:5997-6003.
Reference in New Issue View Git Blame Copy Permalink