The following is the file "README.txt" that accompanies the version 14.0 of Consed. It has been edited to reflect minor changes in installation and contact information that apply to the CodonCode distribution.
For support, please send email to support@codoncode.com.
CONTENTS:
WHAT IS NEW IN CONSED 14.0
WHAT IS AUTOFINISH
INSTALLING CONSED<---------------------
QUICK TOUR OF CONSED
USING AUTOFINISH
USING AUTOMATED ADD NEW READS
USING AUTOPCRAMPLIFY
USING AUTOEDIT
ADVANCED PHRAP/CONSED USAGE
NOTE TO LINUX USERS
NOTE TO ITANIUM LINUX USERS
NOTE TO SGI USERS
NOTE TO SOLARIS USERS
NOTE TO MACOSX USERS
CONSED CUSTOMIZATION
CONSED FOR LARGE ASSEMBLIES
FOR PROGRAMMERS AND FELLOW TRAVELLERS ONLY
MONITORS AND MICE FOR CONSED
HUGE ASSEMBLIES
AUTOFINISH AND PRIMER-PICKING PARAMETERS
ACE FILE FORMAT
WHAT THE COLORS MEAN
----------------------------------------------------------------------------
WHAT IS NEW IN CONSED 14.0
Itanium (64-bit linux) is now available.
Assembly View
Sequence matches are shown, and can be used to make joins.
Can show restriction digest cut sites. More probably coming on
this.
Tags Can be Shown. You can customize which tag types are shown.
Clicking on a tag shows more information about it and displays
that location in the Aligned Reads Window.
Can more easily specify which contigs to show.
Show Traces for All Reads
If you are examining traces hundreds of reads deep, then Consed
will now show you the reads in batches of 100 at a time. It will
optionally not show reads that have no base at the cursor position,
have no trace near the cursor position, or that have a user-specified
tag at the cursor position. This helps focus on the reads you need to
look at.
There were some bugs with this feature--I believe they are all
fixed.
AutoPCRAmplify
Can now be used in cases in which you only have a fasta file of
the region to amplify, such as when you download the sequence
from Genbank or UCSC.
Now more likely to find pcr primer pairs.
Can optionally use a fixed primer for one end, such as when using
a transplice leader sequence.
User feedback helped me improve the criteria for rejecting PCR
primer pairs.
Improvement for Huge Assemblies
You can concatentate all of the phd files into a single file and
delete the many phd files.
You can type in the contig name rather than selecting it from a
list--good if the list is hundreds of contigs long.
You can filter which ace files you want displayed--useful if there
are a huge number of ace files.
Custom Data Within Tags
Consed has always allowed user-defined tag types. Now Consed
allows you to defined fields within user-defined tags. For
example, you might define an exon tag with a field which is a
pointer to a gene tag.
Miniassemblies
Users have found they sometimes only recognize they made a mistake
after starting a miniassembly. But they couldn't stop it. Now
there is a "Cancel Miniassembly" button.
Autofinish
To prevent Autofinish from trying to do pcr with a particular
contig-end, there is now a doNotDoPCR tag that Autofinish will
obey.
Bugs fixed
Macosx Autofinish crash
Linux AutoPCRAmplify crash
One caused a corrupted ace file.
----------------------------------------------------------------------------
WHAT IS AUTOFINISH?
Autofinish automatically chooses reads for finishing. Autofinish
sometimes is able to completely finish a project with no human
decisions. In other cases Autofinish mostly finishes a project, and a
human just needs to do the final difficult problems since all the
routine problems have already been completed by Autofinish. Thus a
human finisher is able to complete far more projects in the same
length of time.
Autofinish is flexible to the finishing strategy of your lab. It can
be used to finish with just universal primer reads, just oligo walks,
just minilibraries, or a combination of these. It can be used to
finish either genomic or cDNA.
Autofinish will do the following:
-close gaps
-improve sequence quality
-determine the relative orientation of contigs
-ensure that, at each consensus base, at least 2 reads from different
templates are aligned
(You can configure Autofinish to do any combination of these tasks.)
Autofinish will suggest the following types of experiments:
-universal primer reads (forward or reverse)
-custom primer reads with subclone templates
-custom primer reads with whole clone templates
-minilibraries (transposon or shatter) from subclone templates
-PCR
(You can configure Autofinish to suggestion any combination of these
experiments.)
------------------------------------------------------------------------
INSTALLING CONSED
(If you are running Solaris, see NOTE TO SOLARIS USERS. If you are
running SGI, see NOTE TO SGI USERS. If you are running MACOSX, see
NOTE TO MACOSX USERS. If you are running regulare Linux, see NOTE TO
LINUX USERS. If you are running Linux on an Itanium (a big
64 bit box), see NOTE TO ITANIUM LINUX USERS.
You MUST have the following phred, phrap, phd2fasta, and crossmatch in
order to use this version of Consed:
000925.c or later for phred
0.990319 or later for phrap and crossmatch
0.990622.e or later for phd2fasta (supplied with this version of consed)
any version of addReads2Consed.perl (supplied with this version
of consed)
030326 of phredPhrap (supplied with this version of consed)
(Note: if you have an older version of phredPhrap, some of the
more recent Consed features, such as miniassemblies, will not
work. Note to existing polyphred users: phredPhrap now calls
polyphred with different parameters which will give cause it to apply
different tags than it used to, but these different tags will
give it behavior consistent with that described below in
CONSED-POLYPHRED INTERACTION. For more information, see
http://droog.gs.washington.edu/PolyPhred.html )
011130 or later for transferConsensusTags.perl (supplied with this
version of consed)
any verion of tagRepeats.perl (supplied with this version of consed)
any version of determineReadTypes.perl (or your own custom
modified version)
Contact sales@codoncode.com if you need any of the other programs.
Summary of files your must edit (instructions are below):
addReads2Consed.perl
determineReadTypes.perl
phredPhrap
primerCloneScreen.seq
primerSubcloneScreen.seq
repeats.fasta
vector.seq
In order to run the gauntlet of phred/phd2fasta/crossmatch/phrap,
there is a perl script phredPhrap supplied with Consed (above). YOU
MUST USE THIS PERL SCRIPT. If you try to run each of these programs
directly, you are on your own and you will probably spend a lot of
time needlessly.
If you want to try to take a short-cut through the steps below, and
you are using Redhat Linux, a fellow Consed user has written a script
and instructions to allow you to take this short-cut. I have not
studied nor tested his script, but he usually does good work. If you
want to try it, see:
The instructions are at:
http://ludwig.ucdavis.edu/consed-install.txt
The script is at:
http://ludwig.ucdavis.edu/install-PHUI.pl
Alternatively, follow the instructions below:
1) After downloading the distribution with netscape (see www.phrap.org
and click on 'Consed'), copy the distribution to a unix computer (if
it is not already on one). Unpack the files by typing the appropriate
line below (which one depends on what you named the file downloaded by
netscape):
gunzip Linux_Consed.tar.gz; tar -xvf Linux_Consed.tar
(the name of the files will differ on other operating systems)
For additional instructions, please visit:
http://www.codoncode.com/support/install_phredphrap.htm.
Note: You must run tar on a UNIX computer--not on an Windows computer,
due to a difference in the handling of breaks between lines.
2) I suggest you put Consed, phred, crossmatch, phrap, the perl
scripts, and other executables into /usr/local/genome/bin. So create
/usr/local/genome/bin and /usr/local/genome/lib
If you can't actually use /usr/local/genome, then you could make
/usr/local/genome be a link to the real location--that will work just
as well.
If you want to have another location xxx, then put:
setenv CONSED_HOME xxx
into the .cshrc (or equivalent if you are using bash or shell other
than csh or tcsh) of all Consed users
and create $CONSED_HOME/bin and $CONSED_HOME/lib and put all of these
programs into $CONSED_HOME/bin
3) Make sure that /usr/local/genome/bin (or $CONSED_HOME/bin) is in
every Consed users' PATH.
4) Put the Consed executable in /usr/local/genome/bin (or $CONSED_HOME/bin)
5) Check this by logging on as a user and typing:
consed -V
You should see 'Version 14.0'. If you see something else, you have
some debugging to do.
6) TESTING CONSED
Follow the first few steps of USING CONSED GRAPHICALLY of the
Quick Tour (below). If you have problems, it may be due to your X
emulator. See 'MONITORS AND MICE FOR CONSED' below.
If you get some error such as:
Error: Can't open display:
then the problem may have nothing to do with Consed, but rather with
X. To test this, run some other X application (such as xclock, xterm,
xeyes, or xcalc) and see if you get the same error.
7) Build phd2fasta:
Go to the misc/phd2fasta directory and type 'make'
Move the phd2fasta executable to /usr/local/genome/bin (or $CONSED_HOME/bin)
8) Build mktrace:
Go to the misc/mktrace directory and type 'make'
Move the mktrace executable to /usr/local/genome/bin (or $CONSED_HOME/bin)
9) Move all perl scripts from the scripts directory to
/usr/local/genome/bin (or $CONSED_HOME/bin)
Make sure all are executable (chmod a+x *)
DELETE ANY PREVIOUS VERSIONS OF THESE SCRIPTS OR YOU WILL BE SORRY!
(Bugs have been fixed.)
10) Get perl 5. (If you have Linux, you already have it so you can
skip this step.) You can check where to get perl via the perl web
site:
http://www.perl.com/perl/info/software.html
(If you don't know about perl, try it--it will save you a
huge amount of time over developing the same utilities in C, awk, or
csh or sh.)
Regardless where you put perl, put a link to it in /usr/bin so that
all of the scripts with
#!/usr/bin/perl
will work and you won't have to edit all of them everytime a new
Consed release comes out.
11) Create a subdirectory /usr/local/genome/lib/screenLibs. (If you
are using a location other than /usr/local/genome for the root of all
Phred/Phrap/Consed programs, create $CONSED_HOME/lib/screenLibs). From
the misc subdirectory, copy primerCloneScreen.seq and
primerSubcloneScreen.seq to the directory
/usr/local/genome/lib/screenLibs (or $CONSED_HOME/lib/screenLibs).
primerCloneScreen.seq is used to screen candidate primers when you use
Consed's function "Pick Primer from Clone Template" (on the Aligned
Reads Window).
primerSubcloneScreen.seq is used to screen candidate primers when you
use Consed's function "Pick Primer from Subclone Template" (on the
Aligned Reads Window).
Take a look at these files. They are dummy files indicating the fasta
format of the sequences that should be put in them. You should put
into primerCloneScreen.seq the vector sequence of the cloning vectors
you are using (BAC or cosmid) and into primerSubcloneScreen.seq the
sequencing vectors you are using (plasmid, M13, etc). Don't be too
generous in putting lots of vectors into the files! The larger they
are, the slower primer picking will be. Our files are only this big:
-rw-r--r-- 1 root root 29938 Nov 7 1997 primerCloneScreen.seq
-rw-r--r-- 1 root root 7381 Aug 13 1997 primerSubcloneScreen.seq
and primer picking is quite fast enough.
Now that you have set this up, you should try the PRIMER PICKING
sections in the Quick Tour (above) to make sure this works.
12) You should create a file
/usr/local/genome/lib/screenLibs/vector.seq
(or $CONSED_HOME/lib/screenLibs/vector.seq if you are not using
/usr/local/genome for the root of the Phred/Phrap/Consed files.)
This contains all the vector sequences (in FASTA format) that you want
to mask out before phrapping. In general, it is the combination of
primerCloneScreen.seq and primerSubcloneScreen.seq
13) You should create a file
/usr/local/genome/lib/screenLibs/repeats.fasta
(or $CONSED_HOME/lib/screenLibs/repeats.fasta if you are not using
/usr/local/genome for the root of the Phred/Phrap/Consed files.)
In this file, put any sequences (in FASTA format) that you want to
have automatically tagged. These typically are ALU sequences. If you
don't want to tag anything, then comment out (put '#' as the first
character of the line) the following lines in phredPhrap:
Change:
!system( "$tagRepeats $szAceFileToBeProduced" )
|| die "some problem running $tagRepeats";
to:
#!system( "$tagRepeats $szAceFileToBeProduced" )
# || die "some problem running $tagRepeats";
14) You should create a file
/usr/local/genome/lib/screenLibs/singleVectorForRestrictionDigest.fasta
containing the cloning vector sequence. This is used for doing in-silico
restriction digests. Thus this cloning vector must start at precisely
the site where you cut the vector to ligate the insert. It is not
sufficient to just download the vector sequence from Genbank. You
might need to have it start in a different place.
15) SETTING UP TEST DIRECTORIES
Copy the test directories and their contents to some location where
the users have write access. Copy--do not move them--because the
users will occasionally want a fresh copy.
cp -r standard new_location
cp -r autofinish new_location
cp -r assembly_view new_location
cp -r polyphred new_location
cd new_location
chmod -R a+w *
16) MODIFYING determineReadTypes.perl
Read the comments in determineReadTypes.perl
Phrap, Consed's primer picking, and Consed/Autofinish all need the
following information for each read:
is it a univeral primer forward, a universal primer reverse,
or a walking read?
what is its template name?
If you are using different libraries that have different insert sizes,
then Consed/Autofinish also need the library name for each read.
Generally this information can be determined from the read name, using
*your* naming convention. Modify the perl script
determineReadTypes.perl to put this information at the end of the phd
file using WR info items.
If you don't want to do much perl programming and all your libraries
have the same insert size, you have the option of using the St Louis
naming convention. In this case, the only perl programming you
need do is to comment out (put a "#" in front) the line in
determineReadTypes.perl that starts with:
die "You must edit determineReadTypes.perl
You must also uncomment (remove the "#"s in column 1) the lines in
the phredPhrap script that say roughly:
#print "\n\n--------------------------------------------------------\n";
#print "Now running determineReadTypes.perl...\n";
#print "--------------------------------------------------------\n\n\n";
#!system( "$determineReadTypes" ) || die "some problem running determineReadTypes.perl $!\n";
But what is the St Louis naming convention? Most of it (but not all)
is explaned in the phrap documentation. In addition, you must never
use an underscore in the name if the read is a universal primer
forward or universal primer reverse read. If the read is a walk, then
you must have an underscore (_) follow the template name and then have
a number (the oligo number).
Examples of reads in the St Louis naming convention:
read eeq03a01.g1.phd.1 is univ rev template: eeq03a01 library: eeq03
read eeq03a02.b1.phd.1 is univ fwd template: eeq03a02 library: eeq03
read eeq03a02.g1.phd.1 is univ rev template: eeq03a02 library: eeq03
read eeq03a03.b1.phd.1 is univ fwd template: eeq03a03 library: eeq03
read eej45h07_2.i1.phd.1 is walk template: eej45h07 library: eej45
read eej46c12_1.i1.phd.1 is walk template: eej46c12 library: eej46
Once you have correctly customized determineReadTypes.perl, then
uncomment the line in phredPhrap which calls determineReadTypes.perl
It is fine to assume the St Louis for the purpose of the sample
dataset directories that come with Consed ("standard",
"assembly_view", "autofinish", and "polyphred").
17) TROUBLESHOOTING YOUR CHANGES TO determineReadTypes.perl
Consed allows you to check that you have correctly modified
determineReadTypes.perl: On the Consed Main Window, point to 'Info',
hold down the left mouse button, and release on 'Show Info for Each
Read'. Study all the information and check that the information
presented is correct. If, for example, Consed thinks that there are
templates that have 9 or more reads, it is likely that you have not
correctly customized determineReadTypes.perl
You will see a section that looks like this:
template djs736a2_fp04q286 with 2 reads
djs736a2_fp04q286.x2 term universal forward (from phd file)
djs736a2_fp04q286.y2 term universal reverse (from phd file)
You want to see the "from phd file" part. If, instead of "from phd
file", it says "inferred from name", that means that
determineReadTypes.perl couldn't figure out what kind of read it was.
If you think you have made a mistake in customizing
determineReadTypes.perl, it is best to delete the PHD files (and
phd.ball if you are using that) and run phredPhrap again since the
otherwise incorrect WR items will be left in the PHD files.
There is more specific documentation within the script
determineReadTypes.perl for more information about how to customize
it.
CUSTOMIZING determineReadTypes.perl: SPECIAL CASES
18) FAKE READS
By "fake reads" I mean reads such as those created from a Genbank
reference sequence or a consensus from some other assembly... or others
for which there is no chromatogram (and there never was any
chromatogram). If you don't use any such reads, you can skip this
step.
In the past, any read that ended with a .a2 or .c3 (where 2 and 3
could be any numbers), was considered a fake read. Now you can make
Autofinish not assume this using the .consedrc parameter (see CONSED
CUSTOMIZATION):
consed.fakeReadsSpecifiedByFilenameExtension: false
Instead, you must have determineReadTypes.perl put "fake" into the
"type:" field of a "template" WR item. See determineReadTypes.perl for
more information.
After installing Consed, you should run all the following tests to
make sure you have installed everything correctly:
19) APPENDING EXPID TO THE PHD FILES
If you are using Autofinish, and would like Autofinish to tell you how
well your reads are succeeding, then the phd files must be appended
with the experiment id's. In the 3 Autofinish summary files
(*.univReverse, *.univForwards, and *.customPrimers), you will see
information like this:
univ rev,,,->,-329,-249,71,Contig1,3,djs228_1034
or this:
tgaagaaatggctgactcc,56,1,->,3258,3338,3658,Contig1,4,djs228_2813,5,djs228_168,6,djs228_1248
The '3' just before the djs228_1034 is an experiment id. There is
also an expid '4' just before djs228_2813, an expid '5' before
djs228_168, and an expid '6' just before djs228_1248.
Autofinish doesn't know what you will end up calling these reads it is
telling you to make. Autofinish only knows those reads by the numbers
3, 4, 5, and 6. So when you make the reads, Autofinish needs to be
informed that this is 'experiment 3' or whatever. You do this by
appending in the phd file the following structure:
WR{
expid addExpid 990811:140818
5
}
where WR stands for 'whole read item',
expid for 'expid'
addExpid is the name of the program that you will write that
will append this information
990811:140818 is the date and time in format YYMMDD:HHMISS
5 is the expid
This program must be run *after* phred runs to create the phd files.
Thus your program must have some method of determining what the expid
of each read is. What the University of Washington Genome Center does
is to have the finishers put the expid as part of the filename. This
makes it easy for a program to look at the phd file and figure out
what the expid is and then write the WR item into that phd file.
Alternatively, you could keep a database and, after the phd file is
created, look into the database to see what the expid is.
When you have successfully added expid's to the phd files, the next
time you run Autofinish on this project, it will have in the
'EVALUATE' section of the Autofinish output file, lots of interesting
information about how well the reads succeeded.
TESTING
The following tests must be done to insure that the installation was
done correctly.
20) TESTING RESTRICTION DIGEST
Try the restriction digest feature (RESTRICTION DIGEST above) to make
sure this works.
21) TESTING ADDING NEW READS
It will make your life easier if phred, phrap, and crossmatch are
all where Consed expects them: in /usr/local/genome/bin
22) Decide where to put phred's parameter file phredpar.dat and edit
both addReads2Consed.perl and phredPhrap to reflect this location. I
generally prefer to put it in /usr/local/genome/lib to keep all of the
Phred/Phrap/Consed files in one place. Alternatively, you could put
it in /usr/local/etc/PhredPar/phredpar.dat which is the historical
location of this file.
23) Next you should test the ADDING NEW READS step in the Quick Tour
(above). This step requires that everything be set up correctly and
in the correct location. Hopefully the error messages are clear
enough to help you if you have set up anything incorrectly.
24) TESTING RUNNING CROSSMATCH FROM ASSEMBLY VIEW
See RUNNING CROSSMATCH FOR SEQUENCE MATCHES (above) and make sure
that step works.
25) TEST RUNNING PHREDPHRAP
See the section RUNNING PHRED and PHRAP in the Quick Tour (above)
26) TESTING MINIASSEMBLIES
See PULLING OUT READS AND RE-ASSEMBLYING THEM (MINIASSEMBLIES) and
MINIASSEMBLIES (above) and make sure those steps work.
The newer version of phredPhrap is required for this. If you have
invested a lot of work customizing some old version of phredPhrap, and
don't want to upgrade, you do have the option of keeping your
customized version of phredPhrap for regular assemblies, and using the
new version of phredPhrap for miniassemblies. To do this, you must
specify the alternate name/location of phredPhrap by the .consedrc
parameter:
consed.fullPathnameOfMiniassemblyScript: /usr/local/genome/bin/phredPhrap
(See CONSED CUSTOMIZATION below.)
USING YOUR OWN DATA
27) Create the following directory structure, which can be anywhere
on any disk:
Directory structure:
top level directory (generally named after the BAC or cosmid)
subdirectory 'chromat_dir'--chromatograms go in here
subdirectory 'phd_dir'--phd files will automatically be put here
subdirectory 'edit_dir'--ace files will automatically be put here
Put the chromatogram files (e.g., .ab1 or .scf files) into the
chromat_dir directory. Keep phd_dir and edit_dir empty.
If you already have your chromatograms somewhere else, you can make
chromat_dir be a link to wherever you have them.
The various phrap and crossmatch files will be put into edit_dir by
the phredPhrap script.
28) cd to the edit_dir directory, and type:
phredPhrap
If you are successful, the script will tell you so. (You can also
look in phd_dir and you will see phd files for each of the
chromatograms you added in chromat_dir. If the phd files are missing,
then phred was unable to call bases from the chromatograms in
chromat_dir and you will need to figure out why not). Make sure you
are in edit_dir and bring up Consed on the ace file:
29) Type:
consed
You should see a file with the extension .ace.1
Double click on it.
You should see a list of contigs.
Double click on the one you want to see.
Follow the first few steps of the Quick Tour under USING CONSED
GRAPHICALLY (above). You should at least go as far as viewing traces.
----------------------------------------------------------------------------
QUICK TOUR OF CONSED
Release 14.0
Consed is a program for viewing and editing assemblies assembled with
the phrap assembly program.
If you are already an advanced Consed user, you should read through
this and do any of the exercises on features that you are unfamiliar
with. I frequently run across people who are doing something in
Consed a hard way month after month, and request a new feature to make
things easier, when that new feature is already in Consed.
If you have never used Consed before, to follow this Quick Tour will
take you less than 6 hours. However, it will save you approximately 2
days in agony. If you have 2 extra days to spare, and prefer to waste
them in agony, then do not do this Quick Tour and instead immediately
skip down to 'INSTALLING CONSED' above.
If you do the Quick Tour, start your system administrator installing
consed (see INSTALLING CONSED (above)) because you will
need to have completed that for some of the more advanced sections of
the Quick Tour.
When you do the quick tour, I encourage you to be free about changing
the data set. If you really mess things up (such as changing all a
read's bases to N's), no problem--just delete the data set and start
again with a fresh copy.
USING CONSED GRAPHICALLY
30) Type the following:
cd standard/edit_dir
31) start Consed by typing the appropriate command below:
../../consed_solaris
../../consed_alpha
../../consed_hp
../../consed_sgi
../../consed_linux
../../consed_mac
../../consed_linux_itanium
../../consed_solaris_intel
../../consed_ibm
(Don't worry about a message like:
Warning: Cannot convert string "helvetica" to type FontStruct )
Two windows will appear. One of these will have the list of .ace
files and say 'select assembly file to open' and
'standard.fasta.screen.ace.1'. Double click on
"standard.fasta.screen.ace.1". The first window goes away.
You will now see a list of one contig and a list of reads. This is the
'Consed Main Window'.
Double click on 'Contig1'.
The 'Aligned Reads Window' will appear.
32) SCROLLING
Try scrolling back and forth. Try scrolling by dragging the thumb of
the scrollbar. Also try scrolling by clicking on the 4 << < > >>
buttons for scrolling by small amounts. For scrolling by tiny
amounts, click on the arrows at either end of the scrollbar. For
scrolling by huge amounts, use the middle mouse button and just click
on some location on the scrollbar. For scrolling to the beginning or
end of the contig, use the <<< or >>> buttons.
(Question: why can't you just move the scrollbar to the extreme right
in order to go to the end of the contig? Answer: in typical
assemblies, there are reads that protrude beyond the beginning of the
contig and reads that protrude beyond the end of the contig. Moving
the scrollbar to the extreme right will scroll the contig to the
end of the rightmost read--typically far to the right of the
end of the contig. Thus you should get in the habit of using
the <<< and >>> buttons.)
GOTO POSITION
33) In the Aligned Reads Window, click in the 'Pos:' box in the upper
right-hand corner. Type in a number, such as 540, and push the
'Return' or 'Enter' key. The Aligned Reads Window will scroll to
position 540. We find this feature is particularly useful when one
person wants another person to look at something in the sequence.
34) COLORS
Notice the colors. Scroll to position 937 and notice the read 'a'.
The red bases are the ones that disagree with the consensus.
Notice the different shades of grey background (around the bases).
They have the following meanings, but first, you need to understand
the meaning of the quality values:
A quality value of 10 means 1 error in ten to the 1.0 power
A quality value of 20 means 1 error in ten to the 2.0 power
A quality value of 30 means 1 error in ten to the 3.0 power
A quality value of 40 means 1 error in ten to the 4.0 power
and for quality values in between:
A quality value of 25 means 1 error in ten to the 2.5 power
Get the idea?
(These have actually been empirically verified--if you are interested
in the gory details, read the phred papers:
Ewing B, Hillier L, Wendl M, Green P: Basecalling of automated
sequencer traces using phred. I. Accuracy assessment. Genome Research
8, 175-185 (1998).
Ewing B, Green P: Basecalling of automated sequencer traces using
phred. II. Error probabilities. Genome Research 8, 186-194 (1998).
In that same copy of the journal is a paper about Consed, as well.)
Also notice the upper and lowercase. This is just a cruder indication
of the quality of the bases.
35) To see the quality value of a particular base, point at it and
click with the left mouse button. You will see the quality displayed
in the Info Box at the bottom of the Aligned Reads Window.
These quality values are shown in grey scales:
Quality 0 through 4 is given by dark grey
Quality 5 through 9 is given by a shade lighter
Quality 10 through 14 is given by a shade still lighter
.
.
.
Quality of 40 through 97 is given by white (the brightest shade)
A quality value of 99 is reserved for bases that have been edited and
the user is absolutely sure of the base ('high quality edited').
A quality value of 98 is reserved for bases that have been edited and
the user is not sure of the base ('low quality edit').
The ends of the reads shows bases that are grey and have a black
background. These are the low quality ends of the reads or the
unaligned ends of reads, as determined by phrap.
36) Click on a base on a read. Then hold down the control key and
type 'a'. You will move to the beginning of the read. Hold down the
control key and type 'e'. You will move to the end of the read.
(Emacs users will recognize these commands.)
37) HIGHLIGHTING READ NAMES
In the Aligned Reads Window, click on a read name with the left mouse
button. The name will turn magenta. Click again and it will turn
yellow again. Try turning it magenta and then scrolling. This
feature is helpful in keeping track of a particular read as you
scroll.
If you have an emacs window open (or any editor window), you can paste
the read name in by just clicking with the middle mouse button.
When you clicked on the read name in the Aligned Reads Window with the
left mouse button, the read name was loaded into the paste buffer.
38) DIMMING ENDS OF READS
Scroll so that location 490 is about in the middle of the aligned
reads window. Push the left mouse button down on the menu item 'Dim'.
There will be a list of choices that will appear. Drag the cursor
down to 'Dim Nothing' and release. Now look what happened to the
color of the bases. The ends of the reads that used to be with a
black background now appear red with a grey background. You are
seeing the clipped-off bases with all the same information as any
other base. Since there is a huge amount of red (discrepant) bases,
the screen becomes distracting and busy. Thus by default the low
quality clipped-off bases are made with a black background and a grey
foreground so they don't distract you.
Notice there is a distinction here between 'low quality ends of
reads' and 'unaligned ends of reads'. Unaligned ends of reads can be
low quality as well, or they can be high quality, as in the case of
chimeric reads.
Point with the mouse to a read name and hold down the right mouse
button. You will notice there is a line that says "high quality from
nnn to nnn; aligned from nnn to nnn; chem: prim". This is giving the
same information in number form. Highlight the read name first (see
HIGHLIGHTING READ NAMES above) so you don't lose the read as you
scroll. Then check that the numbers agree with the dimming.
You can play with the dimming options a bit. Then return it to 'Dim
Low Quality' for the rest of this tour.
TRACES AND EDITING
39) Point with the mouse at a base of one of the reads and click with the
middle mouse button. (If you have a 2 button mouse, see MONITORS AND
MICE FOR CONSED below.) The Trace Window showing the traces for that
stretch of read should popup.
There are 2 rows of numbers:
'con' are the consensus positions
'rd' are the read positions
There are 3 rows of bases in the trace window:
'con' is the consensus
'edt' is where you can edit the base calls of the read
'phd' is the original phred base calls
Notice that a red rectangle blinks (the 'cursor') in the corresponding
positions of the Aligned Reads Window and the Trace Window.
40) Try editing in the Trace Window. You can click the left mouse
button on a base in the 'edt' line to set the cursor (a blinking red
rectangle). You can directly overstrike a base by typing a letter.
Try this. Try undoing it (by clicking on 'undo' ). If you want to
undo more than one edit, you will have to go back to the main Consed
window and click on the button labeled 'Undo Edit...'--you will learn
that later. You can overstrike with the following characters: acgt
(bases), * (a pad, in effect deleting the base), and mrwsykvhdb (IUB
ambiguity codes).
You can move left and right with the arrow keys.
We believe that the user should change a base call only while
examining the traces. That is why editing is done here--not in the
Aligned Reads Window.
41) You can insert a column of pads by pushing the space bar. Try
this. (You may need to click on a base on the 'edt' line first.)
(For those of you new to editing assemblies, a 'pad', which in Consed
and phrap is represented by the '*' character, is used to align
two or more sequences such as these:
gttgacagtaatcta
gttgacataatcta
in which one sequence has an inserted or deleted base with respect to
the other. By inserting the pad character, it is possible to get a
good alignment:
gttgacagtaatcta
gttgaca*taatcta
This is the purpose of pad character--it is just a placeholder.)
You can then overstrike a pad with a base. In this way you
can insert a base, and still preserve the alignment.
42) Try highlighting a stretch of a read on the edt line by holding
down the middle mouse button and dragging the cursor over some bases.
They will turn yellow as you drag. Then release the mouse button. A
window will pop up giving you some choices of what to do with those
(yellow) bases.:
Make High Quality--makes the highlighted bases edited high quality
(99). This tells phrap (when it reassembles) that you are
sure of the sequence here.
Change Consensus--make the highlighted bases edited high quality and
change the consensus to agree with that stretch of the read.
This is a directive to phrap (upon reassembly) to use that
stretch of that read to be the consensus.
Make low quality--makes the highlighted bases edited low quality.
This tells phrap (when it reassembles) that you are not sure
of the bases here and phrap can go ahead and make a join even
if the bases in this region don't match perfectly.
Make Low Quality to Left End--same as above, but all the way to
the left end of the read.
Make Low Quality to Right End--same as above, but all the way to
the right end of the read.
Change to n's--Change the highlighted bases to n's which means
they are unknown bases. This tells phrap (when it
reassembles) to not make any join based on these bases. It is
useful when you believe the bases may be in the chimeric
portion of a read.
Change to n's to left--same as above but to left end.
Change to n's to right--same as above but to right end.
Change to x's to left--Change the highlighted bases to x's which
means they are vector. This tells phrap to ignore these bases
for the purpose of determining overlap.
Change to x's to right--same as above but to right end.
Add Tag--allows user to add any tag to a stretch of read bases.
Dismiss--you decided you don't really want to do anything with
this stretch of bases.
This popup is made so that nothing else works until you choose
something. Try each of these choices, except for tags, which you'll
try below.
'Change Consensus' has an additional function--if a read extends out
on the right beyond the end of the consensus, you can extend the
consensus by using this function. You might want to do this, for
example, if crossmatch did not correctly find the cloning site and
thus clipped too much. You can add these bases to the consensus
by using 'Change Consensus'. Typically, the quality of these bases in
the read and in the consensus is 99. That is so that next time phrap
runs, it will correctly extend the consensus.
However, if you aren't going to reassemble, you might want to just
leave the quality values the way phred originally called them. You
can do this by using a Consed parameter
(consed.extendConsensusWithHighQuality), which you will learn more
about later (see CONSED CUSTOMIZATION).
43) To delete a base, overstrike it with a '*' character. (Phrap
ignores '*', so this is the same as deleting the character.) If you
overstrike all bases in a column with * characters so the entire
column consists of *'s (including the consensus base), there is no way
to remove the column. This is OK since when you export the consensus
(try the exercise on EXPORTING THE CONSENSUS), the *'s are not
exported. While you are editing in Consed, we believe there should be
a visual indication that a base was deleted.
SAVING THE ASSEMBLY
44) To save the assembly, pull down the 'File' menu on the Aligned
Reads Window, and release on 'Save assembly'. A box will pop up with
a suggested name. I suggest you always use the one it suggests. The
idea is that the ace files:
(project).fasta.screen.ace.1
(project).fasta.screen.ace.2
(project).fasta.screen.ace.3
(project).fasta.screen.ace.4
(project).fasta.screen.ace.5
are in order of how old they are. If you feel you are taking up too
much disk space, then start deleting the ace files starting at the
oldest. I do not recommend that you overwrite existing ace files.
The version numbers just keep growing, and that is not a problem.
EXPORTING THE CONSENSUS
45) Exporting the consensus. Bring the Aligned Reads Window into view
again. Hold down the left mouse button on the 'File' menu and
release the button on 'Export consensus sequence'. Notice that the
consensus will be stored (in this case) in a file called
'Contig1.fasta'. Click 'OK'. There is now a file in your edit_dir
directory called 'Contig1.fasta' that has the consensus sequence in
it. If you want to see the file, bring up another Xterm (if you are
UNIX literate), and type:
cd standard/edit_dir
more Contig1.fasta
46) Fancier exporting the consensus. Bring the Aligned Reads Window
into view again. Hold down the left mouse button on the 'File' menu
but this time release on 'Export consensus sequence (with
options)...'. Just export a little snip of the consensus, from 400 to
410. (You will notice this contains a pad * character.) Under "Write
Both Bases File and Qual File or Just Bases File?" click "Both Files"
Click 'OK'. Consed will want to call this file 'Contig1.fasta' again.
You can overwrite the existing file.
Look in your other Xterm at these files:
more Contig1.fasta
more Contig1.fasta.qual
The one file contains the bases (but no * pads) and the other
contains the corresponding qualities of those bases.
47) Exporting the consensus of all contigs at once: Go to the Main
Consed Window. Point to 'File', hold down the left mouse button, and
release on 'Write all contigs to fasta file'. You then can choose a
filename for all contigs to be written to. (In this project there is
only 1 contig, so there is no difference between this option and just
exporting a contig at a time.)
48) COMPLEMENTING THE CONTIG
Push 'Compl Cont' in the Aligned Reads Window to complement the
contig. This displays the opposite strand of the contig including the
consensus and all reads. Push this button again to uncomplement it.
49) COLOR MEANS EDITED AND TAGS
(For this step, first click on the 'Dim' menu and release on 'Dim
Nothing'.) Point to the 'Color' menu, hold down the left mouse button
and release on 'Color Means Edited and Tags'. Notice that the bases
that you have edited (make sure you have edited some bases) will stand
out in either white or grey (depending on whether the base was made
high quality or low quality). Observe this both in the Trace Window
and the Aligned Reads window. This colormode is useful if you are
interested in easily spotting which bases are edited.
Return to the 'Color Means Quality and Tags' colormode by the
following: point to the 'Color' menu, hold down the left mouse button
and release on 'Color Means Quality and Tags'.
FIND MAIN WINDOW
50) On the Aligned Reads window, click on 'Find Main Win'. This will
cause the Consed Main Window to pop up in the event you have buried it under
other windows or iconified it. (This may not work with some settings of
your X emulator. In that case you will have to find and click on the
Main Window to bring it up.)
MULTIPLE UNDO EDIT
51) Now that the Consed Main Window is visible, click the 'Undo
Edit...' button. There will be a popup indicating the most recent
edit. (If it says "no edits so far", then bring up a trace and make
several edits. Then click on 'Undo Edit...' again.) Click 'undo'.
Then you will see the edit that was done before that. Click 'undo'.
You can continue undoing if you like. You now know how to undo more
than one edit. You cannot choose which edits to undo and which to not
undo--edits can only be undone in precisely reverse order from the
order you made them. Once you save the assembly, you cannot undo
prior edits.
SCROLLING TRACES AND ALIGNED READS TOGETHER
52) In the Aligned Reads window, scroll along the contig to a
different point. Click the left mouse button on a read whose trace is
already up. Notice that the existing trace instantly scrolls to the
corresponding location. Now go to the Trace Window and scroll the
traces to a new location. Click on the edt line with the left mouse
button. You will notice that the Aligned Reads window will instantly
scroll to the corresponding location. Thus you can keep the Aligned
Reads window and the traces scrolled to the same location.
EXAMINING ALL TRACES
53) Go to a region where there are lots of reads, say base 1660. Push
down the right mouse button and release on 'Display traces for all
reads'. You will see all traces displayed in a scrolling window. You
can drag the scrollbar on the right down and up to see all the traces.
This feature is particularly useful for polymorphism/mutation
detection work. This feature was added to work in cooperation with
polyphred. (See CONSED-POLYPHRED intereaction below.)
In this Traces Window, point at one of the bases of one of the reads
and click with the left mouse button. The base should start blinking
in red. Now push the down arrow key on your keyboard. The cursor
should move to the next read. Repeatedly type the down arrow key.
Eventually the display should scroll so you can continue to see the
read the cursor is on. Try the up arrow key as well.
EXITING CONSED
54) On the Aligned Reads Window, point to 'File' menu, hold down the
left button and release on 'Quit Consed'. If it asks you some
questions, answer 'Quit Without Saving and Discard .wrk File'.
ASSEMBLY VIEW
55) Consed can show you a bird's eye view of the Assembly using
forward/reverse pair information, sequence match information, read
depth, etc. We have a test database which shows its features.
Type:
cd assembly_view/edit_dir
(You might need to type "cd ../.." first depending on where you are.)
ls
Restart consed
Double click on "assembly_view.fasta.screen.ace.1"
In the Consed Main Window, click on the button "Assembly View" which is
near the upper left corner of the window.
You should see 3 grey bars with pink labels "2", "3", and "1". The
bars are the contigs: Pink "1" means Contig1, pink "2" means Contig2,
etc. Notice the scale on the contigs. This gives the contig
position.
READ DEPTH
56) You should see two graphs above the contig bars: one bright green
and one dark green. The dark green graph indicates read depth--the
depth of the quality 20 (by default) region of reads. Turn off read
depth as follows: Click on the button labelled "What to Show". A menu
will popup at that location. Click on the "Read Depth" menu item. A
box will appear labelled "Show Read Depth". It has a square (a toggle
button) with "show read depth" to the right of the toggle button.
Click on the toggle button to change it from appearing pushed in to
appearing sticking out. Then click on "Apply". The read depth graph
should disappear. If you would like, you can try showing read depth
for other qualities other than 20.
FORWARD/REVERSE PAIR DEPTH
A "forward/reverse pair" is a pair of reads from the same subclone
template, each of which is primed within the subclone vector, but one
is primed on one side of the insert and the other is primed on the
other end of the insert. A forward/reverse pair may both be assembled
into the same contig, in which case they should point towards each
other and be approximately the insert size apart. A forward reverse
pair also might be in different contigs on different sides of a gap.
57) The bright green graph is highest around 7000 to 10000 of Contig2
and around 14000 of Contig3. The bright green graph indicates, for
each base, the depth of subclone templates that have a consistent
forward/reverse pair. A forward/reverse pair is "consistent" if the
forward and reverse are pointing towards each other and are not too
far away from each other. ("Too far" is defined as 3 or more standard
deviations from the mean of the insert size of templates from a
particular library.) In other words, the green graph tells for each
base, how many consistent forward/reverse pairs have that base between
the forward read and the reverse read. This forward/reverse pair
depth is not the same as read depth, which is typically much less.
Forward/reverse pair depth is important in that it gives a measure of
the confidence of the assembly at a base. If the forward/reverse pair
depth is close to zero, as it is in Contig1 position about 9300, there
is a likelihood that phrap has made an incorrect join. When the
forward/reverse pair depth is zero, the green line turns red, as it
does on the right end of Contig3.
INCONSISTENT FORWARD/REVERSE PAIRS
58) The red lines connect the right end of Contig3 with the middle of
Contig1. These are filtered inconsisent forward/reverse pairs--they
are "inconsistent" because they are not consistent (see above) and
they are "filtered" in that they have another inconsistent read
close by (at both ends) that is inconsistent for the same reason. If
two red lines are on top of one another, it is displayed in purple so
you know there is more than one there.
This is a good example of a misassembly. There are many many reads at
the right end of Contig3 that are paired with reads in the middle of
Contig1. Notice that the forward/reverse pair depth of Contig1 is
close to zero around base 9300. (You can use the "Zoom In" button to
see this in more detail, but when you are done experimenting with the
Zoom buttons and the scroll bar, click on "Zoom Orig" for the rest of
this exercise.) This is where phrap made a bad join. If you tear the
contig apart there, complement the left part of Contig1, and then join
it to the right end of Contig3, the forward/reverse pairs will change
from inconsistent to consistent. You will learn later how to do that.
59) Point to one of the red lines. You will notice that it turns yellow.
the box near the bottom of the screen tells you a little more about
what you have "highlighted" (turned yellow). If you want more
information, click with the left mouse button. A window "Clicked
Forward/Reverse Pairs" will appear giving information about each
highlighted read. Try this. In the "Clicked Forward/Reverse Pairs"
Window double click on one of the reads. The Aligned Reads Window
should appear with the cursor on that read. This shows how to go from
the Assembly View Window to the Aligned Reads Window.
60) You can also go from the Aligned Reads Window to the Assembly View
Window. First you must make sure the Assembly View Window is already
open (or else open it by clicking on Assembly View in the Consed Main
Window). In the Aligned Reads Window, point to a read name, hold down
the right mouse button, and release on "Find Read in Assembly View"
(one of the last items in the menu the appears when you push down with
the right mouse button). If the read is from a subclone that has a
forward/reverse pair in the assembly, then the same "Clicked
Forward/Reverse Pairs" Window will appear. It will contain not only
the read that you pointed to, but all of the other reads from the same
subclone as the one you pointed to. In the Assembly View Window, all
of these reads will blink yellow. You can use this procedure to go
within the Aligned Reads Window from forward read to reverse read or
visa versa.
61) Notice the aqua and purple lines that connect the right end of
Contig2 to the left end of Contig3. These are consistent gap-spanning
forward/reverse pairs. If there is more than one pair on top of each
other, the color is purple. These are the reads that tell you (and Consed,
Autofinish, and Phrap) that the right end of Contig2 is connected to
the left end of Contig3. As above, point to one to highlight it and
click on it to see more information.
62) You can see much more information by clicking on the "What to
Show" button, and then when the menu pops up, click on the "Fwd/Rev
Pairs" menu item. Up will pop the "Which Fwd/Rev Pairs to Show in
Assembly View" Window. Click on "All" next to "Show Inconsistent
Forward/Reverse Pairs". Then click "Apply" at the bottom of this
window. In this particular example, you just see a few more stray red
lines. In a real example, you would probably see so many red lines
that it would be a mess. In most cases those inconsistent
forward/reverse pairs would be just caused by some laboratory problem
(turning a plate around, mislabelling, etc) and not to any
misassembly. Thus I suggest that you only generally leave "Show
Inconsistent Forward/Reverse Pairs" to "Filtered".
63) Still in the "Which Fwd/Rev Pairs to Show in Assembly View"
Window, click on "Show each consistent fwd/rev pair within contigs"
(so the button looks as though it is pushed in) and click "Apply".
This will show a blue (or purple if there is more than one at a
location) square for each consistent forward/reverse pair within a
contig. The horizontal position of the square is the center of the
subclone (midway between the forward and reverse read) and the
vertical position of the square indicates the size of the subclone
(higher means a larger subclone). If you really want to see the
position of the forward and reverse reads, you can do that too: Click
on "Show legs on squares for consistent fwd/rev pairs" ("Show each
consistent fwd/rev pair within contigs" must be still on) and click
"Apply". What a mess! I believe most of this information is much
more easily understood by just showing the "consistent fwd/rev pair
depth" (the bright green graph described above). But it is your
choice. When you want to highlight a consistent fwd/rev pair, you
must point to the square--not the legs. Try it so you understand.
64) Suppose you have an assembly and there are some forward/reverse
pairs that you specifically do not want to see in the Assembly View
Window. For example, perhaps they are from a plate that was misnamed
(or turned around) or from a library that is somehow less reliable.
By hiding these forward/reverse pairs, the more reliable/important
ones can more easily be seen. This is how you can do that:
In the "Which Fwd/Rev Pairs to Show in Assembly View" Window, notice
the line that says:
Do not show templates in file doNotShowInAssemblyView.fof
Underneath this are 3 buttons and probably the one that is selected is
"show all templates". Try clicking "do not show specified templates"
and click 'Apply'. See if you notice that anything changed in which
forward/reverse pairs are displayed. If not, switch back and forth
between "show all templates" and "do not show specified templates",
each time clicking 'Apply'. When you see a line that appears and
disappears, click on it to find what template it is. For example,
djs736a2_fp04q146 is one such template. Then from an xterm in the
assembly_view/edit_dir directory, type:
more doNotShowInAssemblyView.fof
You will see the names of the templates that are displayed/hidden.
In order to hide particular forward/reverse pairs, put them into
this file. This file can also contain the character '*' which means
"match any characters". For example, djs736a1_fp* would match the template
djs736a1_fp04q206
but not
djs736a2_fp01q127
65) Try turning on/off each of the Fwd/Rev Pair options so you
understand them. (In this example, there are no "consistent fwd/rev
pairs between different scaffolds.")
SEQUENCE MATCHES
66) Notice the curvy orange lines connecting Contig1 with Contig2 and
Contig3. These show sequence matches. Point at the one connecting
Contig1 and Contig2 and click on it. A "Sequence Matches" box will
popup saying that this match has 119 bases and has a similarity of
90.8%. Click on that line so its background turns black. Then click
on the button "Show Alignment". Up will pop the Compare Contigs
Window with the alignment shown in the lower half of this box. You
will learn more about this later (see "JOIN CONTIGS"). For now,
dismiss this window.
67) In the Assembly View Window, click on "What to Show" and then when
the menu pops up, click on "Sequence Matches". In the "Which Sequence
Matches to Show in Assembly View" Window, try clicking off "ok to show
sequence matches between contigs". Then click the "Apply" button.
You should see the orange lines disappear. (Any highlighted lines
will not disappear.) Click "ok to show sequence matches between
contigs" back on, and click "Apply" and the lines should be back.
68) Also in the "Which Sequence Matches to Show in Assembly View"
Window, change the minimum similarity from 90 to 85. Click "Apply".
You should see a lot more orange curvy lines, and now you should also
see black curvy lines. If you look carefully, you will see that 2
lines within each pair of orange curvy lines do not cross each other
but the 2 lines within each pair of black curvy lines do. This is
because orange is used to show direct repeats and black is used to
show inverted repeats (relative to the orientation of the contigs in
the Assembly View Window).
69) Also in the "Which Sequence Matches to Show in Assembly View"
Window, click on "filter seq matches by size" and set the min size to
400 and the max size to some huge number (e.g., 1000000) and click
"Apply". You will see just one direct repeat (orange curvy lines) of
size 745.
70) Try some of the other ways of filtering the sequence matches on
"Which Sequence Matches to Show in Assembly View".
71) You must learn this step if you are going to ever see sequence
matches with your own data, so don't skip this step. If you have
problems, it is likely that the phred/phrap/consed package has not
been installed correctly and you will need help from your system
administrator. Exit Consed and look at the files in
assembly_view/edit_dir.
Notice there is a file: assembly_view.fasta.screen.ace.1.aview
This is what Consed uses to show sequence matches in the Assembly
View Window.
When you use your own data, you will not have this file so you will
need to learn how to create it. Hide it from Consed by (in practice
you will never do this step--this is just to simulate the .aview file
not being there):
mv assembly_view.fasta.screen.ace.1.aview assembly_view.fasta.screen.ace.1.aview_hide
Now restart consed and select ace file
assembly_view.fasta.screen.ace.1
If you are asked if you want to apply edits, click the "No" button.
Click on "Assembly View" in the Consed Main Window.
You will get the error message:
"Sequence matches will not be shown in Assembly View because there is
no file
assembly_view.fasta.screen.ace.1.aview
If you want sequence matches to be shown, click on "What to show:
Sequence Matches" and then "run crossmatch"
72) RUNNING CROSSMATCH FOR SEQUENCE MATCHES
Just as the instructions (above) say, click on "What to show" and then
when the popup menu appears, click on "Sequence Matches" and then when
the "Which Sequence Matches to Show In Assembly View" Window comes up,
click on the "Run Crossmatch" button.
Watch the action in the xterm. There should be several pages worth of
output from crossmatch that scrolls by in the xterm. If you get an
error, it is likely that the phred/phrap/consed package is not
correctly installed. You (or your system administrator) should track
down the problems and correct them.
If you are successful, then 3 orange pairs of curvy lines will appear
in the Assembly View Window--the same as you saw in the steps above.
PULLING OUT READS AND RE-ASSEMBLYING THEM (MINIASSEMBLIES)
When the Assembly View Window indicates, using forward-reverse pair
information, that there is a misassembly, Consed provides the tools to
correct that misassembly: you can first pull out the the misassembled
reads from their current contigs into individual contigs, with a
single read per contig. Then you can reassemble those new contigs
that each contain a single read. Let's do this:
73) In the Assembly View Window move your cursor so that the red and
purple forward/reverse pair lines turn yellow. You will be unable to
get them all yellow, but get as many as you can. Then click with the
left mouse button. A window labelled "Clicked Fwd/Rev Pairs" should
appear with a very long list of reads in it (around 53 reads).
74) In the "Clicked Fwd/Rev Pairs" Window, click on the button labelled
"Pull out reads". A window labelled "Put Reads into Their Own Contigs"
should appear.
75) In the "Put Reads into Their Own Contigs" Window, select all of
the reads. You can do that by clicking with the left mouse button on
the first read and then scrolling down to the bottom of the list of
reads, holding down the shift key and clicking with the left mouse
button on the last read. (When a read is selected, its background
should be black.) Click on the button "Remove Highlighted Reads".
The Assembly View Window will close and reopen after a few seconds and
will complain about not being able to show sequence matches. Save the
assembly (see "SAVING THE ASSEMBLY" above) and follow the instructions
in "RUNNING CROSSMATCH FOR SEQUENCE MATCHES" (above).
The assembly will now probably contain 4 contigs: 2-3-1c in one scaffold
and 4 in the other. That is because when the misassembled reads were
pulled out of Contig1, it fell into two new contigs: the new contig 1
and contig 4. All of the reads you pulled out have created Contig5,
Contig6, ... and approximately Contig58, each of which contain only a
single read.
MINIASSEMBLIES
76) On the Consed Main Window, click the button "Miniassembly". A box
will popup labelled "Reassemble Some Contigs". On the left part of
the box will be all contigs, from Contig1 to about Contig58. Notice
that starting with Contig5 will be contigs that contain only a single
read. On the right will be Contig5 through approximately Contig58.
You add or delete from the list on the right. For example, to delete
Contig5 from the list on the right, click on it, and then click "Clear
Highlighted". The right list should now only contain Contig6 through
the last contig. Add Contig5 back to the right list by clicking on
Contig5 in the left list and then clicking on the button labelled
"Move Highlighted to Right". Contig5 will now appear at the bottom of
the list on the right.
77) Leave all of these boxes blank: "-minscore", "-minmatch",
"-forcelevel", and "other phrap options:". Keep "Put into separate
contigs" selected rather than "Disgard from assembly". Click the
"Reassemble" button. If you haven't saved the assembly, a box will
popup saying "Error You must first save the assembly before making a
miniassembly". Follow the instructions you learned above ("SAVING THE
ASSEMBLY") to save the assembly. Then click the "Reassemble" button
again and watch the action in the xterm. Lots of output from
determineReadTypes.perl, phrap, crossmatch will scroll by in the xterm
as those programs run. (If they don't, you haven't correctly
installed all of the Consed package.)
78) When the miniassembly is complete, a box will popup asking "Would
you prefer to discard this miniassembly and reassemble again?" Click
the "No" button.
79) On the Consed Main Window, click the "Assembly View" button.
Consed will complain about not being able to show Sequence Matches so
save the assembly and follow the instructions in "RUNNING CROSSMATCH
FOR SEQUENCE MATCHES" (above). In the Assembly View Window in
addition to Contig1, Contig2, Contig3, and Contig4, you should see a
few more contigs. These are the result of the miniassembly of all
those individual reads.
CONTIG ARRANGEMENT--REORDER CONTIGS
Contigs are arranged by Consed into "scaffolds" using forward/reverse pair
information. However, you might have some external information (such
as digest information) that tells you a different arrangement. You
can use Consed to rearrange the contigs. This new arrangement will be
preserved even if you reassemble.
80) Exit Consed and then restart Consed.
Double click on "assembly_view.fasta.screen.ace.1"
(If a window pops up saying "There is an edit history file ( a .wrk
file )...", click the "No" button.)
Click on the "Assembly View" button. You will see two scaffolds: one
on the top row with Contig2 and Contig3, and one on the bottom row
with just Contig1. Now suppose that you believe that Contig2 and
Contig1 are connected together instead of Contig2 and Contig3. To do
this:
81) Within the Assembly View Window, click on the "Contig Arrangement"
button. Up will pop a menu. Click on "Reorder Contigs". A "Reorder
Contigs" Window will pop up. Enter the following information:
Contig: 2 [Right End] connected to Contig: 1 [Left End]
That is, you must enter "2" and "1" in the contig boxes, and you must
click on the first "right end" button.
Then click on the "Add and Restart Assembly View" button. A warning
box will pop up telling you that you are crazy, because there are 12
forward/reverse pairs as evidence that the scaffold as displayed in
the Assembly View Window is already correct. Click on "yes"--that you
are sure.
The Assembly View Window will disappear for a second and reappear,
with Consed2 and Contig1 connected together, just as you wanted.
CONTIG ORIENTATION
82) Some users want a scaffold oriented a particular way. For
example, one user might be working on a particular gene so wants to
always view the top strand of that gene. Another user might be
finishing a BAC and wants the 5' end of the BAC on the left of the
scaffold. Phrap, however, may not respect their wishes and might have
contigs complemented from the way the users want to view them. Consed
provides a way for the user to indicate his/her desired orientation,
and thereafter if phrap complements a contig from that desired
orientation, Consed will complement the contig back when Consed starts
up.
To demonstrate this, exit Consed and then restart Consed.
Double click on "assembly_view.fasta.screen.ace.1"
In the Consed Main Window, double click on Contig1. You will see read
djs736a2_fp02q494.y1 pointing left. But let's suppose that you would
rather the Contig be in the other orientation, with read
djs736a2_fp02q494.y1 pointing right.
In the Consed Main Window, click on Assembly View. Then click on the
button labelled "contig arrangement". When a popup menu comes up,
click on "Reorient Contigs". The "Reorient Contigs Window" should
come up. Highlight the scaffold labelled "1" under "Select a
scaffold". Click on "flip scaffold". Then push the button labelled
"Apply and Restart Assembly View". There will be an error box
complaining about not being able to show sequence matches. To fix
that, save the assembly and follow the instructions in "RUNNING
CROSSMATCH FOR SEQUENCE MATCHES" (above). In the Consed Main Window,
double click on Contig1 so the Aligned Reads Window comes up. Scroll
to the right end. You will notice that djs736a2_fp02q494.y1 is now on
the right end pointing right.
What is the difference between doing this and just complementing the
contig, which just takes the click of a button? The difference is
that complementing the contig will be undone the next time phrap runs,
but using this procedure will be permanent, even if phrap complements
the contig.
CONSED-POLYPHRED INTERACTION
Polyphred is a program for finding polymorphic sites; it was developed by
Debbie Nickerson's group (contact them at http://droog.mbt.washington.edu).
We have a test database, 'polyphred', which has had polyphred run on
it already. Polyphred has put a polymorphism tag on each polymorphic
site.
If Consed is running, exit it.
Type:
cd polyphred/edit_dir
(You might need to first type "cd ../.." depending on where you are.)
ls
Restart Consed.
Double click on example2.fasta.screen.ace.1
When Consed comes up, you should see 2 contigs.
Double click on Contig2
In the Aligned Reads Window, push the left mouse button while pointing
to the 'Navigate' menu and release on:
'Toggle feature: when navigating to consensus location, pop up all
traces (currently off)'
That will turn this feature on.
Now push the left mouse button while pointing to the 'Navigate' menu
and release on 'Tags'. Up should pop a list of tag types. Double
click on 'polymorphism'. Polyphred has already been run so the
consensus is tagged with polymorphism tags at each polymorphic site.
Up will pop a window labelled 'Polymorphism Tags' with a list of
sites. Click on 'Next'.
If you correctly followed the instructions above, all the traces should
pop up at the first polymorphic site. You may want to reposition the
traces window to see it better.
Now ignore the original 'Polymorphism Tags' window and instead click
on 'Next' in the *traces* window. This will take you to the next
polymorphic site. Pretty nice, huh?
Dismiss the Traces Window.
83) ALPHABETICAL ORDERING OF READS
The reads can be ordered in two ways:
a) alphabetically
b) first all the top strand reads and then all the bottom
strand reads. The top strand reads are then ordered
by the left end of the reads. Same with the bottom
strand reads.
Try changing between a) and b). In the Consed Main Window (click on
'Find Main Win' on the Aligned Reads Window if you can't find the Main
Consed Window because it is covered up with other windows), pull down
the 'Options' menu, and release on 'General Preferences'. Scroll down
until you find 'Display reads sorted alphabetically or by strand/left
end of read.' Switch it between 'alpha' and 'strand'. Then click
'Apply and Dismiss'. Notice the effect in the Aligned Reads Window.
Many polymorphism and mutation detection labs find that alphabetically
sorting is most useful, while many genomic sequencing labs find that
sorting by strand/left end of read is most useful.
After you are done playing with these features, exit Consed and go back
to the previous database:
cd standard/edit_dir
(You might need to first type "cd ../.." depending on where you are.)
ls
Restart Consed.
Double click on standard.fasta.screen.ace.1
When it says "There is an edit history file (a .wrk file)...Do you
want to apply those edits?", click on "no".
Double click on Contig1 to bring up the Aligned Reads Window again in
preparation for the next step.
NAVIGATING
84) In the Aligned Reads window, pull down the Navigate menu and
release on 'Low consensus quality'. You will see a list of locations.
Move the 'Low consensus quality' window down so you can see the
Aligned Reads window.
Repeatedly click on 'Next' until you reach the end of the list. (Low
consensus quality means an area in which the bases each have too high
probability of being wrong.) This saves you from having to look
through large amounts of high quality data trying to find problem
areas.
There are 2 'Next' buttons--one on the Aligned Reads Window and one on
the Low Consensus Quality Window. You can click on either, but it is
probably more convenient to use the 'Next' button on the Aligned Reads
Window. Thus you can keep the Aligned Reads Window in
front with input focus and keep the Low consensus quality window
pushed out of the way.
You may want to click on the 'Save' button in the Low consensus
quality Window to save to a file a copy of this list of problem areas
as you work through them.
In our experience, this will be the most important navigate list you
will use. In fact, finishing partly consists mainly of adding reads
and rephrapping until this list is reduced to nothing.
85) Dismiss the Low consensus quality window. Pull down the
'Navigate' menu again and release on 'High quality discrepancies as
above, but omitting tagged compressions and G_dropouts'. You will
probably notice there are no entries (unless you created some yourself
by editing). That is because there are no high quality discrepancies
with this dataset. So let's force there to be some by lowering the
quality threshold. First, dismiss the High quality discrepancies
window.
Click on 'Find Main Win'. In the Consed Main Window, pulldown the
'Options' menu and release on 'General Preferences'. Notice that the
default for 'Threshold for High Quality Discrepancy' is 40. Change it
to 15 and click 'Apply & Dismiss'.
Then follow the steps above to bring up the High quality discrepancies
menu. Now you will see several entries. Click 'next' repeatedly to
go successively to the next high quality discrepancy in the Aligned
Reads Window.
You can also double click on a particular line in the High quality
discrepancies window to go to that location. Alternatively, you can
single click on a line and then click the 'Go' button.
Dismiss the High quality discrepancies window.
86) Similarly, try the other navigate lists: Unaligned high quality
regions (this list will be empty with this data set), Edits, Regions
covered by only 1 strand and only 1 chemistry, and Regions covered by only 1
subclone.
Unaligned high quality regions are regions in which the traces are
high quality so there is no question of the bases, but the region
differs so much from other reads that phrap has given up trying to
align the region with the consensus. This could be due to a chimeric
read, or perhaps the read belongs somewhere else.
We believe that regions covered by only 1 subclone should be covered
by a 2nd subclone to prevent the possibility of there being a deletion
in the single subclone.
There are so many different problem lists that you may forget to check
one of them and thus miss a serious problem. Thus we combined them
all into a single list. This is the first menu item: 'Low Cons/High
Qual Discrep/Single Stranded/Single Subclone/Unaligned High'. We
suggest you use this list.
87) Also try navigate by tags by selecting 'tags' under navigate: when
the Select Tag Type Window appears, double click on 'compression'.
(Note that you can't do anything else until you deal with this
window.) This gives a list of a particular tag type in a particular
contig.
88) There is also a way of getting a list of a particular tag type in
all contigs: Click on 'Find Main Win'. In the Consed Main Window,
point to the 'Navigate' menu, hold down the left mouse button, and
release on 'Tags in all contigs'. Continue as in the previous step.
(Since there is only one contig, this list will not be any different
than the corresponding list for Contig1.)
PRIMER-PICKING
89) Go to some location near the right end of the contig, say base
2470. Click with the right mouse button on the consensus and click on
either one of the top strand primer choices (either from subclone
template or from clone template). Consed will pause a moment, and
then there will appear a selection of primers that pass all of
Consed's requirements. (If you get an error message, Consed might not
have been correctly installed. See INSTALLING CONSED above.)
Templates are also chosen for each primer. You may have to scroll the
primer list to the right to see the templates. Consed lists these
templates in order of quality--all of them will cover the read you
want to make.
Double click on one of the primers in the Primers Window. That will
cause the Aligned Reads Window to scroll to show that oligo in
context. Click on 'Accept Primer'. A comment box will pop up. Enter
some comment and click 'OK'. Notice that a yellow oligo tag, with a
little red end, is created on the consensus for that primer. The red
end points in the direction of the oligo. The tag contains all the
information you need to order that oligo and do the reaction--you will
learn how to pop it up below under 'tags'.
What is the difference between 'Pick Primer from Subclone Template'
and 'Pick Primer from Clone Template'?
There are 3 differences:
A. which vector file the primers are screened against. In the former
case, the primer is screened against the file primerSubcloneScreen.seq
and in the latter case against the file primerCloneScreen.seq
B. In checking for false matches elsewhere in the assembly, if the
template is the whole clone, then Consed must check for false matches
in the *entire* assembly, including all other contigs. But if the
template is just going to be a subclone, Consed only needs to check
elsewhere in that subclone. Actually, to be conservative, Consed
checks for false matches +/- the maximum insert size of a subclone.
C. If you are picking primers for subclone template, then the primer
picker can also pick the subclone templates. If it doesn't find any
suitable subclone template, it will reject the primer. (By default,
picking of subclone templates is turned on. If you prefer to pick
your own primers, and want Consed's primer picker to be much faster,
you can turn it off temporarily or permanently. To turn it off
temporarily, go to the Consed Main Window, point to the Options menu,
hold down the left mouse button and release on 'Primer Picking
Preferences'. Scroll down to 'Pick Subclone Templates for Primers'
and click 'False'. Click on 'Apply and Dismiss'. To change this
permanently, see CONSED CUSTOMIZATION below. Beware: you must
correctly customize determineReadTypes.perl for template picking to
work. See INSTALLING CONSED above.)
If you are interested in the details of primer-picking, see the
section 'AUTOFINISH AND PRIMER PARAMETERS' (below).
When you are done editing and have saved the assembly and exited
Consed, run ace2Oligos.perl (supplied with this distribution--make
sure your system administrator installed it) which will extract all
the oligos you just created. This is handy for email ordering of
oligos.
In the xterm, type:
ace2Oligos.perl standard.fasta.screen.ace.2 oligos.txt
where standard.fasta.screen.ace.2 is whatever the name is of the ace
file you just saved.
ace2Oligos.perl does not record the comments that the finisher
entered when creating the oligo. If you want to record that as well,
you could use the script ace2OligosWithComments.perl which was written
by a Consed user and thus is found in the "contributions" directory.
90) WHEN CONSED CAN'T FIND AN ACCEPTABLE PRIMER
Sometimes Consed refuses to pick a primer. This is because it has
tried every possible primer and rejected it for one reason or
another. If you don't understand why it didn't pick a particular
primer, you can ask it as follows:
In the Aligned Reads Window, point to the "Misc" menu, hold down the
left mouse button and release on "Check Primer". Enter the left and
right consensus positions of the primer, check which strand, and
whether the primer is to use subclone templates or the whole clone as
a template. Consed will tell you all that is wrong with that primer.
Try looking at a top strand subclone primer from 2340 to 2360.
91) PICKING PCR PRIMER PAIRS
In the Aligned Reads Window, go to the location where you want to pick
the first PCR primer, say base 500. Point to the consensus, hold down
the right mouse button and release on "Top Strand PCR Primer". Then
scroll to the location where you want to pick the second PCR primer,
say base 2200. Point to the consensus, hold down the right mouse
button and release on "Bottom Strand PCR Primer". There will be a
pause and then there will be a list of PCR primer pairs. Click on the
pair you want and click "Accept Pair".
You can modify the parameters for choosing PCR primer pairs by going
to the Consed Main Window, pointing to "Options", holding down the
left mouse button, and releasing on "Primer Picking Preferences." For
example, by default Consed does not display all PCR primer pairs--this
would take too long and give you too many. However, you can ask it to
show you all such pairs. In the Primer Picking Preferences, scroll
down to "Check All PCR Pairs (huge) or Just Sample?" and click on
"All". Then click on "Apply and Dismiss". Then pick PCR primers
again, as above. Don't be surprised if you get 10,000 or more pairs
of primers!
SEARCH FOR STRING
92) Try the 'Search for String' button (left side of the Aligned Reads
Window). Type in a string (such as aaaca), and click 'ok'. There
should be a list of 'hits'. Double click on one of the hits (or
single click on it and click on 'go'.) Notice that the Aligned Reads
Window scrolls to that position and has the cursor on the found
string. (It might be complemented.)
Dismiss this window. Try this again, only this time in the Search For
String Window select 'Search Just Reads'. Then click 'OK'. You will
notice there are many more hits. This is because this shows hits in
each read, even if they are at the same consensus position.
You can also try the approximate match search for string by clicking
on 'Approximate' instead of 'Exact'. The 'Per Cent Mismatch' only
applies to the Approximate match search.
COPY AND PASTE
93) In the Aligned Reads Window, swipe some bases by holding down the
left mouse button. You should see the bases turn yellow, at least
temporarily. Then click the 'Search for String' button. Use the
middle mouse button to paste the bases you have just swiped into the
'Query string:' box. Notice that you can swipe bases either from the
consensus or from a read.
The search for string is case-insensitive so don't worry about the
pasting being upper or lowercase.
CORRECTING FALSE JOINS MADE BY PHRAP
94) Phrap may put several reads together that you believe do not belong
together. (For example, you may see several high quality
discrepancies between the reads.) If you are sure these reads do not
belong together, you can force a subsequent reassembly by phrap to not
assemble those reads together. You do this by finding a location
where there is a high quality discrepancy. Then click on the read
with the right mouse button and release on 'Tell phrap not to overlap
reads discrepant at this location'. There are no high quality
discrepancies with this dataset so Consed won't let you do this.
(Try it and see.) However, when you use your own data, you may get
the chance!
It is possible to automate this procedure using AutoEdit (see USING
AUTOEDIT).
ADDING NEW READS
95) For this to work, your system administrator must have set up
everything correctly. (See below in INSTALLING CONSED.) Assuming you
have set everything up correctly, you can now experiment with adding
reads.
From a UNIX prompt, copy the new chromatograms into the chromat_dir
directory:
cp ../chromats_to_add/* ../chromat_dir
Exit Consed and bring it up again using the original ace file
standard.fasta.screen.ace.1
If it asks if you want to apply edits, just say 'no'.
On the Main Window, click on the Add New Reads button. There will
appear a list of files ending with .fof. These are files that contain
lists of chromatograms. Double click on 'reads_to_add.fof' Then
Consed will ask "If a read doesn't align against any existing contig,
do you want to have it go into a contig by itself? (otherwise it will
just not be put into the assembly)" Users usually prefer to answer
"yes". Consed will ask "Do you want to recalculate the consensus
quality values where each of the new reads is aligned?" Answer yes or
no, but in practice you should generally answer "yes." There should
be lots of progress output in the xterm from which you started Consed.
When it completes, there will be a Reads Added Window popup with a
report of which reads were added. In this case, it should say that 9
reads were successfully added and list them.
If you get an error message, look carefully at the full error message
in the xterm to diagnose the problem. Probably there is some mistake
in how you installed Consed. See INSTALLING CONSED (above).
TEAR CONTIG
Just so you get the same results as I do, exit Consed and bring it up
again using the original ace file
standard.fasta.screen.ace.1
If it asks if you want to apply edits, just say 'no'.
96) When phrap really screws up, you may want to just tear the contig
apart in several places and then join the pieces back together in a
different way. Let's try it:
Go to location 1500. Point the mouse at the consensus base at 1500
and push the right mouse button down. Release the button on 'Tear
Contig at This Consensus Position'. Up will pop a list of reads with
2 little buttons next to them <- and ->. Leave everything as it is
and just click 'Do Tear'. (If you want to play around with which
reads goes into which contig, do that another time.)
Now you should have 2 Aligned Reads Windows on top of each other. One
should contain 'Contig2' and the other 'Contig3'. Dismiss the little
window that says 'Tear Complete'.
JOIN CONTIGS
97) Now let's join these 2 contigs back together:
Click on 'Search for String' and type in the following bases:
agctgccatc
Click 'OK'.
Search for string should find 2 locations, one in Contig2 and one in
Contig3:
Contig2 (consensus) 1447-1456 (uncomplemented)
Contig3 (consensus) 829-838 (uncomplemented)
Double click on the first one. The Aligned Reads Window for Contig2
will scroll to location 1447 and the window will raise up. In that
Aligned Reads Window, click on 'Compare Cont'.
Now double click on the 'Contig3' line in the above Search for String
results. The Aligned Reads Window for Contig3 will scroll to location
829 and lift up. In that Aligned Reads Window, click on 'Compare
Cont'.
Now the Compare Contigs Window should be visible. In the Compare
Contigs Window, try scrolling back and forth. You can change the
cursors (blinking red), but if you do, please return them to the
locations 1447 and 829 for the next step. The cursors 'pin' these
bases together when doing an alignment. (The algorithm is a pinned
and banded Smith-Waterman alignment.)
Click on Align. Try scrolling the alignment by dragging the thumb in
the lower half of the Compare Contigs. An 'X' means there is a
discrepancy between the 2 contigs. There is also a 'P' (see if you
can find it!) The P indicates the bases that you pinned together.
You will also notice that some bases are lighter and some are darker.
This indicates quality just as in the Aligned Reads Window. You will
notice that wherever there an is a discrepancy (an 'X') one of the
bases is low quality. This is your cue that the discrepancy is just a
base calling error rather than indicating that the two contigs really
are different but similar locations.
Click with the left mouse button on either contig in the bottom
alignment. You will notice that both contigs will have the red
blinking cursor in the same position. Click on 'Scroll Both Aligned
Reads Windows' and look at the Aligned Reads Windows to see that they
scroll to the corresponding positions. You can have traces up for the
contigs, and they will scroll as well. Experiment with this. Then
click 'Join Contigs'. The 2 previous Aligned Reads Windows will
disappear and there will be a new one which has a new contig
'Contig4'. You have made a join!
Scroll left and right. You will notice that many of the reads are
highlighted. These are the reads that came from the previous "right"
contig. To unhighlight all of these reads at once, point to the
"Misc" menu, hold down the left mouse button and release on
"Unhighlight All Reads".
It is possible to have more than one Compare Contigs Windows up at a
time. This allows you to investigate a repeat that has more than 2
copies.
COMPARE CONTIGS WINDOW AND INVERTED REPEATS
In the above example, we used the Compare Contigs Window to
examine a sequence match between two different contigs. It is also
possible to use the Compare Contigs Window to examine a sequence
match between two copies of a repeat within the same contig, either
direct or inverted.
98) To see this, restart Consed:
../../consed_(computer type)
Double click on standard.fasta.screen.ace.1
When it says "There is an edit history file (a .wrk file)...Do you
want to apply those edits?", click on "no".
Double click on Contig1 to bring up the Aligned Reads Window. Go to
position 69 (use the "Pos:" box described above). Click the "Compare
Cont" button on the Aligned Reads Window. The Compare Contigs Window
will popup, but move it aside. Go to position 2035 in the Aligned
Reads Window. Click the "Compare Contig" button again on the Aligned
Reads Window. In the Compare Contigs Window there are two copies of
Contig1--one on top and one on the bottom. Each has a "complement
just in this window" button. Click on the bottom one (the one that
has position 2035 blinking red). After clicking on it, you should
notice that the numbers on the bottom contig are reversed to they
decrease to the right--a copy of Contig1 has been reversed and
complemented. Now click the "Align" button. Suddenly, you should see
the alignment appear in the bottom half of the Compare Contigs Window.
You should see bases between 69-78 aligned against the reversed
complement of bases from 2026-2035.
This has shown how you explore an inverted repeat. If you wanted to
examine a direct repeat, you would use the same method except you
wouldn't click on the "complement just in this window" button.
Compare Contigs is one method of exploring joins of contigs that were
not made by phrap. Another method is to use the Assembly View Window
(above). They are designed to work together: the Assembly View Window
gives a high level view of all sequence matches and takes you to the
Compare Contigs Window which shows the alignment of a single sequence
match and, if the user so desires, makes a join.
REMOVING READS
99) You can remove individual reads and put them into their own
contigs. For example, in the Aligned Reads Window, go to location
2000. Point to the read name of read djs74_2664.s1 and hold down the
right mouse button. Release on 'Put read djs74_2664.s1 into its own
contig.' Presto-chango! The read is put into its own contig and the
old contig is redrawn without the read in it. At this point you
should save the assembly--you should always save the assembly after
removing reads.
100) You can also remove many reads at once.
Look at the Consed Main Window. Click on "Remove Reads". Type into
the "File of read names:" box "reads_to_remove.fof" and either push
the "Enter" key or click on "Read File". You should see a list of 2
reads:
djs74-2231.s1
djs74-3174.s1
You can click back and forth between the choices of "Delete Reads from
Assembly" and "Just Put Each Read into Its Own Contig". Try each
one.
Delete Reads from Assembly means that the read will no longer appear
in Consed. When you are using your own data and you really want to
remove reads from the assembly, you must also use the UNIX "rm"
command to remove the corresponding phd files from phd_dir and the
chromatograms from chromat_dir. Otherwise, the next time you run
phredPhrap, the reads, like Phoenix, will rise again to become part of
the next assembly.
After you have completed this exercise, restart Consed so that you
have all the reads in their original locations for the following
exercises.
TAGS
101) Bring up a trace for a read (as above). Swipe some bases on the
'edt' line while holding the middle mouse button down. A list of
choices will pop up. Select 'Add Tag'. Type in a comment in the box
at the bottom, and select 'comment' from the list of tag types. You
will now see a blue box both in the Aligned Reads Window and in the
Traces Window on that read.
To see the comment, you can just point to it in the Aligned Reads
Window and you will see the comment in the lower right hand corner of
the Aligned Reads Window. Alternatively, you can click on that blue
tag in the Aligned Reads Window with the right mouse button and
release on 'Tag: comment Show more info?'. Alternatively, you can
click on the blue tag in the Traces Window with the right mouse
button.
Try creating some other kinds of tags: again swipe some bases in the
Trace Window by selecting a different tag type. You will notice that
different tags are in different colors. You can always use the
methods above to see what kind of tag it is if you forget what a
particular color means.
You can also define your own tag types. See below CREATING CUSTOM TAG
TYPES for how to do that.
CREATING LONG TAGS
102) You can create really, really long tags as follows: Just create a
short version of the tag as above for where you want the tag to start.
Then figure out the consensus position of where you want the tag to
end. In the Aligned Reads Window, click on the short tag with the
right mouse button and release on 'tag: show more info?' (as above).
A Tag Window will appear for that tag. In the Tag Window, simply
change the End Unpadded Consensus Position to the place you want it to
end. Then click 'OK'. You will now notice that the tag will be as
long as you wanted.
CONSENSUS TAGS
103) You can create tags on the consensus in the same way. In the
Aligned Reads Window, use the middle mouse button to swipe some bases
on the consensus in the Aligned Reads Window. Up will pop a list of
tag types. Click on one of them. Try it again somewhere else. Try
it with the tag type being 'comment'. In this case, you must enter a
comment. Notice the pretty colors! If you forget which tag type a particular
color represents, just point at the colored tag with the mouse and the
tag type will be displayed at the bottom of the Aligned Reads Window.
104) Try creating some tags that overlap each other. You will notice
that the overlapping region will be purple. If you want to know which
tags overlap, you can use any of the methods already discussed.
SEARCH FOR READ NAME
105) Restart Consed using the original ace file
standard.fasta.screen.ace.1
If it asks if you want to apply edits, just say 'no'.
Instead of clicking on a read or contig name, type a read name into
the "Find reads containing (*'s allowed):". If you want to look at
the location containing read djs74-2689.s1, you can just type "2689"
and then push the "Enter" key and Consed will immediately bring up the
Aligned Reads Window with the cursor on read djs74-2689.s1. Suppose
that there were more than one read that matched? For example, suppose
you type: "26" and then push the "Enter" key. This matches 3 reads:
djs74-2689.s1
djs74-2679.s1
djs74-2664.s1
Try it and see what happens...
Try entering "26*9" and see what happens. What does the "*" mean?
Try using "Find 1st read starting with:". Try typing djs74-2 You will
notice that as you type each letter, the first item in the list that
matches the letters typed will be highlighted. Experiment with
deleting a few letters and typing others. This is a powerful method
of quickly getting to the read name you are interested in. When you
get to the name in the list, you do not have to type the rest of the
name--just type carriage return or else click on 'OK'.
ONLINE DOCUMENTATION
106) On the Aligned Reads Window or on the Consed Main Window, click on
the 'Help' menu and release on 'Show Documentation'. You will see
this document. You can search for keywords in it.
THE .WRK FILE
107) Consed keeps a log of all changes you make to an assembly: adding
new reads, putting reads into their own contigs, making joins and
tears, adding and removing tags, and changing bases. This log is kept
in a file ending with ".wrk".
If Consed (or your computer) crashes, Consed can use the .wrk file to
recover your edits (base changes, adding and removing tags, and
complementing contigs).
Make an edit (remember, edits are made in the Trace Window) and jot
down its location. Also note the name of the ace file which is
displayed in the upper left box in the Aligned Reads Window. Then
simulate a crash by going to the xterm where you started Consed and
typing control-C (holding down the control key, and typing C).
Restart Consed and select the same ace file you noted (above). A box
will come up saying 'There is an edit history (a .wrk file) Consed may
have crashed during a previous session with this same file. Do you
want to apply those edits?' Click on 'yes'. Go and find the edits
you made before Consed crashed--you will find them.
108) You should save your edits by pulling open the 'File' menu on the
Aligned Reads Window, and releasing on 'Save assembly'.
RESTRICTION DIGEST
109) Restart Consed.
Double click on "standard.fasta.screen.ace.1"
In the Consed Main Window, click the "Digest" button. For the
purpose of this exercise, the full pathname of file of vector sequence
can refer to any file of sequence in fasta format. However, when you
are using it with your own data it should refer to a file that
contains the sequence of your cloning vector. For example, if you are
sequencing a BAC, it should contain BAC vector. The sequence must
start at the vector/insert junction that you used when you ligated the
insert.
Click "OK". You will see a comparison of in-silico fragments (those
calculated from the sequence) and real fragments (those in
fragSizes.txt which supposedly came from a real gel).
* If a band is red, that means that it doesn't match.
* If a band has a "v" on it, that means it is a vector fragment.
* If a band has a "g" on it, that means it is a gap-spanning fragment.
Move the pointer over the fragments, and you will see the fragment
sizes appear. Move the pointer to the in-silico fragment with size
2299. Click on it. You will see the fragment on the left size of the
window become highlighted. Click on the button labeled "right end"
(2nd row from the bottom of the window) and the Aligned Reads Window
will pop up, with the cursor on the right end of the fragment.
Click on "show problems" and navigate through the list of problems by
clicking on "next". You will notice that the Gel Window is zoomed
in. To return to the original zoom, click on "Zoom Original".
Where it says "Select Enzyme:", point to "EcoRV", hold down the left
mouse button and release on "HindIII". This is how you change
enzymes.
Click on the button labeled "Text Output". This can be saved to a
file and printed out.
Dismiss the restriction digest window. On the Consed Main Window,
click the "Digest" button again. Notice the file "fragSizes.txt".
This is a file of actual gel fragment sizes. If you don't have an
actual gel, but rather you want to just make predictions of fragment
sizes from the sequence, you can leave this box blank (erase the
"fragSizes.txt"). Try that.
fragSizes.txt has the following format:
>EcoRV
448
710
1102
1197
-1
>HindIII
448
508
586
735
801
-1
where EcoRV and HindIII are enzymes and the numbers below them are the
actual fragment sizes. Each enzyme list is terminated by -1.
Consed does its best to try to figure out which end of the clone
insert is connected to which end of the vector. However, it sometimes
is wrong. If you believe it is wrong, you can click "compl vector" to
try connecting the insert to the vector in the opposite orientation
and see if that produces better agreement with the actual digest.
PROTEIN TRANSLATION AND OPEN READING FRAMES
110) If you would like, you can see the amino acid translation of the
consensus in all reading frames. In the Aligned Reads Window, push
down the left mouse button on the 'Misc' menu and release on 'Show Top
Strand Protein Translation'. Try again but this time release on 'Show
Bottom Strand Protein Translation'. Notice that there are 2
characters that are in magenta color. What are those characters? Why
are they made in a different color? To not show the protein
translation, push down the left mouse button on the 'Misc' menu and
release on 'Don't show protein translation'.
111) You can search for open reading frames (a methionine and a stop
codon within the same reading frame) within a contig. In the
Aligned Reads Window, push the left mouse button on 'Navigate' and
release on 'Search for Open Reading Frames'. Notice that the open
reading frames are shown for all 6 reading frames and are sorted by
length.
ERROR RATE
112) In the Aligned Reads Window is a box (upper right) labelled
'Err/10kb'. This is the estimated error rate for this contig, and it
is a good indicator of when you are done (or not done) finishing.
In addition, you can find the error rate for a particular region of
contig as follows: Point at 'Misc' menu, hold down the left mouse
button, pull down and release on 'Show Error Info For Region'. Fill
in the boxes for left and right consensus position, click on
'Calculate' and you will be given the error and single subclone data
for that region.
RUNNING PHRED and PHRAP
phred and phrap *must* be run via the phredPhrap perl script. If you
don't do this, you are on your own. If you run phred on its own, and
then you run phrap on its own, you will get an ace file that will not
be usable by Consed. After you have run into problems (and you
probably will), then do not email us--instead please use the
phredPhrap script. To use the phredPhrap script to run phred and
phrap:
113) Type:
phredPhrap -V
It should say:
030326
(or newer).
If it does not, then you probably have not installed all the perl
scripts from the scripts directory, as directed in INSTALLING CONSED.
114) Make a copy of the standard dataset. E.g.,
(First go up by typing "cd .." until you see "standard" when you type "ls".
Then type:
cp -r standard test
cd test
115) Delete all the files in phd_dir and edit_dir:
rm phd_dir/*
rm edit_dir/*
116) cd edit_dir
117) Run phredPhrap by typing
phredPhrap
That's it--you no longer need to type *any* arguments, and generally
you should not. (Please do *not* use the -notags option any longer.)
If you want to add phrap options, you can do that:
e.g.,
phredPhrap -forcelevel 3
Then run Consed on the resulting ace file as indicated in the beginning of
the Quick Tour (above). If you have any problems, this is the time to
diagnose them before you use your own data.
COMMON PROBLEMS RUNNING PHREDPHRAP
118) Problems that were due to polyphred. To check this, in
phredPhrap, leave the following line:
$bUsingPolyPhred = 0;
This will make polyphred not be used. If the problem then goes away,
you will know the problem has something to do with polyphred so do not
contact any of the phred/phrap/Consed people. Instead, contact the
polyphred people: http://droog.mbt.washington.edu and
dpc@u.washington.edu and debnick@u.washington.edu
119) Permission problems. Check that you have write access to the
phd_dir and edit_dir directories. You can do this by trying to create
a file in those directories:
touch ../phd_dir/xxx
which creates a file
ls -l ../phd_dir/xxx
which checks if the file was created.
Do the same with ../edit_dir/xxx
If you get a permission problem, do not contact me. UNIX permission
problems are very simple for anyone who knows UNIX--get someone
locally who understands UNIX and can help you solve the permission
problem.
----------------------------------------------------------------------------
USING AUTOFINISH
Note: Before you use Autofinish on your own data, you must modify
determineReadTypes.perl. See INSTALLING CONSED above for information
about this.
To do the exercises in this section, it would help to be able to edit
a file under UNIX and run a program under UNIX. If you can't do that,
have someone teach you. (It will not work to edit a file on Windows
and then transfer to UNIX.) Typical editors on UNIX are vi and emacs,
but pico is probably the simplest for occasional users. You can find
more information on pico from:
http://www.strath.ac.uk/IT/Docs/IntroToUnix/node122.html
You should also learn how to examine a file in UNIX, how to move
around the filesystem, etc. If you don't know how to do this,
consult:
http://www.washington.edu/computing/unix/startdoc/files.html
and
http://www.washington.edu/computing/unix/startdoc/directories.html
There are also many books about Unix at bookstores.
120) Type:
cd autofinish/edit_dir
(You might need to first type "cd ../.." depending on where you are.)
121) Try starting Autofinish by typing:
../../consed -ace autofinish.fasta.screen.ace.1 -autofinish
(Note 'consed' above may be 'consed_solaris', 'consed_alpha',
'consed_hp', 'consed_sgi', or 'consed_linux' depending on your
executable. If you have trouble, use that 'ls' command (see above) or
consult the person who installed Consed! )
If Autofinish says:
Run-time exception error; current exception: InputDataError
No handler for exception.
Abort
that means that you have not followed the instructions under
'INSTALLING CONSED' above. Please follow those instructions and then
try this again.
When you have successfully run the above command, Autofinish will
create 7 files:
autofinish.fof
(project name).001014.155627.customPrimers
(project name).001014.155627.nav
(project name).001014.155627.out
(project name).001014.155627.sorted
(project name).001014.155627.univForwards
(project name).001014.155627.univReverses
Where '001014.155627' is replaced by your current date and time in
format YYMMDD.HHMISS. The first file, autofinish.fof, is a file of
filenames. It contains the names of the other files.
(project name).001014.155627.univForwards
is the summary file of the suggested universal forward subclone reads
(project name).001014.155627.univReverses
is the summary file of the suggested universal reverse subclone reads
(project name).001014.155627.customPrimers
is the summary file of the suggested custom primer reads
These are the files you will typically use for directing your bench
work. If you like, you can import these files into Excel since the
fields are separated by commas.
The .out file is the Autofinish output file. This is the most
important file to examine while you are evaluating Autofinish. If you
want to know *why* Autofinish picked the reads it did, it will tell
you. Consult this file before you start complaining about
Autofinish's choices. I've had people complain, and then, once they
look in the .out file (*not* any of the other files), they learn
information that persuades them that Autofinish was correct all along.
This is hard to over-emphasize, but I will try:
CONSULT THE .out FILE CAREFULLY IF YOU DISAGREE WITH ANY OF AUTOFINISH'S
CHOICES!
It will tell you lots more, such as the orientation of the contigs.
It will also tell you the value of all Autofinish parameters used. If you
try to customize one of the parameters, check in the .out file to be
sure that Autofinish used the value you intended.
The .sorted file gives the reads sorted by contig and position. This
file is useful if you want to find what reads Autofinish suggested for a
particular location. It is *not* useful for understanding *why*
Consed chose a particular read. It is deliberately terse to make it
useful for automation the ordering of reads.
The .nav file is a custom navigation file (see "CUSTOM NAVIGATION" far
below). This file allows a Consed user to just click 'next', 'next',
... to review all of Autofinish's suggestions in context. This is a
great way to quickly and easily review all of the reads suggested by
Autofinish.
This finishing tool is designed to be run in batch after each
assembly. In a high throughput operation, the production people can
make these reads without anyone using Consed to examine the assembly
interactively. Only when Autofinish cannot help you any longer
(generally after 3 or more times of running Autofinish, making the
reads, and re-assembling), must you bring up Consed graphically and
examine the assembly.
We suggest that you write some of your own software to parse the
summary files to automatically order primers and reads. The summary
files (.customPrimers, .univForwards, .univReverses) will not change
much but the .out file is constantly changing, so don't try to parse
it.
AUTOFINISH: MINIMUM NUMBER OF ERRORS FIXED PER READ
122) By default, the minimum number of errors fixed by an experiment is
0.02
Human finishers typically look for low consensus quality
regions--regions that have one or more bases below a particular
quality threshold. However, Autofinish can do better: it can find
regions where the *total* number of errors is greater than some
particular cutoff value. This method can find regions where none of
the bases are low quality, but many are nearly low quality and thus
the total number of errors in the region is high. Autofinish will
also ignore regions that have a very few low quality bases, as long as
the total number of errors is smaller than your cutoff. This is a
better critereon because it is the total number of errors that you are
trying to reduce when finishing--not the number of bases with quality
below some arbitrary cutoff.
Two bases of quality 20 have 0.02 errors (on average). Similarly, 20
bases of quality 30 have 0.02 errors (on average). (Quality values
were explained at the beginning of this document.) Suppose that you
want Autofinish to suggest an additional read for an area that even
just has one quality 20 base. (Be aware that Autofinish will consider
10 quality 30 bases to be just as severe as 1 quality 20 base since,
on average, they will both have precisely the same number of errors:
0.01)
123) EDIT PARAMETERS: HOW TO CHANGE CONSED/AUTOFINISH PARAMETERS
This shows how to change
consed.autoFinishMinNumberOfErrorsFixedByAnExp. To change any other
parameter, follow these same instructions replacing
consed.autoFinishMinNumberOfErrorsFixedByAnExp with the parameter you
want to change.
In the edit_dir directory is a file called ".consedrc" which you will
only see if you use "ls -a" instead of just "ls". In that .consedrc,
add the following line:
consed.autoFinishMinNumberOfErrorsFixedByAnExp: 0.01
You can do this using an editor, such as pico, or you can do it with
Consed. To do it with Consed, bring up consed as follows:
consed -ace autofinish.fasta.screen.ace.1
On the Consed Main Window, point to the "Options" menu, push down the
left mouse button and release on "Edit Consed/Autofinish Parameters".
Up will pop the "Edit Parameters" window. Near the top is
"consed.autoFinishMinNumberOfErrorsFixedByAnExp". Point and click in
the box on the left containing 0.02 just underneath
"consed.autoFinishMinNumberOfErrorsFixedByAnExp". After clicking, the
box outline should turn bold and the cursor should start blinking.
Change the 0.02 to 0.01. Click on "just project" near the bottom of
the window. The box containing 0.01 should turn red indicating that
it is now different than the default. Then click "save". A box
titled "Name of parameter file to write" should pop up. Click "ok".
Note: you can changed more than one of these values before clicking
"save".
When using the Edit Parameters Window, I suggest that you do not click
on the up and down arrows of the vertical scrollbar because these will
scroll by too much. Instead, I suggest you point to the thumb of the
vertical slider, hold down the left mouse button and drag the thumb.
Alternatively, point to the black space above or below the thumb and
click with the left mouse button. You need to try this to understand.
To be sure that everything happened correctly, look at .consedrc file.
It should contain the line:
consed.autoFinishMinNumberOfErrorsFixedByAnExp: 0.01
(If you don't know how to view a file, get a UNIX book and learn the
commands "less", "more", "pico", "vi", or "emacs".)
(Get in the habit of checking .consedrc
after using Consed's Edit Parameter Window.)
AUTOFINISH: MINIMUM NUMBER OF ERRORS FIXED PER READ (continued)
Then run Autofinish again:
consed -ace autofinish.fasta.screen.ace.1 -autofinish
Look at the files just created by typing 'ls -tlr' and look at the
.out file by bringing it up with your favorite UNIX editor. You
should see:
PARAMETERS_CHANGED_FROM_DEFAULTS {
.
consed.autoFinishMinNumberOfErrorsFixedByAnExp: 0.010
.
.
Further down is a section:
PARAMETERS {
! If you want to modify any of these parameters, just cut/paste
! the relevant line into your ~/.consedrc file
! (or into the edit_dir/.consedrc file)
! In the following, I have annotated the parameters with the following
! symbols:
!
! (YES) freely customize to your own site
! (OK) don't change unless you have a specific need and know what you
! are doing
! (NO) don't change this!
This section contains all Autofinish parameters, whether you have
changed them or not. Thus a changed parameter will be in both lists.
Find
consed.autoFinishMinNumberOfErrorsFixedByAnExp: 0.010
in this second list.
Then compare the .sorted files from this run of Autofinish and the
previous run of Autofinish in which the
consed.autoFinishMinNumberOfErrorsFixedByAnExp value 0.02 You will
notice that there are 2 additional reads suggested when the parameter
is 0.01. There is a resequence with dye terminator chemistry of the
djs228_474 template and a de novo reverse on template djs228_2632.
Look at the .out file to see why Autofinish chose these reads. It
will indicate that the first read is mainly to fix 0.01 errors in the
region from 2536 to 2545 and the second read to mainly fix 0.01 errors
from 969 to 978.
Bring up Consed to see what is in the 10 base region from 2536 to
2545. You will see that there is a quality 25 base at 2539 and a
quality 21 base at 2540. After that come some bases whose qualities
are in the high 30s.
In the Aligned Reads Window, point at the Misc menu, hold down the
left mouse button, and release on Show Error for a Region. Enter 2539
and 2549 for the "Left Consensus Position of Region" and "Right
Consensus Position of Region" respectively and click on "Calculate".
You will see that there are .0135 errors in this region. This is less
than 0.02 so Autofinish will not try to fix this region unless you
reduce consed.autoFinishMinNumberOfErrorsFixedByAnExp to 0.01
The default is 0.02 because most labs do not want to fix regions that
have less than 0.02 errors.
124) DIVERSION: UNIX LESSON
Note for UNIX novices: Earlier, I said that you only needed to know 3
UNIX commands: pwd, ls, and cd. Then I added "ls -a", "less" and an
editor (such as pico). Now I want you to learn one more:
ls -tlr
This is the same as ls, but it puts one file on a line and prints the
lines so that the most recent files are on the bottom. Since you will
be creating many, many files as you work through these Autofinish
exercises, this command gives an easy way to see the files you have
just created, without having to always look at autofinish.fof to look
for the names of the files you just created.
AUTOFINISH: CHANGING MELTING TEMPERATURES
125) Use 'ls -tlr' to find the most recent .out file. Search in the
.out file (using your favorite editor) for MeltingTemp and you will
find the following lines:
consed.primersMinMeltingTemp: 55
consed.primersMaxMeltingTemp: 60
Some labs prefer to use primers with lower melting temperatures. In
your .consedrc file, put the following lines:
consed.primersMinMeltingTemp: 50
consed.primersMaxMeltingTemp: 55
You can do this by following the instructions above under HOW TO
CHANGE CONSED/AUTOFINISH PARAMETERS. When you are done doing that,
look in the .consedrc file to make sure it contains the above 2 lines.
Then run Autofinish again:
consed -ace autofinish.fasta.screen.ace.1 -autofinish
Using your favorite editor, check that the .out file you just
created says:
consed.primersMinMeltingTemp: 50
consed.primersMaxMeltingTemp: 55
(You can find the most recent .out file by typing 'ls -tlr'.)
Compare the .sorted files from this run of Autofinish and the previous
run. The difference should be the custom primer read:
The previous .sorted file had:
tcttttgtctttccatatacatttt,56
which means the melting temperature is 56.
The latest .sorted file had:
cattttagaatcagtttgttg,50
which means the melting temperature is 50.
126) AUTOFINISH: JUST CLOSING GAPS
You could use Autofinish to just close gaps (you are not interested in
fixing single subclone regions or weak regions). Add the following
to the .consedrc file (and remove everything else so that Autofinish
uses the default values for everything else):
consed.autoFinishCoverLowConsensusQualityRegions: false
consed.autoFinishCoverSingleSubcloneRegions: false
If you are using the Edit Parameter Window to change these values, you
will find them when scrolling about 1/3 way down. Change the
consed.primersMinMeltingTemp and consed.primersMaxMeltingTemp back to
their original values. Then check the .consedrc to make sure it
contains the above 2 lines. (Get in the habit of checking .consedrc
after using Consed's Edit Parameter Window.)
Now you should see in the .sorted file just 4 reads: one custom
primer read pointing out the left end of the contig and 3 reverses off
the left end of the contig. The right end is not extended because
Autofinish recognizes that it is the end of the BAC.
You can change any of the parameters listed at the top of the
Autofinish output file (or actually any of the more exhaustive list of
parameters listed in the 'Info' menu, 'Show Consed Parameters' list.)
We believe the defaults are an excellent starting point.
127) AUTOFINISH: JUST CLOSING GAPS JUST USING WALKS
One high-throughput operation was only interested in closing gaps and
only interested in using walks to close those gaps. This is the
appropriate set of Autofinish parameters to do this:
consed.autoFinishCoverSingleSubcloneRegions: false
consed.autoFinishCoverLowConsensusQualityRegions: false
consed.autoFinishAllowDeNovoUniversalPrimerSubcloneReads: false
consed.autoFinishAllowPCR: false
consed.autoFinishAllowResequencingReads: false
consed.autoFinishAllowMinilibraries: false
consed.autoFinishNearGapsSuggestEachMissingReadOfReadPairs: false
consed.autoFinishCallReversesToFlankGaps: false
(and every other parameter left the default value).
The first 2 parameters are the same as the "AUTOFINISH: JUST CLOSING
GAPS" section (above). The other parameters tell Autofinish all of
the types of reactions it is not allowed to use, leaving just walks.
Try this. Now you should see only a single read, a walk, pointing
left off the left end of the contig.
128) AUTOFINISH: NOT REPEATING FAILED EXPERIMENTS
For this exercise, keep a backup copy of the ace file:
cp autofinish.fasta.screen.ace.1 autofinish.fasta.screen.ace.1.save
If you run Autofinish with the -doExperiments parameter (see below),
-doExperiments causes Autofinish to record its suggestions in the ace
file (hence changing the ace file). If one of these suggested reads
fails to fix a problem, when Autofinish is run again it won't pick the
same read again.
consed -ace (ace file name) -autofinish -doExperiments
If a forward or reverse universal primer read failed, Autofinish (when
run in a subsequent round) will not suggest that same experiment. If
a custom primer read fails, Autofinish will not pick that same
experiment again, and it won't pick a custom primer read that is even
close to the failed one. 'Close' is defined by the parameter:
consed.autoFinishNewCustomPrimerReadThisFarFromOldCustomPrimerRead: 50
In addition, Autofinish (the next time it is run) will tell you how
well each experiment did in solving the problem it was intended to
solve.
Return the parameters to the defaults and try this by running
Autofinish twice like this:
consed -ace autofinish.fasta.screen.ace.1 -autofinish -doExperiments
consed -ace autofinish.fasta.screen.ace.1 -autofinish -doExperiments
and look at the .out file from the 2nd run. (You can find the most
recent .out file by typing 'ls -tlr'.) You should see lines such
as this:
rejecting experiment: reverse universal primer read with template djs228_1094
because an earlier round of autofinish called this with expid: 1
rejecting experiment: reverse universal primer read with template djs228_1422
because an earlier round of autofinish called this with expid: 2
rejecting experiment: reverse universal primer read with template djs228_1034
because an earlier round of autofinish called this with expid: 3
This is Autofinish trying experiment after experiment but finding they
were already suggested in an earlier round of Autofinish.
You should not type '-doExperiments' if you do not intend to do the
experiments Autofinish suggests. If you use -doExperiments, but you
don't really do the experiments, and then you run Autofinish again,
Autofinish will be very upset--it will think that all of its suggested
experiments failed (because it can't find them). It will see that all
of the problems are still present but it will think that it should not
choose any of those same experiments again so it will suggest
different experiments that will not be as good as its original
suggestions.
-doExperiments will also cause oligos to be tagged.
Primer id's created by Autofinish use the same naming scheme as
primers created in Consed and they will not conflict with each other.
For example, if Autofinish creates oligos djs14.1, djs14.2, and
djs14.3, then the next primer that a user accepts will be djs14.4. If
Autofinish is run a second time, it will start with primer djs14.5.
When you have completed this exercise with -doExperiments, replace the
original .ace file by typing:
cp autofinish.fasta.screen.ace.1.save autofinish.fasta.screen.ace.1
129) AUTOFINISH: doNotFinish particular regions
If there is a region that you don't care to finish (e.g., it has
already been finished by an overlapping clone or you know there is no
gene there), then you can put a doNotFinish tag on the consensus and
Autofinish will not try to finish this area.
First, delete the .consedrc file (or, if you are using the Edit
Parameter Window of Consed, restore the parameters to their default
values) and run Autofinish again:
consed -ace autofinish.fasta.screen.ace.1 -autofinish
Bring up consed:
consed -ace autofinish.fasta.screen.ace.1
and put a doNotFinish tag on the region from 2000 to 4000. (If you
don't know how to do that, read through the Consed Quick Tour, above.)
Save the assembly as autofinish.fasta.screen.ace.2
Run Autofinish again:
consed -ace autofinish.fasta.screen.ace.2 -autofinish
Look at the .out files for each of the 2 runs of Autofinish. (You can
find the most recent .out files by typing 'ls -tlr'.) You will notice
in the .out file for the 2nd run of Autofinish that, in the other than
the experiments to extend the contig to the left, there is only one
experiment which is from 315 to 1662. If you find that experiment in
the .out file, it will say Contig1 0.05 errors fixed in region from
315 to 1662 fixing 0.05 errors from 969 to 978
The "969 to 978" gives the worst 10 base window that the read is
intended to fix. If you look with Consed, you will see that
there is a quality 12 base at 974.
You can also use doNotFinish tags to prevent Autofinish from
*extending* a contig into a gap by putting a doNotFinish tag near the
end of the contig and setting the following Autofinish parameters:
consed.autoFinishDoNotExtendContigsWhereTheseTagsAre: doNotFinish
consed.autoFinishDoNotExtendContigsIfTagsAreThisCloseToContigEnd: 50
130) AUTOFINISH: NOT USING PARTICULAR SUBCLONE TEMPLATES
If you no longer have a template that was used in shotgun, and thus
you don't want Autofinish to pick that template, you can put it in a
file badTemplates.txt in edit_dir. This is a simple file with one
name per line.
Using your favorite UNIX editor, create a file called
"badTemplates.txt" in edit_dir. Make it contain a single line:
djs228_1094
Delete .consedrc (or, if you are using the Edit Parameter Window,
restore the parameters to their defaults) and run autofinish again:
consed -ace autofinish.fasta.screen.ace.1 -autofinish
Search the .out file for djs228_1094. You will find one line like
this:
not using template: djs228_1094 because in bad templates file
Now try deleting badTemplates.txt and running autofinish again the
same way. You will notice there are many differences in reads chosen,
since djs22_1094 is now available again for making reverses as well as
a template for custom primer walks.
badTemplates.txt can accept "*" (match any characters) as part of the
name. For example, djs140_23* will eliminate templates:
djs140_235684
djs140_235783
djs140_2326
etc.
131) AUTOFINISH: NOT USING ENTIRE LIBRARIES FOR FINISHING
In addition to the badTemplates.txt file, you can use a
badLibraries.txt file which contains a list of all libraries that are
off-limits to Autofinish (e.g., you threw away all subclone templates
from this library or they are from a different lab which gave you the
chromatograms but not the templates). Autofinish determines the
library of a read by the following in the PHD file:
WR{
template dscript 990603:090231
name: djs366_101
lib: library1
}
where "library1" is replaced by the actual library name. Take a look
at any phd file in autofinish/phd_dir and you will see this.
Make sure that badTemplates.txt is deleted and .consedrc is either
deleted (or use the Edit Parameter Window to restore the defaults) and
run Autofinish again.
consed -ace autofinish.fasta.screen.ace.1 -autofinish
Now create a file badLibraries.txt containing a single line:
lib1
and run autofinish again:
consed -ace autofinish.fasta.screen.ace.1 -autofinish
Look at the .out file. You will see lines like this:
not using template: djs228_1034 because in bad libraries file
not using template: djs228_1051 because in bad libraries file
not using template: djs228_1094 because in bad libraries file
.
.
.
You will see that there are no reads suggested that use any of these
templates, even though some of them (e.g.., djs228_1034) were used in
the Autofinish run (above) before you created the badLibraries.txt
file.
When you start doing this with your own data, you must put the lib:
line into your phd files. Do this by modifying determineReadTypes.perl.
132) MULTIPLE LIBRARIES WITH DIFFERENT INSERT SIZES
If different libraries have different insert sizes, Autofinish must
know the insert size of each library. If there are 5 or more
forward-reverse pairs, where the forward and reverse are both in the
same contig, then Consed/Autofinish calculates the insert size of the
library by finding the mean and standard deviation of the insert sizes
of these forward-reverse pairs. The maximum insert size of the
library is set at the mean plus 2.5 times the standard deviation.
If there are fewer than 5 forward-reverse pairs, where the forward and
reverse are both in the same contig, Consed/Autofinish considers this
statistical information unreliable so instead relies on a
file called 'librariesInfo.txt" which must be placed in edit_dir
(where the ace file is). This file looks like this:
LIB{
name: lib0
avgInsertSize: 1500
maxInsertSize: 3000
stranded: double
cost: 600.0
}
LIB{
name: lib1
avgInsertSize: 3000
maxInsertSize: 5000
stranded: double
cost: 1000.0
}
LIB{
name: lib2
avgInsertSize: 10000
maxInsertSize: 12000
stranded: double
cost: 5000.0
}
'name' is the name of the library. This is the name that goes into
the PHD file after the 'lib:' keyword (see AUTOFINISH: NOT USING
ENTIRE LIBRARIES FOR FINISHING above). 'avgInsertSize' is the
average insert size of the library--the figure to be used by
Autofinish if there are not enough forward/reverse pairs for
Autofinish to calculate the mean insert size of the library.
'maxInsertSize' is the maximum insert size--if forward/reverse pairs
are further apart than this, Autofinish will assume these reads are
misassembled. 'stranded' is whether this template is single or double
stranded. 'cost' is the cost of making a minilibrary out of a
template from this library.
In .consedrc, there must be a line like this:
consed.primersMaxInsertSizeOfASubclone: 5000
where 5000 is replaced by whatever the maximum insert size of all of
your different libraries.
For this exercise make .consedrc have a single line:
consed.primersMaxInsertSizeOfASubclone: 12000
Alternatively, use the Edit Parameter Window to set
consed.primersMaxInsertSizeOfASubclone to 12000.
For this exercise I have a file in edit_dir called
"librariesInfo.txt_hide". To make Autofinish pay attention to it, do
the following:
cp librariesInfo.txt_hide librariesInfo.txt
Delete badLibraries.txt:
rm badLibraries.txt
Before you run Autofinish again, first restart Consed:
consed -ace autofinish.fasta.screen.ace.1
On Consed's Main Window, point to 'Info', hold down the left mouse
button, and release on 'Show Library Info'. You should see the names
of your libraries and the correct number of reads in each library.
This feature will be useful in debugging your use of librariesInfo.txt
Then run Autofinish again:
consed -ace autofinish.fasta.screen.ace.1 -autofinish
Look at the .out file. Look for the following:
"Choosing de novo universal primer reads to try to close gaps"
You will see there are many reads under this heading. These are
the lib1 and lib2 reads that have a large average insert size and thus
span the gap. Autofinish did not choose some of these reads before
because, if the insert size were only 1500 bases, these reads would
not have helped to close the gap.
When you are done with this exercise, delete librariesInfo.txt and
.consedrc
When there are many reads from the same library, Consed/Autofinish
will look at the forward/reverse pairs that are within the same contig
(so the insert size of that template can be directly measured) and do
figure out the mean and standard deviation of the insert size of
templates from that library. Consed/Autofinish will use these numbers
rather than the number from librariesInfo.txt
133) AUTOFINISH CLOSING GAPS WITH MINILIBRARIES
If you wanted Autofinish to *only* suggest minilibraries to close
gaps, use the following parameters:
consed.autoFinishAllowWholeCloneReads: false
consed.autoFinishAllowCustomPrimerSubcloneReads: false
consed.autoFinishAllowResequencingReads: false
consed.autoFinishAllowDeNovoUniversalPrimerSubcloneReads: false
consed.autoFinishAllowPCR: false
consed.autoFinishAllowResequencingAUniversalPrimerAutofinishRead: false
consed.autoFinishCallReversesToFlankGaps: false
consed.autoFinishAllowMinilibraries: true
consed.autoFinishAlwaysCloseGapsUsingMinilibraries: true
consed.autoFinishPrintMinilibrariesSummaryFile: true
For this exercise, type:
cd assembly_view/edit_dir
(You might need to first type "cd ../.." depending on where you are.)
Attention! This is *not* the same directory you have been using. It,
autofinish/edit_dir, does not have any gaps so it cannot be used for
this exercise.
Create a .consedrc file with the parameters above in it.
Alternatively, start Consed in this directory and use the Edit
Parameter Window to modify the parameters as above. Then run
Autofinish:
consed -ace assembly_view.fasta.screen.ace.1 -autofinish
When it has completed, look in the .out file. You will see the
following:
Enough existing fwd/rev pairs to establish:
Left end of Contig3 has 13 fwd/rev pairs connecting it to
Right end of Contig2 with gap size -460 (contigs overlap)
Trying to suggest minilibrary for gap between right end of Contig2 and left end of
Contig3
MINILIBRARY{
best template: djs736a2_fp04q274 from lib djs736a2
size: 3607 errors fixed: 0.01 errors fixed per dollar: 0.00
connecting right end of Contig2 to left end of Contig3 with estimated gap size -460
alternative template: djs736a1_fp02q472 from lib djs736a1
size: 1184 errors fixed: 0.01 errors fixed per dollar: 0.00
}
You will also see a more terse (but more easily parseable) description
in the .minilibraries file.
The parameter:
consed.autoFinishPrintMinilibrariesSummaryFile: true
will cause Autofinish to print a file with name similar to:
(project name).001014.155627.minilibraries
Or you could be more sparing in which gaps you close with
minilibraries and which you do not:
consed.autoFinishAlwaysCloseGapsUsingMinilibraries: false
If the parameter above is set to false, then Autofinish will only
choose minilibraries if the gap is the size below or larger:
consed.autoFinishSuggestMinilibraryIfGapThisManyBasesOrLarger: 800
If you try this in this example, you will see that Autofinish will not
suggest a minilibrary because the ga