Gene-Ontologies and Annotation
In the early days of microarray analysis, people were happy if they got a handful of differentially-expressed genes that they could validate or follow-up. However, with later technologies (and depending on the experimental setup) we might have thousands of statistically-significant results, which no-one has the time to follow-up. Also, we might be interested in pathways / mechanisms that are altered and not just individual genes.
In this section we move towards discovering if our results are biologically significant. Are the genes that we have picked statistical flukes, or are there some commonalities.
There are two different approaches one might use, and we will cover the theory behind both. The distinction is whether you are happy to use a hard (and arbitrary) threshold to identify DE genes.
Threshold-free analysis
This type of analyis is popular for datasets where differential expression analysis does not reveal many genes that are differentially-expressed on their own. Instead, it seeks to identify genes that as a group have a tendancy to be near the extremes of the log-fold changes.
The Broad institute has made this analyis method popular and provides a version of GSEA that can be run via a java application.
However, we are going to use an open-source version of the algorithm (fgsea) that is implmented through Galaxy.
This method requires a sorted list of genes, so we will use a tool within Galaxy to sort our background.tsv
Degust results by log fold change
Filter and Sort -> Sort Text Manipulation -> Cut
- Use Filter and Sort -> Sort to sort the
background.tsv
on the column containing the log fold change. Make sure the number of header lines is set to 1.
- The use the Text Manipulation -> Cut tool to cut columns
c1
and c3
from the output.
You will also need a set of pre-defined gene-sets to apply this method. These can be obtained from the MSigDB website for Human. Alternatively, the WEHI has made Human and Mouse versions available on their website. If you are using a different organism, you may have to search around for equivalent files or create your own.
Go to the WEHI website and download the H hallmark gene sets (rdata file) for Mouse
. Upload this to Galaxy using the Get Data -> Upload File tool.
The fgsea
tool can now be run as follows:-
Annotation -> fgsea - fast preranked gene set
- Ranked Genes:- Your sorted and filtered file
- Genes sets:- The rdata file you Downloaded from the WEHI.
- Output plots:- Yes
Over-representation analysis
“Threshold-based” methods require defintion of a statistical threshold to define list of genes to test (e.g. FDR < 0.01). Then a hypergeometric test or Fisher’s Exact test generally used. They are typically used in situations where plenty of DE genes have been identified, and people often use quite relaxed criteria for identifying DE genes (e.g. raw rather than adjusted p-values or FDR value)
The question we are asking here is;
“Are the number of DE genes associated with Theme X significantly greater than what we might expect by chance alone?”
We can answer this question by knowing
- the total number of DE genes
- the number of genes in the gene set (pathway or process)
- the number of genes in the gene set that are found to be DE
- the total number of tested genes (background)
The formula for Fishers exact test is;
\[ p = \frac{\binom{a + b}{a}\binom{c +d}{c}}{\binom{n}{a +c}} = \frac{(a+b)!(c+d)!(a+c)!(b+d)!}{a!b!c!d!n!} \]
with:-
In.DE.List Not.in.DE.list RowTotal
In Gene Set a b a +b
Not in Gene Set c d c+d
Column Total a+c b+d a+b+c+d (=n)
In this first test, our genes will be grouped together according to their Gene Ontology (GO) terms:- http://www.geneontology.org/
goseq
Using GOrilla
There are several popular online tools for performing enrichment analysis
We will be using the online tool GOrilla to perform the pathways analysis. It has two modes; the first of which accepts a list of background and target genes. However, the input IDs need to be Gene Symbols.
Exercise: Download your annotated background.tsv file. This will act as the background list for the analysis. Now use the annotateMyIDs to convert the Entrez IDs in the file B.preg_vs_lactation.tsv
into gene symbols and download the result.
- Go to http://cbl-gorilla.cs.technion.ac.il/
- Read the “Running Example”
- Choose Organism:
Mus Musculus
- Choose running mode:
Two unranked lists of genes
- Paste the gene symbols corresponding to DE genes into the Target set.
- Paste the gene symbols from the Background set into the other box.
- Choose an Ontology:
Process
Search Enriched GO terms
You should be presented with a graph of enriched GO terms showing the relationship between the terms. Each GO term is coloured according to its statistical significance.
Below the figure is the results table. This links to more information about each GO term, and lists each gene in the category that was found in your list. The enrichment column gives 4 numbers that are used to determine enrichment (similar to the Fisher exact test we saw earlier)
- N, total number of genes (should be the same in all rows)
- B, total number of genes annotated with the GO term
- n, total number of genes that were found in the list you uploaded (same for all rows)
- b, number of genes in the list you uploaded that intersect with this GO term
If you have time, you can also experiment uploading the same genes lists to the online tools DAVID and GeneTrail
Using EnrichR
Question: EnrichR is another online tool for performing enrichment analysis against a large collection of databases. Go to the website, and paste-in your list of differentially-expressed genes. Explore the results that EnrichR provides
LS0tCnRpdGxlOiAiT250b2xvZ2llcyBhbmQgRW5yaWNobWVudCIKYXV0aG9yOiAiTWFyayBEdW5uaW5nIgpkYXRlOiAnYHIgZm9ybWF0KFN5cy50aW1lKCksICJMYXN0IG1vZGlmaWVkOiAlZCAlYiAlWSIpYCcKb3V0cHV0OiAKICBodG1sX25vdGVib29rOiAKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwotLS0KCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFfQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IEZBTFNFKQpgYGAKCgpgYGB7ciBlY2hvPUZBTFNFLG1lc3NhZ2U9RkFMU0V9CmxpYnJhcnkobGltbWEpCmBgYAoKIyBSZWNhcAoKSW4gdGhpcyBzZWN0aW9uIHdlIHdpbGwgdXNlIHRoZSBmb2xsb3dpbmcgdGhyZWUgZmlsZXMKCi0gYGJhY2tncm91bmQudHN2YCBjb250YWluaW5nIG9uZSByb3cgZm9yIGVhY2ggZ2VuZSBpbiB0aGUgY29tcGFyaXNvbiBCYXNhbC5wcmVnbmFudCB2cyBCYXNhbC5sYWN0YXRpb24gKDI3LDE3OSByb3dzKS4KLSBgQi5wcmVnX3ZzX2xhY3RhdGlvbi50c3ZgIGNvbnRhaW5pbmcgb25lIHJvdyBmb3IgZWFjaCBmb3VuZCB0byBiZSBERSBpbiB0aGUgY29udHJhc3QgQmFzYWwucHJlZ25hbnQgdnMgQmFzYWwuTGFjdGF0aW9uLgotIGBvdmVybGFwLWdlbmVzLnRzdmAgdGhlIGdlbmVzIHdlIGlkZW50aWZpZWQgdXNpbmcgKnZlbm55KiBhcyBiZWluZyBERSBpbiBib3RoIEJhc2FsIGFuZCBMdW1pbmFsIGNlbGwgdHlwZXMuCgo8ZGl2IGNsYXNzPSJhbGVydCBhbGVydC1pbmZvIj4KVGhlc2UgZmlsZXMgYXJlIGF2YWlsYWJsZSBmcm9tIHRoZSBnb29nbGUgZHJpdmUgaWYgeW91IGhhdmUgcHJvYmxlbXMuCjwvZGl2PgoKCiMgQWRkaW5nIGV4dHJhIGFubm90YXRpb24gdG8gcmVzdWx0cwoKVGhlIHJlc3VsdHMgZnJvbSBkZWd1c3QgaGF2ZSAqRW50cmV6IElEKiBpbiB0aGUgZmlyc3QgY29sdW1uLCBidXQgbm8gb3RoZXIgdXNlZnVsIGFubm90YXRpb24uIFNvbWV0aW1lcyB3ZSBtaWdodCB3YW50IG90aGVyIElEcyB0byBiZSBhZGRlZCBpbiBvcmRlciB0byBpbnRlcnByZXQgb3VyIHJlc3VsdHMuIEluZGl2aWR1YWwgcXVlcmllcyBjYW4gYmUgbWFkZSBvbmxpbmUgKGUuZy4gRW5zZW1ibCwgYmlvbWFydCkgYnV0IHRlZGlvdXMgZm9yIGxhcmdlIG51bWJlcnMgb2YgZ2VuZXMuIFRoZSBHYWxheHkgdG9vbCAqKmFubm90YXRlTXlJRHMqKiBpcyBhIHNpbXBsZSB0b29sIGZvciBhbm5vdGF0aW5nIGEgZmlsZSBjb250YWluaW5nIGEgY29sdW1uIG9mIElEcy4KCldlIGFyZSBnb2luZyB0byB1c2UgdGhpcyB0byBnZXQgbW9yZSBpbmZvcm1hdGlvbiBvbiB0aGUgZ2VuZXMgd2UgaWRlbnRpZmllZCBhcyBiZWluZyBpbiBjb21tb24gYmV0d2VlbiBCYXNhbCBhbmQgTHVtaW5hbCBjZWxscyBpbiB0aGUgY29udHJhc3QgcHJlZ25hbnQgdnMgbGFjdGF0aW9uCgoKCjxkaXYgY2xhc3M9ImFsZXJ0IGFsZXJ0LWluZm8iPgoqKkFubm90YXRpb24qKiAtPiAqKmFubm90YXRlTXlJRHMqKgo8L2Rpdj4KCi0gRmlsZSB3aXRoIElEczogZ2VuZS1vdmVybGFwLmNzdgotIE9yZ2FuaXNtOiBNb3VzZQotIElEIFR5cGU6IEVudHJleklECi0gS2VlcCBhbGwgb3RoZXIgZGVmYXVsdHMKClRoaXMgc2hvdWxkIHByb2R1Y2UgYW4gb3V0cHV0IHRhYmxlIGNvbnRhaW5pbmcgdGhlIG9yaWdpbmFsIElELCBhbmQgdGhlIGVxdWl2bGVudCBHZW5lIFN5bWJvbCwgRW5zZW1ibCBJRC4KCjxkaXYgY2xhc3M9ImFsZXJ0IGFsZXJ0LXdhcm5pbmciPgoqKlF1ZXN0aW9uKio6IFVzZSB0aGUgR2V0IERhdGEgdG9vbCB0byB1cGxvYWQgdGhlIGZpbGUgYGJhY2tncm91bmQudHN2YCB0byBHYWxheHkgYW5kIHByb2R1Y2UgYW4gYW5ub3RhdGlvbiB0YWJsZS4KPC9kaXY+CgoKVGhlIG91dHB1dCBjYW4gdGhlbiBiZSAqam9pbmVkKiB0byB0aGUgb3JpZ2luYWwgcmVzdWx0cyBmaWxlIHRvIHByb2R1Y2UgYSBtb3JlIGRldGFpbGVkIHJlc3VsdAoKPGRpdiBjbGFzcz0iYWxlcnQgYWxlcnQtaW5mbyI+CioqVGV4dCBNYW5pcHVsYXRpb24qKiAtPiAqKkpvaW4gdHdvIGZpbGVzKioKPC9kaXY+CgotIDFzdCBmaWxlOiBiYWNrZ3JvdW5kLnRzdgotIENvbHVtbiB0byB1c2UgZnJvbSAxc3QgZmlsZTogQ29sdW1uIDEKLSAybmQgZmlsZTogcmVzdWx0IGZyb20gKmFubm90YXRlTXlJRHMqCi0gQ29sdW1uIHRvIHVzZSBmcm9tIDJuZCBmaWxlOiBDb2x1bW4gMQoKPGRpdiBjbGFzcz0iYWxlcnQgYWxlcnQtaW5mbyI+ClRoZSBzdGVwIG9mIGFubm90YXRpbmcgeW91ciByZXN1bHRzIG1heSBuZWVkIG1vZGlmeWluZyAob3IgbWF5IG5vdCBiZSBuZWNlc3NhcnkpIGRlcGVuZGluZyBvbiB3aGF0IElEcyB5b3UgaGF2ZSB1c2VkIGluIHRoZSBjb3VudGluZy4gTWFrZSBzdXJlIHlvdSBjaG9vc2UgdGhlIG9wdGlvbnMgY2FyZWZ1bGx5IHdoZW4gdXNpbmcgdGhpcyB0b29sLgo8L2Rpdj4KCiMgR2VuZS1PbnRvbG9naWVzIGFuZCBBbm5vdGF0aW9uCgoKSW4gdGhlIGVhcmx5IGRheXMgb2YgbWljcm9hcnJheSBhbmFseXNpcywgcGVvcGxlIHdlcmUgaGFwcHkgaWYgdGhleSBnb3QgYSBoYW5kZnVsIG9mIGRpZmZlcmVudGlhbGx5LWV4cHJlc3NlZCBnZW5lcyB0aGF0IHRoZXkgY291bGQgdmFsaWRhdGUgb3IgZm9sbG93LXVwLiBIb3dldmVyLCB3aXRoIGxhdGVyIHRlY2hub2xvZ2llcyAoYW5kIGRlcGVuZGluZyBvbiB0aGUgZXhwZXJpbWVudGFsIHNldHVwKSB3ZSBtaWdodCBoYXZlIHRob3VzYW5kcyBvZiBzdGF0aXN0aWNhbGx5LXNpZ25pZmljYW50IHJlc3VsdHMsIHdoaWNoIG5vLW9uZSBoYXMgdGhlIHRpbWUgdG8gZm9sbG93LXVwLiBBbHNvLCB3ZSBtaWdodCBiZSBpbnRlcmVzdGVkIGluIHBhdGh3YXlzIC8gbWVjaGFuaXNtcyB0aGF0IGFyZSBhbHRlcmVkIGFuZCBub3QganVzdCBpbmRpdmlkdWFsIGdlbmVzLgoKSW4gdGhpcyBzZWN0aW9uIHdlIG1vdmUgdG93YXJkcyBkaXNjb3ZlcmluZyBpZiBvdXIgcmVzdWx0cyBhcmUgKioqYmlvbG9naWNhbGx5IHNpZ25pZmljYW50KioqLiBBcmUgdGhlIGdlbmVzIHRoYXQgd2UgaGF2ZSBwaWNrZWQgc3RhdGlzdGljYWwgZmx1a2VzLCBvciBhcmUgdGhlcmUgc29tZSBjb21tb25hbGl0aWVzLiAKClRoZXJlIGFyZSB0d28gZGlmZmVyZW50IGFwcHJvYWNoZXMgb25lIG1pZ2h0IHVzZSwgYW5kIHdlIHdpbGwgY292ZXIgdGhlIHRoZW9yeSBiZWhpbmQgYm90aC4gVGhlIGRpc3RpbmN0aW9uIGlzIHdoZXRoZXIgeW91IGFyZSBoYXBweSB0byB1c2UgYSBoYXJkIChhbmQgYXJiaXRyYXJ5KSB0aHJlc2hvbGQgdG8gaWRlbnRpZnkgREUgZ2VuZXMuCgojIyBUaHJlc2hvbGQtZnJlZSBhbmFseXNpcwoKVGhpcyB0eXBlIG9mIGFuYWx5aXMgaXMgcG9wdWxhciBmb3IgZGF0YXNldHMgd2hlcmUgZGlmZmVyZW50aWFsIGV4cHJlc3Npb24gYW5hbHlzaXMgZG9lcyBub3QgcmV2ZWFsIG1hbnkgZ2VuZXMgdGhhdCBhcmUgZGlmZmVyZW50aWFsbHktZXhwcmVzc2VkIG9uIHRoZWlyIG93bi4gSW5zdGVhZCwgaXQgc2Vla3MgdG8gaWRlbnRpZnkgZ2VuZXMgdGhhdCBhcyBhIGdyb3VwIGhhdmUgYSB0ZW5kYW5jeSB0byBiZSBuZWFyIHRoZSBleHRyZW1lcyBvZiB0aGUgbG9nLWZvbGQgY2hhbmdlcy4KClRoZSBCcm9hZCBpbnN0aXR1dGUgaGFzIG1hZGUgdGhpcyBhbmFseWlzIG1ldGhvZCBwb3B1bGFyIGFuZCBwcm92aWRlcyBbYSB2ZXJzaW9uIG9mIEdTRUFdKGh0dHA6Ly9zb2Z0d2FyZS5icm9hZGluc3RpdHV0ZS5vcmcvZ3NlYS9pbmRleC5qc3ApIHRoYXQgY2FuIGJlIHJ1biB2aWEgYSBqYXZhIGFwcGxpY2F0aW9uLiAKCiFbXShtZWRpYS9vdmVyZXhwcmVzc2VkLWdzZWEucG5nKQoKSG93ZXZlciwgd2UgYXJlIGdvaW5nIHRvIHVzZSBhbiBvcGVuLXNvdXJjZSB2ZXJzaW9uIG9mIHRoZSBhbGdvcml0aG0gKGZnc2VhKSB0aGF0IGlzIGltcGxtZW50ZWQgdGhyb3VnaCBHYWxheHkuIAoKVGhpcyBtZXRob2QgcmVxdWlyZXMgYSBzb3J0ZWQgbGlzdCBvZiBnZW5lcywgc28gd2Ugd2lsbCB1c2UgYSB0b29sIHdpdGhpbiBHYWxheHkgdG8gc29ydCBvdXIgYCBiYWNrZ3JvdW5kLnRzdmAgRGVndXN0IHJlc3VsdHMgYnkgbG9nIGZvbGQgY2hhbmdlCgo8ZGl2IGNsYXNzPSJhbGVydCBhbGVydC1pbmZvIj4KKipGaWx0ZXIgYW5kIFNvcnQqKiAtPiAqKlNvcnQqKgoqKlRleHQgTWFuaXB1bGF0aW9uKiogLT4gKipDdXQqKgo8L2Rpdj4KCi0gVXNlICoqRmlsdGVyIGFuZCBTb3J0KiogLT4gKipTb3J0KiogdG8gc29ydCB0aGUgYGJhY2tncm91bmQudHN2YCBvbiB0aGUgY29sdW1uIGNvbnRhaW5pbmcgdGhlIGxvZyBmb2xkIGNoYW5nZS4gKk1ha2Ugc3VyZSB0aGUgbnVtYmVyIG9mIGhlYWRlciBsaW5lcyBpcyBzZXQgdG8gMSouCi0gVGhlIHVzZSB0aGUgKipUZXh0IE1hbmlwdWxhdGlvbioqIC0+ICoqQ3V0KiogdG9vbCB0byBjdXQgY29sdW1ucyBgYzFgIGFuZCBgYzNgIGZyb20gdGhlIG91dHB1dC4KCllvdSB3aWxsIGFsc28gbmVlZCBhIHNldCBvZiBwcmUtZGVmaW5lZCBnZW5lLXNldHMgdG8gYXBwbHkgdGhpcyBtZXRob2QuIFRoZXNlIGNhbiBiZSBvYnRhaW5lZCBmcm9tIHRoZSBbTVNpZ0RCIHdlYnNpdGVdKGh0dHA6Ly9zb2Z0d2FyZS5icm9hZGluc3RpdHV0ZS5vcmcvZ3NlYS9tc2lnZGIvaW5kZXguanNwKSBmb3IgSHVtYW4uIEFsdGVybmF0aXZlbHksIHRoZSBXRUhJIGhhcyBtYWRlIEh1bWFuIGFuZCBNb3VzZSB2ZXJzaW9ucyBhdmFpbGFibGUgb24gW3RoZWlyIHdlYnNpdGVdKGh0dHA6Ly9iaW9pbmYud2VoaS5lZHUuYXUvc29mdHdhcmUvTVNpZ0RCLykuIElmIHlvdSBhcmUgdXNpbmcgYSBkaWZmZXJlbnQgb3JnYW5pc20sIHlvdSBtYXkgaGF2ZSB0byBzZWFyY2ggYXJvdW5kIGZvciBlcXVpdmFsZW50IGZpbGVzIG9yIGNyZWF0ZSB5b3VyIG93bi4KCgo8ZGl2IGNsYXNzPSJhbGVydCBhbGVydC1pbmZvIj4KR28gdG8gdGhlIFtXRUhJIHdlYnNpdGVdKGh0dHA6Ly9iaW9pbmYud2VoaS5lZHUuYXUvc29mdHdhcmUvTVNpZ0RCLykgYW5kIGRvd25sb2FkIHRoZSBgSCBoYWxsbWFyayBnZW5lIHNldHMgKHJkYXRhIGZpbGUpIGZvciBNb3VzZWAuIFVwbG9hZCB0aGlzIHRvIEdhbGF4eSB1c2luZyB0aGUgKipHZXQgRGF0YSAtPiBVcGxvYWQgRmlsZSoqIHRvb2wuCjwvZGl2PgoKVGhlIGBmZ3NlYWAgdG9vbCBjYW4gbm93IGJlIHJ1biBhcyBmb2xsb3dzOi0KCjxkaXYgY2xhc3M9ImFsZXJ0IGFsZXJ0LWluZm8iPgoqKkFubm90YXRpb24qKiAtPiAqKmZnc2VhIC0gZmFzdCBwcmVyYW5rZWQgZ2VuZSBzZXQqKgo8L2Rpdj4KCi0gUmFua2VkIEdlbmVzOi0gWW91ciBzb3J0ZWQgYW5kIGZpbHRlcmVkIGZpbGUKLSBHZW5lcyBzZXRzOi0gVGhlIHJkYXRhIGZpbGUgeW91IERvd25sb2FkZWQgZnJvbSB0aGUgV0VISS4KLSBPdXRwdXQgcGxvdHM6LSBZZXMKCgojIyBPdmVyLXJlcHJlc2VudGF0aW9uIGFuYWx5c2lzCgoiVGhyZXNob2xkLWJhc2VkIiBtZXRob2RzIHJlcXVpcmUgZGVmaW50aW9uIG9mIGEgc3RhdGlzdGljYWwgdGhyZXNob2xkIHRvIGRlZmluZSBsaXN0IG9mIGdlbmVzIHRvIHRlc3QgKGUuZy4gRkRSIDwgMC4wMSkuIFRoZW4gYSAqaHlwZXJnZW9tZXRyaWMqIHRlc3Qgb3IgKkZpc2hlcidzIEV4YWN0KiB0ZXN0IGdlbmVyYWxseSB1c2VkLiBUaGV5IGFyZSB0eXBpY2FsbHkgdXNlZCBpbiBzaXR1YXRpb25zIHdoZXJlIHBsZW50eSBvZiBERSBnZW5lcyBoYXZlIGJlZW4gaWRlbnRpZmllZCwgYW5kIHBlb3BsZSBvZnRlbiB1c2UgcXVpdGUgcmVsYXhlZCBjcml0ZXJpYSBmb3IgaWRlbnRpZnlpbmcgREUgZ2VuZXMgKGUuZy4gcmF3IHJhdGhlciB0aGFuIGFkanVzdGVkIHAtdmFsdWVzIG9yIEZEUiB2YWx1ZSkKClRoZSBxdWVzdGlvbiB3ZSBhcmUgYXNraW5nIGhlcmUgaXM7Cgo+ICoqKiJBcmUgdGhlIG51bWJlciBvZiBERSBnZW5lcyBhc3NvY2lhdGVkIHdpdGggVGhlbWUgWCBzaWduaWZpY2FudGx5IGdyZWF0ZXIgdGhhbiB3aGF0IHdlIG1pZ2h0IGV4cGVjdCBieSBjaGFuY2UgYWxvbmU/IioqKgoKV2UgY2FuIGFuc3dlciB0aGlzIHF1ZXN0aW9uIGJ5IGtub3dpbmcKCi0gdGhlIHRvdGFsIG51bWJlciBvZiBERSBnZW5lcwotIHRoZSBudW1iZXIgb2YgZ2VuZXMgaW4gdGhlIGdlbmUgc2V0IChwYXRod2F5IG9yIHByb2Nlc3MpCi0gdGhlIG51bWJlciBvZiBnZW5lcyBpbiB0aGUgZ2VuZSBzZXQgdGhhdCBhcmUgZm91bmQgdG8gYmUgREUKLSB0aGUgdG90YWwgbnVtYmVyIG9mIHRlc3RlZCBnZW5lcyAoYmFja2dyb3VuZCkKClRoZSBmb3JtdWxhIGZvciBGaXNoZXJzIGV4YWN0IHRlc3QgaXM7CgokJCBwID0gXGZyYWN7XGJpbm9te2EgKyBifXthfVxiaW5vbXtjICtkfXtjfX17XGJpbm9te259e2EgK2N9fSA9IFxmcmFjeyhhK2IpIShjK2QpIShhK2MpIShiK2QpIX17YSFiIWMhZCFuIX0gJCQKCndpdGg6LQoKYGBge3IgZWNobz1GQUxTRX0KZGYgPC0gZGF0YS5mcmFtZShgSW4gREUgTGlzdGAgID0gYygiYSIsImMiLCJhK2MiKSwgYE5vdCBpbiBERSBsaXN0YCA9IGMoImIiLCJkIiwiYitkIiksIFJvd1RvdGFsID0gYygiYSArYiIsImMrZCIsImErYitjK2QgKD1uKSIpKQpyb3duYW1lcyhkZikgPC0gYygiSW4gR2VuZSBTZXQiLCAiTm90IGluIEdlbmUgU2V0IiwiQ29sdW1uIFRvdGFsIikKZGYKYGBgCgpJbiB0aGlzIGZpcnN0IHRlc3QsIG91ciBnZW5lcyB3aWxsIGJlIGdyb3VwZWQgdG9nZXRoZXIgYWNjb3JkaW5nIHRvIHRoZWlyIEdlbmUgT250b2xvZ3kgKEdPKSB0ZXJtczotIGh0dHA6Ly93d3cuZ2VuZW9udG9sb2d5Lm9yZy8KCgojIyBnb3NlcQoKCgojIyBVc2luZyBHT3JpbGxhCgpUaGVyZSBhcmUgc2V2ZXJhbCBwb3B1bGFyIG9ubGluZSB0b29scyBmb3IgcGVyZm9ybWluZyBlbnJpY2htZW50IGFuYWx5c2lzCgpXZSB3aWxsIGJlIHVzaW5nIHRoZSBvbmxpbmUgdG9vbCBbR09yaWxsYV0oaHR0cDovL2NibC1nb3JpbGxhLmNzLnRlY2huaW9uLmFjLmlsLykgdG8gcGVyZm9ybSB0aGUgcGF0aHdheXMgYW5hbHlzaXMuIEl0IGhhcyB0d28gbW9kZXM7IHRoZSBmaXJzdCBvZiB3aGljaCBhY2NlcHRzIGEgbGlzdCBvZiAqYmFja2dyb3VuZCogYW5kICp0YXJnZXQqIGdlbmVzLiAqKipIb3dldmVyLCB0aGUgaW5wdXQgSURzIG5lZWQgdG8gYmUgR2VuZSBTeW1ib2xzKioqLgoKPGRpdiBjbGFzcz0iYWxlcnQgYWxlcnQtd2FybmluZyI+CioqRXhlcmNpc2U6KiogRG93bmxvYWQgeW91ciBhbm5vdGF0ZWQgYmFja2dyb3VuZC50c3YgZmlsZS4gVGhpcyB3aWxsIGFjdCBhcyB0aGUgYmFja2dyb3VuZCBsaXN0IGZvciB0aGUgYW5hbHlzaXMuIE5vdyB1c2UgdGhlICphbm5vdGF0ZU15SURzKiB0byBjb252ZXJ0IHRoZSBFbnRyZXogSURzIGluIHRoZSBmaWxlIGBCLnByZWdfdnNfbGFjdGF0aW9uLnRzdmAgaW50byBnZW5lIHN5bWJvbHMgYW5kIGRvd25sb2FkIHRoZSByZXN1bHQuCgo8L2Rpdj4KCjEuIEdvIHRvIGh0dHA6Ly9jYmwtZ29yaWxsYS5jcy50ZWNobmlvbi5hYy5pbC8KMi4gUmVhZCB0aGUg4oCcUnVubmluZyBFeGFtcGxl4oCdCgohW10obWVkaWEvZ29yaWxsYS1leGFtcGxlLnBuZykKCjMuIENob29zZSBPcmdhbmlzbTogYE11cyBNdXNjdWx1c2AKNC4gQ2hvb3NlIHJ1bm5pbmcgbW9kZTogYFR3byB1bnJhbmtlZCBsaXN0cyBvZiBnZW5lc2AKNS4gUGFzdGUgdGhlIGdlbmUgc3ltYm9scyBjb3JyZXNwb25kaW5nIHRvIERFIGdlbmVzIGludG8gdGhlIFRhcmdldCBzZXQuCjYuIFBhc3RlIHRoZSBnZW5lIHN5bWJvbHMgZnJvbSB0aGUgQmFja2dyb3VuZCBzZXQgaW50byB0aGUgb3RoZXIgYm94Lgo3LiBDaG9vc2UgYW4gT250b2xvZ3k6IGBQcm9jZXNzYAo4LiBgU2VhcmNoIEVucmljaGVkIEdPIHRlcm1zYAoKWW91IHNob3VsZCBiZSBwcmVzZW50ZWQgd2l0aCBhIGdyYXBoIG9mIGVucmljaGVkIEdPIHRlcm1zIHNob3dpbmcgdGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZSB0ZXJtcy4gRWFjaCBHTyB0ZXJtIGlzIGNvbG91cmVkIGFjY29yZGluZyB0byBpdHMgc3RhdGlzdGljYWwgc2lnbmlmaWNhbmNlLgoKQmVsb3cgdGhlIGZpZ3VyZSBpcyB0aGUgcmVzdWx0cyB0YWJsZS4gVGhpcyBsaW5rcyB0byBtb3JlIGluZm9ybWF0aW9uIGFib3V0IGVhY2ggR08gdGVybSwgYW5kIGxpc3RzIGVhY2ggZ2VuZSBpbiB0aGUgY2F0ZWdvcnkgdGhhdCB3YXMgZm91bmQgaW4geW91ciBsaXN0LiBUaGUgZW5yaWNobWVudCBjb2x1bW4gZ2l2ZXMgNCBudW1iZXJzIHRoYXQgYXJlIHVzZWQgdG8gZGV0ZXJtaW5lIGVucmljaG1lbnQgKHNpbWlsYXIgdG8gdGhlIEZpc2hlciBleGFjdCB0ZXN0IHdlIHNhdyBlYXJsaWVyKQoKLSBOLCB0b3RhbCBudW1iZXIgb2YgZ2VuZXMgKHNob3VsZCBiZSB0aGUgc2FtZSBpbiBhbGwgcm93cykKLSBCLCB0b3RhbCBudW1iZXIgb2YgZ2VuZXMgYW5ub3RhdGVkIHdpdGggdGhlIEdPIHRlcm0KLSBuLCB0b3RhbCBudW1iZXIgb2YgZ2VuZXMgdGhhdCB3ZXJlIGZvdW5kIGluIHRoZSBsaXN0IHlvdSB1cGxvYWRlZCAoc2FtZSBmb3IgYWxsIHJvd3MpCi0gYiwgbnVtYmVyIG9mIGdlbmVzIGluIHRoZSBsaXN0IHlvdSB1cGxvYWRlZCB0aGF0IGludGVyc2VjdCB3aXRoIHRoaXMgR08gdGVybQoKSWYgeW91IGhhdmUgdGltZSwgeW91IGNhbiBhbHNvIGV4cGVyaW1lbnQgdXBsb2FkaW5nIHRoZSBzYW1lIGdlbmVzIGxpc3RzIHRvIHRoZSBvbmxpbmUgdG9vbHMgW0RBVklEXShodHRwczovL2RhdmlkLm5jaWZjcmYuZ292L3Rvb2xzLmpzcCkgYW5kIFtHZW5lVHJhaWxdKGh0dHBzOi8vZ2VuZXRyYWlsMi5iaW9pbmYudW5pLXNiLmRlLykKCgojIyBVc2luZyBFbnJpY2hSCgo8ZGl2IGNsYXNzPSJhbGVydCBhbGVydC13YXJuaW5nIj4KKipRdWVzdGlvbjoqKiBbRW5yaWNoUl0oaHR0cDovL2FtcC5waGFybS5tc3NtLmVkdS9FbnJpY2hyLykgaXMgYW5vdGhlciBvbmxpbmUgdG9vbCBmb3IgcGVyZm9ybWluZyBlbnJpY2htZW50IGFuYWx5c2lzIGFnYWluc3QgYSBsYXJnZSBjb2xsZWN0aW9uIG9mIGRhdGFiYXNlcy4gR28gdG8gdGhlIHdlYnNpdGUsIGFuZCBwYXN0ZS1pbiB5b3VyIGxpc3Qgb2YgZGlmZmVyZW50aWFsbHktZXhwcmVzc2VkIGdlbmVzLiBFeHBsb3JlIHRoZSByZXN1bHRzIHRoYXQgRW5yaWNoUiBwcm92aWRlcwoKPC9kaXY+CgoKCgo=