Original Authors: Belinda Phipson, Anna Trigos, Matt Ritchie, Maria Doyle, Harriet Dashnow, Charity Law, Stephane Ballereau, Oscar Rueda, Ashley Sawle Based on the course RNAseq analysis in R delivered on May 11/12th 2016 and modified by Cancer Research Uk Cambridge Centre for the Functional Genomics Autumn School 2017

Overview

  • Reading in table of counts
  • Filtering lowly expressed genes
  • Quality control
  • Normalisation for composition bias

Introduction

Measuring gene expression on a genome-wide scale has become common practice over the last two decades or so, with microarrays predominantly used pre-2008. With the advent of next generation sequencing technology in 2008, an increasing number of scientists use this technology to measure and understand changes in gene expression in often complex systems. As sequencing costs have decreased, using RNA-Seq to simultaneously measure the expression of tens of thousands of genes for multiple samples has never been easier. The cost of these experiments has now moved from generating the data to storing and analysing it.

There are many steps involved in analysing an RNA-Seq experiment. Analysing an RNAseq experiment begins with sequencing reads. Traditionally, these are aligned to a reference genome, then the number of reads mapped to each gene can be counted. More modern approaches such as salmon quantify transcripts directly and do not require genome aligment to have taken place. Either approach results in a table of counts, which is what we perform statistical analyses on in R. While mapping and counting are important and necessary tasks, this session will be starting from the count data and getting stuck into analysis.

We will be following a workflow that uses the DESeq2 package. An alternative and well-respected workflow is based on the edgeR and limma packages.

Mouse mammary gland dataset

The data for this tutorial comes from a Nature Cell Biology paper, EGF-mediated induction of Mcl-1 at the switch to lactation is essential for alveolar cell survival [@Fu2015].

This study examines the expression profiles of basal stem-cell enriched cells (B) and committed luminal cells (L) in the mammary gland of virgin, pregnant and lactating mice. Six groups are present, with one for each combination of cell type and mouse status. Each group contains two biological replicates.

The sequencing reads for this experiment were uploaded to the Sequencing Read Archive (SRA) and details of how to obtain the raw reads and process are given in the session “Preparing fastq files for analysis in R”.

Obtaining the metadata

The sampleInfo.txt in the meta_data folder contains basic information about the samples that we will need for the analysis today. This includes the ID for the sample from SRA, an ID assigned by the researcher, and the cell type and developmental stage for each sample.

# Read the sample information into R
sampleinfo <- read.delim("meta_data/sampleInfo.txt")
View(sampleinfo)
sampleinfo
          run    Name CellType     Status
1  SRR1552444 MCL1-LA    basal     virgin
2  SRR1552445 MCL1-LB  luminal     virgin
3  SRR1552446 MCL1-LC  Luminal  pregnancy
4  SRR1552447 MCL1-LD  Luminal  pregnancy
5  SRR1552448 MCL1-LE  luminal  lactation
6  SRR1552449 MCL1-LF  luminal  lactation
7  SRR1552450 MCL1-DG    basal     virgin
8  SRR1552451 MCL1-DH  luminal     virgin
9  SRR1552452 MCL1-DI    basal  pregnancy
10 SRR1552453 MCL1-DJ    basal  pregnancy
11 SRR1552454 MCL1-DK    basal  lactation
12 SRR1552455 MCL1-DL    basal  lactation
rownames(sampleinfo) <- sampleinfo$run

Reading in the count data

Overview

We are going to use the tximport package to import the count data into R and collapse the data to the gene level. This requires us to run a function in the following form:-

txi <- tximport(files=..., type="salmon", tx2gene=...)

So we will need to define the files that we want to import and a transcript mapping data frame. The transcript mapping takes the form

 | TXNAME | GENEID
1| ENST00000456328.2 |  ENSG00000223972.5
2| ENST00000450305.2 | ENSG00000223972.5
3| ENST00000473358.1 | ENSG00000243485.5
4| ENST00000469289.1 | ENSG00000243485.5
5| ENST00000607096.1 | ENSG00000284332.1
6| ENST00000606857.1 | ENSG00000268020.3

tximport is able to import counts produced by different software, and different workflows are described for each in the tximport vignette.

Identifying the files

The samples from this study have been quantified using salmon. For details on how this is done, please see the previous session on preparing fastq files for analysis.

The script we used to run salmon (run_salmon.sh) created a separate folder for each sample (named according to the SRA ID), and inside each of these folders we find the salmon quantification file. Note that the salmon analysis produced many other files (e.g. log files), but we will only need the quant.sf.gz files for analysis.

The function we are going to use to import the salmon files requires a vector comprising the paths to the files that are to be imported. To construct such a vector we can use the following code chunk. Furthermore, we can name each item in the vector according to the directory name. These names will be used eventually to name the columns of our count matrices.

dirs <- list.files("salmon_quant/")
quant_files <- list.files("salmon_quant/",pattern="quant.sf.gz",recursive = TRUE,full.names = TRUE)
names(quant_files) <- dirs
quant_files
                            SRR1552444 
"salmon_quant//SRR1552444/quant.sf.gz" 
                            SRR1552445 
"salmon_quant//SRR1552445/quant.sf.gz" 
                            SRR1552446 
"salmon_quant//SRR1552446/quant.sf.gz" 
                            SRR1552447 
"salmon_quant//SRR1552447/quant.sf.gz" 
                            SRR1552448 
"salmon_quant//SRR1552448/quant.sf.gz" 
                            SRR1552449 
"salmon_quant//SRR1552449/quant.sf.gz" 
                            SRR1552450 
"salmon_quant//SRR1552450/quant.sf.gz" 
                            SRR1552451 
"salmon_quant//SRR1552451/quant.sf.gz" 
                            SRR1552452 
"salmon_quant//SRR1552452/quant.sf.gz" 
                            SRR1552453 
"salmon_quant//SRR1552453/quant.sf.gz" 
                            SRR1552454 
"salmon_quant//SRR1552454/quant.sf.gz" 
                            SRR1552455 
"salmon_quant//SRR1552455/quant.sf.gz" 

Inspecting the salmon output

The quant files are simple tab-delimited files that tabulate the counting results for each transcript in our chosen organism. Although we will use a specialised Bioconductor package (tximport) to import the counts for entire dataset into R, we can inspect the first of the files using the standard read_tsv function from the readr package.

library(readr)
quants <- read_tsv(quant_files[1])
Parsed with column specification:
cols(
  Name = col_character(),
  Length = col_double(),
  EffectiveLength = col_double(),
  TPM = col_double(),
  NumReads = col_double()
)
head(quants)

Various methods have been proposed to account for the two main known biases in RNA-seq; library composition and gene length. A nice summary is presented on this blog. The current favourite is TPM (transcripts per million) which is similar in concept to RPKM, but the different order in which operations are applied makes it easier to compare across samples.

The TPM are giving in the table, but we can demonstrate the calculation in a few steps

  • divide the number of reads for each transcript by it’s length (reads per kilobase - RPK)
  • sum the RPK values and divide by 1 million to get a scaling factor
  • divide the RPK values by the scaling factor to get the TPM
rpk <- quants$NumReads / quants$EffectiveLength
scale_factor <- sum(rpk) / 1e6
tpm <- rpk / scale_factor

Defining the transcript mapping

In order for tximport to give gene-level counts, we need to supply a data frame that can be used to associate each transcript name with a gene identifier. It is important to use a transcript file that corresponds to the name genome build as the file used to count the transcripts.

We can check if the gtf file exists in the directory we expect by running the file.exists function; returning TRUE or FALSE

gtf_file <- "Mus_musculus.GRCm38.91.chr.gtf.gz"
file.exists(gtf_file)
[1] TRUE

If required, we can download from the Ensembl FTP site.

download.file("ftp://ftp.ensembl.org/pub/release-91/gtf/mus_musculus/Mus_musculus.GRCm38.91.chr.gtf.gz",destfile = gtf_file)

Note on analysing your own data

[images/download_gtf.png]

If analysing your own data, you will have to locate the gtf file on the Ensembl FTP site. If you enter ftp://ftp.ensembl.org/pub/release-91/gtf into a web browser you will be able to navigate the site and find your organism of interest. By right-clicking on the name of the gtf you will be able to copy the URL and then paste into RStudio.

gtf_file <- "ensembl_ref/my_ref.gtf"
download.file(PASTE_LINK_FROM_ENSEMBL_HERE,destfile = gtf_file)

Creating a transcript database

The Bioconducor website provides many pre-built transcript databases for some organisms (Human, Mouse,Rat etc) which provide transcript definitions and allow users to query the locations of particular genes, exons and other genomic features. You may find a pre-built package that already has the transcript locations required to create the transcript mapping file. Check out the annotation section of the Bioconductor website - http://bioconductor.org/packages/release/BiocViews.html#___AnnotationData and look for packages starting TxDb...

However, it is quite easy to build such a database if we have a gtf file using the GenomicFeatures infrastructure.

## Could take a few minutes to run the makeTxDbFromGFF command
library(GenomicFeatures)
txdb <- makeTxDbFromGFF(gtf_file)
The "phase" metadata column contains non-NA values for features
  of type stop_codon. This information was ignored.

The database has a number of predefined “keys” and “columns” that have to be specified when creating a query

keytypes(txdb)
[1] "CDSID"    "CDSNAME"  "EXONID"   "EXONNAME" "GENEID"   "TXID"    
[7] "TXNAME"  
columns(txdb)
 [1] "CDSCHROM"   "CDSEND"     "CDSID"      "CDSNAME"    "CDSPHASE"  
 [6] "CDSSTART"   "CDSSTRAND"  "EXONCHROM"  "EXONEND"    "EXONID"    
[11] "EXONNAME"   "EXONRANK"   "EXONSTART"  "EXONSTRAND" "GENEID"    
[16] "TXCHROM"    "TXEND"      "TXID"       "TXNAME"     "TXSTART"   
[21] "TXSTRAND"   "TXTYPE"    

Sometimes we would want to query the positions for a limited set of selected genes (perhaps the results of a differential-expression analysis), but in this case we want the gene names that correspond to every transcript in the database. To get the names of all transcripts we can use the keys function. We then compose the query using the select function to return a data frame

k <- keys(txdb, keytype="TXNAME")
tx_map <- select(txdb, keys = k, columns="GENEID", keytype = "TXNAME")
'select()' returned 1:1 mapping between keys and columns
head(tx_map)
              TXNAME             GENEID
1 ENSMUST00000193812 ENSMUSG00000102693
2 ENSMUST00000082908 ENSMUSG00000064842
3 ENSMUST00000192857 ENSMUSG00000102851
4 ENSMUST00000161581 ENSMUSG00000089699
5 ENSMUST00000192183 ENSMUSG00000103147
6 ENSMUST00000193244 ENSMUSG00000102348

Such a data frame should be sufficient to allow us to use the tximport package. There is a bit of a problem though..

library(tximport)
tx2gene <- tx_map
write.csv(tx2gene,file="tx2gene.csv",row.names = FALSE,quote=FALSE)
txi <- tximport(quant_files,type="salmon",tx2gene = tx2gene)

In this case R is reporting a useful error message; the IDs we have supplied in the tx2gene data frame do not correspond to the transcript names in our files.

table(tx_map$TXNAME %in% quants$Name)

 FALSE 
133849 

Fortunately the authors of tximport have recognised this as a common problem and added an argument to the tximport function that can be used to overcome the error.

library(tximport)
tx2gene <- tx_map
txi <- tximport(quant_files,type="salmon",tx2gene = tx2gene,ignoreTxVersion = TRUE)
reading in files with read_tsv
1 2 3 4 5 6 7 8 9 10 11 12 
transcripts missing from tx2gene: 147
summarizing abundance
summarizing counts
summarizing length

The resulting object is a simple list structure in R which contains a number of components that we can access using a $ operator

names(txi)
[1] "abundance"           "counts"              "length"             
[4] "countsFromAbundance"
head(txi$counts)
                   SRR1552444 SRR1552445 SRR1552446 SRR1552447
ENSMUSG00000000001   5443.000   6085.000   5773.000       5166
ENSMUSG00000000003      0.000      0.000      0.000          0
ENSMUSG00000000028    259.000    329.753    267.000        102
ENSMUSG00000000037     66.000     97.999     65.999         42
ENSMUSG00000000049      2.000      3.000      0.000          0
ENSMUSG00000000056    266.253    326.000    589.000        464
                   SRR1552448 SRR1552449 SRR1552450 SRR1552451
ENSMUSG00000000001       2191       2189   4354.000       4369
ENSMUSG00000000003          0          0      0.000          0
ENSMUSG00000000028        231        206    304.694        259
ENSMUSG00000000037         24         21    182.000        162
ENSMUSG00000000049          0          0      4.000          6
ENSMUSG00000000056        402        360    770.000        522
                   SRR1552452 SRR1552453 SRR1552454 SRR1552455
ENSMUSG00000000001   4556.000       4079       3550   2800.000
ENSMUSG00000000003      0.000          0          0      0.000
ENSMUSG00000000028    199.000        152         44     63.000
ENSMUSG00000000037    230.000        198        238    192.000
ENSMUSG00000000049      1.000          1          3      2.000
ENSMUSG00000000056    382.998        230        441    398.001
all(rownames(sampleinfo) == colnames(txi$counts))
[1] TRUE

This was a fairly simple fix, but we could have also used some functionality from the tidyverse collection of packages to make the names compatible.

Fixing the problem with transcript names - the hard way!

None of the transcript names are matching up. If we look at the transcript names in the quant files they have a . symbol followed by a number after the transcript name. The tidyr package contains several useful functions for tidying a data frame. One function we will use in particular is to split a column of value that contain multiple pieces of information separated by a common character. In this case the character is “.” but other common separating characters include -, _, :. The separate function is able to detect which character is being used.

Here we create two new columns, TXNAME and Number. The TXNAME being the column that should match entries in the tx_map data frame.

library(tidyr)

Attaching package: ‘tidyr’

The following object is masked from ‘package:S4Vectors’:

    expand
quants <- separate(quants, Name, c("TXNAME","Number"),remove = FALSE)
head(quants)

Another useful piece of tidyverse functionality is to join two tables together based on a common column. We want to create a data frame that has the transcript name as it appears in the quant files and the gene name. This can be achieved by a left_join to ensure that all the transcript IDs in the quant files have an entry in the combined table.

N.B. those already familiar with R might have encountered the merge function for achieving a similar result.

library(dplyr)

Attaching package: ‘dplyr’

The following object is masked from ‘package:AnnotationDbi’:

    select

The following object is masked from ‘package:Biobase’:

    combine

The following objects are masked from ‘package:GenomicRanges’:

    intersect, setdiff, union

The following object is masked from ‘package:GenomeInfoDb’:

    intersect

The following objects are masked from ‘package:IRanges’:

    collapse, desc, intersect, setdiff, slice, union

The following objects are masked from ‘package:S4Vectors’:

    first, intersect, rename, setdiff, setequal, union

The following objects are masked from ‘package:BiocGenerics’:

    combine, intersect, setdiff, union

The following objects are masked from ‘package:stats’:

    filter, lag

The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union
quants <- left_join(quants, tx_map, by="TXNAME")
head(quants)

To create the tx2gene we can pull out the columns that we require from this merged data frame. There is another dplyr function for doing this that has simpler syntax than the “base R” method of subsetting a data frame.

tx2gene <- dplyr:::select(quants, Name, GENEID)
head(tx2gene)

We might want to further check that there are no “missing values” in the Gene ID column and remove them from the data frame

any(is.na(tx2gene$GENEID))
[1] TRUE
tx2gene <- filter(tx2gene, !is.na(GENEID))
library(tximport)
txi <- tximport(quant_files,type="salmon",tx2gene = tx2gene)
reading in files with read_tsv
1 2 3 4 5 6 7 8 9 10 11 12 
transcripts missing from tx2gene: 147
summarizing abundance
summarizing counts
summarizing length

Quality control of the imported counts

We will be using the DESeq2 library to analyse this dataset. As part of the DESeq2 vignette you will see examples of importing count data from different sources. In all cases, raw count data are expected.

For this workflow we are going to import data from tximport with the DESeqDataSetFromTximport function along with the sample information that we created earlier.

A design for the experiment also needs to be specified. This will define how the differential expression analysis is carried out, but can be changed at a later stage so we will use CellType for now as our factor of interest.

library(DESeq2)
dds <- DESeqDataSetFromTximport(txi, 
                                colData = sampleinfo,
                                design <- ~CellType)

The object contains all the metadata for the experiment, along with the counts.

colData(dds)
DataFrame with 12 rows and 4 columns
                  run     Name CellType     Status
             <factor> <factor> <factor>   <factor>
SRR1552444 SRR1552444  MCL1-LA    basal     virgin
SRR1552445 SRR1552445  MCL1-LB  luminal     virgin
SRR1552446 SRR1552446  MCL1-LC  Luminal  pregnancy
SRR1552447 SRR1552447  MCL1-LD  Luminal  pregnancy
SRR1552448 SRR1552448  MCL1-LE  luminal  lactation
...               ...      ...      ...        ...
SRR1552451 SRR1552451  MCL1-DH  luminal     virgin
SRR1552452 SRR1552452  MCL1-DI    basal  pregnancy
SRR1552453 SRR1552453  MCL1-DJ    basal  pregnancy
SRR1552454 SRR1552454  MCL1-DK    basal  lactation
SRR1552455 SRR1552455  MCL1-DL    basal  lactation

We will be using these raw counts throughout the workshop and transforming them using methods in the DESeq2 package. If TPM values are desired for some other application, we can extract them from the tximport object. These are the transcript-level TPM values that we saw earlier in the quant files that have been summarised to the gene-level.

tpm <- txi$abundance
write.csv(tpm, file="tpm_values.csv",quote=FALSE)

At the time of writing, TPM is the recommended way of transforming RNA-seq counts to account for gene length and library composition biases. DESeq2 also provides methods for extracting normalised counts as FPKM (fragments per kilobase per million mapped fragments) and FPM (fragments per million mapped fragments) which might be required for some other analysis or visualisation outside of Bioconductor or DESeq2.

fpm <- fpm(dds)
write.csv(fpm, file="fpm_values.csv",quote=FALSE)
fpkm <- fpkm(dds)
write.csv(fpkm, file="fpkm_values.csv",quote=FALSE)

Visualising library sizes

We can look at a few different plots to check that the data is good quality, and that the samples are as we would expect. First, we can check how many reads we have for each sample in the DESeqDataSet. The counts themselves are accessed using the assay function; giving a matrix of counts. The sum of a particular column is therefore the total number of reads for that sample.

sum(assay(dds)[,1])
[1] 24444863

A convenience function colSums exists for calculating the sum of each column in a matrix, returning a vector as a result.

colSums(assay(dds))
SRR1552444 SRR1552445 SRR1552446 SRR1552447 SRR1552448 SRR1552449 
  24444863   25995520   26800109   26378758   29201722   29106858 
SRR1552450 SRR1552451 SRR1552452 SRR1552453 SRR1552454 SRR1552455 
  27070438   25504036   27990146   26185840   24395121   22722335 

Challenge 1

  1. Produce a bar plot to show the Millions of reads for each sample
  2. Change the names under the plot so they are the “Name” of the sample rather than Run Name
  3. Add a horizontal line at 20 million reads

HINT: look at the help for the base graphics functions barplot and abline (or try and use ggplot2 if you prefer)

Filtering non-expressed genes

The dataset is usually filtered at this stage to remove any genes that are not expressed. Although not strictly required for the DESeq2 differential expression algorithm, it can reduce the time and memory required to perform some of the analysis. Let’s say that for a gene to be “expressed” in a particular sample we need to see 5 or more counts

is_expressed <- assay(dds) >= 5
head(is_expressed)
                   SRR1552444 SRR1552445 SRR1552446 SRR1552447
ENSMUSG00000000001       TRUE       TRUE       TRUE       TRUE
ENSMUSG00000000003      FALSE      FALSE      FALSE      FALSE
ENSMUSG00000000028       TRUE       TRUE       TRUE       TRUE
ENSMUSG00000000037       TRUE       TRUE       TRUE       TRUE
ENSMUSG00000000049      FALSE      FALSE      FALSE      FALSE
ENSMUSG00000000056       TRUE       TRUE       TRUE       TRUE
                   SRR1552448 SRR1552449 SRR1552450 SRR1552451
ENSMUSG00000000001       TRUE       TRUE       TRUE       TRUE
ENSMUSG00000000003      FALSE      FALSE      FALSE      FALSE
ENSMUSG00000000028       TRUE       TRUE       TRUE       TRUE
ENSMUSG00000000037       TRUE       TRUE       TRUE       TRUE
ENSMUSG00000000049      FALSE      FALSE      FALSE       TRUE
ENSMUSG00000000056       TRUE       TRUE       TRUE       TRUE
                   SRR1552452 SRR1552453 SRR1552454 SRR1552455
ENSMUSG00000000001       TRUE       TRUE       TRUE       TRUE
ENSMUSG00000000003      FALSE      FALSE      FALSE      FALSE
ENSMUSG00000000028       TRUE       TRUE       TRUE       TRUE
ENSMUSG00000000037       TRUE       TRUE       TRUE       TRUE
ENSMUSG00000000049      FALSE      FALSE      FALSE      FALSE
ENSMUSG00000000056       TRUE       TRUE       TRUE       TRUE

R is happy to think of logical values (TRUE or FALSE) as the integers 0 or 1. Therefore if we calculate the sum across a particular row it will give the number of samples that gene is expressed in.

sum(is_expressed[1,])
[1] 12
sum(is_expressed[2,])
[1] 0

In a similar manner to colSums, rowSums will give the sum of each row in a matrix and return the result as a vector.

hist(rowSums(is_expressed),main="Number of samples a gene is expressed in",xlab="Sample Count")

It seems that genes are either expressed in all samples, or not expressed at all. We will decide to keep genes that are expressed in at least 2 samples.

keep <- rowSums(assay(dds) >= 5) >= 2
table(keep)
keep
FALSE  TRUE 
16479 17973 
dds <- dds[keep,]

Visualising count distributions

We typically use a boxplot to visualise difference the distributions of the columns of a numeric data frame. Applying the boxplot function to the raw counts from our dataset reveals something about the nature of the data; the distributions are dominated by a few genes with very large counts.

boxplot(assay(dds))

boxplot(log10(assay(dds)))

We can use the vst or rlog function from DESeq2to compensate for the effect of different library sizes and put the data on the log\(_2\) scale. The effect is to remove the dependence of the variance on the mean, particularly the high variance of the logarithm of count data when the mean is low. For more details see the DESeq2 vignette

# Get log2 counts
vsd <- vst(dds,blind=TRUE)
using 'avgTxLength' from assays(dds), correcting for library size
# Check distributions of samples using boxplots
boxplot(assay(vsd), xlab="", ylab="Log2 counts per million",las=2,main="Normalised Distributions")
# Let's add a blue horizontal line that corresponds to the median logCPM
abline(h=median(assay(vsd)), col="blue")

Heatmap of the sample-to-sample distances

Another use of the transformed data is sample clustering. Here, we apply the dist function to the transpose of the transformed count matrix to get sample-to-sample distances.

sampleDists <- dist(t(assay(vsd)))

A heatmap of this distance matrix gives us an overview over similarities and dissimilarities between samples. By re-naming the rows and columns of the distance matrix we can make the plot easier to interpret.

library(RColorBrewer)
library(pheatmap)
sampleDistMatrix <- as.matrix(sampleDists)
rownames(sampleDistMatrix) <- paste(colData(dds)$CellType, colData(dds)$Status, sep="-")
colnames(sampleDistMatrix) <- colData(dds)$Name
colors <- colorRampPalette( rev(brewer.pal(9, "Blues")) )(255)
pheatmap(sampleDistMatrix,
         col=colors)

Principal component (PCA)

Related to the distance matrix heatmap is the (Principal Components Analysis) PCA plot, which shows the samples in the 2D plane spanned by their first two principal components. A principle components analysis is an example of an unsupervised analysis, where we don’t need to specify the groups. If your experiment is well controlled and has worked well, what we hope to see is that the greatest sources of variation in the data are the treatments/groups we are interested in. It is also an incredibly useful tool for quality control and checking for outliers

DESeq2 has a convenient plotPCA function for making the PCA plot, which makes use of the ggplot2 graphics package.

plotPCA(vsd,intgroup="CellType")

Challenge 2

  1. Is the plotPCA plot based on all genes in the dataset? How can we change how many genes are used for the PCA analysis? Does this significantly change the plot? (HINT: check the documentation for the plotPCA function.)
  2. Change the intgroup parameter so that both CellType and Status are used for grouping. (See the documentation again)
  3. Is there something strange going on with the samples?
  4. Identify the two samples that don’t appear to be in the right place.
  5. What other problems can you see with the metadata?

Note about batch effects

In our unsupervised analysis we should see that the main source of variation is due to biological effects, and not technical variation such as when the libraries were sequenced. If we do observe high technical variation in our data, it is not a complete disaster provided that we have designed our experiment propery. In particular the sva Bioconductor package can correct for batch effects provided that representatives of the groups of interest appear in each batch. Alternatively, the batch or confounding factor may be incorporated into the differential expression analysis.

Correcting the sample information

Hopefully we have spotted a potential sample swap in the dataset. The mislabelled samples are MCL1.DH, which is labelled as luminal but should be basal, and MCL1.LA, which is labelled as basal but should be luminal. Such errors are not uncommon when handling large numbers of samples and sometimes we need to go back to the lab books and verify that a swap has been made. If there is no supporting evidence for a swap then it can be safer to exclude the samples.

Furthermore, the person creating the sample sheet has been inconsistent about the way that values of CellType and Status have been entered into the metadata. Such errors can be annoying when labelling plots, but have more serious consequences when attempting to fit statistical models to the data.

library(stringr)
sampleinfo_corrected <- sampleinfo
sampleinfo_corrected <- mutate(sampleinfo_corrected, CellType = str_to_lower(CellType))
sampleinfo_corrected <- mutate(sampleinfo_corrected, Status = str_trim(Status))
sampleinfo_corrected <- mutate(sampleinfo_corrected, CellType = ifelse(Name == "MCL1-DH","basal",CellType))
sampleinfo_corrected <- mutate(sampleinfo_corrected, CellType= ifelse(Name == "MCL1-LA","luminal",CellType))
write.table(sampleinfo_corrected, file="meta_data/sampleInfo_corrected.txt",sep="\t",row.names = FALSE)

Challenge 3

  1. Re-create the DESeqDataset object to include the corrected sample information
  2. Re-run the plotPCA function on the new data and verify that the sample groups now look correct

A note about “pipes” and ggplot2

It is common practice when using a series of dplyr data frame manipulations (such as mutate above) to create a workflow with the %>% operation from the magrittr package. The result is a more-readable chunk of code, although the result is exactly the same.

sampleinfo_corrected <- mutate(sampleinfo, CellType = str_to_lower(CellType)) %>% 
  mutate(Status = str_trim(Status)) %>% 
  mutate(CellType = ifelse(Name == "MCL1-DH","basal",CellType)) %>% 
  mutate(CellType= ifelse(Name == "MCL1-LA","luminal",CellType))
sampleinfo_corrected
          run    Name CellType    Status
1  SRR1552444 MCL1-LA  luminal    virgin
2  SRR1552445 MCL1-LB  luminal    virgin
3  SRR1552446 MCL1-LC  luminal pregnancy
4  SRR1552447 MCL1-LD  luminal pregnancy
5  SRR1552448 MCL1-LE  luminal lactation
6  SRR1552449 MCL1-LF  luminal lactation
7  SRR1552450 MCL1-DG    basal    virgin
8  SRR1552451 MCL1-DH    basal    virgin
9  SRR1552452 MCL1-DI    basal pregnancy
10 SRR1552453 MCL1-DJ    basal pregnancy
11 SRR1552454 MCL1-DK    basal lactation
12 SRR1552455 MCL1-DL    basal lactation

The plotPCA function produces a ggplot2 plot (rather than “base” graphics) and we can also get the function to return the data used to create the plot by changing the returnData argument.

plot_data <- plotPCA(vsd,intgroup=c("CellType","Status"),returnData=TRUE)
plot_data <- bind_cols(plot_data,sampleinfo_corrected)

With ggplot2 plot can be created by mapping various aesthetics of the plot (colour, shape, x- and y-coordinates) to columns in the data frame.

library(ggplot2)
ggplot(plot_data, aes(x = PC1,y=PC2, col=CellType)) + geom_point()

Having the data in this form allows us to customise the plot in lots of ways

ggplot(plot_data, aes(x = PC1,y=PC2, col=CellType,pch=Status)) + geom_point(size=5)

ggplot(plot_data, aes(x = PC1,y=PC2, col=CellType,pch=Status,label=Name)) + geom_point() + geom_text(alpha=0.4)

LS0tCnRpdGxlOiAiUk5BLXNlcSBhbmFseXNpcyBpbiBSIgphdXRob3I6ICJNYXJrIER1bm5pbmciCmRhdGU6ICJGZWJydWFyeSAyMDIwIgpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazoKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwogICAgdG9jX2Zsb2F0OiB5ZXMKbWludXRlczogMzAwCmxheW91dDogcGFnZQpzdWJ0aXRsZTogUHJlLXByb2Nlc3NzaW5nIFJOQS1zZXEgZGF0YQoKLS0tCgoqKk9yaWdpbmFsIEF1dGhvcnM6IEJlbGluZGEgUGhpcHNvbiwgQW5uYSBUcmlnb3MsIE1hdHQgUml0Y2hpZSwgTWFyaWEgRG95bGUsIEhhcnJpZXQgRGFzaG5vdywgQ2hhcml0eSBMYXcqKiwgKipTdGVwaGFuZSBCYWxsZXJlYXUsIE9zY2FyIFJ1ZWRhLCBBc2hsZXkgU2F3bGUqKgpCYXNlZCBvbiB0aGUgY291cnNlIFtSTkFzZXEgYW5hbHlzaXMgaW4gUl0oaHR0cDovL2NvbWJpbmUtYXVzdHJhbGlhLmdpdGh1Yi5pby8yMDE2LTA1LTExLVJOQXNlcS8pIGRlbGl2ZXJlZCBvbiBNYXkgMTEvMTJ0aCAyMDE2IGFuZCBtb2RpZmllZCBieSBDYW5jZXIgUmVzZWFyY2ggVWsgQ2FtYnJpZGdlIENlbnRyZSBmb3IgdGhlIFtGdW5jdGlvbmFsIEdlbm9taWNzIEF1dHVtbiBTY2hvb2wgMjAxN10oaHR0cHM6Ly9iaW9pbmZvcm1hdGljcy1jb3JlLXNoYXJlZC10cmFpbmluZy5naXRodWIuaW8vY3J1ay1hdXR1bW4tc2Nob29sLTIwMTcvKQoKYGBge3Iga25pdHJPcHRzLCBpbmNsdWRlPUZBTFNFfQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IFRSVUUpCmBgYAoKCgojIyBSZXNvdXJjZXMgYW5kIGRhdGEgZmlsZXMKClRoaXMgbWF0ZXJpYWwgaGFzIGJlZW4gY3JlYXRlZCB1c2luZyB0aGUgZm9sbG93aW5nIHJlc291cmNlczogIAoKLSBodHRwOi8vd3d3LnN0YXRzY2kub3JnL3NteXRoL3B1YnMvUUxlZGdlUlByZXByaW50LnBkZiBbQEx1bjIwMTZdICAKLSBodHRwOi8vbW9uYXNoYmlvaW5mb3JtYXRpY3NwbGF0Zm9ybS5naXRodWIuaW8vUk5Bc2VxLURFLWFuYWx5c2lzLXdpdGgtUi85OS1STkFzZXFfREVfYW5hbHlzaXNfd2l0aF9SLmh0bWwgIAotIGh0dHA6Ly9iaW9jb25kdWN0b3Iub3JnL3BhY2thZ2VzL2RldmVsL2Jpb2MvdmlnbmV0dGVzL0RFU2VxMi9pbnN0L2RvYy9ERVNlcTIuaHRtbAotIGh0dHBzOi8vYmlvY29uZHVjdG9yLmdpdGh1Yi5pby9CaW9jV29ya3Nob3BzL3JuYS1zZXEtZGF0YS1hbmFseXNpcy13aXRoLWRlc2VxMi5odG1sCgoKIyMgT3ZlcnZpZXcKCiogUmVhZGluZyBpbiB0YWJsZSBvZiBjb3VudHMKKiBGaWx0ZXJpbmcgbG93bHkgZXhwcmVzc2VkIGdlbmVzCiogUXVhbGl0eSBjb250cm9sCiogTm9ybWFsaXNhdGlvbiBmb3IgY29tcG9zaXRpb24gYmlhcwoKCiMjIEludHJvZHVjdGlvbgoKTWVhc3VyaW5nIGdlbmUgZXhwcmVzc2lvbiBvbiBhIGdlbm9tZS13aWRlIHNjYWxlIGhhcyBiZWNvbWUgY29tbW9uIHByYWN0aWNlIG92ZXIgdGhlIGxhc3QgdHdvIGRlY2FkZXMgb3Igc28sIHdpdGggbWljcm9hcnJheXMgcHJlZG9taW5hbnRseSB1c2VkIHByZS0yMDA4LiBXaXRoIHRoZSBhZHZlbnQgb2YgbmV4dCBnZW5lcmF0aW9uIHNlcXVlbmNpbmcgdGVjaG5vbG9neSBpbiAyMDA4LCBhbiBpbmNyZWFzaW5nIG51bWJlciBvZiBzY2llbnRpc3RzIHVzZSB0aGlzIHRlY2hub2xvZ3kgdG8gbWVhc3VyZSBhbmQgdW5kZXJzdGFuZCBjaGFuZ2VzIGluIGdlbmUgZXhwcmVzc2lvbiBpbiBvZnRlbiBjb21wbGV4IHN5c3RlbXMuIEFzIHNlcXVlbmNpbmcgY29zdHMgaGF2ZSBkZWNyZWFzZWQsIHVzaW5nIFJOQS1TZXEgdG8gc2ltdWx0YW5lb3VzbHkgbWVhc3VyZSB0aGUgZXhwcmVzc2lvbiBvZiB0ZW5zIG9mIHRob3VzYW5kcyBvZiBnZW5lcyBmb3IgbXVsdGlwbGUgc2FtcGxlcyBoYXMgbmV2ZXIgYmVlbiBlYXNpZXIuIFRoZSBjb3N0IG9mIHRoZXNlIGV4cGVyaW1lbnRzIGhhcyBub3cgbW92ZWQgZnJvbSBnZW5lcmF0aW5nIHRoZSBkYXRhIHRvIHN0b3JpbmcgYW5kIGFuYWx5c2luZyBpdC4KClRoZXJlIGFyZSBtYW55IHN0ZXBzIGludm9sdmVkIGluIGFuYWx5c2luZyBhbiBSTkEtU2VxIGV4cGVyaW1lbnQuIEFuYWx5c2luZyBhbiBSTkFzZXEgZXhwZXJpbWVudCBiZWdpbnMgd2l0aCBzZXF1ZW5jaW5nIHJlYWRzLiBUcmFkaXRpb25hbGx5LCB0aGVzZSBhcmUgYWxpZ25lZCB0byBhIHJlZmVyZW5jZSBnZW5vbWUsIHRoZW4gdGhlIG51bWJlciBvZiByZWFkcyBtYXBwZWQgdG8gZWFjaCBnZW5lIGNhbiBiZSBjb3VudGVkLiBNb3JlIG1vZGVybiBhcHByb2FjaGVzIHN1Y2ggYXMgYHNhbG1vbmAgcXVhbnRpZnkgdHJhbnNjcmlwdHMgZGlyZWN0bHkgYW5kIGRvIG5vdCByZXF1aXJlIGdlbm9tZSBhbGlnbWVudCAgdG8gaGF2ZSB0YWtlbiBwbGFjZS4gRWl0aGVyIGFwcHJvYWNoIHJlc3VsdHMgaW4gYSB0YWJsZSBvZiBjb3VudHMsIHdoaWNoIGlzIHdoYXQgd2UgcGVyZm9ybSBzdGF0aXN0aWNhbCBhbmFseXNlcyBvbiBpbiBSLiBXaGlsZSBtYXBwaW5nIGFuZCBjb3VudGluZyBhcmUgaW1wb3J0YW50IGFuZCBuZWNlc3NhcnkgdGFza3MsIHRoaXMgc2Vzc2lvbiB3aWxsIGJlIHN0YXJ0aW5nIGZyb20gdGhlIGNvdW50IGRhdGEgYW5kIGdldHRpbmcgc3R1Y2sgaW50byBhbmFseXNpcy4KCldlIHdpbGwgYmUgZm9sbG93aW5nIGEgd29ya2Zsb3cgdGhhdCB1c2VzIHRoZSBgREVTZXEyYCBwYWNrYWdlLiBBbiBhbHRlcm5hdGl2ZSBhbmQgd2VsbC1yZXNwZWN0ZWQgd29ya2Zsb3cgaXMgYmFzZWQgb24gdGhlIFtlZGdlUiBhbmQgbGltbWEgcGFja2FnZXNdKGh0dHBzOi8vYmlvY29uZHVjdG9yLmdpdGh1Yi5pby9CaW9jV29ya3Nob3BzL3JuYS1zZXEtYW5hbHlzaXMtaXMtZWFzeS1hcy0xLTItMy13aXRoLWxpbW1hLWdsaW1tYS1hbmQtZWRnZXIuaHRtbCkuCgojIyMgTW91c2UgbWFtbWFyeSBnbGFuZCBkYXRhc2V0CgpUaGUgZGF0YSBmb3IgdGhpcyB0dXRvcmlhbCBjb21lcyBmcm9tIGEgTmF0dXJlIENlbGwgQmlvbG9neSBwYXBlciwgWypFR0YtbWVkaWF0ZWQgaW5kdWN0aW9uIG9mIE1jbC0xIGF0IHRoZSBzd2l0Y2ggdG8gbGFjdGF0aW9uIGlzIGVzc2VudGlhbCBmb3IgYWx2ZW9sYXIgY2VsbCBzdXJ2aXZhbCpdKGh0dHA6Ly93d3cubmNiaS5ubG0ubmloLmdvdi9wdWJtZWQvMjU3MzA0NzIpIFtARnUyMDE1XS4gCgpUaGlzIHN0dWR5IGV4YW1pbmVzIHRoZSBleHByZXNzaW9uIHByb2ZpbGVzIG9mIGJhc2FsIHN0ZW0tY2VsbCBlbnJpY2hlZCBjZWxscyAoQikgYW5kIGNvbW1pdHRlZCBsdW1pbmFsIGNlbGxzIChMKSBpbiB0aGUgbWFtbWFyeSBnbGFuZCBvZiB2aXJnaW4sIHByZWduYW50IGFuZCBsYWN0YXRpbmcgbWljZS4gU2l4IGdyb3VwcyBhcmUgcHJlc2VudCwgd2l0aCBvbmUgZm9yIGVhY2ggY29tYmluYXRpb24gb2YgY2VsbCB0eXBlIGFuZCBtb3VzZSBzdGF0dXMuIEVhY2ggZ3JvdXAgY29udGFpbnMgdHdvIGJpb2xvZ2ljYWwgcmVwbGljYXRlcy4KClRoZSBzZXF1ZW5jaW5nIHJlYWRzIGZvciB0aGlzIGV4cGVyaW1lbnQgd2VyZSB1cGxvYWRlZCB0byB0aGUgW1NlcXVlbmNpbmcgUmVhZCBBcmNoaXZlIChTUkEpXShodHRwczovL3d3dy5uY2JpLm5sbS5uaWguZ292L3NyYT90ZXJtPVNSUDA0NTUzNCkgYW5kIGRldGFpbHMgb2YgaG93IHRvIG9idGFpbiB0aGUgcmF3IHJlYWRzIGFuZCBwcm9jZXNzIGFyZSBnaXZlbiBpbiB0aGUgc2Vzc2lvbiAiUHJlcGFyaW5nIGZhc3RxIGZpbGVzIGZvciBhbmFseXNpcyBpbiBSIi4KCiMjIE9idGFpbmluZyB0aGUgbWV0YWRhdGEKCgpUaGUgYHNhbXBsZUluZm8udHh0YCBpbiB0aGUgYG1ldGFfZGF0YWAgZm9sZGVyIGNvbnRhaW5zIGJhc2ljIGluZm9ybWF0aW9uIGFib3V0IHRoZSBzYW1wbGVzIHRoYXQgd2Ugd2lsbCBuZWVkIGZvciB0aGUgYW5hbHlzaXMgdG9kYXkuIFRoaXMgaW5jbHVkZXMgdGhlIElEIGZvciB0aGUgc2FtcGxlIGZyb20gU1JBLCBhbiBJRCBhc3NpZ25lZCBieSB0aGUgcmVzZWFyY2hlciwgYW5kIHRoZSBjZWxsIHR5cGUgYW5kIGRldmVsb3BtZW50YWwgc3RhZ2UgZm9yIGVhY2ggc2FtcGxlLgoKYGBge3IgbG9hZFNhbXBsZUluZm99CiMgUmVhZCB0aGUgc2FtcGxlIGluZm9ybWF0aW9uIGludG8gUgpzYW1wbGVpbmZvIDwtIHJlYWQuZGVsaW0oIm1ldGFfZGF0YS9zYW1wbGVJbmZvLnR4dCIpClZpZXcoc2FtcGxlaW5mbykKc2FtcGxlaW5mbwpyb3duYW1lcyhzYW1wbGVpbmZvKSA8LSBzYW1wbGVpbmZvJHJ1bgpgYGAKCgoKIyMgUmVhZGluZyBpbiB0aGUgY291bnQgZGF0YQoKIyMjIE92ZXJ2aWV3CgpXZSBhcmUgZ29pbmcgdG8gdXNlIHRoZSBbYHR4aW1wb3J0YF0oaHR0cDovL2R4LmRvaS5vcmcvMTAuMTI2ODgvZjEwMDByZXNlYXJjaC43NTYzLjEpIHBhY2thZ2UgdG8gaW1wb3J0IHRoZSBjb3VudCBkYXRhIGludG8gUiBhbmQgY29sbGFwc2UgdGhlIGRhdGEgdG8gdGhlICpnZW5lIGxldmVsKi4gVGhpcyByZXF1aXJlcyB1cyB0byBydW4gYSBmdW5jdGlvbiBpbiB0aGUgZm9sbG93aW5nIGZvcm06LQoKYGBge3IgZXZhbD1GQUxTRX0KdHhpIDwtIHR4aW1wb3J0KGZpbGVzPS4uLiwgdHlwZT0ic2FsbW9uIiwgdHgyZ2VuZT0uLi4pCmBgYAoKU28gd2Ugd2lsbCBuZWVkIHRvIGRlZmluZSB0aGUgZmlsZXMgdGhhdCB3ZSB3YW50IHRvIGltcG9ydCBhbmQgYSB0cmFuc2NyaXB0IG1hcHBpbmcgZGF0YSBmcmFtZS4gVGhlIHRyYW5zY3JpcHQgbWFwcGluZyB0YWtlcyB0aGUgZm9ybSAKCmBgYAogfCBUWE5BTUUgfCBHRU5FSUQKMXwgRU5TVDAwMDAwNDU2MzI4LjIgfCAgRU5TRzAwMDAwMjIzOTcyLjUKMnwgRU5TVDAwMDAwNDUwMzA1LjIgfCBFTlNHMDAwMDAyMjM5NzIuNQozfCBFTlNUMDAwMDA0NzMzNTguMSB8IEVOU0cwMDAwMDI0MzQ4NS41CjR8IEVOU1QwMDAwMDQ2OTI4OS4xIHwgRU5TRzAwMDAwMjQzNDg1LjUKNXwgRU5TVDAwMDAwNjA3MDk2LjEgfCBFTlNHMDAwMDAyODQzMzIuMQo2fCBFTlNUMDAwMDA2MDY4NTcuMSB8IEVOU0cwMDAwMDI2ODAyMC4zCmBgYAoKYHR4aW1wb3J0YCBpcyBhYmxlIHRvIGltcG9ydCBjb3VudHMgcHJvZHVjZWQgYnkgZGlmZmVyZW50IHNvZnR3YXJlLCBhbmQgZGlmZmVyZW50IHdvcmtmbG93cyBhcmUgZGVzY3JpYmVkIGZvciBlYWNoIGluIHRoZSBbdHhpbXBvcnQgdmlnbmV0dGVdKGh0dHBzOi8vYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9kZXZlbC9iaW9jL3ZpZ25ldHRlcy90eGltcG9ydC9pbnN0L2RvYy90eGltcG9ydC5odG1sKS4KCiMjIElkZW50aWZ5aW5nIHRoZSBmaWxlcwoKVGhlIHNhbXBsZXMgZnJvbSB0aGlzIHN0dWR5IGhhdmUgYmVlbiAqcXVhbnRpZmllZCogdXNpbmcgYHNhbG1vbmAuIEZvciBkZXRhaWxzIG9uIGhvdyB0aGlzIGlzIGRvbmUsIHBsZWFzZSBzZWUgdGhlIHByZXZpb3VzIHNlc3Npb24gb24gcHJlcGFyaW5nIGZhc3RxIGZpbGVzIGZvciBhbmFseXNpcy4KClRoZSBzY3JpcHQgd2UgdXNlZCB0byBydW4gc2FsbW9uIChgcnVuX3NhbG1vbi5zaGApIGNyZWF0ZWQgYSBzZXBhcmF0ZSBmb2xkZXIgZm9yIGVhY2ggc2FtcGxlIChuYW1lZCBhY2NvcmRpbmcgdG8gdGhlIFNSQSBJRCksIGFuZCBpbnNpZGUgZWFjaCBvZiB0aGVzZSBmb2xkZXJzIHdlIGZpbmQgdGhlIHNhbG1vbiBxdWFudGlmaWNhdGlvbiBmaWxlLiBOb3RlIHRoYXQgdGhlIHNhbG1vbiBhbmFseXNpcyBwcm9kdWNlZCBtYW55IG90aGVyIGZpbGVzIChlLmcuIGxvZyBmaWxlcyksIGJ1dCB3ZSB3aWxsIG9ubHkgbmVlZCB0aGUgYHF1YW50LnNmLmd6YCBmaWxlcyBmb3IgYW5hbHlzaXMuCgoKVGhlIGZ1bmN0aW9uIHdlIGFyZSBnb2luZyB0byB1c2UgdG8gaW1wb3J0IHRoZSBzYWxtb24gZmlsZXMgcmVxdWlyZXMgYSBgdmVjdG9yYCBjb21wcmlzaW5nIHRoZSBwYXRocyB0byB0aGUgZmlsZXMgdGhhdCBhcmUgdG8gYmUgaW1wb3J0ZWQuIFRvIGNvbnN0cnVjdCBzdWNoIGEgdmVjdG9yIHdlIGNhbiB1c2UgdGhlIGZvbGxvd2luZyBjb2RlIGNodW5rLiBGdXJ0aGVybW9yZSwgd2UgY2FuIG5hbWUgZWFjaCBpdGVtIGluIHRoZSB2ZWN0b3IgYWNjb3JkaW5nIHRvIHRoZSBkaXJlY3RvcnkgbmFtZS4gVGhlc2UgbmFtZXMgd2lsbCBiZSB1c2VkIGV2ZW50dWFsbHkgdG8gbmFtZSB0aGUgY29sdW1ucyBvZiBvdXIgY291bnQgbWF0cmljZXMuCgpgYGB7cn0KZGlycyA8LSBsaXN0LmZpbGVzKCJzYWxtb25fcXVhbnQvIikKcXVhbnRfZmlsZXMgPC0gbGlzdC5maWxlcygic2FsbW9uX3F1YW50LyIscGF0dGVybj0icXVhbnQuc2YuZ3oiLHJlY3Vyc2l2ZSA9IFRSVUUsZnVsbC5uYW1lcyA9IFRSVUUpCm5hbWVzKHF1YW50X2ZpbGVzKSA8LSBkaXJzCnF1YW50X2ZpbGVzCmBgYAoKIyMjIEluc3BlY3RpbmcgdGhlIHNhbG1vbiBvdXRwdXQKClRoZSBxdWFudCBmaWxlcyBhcmUgc2ltcGxlIHRhYi1kZWxpbWl0ZWQgZmlsZXMgdGhhdCB0YWJ1bGF0ZSB0aGUgY291bnRpbmcgcmVzdWx0cyBmb3IgZWFjaCB0cmFuc2NyaXB0IGluIG91ciBjaG9zZW4gb3JnYW5pc20uIEFsdGhvdWdoIHdlIHdpbGwgdXNlIGEgc3BlY2lhbGlzZWQgQmlvY29uZHVjdG9yIHBhY2thZ2UgKGB0eGltcG9ydGApIHRvIGltcG9ydCB0aGUgY291bnRzIGZvciBlbnRpcmUgZGF0YXNldCBpbnRvIFIsIHdlIGNhbiBpbnNwZWN0IHRoZSBmaXJzdCBvZiB0aGUgZmlsZXMgdXNpbmcgdGhlIHN0YW5kYXJkIGByZWFkX3RzdmAgZnVuY3Rpb24gZnJvbSB0aGUgYHJlYWRyYCBwYWNrYWdlLgoKCgpgYGB7cn0KbGlicmFyeShyZWFkcikKcXVhbnRzIDwtIHJlYWRfdHN2KHF1YW50X2ZpbGVzWzFdKQpoZWFkKHF1YW50cykKCmBgYAoKClZhcmlvdXMgbWV0aG9kcyBoYXZlIGJlZW4gcHJvcG9zZWQgdG8gYWNjb3VudCBmb3IgdGhlIHR3byBtYWluIGtub3duIGJpYXNlcyBpbiBSTkEtc2VxOyBsaWJyYXJ5IGNvbXBvc2l0aW9uIGFuZCBnZW5lIGxlbmd0aC4gQSBuaWNlIHN1bW1hcnkgaXMgcHJlc2VudGVkIG9uIFt0aGlzIGJsb2ddKGh0dHBzOi8vd3d3LnJuYS1zZXFibG9nLmNvbS9ycGttLWZwa20tYW5kLXRwbS1jbGVhcmx5LWV4cGxhaW5lZC8pLiBUaGUgY3VycmVudCBmYXZvdXJpdGUgaXMgKlRQTSogKHRyYW5zY3JpcHRzIHBlciBtaWxsaW9uKSB3aGljaCBpcyBzaW1pbGFyIGluIGNvbmNlcHQgdG8gKlJQS00qLCBidXQgdGhlIGRpZmZlcmVudCBvcmRlciBpbiB3aGljaCBvcGVyYXRpb25zIGFyZSBhcHBsaWVkIG1ha2VzIGl0IGVhc2llciB0byBjb21wYXJlIGFjcm9zcyBzYW1wbGVzLgoKVGhlICpUUE0qIGFyZSBnaXZpbmcgaW4gdGhlIHRhYmxlLCBidXQgd2UgY2FuIGRlbW9uc3RyYXRlIHRoZSBjYWxjdWxhdGlvbiBpbiBhIGZldyBzdGVwcwoKLSBkaXZpZGUgdGhlIG51bWJlciBvZiByZWFkcyBmb3IgZWFjaCB0cmFuc2NyaXB0IGJ5IGl0J3MgbGVuZ3RoICgqcmVhZHMgcGVyIGtpbG9iYXNlKiAtIFJQSykKLSBzdW0gdGhlIFJQSyB2YWx1ZXMgYW5kIGRpdmlkZSBieSAxIG1pbGxpb24gdG8gZ2V0IGEgc2NhbGluZyBmYWN0b3IKLSBkaXZpZGUgdGhlIFJQSyB2YWx1ZXMgYnkgdGhlIHNjYWxpbmcgZmFjdG9yIHRvIGdldCB0aGUgKlRQTSoKCgpgYGB7cn0KcnBrIDwtIHF1YW50cyROdW1SZWFkcyAvIHF1YW50cyRFZmZlY3RpdmVMZW5ndGgKCnNjYWxlX2ZhY3RvciA8LSBzdW0ocnBrKSAvIDFlNgoKdHBtIDwtIHJwayAvIHNjYWxlX2ZhY3RvcgoKYGBgCgoKIyMjIERlZmluaW5nIHRoZSB0cmFuc2NyaXB0IG1hcHBpbmcKCgpJbiBvcmRlciBmb3IgYHR4aW1wb3J0YCB0byBnaXZlICpnZW5lLWxldmVsKiBjb3VudHMsIHdlIG5lZWQgdG8gc3VwcGx5IGEgZGF0YSBmcmFtZSB0aGF0IGNhbiBiZSB1c2VkIHRvIGFzc29jaWF0ZSBlYWNoIHRyYW5zY3JpcHQgbmFtZSB3aXRoIGEgZ2VuZSBpZGVudGlmaWVyLiAqKkl0IGlzIGltcG9ydGFudCB0byB1c2UgYSB0cmFuc2NyaXB0IGZpbGUgdGhhdCBjb3JyZXNwb25kcyB0byB0aGUgbmFtZSBnZW5vbWUgYnVpbGQgYXMgdGhlIGZpbGUgdXNlZCB0byBjb3VudCB0aGUgdHJhbnNjcmlwdHMqKi4gCgpXZSBjYW4gY2hlY2sgaWYgdGhlIGBndGZgIGZpbGUgZXhpc3RzIGluIHRoZSBkaXJlY3Rvcnkgd2UgZXhwZWN0IGJ5IHJ1bm5pbmcgdGhlIGBmaWxlLmV4aXN0c2AgZnVuY3Rpb247IHJldHVybmluZyBgVFJVRWAgb3IgYEZBTFNFYAoKYGBge3J9Cmd0Zl9maWxlIDwtICJNdXNfbXVzY3VsdXMuR1JDbTM4LjkxLmNoci5ndGYuZ3oiCmZpbGUuZXhpc3RzKGd0Zl9maWxlKQpgYGAKCklmIHJlcXVpcmVkLCB3ZSBjYW4gZG93bmxvYWQgZnJvbSB0aGUgRW5zZW1ibCBGVFAgc2l0ZS4gCgoKYGBge3IgZXZhbD1GQUxTRX0gCmRvd25sb2FkLmZpbGUoImZ0cDovL2Z0cC5lbnNlbWJsLm9yZy9wdWIvcmVsZWFzZS05MS9ndGYvbXVzX211c2N1bHVzL011c19tdXNjdWx1cy5HUkNtMzguOTEuY2hyLmd0Zi5neiIsZGVzdGZpbGUgPSBndGZfZmlsZSkKCmBgYAoKCiMjIyBOb3RlIG9uIGFuYWx5c2luZyB5b3VyIG93biBkYXRhCgohW2ltYWdlcy9kb3dubG9hZF9ndGYucG5nXQoKSWYgYW5hbHlzaW5nIHlvdXIgb3duIGRhdGEsIHlvdSB3aWxsIGhhdmUgdG8gbG9jYXRlIHRoZSBndGYgZmlsZSBvbiB0aGUgRW5zZW1ibCBGVFAgc2l0ZS4gSWYgeW91IGVudGVyIGBmdHA6Ly9mdHAuZW5zZW1ibC5vcmcvcHViL3JlbGVhc2UtOTEvZ3RmYCBpbnRvIGEgd2ViIGJyb3dzZXIgeW91IHdpbGwgYmUgYWJsZSB0byBuYXZpZ2F0ZSB0aGUgc2l0ZSBhbmQgZmluZCB5b3VyIG9yZ2FuaXNtIG9mIGludGVyZXN0LiBCeSByaWdodC1jbGlja2luZyBvbiB0aGUgbmFtZSBvZiB0aGUgZ3RmIHlvdSB3aWxsIGJlIGFibGUgdG8gY29weSB0aGUgVVJMIGFuZCB0aGVuIHBhc3RlIGludG8gUlN0dWRpby4KCmBgYHtyIGV2YWw9RkFMU0V9Cmd0Zl9maWxlIDwtICJlbnNlbWJsX3JlZi9teV9yZWYuZ3RmIgpkb3dubG9hZC5maWxlKFBBU1RFX0xJTktfRlJPTV9FTlNFTUJMX0hFUkUsZGVzdGZpbGUgPSBndGZfZmlsZSkKYGBgCgojIyMgQ3JlYXRpbmcgYSB0cmFuc2NyaXB0IGRhdGFiYXNlCgpUaGUgQmlvY29uZHVjb3Igd2Vic2l0ZSBwcm92aWRlcyBtYW55IHByZS1idWlsdCB0cmFuc2NyaXB0IGRhdGFiYXNlcyBmb3Igc29tZSBvcmdhbmlzbXMgKEh1bWFuLCBNb3VzZSxSYXQgZXRjKSAgd2hpY2ggcHJvdmlkZSB0cmFuc2NyaXB0IGRlZmluaXRpb25zIGFuZCBhbGxvdyB1c2VycyB0byBxdWVyeSB0aGUgbG9jYXRpb25zIG9mIHBhcnRpY3VsYXIgZ2VuZXMsIGV4b25zIGFuZCBvdGhlciBnZW5vbWljIGZlYXR1cmVzLiBZb3UgbWF5IGZpbmQgYSBwcmUtYnVpbHQgcGFja2FnZSB0aGF0IGFscmVhZHkgaGFzIHRoZSB0cmFuc2NyaXB0IGxvY2F0aW9ucyByZXF1aXJlZCB0byBjcmVhdGUgdGhlIHRyYW5zY3JpcHQgbWFwcGluZyBmaWxlLiBDaGVjayBvdXQgdGhlIGFubm90YXRpb24gc2VjdGlvbiBvZiB0aGUgQmlvY29uZHVjdG9yIHdlYnNpdGUgLSBodHRwOi8vYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9yZWxlYXNlL0Jpb2NWaWV3cy5odG1sI19fX0Fubm90YXRpb25EYXRhIGFuZCBsb29rIGZvciBwYWNrYWdlcyBzdGFydGluZyBgVHhEYi4uLmAKCkhvd2V2ZXIsIGl0IGlzIHF1aXRlIGVhc3kgdG8gYnVpbGQgc3VjaCBhIGRhdGFiYXNlIGlmIHdlIGhhdmUgYSBgZ3RmYCBmaWxlIHVzaW5nIHRoZSBgR2Vub21pY0ZlYXR1cmVzYCBpbmZyYXN0cnVjdHVyZS4KCmBgYHtyIG1lc3NhZ2U9RkFMU0V9CiMjIENvdWxkIHRha2UgYSBmZXcgbWludXRlcyB0byBydW4gdGhlIG1ha2VUeERiRnJvbUdGRiBjb21tYW5kCmxpYnJhcnkoR2Vub21pY0ZlYXR1cmVzKQp0eGRiIDwtIG1ha2VUeERiRnJvbUdGRihndGZfZmlsZSkKCmBgYAoKVGhlIGRhdGFiYXNlIGhhcyBhIG51bWJlciBvZiBwcmVkZWZpbmVkICJrZXlzIiBhbmQgImNvbHVtbnMiIHRoYXQgaGF2ZSB0byBiZSBzcGVjaWZpZWQgd2hlbiBjcmVhdGluZyBhIHF1ZXJ5CgpgYGB7cn0Ka2V5dHlwZXModHhkYikKYGBgCgpgYGB7cn0KY29sdW1ucyh0eGRiKQpgYGAKClNvbWV0aW1lcyB3ZSB3b3VsZCB3YW50IHRvIHF1ZXJ5IHRoZSBwb3NpdGlvbnMgZm9yIGEgbGltaXRlZCBzZXQgb2Ygc2VsZWN0ZWQgZ2VuZXMgKHBlcmhhcHMgdGhlIHJlc3VsdHMgb2YgYSBkaWZmZXJlbnRpYWwtZXhwcmVzc2lvbiBhbmFseXNpcyksIGJ1dCBpbiB0aGlzIGNhc2Ugd2Ugd2FudCB0aGUgZ2VuZSBuYW1lcyB0aGF0IGNvcnJlc3BvbmQgdG8gZXZlcnkgdHJhbnNjcmlwdCBpbiB0aGUgZGF0YWJhc2UuIFRvIGdldCB0aGUgbmFtZXMgb2YgYWxsIHRyYW5zY3JpcHRzIHdlIGNhbiB1c2UgdGhlIGBrZXlzYCBmdW5jdGlvbi4gV2UgdGhlbiBjb21wb3NlIHRoZSBxdWVyeSB1c2luZyB0aGUgYHNlbGVjdGAgZnVuY3Rpb24gdG8gcmV0dXJuIGEgZGF0YSBmcmFtZQoKYGBge3J9CmsgPC0ga2V5cyh0eGRiLCBrZXl0eXBlPSJUWE5BTUUiKQp0eF9tYXAgPC0gc2VsZWN0KHR4ZGIsIGtleXMgPSBrLCBjb2x1bW5zPSJHRU5FSUQiLCBrZXl0eXBlID0gIlRYTkFNRSIpCmhlYWQodHhfbWFwKQpgYGAKClN1Y2ggYSBkYXRhIGZyYW1lICpzaG91bGQqIGJlIHN1ZmZpY2llbnQgdG8gYWxsb3cgdXMgdG8gdXNlIHRoZSBgdHhpbXBvcnRgIHBhY2thZ2UuIFRoZXJlIGlzIGEgYml0IG9mIGEgcHJvYmxlbSB0aG91Z2guLgoKYGBge3IgZXZhbD1GQUxTRX0KbGlicmFyeSh0eGltcG9ydCkKdHgyZ2VuZSA8LSB0eF9tYXAKd3JpdGUuY3N2KHR4MmdlbmUsZmlsZT0idHgyZ2VuZS5jc3YiLHJvdy5uYW1lcyA9IEZBTFNFLHF1b3RlPUZBTFNFKQp0eGkgPC0gdHhpbXBvcnQocXVhbnRfZmlsZXMsdHlwZT0ic2FsbW9uIix0eDJnZW5lID0gdHgyZ2VuZSkKYGBgCgpJbiB0aGlzIGNhc2UgUiBpcyByZXBvcnRpbmcgYSB1c2VmdWwgZXJyb3IgbWVzc2FnZTsgdGhlIElEcyB3ZSBoYXZlIHN1cHBsaWVkIGluIHRoZSBgdHgyZ2VuZWAgZGF0YSBmcmFtZSBkbyBub3QgY29ycmVzcG9uZCB0byB0aGUgdHJhbnNjcmlwdCBuYW1lcyBpbiBvdXIgZmlsZXMuCgpgYGB7cn0KdGFibGUodHhfbWFwJFRYTkFNRSAlaW4lIHF1YW50cyROYW1lKQpgYGAKCkZvcnR1bmF0ZWx5IHRoZSBhdXRob3JzIG9mIGB0eGltcG9ydGAgaGF2ZSByZWNvZ25pc2VkIHRoaXMgYXMgYSBjb21tb24gcHJvYmxlbSBhbmQgYWRkZWQgYW4gYXJndW1lbnQgdG8gdGhlIGB0eGltcG9ydGAgZnVuY3Rpb24gdGhhdCBjYW4gYmUgdXNlZCB0byBvdmVyY29tZSB0aGUgZXJyb3IuCgpgYGB7cn0KbGlicmFyeSh0eGltcG9ydCkKdHgyZ2VuZSA8LSB0eF9tYXAKdHhpIDwtIHR4aW1wb3J0KHF1YW50X2ZpbGVzLHR5cGU9InNhbG1vbiIsdHgyZ2VuZSA9IHR4MmdlbmUsaWdub3JlVHhWZXJzaW9uID0gVFJVRSkKYGBgCgpUaGUgcmVzdWx0aW5nIG9iamVjdCBpcyBhIHNpbXBsZSBgbGlzdGAgc3RydWN0dXJlIGluIFIgd2hpY2ggY29udGFpbnMgYSBudW1iZXIgb2YgY29tcG9uZW50cyB0aGF0IHdlIGNhbiBhY2Nlc3MgdXNpbmcgYSBgJGAgb3BlcmF0b3IKCmBgYHtyfQpuYW1lcyh0eGkpCmBgYAoKYGBge3J9CmhlYWQodHhpJGNvdW50cykKYGBgCgpgYGB7cn0KYWxsKHJvd25hbWVzKHNhbXBsZWluZm8pID09IGNvbG5hbWVzKHR4aSRjb3VudHMpKQpgYGAKCgpUaGlzIHdhcyBhIGZhaXJseSBzaW1wbGUgZml4LCBidXQgd2UgY291bGQgaGF2ZSBhbHNvIHVzZWQgc29tZSBmdW5jdGlvbmFsaXR5IGZyb20gdGhlIGB0aWR5dmVyc2VgIGNvbGxlY3Rpb24gb2YgcGFja2FnZXMgdG8gbWFrZSB0aGUgbmFtZXMgY29tcGF0aWJsZS4KCiMjIyBGaXhpbmcgdGhlIHByb2JsZW0gd2l0aCB0cmFuc2NyaXB0IG5hbWVzIC0gdGhlIGhhcmQgd2F5IQoKTm9uZSBvZiB0aGUgdHJhbnNjcmlwdCBuYW1lcyBhcmUgbWF0Y2hpbmcgdXAuIElmIHdlIGxvb2sgYXQgdGhlIHRyYW5zY3JpcHQgbmFtZXMgaW4gdGhlIGBxdWFudGAgZmlsZXMgdGhleSBoYXZlIGEgYC5gIHN5bWJvbCBmb2xsb3dlZCBieSBhIG51bWJlciBhZnRlciB0aGUgdHJhbnNjcmlwdCBuYW1lLiBUaGUgYHRpZHlyYCBwYWNrYWdlIGNvbnRhaW5zIHNldmVyYWwgdXNlZnVsIGZ1bmN0aW9ucyBmb3IgKnRpZHlpbmcqIGEgZGF0YSBmcmFtZS4gT25lIGZ1bmN0aW9uIHdlIHdpbGwgdXNlIGluIHBhcnRpY3VsYXIgaXMgdG8gc3BsaXQgYSBjb2x1bW4gb2YgdmFsdWUgdGhhdCBjb250YWluIG11bHRpcGxlIHBpZWNlcyBvZiBpbmZvcm1hdGlvbiBzZXBhcmF0ZWQgYnkgYSBjb21tb24gY2hhcmFjdGVyLiBJbiB0aGlzIGNhc2UgdGhlIGNoYXJhY3RlciBpcyAiLiIgYnV0IG90aGVyIGNvbW1vbiBzZXBhcmF0aW5nIGNoYXJhY3RlcnMgaW5jbHVkZSBgLWAsIGBfYCwgYDpgLiBUaGUgYHNlcGFyYXRlYCBmdW5jdGlvbiBpcyBhYmxlIHRvIGRldGVjdCB3aGljaCBjaGFyYWN0ZXIgaXMgYmVpbmcgdXNlZC4KCkhlcmUgd2UgY3JlYXRlIHR3byBuZXcgY29sdW1ucywgYFRYTkFNRWAgYW5kIGBOdW1iZXJgLiBUaGUgYFRYTkFNRWAgYmVpbmcgdGhlIGNvbHVtbiB0aGF0IHNob3VsZCBtYXRjaCBlbnRyaWVzIGluIHRoZSBgdHhfbWFwYCBkYXRhIGZyYW1lLgoKYGBge3J9CmxpYnJhcnkodGlkeXIpCnF1YW50cyA8LSBzZXBhcmF0ZShxdWFudHMsIE5hbWUsIGMoIlRYTkFNRSIsIk51bWJlciIpLHJlbW92ZSA9IEZBTFNFKQpoZWFkKHF1YW50cykKYGBgCgpBbm90aGVyIHVzZWZ1bCBwaWVjZSBvZiBgdGlkeXZlcnNlYCBmdW5jdGlvbmFsaXR5IGlzIHRvIGpvaW4gdHdvIHRhYmxlcyB0b2dldGhlciBiYXNlZCBvbiBhIGNvbW1vbiBjb2x1bW4uIFdlIHdhbnQgdG8gY3JlYXRlIGEgZGF0YSBmcmFtZSB0aGF0IGhhcyB0aGUgdHJhbnNjcmlwdCBuYW1lIGFzIGl0IGFwcGVhcnMgaW4gdGhlIHF1YW50IGZpbGVzIGFuZCB0aGUgZ2VuZSBuYW1lLiBUaGlzIGNhbiBiZSBhY2hpZXZlZCBieSBhIGBsZWZ0X2pvaW5gIHRvIGVuc3VyZSB0aGF0IGFsbCB0aGUgdHJhbnNjcmlwdCBJRHMgaW4gdGhlIHF1YW50IGZpbGVzIGhhdmUgYW4gZW50cnkgaW4gdGhlIGNvbWJpbmVkIHRhYmxlLgoKTi5CLiB0aG9zZSBhbHJlYWR5IGZhbWlsaWFyIHdpdGggUiBtaWdodCBoYXZlIGVuY291bnRlcmVkIHRoZSBgbWVyZ2VgIGZ1bmN0aW9uIGZvciBhY2hpZXZpbmcgYSBzaW1pbGFyIHJlc3VsdC4KCmBgYHtyfQpsaWJyYXJ5KGRwbHlyKQpxdWFudHMgPC0gbGVmdF9qb2luKHF1YW50cywgdHhfbWFwLCBieT0iVFhOQU1FIikKaGVhZChxdWFudHMpCmBgYAoKVG8gY3JlYXRlIHRoZSBgdHgyZ2VuZWAgd2UgY2FuIHB1bGwgb3V0IHRoZSBjb2x1bW5zIHRoYXQgd2UgcmVxdWlyZSBmcm9tIHRoaXMgbWVyZ2VkIGRhdGEgZnJhbWUuIFRoZXJlIGlzIGFub3RoZXIgYGRwbHlyYCBmdW5jdGlvbiBmb3IgZG9pbmcgdGhpcyB0aGF0IGhhcyBzaW1wbGVyIHN5bnRheCB0aGFuIHRoZSAqImJhc2UgUiIqIG1ldGhvZCBvZiBzdWJzZXR0aW5nIGEgZGF0YSBmcmFtZS4KCmBgYHtyfQp0eDJnZW5lIDwtIGRwbHlyOjo6c2VsZWN0KHF1YW50cywgTmFtZSwgR0VORUlEKQpoZWFkKHR4MmdlbmUpCmBgYAoKV2UgbWlnaHQgd2FudCB0byBmdXJ0aGVyIGNoZWNrIHRoYXQgdGhlcmUgYXJlIG5vICJtaXNzaW5nIHZhbHVlcyIgaW4gdGhlIEdlbmUgSUQgY29sdW1uIGFuZCByZW1vdmUgdGhlbSBmcm9tIHRoZSBkYXRhIGZyYW1lIAoKYGBge3J9CmFueShpcy5uYSh0eDJnZW5lJEdFTkVJRCkpCgp0eDJnZW5lIDwtIGZpbHRlcih0eDJnZW5lLCAhaXMubmEoR0VORUlEKSkKCmBgYAoKCmBgYHtyfQpsaWJyYXJ5KHR4aW1wb3J0KQoKdHhpIDwtIHR4aW1wb3J0KHF1YW50X2ZpbGVzLHR5cGU9InNhbG1vbiIsdHgyZ2VuZSA9IHR4MmdlbmUpCgpgYGAKCgoKCiMjIFF1YWxpdHkgY29udHJvbCBvZiB0aGUgaW1wb3J0ZWQgY291bnRzCgpXZSB3aWxsIGJlIHVzaW5nIHRoZSBgREVTZXEyYCBsaWJyYXJ5IHRvIGFuYWx5c2UgdGhpcyBkYXRhc2V0LiBBcyBwYXJ0IG9mIHRoZSBbREVTZXEyIHZpZ25ldHRlXShodHRwOi8vYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9kZXZlbC9iaW9jL3ZpZ25ldHRlcy9ERVNlcTIvaW5zdC9kb2MvREVTZXEyLmh0bWwpIHlvdSB3aWxsIHNlZSBleGFtcGxlcyBvZiBpbXBvcnRpbmcgY291bnQgZGF0YSBmcm9tIGRpZmZlcmVudCBzb3VyY2VzLiBJbiBhbGwgY2FzZXMsIFtyYXcgY291bnQgZGF0YSBhcmUgZXhwZWN0ZWRdKGh0dHA6Ly9iaW9jb25kdWN0b3Iub3JnL3BhY2thZ2VzL2RldmVsL2Jpb2MvdmlnbmV0dGVzL0RFU2VxMi9pbnN0L2RvYy9ERVNlcTIuaHRtbCN3aHktdW4tbm9ybWFsaXplZC1jb3VudHMpLiAKCkZvciB0aGlzIHdvcmtmbG93IHdlIGFyZSBnb2luZyB0byBpbXBvcnQgZGF0YSBmcm9tIGB0eGltcG9ydGAgd2l0aCB0aGUgYERFU2VxRGF0YVNldEZyb21UeGltcG9ydGAgZnVuY3Rpb24gYWxvbmcgd2l0aCB0aGUgc2FtcGxlIGluZm9ybWF0aW9uIHRoYXQgd2UgY3JlYXRlZCBlYXJsaWVyLiAKCkEgKmRlc2lnbiogZm9yIHRoZSBleHBlcmltZW50IGFsc28gbmVlZHMgdG8gYmUgc3BlY2lmaWVkLiBUaGlzIHdpbGwgZGVmaW5lIGhvdyB0aGUgZGlmZmVyZW50aWFsIGV4cHJlc3Npb24gYW5hbHlzaXMgaXMgY2FycmllZCBvdXQsIGJ1dCBjYW4gYmUgY2hhbmdlZCBhdCBhIGxhdGVyIHN0YWdlIHNvIHdlIHdpbGwgdXNlIGBDZWxsVHlwZWAgZm9yIG5vdyBhcyBvdXIgZmFjdG9yIG9mIGludGVyZXN0LgoKYGBge3IgbWVzc2FnZT1GQUxTRX0KbGlicmFyeShERVNlcTIpCmRkcyA8LSBERVNlcURhdGFTZXRGcm9tVHhpbXBvcnQodHhpLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb2xEYXRhID0gc2FtcGxlaW5mbywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkZXNpZ24gPC0gfkNlbGxUeXBlKQpgYGAKClRoZSBvYmplY3QgY29udGFpbnMgYWxsIHRoZSBtZXRhZGF0YSBmb3IgdGhlIGV4cGVyaW1lbnQsIGFsb25nIHdpdGggdGhlIGNvdW50cy4gCgpgYGB7cn0KY29sRGF0YShkZHMpCmBgYAoKCldlIHdpbGwgYmUgdXNpbmcgdGhlc2UgcmF3IGNvdW50cyB0aHJvdWdob3V0IHRoZSB3b3Jrc2hvcCBhbmQgdHJhbnNmb3JtaW5nIHRoZW0gdXNpbmcgbWV0aG9kcyBpbiB0aGUgYERFU2VxMmAgcGFja2FnZS4gSWYgVFBNIHZhbHVlcyBhcmUgZGVzaXJlZCBmb3Igc29tZSBvdGhlciBhcHBsaWNhdGlvbiwgd2UgY2FuIGV4dHJhY3QgdGhlbSBmcm9tIHRoZSBgdHhpbXBvcnRgIG9iamVjdC4gVGhlc2UgYXJlIHRoZSB0cmFuc2NyaXB0LWxldmVsIFRQTSB2YWx1ZXMgdGhhdCB3ZSBzYXcgZWFybGllciBpbiB0aGUgcXVhbnQgZmlsZXMgdGhhdCBoYXZlIGJlZW4gc3VtbWFyaXNlZCB0byB0aGUgZ2VuZS1sZXZlbC4KCmBgYHtyfQp0cG0gPC0gdHhpJGFidW5kYW5jZQp3cml0ZS5jc3YodHBtLCBmaWxlPSJ0cG1fdmFsdWVzLmNzdiIscXVvdGU9RkFMU0UpCmBgYAoKQXQgdGhlIHRpbWUgb2Ygd3JpdGluZywgKlRQTSogaXMgdGhlIHJlY29tbWVuZGVkIHdheSBvZiB0cmFuc2Zvcm1pbmcgUk5BLXNlcSBjb3VudHMgdG8gYWNjb3VudCBmb3IgZ2VuZSBsZW5ndGggYW5kIGxpYnJhcnkgY29tcG9zaXRpb24gYmlhc2VzLiBERVNlcTIgYWxzbyBwcm92aWRlcyBtZXRob2RzIGZvciBleHRyYWN0aW5nIG5vcm1hbGlzZWQgY291bnRzIGFzICpGUEtNKiAoZnJhZ21lbnRzIHBlciBraWxvYmFzZSBwZXIgbWlsbGlvbiBtYXBwZWQgZnJhZ21lbnRzKSBhbmQgKkZQTSogKGZyYWdtZW50cyBwZXIgbWlsbGlvbiBtYXBwZWQgZnJhZ21lbnRzKSB3aGljaCBtaWdodCBiZSByZXF1aXJlZCBmb3Igc29tZSBvdGhlciBhbmFseXNpcyBvciB2aXN1YWxpc2F0aW9uIG91dHNpZGUgb2YgQmlvY29uZHVjdG9yIG9yIGBERVNlcTJgLgoKYGBge3J9CmZwbSA8LSBmcG0oZGRzKQp3cml0ZS5jc3YoZnBtLCBmaWxlPSJmcG1fdmFsdWVzLmNzdiIscXVvdGU9RkFMU0UpCmZwa20gPC0gZnBrbShkZHMpCndyaXRlLmNzdihmcGttLCBmaWxlPSJmcGttX3ZhbHVlcy5jc3YiLHF1b3RlPUZBTFNFKQpgYGAKCgojIyMgVmlzdWFsaXNpbmcgbGlicmFyeSBzaXplcwoKV2UgY2FuIGxvb2sgYXQgYSBmZXcgZGlmZmVyZW50IHBsb3RzIHRvIGNoZWNrIHRoYXQgdGhlIGRhdGEgaXMgZ29vZCBxdWFsaXR5LCBhbmQgdGhhdCB0aGUgc2FtcGxlcyBhcmUgYXMgd2Ugd291bGQgZXhwZWN0LiBGaXJzdCwgd2UgY2FuIGNoZWNrIGhvdyBtYW55IHJlYWRzIHdlIGhhdmUgZm9yIGVhY2ggc2FtcGxlIGluIHRoZSBgREVTZXFEYXRhU2V0YC4gVGhlIGNvdW50cyB0aGVtc2VsdmVzIGFyZSBhY2Nlc3NlZCB1c2luZyB0aGUgYGFzc2F5YCBmdW5jdGlvbjsgZ2l2aW5nIGEgbWF0cml4IG9mIGNvdW50cy4gVGhlIHN1bSBvZiBhIHBhcnRpY3VsYXIgY29sdW1uIGlzIHRoZXJlZm9yZSB0aGUgdG90YWwgbnVtYmVyIG9mIHJlYWRzIGZvciB0aGF0IHNhbXBsZS4KCmBgYHtyfQpzdW0oYXNzYXkoZGRzKVssMV0pCmBgYAoKQSBjb252ZW5pZW5jZSBmdW5jdGlvbiBgY29sU3Vtc2AgZXhpc3RzIGZvciBjYWxjdWxhdGluZyB0aGUgc3VtIG9mIGVhY2ggY29sdW1uIGluIGEgbWF0cml4LCByZXR1cm5pbmcgYSBgdmVjdG9yYCBhcyBhIHJlc3VsdC4KCmBgYHtyIGRnZUxpYnJhcnlTaXplc30KY29sU3Vtcyhhc3NheShkZHMpKQoKYGBgCgoKPiAjIyBDaGFsbGVuZ2UgMSB7LmNoYWxsZW5nZX0KPgo+IDEuIFByb2R1Y2UgYSBiYXIgcGxvdCB0byBzaG93IHRoZSBNaWxsaW9ucyBvZiByZWFkcyBmb3IgZWFjaCBzYW1wbGUKPiAyLiBDaGFuZ2UgdGhlIG5hbWVzIHVuZGVyIHRoZSBwbG90IHNvIHRoZXkgYXJlIHRoZSAiTmFtZSIgb2YgdGhlIHNhbXBsZSByYXRoZXIgdGhhbiBSdW4gTmFtZQo+IDMuIEFkZCBhIGhvcml6b250YWwgbGluZSBhdCAyMCBtaWxsaW9uIHJlYWRzCgo+IEhJTlQ6IGxvb2sgYXQgdGhlIGhlbHAgZm9yIHRoZSBiYXNlIGdyYXBoaWNzIGZ1bmN0aW9ucyBgYmFycGxvdGAgYW5kIGBhYmxpbmVgIChvciB0cnkgYW5kIHVzZSBgZ2dwbG90MmAgaWYgeW91IHByZWZlcikKCiFbXShpbWFnZXMvbGliX3NpemUucG5nKQoKIyMjIEZpbHRlcmluZyBub24tZXhwcmVzc2VkIGdlbmVzCgpUaGUgZGF0YXNldCBpcyB1c3VhbGx5IGZpbHRlcmVkIGF0IHRoaXMgc3RhZ2UgdG8gcmVtb3ZlIGFueSBnZW5lcyB0aGF0IGFyZSBub3QgZXhwcmVzc2VkLiBBbHRob3VnaCBub3Qgc3RyaWN0bHkgcmVxdWlyZWQgZm9yIHRoZSBERVNlcTIgZGlmZmVyZW50aWFsIGV4cHJlc3Npb24gYWxnb3JpdGhtLCBpdCBjYW4gcmVkdWNlIHRoZSB0aW1lIGFuZCBtZW1vcnkgcmVxdWlyZWQgdG8gcGVyZm9ybSBzb21lIG9mIHRoZSBhbmFseXNpcy4gTGV0J3Mgc2F5IHRoYXQgZm9yIGEgZ2VuZSB0byBiZSAiZXhwcmVzc2VkIiBpbiBhIHBhcnRpY3VsYXIgc2FtcGxlIHdlIG5lZWQgdG8gc2VlIDUgb3IgbW9yZSBjb3VudHMgIAoKCmBgYHtyfQppc19leHByZXNzZWQgPC0gYXNzYXkoZGRzKSA+PSA1CmhlYWQoaXNfZXhwcmVzc2VkKQpgYGAKClIgaXMgaGFwcHkgdG8gdGhpbmsgb2YgbG9naWNhbCB2YWx1ZXMgKGBUUlVFYCBvciBgRkFMU0VgKSBhcyB0aGUgaW50ZWdlcnMgYDBgIG9yIGAxYC4gVGhlcmVmb3JlIGlmIHdlIGNhbGN1bGF0ZSB0aGUgc3VtIGFjcm9zcyBhIHBhcnRpY3VsYXIgcm93IGl0IHdpbGwgZ2l2ZSB0aGUgbnVtYmVyIG9mIHNhbXBsZXMgdGhhdCBnZW5lIGlzIGV4cHJlc3NlZCBpbi4KCmBgYHtyfQpzdW0oaXNfZXhwcmVzc2VkWzEsXSkKc3VtKGlzX2V4cHJlc3NlZFsyLF0pCgpgYGAKCkluIGEgc2ltaWxhciBtYW5uZXIgdG8gYGNvbFN1bXNgLCBgcm93U3Vtc2Agd2lsbCBnaXZlIHRoZSBzdW0gb2YgZWFjaCByb3cgaW4gYSBtYXRyaXggYW5kIHJldHVybiB0aGUgcmVzdWx0IGFzIGEgYHZlY3RvcmAuCgpgYGB7cn0KaGlzdChyb3dTdW1zKGlzX2V4cHJlc3NlZCksbWFpbj0iTnVtYmVyIG9mIHNhbXBsZXMgYSBnZW5lIGlzIGV4cHJlc3NlZCBpbiIseGxhYj0iU2FtcGxlIENvdW50IikKYGBgCgpJdCBzZWVtcyB0aGF0IGdlbmVzIGFyZSBlaXRoZXIgZXhwcmVzc2VkIGluIGFsbCBzYW1wbGVzLCBvciBub3QgZXhwcmVzc2VkIGF0IGFsbC4gV2Ugd2lsbCBkZWNpZGUgdG8ga2VlcCBnZW5lcyB0aGF0IGFyZSBleHByZXNzZWQgaW4gYXQgbGVhc3QgMiBzYW1wbGVzLgoKYGBge3J9CmtlZXAgPC0gcm93U3Vtcyhhc3NheShkZHMpID49IDUpID49IDIKdGFibGUoa2VlcCkKZGRzIDwtIGRkc1trZWVwLF0KCmBgYAoKIyMjIFZpc3VhbGlzaW5nIGNvdW50IGRpc3RyaWJ1dGlvbnMKCldlIHR5cGljYWxseSB1c2UgYSBgYm94cGxvdGAgdG8gdmlzdWFsaXNlIGRpZmZlcmVuY2UgdGhlIGRpc3RyaWJ1dGlvbnMgb2YgdGhlIGNvbHVtbnMgb2YgYSBudW1lcmljIGRhdGEgZnJhbWUuIEFwcGx5aW5nIHRoZSBgYm94cGxvdGAgZnVuY3Rpb24gdG8gdGhlIHJhdyBjb3VudHMgZnJvbSBvdXIgZGF0YXNldCByZXZlYWxzIHNvbWV0aGluZyBhYm91dCB0aGUgbmF0dXJlIG9mIHRoZSBkYXRhOyB0aGUgZGlzdHJpYnV0aW9ucyBhcmUgZG9taW5hdGVkIGJ5IGEgZmV3IGdlbmVzIHdpdGggdmVyeSBsYXJnZSBjb3VudHMuCgpgYGB7cn0KYm94cGxvdChhc3NheShkZHMpKQpgYGAKCmBgYHtyIG1lc3NhZ2U9RkFMU0V9CmJveHBsb3QobG9nMTAoYXNzYXkoZGRzKSkpCmBgYAoKCgogV2UgY2FuIHVzZSB0aGUgYHZzdGAgb3IgYHJsb2dgIGZ1bmN0aW9uIGZyb20gYERFU2VxMmB0byBjb21wZW5zYXRlIGZvciB0aGUgZWZmZWN0IG9mIGRpZmZlcmVudCBsaWJyYXJ5IHNpemVzIGFuZCBwdXQgdGhlIGRhdGEgb24gdGhlIGxvZyRfMiQgc2NhbGUuIFRoZSBlZmZlY3QgaXMgdG8gcmVtb3ZlIHRoZSBkZXBlbmRlbmNlIG9mIHRoZSB2YXJpYW5jZSBvbiB0aGUgbWVhbiwgcGFydGljdWxhcmx5IHRoZSBoaWdoIHZhcmlhbmNlIG9mIHRoZSBsb2dhcml0aG0gb2YgY291bnQgZGF0YSB3aGVuIHRoZSBtZWFuIGlzIGxvdy4gRm9yIG1vcmUgZGV0YWlscyBzZWUgdGhlIFtERVNlcTIgdmlnbmV0dGVdKGh0dHA6Ly9iaW9jb25kdWN0b3Iub3JnL3BhY2thZ2VzL2RldmVsL2Jpb2MvdmlnbmV0dGVzL0RFU2VxMi9pbnN0L2RvYy9ERVNlcTIuaHRtbCNjb3VudC1kYXRhLXRyYW5zZm9ybWF0aW9ucykKCgoKYGBge3J9CiMgR2V0IGxvZzIgY291bnRzCnZzZCA8LSB2c3QoZGRzLGJsaW5kPVRSVUUpCiMgQ2hlY2sgZGlzdHJpYnV0aW9ucyBvZiBzYW1wbGVzIHVzaW5nIGJveHBsb3RzCmJveHBsb3QoYXNzYXkodnNkKSwgeGxhYj0iIiwgeWxhYj0iTG9nMiBjb3VudHMgcGVyIG1pbGxpb24iLGxhcz0yLG1haW49Ik5vcm1hbGlzZWQgRGlzdHJpYnV0aW9ucyIpCiMgTGV0J3MgYWRkIGEgYmx1ZSBob3Jpem9udGFsIGxpbmUgdGhhdCBjb3JyZXNwb25kcyB0byB0aGUgbWVkaWFuIGxvZ0NQTQphYmxpbmUoaD1tZWRpYW4oYXNzYXkodnNkKSksIGNvbD0iYmx1ZSIpCmBgYAoKIyMjIEhlYXRtYXAgb2YgdGhlIHNhbXBsZS10by1zYW1wbGUgZGlzdGFuY2VzCgpBbm90aGVyIHVzZSBvZiB0aGUgdHJhbnNmb3JtZWQgZGF0YSBpcyBzYW1wbGUgY2x1c3RlcmluZy4gSGVyZSwgd2UgYXBwbHkgdGhlIGBkaXN0YCBmdW5jdGlvbiB0byB0aGUgdHJhbnNwb3NlIG9mIHRoZSB0cmFuc2Zvcm1lZCBjb3VudCBtYXRyaXggdG8gZ2V0IHNhbXBsZS10by1zYW1wbGUgZGlzdGFuY2VzLgoKCmBgYHtyfQpzYW1wbGVEaXN0cyA8LSBkaXN0KHQoYXNzYXkodnNkKSkpCmBgYAoKQSBoZWF0bWFwIG9mIHRoaXMgZGlzdGFuY2UgbWF0cml4IGdpdmVzIHVzIGFuIG92ZXJ2aWV3IG92ZXIgc2ltaWxhcml0aWVzIGFuZCBkaXNzaW1pbGFyaXRpZXMgYmV0d2VlbiBzYW1wbGVzLiBCeSByZS1uYW1pbmcgdGhlIHJvd3MgYW5kIGNvbHVtbnMgb2YgdGhlIGRpc3RhbmNlIG1hdHJpeCB3ZSBjYW4gbWFrZSB0aGUgcGxvdCBlYXNpZXIgdG8gaW50ZXJwcmV0LgoKYGBge3J9CmxpYnJhcnkoUkNvbG9yQnJld2VyKQpsaWJyYXJ5KHBoZWF0bWFwKQpzYW1wbGVEaXN0TWF0cml4IDwtIGFzLm1hdHJpeChzYW1wbGVEaXN0cykKcm93bmFtZXMoc2FtcGxlRGlzdE1hdHJpeCkgPC0gcGFzdGUoY29sRGF0YShkZHMpJENlbGxUeXBlLCBjb2xEYXRhKGRkcykkU3RhdHVzLCBzZXA9Ii0iKQpjb2xuYW1lcyhzYW1wbGVEaXN0TWF0cml4KSA8LSBjb2xEYXRhKGRkcykkTmFtZQpjb2xvcnMgPC0gY29sb3JSYW1wUGFsZXR0ZSggcmV2KGJyZXdlci5wYWwoOSwgIkJsdWVzIikpICkoMjU1KQoKCnBoZWF0bWFwKHNhbXBsZURpc3RNYXRyaXgsCiAgICAgICAgIGNvbD1jb2xvcnMpCmBgYAoKCgojIyMgUHJpbmNpcGFsIGNvbXBvbmVudCAoUENBKSAKClJlbGF0ZWQgdG8gdGhlIGRpc3RhbmNlIG1hdHJpeCBoZWF0bWFwIGlzIHRoZSBbKFByaW5jaXBhbCBDb21wb25lbnRzIEFuYWx5c2lzKSBQQ0FdKGh0dHA6Ly9zZXRvc2EuaW8vZXYvcHJpbmNpcGFsLWNvbXBvbmVudC1hbmFseXNpcy8pIHBsb3QsIHdoaWNoIHNob3dzIHRoZSBzYW1wbGVzIGluIHRoZSAyRCBwbGFuZSBzcGFubmVkIGJ5IHRoZWlyIGZpcnN0IHR3byBwcmluY2lwYWwgY29tcG9uZW50cy4gQSBwcmluY2lwbGUgY29tcG9uZW50cyBhbmFseXNpcyBpcyBhbiBleGFtcGxlIG9mIGFuIHVuc3VwZXJ2aXNlZCBhbmFseXNpcywgd2hlcmUgd2UgZG9u4oCZdCBuZWVkIHRvIHNwZWNpZnkgdGhlIGdyb3Vwcy4gSWYgeW91ciBleHBlcmltZW50IGlzIHdlbGwgY29udHJvbGxlZCBhbmQgaGFzIHdvcmtlZCB3ZWxsLCB3aGF0IHdlIGhvcGUgdG8gc2VlIGlzIHRoYXQgdGhlIGdyZWF0ZXN0IHNvdXJjZXMgb2YgdmFyaWF0aW9uIGluIHRoZSBkYXRhIGFyZSB0aGUgdHJlYXRtZW50cy9ncm91cHMgd2UgYXJlIGludGVyZXN0ZWQgaW4uIEl0IGlzIGFsc28gYW4gaW5jcmVkaWJseSB1c2VmdWwgdG9vbCBmb3IgcXVhbGl0eSBjb250cm9sIGFuZCBjaGVja2luZyBmb3Igb3V0bGllcnMKCmBERVNlcTJgIGhhcyBhIGNvbnZlbmllbnQgYHBsb3RQQ0FgIGZ1bmN0aW9uIGZvciBtYWtpbmcgdGhlIFBDQSBwbG90LCB3aGljaCBtYWtlcyB1c2Ugb2YgdGhlIGBnZ3Bsb3QyYCBncmFwaGljcyBwYWNrYWdlLgoKYGBge3J9CnBsb3RQQ0EodnNkLGludGdyb3VwPSJDZWxsVHlwZSIpCmBgYAoKPiAjIyBDaGFsbGVuZ2UgMiB7LmNoYWxsZW5nZX0KPgo+IDEuIElzIHRoZSBgcGxvdFBDQWAgcGxvdCBiYXNlZCBvbiBhbGwgZ2VuZXMgaW4gdGhlIGRhdGFzZXQ/IEhvdyBjYW4gd2UgY2hhbmdlIGhvdyBtYW55IGdlbmVzIGFyZSB1c2VkIGZvciB0aGUgUENBIGFuYWx5c2lzPyBEb2VzIHRoaXMgc2lnbmlmaWNhbnRseSBjaGFuZ2UgdGhlIHBsb3Q/IChISU5UOiBjaGVjayB0aGUgZG9jdW1lbnRhdGlvbiBmb3IgdGhlIGBwbG90UENBYCBmdW5jdGlvbi4pCj4gMi4gQ2hhbmdlIHRoZSBgaW50Z3JvdXBgIHBhcmFtZXRlciBzbyB0aGF0IGJvdGggQ2VsbFR5cGUgYW5kIFN0YXR1cyBhcmUgdXNlZCBmb3IgZ3JvdXBpbmcuIChTZWUgdGhlIGRvY3VtZW50YXRpb24gYWdhaW4pCj4gMy4gSXMgdGhlcmUgc29tZXRoaW5nIHN0cmFuZ2UgZ29pbmcgb24gd2l0aCB0aGUgc2FtcGxlcz8KPiA0LiBJZGVudGlmeSB0aGUgdHdvIHNhbXBsZXMgdGhhdCBkb24ndCBhcHBlYXIgdG8gYmUgaW4gdGhlIHJpZ2h0IHBsYWNlLgo+IDUuIFdoYXQgb3RoZXIgcHJvYmxlbXMgY2FuIHlvdSBzZWUgd2l0aCB0aGUgbWV0YWRhdGE/CgojIyMgTm90ZSBhYm91dCBiYXRjaCBlZmZlY3RzCgpJbiBvdXIgdW5zdXBlcnZpc2VkIGFuYWx5c2lzIHdlIHNob3VsZCBzZWUgdGhhdCB0aGUgbWFpbiBzb3VyY2Ugb2YgdmFyaWF0aW9uIGlzIGR1ZSB0byBiaW9sb2dpY2FsIGVmZmVjdHMsIGFuZCBub3QgdGVjaG5pY2FsIHZhcmlhdGlvbiBzdWNoIGFzIHdoZW4gdGhlIGxpYnJhcmllcyB3ZXJlIHNlcXVlbmNlZC4gSWYgd2UgZG8gb2JzZXJ2ZSBoaWdoIHRlY2huaWNhbCB2YXJpYXRpb24gaW4gb3VyIGRhdGEsIGl0IGlzIG5vdCBhIGNvbXBsZXRlIGRpc2FzdGVyIHByb3ZpZGVkIHRoYXQgd2UgaGF2ZSBkZXNpZ25lZCBvdXIgZXhwZXJpbWVudCBwcm9wZXJ5LiBJbiBwYXJ0aWN1bGFyIHRoZSBbc3ZhIEJpb2NvbmR1Y3RvciBwYWNrYWdlXShodHRwczovL2Jpb2NvbmR1Y3Rvci5vcmcvcGFja2FnZXMvcmVsZWFzZS9iaW9jL3ZpZ25ldHRlcy9zdmEvaW5zdC9kb2Mvc3ZhLnBkZikgY2FuIGNvcnJlY3QgZm9yIGJhdGNoIGVmZmVjdHMgcHJvdmlkZWQgdGhhdCByZXByZXNlbnRhdGl2ZXMgb2YgdGhlIGdyb3VwcyBvZiBpbnRlcmVzdCBhcHBlYXIgaW4gZWFjaCBiYXRjaC4gQWx0ZXJuYXRpdmVseSwgdGhlIGJhdGNoIG9yIGNvbmZvdW5kaW5nIGZhY3RvciBtYXkgYmUgaW5jb3Jwb3JhdGVkIGludG8gdGhlIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGFuYWx5c2lzLgoKIyMjIENvcnJlY3RpbmcgdGhlIHNhbXBsZSBpbmZvcm1hdGlvbgoKSG9wZWZ1bGx5IHdlIGhhdmUgc3BvdHRlZCBhIHBvdGVudGlhbCBzYW1wbGUgc3dhcCBpbiB0aGUgZGF0YXNldC4gVGhlIG1pc2xhYmVsbGVkIHNhbXBsZXMgYXJlIE1DTDEuREgsIHdoaWNoIGlzIGxhYmVsbGVkIGFzIGx1bWluYWwgYnV0IHNob3VsZCBiZSBiYXNhbCwgYW5kIE1DTDEuTEEsIHdoaWNoIGlzIGxhYmVsbGVkIGFzIGJhc2FsIGJ1dCBzaG91bGQgYmUgbHVtaW5hbC4gIFN1Y2ggZXJyb3JzIGFyZSBub3QgdW5jb21tb24gd2hlbiBoYW5kbGluZyBsYXJnZSBudW1iZXJzIG9mIHNhbXBsZXMgYW5kIHNvbWV0aW1lcyB3ZSBuZWVkIHRvIGdvIGJhY2sgdG8gdGhlIGxhYiBib29rcyBhbmQgdmVyaWZ5IHRoYXQgYSBzd2FwIGhhcyBiZWVuIG1hZGUuICpJZiB0aGVyZSBpcyBubyBzdXBwb3J0aW5nIGV2aWRlbmNlIGZvciBhIHN3YXAgdGhlbiBpdCBjYW4gYmUgc2FmZXIgdG8gZXhjbHVkZSB0aGUgc2FtcGxlcyouIAoKRnVydGhlcm1vcmUsIHRoZSBwZXJzb24gY3JlYXRpbmcgdGhlIHNhbXBsZSBzaGVldCBoYXMgYmVlbiBpbmNvbnNpc3RlbnQgYWJvdXQgdGhlIHdheSB0aGF0IHZhbHVlcyBvZiBgQ2VsbFR5cGVgIGFuZCBgU3RhdHVzYCBoYXZlIGJlZW4gZW50ZXJlZCBpbnRvIHRoZSBtZXRhZGF0YS4gU3VjaCBlcnJvcnMgY2FuIGJlIGFubm95aW5nIHdoZW4gbGFiZWxsaW5nIHBsb3RzLCBidXQgaGF2ZSBtb3JlIHNlcmlvdXMgY29uc2VxdWVuY2VzIHdoZW4gYXR0ZW1wdGluZyB0byBmaXQgc3RhdGlzdGljYWwgbW9kZWxzIHRvIHRoZSBkYXRhLgoKCmBgYHtyIGNvcnJlY3RTYW1wbGVTaGVldH0KbGlicmFyeShzdHJpbmdyKQpzYW1wbGVpbmZvX2NvcnJlY3RlZCA8LSBzYW1wbGVpbmZvCgpzYW1wbGVpbmZvX2NvcnJlY3RlZCA8LSBtdXRhdGUoc2FtcGxlaW5mb19jb3JyZWN0ZWQsIENlbGxUeXBlID0gc3RyX3RvX2xvd2VyKENlbGxUeXBlKSkKc2FtcGxlaW5mb19jb3JyZWN0ZWQgPC0gbXV0YXRlKHNhbXBsZWluZm9fY29ycmVjdGVkLCBTdGF0dXMgPSBzdHJfdHJpbShTdGF0dXMpKQpzYW1wbGVpbmZvX2NvcnJlY3RlZCA8LSBtdXRhdGUoc2FtcGxlaW5mb19jb3JyZWN0ZWQsIENlbGxUeXBlID0gaWZlbHNlKE5hbWUgPT0gIk1DTDEtREgiLCJiYXNhbCIsQ2VsbFR5cGUpKQpzYW1wbGVpbmZvX2NvcnJlY3RlZCA8LSBtdXRhdGUoc2FtcGxlaW5mb19jb3JyZWN0ZWQsIENlbGxUeXBlPSBpZmVsc2UoTmFtZSA9PSAiTUNMMS1MQSIsImx1bWluYWwiLENlbGxUeXBlKSkKCndyaXRlLnRhYmxlKHNhbXBsZWluZm9fY29ycmVjdGVkLCBmaWxlPSJtZXRhX2RhdGEvc2FtcGxlSW5mb19jb3JyZWN0ZWQudHh0IixzZXA9Ilx0Iixyb3cubmFtZXMgPSBGQUxTRSkKYGBgCgo+ICMjIENoYWxsZW5nZSAzIHsuY2hhbGxlbmdlfQo+Cj4gMS4gUmUtY3JlYXRlIHRoZSBERVNlcURhdGFzZXQgb2JqZWN0IHRvIGluY2x1ZGUgdGhlIGNvcnJlY3RlZCBzYW1wbGUgaW5mb3JtYXRpb24KPiAyLiBSZS1ydW4gdGhlIHBsb3RQQ0EgZnVuY3Rpb24gb24gdGhlIG5ldyBkYXRhIGFuZCB2ZXJpZnkgdGhhdCB0aGUgc2FtcGxlIGdyb3VwcyBub3cgbG9vayBjb3JyZWN0Cj4KCiMjIyBBIG5vdGUgYWJvdXQgInBpcGVzIiBhbmQgZ2dwbG90MgoKSXQgaXMgY29tbW9uIHByYWN0aWNlIHdoZW4gdXNpbmcgYSBzZXJpZXMgb2YgYGRwbHlyYCBkYXRhIGZyYW1lIG1hbmlwdWxhdGlvbnMgKHN1Y2ggYXMgYG11dGF0ZWAgYWJvdmUpIHRvIGNyZWF0ZSBhIHdvcmtmbG93IHdpdGggdGhlIGAlPiVgIG9wZXJhdGlvbiBmcm9tIHRoZSBgbWFncml0dHJgIHBhY2thZ2UuIFRoZSByZXN1bHQgaXMgYSBtb3JlLXJlYWRhYmxlIGNodW5rIG9mIGNvZGUsIGFsdGhvdWdoIHRoZSByZXN1bHQgaXMgZXhhY3RseSB0aGUgc2FtZS4KCmBgYHtyfQpzYW1wbGVpbmZvX2NvcnJlY3RlZCA8LSBtdXRhdGUoc2FtcGxlaW5mbywgQ2VsbFR5cGUgPSBzdHJfdG9fbG93ZXIoQ2VsbFR5cGUpKSAlPiUgCiAgbXV0YXRlKFN0YXR1cyA9IHN0cl90cmltKFN0YXR1cykpICU+JSAKICBtdXRhdGUoQ2VsbFR5cGUgPSBpZmVsc2UoTmFtZSA9PSAiTUNMMS1ESCIsImJhc2FsIixDZWxsVHlwZSkpICU+JSAKICBtdXRhdGUoQ2VsbFR5cGU9IGlmZWxzZShOYW1lID09ICJNQ0wxLUxBIiwibHVtaW5hbCIsQ2VsbFR5cGUpKQoKc2FtcGxlaW5mb19jb3JyZWN0ZWQKYGBgCgpUaGUgYHBsb3RQQ0FgIGZ1bmN0aW9uIHByb2R1Y2VzIGEgYGdncGxvdDJgIHBsb3QgKHJhdGhlciB0aGFuICJiYXNlIiBncmFwaGljcykgYW5kIHdlIGNhbiBhbHNvIGdldCB0aGUgZnVuY3Rpb24gdG8gcmV0dXJuIHRoZSBkYXRhIHVzZWQgdG8gY3JlYXRlIHRoZSBwbG90IGJ5IGNoYW5naW5nIHRoZSBgcmV0dXJuRGF0YWAgYXJndW1lbnQuCgpgYGB7cn0KcGxvdF9kYXRhIDwtIHBsb3RQQ0EodnNkLGludGdyb3VwPWMoIkNlbGxUeXBlIiwiU3RhdHVzIikscmV0dXJuRGF0YT1UUlVFKQpwbG90X2RhdGEgPC0gYmluZF9jb2xzKHBsb3RfZGF0YSxzYW1wbGVpbmZvX2NvcnJlY3RlZCkKYGBgCgpXaXRoIGBnZ3Bsb3QyYCBwbG90IGNhbiBiZSBjcmVhdGVkIGJ5IG1hcHBpbmcgdmFyaW91cyAqYWVzdGhldGljcyogb2YgdGhlIHBsb3QgKGNvbG91ciwgc2hhcGUsIHgtIGFuZCB5LWNvb3JkaW5hdGVzKSB0byBjb2x1bW5zIGluIHRoZSBkYXRhIGZyYW1lLgoKYGBge3J9CmxpYnJhcnkoZ2dwbG90MikKZ2dwbG90KHBsb3RfZGF0YSwgYWVzKHggPSBQQzEseT1QQzIsIGNvbD1DZWxsVHlwZSkpICsgZ2VvbV9wb2ludCgpCmBgYAoKSGF2aW5nIHRoZSBkYXRhIGluIHRoaXMgZm9ybSBhbGxvd3MgdXMgdG8gY3VzdG9taXNlIHRoZSBwbG90IGluIGxvdHMgb2Ygd2F5cwoKYGBge3J9CmdncGxvdChwbG90X2RhdGEsIGFlcyh4ID0gUEMxLHk9UEMyLCBjb2w9Q2VsbFR5cGUscGNoPVN0YXR1cykpICsgZ2VvbV9wb2ludChzaXplPTUpCgpgYGAKCmBgYHtyfQpnZ3Bsb3QocGxvdF9kYXRhLCBhZXMoeCA9IFBDMSx5PVBDMiwgY29sPUNlbGxUeXBlLHBjaD1TdGF0dXMsbGFiZWw9TmFtZSkpICsgZ2VvbV9wb2ludCgpICsgZ2VvbV90ZXh0KGFscGhhPTAuNCkKCmBgYAoK