Introduction

This is a brief tutorial on automatic cell type annotation of single-cell RNA sequencing (scRNA-seq) data. The primary dataset used here contains hematopoietic and stromal bone marrow populations (Baccin et al.). The version used here is a subset of the original dataset to have more similar population sizes and speed up processing.

Load data

This tutorial includes some Bioconductor dependencies. Before proceeding, confirm that Bioconductor is installed and its version is at least 3.10.

if (!requireNamespace("BiocManager", quietly = TRUE)) {
  install.packages("BiocManager")
}
BiocManager::version()
[1] ‘3.11’

If this is not the case, remove all versions of BiocVersion with remove.packages("BiocVersion"). Then update Bioconductor packages using BiocManager::install().

Since we are using a Seurat object, load Seurat and related packages.

library(Seurat)
library(ggplot2)
library(cowplot)
library(ggsci)
library(dplyr)
library(stringr)

Load the dataset.

so = readRDS(url("https://osf.io/cvnqb/download"))
so
An object of class Seurat 
16701 features across 2821 samples within 1 assay 
Active assay: RNA (16701 features, 2872 variable features)
 3 dimensional reductions calculated: pca, tsne, umap

Check the experiment labels overlaid onto the UMAP visualization.

DimPlot(so, reduction = "umap", group.by = "experiment", cells = sample(colnames(so))) +
  scale_color_nejm()

Check the experiment labels overlaid onto the tSNE visualization, as shown in the original publication (original figure).

DimPlot(so, reduction = "tsne", group.by = "experiment", cells = sample(colnames(so))) +
  scale_color_nejm()

Check the cell type labels overlaid onto the tSNE visualization.

DimPlot(so, reduction = "tsne", group.by = "celltype", cells = sample(colnames(so))) +
  scale_color_igv()

Annotation using SingleR

Load SingleR (install with BiocManager::install("SingleR")).

library(SingleR)

SingleR expects the input as a matrix or a SummarizedExperiment object.

exp_mat = GetAssayData(so, assay = "RNA", slot = "data")
exp_mat = as.matrix(exp_mat)
dim(exp_mat)
[1] 16701  2821

MouseRNAseqData cell types

The SingleR package provides normalized expression values and cell types labels based on bulk RNA-seq, microarray, and single-cell RNA-seq data from several different datasets. See the SingleR vignette for the description.

We will try the mouse dataset from 358 bulk RNA-seq samples of sorted cell populations as the reference. It provides normalized expression values for samples that have been assigned to one of 18 main cell types and 28 subtypes.

It is possible that this fails with a No internet connection using 'localHub=TRUE' error. This may be resolved by running ExperimentHub::setExperimentHubOption("PROXY", "http://127.0.0.1:10801").

singler_se
class: SummarizedExperiment 
dim: 21214 358 
metadata(0):
assays(1): logcounts
rownames(21214): Xkr4 Rp1 ... LOC100039574 LOC100039753
rowData names(0):
colnames(358): ERR525589Aligned ERR525592Aligned ... SRR1044043Aligned
  SRR1044044Aligned
colData names(3): label.main label.fine label.ont

Restrict to common genes between the test and reference datasets.

common_genes = intersect(rownames(exp_mat), rownames(singler_se))
common_genes = sort(common_genes)
exp_common_mat = exp_mat[common_genes, ]
singler_se = singler_se[common_genes, ]
length(common_genes)
[1] 13929

Perform SingleR annotation.

singler_pred = SingleR(
  test = exp_common_mat,
  ref = singler_se,
  labels = singler_se$label.main
)

Each row of the output data frame contains prediction results for a single cell.

head(as.data.frame(singler_pred))

Add the assigned labels to the Seurat object.

so_labeled = AddMetaData(so, as.data.frame(singler_pred))

Check the assigned labels to the original labels.

so_labeled@meta.data %>% select(labels) %>% table(useNA = "ifany")
.
       Adipocytes           B cells    Cardiomyocytes   Dendritic cells Endothelial cells 
                6               400                 3                 1               201 
     Erythrocytes       Fibroblasts      Granulocytes       Macrophages         Monocytes 
              372               815               191                 2               534 
         NK cells  Oligodendrocytes           T cells 
               72                90               134 

SingleR also attempts to remove low quality or ambiguous assignments. Ambiguous assignments are based on the difference between the score for the assigned label and the median across all labels for each cell. Tuning parameters can be adjusted with pruneScores(). We can check how many cells are considered ambiguous and which populations they potentially belong to.

so_labeled@meta.data %>% select(pruned.labels) %>% table(useNA = "ifany")
.
       Adipocytes           B cells    Cardiomyocytes   Dendritic cells Endothelial cells 
                6               400                 3                 1               197 
     Erythrocytes       Fibroblasts      Granulocytes       Macrophages         Monocytes 
              372               815               191                 2               534 
         NK cells  Oligodendrocytes           T cells              <NA> 
               66                88               115                31 

Visualize MouseRNAseqData cell type labels.

DimPlot(so_labeled, reduction = "tsne", group.by = "labels") +
  scale_color_igv()

Annotation using clustermole

SingleR is able to label cells, but it requires a reference dataset.

A more exploratory and unbiased approach is possible with clustermole, an R package that provides a collection of cell type markers for thousands of human and mouse cell populations sourced from a variety of databases as well as methods to query them.

Load clustermole.

library(clustermole)

Available cell types

We can retrieve a list of all markers in the clustermole database to see all the available options. Each row in the returned data frame is a combination of each cell type and its genes.

markers_tbl = clustermole_markers()
head(markers_tbl)

Check the available cell types (ignore gene columns).

markers_tbl %>% distinct(celltype, organ, db)

Marker gene overlaps

Calculate the average expression levels for the clusters for enrichment analysis using Seurat’s AverageExpression function, convert to a matrix, and log-transform.

Idents(so) = "celltype"
avg_exp_mat = AverageExpression(so, assays = "RNA", slot = "data")
Finished averaging RNA for cluster Adipo-CAR
Finished averaging RNA for cluster Arteriolar-ECs
Finished averaging RNA for cluster Arteriolar-fibro
Finished averaging RNA for cluster B-cell
Finished averaging RNA for cluster Chondrocytes
Finished averaging RNA for cluster Dendritic-cells
Finished averaging RNA for cluster Endosteal-fibro
Finished averaging RNA for cluster Eo-Baso-prog
Finished averaging RNA for cluster Ery-Mk-prog
Finished averaging RNA for cluster Ery-prog
Finished averaging RNA for cluster Erythroblasts
Finished averaging RNA for cluster Fibro-Chondro-p
Finished averaging RNA for cluster Gran-Mono-prog
Finished averaging RNA for cluster large-pre-B
Finished averaging RNA for cluster LMPPs
Finished averaging RNA for cluster Mk-prog
Finished averaging RNA for cluster Mono-prog
Finished averaging RNA for cluster Monocytes
Finished averaging RNA for cluster Myofibroblasts
Finished averaging RNA for cluster Neutro-prog
Finished averaging RNA for cluster Neutrophils
Finished averaging RNA for cluster Ng2-MSCs
Finished averaging RNA for cluster NK-cells
Finished averaging RNA for cluster Osteo-CAR
Finished averaging RNA for cluster Osteoblasts
Finished averaging RNA for cluster pro-B
Finished averaging RNA for cluster Schwann-cells
Finished averaging RNA for cluster Sinusoidal-ECs
Finished averaging RNA for cluster small-pre-B
Finished averaging RNA for cluster Smooth-muscle
Finished averaging RNA for cluster Stromal-fibro
Finished averaging RNA for cluster T-cells
avg_exp_mat = as.matrix(avg_exp_mat$RNA)
avg_exp_mat = log1p(avg_exp_mat)

Check the average expression matrix.

avg_exp_mat[1:5, 1:5]
        Adipo-CAR Arteriolar-ECs Arteriolar-fibro    B-cell Chondrocytes
Sox17   0.0000000      2.5800606        0.0000000 0.0000000   0.00000000
Mrpl15  0.4315376      0.5284282        0.3304569 0.8481054   0.07307574
Lypla1  0.1990537      0.4477973        0.2448583 0.6352917   0.09397463
Gm37988 0.0000000      0.0000000        0.0000000 0.0000000   0.00000000
Tcea1   0.5620502      0.7077588        0.6135480 0.7798060   0.52126901

Check the cell type names.

levels(Idents(so))
 [1] "Adipo-CAR"        "Arteriolar-ECs"   "Arteriolar-fibro" "B-cell"          
 [5] "Chondrocytes"     "Dendritic-cells"  "Endosteal-fibro"  "Eo-Baso-prog"    
 [9] "Ery-Mk-prog"      "Ery-prog"         "Erythroblasts"    "Fibro-Chondro-p" 
[13] "Gran-Mono-prog"   "large-pre-B"      "LMPPs"            "Mk-prog"         
[17] "Mono-prog"        "Monocytes"        "Myofibroblasts"   "Neutro-prog"     
[21] "Neutrophils"      "Ng2-MSCs"         "NK-cells"         "Osteo-CAR"       
[25] "Osteoblasts"      "pro-B"            "Schwann-cells"    "Sinusoidal-ECs"  
[29] "small-pre-B"      "Smooth-muscle"    "Stromal-fibro"    "T-cells"         

Find markers for the B-cell cluster.

b_genes = rownames(avg_exp_mat[avg_exp_mat[, "B-cell"] == rowMaxs(avg_exp_mat), ])
length(b_genes)
[1] 517
b_markers_df = FindMarkers(so, ident.1 = "B-cell", features = b_genes, verbose = FALSE)
nrow(b_markers_df)
[1] 165

Check the markers table.

head(b_markers_df)

With the default cutoffs, this gives us a data frame with hundreds of genes. Let’s subset to just the top 20 genes.

b_markers = rownames(b_markers_df)
b_markers = head(b_markers, 20)
b_markers
 [1] "Ms4a1"         "Fcmr"          "Cd74"          "Ly6d"          "Gm43603"      
 [6] "Bank1"         "2010309G21Rik" "Fcer2a"        "H2-DMb2"       "H2-Eb1"       
[11] "Cd79a"         "H2-Aa"         "Tnfrsf13c"     "Ltb"           "Cd79b"        
[16] "Ccr7"          "Fcrl1"         "Siglecg"       "Cd83"          "Srpk3"        

Check overlap of our B-cell markers with all cell type signatures.

overlaps_tbl = clustermole_overlaps(genes = b_markers, species = "mm")
Registered S3 method overwritten by 'cli':
  method     from    
  print.boxx spatstat

Check the top scoring cell types for the B-cell cluster.

head(overlaps_tbl, 10)

Find markers for the Adipo-CAR cluster.

acar_genes = rownames(avg_exp_mat[avg_exp_mat[, "Adipo-CAR"] == rowMaxs(avg_exp_mat), ])
length(acar_genes)
[1] 888
acar_markers_df = FindMarkers(so, ident.1 = "Adipo-CAR", features = acar_genes, verbose = FALSE)
nrow(acar_markers_df)
[1] 326

Check the markers table.

head(acar_markers_df)

With the default cutoffs, this gives us a data frame with hundreds of genes. Let’s subset to just the top 20 genes.

acar_markers = rownames(acar_markers_df)
acar_markers = head(acar_markers, 20)
acar_markers
 [1] "Adipoq"        "Kng1"          "Kng2"          "Esm1"          "Cxcl12"       
 [6] "Lpl"           "Gdpd2"         "Agt"           "Dpep1"         "Lepr"         
[11] "Fst"           "Chrdl1"        "Pdzrn4"        "Kitl"          "Ccl19"        
[16] "Ptx3"          "Ackr4"         "1500009L16Rik" "Gas6"          "Serpina12"    

Check overlap of our Adipo-CAR markers with all cell type signatures.

overlaps_tbl = clustermole_overlaps(genes = acar_markers, species = "mm")

Check the top scoring cell types for the Adipo-CAR cluster.

head(overlaps_tbl, 10)

Find markers for the Osteoblasts cluster.

o_genes = rownames(avg_exp_mat[avg_exp_mat[, "Osteoblasts"] == rowMaxs(avg_exp_mat), ])
length(o_genes)
[1] 390
o_markers_df = FindMarkers(so, ident.1 = "Osteoblasts", features = o_genes, verbose = FALSE)
nrow(o_markers_df)
[1] 182

Check the markers table.

head(o_markers_df)

With the default cutoffs, this gives us a data frame with hundreds of genes. Let’s subset to just the top 20 genes.

o_markers = rownames(o_markers_df)
o_markers = head(o_markers, 20)
o_markers
 [1] "Cpz"           "Smpd3"         "Col22a1"       "Ifitm5"        "Mlip"         
 [6] "Bglap"         "Lipc"          "Cgref1"        "Col13a1"       "Entpd3"       
[11] "Fabp3"         "Bglap2"        "Cthrc1"        "Bglap3"        "Rerg"         
[16] "Cdo1"          "Car3"          "RP23-457J22.1" "Col24a1"       "Bmp3"         

Check overlap of our Osteoblasts markers with all cell type signatures.

overlaps_tbl = clustermole_overlaps(genes = o_markers, species = "mm")

Check the top scoring cell types for the Osteoblasts cluster.

head(overlaps_tbl, 10)

Enrichment of markers

Run enrichment of all cell type signatures across all clusters.

enrich_tbl = clustermole_enrichment(expr_mat = avg_exp_mat, species = "mm")
Some gene sets have size one. Consider setting 'min.sz > 1'.Selecting by score

Most enriched cell types for the B-cell cluster.

enrich_tbl %>% filter(cluster == "B-cell") %>% select(-cluster) %>% head(10)

Most enriched cell types for the Adipo-CAR cluster.

enrich_tbl %>% filter(cluster == "Adipo-CAR") %>% select(-cluster) %>% head(10)

Most enriched cell types for the Osteoblasts cluster.

enrich_tbl %>% filter(cluster == "Osteoblasts") %>% select(-cluster) %>% head(10)

previous tutorials

LS0tCnRpdGxlOiAiQ2VsbCBUeXBlIEFubm90YXRpb24iCm91dHB1dDoKICBodG1sX25vdGVib29rOgogICAgdGhlbWU6IHJlYWRhYmxlCiAgICB0b2M6IHllcwogICAgdG9jX2Zsb2F0OiB5ZXMKICAgIGNvZGVfZm9sZGluZzogbm9uZQotLS0KCgojIyBJbnRyb2R1Y3Rpb24KClRoaXMgaXMgYSBicmllZiB0dXRvcmlhbCBvbiBhdXRvbWF0aWMgY2VsbCB0eXBlIGFubm90YXRpb24gb2Ygc2luZ2xlLWNlbGwgUk5BIHNlcXVlbmNpbmcgKHNjUk5BLXNlcSkgZGF0YS4gVGhlIHByaW1hcnkgZGF0YXNldCB1c2VkIGhlcmUgY29udGFpbnMgaGVtYXRvcG9pZXRpYyBhbmQgc3Ryb21hbCBib25lIG1hcnJvdyBwb3B1bGF0aW9ucyAoW0JhY2NpbiBldCBhbC5dKGh0dHBzOi8vZG9pLm9yZy8xMC4xMDM4L3M0MTU1Ni0wMTktMDQzOS02KSkuIFRoZSB2ZXJzaW9uIHVzZWQgaGVyZSBpcyBhIHN1YnNldCBvZiB0aGUgb3JpZ2luYWwgZGF0YXNldCB0byBoYXZlIG1vcmUgc2ltaWxhciBwb3B1bGF0aW9uIHNpemVzIGFuZCBzcGVlZCB1cCBwcm9jZXNzaW5nLgoKIyMgTG9hZCBkYXRhCgpUaGlzIHR1dG9yaWFsIGluY2x1ZGVzIHNvbWUgQmlvY29uZHVjdG9yIGRlcGVuZGVuY2llcy4gQmVmb3JlIHByb2NlZWRpbmcsIGNvbmZpcm0gdGhhdCBCaW9jb25kdWN0b3IgaXMgaW5zdGFsbGVkIGFuZCBpdHMgdmVyc2lvbiBpcyBhdCBsZWFzdCAzLjEwLgoKYGBge3IgYmlvYy12ZXJzaW9ufQppZiAoIXJlcXVpcmVOYW1lc3BhY2UoIkJpb2NNYW5hZ2VyIiwgcXVpZXRseSA9IFRSVUUpKSB7CiAgaW5zdGFsbC5wYWNrYWdlcygiQmlvY01hbmFnZXIiKQp9CkJpb2NNYW5hZ2VyOjp2ZXJzaW9uKCkKYGBgCgpJZiB0aGlzIGlzIG5vdCB0aGUgY2FzZSwgcmVtb3ZlIGFsbCB2ZXJzaW9ucyBvZiBCaW9jVmVyc2lvbiB3aXRoIGByZW1vdmUucGFja2FnZXMoIkJpb2NWZXJzaW9uIilgLiBUaGVuIHVwZGF0ZSBCaW9jb25kdWN0b3IgcGFja2FnZXMgdXNpbmcgYEJpb2NNYW5hZ2VyOjppbnN0YWxsKClgLgoKU2luY2Ugd2UgYXJlIHVzaW5nIGEgU2V1cmF0IG9iamVjdCwgbG9hZCBTZXVyYXQgYW5kIHJlbGF0ZWQgcGFja2FnZXMuCgpgYGB7ciBsb2FkLWxpYnJhcmllcywgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KbGlicmFyeShTZXVyYXQpCmxpYnJhcnkoZ2dwbG90MikKbGlicmFyeShjb3dwbG90KQpsaWJyYXJ5KGdnc2NpKQpsaWJyYXJ5KGRwbHlyKQpsaWJyYXJ5KHN0cmluZ3IpCmBgYAoKTG9hZCB0aGUgZGF0YXNldC4KCmBgYHtyIGxvYWQtc2V1cmF0LW9iamVjdCwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0Kc28gPSByZWFkUkRTKHVybCgiaHR0cHM6Ly9vc2YuaW8vY3ZucWIvZG93bmxvYWQiKSkKc28KYGBgCgpDaGVjayB0aGUgZXhwZXJpbWVudCBsYWJlbHMgb3ZlcmxhaWQgb250byB0aGUgVU1BUCB2aXN1YWxpemF0aW9uLgoKYGBge3IgdW1hcC1leHB9CkRpbVBsb3Qoc28sIHJlZHVjdGlvbiA9ICJ1bWFwIiwgZ3JvdXAuYnkgPSAiZXhwZXJpbWVudCIsIGNlbGxzID0gc2FtcGxlKGNvbG5hbWVzKHNvKSkpICsKICBzY2FsZV9jb2xvcl9uZWptKCkKYGBgCgpDaGVjayB0aGUgZXhwZXJpbWVudCBsYWJlbHMgb3ZlcmxhaWQgb250byB0aGUgdFNORSB2aXN1YWxpemF0aW9uLCBhcyBzaG93biBpbiB0aGUgb3JpZ2luYWwgcHVibGljYXRpb24gKFtvcmlnaW5hbCBmaWd1cmVdKGh0dHBzOi8vd3d3Lm5hdHVyZS5jb20vYXJ0aWNsZXMvczQxNTU2LTAxOS0wNDM5LTYvZmlndXJlcy8xKSkuCgpgYGB7ciB0c25lLWV4cH0KRGltUGxvdChzbywgcmVkdWN0aW9uID0gInRzbmUiLCBncm91cC5ieSA9ICJleHBlcmltZW50IiwgY2VsbHMgPSBzYW1wbGUoY29sbmFtZXMoc28pKSkgKwogIHNjYWxlX2NvbG9yX25lam0oKQpgYGAKCkNoZWNrIHRoZSBjZWxsIHR5cGUgbGFiZWxzIG92ZXJsYWlkIG9udG8gdGhlIHRTTkUgdmlzdWFsaXphdGlvbi4KCmBgYHtyIHRzbmUtY2VsbHR5cGV9CkRpbVBsb3Qoc28sIHJlZHVjdGlvbiA9ICJ0c25lIiwgZ3JvdXAuYnkgPSAiY2VsbHR5cGUiLCBjZWxscyA9IHNhbXBsZShjb2xuYW1lcyhzbykpKSArCiAgc2NhbGVfY29sb3JfaWd2KCkKYGBgCgojIyBBbm5vdGF0aW9uIHVzaW5nIFNpbmdsZVIKCkxvYWQgU2luZ2xlUiAoaW5zdGFsbCB3aXRoIGBCaW9jTWFuYWdlcjo6aW5zdGFsbCgiU2luZ2xlUiIpYCkuCgpgYGB7ciBsb2FkLXNpbmdsZXIsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9CmxpYnJhcnkoU2luZ2xlUikKYGBgCgpTaW5nbGVSIGV4cGVjdHMgdGhlIGlucHV0IGFzIGEgbWF0cml4IG9yIGEgU3VtbWFyaXplZEV4cGVyaW1lbnQgb2JqZWN0LgoKYGBge3IgZXhwLW1hdH0KZXhwX21hdCA9IEdldEFzc2F5RGF0YShzbywgYXNzYXkgPSAiUk5BIiwgc2xvdCA9ICJkYXRhIikKZXhwX21hdCA9IGFzLm1hdHJpeChleHBfbWF0KQpkaW0oZXhwX21hdCkKYGBgCgojIyMgTW91c2VSTkFzZXFEYXRhIGNlbGwgdHlwZXMKClRoZSBTaW5nbGVSIHBhY2thZ2UgcHJvdmlkZXMgbm9ybWFsaXplZCBleHByZXNzaW9uIHZhbHVlcyBhbmQgY2VsbCB0eXBlcyBsYWJlbHMgYmFzZWQgb24gYnVsayBSTkEtc2VxLCBtaWNyb2FycmF5LCBhbmQgc2luZ2xlLWNlbGwgUk5BLXNlcSBkYXRhIGZyb20gc2V2ZXJhbCBkaWZmZXJlbnQgZGF0YXNldHMuIFNlZSB0aGUgW1NpbmdsZVIgdmlnbmV0dGVdKGh0dHBzOi8vYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy8zLjEwL2Jpb2MvdmlnbmV0dGVzL1NpbmdsZVIvaW5zdC9kb2MvU2luZ2xlUi5odG1sIzVfYXZhaWxhYmxlX3JlZmVyZW5jZXMpIGZvciB0aGUgZGVzY3JpcHRpb24uCgpXZSB3aWxsIHRyeSB0aGUgbW91c2UgZGF0YXNldCBmcm9tIDM1OCBidWxrIFJOQS1zZXEgc2FtcGxlcyBvZiBzb3J0ZWQgY2VsbCBwb3B1bGF0aW9ucyBhcyB0aGUgcmVmZXJlbmNlLiBJdCBwcm92aWRlcyBub3JtYWxpemVkIGV4cHJlc3Npb24gdmFsdWVzIGZvciBzYW1wbGVzIHRoYXQgaGF2ZSBiZWVuIGFzc2lnbmVkIHRvIG9uZSBvZiAxOCBtYWluIGNlbGwgdHlwZXMgYW5kIDI4IHN1YnR5cGVzLgoKYGBge3IgbW91c2VybmFzZXEtc2UtbG9hZCwgaW5jbHVkZT1GQUxTRX0Kc2luZ2xlcl9zZSA9IE1vdXNlUk5Bc2VxRGF0YSgpCmBgYAoKSXQgaXMgcG9zc2libGUgdGhhdCB0aGlzIGZhaWxzIHdpdGggYSBgTm8gaW50ZXJuZXQgY29ubmVjdGlvbiB1c2luZyAnbG9jYWxIdWI9VFJVRSdgIGVycm9yLiBUaGlzIG1heSBiZSByZXNvbHZlZCBieSBydW5uaW5nIGBFeHBlcmltZW50SHViOjpzZXRFeHBlcmltZW50SHViT3B0aW9uKCJQUk9YWSIsICJodHRwOi8vMTI3LjAuMC4xOjEwODAxIilgLgoKYGBge3IgbW91c2VybmFzZXEtc2Utc2hvd30Kc2luZ2xlcl9zZQpgYGAKClJlc3RyaWN0IHRvIGNvbW1vbiBnZW5lcyBiZXR3ZWVuIHRoZSB0ZXN0IGFuZCByZWZlcmVuY2UgZGF0YXNldHMuCgpgYGB7ciBtb3VzZXJuYXNlcS1jb21tb24tZ2VuZXN9CmNvbW1vbl9nZW5lcyA9IGludGVyc2VjdChyb3duYW1lcyhleHBfbWF0KSwgcm93bmFtZXMoc2luZ2xlcl9zZSkpCmNvbW1vbl9nZW5lcyA9IHNvcnQoY29tbW9uX2dlbmVzKQpleHBfY29tbW9uX21hdCA9IGV4cF9tYXRbY29tbW9uX2dlbmVzLCBdCnNpbmdsZXJfc2UgPSBzaW5nbGVyX3NlW2NvbW1vbl9nZW5lcywgXQpsZW5ndGgoY29tbW9uX2dlbmVzKQpgYGAKClBlcmZvcm0gU2luZ2xlUiBhbm5vdGF0aW9uLgoKYGBge3IgbW91c2VybmFzZXEtc2luZ2xlcn0Kc2luZ2xlcl9wcmVkID0gU2luZ2xlUigKICB0ZXN0ID0gZXhwX2NvbW1vbl9tYXQsCiAgcmVmID0gc2luZ2xlcl9zZSwKICBsYWJlbHMgPSBzaW5nbGVyX3NlJGxhYmVsLm1haW4KKQpgYGAKCkVhY2ggcm93IG9mIHRoZSBvdXRwdXQgZGF0YSBmcmFtZSBjb250YWlucyBwcmVkaWN0aW9uIHJlc3VsdHMgZm9yIGEgc2luZ2xlIGNlbGwuCgpgYGB7ciBtb3VzZXJuYXNlcS1kZn0KaGVhZChhcy5kYXRhLmZyYW1lKHNpbmdsZXJfcHJlZCkpCmBgYAoKQWRkIHRoZSBhc3NpZ25lZCBsYWJlbHMgdG8gdGhlIFNldXJhdCBvYmplY3QuCgpgYGB7ciBtb3VzZXJuYXNlcS1hZGRtZXRhZGF0YX0Kc29fbGFiZWxlZCA9IEFkZE1ldGFEYXRhKHNvLCBhcy5kYXRhLmZyYW1lKHNpbmdsZXJfcHJlZCkpCmBgYAoKQ2hlY2sgdGhlIGFzc2lnbmVkIGxhYmVscyB0byB0aGUgb3JpZ2luYWwgbGFiZWxzLgoKYGBge3IgbW91c2VybmFzZXEtdGFibGV9CnNvX2xhYmVsZWRAbWV0YS5kYXRhICU+JSBzZWxlY3QobGFiZWxzKSAlPiUgdGFibGUodXNlTkEgPSAiaWZhbnkiKQpgYGAKClNpbmdsZVIgYWxzbyBhdHRlbXB0cyB0byByZW1vdmUgbG93IHF1YWxpdHkgb3IgYW1iaWd1b3VzIGFzc2lnbm1lbnRzLiBBbWJpZ3VvdXMgYXNzaWdubWVudHMgYXJlIGJhc2VkIG9uIHRoZSBkaWZmZXJlbmNlIGJldHdlZW4gdGhlIHNjb3JlIGZvciB0aGUgYXNzaWduZWQgbGFiZWwgYW5kIHRoZSBtZWRpYW4gYWNyb3NzIGFsbCBsYWJlbHMgZm9yIGVhY2ggY2VsbC4gVHVuaW5nIHBhcmFtZXRlcnMgY2FuIGJlIGFkanVzdGVkIHdpdGggYHBydW5lU2NvcmVzKClgLiBXZSBjYW4gY2hlY2sgaG93IG1hbnkgY2VsbHMgYXJlIGNvbnNpZGVyZWQgYW1iaWd1b3VzIGFuZCB3aGljaCBwb3B1bGF0aW9ucyB0aGV5IHBvdGVudGlhbGx5IGJlbG9uZyB0by4KCmBgYHtyIG1vdXNlcm5hc2VxLXBydW5lZC10YWJsZX0Kc29fbGFiZWxlZEBtZXRhLmRhdGEgJT4lIHNlbGVjdChwcnVuZWQubGFiZWxzKSAlPiUgdGFibGUodXNlTkEgPSAiaWZhbnkiKQpgYGAKClZpc3VhbGl6ZSBNb3VzZVJOQXNlcURhdGEgY2VsbCB0eXBlIGxhYmVscy4KCmBgYHtyIG1vdXNlcm5hc2VxLXRzbmV9CkRpbVBsb3Qoc29fbGFiZWxlZCwgcmVkdWN0aW9uID0gInRzbmUiLCBncm91cC5ieSA9ICJsYWJlbHMiKSArCiAgc2NhbGVfY29sb3JfaWd2KCkKYGBgCgojIyBBbm5vdGF0aW9uIHVzaW5nIGNsdXN0ZXJtb2xlCgpTaW5nbGVSIGlzIGFibGUgdG8gbGFiZWwgY2VsbHMsIGJ1dCBpdCByZXF1aXJlcyBhIHJlZmVyZW5jZSBkYXRhc2V0LgoKQSBtb3JlIGV4cGxvcmF0b3J5IGFuZCB1bmJpYXNlZCBhcHByb2FjaCBpcyBwb3NzaWJsZSB3aXRoIFtjbHVzdGVybW9sZV0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvcGFja2FnZT1jbHVzdGVybW9sZSksIGFuIFIgcGFja2FnZSB0aGF0IHByb3ZpZGVzIGEgY29sbGVjdGlvbiBvZiBjZWxsIHR5cGUgbWFya2VycyBmb3IgdGhvdXNhbmRzIG9mIGh1bWFuIGFuZCBtb3VzZSBjZWxsIHBvcHVsYXRpb25zIHNvdXJjZWQgZnJvbSBhIHZhcmlldHkgb2YgZGF0YWJhc2VzIGFzIHdlbGwgYXMgbWV0aG9kcyB0byBxdWVyeSB0aGVtLgoKTG9hZCBjbHVzdGVybW9sZS4KCmBgYHtyIGxvYWQtY2x1c3Rlcm1vbGUsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9CmxpYnJhcnkoY2x1c3Rlcm1vbGUpCmBgYAoKIyMjIEF2YWlsYWJsZSBjZWxsIHR5cGVzCgpXZSBjYW4gcmV0cmlldmUgYSBsaXN0IG9mIGFsbCBtYXJrZXJzIGluIHRoZSBjbHVzdGVybW9sZSBkYXRhYmFzZSB0byBzZWUgYWxsIHRoZSBhdmFpbGFibGUgb3B0aW9ucy4gRWFjaCByb3cgaW4gdGhlIHJldHVybmVkIGRhdGEgZnJhbWUgaXMgYSBjb21iaW5hdGlvbiBvZiBlYWNoIGNlbGwgdHlwZSBhbmQgaXRzIGdlbmVzLgoKYGBge3IgY2x1c3Rlcm1vbGUtbWFya2Vycy10YWJsZX0KbWFya2Vyc190YmwgPSBjbHVzdGVybW9sZV9tYXJrZXJzKCkKaGVhZChtYXJrZXJzX3RibCkKYGBgCgpDaGVjayB0aGUgYXZhaWxhYmxlIGNlbGwgdHlwZXMgKGlnbm9yZSBnZW5lIGNvbHVtbnMpLgoKYGBge3IgY2x1c3Rlcm1vbGUtbWFya2Vycy1zdW1tYXJ5fQptYXJrZXJzX3RibCAlPiUgZGlzdGluY3QoY2VsbHR5cGUsIG9yZ2FuLCBkYikKYGBgCgojIyMgTWFya2VyIGdlbmUgb3ZlcmxhcHMKCkNhbGN1bGF0ZSB0aGUgYXZlcmFnZSBleHByZXNzaW9uIGxldmVscyBmb3IgdGhlIGNsdXN0ZXJzIGZvciBlbnJpY2htZW50IGFuYWx5c2lzIHVzaW5nIFNldXJhdCdzIGBBdmVyYWdlRXhwcmVzc2lvbmAgZnVuY3Rpb24sIGNvbnZlcnQgdG8gYSBtYXRyaXgsIGFuZCBsb2ctdHJhbnNmb3JtLgoKYGBge3IgYXZnLWV4cH0KSWRlbnRzKHNvKSA9ICJjZWxsdHlwZSIKYXZnX2V4cF9tYXQgPSBBdmVyYWdlRXhwcmVzc2lvbihzbywgYXNzYXlzID0gIlJOQSIsIHNsb3QgPSAiZGF0YSIpCmF2Z19leHBfbWF0ID0gYXMubWF0cml4KGF2Z19leHBfbWF0JFJOQSkKYXZnX2V4cF9tYXQgPSBsb2cxcChhdmdfZXhwX21hdCkKYGBgCgpDaGVjayB0aGUgYXZlcmFnZSBleHByZXNzaW9uIG1hdHJpeC4KCmBgYHtyIGF2Zy1leHAtaGVhZH0KYXZnX2V4cF9tYXRbMTo1LCAxOjVdCmBgYAoKQ2hlY2sgdGhlIGNlbGwgdHlwZSBuYW1lcy4KCmBgYHtyfQpsZXZlbHMoSWRlbnRzKHNvKSkKYGBgCgpGaW5kIG1hcmtlcnMgZm9yIHRoZSBCLWNlbGwgY2x1c3Rlci4KCmBgYHtyfQpiX2dlbmVzID0gcm93bmFtZXMoYXZnX2V4cF9tYXRbYXZnX2V4cF9tYXRbLCAiQi1jZWxsIl0gPT0gcm93TWF4cyhhdmdfZXhwX21hdCksIF0pCmxlbmd0aChiX2dlbmVzKQpgYGAKCmBgYHtyIGZpbmQtbWFya2Vycy1ifQpiX21hcmtlcnNfZGYgPSBGaW5kTWFya2VycyhzbywgaWRlbnQuMSA9ICJCLWNlbGwiLCBmZWF0dXJlcyA9IGJfZ2VuZXMsIHZlcmJvc2UgPSBGQUxTRSkKbnJvdyhiX21hcmtlcnNfZGYpCmBgYAoKQ2hlY2sgdGhlIG1hcmtlcnMgdGFibGUuCgpgYGB7ciBmaW5kLW1hcmtlcnMtYi1oZWFkfQpoZWFkKGJfbWFya2Vyc19kZikKYGBgCgpXaXRoIHRoZSBkZWZhdWx0IGN1dG9mZnMsIHRoaXMgZ2l2ZXMgdXMgYSBkYXRhIGZyYW1lIHdpdGggaHVuZHJlZHMgb2YgZ2VuZXMuIExldCdzIHN1YnNldCB0byBqdXN0IHRoZSB0b3AgMjAgZ2VuZXMuCgpgYGB7ciBtYXJrZXJzLXRvcC1nZW5lc30KYl9tYXJrZXJzID0gcm93bmFtZXMoYl9tYXJrZXJzX2RmKQpiX21hcmtlcnMgPSBoZWFkKGJfbWFya2VycywgMjApCmJfbWFya2VycwpgYGAKCkNoZWNrIG92ZXJsYXAgb2Ygb3VyIEItY2VsbCBtYXJrZXJzIHdpdGggYWxsIGNlbGwgdHlwZSBzaWduYXR1cmVzLgoKYGBge3IgY2x1c3Rlcm1vbGUtb3ZlcmxhcHN9Cm92ZXJsYXBzX3RibCA9IGNsdXN0ZXJtb2xlX292ZXJsYXBzKGdlbmVzID0gYl9tYXJrZXJzLCBzcGVjaWVzID0gIm1tIikKYGBgCgpDaGVjayB0aGUgdG9wIHNjb3JpbmcgY2VsbCB0eXBlcyBmb3IgdGhlIEItY2VsbCBjbHVzdGVyLgoKYGBge3IgY2x1c3Rlcm1vbGUtb3ZlcmxhcHMtcmVzdWx0fQpoZWFkKG92ZXJsYXBzX3RibCwgMTApCmBgYAoKRmluZCBtYXJrZXJzIGZvciB0aGUgQWRpcG8tQ0FSIGNsdXN0ZXIuCgpgYGB7cn0KYWNhcl9nZW5lcyA9IHJvd25hbWVzKGF2Z19leHBfbWF0W2F2Z19leHBfbWF0WywgIkFkaXBvLUNBUiJdID09IHJvd01heHMoYXZnX2V4cF9tYXQpLCBdKQpsZW5ndGgoYWNhcl9nZW5lcykKYGBgCgpgYGB7cn0KYWNhcl9tYXJrZXJzX2RmID0gRmluZE1hcmtlcnMoc28sIGlkZW50LjEgPSAiQWRpcG8tQ0FSIiwgZmVhdHVyZXMgPSBhY2FyX2dlbmVzLCB2ZXJib3NlID0gRkFMU0UpCm5yb3coYWNhcl9tYXJrZXJzX2RmKQpgYGAKCkNoZWNrIHRoZSBtYXJrZXJzIHRhYmxlLgoKYGBge3J9CmhlYWQoYWNhcl9tYXJrZXJzX2RmKQpgYGAKCldpdGggdGhlIGRlZmF1bHQgY3V0b2ZmcywgdGhpcyBnaXZlcyB1cyBhIGRhdGEgZnJhbWUgd2l0aCBodW5kcmVkcyBvZiBnZW5lcy4gTGV0J3Mgc3Vic2V0IHRvIGp1c3QgdGhlIHRvcCAyMCBnZW5lcy4KCmBgYHtyfQphY2FyX21hcmtlcnMgPSByb3duYW1lcyhhY2FyX21hcmtlcnNfZGYpCmFjYXJfbWFya2VycyA9IGhlYWQoYWNhcl9tYXJrZXJzLCAyMCkKYWNhcl9tYXJrZXJzCmBgYAoKQ2hlY2sgb3ZlcmxhcCBvZiBvdXIgQWRpcG8tQ0FSIG1hcmtlcnMgd2l0aCBhbGwgY2VsbCB0eXBlIHNpZ25hdHVyZXMuCgpgYGB7cn0Kb3ZlcmxhcHNfdGJsID0gY2x1c3Rlcm1vbGVfb3ZlcmxhcHMoZ2VuZXMgPSBhY2FyX21hcmtlcnMsIHNwZWNpZXMgPSAibW0iKQpgYGAKCkNoZWNrIHRoZSB0b3Agc2NvcmluZyBjZWxsIHR5cGVzIGZvciB0aGUgQWRpcG8tQ0FSIGNsdXN0ZXIuCgpgYGB7cn0KaGVhZChvdmVybGFwc190YmwsIDEwKQpgYGAKCkZpbmQgbWFya2VycyBmb3IgdGhlIE9zdGVvYmxhc3RzIGNsdXN0ZXIuCgpgYGB7cn0Kb19nZW5lcyA9IHJvd25hbWVzKGF2Z19leHBfbWF0W2F2Z19leHBfbWF0WywgIk9zdGVvYmxhc3RzIl0gPT0gcm93TWF4cyhhdmdfZXhwX21hdCksIF0pCmxlbmd0aChvX2dlbmVzKQpgYGAKCmBgYHtyfQpvX21hcmtlcnNfZGYgPSBGaW5kTWFya2VycyhzbywgaWRlbnQuMSA9ICJPc3Rlb2JsYXN0cyIsIGZlYXR1cmVzID0gb19nZW5lcywgdmVyYm9zZSA9IEZBTFNFKQpucm93KG9fbWFya2Vyc19kZikKYGBgCgpDaGVjayB0aGUgbWFya2VycyB0YWJsZS4KCmBgYHtyfQpoZWFkKG9fbWFya2Vyc19kZikKYGBgCgpXaXRoIHRoZSBkZWZhdWx0IGN1dG9mZnMsIHRoaXMgZ2l2ZXMgdXMgYSBkYXRhIGZyYW1lIHdpdGggaHVuZHJlZHMgb2YgZ2VuZXMuIExldCdzIHN1YnNldCB0byBqdXN0IHRoZSB0b3AgMjAgZ2VuZXMuCgpgYGB7cn0Kb19tYXJrZXJzID0gcm93bmFtZXMob19tYXJrZXJzX2RmKQpvX21hcmtlcnMgPSBoZWFkKG9fbWFya2VycywgMjApCm9fbWFya2VycwpgYGAKCkNoZWNrIG92ZXJsYXAgb2Ygb3VyIE9zdGVvYmxhc3RzIG1hcmtlcnMgd2l0aCBhbGwgY2VsbCB0eXBlIHNpZ25hdHVyZXMuCgpgYGB7cn0Kb3ZlcmxhcHNfdGJsID0gY2x1c3Rlcm1vbGVfb3ZlcmxhcHMoZ2VuZXMgPSBvX21hcmtlcnMsIHNwZWNpZXMgPSAibW0iKQpgYGAKCkNoZWNrIHRoZSB0b3Agc2NvcmluZyBjZWxsIHR5cGVzIGZvciB0aGUgT3N0ZW9ibGFzdHMgY2x1c3Rlci4KCmBgYHtyfQpoZWFkKG92ZXJsYXBzX3RibCwgMTApCmBgYAoKIyMjIEVucmljaG1lbnQgb2YgbWFya2VycwoKUnVuIGVucmljaG1lbnQgb2YgYWxsIGNlbGwgdHlwZSBzaWduYXR1cmVzIGFjcm9zcyBhbGwgY2x1c3RlcnMuCgpgYGB7cn0KZW5yaWNoX3RibCA9IGNsdXN0ZXJtb2xlX2VucmljaG1lbnQoZXhwcl9tYXQgPSBhdmdfZXhwX21hdCwgc3BlY2llcyA9ICJtbSIpCmBgYAoKTW9zdCBlbnJpY2hlZCBjZWxsIHR5cGVzIGZvciB0aGUgQi1jZWxsIGNsdXN0ZXIuCgpgYGB7cn0KZW5yaWNoX3RibCAlPiUgZmlsdGVyKGNsdXN0ZXIgPT0gIkItY2VsbCIpICU+JSBzZWxlY3QoLWNsdXN0ZXIpICU+JSBoZWFkKDEwKQpgYGAKCk1vc3QgZW5yaWNoZWQgY2VsbCB0eXBlcyBmb3IgdGhlIEFkaXBvLUNBUiBjbHVzdGVyLgoKYGBge3J9CmVucmljaF90YmwgJT4lIGZpbHRlcihjbHVzdGVyID09ICJBZGlwby1DQVIiKSAlPiUgc2VsZWN0KC1jbHVzdGVyKSAlPiUgaGVhZCgxMCkKYGBgCgpNb3N0IGVucmljaGVkIGNlbGwgdHlwZXMgZm9yIHRoZSBPc3Rlb2JsYXN0cyBjbHVzdGVyLgoKYGBge3J9CmVucmljaF90YmwgJT4lIGZpbHRlcihjbHVzdGVyID09ICJPc3Rlb2JsYXN0cyIpICU+JSBzZWxlY3QoLWNsdXN0ZXIpICU+JSBoZWFkKDEwKQpgYGAKCi0tLQoKW3ByZXZpb3VzIHR1dG9yaWFsc10oaHR0cHM6Ly9pZ29yZG90LmdpdGh1Yi5pby90dXRvcmlhbHMvKQoKCg==