Adding and managing files

Supported file formats

Tracks:

Features:

Files must be formatted according to UCSC guidelines. All widely used chromosome names conventions are accepted, e.g. for human files either ‘chr1’ or ‘1’ can be used, however these conventions should not be mixed within single files.

Adding files

Press the Add files button to bring up the file upload panel.

File upload panel

You can drag and drop files here or press the Add files... button to open a file selection menu. Before starting the upload the following mandatory information must be provided about each file:

Comments are optional.

The contents of the text field can be copied to all files by clicking the icon at the left of the field. The default values can be set using Set defaults... button. Default values are stored using the browser cookies, and the settings will be remembered across different sessions as long as the same web browser is used. File extensions that are not supported will raise an error.

File upload panel with 4 files selected

Individual files can be uploaded by pressing ‘start’ next to the file name or all files can be uploaded at once by pressing the Start upload button at the top of file upload panel.

During the upload process a progress bar is displayed. After upload SeqPlots gives a message that upload was successful or or gives an error message. Common errors are misformatted file formats or chromosome names do not matched the reference genome. For more information please refer to Error explained chapter.

A feedback on successfully upload files

To dismiss the upload window, click on X or outside the window.

Downloading and removing files

Clicking the New plot set button brings up the file collection window. The primary function of this window is to choose signal tracks and feature files to use for calculating the plots. However, it also provides basic file management capabilities. Information on files can be reviewed and files can be downloaded or deleted. Fields can be searched, filtered and sorted by any column. The red x button on the right site of file table removes a single file from the collection, while Remove selected files button will erase all selected files.

The file collection window

Running the plot-set jobs

Pressing the New plot set button brings up the file collection window from which you can choose signal tracks and feature files to calculate average plots and heat maps. If you wish to upload more files please refer to adding new files documentation. The file collection window has three tabs:

The file collection modal

Selecting files

The Tracks and Features tabs displays information about the files and allows you to filter and sort by any column. The “Search:” dialog allows you to find any keyword in any field, while dropdowns below the file grid allow for more advanced filtering on specific columns.

Select files by clicking on the file name or any other part of the row beside Show comment and Download or Remove buttons. Chosen files are highlighted in light blue. Clicking the file name again will cancel the selection. At least one signal track or motif and one feature file must be selected before starting the calculation.

Setting up plot options

Options controlling the plot settings is found below the file selection window:

  1. Bin track @ [bp]: - this numeric input determines the resolution of data acquisition; the default value 10 means that 10bp intervals within the plotting range will be summarized by calculating the mean. Higher values increases the speed of calculation, but decreases resolution. See the explanations.
  2. Choose the plot type - there are three options:
    • Point Features - anchor plot on the start of a feature. By default, plot will be directional if strand information is present (i.e, use start position and plot on positive strand for + strand features and use end position and plot on negative strand for minus strand features). If strand information is not present in the feature file (or if the “ignore strand” option is chosen), plot will use start position of feature and be plotted on the positive strand (see explanations). User chooses length of upstream and downstream sequence to plot.
    • Midpoint Features - similar to point feature, but plot is centered on the midpoint of the feature.
    • Endpoint Features - similar to point feature, but plot is centered on the end of the feature. Strand information is used by default to determine the end side.
    • Anchored Features - features are anchored at start and stop positions and given pseudo-length chosen by the user. Additionally, the user chooses the length of sequence upstream of the start and downstream of the end to plot.
  3. Ignore strand - the directionality (strand) will be ignored all features plotted on the positive strand.
  4. Ignore zeros - signal values of 0 in the track will be be excluded from calculations
  5. Calculate heatmap - selecting this generates and saves a heat map matrix. Select if you wish to generate heatmap; uncheck if you only wish to generate average plots, as this will speed calculations.
  6. Plotting distances in [bp] - the distances in to be plotted:
    • Upstream - the plotting distance in base pairs upstream to the feature
    • Anchored - the pseudo-length, to which the features will be extended or shrunk using linear approximation (only for anchored plots)
    • Downstream - the plotting distance in base pairs downstream to the feature

Plotting sequence motif density

The Sequence features tab allows you to calculate and plot the density of any user-defined motif around the chosen genomic feature using the reference sequence package. Motif plots can be mixed with track files’ signal plots. The following options can be set:

  1. DNA motif - the DNA motif
  2. Sliding window size in base pairs [bp] - the size of the sliding window for motif calculation. The value (number of matching motifs within the window) is reported in the middle of the window, e.g. if window is set to 200bp, DNA motif is “GC” and there are 8 CpGs in first 200 bp of the chromosome the value 8 will be reported at 100th bp.
  3. Display name - The name of the motif that will be shown in key and heatmap labels. Leave blank to use DNA motif value.
  4. Plot heatmap or error estimates - this checkbox determines if heatmap matrix and error estimates should be calculated. If unchecked much faster algorithm will be used for motif density calculation, but only the average plot without the error estimates will be available.
  5. Match reverse complement as well - select if reverse complement motif should be reported as well. For example the TATA motif will report both TATA and ATAT with this option selected.

Sequence motifs selection tab

Clicking Add button adds the motif to plot set, while Reset All clears the motif selection. On the right side of the motif setting panel gives a list summary of included motifs.

Starting the plot set calculation

The options are executed by pressing the Run calculation button. This dismisses the file collection window and brings up the calculation dialog, which shows the progress. On Linux and Mac OS X (systems supporting fork based parallelization) the calculation can be stopped using the Cancel button - this will bring back all settings in file collection window.

The calculation progress dialog

After successful execution the plot array and plot preview panel will appear. In case of error an informative error pop-up will explain the problem. Please refer to the error section for further information.

The plot array

Plotting

This section focuses on average (line) plots and options common between these and heatmaps. For heatmap options please refer to heatmap documentation.

Previewing plot

After calculating or loading a plot set, a plot array of checkboxes is displayed to select the desired pairs of features and tracks/motifs. Clicking on the column name (tracks/motifs) or row name (features) selects/deselects the whole column or row. Clicking on top-left most cell of plot array toggles the selection of whole array.

Plot preview plus Line plot, Heatmap and refresh buttons

If at least one pair on plot array is selected pressing the Line plot button produces an average plot preview and the Heatmap button produces a heatmap preview. Alternatively, pressing the [RETURN] key will also produce the new selection and options. These operations are done automatically in reactive mode (see Advanced options chapter). Plots can be downloaded as PDF files using the Line plot and Heatmap buttons next to Download (at the top of the panel).

Below the plotting buttons are options for labeling plots and setting axes. On application start the first panel responsible for bringing file upload, management and plot set calculation modals is active. The further three panels hold common plot settings.

Titles and axis panel

The view on titles and axis panel

This panel groups settings influencing the plot main title, axis labels, various font sizes plus vertical and horizontal plot limits.

Guide lines and data scaling

The view on guide lines and data scaling

Controls in this panel controls the display of guide lines and error estimates, and allows to log scale the signal prior to plotting.

Keys, labels and colors panel

The view on keys, labels and colors panel (left). Color picker, label text input and Priority/Order checkboxes reviled on plot set grid (right).

This panel groups two types of controls. Colors, Label and Priority/Order are a checkboxes revealing further controls on plot set grid, specific for a feature-track pair or sub-heatmap. Show plot key, Show error estimate key and Legend font size re global controls specific for average plots. Inputs on plot set grid do not have specific labels, but the tooltip explaining their meaning is shown on mouse cursor hover.

Heatmaps are often more informative than average plots. If there is variability in signal along individual instances of a given genomic feature (e.g., because there are different biological classes), an average plot might not represent the behavior of any individual feature and could even give a misleading picture. SeqPlots plots track-feature pairs as sub-heatmaps horizontally aligned on single figure. All sub-heatmaps must have the same number of data rows, hence in single plot mode simultaneous plotting is possible only on single features or feature files containing exact same number of rows. The heatmaps can be sorted and clustered by k-means, hierarchical clustering or super self organising maps (SupreSOM).

Heatmap setup tab

This tab has heatmap specific options for data processing and display.

The view on Heatmap setup tab (left). Color picker, Label text input, Priority/Order checkboxes, Choose individual heatmaps for sorting/clustering control and Set individual color key limits numric inputs reviled on plot set grid (right).

Other options controlling heatmap appearance

The heatmap output shares many display options from other tabs. Here we provide a list of these inputs, please refer to “Viewing and manipulating plots” for further reference.

Getting PDFs and cluster info

Plots can be downladed as PDFs by clicking Line plot or Heatmap buttons in the “Download:” section of the tool panel (above the plot preview).

Download:" section of tool panel with Line plot and Heatmap buttons

The small buttons next to Line plot and Heatmap produce additional output files:

The cluster report contains following columns:

Sample report:

chromosome  start   end     width   strand  metadata_group  originalOrder   ClusterID   SortingOrder    FinalOrder
chrI        9065087 9070286 5200    +       g1              1               1           3               3
chrI        5171285 5175522 4238    -       g1              2               3           50              43
chrI        9616508 9618109 1602    -       g1              3               3           13              43
chrI        3608395 3611844 3450    +       g1              4               3           11              12

Table view:

chromosome start end width strand metadata_group originalOrder ClusterID SortingOrder FinalOrder
chrI 9065087 9070286 5200 + g1 1 1 3 3
chrI 5171285 5175522 4238 - g1 2 3 50 43
chrI 9616508 9618109 1602 - g1 3 3 13 43
chrI 3608395 3611844 3450 + g1 4 3 11 12

PDF output size

The last tab (Batch operation and setup) on the tool panel includes batch operations and various other settings including PDF output size. By default the output PDF will be A4 landscape. This can be changed using the drop-down list to following settings:

The view on top part of batch operation and setup panel

Batch operations

Controls to plot multiple plots at once are located on the Batch operation and setup tab, just below PDF paper options. It is possible to output the plots to multipage PDF, plot an array of plots on a single page (for average plots) or mix these options together.

The view on bottom part of batch operation and setup panel

The first drop-down controls the type of the plot - either average or heatmap. The second drop down determines the strategy to traverse the plot grid. The options include:

The multi plot grid option controls how many plots will be placed on each page of the PDF output, e.g. 1x1 means one plot per one page, while 3x4 means 3 columns and 4 rows of plots. If number of plots exceeds the number of slots on page the new page will be added to the PDF.

Filter names will apply a filter to plot titles, which are based on on uploaded file names. For example, if you uploaded 100 files starting with a prefix of “my_experiment_”, you can remove this fragment from each plot title and/or heatmap caption by putting this string in Filter names.

Finally, pressing Get PDF produces the final output file. Please see example below:

Batch plot usage example - multiple average plots aranged in 6x2 plot grid Saving and loading plotsets ============================

If desired, SeqPlots will save plot sets as binary R files, allowing you to quickly load the pre-calculated set for replotting. Saved plot sets can also be shared with other SeqPlots users.

Load or save plotset

Controls available on the “Load or save plotset” panel:

All saved dataset can be found in data location/publicFiles. Any SeqPlots Rdata binaries put in the folder will become available for loading in Load saved plot set control.

The view on the Load or save plotset panel

Plot set files structure

The plot sets files can be also directly loaded in R. This allows further processing and customization of the plots. Data structure is a nested list, which elements be accessed by [[ R operator. The nesting goes as follow:

The example structure:

List of 2
 $ HTZ1_Differential_genes_TOP100_v2.gff:List of 2
  ..$ HTZ1_JA00001_IL1andIL2_F_N2_L3_NORM_linear_1bp_IL010andIL009_averaged.bw    :List of 7
  .. ..$ means   : num [1:501] 2.52 2.52 2.52 2.53 2.54 ...
  .. ..$ stderror: num [1:501] 0.114 0.112 0.111 0.11 0.109 ...
  .. ..$ conint  : num [1:501] 0.226 0.223 0.221 0.218 0.217 ...
  .. ..$ all_ind : num [1:501] -1000 -995 -990 -985 -980 -975 -970 -965 -960 -955 ...
  .. ..$ e       : NULL
  .. ..$ desc    : chr "HTZ1_JA00001_IL1andIL2...\n@HTZ1_Differential_genes_TOP100_v2"
  .. ..$ heatmap : num [1:100, 1:501] 2.36 5.25 2.2 3.48 4.32 ...
  ..$ HTZ1_JA00001_IL3andIIL5_F_lin35_L3_NORM_linear_1bp_IL008andIL011_averaged.bw:List of 7
  .. ..$ means   : num [1:501] 2.36 2.35 2.35 2.36 2.38 ...
  .. ..$ stderror: num [1:501] 0.126 0.125 0.125 0.126 0.125 ...
  .. ..$ conint  : num [1:501] 0.249 0.249 0.247 0.251 0.249 ...
  .. ..$ all_ind : num [1:501] -1000 -995 -990 -985 -980 -975 -970 -965 -960 -955 ...
  .. ..$ e       : NULL
  .. ..$ desc    : chr "HTZ1_JA00001_IL3andIIL5...\n@HTZ1_Differential_genes_TOP100_v2"
  .. ..$ heatmap : num [1:100, 1:501] 2.61 3.17 1.42 2.46 4.26 ...
 $ HTZ1_Differential_genes_BOTTOM100.gff:List of 2
  ..$ HTZ1_JA00001_IL1andIL2_F_N2_L3_NORM_linear_1bp_IL010andIL009_averaged.bw    :List of 7
  .. ..$ means   : num [1:501] 1.57 1.57 1.58 1.6 1.62 ...
  .. ..$ stderror: num [1:501] 0.0996 0.0985 0.1003 0.1022 0.1018 ...
  .. ..$ conint  : num [1:501] 0.198 0.195 0.199 0.203 0.202 ...
  .. ..$ all_ind : num [1:501] -1000 -995 -990 -985 -980 -975 -970 -965 -960 -955 ...
  .. ..$ e       : NULL
  .. ..$ desc    : chr "HTZ1_JA00001_IL1andIL2...n@HTZ1_Differential_genes_BOTTOM100"
  .. ..$ heatmap : num [1:100, 1:501] 1.64 1.37 1.61 1.77 1.86 ...
  ..$ HTZ1_JA00001_IL3andIIL5_F_lin35_L3_NORM_linear_1bp_IL008andIL011_averaged.bw:List of 7
  .. ..$ means   : num [1:501] 1.94 1.94 1.95 1.96 1.97 ...
  .. ..$ stderror: num [1:501] 0.123 0.123 0.124 0.126 0.128 ...
  .. ..$ conint  : num [1:501] 0.244 0.245 0.246 0.251 0.253 ...
  .. ..$ all_ind : num [1:501] -1000 -995 -990 -985 -980 -975 -970 -965 -960 -955 ...
  .. ..$ e       : NULL
  .. ..$ desc    : chr "HTZ1_JA00001_IL3andIIL5...\n@HTZ1_Differential_genes_BOTTOM100"
  .. ..$ heatmap : num [1:100, 1:501] 1.61 1.37 1.29 3.04 3.77 ...

Advanced options

Some additional SeqPlots options are located at very bottom of the Batch operation and setup tab:

The view on Advanced options section of the batch operation and setup panel

Genomes managment

The Manage reference genomes tab allows to add or remove reference genomes installed with SeqPlots. New genomes are added to user data folder, so they can be easily moved along with data and would not be removed when updating R or SeqPlots. User can choose to install standard genomes package available with Bioconductor or install forged genomic package from local file.

The view on Manage reference genomes panel

Error messages

Problem with line N: "line_text" [internal_error]

The import of feature file (GFF or BED) was not successful due to mis-formatted file.


Chromosome names provided in the file does not match ones defined in reference genome. 
INPUT: [chr3R, chr2L, chr2R, chr3L] 
GENOME: [chrI, chrII, chrIII, chrIV, chrV, ...]

There are unexpected chromosome names in input file. Following genomes: Arabidopsis thaliana, Caenorhabditis elegans, Cyanidioschyzon_merolae, Drosophila melanogaster, Homo sapiens, Oryza sativa, Populus trichocarpa, Saccharomyces cerevisiae and Zea mays support chromosome names remapping between different naming conventions, including: AGPv2, ASM9120v1, Ensembl, JGI2_0, MSU6, NCBI, TAIR10 and UCSC. If you see above error in one of these genomes there are still unexpected names after the correction. The problematic chromosome names are given in the error message. Remove GFF/BED lines corresponding to them or upgrade the genome to one containing proper naming. Alternatively set genome to NA.


File already exists, change the name or remove old one.

File named like this already exists in the database, it is impossible to have two files sharing same filename.


ERROR: solving row 300: negative widths are not allowed

The the row 300 have end coordinate smaller than beginning, hence the width in negative. To fix it the start and stop indicates should be swapped. This error often happens when negative strand (-) ranges are misformatted.


Explanations


References

This section lists R, JavaScript and CSS libraries used by SeqPlots, important conceptual contributions to the software, and publication, where figures generated by SeqPlots are featured.

R project and Bioconductor

JavaScript and CSS

Important conceptual contribution to the project

Server deployment

Publications containing figures made by SeqPlots

Session Information

## R version 3.2.2 (2015-08-14)
## Platform: x86_64-apple-darwin13.4.0 (64-bit)
## Running under: OS X 10.10.2 (Yosemite)
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats4    stats     graphics  grDevices utils     datasets  methods  
## [8] base     
## 
## other attached packages:
##  [1] seqplots_1.7.2                    ggplot2_1.0.1                    
##  [3] BSgenome.Celegans.UCSC.ce10_1.4.0 BSgenome_1.36.3                  
##  [5] rtracklayer_1.28.10               Biostrings_2.36.4                
##  [7] XVector_0.8.0                     GenomicRanges_1.20.6             
##  [9] GenomeInfoDb_1.4.2                IRanges_2.2.7                    
## [11] S4Vectors_0.6.5                   testthat_0.10.0                  
## [13] BiocGenerics_0.14.0               devtools_1.8.0                   
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_0.12.0             Rsamtools_1.20.4       
##  [3] class_7.3-14            digest_0.6.8           
##  [5] mime_0.4                R6_2.1.1               
##  [7] plyr_1.8.3              futile.options_1.0.0   
##  [9] evaluate_0.7.2          RSQLite_1.0.0          
## [11] spam_1.0-1              httr_1.0.0             
## [13] zlibbioc_1.14.0         lazyeval_0.1.10        
## [15] curl_0.9.3              rstudioapi_0.3.1       
## [17] DT_0.1.31               rmarkdown_0.8          
## [19] proto_0.3-10            BiocParallel_1.2.21    
## [21] stringr_1.0.0           htmlwidgets_0.5        
## [23] RCurl_1.95-4.7          munsell_0.4.2          
## [25] shiny_0.12.2            httpuv_1.3.3           
## [27] rversions_1.0.2         htmltools_0.2.6        
## [29] gridExtra_2.0.0         roxygen2_4.1.1         
## [31] codetools_0.2-14        XML_3.98-1.3           
## [33] crayon_1.3.1            GenomicAlignments_1.4.1
## [35] MASS_7.3-44             bitops_1.0-6           
## [37] grid_3.2.2              jsonlite_0.9.17        
## [39] xtable_1.7-4            gtable_0.1.2           
## [41] DBI_0.3.1               formatR_1.2            
## [43] git2r_0.11.0            magrittr_1.5           
## [45] covr_1.2.0              scales_0.3.0           
## [47] stringi_0.5-5           kohonen_2.0.19         
## [49] reshape2_1.4.1          rex_1.0.1              
## [51] xml2_0.1.2              futile.logger_1.4.1    
## [53] BiocStyle_1.6.0         lambda.r_1.1.7         
## [55] RColorBrewer_1.1-2      tools_3.2.2            
## [57] Cairo_1.5-8             maps_2.3-11            
## [59] fields_8.2-1            yaml_2.1.13            
## [61] plotrix_3.5-12          parallel_3.2.2         
## [63] colorspace_1.2-6        memoise_0.2.1          
## [65] knitr_1.11