Discussion: Directory structure for modules on server

Open
  • Is something wrong?
#246 5 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
20/100
Issue type
Refactor
Clarity
Needs clarification
Activity status
Stale
Tech stack
r
Domain
documentation

Research direction

Start with the scRNA-seq module tree, its README.md, and the Rmd notebooks that reference the current data paths. Compare the direct shared-data paths with the proposed symlink layout, including the RStudio visibility and read-only data considerations. Done means the project has agreed on one structure and the affected module paths and documentation consistently follow it.

Written by the indexing model from the issue text.

Description

discussion

The current directory structure for the scRNA module set looks something like this:

training-modules/scRNA-seq$ tree
.
├── 00-scRNA-seq_introduction.md
├── 01-normalizing_scRNA-seq.Rmd
├── 01-normalizing_scRNA-seq.nb.html
├── 02-tag-based_pre-processing_scRNA-seq.html
├── 02-tag-based_pre-processing_scRNA-seq.md
├── 03-dimension_reduction_scRNA-seq.Rmd
├── 03-dimension_reduction_scRNA-seq.nb.html
├── 04-scrnaseq_exercise.Rmd
├── README.md
├── data
│   ├── glioblastoma
│   │   └── raw
│   │       ├── unfiltered_darmanis_counts.tsv
│   │       └── unfiltered_darmanis_metadata.tsv
│   └── tabula_muris
│       ├── fastq
│       │   ├── tab_mur_10X_P4_3_L001_R1_subset.fastq.gz
│       │   └── tab_mur_10X_P4_3_L001_R2_subset.fastq.gz
│       ├── normalized
│       │   ├── scran_norm_tab_mur.tsv
│       │   └── tab_mur_metadata.tsv
│       └── qc_reports
│           ├── 10X_P4_3_qc_report.html
│           └── Bad_Example_10X_P4_2_qc_report.html
├── diagrams
├── figures
│   ├── gbm_figure.jpg
│   └── sce_structure.png
├── index
├── scripts
│   ├── gene_matrix_filter.R
│   └── read_alevin.R
└── setup
    ├── README.md
    ├── glioblastoma
    └── tabula-muris

There isn't anything inherently wrong with this, but with using the Rstudio server, we now plan to have the raw data stored in shared/data/ to avoid unnecessary duplication.

There are two options that I see: the first is to leave things mostly as they are, but remove the raw data and put the paths to the files directly in the notebooks. This would look something like this, using the current paths in /shared/data:

data_dir <- file.path("~", "shared-data", "training-data", "darmanis")
gene_matrix_file <- file.path(data_dir, "tximport", "count_matrix.tsv")

Note that this uses ~/shared-data/ for the reason that this symlink that we set up for every user is visible in the RStudio file browser without having to use "Go to folder".

The other option is to add moar symlinks within the module, so the new directory structure might look like this:

├── 00-scRNA-seq_introduction.md
├── 01-normalizing_scRNA-seq.Rmd
├── 01-normalizing_scRNA-seq.nb.html
├── 02-tag-based_pre-processing_scRNA-seq.html
├── 02-tag-based_pre-processing_scRNA-seq.md
├── 03-dimension_reduction_scRNA-seq.Rmd
├── 03-dimension_reduction_scRNA-seq.nb.html
├── 04-scrnaseq_exercise.Rmd
├── README.md
├── analysis
│   └── glioblastoma
│       └── normalized
├── data
│   ├── glioblastoma -> ~/shared-data/training-data/darmanis/
│   └── tabula_muris -> ~/shared-data/training-data/tabula_muris/

This would more closely mirror a real project, as the data would appear to be in the module folder and could be referred to that way. It can also encourage the practice that a data folder is for reading only, and anything you write goes in a separate location (though I am not too strict about this myself, as long as there is a raw subfolder of some kind within data).

Thoughts on which of these two options is preferred, or modifications to the proposed structures?

Dominant language
HTML
Stars
80
Forks
35
Avg merge
5h 54m
Merged PRs (30d)
19

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from AlexsLemonade/training-modules

All issues in AlexsLemonade/training-modules

Similar issues

More Documentation issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.