From a146728c3aa8445aab3f510fda4c5978b240318c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Fri, 31 Jul 2026 15:33:26 +0200 Subject: [PATCH 1/3] Remove page about DICOM archival This page is more of a historical record as it contained a proposal for a particular solution. At the time of writing, it was included in the docs for better visibility, but now feels outdated. AFAIK all sites have now adapted their workflows to their preferences, but I am not privy to the details. --- .../work-with-raw-mri/archive-dicoms/index.md | 45 ------------------- 1 file changed, 45 deletions(-) delete mode 100644 content/howtos/work-with-raw-mri/archive-dicoms/index.md diff --git a/content/howtos/work-with-raw-mri/archive-dicoms/index.md b/content/howtos/work-with-raw-mri/archive-dicoms/index.md deleted file mode 100644 index 57baeac..0000000 --- a/content/howtos/work-with-raw-mri/archive-dicoms/index.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -title: Archive DICOMs ---- - -As the "true" raw data, DICOMs are rarely (re)accessed and hardly ever need to change. -However, they need to be stored somewhere. -Tracking DICOMs in DataLad datasets allows dependency tracking for conversion to NIfTI. -However, it's good to keep DataLad optional (allow DataLad and non-DataLad access). - -## Historical precedent: ICF at FZJ - -The following solution has been proposed for the Imaging Core Facility at FZJ: - -- DICOMs are packed into tar files (tarballs) [^1] -- the tarballs are placed on a web server (intranet only), organized - by project (HTTP Basic Authentication for access management) -- DataLad datasets record availability via - [archivist](https://docs.datalad.org/projects/next/en/latest/generated/datalad_next.annexremotes.archivist.html) - and - [uncurl](https://docs.datalad.org/projects/next/en/latest/generated/datalad_next.annexremotes.uncurl.html) - special remotes, which translates to: - - a file is available from a tarball (archivist special remote) - - a tarball is available from a given URL, pointing to the web server (uncurl special remote)[^2]. -- Only the Git repository (no annex) is stored by the consuming institute; - the ICF web server is the permanent DICOM storage. - -The system has been documented in https://inm-icf-utilities.readthedocs.io/en/latest/ -and the tarball & dataset generation tools implementation is in https://github.com/psychoinformatics-de/inm-icf-utilities. - -[^1]: timestamps are normalized to ensure re-packing the same data does not change tarball checksums -[^2]: uncurl is chosen because it allows re-writing patterns with just configuration, e.g., should the base URL change - -## TRR reimplementation - -One of the TRR sites indicated intent to use a Forgejo instance for DICOM storage. -A particular challenge for the underlying system was [inode](https://en.wikipedia.org/wiki/Inode) limitation. -For this reason, an adaptation of the ICF system has been proposed: - -- a dataset is generated upfront, and DICOM tarball is stored with the dataset in Forgejo (as annex) -- we keep using the archivist remote (file in tarball) to avoid using up thousands of inodes for individual files - (Git can pack its repository into several files, so we only add one more for the tarball). - -A proof of principle for dataset generation (using re-written ICF -code) has been proposed in https://hub.trr379.de/q02/dicom-utilities. See the README for more -detailed explanations (and commit messages for even more detail). -- 2.52.0 From 9312103e7af271d423d2d98917574c11ec90fc9b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Fri, 31 Jul 2026 16:54:15 +0200 Subject: [PATCH 2/3] Rewrite the BIDS conversion page The BIDS conversion page contained a mix of general statements and fine details (reflecting open discussions at the time of writing). Now, considering the "less is more" principle, the introduction is kept, but the other sections are changed to communicate the following: - selected tooling (heudiconv / dcm2niix), what id does - shared resources (public repos, reused across sites) - DataLad layout of a BIDS dataset (what subdatasets and why) --- .../bids-conversion/index.md | 77 +++++-------------- 1 file changed, 19 insertions(+), 58 deletions(-) diff --git a/content/howtos/work-with-raw-mri/bids-conversion/index.md b/content/howtos/work-with-raw-mri/bids-conversion/index.md index cc3eb14..8158923 100644 --- a/content/howtos/work-with-raw-mri/bids-conversion/index.md +++ b/content/howtos/work-with-raw-mri/bids-conversion/index.md @@ -10,74 +10,35 @@ TRR379 uses the [Brain Imaging Data Structure](bids.neuroimaging.io) (BIDS) as t ## Conversion to BIDS The conversion of raw MRI data in DICOM format to a BIDS-compliant dataset is a largely automated process. -The recommended software to be used for conversion is [Heudiconv](https://heudiconv.readthedocs.io). - +The recommended software to be used for conversion is [heudiconv](https://heudiconv.readthedocs.io). Heudiconv uses [dcm2niix](https://github.com/rordenlab/dcm2niix) as the actual DICOM→NIfTI converter. -In our experience, `dcm2niix` is the most robust and most correct tool available for this task. +[Heudiconv tutorials](https://heudiconv.readthedocs.io/en/latest/tutorials.html) further illustrate how the software works. -Heudiconv does the job of mapping DICOM series to BIDS entities (ie. determine BIDS-compliant file names). -A key heudiconv concept is a **heuristic**: a Python program (function) which looks at the DICOM series properties and matches it with a file naming pattern. +Heudiconv performs the task of mapping DICOM series to BIDS entities (ie. determining BIDS-compliant file names). +A key heudiconv concept is a **heuristic**: a Python program (function) which receives the DICOM series properties and matches them with a file naming pattern. A heuristic typically relies on DICOM series naming (set at the scanner console), but it can also use other properties such as number of images or acquisition parameters. -Because TRR379 uses its own conventions, a matching heuristic needs to be provided (possibly one for each TRR379 site). -An [implementation of such a heuristic](https://hub.trr379.de/q02/phantom-mri-bids/src/branch/main/code/heuristic-q01.py) has been created, and was tested on phantom MRI acquisitions from all sites (see below). -Using this heuristic, MRI data from all sites can be BIDS-standardized. -As with any automation, caution and oversight is needed for edge cases (e.g. repeated / discarded acquisitions). +Heudiconv workflows have been implemented at each MRI acquisition site, and reuse shared components. -[Heudiconv tutorials](https://heudiconv.readthedocs.io/en/latest/tutorials.html) further illustrate the process and capabilities of the software. +## Shared Resources -### Good practices +- [q02/heudiconv-container](https://hub.trr379.de/q02/heudiconv-container) is a DataLad dataset providing a heudiconv container. +- [q02/rdmtools](https://hub.trr379.de/q02/rdmtools) contains Python scripts which further automate BIDS conversion, pushing created datasets to dedicated locations, and other data curation tasks. +- [q01/phantom-mri-dicoms](https://hub.trr379.de/q01/phantom-mri-dicoms) is a superdataset grouping phantom MRI scans from all sites, used to develop and test the conversion workflow; it also contains the [reference TRR379 heudiconv heuristic](https://hub.trr379.de/q01/phantom-mri-dicoms/src/branch/main/code/heuristic-q01.py). -- Use heudiconv as a containerized application. [Q02]({{% ref "/references/resources/projects/q02/" %}}) provides a [readily usable utility dataset with a - configured container](https://hub.trr379.de/q02/heudiconv-container). See that repository for - an example usage. -- DICOMs as subdatasets helps with provenance, even if those DICOMs are never accessed outside -- Heudiconv takes paths and (optionally) intended subject IDs as input - - if paths contain identifying information, this would leak into DataLad run records - - having a helper script / lookup table in the (private) DICOM dataset can hide this information +> [!important] +> Each site maintains *its own copy of the heuristic* (as part of the site-specific DICOM superdataset) to account for site-specific differences. -### Caveats +## Details -- https://hub.trr379.de/q02/phantom-mri-bids used dcm2niix v1.0.20240202 - - current latest is v1.0.20250506 - - potential impact discussed in https://hub.trr379.de/q02/phantom-mri-bids/issues/8 +### Modular datasets -## Demonstrators and resources +A DataLad dataset containing a BIDS dataset will typically have these subdatasets. +Subdatasets help in recording provenance while enabling different access scopes for different components: -### TRR phantom DICOMs - -Scans of MRI phantoms were carried out using the intended sequences (presumably - see caveats section below). -These were shared with Q02 and uploaded to the TRR Hub forgejo instance: - -- https://hub.trr379.de/q01/phantom-mri-dicom-aachen -- https://hub.trr379.de/q01/phantom-mri-dicom-frankfurt -- https://hub.trr379.de/q01/phantom-mri-dicom-heidelberg -- https://hub.trr379.de/q01/phantom-mri-dicom-mannheim - -Note: Aachen did a re-scan which was shared by e-mail / cloud (June 03, 2025). -This has not been uploaded to forgejo (permissions + size). - -### TRR phantom BIDS - -- A BIDS-compliant dataset from these dicoms (3/4 sites): https://hub.trr379.de/q02/phantom-mri-bids -- The heuristic used: https://hub.trr379.de/q02/phantom-mri-bids/src/branch/main/code/heuristic-q01.py -- Issue tracker: https://hub.trr379.de/q02/phantom-mri-bids/issues - -Conversion of re-scanned Aachen phantom is in https://hub.trr379.de/q02/tmp-phantom-bids (separate from the above -because input data is not available as a DataLad dataset) - -### Data consistency - -- the phantom datasets are not the same: https://hub.trr379.de/q02/phantom-mri-bids/issues/6 -- re-scan from Aachen has more sequences than the initial scan, but lacks T2w -- heudiconv fails to parse some Heidelberg dicoms, and dcm2niix raises warnings; - unclear whether this is data issue or software issue: https://hub.trr379.de/q02/phantom-mri-bids/issues/5 - -### Conversion: technical issues - -These are open questions: - -- Technical: BIDS validator errors https://hub.trr379.de/q02/phantom-mri-bids/issues/7 -- Technical: re-run with the latest dcm2niix https://hub.trr379.de/q02/phantom-mri-bids/issues/8 +- `code/heudiconv`: heudiconv container ([q02/heudiconv-container](https://hub.trr379.de/q02/heudiconv-container)) +- `code/rdmtools`: optional, shared scripts for data curation tasks ([q02/rdmtools](https://hub.trr379.de/q02/rdmtools)) +- `sourcedata/dicoms`: a site-specific DICOM dataset, itself containing one subdataset per session +- `.heudiconv`: heudiconv cache / intermediate outputs are stored in a separate dataset to avoid including DICOM information in the BIDS dataset -- 2.52.0 From bef390a02b6793818fe9aef38f702960de884870 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Szczepanik?= Date: Fri, 31 Jul 2026 17:15:29 +0200 Subject: [PATCH 3/3] Fix typos found with codespell --- content/advanced/deploy-services/dumpthings/index.md | 2 +- content/advanced/deploy-services/neurobagel/index.md | 2 +- content/howtos/pool-tool/report-publications/index.md | 4 ++-- content/references/goals/rdm-infrastructure/_index.md | 2 +- content/references/services/calendars/_index.md | 4 ++-- .../references/services/cohort-data-discovery/_index.md | 2 +- content/references/services/matrix/_index.md | 2 +- content/support/pool-tips/index.md | 8 ++++---- 8 files changed, 13 insertions(+), 13 deletions(-) diff --git a/content/advanced/deploy-services/dumpthings/index.md b/content/advanced/deploy-services/dumpthings/index.md index 51725bc..b87d0e8 100644 --- a/content/advanced/deploy-services/dumpthings/index.md +++ b/content/advanced/deploy-services/dumpthings/index.md @@ -22,7 +22,7 @@ However, the service could also run under any other (existing) user account. # new user, prohibit login, disable password sudo adduser dumpthing --disabled-password --disabled-login -# allow this user to run prcoess while not logged in +# allow this user to run process while not logged in sudo loginctl enable-linger dumpthing # allow this user to execute systemd commands interactively. diff --git a/content/advanced/deploy-services/neurobagel/index.md b/content/advanced/deploy-services/neurobagel/index.md index 5132abf..f6d5eb6 100644 --- a/content/advanced/deploy-services/neurobagel/index.md +++ b/content/advanced/deploy-services/neurobagel/index.md @@ -39,7 +39,7 @@ NeuroBagel will be deployed under this user account, and all software and data w # new user, prohibit login, disable password sudo adduser neurobagel --disabled-password --disabled-login -# allow this user to run prcoess while not logged in +# allow this user to run process while not logged in sudo loginctl enable-linger neurobagel # allow this user to execute systemd commands interactively. diff --git a/content/howtos/pool-tool/report-publications/index.md b/content/howtos/pool-tool/report-publications/index.md index 074f692..46e851e 100644 --- a/content/howtos/pool-tool/report-publications/index.md +++ b/content/howtos/pool-tool/report-publications/index.md @@ -32,7 +32,7 @@ Enter just the DOI without the URL project (i.e., `10.1016/j.neubiorev.2025.1063 7. Navigate to your new Publication record in the pool's Publication records list, and click the pencil icon to edit it. -![Screenshot hilighting the edit button to edit an existing Publication record](edit-publication.webp) +![Screenshot highlighting the edit button to edit an existing Publication record](edit-publication.webp) 8. For publications to make the best impressions and live up to their fullest metadata potential, proceed to add relevant information to complete the Publication record as follows: @@ -74,7 +74,7 @@ Enter just the DOI without the URL project (i.e., `10.1016/j.neubiorev.2025.1063 {{% /expand %}} -{{% expand title="Funding acknowledgement" %}} +{{% expand title="Funding acknowledgment" %}} - At the very bottom of your Publication record editing screen, toggle the "All fields" option to on (indicated in blue). - Within the "Annotations" box, search "TRR funding" and select whether TRR funding is or is NOT acknowledged in the publication. diff --git a/content/references/goals/rdm-infrastructure/_index.md b/content/references/goals/rdm-infrastructure/_index.md index 31ca466..ea22fdc 100644 --- a/content/references/goals/rdm-infrastructure/_index.md +++ b/content/references/goals/rdm-infrastructure/_index.md @@ -148,7 +148,7 @@ of TRR379. Sites are free to implement any RDM solutions, as long as that infrastructure provides -- (programmatically) queriable metadata of a previously agreed upon nature +- (programmatically) queryable metadata of a previously agreed upon nature - (programmatically) accessible data to any authorized members of TRR379 with the aim to enable reproducible research from primary data to published diff --git a/content/references/services/calendars/_index.md b/content/references/services/calendars/_index.md index fa63f5f..6b2470f 100644 --- a/content/references/services/calendars/_index.md +++ b/content/references/services/calendars/_index.md @@ -18,7 +18,7 @@ Contact the management team to request a dedicated calendar. ## Public calendars -All public calenders are available via CalDAV URL and can be included in any +All public calendars are available via CalDAV URL and can be included in any calendar solution, such as Google calendar. The URLs follow the pattern `https://cal.trr379.de/public/`, @@ -26,7 +26,7 @@ where `` is the calendar name, as stated in the list below. For the events calendar this is `https://cal.trr379.de/public/events`. For use with Google calendar, replace `https://` with `webcal://`. For example, -the events calendar can be added to Google calender with the URL +the events calendar can be added to Google calendar with the URL `webcal://cal.trr379.de/public/events`. Available public calendars: diff --git a/content/references/services/cohort-data-discovery/_index.md b/content/references/services/cohort-data-discovery/_index.md index cf0b867..945385a 100644 --- a/content/references/services/cohort-data-discovery/_index.md +++ b/content/references/services/cohort-data-discovery/_index.md @@ -4,6 +4,6 @@ weight: 70 --- https://nb-query.trr379.de hosts the central [NeuroBagel](https://neurobagel.org) query interface of the TRR379. -With this solution, data availability can be queried dynamically, to faciliate data access requests. +With this solution, data availability can be queried dynamically, to facilitate data access requests. NeuroBagel supports federated queries and is therefore ideally suited for the decentralized research data management approach of TRR379. diff --git a/content/references/services/matrix/_index.md b/content/references/services/matrix/_index.md index 1f5a4c4..9a67308 100644 --- a/content/references/services/matrix/_index.md +++ b/content/references/services/matrix/_index.md @@ -12,6 +12,6 @@ Many institutions provide their own [homeservers](https://matrix.org/docs/matrix If not, anyone can create a free account at https://app.element.io/. The TRR379 Matrix space is only for TRR379 consortium members. -To get access, email [trr379@lists.fz-juelich.de](mailto:trr379@lists.fz-juelich.de?subject=TRR379%20Matrix%20access) with your Matrix acccount name included, or contact @mih:matrix.org directly on matrix. +To get access, email [trr379@lists.fz-juelich.de](mailto:trr379@lists.fz-juelich.de?subject=TRR379%20Matrix%20access) with your Matrix account name included, or contact @mih:matrix.org directly on matrix. diff --git a/content/support/pool-tips/index.md b/content/support/pool-tips/index.md index 816f567..e504a66 100644 --- a/content/support/pool-tips/index.md +++ b/content/support/pool-tips/index.md @@ -9,7 +9,7 @@ To assist with common issues you may encounter while working with the [knowledge - It is **highly recommended** to use the knowledge pool's web UI with a desktop browser; the form offers rich contextual information that is hard to access on mobile devices. - Submitting records will only be possible using a [valid access token]({{% ref "/start/accounts/generate-token/" %}}); the token can be entered by navigating to "Settings" (gear icon) and then pasting your token in the "TOKENS" tab. -- When editing records, you cna hover your mouse pointer over field labels to display help text describing expected values and controlled vocabularies. +- When editing records, you can hover your mouse pointer over field labels to display help text describing expected values and controlled vocabularies. - For some fields (e.g., journal name when filling out a Publication record), one or more choices have to be selected. The selector offers type-ahead search of choices. When searching: - Try entering full names instead of abbreviations. - If an item is not found in one language, try searching its English version. @@ -19,13 +19,13 @@ To assist with common issues you may encounter while working with the [knowledge ## Common errors -- **Authorization issues**: If the [access token]({{% ref "/start/accounts/generate-token/" %}}) you are providing is incorrect or does not have approriate permissions, the interface may desplay one or more of the following errors during metadata retrieval or submission: +- **Authorization issues**: If the [access token]({{% ref "/start/accounts/generate-token/" %}}) you are providing is incorrect or does not have appropriate permissions, the interface may display one or more of the following errors during metadata retrieval or submission: ![Screenshot of the "Token not set" error message](token-not-set.webp) ![Screenshot of the token prompt displaying a message about token requirements](token-management.webp) -![Screenshot of an invalid token error message displayed during a record submission attemp](invalid-token.webp) +![Screenshot of an invalid token error message displayed during a record submission attempt](invalid-token.webp) -- **Incomplete information**: If a metadata record is missing required information, you will not be able to save it. Required fields - highlighted with a red asterix - and an warning sign will alert you to this. +- **Incomplete information**: If a metadata record is missing required information, you will not be able to save it. Required fields - highlighted with a red asterisk - and an warning sign will alert you to this. - **Internal errors**: If there is an unexpected error during metadata submission, the submission interface will report technical information. Please [contact the Q02 project team]({{% ref "/support/" %}}) and copy the entire error message. -- 2.52.0