Add exdf-tree CLI to visualize source and key structure
This has been my procrastination side project for the past few weeks, escalating slowly and slowly to much more than I initially planned to.
exdf-tree
is a visualization tool for the source and key structure of EXDF files as trees, with as little EuXFEL-assumption as possible while offering various convenience functionality.
Basic Examples
% exdf-tree /gpfs/exfel/exp/SQS/202501/p007901/raw/r0100/RAW-R0100-DA04-S00000.h5
DataCollection
├── SQS_AQS_GDS/GAUGE/FORE
├── SQS_AQS_LAS/MOTOR/IRIS_800
...
├── SQS_ILH_LAS/MOTOR/DELAY_AX_1030
└── SQS_REMI_MOLB/TSENS/T_GAS_2
% exdf-tree /gpfs/exfel/exp/SQS/202501/p007901/raw/r0100/RAW-R0100-DA04-S00000.h5 -S
DataCollection
...
├── SQS_AQS_LAS
│ └── MOTOR
│ ├── IRIS_800
...
│ └── DELAY_AX_1030
└── SQS_REMI_MOLB
└── TSENS
└── T_GAS_2
% exdf-tree /gpfs/exfel/exp/SQS/202501/p007901/raw/r0100/RAW-R0100-DA04-S00000.h5 -t
DataCollection
...
├── MDL
│ ├── SQS_ILH_LAS/ /DELAY_AX_800
│ └── SQS_ILH_LAS/ /DELAY_AX_1030
├── MOTOR
│ ├── SQS_AQS_LAS/ /IRIS_800
...
│ └── SQS_ILH_LAS/ /DELAY_AX_1030
└── TSENS
├── SQS_AQS_OVEN/ /I_OVEN_1
├── SQS_DPU_LIC/ /T_MIRROR_1
└── SQS_REMI_MOLB/ /T_GAS_2
Grouping by aggregator (adding a level in the front) and domain (the first part of a Karabo device ID) is also supported. The underlying code is capable of grouping on any level, but only those seem useful.
Keys are omitted by default and can be added with -k
(flat) and -K
(as a tree). The tree can be filtered down by --match, -m
based on fnmatch
.
Special sauce
If possible, items are compacted to shorten the tree as much as possible. There are two ways of compaction supported right now:
-
Combine identical prefixes for at least two sources, most commonly if there is both a
CONTROL
andINSTRUMENT
source present:% exdf-tree /gpfs/exfel/exp/SQS/202501/p007901/raw/r0100/RAW-R0100-DA01-S00000.h5 DataCollection ├── SA1_XTD2_XGM/XGM/DOOCS(:output) ├── SA3_XTD10_CALL/MDL/ELCOMAT_AUTOCOLLIMATOR ├── SA3_XTD10_DOOCS/BAM/LCAT414_B2(:output) ├── SA3_XTD10_DOOCS/BAM/LCAT1932[MS]_TL(:output) ...
-
Combine source names identical beyond a single character:
% exdf-tree /gpfs/exfel/exp/SQS/202501/p007901/raw/r0100/RAW-R0100-DA01-S00000.h5 ├── SQS_REMI_MOLB/GAUGE/JET_GASLINE_[123] ... ├── SQS_REMI_MOLB/TSENS/T_NOZZLE_[12] ├── SQS_REMI_VAC/GAUGE/DIFF_[12] ├── SQS_REMI_VAC/GAUGE/DUMP_[12]