Skip to content

Updates, additions and fixes

Current stable version: v0.25.5 (main downloads page)

v0.25.5 (24-May-2021)

Major new functionality

  • Major changes to the epoch-wise artifact detection via SIGSTATS, which is now implemented via the CHEP-MASK and CHEP commands, as illustrated in this vignette

  • Significant speed improvements for spectral analyses (PSD and especially MTM) and other (e.g. PSC, SUDS, ICA, CPT) commands through code refactoring, tweaking our use of FFTW and the incorporation of the Eigen matrix library

  • A beta-version of our new Luna-powered NAP (NSRR Automated Pipeline) cloud-based portal, described here, and available for testing from this URL http://remnrem.net/nap/ populated with public Sleep-EDF dataset.

  • Principal spectral components analysis via the PSC command

  • Phase slope index connectivity via the PSI command

  • Epoch-wise spherical spline interpolation via the INTERPOLATE command

  • Independent components analysis via the ICA and ADJUST commands

  • EEG microstate analysis via the MS command

  • Sleep stage prediction and evaluation via the SUDS and SOAP commands (with supporting RESOAP and --copy-suds commands)

  • Time-series clustering of epochs and/or channels via the EXE command

  • Cluster-based non-parametric linear models for association analysis via the CPT command

Other new commands/functionality

  • Added cache-metrics options for PSD, MTM, COH, PSI, SPINDLES and SO (i.e. primarily for use with the PSC command)

  • SO caches negative peaks (with the cache option)

  • New MEANS command to give signal means conditional on annotations

  • New special variables describing topographical EEG 64-channel groups, e.g. ${anterior}, ${midline}, etc,

  • Find peaks in signals and subsequently produced peak-locked averages across other channels, via the PEAKS command and the TLOCK command

  • Added a peaks option to the PSD command

  • Cross-correlation and phase synchrony metrics, via the new TSYNC command

  • New A2S and S2A commands to convert anntotations to binary EDF signals channels, and vice versa

  • Restructure EDFs to align records, annotations and epochs via the ALIGN command

  • Added the EDF command to convert EDF+C and EDF+D to standard EDF

  • Added the LINE-DENOISE command, using spectrum interpolation to reduce line noise

  • Added the SEDF command to produce thumbnail-like summary EDFs (to be used in future NAP iterations)

  • Added a simple SIGGEN command to generate test signals (currently only sine waves)

  • Added the CONTAINS command that indicates (via Luna's exit code) whether one or more signals are present in the EDF (i.e. helps guide programmatic driving of Luna pipelines, to be used in future NAP iterations)

  • new output/options for the HYPNO command: 1) stage transition counts and probabilities, with PRE and POST factors; also with conditional probabilities P_PRE_COND_POST and P_POST_COND_PRE; 2) it now outputs epoch-level stage transition information in full and with different naming scheme: TR_NR2REM, etc and TOT_NR2REM. 3) FLANKING_MIN and FLANKING_ALL instead of FLANKING_SIM; 4) option flanking-collapse-nrem (default T), i.e. flank based on any NREM stage (1, 2 or 3) 4) option req-pre-post, to only consider stage transitions that have FLANKING_ALL >= x for the POST stage; defaults to 4 (2 mins). Added 'CONFLICT' and TOTHR outputs.

Minor modifications/fixes

  • the epoch argument of MASK accepts end as the final epoch number
  • added the force-edf and EDF+D options to WRITE to force writing as EDF (or EDF+D); this command also now properly supports EDF+C, EDF+D files properly (e.g. in how the EDF start time is changed, etc)

  • one can now add generic canonical signals via CANONICAL; it will not change the SR if set to missing (.); the canonical signal definition now includes units

  • channel types now include specific reference and independent component types; further, channel type variables are now automatically updated after adding new channels (e.g. added IC to types; when channels added, now they are typed (i.e. the variable is updated by any commands that add channels, ICA --> ${ic} will be available afterwards to match on IC_1, IC_2, etc).

  • we now allow .eannot to be attached with EDF+C but not EDF+D files

  • SPINDLES now has an annot function to generate an internal annotation track (which should be used by WRITE-ANNOTS but can also be used by, e.g. TLOCK, etc. The old ftr format is now retired.

  • all commands that use epoch-level sleep stages (SOAP, HYPNO, STAGE, etc) will note if an epoch has multiple spanning stage annotations (i.e. which might happen if stages and epochs are not temporally well-aligned); it now reports tje CONF variable describing the overlap

  • fixed an issue in selecting ranges when annotations do not align with sample points (fixed for continuous EDFs; need to check whether EDF+D requires additional tweaks)

  • added the epoch option to HYPNO, which now give less verbose output by default

  • added offset and align to EPOCH

  • added annot output for the SOAPcommand

  • added a regional MASK which masks epochs surrounded by masked epochs

  • added annot, hms and no-specials options to WRITE-ANNOTS

  • now the HEADERS command respects the sig option; also, some changes to variable output names

  • fixed the mkdir system call during WRITE for the Windows platform

  • the ANON command now conforms with EDF+ specifications for the null ID

  • added the root option to the ANON command, to specify "dummy" IDs, in the form root_N where N is 1, 2, 3, etc. As the N count is always from 1 for a given run of luna, this can be inconvenient if splitting a sample-list and running in parallel: thus the ids option as above is also given.

  • added the ids command-line option to supply an ID mapper [ old -> new , tab-delim file ]; this can be used with the ANON to set the EDF header IDs

  • --build has an option to add quotes around file paths if they contain spaces

  • the CLOCS command now has an option verbose, to dump pairwise channel distances

  • multiple changes to the .annot file format: allow durations (+seconds); allow hh:mm:ss elapsed time (0+hh:mm:ss offsets); allow fractional seconds in hh:mm:ss specfication; added channel labels and six-column format (but allowing reduced 3-col and 4-col formats for backwards compatibility); headers now optional; allow space/tab or tab-only delimiters; fixed an off-by-one-time-point glitch with clocktime specification; now fully allow 0-duration annotations [a,a); allow ... in the stop field to read until the start of the next; in all imports, times are scalled to 1/1000th second resolution to avoid floating-point nastiness to cause too much trouble; added sep-dp=N option to control the decimal place in .annot outputs;

  • added fac instead of bin for PSD (but, in general, removing support/documentation around this).

  • for SIGNALS command, added the req option to require that specified signals are present/kept; in contrast, the original keep option now will not complain if a requested signal is not present (i.e. keep implies keep if present)

  • added the id command-line option, so that a numeric ID can be specified (i.e. and not be interpreted as a sample-list line number)

  • added a min option to EPOCH, which gives minimal output: just the epoch count to standard output

  • added a dump option to STAGE to write stages to standard out (i.e. parallels eannot option, but does not write to a file)

  • added guess (and eeg) options to the CANONICAL command, to guess csEEG without a file/group specified

Internal

  • added cache_t for PSC command SPINDLES/SO, and [in flight] adding a COUPL command to separate out spindle/SO detection from coupling analyses

  • trims trailing whitespace from the physical dimension and transducer type fields in the EDF header; this was already done for channel labels

  • swapped in functions and classes from the Eigen matrix library for many numerical intensive procedures

  • added a runs test to the stats helper function library

v0.24 (22-August-2020)

New commands

  • ALIASES command to list any channel/annotation aliases done for that individual

  • MINMAX command to set EDF header digital/physical min/max to same values across multiple channels

  • TYPES command and accompanying options ch-match, ch-exact and ch-clear, which groups channels by types (e.g. EEG, EMG, respiratory effort, etc) and defines correspondong automatic variables (e.g. ${eeg} and ${emg}); the HEADERS command now also outputs a TYPE field

  • CANONICAL command to generate canonical signals, cs_EEG, cs_LOC, cs_ROC, cs_EMG and cs_ECG

  • VARS command to output all variables defined for a given individual (designed to be paired with the new individual-level variables feature, via the vars option)

  • CC command for connectivity and coupling statistics

  • ACF command to calculate signal autocorrelation

New options/behaviors for existing commands

  • added SENS and POS variables to HEADERS channel-level output

  • added flat and max options to SIGSTATS

  • CWT-DESIGN now outputs FWHM in time and frequency domains, and accepts newer FWHM-basewd specification (instead of cycles); see this manuscript

  • added the wrapped and fwhm options to CWT

  • added the fft option to FILTER for FFT convolution

  • FILTER and FILTER-DESIGN can now read a FIR from a file; also, instead of Kaiser windows to design a FIR, one can now fix the filter order and specify the type of window (e.g. Hamming, Blackman, etc)

  • added a new option for the REFERENCE command, and the ability to specify . as a reference (i.e. to do nothing, to facilitate automatic processing)

  • added summaries to CORREL, and improved the speed of this command

  • allow now a comma-delimited list when defining stage labels for STAGE and HYPNO, e.g. N1=n1,NREM2

  • added epoch and per-spindle options to the SPINDLES, making this level of output optinal, and omitted by default (i.e. for ExFxCH and NxFxCH)

  • new added cstats and astats options for SIGSTATS

  • added the channels option to the DESC command (to write a simple list of all channel labels to standard output)

  • the ANNOTS command now outputs START_HMS/STOP_HMS and START_ELAPSED_HMS/STOP_ELAPSED_HMS under ANNOT x INST x T output strata, which are the hh:mm:ss version of clock-time and elapsed time since EDF start, respectively

Other changes

  • default log output is now less verbose (unless verbose=1 set)

  • small fixes in processing EDF fields: the reserved field is now set to space (rather than null) characters; also, all EDF header fields (including the reserved field) are now checked for being within the 7-bit US-ASCII range 32-126 (any character outside this range is changed to a ? character)

  • changed behavior to replace spaces in channel names with underscore (_) characters for ease of processing; setting the spaces variable can specify alternate replacement characters; keep-spaces option, if true, means that spaces are retained

  • added support for individual-level variables to be loaded from a file (via the vars option)

  • added automatic special variable ${id}, which can act in the same manner as the ^ special character in scripts, i.e. to represent the current EDF ID

  • added the add= option to turn on/off conditional blocks

  • added a check that any variable defined in a script does not clash with a special variable (i.e. command line option).

  • added include={file list} option, to mirror exclude={file list} (nb. you cannot have both exclude and include)

  • Luna can now read gzipped (compressed) ASCIIs directly

  • added a check that named TAGs do not clash with existing, internal tags, e.g. F or CH

  • fixed an issue with 0-duration annotations, e.g. as may occur with marks from EDF Annotations channel

  • fixed an issue with the ANNOTS command not reporting all output if the EPOCH command hadn't been explicitly called first

  • Luna now explicitly gives an error message if trying to use an .eannot file (or e:1 notation in an .annot file) with EDF+ (i.e. which may be discontinuous). That is, if the input file is an EDF+, Luna requires annotation formats with exact times (i.e. either seconds elapsed since the start of the EDF, or hh:mm:ss format).

  • added EDF+ Annotation parsing; Luna now reads all EDF Annotations events as an annotation track, which is automatically combined with any other annotations (e.g. from XML or .annot files). Currently, all annotations have the class edf_annot, which values are instance IDs/label.

  • added new options to force/skip annotations: force-edf, skip-annots, skip-edf-annots and skip-all-annots.

  • added support for using braces { and } instead of ' to denote string literals in eval expressions (i.e. which can be convenient if writing expressions on the command line with -s where ' is already used)

  • changed formatting of -t text table file names; added --tt-prepend and --tt-append options (equivalently, --tt-prefix and --tt-suffix)

  • SPANNING now works on EDF+D files

  • HEADERS now reports EDF_TYPE

v0.23 (15-Jan-2020)

  • new help function (-h command line argument), to list commands, parameters and their output tables/variables

  • new spindle/slow oscillation coupling permutation option and output variables

  • provisional support for text-table output mode with -t option, optionally writing gzipped text (with corresponding lunaR ltxttab() function

  • new option to read data from a plain text file rather than an EDF

  • automatically generate sample lists with a new --build option, i.e. to recursively find all EDFs and match annotation files in a set of folders

  • added support to read and write compressed EDF files, via the EDFZ format, as highlighed in this vignette

  • new scripting features: command files now allow conditional clauses, via the [[var syntax, within-script variable definitions, via the ${var=xyz}syntax, and expanding numeric sequences, e.g. [ICA][1:10] expands to ICA1,ICA2,ICA3,...

  • new SEGMENTS command to show contiguous time intervals within a discontinuous EDF+

  • can now write EDF+ files

  • new experimental commands for multi-channel EEG: CLOCS (to read channel locations), CHEP (channel/epoch masks), INTERPOLATE (to interpolate missing channels/epoch based on spherical splines), SL (to compute the surface Laplacian), and ICA (implementation of the fastICA algorithm).

  • added individual-ID wildcards in the specification of output database names (-o or -a), i.e. to write to a separate database for each EDF/individual, e.g. -o path/to/out-^.db

  • COH performance has increased, and it now reports imaginary and lagged coherence; also, can accept sig1 and sig2 parameters to more flexibly specify which pairs of channels are considered

  • the CHS output factor is now split into two separate factors, CH1 and CH2, for COH CORREL and MI

  • added SPANNING command to report on "coverage" of an EDF by one or more annotations

  • added special variables silent to turn off all console logging

  • added --xml2 command line option for verbose view of XML tree

  • removed the skip-annots special variable; replaced with two new variables: skip-edf-annots and skip-all-annots

  • the MASK epoch and mask-epoch parameters can now take comma-delimited lists of ranges, e.g. epoch=1,6,8-10,22

  • changed the behavior of assume-pm-start; it is now off by default, and accepts a time parameter to define whether ambiguous times are assigned AM or PM values

  • added START_TIME, START_DATE and (optionally) SIGNALS output variables to the HEADERS command

  • now make new, unique labels for any duplicate channel labels found in an EDF (e.g. EEG, EEG.1, EEG.2, etc) and writes a message to the log

  • now check whether the same channel alias points to more than one channel, and if so, will give an error message

  • can now use hh:mm:ss clock-time format in .annot files (assumes 24 hour; requires start & stop specified)

  • changed clock-time format from hh:mm:ss to hh.mm.ss for compliance with the EDF spec; Luna can still read clock-times with the colon-delimiter however

  • new mode (-a instead of -o) to append to, rather than overwrite, output databases

  • fixed an issue with the MTM command's epoch option, when used with multiple signals

  • fixed issue with command parameters not being recorded in the output database

  • added the nsrr-remap option to turn off auto-remapping of NSRR annotations

  • new RECS and SEGMENTS commands to dump basic EDF record and interval info to stdout

  • added a fail-list option [note: need to check implementation]

  • fixed issue with ANNOTS where an annotation was flagged as overlapping a region/epoch is it ended exactly 1 time-unit beforehand (i.e. ignored convention that intervals are internally represented as [start] to [one past the end] of the interval).

v0.22 (31-March-2019)

Initial public release.