MUX EPI

Simultaneous Multi-Slice (SMS) EPI at CNI (a.k.a., MUX EPI)

NOTE: if nothing else, be sure to read and understand the Calibration Images section!

Overview

The CNI, in collaboration with GE, has implemented simultaneous multi-slice EPI (also known as multiband EPI, multiplexed EPI, or, as we like to call it, "mux EPI"). We have functioning sequences for both gradient-echo (GRE) and spin-echo (SE) EPI. GRE-EPI is useful for BOLD imaging and SE-EPI is useful for diffusion imaging and other quantitative anatomical methods.

Useful Parameters

• number of simultaneously-acquired bands (mux factor): How many slices to acquire simultaneously.
• inplane acceleration (arc factor): Determines the ky undersampling. Currently limited to an integer. E.g., 2 will acquire every other ky line.
• slice separation (mm): For a contiguous set of slices, this must equal (slice_thickness + gap) * num_slices. (Note that setting this to zero will automatically compute the separation needed for contiguous slices.)
• CAIPI z-blip flag: enables CAIPI z-blips (see Setsompop et. al. 2012)
• CAIPI FOV shift: controls the CAIPI field shift. In general, this should be set to 3 (or maybe 4 for higher mux factors)

Calibration Images

The first few acquired images are fully-sampled and used as reference images for the reconstruction. It is critical that the subject not move during this calibration phase, or the whole dataset may be impossible to reconstruct without significant artifacts (if at all). The number of reference images depends on the mux and arc factors as well as the CV that controls the number of reference scan repetitions (stop_mux_cycling_rf1, exposed in the User CVs as "Number of calibration scans"). The number of TRs needed for the reference images is: arc * mux * num_mux_cycle. (num_mux_cycle default is 2). The NIFTI file that you get will include these extra images at the beginning, so you should exclude them from your analysis! (See the next section for details.)

Timing and Number of TRs

Here are some useful equations to keep in mind if you are setting up a mux sequence for an fMRI experiment that is n TRs long per run, not including calibration time (i.e., you want to have n usable TRs from a scan):

• total number of TRs acquired = n + calibration TRs
• number of calibration TRs = mux * arc * num_mux_cycle (see Reference Images section)
• number of TRs in nifti = n + num_mux_cycle
• number in the phases per location field: n + mux * n_mux_cycle

For example, in order to have 100 usable TRs from a run with mux=3, arc=2, and num_mux_cycle=2, you would manually enter 106 in the phases per location field, the total scan time would automatically adjust to be 112 TRs, and your experiment should begin 12 TRs after you trigger the scan to begin. The nifti for this run would contain 102 volumes, the first two of which are the reconstructed calibration scans that should be skipped for bold analysis.

Reconstruction

The mux epi recon is started automatically by NIMS at the end of a mux scan. It can take anywhere from a few minutes to an hour per scan, depending on the resolution and number of TRs acquired.

(Describe Kangrong's recon: reference images, virtual coils, GRAPPA vs. SENSE)

Slice acquisition order

The multiband sequences acquire slices in either ascending or descending interleaved order, depending on which direction the slices were prescribed (the nifti header should indicate whether it's ascending or descending-- see the nifti faq for a description of the two orders). And, of course, for mux>1, multiple slices are acquired simultaneously. E.g., for a mux 3 scan with 14 muxed slices (that will result in 14*3=42 unmuxed slices), the acq. order is [1,3,5,7,9,11,13,2,4,6,8,10,12,14]. And mux slice 1 will get reconstructed as unmuxed slices [1,15,29], which are all acquired simultaneously. Likewise, mux slice 2 will get deconvolved into unmuxed slices [2,16,30]. And so on.

Here's an algorithm to compute the relative slice acquisition time within the TR:

   mux = 3
   nslices = 5
   tr = 1.0
   mux_slice_acq_order = range(0,nslices,2) + range(1,nslices,2)
   mux_slice_acq_time = [float(s)/nslices*tr for s in xrange(nslices)]
   unmux_slice_acq_order = [nslices*m+s for m in xrange(mux) for s in mux_slice_acq_order]
   slice_time = {slice_num:slice_time for slice_num,slice_time in zip(unmux_slice_acq_order, mux_slice_acq_time*3)}
   for slice,acq in sorted(slice_time.items()):
       print "    slice %02d acquired at time %.3f sec" % (slice+1,acq)

   slice 01 acquired at time 0.000 sec
   slice 03 acquired at time 0.200 sec
   slice 05 acquired at time 0.400 sec
   slice 02 acquired at time 0.600 sec
   slice 04 acquired at time 0.800 sec
   slice 06 acquired at time 0.000 sec
   slice 08 acquired at time 0.200 sec
   slice 10 acquired at time 0.400 sec
   slice 07 acquired at time 0.600 sec
   slice 09 acquired at time 0.800 sec
   slice 11 acquired at time 0.000 sec
   slice 13 acquired at time 0.200 sec
   slice 15 acquired at time 0.400 sec
   slice 12 acquired at time 0.600 sec
   slice 14 acquired at time 0.800 sec

To generate an FSL slice time file:

   mux = 3
   nslices = 15
   descending = False
   filename = 'slicetime.txt'
   mux_slice_acq_order = range(0,nslices,2) + range(1,nslices,2)
   if descending:
       mux_slice_acq_order = mux_slice_acq_order[::-1]
   unmux_slice_acq_rel_time = [float(s)/nslices - 0.5 for s in xrange(nslices)] * mux
   unmux_slice_acq_order = [nslices*m+s for m in xrange(mux) for s in mux_slice_acq_order]
   slice_time = {slice_num:slice_time for slice_num,slice_time in zip(unmux_slice_acq_order, unmux_slice_acq_rel_time)}
   with open(filename,'w') as fp:
       for slice,acq in sorted(slice_time.items()):
           fp.write("%.4f\n" % acq)

Sharing

We are happy to share our SMS-EPI pulse sequences (and associated reconstruction code) with other GE research sites running a compatible platform (e.g., MR750 and related variants). We have two versions, one for BOLD fMRI and one for diffusion imaging. To request access to the pulse sequence (PSD) binaries and associated image reconstruction software, please send a note to Bob Dougherty (bobd@stanford.edu). You will also need to complete the sharing agreement and verify that you have an active research agreement with GE. Once the paperwork is out of the way, we will share the PSDs and recon code via a private Github repo, so you will need to send us your Github username and we will grant you access to the current code and future updates. The current versions of the PSDs run on both DV24 and DV25.

Please note that this is very much "research grade" software and as such, it is not a polished "turn-key" package. You will need someone with experience installing research PSDs on a GE platform to get the sequences working, and will also need to set a mechanism for collecting the raw data files from the scanner and configuring the off-line reconstruction software on a Linux machine.

Reconstruction Hardware

Our recon should run on any modern linux system. However, it may be pretty slow (hours per fMRI scan), unless you have a beefy Intel CPU (e.g., a 12-core Sandy Bridge will get you down to 30 minutes or so) and/or good GPU cards (a Sandy Bridge CPU with three Nvidia Titan-X cards will get the recon down to a few minutes and allow it to run in real-time). We have spec'ed a linux workstation that will cost under $8k to build and can do real-time recons. All components are available on Amazon and NewEgg:

• ASRock X99 Extreme6 LGA 2011-v3 Intel X99 SATA 6Gb/s USB 3.0 ATX Intel Motherboard
• Intel Xeon E5-2697 v3 Haswell 2.6 GHz 35MB L3 Cache LGA 2011-3 145W BX80644E52697V3 Server Processor (14 core)
• G.SKILL Ripjaws 4 Series 64GB (8 x 8GB) 288-Pin DDR4 SDRAM DDR4 2800 (PC4 22400) Intel X99 Desktop Memory
• Noctua NH-U14S 140x150x25 (NF-A15 PWM) SSO2-Bearing CPU Cooler
• EVGA SuperNOVA 1600 G2 80+ GOLD, 1600W Power Supply
• 3 X EVGA GeForce GTX TITAN X 12GB (superclocked)
• Samsung 850 Pro 1 TB 2.5-Inch SATA III Internal SSD
• 2.5” to 3.5” mounting bracket
• Corsair Obsidian Series 750D Performance Full Tower Case
• keyboard & mouse

Notes:

• The 14-core CPU (currently $2700) could probably be replaced by a 10 or 12 core 2011-v3 CPU to reduce the cost (e.g., Intel Xeon E5-2680 v3 Haswell 2.5 GHz 12 x 256KB L2 Cache 30 MB L3 Cache LGA 2011-3120W BX80644E52680V3 Server Processor). We haven’t tested this, but we suspect such a processor would be adequate for real-time reconstruction based on the spare capacity of the 14-core CPU when running a recon.
• The power supply is also bigger than needed (we got this one to allow for upgrade to a dual-processor motherboard and a fourth GPU card, if needed). Total system power should be well under 1000 Watts at full load, so a 1000W supply would be fine (e.g., EVGA 120-G2-1000-XR 80 PLUS GOLD 1000 W).
• A smaller case could also be used (again, we got this one to allow for an upgrade with an an extended ATX motherboard). Just be sure there is enough room between the motherboard and the power supply to intall the third GPU card.
• Do NOT populate the Ultra M.2 socket on this ASRock motherboard, as doing so disables the third PCIe 3.0 x16 slot. (This is mentioned in the manual, but we didn't read that until after many hours of frustrated debugging!)