Skip to content

suite2p_loader.py

PlaneSuite2p

Parse the suite2p output directory and load data, per plane.

Suite2p output doc: https://suite2p.readthedocs.io/en/latest/outputs.html

Expecting the following files
  • ops: Options file
  • Fneu: Neuropil traces file for functional channel
  • Fneu_chan2: Neuropil traces file for channel 2
  • F: Fluorescence traces for functional channel
  • F_chan2: Fluorescence traces for channel 2
  • iscell: Array of (user curated) cells and probability of being a cell
  • spks: Spikes (raw deconvolved with OASIS package)
  • stat: Various statistics for each cell
  • redcell: "Red cell" (second channel) stats

Attributes:

Name Type Description
alignment_channel

ops["align_by_chan"] as zero-indexed
cell_prob
correlation_map

ops["Vcorr"]
creation_time

earliest file creation time across planes
curation_time

latest curation time across planes
F

Fluorescence traces for functional channel as numpy array if exists If does not exist, returns empty list
F_chan2

Fluorescence traces for channel 2 as numpy array if exists If does not exist, returns empty lists
Fneu

Neuropil traces file for functional channel as numpy array if exists If does not exist, returns empty list
Fneu_chan2

Neuropil traces file for channel 2 as numpy array if exists If does not exist, returns empty list
fpath

path to plane folder
iscell
max_proj_image

ops["max_proj"] if exists. Else np.full_like(mean_image))
mean_image

ops["meanImg"]
ops

Options file as numpy array
plane_idx

plane index. -1 if combined, else number in path
redcell

"Red cell" (second channel) stats as numpy array if exists If does not exist, returns empty list
ref_image

ops["refImg"]
segmentation_channel

ops["functional_chan"] as zero-indexed
spks

Spikes (raw deconvolved with OASIS package) as numpy array if exists If does not exist, returns empty lists
stat

Various statistics for each cell as numpy array if exists If does not exist, returns empty lists
Source code in element_interface/suite2p_loader.py 
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
class PlaneSuite2p:
    """Parse the suite2p output directory and load data, ***per plane***.

    Suite2p output doc: https://suite2p.readthedocs.io/en/latest/outputs.html

    Expecting the following files:
        - ops:        Options file
        - Fneu:       Neuropil traces file for functional channel
        - Fneu_chan2: Neuropil traces file for channel 2
        - F:          Fluorescence traces for functional channel
        - F_chan2:    Fluorescence traces for channel 2
        - iscell:     Array of (user curated) cells and probability of being a cell
        - spks:       Spikes (raw deconvolved with OASIS package)
        - stat:       Various statistics for each cell
        - redcell:    "Red cell" (second channel) stats

    Attributes:
        alignment_channel: ops["align_by_chan"] as zero-indexed
        cell_prob:
        correlation_map: ops["Vcorr"]
        creation_time: earliest file creation time across planes
        curation_time: latest curation time across planes
        F: Fluorescence traces for functional channel as numpy array if exists
            If does not exist, returns empty list
        F_chan2: Fluorescence traces for channel 2 as numpy array if exists
            If does not exist, returns empty lists
        Fneu: Neuropil traces file for functional channel as numpy array if exists
            If does not exist, returns empty list
        Fneu_chan2: Neuropil traces file for channel 2 as numpy array if exists
            If does not exist, returns empty list
        fpath: path to plane folder
        iscell:
        max_proj_image: ops["max_proj"] if exists. Else np.full_like(mean_image))
        mean_image: ops["meanImg"]
        ops: Options file as numpy array
        plane_idx: plane index. -1 if combined, else number in path
        redcell: "Red cell" (second channel) stats as numpy array if exists
            If does not exist, returns empty list
        ref_image: ops["refImg"]
        segmentation_channel: ops["functional_chan"] as zero-indexed
        spks: Spikes (raw deconvolved with OASIS package) as numpy array if exists
            If does not exist, returns empty lists
        stat:  Various statistics for each cell as numpy array if exists
            If does not exist, returns empty lists
    """

    def __init__(self, suite2p_plane_dir: str):
        """Initialize PlaneSuite2p class given a plane directory

        Args:
            suite2p_plane_dir (str): Suite2p plane directory

        Raises:
            FileNotFoundError: No "ops.npy" found. Invalid suite2p plane folder
            FileNotFoundError: No "iscell.npy" found. Invalid suite2p plane folder
        """
        self.fpath = pathlib.Path(suite2p_plane_dir)

        # -- Verify dataset exists --
        ops_fp = self.fpath / "ops.npy"
        if not ops_fp.exists():
            raise FileNotFoundError(
                'No "ops.npy" found. Invalid suite2p plane folder: {}'.format(
                    self.fpath
                )
            )
        self.creation_time = datetime.fromtimestamp(ops_fp.stat().st_ctime)

        iscell_fp = self.fpath / "iscell.npy"
        if not iscell_fp.exists():
            raise FileNotFoundError(
                'No "iscell.npy" found. Invalid suite2p plane folder: {}'.format(
                    self.fpath
                )
            )
        self.curation_time = datetime.fromtimestamp(iscell_fp.stat().st_ctime)

        # -- Initialize attributes --
        for s2p_type in _suite2p_ftypes:
            setattr(self, "_{}".format(s2p_type), None)
        self._cell_prob = None

        self.plane_idx = (
            -1
            if self.fpath.name == "combined"
            else int(self.fpath.name.replace("plane", ""))
        )

    # -- load core files --

    @property
    def ops(self):
        if self._ops is None:
            fp = self.fpath / "ops.npy"
            self._ops = np.load(fp, allow_pickle=True).item()
        return self._ops

    @property
    def Fneu(self):
        if self._Fneu is None:
            fp = self.fpath / "Fneu.npy"
            self._Fneu = np.load(fp) if fp.exists() else []
        return self._Fneu

    @property
    def Fneu_chan2(self):
        if self._Fneu_chan2 is None:
            fp = self.fpath / "Fneu_chan2.npy"
            self._Fneu_chan2 = np.load(fp) if fp.exists() else []
        return self._Fneu_chan2

    @property
    def F(self):
        if self._F is None:
            fp = self.fpath / "F.npy"
            self._F = np.load(fp) if fp.exists() else []
        return self._F

    @property
    def F_chan2(self):
        if self._F_chan2 is None:
            fp = self.fpath / "F_chan2.npy"
            self._F_chan2 = np.load(fp) if fp.exists() else []
        return self._F_chan2

    @property
    def iscell(self):
        if self._iscell is None:
            fp = self.fpath / "iscell.npy"
            d = np.load(fp)
            self._iscell = d[:, 0].astype(bool)
            self._cell_prob = d[:, 1]
        return self._iscell

    @property
    def cell_prob(self):
        if self._cell_prob is None:
            fp = self.fpath / "iscell.npy"
            if fp.exists():
                d = np.load(fp)
                self._iscell = d[:, 0].astype(bool)
                self._cell_prob = d[:, 1]
        return self._cell_prob

    @property
    def spks(self):
        if self._spks is None:
            fp = self.fpath / "spks.npy"
            self._spks = np.load(fp) if fp.exists() else []
        return self._spks

    @property
    def stat(self):
        if self._stat is None:
            fp = self.fpath / "stat.npy"
            self._stat = np.load(fp, allow_pickle=True) if fp.exists() else []
        return self._stat

    @property
    def redcell(self):
        if self._redcell is None:
            fp = self.fpath / "redcell.npy"
            self._redcell = np.load(fp) if fp.exists() else []
        return self._redcell

    # -- image property --

    @property
    def ref_image(self):
        return self.ops["refImg"]

    @property
    def mean_image(self):
        return self.ops["meanImg"]

    @property
    def max_proj_image(self):
        return self.ops.get("max_proj", np.full_like(self.mean_image, np.nan))

    @property
    def correlation_map(self):
        return self.ops["Vcorr"]

    @property
    def alignment_channel(self):
        return self.ops["align_by_chan"] - 1  # suite2p is 1-based, convert to 0-based

    @property
    def segmentation_channel(self):
        return self.ops["functional_chan"] - 1  # suite2p is 1-based, convert to 0-based

__init__(suite2p_plane_dir)

Initialize PlaneSuite2p class given a plane directory

Parameters:

Name Type Description Default
suite2p_plane_dir str

Suite2p plane directory

 required

Raises:

Type Description
FileNotFoundError

No "ops.npy" found. Invalid suite2p plane folder
FileNotFoundError

No "iscell.npy" found. Invalid suite2p plane folder
Source code in element_interface/suite2p_loader.py 
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
def __init__(self, suite2p_plane_dir: str):
    """Initialize PlaneSuite2p class given a plane directory

    Args:
        suite2p_plane_dir (str): Suite2p plane directory

    Raises:
        FileNotFoundError: No "ops.npy" found. Invalid suite2p plane folder
        FileNotFoundError: No "iscell.npy" found. Invalid suite2p plane folder
    """
    self.fpath = pathlib.Path(suite2p_plane_dir)

    # -- Verify dataset exists --
    ops_fp = self.fpath / "ops.npy"
    if not ops_fp.exists():
        raise FileNotFoundError(
            'No "ops.npy" found. Invalid suite2p plane folder: {}'.format(
                self.fpath
            )
        )
    self.creation_time = datetime.fromtimestamp(ops_fp.stat().st_ctime)

    iscell_fp = self.fpath / "iscell.npy"
    if not iscell_fp.exists():
        raise FileNotFoundError(
            'No "iscell.npy" found. Invalid suite2p plane folder: {}'.format(
                self.fpath
            )
        )
    self.curation_time = datetime.fromtimestamp(iscell_fp.stat().st_ctime)

    # -- Initialize attributes --
    for s2p_type in _suite2p_ftypes:
        setattr(self, "_{}".format(s2p_type), None)
    self._cell_prob = None

    self.plane_idx = (
        -1
        if self.fpath.name == "combined"
        else int(self.fpath.name.replace("plane", ""))
    )

Suite2p

Wrapper class containing all suite2p outputs from one suite2p analysis routine.

This wrapper includes outputs from the individual plane, with plane indexing starting from 0 Plane index of -1 indicates a suite2p "combined" outputs from all planes, thus saved in the "planes_combined" attribute. See also PlaneSuite2p class.

Directory example
  • plane0: ops.npy, F.npy, etc.
  • plane1: ops.npy, F.npy, etc.
  • combined: ops.npy, F.npy, etc.
Example

loaded_dataset = suite2p_loader.Suite2p(output_dir)

Source code in element_interface/suite2p_loader.py 
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
class Suite2p:
    """Wrapper class containing all suite2p outputs from one suite2p analysis routine.

    This wrapper includes outputs from the individual plane, with plane indexing
    starting from 0 Plane index of -1 indicates a suite2p "combined" outputs from all
    planes, thus saved in the "planes_combined" attribute. See also PlaneSuite2p class.

    Directory example:
        - plane0: ops.npy, F.npy, etc.
        - plane1: ops.npy, F.npy, etc.
        - combined: ops.npy, F.npy, etc.

    Example:
        > loaded_dataset = suite2p_loader.Suite2p(output_dir)

    """

    def __init__(self, suite2p_dir: str):
        """Initialize Suite2p class

        Args:
            suite2p_dir (str): Suite2p directory

        Raises:
            FileNotFoundError: Could not find Suite2p results
        """
        self.suite2p_dir = pathlib.Path(suite2p_dir)

        ops_filepaths = list(self.suite2p_dir.rglob("*ops.npy"))

        if not len(ops_filepaths):
            raise FileNotFoundError(
                "Suite2p output result files not found at {}".format(suite2p_dir)
            )

        self.planes = {}
        self.planes_combined = None
        for ops_fp in ops_filepaths:
            plane_s2p = PlaneSuite2p(ops_fp.parent)
            if plane_s2p.plane_idx == -1:
                self.planes_combined = plane_s2p
            else:
                self.planes[plane_s2p.plane_idx] = plane_s2p
        self.planes = OrderedDict({k: self.planes[k] for k in sorted(self.planes)})

        self.creation_time = min(
            [p.creation_time for p in self.planes.values()]
        )  # ealiest file creation time
        self.curation_time = max(
            [p.curation_time for p in self.planes.values()]
        )  # most recent curation time

__init__(suite2p_dir)

Initialize Suite2p class

Parameters:

Name Type Description Default
suite2p_dir str

Suite2p directory

 required

Raises:

Type Description
FileNotFoundError

Could not find Suite2p results
Source code in element_interface/suite2p_loader.py 
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
def __init__(self, suite2p_dir: str):
    """Initialize Suite2p class

    Args:
        suite2p_dir (str): Suite2p directory

    Raises:
        FileNotFoundError: Could not find Suite2p results
    """
    self.suite2p_dir = pathlib.Path(suite2p_dir)

    ops_filepaths = list(self.suite2p_dir.rglob("*ops.npy"))

    if not len(ops_filepaths):
        raise FileNotFoundError(
            "Suite2p output result files not found at {}".format(suite2p_dir)
        )

    self.planes = {}
    self.planes_combined = None
    for ops_fp in ops_filepaths:
        plane_s2p = PlaneSuite2p(ops_fp.parent)
        if plane_s2p.plane_idx == -1:
            self.planes_combined = plane_s2p
        else:
            self.planes[plane_s2p.plane_idx] = plane_s2p
    self.planes = OrderedDict({k: self.planes[k] for k in sorted(self.planes)})

    self.creation_time = min(
        [p.creation_time for p in self.planes.values()]
    )  # ealiest file creation time
    self.curation_time = max(
        [p.curation_time for p in self.planes.values()]
    )  # most recent curation time