9. Revert downsampled, symmetry expanded particles

Re-extracting particles after symmetry expansion is wasteful, since it creates a particle stack N times larger than is necessary, where N is the number of symmetry-related positions. However, during particle cleanup it is often helpful to work with smaller images, since most cleanup operations do not require high resolution information.

This notebook illustrates the process of using a subset of particles (presumably after some junk has been removed) to filter the full-size particle images.

We presume the following jobs have already been run in CryoSPARC:

• an initial consensus refinement

• a symmetry expansion of this consensus refinement. This is the full-size symmetry expansion, in our case Job 31.

• a Downsample job of the consensus refinement

• a symmetry expansion of this Downsample job.

• some number of jobs to curate and filter the downsampled, symmetry-expanded particles. This is the downsampled particle stack, in our case Job 34.

Warning

The results of this notebook will retain all poses, CTF fit, and other information from the full-size symmetry expansion. If the poses or other information from the jobs performed on the downsampled, symmetry-expanded particles are required, they will need to be transferred to the final result.

from cryosparc.tools import CryoSPARC
import numpy as np

cs = CryoSPARC(base_port=40000)
cs.test_connection()

Connection succeeded to CryoSPARC command_core at http://localhost:40002
Connection succeeded to CryoSPARC command_vis at http://localhost:40003
Connection succeeded to CryoSPARC command_rtp at http://localhost:40005

True

First, we load the relevant projects and jobs. Storing the project and job IDs in variables makes later steps easier.

select_project = "P294"
select_workspace = "W2"
job_fullsize_particles = "J31"
job_downsampled_particles = "J34"

ds_particles = cs.find_job(select_project, job_downsampled_particles).load_output("particles")
full_particles = cs.find_job(select_project, job_fullsize_particles).load_output("particles")

Symmetry expansion assigns a new UID to each particle, so we cannot simply use UIDs to filter the fullsize particles. Instead, we must use the combination of two other fields created during symmetry expansion:

• sym_expand/src_uid, which stores the UID of the original, unexpanded particle image from which this symmetry copy was created

• sym_expand/idx, which stores the index for this symmetry copy. For example, in a C2 symmetry expansion, sym_expand/idx alternates between 0 and 1; in a C6 expansion it would range from 0 to 5.

The combination of these two fields uniquely identifies a symmetry expansion of a particle image. We add a field to both the fullsize and downsampled symmetry-expanded particles containing the combination of these fields.

full_particles.add_fields(["intersect_field"], ["str"])
full_particles["intersect_field"] = [
    # this list comprehension creates a combined field, like 123456.0 for a particle
    # with src_uid = 123456 and idx = 0
    f"{r['sym_expand/src_uid']}.{r['sym_expand/idx']}"
    for r in full_particles.rows()
]
ds_particles.add_fields(["intersect_field"], ["str"])
ds_particles["intersect_field"] = [f"{r['sym_expand/src_uid']}.{r['sym_expand/idx']}" for r in ds_particles.rows()]

Next, we keep only the particles with an intersect_field value that is in the downsampled particles as well. This step is a little slow, since it is performing a great number of string comparisons.

intersection = full_particles.query({"intersect_field": ds_particles["intersect_field"]})

Finally, we use save_external_result to create an External Job in our CryoSPARC instance containing only the filtered particles.

Note that we pass the fullsize particles’ poses through. If the poses from the downsampled particles were desired, first the pixel size and shifts would have to be adjusted for the new, fullsize images.

cs.save_external_result(
    select_project,
    select_workspace,
    intersection,
    type="particle",
    name="sym_expand_intersection",
    slots=["blob"],
    passthrough=(job_fullsize_particles, "particles"),
    title="Filtered Subset",
)

'J55'

Checking J55, we see that we have successfully filtered the particles to the desired subset: