Add better support for correcting for shift of diffraction pattern as function of probe position

[magnunor]

Background

A common artifact in scanning electron diffraction is dscan, where the center position of the diffraction position shifts as a function of scan position. This effect is typically a slow and monotonic "shift" of the center position, in (most) cases being close to linear. There are ways of removing this effect in pyxem, for example via Diffraction2D.center_direct_beam. It works by finding the direct beam position in each scan position, and moving this to the center of the signal dimensions. This works for a large number of samples, where the direct beam is clearly visible, and fairly homogeneous.

However, for parts of some samples, the material can be so thick or heavily diffracting, that the center position of the direct beam is hard to resolve. In that situation, we can use some more ideal reference region to find the amount of dscan. Next, a linear plane can be fitted to these regions to find the dscan in the thick/heavily diffracting regions.

---

Changes

This pull request implements a new class, BeamShift, to handle exactly this. It inherits Signal1D, and has the beam position in the signal dimension, and the scan positions in the navigation dimensions.

• get_direct_beam_position now return this new class
• The class has the make_linear_plane method which fits a linear plane to the x and y beam positions, and replaces the original data with this plane. This function has a mask parameter, which is used to mask any "bad" regions.
  • This can then be passed as center_direct_beam(shifts=s_beam_shift)
• The class also has a to_dpcsignal(), which transposes the BeamShift class, and returns a DPCSignal object (which is more focused on analysing the beam shift itself, for example magnetic or ferroelectric materials).

In addition, the pull request changes the _fit_ramp_to_image to make it more general, to facilitate the aforementioned processing.

---

Todo list

• Is BeamShift a good class name?
• Finish implementation
• Change and add new unit tests
• Add this functionality to a notebook

---

Comments

• This type of processing might be tricky for lazy processing, as the scan positions are no longer independent from each other. Thus, currently make_linear_plane does not have a lazy version. In practice, the easiest way of doing this is by first running s_beam_shift.compute().

The functionality can also be useful for getting potentially better direct beam centering, since currently inhomogeneities in the direct beam can lead to erroneous shifting of the direct beam. For example in the image, where many of the variations are due to grain boundaries:

By replacing this "raw" direct beam position with a fitted linear plane, the effect from the grain boundaries will be diminished.

---

Future work

• For larger scan areas, a linear plane isn't enough to model the dscan. For this, a vacuum scan might be necessary. There should be some functionality for handling this.

[pc494]

I'm happy with the design of this. I agree that I would expect a user to be able to compute a BeamShift object (and so its methods don't need to work lazily)

[coveralls]

Coverage increased (+0.01%) to 100.0% when pulling e5f25d1 on magnunor:beam_shift_signal into 42134d8 on pyxem:master.

[magnunor]

This should be good to go! Not sure how best to add this to a notebook. In theory having it as a part of a bigger one is nice, but I could make a new one showing some of the processing "pipeline" I've been working on.

[pc494]

Great, thanks @magnunor, hoping to get this reviewed by the end of the week. In terms of the notebooks, I think it fits nicely in one of the subsections of demo 10: "correcting d-scan"

[pc494]

This is basically good to go, but it needs docstrings (for make_linear_plane and the to/from dpc/beam_shift) functions and a CHANGELOG update.

[dnjohnstone]

Correcting for a systematic shift in the direct beam position in the diffraction pattern plane is a good feature, thanks!

But please can we tighten up the language here before this is merged, which I think is scientifically/technically dubious right now.

I don't really know what "A common artifact in scanning electron diffraction is dscan" means, partly because I've not seen "dscan" used like that (can you provide a reference for that usage?) - to me, the artifact is "a shift in the direct beam position in the diffraction plane resulting from the electron beam straying from the optical axis across a scanned region". I.e. a shift in the direct beam position resulting from imperfect electron-optical alignment rather than any property of the sample.

So I think what you're trying to do is to apply a digital correction for this case which could be considered as the case when the physical "de-scan" correction (https://www.jeol.co.jp/en/words/emterms/search_result.html?keyword=de-scan) is imperfect.

In other words, it would make more sense to me to use a phrase like "apply a digital de-scan correction" than to say "correct for a dscan artefact". Can you provide a reference justifying that this should be a linear correction on electron-optical grounds? Or should we be making clear that this is an approximation?

Am I making sense or am I being silly?

[magnunor]

This is basically good to go, but it needs docstrings (for make_linear_plane and the to/from dpc/beam_shift) functions and a CHANGELOG update.

Indeed! I'll fix that.

But please can we tighten up the language here before this is merged, which I think is scientifically/technically dubious right now.

Yeah, I agree. I wrote this perhaps a bit too colloquially, I'll make the language a bit more precise.

I don't really know what "A common artifact in scanning electron diffraction is dscan" means, partly because I've not seen "dscan" used like that (can you provide a reference for that usage?) - to me, the artifact is "a shift in the direct beam position in the diffraction plane resulting from the electron beam straying from the optical axis across a scanned region". I.e. a shift in the direct beam position resulting from imperfect electron-optical alignment rather than any property of the sample.

So I think what you're trying to do is to apply a digital correction for this case which could be considered as the case when the physical "de-scan" correction (https://www.jeol.co.jp/en/words/emterms/search_result.html?keyword=de-scan) is imperfect.

In other words, it would make more sense to me to use a phrase like "apply a digital de-scan correction" than to say "correct for a dscan artefact". Can you provide a reference justifying that this should be a linear correction on electron-optical grounds? Or should we be making clear that this is an approximation?

This type of "artefact" is very common and noticeable when doing STEM-DPC with a fast pixelated electron detector to image for example magnetic fields. It is (as you write here), an effect of the centre of the diffraction pattern changing as a function of probe position is a STEM scan. As far as I know, it is inevitable when scanning over large regions in the STEM, and it gets worse the bigger region you're scanning. Thus, it is not noticeable of you're scanning 10 x 10 nanometers, but becomes much more noticable when scanning 10 x 10 micrometers. The reason I called it an artefact, is that it is kind of an artefact when working with STEM-DPC data, as the "de-scan" will look like a magnetic or electric field. This is seen in the images here, which shows the beam shift from a magnetic STEM-DPC dataset. The ferromagnetic domains are "on top of" the slowly varying beam shift caused by the scanning system.

As far as I know, the only way to compensate for it is having dedicated "de-scan" correction coils below the sample. However, a large number of microscopes does not have this. For example 5+ year old Jeol microscopes. In addition, for sufficiently large scan regions, even the de-scan correction coils can't remove all of this, thus requiring some type of correction in the post processing.

For medium sized regions (100-500 nm), a linear plane seems to work fine for most cases. For larger regions (1 um and upwards), quadratic or higher order planes are sometimes necessary. At some point, the best solution is to acquire a vacuum dataset, and use that as the correction.

I'll improve up the docstring texts a bit, to make them more precise.

[magnunor]

I added a (fairly) extensive docstring for make_linear_plane, the to_... and did some minor changes to the correct_ramp.

@dnjohnstone, do you think the make_linear_plane docstring text works better now?