Calibrators#

The main focus of the euv_fitting package is finding the calibration, or the relationship between channel number and wavelength.

Calibration Process#

Each calibration line will have a certain channel position (given by the peak fitting) and a wavelength. If we make a graph with channel position on the x-axis and wavelength on the y-axis, a line at channel number 1000 and an 8.8 nm wavelength looks like this:

_images/calibration_0_0.png

The channel uncertainty is calculated in the peak fitting process, while the wavelength comes from the literature source for that line. For a more detailed description of how uncertainties and outliers are handled, see the uncertainties page.

If multiple calibration lines have been identified, that gives us many (x, y) points we can use to estimate the calibration curve. The Calibrator class stores identified calibration lines in Calibrator.solution_arr. .solution_arr has the shape (N x 4), where N is the number of calibration lines used. Each row has the form [wavelength, wavelength_uncertainty, channel, channel_uncertainty].

As an example, consider these neon and background spectra which were taken at roughly the same time:

from euv_fitting.calibrate.utils import SpeReader, CosmicRayFilter
from euv_fitting.calibrate.calibrators import Distance_Calibrator

CRF = CosmicRayFilter()

Ne_spe = SpeReader('./example_data/Example_Neon.SPE')
Bkg_spe = SpeReader('./example_data/Example_Background.SPE')

Ne_img = CRF.apply(Ne_spe.load_img())
Bkg_img = CRF.apply(Bkg_spe.load_img())

plt.plot(Ne_img, label = 'Ne')
plt.plot(Bkg_img, label = 'Bkg')
plt.xlabel('Channel')
plt.legend()
plt.show()
_images/calibration_1_0.png

To identify the lines in these spectra, we create Distance_Calibrator objects and call their .calibrate method.

Ne_cal = Distance_Calibrator(Ne_img, 'Ne', num_peaks = 25) #specify element and #peaks
Bkg_cal = Distance_Calibrator(Bkg_img, 'Ba', num_peaks = 25) #specify element and #peaks

Ne_cal.calibrate()
Bkg_cal.calibrate()

print('Neon Solution Array')
print(Ne_cal.solution_arr)

print('Background Solution Array')
print(Bkg_cal.solution_arr)

selecting best peaks
selecting best peaks

Neon Solution Array
[[8.80929000e+00 1.40000000e-04 7.62421231e+02 2.58781802e-02]
 [9.75020000e+00 4.00000000e-04 8.90567351e+02 3.55102613e-02]
 [1.11136000e+01 1.80000000e-03 1.06748262e+03 3.36749072e-02]
 [1.16691000e+01 5.00000000e-04 1.13674959e+03 2.73318275e-02]
 [1.27676000e+01 7.00000000e-04 1.26986371e+03 1.56628562e-02]
 [1.43314000e+01 7.00000000e-04 1.45125753e+03 2.30675110e-02]
 [1.47138000e+01 7.00000000e-04 1.49425996e+03 4.19253220e-02]
 [1.76186000e+01 2.80000000e-04 1.80726417e+03 2.92112055e-02]
 [1.95004000e+01 8.00000000e-04 1.99888699e+03 2.73615248e-02]]
Background Solution Array
[[5.21540000e+00 2.50000000e-03 2.09996430e+02 6.03119420e-02]
 [5.67700000e+00 1.00000000e-03 2.88094113e+02 4.50670806e-02]
 [5.98460000e+00 2.00000000e-04 3.38510633e+02 7.24005140e-02]
 [6.16220000e+00 2.50000000e-03 3.67006855e+02 9.25922924e-02]
 [6.28800000e+00 3.00000000e-03 3.88270099e+02 5.29248232e-02]
 [6.66230000e+00 7.00000000e-04 4.46800576e+02 3.19296907e-02]
 [1.17686000e+01 1.00000000e-03 1.14876942e+03 3.39887742e-02]
 [1.23920000e+01 3.00000000e-03 1.22513586e+03 2.31265415e-02]
 [1.25818000e+01 5.00000000e-03 1.24772767e+03 4.11585546e-02]
 [1.29930000e+01 3.00000000e-03 1.29716876e+03 3.89691709e-02]
 [1.33246000e+01 1.40000000e-03 1.33537973e+03 5.23796173e-02]
 [1.50101000e+01 5.00000000e-04 1.52770709e+03 3.52752553e-02]
 [1.72169000e+01 3.00000000e-04 1.76518033e+03 8.77637881e-02]
 [1.73081000e+01 5.00000000e-04 1.77455803e+03 1.19250428e-01]]

If we were to use each of these files individually, we would only get part of the story; In general, Neon files have more calibration lines in the high wavelength region, while Background files fill out the lower wavelengths. To take advantage of all information, we can join these two calibrators together:

Ne_cal.join([Bkg_cal]) #Could add more calibrators if available to this list.

print('Total Solution Array')
print(Ne_cal.solution_arr)

Total Solution Array
[[8.80929000e+00 1.40000000e-04 7.62421231e+02 2.58781802e-02]
 [9.75020000e+00 4.00000000e-04 8.90567351e+02 3.55102613e-02]
 [1.11136000e+01 1.80000000e-03 1.06748262e+03 3.36749072e-02]
 [1.16691000e+01 5.00000000e-04 1.13674959e+03 2.73318275e-02]
 [1.27676000e+01 7.00000000e-04 1.26986371e+03 1.56628562e-02]
 [1.43314000e+01 7.00000000e-04 1.45125753e+03 2.30675110e-02]
 [1.47138000e+01 7.00000000e-04 1.49425996e+03 4.19253220e-02]
 [1.76186000e+01 2.80000000e-04 1.80726417e+03 2.92112055e-02]
 [1.95004000e+01 8.00000000e-04 1.99888699e+03 2.73615248e-02]
 [5.21540000e+00 2.50000000e-03 2.09996430e+02 6.03119420e-02]
 [5.67700000e+00 1.00000000e-03 2.88094113e+02 4.50670806e-02]
 [5.98460000e+00 2.00000000e-04 3.38510633e+02 7.24005140e-02]
 [6.16220000e+00 2.50000000e-03 3.67006855e+02 9.25922924e-02]
 [6.28800000e+00 3.00000000e-03 3.88270099e+02 5.29248232e-02]
 [6.66230000e+00 7.00000000e-04 4.46800576e+02 3.19296907e-02]
 [1.17686000e+01 1.00000000e-03 1.14876942e+03 3.39887742e-02]
 [1.23920000e+01 3.00000000e-03 1.22513586e+03 2.31265415e-02]
 [1.25818000e+01 5.00000000e-03 1.24772767e+03 4.11585546e-02]
 [1.29930000e+01 3.00000000e-03 1.29716876e+03 3.89691709e-02]
 [1.33246000e+01 1.40000000e-03 1.33537973e+03 5.23796173e-02]
 [1.50101000e+01 5.00000000e-04 1.52770709e+03 3.52752553e-02]
 [1.72169000e+01 3.00000000e-04 1.76518033e+03 8.77637881e-02]
 [1.73081000e+01 5.00000000e-04 1.77455803e+03 1.19250428e-01]]

The Calibrator class also handles calculating the calibration from that array. The final step is to call the .fit() method and print out information about how the fit went.

Ne_cal.fit()
Ne_cal.plot()
Ne_cal.residual_plot()
Ne_cal.print_info()
_images/calibration_4_0.png _images/calibration_4_1.png 
K0 = 4.051760979290335±0.003037473135061028
K1 = 0.005274290727274932±1.1718352123140706e-05
K2 = 1.2887169025469727e-06±1.2352525685988656e-08
K3 = -2.9330230066183045e-11±3.842164006338311e-12
Highest/Average channel uncertainty 1.14e-03 / 3.63e-04
Highest/Average wavelength uncertainty 5.00e-03 / 1.36e-03
Systematic Uncertainty: 0.0010382812499999975
Unadjusted Chi-square: 2.41881744438093

Calibrator#

Includes general calibration functionality such as uncertainty analysis, intensity calibration, printing, and plotting.

class Calibrator(arr)#

Parent class of DistanceCalibrator and DFSCalibrator. Includes uncertainty analysis, plotting, and outlier detection for final analysis.

If initializing from array, the calibrator class by itself expects the format: [[y, y_uncertainty, x, x_uncertainty], [y, y_uncertainty, x, x_uncertainty], [y, y_uncertainty, x, x_uncertainty]]

confidence_bands(n_sigma=2, func=None)#

Finds confidence bands at each wavelength and stores it in self.cb

Inputs: n_sigma: float, number of sigma for bands func: calibration function

eval_chi_square(i, func, target=1, verbose=False)#

Finds distance of chi-squared from target given systematic uncertainty i

Parameters:

i: float

Systematic uncertainty

func: callable

Calibration function, takes in x and returns y. Must have the signature:

func(x, *args).
target: float

goal value of chi-squared

verbose: boolean

If true, print out additional information during the fitting process.

fit(func=None, outlier_sigma=3, iterations=0, const_uncertainty=15, verbose=False)#

Main call of the calibrator class. Computes the calibration coefficients and covariance.

This function should only be called after the .calibrate method. It relies on data stored on self.solution_arr and self.order_data to be present. It completes automatic outlier removal, can penalize first and second order line differences, and calculates confidence bands of the resulting fit.

After this call, the following attributes are created:

self.popt - coefficients of the calibration function.

self.pcov - covariance matrix of the fit.

Parameters:

  • func (callable) –

    Calibration function, takes in x and returns y. Must have the signature:

    func(x, *args).

  • outlier_sigma ({float, None}) – Number of sigmas away a point can be before being removed. Lower values indicate more aggressive outlier removal. See the remove_outliers function for more details. If None, no outlier removal is completed.

  • iterations (int) – Number of iterations of intensity calibration to complete. This uses the calibration function from the previous function to weight peak fitting of the next iteration.

  • const_uncertainty (float) – Constant uncertainty in adu to add to all channels. Estimate for the electronic noise present in the CCD camera.

  • verbose (boolean) – If true, print out additional information during the fitting process.

fit_iter(func=None, outlier_sigma=3, verbose=False)#

Fits current solution array. Handles outlier removal and function fitting.

Parameters:

  • func (callable) –

    Calibration function, takes in x and returns y. Must have the signature:

    func(x, *args).

  • outlier_sigma ({float, None}) – Number of sigmas away a point can be before being removed. Lower values indicate more aggressive outlier removal. See the remove_outliers function for more details. If None, no outlier removal is completed.

  • verbose (boolean) – If true, print out additional information during the fitting process.

plot(func=None, show=True, ax=None)#

plots calibration function with x on x-axis and y on y-axis

remove_outliers(func=None, outlier_sigma=3, verbose=False)#

Removes outliers from solution_arr and order_data using Leave One Out.

This function removes one point from the calibration, completes the calibration process without that point, and tests to see if the removed point is now outlier_sigma away. If multiple points are outliers, the worst point is removed and the process begins again.

The number of sigmas away the removed point is is computed by:

n_sigmas_away = removed_line_residual / (removed_line_uncertainty ** 2 + calibration_uncertainty ** 2) ** (1/2)

Parameters:

  • func (callable) –

    Calibration function, takes in x and returns y. Must have the signature:

    func(x, *args).

  • outlier_sigma ({float, None}) – Number of sigmas away a point can be before being removed. Lower values indicate more aggressive outlier removal. If None, no outlier removal is completed.

  • verbose (boolean) – If true, print out additional information during the fitting process.

residual_plot(func=None, n_sigma=2, title='Residual Plot', show=True, ax=None, linestyle='-b')#

plots residuals, uncertainties, and confidence bands at n_sigma

solve_chi_square(func, target=1, x0=0.002, verbose=False)#

Finds value of systematic uncertainty that gets the chi-squared as close to target as possible.

If the chi square is already less than target, returns 0.

Parameters:

  • func (callable) –

    Calibration function, takes in x and returns y. Must have the signature:

    func(x, *args).

  • target (float) – Goal value of chi_squared, typically 1.

  • x0 (Float) – Initial guess of systematic uncertainty.

  • verbose (boolean) – If true, print out additional information during the fitting process.

uncertainty_fit(popt, pcov, func, verbose=False)#

Finds the best fit parameters given the uncertainty in x, y, and systematics.

Parameters:

  • popt (array-like) – Calibration coefficients only considering y-uncertainty. Must have length N, where N is the number of calibration parameters.

  • pcov (array-like) – Calibration matrix only considering y-uncertainty. Must have shape N x N, where N is the number of calibration parameters.

  • func (callable) –

    Calibration function, takes in x and returns y. Must have the signature:

    func(x, *args).

  • verbose (boolean) – If true, print out additional information during the fitting process.

Returns:

  • popt (array-like) – Array of the best calibration parameters given the uncertainty in x, y, and systematics.

  • pcov (array-like) – Array of the covariance matrix given the uncertainty in x, y, and systematics.

Distance_Calibrator#

Currently, the only supported method of line identification is Distance Calibration, which analyzes the relative distance between peaks to identify well-known calibration lines. In any given spectra, the channel distance between peaks stays relatively constant. Arrays of well known peaks for Ne, Bkg, and Ar files are stored in distance files.

For example the distance file for neon looks like:

import numpy as np
ne_dis = np.loadtxt('../euv_fitting/calibrate/dis_files/NeO_distances.csv', delimiter = ',')

print(ne_dis)

[[ 7.57640000e+00  4.00000000e-04 -1.76901644e+02]
 [ 8.80929000e+00  1.40000000e-04  0.00000000e+00]
 [ 9.75020000e+00  4.00000000e-04  1.27844000e+02]
 [ 1.11136000e+01  1.80000000e-03  3.04301000e+02]
 [ 1.16691000e+01  5.00000000e-04  3.73434000e+02]
 [ 1.27676000e+01  7.00000000e-04  5.06394000e+02]
 [ 1.43314000e+01  7.00000000e-04  6.87520000e+02]
 [ 1.47138000e+01  7.00000000e-04  7.30589000e+02]
 [ 1.50101000e+01  5.00000000e-04  7.63667000e+02]
 [ 1.76186000e+01  2.80000000e-04  1.04329000e+03]
 [ 1.95004000e+01  8.00000000e-04  1.23294000e+03]
 [ 2.33382000e+01  1.00000000e-03  1.60221000e+03]
 [ 2.45404000e+01  3.40000000e-04  1.71233000e+03]
 [ 2.55352000e+01  1.40000000e-03  1.80123000e+03]]

Each row represents a different calibration line. The first and second column are the wavelength and wavelength uncertainty of the line. The last column is the channel distance from a given line to the reference line. For neon, the reference line is 8.80929 nm, as indicated by the 0 in the last column (the distance from the reference line to itself has to be 0). Distance_Calibrator.calibrate() tries each peak in a spectra as the reference line, and looks for other peaks at the channel distances given in the calibration file.

class Distance_Calibrator(arr, element='Ne', xs=None, num_peaks=30, timeout=None, hdf_dataset=None, gauss_locs=[], function='gaussian', num_poly=2)#

Current (as of 7/1/2022) preferred calibrator. Takes in 1d array of intensity values, fits it with a sum-of-gaussians (multipeakgaussian), and identifies features based on their relative positions.

calibrate(tolerance=2, outfile=None, return_dict=None)#

Tries each peak as the reference line and chooses the best match.

Parameters:

  • tolerance (float) – Maximum channel distance that can still yield a match.

  • outfile (string) – File location to store data.

  • return_dict (dict) – If given, will store calibration data in [‘data’] key of the dictionary.

Returns:

solution_arr – Identified lines and their channel positions. Each row in the array has form [wavelength, wavelength_uncertainty, channel, channel_uncertainty]. This array is also stored in self.solution_arr for later use.

Return type:

array-like

improve_calibration(ref_ch, tolerance=5, func=None)#

Checks to see if, given the lines that have been identified so far, any lines currently lie on the calibration curve. If so, it adds them to the detected lines.