...
It is very difficult for me to see a correlation. It is especially unfortunate that only the first two months of data had these numbers recorded since most of those images are in focus anyway.
Update 4/2/15
Beginning analysis of entire set
I let the code run continuously for a few days, to see how many of the files it could analyze and to see if any errors need to be fixed in the code.
For each day of data (~1000 images), it analyzes every image and looks for a complete spectrum of Polaris. If a full spectrum is found, it goes through all the steps previously described to extract the equivalent widths of the six features (Ox A, Ox B, and 4 water features). For each day, it writes a text file with each equivalent width. It then calculates the average equivalent width for each feature over the entire day, along with a purely statistical uncertainty, and keeps that info in a separate text file.
Below I plot the average equivalent widths of the six features over a ~three month period (took three days of computation on my Mac).
The two plots are over the same time period, I simply divided the six lines into two plots for clarity (Ox A is in both for reference).
I think a promising sign from these plots is the correlation in the water lines. While the oxygen lines stay constant over time, not only do the four H2O lines vary significantly over time, but they generally vary in the same way. This makes me think the code is doing a good job at extracting the equivalent widths without too much noise.
A few tweaks to do before applying the code to the entire set:
- Only ~20% of the images have a complete, "good" spectrum with all features in the CCD. Is there a way to increase this number? I.e., can we make the "chopped" or cut-off spectra described in previous entries useful?
- Decrease noise by improving how the algorithm calculates dλ/dpixel. It currently calculates dλ/dpixel by finding Ox A and Ox B in pixel space, and using their known wavelengths. I have confidence that the code finds Ox A very well, but about ~10% of the time it fails to find Ox B. Matt gave a few suggestions for improving how to find Ox B (e.g. cross correlation with typical "good" spectra) that I can quickly implement to improve efficiency.
- Compare three different methods for finding equivalent width of a certain line:
- My current method (using a spline as the continuum; probably the most accurate but computationally intensive)
- fit a sloped line using the spectrum just before and just after the feature, and use that line as the continuum
- fit a flat line, i.e. same as above except simply use median of neighboring points. Computationally quick but probably less accurate since some water features are at high slope.