Day 1: 12/06/22:
...
Day 1: 12/06/22:
- Transferred the DIMSUM code to my external hard drive. Any Mac Mini that was setup according to the Mac Mini Setup in this wiki will be able to use it. In the device open the terminal and run the command:
cd /Volumes/Ali Media/stubbs_code . After this, you can execute the desired script. The images will be saved in the directory stubbs_code which is in the external disk. Failed to mount the camera. The clamp was not strong enough to hold the camera on its own.
...
Summary of the current situation after three days:
The logistical situation:
- I have only 6 days left to get this done.
- Hopefully I will be able to stay a few night next week and work on the project 17 hours rather than only 7 hours.
What is working or has worked before:
- We found ways to mount the camera and pegboard without a tripod which would possibly disturb movement in the AuxTel Dome
- We found ways to make the pegboard and camera face each other, above ground. It needs further stabilization.
- We can take pictures of the flash sources and relatively make instant analysis with highly abstracted code.
- The extension chords and the lengths of the connections (both optical fibers and the length of power chord of the Xenon lamp) makes it possible to wires to have an indirect path, such as under the gratings, and still get to their destination.
- It is not interfering with the dome illumination system that Patrick and Parker are installing!
- The data can be taken directly to an external disk without filling the memory of the computer (we need another external disk other than personal one).
What has to be fixed:
- Firstly, the most importantly, we need to be able to take pictures of the flashlights again.
- We need to stabilize the camera.
- Figure out the focusing issue and make sure to get centeroidable sources.
- Optionally add another type of support to pegboard.
Plan forward:
...
- Tie the optical fibers to the rail to address the trip hazard.
- Run the cables under the grating.
...
- Either increase the value of the ND filters.
- Put SM1 tubes to the sources and wiling to be sacrifice the number of sources.
...
Day 4: 12/09/22:
...
We tested how much do the flashes from the circuit affect the observations in terms of background light. This was to determine whether we would need a server that communicates with the telescope. Below is a table of the runs we made. We took flats with and without the flashes on and compared the ADU. Auxiliary Telescope (AT) elevation of 20 degrees is the minimum it would get, that is also the closest height it would be to the flashing lights. AT azimuth of 215 is the most direct pointing of AT mirror to the flash light. The reason we took many trials was simply a failure to get ideal conditions such as screens being turned on.
...
Overall results of the daytime testing were:
...
- Fix the pegboard orientation.
12/12/22: Day 5:
- Wrote a script in the weekend to transfer initially internally written files to an external disk while keep taking data. It takes four arguments, path/to/source (where the files are internally saved), path/to/target (the directory of the external disk), file type, number of files to transfer after. That would let us write the files into the internal disk and transfer them over while keep taking more data and never run out of local memory. The video below is a demonstration with a threshold of 50 files. I wrote a dummy bash script to create files with increasing numbers and wrote it to a directory:
View file name IMG_2319.MOV height 250 - There was a mild earthquake during the weekend. Nothing significant happened to the setup.
- We further stabilized the pegboard. It still moves but is not completely like a sail.
- Added additional sources by adding 1 to 4 optical fibers to the ends of 1 to 7 optical fibers. Some of the 1 to 4 optical fibers did not work anymore.
- We technically should be able to get 18 sources. However, while some sources after being divided the 4 are sufficiently bright, others could barely be seen. I do not currently know the reason behind this problem.
- Tomorrow will be the observing night of DIMSUM, Elana and I are dealing with ssh issues. I am trying to get access to the actual network rather than a guest.
- Currently sources are not perfect Gaussians, some of them are decent. It may be a plane of focus issue.
- If the issues are not fixed, I will only work with clearly seen sources, Get a mock DIMSUM run and hope for the best for a data that would be good for a paper.
12/13/22: Day 6:
- Today was the final day before the observations.
- Make the pegboard more stable by mounting onto ground from additional places.
- Ran the test before, everything was successful.
- The data was noisy, the night should be better.
The night of observing:
- The sources and trials are documented in DIMSUM_observing_analysis.ipynb
- The sources were out of focus and not all sources were identifed.
- Tried to fix it as best I as I could. Seem to be having no doughnuts but sources are worse than the trial.
- I will still try to run the analysis and see if there is anything significant. Source extractor does a pretty good job of identifying bad sources.
Notes from the observations themselves:
...
Flat file trials:
Took flat files with dome closed to see the background light. Here is the summary table for the night of 12/13/22:
...
We were only able to check the count of the first two images, they were both approximately 4. The absolute difference was negligible, Tiago suggested dividing the flash pictures by non flash pictures.
Images From Different Times of Focusing from the night of 13/12/22:
- The different centeroids will be available in the final jupyter notebook.
- The fixed focus before the trials start:
That was the best I got at the limited time. It still gave decent sources. - Focus when I checked to see what went wrong around midnight:
At that point, it looked good which was shocking but it may have been slightly tilted. I readjusted the focus when I went up just in case. - The final state of the pegboard after readjustment:
I am aware I should have documented the final focus as well. I unfortunately did not. It had looked okay and I wanted to get out of the dome to not have any background light while the observations were running. - A possible camera issue: The camera names files from 0 9999. After it reaches 9999 it goes back to 0. If we get more than 10000 photos in a run, that may be a problem because the files with the same name will be in the same directory. I can try to write a python script to add some sort of an additional extension or time stamp to the names of the file.
Observation notes after and during analysis:
- With the excitement of last night, I had confused the meaning of the location of an object in terms of rows and columns to x and y.
- This means we have the camera shaking up and down, the row values are oscillating, but the columns are decreasing. Which means the image is moving left because the camera is rotating more right.
- Remark, these are unofficial notes while making the analysis. I am documenting the entire thought process as Chris Stubbs did in strobed analysis page.
Jupyter notebook for the night of observation:
| View file | ||||
|---|---|---|---|---|
|
12/14/22: Day 7:
...
- Take the time stamps of all the files, reorganize the files in a way that we only use good files.
- Make it very clear in jupyter notebook what I did.
- Back all the data up into the cluster
- Run the analysis.
- If additional time, work to make DIMSUM sturdier.
...
three days:
The logistical situation:
- I have only 6 days left to get this done.
- Hopefully I will be able to stay a few night next week and work on the project 17 hours rather than only 7 hours.
What is working or has worked before:
- We found ways to mount the camera and pegboard without a tripod which would possibly disturb movement in the AuxTel Dome
- We found ways to make the pegboard and camera face each other, above ground. It needs further stabilization.
- We can take pictures of the flash sources and relatively make instant analysis with highly abstracted code.
- The extension chords and the lengths of the connections (both optical fibers and the length of power chord of the Xenon lamp) makes it possible to wires to have an indirect path, such as under the gratings, and still get to their destination.
- It is not interfering with the dome illumination system that Patrick and Parker are installing!
- The data can be taken directly to an external disk without filling the memory of the computer (we need another external disk other than personal one).
What has to be fixed:
- Firstly, the most importantly, we need to be able to take pictures of the flashlights again.
- We need to stabilize the camera.
- Figure out the focusing issue and make sure to get centeroidable sources.
- Optionally add another type of support to pegboard.
Plan forward:
- Fix the emergency fixes.
- Add all the possible sources to the pegboard. The more the merrier.
- Find out an ND filter combination that accounts for the divided light that have different brightness. The different brightness should not matter because we expect the shape to be the same as long as they are visible and not saturated.
- Make sure we get a distribution of sources that makes the DIM value with respect to distance saturate so that we can correlate with fwhm of the observations when the time comes.
- At this point it should be ready for further tests and data collection
- Test whether the light from the sources affect the telescope dome.
- If not:
- Tie the optical fibers to the rail to address the trip hazard.
- Run the cables under the grating.
- If it is, try not to make it affect:
- Either increase the value of the ND filters.
- Put SM1 tubes to the sources and wiling to be sacrifice the number of sources.
- If nothing works, do the server connection Tiago asked.
- If not:
- Once the test of dome interference is done, make a night test taking actual data and try to correlate the saturated differential image motion data with the fwhm.
- Do not forget to enjoy the stars and the experience the Magellanic Clouds.
Day 4: 12/09/22:
- The sources from the day before with ghosting effects before the camera and flashes get out of phase, for some reason camera did not take pictures of the flashes:
I only was able to take a screenshot of the 4 sources but they give the essential idea. These images should not be considered other than observing that sources were initially not good. The left panels is the initial trial and right one is after a focus change. These trials made us realize that we need to fix the camera movement issue. - We tried to figure out why the photos were out of sync with the sources. We could not figure it out why it had worked before and stopped working. Then we just decided to average out any latency by just making the exposure 1/200 seconds rather than 1/2000. That made it work.
- We remounted the camera and made it more sturdy. We discovered we cannot fully get rid of the azimuthal rotation since that is required to unscrew the camera from its mount. If the mount would not let it do that, then it would really be wrong design.
- We reoriented the camera and its mount in a way that camera can be tightened, loosened, and moved much easier compared to previous style. Also, now the camera is sturdy enough to hold itself up by only tightening the big screw on its side. The set screw only gives additional security. In the images below it can be seen that the set screw is easily reachable.
- I had designed the system in a way that the images are written directly into the external disk. That made the external disk and the program crash eventually. So now the images are stored in the internal disk and then transferred when the data collection is done. An hour of data collection approximately takes 60GB so if there is nothing else on the computer a 4 hour run will be possible before the computer runs out of memory. If needed, we may write a python script to automate the process. We also bought a solid state disk to avoid disk crashes.
We tested how much do the flashes from the circuit affect the observations in terms of background light. This was to determine whether we would need a server that communicates with the telescope. Below is a table of the runs we made. We took flats with and without the flashes on and compared the ADU. Auxiliary Telescope (AT) elevation of 20 degrees is the minimum it would get, that is also the closest height it would be to the flashing lights. AT azimuth of 215 is the most direct pointing of AT mirror to the flash light. The reason we took many trials was simply a failure to get ideal conditions such as screens being turned on.
Exposure ID Exposure Time (s) filter AT azimuth AT elevation flash (y/n) ND filter computer screen status 10 60 none 270 20 n 0 all on 11 60 none 270 20 y 0 all on 12 60 none 270 20 n 1 all on 13 60 none 270 20 y 1 all on 14 50 none 270 20 n 1 Ali, Elana screens off 15 50 none 270 20 n 1 all screens off 16 60 none 270 20 y 1 all screens off 17 30 none 270 20 n 1 Elana's screen on for a bit 18 30 none 270 20 y 1 all screens off 19 30 none 270 20 n 1 all screens off 20 30 none 215 20 n 1 all screens off 21 30 none 215 20 y 1 all screens off Overall results of the daytime testing were:
- If there is no ND filter lights do have a significant affect.
- With ND 1, which is way smaller than what we are going to have, the flashes do not cause additional background lighting that would possibly disturb the observations azimuth of 270 and elevation at 20. At azimuth of 215, flashlight do have an effect but it is still less compared to the typical background of 50 ADU.
- We used 7 sources with ND1, in the actual setup, we will use an ND 1.6 filter. It will be true that there will be more sources, but the total light would be the same.
- These suggest that we don't need to be concerned about DIMSUM affecting auxtel observations.
- We will verify these numbers in a fully dark night dome.
- We did not test shorter observations because AuxTel does not take shorter observations than 30 seconds "to do science" as Craig said.
- We took data with the new fixed camera and the pegboard. Below are the sources with ND 1.6:
The bottom source looks very nice. Being able to get such a nice source shows that we have a focusing issue and pegboard and cameras line of sight may not be fully perpendicular. Also, brightness of the sources are different which should not be the case. In the Clay Telescope observations, I was able to get all the sources to some extent focused, however, they were closer to each other in the image space and on the pegboard itself. It is harder to keep focus on a wider range. That said, the sources are not saturated with the ND 1.6 filter. Below are the graphs of the nice source, the source above the nice one, and a dimmer source:
They all look Gaussian. - Finally, I took a picture of the pegboard just before I closed everything. I am going to do the same thing again on Monday and compare the two images. Below is the final picture.
- Things to do on Monday with priority different than the plan forward above:
- Fix the pegboard orientation.
12/12/22: Day 5:
- Wrote a script in the weekend to transfer initially internally written files to an external disk while keep taking data. It takes four arguments, path/to/source (where the files are internally saved), path/to/target (the directory of the external disk), file type, number of files to transfer after. That would let us write the files into the internal disk and transfer them over while keep taking more data and never run out of local memory. The video below is a demonstration with a threshold of 50 files. I wrote a dummy bash script to create files with increasing numbers and wrote it to a directory:
View file name IMG_2319.MOV height 250 - There was a mild earthquake during the weekend. Nothing significant happened to the setup.
- We further stabilized the pegboard. It still moves but is not completely like a sail.
- Added additional sources by adding 1 to 4 optical fibers to the ends of 1 to 7 optical fibers. Some of the 1 to 4 optical fibers did not work anymore.
- We technically should be able to get 18 sources. However, while some sources after being divided the 4 are sufficiently bright, others could barely be seen. I do not currently know the reason behind this problem.
- Tomorrow will be the observing night of DIMSUM, Elana and I are dealing with ssh issues. I am trying to get access to the actual network rather than a guest.
- Currently sources are not perfect Gaussians, some of them are decent. It may be a plane of focus issue.
- If the issues are not fixed, I will only work with clearly seen sources, Get a mock DIMSUM run and hope for the best for a data that would be good for a paper.
12/13/22: Day 6:
- Today was the final day before the observations.
- Make the pegboard more stable by mounting onto ground from additional places.
- Ran the test before, everything was successful.
- The data was noisy, the night should be better.
The night of observing:
- The sources and trials are documented in DIMSUM_observing_analysis.ipynb
- The sources were out of focus and not all sources were identifed.
- Tried to fix it as best I as I could. Seem to be having no doughnuts but sources are worse than the trial.
- I will still try to run the analysis and see if there is anything significant. Source extractor does a pretty good job of identifying bad sources.
Notes from the observations themselves:
- Transfer code works. The images are written to the internal disk and then transferred to the external disk.
- Sources are consistent through 22:45 Chile time
- The camera is moving to the left with a rate of 2 pixels for 500 frames, this was 488
- Actually no turning left, the images started moving right, this is 1276
- We may see huge differential image motion. Or it may cancel out in the COM frame.
- We were taking 5 second exposures, from 11 until 322.
- Started 1, 10, 30 second exposures with 323
- Started 1, 5, 10, 30 second exposures with 326
- Frame 2110, still all sources detected.
- Time 23:33, rotating the dome and taking pictures with the best exposure. Planning to take an additional hour of data after this.
- 23:48 dome opened up more in addition to rotation.
- 23:50, tried 3072, focus gone, going to run back.
- Went up to dome and refocused. New data is in observe-readjust folder
- We find decent sources but it looks like we have a significant shift to left. This time for real.
- Will start taking flats at 1AM.
- The source locations got fixed again. No left shift.
Flat file trials:
Took flat files with dome closed to see the background light. Here is the summary table for the night of 12/13/22:
Exposure ID Exposure Time (s) filter AT azimuth AT elevation flash (y/n) ND filter computer screen status count 435 60 none 270 20 n 1,50 all screens off 4 436 60 none 270 20 y 1,50 all screens off 4 437 60 none 215 20 n 1,50 all screens off 438 60 none 215 20 y 1,50 all screens off 439 60 none 315 20 n 1,50 all screens off 440 60 none 315 20 y 1,50 all screens off 441 60 none 270 50 n 1,50 all screens off 442 60 none 215 50 y 1,50 all screens off 443 60 none 270 50 n 1,50 all screens off 444 60 none 270 50 y 1,50 all screens off 445 60 none 315 50 n 1,50 all screens off 446 60 none 315 50 y 1,50 all screens off 447 60 none 215 50 n 1,50 all screens off 448 60 none 215 50 y 1,50 all screens off We were only able to check the count of the first two images, they were both approximately 4. The absolute difference was negligible, Tiago suggested dividing the flash pictures by non flash pictures.
Images From Different Times of Focusing from the night of 13/12/22:
- The different centeroids will be available in the final jupyter notebook.
- The fixed focus before the trials start:
That was the best I got at the limited time. It still gave decent sources. - Focus when I checked to see what went wrong around midnight:
At that point, it looked good which was shocking but it may have been slightly tilted. I readjusted the focus when I went up just in case. - The final state of the pegboard after readjustment:
I am aware I should have documented the final focus as well. I unfortunately did not. It had looked okay and I wanted to get out of the dome to not have any background light while the observations were running. - A possible camera issue: The camera names files from 0 9999. After it reaches 9999 it goes back to 0. If we get more than 10000 photos in a run, that may be a problem because the files with the same name will be in the same directory. I can try to write a python script to add some sort of an additional extension or time stamp to the names of the file.
Observation notes after and during analysis:
- With the excitement of last night, I had confused the meaning of the location of an object in terms of rows and columns to x and y.
- This means we have the camera shaking up and down, the row values are oscillating, but the columns are decreasing. Which means the image is moving left because the camera is rotating more right.
- Remark, these are unofficial notes while making the analysis. I am documenting the entire thought process as Chris Stubbs did in strobed analysis page.
Jupyter notebook for the night of observation:
| View file | ||||
|---|---|---|---|---|
|
12/14/22: Day 7:
- Plan for the day:
- Take the time stamps of all the files, reorganize the files in a way that we only use good files.
- Make it very clear in jupyter notebook what I did.
- Back all the data up into the cluster
- Run the analysis.
- If additional time, work to make DIMSUM sturdier.
- Writing the following the next day (12/15/22)
- Ran the analysis and the graphs are available in the jupyter notebook.
- Overall, the data does not seem to useful, as the sources get worse, the differential motion measured increased. I did several cuts to data with respect times of source quality. As the time passes (and source quality decreased) the differential image motion was simply more. I do not know if it can be trusted. I still have all the datas in the csvs.
- At the end of second run we have a lot of dark files, when I checked the hardware out today it seemed fine.
- We attempted to make both the camera and the pegboard more sturdy.
- Mounted the camera directly on top of the L rail. It is harder to adjust but more sturdy. Such a mount made the camera higher than before and unable to tilt. To address that we moved the pegboard up. Today we will see whether the higher pegboard will be blocked by the telescope. Telescope was not able to rotate last night.
- We debugged the focusing and ghosting issue with continuous light sources. Even in the most focused case, the sources seem to have ghosting affects. The ghosting did not get fixed when we relieved pressure from the optical fiber, changed the location of the source in the pegboard, or how far we put through the fiber to the thing it's holding, or whether we are using sm1 holders or what Elana had used in her previous setup. Here are examples of ghosting we see:
- I ran some analysis with the more sturdy setup. It looks decent. I also did two more tests when the fan downstairs was open. Unfortunately in one of the tests the window was not open so the fan did not ork probably. I was also only allowed to work it up to 20Hz.
- Overall, the view from the camera, the camera itself, and the pegboard itself look like this:
- Backed up both of the observing runs to the cluster. Going to back up the additional tests as well.
12/15/22: Day 8:
- Further document the Jupyter notebook about the observations of the tests of the night.
- Back the rest of the data up to cluster.
- Copy the local data to the external disk as well.
- As soon as the telescope is able to rotate again check whether the new setup is blocked by the telescope.
- If not blocked. That is the new final setup of the dimsum.
- Make a test with fan on 100Hz and check if the setup can take it.
- If it passes that test as well, make the setup permanent. This includes:
- Making the cables trip proof by running them under the grating and tying the optical fibers to the rail.
- Put the box and power adaptors to the corner
- Clean up the mess and decide what to bring back.
Newest version of the observing jupyter notebook:
| View file | ||||
|---|---|---|---|---|
|
The moving of the telescope does not affect the new setup. And the camera and pegboard were in the same position in the morning.
| View file | ||||
|---|---|---|---|---|
|
12/15/22: Day 9:
- Last day we could be on the summit.
- Today and last night I got the DIMSUM to its final state and professional looking.
- A problem: Not all sources are identified in a brighter dome.
- The setup is a 1 to 7 optical fiber which have 1 to 4 optical fibers connected to four of the ends. One of the fibers of 1 to 4 optical fibers did not work and one of them was very dim compared to others. So I had omitted those two. That left us with 14 sources. Then I had added the remaining 3 fibers from 1 to 7 to the pegboard. With that I was getting 17 sources.
- That ended up creating a brightness difference between the sources and with an ND filter of avoiding the saturation of brightest sources the dimmest sources were not seen in a brighter dome (before astronomical twilight).
- Changed the setup in the last day and omitted the direct fibers coming from the 1 to 7 fiber. Instead I added dim 1 to 4 fiber which gave us 15 sources. Doing that I decreased the ND filter to 1 from 1.5. Even with the ND filter of 1, the count of the dimmest source was very dim. It was able to be detected but I doubt we may run into the same issue in a brighter dome. I do not understand why some sources coming from the same 1 to 4 optical fiber can have different brightness. That is the case even in the continuous light source. If the fibers of 1 to 7 and 1 to 4 do not divide the light equally, I believe that is a very unnecessary uncertainty.
- Final setup is 15 sources with nd filter 1.
- Having possible plane of focus and ghosting issues made me come up with ways to make my analysis code way more configurable and flexible with equal success. I had an algorithm which simply showed the sources qualitatively without really relating them to particular source identified.
- I made two upgraded functions where when sources are shown qualitatively. I also made finding sources higher abstracted. All source finding is done by python source extractor which is exactly the same as c and part of cython.
- find_sources: is a function that takes a path/to/file and a threshold as an input and returns the sources identified ordered by x and y coordinates.
- show_sources: is a function that shows the sources identified in a box that has dimensions 4 times the fwhm.\
- show_sources_3d: makes 3d plots of the sources to have quantitative understanding of the source quality and effect of ghosting.
- Sample output of show_sources:
View file name 2d_test.pdf height 250 - Sample output of show_sources_3d:
View file name 3d_test.pdf height 250 - I took photographs of the setup for documentation including:
- Taking the lens cap of the camera.
- Turning the camera on.
- Making sure camera setting is in on M.
- Turning on the remote flash trigger.
- Turning on the remote flash receiver.
- Turning on the power source.
- I also took photographs of possible trouble shooting:
- The continuous halogen lamp light source for debugging the focus if needed.
- The screw to easily rotate the plane of pegboard.
- Additional AA batteries in case the receiver or trigger runs out of battery.
- The filter tool that lets us change the ND filtering.
- I am going to write a documentation of the setups needed to take data with the setup.
- I am also going to make a sample jupyter notebook which can be duplicated and the title could be changed which runs through the possible steps of DIMSUM analysis. The notebook will be more concise than the notebook I used for documentation. However, Craig told me that people would actually follow analysis steps through a notebook.
- Finally, we packed everything and left the Auxtel dome cleaner than it was.
- All of the data we took is backed up in the external disk which currently has 1.8Tb of free space and in the cluster.
- The data is in a file called december_22_chile which has subdirectories of:
- december_22_chile_tests: that includes the raw files of the tests we did
- december_22_chile_csvs: results of the analysis of the differential motion tests and time stamps of the files in the test
- december_22_chile_pdfs: figures we made out of the csvs
- actual_observing_run: The main observing run that took place in 12/13/22
- observe-readjust: Continuation of observing run after the camera was adjusted.
- Final version of the jupyter notebook is here:
View file name DIMSUM_observing_analysis.ipynb height 250
From Stubbs:
Ok sounds like a partial success. I think the key think is to get a good dome-closed vs. dome-open comparison.
I don't understand the longer exposures. Is that with a single flash per image? If so then all the longer images would seem to do is add to the background and not the signal. Or are you running with constant light source?
I can certainly believe that dome rotation shakes things- During most image the dome is not rotating.
The focus change could well be due to temperature change of the lens. I can't remember if there is an f-stop ring on that lens or not... I think not. The depth of focus will increase if you make a smaller aperture in front of the lens. There is heavy duty Aluminum foil in the cardboard fan box at the base of the pier. You could try making a snout with a 1 inch off-axis hole and see if that (with reducing ND filter) helps with focus.
...