Guider mode imaging

We have the ability to take guider mode images with AuxTel, ComCam, and LSSTCam. We can take 30s 9 Hz data with 50 x 50 ROI or 3 Hz data with a 400 x 400 ROI. We can get data from every amp simultaneously.

From Gregg Thayer on 2/16/2024:

"The guider system does not store the guide data, it's up to the Client to decide what it wants to do with it. DAQ provides C++ headers and libraries to write your own client to do with the data what you wish. The client registers itself with the guider system for which sensors it wishes to receive data about. The data is not saved anywhere, the interface is in memory only. Obviously the Client code is free to save the data, process/reduce the data, drive actuators in response to the data, or whatever. Depending on the amount of processing done, you may have to split the Clients across multiple nodes/threads/NICs/etc.

The data format that contains all 16 amps is made available through a class called RawStamp which is truly a copy of the data as it is produced by the REBs. I am pretty sure this means it's 18-bit packed and interleaved (not grouped by amplifier). It was intended for debugging purposes, and as such is as little processed as possible. This also means that ROIs that span amps (split-ROIs) are also not reconstructed. Though, IIRC you weren't planning on using that for this mode.

If you wanted the DAQ to put this in a different form, we can negotiate that, though I wouldn't be surprised if you wanted the binary blob to pick apart in your own code. Particularly if you're just spooling it to disk for offline processing where performance isn't as big a concern. I could see it something like an array of 16 Stamps like the one that we provide currently, but to do so for all cases (Sensor type, split-ROI, etc) might be more work than is worth.

I also note that during testing, I've run in to a bit of a bandwidth bottleneck (multicast packets through the embedded switches!?!?!) that limits the number of sensors I can include in the system at a time. I'm currently working on a fix that should remove that limitation at the cost of some latency/rate. This work is not yet complete, is non-trivial, but I don't think has any likely showstoppers.

Having said that, ComCam could be used now with the current system.

To get you started here's the "documentation" The description of the data client/format begins on slide 17. https://www.slac.stanford.edu/~jgt/GDSAPI_v6.pdf

I've put the data from our one successful AuxTel acquisition from last year here: https://www.slac.stanford.edu/~jgt/00014_00_0_0.tgz
I also have some Split-ROI data from IR-2 if you're feeling ambitious. Note, that for the RawStamps, I expect the E2V data to be different in the ordering of the amplifiers in the same way it's different for science data. I haven't had a crack at any E2V data, so I haven't been able to test any.
The code (which you'll want for the definitions of the binary data structures) is here: https://www.slac.stanford.edu/~jgt/R5-V8.2.tgz
Here's the explanation of how to use them:
There are four types of files .stamp , .rstamp , .rstamp.unpacked, .rstamp.decoded The file names are of the form <sequence_number><Raft><REB>_<Sensor>.<type>. The unpacked and decoded files were generated from the .rstamp files after acquisition, and the .rstamp.decoded should be identical to the .stamp file which was recorded from the data stream at acquisition time. I've included the images
The structure of each file is of the following form:
StateMetadata (Type: Start)
SeriesMetadata
StateMetadata (Type: Stamp)
Stamp Data
StateMetadata (Type: Stamp)
Stamp Data
.
.
.
StateMetadata (Type: Stamp)
Stamp Data
I do not record Stop, Pause, or Resume transitions, though there's no reason I couldn't.
The structure of the Metadata blocks can be found in R5-V8.2/include/gds/StateMetadata.hh and R5-V8.2/include/gds/SeriesMetadata.hh They refer to the files RoiCommon.hh and RoiLocation.hh which in turn refer to Location.hh in the same directory. The SeriesMetadata should include all the information necessary to decode the Stamp Data sections.
The structure of the Stamp Data depends on the file type as follows:
In .stamp and .rstamp.decoded they are a 1-D array of pixels (minus the underscan columns) represented as little-endian signed 32-bit integers. The pixels are in the order they are read out, so the orientation of the rows and columns in real space varies in the usual way by segment.
.rstamp contains the data from all 16 segments (including underscan cols) of the sensor interleaved in the order they are read out in 18-bit packed form. This is the raw bitstream produced by the REB.
.rstamp.unpacked contains all the data of .rstamp but the pixels have been unpacked into little-endian signed 32-bit integers. The segments are still interleaved in the usual way.
These files were collected using the utility R5-V8.2/examples/gds/gds_archiver.cc and processed with gds_rstamp_unpack.cc and gds_rstamp_decode.cc