Page Comparison

All Sky camera for LSST site.

Nov 29 2013. Setting up mac mini to run Canon camera:

Nov 29 2013.

Setting up Mac mini for gphoto2 using Dave Monet's notes:

libtool-2.4.2:

./configure --prefix=/home/dgm/linux

make

make install

libusb-1.0.9:

./configure --prefix=/home/dgm/linux

make

make install

libgphoto2-2.5.2:

setenv CPPFLAGS "-I/home/dgm/linux/include"

setenv LDFLAGS "-L/home/dgm/linux/lib"

setenv PKG_CONFIG_PATH /home/dgm/linux/lib/pkgconfig

./configure --prefix=/home/dgm/linux

make

make install

gphoto2-2.5.2:

./configure --prefix=/home/dgm/linux

make

make install

(does not need the CPPFLAGS and LDFLAGS of libgphoto)

------------

downloaded

libgphoto2-2.5.2

gphoto2-2.5.2

libusb-1.0.9

libtool-2.4.2

I am going to set this up like Dave Monet's configuration, in a directory

called linux off the user's home directory.

did sudo xterm

put the directories into /usr/local as root.

trying libtool: rats, no c compiler on the path!

so for that I need Xcode developer's toolkit

downloading that from Apple web site, 1.7 GB!

after some pain downloading and installing Xcode for this version of OSX,

trying again....

nope. It turns out I also need to install "Command Line Tools" for Xcode

ok, now we have gcc.

did same installation as Dave Monet has, except replace

/home/monet/linux

with

/Users/christopherstubbs/linux

seemed to work for libtool and libusb. Had to switch to tcsh for setenv to work.

got a make error for libgphoto2. Try putting pkgconfig on path with

set path = ( $path /Users/christopherstubbs/linux/lib/pkgconfig)

nope- try downloading and installing it

did

./configure --prefix=/Users/christopherstubbs/linux --with-internal-glib

now try libgphoto again

set path = ( $path /Users/christopherstubbs/linux/bin)

did setenv to this

set path = ( $path /Users/christopherstubbs/linux/bin)

got compiler errors for libgphoto2, fixed some undelcared variables in

linux.c but then encountered

libtool: link: gcc -o .libs/ptp2.so -bundle ptp2/.libs/ptp.o ptp2/.libs/library.o ptp2/.libs/usb.o ptp2/.libs/ptpip.o ptp2/.libs/config.o ptp2/.libs/olympus-wrap.o -L/Users/christopherstubbs/linux/lib ../libgphoto2/.libs/libgphoto2.dylib /usr/local/libgphoto2-2.5.2/libgphoto2_port/libgphoto2_port/.libs/libgphoto2_port.dylib -lm ../libgphoto2_port/libgphoto2_port/.libs/libgphoto2_port.dylib /Users/christopherstubbs/linux/lib/libltdl.dylib -lpthread -liconv -O2 -Wl,-dylib_file -Wl,/Users/christopherstubbs/linux/lib/libgphoto2_port.10.dylib:/usr/local/libgphoto2-2.5.2/libgphoto2_port/libgphoto2_port/.libs/libgphoto2_port.dylib -Wl,-exported_symbols_list,.libs/ptp2-symbols.expsym

Undefined symbols for architecture x86_64:

"_olympus_setup", referenced from:

_camera_init in library.o

ld: symbol(s) not found for architecture x86_64

collect2: ld returned 1 exit status

make[3]: *** [ptp2.la] Error 1

make[2]: *** [all-recursive] Error 1

make[1]: *** [all-recursive] Error 1

make: *** [all] Error 2

[server:/usr/local/libgphoto2-2.5.2] root#

and gave up.

---------------

OK so let's try a totally different way- macports.

installed macports for Lion, 10.7

it puts the executable into /opt/local/bin so execute that explicitly,

did port -v selfupdate and that worked.

also did install of gphoto2 using ports and that worked as well,

although it stuffed things into /opt/local/bin, so need to add that to path.

This is a good resource:

http://photolifetoys.blogspot.com/2012/08/control-your-camera-with-gphoto2-via.html

Before running gphoto2 you need to terminate this "hijack" by running the following command:

killall PTPCamera

This will need to be done every time the camera is plugged in or turned on while connected to your mac.

/opt/local/bin/gphoto2 --summary

/opt/local/bin/gphoto2 --capture-image-and-download

works!!

to do next:

add /opt/local/bin to path

get JT's analysis stuff installed here.

Nov 30, 2013.

Twilight times

Pachon is at West long of 70.7333, lat -30.2333

used USNO site http://aa.usno.navy.mil/data/docs/RS_OneYear.php to determine civil twilight times in UT.

Data file is here pachon_twilight.dat

broke that into one file per month, called mxx.dat with xx ranging from 01 to 12. Did

grep -v "%" pachon_twilight.dat | awk '{print $1,$3,$2}' > m01.dat

grep -v "%" pachon_twilight.dat | awk '{print $1,$5,$4}' > m02.dat

note that this intentionally reverses the times so that sundown in column 1 and sunrise in col 2, in UT

can use this to determine when to start taking data.

OSX cron jobs

set up time as GMT
sudo touch /etc/crontab
crontab -e edits the crontab file.
This is a good tutorial: http://rossb.biz/blog/2011/os-x-cron-jobs-a-simple-tutorial/
syntax we want is to start the task at a sensible time before sunset, say 2200 UT, i.e. 6 pm local. The script should then
1. check space avail.
2. create data directory
3. change image prefix
4. cd to data directory
5. compute appropriate number of images to take
6. wait for sunset time according to date
  1. then start collecting N images
  2. end when sunup or N is exceeded
  3. run simple analysis, convert to fits files, compress data, update status web page
  4. end

cron entry time format is

so we want

1 22 * * * /usr/local/bin/startnight.sh

I suppose we could just go ahead and use crontab to initiate each night's data collection at the desired time, within a range of dates. Say we break things up into two week intervals. For example we could do

1 22 1-15 01 * <tasks for first half of Jan>

30 22 16-31 01 * <tasks for second half of Jan>

etc.

So let's make the table of startup times, in the format crontab wants. Make a list of

minute hour dayrange month

15 00 01-15 01

10 00 16-31 01

05 00 01-15 02

45 23 16-28 02

30 23 01-15 03

15 23 16-30 03

00 23 01-15 04

30 22 16-31 04

30 22 01-15 05

15 22 16-31 05

15 22 01-15 06

15 22 16-31 06

40 22 01-15 07

45 22 16-31 07

45 23 01-15 08

45 23 16-31 08

00 23 01-15 09

00 23 16-31 09

15 23 01-15 10

15 23 16-31 10

30 23 01-15 11

45 23 16-31 11

00 00 01-15 12

15 00 16-31 12

gphoto2 commands

shutter speed:

server:~ christopherstubbs$ gphoto2 --get-config=/main/capturesettings/shutterspeed

Label: Shutter Speed

Type: RADIO

Current: 15

Choice: 0 30

Choice: 1 25

Choice: 2 20

Choice: 3 15

Choice: 4 13

Choice: 5 10

Choice: 6 8

Choice: 7 6

Choice: 8 5

Choice: 9 4

Choice: 10 3.2

Choice: 11 2.5

Choice: 12 2

Choice: 13 1.6

Choice: 14 1.3

Choice: 15 1

Choice: 16 0.8

Choice: 17 0.6

Choice: 18 0.5

Choice: 19 0.4

Choice: 20 0.3

Choice: 21 1/4

Choice: 22 1/5

Choice: 23 1/6

Choice: 24 1/8

Choice: 25 1/10

Choice: 26 1/13

Choice: 27 1/15

Choice: 28 1/20

Choice: 29 1/25

Choice: 30 1/30

Choice: 31 1/40

Choice: 32 1/50

Choice: 33 1/60

Choice: 34 1/80

Choice: 35 1/100

Choice: 36 1/125

Choice: 37 1/160

Choice: 38 1/200

Choice: 39 1/250

Choice: 40 1/320

Choice: 41 1/400

Choice: 42 1/500

Choice: 43 1/640

Choice: 44 1/800

Choice: 45 1/1000

Choice: 46 1/1250

Choice: 47 1/1600

Choice: 48 1/2000

Choice: 49 1/2500

Choice: 50 1/3200

Choice: 51 1/4000

Choice: 52 1/5000

Choice: 53 1/6400

Choice: 54 1/8000

server:~ christopherstubbs$ gphoto2 --get-config=/main/capturesettings/aperture

Label: Aperture

Type: RADIO

Current: 2.8

Choice: 0 2.8

Choice: 1 3.2

Choice: 2 3.5

Choice: 3 4

Choice: 4 4.5

Choice: 5 5

Choice: 6 5.6

Choice: 7 6.3

Choice: 8 7.1

Choice: 9 8

Choice: 10 9

Choice: 11 10

Choice: 12 11

Choice: 13 13

Choice: 14 14

Choice: 15 16

Choice: 16 18

Choice: 17 20

Choice: 18 22

server:~ christopherstubbs$ gphoto2 --get-config=/main/imgsettings/iso

Label: ISO Speed

Type: RADIO

Current: 640

Choice: 0 Auto

Choice: 1 100

Choice: 2 125

Choice: 3 160

Choice: 4 200

Choice: 5 250

Choice: 6 320

Choice: 7 400

Choice: 8 500

Choice: 9 640

Choice: 10 800

Choice: 11 1000

Choice: 12 1250

Choice: 13 1600

Choice: 14 2000

Choice: 15 2500

Choice: 16 3200

Choice: 17 4000

Choice: 18 5000

Choice: 19 6400

Choice: 20 Unknown value 007b

Choice: 21 Unknown value 007d

Choice: 22 12800

Choice: 23 Unknown value 0083

Choice: 24 Unknown value 0085

Choice: 25 25600

to set shutter speed to one second: gphoto2 --set-config=/main/capturesettings/shutterspeed=1

(argument is simple shutter speed in seconds, it seems)

to take and store an image with existing settings gphoto2 --capture-image-and-download --filename ut113013.0001.cr2

...

Dec 1 2013

have fixed up cron task so that it works properly. It turns out need to set paths within the cron script, so sourcing bash_profile within script works.

did power cycle on mac mini and camera, and things run fine after power up.

Jan 12 2014, at Pachon.

Have powered up mac mini within the Illinois aeronomy building. Set networking for hardwired ethernet to

IP 139.229.19.220

submet mask 255.255.255.224

router 139.229.19.193

we can also use IP address 139.229.19.221 if need be.

local access is ssh christopherstubbs@139.229.19.220, cat4apple

On Dec 18, 2013, at 12:21 PM, Alan Liu <alan.liu@erau.edu> wrote:

Chris,
Please set up your computer using the following parameters:
Router 139.229.19.193
Mask: 255.255.255.224
DNS: 139.229.15.130, 139.229.2.3
IP 139.229.19.220
If you need additional IP number, 139.229.19.221 can also be used.
In terms of remote access to ALO computers, you also need to obtain information about firewall access from Ron Lambert rlambert@ctio.noao.edu, who is in the process of reconfiguring the firewall at ALO.
Let me know if you have any questions,
Alan
--
Alan Liu, Ph.D.
Associate Professor
Physical Sciences, Daytona Beach
Embry-Riddle Aeronautical University
Daytona Beach, Florida
Office: +1-386-226-6538
email; alan.liu@erau.edu

here are the scripts that run, as of Jan 12:

startnight.sh (takes NN frames as command line argument)

server:~ christopherstubbs$ cat startnight.sh

#!/bin/bash

# modified Dec 12 2013 to allow N frames set from command line, overwrites what was set in initialize.sh

export maxframes=$1

echo first arguent is $1

source ~/initialize.sh

echo maxframes is $1

while [ $framecounter -lt $maxframes ]; do

echo sleeping for $pauseinterval

sleep $pauseinterval

gphoto2 --set-config=/main/capturesettings/shutterspeed=1

# this next trick pads out the framecounter to 4 digits for decent file names

export paddedcounter=`printf "%04i\n" $framecounter`

gphoto2 --capture-image-and-download --filename $dirpath/$dirname/$dirname.$paddedcounter.short.cr2

let framecounter=framecounter+1

export paddedcounter=`printf "%04i\n" $framecounter`

gphoto2 --set-config=/main/capturesettings/shutterspeed=10

export paddedcounter=`printf "%04i\n" $framecounter`

gphoto2 --capture-image-and-download --filename $dirpath/$dirname/$dirname.$paddedcounter.long.cr2

let framecounter=framecounter+1

done

# at this stage, we're done collecting images for the night. Wrap things up...

ls $dirpath/$dirname/*.cr2 | wc | awk '{print $1," images collected"}' >> $dirpath/$dirname/$dirname.log

# now do image conversion to fits, and run some initial analysis.

echo making fits files at `date`

source ~/makefits.sh

echo extracting sky values at `date`

source ~/getsky.sh

echo doing photometry at `date`

source ~/photometry.sh

# copy result files to Amazon Web Services machine

echo copying files to aws at `date`

scp -i ~/aws/aws1.pem.txt -r $dirpath/$dirname/$dirname.* ec2-user@54.200.60.175:~/data/

echo compressing images at `date`

cd $dirpath/$dirname

gzip B/*.fits

gzip G/*.fits

gzip R/*.fits

gzip M/*.fits

gzip CR2/*.cr2

echo moving the log file to appropriate directory at `date`

mv ~/cronlog $dirpath/$dirname/$dirname.cronlog

initialize.sh

server:~ christopherstubbs$ cat initialize.sh

#!/bin/bash

# try to fix up user issues for crontab

source /Users/christopherstubbs/.bash_profile

# set up directory names and data paths

export datadisk=/dev/disk1s2

export dirpath=/Users/christopherstubbs/data

export dirname=`date +"ut%m%d%y"`

export thismonth=`date +"%m"`

# create tonight's directory

mkdir $dirpath/$dirname

cd $dirpath/$dirname

date | awk '{print "started at "$0 }'>> $dirname.log

df -H | grep $datadisk | sed s/G//g | awk '{print $4," GB left on data disk"}' >> $dirname.log

export spaceleft=`df -H | grep $datadisk | sed s/G//g | awk '{print $4}'`

# fix this later

#if ($spaceleft>20); then

# mail -s "allsky camera lots space!" stubbs@physics.harvard.edu

#fi

# wipe out any existing connections to camera

killall PTPCamera

# set up environment variables for camera

export framecounter=0001

export xctr=2875

export yctr=1920

# prefix for images, gets framecounter.cr2 appended

export imageprefix=$dirname.

# interval after end of last image before starting next one, in seconds

export pauseinterval=50

# configure aspects of camera that won't change

# synch camera datetime to ntp-served acquisition computer value, in UT

gphoto2 --set-config syncdatetime=1

gphoto2 --set-config iso=1600

gphoto2 --set-config aperture=2.8

cd ~

photometry.sh:

server:~ christopherstubbs$ cat photometry.sh

#!/bin/bash

# performs quick photometry on each image, using tphot

cd $dirpath/$dirname/M

rm *.phot

rm *.nstars

for i in *.M.fits; do tphot $i -out `basename $i .fits`.phot; done

# how many stars in each image?

for i in *.M.phot; do wc $i | awk '{print ($1-1)}' >> $dirname.nstars ; done

mv $dirname.nstars ..

cd $dirpath/$dirname/B

rm *.phot

for i in *.B.fits; do tphot $i -out `basename $i .fits`.phot; done

cd $dirpath/$dirname/G

rm *.phot

for i in *.G.fits; do tphot $i -out `basename $i .fits`.phot; done

cd $dirpath/$dirname/R

rm *.phot

for i in *.R.fits; do tphot $i -out `basename $i .fits`.phot; done

cd ..

rm temp

# add extra line to nstars file to accommodate header line coming up

echo " " > temp2

cat $dirname.nstars >> temp2

mv temp2 $dirname.nstars

rm temp2

# add another column to the obslog file

paste $dirname.obslog $dirname.nstars >> temp

mv temp $dirname.obslog

# separate out long and short exposures

echo "#image BIAS EXPTIME MJD-OBS M B G R nstars" > $dirname.short.obslog

grep short $dirname.obslog >> $dirname.short.obslog

echo "#image BIAS EXPTIME MJD-OBS M B G R nstars" > $dirname.long.obslog

grep long $dirname.obslog >> $dirname.long.obslog

getsky.sh

server:~ christopherstubbs$ cat getsky.sh

#!/bin/bash

# for a rough measure of sky brightness, extract mean for a region near the center of sensor, in each band. Uses header to subtract bias scalar

cd $dirpath/$dirname/M

rm *.sky.*.dat

rm *.bias.*.dat

rm *debias*.dat

for i in *.M.fits; do getpix $i 800-1200 800-1200 -m | grep Mean | awk '{print $2}' >> $dirname.sky.M.dat ; done

gethead *.fits BIAS EXPTIME MJD-OBS > $dirname.M.obslog

paste $dirname.M.obslog $dirname.sky.M.dat >> temp

awk '{print ($5-$2)/$3}' temp >> $dirname.skydebiased.M.dat

# put sky rate ADU/sec/pix into obslog file

paste $dirname.M.obslog $dirname.skydebiased.M.dat >> temp2

mv temp2 $dirname.M.obslog

rm temp2

rm temp

cd $dirpath/$dirname/B

rm *.sky.*.dat

rm *.bias.*.dat

rm *debias*.dat

for i in *.B.fits; do getpix $i 800-1200 800-1200 -m | grep Mean | awk '{print $2}' >> $dirname.sky.B.dat ; done

gethead *.fits BIAS EXPTIME MJD-OBS > $dirname.B.obslog

paste $dirname.B.obslog $dirname.sky.B.dat >> temp

awk '{print ($5-$2)/$3}' temp >> $dirname.skydebiased.B.dat

# put sky rate ADU/sec/pix into obslog file

paste $dirname.B.obslog $dirname.skydebiased.B.dat >> temp2

mv temp2 $dirname.B.obslog

rm temp2

rm temp

cd $dirpath/$dirname/G

rm *.sky.*.dat

rm *.bias.*.dat

rm *debias*.dat

for i in *.G.fits; do getpix $i 800-1200 800-1200 -m | grep Mean | awk '{print $2}' >> $dirname.sky.G.dat ; done

gethead *.fits BIAS EXPTIME MJD-OBS > $dirname.G.obslog

paste $dirname.G.obslog $dirname.sky.G.dat >> temp

awk '{print ($5-$2)/$3}' temp >> $dirname.skydebiased.G.dat

# put sky rate ADU/sec/pix into obslog file

paste $dirname.G.obslog $dirname.skydebiased.G.dat >> temp2

mv temp2 $dirname.G.obslog

rm temp2

rm temp

cd $dirpath/$dirname/R

rm *.sky.*.dat

rm *.bias.*.dat

rm *debias*.dat

for i in *.R.fits; do getpix $i 800-1200 800-1200 -m | grep Mean | awk '{print $2}' >> $dirname.sky.R.dat ; done

gethead *.fits BIAS EXPTIME MJD-OBS > $dirname.R.obslog

paste $dirname.R.obslog $dirname.sky.R.dat >> temp

awk '{print ($5-$2)/$3}' temp >> $dirname.skydebiased.R.dat

# put sky rate ADU/sec/pix into obslog file

paste $dirname.R.obslog $dirname.skydebiased.R.dat >> temp2

mv temp2 $dirname.R.obslog

rm temp2

rm temp

cd $dirpath/$dirname/R

ls *.R.fits > listing

mv listing ..

cd ..

echo "#image BIAS EXPTIME MJD-OBS M B G R nstars " > $dirname.obslog

paste M/$dirname.M.obslog B/*.skydebiased.B.dat G/*.skydebiased.G.dat R/*.skydebiased.R.dat >> $dirname.obslog

time zone is set to GMT, with automatic time updates enabled

power up reboot is enabled.

also, the mac mini fires up its own wireless called LSSTserver. If I connect to that, wirelessly, I can log in with

ssh christopherstubbs@server.local

and that works, too.

Need to make sure camera is set to M mode, in order to be able to control the shutter speed.

Jan 13 2014

Here is sensitivity curve for Canon 5D color bands:

Image Removed

FOV of 15mm Sigma fisheye lens is as follows:

in long direction: 147.5 degrees, or 73 degrees off zenith

in short direction: 94.3 degrees, or 47 degrees off zenith

corner-to-corner: 184 degrees, or horizon to horizon.

We're currently oriented with EW in long direction, and NS in short direction. So we don't quite get to celestial pole in the S but we do get to the equator in the N

...