First stage of the generic pipeline. The existing scripts parse
filenames, expecting calibrated-T32-...-Luminance-BIN2-W-300-001.fit,
which works for exactly one telescope, one binning and one exposure
length. Three of the four archived sessions have no luminance channel at
all, so that approach cannot process them.
session.py opens each frame and reads FILTER, EXPTIME, XBINNING, OBJECT
and DATE-OBS instead. It groups frames by filter, works out which colour
scheme the session supports (LRGB, RGB, SHO, HOO or mono), and writes a
manifest. Filter aliases are mapped, so a header saying H-alpha, Halpha
or HA all resolve to Ha.
Three real defects were found by running it against the archive rather
than by reasoning about it:
Frame counts came out doubled. iTelescope ships every frame twice, as
'calibrated-*' with bias, dark and flat applied and 'raw-*' without, and
extracting both put two copies of each frame into the stack. Only the
calibrated archives are extracted now, with CALSTAT used as a header
level safety net for sessions that were already extracted the old way.
NGC 5128 now scans as 12/4/4/4, exactly matching the session processed
by hand, which makes it a usable regression test.
A re-mirror over the organised tree produced duplicates and mirror
damage. An interrupted segmented download leaves DIRECTORIES named after
the file being fetched - a directory called something.fit - plus
.cyberducksegment parts. Anything that trusts a file extension will try
to open a directory as a frame. Those are now skipped explicitly at
discovery, ingest and scan, and byte-identical copies that a re-mirror
puts back are removed as they are refiled.
Two T20 archives are corrupt and cannot be extracted. That is real data
loss, not a bug: the session has 14 of its 16 frames and the warning is
printed rather than swallowed.
Not yet done: NGC 6744 mixes bin1 luminance at 4096x4096 with bin2
colour at 2048x2048, so registration will need to resample onto a common
grid rather than assuming every frame shares a shape.
The mechanical merge preserved both histories but left two repositories
sitting side by side rather than one repository. This is the part that
makes it whole.
A top-level README explains the three strands - the telescope network
and campaign, the processing code, and in-person observing plans - and
says plainly that image data does not live here, because that is the
first question anyone will have when they find a pipeline with no
pixels.
state/PROJECT.md was still describing a markdown-only reference project
whose scope explicitly EXCLUDED automating bookings and image
processing. Both of those are now most of what the project does, so the
objective, scope and key facts are rewritten to match reality.
state/DECISIONS.md records why the merge happened - the two repos were
split by chronology rather than design, and the seam was already
leaking, with the campaign's TODO citing results held in the other repo
and the pipeline's README explaining a campaign it did not contain. It
also records that the image data deliberately stays out of the
repository.
state/TODO.md gains the pipeline and observing work that previously had
nowhere to live, including the measurement that changes the pipeline
plan: per-frame processing is 2.9 seconds and the whole 24-frame session
is 1.1 minutes single-threaded, so a scheduler earns nothing on stacking
and should be pointed at the Monte Carlo stages instead.
Internal links fixed for the new paths: pipeline/README.md referred to
session-scripts/ throughout, and itelescope/README.md pointed at a
state/ directory that is now one level up.
The .gitignore conflict between the two repositories is resolved by
combining them, with image formats ignored globally except under docs/,
where documentation figures are committed deliberately.
Preparing to merge this repository into a combined astrophotography
repo. session-scripts/ becomes pipeline/ because the scripts import
layout.py from their own directory and must stay together, and because
'pipeline' says what it is rather than how it came about. observing/
stays at the top level: observing plans are not processing code.
