laurence
3f785136ed
Rule 2 of the workflow is that state lives in the repo rather than the chat, and a long session had put most of the current position only in the chat. This writes it down. TODO.md gains a RESUME HERE block at the top, because the useful question for the next session is not what has been done but what to do next and what is blocking it. It names the three live strands, the three decisions waiting on the user, and the one thing that will break the batch run if it is forgotten: NGC 6744 mixes bin1 luminance at 4096x4096 with bin2 colour at 2048x2048, so registration has to resample onto a common grid. Nothing else in the archive does that, which is exactly why it will be forgotten. It also drops three duplicate "Done (recent)" headings that had accumulated, and tabulates what the five archived sessions actually contain. NOTES.md gains the gotchas, all of which produced plausible wrong answers rather than errors: every frame is delivered twice as calibrated and raw, so extracting both silently doubles the stack; an interrupted segmented download leaves DIRECTORIES named like files, which defeats anything that trusts an extension; a re-mirror duplicates rather than replaces; two T20 archives are genuinely corrupt; extraction triples the disk footprint and filled the drive. Alongside them the analysis lessons from NGC 5128, kept for the same reason - each is a way to be confidently wrong. DECISIONS.md records why the pipeline reads headers rather than filenames, and why imaging is being made push-button while the science stays hand-driven. ARCHITECTURE.md described a documentation-only repository with no code, which stopped being true when the pipeline merged in. It now covers all three parts, the code-and-data separation the pipeline is built around, and the on-disk session layout. Both of this session's own errors are recorded in TODO.md as things not to repeat: measuring a foreground star instead of the galaxy nucleus and booking telescope time to fix it, and telling the user NGC 6744 had not been processed when it had.
75 lines
3.5 KiB
Markdown
75 lines
3.5 KiB
Markdown
# Architecture
|
|
|
|
> How the system is built and why. Update this when the structure changes; a change is
|
|
> not finished until this reflects it.
|
|
|
|
Three parts, which is why the repository has three top-level content
|
|
directories. Until 2026-07-21 the first two lived in separate repositories.
|
|
|
|
## itelescope/ - the network and the campaign
|
|
|
|
Documentation and data, no code.
|
|
|
|
- `TELESCOPES.md`: the deliverable. Per-observatory sections, one entry per
|
|
telescope (spec bullets then a review paragraph), a "choosing a telescope"
|
|
table, and general observations. Compiled from the sources below.
|
|
- `TARGETS.md`: southern targets invisible from the UK, matched to scopes and
|
|
to the time of year.
|
|
- `CAMPAIGN.md`: the points drain campaign - ledger, bookings, queue, rate
|
|
model. Updated after every session from the account's transaction history.
|
|
- `data/itelescope-telescopes.csv`: verbatim CSV export of iTelescope's
|
|
maintained specs sheet. The source of truth for numbers; re-exported to
|
|
refresh (see `state/NOTES.md`).
|
|
- `plans/`: ACP observing plans, one per booked run, uploaded to the scope.
|
|
|
|
## pipeline/ - the processing code
|
|
|
|
Python 3.12. The design rule is that **code and data are separate**: nothing
|
|
here knows an absolute path. A session is located at runtime through the
|
|
`ASTRO_SESSION` environment variable.
|
|
|
|
- `layout.py` maps a **filename** to the subdirectory it belongs in, using the
|
|
same rules the session directories are organised with. A script asks for
|
|
`master-Red.fit` or `_stars.npz` and gets the right path without any call
|
|
site knowing the structure. Unrecognised names resolve to the session root,
|
|
which is visible and correctable rather than silently wrong.
|
|
- `session.py` is the entry point for anything generic. It discovers sessions,
|
|
ingests and uncompresses them, reads every frame's header, groups frames by
|
|
filter, and derives the colour scheme (LRGB / RGB / SHO / HOO / MONO) from
|
|
what is actually present. **Header-driven, never filename-driven** - see
|
|
DECISIONS.
|
|
- `restructure.py` reorganises a flat session directory into the standard
|
|
layout. Idempotent, dry run by default.
|
|
- The remaining scripts are the NGC 5128 session as it was actually run:
|
|
`unzip -> analyse -> stack -> solve -> depth -> compose -> hdr/enhance/
|
|
starless/annotate -> final -> closeup -> triptych`, plus the analysis
|
|
families `gc-*` (globular clusters), `sb_*` (surface photometry) and `mo_*`
|
|
(moving objects and transients). They are kept as provenance for published
|
|
results and are being generalised stage by stage.
|
|
|
|
### Session layout on disk (not in this repo)
|
|
|
|
```
|
|
<session>/
|
|
METHODS.md what was done to this data and what was found
|
|
raw/ exactly what the telescope delivered
|
|
calibrated/ uncompressed calibrated subs
|
|
stacks/masters/ per-filter registered, combined, plate-solved
|
|
stacks/original/ alignment-only baseline, no other processing
|
|
final/ the deliverable renderings
|
|
renderings/ other finished images
|
|
science/figures|catalogues|data|notes
|
|
intermediates/ caches; deletable, regenerated by a re-run
|
|
```
|
|
|
|
## observing/ - in-person plans
|
|
|
|
Field plans worked out from real numbers, revised as a date approaches. The
|
|
reasoning is the part worth keeping, so plans record why a decision was made,
|
|
not only what was decided.
|
|
|
|
## The workflow itself
|
|
|
|
`CLAUDE.md` + `docs/`: the Default Workflow, copied in so the project is
|
|
self-contained. `state/`: project memory - objective, current work, decisions,
|
|
notes.
|