Processing and analysis code for remote-telescope imaging sessions
The scripts that processed the NGC 5128 session of 2026-07-21 previously lived inside the data directory and addressed it with absolute paths. Code and data are now separated: the code lives here, and a session is located at runtime through the ASTRO_SESSION environment variable. layout.py is what makes that work. It maps a FILENAME to the subdirectory that file belongs in, using the same rules the session directories are organised with, so a script can go on asking for 'master-Red.fit' or '_stars.npz' without any call site knowing the directory structure. Anything unrecognised resolves to the session root, which is visible and correctable rather than silently wrong. restructure.py reorganises a flat session directory into that layout. It is idempotent and dry-run by default. The 50 session scripts are kept as they were run rather than tidied into a library. They were written in sequence as the work went along, several of them by parallel agents, and they show it - but they are the honest provenance of a published set of results, and the productionised pipeline should be able to reproduce those results exactly. Verified before committing: all 51 files compile without warnings, and verify_core.py, closeup.py and triptych.py were run end to end against the reorganised session, correctly finding inputs across calibrated/, stacks/masters/ and final/ and writing outputs back to the right places.
This commit is contained in:
commit
5286a2e81b
53 changed files with 8820 additions and 0 deletions
108
session-scripts/layout.py
Normal file
108
session-scripts/layout.py
Normal file
|
|
@ -0,0 +1,108 @@
|
|||
"""Where every file of a processed session lives.
|
||||
|
||||
The scripts used to sit inside the data directory and address it with absolute
|
||||
paths. Now that they live in a repository and the data is organised into named
|
||||
subdirectories, they need one place that knows the mapping - this module.
|
||||
|
||||
Two rules make it work:
|
||||
|
||||
* The session root comes from the ASTRO_SESSION environment variable, so the
|
||||
same code runs against any session without editing. It falls back to the
|
||||
NGC 5128 session these scripts were written against.
|
||||
* A file's subdirectory is derived from its NAME, using the same rules the
|
||||
directory was organised with. That means a script can go on asking for
|
||||
"master-Red.fit" or "_stars.npz" and get the right path, without every call
|
||||
site having to learn the layout.
|
||||
|
||||
Anything unrecognised lands at the session root, which is visible and easy to
|
||||
correct rather than silently wrong.
|
||||
"""
|
||||
import os
|
||||
|
||||
SESSION = os.environ.get(
|
||||
"ASTRO_SESSION",
|
||||
r"C:\Users\lhorrocks-barlow\Downloads\NGC5128\20260721")
|
||||
|
||||
# Subdirectories, also created on demand by path().
|
||||
RAW = "raw"
|
||||
CALIBRATED = "calibrated"
|
||||
MASTERS = os.path.join("stacks", "masters")
|
||||
ORIGINAL = os.path.join("stacks", "original")
|
||||
FINAL = "final"
|
||||
RENDERINGS = "renderings"
|
||||
FIGURES = os.path.join("science", "figures")
|
||||
CATALOGUES = os.path.join("science", "catalogues")
|
||||
SCIENCE_DATA = os.path.join("science", "data")
|
||||
NOTES = os.path.join("science", "notes")
|
||||
INTERMEDIATES = "intermediates"
|
||||
|
||||
# Names that are directories in their own right, not files.
|
||||
_DIRS = {"final": FINAL, "original": ORIGINAL, "raw": RAW,
|
||||
"calibrated": CALIBRATED, "renderings": RENDERINGS,
|
||||
"_dss": os.path.join(INTERMEDIATES, "_dss")}
|
||||
|
||||
|
||||
def subdir(name):
|
||||
"""The subdirectory a given filename belongs in."""
|
||||
low = name.lower()
|
||||
if name in _DIRS:
|
||||
return _DIRS[name]
|
||||
if name.startswith("_"):
|
||||
return INTERMEDIATES
|
||||
if low.endswith(".zip") or (low.startswith("jpeg-") and
|
||||
low.endswith(".jpg")):
|
||||
return RAW
|
||||
if low.startswith("calibrated-") and low.endswith((".fit", ".tif")):
|
||||
return CALIBRATED
|
||||
if low.startswith("raw-") and low.endswith((".fit", ".tif")):
|
||||
return RAW
|
||||
if low.startswith("master-") and low.endswith(".fit"):
|
||||
return MASTERS
|
||||
if low.startswith("original-"):
|
||||
return ORIGINAL
|
||||
if "final" in low and low.endswith((".png", ".tif", ".jpg")):
|
||||
return FINAL
|
||||
if low.startswith("notes-") and low.endswith(".md"):
|
||||
return NOTES
|
||||
if low.endswith(".csv"):
|
||||
return CATALOGUES
|
||||
if low.startswith("sb-") and low.endswith((".fits", ".npy", ".txt")):
|
||||
return SCIENCE_DATA
|
||||
if low.endswith(".png") and any(f"-{p}-" in low for p in ("gc", "sb", "mo")):
|
||||
return FIGURES
|
||||
if low.startswith(("ngc5128-lrgb", "ngc5128-img-")):
|
||||
return RENDERINGS
|
||||
if low.endswith(".md"):
|
||||
return ""
|
||||
return ""
|
||||
|
||||
|
||||
def path(*parts, make=True):
|
||||
"""Absolute path for a session file, addressed by name alone.
|
||||
|
||||
Extra leading components are accepted and ignored, so calls written
|
||||
against the old flat layout - path("stacked", "_stars.npz") - still land
|
||||
correctly. The last component is the one that decides.
|
||||
"""
|
||||
name = parts[-1]
|
||||
full = os.path.join(SESSION, subdir(name), name)
|
||||
if make:
|
||||
os.makedirs(os.path.dirname(full), exist_ok=True)
|
||||
return full
|
||||
|
||||
|
||||
def describe():
|
||||
lines = [f"session root: {SESSION}"]
|
||||
for label, d in (("raw", RAW), ("calibrated", CALIBRATED),
|
||||
("masters", MASTERS), ("original stacks", ORIGINAL),
|
||||
("final images", FINAL), ("other renderings", RENDERINGS),
|
||||
("science figures", FIGURES),
|
||||
("science catalogues", CATALOGUES),
|
||||
("science data", SCIENCE_DATA), ("notes", NOTES),
|
||||
("intermediates", INTERMEDIATES)):
|
||||
lines.append(f" {label:20s} {d}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(describe())
|
||||
Loading…
Add table
Add a link
Reference in a new issue