Release Notes

Changes in SDM Schemas are generally organized by the type of schema which was affected:

Data Releases - Data Release and Data Preview schemas, having names that typically start with dp or dr (e.g., dp02_dc2)

Science Pipelines - Schemas which are managed through continuous integration with the Science Pipelines (imsim, hsc)

Solar System Production - Solar System Production schema (e.g., sso_base)

Alert Production and Prompt Products - Alert Production Database (APDB) and Prompt Products Database (PPDB) used in nightly data processing (apdb)

ConsDB - Consolidated Database of Image Metadata schema, having names that start with cdb (e.g., cdb_lsstcam)

Miscellaneous - Catch-all category for changes which are not related to particular schemas, typically involving changes to GitHub workflows, project scripts, etc.

v30.0.11 (2026-08-13)

Data Releases

  • Added CoaddPatches table to DP2 schema (dp2.yaml). (DM-55392)
  • Added mpc_orbits table to DP2 schema. (DM-55438)
  • Hid redundant or unnecessary coord_ra, coord_dec columns. (DM-55508)
  • SSO table changes:
    • Restored current_identifications and numbered_identifications SSO tables, less array-valued columns.
    • Reordered tables for final EDP2 presentation sequence. (DM-55597)
  • DP2 Object column presentation order:
    • Restored ugrizy column order throughout.
    • Moved a small number of general columns to the top. (DM-55607)
  • Reordered ObsTAP columns to make DP1/DP2 distinction more visible and moved some standard columns up in the order for clarity. (DM-55637)
  • DP2 presentation changes:
    • Reordered columns in ForcedSourceOnDiaObject to prioritize the psfDiffFlux.
    • Moved up nDiaSources column in DiaObject to ensure LC length is easily seen. (DM-55661)
  • Removed incorrect description of the DiaSource.snr computation from DP1 and drp_base. (DM-55752)
  • Added overall description text for DP2 schema. (DM-55790)

Science Pipelines

  • IsolatedStarStellarMotions changes:
    • Fixed data types.
    • Updated some UCDs.
    • Removed off-diagonal covariance terms from the principal subset.
    • Documented Gaia DR3 as the reference dataset. (DM-55449)
  • Improvements to drp_base:
    • Added principal flag to columns in DiaObject.
    • Improved flux-related UCDs for DiaObject, DiaSource, and ForcedSourceOnDiaObject.
    • Improved DiaSource count UCDs for DiaObject.
    • Applied band and filter UCDs consistently to all DRP tables and ObsCore. (DM-55661)

v30.0.8 (2026-06-10)

Data Releases

  • Added ShearObject table schema for LSSTCam. (DM-53978)
  • Added WCS-rejection columns to CcdVisit on LSSTCam (probably for DP2) and CI. (DM-54286)
  • Added IsolatedStarStellarMotions schema for LSSTCam. (DM-54418)
  • Renamed psfOriginal_e1/e2 to psfOriginal_g1/g2 in ShearObject to denote the change in convention. (DM-54456)
  • Added columns in ShearObject schema to flag duplicate detections. (DM-54960)

Science Pipelines

  • Removed calib_photometry_reserved from Object table, because it is always false, and added cModel_devFlux and cModel_expFlux unforced free measurements to Object table. (DM-53352)
  • Switched Object shape moments to e1/e2 parameterization. (DM-53886)
  • Removed the exponential/Sersic model x/y and Err columns. Added exponential/Sersic model ra_dec_Cov columns. Added ivoa:ucd tags to exponential/Sersic RA/dec columns. (DM-53940)

v30.0.0 (2026-01-16)

Data Releases

  • Added DP1 schema. (DM-51047)
  • DP0.3 type corrections:
    • Corrected type of DiaSource.ssObjectReassocTime to match the true Postgres type, as it was causing run-time errors.
    • Corrected type of DiaSource.ccdVisitId as well, though this was a harmless inconsistency. (DM-51064)
  • Renamed dp02_v2.yaml to dp01.yaml and reformatted using the felis dump command. (DM-51408)
  • Removed the @id fields from the dp1.yaml schema, which should no longer be needed after updates to Felis. (DM-51410)
  • Changed fits:tunit values to ivoa:unit in DP1 schema. (DM-51411)
  • Assigned an order to the DP1 tables, derived from the DP0.2 order. (DM-51414)
  • Added foreign key relationships to DP1 schema, not including the “visit+detector” relationships. (DM-51415)
  • Propagated DP0.2 table service descriptors to DP1. (DM-51416)
  • Revised descriptions for Object columns and added principal flags. (DM-51417)
  • Added DP1 virtual columns. (DM-51449)
  • Added DP0.2 ObsCore table to main DP0.2 schema. (DM-51478)
  • Removed arraysize overrides from DP1 ObsCore table. (DM-51506)
  • Added DP1 static ObsCore table; configured only for data-int. (DM-51507)
  • Added the DP1 and IVOA ObsCore schemas to the idfprod environment (data.lsst.cloud). (DM-51511)
  • Updated DP1 schema and table descriptions for release. (DM-51541)
  • Removed DP1 static ObsCore table from dp1 schema (It is only in the ivoa.ObsCore table in Qserv now.) (DM-51545)
  • Added obs_title to DP1 ObsCore. (DM-51549)
  • Removed decl columns from DP1 that had long since been declared deprecated. (DM-51559)
  • Added CoaddPatches table. (DM-51560)
  • Added index definitions to the DP1 schema. (DM-51573)
  • Copied tap:principal column metadata from DP0.2 to DP1 for ForcedSource, ForcedSourceOnDiaObject, Visit, and CcdVisit and flagged a few additional principal flags to columns which were not present in DP0.2. (DM-51600)
  • Revised foreign-key descriptions to clarify 1:N relationships. (DM-51604)

Science Pipelines

  • Added starEMedian and starUnNormalizedEMedian to lsstcam.yaml, hsc.yaml, and imsim.yaml. (DM-48316)
  • Changed double columns to float. (DM-49074)
  • Separated NO_DATA from EDGE pixel flag in HSC schema. (DM-49274)
  • Added sky-coordinate moments. (DM-49710)
  • Added trailFluxErr column to DiaSource table in imsim.yaml. (DM-49714)
  • Added epoch columns to Object table. (DM-49727)
  • Propagated noData flags into ForcedSource tables. (DM-49729)
  • Upgraded MySQL to version 8.0.41 for TAP_SCHEMA database. (DM-49876)
  • Changed id to column_id in TAP_SCHEMA columns and added api_created column to the TAP_SCHEMA tables table. (DM-50252)
  • Removed fields from the AP DIAObject and DIASource tables which were unpopulated. (DM-50837)
  • Added glint_trail boolean flag column to imsim.yaml, which will now appear in all DiaSource tables. (DM-50988)
  • Added templateFlux and templateFluxErr to imsim.yaml. (DM-51823)
  • Converted DIA timestamp fields to MJD TAI and renamed DiaSource.time_processed to timeProcessedMjdTai. (DM-52215)
  • Added MultiProFit exponential model fit columns to object table. (DM-52462)
  • Added model_extendedness columns, representing a new, continuous classification for whether an object is compact or extended, with one column per band and one with griz combined. (DM-52667)
  • Removed and renamed columns in the Object table per RFC-1131:
    • Removed GAAP fluxes with apertures >= 1.5 and Optimal.
    • Removed reference centroids (x, y) in pixel coords.
    • Renamed reference centroid flag from xy_flag to coord_flag and moved under coord_ra/coord_dec.
    • Removed per-band centroids in pixel coords.
    • Removed calib fluxes.
    • Removed ixyRound et al. (DM-52922)
  • Added default schema for the LSSTCam DRP pipeline and removed forcedSourceId and forcedSourceOnDiaObjectId from respective schemas. (DM-53027)
  • Applied major update to Solar System table schemas as described in RFC-1138:
    • Affected tables included SSObject, SSSource, mpc_orbits, current_identifications and numbered_identifications.
    • mpc_orbits replaced MPCORB. (DM-53310)
  • Added view_target column to TAP_SCHEMA tables table. (DM-53338)
  • Added major/minor/position angle galaxy model ellipse columns, implementing parts of RFC-1081 and most of RFC-1132. (DM-53442)

Alert Production

  • Added starEMedian and starUnNormalizedEMedian to apdb.yaml. (DM-48316)
  • Removed unpopulated fields from the AP DiaObject and DiaSource tables. (DM-50837)
  • Added glint_trail boolean flag column to apdb.yaml, which will now appear in all DiaSource tables. (DM-50988)
  • To match DP1, removed all MPCORB columns except ssObjectId, mpcH, epoch, a, e, incl, node, peri, and M; added q and t_p to include cometary elements. (DM-51864)
  • Corrected SSSource velocity units from AU to AU/d. (DM-51993)
  • Converted DIA timestamp fields to MJD TAI and renamed them:
    • This changed time_processed and time_withdrawn to timeProcessedMjdTai and timeWithdrawnMjdTai in DiaSource and DiaForcedSource.
    • DiaSource.ssObjectReassocTime became ssObjectReassocTimeMjdTai.
    • Similarly, validityStart and validityEnd became validityStartMjdTai and validityEndMjdTai in DIAObject. (DM-52215)
  • Added a new column validityStartMjdTai to DiaObjectLast which represents the start of the latest validity interval for a diaObjectId. (DM-52827)
  • Updates to Solar System related tables from RFC-1138:
    • The tables affected are SSObject, SSSource, mpc_orbits, current_identifications and numbered_identifications.
    • mpc_orbits replaces the MPCORB table.
    • current_identifications replaces the MPCDESIGMAP tables. (DM-53310)

Consolidated Database

  • Added the Transformed EFD schemas. (DM-43722)
  • Added scheduler_note column to the cdb schemas. (DM-47965)
  • Added can_see_sky column to the cdb schemas. (DM-51051)
  • Added zernikes column to the cdb ccdvisit1_quicklook and visit1_quicklook schema. (DM-51220)
  • New primary key columns were added to the Transformed EFD schemas, modifying the basic structure. IVOA metadata was also added and/or updated. (DM-51362)
  • Added tap:table_index values for ConsDB schemas:
    • Exposure and related tables come first (100 series), followed by visit tables (200 series).
    • Per-CCD versions of the above follow (300 and 400 series).
    • Last are the flexible metadata schemas that are currently unused (and may not ever become public, if they are used; 800 and 900 series). (DM-51439)
  • Added aos_fwhm, donut_blur_fwhm and physical_rotator_angle to exposure consdb table. Added m1m3 glycol temperatures, salindex 112 temperature sensors, fan coil unit temperatures, m2 ring temperatures and compensation offsets to transformed_efd consdb. (DM-51455)
  • Added jitter and image degradation columns for the exposure_quicklook table. (DM-51764)
  • Added guider columns to visit1_quicklook table. (DM-52666)
  • Added even more guider columns to visit1_quicklook table. (OSW-1516)

Miscellaneous

  • Removed dependence on the @id field from the schema browser:
    • Felis was changed to generate IDs by default, so these may not be present in the YAML files. (DM-46240)
  • Moved schema descriptions from browser markdown into YAML files:
    • No custom schema descriptions should be present anymore in the schema browser. (DM-46896)
  • Sorted tables by their ‘tap:table_index’ in the schema browser.
    • Tables without this field were listed alphabetically after those which did. (DM-46989)
  • Added index details section to each table in the schema browser. (DM-48366)
  • The dependency on the sdm_tools repository was removed from pyprojec.toml. The GitHub workflows were updated to install this dependency using pip instead. (DM-49962)
  • Added cdb_lsstcam to the schema browser. (DM-50521)
  • Added --force-unbounded-arraysize to tap-schema/build for forcing VOTable arraysize to ‘*’ on columns by default for variable length string types:
    • This change affects the arraysize values in the TAP_SCHEMA SQL output that is generated for the Docker images.
    • This is a temporary workaround for astropy Issue #18099, which will eventually be reverted once there is a permanent fix. (DM-50914)
  • Added the IVOA ObsCore schema (‘ivoa_obscore.yaml’) to the schema browser. (DM-51557)
  • Added containers for both linux/amd64 and linux/arm64. Added schemas for idfdemo environment. (DM-52731)

v29.0.0 (2025-03-31)

Data Releases

  • Ensured (tract, patch) are always displayed, visually, in that order in DP catalogs. (DM-43996)

Science Pipelines

  • Updated description of extendedness column. (DM-46681)
  • Added dipoleFitAttempted flag. (DM-48106)
  • Separated EDGE and NO_DATA pixel flags. (DM-45621)
  • Removed STREAK mask columns. (DM-46933)
  • Removed calib_detected column from Source table. (DM-46991)
  • Removed localWcs and localPhotoCalib columns from ImSim Sources table. (DM-48117)
  • Added contributor documentation for DRP schemas. (DM-48178)
  • Added multiprofit two-Gaussian PSF and Sersic columns. (DM-48591)

Alert Production

  • Updated description of extendedness column. (DM-46681)
  • Added dipoleFitAttempted flag. (DM-48106)
  • Separated EDGE and NO_DATA pixel flags. (DM-45621)
  • Added nDiaSources column to DiaObjectLast table. (DM-44098)
  • Added column with is_negative flag. (DM-48437)

Consolidated Database

  • Added several missing ConsDB schemas to the schema browser. (DM-46895)
  • Added pixelScale and PSF model delta metric columns. (DM-47002)
  • Updated units in cdb_lsstcomcam.visit1_quicklook. (DM-47044)
  • Added electron UCDs in cdb_lsstcomcam. (DM-47308)
  • Moved mount and jitter columns to exposure_quicklook table. (DM-47443)

Miscellaneous

  • Added Python packaging with support for resource paths. (DM-46273)
  • Improved the schema browser. (DM-41867)
  • Added toggle for table pagination in the schema browser. (DM-46982)
  • Added EUPS and SCons configuration for resource path support. (DM-47069)
  • Moved YAML files into Python source tree. (DM-47147)
  • Fixed problems with GitHub build workflow. (DM-47403)
  • Added workflow to perform schema comparisons and check for changes to deployed schemas. (DM-46158)
  • Added GitHub workflow that runs extra validation checks. (DM-47844)
  • Added contribution guide and pull request template. (DM-48013)
  • Changed the TAP_SCHEMA build to use the new Felis command load-tap-schema. (DM-46957)
  • Added api_created field to tap_schema.schemas table definition. (DM-48979)
  • Added the sdm_tools dependency, which provides a new command line utility for processing schema files, and ported the script for generating the Datalink snippets to the new repository. (DM-41290)
  • Added support for towncrier. (DM-49711)

v28.0.0 (2025-01-23)

Data Releases

  • Added missing primary keys to DP0.2. (DM-43115)
  • Removed DP0.1 from TAP_SCHEMA and DataLink builds. (DM-44884)
  • Fixed lengths and datatypes of string columns. (DM-43946)
  • Added missing precision values on timestamps. (DM-44825)
  • Removed datatype overrides on timestamp columns and specified precision. (DM-44825)
  • Removed length fields and type overrides from timestamp columns. (DM-44059)
  • Removed unused mysql:datatype overrides on string columns. (DM-44637)
  • Removed unused mysql:datatype overrides on numeric columns. (DM-44241)
  • Removed redundant mysql:datatype overrides for Felis numeric types. (DM-43716)
  • Removed redundant mysql:datatype overrides for Felis boolean types. (DM-43958)
  • Removed redundant mysql:datatype overrides for Felis fixed length strings. (DM-43956)

Science Pipelines

  • Fixed lengths and datatypes of string columns. (DM-43946)
  • Migrated from ccdVisitId to (visit, detectors). (DM-42435)
  • Added invalidPsfFlag column. (DM-44167)
  • Removed datatype overrides on timestamp columns and specified precision. (DM-44825)
  • Removed length fields and type overrides from timestamp columns. (DM-44059)
  • Added pixelScale to visitSummary and ccdVisit tables. (DM-44854)
  • Added psfApCorrDelta and psfApFluxDelta metrics to exposure summaries. (DM-37952)
  • Removed unused mysql:datatype overrides on numeric columns. (DM-44241)
  • Added normalized, compensated tophat fluxes to Source tables. (DM-38632)
  • Removed redundant mysql:datatype overrides for Felis numeric types. (DM-43716)
  • Removed redundant mysql:datatype overrides for Felis boolean types. (DM-43958)
  • Removed redundant mysql:datatype overrides for Felis fixed length strings. (DM-43956)

Alert Production

  • Corrected column value fields to use correct type, so 0 instead of "0". (DM-46073)
  • Migrated from ccdVisitId to (visit, detectors). (DM-42435)
  • Added individual flag fields. (DM-41530)
  • Removed placeholder time series feature columns from DiaObject table. (DM-44092)
  • Removed length fields and type overrides from timestamp columns. (DM-44059)
  • Added ra and dec and removed x and y from DiaForcedSource. (DM-44470)
  • Added psfApCorrDelta and psfApFluxDelta metrics to exposure summaries. (DM-37952)
  • Removed unused mysql:datatype overrides on numeric columns. (DM-44241)
  • Removed redundant mysql:datatype overrides for Felis numeric types. (DM-43716)
  • Removed redundant mysql:datatype overrides for Felis fixed length strings. (DM-43956)

Consolidated Database

  • Implemented multi-column primary key wth day_obs and seq_num. (DM-46073)
  • Added exposure_quicklook table to LATISS and ComCam. (DM-46628)
  • Added StarTracker schemas. (DM-45893)
  • Added mount jitter to LATISS and ComCam. (DM-46009)
  • Added magnitude limit columns from exposure summary stats. (DM-45573)
  • Added postisr_pixel_median columns. (DM-45848)
  • Made constraint names unique. (DM-45623)
  • Removed datatype overrides on timestamp columns and specified precision. (DM-44825)
  • Added vignette columns. (DM-44967)
  • Updated descriptions for eff_time_*_scale columns. (DM-44958)
  • Updated descriptions for eff_time metric columns. (DM-44955)
  • Added a representative ConsDB schema. (DM-44161)
  • Added CcdExposure tables and LSSTComCamSim schema. (DM-44429)
  • Added visit1 and ccdvisit1 tables. (DM-44489)

Miscellaneous

  • Changed the TAP_SCHEMA workflow to set the schema index with a command line argument to Felis. (DM-42935)
  • Added redundant datatype checks to validation workflow. (DM-44058)
  • Deleted UWS creation script from tap-schema/sql directory. (DM-45099)
  • Removed DP0.1 from TAP_SCHEMA on IDF int and dev. (DM-44884)
  • Added database tests to GitHub workflows. (DM-44158)
  • Updated the schema browser to use LTD. (DM-41310)

SDM Schemas is maintained by Rubin Observatory.