desitarget Change Log

5.2.1 (unreleased)

  • No changes yet.

5.2.0 (2026-06-09)

  • Make target/sky subpriorities depend on Data Release [PR #888].

5.1.0 (2026-06-04)

  • Final updates to run targeting files on DR11 [PR #887]. Includes:
    • New RELEASE number/schema for SAFE / BAD_SKY skies.

      • They now get assigned a release number of RELEASE//1000.

      • But only if RELEASE//1000 > 10 for backward compatibility.

      • Prevents TARGETID duplication with DR9 SAFE locations.

      • As SAFE / BAD_SKY skies used to receive RELEASE==0.

    • Update supplemental skies to use Gaia DR3.

      • This guarantees SKY targets won’t duplicate a DR9 TARGETID.

    • Add gnu parallel for supplemental skies in bundling function.

    • Functionality to only add DR11 targets to MTLs in DR11 bricks.

    • New code to turn off DR9 targets in DR11 bricks.

  • Update the default set of imaging mask bits for DR11 [PR #886]:

    • Now always include new RESOLVED, MCLOUDS and WISE_GAIA bits.

5.0.0 (2026-05-28)

  • Update code to process targets and skies from DR11 [PR #885]:

    • Mostly handles updates to the DR11 imaging data model.

    • In particular, adds new RELEASE numbers for DR11/south.

    • Also updates the FITS extension from which the blob maps are read.

    • And adds some slurm scripts to improve batching using GNU parallel.

  • Correct the concatenation of MTL ledgers in read_mtl_in_hp [PR #884].

  • Catch further reprocessing bugs related to PR #879 [PR #882]:

    • PR #879 worked as buggy tiles were batched with other tiles.

    • For individual buggy 1B tiles, reprocessing can still fail.

    • This PR catches 2 new corner cases for such individual tiles.

    • The case caught in PR #879.

    • The case for which a 1B tile contains no secondary targets.

  • Add code to make a DR9/DR11 bricks file [PR #881].

4.7.2 (2026-04-29)

  • Revert ToO changes between desitarget/4.5.0 and 4.6.0 [PR #880].

    • This will allow a single tag for both ToO and MTL updates.

  • Catch reprocessing bug where 1B tile list can change size [PR #879].

    • The input tile list can be bigger than the output tile list.

    • This is because not every 1B tile changes a 1A target.

    • Notably M31 BRIGHT1B tiles which don’t overlap any BRIGHT tiles.

    • We don’t write any tile information when ext=True anyway.

4.7.1 (2026-04-23)

  • Catch case where NUMOBS_MORE drops to -1 for LyA quasars [PR #878].

4.7.0 (2026-04-20)

  • Add new, main4 secondary targets [PR #877]. Includes:

    • Add BRIGHT1B and DARK1B secondary targets.

    • Allow and add BACKUP secondary targets.

    • Add DARK and BRIGHT targets for the new DR11 imaging area.

      • Likely include by adding 1B targets to DARK/BRIGHT tiles.

      • See related fiberassign issue #503.

    • Facilitate running the MTL loop for secondary 1B targets.

      • Notably if we’ve already observed corresponding primary tiles.

    • Also includes new routines to make “lightweight” Gaia files.

      • These files can be used for fast matching to bright stars.

      • Routines to read and match to the lightweight files also added.

      • Can be used to flag secondaries that are close to bright stars.

  • Circumvent warnings when running with Python >3.12 [PR #870].

4.6.0 (2026-01-30)

  • Spin currently observable ToOs off into a smaller file [PR #872].

4.5.0 (2025-12-16)

  • Allow HI priority FIBER ToOs with minimal oversight [PR #869].

4.4.0 (2025-11-24)

Note: There are no significant code changes compared to 4.3.0. The only changes are package metadata to support pip installation, etc. This tag the first version of desitarget to appear on PyPI.

  • Final preparation of package metadata for upload to PyPI [PR #868].

  • Add pyproject.toml file and update other top-level files [PR #864].

4.3.0 (2025-11-04)

  • Set up the capability to veto targets dynamically in svn [PR #863].

    • Initially for the M31 BRIGHT1B program.

  • Smarter read/merge of files with different data models [PR #861].

    • This deprecates [PR #860].

    • We don’t actually need to merge the M31 and C-19 stream programs.

    • Tiles can touch HEALPixels in these disparate programs…

    • …but can ever touch targets from both of these programs…

    • …as the programs are so widely separated on the sky.

    • So, we can allow a read of both programs with careful checks on…

    • …whether targets are touched by a tile/HEALPixel rather than…

    • …whether HEALPixels are touched by a tile/HEALPixel.

    • This PR is also backward-compatible, where [PR #860] was not.

4.2.0 (2025-10-08)

  • Version 4.2.0 was deprecated by [PR #861] and should never be used!!

  • Allow target files with different data models to be read [PR #860].

    • Combined in a HEALPixel or a tile.

    • But only for the very special case of M31-like target files.

4.1.1 (2025-10-06)

4.1.0 (2025-09-29)

  • Update read_one_mtl_ledger() to read gzipped files [PR #858].

  • Add the BRIGHT1B targets for the C-19 stream in the SGC [PR #857].

4.0.0 (2025-09-19)

  • Fix ordering bug when reading two MTL ledgers [PR #856].

    • Note that this fix is not backwards compatible.

    • Specifically for the io.read_two_mtl_ledgers() function.

  • Set SKYHEALPIXS_DIR in etc/desitarget.module [PR #853].

3.6.1 (2025-08-11)

  • 3.6.0 facilitated combining M31 targets and calibrations [PR #851].

    • But these can’t be combined in the MTLs as they have different DRs.

    • This PR allows mixed DRs for MTLs when M31 targets are present.

3.6.0 (2025-08-08)

  • Add BACKUP standards to the BRIGHT1B M31/M33 target files [PR #850].

3.5.0 (2025-08-07)

  • Code to generate BRIGHT1B M31/M33 targets and update MTLs [PR #849].

3.4.2 (2025-07-21)

  • Another bug fix related to PR #845 [PR #848].

    • Reprocessed quasars can have two UNOBS entries in a ledger.

3.4.1 (2025-07-21)

  • Bug fix related to PR #845 [PR #847].

    • The code crashed for MTLs that don’t contain any quasars.

    • As it didn’t know how to append zero information to a ledger.

3.4.0 (2025-07-21)

  • Facilitate running MTL updates for the BACKUP program [PR #846].

  • Change LyA decisions for DESI 1B phase [PR #845].
    • Include function to add 2 observations for every LyA QSO.

      • Also sets priority for these QSOs to most recent LyA-like one.

    • MTL logic sets 6 observations for ELG targets that are LyA QSOs.

      • And sets priority to just below a QSO target that is a LyA QSO.

      • Introduces new priority level LY_1B_ELG.

3.3.0 (2025-06-21)

  • Expand program string in mtl-done file to allow BRIGHT1B [PR #843].

  • Add numpy 2 compatibility; remove mockmaker.py/lyazcat.py [PR #838].

3.2.0 (2025-06-08)

  • This is the last tag that will include mockmaker.py and lyazcat.py.

  • Update BRIGHT1B data model to better match DARK1B [PR #842].

3.1.0 (2025-06-05)

  • Finalize target files, initial MTL creation for BRIGHT1B [PR #841].

  • Add target selection for streams and dSphs for BRIGHT1B [PR #835].

    • Also updates mtl.check_archiving() for 1B ledger checks.

3.0.0 (2025-04-25)

  • No code changes, moving to 3.0.0 to tag first 1B targets.

2.9.0 (2025-03-14)

Major:

  • Implement multi-MTL procedure for DESI 1B [PR #833]

Minor:

  • Fix installation when using desiutil/3.5.0, dropping support of python setup.py test as a side-effect [PR #832]

  • Update readthedocs configuration [PR #834]

2.8.0 (2024-08-23)

  • Implement multi-MTL procedure for DESI 1B [PR #833].

    • New dark-time targets for DESI 1B.

    • New ledger types for DARK1B and BRIGHT1B.

    • Update io.read_mtl_ledger() to handle two sets of ledgers.

    • Upate readers that wrap io.read_mtl_ledger().

    • Add HIGHEST/WANTED/CONTAINS columns in multi-MTL mode.

    • MTL loop updates Main Survey and 1B ledgers for 1B tiles.

  • Attempt to fix coverage tests [PR #831]

    • Didn’t appear to fully work, but partial progress so merged.

  • Fix pkg_resources/scipy.ndimage/pytest deprecations [PR #829].

  • Change match routines to test input arrays for duplicates [PR #828].
  • Add GAIA_ prefix to PHOT_*_MEAN_MAG column names [PR #827].

    • So GD1/streams files conform to downstream data model.

  • Module file removed vars that are now defined in desimodules: DUST_DIR, QN_MODEL_FILE, SQ_MODEL_FILE [PR #826].

  • Add different Table reader options to mtl.update_ledger() [PR #825].

  • Add function to check format of secondary target files [PR #824].

  • When resolving, only set bits for northern cuts/imaging [PR #823].

    • And, similarly, for southern cuts in southern imaging.

    • Addresses issue #821.

  • Allow different Table reader options when reading MTLs [PR #822].

  • Function to match RA/Dec positions to Main Survey targets [PR #820].

  • Bump astropy from 5.0 to 5.3.3 (dependabot) [PR #815].

  • Add the GD1 dark matter streams program [PR #814]. * Substantial new code in the streams directory. * Addresses issue #812. * General unit tests fixes, including one introduced in PR #823

2.7.0 (2023-12-05)

  • Option to write all columns for Gaia DR3 FITS files [PR #809].

    • Also adds a new higher-priority bit for the BACKUP DIB program.

  • Add check that archived tiles match tiles-specstatus file [PR #808].

  • Reproducible output from io.read_targets_in_hp() [PR #806].

  • Option to read subset of columns from HEALPix-split MTLs [PR #805].

    • Specifically in io.read_mtl_in_hp().

    • Addresses issue #802.

    • Partial fix for issue #804.

    • Also address some deprecation warnings.

2.6.1 (2023-08-23)

  • Fix bug where empty override MTLs can alter data types [PR #803].

  • Updates to run targets for Legacy Surveys DR10 [PR #801]. Includes:

    • Add RELEASE number of 10000 (a “southern” release).

    • DR10 has REF_CAT == ‘GE’ for Gaia EDR3 instead of ‘G2.

    • Update data model to reflect altered columns in DR10.

    • Parse both drXX and drX strings in output filenames.

2.6.0 (2022-12-08)

  • Updates to run random catalogs for Legacy Surveys DR10 [PR #800].

    • Also stop applying override ledgers once NUMOVERRIDE is zero.

  • Restore coverage for svX code [PR #799].

  • Address deprecated np.int and np.float dependencies [PR #797].

  • Functionality to download and reformat files for Gaia DR3 [PR #796].

2.5.0 (2022-04-22)

  • Better functionality for fiber-based ToOs and overrides [PR #795]:

    • Append new ToOs to the ledger rather than fully overwriting it.

    • Create a separate ToO-fiber file for TOO_TYPE==FIBER targets.

    • Don’t allow high-priority fiber-override ToOs in the Main Survey.

    • Create new mtl-done files to log when MTL overrides were forced.

    • When forcing MTL overrides, always process all ledgers:

      • This makes reproducibility easier to track for alt MTL ledgers.

  • Added a notebook about running Main Survey targeting [PR #791].

    • To accompany the DESI target selection pipeline paper.

2.4.0 (2022-01-20)

  • A couple of MTL maintenance updates [PR #789]. Includes:

    • Turn off BACKUP MTL processing for the Main Survey (for now).

    • Allow the file format for override ledgers to be forced.

  • Restore select_mock_targets to a working state [PR #788].

  • Add optional [grz]fiberflux arguments to the ELG, LRG, and BGS color-cut selection functions to support the desisim template-generating code (see desisim/PR #556) [PR #786].

  • Extra functionality for splitting randoms by HEALPixel [PR #785]:

    • Correctly process catalogs with an existing HPXPIXEL column.

  • Functionality to split random catalogs by HEALPixel [PR #783].

    • Allows the io/reading utilities to be used on the resulting files.

  • More accurately round pixel coordinates for randoms [PR #782].

    • Alters brick-based values for randoms (e.g. NOBS/MASKBITS/PSFSIZE).

2.3.0 (2021-12-14)

  • Fix ragged array warning in geomask.circle_boundaries() [PR #781]:
  • Finalize MTL logic for reprocessed tiles [PR #780]. Logic is:
    • Assemble all previous observations that touch a reprocessed tile.

      • Include any new, reprocessed observations.

      • Find the most recent for unique TILEID + TARGETID.

      • Store these in a redshift catalog (an “all-zcat”).

    • Determine the order in which tiles were originally processed.

    • Loop through the tiles in this original order.

      • Start with the UNOBS state.

      • Update the UNOBS state with entries in the “all-zcat”.

      • Recover the final state for each TARGETID.

      • Add the progression, WITH BAD observations, to the ledgers.

    • Also includes:

      • Mock-up (unique) TIMESTAMPs instead of delaying the code.

      • Deprecate numobsfromzcat as a user-specified option.

        • as we now always retrieve NUMOBS from the ledger.

2.2.1 (2021-11-22)

  • Small modification on skyhealpixs.py [PR #776]:

    • Adjust the per-healpix file name.

    • Set (nside, nest) arguments, defaulting to (64, True).

2.2.0 (2021-11-21)

  • Add SCND_TARGET for backup targets if nosec is passed [PR #775].

  • Add another MWS backup object type BACKUP_GIANT_LOP. The default BACKUP_GIANT category is now downsampled with galactic latitude to avoid having large densities [PR #772].

  • Change backup target priorities to be between 15-30 to be able to place new object types in between [PR #772].

  • Use Gaia to look up good sky positions for stuck fibers [PR #771].

2.1.0 (2021-11-16)

  • First steps towards MTL for reprocessed tiles [PR #774]. Includes:

    • Finding reprocessed tiles using ARCHIVEDATE and TIMESTAMP.

    • Adding a --reprocess option to the run_mtl_loop script.

      • MTL will only reprocess when forced to do so.

    • Keep reprocessing completely separate from “standard” MTL looping.

      • So the approaches can be run independently.

    • Looser error checking because we may be reprocessing piecemeal.

      • i.e. zdone could change from true to false for some tiles.

    • The skeleton of the logic for updating reprocessed targets.

      • In the mtl.reprocess_ledger() function.

      • This will need tested and updated in a future PR.

2.0.0 (2021-11-11)

  • Add ARCHIVEDATE to the mtl done files and data model [PR #773].

    • Also change the type of ZDATE to int64.

    • These changes will not generally be backward compatible for MTL.

  • Fix some typos in variables’ names in the targeting cuts [PR #770]. All of those likely never were triggered in production.

  • Find MTL-processed tiles that don’t overlap future tiles [PR #768]:

    • Add code to purge such tiles from the MTL done files and ledgers.

    • Also improve reading headers and header values from .ecsv files.

    • Also update GAIA EDR3 files to include RA/Dec errors and REF_EPOCH.

  • Update targetmask and cuts for backup program [PR #766]:

    • Matches description in backup program document.

  • Also use the ops/tiles-specstatus.ecsv tile file for SV [PR #765].

  • Fix a few variable name typos in the target selection code [PR #770]. All of those likely never were triggered in production.

1.3.0 (2021-09-20)

  • Add secondaries to MTL ledgers without merging [PR #764]:
    • Fix a bug reading ecsv headers with io.read_ecsv_header():

      • Now strictly extracts dictionaries after the meta keyword.

    • Add new targeting bits for RR_LYRAE and MWS_FAINT_*.

    • Update secondary.select_secondary() to ignore primaries:

      • Every target is effectively treated as an OVERRIDE target.

      • Adds a TARGETID based on main2 to prevent duplicates.

    • Add a new add_secondary_no_merge script.

    • Functionality to append to ledgers in io.write_mtl().

    • Similarly, add an append option to make_initial_ledger script.

  • Functionality to override MTL ledger entries [PR #763]. Includes:

    • add_to_override_ledgers to create or expand override ledgers.

    • force_mtl_overrides to force overrides into the MTL ledgers.

    • Processing overrides automatically as part of the MTL loop.

    • Override ledgers can be read using MTL utilities in desitarget.io.

  • Add a TIMESTAMP to the Main Survey ToO Ledgers [PR #761].

  • Add POORDATA bit 11 to zwarn_mask for consistency with redrock [PR #757].

1.2.2 (2021-07-08)

  • No code changes. We tagged 1.2.2 due to a glitch while tagging 1.2.1.

    • 1.2.1 exists on GitHub, but not for local developer updates.

    • Do NOT use 1.2.1 for any purpose.

    • ALWAYS use 1.2.2 instead.

1.2.1 (2021-07-08)

  • Speed-ups and bug-fixes for I/O [PR #753]. Includes:
    • Read all types of MTL ledgers, including SV3 and Main Survey 1.0.0.
    • Implement faster version of io.read_targets_in_tiles().

      • Speed-ups are in new io.read_targets_in_tiles_quick().

    • Implement faster version of mtl.inflate_ledger().

      • Speed-ups are in new mtl.match_ledger_to_targets().

1.2.0 (2021-06-19)

  • Update the data model for Main Survey MTL [PR #755]. Includes:
    • Read all types of MTL ledgers, including SV3 and Main Survey 1.0.0.
    • Change the location of the “ZTILE” (zdone=True/False) file:

      • For sv it was $ZCAT_DIR/tiles.csv

      • For main it will be $MTL_DIR/../ops/tiles-specstatus.ecsv

    • A workaround to process SV3 files that have Main-like fibermaps.

  • Add new ZWARN bits BAD_SPECQA and BAD_PETALQA [PR #754].

  • make_zqso_files more robust to individual tile,night,petal problems while still processing the others [PR #752].

  • Add final QSO decision logic to full Main Survey MTL loop [PR #751].

  • Make creation of zqso catalogs robust to missing columns [PR #750].

1.1.1 (2021-05-29)

  • Turn off the MWS_FAINT target classes [PR #748]. Includes:

    • Don’t set any of the MWS_FAINT target bits.

    • No need to exempt MWS_FAINT from forced ledger TIMESTAMPs.

    • Remove special logic for merging MWS_FAINT with secondaries.

1.1.0 (2021-05-29)

  • Don’t allow MWS_FAINT classes to affect secondaries [PR #746].

  • Set up the end-to-end MTL Main Survey loop [PR #744]. Includes:

    • mtl-done-tiles file TIMESTAMP is later than any ledger entry.

    • Read the zcats from the zqso files instead of making a “backstop”.

    • Add IS_QSO_QN column to the initial ledgers.

      Again, this is not backwards-compatible to version 1.0.0.

  • override_subpriority support duplicate input TARGETID from secondaries [PR #743].

  • Support reading mtl 1.0.0 format with different columns [PR #742].

  • Enable optional subpriority overrides [PR #740, PR #741].

  • Allow initial ledgers to use a preordained timestamp [PR #739].

    • MWS_FAINT targets can be exempted from this timestamp.

    • Also change data model for initial ledgers:

      • ZS and ZINFO are replaced by Z_QN

      • This is not backwards-compatible to version 1.0.0

    • Fix PR #734 bug where hpxlist was used in write_secondary.

  • Ensure fixed order of input files for reproducible outputs [PR #738].

  • Refactor data model and I/O for the QSO zcats [PR #737]. Includes:

    • New directory structures.

    • Allow filenames to be passed instead of TILE/NIGHT combinations.

    • Always run QuasarNP (QN) by default.

    • Update which redrock columns are populated in the output files.

    • Only include output columns for afterburners that are run.

  • Add option to use Gaia EDR3 when selecting GFAs [PR #734]. Also:
    • Initialize SUBPRIORITY with better random seeds.

      • This is crucial when parallelizing across HEALPixels.

    • Document that the io.write_ routines alter SUBPRIORITY.

      • Plus, add a keyword to turn that overwriting behavior off.

    • Add leq kwarg when reading ledgers with a specific isodate.

      • Allows ledger entries to be read BEFORE or ON that date.

      • Supplements the default of reading STRICTLY BEFORE that date.

  • Fix bug leading to zero MWS_FAINT targets [PR #733].

1.0.1 (2021-05-14)

  • Add $SKYBRICKS_DIR to module config [PR #732].

    • Also, make more portable by using $DESI_ROOT.

1.0.0 (2021-05-12)

  • Near-final Main Survey MTL logic for repeats [PR #731]. Includes:
    • All MWS targets are observed twice before dropping to DONE.

      • The repeat is at an interstitial, relatively low priority.

      • Except MWS_BHB, which is repeated at relatively high priority.

    • All QSO targets are observed twice before dropping to DONE.

      • The repeat is at an interstitial, relatively low priority.

      • Except z > 1.6 quasars, which receive 4 observations:

        • QSOs at z > 2.1 are repeated at high priority.

        • QSOs at 1.6 < z < 2.1 repeated at relatively low priority.

      • Same logic is applied for secondaries that have flavor QSO.

    • All other primaries are observed once before dropping to DONE.

    • Expand TARGET_STATE to include full bit-names for all targets.

      • Previously, BGS, MWS, secondary only had “BGS”, “MWS”, “SCND”.

    • Add some extra informational redshift columns in the ledgers:

      • ZS intended to hold RR for redrock, QN for QuasarNET, etc.

      • ZINFO intended as a catch-all for algorithmic updates.

    • Also fix a bug when substituting Gaia EDR3 values into the sweeps.

      • REF_EPOCH wasn’t being written.

  • Update desitarget.skybricks to support skybricks/v3 [PR #730].

    • The v3 format uses fits.fz files instead of .fits.gz.

0.58.0 (2021-05-09)

  • Update Main Survey bit-mask information [PR #729]. Includes:

    • Add new secondary targets (Globular Clusters and Dwarfs).

    • Update priorities and numobs for each bit.

    • Debug code that sets which secondaries can override MWS targets.

  • Update PR #723 to fix a transcription bug [PR #728].

  • Clean up the QSO code for the Main Survey [PR #727]. Includes:

    • Remove QSO selection code for data releases prior to DR9.

    • Remove code that selects high-redshift quasars (QSO_HIZ).

    • Also, change the initial priorities for some of the ELG classes:

      • ELG_VLO is now 3000, ELG_LOP is now 3100.

  • Update the ELG/LRG code for the Main Survey [PR #726]. Includes:

    • Deprecate the LRG_LOWDENS targeting bit. It was never used.

    • Upweight 10% of the “filler” ELG sample to the LRG priority.

  • New BGS faint limits to obtain a target density of ~1400 [PR #725].

    • new limits are r< 20.175 for DECaLS and r<20.22 for BASS/MzLS.

  • Add utility functions decode/encode_negative_targetid(ra,dec,group) unique to at least 2 milliarcsec [PR #724].

  • Update baseline LRG selection [PR #723]. Changes from SV3 include:

    • Change the zfiber faint limit from 21.7 to 21.6.

    • Change the low-z limit from z>0.3 to z>0.4.

    • Change the overall density from 800/sq.deg. to 600/sq.deg.

    • Remove the LRG_LOWDENS target bit.

  • Add desispec.skybricks to lookup ra,dec sky locations [PR #722].

  • Update MWS cuts to Gaia EDR3 [PR #720]. Includes:

    • AEN stellarity cut now 2 (previously 3).

    • Parallax floor now 0.3 mas (previously 1 mas).

    • RED/BROAD proper motion split now a function of magnitude.

    • ASTROMETRIC_PARAMS_SOLVED checks account for value in EDR3.

  • Add MWS-FAINT-RED, MWS-FAINT-BLUE Main Survey classes [PR #719].

  • Prepare the Main Survey cuts and bit-masks [PR #718]. Includes:

    • Transfer the SV3 cuts and extra SV3 bits into the Main Survey files.

    • Include the full slate of secondary target bits for the Main Survey.

    • Add formalism to up the priority of some ELGs to the LRG priority.

    • Include a new “filler” ELG class (ELG_VLO).

    • Remove any individual references to a GRAY program for bits.

    • Increase the default density of sky fibers by a factor of 4.

    • Add a kwarg to restore the initial ledger state when reading MTLs.

    • Speed up the unit test for selecting targets in caps/boxes/pixels.

    • Fix a bug where WD standards weren’t being selected in BACKUP tiles.

  • Update LyA QSO zcats with MgII absorption redshifts [PR #717].

  • Explicitly permit secondary bits that can drive updates to MWS primary targets by adding an updatemws flag to targetmask entries [PR #716].

  • Add EDR3 options to code that writes Gaia-matched sweeps [PR #715].

    • Also add gaiasub option when selecting targets, which is used to swap EDR3 proper motions/parallaxes for values in sweeps files.

  • New function and bin script to make QSO redshift catalogs [PR #714].

    • Incorporates functionality from QuasarNET and SQUEzE.

0.57.2 (2021-04-18)

  • Update data model of MTL ledgers for the Main Survey [PR #712]:

    • Express the TIMESTAMP in strict ISO format.

    • Ensure TARGET_STATE is a long enough string for all states.

    • Allow new columns to be easily added to the zcat/MTL ledgers.

      • In preparation for SQUEzE, QuasarNET, etc.

    • Don’t assume first 500 fibermap rows are unique targets for a petal.

    • Simplify the data model for ToO ledgers and default to .ecsv format.

      • In preparation for committing ToO ledgers to svn.

  • Fix new ZWARN unit test from PR #710 [PR #711].

  • General improvements to MTL functionality [PR #710]. Includes:

    • Significant speed-up of mtl.inflate_ledger().

    • Unit test to compare the desitarget ZWARN bit-mask to redrock.

0.57.1 (2021-04-07)

  • Late-breaking updates to MTL for SV3 [PR #709]:
    • Never run the secondary ledgers for BACKUP targets.

      • Because they have no secondary ledgers.

    • Force anything with NUMOBS_INIT = 9 to NUMOBS_INIT = 3.

      • As we made a late decision to have 3 passes rather than 9.

      • The first ledger row will reflect 9 to match the target files.

        • Subsequent rows will change to NUMOBS_INIT = 3.

        • And NUMOBS_MORE will appropriately drop to 2.

  • When making a zcat, update ZWARN using DELTACHI2 [PR #707]:

    • Flag ZWARN for all targets with DELTACHI2 < 25.

    • Also flag BGS targets in bright-time with DELTACHI2 < 40.

0.57.0 (2021-04-04)

  • Updates to MTL schema for SV3 [PR #706]. Includes:

    • Add priorities and initial numbers of observations for SV3.

    • Ensure targets that are both of ELG/QSO revert to QSO if z > 1.6.

    • Turn off the “unlimited BGS observations” Main Survey code for SV3.

    • Add the ZWARN bitmask to the desitarget bitmask yaml file.

    • Fix a bug where SV2 secondaries didn’t have a flavor property.

      • flavor in this context is, e.g., SPARE, DEDICATED.

      • Just caught this now as we didn’t have secondaries for SV2.

    • Don’t process observations if ZWARN includes NODATA.

      • Such observations are now completely ignored in the MTL loop.

    • Create a system to decrement priorities for MORE_ZWARN sources.

      • The decrement amount can be controlled by a bit-property.

      • This property is called ZWARN_DECREMENT.

  • Updates to BGS for SV3 [PR #705].

    • Updating the BGS bright limit to r > 12 & rfibertotmag > 15.

    • Slightly different faint limits for BASS/MzLS and DECaLS.

      • For both r and rfibermag.

  • Bright limit (GAIA_G > 15) AND (ZFIBERTOT > 15) for MWS [PR #704].

    • Applied to all MWS targets (BROAD/NEARBY/WD/BHB).

  • Further updates to the LRG selection for SV3 [PR #703]. Includes:

    • Change bright-end cut to zfibertot > 17.5 (instead of > 16).

    • Add low-density (600-per-sq.-deg.) LRG sample (LRG_LOWDENS).

      • Bit is informational as LRG_LOWDENS is a subset of LRG.

    • Update the intersphinx URLs to fix the online documentation builds.

0.56.0 (2021-03-31)

  • Update the MTL loop to work for secondary ledgers for sv3 [PR #702]:
    • Add tile file to check secondary processing tracks with primaries.

      • called scnd-mtl-done-tiles.ecsv.

    • Default to running secondary ledgers whenever primaries are run.

      • i.e. specifically in the run_mtl_loop script.

    • Catch some special cases for secondaries.

      • e.g. secondary QSOs should update like primary QSOs.

    • Set NUMOBS to more sensible numbers for secondary targets.

      • let NUMOBS_MORE drop to zero and PRIORITY to DONE.

  • Revert ELG_LOP back to being its own target class [PR #701].

    • But maintain the correct behavior for ELG_HIP when making MTLs.

  • Some bug fixes for SV3 [PR #700]. Includes:

    • Turn on the BGS_WISE bit, which had been deprecated.

    • Correct behavior for ELG_HIP when making MTLs.

      • Also change ELG_LOP to just ELG for simplicity.

  • Update the LRG selection for SV3 (The 1% Survey) [PR #699]:

    • Replace zfiber>16 with zfibertot>16 to reject bright star shreds.

0.55.0 (2021-03-29)

  • Another update to the data model for the MTL loop [PR #698].

  • Update the secondary target bits for SV3 [PR #697].

    • Also make FIBERTOTFLUX_* available for target cuts.

  • Update the ELG selection for SV3 [PR #696]:

    • Add functionality for low (LOP) and high (HIP) priority ELGs.

  • Update the BGS selection for SV3 [PR #695]. Main changes include:

    • BGS_FAINT goes as deep as r-tot mag of 20.3.

    • BGS_FAINT also includes a fibermag-color based selection.

    • BGS_BRIGHT maintains the same r-tot mag limits (< 19.5).

    • Revive the BGS_WISE bit/TARGTYPE to select AGN (~3 /deg^2).

    • Increase the r-tot mag faint limit by ~0.025 mags for BASS/MzLS.

      • Now goes to r-tot < 19.5+0.025 for BGS_BRIGHT sample and to 19.5+0.025 < r-tot < 20.3+0.025 for BGS_FAINT.

    • Promote 20% of BGS_FAINT to BGS_FAINT_HIP (instead of 10%).

  • Update the LRG selection for SV3 (The 1% Survey) [PR #694]:

    • Add fainter (and higher redshift) LRG targets.

0.54.0 (2021-03-26)

  • Update MTL loop for new data model [PR #693]. Includes:

    • Functionality to make initial ledgers for secondary targets.

    • Use the ZTILE file to look-up redshift “done-ness” (zdone)

      • instead of relying on a “done” directory.

    • Also derive the survey and program/obscon from the ZTILE file.

    • Switch to reading the zbest files in the cumulative directory.

    • Remove -ve TARGETIDs before matching.

    • Add the date that redshifts were extracted to the mtl tiles file.

    • Add the root redshift directory (ZCAT_DIR) to the manifest.

  • Change minimum r_value for QSO RF from 17.5 to 16.5 [PR #692].

  • Some clean-up for the 1% Survey [PR #691]. Includes:

    • Don’t allow BGS_FAINT targets to be observed in DARK.

    • Warn about primary targets that might be too bright.

    • Have a single function for calculating UTC time stamps.

    • Functionality to read ledgers strictly before a certain UTC time.

    • Centralize and speed up routines to match arrays on TARGETID.

  • Update ToO Ledger with TOOID and HI/LO priority options [PR #690].

  • Add an sv3_cuts.py module and sv3 bitmask yaml file [PR #689].

  • Don’t pass the DR when constructing MTL filenames [PR #688].

  • Don’t insist that ZTILEID has to be in the zcat [PR #687].

  • Install the SV2-related data files [PR #686].

0.53.0 (2021-03-18)

  • Update the ELG selection for SV2 [PR #685].

  • Implement full MTL loop [PR #684]. Includes:

    • Modify ledgers based on any new tiles in a zcat directory.

    • An MTL tile file to track which tiles have been processed by MTL.

    • Read standard tile file to get observing conditions for each tile.

      • Only update ledgers for tiles with the appropriate conditions.

    • Option to use the ledgers themselves to updated NUMOBS

      • instead of expecting NUMOBS to be in the zcat.

    • A command-line script to execute the full loop.

      • Input directories and files can be environment variables.

    • The zcat now requires a ZTILEID

      • indicating which tile was used to update the redshifts.

  • Update the LRG selection for SV2 [PR #683]:

    • Switch from optical to IR as the (tentative) baseline selection.

    • Tweak the low-z and and IR sliding cuts.

    • Remove bright (G<18) GAIA sources.

0.52.0 (2021-03-13)

  • New secondary bits for COSMOS/unusual point sources [PR #682].

  • Add formalism to make ledger for BACKUP targets [PR #681].

  • New QSO target selection in SV2 [PR #680] for validation:

    • RF file (dr9_final) trained with significance > 10 and SV QSOs.

    • push selection to r < 23.

    • add W1/W2 cut (< 22.3).

    • update notebook with example.

  • Prepare SV2 for the “.1% Survey” [PR #679]. Includes:

    • Add an sv2_cuts.py module and an sv2 bitmask yaml file.

    • Rename POOR observing conditions to be BACKUP throughout.

    • Retain BACKUP targets that are duplicates. Specifically:

      • duplicates between Gaia-only and Legacy Surveys targets.

      • these are the same source with different TARGETID

        • (one derived from Gaia and one from the Legacy Surveys).

      • these can be removed using REF_ID (Gaia SOURCE_ID).

0.51.0 (2021-03-07)

  • Finalize dedicated secondaries for sv1->sv2 transition [PR #678].

    • Deprecates Targets of Opportunity as standard secondary targets.

    • ToOs will be handled by a separate ledger-based mechanism.

  • Main Survey, secondary, outside-footprint target updates [PR #677]:

    • Update priorities for some secondary programs.

    • MTL changes to reobserve 0.7 < z < 2.1 QSOs at low priority.

      • Requested by two secondary programs.

    • Allow secondaries to be truncated to first N targets in a file.

    • Add fluxes/mags to the file of “standalone” secondaries.

    • Apply bright star mask and a mag > 16 limit to secondary targets.

    • Mask Main Survey MWS targets on BRIGHT and GALAXY using MASKBITS.

    • Debug reading MTL ledger headers, now that they have units.

    • Apply non-linear correction to de-extinct Gaia magnitudes TWICE.

    • Add PHOT_G_N_OBS to the Gaia edr3 data model.

  • Add formalism for Targets of Opportunity [PR #676]. Includes:

    • New ToO module.

    • Function and script to write an initial ToO .ecsv ledger.

    • Function and script to convert the ledger to a targets file.

      • Includes checks on the ledger format and fiber allocations.

  • Migrated unit tests to GitHub Actions [PR #675].

0.50.0 (2021-01-29)

  • Improved functionality for Gaia-only targets [PR #673]. Includes:
    • Deredden Gaia colors when selecting Gaia-only standards.

      • Add a general function to de-extinct Gaia magnitudes.

      • Add E(B-V) to supplemental targets for downstream calibrations.

    • Add a flavor property to the secondary targets.

      • Flavor can be “SPARE”, “DEDICATED”, “SSV”, “QSO”, “TOO”.

      • Add unit test to check allowed values of flavor.

    • Update priorities for dedicated secondary targets.

      • Now they can be distinguished from spare targets by flavor.

    • Include PHOTSYS of “G” for Gaia-only targets.

    • Fix minor bugs that arose due to recent changes in the data model.

    • Add code to scrape Gaia EDR3 flat (csv) files from the web.

      • And reformat them as FITS files, and by HEALPixel.

  • Updates to MWS main survey target cuts [PR #672]. Includes:

    • Add bright limit to MWS_NEARBY

    • Add MWS_BHB as main survey class

0.49.0 (2021-01-18)

  • General clean-up for final DR9 imaging [PR #670]. Includes:
    • Debug primary-secondary cross-matching:

      • remove duplicate secondaries that match two primaries…

      • …NOT duplicate primaries that match two secondaries.

    • Catch if no Gaia sources are found when making Gaia-only standards.

    • Shift Gaia-based morphological cuts to a single function.

    • Add or update wiki versions referenced in doc strings.

    • Change cuts for bright, Main Survey standards to G > 16.

    • Debug and streamline “outside-of-the-footprint” randoms.

    • Read the actual RELEASE number for randoms from file headers.

      • Rather than assuming a single, canonical North/South RELEASE.

    • Add new WD_BINARIES secondary program that is split by DARK/BRIGHT.

0.48.0 (2021-01-09)

  • First run of secondaries with real target files [PR #669]. Includes:
    • Add Gaia-only standard stars to the MWS masks for SV, Main Survey:

      • GAIA_STD_FAINT, GAIA_STD_BRIGHT, GAIA_STD_WD.

    • General optimization, updating and debugging of the secondary code.

    • Get TARGETIDs from the input sweeps, not just the actual targets.

    • Add the first full bitmask for the SV1 secondary target files.

    • Updates to the data model to better reflect the primary targets.

  • Clean-up minor style and doc issues from PR #636 [PR #668].

  • Updates and bug fixes for DR9 now SV is on-sky [PR #665]. Includes:

    • Pass MASKBITS column forward for GFAs.

    • Bug fixes necessitated by target files having a second extension.

      • Notably, not all shasums were checked in North/South overlaps.

    • Some minor additional functionality for creating randoms.

    • Clean-up code style and syntax errors introduced in PR #664.

  • Tutorial (and initial code) to train DR9 Random Forests [PR #664].

  • Simplify stellar SV bits [PR #636]:

    • Secondary bit requirement for main stellar SV program to 4 bits.

    • Primary bright science WDs use the old algorithmic selection.

0.47.0 (2020-12-10)

  • Update the gr_blue ELG cut for DR9 imaging for SV [PR #663]:

0.46.0 (2020-12-10)

  • Update ELG cuts for DR9 imaging for SV and Main Survey [PR #662].

  • Retune LRG cuts for DR9 and update the LRG SV target bits [PR #661]:
    • Only use the default BRIGHT, GALAXY and CLUSTER masks.

      • i.e. ignore ALLMASK and MEDIUM.

    • Increase the SV faint limits from z < 20.5 to z < 21.0.

    • Increase the SV faint limits from zfiber < 21.9 to zfiber < 22.0.

  • PR #660: Work completed in PR #661.

  • Two main changes for BGS SV selection for DR9 [PR #659]:

    • Remove FRACS* cuts, except for LOWQ superset.

    • Limit FIBMAG superset to r < 20.5 instead of r < 21.0.

  • General clean-ups and speed-ups for DR9 work [PR #658]. Includes:

    • Corrected data model when repartitioning skies into HEALPixels.

    • Faster versions of all of the read_targets_in_X functions:

      • e.g., in_box, in_cap, in_tiles, in_hp.

      • less general, but run faster by assuming the data model.

      • Speed-up is 10x or more for files pixelized at higher nsides.

    • Read “standard” MASKBITS cuts automatically for pixweight files.

    • Catch if MTL ledgers are at a lower resolution that target files.

  • Extension of mag limit to 22.3 for RF selection [PR #655].

  • Add input sweep files and their checksums to target files [PR #641].

0.45.1 (2020-11-22)

  • Add RA/Dec to the Main Survey calls for the QSO RF in cmx [PR #654].

0.45.0 (2020-11-22)

  • Clean-up for DR9-based commissioning [PR #653]. Includes:
    • Use HEALPixels instead of BRICKIDs for supp_skies.

      • This avoids duplicated TARGETIDs where bricks span pixels.

      • Addresses issue #647.

    • G < 19 for STD_DITHER_GAIA cmx targets near the Galaxy.

    • Allow gather_targets to restrict to a subset of columns.

    • Ignore new “light-curve” and “extra” flavors when finding sweeps.

    • Smarter processing of randoms when writing “bundled” slurm file.

      • Split pixelized files into N smaller files first…

      • …then combine across pixels to make N random catalogs.

      • Never requires memory to write a very large random catalog.

  • Tune the RF selection for QSOs in SV using DR9 imaging [PR #652].

  • Add RF files and threshold for each DR9 sub-footprint [PR #648].

0.44.0 (2020-11-12)

  • Clean-up targets and randoms for the internal DR9 release [PR #649]:
    • Add function geomask.imaging_mask():

      • Allows easier parsing of maskbits by string (“BRIGHT”, etc.)

      • Establishes a default set of cuts on maskbits.

    • New executable alt_split_randoms (slower but saves memory).

    • Flexibility when adding MTL columns to randoms, to save memory:

      • MTL columns can still be added when running the randoms.

      • Or, can now be added when splitting a larger random catalog.

  • Add notebook demonstrating ledgers [PR #642].

0.43.0 (2020-10-27)

  • Add the STD_DITHER_GAIA target class for CMX [PR #644].

    • For dither tests outside the Legacy Surveys footprint.

  • Tune shifts between southern and northern imaging for DR9 [PR #643].

  • Update Travis for Py3.8/Astropy 4.x (fixes issue #639) [PR #640].

    • Also adds a useful script for recovering the QSO RF probabilities.

  • Add units to all output files (addresses issue #356) [PR #638]:

    • Units for all output quantities are stored in data/units.yaml.

    • Unit tests check that output quantities have associated units.

    • Unit tests also check that all units are valid astropy units.

    • Also some more minor cleanup and speedups.

0.42.0 (2020-08-17)

  • Update the data model to address issue #633 [PR #637].

  • Major refactor to MTL to implement ledgers [PR #635]. Includes:
    • Code to make initial HEALPix-split ledger files from target files.

      • Ledgers can be produced for each observing layer.

      • Also includes an easy-to-use binary executable script.

      • New data model with timestamp, code version and target states.

    • Code to rapidly update MTL information by appending to a ledger.

      • Uses targets and a zcat with the current standard columns.

    • Functionality that works with either FITS or ECSV files.

    • Automatic trimming of target columns in mtl.make_mtl().

      • Saves memory, which may help with processing of mocks.

    • mtl.inflate_ledger() to re-obtain trimmed target columns.

    • Code to write MTL files in a standard format.

    • Utility functions to read (FITS or ECSV) MTL ledgers:

      • In a set of HEALPixels (io.read_mtl_in_hp())

      • In a set of tiles (read_targets_in_tiles() with mtl=True)

      • In a box (read_targets_in_box() with mtl=True)

      • In a cap (read_targets_in_cap() with mtl=True)

    • Can read entire ledger, or most recent entry for each TARGETID.

0.41.0 (2020-08-04)

  • Support for python/3.8 and numpy/1.18, including new tests [PR #631, PR #634]

  • Minor data model fixes, error checks and streamlining [PR #627].

    • The most important change is that MWS science targets are no longer observed in GRAY or DARK, except for MWS_WDs.

  • Cleanup: Avoid absolute path in resource_filename [PR #626].

  • Update masking to be “all-sky” using Gaia/Tycho/URAT [PR #625]:

    • General desitarget functionality to work with Tycho files.

    • Deprecate using the sweeps to mask bright objects as this is now being done using MASKBITS from the imaging catalogs.

    • Functionality to allow masks to be built at different epochs, via careful treatment of Tycho/Gaia/URAT proper motions.

    • Bright star masks are now explicitly written to a $MASK_DIR.

    • The radius-magnitude relationship is now a single function.

    • Refactoring of unit tests to be simpler and have more coverage.

    • Skies and supplemental skies are now always masked by default.

    • A lack of backward compatibility, which should be OK as the masking formalism wasn’t being extensively used.

  • Functionality for iterations of SV beyond sv1 [PR #624]. Includes:

    • A script to create the necessary files for new iterations of SV.

    • Generalized mask/cuts handling for survey=svX, X being any integer.

    • targets.main_cmx_or_sv() also updated to handle survey=svX.

    • Alter the automated creation of output SV target directory names:

      • write svX targets to /targets/svX/ instead of just targets/sv/.

    • Make TARGETID for secondary targets unique for iterations of SVX:

      • Schema is RELEASE=(X-1)*100 + SCND_BIT for SVX-like surveys…

      • …and RELEASE=5*100 + SCND_BIT for the Main Survey.

  • Adjust MWS SV1 target classes for new SV schedule [PR #623]:

    • More generic names for clusters, stream, dwarf targets.

    • Remove ORPHAN, add CV.

    • Lower priority for SEGUE targets.

0.40.0 (2020-05-26)

  • Add RELEASE for dr9i, dr9j (etc.) of the Legacy Surveys [PR #622].

  • Repartition sky files so skies lie in HEALPix boundaries [PR #621]:

    • Previously, unlike other target classes, skies were written such that the brick centers in which they were processed, rather than the sky locations themselves, lay within given HEALPixels.

    • is_sky_dir_official() now checks skies are partitioned right.

    • bin/repartition_skies now reassigns skies to correct HEALPixels.

    • In addition, also includes:

      • Significant (5-10x) speed-ups in read_targets_in_hp().

      • Remove supplemental skies that are near existing sky locations. (which addresses issue #534).

      • A handful of more minor fixes and speed-ups.

  • Various updates to targeting bits and MTL [PR #619]. Includes:

    • Don’t select any BGS_WISE targets in the Main Survey.

    • Always set BGS targets with a ZWARN > 0 to a priority of DONE.

    • Add an informational bit for QSOs selected with the high-z RF (addresses issue #349).

    • MWS targets should drop to a priority of DONE after one observation (but will always be higher priority than BGS for that observation).

    • Update the default priorities for reobserving Lyman-alpha QSOs (as described in issue #486, which this addresses).

  • NUMOBS_MORE for tracer QSOs that are also other targets [PR #617]:

    • Separate the calculation of NUMOBS_MORE into its own function.

    • Consistently use zcut = 2.1 to define Lyman-Alpha QSOs.

    • Check tracer QSOs that are other targets drop to NUMOBS_MORE = 0.

    • New unit test to enforce that check on such tracer QSOs.

    • New unit test to check BGS always gets NUMOBS_MORE = 1 in BRIGHT.

    • Enforce maximum seed in randoms_in_a_brick_from_edges().

  • Update masks for QSO Random Forest selection for DR8 [PR #615]

  • Add a new notebook tutorial about the Merged Target List [PR #614].

  • Recognize (and skip) existing (completed) healpixels when running select_mock_targets [PR #591].

0.39.0 (2020-05-01)

  • Help the mocks run on pixel-level imaging data [PR #611]. Includes:

    • New geomask.get_brick_info() function to look up the brick names associated with each HEALPixel.

    • In randoms.quantities_at_positions_in_a_brick(), add a justlist option to list the (maximal) required input files.

    • Minor bug fixes and documentation updates.

  • Update QSO Random Forest selection (and files) for DR8 [PR #610].

0.38.0 (2020-04-23)

  • Minor updates for latest DR9 imaging versions (dr9f/dr9g) [PR #607].

  • Extra columns and features in the random catalogs [PR #606]:

    • Better error messages and defaults for bin/supplement_randoms.

    • Don’t calculate APFLUX quantities if aprad=0 is passed.

    • Pass the randoms through the finalize and make_mtl functions:

      • To populate columns needed to run fiberassign on the randoms.

      • Addresses issue #597.

    • Add the BRICKID column to the random catalogs.

    • Also add a realistic TARGETID (and RELEASE, BRICK_OBJID).

    • Recognize failure modes more quickly (and fail more quickly).

    • Write out both “resolve” and “noresolve” (North/South) catalogs.

  • Fixes a typo in the priority of MWS_WD_SV targets [PR #601].

  • Fixes calc_priority logic for MWS CMX targets [PR #601].

  • Separate calc_priority() for CMX into a separate function [PR #601].

  • Alter cmx targetmask such that obsconditions can be used to work around MWS/BGS conflicts on MWS CMX tiles [PR #601].

  • Update test_priorities() for new MWS CMX targets scheme [PR #601].

  • Adds SV0_MWS_FAINT bit [PR #601].

0.37.3 (2020-04-15)

  • Update QA now basemap dependency is removed [PR #605]:

    • Also reintroduce unit tests in test_qa.py.

    • basemap dependency was removed in desiutil PR #141

0.37.2 (2020-04-13)

  • Fix select_mock_targets I/O bug reported in #603 [PR #604].

0.37.1 (2020-04-07)

  • Fix mock QSO density bug reported in #594 [PR #602].

  • Fixes a typo in the priority of MWS_WD_SV targets [PR #600].

0.37.0 (2020-03-12)

  • Add SV0_MWS_CLUSTER_ target classes for commissioning [PR #599].

  • Flag the high-z quasar selection in CMX (as SV0_QSO_Z5) [PR #598].

  • Leak of Bright Stars in BGS Main Survey and BGS SV fixed [PR #596].

  • Restrict skies to the geometric boundaries of their brick [PR #595].

  • Changes in CMX after running code for Mini-SV [PR #592]. Includes:

    • g/G >= 16 for SV0_BGS/SV0_MWS/SV0_WD/MINI_SV_BGS_BRIGHT.

    • Remove the LRG LOWZ_FILLER class (both in SV and CMX).

    • Mask on bright in MASKBITS for z~5 QSOs (both in SV and CMX).

    • Remove the ‘low quality’ (lowq) component of SV0_BGS.

    • Add optical MASKBITS flags for LRGs (in Main Survey, SV and CMX).

0.36.0 (2020-02-16)

  • Add Main Survey LRG/ELG/QSO/BGS cuts to CMX for Mini-SV [PR #590].

  • Cut on NOBS > 0 for QSOs and LRGs for Main Survey and SV [PR #589].

  • Fix bug when adding LSLGA galaxies into Main Survey BGS [PR #588]:

    • Catch cases of bytes/str types as well as zero-length strings.

  • Noting (here) that we used the BFG to excise lots of junk [PR #587].

  • Updates and fixes to QA for DR9 [PR #584]. Includes:

    • Options to pre-process and downsample input files to speed testing.

    • Better labeling of QA output, including cleaning up labeling bugs.

    • Make points in scatter plots black to contrast with blue contours.

    • Smarter clipping of dense pixels in histogram plots and sky maps.

    • Print out densest pixels for each target class, with viewer links.

  • Update BGS Main target selection as stated in [PR #581]. Includes:

    • Changes in Fibre Magnitude Cut.

    • LSLGA galaxies manually added to BGS.

      • Future-proof LSLGA object references changing (‘L2’ –> ‘LX’).

    • ‘REF_CAT’ information passed to throught ‘_prepare_optical_wise’.

  • Tune QSO SV selection for both North and South for dr9d [PR #580].

0.35.3 (2020-02-03)

  • Further fixes for DR9 [PR #579]. Includes:

    • Add SERSIC columns for the DR9 data model.

    • Read the bricks file in lower-case in get_brick_info():

      • As, during DR9 testing, it’s been both upper- and lower-case.

    • Set the default nside to None for the randoms:

      • To force the user to choose an nside, or fail otherwise.

    • Fix a numpy future/deprecation warning.

  • Load yaml config file safely in mpi_select_mock_targets [PR #577].

  • Fix bugs in updating primary targets with secondary bits set [PR #574].

  • Adds more stellar SV targets [PR #574].

  • Add LyA features to select_mock_targets [PR #565].

0.35.2 (2019-12-20)

  • Fix z~5 QSO bug in CMX/SV0 that was already fixed for SV [PR #576].

0.35.1 (2019-12-16)

  • Fix bugs triggered by empty files or regions of the sky [PR #575].

0.35.0 (2019-12-15)

  • Preparation for DR9 [PR #573]. Includes:

    • Update data model, maintaining backwards compatibility with DR8.

    • Don’t set the SKY bit when setting the SUPP_SKY bit.

    • Users can input a seed (1, 2, 3, etc.) to bin/select_randoms:

      • This user-provided seed is added to the output file name.

      • Facilitates generating a range of numbered random catalogs.

    • Write out final secondaries using io.find_target_files().

  • More clean-up of glitches and minor bugs [PR #570]. Includes:

    • Remove Python 3.5 unit tests.

    • Catch AssertionError if NoneType input directory when writing.

      • Later (correctly) updated to AttributeError directly in master.

    • Assert the data model when reading secondary target files.

    • Use io.find_target_files() to name priminfo file for secondaries.

    • Allow N < 16 nodes when bundling files for slurm.

    • Use the DR14Q file for SV, not the DR16Q file.

  • Fix bug where wrong SNRs were passed to z~5 QSO selection [PR #569].

  • General clean-up of glitches and minor bugs [PR #564]. Includes:

    • Don’t include BACKUP targets in the pixweight files.

    • Correctly write all all-sky pixels outside of the Legacy Surveys.

    • Propagate flags like –nosec, –nobackup, –tcnames when bundling.

    • Write –tcnames options to header of output target files.

    • Deprecate the sandbox and file-format-check function.

    • Find LSLGAs using ‘L’ in REF_CAT not ‘L2’ (to prepare for ‘L3’).

    • Refactor to guard against future warnings and overflow warnings.

    • Return all HEALpixels at nside in sweep_files_touch_hp().

  • Strict NoneType checking and testing for fiberfluxes [PR #563]:

    • Useful to ensure ongoing compatibility with the mocks.

  • Bitmasks (1,12,13), rfiberflux cut for BGS Main Survey [PR #562].

  • Implement a variety of fixes to select_mock_targets [PR #561].

  • Fixes and updates to secondary.py [PR #530]:

    • Fix a bug that led to incorrect OBSCONDITIONS for secondary-only targets.

    • Secondary target properties can override matched primary properties, but only for restricted combinations of DESI_TARGET bits (MWS and STD).

  • Add stellar SV targets [PR #530]:

    • Add MWS SV target definitions in sv1_targetmask and cuts.

    • Science WDs are now a secondary target class.

    • Adds a bright limit to the MWS-NEARBY sample.

    • Add stellar SV secondary targets in sv1_targetmask.

    • Remove the BACKSTOP secondary bit.

0.34.0 (2019-11-03)

  • Update SV0 (BGS, ELG, LRG, QSO) classes for commissioning [PR #560].

    • Also add new STD_DITHER target class for commissioning.

  • All-sky/backup targets, new output data model [PR #558]. Includes:

    • Add all-sky/backup/supplemental targets for SV.

    • Add all-sky/backup/supplemental targets for the Main survey.

    • Write dark/bright using, e.g. targets/dark/targets-*.fits format.

    • New targets/targets-supp/targets-*.fits format for output.

    • Add io.find_target_files() to parse output data model.

    • File names now generated automatically in io.write_* functions:

      • File-name-generation used by randoms, skies, targets and gfas.

      • select_* binaries for these classes use this functionality.

    • Change CMX BACKUP_FAINT limit to G < 19.

0.33.3 (2019-10-31)

  • Add cuts for z = 4.3-4.8 quasar into the z5QSO selection [PR #559].

0.33.2 (2019-10-17)

  • Add FIBERFLUX_IVAR_G/R/Z to mock skies when merging [PR #556].

  • Fix minor bugs in select_mock_targets [PR #555].

  • Update the ELG selections for SV [PR #553]. Includes:
    • Four new bit names:

      • ELG_SV_GTOT, ELG_SV_GFIB.

      • ELG_FDR_GTOT, ELG_FDR_GFIB.

    • Associated new ELG selections with north/south differences.

    • Propagate FIBERFLUX_G from the sweeps for SV ELG cuts.

    • Increase the default sky densities by a factor of 4x.

    • Relax CMX BACKUP_FAINT limit to G < 21 to test fiber assign.

  • Bright-end FIBERFLUX_R cut on BGS_FAINT_EXT in SV [PR #552].

  • Update LRG selections for SV [PR #550]. Includes:

    • The zfibermag faint limit is changed from 21.6 to 21.9.

    • IR-selected objects with r-W1>3.1 not subjected to the sliding cut.

0.33.1 (2019-10-13)

  • Enhancements and on-sky clean-up for SV and CMX [PR #551]. Includes:

    • Add areas contingent on MASKBITS to the pixweight- files.

    • Change APFLUX to FIBERFLUX for skies and supp-skies.

    • Add new M33 First Light program.

    • Change priorities for the First Light programs.

    • Retain Tycho, and sources with no measured proper motion, in GFAs.

    • Add the REF_EPOCH column to all target files.

0.33.0 (2019-10-06)

  • Update skies, GFAs and CMX targets for all-sky observing [PR #548]:

    • Process and output GFAs, skies and CMX targets split by HEALPixel.

    • “bundling” scripts to parallelize GFAs, skies, CMX by HEALPixel.

    • Bundle across all HEALPixels (not just those in the footprint).

    • Add pixel information to file headers for GFAs, skies and CMX.

    • Write all-sky CMX targets separately from in-footprint targets.

    • Add back-up and first light targets for commissioning.

    • New TARGETID encoding scheme for Gaia-only and first light targets.

    • Resolve commissioning targets from the Legacy Surveys.

    • io.read functions can now process SKY and GFA target files.

    • New function to read in targets restricted to a set of DESI tiles.

    • Implement Boris Gaensicke’s geographical cuts for Gaia.

    • Update unit tests to use DR8 files.

  • Further updates to changes in PR #531, [PR #544]. Includes:

    • A –writeall option to select_secondary writes a unified target file without the BRIGHT/DARK split, as for select_targets

    • Removes duplicate secondaries that arise from multiple matches to one primary and secondary targets appearing in more than one input file. The duplciate with highest PRIORTIY_INIT is retained.

  • Update mocks to match latest data-based targets catalogs [PR #543].

  • Add new redshift 5 (QSO_Z5) SV QSO selection [PR #539]. Also:

    • Remove all Tycho and LSLGA sources from the GFA catalog.

    • Minor improvements to documentation for secondary targets.

    • Use N/S bricks for skies when S/N bricks aren’t available.

  • Tune, high-z, faint (QSO_HZ_F) SV QSO selection [PR #538]

  • Use SPECTYPE from zcat to set NUMOBS_MORE [PR #537]:

    • Updates behavior for tracer QSOs vs. LyA QSOs in MTL.

  • Update LRG selections for DR8 [PR #532]. Includes:

    • New LRG selection for SV with fewer bits.

    • New LOWZ_FILLER class for SV.

    • Add LRG 4PASS and 8PASS bits/classes using cuts on FLUX_Z.

    • New and simplified LRG selection for the Main Survey.

    • Deprecate Main Survey 1PASS/2PASS LRGs, all LRGs now have one pass.

    • Deprecate some very old code in desitarget.targets.

  • Finalize secondaries, add BRIGHT/DARK split [PR #531]. Includes:

    • Updated data model for secondaries.

    • New secondary output columns (OBSCONDITIONS, proper motions).

    • Add a cached file of primary TARGETIDs to prevent duplicates.

    • Create a more reproducible TARGETID for secondaries.

    • Automatically write secondaries split by BRIGHT/DARK.

    • Add option to pass secondary file in MTL.

    • Insist on observing layer/conditions for MTL:
      • Ensures correct behavior for dark targets in bright time…

        • …and bright-time targets observed in dark-time.

    • Minor update to the MWS_BROAD class.

  • Add info on versioning, main_cmx_or_sv to bitmask notebook [PR #527]

0.32.0 (2019-08-07)

  • Add URAT catalog information [PR #526]. Includes:

    • New module to retrieve URAT data from Vizier and reformat it.

    • Code to match RAs/Decs to URAT, as part of that new URAT module.

    • Substitute URAT PMs for GFAs where Gaia has not yet measured PMs.

  • Update CMX and Main Survey target classes [PR #525]. Includes:

    • New SV0_WD, SV0_STD_FAINT target classes for commissioning.

    • Mild updates to SV0_BGS and SV0_MWS for commissioning.

    • New BGS_FAINT_HIP (high-priority BGS) Main Survey class.

    • Explicit checking on ASTROMETRIC_PARAMS_SOLVED for MWS targets.

    • Add 3-sigma parallax slop in MWS_MAIN survey target class.

  • Add OBSCONDITIONS to target files [PR #523] Also includes:

    • Split target files explicitly into bright and “graydark” surveys.

    • Default to such a file-spilt for SV and Main (not for cmx).

    • Adds an informational bit for supplemental sky locations.

  • Use MASKBITS instead of BRIGHTSTARINBLOB [PR #521]. Also:

    • Extra options and checks when making and vetting bundling scripts.

    • Option to turn off commissioning QSO cuts to speed unit tests.

  • Add ELG/LRG/QSO/STD selection cuts for commissioning [PR #519].

  • Add full set of columns to supplemental skies file [PR #518].

  • Fix some corner cases when reading HEALPixel-split files [PR #518].

0.31.1 (2019-07-05)

  • Pass Gaia astrometric excess noise in cmx MWS SV0 [PR #516].

0.31.0 (2019-06-30)

  • MASKBITS of BAILOUT for randoms when no file is found [PR #515].

  • Near-trivial fix for an unintended change to the isELG API introduced in PR #513 [PR #514].

  • Preliminary ELG cuts for DR8 imaging for main and SV [PR #513].

    • Don’t deprecate wider SV bits, yet, ELGs may still need them.

  • Further updates to generating randoms for DR8. [PR #512]. Includes:

    • Add WISE depth maps to random catalogs and pixweight files.

    • Code to generate additional supplemental randoms catalogs.

      • Supplemental, here, means (all-sky) outside of the footprint.

    • Executable to split a random catalog into N smaller catalogs.

    • Fixes a bug in targets.main_cmx_or_sv().

      • Secondary columns now aren’t the default if rename is True.

    • Better aligns data model with expected DR8 directory structure.

      • Also fixes directory-not-found bugs when generating skies.

  • Add “supplemental” (outside-of-footprint) skies [PR #510]:

    • Randomly populates sky area beyond some minimum Dec and Galactic b.

    • Then avoids all Gaia sources at some specified radius.

    • Fixes a bug where geomask.hp_in_box() used geodesics for Dec.

      • Dec cuts should be small circles, not geodesics.

  • First implementation for secondary targets [PR #507]. Includes:

    • Framework and design for secondary targeting process.

    • Works automatically for both Main Survey and SV files.

    • New bitmasks for secondaries that populate SCND_TARGET column.

      • can have any PRIORITY_INIT and NUMOBS_INIT.

    • A reserved “veto” bit to categorically reject targets.

    • Rigorous checking of file formats…

      • …and that files correspond to secondary bits.

    • Example files and file structure (at NERSC) in SCND_DIR.

      • /project/projectdirs/desi/target/secondary.

    • Secondary targets are matched to primary targets on RA/Dec.

      • unless a (per-source) OVERRIDE column is set to True.

    • Secondary-primary matches share the primary TARGETID.

    • Non-matches and overrides have their own TARGETID.

      • with RELEASE == 0.

    • Non-override secondary targets are also matched to themselves.

      • TARGETID and SCND_TARGET correspond for matches.

0.30.1 (2019-06-18)

  • Fix normalization bug in QSO tracer/Lya mock target densities [PR #509].

  • Tune “Northern” QSO selection and color shifts for Main and SV [PR #506]

  • Follow-up PR to PR #496 with two changes and bug fixes [PR #505]:

    • Select QSO targets using random forest by default.

    • Bug fix: Correctly populate REF_CAT column (needed to correctly set MWS targeting bits).

0.30.0 (2019-05-30)

  • Drop Gaia fields with np.rfn to fix Python 3.6/macOS bug [PR #502].

  • Apply the same declination cut to the mocks as to the data [PR #501].

  • Add information to GFA files [PR #498]. Includes:

    • Add columns PARALLAX, PARALLAX_IVAR, REF_EPOCH.

    • Remove REF_EPOCH from GFA file header, as it’s now a column.

    • Sensible defaults for Gaia-only REF_EPOCH, RA/DEC_IVAR.

    • Use fitsio.read() instead of desitarget.io.read_tractor().

      • It’s faster and special handling of input files isn’t needed.

  • General clean-up of target selection code [PR #497]. Includes:

    • Deprecate old functions in desitarget.gfa.

    • Greatly simplify io.read_tractor().

      • Backwards-compatability is now only guaranteed for DR6-8.

    • Guard against warnings (e.g. divide-by-zero) in cuts and SV cuts.

    • Default to only passing North (S) sources through North (S) cuts.

      • Retain previous behavior if --noresolve flag is passed.

  • Add SV support to select_mock_targets [PR #496]

  • A few more updates and enhancements for DR8 [PR #494]. Includes:

    • Add WISEMASK_W1 and WISEMASK_W2 to random catalogs.

    • Deprecate BRIGHTBLOB in favor of MASKBITS for targets.

    • Add qso_selection==colorcuts in set_target_bits.sv1_cuts()

      • This should facilitate QSO selection for SV mocks.

  • Add REF_CAT and Gaia BP and RP mags and errors to GFAs [PR #493].

  • Minor bug fix in how select_mock_targets handles Lya targets [PR #444].

  • Further updates and enhancements for DR8 [PR #490]. Includes:

    • Resolve sky locations and SV targets in North/South regions.

    • Update sky and SV slurming for DR8-style input (two directories).

    • Write both of two input directories to output file headers.

    • Parallelize plot production to speed-up QA by factors of 8.

    • Add PSFSIZE to randoms, pixweight maps and QA plots.

    • QA and pixweight maps work fully for SV-style files and bits.

    • Pixweight code can now take HEALpixel-split targets as input.

    • Add aperture-photometered background flux to randoms catalogs.

    • Additional unit test module (test.test_geomask()).

    • Deprecate make_hpx_density_file; use make_imaging_weight_map.

    • io.read_targets_in_a_box() can now read headers.

    • Update unit test data for new DR8 columns and functionality.

  • Update QSO targeting algorithms for DR8 [PR #489]. Includes:

    • Update baseline quasar selection for the main survey.

    • Update QSO bits and selection algorithms for SV.

  • Remove GFA/Gaia duplicates on REF_ID not BRICKID [PR #488].

  • Various bug and feature fixes [PR #484]. Includes:

    • Fix crash when using sv_select_targets with –tcnames.

    • Only import matplotlib where explicitly needed.

  • Update select_mock_targets to (current) DR8 data model [PR #480].

0.29.1 (2019-03-26)

  • Add REF_CAT, WISEMASK_W1/W2 to DR8 data model [PR #479].

  • Use speed of light from scipy [PR #478].

0.29.0 (2019-03-22)

  • Update SV selection for DR8 [PR #477]. Includes:

    • New SV targeting bits for QSOs and LRGs.

    • New SV selection algorithms for QSOs, ELGs and LRGs.

    • MTL fixes to handle SV LRGs (which are now not 1PASS/2PASS).

    • QA can now interpret HEALPixel-split targeting files.

    • Updated test files for the quasi-DR8 imaging data model.

    • SKY and BAD_SKY added to commissioning bits yaml file.

    • Randoms in overlap regions, and for DR8 dual directory structure.

    • Write overlap regions in addition to resolve for targets/randoms.

  • Change instances of yaml.load to yaml.safe_load [PR #475].

  • Fix Gaia files format in doc string (healpix not healpy) [PR #474].

  • Write Gaia morphologies and allow custom tilings for GFAs [PR #467].

  • Initial updates for DR8 [PR #466]. Includes:

    • DR8 data model updates (e.g BRIGHTSTARBLOB -> bitmask BRIGHTBLOB).

    • Apply resolve capability to targets and randoms.

    • Handle BASS/MzLS and DECaLS existing in the same input directory.

  • New resolve capability for post-DR7 imaging [PR #462]. Includes:

    • Add RELEASE to GFA data model to help resolve duplicates.

    • Resolve N/S duplicates by combining RELEASE and areal cuts.

    • Apply the new resolve code (targets.resolve()) to GFAs.

    • Deprecate Gaia-matching code for GFAs, as we no longer need it.

  • Add code to select GFAs for cmx across wider sky areas [PR #461].

0.28.0 (2019-02-28)

  • desitarget.mock.build.targets_truth fixes for new priority calcs [PR #460].

  • Updates to GFAs and skies for some cmx issues [PR #459]. Includes:

    • Assign BADSKY using BLOBDIST rather than aperture fluxes.

    • Increase default density at which sky locations are generated.

    • Store only aperture fluxes that match the DESI fiber radius.

    • Ensure GFAs exist throughout the spectroscopic footprint.

  • Refactor SV/main targeting for spatial queries [PR #458]. Includes:

    • Many new spatial query capabilities in desitarget.geomask.

    • Parallelize target selection by splitting across HEALPixels.

    • Wrappers to read in HEALPix-split target files split by:

      • HEALPixels, RA/Dec boxes, RA/Dec/radius caps, column names.

    • Only process subsets of targets in regions of space, again including:

      • HEALPixels, RA/Dec boxes, RA/Dec/radius caps.

    • New unit tests to check these spatial queries.

    • Updated notebook including tutorials on spatial queries.

  • Update the SV selections for BGS [PR #457].

  • Update MTL to work for SV0-like cmx and SV1 tables [PR #456]. Includes:

    • Make SUBPRIORITY a random number (0->1) in skies output.

    • New targets.main_cmx_or_sv() to parse flavor of survey.

    • Update targets.calc_priority() for SV0-like cmx and SV1 inputs.

    • mtl.make_mtl() can now process SV0-like cmx and SV1 inputs.

    • New unit tests for SV0-like cmx and SV1 inputs to MTL.

  • Deprecate targets.calc_priority() that had table copy [PR #452].

  • Update SV QSO selections, add seed and DUST_DIR for randoms [PR #449].

  • Style changes to conform to PEP 8 [PR #446], [PR #447], [PR #448].

0.27.0 (2018-12-14)

  • Remove reliance on Legacy Surveys for Gaia data [PR #438]. Includes:

    • Use $GAIA_DIR environment variable instead of passing a directory.

    • Functions to wget Gaia DR2 CSV files and convert them to FITS.

    • Function to reorganize Gaia FITS files into (NESTED) HEALPixels.

    • Use the NESTED HEALPix scheme for Gaia files throughout desitarget.

    • Change output column TYPE to MORPHTYPE for GFAs.

  • Move select-mock-targets.yaml configuration file to an installable location for use by desitest [PR #436].

  • Significant enhancement and refactor of select_mock_targets to include stellar and extragalactic contaminants [PR #427].

0.26.0 (2018-12-11)

  • Refactor QSO color cuts and add hard r > 17.5 limit [PR #433].

  • Refactor of MTL and MTL-related enhancements [PR #429]. Includes:

    • Use targets file NUMOBS_INIT not targets.calc_numobs().

    • Use targets file PRIORITY_INIT not targets.calc_priority().

    • Remove table copies from desitarget.mtl to use less memory.

    • New function targets.calc_priority_no_table() to use less memory.

    • Set informational (NORTH/SOUTH) bits to 0 PRIORITY and NUMOBS.

    • Set priorities using LRG_1PASS/2PASS bits rather than on LRG.

  • Minor updates to select_mock_targets [PR #425].

    • Use pre-computed template photometry (requires v3.1 basis templates).

    • Include MW dust extinction in the spectra.

    • Randomly assign a radial velocity to superfaint mock targets.

  • Update default mock catalogs used by select_mock_targets [PR #424]

  • Update Random Forests for DR7 quasar selection [PR #423]

  • Fix bugs in main MWS selections [PR #422].

  • Fix python setup.py install for cmx and sv1 directories [PR #421].

  • More updates to target classes, mainly for SV [PR #418]. Includes:

    • First full implementations of QSO, LRG, ELG, and STD for SV.

    • Update and refactor of MWS and BGS classes for the main survey.

    • Change name of main survey MWS_MAIN class to MWS_BROAD.

    • Augment QA code to handle SV sub-classes such as ELG_FDR_FAINT.

0.25.0 (2018-11-07)

  • Randomize mock ordering for Dark Sky mocks which aren’t random [PR #416].

  • Updates to several target classes [PR #408]. Includes:

    • Refactor of the ELG and MWS_MAIN selection algorithms.

    • Update of the ELG and MWS_MAIN selection cuts.

    • Change MWS_WD priority to be higher than that of BGS target classes.

    • Set skies to BAD only if both g-band and r-band are missing.

  • Refactor of BGS selections to separate masking and color cuts [PR #407].

  • Quicksurvey MTL fix [PR #405].

  • Mocks use QSO color cuts instead of random forest [PR #403].

  • Updates to Bright Galaxy Survey and QSO selections [PR #402]. Includes:

    • Updates to BGS_FAINT and BGS_BRIGHT target selections.

    • New BGS_WISE selection and implementation.

    • New data model columns BRIGHTSTARINBLOB and FRACIN_.

    • Add cut on BRIGHTSTARINBLOB to QSO selection.

    • Modify I/O to retain (some) backwards-compatibility between DR6 and DR7.

    • Updated unit test example files with appropriate columns.

    • Speed-up of cuts unit tests without loss of coverage.

  • Updated mock sky catalog with positions over a larger footprint [PR #398].

  • Major update to select_mock_targets to use the latest (v3.0) basis templates [PR #395].

  • Propagate per-class truth HDUs into final merged truth file [PR #393].

  • Incorporate simple WISE depth model in select_mock_targets which depends on ecliptic latitude [PR #391].

0.24.0 (2018-09-26)

  • Fix bug in code that produces data for unit tests [PR #387].

  • Rescale spectral parameters when generating and querying kd-trees in select_mock_targets [PR #386].

  • Bug fixes: [PR #383].

    • Use parallax_err when selecting MWS_NEARBY targets.

    • In select_mock_targets do not use Galaxia to select WDs and 100pc targets.

  • Refactor QA to work with commissioning and SV files and add (first) unit tests for QA. [PR #382].

  • Estimate FIBERFLUX_[G,R,Z] for mock targets. [PR #381].

  • First fully working version of SV code [PR #380]. Includes:

    • (Almost) the only evolving part of the code for SV is now the cuts.

    • Unit tests for SV that should be easy to maintain.

    • Bit and column setting for SV that should be maintainable.

    • SV0 (commissioning) MWS cuts.

    • Updated STD cuts to fix a fracmasked typo.

    • Alterations to Travis coverage to exclude some external code.

  • Fix a bug which resulted in far too few standard stars being selected in the mocks [PR #378].

  • Fix a bug in how the objtruth tables are written out to by select_mock_targets [PR #374].

  • Remove Python 2.7 from Travis, add an allowed-to-fail PEP 8 check [PR #373].

  • Function to read RA, DEC from non-standard external files [PR #372].

  • Update the data model for output target files [PR #372]:

    • Change TYPE to MORPHTYPE.

    • Add EBV, FIBERFLUX_G,R,Z and FIBERTOTFLUX_G,R,Z.

  • Additional commissioning (cmx) classes and priorities [PR #370]. Includes:

    • New functions to define several more commissioning classes.

    • A $CMX_DIR to contain files of cmx sources to which to match.

    • An example $CMX_DIR is /project/projectdirs/desi/target/cmx_files.

    • Functionality to reset initial priorities for commissioning targets.

    • Downloading fitsio using pip/astropy to fix Travis.

  • Significant enhancement of select_mock_targets (see PR for details) [PR #368].

  • Include per-band number counts for targets on the QA pages [PR #367].

  • Use new desiutil.dust.SFDMap() module [PR #366].

  • Set the STD_WD bit (it’s identical to the MWS_WD bit) [PR #364].

  • Add notebook for generating Gaussian mixture models from DR7 photometry and morphologies of ELG, LRG, and BGS targets [PR #363 and PR #365].

  • Make commissioning (cmx) target selection fully functional [PR #359]. Includes:

    • Initial target selection algorithms.

    • First unit tests for cmx (> 90% coverage).

    • SV_TARGET and CMX_TARGET as output columns instead of as a bit.

  • Remove “legacy” code in QA [PR #359].

  • Add isELG_colors functions [PR #357].

  • Adapt cuts.isSTD_colors to deal with different north/south color-cuts [PR #355].

  • Refactor to allow separate commissioning and SV target selections [PR #346]:

    • Added sv and commissioning directories.

    • New infrastructure to have different cuts for SV and commissioning:

      • separate target masks (e.g. sv/data/sv_targetmask.yaml).

      • separate cuts modules (e.g. sv_cuts.py).

    • Added executables for SV/commissioning (e.g. select_sv_targets).

    • Initial NUMOBS and PRIORITY added as columns in targets- files.

    • Initial NUMOBS is now hardcoded in target masks, instead of being set by MTL.

    • SV bits added to target masks to track if targets are from SV/comm/main.

    • sv/comm/main can now be written to the header of the targets- files.

    • SUBPRIORITY is set when writing targets to facilitate reproducibility.

  • Set NUMOBS for LRGs in MTL using target bits instead of magnitude [PR #345].

  • Update GFA targets [PR #342]:

    • Handle reading Gaia from sweeps as well as matching. Default to not matching.

    • Makes Gaia matching radius stricter to return only the best Gaia objects.

    • Retains Gaia RA/Dec when matching, instead of RA/Dec from sweeps.

    • Fixes a bug where Gaia objects in some HEALPixels weren’t being read.

    • Add Gaia epoch to the GFA file header (still needs passed from the sweeps).

0.23.0 (2018-08-09)

Includes non-backwards compatible changes to standard star bit names.

  • STD/STD_FSTAR -> STD_FAINT, with corresponding fixes for mocks [PR #341].

  • Match sweeps to Gaia and write new sweeps with Gaia columns [PR #340]:

    • Also add BRIGHTSTARINBLOB (if available) to target output files.

    • And include a flag to call STD star cuts function without Gaia columns.

0.22.0 (2018-08-03)

Includes non-backwards compatible changes to standard star target mask bit names and selection function names.

  • Produce current sets of target bits for DR7 [PR #338]:

    • Update the LRG, QSO, STD and MWS algorithms to align with the wiki.

    • In particular, major updates to the STD and MWS selections.

    • Don’t match to Gaia by default, only if requested.

    • Maintain capability to match to Gaia if needed for earlier Data Releases.

    • Run subsets of target classes by passing, e.g.. --tcnames STD,QSO.

    • Update unit test files to not rely on Gaia.

    • Bring Data Model into agreement with Legacy Surveys sweeps files.

    • Rename FSTD to be STD throughout.

    • QA fails gracefully if weight maps for systematics aren’t passed.

0.21.1 (2018-07-26)

  • Update the schema for target selection QA [PR #334]:

    • Sample imaging pixels from the Legacy Surveys to make random catalogs.

    • Add E(B-V) from SFD maps and stellar densities from Gaia to the randoms.

    • Sample randoms to make HEALpixel maps of systematics and target densities.

    • Sample randoms in HEALPixels to precisely estimate imaging footprint areas.

    • Make several new systematics plots.

    • Make new plots of parallax and proper motion information from Gaia.

0.21.0 (2018-07-18)

  • Fix bug when generating targeting QA for mock catalogs [PR #332].

  • Add support for GAMA/BGS mocks and new calib_only option in mock.targets_truth [PR #331].

  • Add RA_IVAR and DEC_IVAR to the GFA Data Model [PR #329].

  • Update the Gaia Data Model [PR #327]:

    • Output columns formatted as expected downstream for GFA assignment.

    • Align Gaia Data Model in matching and I/O with the Legacy Surveys.

  • Allow environment variables in select_mock_targets config file [PR #325].

  • First version of Milky Way Survey selection [PR #324]:

    • Catalog-matches to Gaia using desitarget.gaimatch.

    • Sets MWS_MAIN, MWS_WD and MWS_NEARBY bits.

    • Makes individual QA pages for MWS (and other) bits.

  • Change GFA selection to be Gaia-based [PR #322]:

    • Update the select_gfas binary to draw from Gaia DR2.

    • Parallelize across sweeps files to add fluxes from the Legacy Surveys.

    • Gather all Gaia objects to some magnitude limit in the sweeps areas.

  • Add desitarget.gaimatch for matching to Gaia [PR #322]:

    • Can perform object-to-object matching between Gaia and the sweeps.

    • Can, in addition, retain all Gaia objects in an RA/Dec box.

  • Mock targets bug fixes [PR #318].

  • Add missing GMM files to installations [PR #316].

  • Introduction of pixel-level creation of sky locations [PR #313]:

    • Significant update of desitarget.skyfibers

    • desitarget.skyutilities.astrometry to remove astrometry.net dependency.

    • desitarget.skyutilities.legacypipe to remove legacypipe dependency.

    • Grids sky locations by applying a binary erosion to imaging blob maps.

    • Sinks apertures at the resulting sky locations to derive flux estimates.

    • Sets the BAD_SKY bit using high flux levels in those apertures.

    • desitarget.skyfibers.bundle_bricks() to write a slurm script.

    • Parallelizes via HEALPixels to run in a few hours on interactive nodes.

    • Adds the select_skies binary to run from the command line.

    • Includes gather_skies binary to collect results from parallelization.

    • Adds functionality to plot good/bad skies against Legacy Survey images.

  • select_mock_targets full footprint updates [PR #312].

  • QA fix for testing without healpix weight map [PR #311].

  • New QSO random forest [PR #309].

  • Restore the no-spectra option of select_mock_targets, for use with quicksurvey [PR #307].

  • Better handling of imaging survey areas for QA [PR #304]:

    • desitarget.imagefootprint to build HEALPix weight maps of imaging.

    • Executable (bin) interface to make weight maps from the command line.

    • desitarget.io loader to resample maps to any HEALPix nside.

    • Update desitarget.QA to handle new imaging area weight maps.

  • Improve north/south split functions for LRG and QSO color cuts [PR #302].

  • Minor QA and selection cuts updates [PR #297]:

    • QA matrix of target densities selected in multiple classes.

    • Functions to allow different north/south selections for LRGs.

0.20.1 (2018-03-29)

  • Add a bright (g>21) flux cut for ELGs. [PR #296].

0.20.0 (2018-03-24)

  • Added compare_target_qa script [PR #289].

  • Astropy 2.x compatibility [PR #291].

  • Update of sky selection code [PR #290]. Includes:

    • Use the desitarget.brightmask formalism to speed calculations.

    • Pass around a magnitude limit on masks from the sweeps (to better avoid only objects that are genuinely detected in the sweeps).

    • Reduce the default margin to produce ~1700 sky positions per sq. deg.

  • Retuning of DR6 target densities [PR #294]. Includes:

    • Tweaking the QSO random forest probability.

    • Adding a new ELG selection for the northern (MzLS/BASS) imaging.

    • Slight flux shifts to reconcile the northern and southern (DECaLS) imaging.

    • Initial functionality for different North/South selections.

  • Some reformatting of output target files and bits [PR #294]:

    • Introducing a NO_TARGET bit.

    • Renaming the BADSKY bit BAD_SKY for consistency with other bits.

    • Including FRACDEV and FRACDEV_IVAR as outputs.

0.19.1 (2018-03-01)

  • Fix bug whereby FLUX and WAVE weren’t being written to truth.fits files [PR #287].

  • Include OBSCONDITIONS in mock sky/stdstar files for fiberassign [PR #288].

0.19.0 (2018-02-27)

This release includes significant non-backwards compatible changes to importing target mask bits and how mock spectra are generated.

  • Major refactor of select_mock_targets code infrastructure [PR #264].

  • Restructure desi_mask, bgs_mask, etc. imports to fix readthedocs build [PR #282].

  • Update RELEASE dictionary with 6000 (northern) for DR6 [PR #281].

0.18.1 (2018-02-23)

  • Open BGS hdf5 mocks read-only to fix parallelism bug [PR #278].

0.18.0 (2018-02-23)

  • New target density fluctuations model based on DR5 healpixel info [PR #254].

  • Include (initial) mock QA plots on targeting QA page [PR #262]

  • Added select_gfa script [PR #275]

  • Update masking for ellipses (“galaxies”) in addition to circles (“stars”) [PR #277].

0.17.1 (2017-12-20)

  • HPXNSIDE and HPXPIXEL as header keywords for mocks too [PR #246].

0.17.0 (2017-12-20)

  • Support LyA skewers v2.x format [PR #244].

  • Split LRGs into PASS1/PASS2 separate bits [PR #245].

  • Sky locations infrastructure [PR #248].

  • Mock targets densities fixes [PR #241 and PR #242].

0.16.2 (2017-11-16)

  • Allows different star-galaxy separations for quasar targets for different release numbers [PR #239].

0.16.1 (2017-11-09)

  • fixes to allow QA to work with mock data [PR #235].

  • cleanup of mpi_select_mock_targets [PR #235].

  • adds BGS properties notebook documentation [PR #236].

0.16.0 (2017-11-01)

  • General clean-up prior to running DR5 targets [PR #229].

    • Use desiutil.log instead of verbose (everywhere except mocks)

    • Change HEALPix references to header keywords instead of dependencies

    • Include SUBPRIORITY and shape parameter IVARs in target outputs

  • Include GMM model data for mocks when installing [PR #222].

  • Initial simplistic code for generating sky positions [PR #220]

0.15.0 (2017-09-29)

  • Refactored desitarget.QA to calculate density fluctuations in HEALPixels instead of in bricks [PR #217]:

  • Updated desitarget.io for the DR5 RELEASE number [PR #214]:

  • Updated desitarget.QA to produce QA plots [PR #210]:
  • Changes for mocks [PR #200]:

    • Fix isLRG vs. isLRG_colors

    • Correct random seeds when processing pix in parallel

    • Misc other small bug fixes

  • Added mpi_select_mock_targets

  • Changes for mocks [PR #228]:

    • Refactor of targets_truth_no_spectra

    • Solves bug of healpix patterns present in target mocks.

    • Removes current implementation for target fluctuations.

  • Added desitarget.mock.sky.random_sky [PR #219]

0.14.0 (2017-07-10)

  • Significant update to handle transition from pre-DR4 to post-DR4 data model [PR #189]:

    • desitarget.io can now read old DR3-style and new DR4-style tractor and sweeps files

    • desitarget.cuts now always uses DR4-style column names and formats

    • new 60-bit TARGETID schema that incorporates RELEASE column from imaging surveys

    • desitarget.brightstar builds masks on DR4-style data, uses RELEASE to set DR

    • HEALPix pixel number (current nside=64) added to output target files

    • select_targets passes around information related to HEALPix

    • column PHOTSYS added to output files, recording North or South for the photometric system

    • unit tests that explicitly used columns and formats from the data model have been updated

0.13.0 (2017-06-15)

  • Fix bug when no Lya QSOs are on a brick [PR #191].

  • Additional QA plots for mock target catalogs [PR #190]

  • Additional debugging and support for healpix input to select_mock_targets [PR #186].

  • Set specific DONE, OBS, and DONOTOBSERVE priorities [PR #184].

0.12.0 (2017-06-05)

  • Changed refs to desispec.brick to its new location at desiutil.brick [PR #182].

  • Fix ELG and stdstar mock densities; add mock QA [PR #181].

  • Updated LRG cuts significantly to match proposed shift in LRG target density [PR #179].

  • Major expansion of bright object masking functionality (for circular masks) [PR #176]:

    • Generate SAFE/BADSKY locations around mask perimeters

    • Set the target bits (including TARGETID) for SAFE/BADSKY sky locations

    • Set a NEAR_RADIUS warning for objects close to (but not in) a mask

    • Plot more realistic mask shapes by using ellipses

  • Significant expansion of the mocks-to-targets code [PR #173 and PR #177]:

    • Better and more graceful error handling.

    • Now includes contaminants.

    • Much better memory usage.

    • Updated QA notebook.

  • Add Random Forest selection for ELG in the sandbox [PR #174].

  • Fix ELG and stdstar mock densities; add mock QA [PR #181].

0.11.0 (2017-04-14)

  • New cuts for standards [PR #167]

  • Ensured objtype was being passed to isFSTD().

  • Added mock -> targets+spectra infrastructure

0.10.0 (2017-03-27)

  • Update Travis configuration to catch documentation errors.

  • WIP: refactor of mock.build

  • added mock.spectra module to connect mock targets with spectra

  • fix overflow in LRG sandbox cuts [PR #160]

  • fixed many documentation syntax errors

0.9.0 (2017-03-03)

  • Include mapping from MOCKID -> TARGETID.

  • Added shapes to gaussian mixture model of target params [PR #150].

  • Added basic bright star masking.

  • Updates for mock targets.

  • Added desitarget.sandbox.cuts area for experimental work.

  • Add ELG XD and new LRG to sandbox.

0.8.2 (2016-12-03)

  • Updates for mocks integrated with quicksurvey.

0.8.1 (2016-11-23)

0.8.0 (2016-11-23)

  • Adds DESI_TARGET bits for bright object masking.

  • MTL sets priority=-1 for any target with IN_BRIGHT_OBJECT set.

  • Many updates for reading and manipulating mock targets.

  • Adds BGS_FAINT target selection.

0.7.0 (2016-10-12)

  • Added functionality for Random Forest into quasar selection.

  • Updates to be compatible with Python 3.5.

  • Refactor of merged target list (mtl) code.

  • Update template module file to DESI+Anaconda standard.

0.6.1 (2016-08-18

  • PR #59: fix LRG selection (z < 20.46 not 22.46).

0.6.0 (2016-08-17)

  • Big upgrade for how Tractor Catalogues are loaded to DB. Only the mapping between Catalogue and DB naming is hardcoded. Compatible DR2.

  • Python parallelism. Can choose mulprocessing OR mpi4py.

  • Unit test script that compares random rows from random Catalogues with what is in the DB.

0.5.0 (2016-08-16)

  • Added obscondition and truesubtype to mocks (PR #55; JFR).

  • refactored cut functions to take all fluxes so that they have same call signature (PR #56; JM).

  • Move data into Python package to aid pip installs (PR #47; BAW).

  • Support for Travis, Coveralls and ReadTheDocs (BAW).

0.4.0 (2016-07-12)

  • Updated code from DECaLS DR1 to load DR2 tractor catalaogues to psql db.

  • Basic unit test script for checking that db rows match tractor catalogues.

0.3.3 (2016-03-08)

  • Added isMWSSTAR_colors().

  • Allow user to specify columns when reading tractor files.

  • New code for generating merged target list (MTL).

  • Removed unused npyquery code.

0.3.2 (2016-02-15)

  • Add this changes.rst; fix _version.py.

0.3.1 (2016-02-14)

0.3 (2016-02-14)

  • PR #29 and PR #27 refactor desitarget.cuts to include per-class functions.

  • Other changes in git log before (this changes.rst didn’t exist yet).

  • _version.py is wrong in this tag.