desitarget Change Log

0.41.1 (unreleased)

  • No changes yet.

0.41.0 (2020-08-04)

  • Support for python/3.8 and numpy/1.18, including new tests [PR #631, PR #634]
  • Minor data model fixes, error checks and streamlining [PR #627].
    • The most important change is that MWS science targets are no longer observed in GRAY or DARK, except for MWS_WDs.
  • Cleanup: Avoid absolute path in resource_filename [PR #626].
  • Update masking to be “all-sky” using Gaia/Tycho/URAT [PR #625]:
    • General desitarget functionality to work with Tycho files.
    • Deprecate using the sweeps to mask bright objects as this is now being done using MASKBITS from the imaging catalogs.
    • Functionality to allow masks to be built at different epochs, via careful treatment of Tycho/Gaia/URAT proper motions.
    • Bright star masks are now explicitly written to a $MASK_DIR.
    • The radius-magnitude relationship is now a single function.
    • Refactoring of unit tests to be simpler and have more coverage.
    • Skies and supplemental skies are now always masked by default.
    • A lack of backward compatibility, which should be OK as the masking formalism wasn’t being extensively used.
  • Functionality for iterations of SV beyond sv1 [PR #624]. Includes:
    • A script to create the necessary files for new iterations of SV.
    • Generalized mask/cuts handling for survey=svX, X being any integer.
    • targets.main_cmx_or_sv() also updated to handle survey=svX.
    • Alter the automated creation of output SV target directory names:
      • write svX targets to /targets/svX/ instead of just targets/sv/.
    • Make TARGETID for secondary targets unique for iterations of SVX:
      • Schema is RELEASE=(X-1)*100 + SCND_BIT for SVX-like surveys…
      • …and RELEASE=5*100 + SCND_BIT for the Main Survey.
  • Adjust MWS SV1 target classes for new SV schedule [PR #623]:
    • More generic names for clusters, stream, dwarf targets.
    • Remove ORPHAN, add CV.
    • Lower priority for SEGUE targets.

0.40.0 (2020-05-26)

  • Add RELEASE for dr9i, dr9j (etc.) of the Legacy Surveys [PR #622].
  • Repartition sky files so skies lie in HEALPix boundaries [PR #621]:
    • Previously, unlike other target classes, skies were written such that the brick centers in which they were processed, rather than the sky locations themselves, lay within given HEALPixels.
    • is_sky_dir_official() now checks skies are partitioned right.
    • bin/repartition_skies now reassigns skies to correct HEALPixels.
    • In addition, also includes:
      • Significant (5-10x) speed-ups in read_targets_in_hp().
      • Remove supplemental skies that are near existing sky locations. (which addresses issue #534).
      • A handful of more minor fixes and speed-ups.
  • Various updates to targeting bits and MTL [PR #619]. Includes:
    • Don’t select any BGS_WISE targets in the Main Survey.
    • Always set BGS targets with a ZWARN > 0 to a priority of DONE.
    • Add an informational bit for QSOs selected with the high-z RF (addresses issue #349).
    • MWS targets should drop to a priority of DONE after one observation (but will always be higher priority than BGS for that observation).
    • Update the default priorities for reobserving Lyman-alpha QSOs (as described in issue #486, which this addresses).
  • NUMOBS_MORE for tracer QSOs that are also other targets [PR #617]:
    • Separate the calculation of NUMOBS_MORE into its own function.
    • Consistently use zcut = 2.1 to define Lyman-Alpha QSOs.
    • Check tracer QSOs that are other targets drop to NUMOBS_MORE = 0.
    • New unit test to enforce that check on such tracer QSOs.
    • New unit test to check BGS always gets NUMOBS_MORE = 1 in BRIGHT.
    • Enforce maximum seed in randoms_in_a_brick_from_edges().
  • Update masks for QSO Random Forest selection for DR8 [PR #615]
  • Add a new notebook tutorial about the Merged Target List [PR #614].
  • Recognize (and skip) existing (completed) healpixels when running select_mock_targets [PR #591].

0.39.0 (2020-05-01)

  • Help the mocks run on pixel-level imaging data [PR #611]. Includes:
    • New geomask.get_brick_info() function to look up the brick names associated with each HEALPixel.
    • In randoms.quantities_at_positions_in_a_brick(), add a justlist option to list the (maximal) required input files.
    • Minor bug fixes and documentation updates.
  • Update QSO Random Forest selection (and files) for DR8 [PR #610].

0.38.0 (2020-04-23)

  • Minor updates for latest DR9 imaging versions (dr9f/dr9g) [PR #607].
  • Extra columns and features in the random catalogs [PR #606]:
    • Better error messages and defaults for bin/supplement_randoms.
    • Don’t calculate APFLUX quantities if aprad=0 is passed.
    • Pass the randoms through the finalize and make_mtl functions:
      • To populate columns needed to run fiberassign on the randoms.
      • Addresses issue #597.
    • Add the BRICKID column to the random catalogs.
    • Also add a realistic TARGETID (and RELEASE, BRICK_OBJID).
    • Recognize failure modes more quickly (and fail more quickly).
    • Write out both “resolve” and “noresolve” (North/South) catalogs.
  • Fixes a typo in the priority of MWS_WD_SV targets [PR #601].
  • Fixes calc_priority logic for MWS CMX targets [PR #601].
  • Separate calc_priority() for CMX into a separate function [PR #601].
  • Alter cmx targetmask such that obsconditions can be used to work around MWS/BGS conflicts on MWS CMX tiles [PR #601].
  • Update test_priorities() for new MWS CMX targets scheme [PR #601].
  • Adds SV0_MWS_FAINT bit [PR #601].

0.37.3 (2020-04-15)

  • Update QA now basemap dependency is removed [PR #605]:
    • Also reintroduce unit tests in test_qa.py.
    • basemap dependency was removed in desiutil PR #141

0.37.2 (2020-04-13)

  • Fix select_mock_targets I/O bug reported in #603 [PR #604].

0.37.1 (2020-04-07)

  • Fix mock QSO density bug reported in #594 [PR #602].
  • Fixes a typo in the priority of MWS_WD_SV targets [PR #600].

0.37.0 (2020-03-12)

  • Add SV0_MWS_CLUSTER_ target classes for commissioning [PR #599].
  • Flag the high-z quasar selection in CMX (as SV0_QSO_Z5) [PR #598].
  • Leak of Bright Stars in BGS Main Survey and BGS SV fixed [PR #596].
  • Restrict skies to the geometric boundaries of their brick [PR #595].
  • Changes in CMX after running code for Mini-SV [PR #592]. Includes:
    • g/G >= 16 for SV0_BGS/SV0_MWS/SV0_WD/MINI_SV_BGS_BRIGHT.
    • Remove the LRG LOWZ_FILLER class (both in SV and CMX).
    • Mask on bright in MASKBITS for z~5 QSOs (both in SV and CMX).
    • Remove the ‘low quality’ (lowq) component of SV0_BGS.
    • Add optical MASKBITS flags for LRGs (in Main Survey, SV and CMX).

0.36.0 (2020-02-16)

  • Add Main Survey LRG/ELG/QSO/BGS cuts to CMX for Mini-SV [PR #590].
  • Cut on NOBS > 0 for QSOs and LRGs for Main Survey and SV [PR #589].
  • Fix bug when adding LSLGA galaxies into Main Survey BGS [PR #588]:
    • Catch cases of bytes/str types as well as zero-length strings.
  • Noting (here) that we used the BFG to excise lots of junk [PR #587].
  • Updates and fixes to QA for DR9 [PR #584]. Includes:
    • Options to pre-process and downsample input files to speed testing.
    • Better labeling of QA output, including cleaning up labeling bugs.
    • Make points in scatter plots black to contrast with blue contours.
    • Smarter clipping of dense pixels in histogram plots and sky maps.
    • Print out densest pixels for each target class, with viewer links.
  • Update BGS Main target selection as stated in [PR #581]. Includes:
    • Changes in Fibre Magnitude Cut.
    • LSLGA galaxies manually added to BGS.
      • Future-proof LSLGA object references changing (‘L2’ –> ‘LX’).
    • ‘REF_CAT’ information passed to throught ‘_prepare_optical_wise’.
  • Tune QSO SV selection for both North and South for dr9d [PR #580].

0.35.3 (2020-02-03)

  • Further fixes for DR9 [PR #579]. Includes:
    • Add SERSIC columns for the DR9 data model.
    • Read the bricks file in lower-case in get_brick_info():
      • As, during DR9 testing, it’s been both upper- and lower-case.
    • Set the default nside to None for the randoms:
      • To force the user to choose an nside, or fail otherwise.
    • Fix a numpy future/deprecation warning.
  • Load yaml config file safely in mpi_select_mock_targets [PR #577].
  • Fix bugs in updating primary targets with secondary bits set [PR #574].
  • Adds more stellar SV targets [PR #574].
  • Add LyA features to select_mock_targets [PR #565].

0.35.2 (2019-12-20)

  • Fix z~5 QSO bug in CMX/SV0 that was already fixed for SV [PR #576].

0.35.1 (2019-12-16)

  • Fix bugs triggered by empty files or regions of the sky [PR #575].

0.35.0 (2019-12-15)

  • Preparation for DR9 [PR #573]. Includes:
    • Update data model, maintaining backwards compatibility with DR8.
    • Don’t set the SKY bit when setting the SUPP_SKY bit.
    • Users can input a seed (1, 2, 3, etc.) to bin/select_randoms:
      • This user-provided seed is added to the output file name.
      • Facilitates generating a range of numbered random catalogs.
    • Write out final secondaries using io.find_target_files().
  • More clean-up of glitches and minor bugs [PR #570]. Includes:
    • Remove Python 3.5 unit tests.
    • Catch AssertionError if NoneType input directory when writing.
      • Later (correctly) updated to AttributeError directly in master.
    • Assert the data model when reading secondary target files.
    • Use io.find_target_files() to name priminfo file for secondaries.
    • Allow N < 16 nodes when bundling files for slurm.
    • Use the DR14Q file for SV, not the DR16Q file.
  • Fix bug where wrong SNRs were passed to z~5 QSO selection [PR #569].
  • General clean-up of glitches and minor bugs [PR #564]. Includes:
    • Don’t include BACKUP targets in the pixweight files.
    • Correctly write all all-sky pixels outside of the Legacy Surveys.
    • Propagate flags like –nosec, –nobackup, –tcnames when bundling.
    • Write –tcnames options to header of output target files.
    • Deprecate the sandbox and file-format-check function.
    • Find LSLGAs using ‘L’ in REF_CAT not ‘L2’ (to prepare for ‘L3’).
    • Refactor to guard against future warnings and overflow warnings.
    • Return all HEALpixels at nside in sweep_files_touch_hp().
  • Strict NoneType checking and testing for fiberfluxes [PR #563]:
    • Useful to ensure ongoing compatibility with the mocks.
  • Bitmasks (1,12,13), rfiberflux cut for BGS Main Survey [PR #562].
  • Implement a variety of fixes to select_mock_targets [PR #561].
  • Fixes and updates to secondary.py [PR #530]:
    • Fix a bug that led to incorrect OBSCONDITIONS for secondary-only targets.
    • Secondary target properties can override matched primary properties, but only for restricted combinations of DESI_TARGET bits (MWS and STD).
  • Add stellar SV targets [PR #530]:
    • Add MWS SV target definitions in sv1_targetmask and cuts.
    • Science WDs are now a secondary target class.
    • Adds a bright limit to the MWS-NEARBY sample.
    • Add stellar SV secondary targets in sv1_targetmask.
    • Remove the BACKSTOP secondary bit.

0.34.0 (2019-11-03)

  • Update SV0 (BGS, ELG, LRG, QSO) classes for commissioning [PR #560].
    • Also add new STD_DITHER target class for commissioning.
  • All-sky/backup targets, new output data model [PR #558]. Includes:
    • Add all-sky/backup/supplemental targets for SV.
    • Add all-sky/backup/supplemental targets for the Main survey.
    • Write dark/bright using, e.g. targets/dark/targets-*.fits format.
    • New targets/targets-supp/targets-*.fits format for output.
    • Add io.find_target_files() to parse output data model.
    • File names now generated automatically in io.write_* functions:
      • File-name-generation used by randoms, skies, targets and gfas.
      • select_* binaries for these classes use this functionality.
    • Change CMX BACKUP_FAINT limit to G < 19.

0.33.3 (2019-10-31)

  • Add cuts for z = 4.3-4.8 quasar into the z5QSO selection [PR #559].

0.33.2 (2019-10-17)

  • Add FIBERFLUX_IVAR_G/R/Z to mock skies when merging [PR #556].
  • Fix minor bugs in select_mock_targets [PR #555].
  • Update the ELG selections for SV [PR #553]. Includes:
    • Four new bit names:
      • ELG_SV_GTOT, ELG_SV_GFIB.
      • ELG_FDR_GTOT, ELG_FDR_GFIB.
    • Associated new ELG selections with north/south differences.
    • Propagate FIBERFLUX_G from the sweeps for SV ELG cuts.
    • Increase the default sky densities by a factor of 4x.
    • Relax CMX BACKUP_FAINT limit to G < 21 to test fiber assign.
  • Bright-end FIBERFLUX_R cut on BGS_FAINT_EXT in SV [PR #552].
  • Update LRG selections for SV [PR #550]. Includes:
    • The zfibermag faint limit is changed from 21.6 to 21.9.
    • IR-selected objects with r-W1>3.1 not subjected to the sliding cut.

0.33.1 (2019-10-13)

  • Enhancements and on-sky clean-up for SV and CMX [PR #551]. Includes:
    • Add areas contingent on MASKBITS to the pixweight- files.
    • Change APFLUX to FIBERFLUX for skies and supp-skies.
    • Add new M33 First Light program.
    • Change priorities for the First Light programs.
    • Retain Tycho, and sources with no measured proper motion, in GFAs.
    • Add the REF_EPOCH column to all target files.

0.33.0 (2019-10-06)

  • Update skies, GFAs and CMX targets for all-sky observing [PR #548]:
    • Process and output GFAs, skies and CMX targets split by HEALPixel.
    • “bundling” scripts to parallelize GFAs, skies, CMX by HEALPixel.
    • Bundle across all HEALPixels (not just those in the footprint).
    • Add pixel information to file headers for GFAs, skies and CMX.
    • Write all-sky CMX targets separately from in-footprint targets.
    • Add back-up and first light targets for commissioning.
    • New TARGETID encoding scheme for Gaia-only and first light targets.
    • Resolve commissioning targets from the Legacy Surveys.
    • io.read functions can now process SKY and GFA target files.
    • New function to read in targets restricted to a set of DESI tiles.
    • Implement Boris Gaensicke’s geographical cuts for Gaia.
    • Update unit tests to use DR8 files.
  • Further updates to changes in PR #531, [PR #544]. Includes:
    • A –writeall option to select_secondary writes a unified target file without the BRIGHT/DARK split, as for select_targets
    • Removes duplicate secondaries that arise from multiple matches to one primary and secondary targets appearing in more than one input file. The duplciate with highest PRIORTIY_INIT is retained.
  • Update mocks to match latest data-based targets catalogs [PR #543].
  • Add new redshift 5 (QSO_Z5) SV QSO selection [PR #539]. Also:
    • Remove all Tycho and LSLGA sources from the GFA catalog.
    • Minor improvements to documentation for secondary targets.
    • Use N/S bricks for skies when S/N bricks aren’t available.
  • Tune, high-z, faint (QSO_HZ_F) SV QSO selection [PR #538]
  • Use SPECTYPE from zcat to set NUMOBS_MORE [PR #537]:
    • Updates behavior for tracer QSOs vs. LyA QSOs in MTL.
  • Update LRG selections for DR8 [PR #532]. Includes:
    • New LRG selection for SV with fewer bits.
    • New LOWZ_FILLER class for SV.
    • Add LRG 4PASS and 8PASS bits/classes using cuts on FLUX_Z.
    • New and simplified LRG selection for the Main Survey.
    • Deprecate Main Survey 1PASS/2PASS LRGs, all LRGs now have one pass.
    • Deprecate some very old code in desitarget.targets.
  • Finalize secondaries, add BRIGHT/DARK split [PR #531]. Includes:
    • Updated data model for secondaries.
    • New secondary output columns (OBSCONDITIONS, proper motions).
    • Add a cached file of primary TARGETIDs to prevent duplicates.
    • Create a more reproducible TARGETID for secondaries.
    • Automatically write secondaries split by BRIGHT/DARK.
    • Add option to pass secondary file in MTL.
    • Insist on observing layer/conditions for MTL:
      • Ensures correct behavior for dark targets in bright time…
        • …and bright-time targets observed in dark-time.
    • Minor update to the MWS_BROAD class.
  • Add info on versioning, main_cmx_or_sv to bitmask notebook [PR #527]

0.32.0 (2019-08-07)

  • Add URAT catalog information [PR #526]. Includes:
    • New module to retrieve URAT data from Vizier and reformat it.
    • Code to match RAs/Decs to URAT, as part of that new URAT module.
    • Substitute URAT PMs for GFAs where Gaia has not yet measured PMs.
  • Update CMX and Main Survey target classes [PR #525]. Includes:
    • New SV0_WD, SV0_STD_FAINT target classes for commissioning.
    • Mild updates to SV0_BGS and SV0_MWS for commissioning.
    • New BGS_FAINT_HIP (high-priority BGS) Main Survey class.
    • Explicit checking on ASTROMETRIC_PARAMS_SOLVED for MWS targets.
    • Add 3-sigma parallax slop in MWS_MAIN survey target class.
  • Add OBSCONDITIONS to target files [PR #523] Also includes:
    • Split target files explicitly into bright and “graydark” surveys.
    • Default to such a file-spilt for SV and Main (not for cmx).
    • Adds an informational bit for supplemental sky locations.
  • Use MASKBITS instead of BRIGHTSTARINBLOB [PR #521]. Also:
    • Extra options and checks when making and vetting bundling scripts.
    • Option to turn off commissioning QSO cuts to speed unit tests.
  • Add ELG/LRG/QSO/STD selection cuts for commissioning [PR #519].
  • Add full set of columns to supplemental skies file [PR #518].
  • Fix some corner cases when reading HEALPixel-split files [PR #518].

0.31.1 (2019-07-05)

  • Pass Gaia astrometric excess noise in cmx MWS SV0 [PR #516].

0.31.0 (2019-06-30)

  • MASKBITS of BAILOUT for randoms when no file is found [PR #515].
  • Near-trivial fix for an unintended change to the isELG API introduced in PR #513 [PR #514].
  • Preliminary ELG cuts for DR8 imaging for main and SV [PR #513].
    • Don’t deprecate wider SV bits, yet, ELGs may still need them.
  • Further updates to generating randoms for DR8. [PR #512]. Includes:
    • Add WISE depth maps to random catalogs and pixweight files.
    • Code to generate additional supplemental randoms catalogs.
      • Supplemental, here, means (all-sky) outside of the footprint.
    • Executable to split a random catalog into N smaller catalogs.
    • Fixes a bug in targets.main_cmx_or_sv().
      • Secondary columns now aren’t the default if rename is True.
    • Better aligns data model with expected DR8 directory structure.
      • Also fixes directory-not-found bugs when generating skies.
  • Add “supplemental” (outside-of-footprint) skies [PR #510]:
    • Randomly populates sky area beyond some minimum Dec and Galactic b.
    • Then avoids all Gaia sources at some specified radius.
    • Fixes a bug where geomask.hp_in_box() used geodesics for Dec.
      • Dec cuts should be small circles, not geodesics.
  • First implementation for secondary targets [PR #507]. Includes:
    • Framework and design for secondary targeting process.
    • Works automatically for both Main Survey and SV files.
    • New bitmasks for secondaries that populate SCND_TARGET column.
      • can have any PRIORITY_INIT and NUMOBS_INIT.
    • A reserved “veto” bit to categorically reject targets.
    • Rigorous checking of file formats…
      • …and that files correspond to secondary bits.
    • Example files and file structure (at NERSC) in SCND_DIR.
      • /project/projectdirs/desi/target/secondary.
    • Secondary targets are matched to primary targets on RA/Dec.
      • unless a (per-source) OVERRIDE column is set to True.
    • Secondary-primary matches share the primary TARGETID.
    • Non-matches and overrides have their own TARGETID.
      • with RELEASE == 0.
    • Non-override secondary targets are also matched to themselves.
      • TARGETID and SCND_TARGET correspond for matches.

0.30.1 (2019-06-18)

  • Fix normalization bug in QSO tracer/Lya mock target densities [PR #509].
  • Tune “Northern” QSO selection and color shifts for Main and SV [PR #506]
  • Follow-up PR to PR #496 with two changes and bug fixes [PR #505]:
    • Select QSO targets using random forest by default.
    • Bug fix: Correctly populate REF_CAT column (needed to correctly set MWS targeting bits).

0.30.0 (2019-05-30)

  • Drop Gaia fields with np.rfn to fix Python 3.6/macOS bug [PR #502].
  • Apply the same declination cut to the mocks as to the data [PR #501].
  • Add information to GFA files [PR #498]. Includes:
    • Add columns PARALLAX, PARALLAX_IVAR, REF_EPOCH.
    • Remove REF_EPOCH from GFA file header, as it’s now a column.
    • Sensible defaults for Gaia-only REF_EPOCH, RA/DEC_IVAR.
    • Use fitsio.read() instead of desitarget.io.read_tractor().
      • It’s faster and special handling of input files isn’t needed.
  • General clean-up of target selection code [PR #497]. Includes:
    • Deprecate old functions in desitarget.gfa.
    • Greatly simplify io.read_tractor().
      • Backwards-compatability is now only guaranteed for DR6-8.
    • Guard against warnings (e.g. divide-by-zero) in cuts and SV cuts.
    • Default to only passing North (S) sources through North (S) cuts.
      • Retain previous behavior if --noresolve flag is passed.
  • Add SV support to select_mock_targets [PR #496]
  • A few more updates and enhancements for DR8 [PR #494]. Includes:
    • Add WISEMASK_W1 and WISEMASK_W2 to random catalogs.
    • Deprecate BRIGHTBLOB in favor of MASKBITS for targets.
    • Add qso_selection==colorcuts in set_target_bits.sv1_cuts()
      • This should facilitate QSO selection for SV mocks.
  • Add REF_CAT and Gaia BP and RP mags and errors to GFAs [PR #493].
  • Minor bug fix in how select_mock_targets handles Lya targets [PR #444].
  • Further updates and enhancements for DR8 [PR #490]. Includes:
    • Resolve sky locations and SV targets in North/South regions.
    • Update sky and SV slurming for DR8-style input (two directories).
    • Write both of two input directories to output file headers.
    • Parallelize plot production to speed-up QA by factors of 8.
    • Add PSFSIZE to randoms, pixweight maps and QA plots.
    • QA and pixweight maps work fully for SV-style files and bits.
    • Pixweight code can now take HEALpixel-split targets as input.
    • Add aperture-photometered background flux to randoms catalogs.
    • Additional unit test module (test.test_geomask()).
    • Deprecate make_hpx_density_file; use make_imaging_weight_map.
    • io.read_targets_in_a_box() can now read headers.
    • Update unit test data for new DR8 columns and functionality.
  • Update QSO targeting algorithms for DR8 [PR #489]. Includes:
    • Update baseline quasar selection for the main survey.
    • Update QSO bits and selection algorithms for SV.
  • Remove GFA/Gaia duplicates on REF_ID not BRICKID [PR #488].
  • Various bug and feature fixes [PR #484]. Includes:
    • Fix crash when using sv_select_targets with –tcnames.
    • Only import matplotlib where explicitly needed.
  • Update select_mock_targets to (current) DR8 data model [PR #480].

0.29.1 (2019-03-26)

  • Add REF_CAT, WISEMASK_W1/W2 to DR8 data model [PR #479].
  • Use speed of light from scipy [PR #478].

0.29.0 (2019-03-22)

  • Update SV selection for DR8 [PR #477]. Includes:
    • New SV targeting bits for QSOs and LRGs.
    • New SV selection algorithms for QSOs, ELGs and LRGs.
    • MTL fixes to handle SV LRGs (which are now not 1PASS/2PASS).
    • QA can now interpret HEALPixel-split targeting files.
    • Updated test files for the quasi-DR8 imaging data model.
    • SKY and BAD_SKY added to commissioning bits yaml file.
    • Randoms in overlap regions, and for DR8 dual directory structure.
    • Write overlap regions in addition to resolve for targets/randoms.
  • Change instances of yaml.load to yaml.safe_load [PR #475].
  • Fix Gaia files format in doc string (healpix not healpy) [PR #474].
  • Write Gaia morphologies and allow custom tilings for GFAs [PR #467].
  • Initial updates for DR8 [PR #466]. Includes:
    • DR8 data model updates (e.g BRIGHTSTARBLOB -> bitmask BRIGHTBLOB).
    • Apply resolve capability to targets and randoms.
    • Handle BASS/MzLS and DECaLS existing in the same input directory.
  • New resolve capability for post-DR7 imaging [PR #462]. Includes:
    • Add RELEASE to GFA data model to help resolve duplicates.
    • Resolve N/S duplicates by combining RELEASE and areal cuts.
    • Apply the new resolve code (targets.resolve()) to GFAs.
    • Deprecate Gaia-matching code for GFAs, as we no longer need it.
  • Add code to select GFAs for cmx across wider sky areas [PR #461].

0.28.0 (2019-02-28)

  • desitarget.mock.build.targets_truth fixes for new priority calcs [PR #460].
  • Updates to GFAs and skies for some cmx issues [PR #459]. Includes:
    • Assign BADSKY using BLOBDIST rather than aperture fluxes.
    • Increase default density at which sky locations are generated.
    • Store only aperture fluxes that match the DESI fiber radius.
    • Ensure GFAs exist throughout the spectroscopic footprint.
  • Refactor SV/main targeting for spatial queries [PR #458]. Includes:
    • Many new spatial query capabilities in desitarget.geomask.
    • Parallelize target selection by splitting across HEALPixels.
    • Wrappers to read in HEALPix-split target files split by:
      • HEALPixels, RA/Dec boxes, RA/Dec/radius caps, column names.
    • Only process subsets of targets in regions of space, again including:
      • HEALPixels, RA/Dec boxes, RA/Dec/radius caps.
    • New unit tests to check these spatial queries.
    • Updated notebook including tutorials on spatial queries.
  • Update the SV selections for BGS [PR #457].
  • Update MTL to work for SV0-like cmx and SV1 tables [PR #456]. Includes:
    • Make SUBPRIORITY a random number (0->1) in skies output.
    • New targets.main_cmx_or_sv() to parse flavor of survey.
    • Update targets.calc_priority() for SV0-like cmx and SV1 inputs.
    • mtl.make_mtl() can now process SV0-like cmx and SV1 inputs.
    • New unit tests for SV0-like cmx and SV1 inputs to MTL.
  • Deprecate targets.calc_priority() that had table copy [PR #452].
  • Update SV QSO selections, add seed and DUST_DIR for randoms [PR #449].
  • Style changes to conform to PEP 8 [PR #446], [PR #447], [PR #448].

0.27.0 (2018-12-14)

  • Remove reliance on Legacy Surveys for Gaia data [PR #438]. Includes:
    • Use $GAIA_DIR environment variable instead of passing a directory.
    • Functions to wget Gaia DR2 CSV files and convert them to FITS.
    • Function to reorganize Gaia FITS files into (NESTED) HEALPixels.
    • Use the NESTED HEALPix scheme for Gaia files throughout desitarget.
    • Change output column TYPE to MORPHTYPE for GFAs.
  • Move select-mock-targets.yaml configuration file to an installable location for use by desitest [PR #436].
  • Significant enhancement and refactor of select_mock_targets to include stellar and extragalactic contaminants [PR #427].

0.26.0 (2018-12-11)

  • Refactor QSO color cuts and add hard r > 17.5 limit [PR #433].
  • Refactor of MTL and MTL-related enhancements [PR #429]. Includes:
    • Use targets file NUMOBS_INIT not targets.calc_numobs().
    • Use targets file PRIORITY_INIT not targets.calc_priority().
    • Remove table copies from desitarget.mtl to use less memory.
    • New function targets.calc_priority_no_table() to use less memory.
    • Set informational (NORTH/SOUTH) bits to 0 PRIORITY and NUMOBS.
    • Set priorities using LRG_1PASS/2PASS bits rather than on LRG.
  • Minor updates to select_mock_targets [PR #425].
    • Use pre-computed template photometry (requires v3.1 basis templates).
    • Include MW dust extinction in the spectra.
    • Randomly assign a radial velocity to superfaint mock targets.
  • Update default mock catalogs used by select_mock_targets [PR #424]
  • Update Random Forests for DR7 quasar selection [PR #423]
  • Fix bugs in main MWS selections [PR #422].
  • Fix python setup.py install for cmx and sv1 directories [PR #421].
  • More updates to target classes, mainly for SV [PR #418]. Includes:
    • First full implementations of QSO, LRG, ELG, and STD for SV.
    • Update and refactor of MWS and BGS classes for the main survey.
    • Change name of main survey MWS_MAIN class to MWS_BROAD.
    • Augment QA code to handle SV sub-classes such as ELG_FDR_FAINT.

0.25.0 (2018-11-07)

  • Randomize mock ordering for Dark Sky mocks which aren’t random [PR #416].
  • Updates to several target classes [PR #408]. Includes:
    • Refactor of the ELG and MWS_MAIN selection algorithms.
    • Update of the ELG and MWS_MAIN selection cuts.
    • Change MWS_WD priority to be higher than that of BGS target classes.
    • Set skies to BAD only if both g-band and r-band are missing.
  • Refactor of BGS selections to separate masking and color cuts [PR #407].
  • Quicksurvey MTL fix [PR #405].
  • Mocks use QSO color cuts instead of random forest [PR #403].
  • Updates to Bright Galaxy Survey and QSO selections [PR #402]. Includes:
    • Updates to BGS_FAINT and BGS_BRIGHT target selections.
    • New BGS_WISE selection and implementation.
    • New data model columns BRIGHTSTARINBLOB and FRACIN_.
    • Add cut on BRIGHTSTARINBLOB to QSO selection.
    • Modify I/O to retain (some) backwards-compatibility between DR6 and DR7.
    • Updated unit test example files with appropriate columns.
    • Speed-up of cuts unit tests without loss of coverage.
  • Updated mock sky catalog with positions over a larger footprint [PR #398].
  • Major update to select_mock_targets to use the latest (v3.0) basis templates [PR #395].
  • Propagate per-class truth HDUs into final merged truth file [PR #393].
  • Incorporate simple WISE depth model in select_mock_targets which depends on ecliptic latitude [PR #391].

0.24.0 (2018-09-26)

  • Fix bug in code that produces data for unit tests [PR #387].
  • Rescale spectral parameters when generating and querying kd-trees in select_mock_targets [PR #386].
  • Bug fixes: [PR #383].
    • Use parallax_err when selecting MWS_NEARBY targets.
    • In select_mock_targets do not use Galaxia to select WDs and 100pc targets.
  • Refactor QA to work with commissioning and SV files and add (first) unit tests for QA. [PR #382].
  • Estimate FIBERFLUX_[G,R,Z] for mock targets. [PR #381].
  • First fully working version of SV code [PR #380]. Includes:
    • (Almost) the only evolving part of the code for SV is now the cuts.
    • Unit tests for SV that should be easy to maintain.
    • Bit and column setting for SV that should be maintainable.
    • SV0 (commissioning) MWS cuts.
    • Updated STD cuts to fix a fracmasked typo.
    • Alterations to Travis coverage to exclude some external code.
  • Fix a bug which resulted in far too few standard stars being selected in the mocks [PR #378].
  • Fix a bug in how the objtruth tables are written out to by select_mock_targets [PR #374].
  • Remove Python 2.7 from Travis, add an allowed-to-fail PEP 8 check [PR #373].
  • Function to read RA, DEC from non-standard external files [PR #372].
  • Update the data model for output target files [PR #372]:
    • Change TYPE to MORPHTYPE.
    • Add EBV, FIBERFLUX_G,R,Z and FIBERTOTFLUX_G,R,Z.
  • Additional commissioning (cmx) classes and priorities [PR #370]. Includes:
    • New functions to define several more commissioning classes.
    • A $CMX_DIR to contain files of cmx sources to which to match.
    • An example $CMX_DIR is /project/projectdirs/desi/target/cmx_files.
    • Functionality to reset initial priorities for commissioning targets.
    • Downloading fitsio using pip/astropy to fix Travis.
  • Significant enhancement of select_mock_targets (see PR for details) [PR #368].
  • Include per-band number counts for targets on the QA pages [PR #367].
  • Use new desiutil.dust.SFDMap() module [PR #366].
  • Set the STD_WD bit (it’s identical to the MWS_WD bit) [PR #364].
  • Add notebook for generating Gaussian mixture models from DR7 photometry and morphologies of ELG, LRG, and BGS targets [PR #363 and PR #365].
  • Make commissioning (cmx) target selection fully functional [PR #359]. Includes:
    • Initial target selection algorithms.
    • First unit tests for cmx (> 90% coverage).
    • SV_TARGET and CMX_TARGET as output columns instead of as a bit.
  • Remove “legacy” code in QA [PR #359].
    • Weight maps can now be made with desitarget.randoms.pixmap().
  • Add isELG_colors functions [PR #357].
  • Adapt cuts.isSTD_colors to deal with different north/south color-cuts [PR #355].
  • Refactor to allow separate commissioning and SV target selections [PR #346]:
    • Added sv and commissioning directories.
    • New infrastructure to have different cuts for SV and commissioning:
      • separate target masks (e.g. sv/data/sv_targetmask.yaml).
      • separate cuts modules (e.g. sv_cuts.py).
    • Added executables for SV/commissioning (e.g. select_sv_targets).
    • Initial NUMOBS and PRIORITY added as columns in targets- files.
    • Initial NUMOBS is now hardcoded in target masks, instead of being set by MTL.
    • SV bits added to target masks to track if targets are from SV/comm/main.
    • sv/comm/main can now be written to the header of the targets- files.
    • SUBPRIORITY is set when writing targets to facilitate reproducibility.
  • Set NUMOBS for LRGs in MTL using target bits instead of magnitude [PR #345].
  • Update GFA targets [PR #342]:
    • Handle reading Gaia from sweeps as well as matching. Default to not matching.
    • Makes Gaia matching radius stricter to return only the best Gaia objects.
    • Retains Gaia RA/Dec when matching, instead of RA/Dec from sweeps.
    • Fixes a bug where Gaia objects in some HEALPixels weren’t being read.
    • Add Gaia epoch to the GFA file header (still needs passed from the sweeps).

0.23.0 (2018-08-09)

Includes non-backwards compatible changes to standard star bit names.

  • STD/STD_FSTAR -> STD_FAINT, with corresponding fixes for mocks [PR #341].
  • Match sweeps to Gaia and write new sweeps with Gaia columns [PR #340]:
    • Also add BRIGHTSTARINBLOB (if available) to target output files.
    • And include a flag to call STD star cuts function without Gaia columns.

0.22.0 (2018-08-03)

Includes non-backwards compatible changes to standard star target mask bit names and selection function names.

  • Produce current sets of target bits for DR7 [PR #338]:
    • Update the LRG, QSO, STD and MWS algorithms to align with the wiki.
    • In particular, major updates to the STD and MWS selections.
    • Don’t match to Gaia by default, only if requested.
    • Maintain capability to match to Gaia if needed for earlier Data Releases.
    • Run subsets of target classes by passing, e.g.. --tcnames STD,QSO.
    • Update unit test files to not rely on Gaia.
    • Bring Data Model into agreement with Legacy Surveys sweeps files.
    • Rename FSTD to be STD throughout.
    • QA fails gracefully if weight maps for systematics aren’t passed.

0.21.1 (2018-07-26)

  • Update the schema for target selection QA [PR #334]:
    • Sample imaging pixels from the Legacy Surveys to make random catalogs.
    • Add E(B-V) from SFD maps and stellar densities from Gaia to the randoms.
    • Sample randoms to make HEALpixel maps of systematics and target densities.
    • Sample randoms in HEALPixels to precisely estimate imaging footprint areas.
    • Make several new systematics plots.
    • Make new plots of parallax and proper motion information from Gaia.

0.21.0 (2018-07-18)

  • Fix bug when generating targeting QA for mock catalogs [PR #332].
  • Add support for GAMA/BGS mocks and new calib_only option in mock.targets_truth [PR #331].
  • Add RA_IVAR and DEC_IVAR to the GFA Data Model [PR #329].
  • Update the Gaia Data Model [PR #327]:
    • Output columns formatted as expected downstream for GFA assignment.
    • Align Gaia Data Model in matching and I/O with the Legacy Surveys.
  • Allow environment variables in select_mock_targets config file [PR #325].
  • First version of Milky Way Survey selection [PR #324]:
    • Catalog-matches to Gaia using desitarget.gaimatch.
    • Sets MWS_MAIN, MWS_WD and MWS_NEARBY bits.
    • Makes individual QA pages for MWS (and other) bits.
  • Change GFA selection to be Gaia-based [PR #322]:
    • Update the select_gfas binary to draw from Gaia DR2.
    • Parallelize across sweeps files to add fluxes from the Legacy Surveys.
    • Gather all Gaia objects to some magnitude limit in the sweeps areas.
  • Add desitarget.gaimatch for matching to Gaia [PR #322]:
    • Can perform object-to-object matching between Gaia and the sweeps.
    • Can, in addition, retain all Gaia objects in an RA/Dec box.
  • Mock targets bug fixes [PR #318].
  • Add missing GMM files to installations [PR #316].
  • Introduction of pixel-level creation of sky locations [PR #313]:
    • Significant update of desitarget.skyfibers
    • desitarget.skyutilities.astrometry to remove astrometry.net dependency.
    • desitarget.skyutilities.legacypipe to remove legacypipe dependency.
    • Grids sky locations by applying a binary erosion to imaging blob maps.
    • Sinks apertures at the resulting sky locations to derive flux estimates.
    • Sets the BAD_SKY bit using high flux levels in those apertures.
    • desitarget.skyfibers.bundle_bricks() to write a slurm script.
    • Parallelizes via HEALPixels to run in a few hours on interactive nodes.
    • Adds the select_skies binary to run from the command line.
    • Includes gather_skies binary to collect results from parallelization.
    • Adds functionality to plot good/bad skies against Legacy Survey images.
  • select_mock_targets full footprint updates [PR #312].
  • QA fix for testing without healpix weight map [PR #311].
  • New QSO random forest [PR #309].
  • Restore the no-spectra option of select_mock_targets, for use with quicksurvey [PR #307].
  • Better handling of imaging survey areas for QA [PR #304]:
    • desitarget.imagefootprint to build HEALPix weight maps of imaging.
    • Executable (bin) interface to make weight maps from the command line.
    • desitarget.io loader to resample maps to any HEALPix nside.
    • Update desitarget.QA to handle new imaging area weight maps.
  • Improve north/south split functions for LRG and QSO color cuts [PR #302].
  • Minor QA and selection cuts updates [PR #297]:
    • QA matrix of target densities selected in multiple classes.
    • Functions to allow different north/south selections for LRGs.

0.20.1 (2018-03-29)

  • Add a bright (g>21) flux cut for ELGs. [PR #296].

0.20.0 (2018-03-24)

  • Added compare_target_qa script [PR #289].
  • Astropy 2.x compatibility [PR #291].
  • Update of sky selection code [PR #290]. Includes:
    • Use the desitarget.brightmask formalism to speed calculations.
    • Pass around a magnitude limit on masks from the sweeps (to better avoid only objects that are genuinely detected in the sweeps).
    • Reduce the default margin to produce ~1700 sky positions per sq. deg.
  • Retuning of DR6 target densities [PR #294]. Includes:
    • Tweaking the QSO random forest probability.
    • Adding a new ELG selection for the northern (MzLS/BASS) imaging.
    • Slight flux shifts to reconcile the northern and southern (DECaLS) imaging.
    • Initial functionality for different North/South selections.
  • Some reformatting of output target files and bits [PR #294]:
    • Introducing a NO_TARGET bit.
    • Renaming the BADSKY bit BAD_SKY for consistency with other bits.
    • Including FRACDEV and FRACDEV_IVAR as outputs.

0.19.1 (2018-03-01)

  • Fix bug whereby FLUX and WAVE weren’t being written to truth.fits files [PR #287].
  • Include OBSCONDITIONS in mock sky/stdstar files for fiberassign [PR #288].

0.19.0 (2018-02-27)

This release includes significant non-backwards compatible changes to importing target mask bits and how mock spectra are generated.

  • Major refactor of select_mock_targets code infrastructure [PR #264].
  • Restructure desi_mask, bgs_mask, etc. imports to fix readthedocs build [PR #282].
  • Update RELEASE dictionary with 6000 (northern) for DR6 [PR #281].

0.18.1 (2018-02-23)

  • Open BGS hdf5 mocks read-only to fix parallelism bug [PR #278].

0.18.0 (2018-02-23)

  • New target density fluctuations model based on DR5 healpixel info [PR #254].
  • Include (initial) mock QA plots on targeting QA page [PR #262]
  • Added select_gfa script [PR #275]
  • Update masking for ellipses (“galaxies”) in addition to circles (“stars”) [PR #277].

0.17.1 (2017-12-20)

  • HPXNSIDE and HPXPIXEL as header keywords for mocks too [PR #246].

0.17.0 (2017-12-20)

  • Support LyA skewers v2.x format [PR #244].
  • Split LRGs into PASS1/PASS2 separate bits [PR #245].
  • Sky locations infrastructure [PR #248].
  • Mock targets densities fixes [PR #241 and PR #242].

0.16.2 (2017-11-16)

  • Allows different star-galaxy separations for quasar targets for different release numbers [PR #239].

0.16.1 (2017-11-09)

  • fixes to allow QA to work with mock data [PR #235].
  • cleanup of mpi_select_mock_targets [PR #235].
  • adds BGS properties notebook documentation [PR #236].

0.16.0 (2017-11-01)

  • General clean-up prior to running DR5 targets [PR #229].
    • Use desiutil.log instead of verbose (everywhere except mocks)
    • Change HEALPix references to header keywords instead of dependencies
    • Include SUBPRIORITY and shape parameter IVARs in target outputs
  • Include GMM model data for mocks when installing [PR #222].
  • Initial simplistic code for generating sky positions [PR #220]

0.15.0 (2017-09-29)

  • Refactored desitarget.QA to calculate density fluctuations in HEALPixels instead of in bricks [PR #217]:
  • Updated desitarget.io for the DR5 RELEASE number [PR #214]:
  • Updated desitarget.QA to produce QA plots [PR #210]:
  • Changes for mocks [PR #200]:
    • Fix isLRG vs. isLRG_colors
    • Correct random seeds when processing pix in parallel
    • Misc other small bug fixes
  • Added mpi_select_mock_targets
  • Changes for mocks [PR #228]:
    • Refactor of targets_truth_no_spectra
    • Solves bug of healpix patterns present in target mocks.
    • Removes current implementation for target fluctuations.
  • Added desitarget.mock.sky.random_sky [PR #219]

0.14.0 (2017-07-10)

  • Significant update to handle transition from pre-DR4 to post-DR4 data model [PR #189]:
    • desitarget.io can now read old DR3-style and new DR4-style tractor and sweeps files
    • desitarget.cuts now always uses DR4-style column names and formats
    • new 60-bit TARGETID schema that incorporates RELEASE column from imaging surveys
    • desitarget.brightstar builds masks on DR4-style data, uses RELEASE to set DR
    • HEALPix pixel number (current nside=64) added to output target files
    • select_targets passes around information related to HEALPix
    • column PHOTSYS added to output files, recording North or South for the photometric system
    • unit tests that explicitly used columns and formats from the data model have been updated

0.13.0 (2017-06-15)

  • Fix bug when no Lya QSOs are on a brick [PR #191].
  • Additional QA plots for mock target catalogs [PR #190]
  • Additional debugging and support for healpix input to select_mock_targets [PR #186].
  • Set specific DONE, OBS, and DONOTOBSERVE priorities [PR #184].

0.12.0 (2017-06-05)

  • Changed refs to desispec.brick to its new location at desiutil.brick [PR #182].
  • Fix ELG and stdstar mock densities; add mock QA [PR #181].
  • Updated LRG cuts significantly to match proposed shift in LRG target density [PR #179].
  • Major expansion of bright object masking functionality (for circular masks) [PR #176]:
    • Generate SAFE/BADSKY locations around mask perimeters
    • Set the target bits (including TARGETID) for SAFE/BADSKY sky locations
    • Set a NEAR_RADIUS warning for objects close to (but not in) a mask
    • Plot more realistic mask shapes by using ellipses
  • Significant expansion of the mocks-to-targets code [PR #173 and PR #177]:
    • Better and more graceful error handling.
    • Now includes contaminants.
    • Much better memory usage.
    • Updated QA notebook.
  • Add Random Forest selection for ELG in the sandbox [PR #174].
  • Fix ELG and stdstar mock densities; add mock QA [PR #181].

0.11.0 (2017-04-14)

  • New cuts for standards [PR #167]
  • Ensured objtype was being passed to isFSTD().
  • Added mock -> targets+spectra infrastructure

0.10.0 (2017-03-27)

  • Update Travis configuration to catch documentation errors.
  • WIP: refactor of mock.build
  • added mock.spectra module to connect mock targets with spectra
  • fix overflow in LRG sandbox cuts [PR #160]
  • fixed many documentation syntax errors

0.9.0 (2017-03-03)

  • Include mapping from MOCKID -> TARGETID.
  • Added shapes to gaussian mixture model of target params [PR #150].
  • Added basic bright star masking.
  • Updates for mock targets.
  • Added desitarget.sandbox.cuts area for experimental work.
  • Add ELG XD and new LRG to sandbox.

0.8.2 (2016-12-03)

  • Updates for mocks integrated with quicksurvey.

0.8.1 (2016-11-23)

0.8.0 (2016-11-23)

  • Adds DESI_TARGET bits for bright object masking.
  • MTL sets priority=-1 for any target with IN_BRIGHT_OBJECT set.
  • Many updates for reading and manipulating mock targets.
  • Adds BGS_FAINT target selection.

0.7.0 (2016-10-12)

  • Added functionality for Random Forest into quasar selection.
  • Updates to be compatible with Python 3.5.
  • Refactor of merged target list (mtl) code.
  • Update template module file to DESI+Anaconda standard.

0.6.1 (2016-08-18

  • PR #59: fix LRG selection (z < 20.46 not 22.46).

0.6.0 (2016-08-17)

  • Big upgrade for how Tractor Catalogues are loaded to DB. Only the mapping between Catalogue and DB naming is hardcoded. Compatible DR2.
  • Python parallelism. Can choose mulprocessing OR mpi4py.
  • Unit test script that compares random rows from random Catalogues with what is in the DB.

0.5.0 (2016-08-16)

  • Added obscondition and truesubtype to mocks (PR #55; JFR).
  • refactored cut functions to take all fluxes so that they have same call signature (PR #56; JM).
  • Move data into Python package to aid pip installs (PR #47; BAW).
  • Support for Travis, Coveralls and ReadTheDocs (BAW).

0.4.0 (2016-07-12)

  • Updated code from DECaLS DR1 to load DR2 tractor catalaogues to psql db.
  • Basic unit test script for checking that db rows match tractor catalogues.

0.3.3 (2016-03-08)

  • Added isMWSSTAR_colors().
  • Allow user to specify columns when reading tractor files.
  • New code for generating merged target list (MTL).
  • Removed unused npyquery code.

0.3.2 (2016-02-15)

  • Add this changes.rst; fix _version.py.

0.3.1 (2016-02-14)

0.3 (2016-02-14)

  • PR #29 and PR #27 refactor desitarget.cuts to include per-class functions.
  • Other changes in git log before (this changes.rst didn’t exist yet).
  • _version.py is wrong in this tag.