Skip to content

RaBitQ Library 0.8.3

This release adds rabitqlib.quant, a provisional Python module with the quantization building blocks behind the indexes. Use it to store RaBitQ codes in your own database or file format and to score them with your own candidate selection, without building an index. Existing APIs, defaults, results and index files are unchanged.

from rabitqlib import quant

rot = quant.Rotator(dim, seed=42)
residuals = rot.rotate(data) - centroid       # centroid in the rotated space
codes = quant.quantize(residuals, nbits=4)
factors = quant.code_factors(residuals, codes, nbits=4, metric="l2", centroid=centroid)

See Python building blocks for encoding, two-stage estimation with error bounds, refinement and FastScan.

Python changes

  • New module rabitqlib.quant, imported explicitly with from rabitqlib import quant; import rabitqlib does not load it.
  • Rotation: Rotator (seeded FHT-Kac or matrix rotation) with rotate, to_bytes/from_bytes, copying and pickling; padded_dim.
  • Encoding: quantize, split (prefix and extra bits), code_factors, pack_codes/unpack_codes.
  • Estimation: Query with estimate_bitset, estimate_codes, refine, refine_packed and set_centroid; query_terms and estimate_from_ip for inner products from your own kernel.
  • FastScan: pack_fastscan_blocks and Query.estimate_fastscan, which need AVX2 with FMA, AVX-512 or NEON.
  • The module is provisional in the sense of PEP 411: any minor release may change it, with a note in the release notes.
  • Its byte formats (packed codes, factor records, FastScan blocks and rotator blobs) are versioned from the start and frozen by golden files. Bytes of an unknown version raise quant.FormatError, a subclass of ValueError.
  • Every array-returning call accepts out= and num_threads (all CPUs by default for build-side calls, one for scoring calls), and the GIL is released during computation. Invalid inputs raise ValueError and name the first failing row whatever the thread count; values not representable in float32 raise OverflowError.

C++ changes

  • New provisional header rabitqlib/core/provisional/quantize.hpp with rabitqlib::quant::provisional::combined_code and code_factors, which back rabitqlib.quant. It is not part of the stable v1 API and may change in any minor release.
  • quant::v1 results are unchanged.

Upgrading

From 0.8.2, index file formats, the stable core API v1 and existing Python signatures are unchanged; this release only adds the module and header above. Rebuild C++ consumers with matching headers and libraries; no cross-version C++ binary ABI is promised.

When upgrading from 0.7, follow the 0.8.0 release notes and the migration guide.