Migrating Python code from 0.7 to 0.8¶
In 0.8, IvfIndex, HnswIndex and SymqgIndex follow a train → add →
search workflow, as Faiss indexes do. IVF-RaBitQ and HNSW-RaBitQ learn their
own centroids, so you no longer run k-means and pass the centroids and
assignments to build; SymphonyQG has nothing to train and builds its graph in
add. Code written for 0.7
still runs in 0.8. Each old form raises a DeprecationWarning at your call site
and will be removed in 0.9.
Unless an item below says otherwise, a 0.7 call that raises a deprecation
warning keeps its 0.7 behavior. Calls that are valid in both versions, such as
add(x) and search, follow the 0.8 rules.
IVF-RaBitQ and HNSW-RaBitQ¶
Old and new calls¶
| 0.7 | 0.8 |
|---|---|
IvfIndex(dim, max_elements, num_clusters, nbits, metric, initializer) |
IvfIndex(dim, num_clusters, nbits=..., metric=..., initializer=..., seed=None); storage grows on add |
IvfIndex(dim=..., max_elements=..., num_clusters=..., nbits=...) |
the same, without max_elements |
HnswIndex(dim, max_elements, M=16, ef_construction=200, nbits=8, metric="l2", random_seed=100) |
HnswIndex(dim, nbits=..., num_clusters=16, M=16, ef_construction=200, metric="l2", seed=100); capacity grows on add, and resize(n) preallocates it |
HnswIndex(..., random_seed=s) |
HnswIndex(..., seed=s) |
kmeans = RaBitQKMeans(dim, k)kmeans.train(x)index.build(x, kmeans.centroids, kmeans.assignments) |
index.train(x)index.add(x) |
index.build(x, centroids, cluster_ids) |
index.set_centroids(centroids)index.add(x, cluster_ids=cluster_ids) |
index.add(x, cluster_ids, num_threads, fast_quantization) |
index.add(x, cluster_ids=..., num_threads=..., fast_quantization=...) |
index.is_built |
index.is_trained, or len(index) > 0 for "holds vectors" |
Except for dim and IVF-RaBitQ's num_clusters, constructor arguments are
keyword-only in 0.8, and nbits has no default. HNSW-RaBitQ used to default to
nbits=8; the old form keeps that default.
train uses the settings in index.cp: 25 iterations, at most 256 sampled
vectors per centroid, spherical k-means for metric="ip", and the index seed.
See IVF-RaBitQ training for the
details and for set_centroids. The clustering classes still
work if you want to run k-means yourself.
Behavior changes¶
- Thread defaults.
trainandaddnow default to every available CPU (num_threads=0), capped byOMP_NUM_THREADSsince 0.8.1; 0.7buildandadddefaulted to one thread. This applies to everyaddcall that does not passnum_threads, including 0.7 code such asindex.add(x), without a warning.searchstill defaults to one thread, and the deprecatedbuildkeeps its default of 1. - HNSW-RaBitQ builds are not reproducible by default. With more than one
thread,
addlinks points in a nondeterministic order. Passnum_threads=1toaddfor the same graph on every run. With one thread and the same seed,set_centroidsfollowed byaddproduces exactly the index that 0.7buildproduced from the same centroids and cluster IDs. - HNSW-RaBitQ
searchlimitskto the stored point count (num_stored_points) instead of the capacity, since capacity now grows geometrically. Akbetween the two used to return padding rows and now raisesValueError, as it already did for IVF-RaBitQ. - HNSW-RaBitQ capacity grows.
addresizes the index, at least doubling it, instead of raisingValueErrorwhen the batch does not fit. This also applies toaddon indexes created in the old form, and to the deprecatedbuild, which no longer rejects more rows thanmax_elements. - State errors. Calling
add,search,remove,reserveorsavebefore training raisesRuntimeError("IvfIndex must be trained or loaded before ..."); the 0.7 message said "built". A trained index without vectors cannot be saved.reprshowsis_trainedinstead ofis_built. - Seeds.
IvfIndexstill draws a random rotation seed by default, and now reports it asindex.seed; pass it back asseed=to reproduce an index. Indexes loaded from files reportseed=None. - HNSW-RaBitQ
num_clustersreports the configured count (16 by default) before training, where 0.7 reported 0 beforebuild. - An
IvfIndexcreated withmax_elementsis filled bybuild;trainandset_centroidsraiseRuntimeErrorfor it. Create the index in the 0.8 form to train it.
SymphonyQG¶
| 0.7 | 0.8 |
|---|---|
SymqgIndex(dim, max_degree, metric, quantization_bits) |
SymqgIndex(dim, max_degree=..., metric=..., nbits=...); every argument after dim is keyword-only |
quantization_bits=0 (raw vectors), 4, 8 |
nbits=32 (raw vectors, the default), 4, 8 |
index.build(x, ef_construction, num_threads, init) |
SymqgIndex(dim, ef_construction=..., init=..., seed=...)index.add(x, num_threads=...) |
index.search_batch(q, k, ef) |
index.search(q, k, ef); ef is now optional |
index.quantization_bits |
index.nbits (32 instead of 0 for raw vectors) |
index.is_built |
len(index) > 0 |
train(x) exists for a uniform workflow but only checks x as add would,
and is_trained is always True. add builds the graph from all vectors and can run only once;
a second add, or add on a loaded index, raises RuntimeError, as Faiss's
NSG index does. The deprecated build keeps its 0.7 behavior: it replaces any
existing graph and, unless you passed seed, draws a new random seed each time.
search now treats ef like HNSW-RaBitQ: an omitted or zero ef uses
max(k, 10), and a smaller one is raised to k. In 0.7, both raised
ValueError. add uses every CPU by default, while the deprecated build
keeps one thread. The seed now also drives graph construction: without seed,
each index draws one random seed and reports it as index.seed; 0.7 drew
separate random seeds for the rotation and the graph. Before vectors are added,
search and save raise RuntimeError("SymqgIndex has no vectors; ...").
Files and C++¶
Saved files, the max_elements property, and search results for the same
index are unchanged. Files written by 0.7 load as trained indexes.
For C++, the existing constructors and construct are unchanged; see
IVF-RaBitQ and
HNSW-RaBitQ for the new training entry
points. The one C++ behavior change is that saving an index without stored
points now throws std::logic_error.