jruby/docs BETA
Navigation
org.apache.lucene.util.hnsw 25
C BlockingFloatHeap
I CloseableRandomVectorScorerSupplier
C ConcurrentHnswMerger
C FilteredHnswGraphSearcher
C FloatHeap
I HasKnnVectorValues
I HnswBuilder
C HnswConcurrentMergeBuilder
C HnswGraph
C HnswGraphBuilder
I HnswGraphMerger
C HnswGraphSearcher
C HnswUtil
C IncrementalHnswGraphMerger
C InitializedHnswGraphBuilder
I IntToIntFunction
C MergingHnswGraphBuilder
C NeighborArray
C NeighborQueue
C OnHeapHnswGraph
C OrdinalTranslatedKnnCollector
I RandomVectorScorer
I RandomVectorScorerSupplier
C UpdateGraphsUtils
I UpdateableRandomVectorScorer
MergingHnswGraphBuilder — members 2+
M build(max_ord) OnHeapHnswGraph
M from_graphs(scorer_supplier, m, beam_width, seed, graphs, ord_maps, total_number_of_vectors, initialized_nodes) MergingHnswGraphBuilder
from HnswGraphBuilder
F DEFAULT_BEAM_WIDTH int
F DEFAULT_MAX_CONN int
F HNSW_COMPONENT String
F randSeed long
C add_graph_node(node)
C add_graph_node(node, eps0)
M completed_graph() OnHeapHnswGraph
M graph() OnHeapHnswGraph
C info_stream=(info_stream)

org.apache.lucene.util.hnsw.MergingHnswGraphBuilder

class final extends HnswGraphBuilder 2 members

A graph builder that is used during segments' merging.

This builder uses a smart algorithm to merge multiple graphs into a single graph. The algorithm is based on the idea that if we know where we want to insert a node, we have a good idea of where we want to insert its neighbors.

The algorithm is based on the following steps:

  • Get all graphs that don't have deletions and sort them by size desc.
  • Copy the largest graph to the new graph (gL).
  • For each remaining small graph (gS):
    • Find the nodes that best cover gS: join set `j`. These nodes will be inserted into gL as usual: by searching gL to find the best candidates `w` to which connect the nodes.
    • For each remaining node in gS:
      • We provide eps to search in gL. We form `eps` by the union of the node's neighbors in gS and the node's neighbors' neighbors in gL. We also limit beamWidth (efConstruction to M*3)

We expect the size of join set `j` to be small, around 1/5 to 1/2 of the size of gS. For the rest of the nodes in gS, we expect savings by performing lighter searches in gL.

Class Methods

from_graphs

MergingHnswGraphBuilder from_graphs ( RandomVectorScorerSupplier scorer_supplier, int m, int beam_width, long seed, HnswGraph[] graphs, int[][] ord_maps, int total_number_of_vectors, BitSet initialized_nodes )
Java: fromGraphs(org.apache.lucene.util.hnsw.RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, org.apache.lucene.util.hnsw.HnswGraph[] graphs, int[][] ordMaps, int totalNumberOfVectors, org.apache.lucene.util.BitSet initializedNodes)

Create a new HnswGraphBuilder that is initialized with the provided HnswGraph.

nametypedescription
scorer_supplierorg.apache.lucene.util.hnsw.RandomVectorScorerSupplierthe scorer to use for vectors
beam_widthintthe number of nodes to explore in the search
seedlongthe seed for the random number generator
graphsorg.apache.lucene.util.hnsw.HnswGraph[]the graphs to merge
ord_mapsint[][]the ordinal maps for the graphs
total_number_of_vectorsintthe total number of vectors in the new graph, this should include all vectors expected to be added to the graph in the future
initialized_nodesorg.apache.lucene.util.BitSetthe nodes will be initialized through the merging, if null, all nodes should be already initialized after #updateGraph(HnswGraph, int[]) being called

Returns: a new HnswGraphBuilder that is initialized with the provided HnswGraph

Throws

IOException when reading the graph fails

Instance Methods

build

OnHeapHnswGraph build ( int max_ord )
Java: build(int maxOrd)

Inherited

from HnswGraphBuilder

fieldtypenote
DEFAULT_BEAM_WIDTH int Default number of the size of the queue maintained while searching during a graph construction.
DEFAULT_MAX_CONN int Default number of maximum connections per node
HNSW_COMPONENT String A name for the HNSW component for the info-stream *
randSeed long Random seed for level generation; public to expose for testing *

add_graph_node

add_graph_node ( int node )
Java: addGraphNode(int node)

add_graph_node

add_graph_node ( int node, IntHashSet eps0 )
Java: addGraphNode(int node, org.apache.lucene.internal.hppc.IntHashSet eps0)

completed_graph

OnHeapHnswGraph completed_graph ( )
Java: getCompletedGraph() · also: get_completed_graph

graph

OnHeapHnswGraph graph ( )
Java: getGraph() · also: get_graph

info_stream=

info_stream= ( InfoStream info_stream )
Java: setInfoStream(org.apache.lucene.util.InfoStream infoStream) · also: set_info_stream
this work for additional information regarding copyright ownership.