Module org.apache.lucene.core
Package org.apache.lucene.util.hnsw
Class HnswConcurrentMergeBuilder.ConcurrentMergeWorker
java.lang.Object
org.apache.lucene.util.hnsw.HnswGraphBuilder
org.apache.lucene.util.hnsw.HnswConcurrentMergeBuilder.ConcurrentMergeWorker
- All Implemented Interfaces:
HnswBuilder
- Enclosing class:
- HnswConcurrentMergeBuilder
private static final class HnswConcurrentMergeBuilder.ConcurrentMergeWorker
extends HnswGraphBuilder
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.util.hnsw.HnswGraphBuilder
HnswGraphBuilder.GraphBuilderKnnCollector
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate int
private final BitSet
private final AtomicInteger
A common AtomicInteger shared among all workers, used for tracking what's the next vector to be added to the graph.Fields inherited from class org.apache.lucene.util.hnsw.HnswGraphBuilder
DEFAULT_BEAM_WIDTH, DEFAULT_MAX_CONN, hnsw, HNSW_COMPONENT, randSeed
-
Constructor Summary
ConstructorsModifierConstructorDescriptionprivate
ConcurrentMergeWorker
(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, OnHeapHnswGraph hnsw, BitSet initializedNodes, AtomicInteger workProgress) -
Method Summary
Modifier and TypeMethodDescriptionvoid
addGraphNode
(int node) Inserts a doc with vector value to the graphprivate int
getStartPos
(int maxOrd) Reserve the work by atomically increment theworkProgress
private void
run
(int maxOrd) This method first try to "reserve" part of work by callinggetStartPos(int)
and then callingHnswGraphBuilder.addVectors(int, int)
to actually add the nodes to the graph.Methods inherited from class org.apache.lucene.util.hnsw.HnswGraphBuilder
addVectors, build, create, create, getGraph, setInfoStream
-
Field Details
-
workProgress
A common AtomicInteger shared among all workers, used for tracking what's the next vector to be added to the graph. -
initializedNodes
-
batchSize
private int batchSize
-
-
Constructor Details
-
ConcurrentMergeWorker
private ConcurrentMergeWorker(RandomVectorScorerSupplier scorerSupplier, int M, int beamWidth, long seed, OnHeapHnswGraph hnsw, BitSet initializedNodes, AtomicInteger workProgress) throws IOException - Throws:
IOException
-
-
Method Details
-
run
This method first try to "reserve" part of work by callinggetStartPos(int)
and then callingHnswGraphBuilder.addVectors(int, int)
to actually add the nodes to the graph. By doing this we are able to dynamically allocate the work to multiple workers and try to make all of them finishing around the same time.- Throws:
IOException
-
getStartPos
private int getStartPos(int maxOrd) Reserve the work by atomically increment theworkProgress
-
addGraphNode
Description copied from interface:HnswBuilder
Inserts a doc with vector value to the graph- Specified by:
addGraphNode
in interfaceHnswBuilder
- Overrides:
addGraphNode
in classHnswGraphBuilder
- Throws:
IOException
-