CSHARP-4768: Introduce $vectorSearch aggregation stage#1187
Conversation
BorisDog
requested review from
JamesKovacs and
adelinowona
and removed request for
a team
adelinowona
left a comment
adelinowona
left a comment
There was a problem hiding this comment.
LGTM + minor comments
| operatorName, | ||
| (s, sr, linqProvider) => | ||
| { | ||
| var vectorSearchOperator = new BsonDocument() |
There was a problem hiding this comment.
Redundant () after BsonDocument.
| } | ||
|
|
||
| [Fact] | ||
| public void VectorSearch_should_add_expected_stage_with_floats_vector() |
There was a problem hiding this comment.
This test and VectorSearch_should_add_expected_stage seem to be exact except for the queryVector used. Maybe combine them into one method and use parameters for the different queryVectors?
There was a problem hiding this comment.
Discussing offline.
| /// Creates a $vectorSearch stage. | ||
| /// </summary> | ||
| /// <typeparam name="TInput">The type of the input documents.</typeparam> | ||
| /// <returns>The stage.</returns> |
There was a problem hiding this comment.
Xml docs for parameters are missed.
| public static PipelineStageDefinition<TInput, TInput> VectorSearch<TInput>( | ||
| FieldDefinition<TInput> field, | ||
| VectorSearchQueryVector queryVector, | ||
| int limit, |
There was a problem hiding this comment.
Should we validate limit parameter's value?
|
|
||
| for (int i = 0; i < value.Count; i++) | ||
| { | ||
| bsonWriter.WriteDouble(span[i].ToDouble(null)); |
There was a problem hiding this comment.
Should we use QueryVectorBsonArray<T>.Values property here instead to localize all convertion logic there?
There was a problem hiding this comment.
Could be a good idea, but we want to skip allocation for BsonDouble in .Values (it's implicit allocation)
JamesKovacs
left a comment
JamesKovacs
left a comment
There was a problem hiding this comment.
Minor feedback. Plus consider Robert's idea of supporting casts from IEnumerable<T> to QueryVector.
| /// <returns>The stage.</returns> | ||
| /// <returns> | ||
| /// A new pipeline with an additional stage. | ||
| /// </returns> |
There was a problem hiding this comment.
Formatting. Most others have it on a single line.
| /// <returns>The stage.</returns> | ||
| /// <returns> | ||
| /// A new pipeline with an additional stage. | ||
| /// </returns> |
There was a problem hiding this comment.
Formatting. Most others have it on a single line.
| /// <returns>The stage.</returns> | ||
| /// <returns> | ||
| /// A new pipeline with an additional stage. | ||
| /// </returns> |
There was a problem hiding this comment.
Formatting. Most others have it on a single line.
| /// <returns>The stage.</returns> | ||
| /// <returns> | ||
| /// A new pipeline with an additional stage. | ||
| /// </returns> |
There was a problem hiding this comment.
Formatting. Most others have it on a single line.
| /// <returns>The stage.</returns> | ||
| /// <returns> | ||
| /// A new pipeline with an additional stage. | ||
| /// </returns> |
There was a problem hiding this comment.
Formatting. Most others have it on a single line.
| /// <param name="options">The options.</param> | ||
| /// <returns> | ||
| /// The fluent aggregate interface. | ||
| /// A new pipeline with an additional stage. |
There was a problem hiding this comment.
Formatting. Most others have it on a single line.
| /// <param name="options">The options.</param> | ||
| /// <returns> | ||
| /// The fluent aggregate interface. | ||
| /// A new pipeline with an additional stage. |
There was a problem hiding this comment.
Formatting. Most others have it on a single line.
| /// <param name="options">The vector search options.</param> | ||
| /// <returns> | ||
| /// A new pipeline with an additional stage. | ||
| /// </returns> |
There was a problem hiding this comment.
Formatting. Most others have it on a single line.
| /// <param name="options">The vector search options.</param> | ||
| /// <returns> | ||
| /// A new pipeline with an additional stage. | ||
| /// </returns> |
There was a problem hiding this comment.
Formatting. Most others have it on a single line.
| /// <summary> | ||
| /// Vector search query vector. | ||
| /// </summary> | ||
| public sealed class VectorSearchQueryVector |
There was a problem hiding this comment.
VectorSearchQueryVector reads like Vector---Vector to me. Having Vector at both ends of the class name sounds strange to my ear. Is there any ambiguity - either in our API or in C# - if we call it QueryVector?
There was a problem hiding this comment.
Thanks, changed to QueryVector.
| /// <returns> | ||
| /// A new pipeline with an additional stage. | ||
| /// </returns> | ||
| /// <returns>A new pipeline with an additional stage.</returns> |
There was a problem hiding this comment.
This is a lot of diffs unrelated to this PR.
There was a problem hiding this comment.
Yes, just used the opportunity of modifying this file to fix comments.
| /// <summary> | ||
| /// Gets the underlying BSON array. | ||
| /// </summary> | ||
| internal BsonArray Array { get; } |
There was a problem hiding this comment.
Add warning comment so we don't make this public by mistake some day:
internal BsonArray Array { get; } // do not make public because in the case of ReadOnlyMemory the BsonArray subclass is not fully functional
| /// </returns> | ||
| public static implicit operator QueryVector(ReadOnlyMemory<int> readOnlyMemory) => new(readOnlyMemory); | ||
| } | ||
|
|
There was a problem hiding this comment.
Add a warning comment that this subclass of BsonArray is not fully functional and is only intended to be serialized:
// WARNING: this class is a very partial implementation of a BsonArray subclass
// it is not fully functional and is only intended to be serialized
| _memory = memory; | ||
| } | ||
|
|
||
| public override int Count => _memory.Length; |
There was a problem hiding this comment.
I think you can remove this property (but see suggested change below).
| public override int Count => _memory.Length; | ||
|
|
||
| public ReadOnlySpan<T> Span => _memory.Span; | ||
|
|
There was a problem hiding this comment.
Add a comment explaining that this property ONLY exists for testing.
// note: Values is only used in tests
There was a problem hiding this comment.
Perhaps if the tests were written differently this property wouldn't be needed.
There was a problem hiding this comment.
Left the Values and Count properties for simplicity, and added a comment.
| { | ||
| get | ||
| { | ||
| for (int i = 0; i < _memory.Length; i++) |
There was a problem hiding this comment.
for (int i = 0; i < span.Length; i++)
Since the index is being used with Span[i] I would rather control the for loop with span.Length (I realize of course that both evaluate to the same number, but I'd rather not have a level of indirection).
rstam
left a comment
rstam
left a comment
There was a problem hiding this comment.
LGTM
Thanks for walking through the justifications for ReadOnlyMemory in the public API with me. I wish I had known sooner that we were talking about vectors of 2000 numbers instead of less than 10.
5 participants
No description provided.