Skip to content

PHPORM-274 List search indexes in Schema::getIndexes() introspection method #3233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 2, 2025
Merged

PHPORM-274 List search indexes in Schema::getIndexes() introspection method #3233

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5700592
PHPORM-274 List search indexes in Schema::getIndexes introspection me…
GromNaN Dec 31, 2024
d30ab34
Add details on error codes
GromNaN Dec 31, 2024
45603fd
Show dynamic search index and set type null for standard indexes
GromNaN Jan 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 44 additions & 3 deletions src/Schema/Builder.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,17 @@
namespace MongoDB\Laravel\Schema;

use Closure;
use MongoDB\Collection;
use MongoDB\Driver\Exception\ServerException;
use MongoDB\Model\CollectionInfo;
use MongoDB\Model\IndexInfo;

use function array_column;
use function array_fill_keys;
use function array_filter;
use function array_keys;
use function array_map;
use function array_merge;
use function assert;
use function count;
use function current;
Expand Down Expand Up @@ -225,9 +229,11 @@ public function getColumns($table)

public function getIndexes($table)
{
$indexes = $this->connection->getMongoDB()->selectCollection($table)->listIndexes();

$collection = $this->connection->getMongoDB()->selectCollection($table);
assert($collection instanceof Collection);
$indexList = [];

$indexes = $collection->listIndexes();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is outside the diff but I think it's worth mentioning since I see you use an === to check for ['_id' => 1]. If there is ever an issue with that, you may need to relax the comparison on the numeric value. IIRC, mongos used to sometimes return command results as 1.0 instead of 1 and I think there may be similar inconsistencies for index definitions.

That said, it's probably fine to just wait and see if this is ever reported before going out of your way to address it now. It's quite possible that it's a non-issue on recent server versions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The triple === is enforced by phpcs. We will fix it if a new server version produces an incompatible value, but I doubt it.

foreach ($indexes as $index) {
assert($index instanceof IndexInfo);
$indexList[] = [
Expand All @@ -238,12 +244,35 @@ public function getIndexes($table)
$index->isText() => 'text',
$index->is2dSphere() => '2dsphere',
$index->isTtl() => 'ttl',
default => 'default',
default => null,
},
'unique' => $index->isUnique(),
];
}

try {
$indexes = $collection->listSearchIndexes(['typeMap' => ['root' => 'array', 'array' => 'array', 'document' => 'array']]);
foreach ($indexes as $index) {
$indexList[] = [
'name' => $index['name'],
'columns' => match ($index['type']) {
'search' => array_merge(
$index['latestDefinition']['mappings']['dynamic'] ? ['dynamic'] : [],
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • latestDefinition.mappings.dynamic is always defined.
  • dynamic word is added to the list for columns names it there is also explicit mapping.

array_keys($index['latestDefinition']['mappings']['fields'] ?? []),
),
'vectorSearch' => array_column($index['latestDefinition']['fields'], 'path'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per Atlas Vector Search Index Fields, I expect this might return field paths instead of field names. Is that going to cause problems with whatever consumes this?

I assume it may be consistent with how the basic listIndexes() query above runs (using array_keys($index->getKey()) but I'm not sure.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As soon as it is a string, it can be displayed.

},
'type' => $index['type'],
'primary' => false,
'unique' => false,
];
}
} catch (ServerException $exception) {
if (! self::isAtlasSearchNotSupportedException($exception)) {
throw $exception;
}
}

return $indexList;
}

Expand Down Expand Up @@ -290,4 +319,16 @@ protected function getAllCollections()

return $collections;
}

/** @internal */
public static function isAtlasSearchNotSupportedException(ServerException $e): bool
{
return in_array($e->getCode(), [
59, // MongoDB 4 to 6, 7-community: no such command: 'createSearchIndexes'
40324, // MongoDB 4 to 6: Unrecognized pipeline stage name: '$listSearchIndexes'
115, // MongoDB 7-ent: Search index commands are only supported with Atlas.
6047401, // MongoDB 7: $listSearchIndexes stage is only allowed on MongoDB Atlas
31082, // MongoDB 8: Using Atlas Search Database Commands and the $listSearchIndexes aggregation stage requires additional configuration.
], true);
}
}
138 changes: 138 additions & 0 deletions tests/AtlasSearchTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
<?php

namespace MongoDB\Laravel\Tests;

use Illuminate\Support\Facades\Schema;
use MongoDB\Collection as MongoDBCollection;
use MongoDB\Driver\Exception\ServerException;
use MongoDB\Laravel\Schema\Builder;
use MongoDB\Laravel\Tests\Models\Book;

use function assert;
use function usleep;
use function usort;

class AtlasSearchTest extends TestCase
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I plan to test all the search index features in this class. It is also in #3232

{
public function setUp(): void
{
parent::setUp();

Book::insert([
['title' => 'Introduction to Algorithms'],
['title' => 'Clean Code: A Handbook of Agile Software Craftsmanship'],
['title' => 'Design Patterns: Elements of Reusable Object-Oriented Software'],
['title' => 'The Pragmatic Programmer: Your Journey to Mastery'],
['title' => 'Artificial Intelligence: A Modern Approach'],
['title' => 'Structure and Interpretation of Computer Programs'],
['title' => 'Code Complete: A Practical Handbook of Software Construction'],
['title' => 'The Art of Computer Programming'],
['title' => 'Computer Networks'],
['title' => 'Operating System Concepts'],
['title' => 'Database System Concepts'],
['title' => 'Compilers: Principles, Techniques, and Tools'],
['title' => 'Introduction to the Theory of Computation'],
['title' => 'Modern Operating Systems'],
['title' => 'Computer Organization and Design'],
['title' => 'The Mythical Man-Month: Essays on Software Engineering'],
['title' => 'Algorithms'],
['title' => 'Understanding Machine Learning: From Theory to Algorithms'],
['title' => 'Deep Learning'],
['title' => 'Pattern Recognition and Machine Learning'],
]);

$collection = $this->getConnection('mongodb')->getCollection('books');
assert($collection instanceof MongoDBCollection);
try {
$collection->createSearchIndex([
'mappings' => [
'fields' => [
'title' => [
['type' => 'string', 'analyzer' => 'lucene.english'],
['type' => 'autocomplete', 'analyzer' => 'lucene.english'],
],
],
],
]);

$collection->createSearchIndex([
'mappings' => ['dynamic' => true],
], ['name' => 'dynamic_search']);

$collection->createSearchIndex([
'fields' => [
['type' => 'vector', 'numDimensions' => 16, 'path' => 'vector16', 'similarity' => 'cosine'],
['type' => 'vector', 'numDimensions' => 32, 'path' => 'vector32', 'similarity' => 'euclidean'],
],
], ['name' => 'vector', 'type' => 'vectorSearch']);
} catch (ServerException $e) {
if (Builder::isAtlasSearchNotSupportedException($e)) {
self::markTestSkipped('Atlas Search not supported. ' . $e->getMessage());
}

throw $e;
}

// Wait for the index to be ready
do {
$ready = true;
usleep(10_000);
foreach ($collection->listSearchIndexes() as $index) {
if ($index['status'] !== 'READY') {
$ready = false;
}
}
} while (! $ready);
}

public function tearDown(): void
{
$this->getConnection('mongodb')->getCollection('books')->drop();

parent::tearDown();
}

public function testGetIndexes()
{
$indexes = Schema::getIndexes('books');

self::assertIsArray($indexes);
self::assertCount(4, $indexes);

// Order of indexes is not guaranteed
usort($indexes, fn ($a, $b) => $a['name'] <=> $b['name']);

$expected = [
[
'name' => '_id_',
'columns' => ['_id'],
'primary' => true,
'type' => null,
'unique' => false,
],
[
'name' => 'default',
'columns' => ['title'],
'type' => 'search',
'primary' => false,
'unique' => false,
],
[
'name' => 'dynamic_search',
'columns' => ['dynamic'],
'type' => 'search',
'primary' => false,
'unique' => false,
],
[
'name' => 'vector',
'columns' => ['vector16', 'vector32'],
'type' => 'vectorSearch',
'primary' => false,
'unique' => false,
],
];

self::assertSame($expected, $indexes);
}
}
49 changes: 35 additions & 14 deletions tests/SchemaTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -482,20 +482,41 @@ public function testGetIndexes()
$collection->string('mykey3')->index();
});
$indexes = Schema::getIndexes('newcollection');
$this->assertIsArray($indexes);
$this->assertCount(4, $indexes);

$indexes = collect($indexes)->keyBy('name');

$indexes->each(function ($index) {
$this->assertIsString($index['name']);
$this->assertIsString($index['type']);
$this->assertIsArray($index['columns']);
$this->assertIsBool($index['unique']);
$this->assertIsBool($index['primary']);
});
$this->assertTrue($indexes->get('_id_')['primary']);
$this->assertTrue($indexes->get('unique_index_1')['unique']);
self::assertIsArray($indexes);
self::assertCount(4, $indexes);

$expected = [
[
'name' => '_id_',
'columns' => ['_id'],
'primary' => true,
'type' => null,
'unique' => false,
],
[
'name' => 'mykey1_1',
'columns' => ['mykey1'],
'primary' => false,
'type' => null,
'unique' => false,
],
[
'name' => 'unique_index_1',
'columns' => ['unique_index'],
'primary' => false,
'type' => null,
'unique' => true,
],
[
'name' => 'mykey3_1',
'columns' => ['mykey3'],
'primary' => false,
'type' => null,
'unique' => false,
],
];

self::assertSame($expected, $indexes);

// Non-existent collection
$indexes = Schema::getIndexes('missing');
Expand Down
Loading