Document SQL SEQSCAN (#3645)

Resolves #3281

Author: p7nov

doc/how-to/sql/sql_beginners_guide.rst

@@ -11,7 +11,7 @@ Most of the matters in the Beginners' Guide will already be familiar to people w
 
 .. _sql_beginners_sample_table:
 
-Sample Table
+Sample table
 ------------
 
 In football training camp it is traditional for the trainer to begin by showing a football
@@ -270,7 +270,7 @@ NoSQL emphasizes freedom and making your own rules.
 
 .. _sql_beginners_table_relationships:
 
-Table Relationships
+Table relationships
 -------------------
 
 Think about the two tables that have been discussed so far:
@@ -312,6 +312,20 @@ That is wrong, as will be clear in the discussion about what makes a database re
 Selecting with WHERE
 --------------------
 
+.. important::
+
+    By default, Tarantool prohibits ``SELECT`` queries that scan table rows
+    instead of using indexes to avoid unwanted heavy load. For the purposes of
+    this tutorial, allow SQL scan queries in Tarantool by running the command:
+
+    .. code-block:: sql
+
+        SET SESSION "sql_seq_scan" = true;
+
+    Alternatively, you can allow a specific query to perform a table scan by adding
+    the ``SEQSCAN`` keyword before the table name. Learn more about using ``SEQSCAN``
+    in SQL scan queries in the :ref:`SQL FROM clause description <sql_from>`.
+
 We gave a simple example of a SELECT statement earlier:
 
 .. code-block:: sql
@@ -884,7 +898,7 @@ in format clauses. Indexes of NoSQL spaces will be used with SQL if and only if
 
 .. _sql_beginners_relational_databases:
 
-Relational Databases
+Relational databases
 --------------------
 
 Edgar F. Codd, the person most responsible for researching and explaining relational database concepts,

doc/how-to/sql/sql_tutorial.rst

@@ -117,6 +117,55 @@ Then try to put another row:
 This ``INSERT`` fails because of a primary-key violation: the row with the primary
 key ``1, 'AB'`` already exists.
 
+The SEQSCAN keyword
+~~~~~~~~~~~~~~~~~~~
+
+In Tarantool, ``SELECT`` SQL queries that perform sequential scans (that is, go
+through all the table rows instead of using indexes) are prohibited by default.
+For example, this query leads to the error ``Scanning is not allowed for 'table2'``:
+
+.. code-block:: sql
+
+    SELECT * FROM table2;
+
+To execute a scan query, put the ``SEQSCAN`` keyword before the table name:
+
+.. code-block:: sql
+
+    SELECT * FROM SEQSCAN table2;
+
+Try to execute these queries that use indexed ``column1`` in filters:
+
+.. code-block:: sql
+
+    SELECT * FROM table2 WHERE column1 = 1;
+    SELECT * FROM table2 WHERE column1 + 1 = 2;
+
+The result is:
+
+*   The first query returns rows:
+
+    .. code-block:: tarantoolsession
+
+        - [1, 'AB', 'AB', 10.5]
+        - [1, 'CD', '  ', 10005]
+
+*   The second query fails with the error ``Scanning is not allowed for 'TABLE2'``.
+    Although ``column1`` is indexed, the expression ``column1 + 1`` is not calculated
+    from the index, which makes this ``SELECT`` a scan query.
+
+.. note::
+
+    You can allow SQL scan queries without ``SEQSCAN`` for the current session
+    by running the command:
+
+    .. code-block:: sql
+
+        SET SESSION "sql_seq_scan" = true;
+
+
+Learn more about using ``SEQSCAN`` in the :ref:`SQL FROM clause description <sql_from>`.
+
 SELECT with ORDER BY clause
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -127,8 +176,11 @@ Retrieve the 4 rows in the table, in descending order by ``column2``, then
 
 .. code-block:: sql
 
-    SELECT * FROM table2 ORDER BY column2 DESC, column4 ASC;
+    SELECT * FROM SEQSCAN table2 ORDER BY column2 DESC, column4 ASC;
+
+.. important::
 
+    Tarantool has its own
 The result is:
 
 .. code-block:: tarantoolsession
@@ -152,9 +204,9 @@ Retrieve some of what you inserted:
 
 .. code-block:: sql
 
-    SELECT column1, column2, column1 * column4 FROM table2 WHERE column2
+    SELECT column1, column2, column1 * column4 FROM SEQSCAN table2 WHERE column2
     LIKE 'A%';
-    SELECT column1, column2, column3, column4 FROM table2
+    SELECT column1, column2, column3, column4 FROM SEQSCAN table2
         WHERE (column1 < 2 AND column4 < 10)
         OR column3 = X'2020';
 
@@ -185,7 +237,7 @@ The rows which have the same values for ``column2`` are grouped and are aggregat
 .. code-block:: sql
 
     SELECT column2, SUM(column4), COUNT(column4), AVG(column4)
-    FROM table2
+    FROM SEQSCAN table2
     GROUP BY column2;
 
 The result is:
@@ -251,8 +303,8 @@ from the result table.
 
     CREATE TABLE table3 (column1 INTEGER, column2 VARCHAR(100), PRIMARY KEY
     (column2));
-    INSERT INTO table3 SELECT column1, column2 FROM table2 WHERE column1 <> 2;
-    SELECT * FROM table3;
+    INSERT INTO table3 SELECT column1, column2 FROM SEQSCAN table2 WHERE column1 <> 2;
+    SELECT * FROM SEQSCAN table3;
 
 The result is:
 
@@ -274,8 +326,8 @@ present in ``table3``.
 
 .. code-block:: sql
 
-    SELECT * FROM table2 WHERE (column1, column2) NOT IN (SELECT column1,
-    column2 FROM table3);
+    SELECT * FROM SEQSCAN table2 WHERE (column1, column2) NOT IN (SELECT column1,
+    column2 FROM SEQSCAN table3);
 
 The result is the single row that was excluded when inserting the rows with
 the ``INSERT ... SELECT`` statement:
@@ -295,7 +347,7 @@ column values from another table.
 
 .. code-block:: sql
 
-    SELECT * FROM table2, table3
+    SELECT * FROM SEQSCAN table2, table3
         WHERE table2.column1 = table3.column1 AND table2.column2 = table3.column2
         ORDER BY table2.column4;
 
@@ -327,7 +379,7 @@ containing ``13`` in ``column2``. Then try to insert such a row.
     INSERT INTO table4 VALUES (12, 13);
 
 Result: the insert fails, as it should, with the message
-``Check constraint failed ''ck_unnamed_TABLE4_1'': column2 <> 13``.
+``Check constraint 'ck_unnamed_TABLE4_1' failed for tuple``.
 
 CREATE TABLE with a FOREIGN KEY clause
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -348,7 +400,7 @@ Result:
 *   The first ``INSERT`` statement succeeds because
     ``table3`` contains a row with ``[2, 'AB', ' ', 12.34567]``.
 *   The second ``INSERT`` statement, correctly, fails with the message
-    ``Failed to execute SQL statement: FOREIGN KEY constraint failed``.
+    ``Foreign key constraint ''fk_unnamed_TABLE5_1'' failed: foreign tuple was not found``.
 
 UPDATE
 ~~~~~~
@@ -361,7 +413,7 @@ Use ``SELECT`` to see what happened to ``column4``.
 .. code-block:: sql
 
     UPDATE table2 SET column4 = column4 + 5 WHERE column4 <> 0;
-    SELECT column4 FROM table2 ORDER BY column4;
+    SELECT column4 FROM SEQSCAN table2 ORDER BY column4;
 
 The result is: ``{NULL, NULL, 0, 10.5, 17.34567, 10005}``.
 
@@ -385,11 +437,11 @@ Try to delete the last and first of these rows:
 
     DELETE FROM table2 WHERE column1 = 2;
     DELETE FROM table2 WHERE column1 = -1000;
-    SELECT COUNT(column1) FROM table2;
+    SELECT COUNT(column1) FROM SEQSCAN table2;
 
 The result is:
 
-*   The first ``DELETE`` statement causes an error message because
+*   The first ``DELETE`` statement causes an error because
     there's a foreign-key constraint.
 *   The second ``DELETE`` statement succeeds.
 *   The ``SELECT`` statement shows that there are 5 rows remaining.
@@ -461,7 +513,7 @@ in many ways. For example:
 
 .. code-block:: sql
 
-    SELECT column2, column2 || column2, SUBSTR(column2, 2, 1) FROM table2;
+    SELECT column2, column2 || column2, SUBSTR(column2, 2, 1) FROM SEQSCAN table2;
 
 The result is:
 
@@ -479,12 +531,12 @@ Number operations
 You can also manipulate number data (usually defined with ``INTEGER``
 or ``DOUBLE`` data types) in many ways. For example:
 
-* shift lleft with the ``<<`` operator
+* shift left with the ``<<`` operator
 * get modulo with the ``%`` operator
 
 .. code-block:: sql
 
-    SELECT column1, column1 << 1, column1 << 2, column1 % 2 FROM table2;
+    SELECT column1, column1 << 1, column1 << 2, column1 % 2 FROM SEQSCAN table2;
 
 The result is:
 
@@ -516,7 +568,7 @@ with arithmetic on a number column and ordering by a string column.
     INSERT INTO t6 VALUES (+1234567890, 'GD', 1e30);
     INSERT INTO t6 VALUES (10, 'FADEW?', 0.000001);
     INSERT INTO t6 VALUES (5, 'ABCDEFG', NULL);
-    SELECT column1 + 1, column2, column4 * 2 FROM t6 ORDER BY column2;
+    SELECT column1 + 1, column2, column4 * 2 FROM SEQSCAN t6 ORDER BY column2;
 
 The result is:
 
@@ -537,8 +589,8 @@ Create a view ``v3`` based on ``table3`` and select from it:
 
 .. code-block:: sql
 
-    CREATE VIEW v3 AS SELECT SUBSTR(column2,1,2), column4 FROM t6 WHERE
-    column4 >= 0;
+    CREATE VIEW v3 AS SELECT SUBSTR(column2,1,2), column4 FROM SEQSCAN t6
+    WHERE column4 >= 0;
     SELECT * FROM v3;
 
 The result is:
@@ -560,8 +612,8 @@ Create such a view and select from it:
 .. code-block:: sql
 
     WITH cte AS (
-                 SELECT SUBSTR(column2,1,2), column4 FROM t6 WHERE column4
-                 >= 0)
+                 SELECT SUBSTR(column2,1,2), column4 FROM SEQSCAN t6
+                 WHERE column4 >= 0)
     SELECT * FROM cte;
 
 The result is the same as the ``CREATE VIEW`` result:
@@ -593,17 +645,18 @@ The result of both these statements is:
 Metadata
 ~~~~~~~~
 
-
 To find out the internal structure of the Tarantool database with SQL,
-select from the Tarantool system tables:
+select from the Tarantool system tables ``_space``, ``_index``, and ``_trigger``:
+
+.. code-block:: sql
 
-* tables: ``SELECT * FROM "_space";``
-* indexes: ``SELECT * FROM "_index";``
-* triggers: ``SELECT * FROM "_trigger";``
+    SELECT * FROM SEQSCAN "_space";
+    SELECT * FROM SEQSCAN "_index";
+    SELECT * FROM SEQSCAN "_trigger";
 
 Actually, these statements select from NoSQL "system spaces".
 
-Select from ``_space``:
+Select from ``_space`` by a table name:
 
 .. code-block:: sql
 
@@ -636,7 +689,7 @@ You can invoke SQL statements using the Lua function ``box.execute(string)``.
 
 .. code-block:: tarantoolsession
 
-    tarantool> box.execute([[SELECT * FROM table3;]]);
+    tarantool> box.execute([[SELECT * FROM SEQSCAN table3;]]);
 
 The result is:
 
@@ -684,7 +737,7 @@ a bit:
     start_time = os.clock();
     main_function();
     end_time = os.clock();
-    'insert done in ' .. end_time - start_time .. ' seconds';
+    print('insert done in ' .. end_time - start_time .. ' seconds');
 
 The result is: you now have a table with a million rows, with a message saying
 "``insert done in 88.570578 seconds``".
@@ -701,7 +754,7 @@ Check how ``SELECT`` works on the million-row table:
 .. code-block:: lua
 
     box.execute([[SELECT * FROM tester WHERE s1 = 73446;]]);
-    box.execute([[SELECT * FROM tester WHERE s2 LIKE 'QFML%';]]);
+    box.execute([[SELECT * FROM SEQSCAN tester WHERE s2 LIKE 'QFML%';]]);
 
 The result is:
 

doc/reference/reference_lua/box_space/_session_settings.rst

@@ -26,6 +26,7 @@ box.space._session_settings
     *   ``sql_reverse_unordered_selects``: return result rows in reverse order if there is no :ref:`ORDER BY clause <sql_order_by>`.
         Default: ``false``.
     *   ``sql_select_debug``: show execution steps during :ref:`SELECT <sql_select>`. Default:``false``.
+    *   ``sql_seq_scan``: allow sequential scans in SQL :ref:`SELECT <sql_select>`. Default: ``true``.
     *   ``sql_vdbe_debug``: for internal use. Default:``false``.
     *   ``sql_defer_foreign_keys`` (removed in :doc:`2.11.0 </release/2.11.0>`): whether foreign-key checks can wait till
         commit. Default: ``false``.

doc/reference/reference_sql/README.md

@@ -0,0 +1,30 @@
+The diagrams in the SQL reference are generated using [Syntrax](https://github.com/kevinpt/syntrax).
+
+# Install Syntrax
+
+```
+pip install syntrax
+```
+
+## Troubleshooting installation issues
+
+### pip: command not found
+
+- Ensure that Python is installed
+- Try `pip3` instead of `pip` 
+
+### error in syntrax setup command: use_2to3 is invalid.
+
+Run `pip3 install "setuptools<58.0.0"`
+
+### ModuleNotFoundError: No module named 'cairo'|'gi'|'pango'
+
+Search on stackoverflow %)
+
+# Generate a diagram
+
+Generate an SVG with the same file name:
+
+```
+syntrax <filename>.spec -o svg
+```

doc/reference/reference_sql/from.spec

@@ -2,6 +2,10 @@ stack (
   line(
     choice(
       line(
+        choice(
+          None,
+          line(' SEQSCAN ')
+         ),
         'table-name',
         choice(
           None,