Mongo/Driver/Server getInfo() throws exception

[stss-llp]

Description

PHP Fatal error: Uncaught exception 'MongoDB\Driver\Exception\InvalidArgumentException' with message 'Integer overflow detected on your platform: 6558846310068256773' in C:\inetpub\wwwroot\needme.com\vendor\mongodb\mongodb\src\functions.php:178
Stack trace:
#0 C:\inetpub\wwwroot\needme.com\vendor\mongodb\mongodb\src\functions.php(178): MongoDB\Driver\Server->getInfo()
#1 C:\inetpub\wwwroot\needme.com\vendor\mongodb\mongodb\src\Collection.php(521): MongoDB\server_supports_feature(Object(MongoDB\Driver\Server), 4)
#2 C:\inetpub\wwwroot\needme.com\ConsmrStrtdSrchng.php(65): MongoDB\Collection->find(Array)
#3 C:\inetpub\wwwroot\needme.com\ConsmrStrtdSrchng.php(94): PerformAction()
#4 {main}
thrown in C:\inetpub\wwwroot\needme.com\vendor\mongodb\mongodb\src\functions.php on line 178

Environment

Windows 7 (x64)
IIS Server 7.5.7600.16385
PHP 5.6.35 (x86)

MongoDB extension version | 1.4.3
MongoDB extension stability | stable
libbson bundled version | 1.9.4
libmongoc bundled version | 1.9.4
libmongoc SSL | enabled
libmongoc SSL library | OpenSSL
libmongoc crypto | enabled
libmongoc crypto library | libcrypto
libmongoc crypto system profile | disabled
libmongoc SASL | enabled
libmongoc compression | disabled

[jmikola]

This exception is thrown from these macros, which are used when the driver decodes a 64-bit integer from BSON on a 32-bit platform and the value cannot be represented within the range of a signed 32-bit integer.

This exception is entirely appropriate if you were accessing a document from your data set, as we want to warn users up front that their PHP runtime cannot represent the value. Without the exception, it'd be possible for users to attempt to round-trip the improperly decoded value and incur data loss. That said, I agree that this is inconvenient when we're talking about a diagnostic method like Server::getInfo(), which is simply reporting output from isMaster.

Can you attempt to reproduce the exception with mongodb.debug enabled? You can either set it to an output stream (e.g. "stderr") or a writable directory, within which individual log files will be created. I'm curious about the specific field in the isMaster response that produces this error.

Alternatively, you could connect to the same server in a mongo shell and run db.getSiblingDB("admin").runCommand({isMaster:1}), which should also contain the 64-bit value.

I've opened PHPC-1202 to propose that we suppress these exceptions during Server::getInfo() and other diagnostic function calls.

[stss-llp]

Hi Jeremy,

Thanks for your quick response.

Following is the log file created using mongodb.debug setting, as requested by you.

PHO2147.zip

Currently my quick and dirty work-around was to put a "return 1;" statement in the:

---

WEBSITE-PATH\vendor\mongodb\mongodb\src\functions.php

---

function server_supports_feature(Server $server, $feature)
{
return 1; // This line was added because FIND was giving an exception for InvalidArgument
$info = $server->getInfo();
$maxWireVersion = isset($info['maxWireVersion']) ? (integer) $info['maxWireVersion'] : 0;
$minWireVersion = isset($info['minWireVersion']) ? (integer) $info['minWireVersion'] : 0;

return ($minWireVersion <= $feature && $maxWireVersion >= $feature);

}

---

Your converting to string, suggestion is very good.
I actually am now converting FLOATs and INTEGERs to string before storing it in my DB.
And then converting them back to FLOATs and INTEGERs when I retrieve it from DB.

Only problem is, while aggregating I still need them as NON-STRINGs to do the CALC.

Is their anyway of doing that???

Thanks again.
Really Appreciate it.

Regards,
Saurabh Jog
(SaumeeTech Software Solutions LLP)

[jmikola]

@stss-llp: I was able to extract the isMaster responses from your debug logs. For example, this was the first of three:

<?php

$raw = <<<'EOD'
43 03 00 00 04 68 6f 73  74 73 00 a1 00 00 00 02
30 00 2d 00 00 00 63 6c  75 73 74 65 72 30 2d 73
68 61 72 64 2d 30 30 2d  30 30 2d 6b 6a 72 75 6e
2e 6d 6f 6e 67 6f 64 62  2e 6e 65 74 3a 32 37 30
31 37 00 02 31 00 2d 00  00 00 63 6c 75 73 74 65
72 30 2d 73 68 61 72 64  2d 30 30 2d 30 31 2d 6b
6a 72 75 6e 2e 6d 6f 6e  67 6f 64 62 2e 6e 65 74
3a 32 37 30 31 37 00 02  32 00 2d 00 00 00 63 6c
75 73 74 65 72 30 2d 73  68 61 72 64 2d 30 30 2d
30 32 2d 6b 6a 72 75 6e  2e 6d 6f 6e 67 6f 64 62
2e 6e 65 74 3a 32 37 30  31 37 00 00 02 73 65 74
4e 61 6d 65 00 11 00 00  00 43 6c 75 73 74 65 72
30 2d 73 68 61 72 64 2d  30 00 10 73 65 74 56 65
72 73 69 6f 6e 00 03 00  00 00 08 69 73 6d 61 73
74 65 72 00 01 08 73 65  63 6f 6e 64 61 72 79 00
00 02 70 72 69 6d 61 72  79 00 2d 00 00 00 63 6c
75 73 74 65 72 30 2d 73  68 61 72 64 2d 30 30 2d
30 30 2d 6b 6a 72 75 6e  2e 6d 6f 6e 67 6f 64 62
2e 6e 65 74 3a 32 37 30  31 37 00 02 6d 65 00 2d
00 00 00 63 6c 75 73 74  65 72 30 2d 73 68 61 72
64 2d 30 30 2d 30 30 2d  6b 6a 72 75 6e 2e 6d 6f
6e 67 6f 64 62 2e 6e 65  74 3a 32 37 30 31 37 00
07 65 6c 65 63 74 69 6f  6e 49 64 00 7f ff ff ff
00 00 00 00 00 00 00 07  03 6c 61 73 74 57 72 69
74 65 00 87 00 00 00 03  6f 70 54 69 6d 65 00 1c
00 00 00 11 74 73 00 01  00 00 00 f1 99 17 5b 12
74 00 07 00 00 00 00 00  00 00 00 09 6c 61 73 74
57 72 69 74 65 44 61 74  65 00 68 55 31 d4 63 01
00 00 03 6d 61 6a 6f 72  69 74 79 4f 70 54 69 6d
65 00 1c 00 00 00 11 74  73 00 01 00 00 00 f1 99
17 5b 12 74 00 07 00 00  00 00 00 00 00 00 09 6d
61 6a 6f 72 69 74 79 57  72 69 74 65 44 61 74 65
00 68 55 31 d4 63 01 00  00 00 10 6d 61 78 42 73
6f 6e 4f 62 6a 65 63 74  53 69 7a 65 00 00 00 00
01 10 6d 61 78 4d 65 73  73 61 67 65 53 69 7a 65
42 79 74 65 73 00 00 6c  dc 02 10 6d 61 78 57 72
69 74 65 42 61 74 63 68  53 69 7a 65 00 a0 86 01
00 09 6c 6f 63 61 6c 54  69 6d 65 00 4a 58 31 d4
63 01 00 00 10 6c 6f 67  69 63 61 6c 53 65 73 73
69 6f 6e 54 69 6d 65 6f  75 74 4d 69 6e 75 74 65
73 00 1e 00 00 00 10 6d  69 6e 57 69 72 65 56 65
72 73 69 6f 6e 00 00 00  00 00 10 6d 61 78 57 69
72 65 56 65 72 73 69 6f  6e 00 06 00 00 00 08 72
65 61 64 4f 6e 6c 79 00  00 01 6f 6b 00 00 00 00
00 00 00 f0 3f 11 6f 70  65 72 61 74 69 6f 6e 54
69 6d 65 00 01 00 00 00  f1 99 17 5b 03 24 63 6c
75 73 74 65 72 54 69 6d  65 00 58 00 00 00 11 63
6c 75 73 74 65 72 54 69  6d 65 00 01 00 00 00 f1
99 17 5b 03 73 69 67 6e  61 74 75 72 65 00 33 00
00 00 05 68 61 73 68 00  14 00 00 00 00 50 70 cb
6f 3b da e8 05 43 54 37  4f 80 ad d8 a6 2f df d7
ae 12 6b 65 79 49 64 00  05 00 00 00 f8 b3 05 5b
00 00 00                                        
EOD;

$bson = hex2bin(str_replace([' ', "\n"], '', $raw));
var_dump(MongoDB\BSON\toPHP($bson));

It produced the following output:

object(stdClass)#15 (20) {
  ["hosts"]=>
  array(3) {
    [0]=>
    string(44) "cluster0-shard-00-00-kjrun.mongodb.net:27017"
    [1]=>
    string(44) "cluster0-shard-00-01-kjrun.mongodb.net:27017"
    [2]=>
    string(44) "cluster0-shard-00-02-kjrun.mongodb.net:27017"
  }
  ["setName"]=>
  string(16) "Cluster0-shard-0"
  ["setVersion"]=>
  int(3)
  ["ismaster"]=>
  bool(true)
  ["secondary"]=>
  bool(false)
  ["primary"]=>
  string(44) "cluster0-shard-00-00-kjrun.mongodb.net:27017"
  ["me"]=>
  string(44) "cluster0-shard-00-00-kjrun.mongodb.net:27017"
  ["electionId"]=>
  object(MongoDB\BSON\ObjectId)#1 (1) {
    ["oid"]=>
    string(24) "7fffffff0000000000000007"
  }
  ["lastWrite"]=>
  object(stdClass)#8 (4) {
    ["opTime"]=>
    object(stdClass)#3 (2) {
      ["ts"]=>
      object(MongoDB\BSON\Timestamp)#2 (2) {
        ["increment"]=>
        string(1) "1"
        ["timestamp"]=>
        string(10) "1528273393"
      }
      ["t"]=>
      int(7)
    }
    ["lastWriteDate"]=>
    object(MongoDB\BSON\UTCDateTime)#4 (1) {
      ["milliseconds"]=>
      string(13) "1528273393000"
    }
    ["majorityOpTime"]=>
    object(stdClass)#6 (2) {
      ["ts"]=>
      object(MongoDB\BSON\Timestamp)#5 (2) {
        ["increment"]=>
        string(1) "1"
        ["timestamp"]=>
        string(10) "1528273393"
      }
      ["t"]=>
      int(7)
    }
    ["majorityWriteDate"]=>
    object(MongoDB\BSON\UTCDateTime)#7 (1) {
      ["milliseconds"]=>
      string(13) "1528273393000"
    }
  }
  ["maxBsonObjectSize"]=>
  int(16777216)
  ["maxMessageSizeBytes"]=>
  int(48000000)
  ["maxWriteBatchSize"]=>
  int(100000)
  ["localTime"]=>
  object(MongoDB\BSON\UTCDateTime)#9 (1) {
    ["milliseconds"]=>
    string(13) "1528273393738"
  }
  ["logicalSessionTimeoutMinutes"]=>
  int(30)
  ["minWireVersion"]=>
  int(0)
  ["maxWireVersion"]=>
  int(6)
  ["readOnly"]=>
  bool(false)
  ["ok"]=>
  float(1)
  ["operationTime"]=>
  object(MongoDB\BSON\Timestamp)#10 (2) {
    ["increment"]=>
    string(1) "1"
    ["timestamp"]=>
    string(10) "1528273393"
  }
  ["$clusterTime"]=>
  object(stdClass)#14 (2) {
    ["clusterTime"]=>
    object(MongoDB\BSON\Timestamp)#11 (2) {
      ["increment"]=>
      string(1) "1"
      ["timestamp"]=>
      string(10) "1528273393"
    }
    ["signature"]=>
    object(stdClass)#13 (2) {
      ["hash"]=>
      object(MongoDB\BSON\Binary)#12 (2) {
        ["data"]=>
        string(20) "Pp�o;��CT7O��ئ/�׮"
        ["type"]=>
        int(0)
      }
      ["keyId"]=>
      int(6558846310068256773)
    }
  }
}

I believe the overflow error arose from from the $clusterTime.signature.keyId field. The $clusterTime field is included in all command responses from 3.6 replica sets and sharded clusters, so I don't believe this is limited to isMaster; however, the fact that it does occur with isMaster certainly makes it harder to avoid this error when using PHPLIB (as explained in this comment in PHPC-1202). We will certainly need to look into this issue further, so it's too soon to estimate when we might have a fix available.

In the meantime I might suggest using PHP 7 with support for 64-bit integers. Quoting the sidebar on windows.php.net:

PHP 7 provides full 64-bit support. The x64 builds of PHP 7 support native 64-bit integers, LFS, 64-bit memory_limit and much more.

This would allow you to avoid the issue entirely.

---

I actually am now converting FLOATs and INTEGERs to string before storing it in my DB.
And then converting them back to FLOATs and INTEGERs when I retrieve it from DB.
Only problem is, while aggregating I still need them as NON-STRINGs to do the CALC.

For starters, there is no need to convert floats to strings. 32-bit and 64-bit platforms have no trouble converting a PHP float to and from a BSON double. Additionally, I'll note that there is also a MongoDB\BSON\Decimal128 type available if you require a larger decimal type without the approximation issues of regular floating-point types.

Given that you're currently on a 32-bit platform, there should also be no need to convert any PHP integers to strings. Your PHP runtime cannot represent integers outside of the range of signed 32-bit integers, so you should expect no problems converting those value to/from BSON. Those values should all be converted to 32-bit BSON integers and successfully round-trip from the database.

Keeping these floats and integers as numeric types in the database should also ensure that mathematical calculations within an aggregation pipeline work as expected. The only edge case I can think of off hand would be if you were performing an integer computation (e.g. $sum) that caused some result in the aggregation to exceed a 32-bit signed integer. In that case, the MongoDB server will naturally elect to use a 64-bit integer as the result. This will only be a problem if the final result documents sent back to the PHP driver include the 64-bit integer (e.g. they aren't converted to strings or otherwise projected out before the last stage).

MongoDB 4.0 will introduce some aggregation operators to facilitate type conversions (see: SERVER-33169). In MongoDB 3.6, you should be able to utilize the $substr operator for a makeshift conversion. Consider the following shell example:

> db.foo.aggregate([{$project:{x:{$substr:["$x",0,-1]}}}])
{ "_id" : ObjectId("5b17ff4295aaeea0d7dcb471"), "x" : "1" }
> db.foo.drop()
true
> db.foo.insert({x:1})
WriteResult({ "nInserted" : 1 })
> db.foo.aggregate([{$project:{x:{$substr:["$x",0,-1]}}}])
{ "_id" : ObjectId("5b17ff8c95aaeea0d7dcb472"), "x" : "1" }

This idea was lifted from this answer on Stack Overflow. Also note that $substr was recently deprecated in favor of $substrBytes, so I'd suggest using the newer operator if you decide to employ this technique.

[stss-llp]

Hi Jeremy,

Thanks for your quick response.

Sorry for such a late reply, I got really tied up into something.

I have taken your suggestion and moved my server onto PHP 7.2.6 (x64)

Now everything works fine without the work-around that I mentioned in my previous reply.

And, YES it is a SHOW STOPPER, if someone is using the 32 bit PHP.

So thanks for upgrading it to BLOCKER.

Again, very much appreciate your help and support in this regards.

Thanks,
Saurabh Jog
(SaumeeTech Software Solutions LLP)

[jmikola]

Rather than implement a field-specific workaround in 1.4.x (PHPC-1202), we've decided to implement an Int64 BSON type class in 1.5.0, which will be used exclusively to represent 64-bit integers on 32-bit platforms. See PHPC-1211 for more information.