From e1d01a210863c314782329bf65c21f4b93b2e221 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 1 Feb 2010 20:25:18 -0500 Subject: [PATCH 001/442] minor: reorganized yard templates --- source/templates/default/tags/html/core.erb | 8 ++++++++ source/templates/default/tags/setup.rb | 4 ++++ 2 files changed, 12 insertions(+) create mode 100644 source/templates/default/tags/html/core.erb create mode 100644 source/templates/default/tags/setup.rb diff --git a/source/templates/default/tags/html/core.erb b/source/templates/default/tags/html/core.erb new file mode 100644 index 000000000..ae5afd5aa --- /dev/null +++ b/source/templates/default/tags/html/core.erb @@ -0,0 +1,8 @@ +<% if object.has_tag?(:core) %> +

Core docs:

+ +<% end %> diff --git a/source/templates/default/tags/setup.rb b/source/templates/default/tags/setup.rb new file mode 100644 index 000000000..2e989af1c --- /dev/null +++ b/source/templates/default/tags/setup.rb @@ -0,0 +1,4 @@ +def init + super + sections[1].push :core +end From 8a77e50b6e11c2e7da3e2a2fc14e2c7e6869c7a4 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 1 Feb 2010 20:47:17 -0500 Subject: [PATCH 002/442] added yard_ext --- source/yard_ext.rb | 1 + 1 file changed, 1 insertion(+) create mode 100644 source/yard_ext.rb diff --git a/source/yard_ext.rb b/source/yard_ext.rb new file mode 100644 index 000000000..9ddbf56fb --- /dev/null +++ b/source/yard_ext.rb @@ -0,0 +1 @@ +YARD::Tags::Library.define_tag "Core", :core, :with_name From 1ba51a46e50c916860cdedf5483657acc8d227ca Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 1 Feb 2010 21:55:44 -0500 Subject: [PATCH 003/442] minor: core doc fix --- source/templates/default/tags/html/core.erb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/templates/default/tags/html/core.erb b/source/templates/default/tags/html/core.erb index ae5afd5aa..641a7aaca 100644 --- a/source/templates/default/tags/html/core.erb +++ b/source/templates/default/tags/html/core.erb @@ -2,7 +2,7 @@

Core docs:

    <% for tag in object.tags(:core) %> -
  • <%= linkify('http://dochub.mongodb.org/core/' + tag.name, tag.text || tag.name.capitalize) %>
    • +
  • <%= tag.name.capitalize %>
    • <% end %>
<% end %> From 5eec0f5408d4ebf47498f2c974a46fee3948963b Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 25 Oct 2010 11:01:52 -0400 Subject: [PATCH 004/442] minor: doc fix --- source/templates/default/tags/setup.rb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/templates/default/tags/setup.rb b/source/templates/default/tags/setup.rb index 2e989af1c..f67962c7a 100644 --- a/source/templates/default/tags/setup.rb +++ b/source/templates/default/tags/setup.rb @@ -1,4 +1,5 @@ def init super - sections[1].push :core + #sections[1].push :core + sections.push :core end From 5b6642995e463ab3318942f23da7f0b023044373 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 2 Nov 2010 14:25:59 -0400 Subject: [PATCH 005/442] minor: update for YARD 0.6.1 --- source/templates/default/tags/setup.rb | 1 - 1 file changed, 1 deletion(-) diff --git a/source/templates/default/tags/setup.rb b/source/templates/default/tags/setup.rb index f67962c7a..6c48678d8 100644 --- a/source/templates/default/tags/setup.rb +++ b/source/templates/default/tags/setup.rb @@ -1,5 +1,4 @@ def init super - #sections[1].push :core sections.push :core end From 9c4634280298c0decaef829f1e80ff4523d79aa5 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 3 Nov 2010 12:40:58 -0400 Subject: [PATCH 006/442] minor: reorganized yardocs. starting to move tutorial and other docs to this repo. --- source/1.0_UPGRADE.md | 21 +++++++++++++++++++++ source/templates/default/tags/html/core.erb | 8 -------- source/templates/default/tags/setup.rb | 4 ---- source/yard_ext.rb | 1 - 4 files changed, 21 insertions(+), 13 deletions(-) create mode 100644 source/1.0_UPGRADE.md delete mode 100644 source/templates/default/tags/html/core.erb delete mode 100644 source/templates/default/tags/setup.rb delete mode 100644 source/yard_ext.rb diff --git a/source/1.0_UPGRADE.md b/source/1.0_UPGRADE.md new file mode 100644 index 000000000..86e294d0a --- /dev/null +++ b/source/1.0_UPGRADE.md @@ -0,0 +1,21 @@ +You can upgrade freely from v0.20 to v1.0. + +However, if you're running a version < 0.20, upgrade to 0.20 +before upgrading to 1.0. + +The upgrade to 0.20 requires some minor code upgrades. + +1. Note the exception changes in HISTORY. Certain exceptions are now scoped under the BSON +module; if you're catching these, you will need to modify your code. + +2. The BSON types are now scoped under the BSON module. + +3. Note that mongo_ext no longer exists. The new gems are bson and bson_ext. + +4. Indexes on GridFS chunks collections should be unique. If you have existing GridFS +collections, you should drop the current index and replace with a unique one. Before you do this, +make sure that index doesn't exist; no need to go through process unnecessarily. +If you do need to create the index, once you have the chunks collection, here are the commands you can run: + + @chunks.drop_index('files_id_1_n_1') + @chunks.create_index([['files_id', Mongo::ASCENDING], ['n', Mongo::ASCENDING]], :unique => true) diff --git a/source/templates/default/tags/html/core.erb b/source/templates/default/tags/html/core.erb deleted file mode 100644 index 641a7aaca..000000000 --- a/source/templates/default/tags/html/core.erb +++ /dev/null @@ -1,8 +0,0 @@ -<% if object.has_tag?(:core) %> -

Core docs:

- -<% end %> diff --git a/source/templates/default/tags/setup.rb b/source/templates/default/tags/setup.rb deleted file mode 100644 index 6c48678d8..000000000 --- a/source/templates/default/tags/setup.rb +++ /dev/null @@ -1,4 +0,0 @@ -def init - super - sections.push :core -end diff --git a/source/yard_ext.rb b/source/yard_ext.rb deleted file mode 100644 index 9ddbf56fb..000000000 --- a/source/yard_ext.rb +++ /dev/null @@ -1 +0,0 @@ -YARD::Tags::Library.define_tag "Core", :core, :with_name From fa0654b634afe9c66b3a7de8e78ab1fc440d94e3 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 3 Nov 2010 12:41:56 -0400 Subject: [PATCH 007/442] minor: add tutorial --- source/TUTORIAL.md | 248 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 248 insertions(+) create mode 100644 source/TUTORIAL.md diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md new file mode 100644 index 000000000..1acad01d2 --- /dev/null +++ b/source/TUTORIAL.md @@ -0,0 +1,248 @@ +# Tutorial + +This tutorial gives many common examples of using MongoDB with the Ruby driver. If you're looking for information on data modeling, see [MongoDB Data Modeling and Rails](http://www.mongodb.org/display/DOCS/MongoDB+Data+Modeling+and+Rails). Links to the various object mappers are listed on our [object mappers page](http://www.mongodb.org/display/DOCS/Object+Mappers+for+Ruby+and+MongoDB). + +Interested in GridFS? See [GridFS in Ruby](http://www.mongodb.org/display/DOCS/GridFS+in+Ruby). + +As always, the [latest source for the Ruby driver](http://github.com/mongodb/mongo-ruby-driver) can be found on [github](http://github.com/mongodb/mongo-ruby-driver/). + +## Installation + +The mongo-ruby-driver gem is served through Rubygems.org. To install, make sure you have the latest version of rubygems. + gem update --system +Next, install the mongo rubygem: + gem install mongo + +The required `bson` gem will be installed automatically. + +For optimum performance, install the bson_ext gem: + + gem install bson_ext + +After installing, you may want to look at the [examples](http://github.com/mongodb/mongo-ruby-driver/tree/master/examples) directory included in the source distribution. These examples walk through some of the basics of using the Ruby driver. + +## Getting started + +#### Using the gem + +All of the code here assumes that you have already executed the following Ruby code: + + require 'rubygems' # not necessary for Ruby 1.9 + require 'mongo' + +#### Making a Connection + +An `Mongo::Connection` instance represents a connection to MongoDB. You use a Connection instance to obtain an Mongo:DB instance, which represents a named database. The database doesn't have to exist - if it doesn't, MongoDB will create it for you. + +You can optionally specify the MongoDB server address and port when connecting. The following example shows three ways to connect to the database "mydb" on the local machine: + + db = Mongo::Connection.new.db("mydb") + db = Mongo::Connection.new("localhost").db("mydb") + db = Mongo::Connection.new("localhost", 27017).db("mydb") + +At this point, the `db` object will be a connection to a MongoDB server for the specified database. Each DB instance uses a separate socket connection to the server. + +If you're trying to connect to a replica set, see [Replica Sets in Ruby](http://www.mongodb.org/display/DOCS/Replica+Sets+in+Ruby). + +#### Listing All Databases + + connection = Mongo::Connection.new # (optional host/port args) + connection.database_names.each { |name| puts name } + connection.database_info.each { |info| puts info.inspect} + + #### Dropping a Database + connection.drop_database('database_name') + #### Authentication (Optional) + +MongoDB can be run in a secure mode where access to databases is controlled through name and password authentication. When run in this mode, any client application must provide a name and password before doing any operations. In the Ruby driver, you simply do the following with the connected mongo object: + + auth = db.authenticate(my_user_name, my_password) + +If the name and password are valid for the database, `auth` will be `true`. Otherwise, it will be `false`. You should look at the MongoDB log for further information if available. + +#### Getting a List Of Collections + +Each database has zero or more collections. You can retrieve a list of them from the db (and print out any that are there): + + db.collection_names.each { |name| puts name } + +and assuming that there are two collections, name and address, in the database, you would see + + name + address + +as the output. + +#### Getting a Collection + +You can get a collection to use using the `collection` method: + coll = db.collection("testCollection") +This is aliased to the \[\] method: + coll = db["testCollection"] + +Once you have this collection object, you can now do things like insert data, query for data, etc. + +#### Inserting a Document + +Once you have the collection object, you can insert documents into the collection. For example, lets make a little document that in JSON would be represented as + + { + "name" : "MongoDB", + "type" : "database", + "count" : 1, + "info" : { + x : 203, + y : 102 + } + } + +Notice that the above has an "inner" document embedded within it. To do this, we can use a Hash or the driver's OrderedHash (which preserves key order) to create the document (including the inner document), and then just simply insert it into the collection using the `insert()` method. + + doc = {"name" => "MongoDB", "type" => "database", "count" => 1, + "info" => {"x" => 203, "y" => '102'` + coll.insert(doc) + +#### Updating a Document + +We can update the previous document using the `update` method. There are a couple ways to update a document. We can rewrite it: + + doc["name"] = "MongoDB Ruby" + coll.update({"_id" => doc["_id"]}, doc) + +Or we can use an atomic operator to change a single value: + + coll.update({"_id" => doc["_id"]}, {"$set" => {"name" => "MongoDB Ruby"`) + +Read [more about updating documents|Updating]. + +#### Finding the First Document In a Collection using `find_one()` + +To show that the document we inserted in the previous step is there, we can do a simple `find_one()` operation to get the first document in the collection. This method returns a single document (rather than the `Cursor` that the `find()` operation returns). + + my_doc = coll.find_one() + puts my_doc.inspect + +and you should see: + + {"_id"=>#, "name"=>"MongoDB", + "info"=>{"x"=>203, "y"=>102}, "type"=>"database", "count"=>1} + +Note the `\_id` element has been added automatically by MongoDB to your document. + +#### Adding Multiple Documents + +To demonstrate some more interesting queries, let's add multiple simple documents to the collection. These documents will have the following form: + { + "i" : value + } + +Here's how to insert them: + + 100.times { |i| coll.insert("i" => i) } + +Notice that we can insert documents of different "shapes" into the same collection. These records are in the same collection as the complex record we inserted above. This aspect is what we mean when we say that MongoDB is "schema-free". + +#### Counting Documents in a Collection + +Now that we've inserted 101 documents (the 100 we did in the loop, plus the first one), we can check to see if we have them all using the `count()` method. + + puts coll.count() + +and it should print `101`. + +#### Using a Cursor to get all of the Documents + +To get all the documents from the collection, we use the `find()` method. `find()` returns a `Cursor` object, which allows us to iterate over the set of documents that matches our query. The Ruby driver's Cursor implemented Enumerable, which allows us to use `Enumerable#each`, `Enumerable#map}, etc. For instance: + + coll.find().each { |row| puts row.inspect } + +and that should print all 101 documents in the collection. + +#### Getting a Single Document with a Query + +We can create a _query_ hash to pass to the `find()` method to get a subset of the documents in our collection. For example, if we wanted to find the document for which the value of the "i" field is 71, we would do the following ; + + coll.find("i" => 71).each { |row| puts row.inspect } + +and it should just print just one document: + + {"_id"=>#, "i"=>71} + +#### Getting a Set of Documents With a Query + +We can use the query to get a set of documents from our collection. For example, if we wanted to get all documents where "i" > 50, we could write: + + coll.find("i" => {"$gt" => 50}).each { |row| puts row } + +which should print the documents where i > 50. We could also get a range, say 20 < i <= 30: + + coll.find("i" => {"$gt" => 20, "$lte" => 30}).each { |row| puts row } + +#### Selecting a subset of fields for a query + +Use the `:fields` option. If you just want fields "a" and "b": + + coll.find("i" => {"$gt" => 50}, :fields => ["a", "b"]).each { |row| puts row } + +#### Querying with Regular Expressions + +Regular expressions can be used to query MongoDB. To find all names that begin with 'a': + + coll.find({"name" => /^a/}) + +You can also construct a regular expression dynamically. To match a given search string: + + search_string = params['search'] + + # Constructor syntax + coll.find({"name" => Regexp.new(search_string)}) + + # Literal syntax + coll.find({"name" => /#{search_string}/}) + +Although MongoDB isn't vulnerable to anything like SQL-injection, it may be worth checking the search string for anything malicious. + +## Indexing + +#### Creating An Index + +MongoDB supports indexes, and they are very easy to add on a collection. To create an index, you specify an index name and an array of field names to be indexed, or a single field name. The following creates an ascending index on the "i" field: + + # create_index assumes ascending order; see method docs + # for details + coll.create_index("i") +To specify complex indexes or a descending index you need to use a slightly more complex syntax - the index specifier must be an Array of [field name, direction] pairs. Directions should be specified as Mongo::ASCENDING or Mongo::DESCENDING: + + # Explicit "ascending" + coll.create_index([["i", Mongo::ASCENDING]]) + +#### Creating and querying on a geospatial index + +First, create the index on a field containing long-lat values: + + people.create_index([["loc", Mongo::GEO2D]]) + +Then get a list of the twenty locations nearest to the point 50, 50: + + people.find({"loc" => {"$near" => [50, 50]}}, {:limit => 20}).each do |p| + puts p.inspect + end + +#### Getting a List of Indexes on a Collection + +You can get a list of the indexes on a collection using `coll.index_information()`. + +## Database Administration + +A database can have one of three profiling levels: off (:off), slow queries only (:slow_only), or all (:all). To see the database level: + + puts db.profiling_level # => off (the symbol :off printed as a string) + db.profiling_level = :slow_only + +Validating a collection will return an interesting hash if all is well or raise an exception if there is a problem. + p db.validate_collection('coll_name') + +## See Also + +* [MongoDB Koans](http://github.com/chicagoruby/MongoDB_Koans) A path to MongoDB enlightenment via the Ruby driver. +* [MongoDB Manual](http://www.mongodb.org/display/DOCS/Developer+Zone) From f11a78677d4cf57ac2078540ade5046bc0ebd194 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 3 Nov 2010 13:00:34 -0400 Subject: [PATCH 008/442] minor: more reorganization --- source/CREDITS.md | 96 +++++++++++++++ source/HISTORY.md | 215 ++++++++++++++++++++++++++++++++++ source/examples/admin.rb | 43 +++++++ source/examples/capped.rb | 22 ++++ source/examples/cursor.rb | 48 ++++++++ source/examples/gridfs.rb | 44 +++++++ source/examples/index_test.rb | 126 ++++++++++++++++++++ source/examples/info.rb | 31 +++++ source/examples/queries.rb | 70 +++++++++++ source/examples/simple.rb | 24 ++++ source/examples/strict.rb | 35 ++++++ source/examples/types.rb | 36 ++++++ 12 files changed, 790 insertions(+) create mode 100644 source/CREDITS.md create mode 100644 source/HISTORY.md create mode 100644 source/examples/admin.rb create mode 100644 source/examples/capped.rb create mode 100644 source/examples/cursor.rb create mode 100644 source/examples/gridfs.rb create mode 100644 source/examples/index_test.rb create mode 100644 source/examples/info.rb create mode 100644 source/examples/queries.rb create mode 100644 source/examples/simple.rb create mode 100644 source/examples/strict.rb create mode 100644 source/examples/types.rb diff --git a/source/CREDITS.md b/source/CREDITS.md new file mode 100644 index 000000000..816adeab8 --- /dev/null +++ b/source/CREDITS.md @@ -0,0 +1,96 @@ +Adrian Madrid, aemadrid@gmail.com +* bin/mongo_console +* examples/benchmarks.rb +* examples/irb.rb +* Modifications to examples/simple.rb +* Found plenty of bugs and missing features. +* Ruby 1.9 support. +* Gem support. +* Many other code suggestions and improvements. + +Aman Gupta, aman@tmm1.net +* Collection#save +* Noted bug in returning query batch size. + +Jon Crosby, jon@joncrosby.me +* Some code clean-up + +John Nunemaker, http://railstips.org +* Collection#create_index takes symbols as well as strings +* Fix for Collection#save +* Add logger convenience methods to connection and database + +David James, djames@sunlightfoundation.com +* Fix dates to return as UTC + +Paul Dlug, paul.dlug@gmail.com +* Generate _id on the client side if not provided +* Collection#insert and Collection#save return _id + +Durran Jordan, durran@gmail.com +* DB#collections +* Support for specifying sort order as array of [key, direction] pairs +* OrderedHash#update aliases to merge! + +Cyril Mougel, cyril.mougel@gmail.com +* Initial logging support +* Test case + +Honlgi Lai (Phusion) +* Significant performance audit. See commits prior to 1.0.9 + +Jack Chen, chendo on github +* Test case + fix for deserializing pre-epoch Time instances + +Michael Bernstein, mrb on github +* #sort method for Cursor instances + +Paulo Ahahgon, pahagon on github +* removed hard limit + +Les Hill, leshill on github +* OrderedHash#each returns self + +Sean Cribbs, seancribbs on github +* Modified standard_benchmark to allow profiling +* c ext for faster ObjectID creation + +Sunny Hirai +* Suggested hashcode fix for Mongo::ObjectID +* Noted index ordering bug. +* GridFS performance boost + +Christos Trochalakis +* Added map/reduce helper + +Blythe Dunham +* Added finalize option to map/reduce + +Matt Powell (fauxparse) +* Added GridStore#mv + +Patrick Collison +* Added safe mode for Collection#remove + +Chuck Remes +* Extraction of BSON into separate gems +* Extensions compile on Rubinius +* Performance improvements for INT in C extensions +* Performance improvements for JRuby BSON encoder and callback classes + +Dmitrii Golub (Houdini) and Jacques Crocker (railsjedi) +* Support open to exclude fields on query + +dfitzgibbon +* patch for ensuring bson_ext compatibility with early release of Ruby 1.8.5 + +Matt Taylor +* Noticed excessive calls to ObjectId#to_s. As a result, stopped creating +log messages when no logger was passed to Mongo::Connection. Resulted in a significant +performance improvement. + +Hongli Lai (Phusion) +* Significant performance improvements. See commits. + +Mislav Marohnić +* Replaced returning with each_with_object diff --git a/source/HISTORY.md b/source/HISTORY.md new file mode 100644 index 000000000..93305c38a --- /dev/null +++ b/source/HISTORY.md @@ -0,0 +1,215 @@ +1.1.1 2010-10-14 +* Several critical JRuby bug fixes +* Fixes for JRuby in 1.9 mode +* Check keys and move id only when necessary for JRuby encoder + +1.1 2010-10-4 +* Official JRuby support via Java extensons for BSON (beta) +* Connection#lock! and Connection#unlock! for easy fsync lock +* Note: BSON::Code is no longer a subclass of String. + +1.0.9 2010-9-20 +* Significant performance improvements + +1.0.8 2010-8-27 + +* Cursor#rewind! and more consistent Cursor Enumberable behavior +* Deprecated ObjectID for ObjectId +* Numerous minor bug fixes. + +1.0.7 2010-8-4 + +* A few minor test/doc fixes. +* Better tests for replica sets and replication acknowledgment. +* Deprecated DB#error and DB#last_status + +1.0.6 2010-7-26 + +* Replica set support. +* Collection#map_reduce bug fix. + +1.0.5 2010-7-13 + +* Fix for bug introduced in 1.0.4. + +1.0.4 2010-7-13 + +* Removed deprecated + - Cursor admin option + - DB#query + - DB#create_index (use Collection#create_index) + - DB#command only takes hash options now +* j2bson executable (neomantra) +* Fixed bson_ext compilation on Solaris (slyphon) +* System JS helpers (neovintage) +* Use one mutex per thread on pooled connections (cremes) +* Check for CursorNotFound response flag +* MapReduce can return raw command output using :raw +* BSON::OrderedHash equality with other Ruby hashes (Ryan Angilly) +* Fix for broken Socket.send with large payloads (Frédéric De Jaeger) +* Lots of minor improvements. See commmits. + +1.0.3 2010-6-15 + +* Optimiztion for BSON::OrderedHash +* Some important fixes. + +1.0.2 2010-6-5 +This is a minor release for fixing an incompatibility with MongoDB v1.5.2 + +* Fix for boolean response on commands for core server v1.5.2 +* BSON.read_bson_document and b2json executable (neomantra) +* BSON::ObjectID() shortcut for BSON::ObjectID.from_string (tmm1) +* Various bug fixes. + +1.0.1 2010-5-7 + +* set Encoding.default_internal +* DEPRECATE JavaScript string on Collection#find. You now must specify $where explicitly. +* Added Grid#exist? and GridFileSystem#exist? +* Support for replication acknowledgment +* Support for $slice +* Namespaced OrderedHash under BSON (sleverbor) + +1.0 2010-4-29 +Note: if upgrading from versions prior to 0.20, be sure to upgrade +to 0.20 before upgrading to 1.0. + +* Inspected ObjectID is represented in MongoDB extended json format. +* Support for tailable cursors. +* Configurable query response batch size (thx. to Aman Gupta) + +* bson_ext installs on early release of Ruby 1.8.5 (dfitzgibbon) +* Deprecated DB#create_index. Use Collection#create_index index. +* Removed deprecated Grid#put syntax; no longer requires a filename. + +0.20.1 2010-4-7 + * Added bson gem dependency. + +0.20 2010-4-7 +If upgrading from a previous version of the Ruby driver, please read these notes carefully, +along with the 0.20_UPGRADE doc. + +* Support for new commands: + * Collection#find_and_modify + * Collection#stats + * DB#stats +* Query :fields options allows for values of 0 to exclude fields (houdini, railsjedi). +* GridFS + * Option to delete old versions of GridFileSystem entries. + * Filename is now optional for Grid#put. + * Option to write arbitrary attributes to a file: @grid.put(@data, :favorite_phrase => "blimey!") + * Indexes created on the chunks collection are now unique. If you have an existing chunks collection, + you may want to remove +* Removed the following deprecated items: + * GridStore class + * RegexpOfHolding class + * Paired connections must now be initialized with Connection.paired + +* BSON-related code extracted into two separate gems: bson and bson_ext (thx to Chuck Remes). + * mongo_ext no longer exists. + * BSON::Binary constructor can now take a string, which will be packed into an array. + * Exception class adjustments: + * Mongo::InvalidObjectID moved to BSON::InvalidObjectID + * Mongo::InvalidDocument moved to BSON::InvalidDocument + * Mongo::InvalidStringEncoding moved to BSON::InvalidStringEncoding + * Mongo::InvalidName replaced by Mongo::InvalidNSName and BSON::InvalidKeyName + * BSON types are now namespaced under the BSON module. These types include: + * Binary + * ObjectID + * Code + * DBRef + * MinKey and MaxKey + * Extensions compile on Rubinius (Chuck Remes). + +0.19.3 2010-4-5 +* Minor fix for assert_valid_keys. + +0.19.2 2010-4-5 +This release fixes a major bug and is the final release +in the 0.19 series. The next release, 0.20.0, will introduce +separate gems for bson and bson_ext and may require small +changes to existing code bases. Expect that release in the next +few days. +* Fix for Grid#delete bug. +* Log messages read like valid ruby driver code. +* Cursor#has_next. +* Tests for MongoDB 1.4 features. +* Flexible index creation method with Mongo::GEO2D constant. + +0.19.1 2010-3-2 +* Fix for HashWithIndifferentAccess in ActiveSupport-3.0 + +0.19 2010-3-1 +* Deprecated GridFS::GridStore. Grid and GridFileSystem classes replace +the GridFS implementation with a simpler API and vastly-improved performance. +See http://www.mongodb.org/display/DOCS/GridFS+in+Ruby for more details. +* Safe mode for Grid and GridFileSystem. +* Grid and GridFileSystem use Mime/Types to detect content type (if available) +* Connection API simplified. Use Connection.paired for pairs and Connection.from_uri to +use MongoDB's connection URI specification. +* Authentication can be saved so that reauthentication happens automatically +on reconnect. +* Raise exception if authentication fails. +* Raise exception if index creation fails. +* Removed a number of deprecated methods and classes. +* Several bug fixes. +* Nearing 1.0! + +0.18.3 2010-1-25 +* Convert docs to YARD +* Support MongoDB extended JSON on ObjectID#to_json +* ObjectID#from_time for performing date range queries on _id (thx., Sunny Hirai) +* GridStore#mv for renaming files (Matt Powell) +* Safe mode for Collection#remove (Patrick Collison) +* Support BSON types MinKey and MaxKey +* DEPRECATED Admin, XMLToRuby, and RegexpOfHolding classes. +* Handle unsupported Numeric types gracefully (Complex, Rational, BigDecimal) +* Handle unsupported Time types gracefully (Date, DateTime, ActiveSupport::TimeWithZone) +* Numerous small bug fixes +* Minor performance improvements + +0.18.2 2009-12-29 +* Significant GridStore performance improvement (thx., Sunny Hirai) +* Enabled support for keyf on group +* Support :query option for Collection#distinct +* Support :finalize option for Collection#group +* (0.18.1) ObjectID#generation_time returns a created_at timestamp. +* Deprecated Command#group running as a JS eval; should now be run as a command. +* Deprecated Cursor#next_object for Cursor#next_document +* Character encoding fixes for C extension +* Enforce 4MB limit on document creation +* Simplified connection pooling code +* Fixes for connection pooling on Ruby 1.8.6/Windows. + +0.18.1 2009-12-05 +* Fixed issue with negative dates in Ruby 1.9 +* Minor refactorings for C extension and BSON classes +* Ensure UTF-8 in Ruby 1.8 +* Fix for connections on non-default port (Delano Mandelbaum) +* More explicit test suite tasks for running with/without C extension. + +0.18 2009-11-25 +* Connections now support connection pooling. See http://api.mongodb.org/ruby/0.18/classes/Mongo/Connection.html#M000158 +* Deprecated :auto_reconnect option on connection; if the driver fails to + connect, it will automatically try to reconnect on the subsequent operation. + See http://www.mongodb.org/display/DOCS/Replica+Pairs+in+Ruby +* Added Collection#map_reduce helper (Christos Trochalakis) +* Deprecated DB#db_command in favor of DB#command. +* Removed deprecated old sort options, :offset, and Connection#clear. +* Lots of internal code restructuring for better maintainability. + +0.17.1 2009-11-17 +* Index ordering fix +* Notice to install mongo_ext + +0.17 2009-11-16 +* Performance improvements + * large document inserts twice as fast as 0.16 + * queries 18% faster than 0.16 on average + * see benchmark comparison: http://gist.github.com/236062 +* Support for multi-update for Mongo >= 1.1.3 (See Collection#update) +* Collection#distinct +* Connection#copy_database (voodootikigod) +* C optimizations for ByteBuffer#to_s and ObjectID#generate (seancribbs) +* Continue code restructuring for performance and simplicity. diff --git a/source/examples/admin.rb b/source/examples/admin.rb new file mode 100644 index 000000000..e445be9ac --- /dev/null +++ b/source/examples/admin.rb @@ -0,0 +1,43 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) + +require 'mongo' +require 'pp' + +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts "Connecting to #{host}:#{port}" +con = Mongo::Connection.new(host, port) +db = con.db('ruby-mongo-examples') +coll = db.create_collection('test') + +# Erase all records from collection, if any +coll.remove + +admin = con['admin'] + +# Profiling level set/get +puts "Profiling level: #{admin.profiling_level}" + +# Start profiling everything +admin.profiling_level = :all + +# Read records, creating a profiling event +coll.find().to_a + +# Stop profiling +admin.profiling_level = :off + +# Print all profiling info +pp admin.profiling_info + +# Validate returns a hash if all is well and +# raises an exception if there is a problem. +info = db.validate_collection(coll.name) +puts "valid = #{info['ok']}" +puts info['result'] + +# Destroy the collection +coll.drop diff --git a/source/examples/capped.rb b/source/examples/capped.rb new file mode 100644 index 000000000..6cdf07528 --- /dev/null +++ b/source/examples/capped.rb @@ -0,0 +1,22 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) +require 'mongo' + +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts "Connecting to #{host}:#{port}" +db = Connection.new(host, port).db('ruby-mongo-examples') +db.drop_collection('test') + +# A capped collection has a max size and, optionally, a max number of records. +# Old records get pushed out by new ones once the size or max num records is reached. +coll = db.create_collection('test', :capped => true, :size => 1024, :max => 12) + +100.times { |i| coll.insert('a' => i+1) } + +# We will only see the last 12 records +coll.find().each { |row| p row } + +coll.drop diff --git a/source/examples/cursor.rb b/source/examples/cursor.rb new file mode 100644 index 000000000..4bf0fe89c --- /dev/null +++ b/source/examples/cursor.rb @@ -0,0 +1,48 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) + +require 'mongo' +require 'pp' + +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts "Connecting to #{host}:#{port}" +db = Connection.new(host, port).db('ruby-mongo-examples') +coll = db.collection('test') + +# Erase all records from collection, if any +coll.remove + +# Insert 3 records +3.times { |i| coll.insert({'a' => i+1}) } + +# Cursors don't run their queries until you actually attempt to retrieve data +# from them. + +# Find returns a Cursor, which is Enumerable. You can iterate: +coll.find().each { |row| pp row } + +# You can turn it into an array: +array = coll.find().to_a + +# You can iterate after turning it into an array (the cursor will iterate over +# the copy of the array that it saves internally.) +cursor = coll.find() +array = cursor.to_a +cursor.each { |row| pp row } + +# You can get the next object +first_object = coll.find().next_document + +# next_document returns nil if there are no more objects that match +cursor = coll.find() +obj = cursor.next_document +while obj + pp obj + obj = cursor.next_document +end + +# Destroy the collection +coll.drop diff --git a/source/examples/gridfs.rb b/source/examples/gridfs.rb new file mode 100644 index 000000000..a1769f782 --- /dev/null +++ b/source/examples/gridfs.rb @@ -0,0 +1,44 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) +def assert + raise "Failed!" unless yield +end + +require 'mongo' +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts "Connecting to #{host}:#{port}" +db = Connection.new(host, port).db('ruby-mongo-examples') + +data = "hello, world!" + +grid = Grid.new(db) + +# Write a new file. data can be a string or an io object responding to #read. +id = grid.put(data, :filename => 'hello.txt') + +# Read it and print out the contents +file = grid.get(id) +puts file.read + +# Delete the file +grid.delete(id) + +begin +grid.get(id) +rescue => e + assert {e.class == Mongo::GridError} +end + +# Metadata +id = grid.put(data, :filename => 'hello.txt', :content_type => 'text/plain', :metadata => {'name' => 'hello'}) +file = grid.get(id) + +p file.content_type +p file.metadata.inspect +p file.chunk_size +p file.file_length +p file.filename +p file.data diff --git a/source/examples/index_test.rb b/source/examples/index_test.rb new file mode 100644 index 000000000..c9398a8b0 --- /dev/null +++ b/source/examples/index_test.rb @@ -0,0 +1,126 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) + +require 'mongo' + +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts ">> Connecting to #{host}:#{port}" +db = Connection.new(host, port).db('ruby-mongo-index_test') + +class Exception + def errmsg + "%s: %s\n%s" % [self.class, message, (backtrace || []).join("\n") << "\n"] + end +end + +puts ">> Dropping collection test" +begin + res = db.drop_collection('test') + puts "dropped : #{res.inspect}" +rescue => e + puts "Error: #{e.errmsg}" +end + +puts ">> Creating collection test" +begin + coll = db.collection('test') + puts "created : #{coll.inspect}" +rescue => e + puts "Error: #{e.errmsg}" +end + +OBJS_COUNT = 100 + +puts ">> Generating test data" +msgs = %w{hola hello aloha ciao} +arr = (0...OBJS_COUNT).collect {|x| { :number => x, :rndm => (rand(5)+1), :msg => msgs[rand(4)] }} +puts "generated" + +puts ">> Inserting data (#{arr.size})" +coll.insert(arr) +puts "inserted" + +puts ">> Creating index" +#res = coll.create_index "all", :_id => 1, :number => 1, :rndm => 1, :msg => 1 +res = coll.create_index [[:number, 1], [:rndm, 1], [:msg, 1]] +puts "created index: #{res.inspect}" +# ============================ Mongo Log ============================ +# Fri Dec 5 14:45:02 Adding all existing records for ruby-mongo-console.test to new index +# *** +# Bad data or size in BSONElement::size() +# bad type:30 +# totalsize:11 fieldnamesize:4 +# lastrec: +# Fri Dec 5 14:45:02 ruby-mongo-console.system.indexes Assertion failure false jsobj.cpp a0 +# Fri Dec 5 14:45:02 database: ruby-mongo-console op:7d2 0 +# Fri Dec 5 14:45:02 ns: ruby-mongo-console.system.indexes + +puts ">> Gathering index information" +begin + res = coll.index_information + puts "index_information : #{res.inspect}" +rescue => e + puts "Error: #{e.errmsg}" +end +# ============================ Console Output ============================ +# RuntimeError: Keys for index on return from db was nil. Coll = ruby-mongo-console.test +# from ./bin/../lib/mongo/db.rb:135:in `index_information' +# from (irb):11:in `collect' +# from ./bin/../lib/mongo/cursor.rb:47:in `each' +# from ./bin/../lib/mongo/db.rb:130:in `collect' +# from ./bin/../lib/mongo/db.rb:130:in `index_information' +# from ./bin/../lib/mongo/collection.rb:74:in `index_information' +# from (irb):11 + +puts ">> Dropping index" +begin + res = coll.drop_index "number_1_rndm_1_msg_1" + puts "dropped : #{res.inspect}" +rescue => e + puts "Error: #{e.errmsg}" +end + +# ============================ Console Output ============================ +# => {"nIndexesWas"=>2.0, "ok"=>1.0} +# ============================ Mongo Log ============================ +# 0x41802a 0x411549 0x42bac6 0x42c1f6 0x42c55b 0x42e6f7 0x41631e 0x41a89d 0x41ade2 0x41b448 0x4650d2 0x4695ad +# db/db(_Z12sayDbContextPKc+0x17a) [0x41802a] +# db/db(_Z8assertedPKcS0_j+0x9) [0x411549] +# db/db(_ZNK11BSONElement4sizeEv+0x1f6) [0x42bac6] +# db/db(_ZN7BSONObj8getFieldEPKc+0xa6) [0x42c1f6] +# db/db(_ZN7BSONObj14getFieldDottedEPKc+0x11b) [0x42c55b] +# db/db(_ZN7BSONObj19extractFieldsDottedES_R14BSONObjBuilder+0x87) [0x42e6f7] +# db/db(_ZN12IndexDetails17getKeysFromObjectER7BSONObjRSt3setIS0_St4lessIS0_ESaIS0_EE+0x24e) [0x41631e] +# db/db(_Z12_indexRecordR12IndexDetailsR7BSONObj7DiskLoc+0x5d) [0x41a89d] +# db/db(_Z18addExistingToIndexPKcR12IndexDetails+0xb2) [0x41ade2] +# db/db(_ZN11DataFileMgr6insertEPKcPKvib+0x508) [0x41b448] +# db/db(_Z14receivedInsertR7MessageRSt18basic_stringstreamIcSt11char_traitsIcESaIcEE+0x112) [0x4650d2] +# db/db(_Z10connThreadv+0xb4d) [0x4695ad] +# Fri Dec 5 14:45:02 ruby-mongo-console.system.indexes Caught Assertion insert, continuing +# Fri Dec 5 14:47:59 CMD: deleteIndexes ruby-mongo-console.test +# d->nIndexes was 2 +# alpha implementation, space not reclaimed + +puts ">> Gathering index information" +begin + res = coll.index_information + puts "index_information : #{res.inspect}" +rescue => e + puts "Error: #{e.errmsg}" +end +# ============================ Console Output ============================ +# RuntimeError: Keys for index on return from db was nil. Coll = ruby-mongo-console.test +# from ./bin/../lib/mongo/db.rb:135:in `index_information' +# from (irb):15:in `collect' +# from ./bin/../lib/mongo/cursor.rb:47:in `each' +# from ./bin/../lib/mongo/db.rb:130:in `collect' +# from ./bin/../lib/mongo/db.rb:130:in `index_information' +# from ./bin/../lib/mongo/collection.rb:74:in `index_information' +# from (irb):15 + +puts ">> Closing connection" +db.close +puts "closed" diff --git a/source/examples/info.rb b/source/examples/info.rb new file mode 100644 index 000000000..7790e1992 --- /dev/null +++ b/source/examples/info.rb @@ -0,0 +1,31 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) + +require 'mongo' + +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts "Connecting to #{host}:#{port}" +db = Connection.new(host, port).db('ruby-mongo-examples') +coll = db.collection('test') + +# Erase all records from collection, if any +coll.remove + +# Insert 3 records +3.times { |i| coll.insert({'a' => i+1}) } + +# Collection names in database +p db.collection_names + +# More information about each collection +p db.collections_info + +# Index information +coll.create_index('a') +p db.index_information('test') + +# Destroy the collection +coll.drop diff --git a/source/examples/queries.rb b/source/examples/queries.rb new file mode 100644 index 000000000..9707e477a --- /dev/null +++ b/source/examples/queries.rb @@ -0,0 +1,70 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) + +require 'mongo' +require 'pp' + +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts "Connecting to #{host}:#{port}" +db = Connection.new(host, port).db('ruby-mongo-examples') +coll = db.collection('test') + +# Remove all records, if any +coll.remove + +# Insert three records +coll.insert('a' => 1) +coll.insert('a' => 2) +coll.insert('b' => 3) + +# Count. +puts "There are #{coll.count()} records." + +# Find all records. find() returns a Cursor. +cursor = coll.find() + +# Print them. Note that all records have an _id automatically added by the +# database. See pk.rb for an example of how to use a primary key factory to +# generate your own values for _id. +cursor.each { |row| pp row } + +# Cursor has a to_a method that slurps all records into memory. +rows = coll.find().to_a +rows.each { |row| pp row } + +# See Collection#find. From now on in this file, we won't be printing the +# records we find. +coll.find('a' => 1) + +# Find records sort by 'a', skip 1, limit 2 records. +# Sort can be single name, array, or hash. +coll.find({}, {:skip => 1, :limit => 2, :sort => 'a'}) + +# Find all records with 'a' > 1. There is also $lt, $gte, and $lte. +coll.find({'a' => {'$gt' => 1}}) +coll.find({'a' => {'$gt' => 1, '$lte' => 3}}) + +# Find all records with 'a' in a set of values. +coll.find('a' => {'$in' => [1,2]}) + +# Find by regexp +coll.find('a' => /[1|2]/) + +# Print query explanation +pp coll.find('a' => /[1|2]/).explain() + +# Use a hint with a query. Need an index. Hints can be stored with the +# collection, in which case they will be used with all queries, or they can be +# specified per query, in which case that hint overrides the hint associated +# with the collection if any. +coll.create_index('a') +coll.hint = 'a' + +# You will see a different explanation now that the hint is in place +pp coll.find('a' => /[1|2]/).explain() + +# Override hint for single query +coll.find({'a' => 1}, :hint => 'b') diff --git a/source/examples/simple.rb b/source/examples/simple.rb new file mode 100644 index 000000000..76f354c05 --- /dev/null +++ b/source/examples/simple.rb @@ -0,0 +1,24 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) + +require 'mongo' + +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts "Connecting to #{host}:#{port}" +db = Connection.new(host, port).db('ruby-mongo-examples') +coll = db.collection('test') + +# Erase all records from collection, if any +coll.remove + +# Insert 3 records +3.times { |i| coll.insert({'a' => i+1}) } + +puts "There are #{coll.count()} records in the test collection. Here they are:" +coll.find().each { |doc| puts doc.inspect } + +# Destroy the collection +coll.drop diff --git a/source/examples/strict.rb b/source/examples/strict.rb new file mode 100644 index 000000000..5718c7e9c --- /dev/null +++ b/source/examples/strict.rb @@ -0,0 +1,35 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) + +require 'mongo' + +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts "Connecting to #{host}:#{port}" +db = Connection.new(host, port).db('ruby-mongo-examples') + +db.drop_collection('does-not-exist') +db.create_collection('test') + +db.strict = true + +begin + # Can't reference collection that does not exist + db.collection('does-not-exist') + puts "error: expected exception" +rescue => ex + puts "expected exception: #{ex}" +end + +begin + # Can't create collection that already exists + db.create_collection('test') + puts "error: expected exception" +rescue => ex + puts "expected exception: #{ex}" +end + +db.strict = false +db.drop_collection('test') diff --git a/source/examples/types.rb b/source/examples/types.rb new file mode 100644 index 000000000..0589463fa --- /dev/null +++ b/source/examples/types.rb @@ -0,0 +1,36 @@ +$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) + +require 'mongo' +require 'pp' + +include Mongo + +host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' +port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT + +puts "Connecting to #{host}:#{port}" +db = Connection.new(host, port).db('ruby-mongo-examples') +coll = db.collection('test') + +# Remove all records, if any +coll.remove + +# Insert record with all sorts of values +coll.insert('array' => [1, 2, 3], + 'string' => 'hello', + 'hash' => {'a' => 1, 'b' => 2}, + 'date' => Time.now, # milliseconds only; microseconds are not stored + 'oid' => ObjectID.new, + 'binary' => Binary.new([1, 2, 3]), + 'int' => 42, + 'float' => 33.33333, + 'regex' => /foobar/i, + 'boolean' => true, + 'where' => Code.new('this.x == 3'), + 'dbref' => DBRef.new(coll.name, ObjectID.new), + 'null' => nil, + 'symbol' => :zildjian) + +pp coll.find().next_document + +coll.remove From 7f4e731f8079665af5014a8d5cf7a5befb49e1a8 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 3 Nov 2010 13:16:19 -0400 Subject: [PATCH 009/442] minor: credits and history updates. now appear in ydoc --- source/CREDITS.md | 31 +++++++-- source/HISTORY.md | 156 +++++++++++++--------------------------------- 2 files changed, 72 insertions(+), 115 deletions(-) diff --git a/source/CREDITS.md b/source/CREDITS.md index 816adeab8..673457fe3 100644 --- a/source/CREDITS.md +++ b/source/CREDITS.md @@ -1,4 +1,7 @@ +# Credits + Adrian Madrid, aemadrid@gmail.com + * bin/mongo_console * examples/benchmarks.rb * examples/irb.rb @@ -9,88 +12,108 @@ Adrian Madrid, aemadrid@gmail.com * Many other code suggestions and improvements. Aman Gupta, aman@tmm1.net + * Collection#save * Noted bug in returning query batch size. Jon Crosby, jon@joncrosby.me + * Some code clean-up John Nunemaker, http://railstips.org + * Collection#create_index takes symbols as well as strings * Fix for Collection#save * Add logger convenience methods to connection and database David James, djames@sunlightfoundation.com + * Fix dates to return as UTC Paul Dlug, paul.dlug@gmail.com + * Generate _id on the client side if not provided * Collection#insert and Collection#save return _id Durran Jordan, durran@gmail.com + * DB#collections * Support for specifying sort order as array of [key, direction] pairs * OrderedHash#update aliases to merge! Cyril Mougel, cyril.mougel@gmail.com + * Initial logging support * Test case -Honlgi Lai (Phusion) -* Significant performance audit. See commits prior to 1.0.9 - Jack Chen, chendo on github + * Test case + fix for deserializing pre-epoch Time instances Michael Bernstein, mrb on github + * #sort method for Cursor instances Paulo Ahahgon, pahagon on github + * removed hard limit Les Hill, leshill on github + * OrderedHash#each returns self Sean Cribbs, seancribbs on github + * Modified standard_benchmark to allow profiling -* c ext for faster ObjectID creation +* C ext for faster ObjectID creation Sunny Hirai + * Suggested hashcode fix for Mongo::ObjectID * Noted index ordering bug. * GridFS performance boost Christos Trochalakis + * Added map/reduce helper Blythe Dunham + * Added finalize option to map/reduce Matt Powell (fauxparse) + * Added GridStore#mv Patrick Collison + * Added safe mode for Collection#remove Chuck Remes + * Extraction of BSON into separate gems * Extensions compile on Rubinius * Performance improvements for INT in C extensions * Performance improvements for JRuby BSON encoder and callback classes Dmitrii Golub (Houdini) and Jacques Crocker (railsjedi) + * Support open to exclude fields on query dfitzgibbon + * patch for ensuring bson_ext compatibility with early release of Ruby 1.8.5 Matt Taylor + * Noticed excessive calls to ObjectId#to_s. As a result, stopped creating log messages when no logger was passed to Mongo::Connection. Resulted in a significant performance improvement. Hongli Lai (Phusion) + * Significant performance improvements. See commits. Mislav Marohnić + * Replaced returning with each_with_object diff --git a/source/HISTORY.md b/source/HISTORY.md index 93305c38a..be1ce7acf 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,44 +1,57 @@ -1.1.1 2010-10-14 +# MongoDB Ruby Driver History + +### 1.1.1 +2010-10-14 + * Several critical JRuby bug fixes * Fixes for JRuby in 1.9 mode * Check keys and move id only when necessary for JRuby encoder -1.1 2010-10-4 +## 1.1 +2010-10-4 + * Official JRuby support via Java extensons for BSON (beta) * Connection#lock! and Connection#unlock! for easy fsync lock * Note: BSON::Code is no longer a subclass of String. -1.0.9 2010-9-20 -* Significant performance improvements +### 1.0.9 +2010-9-20 + +* Significant performance improvements (with a lot of help from Hongli Lai) -1.0.8 2010-8-27 +### 1.0.8 +2010-8-27 * Cursor#rewind! and more consistent Cursor Enumberable behavior * Deprecated ObjectID for ObjectId * Numerous minor bug fixes. -1.0.7 2010-8-4 +### 1.0.7 +2010-8-4 * A few minor test/doc fixes. * Better tests for replica sets and replication acknowledgment. * Deprecated DB#error and DB#last_status -1.0.6 2010-7-26 +### 1.0.6 +2010-7-26 * Replica set support. * Collection#map_reduce bug fix. -1.0.5 2010-7-13 +### 1.0.5 +2010-7-13 * Fix for bug introduced in 1.0.4. -1.0.4 2010-7-13 +### 1.0.4 +2010-7-13 * Removed deprecated - - Cursor admin option - - DB#query - - DB#create_index (use Collection#create_index) - - DB#command only takes hash options now + * Cursor admin option + * DB#query + * DB#create_index (use Collection#create_index) + * DB#command only takes hash options now * j2bson executable (neomantra) * Fixed bson_ext compilation on Solaris (slyphon) * System JS helpers (neovintage) @@ -49,12 +62,15 @@ * Fix for broken Socket.send with large payloads (Frédéric De Jaeger) * Lots of minor improvements. See commmits. -1.0.3 2010-6-15 +### 1.0.3 +2010-6-15 * Optimiztion for BSON::OrderedHash * Some important fixes. -1.0.2 2010-6-5 +### 1.0.2 +2010-6-5 + This is a minor release for fixing an incompatibility with MongoDB v1.5.2 * Fix for boolean response on commands for core server v1.5.2 @@ -62,7 +78,8 @@ This is a minor release for fixing an incompatibility with MongoDB v1.5.2 * BSON::ObjectID() shortcut for BSON::ObjectID.from_string (tmm1) * Various bug fixes. -1.0.1 2010-5-7 +### 1.0.1 +2010-5-7 * set Encoding.default_internal * DEPRECATE JavaScript string on Collection#find. You now must specify $where explicitly. @@ -71,7 +88,8 @@ This is a minor release for fixing an incompatibility with MongoDB v1.5.2 * Support for $slice * Namespaced OrderedHash under BSON (sleverbor) -1.0 2010-4-29 +## 1.0 +2010-4-29 Note: if upgrading from versions prior to 0.20, be sure to upgrade to 0.20 before upgrading to 1.0. @@ -83,10 +101,14 @@ to 0.20 before upgrading to 1.0. * Deprecated DB#create_index. Use Collection#create_index index. * Removed deprecated Grid#put syntax; no longer requires a filename. -0.20.1 2010-4-7 - * Added bson gem dependency. +### 0.20.1 +2010-4-7 + +* Added bson gem dependency. + +### 0.20 +2010-4-7 -0.20 2010-4-7 If upgrading from a previous version of the Ruby driver, please read these notes carefully, along with the 0.20_UPGRADE doc. @@ -122,94 +144,6 @@ along with the 0.20_UPGRADE doc. * MinKey and MaxKey * Extensions compile on Rubinius (Chuck Remes). -0.19.3 2010-4-5 -* Minor fix for assert_valid_keys. - -0.19.2 2010-4-5 -This release fixes a major bug and is the final release -in the 0.19 series. The next release, 0.20.0, will introduce -separate gems for bson and bson_ext and may require small -changes to existing code bases. Expect that release in the next -few days. -* Fix for Grid#delete bug. -* Log messages read like valid ruby driver code. -* Cursor#has_next. -* Tests for MongoDB 1.4 features. -* Flexible index creation method with Mongo::GEO2D constant. - -0.19.1 2010-3-2 -* Fix for HashWithIndifferentAccess in ActiveSupport-3.0 - -0.19 2010-3-1 -* Deprecated GridFS::GridStore. Grid and GridFileSystem classes replace -the GridFS implementation with a simpler API and vastly-improved performance. -See http://www.mongodb.org/display/DOCS/GridFS+in+Ruby for more details. -* Safe mode for Grid and GridFileSystem. -* Grid and GridFileSystem use Mime/Types to detect content type (if available) -* Connection API simplified. Use Connection.paired for pairs and Connection.from_uri to -use MongoDB's connection URI specification. -* Authentication can be saved so that reauthentication happens automatically -on reconnect. -* Raise exception if authentication fails. -* Raise exception if index creation fails. -* Removed a number of deprecated methods and classes. -* Several bug fixes. -* Nearing 1.0! - -0.18.3 2010-1-25 -* Convert docs to YARD -* Support MongoDB extended JSON on ObjectID#to_json -* ObjectID#from_time for performing date range queries on _id (thx., Sunny Hirai) -* GridStore#mv for renaming files (Matt Powell) -* Safe mode for Collection#remove (Patrick Collison) -* Support BSON types MinKey and MaxKey -* DEPRECATED Admin, XMLToRuby, and RegexpOfHolding classes. -* Handle unsupported Numeric types gracefully (Complex, Rational, BigDecimal) -* Handle unsupported Time types gracefully (Date, DateTime, ActiveSupport::TimeWithZone) -* Numerous small bug fixes -* Minor performance improvements - -0.18.2 2009-12-29 -* Significant GridStore performance improvement (thx., Sunny Hirai) -* Enabled support for keyf on group -* Support :query option for Collection#distinct -* Support :finalize option for Collection#group -* (0.18.1) ObjectID#generation_time returns a created_at timestamp. -* Deprecated Command#group running as a JS eval; should now be run as a command. -* Deprecated Cursor#next_object for Cursor#next_document -* Character encoding fixes for C extension -* Enforce 4MB limit on document creation -* Simplified connection pooling code -* Fixes for connection pooling on Ruby 1.8.6/Windows. - -0.18.1 2009-12-05 -* Fixed issue with negative dates in Ruby 1.9 -* Minor refactorings for C extension and BSON classes -* Ensure UTF-8 in Ruby 1.8 -* Fix for connections on non-default port (Delano Mandelbaum) -* More explicit test suite tasks for running with/without C extension. - -0.18 2009-11-25 -* Connections now support connection pooling. See http://api.mongodb.org/ruby/0.18/classes/Mongo/Connection.html#M000158 -* Deprecated :auto_reconnect option on connection; if the driver fails to - connect, it will automatically try to reconnect on the subsequent operation. - See http://www.mongodb.org/display/DOCS/Replica+Pairs+in+Ruby -* Added Collection#map_reduce helper (Christos Trochalakis) -* Deprecated DB#db_command in favor of DB#command. -* Removed deprecated old sort options, :offset, and Connection#clear. -* Lots of internal code restructuring for better maintainability. - -0.17.1 2009-11-17 -* Index ordering fix -* Notice to install mongo_ext - -0.17 2009-11-16 -* Performance improvements - * large document inserts twice as fast as 0.16 - * queries 18% faster than 0.16 on average - * see benchmark comparison: http://gist.github.com/236062 -* Support for multi-update for Mongo >= 1.1.3 (See Collection#update) -* Collection#distinct -* Connection#copy_database (voodootikigod) -* C optimizations for ByteBuffer#to_s and ObjectID#generate (seancribbs) -* Continue code restructuring for performance and simplicity. +## Prior to 0.20 + +See git revisions. From 7ffdc9c42e110f899bebe4a6d39b032f99c65a65 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 3 Nov 2010 17:50:08 -0400 Subject: [PATCH 010/442] minor: links to internal docs --- source/TUTORIAL.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index 1acad01d2..fa164c4b2 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -1,4 +1,4 @@ -# Tutorial +# MongoDB Ruby Driver Tutorial This tutorial gives many common examples of using MongoDB with the Ruby driver. If you're looking for information on data modeling, see [MongoDB Data Modeling and Rails](http://www.mongodb.org/display/DOCS/MongoDB+Data+Modeling+and+Rails). Links to the various object mappers are listed on our [object mappers page](http://www.mongodb.org/display/DOCS/Object+Mappers+for+Ruby+and+MongoDB). @@ -52,7 +52,6 @@ If you're trying to connect to a replica set, see [Replica Sets in Ruby](http:// #### Dropping a Database connection.drop_database('database_name') - #### Authentication (Optional) MongoDB can be run in a secure mode where access to databases is controlled through name and password authentication. When run in this mode, any client application must provide a name and password before doing any operations. In the Ruby driver, you simply do the following with the connected mongo object: From 6da9d5e189c37d50e2168c1ee485ff0bdcaf1250 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 3 Nov 2010 18:07:50 -0400 Subject: [PATCH 011/442] minor: markdown fix --- source/CREDITS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/CREDITS.md b/source/CREDITS.md index 673457fe3..ac5dcd266 100644 --- a/source/CREDITS.md +++ b/source/CREDITS.md @@ -52,7 +52,7 @@ Jack Chen, chendo on github Michael Bernstein, mrb on github -* #sort method for Cursor instances +* Cursor#sort Paulo Ahahgon, pahagon on github From 2ae040973a04e8513a87bcf92f8ea2e58500c631 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Thu, 4 Nov 2010 17:50:48 -0400 Subject: [PATCH 012/442] BUMP 1.1.2 and HISTORY --- source/HISTORY.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index be1ce7acf..3bed67e0d 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,14 @@ # MongoDB Ruby Driver History +### 1.1.2 +2010-11-4 + +* Two critical fixes to automated failover and replica sets. +* Bug passing :timeout to Cursor. +* Permit safe mode specification on Connection, Collection, and DB levels. +* Specify replica set name on connect to verify connection to the right set. +* Misc. reorganization of project and docs. + ### 1.1.1 2010-10-14 From 656df362cd1d614bdfe1418709d218ccdc3c2f87 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Fri, 12 Nov 2010 17:59:27 -0500 Subject: [PATCH 013/442] RUBY-200 migrating confluence docs to /docs --- source/FAQ.md | 112 +++++++++++++++++++++++++++++ source/GridFS.md | 158 +++++++++++++++++++++++++++++++++++++++++ source/REPLICA_SETS.md | 73 +++++++++++++++++++ 3 files changed, 343 insertions(+) create mode 100644 source/FAQ.md create mode 100644 source/GridFS.md create mode 100644 source/REPLICA_SETS.md diff --git a/source/FAQ.md b/source/FAQ.md new file mode 100644 index 000000000..761e2706d --- /dev/null +++ b/source/FAQ.md @@ -0,0 +1,112 @@ +# Ruby MongoDB FAQ + +This is a list of frequently asked questions about using Ruby with MongoDB. If you have a question you'd like to have answered here, please post your question to the [mongodb-user list](http://groups.google.com/group/mongodb-user). + +#### Can I run (insert command name here) from the Ruby driver? + +Yes. You can run any of the [available database commands|List of Database Commands] from the driver using the DB#command method. The only trick is to use an OrderedHash when specifying the command. For example, here's how you'd run an asynchronous fsync from the driver: + + + # This command is run on the admin database. + @db = Mongo::Connection.new.db('admin') + + # Build the command. + cmd = OrderedHash.new + cmd['fsync'] = 1 + cmd['async'] = true + + # Run it. + @db.command(cmd) + + +It's important to keep in mind that some commands, like `fsync`, must be run on the `admin` database, while other commands can be run on any database. If you're having trouble, check the [command reference|List of Database Commands] to make sure you're using the command correctly. + +#### Does the Ruby driver support an EXPLAIN command? + +Yes. `explain` is, technically speaking, an option sent to a query that tells MongoDB to return an explain plan rather than the query's results. You can use `explain` by constructing a query and calling explain at the end: + + + @collection = @db['users'] + result = @collection.find({:name => "jones"}).explain + + +The resulting explain plan might look something like this: + + + {"cursor"=>"BtreeCursor name_1", + "startKey"=>{"name"=>"Jones"}, + "endKey"=>{"name"=>"Jones"}, + "nscanned"=>1.0, + "n"=>1, + "millis"=>0, + "oldPlan"=>{"cursor"=>"BtreeCursor name_1", + "startKey"=>{"name"=>"Jones"}, + "endKey"=>{"name"=>"Jones"} + }, + "allPlans"=>[{"cursor"=>"BtreeCursor name_1", + "startKey"=>{"name"=>"Jones"}, + "endKey"=>{"name"=>"Jones"`] + } + + +Because this collection has an index on the "name" field, the query uses that index, only having to scan a single record. "n" is the number of records the query will return. "millis" is the time the query takes, in milliseconds. "oldPlan" indicates that the query optimizer has already seen this kind of query and has, therefore, saved an efficient query plan. "allPlans" shows all the plans considered for this query. + +#### I see that BSON supports a symbol type. Does this mean that I can store Ruby symbols in MongoDB? + +You can store Ruby symbols in MongoDB, but only as values. BSON specifies that document keys must be strings. So, for instance, you can do this: + + + @collection = @db['test'] + + boat_id = @collection.save({:vehicle => :boat}) + car_id = @collection.save({"vehicle" => "car"}) + + @collection.find_one('_id' => boat_id) + {"_id" => ObjectID('4bb372a8238d3b5c8c000001'), "vehicle" => :boat} + + + @collection.find_one('_id' => car_id) + {"_id" => ObjectID('4bb372a8238d3b5c8c000002'), "vehicle" => "car"} + + +Notice that the symbol values are returned as expected, but that symbol keys are treated as strings. + +#### Why can't I access random elements within a cursor? + +MongoDB cursors are designed for sequentially iterating over a result set, and all the drivers, including the Ruby driver, stick closely to this directive. Internally, a Ruby cursor fetches results in batches by running a MongoDB `getmore` operation. The results are buffered for efficient iteration on the client-side. + +What this means is that a cursor is nothing more than a device for returning a result set on a query that's been initiated on the server. Cursors are not containers for result sets. If we allow a cursor to be randomly accessed, then we run into issues regarding the freshness of the data. For instance, if I iterate over a cursor and then want to retrieve the cursor's first element, should a stored copy be returned, or should the cursor re-run the query? If we returned a stored copy, it may not be fresh. And if the the query is re-run, then we're technically dealing with a new cursor. + +To avoid those issues, we're saying that anyone who needs flexible access to the results of a query should store those results in an array and then access the data as needed. + +#### Why can't I save an instance of TimeWithZone? + +MongoDB stores times in UTC as the number of milliseconds since the epoch. This means that the Ruby driver serializes Ruby Time objects only. While it would certainly be possible to serialize a TimeWithZone, this isn't preferable since the driver would still deserialize to a Time object. + +All that said, if necessary, it'd be easy to write a thin wrapper over the driver that would store an extra time zone attribute and handle the serialization/deserialization of TimeWithZone transparently. + +#### I keep getting CURSOR_NOT_FOUND exceptions. What's happening? + +The most likely culprit here is that the cursor is timing out on the server. Whenever you issue a query, a cursor is created on the server. Cursor naturally time out after ten minutes, which means that if you happen to be iterating over a cursor for more than ten minutes, you risk a CURSOR_NOT_FOUND exception. + +There are two solutions to this problem. You can either: + +1. Limit your query. Use some combination of `limit` and `skip` to reduce the total number of query results. This will, obviously, bring down the time it takes to iterate. + +2. Turn off the cursor timeout. To do that, invoke `find` with a block, and pass `:timeout => true`: + + @collection.find({}, :timeout => false) do |cursor| + cursor.each do |document + # Process documents here + end + end + +#### I periodically see connection failures between the driver and MongoDB. Why can't the driver retry the operation automatically? + +A connection failure can indicate any number of failure scenarios. Has the server crashed? Are we experiencing a temporary network partition? Is there a bug in our ssh tunnel? + +Without further investigation, it's impossible to know exactly what has caused the connection failure. Furthermore, when we do see a connection failure, it's impossible to know how many operations prior to the failure succeeded. Imagine, for instance, that we're using safe mode and we send an `$inc` operation to the server. It's entirely possible that the server has received the `$inc` but failed on the call to `getLastError`. In that case, retrying the operation would result in a double-increment. + +Because of the indeterminacy involved, the MongoDB drivers will not retry operations on connection failure. How connection failures should be handled is entirely dependent on the application. Therefore, we leave it to the application developers to make the best decision in this case. + +The drivers will reconnect on the subsequent operation. diff --git a/source/GridFS.md b/source/GridFS.md new file mode 100644 index 000000000..2790f8e74 --- /dev/null +++ b/source/GridFS.md @@ -0,0 +1,158 @@ +# GridFS in Ruby + +GridFS, which stands for "Grid File Store," is a specification for storing large files in MongoDB. It works by dividing a file into manageable chunks and storing each of those chunks as a separate document. GridFS requires two collections to achieve this: one collection stores each file's metadata (e.g., name, size, etc.) and another stores the chunks themselves. If you're interested in more details, check out the [GridFS Specification](http://www.mongodb.org/display/DOCS/GridFS+Specification). + +### The Grid class + +The [Grid class](http://api.mongodb.org/ruby/current/Mongo/Grid.html) represents the core GridFS implementation. Grid gives you a simple file store, keyed on a unique ID. This means that duplicate filenames aren't a problem. To use the Grid class, first make sure you have a database, and then instantiate a Grid: + + + @db = Mongo::Connection.new.db('social_site') + @grid = Grid.new(@db) + +#### Saving files +Once you have a Grid object, you can start saving data to it. The data can be either a string or an IO-like object that responds to a #read method: + + + # Saving string data + id = @grid.put("here's some string / binary data") + + # Saving IO data and including the optional filename + image = File.open("me.jpg") + id2 = @grid.put(image, :filename => "me.jpg") + + +Grid#put returns an object id, which you can use to retrieve the file: + + + # Get the string we saved + file = @grid.get(id) + + # Get the file we saved + image = @grid.get(id2) + + +#### File metadata + +There are accessors for the various file attributes: + + + image.filename + # => "me.jpg" + + image.content_type + # => "image/jpg" + + image.file_length + # => 502357 + + image.upload_date + # => Mon Mar 01 16:18:30 UTC 2010 + + # Read all the image's data at once + image.read + + # Read the first 100k bytes of the image + image.read(100 * 1024) + + +When putting a file, you can set many of these attributes and write arbitrary metadata: + + + # Saving IO data + file = File.open("me.jpg") + id2 = @grid.put(file, + :filename => "my-avatar.jpg" + :content_type => "application/jpg", + :_id => 'a-unique-id-to-use-in-lieu-of-a-random-one', + :chunk_size => 100 * 1024, + :metadata => {'description' => "taken after a game of ultimate"}) + + +#### Safe mode + +A kind of safe mode is built into the GridFS specification. When you save a file, and MD5 hash is created on the server. If you save the file in safe mode, an MD5 will be created on the client for comparison with the server version. If the two hashes don't match, an exception will be raised. + + + image = File.open("me.jpg") + id2 = @grid.put(image, "my-avatar.jpg", :safe => true) + + +#### Deleting files + +Deleting a file is as simple as providing the id: + + + @grid.delete(id2) + + +### The GridFileSystem class + +[GridFileSystem](http://api.mongodb.org/ruby/current/Mongo/GridFileSystem.html) is a light emulation of a file system and therefore has a couple of unique properties. The first is that filenames are assumed to be unique. The second, a consequence of the first, is that files are versioned. To see what this means, let's create a GridFileSystem instance: + +#### Saving files + + @db = Mongo::Connection.new.db("social_site") + @fs = GridFileSystem.new(@db) + +Now suppose we want to save the file 'me.jpg.' This is easily done using a filesystem-like API: + + + image = File.open("me.jpg") + @fs.open("me.jpg", "w") do |f| + f.write image + end + + +We can then retrieve the file by filename: + + + image = @fs.open("me.jpg", "r") {|f| f.read } + + +No problems there. But what if we need to replace the file? That too is straightforward: + + + image = File.open("me-dancing.jpg") + @fs.open("me.jpg", "w") do |f| + f.write image + end + + +But a couple things need to be kept in mind. First is that the original 'me.jpg' will be available until the new 'me.jpg' saves. From then on, calls to the #open method will always return the most recently saved version of a file. But, and this the second point, old versions of the file won't be deleted. So if you're going to be rewriting files often, you could end up with a lot of old versions piling up. One solution to this is to use the :delete_old options when writing a file: + + + image = File.open("me-dancing.jpg") + @fs.open("me.jpg", "w", :delete_old => true) do |f| + f.write image + end + + +This will delete all but the latest version of the file. + + +#### Deleting files + +When you delete a file by name, you delete all versions of that file: + + + @fs.delete("me.jpg") + + +#### Metadata and safe mode + +All of the options for storing metadata and saving in safe mode are available for the GridFileSystem class: + + + image = File.open("me.jpg") + @fs.open('my-avatar.jpg', w, + :content_type => "application/jpg", + :metadata => {'description' => "taken on 3/1/2010 after a game of ultimate"}, + :_id => 'a-unique-id-to-use-instead-of-the-automatically-generated-one', + :safe => true) { |f| f.write image } + + +### Advanced Users + +Astute code readers will notice that the Grid and GridFileSystem classes are merely thin wrappers around an underlying [GridIO class](http://api.mongodb.org/ruby/current/Mongo/GridIO.html). This means that it's easy to customize the GridFS implementation presented here; just use GridIO for all the low-level work, and build the API you need in an external manager class similar to Grid or GridFileSystem. + diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md new file mode 100644 index 000000000..48fea7566 --- /dev/null +++ b/source/REPLICA_SETS.md @@ -0,0 +1,73 @@ +# Replica Sets in Ruby + +Here follow a few considerations for those using the MongoDB Ruby driver with [replica sets](http://www.mongodb.org/display/DOCS/Replica+Sets). + +### Setup + +First, make sure that you've configured and initialized a replica set. + +Connecting to a replica set from the Ruby driver is easy. If you only want to specify a single node, simply pass that node to `Connection.new`: + + @connection = Connection.new('foo.local', 27017) + +If you want to pass in multiple seed nodes, use `Connection.multi`: + + @connection = Connection.multi([['n1.mydb.net', 27017], + ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]]) + +In both cases, the driver will attempt to connect to a master node and, when found, will merge any other known members of the replica set into the seed list. + +### Connection Failures + +Imagine that our master node goes offline. How will the driver respond? + +At first, the driver will try to send operations to what was the master node. These operations will fail, and the driver will raise a *ConnectionFailure* exception. It then becomes the client's responsibility to decide how to handle this. + +If the client decides to retry, it's not guaranteed that another member of the replica set will have been promoted to master right away, so it's still possible that the driver will raise another *ConnectionFailure*. However, once a member has been promoted to master, typically within a few seconds, subsequent operations will succeed. + +The driver will essentially cycle through all known seed addresses until a node identifies itself as master. + +### Recovery + +Driver users may wish to wrap their database calls with failure recovery code. Here's one possibility: + + # Ensure retry upon failure + def rescue_connection_failure(max_retries=5) + success = false + retries = 0 + while !success + begin + yield + success = true + rescue Mongo::ConnectionFailure => ex + retries += 1 + raise ex if retries >= max_retries + sleep(1) + end + end + end + end + + # Wrapping a call to #count() + rescue_connection_failure do + @db.collection('users').count() + end + +Of course, the proper way to handle connection failures will always depend on the individual application. We encourage object-mapper and application developers to publish any promising results. + +### Testing + +The Ruby driver (>= 1.0.6) includes some unit tests for verifying replica set behavior. They reside in *tests/replica_sets*. You can run them individually with the following rake tasks: + + rake test:replica_set_count + rake test:replica_set_insert + rake test:pooled_replica_set_insert + rake test:replica_set_query + +Make sure you have a replica set running on localhost before trying to run these tests. + +### Further Reading + +* [Replica Sets](http://www.mongodb.org/display/DOCS/Replica+Set+Configuration) +* [Replics Set Configuration](http://www.mongodb.org/display/DOCS/Replica+Set+Configuration) + From 03f12e71bd08ea6711066a415c0010b720ca99ea Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 24 Nov 2010 13:49:34 -0500 Subject: [PATCH 014/442] Cleanup for distributed reads on replica sets. --- source/examples/replica_set.rb | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 source/examples/replica_set.rb diff --git a/source/examples/replica_set.rb b/source/examples/replica_set.rb new file mode 100644 index 000000000..6ddfd1acf --- /dev/null +++ b/source/examples/replica_set.rb @@ -0,0 +1,24 @@ +# This code assumes a running replica set with at least on node at localhost:27017. +require 'mongo' + +cons = [] + +10.times do + cons << Mongo::Connection.multi([['localhost', 27017]], :read_secondary => true) +end + +ports = cons.map do |con| + con.read_pool.port +end + +puts "These ten connections will read from the following ports:" +p ports + +cons[rand(10)]['foo']['bar'].remove +100.times do |n| + cons[rand(10)]['foo']['bar'].insert({:a => n}) +end + +100.times do |n| + p cons[rand(10)]['foo']['bar'].find_one({:a => n}) +end From 617493304f11cf3ed5efc76a1db406eed10aede4 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 24 Nov 2010 14:01:39 -0500 Subject: [PATCH 015/442] RUBY-174 doc updates for reads from secondaries --- source/REPLICA_SETS.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 48fea7566..dda6ef57c 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -6,16 +6,20 @@ Here follow a few considerations for those using the MongoDB Ruby driver with [r First, make sure that you've configured and initialized a replica set. -Connecting to a replica set from the Ruby driver is easy. If you only want to specify a single node, simply pass that node to `Connection.new`: +Use `Connection.multi` to connect to a replica set: - @connection = Connection.new('foo.local', 27017) + @connection = Connection.multi([['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]]) -If you want to pass in multiple seed nodes, use `Connection.multi`: +The driver will attempt to connect to a master node and, when found, will replace all seed nodes with known members of the replica set. - @connection = Connection.multi([['n1.mydb.net', 27017], - ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]]) +### Read slaves -In both cases, the driver will attempt to connect to a master node and, when found, will merge any other known members of the replica set into the seed list. +If you want to read from a seconday node, you can pass :read_secondary => true. + + @connection = Connection.multi([['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]], + :read_secondary => true) + +A random secondary will be chosen to be read from. In a typical multi-process Ruby application, you'll have a good distribution of reads across secondary nodes. ### Connection Failures From 160a791ff67039f6107749ed323803bb5e460c0e Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 29 Nov 2010 13:56:11 -0500 Subject: [PATCH 016/442] minor: test fix --- source/examples/replica_set.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/examples/replica_set.rb b/source/examples/replica_set.rb index 6ddfd1acf..deb027325 100644 --- a/source/examples/replica_set.rb +++ b/source/examples/replica_set.rb @@ -1,4 +1,4 @@ -# This code assumes a running replica set with at least on node at localhost:27017. +# This code assumes a running replica set with at least one node at localhost:27017. require 'mongo' cons = [] From ddee8e07f75aabea2e1fc57cc19899cadc5555eb Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 29 Nov 2010 14:26:26 -0500 Subject: [PATCH 017/442] History. RS Doc updates. Write concern docs. --- source/HISTORY.md | 12 ++++++++++++ source/REPLICA_SETS.md | 14 +++++++------- source/WRITE_CONCERN.md | 28 ++++++++++++++++++++++++++++ 3 files changed, 47 insertions(+), 7 deletions(-) create mode 100644 source/WRITE_CONCERN.md diff --git a/source/HISTORY.md b/source/HISTORY.md index 3bed67e0d..5e404a6c5 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,17 @@ # MongoDB Ruby Driver History +### 1.1.3 +2010-11-29 + +* Distributed reads for replica set secondaries. See /docs/examples/replica_set.rb and + http://api.mongodb.org/ruby/current/file.REPLICA_SETS.html for details. +* Note: when connecting to a replica set, you must use Connection#multi. +* Cursor#count takes optional skip and limit +* Collection#ensure_index for caching index creation calls +* Collection#update and Collection#remove now return error object when using safe mode +* Important fix for int/long serialization on bug introduced in 1.0.9 +* Numerous tweaks and bug fixes. + ### 1.1.2 2010-11-4 diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index dda6ef57c..4b26a0bb2 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -14,7 +14,7 @@ The driver will attempt to connect to a master node and, when found, will replac ### Read slaves -If you want to read from a seconday node, you can pass :read_secondary => true. +If you want to read from a seconday node, you can pass :read_secondary => true to Connection#multi. @connection = Connection.multi([['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]], :read_secondary => true) @@ -23,9 +23,9 @@ A random secondary will be chosen to be read from. In a typical multi-process Ru ### Connection Failures -Imagine that our master node goes offline. How will the driver respond? +Imagine that either the master node or one of the read nodes goes offline. How will the driver respond? -At first, the driver will try to send operations to what was the master node. These operations will fail, and the driver will raise a *ConnectionFailure* exception. It then becomes the client's responsibility to decide how to handle this. +If any read operation fails, the driver will raise a *ConnectionFailure* exception. It then becomes the client's responsibility to decide how to handle this. If the client decides to retry, it's not guaranteed that another member of the replica set will have been promoted to master right away, so it's still possible that the driver will raise another *ConnectionFailure*. However, once a member has been promoted to master, typically within a few seconds, subsequent operations will succeed. @@ -33,10 +33,11 @@ The driver will essentially cycle through all known seed addresses until a node ### Recovery -Driver users may wish to wrap their database calls with failure recovery code. Here's one possibility: +Driver users may wish to wrap their database calls with failure recovery code. Here's one possibility, which will attempt to connection +every half second and time out after thirty seconds. # Ensure retry upon failure - def rescue_connection_failure(max_retries=5) + def rescue_connection_failure(max_retries=60) success = false retries = 0 while !success @@ -46,7 +47,7 @@ Driver users may wish to wrap their database calls with failure recovery code. H rescue Mongo::ConnectionFailure => ex retries += 1 raise ex if retries >= max_retries - sleep(1) + sleep(0.5) end end end @@ -74,4 +75,3 @@ Make sure you have a replica set running on localhost before trying to run these * [Replica Sets](http://www.mongodb.org/display/DOCS/Replica+Set+Configuration) * [Replics Set Configuration](http://www.mongodb.org/display/DOCS/Replica+Set+Configuration) - diff --git a/source/WRITE_CONCERN.md b/source/WRITE_CONCERN.md new file mode 100644 index 000000000..b68ba28c8 --- /dev/null +++ b/source/WRITE_CONCERN.md @@ -0,0 +1,28 @@ +# Write Concern in Ruby + +## Setting the write concern + +Write concern is set using the `:safe` option. There are several possible options: + + @collection.save({:doc => 'foo'}, :safe => true) + @collection.save({:doc => 'foo'}, :safe => {:w => 2}) + @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200}) + @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200, :fsync => true}) + +The first, `true`, simply indicates that we should request a response from the server to ensure that to errors have occurred. The second, `{:w => 2}`forces the server to wait until at least two servers have recorded the write. The third does the same but will time out if the replication can't be completed in 200 milliseconds. The fourth forces an fsync on each server being written to (note: this option is rarely necessary and will have a dramaticly negative effect on performance). + +## Write concern inheritance + +The Ruby driver allows you to set write concern on each of four levels: the connection, database, collection, and write operation. +Objects will inherit the default write concern from their parents. Thus, if you set a write concern of `{:w => 1}` when creating +a new connection, then all databases and collections created from that connection will inherit the same setting. See this code example: + + @con = Mongo::Connection.new('localhost', 27017, :safe => {:w => 2}) + @db = @con['test'] + @collection = @db['foo'] + @collection.save({:name => 'foo'}) + + @collection.save({:name => 'bar'}, :safe => false) + +Here, the first call to Collection#save will use the inherited write concern, `{:w => 2}`. But notice that the second call +to Collection#save overrides this setting. From 2e4101b4a0e80d7419126cbfd38fbe167d3bbfd1 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 29 Nov 2010 17:04:39 -0500 Subject: [PATCH 018/442] minor: relative paths --- source/GridFS.md | 2 +- source/TUTORIAL.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/source/GridFS.md b/source/GridFS.md index 2790f8e74..03b3eb104 100644 --- a/source/GridFS.md +++ b/source/GridFS.md @@ -4,7 +4,7 @@ GridFS, which stands for "Grid File Store," is a specification for storing large ### The Grid class -The [Grid class](http://api.mongodb.org/ruby/current/Mongo/Grid.html) represents the core GridFS implementation. Grid gives you a simple file store, keyed on a unique ID. This means that duplicate filenames aren't a problem. To use the Grid class, first make sure you have a database, and then instantiate a Grid: +The [Grid class](Mongo/Grid.html) represents the core GridFS implementation. Grid gives you a simple file store, keyed on a unique ID. This means that duplicate filenames aren't a problem. To use the Grid class, first make sure you have a database, and then instantiate a Grid: @db = Mongo::Connection.new.db('social_site') diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index fa164c4b2..4151b9caf 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -2,7 +2,7 @@ This tutorial gives many common examples of using MongoDB with the Ruby driver. If you're looking for information on data modeling, see [MongoDB Data Modeling and Rails](http://www.mongodb.org/display/DOCS/MongoDB+Data+Modeling+and+Rails). Links to the various object mappers are listed on our [object mappers page](http://www.mongodb.org/display/DOCS/Object+Mappers+for+Ruby+and+MongoDB). -Interested in GridFS? See [GridFS in Ruby](http://www.mongodb.org/display/DOCS/GridFS+in+Ruby). +Interested in GridFS? See [GridFS in Ruby](file.GridFS.html). As always, the [latest source for the Ruby driver](http://github.com/mongodb/mongo-ruby-driver) can be found on [github](http://github.com/mongodb/mongo-ruby-driver/). From d2ded59009961c6b6ce0228007769d92ee048360 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 30 Nov 2010 12:50:28 -0500 Subject: [PATCH 019/442] BUMP 1.1.4 --- source/HISTORY.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 5e404a6c5..08975207a 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,11 @@ # MongoDB Ruby Driver History +### 1.1.4 +2010-11-30 + +* Important connection failure fix. +* ObjectId#to_s optimization (David Cuadrado). + ### 1.1.3 2010-11-29 From 3f1548a23daccc17bcc2d4cae5768979901bfc36 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 15 Dec 2010 14:14:06 -0500 Subject: [PATCH 020/442] Updated ReplSetConnection docs --- source/REPLICA_SETS.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 4b26a0bb2..572267660 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -6,17 +6,17 @@ Here follow a few considerations for those using the MongoDB Ruby driver with [r First, make sure that you've configured and initialized a replica set. -Use `Connection.multi` to connect to a replica set: +Use `ReplSetConnection.new` to connect to a replica set: - @connection = Connection.multi([['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]]) + @connection = ReplSetConnection.new(['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]) The driver will attempt to connect to a master node and, when found, will replace all seed nodes with known members of the replica set. ### Read slaves -If you want to read from a seconday node, you can pass :read_secondary => true to Connection#multi. +If you want to read from a seconday node, you can pass :read_secondary => true to ReplSetConnection#new. - @connection = Connection.multi([['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]], + @connection = ReplSetConnection.new(['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017], :read_secondary => true) A random secondary will be chosen to be read from. In a typical multi-process Ruby application, you'll have a good distribution of reads across secondary nodes. @@ -62,14 +62,12 @@ Of course, the proper way to handle connection failures will always depend on th ### Testing -The Ruby driver (>= 1.0.6) includes some unit tests for verifying replica set behavior. They reside in *tests/replica_sets*. You can run them individually with the following rake tasks: +The Ruby driver (>= 1.1.5) includes unit tests for verifying replica set behavior. They reside in *tests/replica_sets*. You can run them as a group with the following rake task: - rake test:replica_set_count - rake test:replica_set_insert - rake test:pooled_replica_set_insert - rake test:replica_set_query + rake test:rs -Make sure you have a replica set running on localhost before trying to run these tests. +The suite will set up a five-node replica set by itself and ensure that driver behaves correctly even in the face +of individual node failures. Node that the `mongod` executable must be in the search path for this to work. ### Further Reading From fafb38e0ba234a344bfc117bccba06ff5ef9f8c3 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 15 Dec 2010 15:29:36 -0500 Subject: [PATCH 021/442] 1.1.5 history and credits. --- source/CREDITS.md | 4 ++++ source/HISTORY.md | 9 +++++++++ 2 files changed, 13 insertions(+) diff --git a/source/CREDITS.md b/source/CREDITS.md index ac5dcd266..0d4dde5cc 100644 --- a/source/CREDITS.md +++ b/source/CREDITS.md @@ -117,3 +117,7 @@ Hongli Lai (Phusion) Mislav Marohnić * Replaced returning with each_with_object + +Alex Stupka + +* Replica set port bug diff --git a/source/HISTORY.md b/source/HISTORY.md index 08975207a..a88a65aa5 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,14 @@ # MongoDB Ruby Driver History +### 1.1.5 +2010-12-15 + +* ReplSetConnection class. This must be used for replica set connections from + now on. You can still use Connection.multi, but that method has been deprecated. +* Automated replica set tests. rake test:rs +* Check that request and response ids match. +* Several bug fixes. See the commit history for details. + ### 1.1.4 2010-11-30 From d21f457f8bc4dee15eba7808ebf9195d128e26ca Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 29 Dec 2010 13:13:54 -0500 Subject: [PATCH 022/442] Use retry instead of while loop for rescuing connection failure (quasor) --- source/REPLICA_SETS.md | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 572267660..12a11b9b8 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -38,18 +38,14 @@ every half second and time out after thirty seconds. # Ensure retry upon failure def rescue_connection_failure(max_retries=60) - success = false - retries = 0 - while !success - begin - yield - success = true - rescue Mongo::ConnectionFailure => ex - retries += 1 - raise ex if retries >= max_retries - sleep(0.5) - end - end + retries = 0 + begin + yield + rescue Mongo::ConnectionFailure => ex + retries += 1 + raise ex if retries > max_retries + sleep(0.5) + retry end end From 5f137cceccad806011c11ea8d666bd7973f152c7 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 5 Jan 2011 12:13:35 -0500 Subject: [PATCH 023/442] BUMP 1.2.rc0 and HISTORY --- source/HISTORY.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index a88a65aa5..b2406f0b1 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,13 @@ # MongoDB Ruby Driver History +1.2.rc0 +2011-1-5 + +Lots of cleanp and minor bug fixes. +* Issues resolved: http://jira.mongodb.org/browse/RUBY/fixforversion/10222 +* Updated Java BSON to Java driver 2.4. +* Platform gem for JRuby bson. + ### 1.1.5 2010-12-15 From d092bdb5ade6a5ccf0db757320858ba831169048 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 18 Jan 2011 11:20:59 -0500 Subject: [PATCH 024/442] BUMP 1.2.0 --- source/HISTORY.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index b2406f0b1..04ad7bf43 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,10 @@ # MongoDB Ruby Driver History +1.2.0 +2011-1-18 +* Some minor improvements. See commit history. +* Since nothing major was reported for the RC, we're releasing. + 1.2.rc0 2011-1-5 From 466d66eacae99c36982e1f83c594ae1c1a77135e Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 18 Jan 2011 11:36:04 -0500 Subject: [PATCH 025/442] minor: HISTORY cleanup --- source/HISTORY.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/source/HISTORY.md b/source/HISTORY.md index 04ad7bf43..7a569c482 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,14 +1,14 @@ # MongoDB Ruby Driver History -1.2.0 +### 1.2.0 2011-1-18 + * Some minor improvements. See commit history. -* Since nothing major was reported for the RC, we're releasing. -1.2.rc0 +### 1.2.rc0 2011-1-5 -Lots of cleanp and minor bug fixes. +Lots of cleanup and minor bug fixes. * Issues resolved: http://jira.mongodb.org/browse/RUBY/fixforversion/10222 * Updated Java BSON to Java driver 2.4. * Platform gem for JRuby bson. From 5abe3868c67125e3931d9d4d516f1e56f7b1677a Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Thu, 10 Feb 2011 15:13:28 -0500 Subject: [PATCH 026/442] minor: docs --- source/FAQ.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/FAQ.md b/source/FAQ.md index 761e2706d..d96b1784a 100644 --- a/source/FAQ.md +++ b/source/FAQ.md @@ -110,3 +110,7 @@ Without further investigation, it's impossible to know exactly what has caused t Because of the indeterminacy involved, the MongoDB drivers will not retry operations on connection failure. How connection failures should be handled is entirely dependent on the application. Therefore, we leave it to the application developers to make the best decision in this case. The drivers will reconnect on the subsequent operation. + +#### I ocassionally get an error saying that responses are out of order. What's happening? + +See (this JIRA issue)[http://jira.mongodb.org/browse/RUBY-221]. From aac635f16492e016a15de500bbab9dc5b03a8dde Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Thu, 10 Feb 2011 15:27:06 -0500 Subject: [PATCH 027/442] BUMP 1.2.1 --- source/HISTORY.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 7a569c482..05fba0876 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,12 @@ # MongoDB Ruby Driver History +### 1.2.1 +2011-1-18 + +* Enable authentication with connection pooling. +* Allow custom logging with Connection#instrument (CodeMonkeySteve) +* Minor fixes and doc improvements. + ### 1.2.0 2011-1-18 From 6ebd49cc4562cdfa4ce8987a46383d04e0cf0d72 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Thu, 10 Feb 2011 17:17:18 -0500 Subject: [PATCH 028/442] minor: doc fix --- source/REPLICA_SETS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 12a11b9b8..dd5df9b64 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -63,7 +63,7 @@ The Ruby driver (>= 1.1.5) includes unit tests for verifying replica set behavio rake test:rs The suite will set up a five-node replica set by itself and ensure that driver behaves correctly even in the face -of individual node failures. Node that the `mongod` executable must be in the search path for this to work. +of individual node failures. Note that the `mongod` executable must be in the search path for this to work. ### Further Reading From 59e6eabe85aada55152146744b07c20f17737e34 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 15 Feb 2011 17:09:52 -0500 Subject: [PATCH 029/442] BUMP 1.2.2 --- source/HISTORY.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 05fba0876..39b2804ed 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,11 @@ # MongoDB Ruby Driver History +### 1.2.2 +2011-2-15 + +* Improved replica set failover for edge case. +* Fix for REE on OSX (Hongli Lai) + ### 1.2.1 2011-1-18 From ca06a162f54e04e21e19e1628e72442cab405f10 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 23 Feb 2011 12:43:00 -0500 Subject: [PATCH 030/442] BUMP 1.2.3 --- source/HISTORY.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 39b2804ed..4a1fb5735 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,11 @@ # MongoDB Ruby Driver History +### 1.2.3 +2011-2-22 + +* Update docs for map-reduce command +* Minor doc fix + ### 1.2.2 2011-2-15 From 10e4a5240a3a7686694e4ab24bb4df1bffc3638b Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 23 Feb 2011 14:51:49 -0500 Subject: [PATCH 031/442] BUMP 1.2.4 --- source/HISTORY.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 4a1fb5735..fed781585 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,13 @@ # MongoDB Ruby Driver History +### 1.2.4 +2011-2-23 + +* Fix the exception message shown when there's an IOError (malditogeek) +* Another update to map-reduce docs for v1.8. Note that if you use the new + output option {:out => {:inline => true}}, then you must also specify + :raw => true. + ### 1.2.3 2011-2-22 From ec35ea0b32a978ca9fa279fe47c32b37cf439e2f Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Sat, 12 Mar 2011 08:40:29 -0500 Subject: [PATCH 032/442] RUBY-247 Improved replica set docs. --- source/REPLICA_SETS.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index dd5df9b64..4818edd5c 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -6,12 +6,13 @@ Here follow a few considerations for those using the MongoDB Ruby driver with [r First, make sure that you've configured and initialized a replica set. -Use `ReplSetConnection.new` to connect to a replica set: +Use `ReplSetConnection.new` to connect to a replica set. This method, which accepts a variable number of arugments, +takes a list of seed nodes followed by any connection options. You'll want to specify at least two seed nodes. This gives +the driver more chances to connect in the event that any one seed node is offline. Once the driver connects, it will +cache the replica set topology as reported by the given seed node and use that information if a failover is later required. @connection = ReplSetConnection.new(['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]) -The driver will attempt to connect to a master node and, when found, will replace all seed nodes with known members of the replica set. - ### Read slaves If you want to read from a seconday node, you can pass :read_secondary => true to ReplSetConnection#new. From 1e066516c33346ab5ed18dec54e17b5532bdffde Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 23 Mar 2011 17:11:21 -0400 Subject: [PATCH 033/442] RUBY-237 RUBY-238 Enforce semantic versioning --- source/RELEASES.md | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 source/RELEASES.md diff --git a/source/RELEASES.md b/source/RELEASES.md new file mode 100644 index 000000000..9c2b6cf3b --- /dev/null +++ b/source/RELEASES.md @@ -0,0 +1,33 @@ +# MongoDB Ruby Driver Release Plan + +This is a description of a formalized release plan that will take effect +with version 1.3.0. + +## Semantic versioning + +The most significant difference is that releases will now adhere to the conventions of +[semantic versioning](http://semver.org). In particular, we will strictly abide by the +following release rules: + +1. Patch versions of the driver (Z in x.y.Z) will be released only when backward-compatible bug fixes are introduced. A bug fix is defined as an internal change that fixes incorrect behavior. + +2. Minor versions (Y in x.Y.z) will be released if new, backward-compatible functionality is introduced to the public API. + +3. Major versions (X in X.y.z) will be incremented if any backward-incompatibl changes are introduced to the public API. + +This policy will clearly indicate to users when an upgrade may affect their code. As a side effect, version numbers will climb more quickly than before. + + +## Release checklist + +Before each relese to Rubygems.org, the following steps will be taken: + +1. All driver tests will be run on Linux, OS X, and Windows via continuous integration system. + +2. HISTORY file will document all significant commits. + +3. Version number will be incremented per the semantic version spec described above. + +4. Appropriate branches and tags will be created in Git repository, as necessary. + +5. Docs will be updated to the latest version of the driver and posted [online](http://api.mongodb.org/ruby/current/index.html). From 75ad7a45ce11b4a15df8a4ef6cd8f18b043db578 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 29 Mar 2011 15:52:38 -0400 Subject: [PATCH 034/442] minor: release notes --- source/HISTORY.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index fed781585..e53a34af1 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,23 @@ # MongoDB Ruby Driver History +### 1.3.0.rc0 + +* Add option to set timeouts on socket read calls. Use + the :op_timeout option when creating a new connection. +* Add StringIO methods to GridIO objects +* Support for BSON timestamp type with BSON::Timestamp +* Change the BSON binary subtype from 2 to 0 +* Remove private method Connection#reset_conection + and deprecate public method ReplSetConnection#reset_connection +* ByteBuffer#== and OrderedHash#dup (FooBarWidget) +* Better check for UTF8 validity in Ruby 1.9 +* Added previously removed Connection#host and Connection#port +* Added transformers to allow Mongo::Cursor to allow instantiated objects (jnunemaker) +* Automated reconnection on fork +* Added Cursor#next alias for Cursor#next_document +* Audit tests after enabling warnings (wpiekutowski) +* Various bug fixes thanks to FooBarWidget, Datanoise, Malitogeek + ### 1.2.4 2011-2-23 From d0acf38141d061e7229463b0dd66f91c5001de67 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 29 Mar 2011 16:34:53 -0400 Subject: [PATCH 035/442] BUMP 1.3.0.rc0 --- source/HISTORY.md | 1 + 1 file changed, 1 insertion(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index e53a34af1..d4c960b86 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,6 +1,7 @@ # MongoDB Ruby Driver History ### 1.3.0.rc0 +2011-3-29 * Add option to set timeouts on socket read calls. Use the :op_timeout option when creating a new connection. From a3e91d025d5092be5c9a78c0bd3d22e01148bec4 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Fri, 1 Apr 2011 12:31:33 -0400 Subject: [PATCH 036/442] minor: history --- source/HISTORY.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/HISTORY.md b/source/HISTORY.md index d4c960b86..b5c677b6e 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -10,19 +10,19 @@ * Change the BSON binary subtype from 2 to 0 * Remove private method Connection#reset_conection and deprecate public method ReplSetConnection#reset_connection -* ByteBuffer#== and OrderedHash#dup (FooBarWidget) +* ByteBuffer#== and OrderedHash#dup (Hongli Lai) * Better check for UTF8 validity in Ruby 1.9 * Added previously removed Connection#host and Connection#port * Added transformers to allow Mongo::Cursor to allow instantiated objects (jnunemaker) * Automated reconnection on fork * Added Cursor#next alias for Cursor#next_document * Audit tests after enabling warnings (wpiekutowski) -* Various bug fixes thanks to FooBarWidget, Datanoise, Malitogeek +* Various bug fixes thanks to Datanoise, Hongli Lai, and Mauro Pompilio ### 1.2.4 2011-2-23 -* Fix the exception message shown when there's an IOError (malditogeek) +* Fix the exception message shown when there's an IOError (Mauro Pompilio) * Another update to map-reduce docs for v1.8. Note that if you use the new output option {:out => {:inline => true}}, then you must also specify :raw => true. From 9341139b6d4bb87b9041296eb2ebe173cbf4819d Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Fri, 1 Apr 2011 13:29:25 -0400 Subject: [PATCH 037/442] minor: removed old 1.0 upgrade notes --- source/1.0_UPGRADE.md | 21 --------------------- 1 file changed, 21 deletions(-) delete mode 100644 source/1.0_UPGRADE.md diff --git a/source/1.0_UPGRADE.md b/source/1.0_UPGRADE.md deleted file mode 100644 index 86e294d0a..000000000 --- a/source/1.0_UPGRADE.md +++ /dev/null @@ -1,21 +0,0 @@ -You can upgrade freely from v0.20 to v1.0. - -However, if you're running a version < 0.20, upgrade to 0.20 -before upgrading to 1.0. - -The upgrade to 0.20 requires some minor code upgrades. - -1. Note the exception changes in HISTORY. Certain exceptions are now scoped under the BSON -module; if you're catching these, you will need to modify your code. - -2. The BSON types are now scoped under the BSON module. - -3. Note that mongo_ext no longer exists. The new gems are bson and bson_ext. - -4. Indexes on GridFS chunks collections should be unique. If you have existing GridFS -collections, you should drop the current index and replace with a unique one. Before you do this, -make sure that index doesn't exist; no need to go through process unnecessarily. -If you do need to create the index, once you have the chunks collection, here are the commands you can run: - - @chunks.drop_index('files_id_1_n_1') - @chunks.create_index([['files_id', Mongo::ASCENDING], ['n', Mongo::ASCENDING]], :unique => true) From e328812e46c8ea91efcd0ea7f1aa0e997b2517f2 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Fri, 1 Apr 2011 13:53:12 -0400 Subject: [PATCH 038/442] minor: history --- source/HISTORY.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/source/HISTORY.md b/source/HISTORY.md index b5c677b6e..0e7a28436 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,10 +1,10 @@ # MongoDB Ruby Driver History -### 1.3.0.rc0 -2011-3-29 +### 1.3.0 +2011-4-04 -* Add option to set timeouts on socket read calls. Use - the :op_timeout option when creating a new connection. +* Add option to set timeouts on socket read calls using the + Mongo::Connection :op_timeout option. * Add StringIO methods to GridIO objects * Support for BSON timestamp type with BSON::Timestamp * Change the BSON binary subtype from 2 to 0 @@ -13,10 +13,10 @@ * ByteBuffer#== and OrderedHash#dup (Hongli Lai) * Better check for UTF8 validity in Ruby 1.9 * Added previously removed Connection#host and Connection#port -* Added transformers to allow Mongo::Cursor to allow instantiated objects (jnunemaker) +* Added transformers to allow Mongo::Cursor to allow instantiated objects (John Nunemaker) * Automated reconnection on fork * Added Cursor#next alias for Cursor#next_document -* Audit tests after enabling warnings (wpiekutowski) +* Audit tests after enabling warnings (Wojciech Piekutowski) * Various bug fixes thanks to Datanoise, Hongli Lai, and Mauro Pompilio ### 1.2.4 From 42a74646ed177f53d147b5fff06b3c3f2064f9d6 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 27 Apr 2011 11:29:57 -0400 Subject: [PATCH 039/442] minor: fix for query examples --- source/examples/queries.rb | 42 +++++++++++++++++++++----------------- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/source/examples/queries.rb b/source/examples/queries.rb index 9707e477a..8e33c9db4 100644 --- a/source/examples/queries.rb +++ b/source/examples/queries.rb @@ -19,52 +19,56 @@ coll.insert('a' => 1) coll.insert('a' => 2) coll.insert('b' => 3) +coll.insert('c' => 'foo') +coll.insert('c' => 'bar') # Count. -puts "There are #{coll.count()} records." +puts "There are #{coll.count} records." # Find all records. find() returns a Cursor. -cursor = coll.find() +puts "Find all records:" +pp cursor = coll.find.to_a # Print them. Note that all records have an _id automatically added by the # database. See pk.rb for an example of how to use a primary key factory to # generate your own values for _id. -cursor.each { |row| pp row } - -# Cursor has a to_a method that slurps all records into memory. -rows = coll.find().to_a -rows.each { |row| pp row } +puts "Print each document individually:" +pp cursor.each { |row| pp row } # See Collection#find. From now on in this file, we won't be printing the # records we find. -coll.find('a' => 1) +puts "Find one record:" +pp coll.find('a' => 1).to_a # Find records sort by 'a', skip 1, limit 2 records. # Sort can be single name, array, or hash. -coll.find({}, {:skip => 1, :limit => 2, :sort => 'a'}) +puts "Skip 1, limit 2, sort by 'a':" +pp coll.find({}, {:skip => 1, :limit => 2, :sort => 'a'}).to_a # Find all records with 'a' > 1. There is also $lt, $gte, and $lte. coll.find({'a' => {'$gt' => 1}}) coll.find({'a' => {'$gt' => 1, '$lte' => 3}}) # Find all records with 'a' in a set of values. -coll.find('a' => {'$in' => [1,2]}) +puts "Find all records where a is $in [1, 2]:" +pp coll.find('a' => {'$in' => [1,2]}).to_a -# Find by regexp -coll.find('a' => /[1|2]/) +puts "Find by regex:" +pp coll.find({'c' => /f/}).to_a # Print query explanation -pp coll.find('a' => /[1|2]/).explain() +puts "Print an explain:" +pp coll.find({'c' => /f/}).explain # Use a hint with a query. Need an index. Hints can be stored with the # collection, in which case they will be used with all queries, or they can be # specified per query, in which case that hint overrides the hint associated # with the collection if any. -coll.create_index('a') -coll.hint = 'a' +coll.create_index('c') +coll.hint = 'c' -# You will see a different explanation now that the hint is in place -pp coll.find('a' => /[1|2]/).explain() +puts "Print an explain with index:" +pp coll.find('c' => /[f|b]/).explain -# Override hint for single query -coll.find({'a' => 1}, :hint => 'b') +puts "Print an explain with natural order hint:" +pp coll.find({'c' => /[f|b]/}, :hint => '$natural').explain From d8e37522a2ba0a1e5c333a7119834913725679d2 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 10 May 2011 16:04:16 -0400 Subject: [PATCH 040/442] 1.3.1 Release notes --- source/HISTORY.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 0e7a28436..1e655656c 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,22 @@ # MongoDB Ruby Driver History +### 1.3.1 +2011-5-10 + +* Fix GridIO#gets infinite loop error (Ryan McGeary) +* Fix BSON::OrderedHash#reject! leaving keys with null values (rpt. by Ben Poweski) +* Minor semantic fix for OrderedHash#reject! +* Fix Mongo::DB to allow symbols in method traversing collection names (rpt. by Chris Griego) +* Support new server regex option "s" (dotall). This is folded in with \m in Ruby. +* Fix so that Cursor#close hits the right node when :read_secondary is enabled. +* Support maxScan, showDiskLoc, and returnKey cursor options. +* Make DB#validate_collection compatible with server v1.9.1. +* Fix so that GridIO#gets returns local md5 with md5 matches server md5 (Steve Tantra). +* Fix bug in BSON::OrderedHash that prevents YAML.load (Ian Warshak). +* Fix example from /examples. +* Ensure that we do not modify hash arguments by calling Hash#dup when appropriate. +* Minor doc fixes. + ### 1.3.0 2011-4-04 From abde8ef3d417a408b225b6d2c6413c5239ebc49b Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 11 May 2011 12:06:43 -0400 Subject: [PATCH 041/442] minor: history --- source/HISTORY.md | 1 + 1 file changed, 1 insertion(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 1e655656c..ae0ad81af 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -15,6 +15,7 @@ * Fix bug in BSON::OrderedHash that prevents YAML.load (Ian Warshak). * Fix example from /examples. * Ensure that we do not modify hash arguments by calling Hash#dup when appropriate. +* Ensure that JRuby deserializer preserves binary subtypes properly. * Minor doc fixes. ### 1.3.0 From 52d51990abee9e665abdb94e4462491d9e63ccd9 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 11 May 2011 12:10:23 -0400 Subject: [PATCH 042/442] minor: history --- source/HISTORY.md | 1 + 1 file changed, 1 insertion(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index ae0ad81af..7dedf348e 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -16,6 +16,7 @@ * Fix example from /examples. * Ensure that we do not modify hash arguments by calling Hash#dup when appropriate. * Ensure that JRuby deserializer preserves binary subtypes properly. +* Fix for streaming an empty file into GridFS (Daniël van de Burgt). * Minor doc fixes. ### 1.3.0 From 5c2684d87590710b3659c16f5ef8002e95201524 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 15 Jun 2011 16:17:42 -0400 Subject: [PATCH 043/442] minor: Deprecate :timeout for :pool_timeout --- source/RELEASES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/RELEASES.md b/source/RELEASES.md index 9c2b6cf3b..71453fae9 100644 --- a/source/RELEASES.md +++ b/source/RELEASES.md @@ -13,7 +13,7 @@ following release rules: 2. Minor versions (Y in x.Y.z) will be released if new, backward-compatible functionality is introduced to the public API. -3. Major versions (X in X.y.z) will be incremented if any backward-incompatibl changes are introduced to the public API. +3. Major versions (X in X.y.z) will be incremented if any backward-incompatible changes are introduced to the public API. This policy will clearly indicate to users when an upgrade may affect their code. As a side effect, version numbers will climb more quickly than before. From 50e4a387c998f515aa956fcefe051d3de006ab02 Mon Sep 17 00:00:00 2001 From: Nathan Ehresman Date: Fri, 24 Jun 2011 08:55:03 -0400 Subject: [PATCH 044/442] minor documentation typo fix --- source/TUTORIAL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index 4151b9caf..2d7051c7e 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -110,7 +110,7 @@ We can update the previous document using the `update` method. There are a coupl Or we can use an atomic operator to change a single value: - coll.update({"_id" => doc["_id"]}, {"$set" => {"name" => "MongoDB Ruby"`) + coll.update({"_id" => doc["_id"]}, {"$set" => {"name" => "MongoDB Ruby"}}) Read [more about updating documents|Updating]. From 256a786630d6fb2cb6e1e1d3b0bad75a9253e639 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Fri, 5 Aug 2011 11:52:45 -0400 Subject: [PATCH 045/442] RUBY-283 add_option for cursors. Deprecate Cursor#query_opts. --- source/HISTORY.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 7dedf348e..c4fa2d2e3 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,10 @@ # MongoDB Ruby Driver History +### 1.4.0 +UNRELEASED + +* Add Cursor#add_option. Deprecate Cursor#query_opts and replace with Cursor#options. + ### 1.3.1 2011-5-10 From b35b83f7a56d9bd446dff9e720f025df28f38356 Mon Sep 17 00:00:00 2001 From: Laurent Arnoud Date: Wed, 24 Aug 2011 23:15:17 +0200 Subject: [PATCH 046/442] Fix some TUTORIAL markdown. * Escape _ for bson_ext. * Fix insert example. * Un-escape _id, not required with backtick quotes. Signed-off-by: Laurent Arnoud --- source/TUTORIAL.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index 2d7051c7e..a589e2ba7 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -15,7 +15,7 @@ Next, install the mongo rubygem: The required `bson` gem will be installed automatically. -For optimum performance, install the bson_ext gem: +For optimum performance, install the bson\_ext gem: gem install bson_ext @@ -98,7 +98,7 @@ Once you have the collection object, you can insert documents into the collectio Notice that the above has an "inner" document embedded within it. To do this, we can use a Hash or the driver's OrderedHash (which preserves key order) to create the document (including the inner document), and then just simply insert it into the collection using the `insert()` method. doc = {"name" => "MongoDB", "type" => "database", "count" => 1, - "info" => {"x" => 203, "y" => '102'` + "info" => {"x" => 203, "y" => '102'}} coll.insert(doc) #### Updating a Document @@ -126,7 +126,7 @@ and you should see: {"_id"=>#, "name"=>"MongoDB", "info"=>{"x"=>203, "y"=>102}, "type"=>"database", "count"=>1} -Note the `\_id` element has been added automatically by MongoDB to your document. +Note the `_id` element has been added automatically by MongoDB to your document. #### Adding Multiple Documents From ec150171354f9568ee674b6543a77944af242aa5 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 6 Sep 2011 15:01:53 -0400 Subject: [PATCH 047/442] minor: doc fix --- source/WRITE_CONCERN.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/source/WRITE_CONCERN.md b/source/WRITE_CONCERN.md index b68ba28c8..b65835630 100644 --- a/source/WRITE_CONCERN.md +++ b/source/WRITE_CONCERN.md @@ -7,9 +7,12 @@ Write concern is set using the `:safe` option. There are several possible option @collection.save({:doc => 'foo'}, :safe => true) @collection.save({:doc => 'foo'}, :safe => {:w => 2}) @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200}) - @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200, :fsync => true}) + @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200, :j => true}) -The first, `true`, simply indicates that we should request a response from the server to ensure that to errors have occurred. The second, `{:w => 2}`forces the server to wait until at least two servers have recorded the write. The third does the same but will time out if the replication can't be completed in 200 milliseconds. The fourth forces an fsync on each server being written to (note: this option is rarely necessary and will have a dramaticly negative effect on performance). +The first, `true`, simply indicates that we should request a response from the server to ensure that to errors have occurred. The second, `{:w => 2}`, forces the server to wait until at least two servers have recorded the write. The third does the same but will time out if the replication can't be completed in 200 milliseconds. +Setting a value for `wtimeout` is encouraed. + +Finally, the fourth example forces the journal to sync to disk if journaling is enabled. ## Write concern inheritance From b621daadd48a372638b61d0f5ca9fb0e40cc0235 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 6 Sep 2011 15:23:51 -0400 Subject: [PATCH 048/442] RUBY-284 RUBY-291 document read preference with secondary ping time --- source/READ_PREFERENCE.md | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 source/READ_PREFERENCE.md diff --git a/source/READ_PREFERENCE.md b/source/READ_PREFERENCE.md new file mode 100644 index 000000000..eedb7c3c1 --- /dev/null +++ b/source/READ_PREFERENCE.md @@ -0,0 +1,39 @@ +# Read Preference in Ruby + +## Setting the read preference + +You can using the `:read` option to specify a query's read preference. There are for now two possible options: + + @collection.find({:doc => 'foo'}, :read => :primary) + @collection.find({:doc => 'foo'}, :read => :secondary) + +In the first case, the query will be directed to the primary node in a replica set. In the second, the query will be sent +to a secondary node. The driver will attempt to choose a secondary node that's nearby, as determined by ping time. If more +than one secondary node is closeby (e.g, responds to pings within 10ms), then a random node within this subset will be chosen. + +## Read preference inheritance + +The Ruby driver allows you to set read preference on each of four levels: the connection, database, collection, and cursor (or read operation). +Objects will inherit the default read preference from their parents. Thus, if you set a read preference of `{:read => :secondary}` when creating +a new connection, then all databases and collections created from that connection will inherit the same setting. See this code example: + + @con = Mongo::ReplSetConnection.new([['localhost', 27017], ['localhost', 27018]], :read => :secondary) + @db = @con['test'] + @collection = @db['foo'] + @collection.find({:name => 'foo'}) + + @collection.find({:name => 'bar'}, :read => :primary) + +Here, the first call to Collection#find will use the inherited read preference, `{:read => :secondary}`. But the second call +to Collection#find overrides this setting by setting the preference to `:primary`. + +You can examine the read preference on any object by calling its `read_preference` method: + + @con.read_preference + @db.read_preference + @collection.read_preference + +## Future work + +In the v2.0 release of the driver, you'll also be able to specify a read preference consisting of a set of tags. This way, +you'll be able to direct reads to a replica set member. You can follow this issue's progress here: (https://jira.mongodb.org/browse/RUBY-326). From 8147415156975b9b7bc2a9a9b9c15f08fd99428f Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 6 Sep 2011 16:33:36 -0400 Subject: [PATCH 049/442] RUBY-295 document tailable cursor API. --- source/TAILABLE_CURSORS.md | 51 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 source/TAILABLE_CURSORS.md diff --git a/source/TAILABLE_CURSORS.md b/source/TAILABLE_CURSORS.md new file mode 100644 index 000000000..033e59536 --- /dev/null +++ b/source/TAILABLE_CURSORS.md @@ -0,0 +1,51 @@ +# Tailable in Ruby + +Tailable cursors are cursors that remain open even after they've returned +a final result. This way, if more documents are added to a collection (i.e., +to the cursor's result set), then you can continue to call `Cursor#next` to +retrieve those results. Here's a complete test case that demonstrates the use +of tailable cursors. + +Note that tailable cursors are for capped collections only. + + require 'mongo' + require 'test/unit' + + class TestTailable < Test::Unit::TestCase + include Mongo + + def test_tailable + + # Create a connection and capped collection. + @con = Connection.new + @db = @con['test'] + @db.drop_collection('log') + @capped = @db.create_collection('log', :capped => true, :size => 1024) + + # Insert 10 documents. + 10.times do |n| + @capped.insert({:n => n}) + end + + # Create a tailable cursor that iterates the collection in natural order + @tail = Cursor.new(@capped, :tailable => true, :order => [['$natural', 1]]) + + # Call Cursor#next 10 times. Each call returns a document. + 10.times do + assert @tail.next + end + + # But the 11th time, the cursor returns nothing. + assert_nil @tail.next + + # Add a document to the capped collection. + @capped.insert({:n => 100}) + + # Now call Cursor#next again. This will return the just-inserted result. + assert @tail.next + + # Close the cursor. + @tail.close + end + + end From e15d496963d7318b546fa3e0e8ff6ef9d4013b12 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 6 Sep 2011 16:36:45 -0400 Subject: [PATCH 050/442] minor: doc fixes --- source/HISTORY.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/HISTORY.md b/source/HISTORY.md index c4fa2d2e3..8cec37ca0 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -48,8 +48,8 @@ UNRELEASED * Fix the exception message shown when there's an IOError (Mauro Pompilio) * Another update to map-reduce docs for v1.8. Note that if you use the new - output option {:out => {:inline => true}}, then you must also specify - :raw => true. + output option `{:out => {:inline => true}}`, then you must also specify + `:raw => true`. ### 1.2.3 2011-2-22 From 92213841fe24cfd2adc2b2f3a45728f917637250 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Thu, 15 Sep 2011 18:44:02 -0400 Subject: [PATCH 051/442] minor fixes and doc updates. :sync is default refresh mode. --- source/REPLICA_SETS.md | 25 +++++++++++++++++++++++-- source/TAILABLE_CURSORS.md | 2 +- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 4818edd5c..01771f973 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -15,10 +15,10 @@ cache the replica set topology as reported by the given seed node and use that i ### Read slaves -If you want to read from a seconday node, you can pass :read_secondary => true to ReplSetConnection#new. +If you want to read from a secondary node, you can pass :read => :secondary to ReplSetConnection#new. @connection = ReplSetConnection.new(['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017], - :read_secondary => true) + :read => :secondary) A random secondary will be chosen to be read from. In a typical multi-process Ruby application, you'll have a good distribution of reads across secondary nodes. @@ -32,6 +32,27 @@ If the client decides to retry, it's not guaranteed that another member of the r The driver will essentially cycle through all known seed addresses until a node identifies itself as master. +### Refresh mode + +You can now specify a refresh mode and refresh interval for a replica set connection. This will help to ensure that +changes to a replica set's configuration are quickly reflected on the driver side. Refresh mode is +enabled in synchronous mode by default. Here's how to specify this explicitly: + + @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :sync) + +If you want to refresh to happen via a background thread, use the `:async` mode: + + @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :async) + +If you want to change the default refresh interval of 90 seconds, you can do so like this: + + @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :async, + :refresh_interval => 60) + +You can also disable refresh mode altogether: + + @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => false) + ### Recovery Driver users may wish to wrap their database calls with failure recovery code. Here's one possibility, which will attempt to connection diff --git a/source/TAILABLE_CURSORS.md b/source/TAILABLE_CURSORS.md index 033e59536..d526ddfff 100644 --- a/source/TAILABLE_CURSORS.md +++ b/source/TAILABLE_CURSORS.md @@ -1,4 +1,4 @@ -# Tailable in Ruby +# Tailable cursors in Ruby Tailable cursors are cursors that remain open even after they've returned a final result. This way, if more documents are added to a collection (i.e., From 4604fd068e5e5f6b63f1b65bc8bba52ca21d0e5a Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 19 Sep 2011 10:05:08 -0400 Subject: [PATCH 052/442] minor: update HISTORY --- source/HISTORY.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 8cec37ca0..49556b1cc 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -3,7 +3,19 @@ ### 1.4.0 UNRELEASED +* Attempt to automatically refresh internal replica set state using ReplSetConnection#refresh. +* Two automated refresh modes: :async and :sync. Automated refresh can also be disabled. +* Choose secondary for reads based on ping time. +* Read preference API: specify whether queries should go to primary or secondary on a per-query basis. +* Enable exhaust-mode queries with OP_QUERY_EXHAUST. +* Collection#count takes a query selector. +* Support continue_on_error flag for bulk inserts (use :continue_on_error => true) * Add Cursor#add_option. Deprecate Cursor#query_opts and replace with Cursor#options. +* Initial SSL support (connect with :ssl => true) +* Update to latest Java driver for JRuby. +* Check max BSON size on a per-connection basis. +* Fixed two platform-specific BSON serialization issues. +* Lots of bug fixes and code cleanup. ### 1.3.1 2011-5-10 From 0cf95c7c9d05ad751d98f6d76aff8c1b2908d957 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 19 Sep 2011 10:13:43 -0400 Subject: [PATCH 053/442] minor: update HISTORY --- source/HISTORY.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/HISTORY.md b/source/HISTORY.md index 49556b1cc..708ef6343 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,12 +1,13 @@ # MongoDB Ruby Driver History ### 1.4.0 -UNRELEASED +2011-9-19 * Attempt to automatically refresh internal replica set state using ReplSetConnection#refresh. * Two automated refresh modes: :async and :sync. Automated refresh can also be disabled. * Choose secondary for reads based on ping time. * Read preference API: specify whether queries should go to primary or secondary on a per-query basis. +* Pass :require_primary => false to ReplSetConnection to connect without requiring a primary node. * Enable exhaust-mode queries with OP_QUERY_EXHAUST. * Collection#count takes a query selector. * Support continue_on_error flag for bulk inserts (use :continue_on_error => true) From d717266e28e007c52b4f0f3c19cccb64772dca96 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Thu, 13 Oct 2011 12:06:39 -0400 Subject: [PATCH 054/442] minor: docs --- source/REPLICA_SETS.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 01771f973..14a20dc1b 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -40,7 +40,8 @@ enabled in synchronous mode by default. Here's how to specify this explicitly: @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :sync) -If you want to refresh to happen via a background thread, use the `:async` mode: +If you want to refresh to happen via a background thread, use the `:async` mode. NOTE: the background +version may be more effective on platforms that use native threads, such as JRuby: @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :async) @@ -49,6 +50,8 @@ If you want to change the default refresh interval of 90 seconds, you can do so @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :async, :refresh_interval => 60) +Do not set this value to anything lower than 30, or you may start to experience performance issues. + You can also disable refresh mode altogether: @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => false) From 976c00165cd7c56055568f9c9609694d32989418 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Sat, 15 Oct 2011 23:53:55 -0400 Subject: [PATCH 055/442] minor: HISTORY --- source/HISTORY.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 708ef6343..661e28163 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,11 @@ # MongoDB Ruby Driver History +### 1.4.1 +2011-10-15 + +* Simplified replica set refresh. +* Fix bugs associated with replica set refresh. + ### 1.4.0 2011-9-19 From 0d9def2228810ef296fc8fd845cb27f69414cc84 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 17 Oct 2011 14:53:14 -0500 Subject: [PATCH 056/442] minor: docs --- source/REPLICA_SETS.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 14a20dc1b..994655dc9 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -28,19 +28,24 @@ Imagine that either the master node or one of the read nodes goes offline. How w If any read operation fails, the driver will raise a *ConnectionFailure* exception. It then becomes the client's responsibility to decide how to handle this. -If the client decides to retry, it's not guaranteed that another member of the replica set will have been promoted to master right away, so it's still possible that the driver will raise another *ConnectionFailure*. However, once a member has been promoted to master, typically within a few seconds, subsequent operations will succeed. +If the client decides to retry, it's not guaranteed that another member of the replica set will have been promoted to master right away, so it's still possible that the driver will raise another *ConnectionFailure*. However, once a member has been promoted to master, typically within a few seconds, subsequent operations will succeed. *Note that this does not prevent +exception in the event of a primary failover.* The driver will essentially cycle through all known seed addresses until a node identifies itself as master. ### Refresh mode You can now specify a refresh mode and refresh interval for a replica set connection. This will help to ensure that -changes to a replica set's configuration are quickly reflected on the driver side. Refresh mode is -enabled in synchronous mode by default. Here's how to specify this explicitly: +changes to a replica set's configuration are quickly reflected on the driver side. In particular, if you change +the state of any secondary node, the automated refresh will ensure that this state is recorded on the client side. +If you add a secondary that responds to pings much faster than the existing nodes, then the new secondary will +be used for reads. + +Refresh mode is enabled in synchronous mode by default. This is the recommended setting, but here's how to specify this explicitly: @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :sync) -If you want to refresh to happen via a background thread, use the `:async` mode. NOTE: the background +If you want to refresh via a background thread, use the `:async` mode. NOTE: the background version may be more effective on platforms that use native threads, such as JRuby: @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :async) @@ -56,6 +61,10 @@ You can also disable refresh mode altogether: @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => false) +And you can call `refresh` manually on any replica set connection: + + @connection.refresh + ### Recovery Driver users may wish to wrap their database calls with failure recovery code. Here's one possibility, which will attempt to connection From 8c05221d1bf868d8f218fd797aa7f87e31accb23 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 17 Oct 2011 14:54:25 -0500 Subject: [PATCH 057/442] minor: HISTORY --- source/HISTORY.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/source/HISTORY.md b/source/HISTORY.md index 661e28163..11ceb85c2 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,10 +1,14 @@ # MongoDB Ruby Driver History ### 1.4.1 -2011-10-15 +2011-10-17 + +If you're using 1.4.0, this is a necessary upgrade. * Simplified replica set refresh. * Fix bugs associated with replica set refresh. +* Make cursor smart enough to continue functioning +even if a refresh is triggered. ### 1.4.0 2011-9-19 From 770fc250df45d79ac6d597469f00bce7a5d592d9 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 26 Oct 2011 10:26:50 -0400 Subject: [PATCH 058/442] Auto-refresh will be disabled by default. --- source/REPLICA_SETS.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 994655dc9..cd490555e 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -41,7 +41,11 @@ the state of any secondary node, the automated refresh will ensure that this sta If you add a secondary that responds to pings much faster than the existing nodes, then the new secondary will be used for reads. -Refresh mode is enabled in synchronous mode by default. This is the recommended setting, but here's how to specify this explicitly: +Refresh mode is disabled by default. + +However, if you expect to make live changes to your secondaries, and you want this to be reflected without +having to manually restart your app server, then you should enable it. You can enable synchronously, which will +refresh the replica set data in a synchronous fashion (which may ocassionally slow down your queries): @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :sync) From f33484f9f7d115863d56c47da0fa382db8740bb1 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Fri, 18 Nov 2011 16:13:19 -0500 Subject: [PATCH 059/442] RUBY-367 deprecate async refresh --- source/REPLICA_SETS.md | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index cd490555e..6ea0bf8ec 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -44,19 +44,15 @@ be used for reads. Refresh mode is disabled by default. However, if you expect to make live changes to your secondaries, and you want this to be reflected without -having to manually restart your app server, then you should enable it. You can enable synchronously, which will -refresh the replica set data in a synchronous fashion (which may ocassionally slow down your queries): +having to manually restart your app server, then you should enable it. You can enable this mode +synchronously, which will refresh the replica set data in a synchronous fashion (which may +ocassionally slow down your queries): @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :sync) -If you want to refresh via a background thread, use the `:async` mode. NOTE: the background -version may be more effective on platforms that use native threads, such as JRuby: - - @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :async) - If you want to change the default refresh interval of 90 seconds, you can do so like this: - @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :async, + @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :sync, :refresh_interval => 60) Do not set this value to anything lower than 30, or you may start to experience performance issues. From a98f794c506aed839fd4177a332d10d5d936820c Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Fri, 18 Nov 2011 17:15:03 -0500 Subject: [PATCH 060/442] BUMP 1.5.0.rc0 --- source/HISTORY.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 11ceb85c2..38c8f1523 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,10 @@ # MongoDB Ruby Driver History +### 1.5.0.rc0 +2011-11-18 + +Fix bugs associated with replica set refresh. + ### 1.4.1 2011-10-17 From a22952bfe19c63cdea7db5441189177e94664f9b Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Mon, 28 Nov 2011 14:09:27 -0500 Subject: [PATCH 061/442] minor: HISTORY --- source/HISTORY.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 38c8f1523..1ccce800b 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,24 @@ # MongoDB Ruby Driver History +### 1.5.0 +2011-11-28 + +This releases fixes bugs introduced in 1.4.0 and 1.4.1 that +were introduced as a result of adding replica set refresh modes. + +* Removed :async refresh mode. +* Disabled auto refresh mode by default. If you want the driver +to automatically check the state of the replica set, you must +use :sync mode. Note that replica set refresh is designed only to +account for benign changes to the replica set (adding and removing +nodes that don't affect current connections). +* Fixed bug with commands being sent to secondary nodes. The next +release will allow you to specify where commands can be sent. +* Support :j safe mode option. +* Fix :max_scan and :show_disk_loc Cursor options. + +You can see the remaining issues at https://jira.mongodb.org/secure/ReleaseNote.jspa?projectId=10005&version=10992 + ### 1.5.0.rc0 2011-11-18 From 344125ebd5130bb7c9489d5f266f7c14b7832b02 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 29 Nov 2011 15:54:08 -0500 Subject: [PATCH 062/442] BUMP 1.5.1 --- source/HISTORY.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 1ccce800b..f47df6e8f 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,12 @@ # MongoDB Ruby Driver History +### 1.5.1 +2011-11-29 + +Release due to corrupted gemspec. This was a bug having +to do with rubygems. Apparently, gems must still be +built with Ruby 1.8. + ### 1.5.0 2011-11-28 From 87f0d9ea6790711b3d676a868ffef8caabc26215 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Tue, 13 Dec 2011 15:35:37 -0500 Subject: [PATCH 063/442] BUMP 1.5.2 --- source/HISTORY.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index f47df6e8f..6e6e0b6e1 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,12 @@ # MongoDB Ruby Driver History +### 1.5.2 +2011-12-13 + +* Lots of fixes for replica set connection edge cases. +* Set default op_timeout and connect_timeout to 30 seconds. +* Support GeoHaystack indexing. + ### 1.5.1 2011-11-29 From 33ed01ae03954e928eb63df4fdbf8400cd6562d9 Mon Sep 17 00:00:00 2001 From: Benjamin Atkin Date: Sun, 12 Feb 2012 20:11:11 -0700 Subject: [PATCH 064/442] typo in docs/WRITE_CONCERN.md --- source/WRITE_CONCERN.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/WRITE_CONCERN.md b/source/WRITE_CONCERN.md index b65835630..c88dddc3b 100644 --- a/source/WRITE_CONCERN.md +++ b/source/WRITE_CONCERN.md @@ -10,7 +10,7 @@ Write concern is set using the `:safe` option. There are several possible option @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200, :j => true}) The first, `true`, simply indicates that we should request a response from the server to ensure that to errors have occurred. The second, `{:w => 2}`, forces the server to wait until at least two servers have recorded the write. The third does the same but will time out if the replication can't be completed in 200 milliseconds. -Setting a value for `wtimeout` is encouraed. +Setting a value for `wtimeout` is encouraged. Finally, the fourth example forces the journal to sync to disk if journaling is enabled. From e0d8a947e4f3da8f586353a2e941b0415e81b360 Mon Sep 17 00:00:00 2001 From: Kyle Banker Date: Wed, 15 Feb 2012 13:14:03 -0500 Subject: [PATCH 065/442] minor: document release process in detail. --- source/RELEASES.md | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/source/RELEASES.md b/source/RELEASES.md index 71453fae9..2d7c25f67 100644 --- a/source/RELEASES.md +++ b/source/RELEASES.md @@ -24,10 +24,24 @@ Before each relese to Rubygems.org, the following steps will be taken: 1. All driver tests will be run on Linux, OS X, and Windows via continuous integration system. -2. HISTORY file will document all significant commits. +2. Update the HISTORY file and document all significant commits. -3. Version number will be incremented per the semantic version spec described above. +3. Update the version in lib/bson.rb, lib/mongo/version.rb, and ext/version.h. -4. Appropriate branches and tags will be created in Git repository, as necessary. +4. Commit: "Release [VERSION]" -5. Docs will be updated to the latest version of the driver and posted [online](http://api.mongodb.org/ruby/current/index.html). +5. git tag [version] + +6. Build gems. Ensure that they have the correct versions. + +7. Push tags and commit to GitHub (git push origin master, git push --tags). + +8. Build and push docs. + +9. Push gems to Rubygems.org. + +10. Test that the gem is downloadable from Rubygems.org. + +11. Close out release in JIRA. + +12. Annouce release on mongodb-user and mongodb-dev. From 3ea206905b6675377aec9c3946da5d2766bff348 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Thu, 16 Feb 2012 14:20:31 -0500 Subject: [PATCH 066/442] minor: documentation and test updates for RUBY-378 --- source/READ_PREFERENCE.md | 2 +- source/REPLICA_SETS.md | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/source/READ_PREFERENCE.md b/source/READ_PREFERENCE.md index eedb7c3c1..7d77a7bc7 100644 --- a/source/READ_PREFERENCE.md +++ b/source/READ_PREFERENCE.md @@ -17,7 +17,7 @@ The Ruby driver allows you to set read preference on each of four levels: the co Objects will inherit the default read preference from their parents. Thus, if you set a read preference of `{:read => :secondary}` when creating a new connection, then all databases and collections created from that connection will inherit the same setting. See this code example: - @con = Mongo::ReplSetConnection.new([['localhost', 27017], ['localhost', 27018]], :read => :secondary) + @con = Mongo::ReplSetConnection.new(['localhost:27017','localhost:27018'], :read => :secondary) @db = @con['test'] @collection = @db['foo'] @collection.find({:name => 'foo'}) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 6ea0bf8ec..c6a1ca30f 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -11,13 +11,13 @@ takes a list of seed nodes followed by any connection options. You'll want to sp the driver more chances to connect in the event that any one seed node is offline. Once the driver connects, it will cache the replica set topology as reported by the given seed node and use that information if a failover is later required. - @connection = ReplSetConnection.new(['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017]) + @connection = ReplSetConnection.new(['n1.mydb.net:27017', 'n2.mydb.net:27017', 'n3.mydb.net:27017']) ### Read slaves If you want to read from a secondary node, you can pass :read => :secondary to ReplSetConnection#new. - @connection = ReplSetConnection.new(['n1.mydb.net', 27017], ['n2.mydb.net', 27017], ['n3.mydb.net', 27017], + @connection = ReplSetConnection.new(['n1.mydb.net:27017', 'n2.mydb.net:27017', 'n3.mydb.net:27017'], :read => :secondary) A random secondary will be chosen to be read from. In a typical multi-process Ruby application, you'll have a good distribution of reads across secondary nodes. @@ -48,18 +48,18 @@ having to manually restart your app server, then you should enable it. You can e synchronously, which will refresh the replica set data in a synchronous fashion (which may ocassionally slow down your queries): - @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :sync) + @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :refresh_mode => :sync) If you want to change the default refresh interval of 90 seconds, you can do so like this: - @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => :sync, + @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :refresh_mode => :sync, :refresh_interval => 60) Do not set this value to anything lower than 30, or you may start to experience performance issues. You can also disable refresh mode altogether: - @connection = ReplSetConnection.new(['n1.mydb.net', 27017], :refresh_mode => false) + @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :refresh_mode => false) And you can call `refresh` manually on any replica set connection: From 895cfc821daef9498880ffce9421731a79734299 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Wed, 22 Feb 2012 10:41:44 -0500 Subject: [PATCH 067/442] Release 1.6.0 --- source/HISTORY.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 6e6e0b6e1..1a3808be2 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,20 @@ # MongoDB Ruby Driver History +### 1.6.0 +2012-02-22 + +* Added Gemfile +* ReplSetConnection seed format is now array of 'host:port' strings +* Added read preference :secondary_only +* Added ability to log duration -- enabled by default (Cyril Mougel) +* Added read_only option for DB#add_user (Ariel Salomon) +* Added :collect_on_error option for bulk-insert (Masahiro Nakagawa) +* Added and updated URI options (now case insensitive) +* Bug fix for ReplSet refresh attempting to close a closed socket +* Default op_timeout for ReplSetConnection is now disabled (was 30 seconds) +* Support db output option for map reduce (John Ewart) +* Support for keeping limited versions of files using GridFS (VvanGemert) + ### 1.5.2 2011-12-13 From f9bd5c39c41d85756533b326661d18cda073e92d Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Wed, 22 Feb 2012 13:47:10 -0500 Subject: [PATCH 068/442] minor: ReplSet doc example update --- source/examples/replica_set.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/examples/replica_set.rb b/source/examples/replica_set.rb index deb027325..2cad06e8c 100644 --- a/source/examples/replica_set.rb +++ b/source/examples/replica_set.rb @@ -4,7 +4,7 @@ cons = [] 10.times do - cons << Mongo::Connection.multi([['localhost', 27017]], :read_secondary => true) + cons << Mongo::ReplSetConnection(['localhost:27017'], :read => :secondary) end ports = cons.map do |con| From 74390b81da4fa045528479faefd34569b7c37d76 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Wed, 7 Mar 2012 14:04:12 -0500 Subject: [PATCH 069/442] minor: update history file --- source/HISTORY.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index 1a3808be2..e5ed9899e 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,13 @@ # MongoDB Ruby Driver History +### 1.6.1 +2012-03-07 + +* Added thread affinity to Mongo::Pool +* Added deploy tasks +* Added Travis CI support (Cyril Mougel) +* Logging warning message is only displayed for level :debug + ### 1.6.0 2012-02-22 From 186f2e01b1b9c9ea6f4bb12a39ad1f9d3719afea Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Wed, 7 Mar 2012 16:38:14 -0500 Subject: [PATCH 070/442] minor: fix for refresh with threads test and rs documentation updates --- source/REPLICA_SETS.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index c6a1ca30f..c401e9f85 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -38,8 +38,16 @@ The driver will essentially cycle through all known seed addresses until a node You can now specify a refresh mode and refresh interval for a replica set connection. This will help to ensure that changes to a replica set's configuration are quickly reflected on the driver side. In particular, if you change the state of any secondary node, the automated refresh will ensure that this state is recorded on the client side. + +There are two secenarios in which refresh is helpful and does not raise exceptions: + +# You add a new secondary node to an existing replica set +# You remove an unused secondary from an existing replica set + +If using MongoDB earlier than 2.0 any changes to replica set state will raise exceptions therefore refresh mode will not be useful. + If you add a secondary that responds to pings much faster than the existing nodes, then the new secondary will -be used for reads. +be used for reads if :read_preference is :secondary or :secondary_only Refresh mode is disabled by default. From 8b5f03442622abed8728a879337e8b8a2bf7bf41 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Thu, 15 Mar 2012 08:32:25 -0400 Subject: [PATCH 071/442] minor: update for replica set docs --- source/REPLICA_SETS.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index c401e9f85..4a9f3c42e 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -41,8 +41,8 @@ the state of any secondary node, the automated refresh will ensure that this sta There are two secenarios in which refresh is helpful and does not raise exceptions: -# You add a new secondary node to an existing replica set -# You remove an unused secondary from an existing replica set +1. You add a new secondary node to an existing replica set +2. You remove an unused secondary from an existing replica set If using MongoDB earlier than 2.0 any changes to replica set state will raise exceptions therefore refresh mode will not be useful. From db9b322a63d3d545582bffb379c3f9afd69c4105 Mon Sep 17 00:00:00 2001 From: Gary Murakami Date: Fri, 30 Mar 2012 16:56:51 -0400 Subject: [PATCH 072/442] fix README gem require for test, typos, JRuby reminders, also add require rubygems to simple.rb --- source/examples/simple.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/source/examples/simple.rb b/source/examples/simple.rb index 76f354c05..4850055e9 100644 --- a/source/examples/simple.rb +++ b/source/examples/simple.rb @@ -1,5 +1,6 @@ $:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) +require 'rubygems' require 'mongo' include Mongo From 9d4ad6849c5dd996d40d66aec9d1a6342512b6e0 Mon Sep 17 00:00:00 2001 From: Gary Murakami Date: Mon, 2 Apr 2012 17:25:01 -0400 Subject: [PATCH 073/442] [Gary] Editing due to minor disconnects of a newbie - might as well make improvements before I'm too tainted by familiarity. Any suggestions for a GFM previewer? --- source/TUTORIAL.md | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index a589e2ba7..3d7a519e2 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -9,8 +9,11 @@ As always, the [latest source for the Ruby driver](http://github.com/mongodb/mon ## Installation The mongo-ruby-driver gem is served through Rubygems.org. To install, make sure you have the latest version of rubygems. + gem update --system + Next, install the mongo rubygem: + gem install mongo The required `bson` gem will be installed automatically. @@ -32,27 +35,33 @@ All of the code here assumes that you have already executed the following Ruby c #### Making a Connection -An `Mongo::Connection` instance represents a connection to MongoDB. You use a Connection instance to obtain an Mongo:DB instance, which represents a named database. The database doesn't have to exist - if it doesn't, MongoDB will create it for you. - -You can optionally specify the MongoDB server address and port when connecting. The following example shows three ways to connect to the database "mydb" on the local machine: - - db = Mongo::Connection.new.db("mydb") - db = Mongo::Connection.new("localhost").db("mydb") - db = Mongo::Connection.new("localhost", 27017).db("mydb") - -At this point, the `db` object will be a connection to a MongoDB server for the specified database. Each DB instance uses a separate socket connection to the server. +An `Mongo::Connection` instance represents a connection to MongoDB. You can optionally specify the MongoDB server address and port when connecting. The following example shows three ways to connect to the local machine: -If you're trying to connect to a replica set, see [Replica Sets in Ruby](http://www.mongodb.org/display/DOCS/Replica+Sets+in+Ruby). + connection = Mongo::Connection.new # (optional host/port args) + connection = Mongo::Connection.new("localhost") + connection = Mongo::Connection.new("localhost", 27017) #### Listing All Databases - connection = Mongo::Connection.new # (optional host/port args) connection.database_names.each { |name| puts name } connection.database_info.each { |info| puts info.inspect} #### Dropping a Database connection.drop_database('database_name') +#### Using a Database + +You use a Connection instance to obtain an Mongo:DB instance, which represents a named database. The database doesn't have to exist - if it doesn't, MongoDB will create it for you. The following examples use the database "mydb": + + db = connection.db("mydb") + db = Mongo::Connection.new.db("mydb") + +At this point, the `db` object will be a connection to a MongoDB server for the specified database. Each DB instance uses a separate socket connection to the server. + +If you're trying to connect to a replica set, see [Replica Sets in Ruby](http://www.mongodb.org/display/DOCS/Replica+Sets+in+Ruby). + +#### Authentication + MongoDB can be run in a secure mode where access to databases is controlled through name and password authentication. When run in this mode, any client application must provide a name and password before doing any operations. In the Ruby driver, you simply do the following with the connected mongo object: auth = db.authenticate(my_user_name, my_password) @@ -75,8 +84,11 @@ as the output. #### Getting a Collection You can get a collection to use using the `collection` method: + coll = db.collection("testCollection") + This is aliased to the \[\] method: + coll = db["testCollection"] Once you have this collection object, you can now do things like insert data, query for data, etc. From 1fca5ff217217c8ba6cd9cbbd0a31f9d1710a876 Mon Sep 17 00:00:00 2001 From: Gary Murakami Date: Tue, 3 Apr 2012 15:14:43 -0400 Subject: [PATCH 074/442] [Gary] Improvements to TUTORIAL, completeness and bug fixes, CRUD reorg, Delete documents, explain, Ruby style --- source/TUTORIAL.md | 190 +++++++++++++++++++++++++++------------------ 1 file changed, 116 insertions(+), 74 deletions(-) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index 3d7a519e2..6e3fbeaab 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -26,9 +26,15 @@ After installing, you may want to look at the [examples](http://github.com/mongo ## Getting started +Note that the output in the following has been updated to Ruby 1.9, so if you are using Ruby 1.8, you will see some minor differences. To follow this tutorial interactively, at the command line, run the Interactive Ruby Shell. + + irb + +As you execute commands, irb will output the result using the `inspect` method. If you are editing and running a script for this tutorial, you can view output using the `puts` or `p` methods. + #### Using the gem -All of the code here assumes that you have already executed the following Ruby code: +Use the `mongo` gem via the `require` kernel method. require 'rubygems' # not necessary for Ruby 1.9 require 'mongo' @@ -43,15 +49,15 @@ An `Mongo::Connection` instance represents a connection to MongoDB. You can opt #### Listing All Databases - connection.database_names.each { |name| puts name } - connection.database_info.each { |info| puts info.inspect} + connection.database_names + connection.database_info.each { |info| puts info.inspect } #### Dropping a Database connection.drop_database('database_name') -#### Using a Database +## Using a Database -You use a Connection instance to obtain an Mongo:DB instance, which represents a named database. The database doesn't have to exist - if it doesn't, MongoDB will create it for you. The following examples use the database "mydb": +You use a Connection instance to obtain an Mongo::DB instance, which represents a named database. The database doesn't have to exist - if it doesn't, MongoDB will create it for you. The following examples use the database "mydb": db = connection.db("mydb") db = Mongo::Connection.new.db("mydb") @@ -68,20 +74,7 @@ MongoDB can be run in a secure mode where access to databases is controlled thro If the name and password are valid for the database, `auth` will be `true`. Otherwise, it will be `false`. You should look at the MongoDB log for further information if available. -#### Getting a List Of Collections - -Each database has zero or more collections. You can retrieve a list of them from the db (and print out any that are there): - - db.collection_names.each { |name| puts name } - -and assuming that there are two collections, name and address, in the database, you would see - - name - address - -as the output. - -#### Getting a Collection +## Using a Collection You can get a collection to use using the `collection` method: @@ -91,11 +84,11 @@ This is aliased to the \[\] method: coll = db["testCollection"] -Once you have this collection object, you can now do things like insert data, query for data, etc. +Once you have this collection object, you can now do create, read, update, and delete (CRUD) functions on persistent storage. -#### Inserting a Document +### Creating Documents -Once you have the collection object, you can insert documents into the collection. For example, lets make a little document that in JSON would be represented as +Once you have the collection object, you can create or `insert` documents into the collection. For example, lets make a little document that in JSON would be represented as { "name" : "MongoDB", @@ -107,112 +100,153 @@ Once you have the collection object, you can insert documents into the collectio } } -Notice that the above has an "inner" document embedded within it. To do this, we can use a Hash or the driver's OrderedHash (which preserves key order) to create the document (including the inner document), and then just simply insert it into the collection using the `insert()` method. +Notice that the above has an "inner" document embedded within it. To do this, we can use a Hash or the driver's OrderedHash (which preserves key order) to create the document (including the inner document), and then just simply insert it into the collection using the `insert` method. - doc = {"name" => "MongoDB", "type" => "database", "count" => 1, - "info" => {"x" => 203, "y" => '102'}} - coll.insert(doc) + doc = {"name" => "MongoDB", "type" => "database", "count" => 1, "info" => {"x" => 203, "y" => '102'}} + id = coll.insert(doc) -#### Updating a Document +We have saved the `id` for future use below. Now the collection has been created and you can list it. -We can update the previous document using the `update` method. There are a couple ways to update a document. We can rewrite it: +#### Getting a List Of Collections - doc["name"] = "MongoDB Ruby" - coll.update({"_id" => doc["_id"]}, doc) +Each database has zero or more collections. You can retrieve a list of them from the db (and print out any that are there): -Or we can use an atomic operator to change a single value: + db.collection_names - coll.update({"_id" => doc["_id"]}, {"$set" => {"name" => "MongoDB Ruby"}}) +You should see -Read [more about updating documents|Updating]. + \["testCollection", "system.indexes"\] -#### Finding the First Document In a Collection using `find_one()` +#### Adding Multiple Documents -To show that the document we inserted in the previous step is there, we can do a simple `find_one()` operation to get the first document in the collection. This method returns a single document (rather than the `Cursor` that the `find()` operation returns). +To demonstrate some more interesting queries, let's add multiple simple documents to the collection. These documents will have the following form: - my_doc = coll.find_one() - puts my_doc.inspect + { + "i" : value + } -and you should see: +Here's how to insert them: - {"_id"=>#, "name"=>"MongoDB", - "info"=>{"x"=>203, "y"=>102}, "type"=>"database", "count"=>1} + 100.times { |i| coll.insert("i" => i) } -Note the `_id` element has been added automatically by MongoDB to your document. +Notice that we can insert documents of different "shapes" into the same collection. These records are in the same collection as the complex record we inserted above. This aspect is what we mean when we say that MongoDB is "schema-free". -#### Adding Multiple Documents +### Reading Documents -To demonstrate some more interesting queries, let's add multiple simple documents to the collection. These documents will have the following form: - { - "i" : value - } +#### Reading the First Document in a Collection using `find_one` -Here's how to insert them: +To retrieve the document that we inserted, we can do a simple `find_one` method to get the first document in the collection. This method returns a single document directly. - 100.times { |i| coll.insert("i" => i) } + coll.find_one -Notice that we can insert documents of different "shapes" into the same collection. These records are in the same collection as the complex record we inserted above. This aspect is what we mean when we say that MongoDB is "schema-free". +and you should something like: -#### Counting Documents in a Collection + {"_id"=>BSON::ObjectId('4f7b1ea6e4d30b35c9000001'), "name"=>"MongoDB", "type"=>"database", "count"=>1, "info"=>{"x"=>203, "y"=>"102"}} + +Note the `_id` element has been added automatically by MongoDB to your document. -Now that we've inserted 101 documents (the 100 we did in the loop, plus the first one), we can check to see if we have them all using the `count()` method. +#### Reading All of the Documents with a Cursor using `find` - puts coll.count() +To get all the documents from the collection, we use the `find` method. `find` returns a `Cursor` object, which allows us to iterate over the set of documents that matches our query. The Ruby driver's Cursor implemented Enumerable, which allows us to use `Enumerable#each`, `Enumerable#map}, etc. For instance: -and it should print `101`. + coll.find.each { |row| puts row.inspect } -#### Using a Cursor to get all of the Documents +and that should print all 101 documents in the collection. You can take advantage of `Enumerable#to_a`. -To get all the documents from the collection, we use the `find()` method. `find()` returns a `Cursor` object, which allows us to iterate over the set of documents that matches our query. The Ruby driver's Cursor implemented Enumerable, which allows us to use `Enumerable#each`, `Enumerable#map}, etc. For instance: + puts coll.find.to_a - coll.find().each { |row| puts row.inspect } +#### Specific Queries -and that should print all 101 documents in the collection. +We can create a _query_ hash to pass to the `find` method to get a subset of the documents in our collection. To check that our update worked, find the document by id: -#### Getting a Single Document with a Query + coll.find("_id" => id).to_a -We can create a _query_ hash to pass to the `find()` method to get a subset of the documents in our collection. For example, if we wanted to find the document for which the value of the "i" field is 71, we would do the following ; +If we wanted to find the document for which the value of the "i" field is 71, we would do the following: - coll.find("i" => 71).each { |row| puts row.inspect } + coll.find("i" => 71).to_a and it should just print just one document: - {"_id"=>#, "i"=>71} + {"_id"=>BSON::ObjectId('4f7b20b4e4d30b35c9000049'), "i"=>71} + +#### Counting Documents in a Collection + +Now that we've inserted 101 documents (the 100 we did in the loop, plus the first one), we can check to see if we have them all using the `count` method. + + coll.count + +and it should print `101`. #### Getting a Set of Documents With a Query We can use the query to get a set of documents from our collection. For example, if we wanted to get all documents where "i" > 50, we could write: - coll.find("i" => {"$gt" => 50}).each { |row| puts row } + puts coll.find("i" => {"$gt" => 50}).to_a which should print the documents where i > 50. We could also get a range, say 20 < i <= 30: - coll.find("i" => {"$gt" => 20, "$lte" => 30}).each { |row| puts row } + puts coll.find("i" => {"$gt" => 20, "$lte" => 30}).to_a -#### Selecting a subset of fields for a query +#### Selecting a Subset of Fields for a Query -Use the `:fields` option. If you just want fields "a" and "b": +Use the `:fields` option to specify fields to return. - coll.find("i" => {"$gt" => 50}, :fields => ["a", "b"]).each { |row| puts row } + puts coll.find("_id" => id, :fields => ["name", "type"]).to_a #### Querying with Regular Expressions Regular expressions can be used to query MongoDB. To find all names that begin with 'a': - coll.find({"name" => /^a/}) + puts coll.find({"name" => /^M/}).to_a You can also construct a regular expression dynamically. To match a given search string: + params = {'search' => 'DB'} search_string = params['search'] # Constructor syntax - coll.find({"name" => Regexp.new(search_string)}) + puts coll.find({"name" => Regexp.new(search_string)}).to_a # Literal syntax - coll.find({"name" => /#{search_string}/}) + puts coll.find({"name" => /#{search_string}/}).to_a Although MongoDB isn't vulnerable to anything like SQL-injection, it may be worth checking the search string for anything malicious. +### Updating Documents + +We can update the previous document using the `update` method. There are a couple ways to update a document. We can rewrite it: + + doc["name"] = "MongoDB Ruby" + coll.update({"_id" => id}, doc) + +Or we can use an atomic operator to change a single value: + + coll.update({"_id" => id}, {"$set" => {"name" => "MongoDB Ruby"}}) + +Verify the update. + + puts coll.find("_id" => id).to_a + +Read [more about updating documents|Updating]. + +### Deleting Documents + +Use the `remove` method to delete documents. + + coll.count + coll.remove("i" => 71) + coll.count + puts coll.find("i" => 71).to_a + +The above shows that the count has been reduced and that the document can no longer be found. + +Without arguments, the `remove` method deletes all documents. + + coll.remove + coll.count + +Please program carefully. + ## Indexing #### Creating An Index @@ -227,7 +261,19 @@ To specify complex indexes or a descending index you need to use a slightly more # Explicit "ascending" coll.create_index([["i", Mongo::ASCENDING]]) -#### Creating and querying on a geospatial index +Use the `explain` method on the cursor to show how MongoDB will run the query. + + coll.find("_id" => id).explain + coll.find("i" => 71).explain + coll.find("type" => "database").explain + +The above shows that the query by `_id` and `i` will use faster indexed BtreeCursor, while the query by `type` will use a slower BasicCursor. + +#### Getting a List of Indexes on a Collection + +You can get a list of the indexes on a collection using `coll.index_information`. + +#### Creating and Querying on a Geospatial Index First, create the index on a field containing long-lat values: @@ -239,10 +285,6 @@ Then get a list of the twenty locations nearest to the point 50, 50: puts p.inspect end -#### Getting a List of Indexes on a Collection - -You can get a list of the indexes on a collection using `coll.index_information()`. - ## Database Administration A database can have one of three profiling levels: off (:off), slow queries only (:slow_only), or all (:all). To see the database level: From 1094be431a8e7a92a3f6e7150877a52c2a377c3c Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Tue, 3 Apr 2012 17:28:42 -0400 Subject: [PATCH 075/442] RUBY-421 fixes for rake reployment tasks --- source/RELEASES.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/source/RELEASES.md b/source/RELEASES.md index 2d7c25f67..9288172de 100644 --- a/source/RELEASES.md +++ b/source/RELEASES.md @@ -45,3 +45,10 @@ Before each relese to Rubygems.org, the following steps will be taken: 11. Close out release in JIRA. 12. Annouce release on mongodb-user and mongodb-dev. + +## Rake Deploy Tasks +1. rake deploy:change_version[1.6.1] +2. rake deploy:git_prepare +3. rake deploy:git_push +4. rake deploy:gem_build +5. rake deploy:gem_push From b811c8e715d92b3b248b1f8bd4c6dd928643574f Mon Sep 17 00:00:00 2001 From: Gary Murakami Date: Wed, 4 Apr 2012 13:25:26 -0400 Subject: [PATCH 076/442] [Gary] more improvements, dropping has its own complete section --- source/TUTORIAL.md | 73 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 58 insertions(+), 15 deletions(-) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index 6e3fbeaab..e40d44376 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -32,14 +32,14 @@ Note that the output in the following has been updated to Ruby 1.9, so if you ar As you execute commands, irb will output the result using the `inspect` method. If you are editing and running a script for this tutorial, you can view output using the `puts` or `p` methods. -#### Using the gem +### Using the gem Use the `mongo` gem via the `require` kernel method. require 'rubygems' # not necessary for Ruby 1.9 require 'mongo' -#### Making a Connection +### Making a Connection An `Mongo::Connection` instance represents a connection to MongoDB. You can optionally specify the MongoDB server address and port when connecting. The following example shows three ways to connect to the local machine: @@ -47,13 +47,12 @@ An `Mongo::Connection` instance represents a connection to MongoDB. You can opt connection = Mongo::Connection.new("localhost") connection = Mongo::Connection.new("localhost", 27017) -#### Listing All Databases +### Listing All Databases connection.database_names connection.database_info.each { |info| puts info.inspect } - #### Dropping a Database - connection.drop_database('database_name') +The `database_info` method returns a hash mapping database names to the size of the database in bytes. ## Using a Database @@ -66,7 +65,7 @@ At this point, the `db` object will be a connection to a MongoDB server for the If you're trying to connect to a replica set, see [Replica Sets in Ruby](http://www.mongodb.org/display/DOCS/Replica+Sets+in+Ruby). -#### Authentication +### Authentication MongoDB can be run in a secure mode where access to databases is controlled through name and password authentication. When run in this mode, any client application must provide a name and password before doing any operations. In the Ruby driver, you simply do the following with the connected mongo object: @@ -86,7 +85,7 @@ This is aliased to the \[\] method: Once you have this collection object, you can now do create, read, update, and delete (CRUD) functions on persistent storage. -### Creating Documents +### Creating Documents with `insert` Once you have the collection object, you can create or `insert` documents into the collection. For example, lets make a little document that in JSON would be represented as @@ -131,7 +130,7 @@ Here's how to insert them: Notice that we can insert documents of different "shapes" into the same collection. These records are in the same collection as the complex record we inserted above. This aspect is what we mean when we say that MongoDB is "schema-free". -### Reading Documents +### Reading Documents with `find_one` and `find` #### Reading the First Document in a Collection using `find_one` @@ -155,6 +154,8 @@ and that should print all 101 documents in the collection. You can take advanta puts coll.find.to_a +Important note - using `to_a` pulls all of the full result set into memory and is inefficient if you can process by each individual document. To process with more memory efficiency, use the `each` method with a code block for the cursor. + #### Specific Queries We can create a _query_ hash to pass to the `find` method to get a subset of the documents in our collection. To check that our update worked, find the document by id: @@ -212,7 +213,7 @@ You can also construct a regular expression dynamically. To match a given search Although MongoDB isn't vulnerable to anything like SQL-injection, it may be worth checking the search string for anything malicious. -### Updating Documents +### Updating Documents with `update` We can update the previous document using the `update` method. There are a couple ways to update a document. We can rewrite it: @@ -229,7 +230,7 @@ Verify the update. Read [more about updating documents|Updating]. -### Deleting Documents +### Deleting Documents with `remove` Use the `remove` method to delete documents. @@ -249,7 +250,7 @@ Please program carefully. ## Indexing -#### Creating An Index +### Creating An Index MongoDB supports indexes, and they are very easy to add on a collection. To create an index, you specify an index name and an array of field names to be indexed, or a single field name. The following creates an ascending index on the "i" field: @@ -269,11 +270,13 @@ Use the `explain` method on the cursor to show how MongoDB will run the query. The above shows that the query by `_id` and `i` will use faster indexed BtreeCursor, while the query by `type` will use a slower BasicCursor. -#### Getting a List of Indexes on a Collection +### Getting a List of Indexes on a Collection + +You can get a list of the indexes on a collection. -You can get a list of the indexes on a collection using `coll.index_information`. + coll.index_information -#### Creating and Querying on a Geospatial Index +### Creating and Querying on a Geospatial Index First, create the index on a field containing long-lat values: @@ -282,9 +285,49 @@ First, create the index on a field containing long-lat values: Then get a list of the twenty locations nearest to the point 50, 50: people.find({"loc" => {"$near" => [50, 50]}}, {:limit => 20}).each do |p| - puts p.inspect + puts p.inspect end +## Dropping + +### Drop an Index + +To drop a secondary index, use the `drop_index` method on the collection. + + coll.drop_index("i_1") + coll.index_information + +The dropped index is no longer listed. + +### Drop All Indexes + +To drop all secondary indexes, use the `drop_indexes` method on the collection. + + coll.drop_indexes + coll.index_information + +Only the primary index "_id_" is listed. + +### Drop a Collection + +To drop a collection, use the `drop` method on the collection. + + coll.drop + db.collection_names + +The dropped collection is no longer listed. The `drop_collection` method can be used on the database as an alternative. + + db.drop_collection("testCollection") + +### Drop a Database + +To drop a database, use the `drop_database` method on the connection. + + connection.drop_database("mydb") + connection.database_names + +The dropped database is no longer listed. + ## Database Administration A database can have one of three profiling levels: off (:off), slow queries only (:slow_only), or all (:all). To see the database level: From 626e4676883289f5cd66d9cd31e9878e19d8033e Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Thu, 5 Apr 2012 13:28:20 -0400 Subject: [PATCH 077/442] RELEASE 1.6.2 --- source/HISTORY.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/source/HISTORY.md b/source/HISTORY.md index e5ed9899e..f14cb4109 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,15 @@ # MongoDB Ruby Driver History +### 1.6.2 +2012-04-05 + +* Implements socket timeouts via non-blocking IO instead of Timeout module +which should greately increase performance in highly threaded applications +* Added ability to authentication via secondary if primary node unavailable +* Replica set refresh interval now enforces a lower bound of 60 seconds +* Added documentation for dropping indexes, collections, databases +* Test output cleanup (...)s unless failure occurs + ### 1.6.1 2012-03-07 From a40741b9f7266903b5d10edc5bc5b04ce9d4d693 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Tue, 10 Apr 2012 23:10:43 -0400 Subject: [PATCH 078/442] minor: added section on sorting to tutorial --- source/TUTORIAL.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index e40d44376..cd09e65e0 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -170,6 +170,18 @@ and it should just print just one document: {"_id"=>BSON::ObjectId('4f7b20b4e4d30b35c9000049'), "i"=>71} +#### Sorting Documents in a Collection + +To sort documents, simply use the `sort` method. The method can either take a key or an array of [key, direction] pairs to sort by. + +Direction defaults to ascending order but can be specified as Mongo::ASCENDING, :ascending, or :asc whereas descending order can be specified with Mongo::DESCENDING, :descending, or :desc. + + # Sort in ascending order by :i + coll.find.sort(:i) + + # Sort in descending order by :i + coll.find.sort([:i, :desc]) + #### Counting Documents in a Collection Now that we've inserted 101 documents (the 100 we did in the loop, plus the first one), we can check to see if we have them all using the `count` method. From 9b0464e28e3c6d9bb34122f5ad1666fde4b80fc4 Mon Sep 17 00:00:00 2001 From: Gary Murakami Date: Mon, 14 May 2012 11:54:28 -0400 Subject: [PATCH 079/442] documentation improvements for Yard/Ydoc newbies, also relative links replace absolute links where possible so that the user is left with the exact version and not forced to current web docs, in accordance with DOCS-197 --- source/GridFS.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/GridFS.md b/source/GridFS.md index 03b3eb104..1ae3cbbf6 100644 --- a/source/GridFS.md +++ b/source/GridFS.md @@ -88,7 +88,7 @@ Deleting a file is as simple as providing the id: ### The GridFileSystem class -[GridFileSystem](http://api.mongodb.org/ruby/current/Mongo/GridFileSystem.html) is a light emulation of a file system and therefore has a couple of unique properties. The first is that filenames are assumed to be unique. The second, a consequence of the first, is that files are versioned. To see what this means, let's create a GridFileSystem instance: +[GridFileSystem](Mongo/GridFileSystem.html) is a light emulation of a file system and therefore has a couple of unique properties. The first is that filenames are assumed to be unique. The second, a consequence of the first, is that files are versioned. To see what this means, let's create a GridFileSystem instance: #### Saving files @@ -154,5 +154,5 @@ All of the options for storing metadata and saving in safe mode are available fo ### Advanced Users -Astute code readers will notice that the Grid and GridFileSystem classes are merely thin wrappers around an underlying [GridIO class](http://api.mongodb.org/ruby/current/Mongo/GridIO.html). This means that it's easy to customize the GridFS implementation presented here; just use GridIO for all the low-level work, and build the API you need in an external manager class similar to Grid or GridFileSystem. +Astute code readers will notice that the Grid and GridFileSystem classes are merely thin wrappers around an underlying [GridIO class](Mongo/GridIO.html). This means that it's easy to customize the GridFS implementation presented here; just use GridIO for all the low-level work, and build the API you need in an external manager class similar to Grid or GridFileSystem. From 9422623f5ff0c17aad14bbb5dfc181b924f53ff2 Mon Sep 17 00:00:00 2001 From: Raul E Rangel Date: Mon, 14 May 2012 14:20:51 -0600 Subject: [PATCH 080/442] Fixed a typo --- source/WRITE_CONCERN.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/WRITE_CONCERN.md b/source/WRITE_CONCERN.md index c88dddc3b..16759dc08 100644 --- a/source/WRITE_CONCERN.md +++ b/source/WRITE_CONCERN.md @@ -9,7 +9,7 @@ Write concern is set using the `:safe` option. There are several possible option @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200}) @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200, :j => true}) -The first, `true`, simply indicates that we should request a response from the server to ensure that to errors have occurred. The second, `{:w => 2}`, forces the server to wait until at least two servers have recorded the write. The third does the same but will time out if the replication can't be completed in 200 milliseconds. +The first, `true`, simply indicates that we should request a response from the server to ensure that no errors have occurred. The second, `{:w => 2}`, forces the server to wait until at least two servers have recorded the write. The third does the same but will time out if the replication can't be completed in 200 milliseconds. Setting a value for `wtimeout` is encouraged. Finally, the fourth example forces the journal to sync to disk if journaling is enabled. From 1ca76adc680607cc54cf59d3addf5c258224ed42 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Tue, 5 Jun 2012 22:17:44 -0400 Subject: [PATCH 081/442] RELEASE 1.6.3 --- source/HISTORY.md | 16 +++++++++++++++- source/RELEASES.md | 2 +- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/source/HISTORY.md b/source/HISTORY.md index f14cb4109..b97aacef1 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,10 +1,24 @@ # MongoDB Ruby Driver History +### 1.6.3 +2012-06-05 + +* Performance measurements and enhancements (especially for C-extensions) +* Bug fixes for checking strings with non UTF-8 forced or implied encodings +* Added refresh support for multiple threaded instances of ReplSetConnection +* Added ability to handle IRB::Abort Exception (ctrl-c) cleanly +* Added support for large dates on 32-bit platforms (Ruby 1.9+) +* Added #to_ary method for BSON::ObjectId (Farrel Lifson) +* Added support for ENV['MONGODB_URI'] (Seamus Abshere) +* Various gridio bug fixes (John Bintz) +* Various logging support improvements +* Various documentation improvements (tutorials, sorting, links) + ### 1.6.2 2012-04-05 * Implements socket timeouts via non-blocking IO instead of Timeout module -which should greately increase performance in highly threaded applications +which should greatly increase performance in highly threaded applications * Added ability to authentication via secondary if primary node unavailable * Replica set refresh interval now enforces a lower bound of 60 seconds * Added documentation for dropping indexes, collections, databases diff --git a/source/RELEASES.md b/source/RELEASES.md index 9288172de..24d90872f 100644 --- a/source/RELEASES.md +++ b/source/RELEASES.md @@ -28,7 +28,7 @@ Before each relese to Rubygems.org, the following steps will be taken: 3. Update the version in lib/bson.rb, lib/mongo/version.rb, and ext/version.h. -4. Commit: "Release [VERSION]" +4. Commit: "RELEASE [VERSION]" 5. git tag [version] From 9eab7ad52f3f37a6c2db584bae1855fb6ce6826b Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Wed, 6 Jun 2012 08:44:59 -0400 Subject: [PATCH 082/442] RELEASE 1.6.4 --- source/RELEASES.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/RELEASES.md b/source/RELEASES.md index 24d90872f..3d54b73f8 100644 --- a/source/RELEASES.md +++ b/source/RELEASES.md @@ -26,7 +26,7 @@ Before each relese to Rubygems.org, the following steps will be taken: 2. Update the HISTORY file and document all significant commits. -3. Update the version in lib/bson.rb, lib/mongo/version.rb, and ext/version.h. +3. Update the version in lib/bson.rb, lib/mongo/version.rb, and ext/cbson/version.h. 4. Commit: "RELEASE [VERSION]" @@ -36,7 +36,7 @@ Before each relese to Rubygems.org, the following steps will be taken: 7. Push tags and commit to GitHub (git push origin master, git push --tags). -8. Build and push docs. +8. Build and push docs. (git: mongodb/apidocs) 9. Push gems to Rubygems.org. @@ -47,7 +47,7 @@ Before each relese to Rubygems.org, the following steps will be taken: 12. Annouce release on mongodb-user and mongodb-dev. ## Rake Deploy Tasks -1. rake deploy:change_version[1.6.1] +1. rake deploy:change_version[x.x.x] 2. rake deploy:git_prepare 3. rake deploy:git_push 4. rake deploy:gem_build From 93649a9da1169581ade84275293a65a2bc710512 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Thu, 7 Jun 2012 10:31:06 -0400 Subject: [PATCH 083/442] minor: tutorial, history, documentation updates --- source/HISTORY.md | 6 ++++++ source/TUTORIAL.md | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/source/HISTORY.md b/source/HISTORY.md index b97aacef1..52c2d4334 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,11 @@ # MongoDB Ruby Driver History +### 1.6.4 +2012-06-06 + +* Added ability to declare sort ordering via an ordered hash +* Addresses major compatability issue with mongoid created by v1.6.3 + ### 1.6.3 2012-06-05 diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index cd09e65e0..714b6ffd5 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -180,7 +180,7 @@ Direction defaults to ascending order but can be specified as Mongo::ASCENDING, coll.find.sort(:i) # Sort in descending order by :i - coll.find.sort([:i, :desc]) + coll.find.sort(:i => :desc) #### Counting Documents in a Collection From f2e645818a0a6c78393d235af9efee10e7f37f1d Mon Sep 17 00:00:00 2001 From: JW Date: Thu, 7 Jun 2012 17:36:42 +0300 Subject: [PATCH 084/442] Small typo. --- source/TUTORIAL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index 714b6ffd5..87bb3b8e8 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -208,7 +208,7 @@ Use the `:fields` option to specify fields to return. #### Querying with Regular Expressions -Regular expressions can be used to query MongoDB. To find all names that begin with 'a': +Regular expressions can be used to query MongoDB. To find all names that begin with 'M': puts coll.find({"name" => /^M/}).to_a From 2c8c4e944c97dff3e4a0e3e5222e19520624438b Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Thu, 7 Jun 2012 10:31:06 -0400 Subject: [PATCH 085/442] minor: tutorial, history, documentation updates --- source/HISTORY.md | 6 ++++++ source/TUTORIAL.md | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/source/HISTORY.md b/source/HISTORY.md index b97aacef1..52c2d4334 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,11 @@ # MongoDB Ruby Driver History +### 1.6.4 +2012-06-06 + +* Added ability to declare sort ordering via an ordered hash +* Addresses major compatability issue with mongoid created by v1.6.3 + ### 1.6.3 2012-06-05 diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index cd09e65e0..714b6ffd5 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -180,7 +180,7 @@ Direction defaults to ascending order but can be specified as Mongo::ASCENDING, coll.find.sort(:i) # Sort in descending order by :i - coll.find.sort([:i, :desc]) + coll.find.sort(:i => :desc) #### Counting Documents in a Collection From 18b42d9ce2d3959ba7dc35c5332a015e76573145 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Mon, 20 Aug 2012 12:15:10 -0400 Subject: [PATCH 086/442] minor: documentation updates --- source/{GridFS.md => GRID_FS.md} | 0 source/HISTORY.md | 10 ++++ source/READ_PREFERENCE.md | 80 ++++++++++++++++++++++++++++---- 3 files changed, 80 insertions(+), 10 deletions(-) rename source/{GridFS.md => GRID_FS.md} (100%) diff --git a/source/GridFS.md b/source/GRID_FS.md similarity index 100% rename from source/GridFS.md rename to source/GRID_FS.md diff --git a/source/HISTORY.md b/source/HISTORY.md index 52c2d4334..2133ce71c 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -1,5 +1,15 @@ # MongoDB Ruby Driver History +### 1.7.0 +2012-08-20 + +* Added testing and full support for MongoDB 2.1 & 2.2 +* Added Aggregation Framework helper method +* Added support for Mongos high availability +* Modified and added new read preferences (details in documentation) +* Added support for data center awareness (tag_sets) +* Fixed bug which attempted to close cursors on wrong replica set member + ### 1.6.4 2012-06-06 diff --git a/source/READ_PREFERENCE.md b/source/READ_PREFERENCE.md index 7d77a7bc7..2bf05a1fc 100644 --- a/source/READ_PREFERENCE.md +++ b/source/READ_PREFERENCE.md @@ -1,15 +1,22 @@ # Read Preference in Ruby -## Setting the read preference +## About Read Preference -You can using the `:read` option to specify a query's read preference. There are for now two possible options: +Read preferences determine the candidate replica set members to which a query or command can be sent. They consist of a *mode* specified as a symbol and an array of hashes known as *tag_sets*. - @collection.find({:doc => 'foo'}, :read => :primary) - @collection.find({:doc => 'foo'}, :read => :secondary) +Read preference mode is configured by providing the read option to a connection, database, collection, or cursor. -In the first case, the query will be directed to the primary node in a replica set. In the second, the query will be sent -to a secondary node. The driver will attempt to choose a secondary node that's nearby, as determined by ping time. If more -than one secondary node is closeby (e.g, responds to pings within 10ms), then a random node within this subset will be chosen. + @collection.find({:doc => 'foo'}, :read => :primary) # read from primary only + @collection.find({:doc => 'foo'}, :read => :secondary) # read from secondaries only + +Used in conjunction with tag_sets: + + @collection.find({:name => 'foo'}, :read => :secondary_preferred, :tag_sets => [{:continent => 'USA'}]) + +*Please Note*: Behavior of some read preference modes have changed in version 1.7.0: + +* `:secondary_preferred` mode is now used to prefer reads from secondary members (before this was the behavior of `:secondary`). +* `:secondary_only` mode (which only allowed reads from secondaries) is now called `:secondary`. ## Read preference inheritance @@ -33,7 +40,60 @@ You can examine the read preference on any object by calling its `read_preferenc @db.read_preference @collection.read_preference -## Future work +## Modes + +You can using the `:read` option to specify a query's read preference mode. There are five possible options. + +### :primary + +With primary, all read operations from the client will use the primary member only. This is the default read preference. + +If the primary is unavailable, all operations with this preference produce an error or throw an exception. Primary read preference modes are not compatible with read preferences modes that use tag sets If you specify a tag set with primary, the driver will produce an error. + +### :primary_preferred + +With the primaryPreferred read preference mode, operations will read from the primary member of the set in most situations. However, if the primary is unavailable, as is the case during failover situations, then these read operations can read from secondary members. + +When the read preference includes a tag set, the client will first read from the primary, if it is available, and then from secondaries that match the specified tags. If there are no secondaries with tags that match the specified tags, this read operation will produce an error. + +### :secondary + +With the secondary read preference mode, operations will read from the secondary member of the set if available. However, if there are no secondaries available, then these operations will produce an error or exception. + +Most sets have at least one secondary, but there are situations where there may not be an available secondary. For example, a set with a primary, a secondary, and an arbiter may not have any secondaries if a member is ever in recovering mode. + +When the read preference includes a tag set, the client will attempt to find a secondary members that match the specified tag set and directs reads to a random secondary from among the nearest group. If there are no secondaries with tags that match the specified tag set, this read operation will produce an error. + +### :secondary_preferred + +With the secondaryPreferred, operations will read from secondary members, but in situations where the set only has a primary instance, the read operation will use the set’s primary. + +When secondaryPreferred reads from a secondary and the read preference includes a tag set, the client will attempt to find a secondary members that match the specified tag set and directs reads to a random secondary from among the nearest group. If there are no secondaries with tags that match the specified tag set, this read operation will produce an error. + +### :nearest + +With the nearest, the driver will read from the nearest member of the set according to the member selection process nearest read operations will not have any consideration for the type of the set member. Reads in nearest mode may read from both primaries and secondaries. + +Set this mode when you want minimize the effect of network latency on read operations without preference for current or stale data. + +If you specify a tag set, the client will attempt to find a secondary members that match the specified tag set and directs reads to a random secondary from among the nearest group. + +## Tag Sets + +Tag sets can be used in for data center awareness by filtering secondary read operations. Primary reads occur independent of any tags. + +A member matches a tag set if its tags match all the tags in the set. For example, a member tagged "{ dc: 'ny', rack: 2, size: 'large' }" matches the tag set "{ dc: 'ny', rack: 2 }". A member's extra tags don't affect whether it's a match. + +Here is an example of a query which sends read operations to members in rack 2. + + @collection.find({:name => 'foo'}, :read => :secondary_preferred, :tag_sets => [{:rack => '2'}]) + +Tag set keys may be symbols or strings. Tag set values should be specified using strings. The `to_s` method will be called on any values provided in the tag set. + +Tag sets are used in conjunction with read preference mode. In this example, because we specified a mode of secondary_preferred, if no secondaries can be found that match the tag_set `{:rack => '2'}` then the primary will be used for the query. + +If only one tag set is provided, the set can be passed as a single hash parameter iteself without the enclosing array. + + @collection.find({:name => 'foo'}, :read => :secondary_preferred, :tag_sets => {:rack => '2'}) -In the v2.0 release of the driver, you'll also be able to specify a read preference consisting of a set of tags. This way, -you'll be able to direct reads to a replica set member. You can follow this issue's progress here: (https://jira.mongodb.org/browse/RUBY-326). +Specifiying tag_sets for mode `:primary` is considered an error and will raise a MongoArgumentError as tag_sets do not affect selection of primary members and only primary members can be selected in that particular mode. From 93e354c1c0bb1ee87a7119313aad4a0230d8fcf4 Mon Sep 17 00:00:00 2001 From: Semyon Perepelitsa Date: Sat, 15 Sep 2012 02:44:43 +0800 Subject: [PATCH 087/442] Tutorial fix: fields option should be in the second hash --- source/TUTORIAL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index 87bb3b8e8..aaa855733 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -204,7 +204,7 @@ which should print the documents where i > 50. We could also get a range, say Use the `:fields` option to specify fields to return. - puts coll.find("_id" => id, :fields => ["name", "type"]).to_a + puts coll.find({"_id" => id}, :fields => ["name", "type"]).to_a #### Querying with Regular Expressions From 536b58ef00a40f54baa8e7e63a028c42373a2785 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Thu, 4 Oct 2012 06:24:44 -0400 Subject: [PATCH 088/442] RUBY-478 organize tests --- source/load/thin/config.ru | 6 +++++ source/load/thin/config.yml.template | 6 +++++ source/load/thin/load.rb | 21 ++++++++++++++++++ source/load/unicorn/config.ru | 6 +++++ source/load/unicorn/load.rb | 23 ++++++++++++++++++++ source/load/unicorn/unicorn.rb.template | 29 +++++++++++++++++++++++++ 6 files changed, 91 insertions(+) create mode 100644 source/load/thin/config.ru create mode 100644 source/load/thin/config.yml.template create mode 100644 source/load/thin/load.rb create mode 100644 source/load/unicorn/config.ru create mode 100644 source/load/unicorn/load.rb create mode 100644 source/load/unicorn/unicorn.rb.template diff --git a/source/load/thin/config.ru b/source/load/thin/config.ru new file mode 100644 index 000000000..aa6ee91e7 --- /dev/null +++ b/source/load/thin/config.ru @@ -0,0 +1,6 @@ +require "rubygems" +require "sinatra" + +require File.join(File.dirname(__FILE__), 'load.rb') + +run Load diff --git a/source/load/thin/config.yml.template b/source/load/thin/config.yml.template new file mode 100644 index 000000000..aa6ee91e7 --- /dev/null +++ b/source/load/thin/config.yml.template @@ -0,0 +1,6 @@ +require "rubygems" +require "sinatra" + +require File.join(File.dirname(__FILE__), 'load.rb') + +run Load diff --git a/source/load/thin/load.rb b/source/load/thin/load.rb new file mode 100644 index 000000000..f97ee83a2 --- /dev/null +++ b/source/load/thin/load.rb @@ -0,0 +1,21 @@ +require File.join(File.dirname(__FILE__), '..', '..', '..', 'lib', 'mongo') +require 'logger' + +$con = Mongo::ReplSetConnection.new(['localhost:30000', 'localhost:30001'], :read => :secondary, :refresh_mode => :sync, :refresh_interval => 30) +$db = $con['foo'] + +class Load < Sinatra::Base + + configure do + LOGGER = Logger.new("sinatra.log") + enable :logging, :dump_errors + set :raise_errors, true + end + + get '/' do + $db['test'].insert({:a => rand(1000)}) + $db['test'].find({:a => {'$gt' => rand(2)}}, :read => :secondary).limit(2).to_a + "ok" + end + +end diff --git a/source/load/unicorn/config.ru b/source/load/unicorn/config.ru new file mode 100644 index 000000000..aa6ee91e7 --- /dev/null +++ b/source/load/unicorn/config.ru @@ -0,0 +1,6 @@ +require "rubygems" +require "sinatra" + +require File.join(File.dirname(__FILE__), 'load.rb') + +run Load diff --git a/source/load/unicorn/load.rb b/source/load/unicorn/load.rb new file mode 100644 index 000000000..f624cb10d --- /dev/null +++ b/source/load/unicorn/load.rb @@ -0,0 +1,23 @@ +require File.join(File.dirname(__FILE__), '..', '..', 'lib', 'mongo') + +$con = Mongo::Connection.new +$db = $con['foo'] + +class Load < Sinatra::Base + + configure do + LOGGER = Logger.new("sinatra.log") + enable :logging, :dump_errors + set :raise_errors, true + end + + get '/' do + 3.times do |n| + if (v=$db.eval("1 + #{n}")) != 1 + n + STDERR << "#{1 + n} expected but got #{v}" + raise StandardError, "#{1 + n} expected but got #{v}" + end + end + end + +end diff --git a/source/load/unicorn/unicorn.rb.template b/source/load/unicorn/unicorn.rb.template new file mode 100644 index 000000000..ede2ae8f3 --- /dev/null +++ b/source/load/unicorn/unicorn.rb.template @@ -0,0 +1,29 @@ +# set path to app that will be used to configure unicorn, +# # note the trailing slash in this example +@dir = "/home/kyle/work/10gen/ruby-driver/test/load/" + +worker_processes 10 +working_directory @dir + +preload_app true + +timeout 30 + +# Specify path to socket unicorn listens to, +# we will use this in our nginx.conf later +listen "#{@dir}tmp/sockets/unicorn.sock", :backlog => 64 + +# Set process id path +pid "#{@dir}tmp/pids/unicorn.pid" + +# # Set log file paths +stderr_path "#{@dir}log/unicorn.stderr.log" +stdout_path "#{@dir}log/unicorn.stdout.log" + +# NOTE: You need this when using forking web servers! +after_fork do |server, worker| + $con.close if $con + $con = Mongo::Connection.new + $db = $con['foo'] + STDERR << "FORKED #{server} #{worker}" +end From 56c333e06cf48393ba8938d1c277c96747c2d07f Mon Sep 17 00:00:00 2001 From: Gary Murakami Date: Tue, 16 Oct 2012 16:47:10 -0400 Subject: [PATCH 089/442] RUBY-486 Emphasize "safe" mode in documentation, explicitly in the tutorial also use :safe => true in examples and other script instances also fix typos in documentation --- source/FAQ.md | 4 ++-- source/GRID_FS.md | 4 ++-- source/HISTORY.md | 6 +++--- source/READ_PREFERENCE.md | 6 +++--- source/RELEASES.md | 2 +- source/REPLICA_SETS.md | 18 +++++++++--------- source/TAILABLE_CURSORS.md | 2 +- source/TUTORIAL.md | 12 ++++++++++-- source/examples/admin.rb | 2 +- source/examples/capped.rb | 2 +- source/examples/cursor.rb | 2 +- source/examples/gridfs.rb | 2 +- source/examples/index_test.rb | 2 +- source/examples/info.rb | 2 +- source/examples/queries.rb | 2 +- source/examples/simple.rb | 2 +- source/examples/strict.rb | 2 +- source/examples/types.rb | 2 +- source/load/thin/load.rb | 2 +- source/load/unicorn/load.rb | 2 +- source/load/unicorn/unicorn.rb.template | 2 +- 21 files changed, 44 insertions(+), 36 deletions(-) diff --git a/source/FAQ.md b/source/FAQ.md index d96b1784a..22061d671 100644 --- a/source/FAQ.md +++ b/source/FAQ.md @@ -8,7 +8,7 @@ Yes. You can run any of the [available database commands|List of Database Comman # This command is run on the admin database. - @db = Mongo::Connection.new.db('admin') + @db = Mongo::Connection.new('localhost', 27017, :safe => true).db('admin') # Build the command. cmd = OrderedHash.new @@ -111,6 +111,6 @@ Because of the indeterminacy involved, the MongoDB drivers will not retry operat The drivers will reconnect on the subsequent operation. -#### I ocassionally get an error saying that responses are out of order. What's happening? +#### I occasionally get an error saying that responses are out of order. What's happening? See (this JIRA issue)[http://jira.mongodb.org/browse/RUBY-221]. diff --git a/source/GRID_FS.md b/source/GRID_FS.md index 1ae3cbbf6..6b8b10cf3 100644 --- a/source/GRID_FS.md +++ b/source/GRID_FS.md @@ -7,7 +7,7 @@ GridFS, which stands for "Grid File Store," is a specification for storing large The [Grid class](Mongo/Grid.html) represents the core GridFS implementation. Grid gives you a simple file store, keyed on a unique ID. This means that duplicate filenames aren't a problem. To use the Grid class, first make sure you have a database, and then instantiate a Grid: - @db = Mongo::Connection.new.db('social_site') + @db = Mongo::Connection.new('localhost', 27017, :safe => true).db('social_site') @grid = Grid.new(@db) #### Saving files @@ -92,7 +92,7 @@ Deleting a file is as simple as providing the id: #### Saving files - @db = Mongo::Connection.new.db("social_site") + @db = Mongo::Connection.new('localhost', 27017, :safe => true).db("social_site") @fs = GridFileSystem.new(@db) Now suppose we want to save the file 'me.jpg.' This is easily done using a filesystem-like API: diff --git a/source/HISTORY.md b/source/HISTORY.md index 2133ce71c..860abfbf7 100644 --- a/source/HISTORY.md +++ b/source/HISTORY.md @@ -253,7 +253,7 @@ Lots of cleanup and minor bug fixes. ## 1.1 2010-10-4 -* Official JRuby support via Java extensons for BSON (beta) +* Official JRuby support via Java extensions for BSON (beta) * Connection#lock! and Connection#unlock! for easy fsync lock * Note: BSON::Code is no longer a subclass of String. @@ -265,7 +265,7 @@ Lots of cleanup and minor bug fixes. ### 1.0.8 2010-8-27 -* Cursor#rewind! and more consistent Cursor Enumberable behavior +* Cursor#rewind! and more consistent Cursor Enumerable behavior * Deprecated ObjectID for ObjectId * Numerous minor bug fixes. @@ -303,7 +303,7 @@ Lots of cleanup and minor bug fixes. * MapReduce can return raw command output using :raw * BSON::OrderedHash equality with other Ruby hashes (Ryan Angilly) * Fix for broken Socket.send with large payloads (Frédéric De Jaeger) -* Lots of minor improvements. See commmits. +* Lots of minor improvements. See commits. ### 1.0.3 2010-6-15 diff --git a/source/READ_PREFERENCE.md b/source/READ_PREFERENCE.md index 2bf05a1fc..d5069f452 100644 --- a/source/READ_PREFERENCE.md +++ b/source/READ_PREFERENCE.md @@ -24,7 +24,7 @@ The Ruby driver allows you to set read preference on each of four levels: the co Objects will inherit the default read preference from their parents. Thus, if you set a read preference of `{:read => :secondary}` when creating a new connection, then all databases and collections created from that connection will inherit the same setting. See this code example: - @con = Mongo::ReplSetConnection.new(['localhost:27017','localhost:27018'], :read => :secondary) + @con = Mongo::ReplSetConnection.new(['localhost:27017','localhost:27018'], :safe => true, :read => :secondary) @db = @con['test'] @collection = @db['foo'] @collection.find({:name => 'foo'}) @@ -92,8 +92,8 @@ Tag set keys may be symbols or strings. Tag set values should be specified using Tag sets are used in conjunction with read preference mode. In this example, because we specified a mode of secondary_preferred, if no secondaries can be found that match the tag_set `{:rack => '2'}` then the primary will be used for the query. -If only one tag set is provided, the set can be passed as a single hash parameter iteself without the enclosing array. +If only one tag set is provided, the set can be passed as a single hash parameter itself without the enclosing array. @collection.find({:name => 'foo'}, :read => :secondary_preferred, :tag_sets => {:rack => '2'}) -Specifiying tag_sets for mode `:primary` is considered an error and will raise a MongoArgumentError as tag_sets do not affect selection of primary members and only primary members can be selected in that particular mode. +Specifying tag_sets for mode `:primary` is considered an error and will raise a MongoArgumentError as tag_sets do not affect selection of primary members and only primary members can be selected in that particular mode. diff --git a/source/RELEASES.md b/source/RELEASES.md index 3d54b73f8..4340b94f5 100644 --- a/source/RELEASES.md +++ b/source/RELEASES.md @@ -44,7 +44,7 @@ Before each relese to Rubygems.org, the following steps will be taken: 11. Close out release in JIRA. -12. Annouce release on mongodb-user and mongodb-dev. +12. Announce release on mongodb-user and mongodb-dev. ## Rake Deploy Tasks 1. rake deploy:change_version[x.x.x] diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md index 4a9f3c42e..023919e36 100644 --- a/source/REPLICA_SETS.md +++ b/source/REPLICA_SETS.md @@ -6,19 +6,19 @@ Here follow a few considerations for those using the MongoDB Ruby driver with [r First, make sure that you've configured and initialized a replica set. -Use `ReplSetConnection.new` to connect to a replica set. This method, which accepts a variable number of arugments, +Use `ReplSetConnection.new` to connect to a replica set. This method, which accepts a variable number of arguments, takes a list of seed nodes followed by any connection options. You'll want to specify at least two seed nodes. This gives the driver more chances to connect in the event that any one seed node is offline. Once the driver connects, it will cache the replica set topology as reported by the given seed node and use that information if a failover is later required. - @connection = ReplSetConnection.new(['n1.mydb.net:27017', 'n2.mydb.net:27017', 'n3.mydb.net:27017']) + @connection = ReplSetConnection.new(['n1.mydb.net:27017', 'n2.mydb.net:27017', 'n3.mydb.net:27017'], :safe => true) ### Read slaves If you want to read from a secondary node, you can pass :read => :secondary to ReplSetConnection#new. @connection = ReplSetConnection.new(['n1.mydb.net:27017', 'n2.mydb.net:27017', 'n3.mydb.net:27017'], - :read => :secondary) + :safe => true, :read => :secondary) A random secondary will be chosen to be read from. In a typical multi-process Ruby application, you'll have a good distribution of reads across secondary nodes. @@ -39,7 +39,7 @@ You can now specify a refresh mode and refresh interval for a replica set connec changes to a replica set's configuration are quickly reflected on the driver side. In particular, if you change the state of any secondary node, the automated refresh will ensure that this state is recorded on the client side. -There are two secenarios in which refresh is helpful and does not raise exceptions: +There are two scenarios in which refresh is helpful and does not raise exceptions: 1. You add a new secondary node to an existing replica set 2. You remove an unused secondary from an existing replica set @@ -54,20 +54,20 @@ Refresh mode is disabled by default. However, if you expect to make live changes to your secondaries, and you want this to be reflected without having to manually restart your app server, then you should enable it. You can enable this mode synchronously, which will refresh the replica set data in a synchronous fashion (which may -ocassionally slow down your queries): +occasionally slow down your queries): - @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :refresh_mode => :sync) + @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :safe => true, :refresh_mode => :sync) If you want to change the default refresh interval of 90 seconds, you can do so like this: - @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :refresh_mode => :sync, + @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :safe => true, :refresh_mode => :sync, :refresh_interval => 60) Do not set this value to anything lower than 30, or you may start to experience performance issues. You can also disable refresh mode altogether: - @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :refresh_mode => false) + @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :safe => true, :refresh_mode => false) And you can call `refresh` manually on any replica set connection: @@ -110,4 +110,4 @@ of individual node failures. Note that the `mongod` executable must be in the se ### Further Reading * [Replica Sets](http://www.mongodb.org/display/DOCS/Replica+Set+Configuration) -* [Replics Set Configuration](http://www.mongodb.org/display/DOCS/Replica+Set+Configuration) +* [Replica Set Configuration](http://www.mongodb.org/display/DOCS/Replica+Set+Configuration) diff --git a/source/TAILABLE_CURSORS.md b/source/TAILABLE_CURSORS.md index d526ddfff..029f00e59 100644 --- a/source/TAILABLE_CURSORS.md +++ b/source/TAILABLE_CURSORS.md @@ -17,7 +17,7 @@ Note that tailable cursors are for capped collections only. def test_tailable # Create a connection and capped collection. - @con = Connection.new + @con = Connection.new('localhost', 27017, :safe => true) @db = @con['test'] @db.drop_collection('log') @capped = @db.create_collection('log', :capped => true, :size => 1024) diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md index aaa855733..3625a8de8 100644 --- a/source/TUTORIAL.md +++ b/source/TUTORIAL.md @@ -41,12 +41,20 @@ Use the `mongo` gem via the `require` kernel method. ### Making a Connection -An `Mongo::Connection` instance represents a connection to MongoDB. You can optionally specify the MongoDB server address and port when connecting. The following example shows three ways to connect to the local machine: +An `Mongo::Connection` instance represents a connection to MongoDB. + + connection = Mongo::Connection.new("localhost", 27017, :safe => true) + +The option :safe => true is highly recommended. This specifies that the driver sends a getlasterror command after every update to ensure that the update succeeded. + +You can optionally specify the MongoDB server address and port when connecting. The following example shows three ways to connect to the local machine: connection = Mongo::Connection.new # (optional host/port args) connection = Mongo::Connection.new("localhost") connection = Mongo::Connection.new("localhost", 27017) +In these cases, the :safe option defaults to false, and updates are fire-and-forget with higher performance. We do not recommend the default :safe => false unless your application can tolerate the potential loss of updates. + ### Listing All Databases connection.database_names @@ -59,7 +67,7 @@ The `database_info` method returns a hash mapping database names to the size of You use a Connection instance to obtain an Mongo::DB instance, which represents a named database. The database doesn't have to exist - if it doesn't, MongoDB will create it for you. The following examples use the database "mydb": db = connection.db("mydb") - db = Mongo::Connection.new.db("mydb") + db = Mongo::Connection.new("localhost", 27017, :safe => true).db("mydb") At this point, the `db` object will be a connection to a MongoDB server for the specified database. Each DB instance uses a separate socket connection to the server. diff --git a/source/examples/admin.rb b/source/examples/admin.rb index e445be9ac..580f7bf6f 100644 --- a/source/examples/admin.rb +++ b/source/examples/admin.rb @@ -9,7 +9,7 @@ port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts "Connecting to #{host}:#{port}" -con = Mongo::Connection.new(host, port) +con = Mongo::Connection.new(host, port, :safe => true) db = con.db('ruby-mongo-examples') coll = db.create_collection('test') diff --git a/source/examples/capped.rb b/source/examples/capped.rb index 6cdf07528..ff63a5ebe 100644 --- a/source/examples/capped.rb +++ b/source/examples/capped.rb @@ -7,7 +7,7 @@ port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port).db('ruby-mongo-examples') +db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') db.drop_collection('test') # A capped collection has a max size and, optionally, a max number of records. diff --git a/source/examples/cursor.rb b/source/examples/cursor.rb index 4bf0fe89c..1dae16757 100644 --- a/source/examples/cursor.rb +++ b/source/examples/cursor.rb @@ -9,7 +9,7 @@ port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port).db('ruby-mongo-examples') +db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') coll = db.collection('test') # Erase all records from collection, if any diff --git a/source/examples/gridfs.rb b/source/examples/gridfs.rb index a1769f782..308eee658 100644 --- a/source/examples/gridfs.rb +++ b/source/examples/gridfs.rb @@ -10,7 +10,7 @@ def assert port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port).db('ruby-mongo-examples') +db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') data = "hello, world!" diff --git a/source/examples/index_test.rb b/source/examples/index_test.rb index c9398a8b0..eb77fe07c 100644 --- a/source/examples/index_test.rb +++ b/source/examples/index_test.rb @@ -8,7 +8,7 @@ port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts ">> Connecting to #{host}:#{port}" -db = Connection.new(host, port).db('ruby-mongo-index_test') +db = Connection.new(host, port, :safe => true).db('ruby-mongo-index_test') class Exception def errmsg diff --git a/source/examples/info.rb b/source/examples/info.rb index 7790e1992..3737fc9e7 100644 --- a/source/examples/info.rb +++ b/source/examples/info.rb @@ -8,7 +8,7 @@ port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port).db('ruby-mongo-examples') +db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') coll = db.collection('test') # Erase all records from collection, if any diff --git a/source/examples/queries.rb b/source/examples/queries.rb index 8e33c9db4..79d9104bc 100644 --- a/source/examples/queries.rb +++ b/source/examples/queries.rb @@ -9,7 +9,7 @@ port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port).db('ruby-mongo-examples') +db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') coll = db.collection('test') # Remove all records, if any diff --git a/source/examples/simple.rb b/source/examples/simple.rb index 4850055e9..907cda56b 100644 --- a/source/examples/simple.rb +++ b/source/examples/simple.rb @@ -9,7 +9,7 @@ port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port).db('ruby-mongo-examples') +db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') coll = db.collection('test') # Erase all records from collection, if any diff --git a/source/examples/strict.rb b/source/examples/strict.rb index 5718c7e9c..9ad80231f 100644 --- a/source/examples/strict.rb +++ b/source/examples/strict.rb @@ -8,7 +8,7 @@ port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port).db('ruby-mongo-examples') +db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') db.drop_collection('does-not-exist') db.create_collection('test') diff --git a/source/examples/types.rb b/source/examples/types.rb index 0589463fa..3ca19bd8c 100644 --- a/source/examples/types.rb +++ b/source/examples/types.rb @@ -9,7 +9,7 @@ port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port).db('ruby-mongo-examples') +db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') coll = db.collection('test') # Remove all records, if any diff --git a/source/load/thin/load.rb b/source/load/thin/load.rb index f97ee83a2..9d471b658 100644 --- a/source/load/thin/load.rb +++ b/source/load/thin/load.rb @@ -1,7 +1,7 @@ require File.join(File.dirname(__FILE__), '..', '..', '..', 'lib', 'mongo') require 'logger' -$con = Mongo::ReplSetConnection.new(['localhost:30000', 'localhost:30001'], :read => :secondary, :refresh_mode => :sync, :refresh_interval => 30) +$con = Mongo::ReplSetConnection.new(['localhost:30000', 'localhost:30001'], :safe => true, :read => :secondary, :refresh_mode => :sync, :refresh_interval => 30) $db = $con['foo'] class Load < Sinatra::Base diff --git a/source/load/unicorn/load.rb b/source/load/unicorn/load.rb index f624cb10d..83da34a26 100644 --- a/source/load/unicorn/load.rb +++ b/source/load/unicorn/load.rb @@ -1,6 +1,6 @@ require File.join(File.dirname(__FILE__), '..', '..', 'lib', 'mongo') -$con = Mongo::Connection.new +$con = Mongo::Connection.new('localhost', 27017, :safe => true) $db = $con['foo'] class Load < Sinatra::Base diff --git a/source/load/unicorn/unicorn.rb.template b/source/load/unicorn/unicorn.rb.template index ede2ae8f3..333df2f6e 100644 --- a/source/load/unicorn/unicorn.rb.template +++ b/source/load/unicorn/unicorn.rb.template @@ -23,7 +23,7 @@ stdout_path "#{@dir}log/unicorn.stdout.log" # NOTE: You need this when using forking web servers! after_fork do |server, worker| $con.close if $con - $con = Mongo::Connection.new + $con = Mongo::Connection.new('localhost', 27017, :safe => true) $db = $con['foo'] STDERR << "FORKED #{server} #{worker}" end From 242bf59102720ba69ad656b3ca1f4e649e8ad423 Mon Sep 17 00:00:00 2001 From: Tyler Brock Date: Wed, 17 Oct 2012 00:36:13 -0400 Subject: [PATCH 090/442] RUBY-474 move docs content to wiki and examples up --- source/CREDITS.md | 123 -------- source/FAQ.md | 116 ------- source/GRID_FS.md | 158 ---------- source/HISTORY.md | 392 ------------------------ source/READ_PREFERENCE.md | 99 ------ source/RELEASES.md | 54 ---- source/REPLICA_SETS.md | 113 ------- source/TAILABLE_CURSORS.md | 51 --- source/TUTORIAL.md | 364 ---------------------- source/WRITE_CONCERN.md | 31 -- source/examples/admin.rb | 43 --- source/examples/capped.rb | 22 -- source/examples/cursor.rb | 48 --- source/examples/gridfs.rb | 44 --- source/examples/index_test.rb | 126 -------- source/examples/info.rb | 31 -- source/examples/queries.rb | 74 ----- source/examples/replica_set.rb | 24 -- source/examples/simple.rb | 25 -- source/examples/strict.rb | 35 --- source/examples/types.rb | 36 --- source/load/thin/config.ru | 6 - source/load/thin/config.yml.template | 6 - source/load/thin/load.rb | 21 -- source/load/unicorn/config.ru | 6 - source/load/unicorn/load.rb | 23 -- source/load/unicorn/unicorn.rb.template | 29 -- 27 files changed, 2100 deletions(-) delete mode 100644 source/CREDITS.md delete mode 100644 source/FAQ.md delete mode 100644 source/GRID_FS.md delete mode 100644 source/HISTORY.md delete mode 100644 source/READ_PREFERENCE.md delete mode 100644 source/RELEASES.md delete mode 100644 source/REPLICA_SETS.md delete mode 100644 source/TAILABLE_CURSORS.md delete mode 100644 source/TUTORIAL.md delete mode 100644 source/WRITE_CONCERN.md delete mode 100644 source/examples/admin.rb delete mode 100644 source/examples/capped.rb delete mode 100644 source/examples/cursor.rb delete mode 100644 source/examples/gridfs.rb delete mode 100644 source/examples/index_test.rb delete mode 100644 source/examples/info.rb delete mode 100644 source/examples/queries.rb delete mode 100644 source/examples/replica_set.rb delete mode 100644 source/examples/simple.rb delete mode 100644 source/examples/strict.rb delete mode 100644 source/examples/types.rb delete mode 100644 source/load/thin/config.ru delete mode 100644 source/load/thin/config.yml.template delete mode 100644 source/load/thin/load.rb delete mode 100644 source/load/unicorn/config.ru delete mode 100644 source/load/unicorn/load.rb delete mode 100644 source/load/unicorn/unicorn.rb.template diff --git a/source/CREDITS.md b/source/CREDITS.md deleted file mode 100644 index 0d4dde5cc..000000000 --- a/source/CREDITS.md +++ /dev/null @@ -1,123 +0,0 @@ -# Credits - -Adrian Madrid, aemadrid@gmail.com - -* bin/mongo_console -* examples/benchmarks.rb -* examples/irb.rb -* Modifications to examples/simple.rb -* Found plenty of bugs and missing features. -* Ruby 1.9 support. -* Gem support. -* Many other code suggestions and improvements. - -Aman Gupta, aman@tmm1.net - -* Collection#save -* Noted bug in returning query batch size. - -Jon Crosby, jon@joncrosby.me - -* Some code clean-up - -John Nunemaker, http://railstips.org - -* Collection#create_index takes symbols as well as strings -* Fix for Collection#save -* Add logger convenience methods to connection and database - -David James, djames@sunlightfoundation.com - -* Fix dates to return as UTC - -Paul Dlug, paul.dlug@gmail.com - -* Generate _id on the client side if not provided -* Collection#insert and Collection#save return _id - -Durran Jordan, durran@gmail.com - -* DB#collections -* Support for specifying sort order as array of [key, direction] pairs -* OrderedHash#update aliases to merge! - -Cyril Mougel, cyril.mougel@gmail.com - -* Initial logging support -* Test case - -Jack Chen, chendo on github - -* Test case + fix for deserializing pre-epoch Time instances - -Michael Bernstein, mrb on github - -* Cursor#sort - -Paulo Ahahgon, pahagon on github - -* removed hard limit - -Les Hill, leshill on github - -* OrderedHash#each returns self - -Sean Cribbs, seancribbs on github - -* Modified standard_benchmark to allow profiling -* C ext for faster ObjectID creation - -Sunny Hirai - -* Suggested hashcode fix for Mongo::ObjectID -* Noted index ordering bug. -* GridFS performance boost - -Christos Trochalakis - -* Added map/reduce helper - -Blythe Dunham - -* Added finalize option to map/reduce - -Matt Powell (fauxparse) - -* Added GridStore#mv - -Patrick Collison - -* Added safe mode for Collection#remove - -Chuck Remes - -* Extraction of BSON into separate gems -* Extensions compile on Rubinius -* Performance improvements for INT in C extensions -* Performance improvements for JRuby BSON encoder and callback classes - -Dmitrii Golub (Houdini) and Jacques Crocker (railsjedi) - -* Support open to exclude fields on query - -dfitzgibbon - -* patch for ensuring bson_ext compatibility with early release of Ruby 1.8.5 - -Matt Taylor - -* Noticed excessive calls to ObjectId#to_s. As a result, stopped creating -log messages when no logger was passed to Mongo::Connection. Resulted in a significant -performance improvement. - -Hongli Lai (Phusion) - -* Significant performance improvements. See commits. - -Mislav Marohnić - -* Replaced returning with each_with_object - -Alex Stupka - -* Replica set port bug diff --git a/source/FAQ.md b/source/FAQ.md deleted file mode 100644 index 22061d671..000000000 --- a/source/FAQ.md +++ /dev/null @@ -1,116 +0,0 @@ -# Ruby MongoDB FAQ - -This is a list of frequently asked questions about using Ruby with MongoDB. If you have a question you'd like to have answered here, please post your question to the [mongodb-user list](http://groups.google.com/group/mongodb-user). - -#### Can I run (insert command name here) from the Ruby driver? - -Yes. You can run any of the [available database commands|List of Database Commands] from the driver using the DB#command method. The only trick is to use an OrderedHash when specifying the command. For example, here's how you'd run an asynchronous fsync from the driver: - - - # This command is run on the admin database. - @db = Mongo::Connection.new('localhost', 27017, :safe => true).db('admin') - - # Build the command. - cmd = OrderedHash.new - cmd['fsync'] = 1 - cmd['async'] = true - - # Run it. - @db.command(cmd) - - -It's important to keep in mind that some commands, like `fsync`, must be run on the `admin` database, while other commands can be run on any database. If you're having trouble, check the [command reference|List of Database Commands] to make sure you're using the command correctly. - -#### Does the Ruby driver support an EXPLAIN command? - -Yes. `explain` is, technically speaking, an option sent to a query that tells MongoDB to return an explain plan rather than the query's results. You can use `explain` by constructing a query and calling explain at the end: - - - @collection = @db['users'] - result = @collection.find({:name => "jones"}).explain - - -The resulting explain plan might look something like this: - - - {"cursor"=>"BtreeCursor name_1", - "startKey"=>{"name"=>"Jones"}, - "endKey"=>{"name"=>"Jones"}, - "nscanned"=>1.0, - "n"=>1, - "millis"=>0, - "oldPlan"=>{"cursor"=>"BtreeCursor name_1", - "startKey"=>{"name"=>"Jones"}, - "endKey"=>{"name"=>"Jones"} - }, - "allPlans"=>[{"cursor"=>"BtreeCursor name_1", - "startKey"=>{"name"=>"Jones"}, - "endKey"=>{"name"=>"Jones"`] - } - - -Because this collection has an index on the "name" field, the query uses that index, only having to scan a single record. "n" is the number of records the query will return. "millis" is the time the query takes, in milliseconds. "oldPlan" indicates that the query optimizer has already seen this kind of query and has, therefore, saved an efficient query plan. "allPlans" shows all the plans considered for this query. - -#### I see that BSON supports a symbol type. Does this mean that I can store Ruby symbols in MongoDB? - -You can store Ruby symbols in MongoDB, but only as values. BSON specifies that document keys must be strings. So, for instance, you can do this: - - - @collection = @db['test'] - - boat_id = @collection.save({:vehicle => :boat}) - car_id = @collection.save({"vehicle" => "car"}) - - @collection.find_one('_id' => boat_id) - {"_id" => ObjectID('4bb372a8238d3b5c8c000001'), "vehicle" => :boat} - - - @collection.find_one('_id' => car_id) - {"_id" => ObjectID('4bb372a8238d3b5c8c000002'), "vehicle" => "car"} - - -Notice that the symbol values are returned as expected, but that symbol keys are treated as strings. - -#### Why can't I access random elements within a cursor? - -MongoDB cursors are designed for sequentially iterating over a result set, and all the drivers, including the Ruby driver, stick closely to this directive. Internally, a Ruby cursor fetches results in batches by running a MongoDB `getmore` operation. The results are buffered for efficient iteration on the client-side. - -What this means is that a cursor is nothing more than a device for returning a result set on a query that's been initiated on the server. Cursors are not containers for result sets. If we allow a cursor to be randomly accessed, then we run into issues regarding the freshness of the data. For instance, if I iterate over a cursor and then want to retrieve the cursor's first element, should a stored copy be returned, or should the cursor re-run the query? If we returned a stored copy, it may not be fresh. And if the the query is re-run, then we're technically dealing with a new cursor. - -To avoid those issues, we're saying that anyone who needs flexible access to the results of a query should store those results in an array and then access the data as needed. - -#### Why can't I save an instance of TimeWithZone? - -MongoDB stores times in UTC as the number of milliseconds since the epoch. This means that the Ruby driver serializes Ruby Time objects only. While it would certainly be possible to serialize a TimeWithZone, this isn't preferable since the driver would still deserialize to a Time object. - -All that said, if necessary, it'd be easy to write a thin wrapper over the driver that would store an extra time zone attribute and handle the serialization/deserialization of TimeWithZone transparently. - -#### I keep getting CURSOR_NOT_FOUND exceptions. What's happening? - -The most likely culprit here is that the cursor is timing out on the server. Whenever you issue a query, a cursor is created on the server. Cursor naturally time out after ten minutes, which means that if you happen to be iterating over a cursor for more than ten minutes, you risk a CURSOR_NOT_FOUND exception. - -There are two solutions to this problem. You can either: - -1. Limit your query. Use some combination of `limit` and `skip` to reduce the total number of query results. This will, obviously, bring down the time it takes to iterate. - -2. Turn off the cursor timeout. To do that, invoke `find` with a block, and pass `:timeout => true`: - - @collection.find({}, :timeout => false) do |cursor| - cursor.each do |document - # Process documents here - end - end - -#### I periodically see connection failures between the driver and MongoDB. Why can't the driver retry the operation automatically? - -A connection failure can indicate any number of failure scenarios. Has the server crashed? Are we experiencing a temporary network partition? Is there a bug in our ssh tunnel? - -Without further investigation, it's impossible to know exactly what has caused the connection failure. Furthermore, when we do see a connection failure, it's impossible to know how many operations prior to the failure succeeded. Imagine, for instance, that we're using safe mode and we send an `$inc` operation to the server. It's entirely possible that the server has received the `$inc` but failed on the call to `getLastError`. In that case, retrying the operation would result in a double-increment. - -Because of the indeterminacy involved, the MongoDB drivers will not retry operations on connection failure. How connection failures should be handled is entirely dependent on the application. Therefore, we leave it to the application developers to make the best decision in this case. - -The drivers will reconnect on the subsequent operation. - -#### I occasionally get an error saying that responses are out of order. What's happening? - -See (this JIRA issue)[http://jira.mongodb.org/browse/RUBY-221]. diff --git a/source/GRID_FS.md b/source/GRID_FS.md deleted file mode 100644 index 6b8b10cf3..000000000 --- a/source/GRID_FS.md +++ /dev/null @@ -1,158 +0,0 @@ -# GridFS in Ruby - -GridFS, which stands for "Grid File Store," is a specification for storing large files in MongoDB. It works by dividing a file into manageable chunks and storing each of those chunks as a separate document. GridFS requires two collections to achieve this: one collection stores each file's metadata (e.g., name, size, etc.) and another stores the chunks themselves. If you're interested in more details, check out the [GridFS Specification](http://www.mongodb.org/display/DOCS/GridFS+Specification). - -### The Grid class - -The [Grid class](Mongo/Grid.html) represents the core GridFS implementation. Grid gives you a simple file store, keyed on a unique ID. This means that duplicate filenames aren't a problem. To use the Grid class, first make sure you have a database, and then instantiate a Grid: - - - @db = Mongo::Connection.new('localhost', 27017, :safe => true).db('social_site') - @grid = Grid.new(@db) - -#### Saving files -Once you have a Grid object, you can start saving data to it. The data can be either a string or an IO-like object that responds to a #read method: - - - # Saving string data - id = @grid.put("here's some string / binary data") - - # Saving IO data and including the optional filename - image = File.open("me.jpg") - id2 = @grid.put(image, :filename => "me.jpg") - - -Grid#put returns an object id, which you can use to retrieve the file: - - - # Get the string we saved - file = @grid.get(id) - - # Get the file we saved - image = @grid.get(id2) - - -#### File metadata - -There are accessors for the various file attributes: - - - image.filename - # => "me.jpg" - - image.content_type - # => "image/jpg" - - image.file_length - # => 502357 - - image.upload_date - # => Mon Mar 01 16:18:30 UTC 2010 - - # Read all the image's data at once - image.read - - # Read the first 100k bytes of the image - image.read(100 * 1024) - - -When putting a file, you can set many of these attributes and write arbitrary metadata: - - - # Saving IO data - file = File.open("me.jpg") - id2 = @grid.put(file, - :filename => "my-avatar.jpg" - :content_type => "application/jpg", - :_id => 'a-unique-id-to-use-in-lieu-of-a-random-one', - :chunk_size => 100 * 1024, - :metadata => {'description' => "taken after a game of ultimate"}) - - -#### Safe mode - -A kind of safe mode is built into the GridFS specification. When you save a file, and MD5 hash is created on the server. If you save the file in safe mode, an MD5 will be created on the client for comparison with the server version. If the two hashes don't match, an exception will be raised. - - - image = File.open("me.jpg") - id2 = @grid.put(image, "my-avatar.jpg", :safe => true) - - -#### Deleting files - -Deleting a file is as simple as providing the id: - - - @grid.delete(id2) - - -### The GridFileSystem class - -[GridFileSystem](Mongo/GridFileSystem.html) is a light emulation of a file system and therefore has a couple of unique properties. The first is that filenames are assumed to be unique. The second, a consequence of the first, is that files are versioned. To see what this means, let's create a GridFileSystem instance: - -#### Saving files - - @db = Mongo::Connection.new('localhost', 27017, :safe => true).db("social_site") - @fs = GridFileSystem.new(@db) - -Now suppose we want to save the file 'me.jpg.' This is easily done using a filesystem-like API: - - - image = File.open("me.jpg") - @fs.open("me.jpg", "w") do |f| - f.write image - end - - -We can then retrieve the file by filename: - - - image = @fs.open("me.jpg", "r") {|f| f.read } - - -No problems there. But what if we need to replace the file? That too is straightforward: - - - image = File.open("me-dancing.jpg") - @fs.open("me.jpg", "w") do |f| - f.write image - end - - -But a couple things need to be kept in mind. First is that the original 'me.jpg' will be available until the new 'me.jpg' saves. From then on, calls to the #open method will always return the most recently saved version of a file. But, and this the second point, old versions of the file won't be deleted. So if you're going to be rewriting files often, you could end up with a lot of old versions piling up. One solution to this is to use the :delete_old options when writing a file: - - - image = File.open("me-dancing.jpg") - @fs.open("me.jpg", "w", :delete_old => true) do |f| - f.write image - end - - -This will delete all but the latest version of the file. - - -#### Deleting files - -When you delete a file by name, you delete all versions of that file: - - - @fs.delete("me.jpg") - - -#### Metadata and safe mode - -All of the options for storing metadata and saving in safe mode are available for the GridFileSystem class: - - - image = File.open("me.jpg") - @fs.open('my-avatar.jpg', w, - :content_type => "application/jpg", - :metadata => {'description' => "taken on 3/1/2010 after a game of ultimate"}, - :_id => 'a-unique-id-to-use-instead-of-the-automatically-generated-one', - :safe => true) { |f| f.write image } - - -### Advanced Users - -Astute code readers will notice that the Grid and GridFileSystem classes are merely thin wrappers around an underlying [GridIO class](Mongo/GridIO.html). This means that it's easy to customize the GridFS implementation presented here; just use GridIO for all the low-level work, and build the API you need in an external manager class similar to Grid or GridFileSystem. - diff --git a/source/HISTORY.md b/source/HISTORY.md deleted file mode 100644 index 860abfbf7..000000000 --- a/source/HISTORY.md +++ /dev/null @@ -1,392 +0,0 @@ -# MongoDB Ruby Driver History - -### 1.7.0 -2012-08-20 - -* Added testing and full support for MongoDB 2.1 & 2.2 -* Added Aggregation Framework helper method -* Added support for Mongos high availability -* Modified and added new read preferences (details in documentation) -* Added support for data center awareness (tag_sets) -* Fixed bug which attempted to close cursors on wrong replica set member - -### 1.6.4 -2012-06-06 - -* Added ability to declare sort ordering via an ordered hash -* Addresses major compatability issue with mongoid created by v1.6.3 - -### 1.6.3 -2012-06-05 - -* Performance measurements and enhancements (especially for C-extensions) -* Bug fixes for checking strings with non UTF-8 forced or implied encodings -* Added refresh support for multiple threaded instances of ReplSetConnection -* Added ability to handle IRB::Abort Exception (ctrl-c) cleanly -* Added support for large dates on 32-bit platforms (Ruby 1.9+) -* Added #to_ary method for BSON::ObjectId (Farrel Lifson) -* Added support for ENV['MONGODB_URI'] (Seamus Abshere) -* Various gridio bug fixes (John Bintz) -* Various logging support improvements -* Various documentation improvements (tutorials, sorting, links) - -### 1.6.2 -2012-04-05 - -* Implements socket timeouts via non-blocking IO instead of Timeout module -which should greatly increase performance in highly threaded applications -* Added ability to authentication via secondary if primary node unavailable -* Replica set refresh interval now enforces a lower bound of 60 seconds -* Added documentation for dropping indexes, collections, databases -* Test output cleanup (...)s unless failure occurs - -### 1.6.1 -2012-03-07 - -* Added thread affinity to Mongo::Pool -* Added deploy tasks -* Added Travis CI support (Cyril Mougel) -* Logging warning message is only displayed for level :debug - -### 1.6.0 -2012-02-22 - -* Added Gemfile -* ReplSetConnection seed format is now array of 'host:port' strings -* Added read preference :secondary_only -* Added ability to log duration -- enabled by default (Cyril Mougel) -* Added read_only option for DB#add_user (Ariel Salomon) -* Added :collect_on_error option for bulk-insert (Masahiro Nakagawa) -* Added and updated URI options (now case insensitive) -* Bug fix for ReplSet refresh attempting to close a closed socket -* Default op_timeout for ReplSetConnection is now disabled (was 30 seconds) -* Support db output option for map reduce (John Ewart) -* Support for keeping limited versions of files using GridFS (VvanGemert) - -### 1.5.2 -2011-12-13 - -* Lots of fixes for replica set connection edge cases. -* Set default op_timeout and connect_timeout to 30 seconds. -* Support GeoHaystack indexing. - -### 1.5.1 -2011-11-29 - -Release due to corrupted gemspec. This was a bug having -to do with rubygems. Apparently, gems must still be -built with Ruby 1.8. - -### 1.5.0 -2011-11-28 - -This releases fixes bugs introduced in 1.4.0 and 1.4.1 that -were introduced as a result of adding replica set refresh modes. - -* Removed :async refresh mode. -* Disabled auto refresh mode by default. If you want the driver -to automatically check the state of the replica set, you must -use :sync mode. Note that replica set refresh is designed only to -account for benign changes to the replica set (adding and removing -nodes that don't affect current connections). -* Fixed bug with commands being sent to secondary nodes. The next -release will allow you to specify where commands can be sent. -* Support :j safe mode option. -* Fix :max_scan and :show_disk_loc Cursor options. - -You can see the remaining issues at https://jira.mongodb.org/secure/ReleaseNote.jspa?projectId=10005&version=10992 - -### 1.5.0.rc0 -2011-11-18 - -Fix bugs associated with replica set refresh. - -### 1.4.1 -2011-10-17 - -If you're using 1.4.0, this is a necessary upgrade. - -* Simplified replica set refresh. -* Fix bugs associated with replica set refresh. -* Make cursor smart enough to continue functioning -even if a refresh is triggered. - -### 1.4.0 -2011-9-19 - -* Attempt to automatically refresh internal replica set state using ReplSetConnection#refresh. -* Two automated refresh modes: :async and :sync. Automated refresh can also be disabled. -* Choose secondary for reads based on ping time. -* Read preference API: specify whether queries should go to primary or secondary on a per-query basis. -* Pass :require_primary => false to ReplSetConnection to connect without requiring a primary node. -* Enable exhaust-mode queries with OP_QUERY_EXHAUST. -* Collection#count takes a query selector. -* Support continue_on_error flag for bulk inserts (use :continue_on_error => true) -* Add Cursor#add_option. Deprecate Cursor#query_opts and replace with Cursor#options. -* Initial SSL support (connect with :ssl => true) -* Update to latest Java driver for JRuby. -* Check max BSON size on a per-connection basis. -* Fixed two platform-specific BSON serialization issues. -* Lots of bug fixes and code cleanup. - -### 1.3.1 -2011-5-10 - -* Fix GridIO#gets infinite loop error (Ryan McGeary) -* Fix BSON::OrderedHash#reject! leaving keys with null values (rpt. by Ben Poweski) -* Minor semantic fix for OrderedHash#reject! -* Fix Mongo::DB to allow symbols in method traversing collection names (rpt. by Chris Griego) -* Support new server regex option "s" (dotall). This is folded in with \m in Ruby. -* Fix so that Cursor#close hits the right node when :read_secondary is enabled. -* Support maxScan, showDiskLoc, and returnKey cursor options. -* Make DB#validate_collection compatible with server v1.9.1. -* Fix so that GridIO#gets returns local md5 with md5 matches server md5 (Steve Tantra). -* Fix bug in BSON::OrderedHash that prevents YAML.load (Ian Warshak). -* Fix example from /examples. -* Ensure that we do not modify hash arguments by calling Hash#dup when appropriate. -* Ensure that JRuby deserializer preserves binary subtypes properly. -* Fix for streaming an empty file into GridFS (Daniël van de Burgt). -* Minor doc fixes. - -### 1.3.0 -2011-4-04 - -* Add option to set timeouts on socket read calls using the - Mongo::Connection :op_timeout option. -* Add StringIO methods to GridIO objects -* Support for BSON timestamp type with BSON::Timestamp -* Change the BSON binary subtype from 2 to 0 -* Remove private method Connection#reset_conection - and deprecate public method ReplSetConnection#reset_connection -* ByteBuffer#== and OrderedHash#dup (Hongli Lai) -* Better check for UTF8 validity in Ruby 1.9 -* Added previously removed Connection#host and Connection#port -* Added transformers to allow Mongo::Cursor to allow instantiated objects (John Nunemaker) -* Automated reconnection on fork -* Added Cursor#next alias for Cursor#next_document -* Audit tests after enabling warnings (Wojciech Piekutowski) -* Various bug fixes thanks to Datanoise, Hongli Lai, and Mauro Pompilio - -### 1.2.4 -2011-2-23 - -* Fix the exception message shown when there's an IOError (Mauro Pompilio) -* Another update to map-reduce docs for v1.8. Note that if you use the new - output option `{:out => {:inline => true}}`, then you must also specify - `:raw => true`. - -### 1.2.3 -2011-2-22 - -* Update docs for map-reduce command -* Minor doc fix - -### 1.2.2 -2011-2-15 - -* Improved replica set failover for edge case. -* Fix for REE on OSX (Hongli Lai) - -### 1.2.1 -2011-1-18 - -* Enable authentication with connection pooling. -* Allow custom logging with Connection#instrument (CodeMonkeySteve) -* Minor fixes and doc improvements. - -### 1.2.0 -2011-1-18 - -* Some minor improvements. See commit history. - -### 1.2.rc0 -2011-1-5 - -Lots of cleanup and minor bug fixes. -* Issues resolved: http://jira.mongodb.org/browse/RUBY/fixforversion/10222 -* Updated Java BSON to Java driver 2.4. -* Platform gem for JRuby bson. - -### 1.1.5 -2010-12-15 - -* ReplSetConnection class. This must be used for replica set connections from - now on. You can still use Connection.multi, but that method has been deprecated. -* Automated replica set tests. rake test:rs -* Check that request and response ids match. -* Several bug fixes. See the commit history for details. - -### 1.1.4 -2010-11-30 - -* Important connection failure fix. -* ObjectId#to_s optimization (David Cuadrado). - -### 1.1.3 -2010-11-29 - -* Distributed reads for replica set secondaries. See /docs/examples/replica_set.rb and - http://api.mongodb.org/ruby/current/file.REPLICA_SETS.html for details. -* Note: when connecting to a replica set, you must use Connection#multi. -* Cursor#count takes optional skip and limit -* Collection#ensure_index for caching index creation calls -* Collection#update and Collection#remove now return error object when using safe mode -* Important fix for int/long serialization on bug introduced in 1.0.9 -* Numerous tweaks and bug fixes. - -### 1.1.2 -2010-11-4 - -* Two critical fixes to automated failover and replica sets. -* Bug passing :timeout to Cursor. -* Permit safe mode specification on Connection, Collection, and DB levels. -* Specify replica set name on connect to verify connection to the right set. -* Misc. reorganization of project and docs. - -### 1.1.1 -2010-10-14 - -* Several critical JRuby bug fixes -* Fixes for JRuby in 1.9 mode -* Check keys and move id only when necessary for JRuby encoder - -## 1.1 -2010-10-4 - -* Official JRuby support via Java extensions for BSON (beta) -* Connection#lock! and Connection#unlock! for easy fsync lock -* Note: BSON::Code is no longer a subclass of String. - -### 1.0.9 -2010-9-20 - -* Significant performance improvements (with a lot of help from Hongli Lai) - -### 1.0.8 -2010-8-27 - -* Cursor#rewind! and more consistent Cursor Enumerable behavior -* Deprecated ObjectID for ObjectId -* Numerous minor bug fixes. - -### 1.0.7 -2010-8-4 - -* A few minor test/doc fixes. -* Better tests for replica sets and replication acknowledgment. -* Deprecated DB#error and DB#last_status - -### 1.0.6 -2010-7-26 - -* Replica set support. -* Collection#map_reduce bug fix. - -### 1.0.5 -2010-7-13 - -* Fix for bug introduced in 1.0.4. - -### 1.0.4 -2010-7-13 - -* Removed deprecated - * Cursor admin option - * DB#query - * DB#create_index (use Collection#create_index) - * DB#command only takes hash options now -* j2bson executable (neomantra) -* Fixed bson_ext compilation on Solaris (slyphon) -* System JS helpers (neovintage) -* Use one mutex per thread on pooled connections (cremes) -* Check for CursorNotFound response flag -* MapReduce can return raw command output using :raw -* BSON::OrderedHash equality with other Ruby hashes (Ryan Angilly) -* Fix for broken Socket.send with large payloads (Frédéric De Jaeger) -* Lots of minor improvements. See commits. - -### 1.0.3 -2010-6-15 - -* Optimiztion for BSON::OrderedHash -* Some important fixes. - -### 1.0.2 -2010-6-5 - -This is a minor release for fixing an incompatibility with MongoDB v1.5.2 - -* Fix for boolean response on commands for core server v1.5.2 -* BSON.read_bson_document and b2json executable (neomantra) -* BSON::ObjectID() shortcut for BSON::ObjectID.from_string (tmm1) -* Various bug fixes. - -### 1.0.1 -2010-5-7 - -* set Encoding.default_internal -* DEPRECATE JavaScript string on Collection#find. You now must specify $where explicitly. -* Added Grid#exist? and GridFileSystem#exist? -* Support for replication acknowledgment -* Support for $slice -* Namespaced OrderedHash under BSON (sleverbor) - -## 1.0 -2010-4-29 -Note: if upgrading from versions prior to 0.20, be sure to upgrade -to 0.20 before upgrading to 1.0. - -* Inspected ObjectID is represented in MongoDB extended json format. -* Support for tailable cursors. -* Configurable query response batch size (thx. to Aman Gupta) - -* bson_ext installs on early release of Ruby 1.8.5 (dfitzgibbon) -* Deprecated DB#create_index. Use Collection#create_index index. -* Removed deprecated Grid#put syntax; no longer requires a filename. - -### 0.20.1 -2010-4-7 - -* Added bson gem dependency. - -### 0.20 -2010-4-7 - -If upgrading from a previous version of the Ruby driver, please read these notes carefully, -along with the 0.20_UPGRADE doc. - -* Support for new commands: - * Collection#find_and_modify - * Collection#stats - * DB#stats -* Query :fields options allows for values of 0 to exclude fields (houdini, railsjedi). -* GridFS - * Option to delete old versions of GridFileSystem entries. - * Filename is now optional for Grid#put. - * Option to write arbitrary attributes to a file: @grid.put(@data, :favorite_phrase => "blimey!") - * Indexes created on the chunks collection are now unique. If you have an existing chunks collection, - you may want to remove -* Removed the following deprecated items: - * GridStore class - * RegexpOfHolding class - * Paired connections must now be initialized with Connection.paired - -* BSON-related code extracted into two separate gems: bson and bson_ext (thx to Chuck Remes). - * mongo_ext no longer exists. - * BSON::Binary constructor can now take a string, which will be packed into an array. - * Exception class adjustments: - * Mongo::InvalidObjectID moved to BSON::InvalidObjectID - * Mongo::InvalidDocument moved to BSON::InvalidDocument - * Mongo::InvalidStringEncoding moved to BSON::InvalidStringEncoding - * Mongo::InvalidName replaced by Mongo::InvalidNSName and BSON::InvalidKeyName - * BSON types are now namespaced under the BSON module. These types include: - * Binary - * ObjectID - * Code - * DBRef - * MinKey and MaxKey - * Extensions compile on Rubinius (Chuck Remes). - -## Prior to 0.20 - -See git revisions. diff --git a/source/READ_PREFERENCE.md b/source/READ_PREFERENCE.md deleted file mode 100644 index d5069f452..000000000 --- a/source/READ_PREFERENCE.md +++ /dev/null @@ -1,99 +0,0 @@ -# Read Preference in Ruby - -## About Read Preference - -Read preferences determine the candidate replica set members to which a query or command can be sent. They consist of a *mode* specified as a symbol and an array of hashes known as *tag_sets*. - -Read preference mode is configured by providing the read option to a connection, database, collection, or cursor. - - @collection.find({:doc => 'foo'}, :read => :primary) # read from primary only - @collection.find({:doc => 'foo'}, :read => :secondary) # read from secondaries only - -Used in conjunction with tag_sets: - - @collection.find({:name => 'foo'}, :read => :secondary_preferred, :tag_sets => [{:continent => 'USA'}]) - -*Please Note*: Behavior of some read preference modes have changed in version 1.7.0: - -* `:secondary_preferred` mode is now used to prefer reads from secondary members (before this was the behavior of `:secondary`). -* `:secondary_only` mode (which only allowed reads from secondaries) is now called `:secondary`. - -## Read preference inheritance - -The Ruby driver allows you to set read preference on each of four levels: the connection, database, collection, and cursor (or read operation). -Objects will inherit the default read preference from their parents. Thus, if you set a read preference of `{:read => :secondary}` when creating -a new connection, then all databases and collections created from that connection will inherit the same setting. See this code example: - - @con = Mongo::ReplSetConnection.new(['localhost:27017','localhost:27018'], :safe => true, :read => :secondary) - @db = @con['test'] - @collection = @db['foo'] - @collection.find({:name => 'foo'}) - - @collection.find({:name => 'bar'}, :read => :primary) - -Here, the first call to Collection#find will use the inherited read preference, `{:read => :secondary}`. But the second call -to Collection#find overrides this setting by setting the preference to `:primary`. - -You can examine the read preference on any object by calling its `read_preference` method: - - @con.read_preference - @db.read_preference - @collection.read_preference - -## Modes - -You can using the `:read` option to specify a query's read preference mode. There are five possible options. - -### :primary - -With primary, all read operations from the client will use the primary member only. This is the default read preference. - -If the primary is unavailable, all operations with this preference produce an error or throw an exception. Primary read preference modes are not compatible with read preferences modes that use tag sets If you specify a tag set with primary, the driver will produce an error. - -### :primary_preferred - -With the primaryPreferred read preference mode, operations will read from the primary member of the set in most situations. However, if the primary is unavailable, as is the case during failover situations, then these read operations can read from secondary members. - -When the read preference includes a tag set, the client will first read from the primary, if it is available, and then from secondaries that match the specified tags. If there are no secondaries with tags that match the specified tags, this read operation will produce an error. - -### :secondary - -With the secondary read preference mode, operations will read from the secondary member of the set if available. However, if there are no secondaries available, then these operations will produce an error or exception. - -Most sets have at least one secondary, but there are situations where there may not be an available secondary. For example, a set with a primary, a secondary, and an arbiter may not have any secondaries if a member is ever in recovering mode. - -When the read preference includes a tag set, the client will attempt to find a secondary members that match the specified tag set and directs reads to a random secondary from among the nearest group. If there are no secondaries with tags that match the specified tag set, this read operation will produce an error. - -### :secondary_preferred - -With the secondaryPreferred, operations will read from secondary members, but in situations where the set only has a primary instance, the read operation will use the set’s primary. - -When secondaryPreferred reads from a secondary and the read preference includes a tag set, the client will attempt to find a secondary members that match the specified tag set and directs reads to a random secondary from among the nearest group. If there are no secondaries with tags that match the specified tag set, this read operation will produce an error. - -### :nearest - -With the nearest, the driver will read from the nearest member of the set according to the member selection process nearest read operations will not have any consideration for the type of the set member. Reads in nearest mode may read from both primaries and secondaries. - -Set this mode when you want minimize the effect of network latency on read operations without preference for current or stale data. - -If you specify a tag set, the client will attempt to find a secondary members that match the specified tag set and directs reads to a random secondary from among the nearest group. - -## Tag Sets - -Tag sets can be used in for data center awareness by filtering secondary read operations. Primary reads occur independent of any tags. - -A member matches a tag set if its tags match all the tags in the set. For example, a member tagged "{ dc: 'ny', rack: 2, size: 'large' }" matches the tag set "{ dc: 'ny', rack: 2 }". A member's extra tags don't affect whether it's a match. - -Here is an example of a query which sends read operations to members in rack 2. - - @collection.find({:name => 'foo'}, :read => :secondary_preferred, :tag_sets => [{:rack => '2'}]) - -Tag set keys may be symbols or strings. Tag set values should be specified using strings. The `to_s` method will be called on any values provided in the tag set. - -Tag sets are used in conjunction with read preference mode. In this example, because we specified a mode of secondary_preferred, if no secondaries can be found that match the tag_set `{:rack => '2'}` then the primary will be used for the query. - -If only one tag set is provided, the set can be passed as a single hash parameter itself without the enclosing array. - - @collection.find({:name => 'foo'}, :read => :secondary_preferred, :tag_sets => {:rack => '2'}) - -Specifying tag_sets for mode `:primary` is considered an error and will raise a MongoArgumentError as tag_sets do not affect selection of primary members and only primary members can be selected in that particular mode. diff --git a/source/RELEASES.md b/source/RELEASES.md deleted file mode 100644 index 4340b94f5..000000000 --- a/source/RELEASES.md +++ /dev/null @@ -1,54 +0,0 @@ -# MongoDB Ruby Driver Release Plan - -This is a description of a formalized release plan that will take effect -with version 1.3.0. - -## Semantic versioning - -The most significant difference is that releases will now adhere to the conventions of -[semantic versioning](http://semver.org). In particular, we will strictly abide by the -following release rules: - -1. Patch versions of the driver (Z in x.y.Z) will be released only when backward-compatible bug fixes are introduced. A bug fix is defined as an internal change that fixes incorrect behavior. - -2. Minor versions (Y in x.Y.z) will be released if new, backward-compatible functionality is introduced to the public API. - -3. Major versions (X in X.y.z) will be incremented if any backward-incompatible changes are introduced to the public API. - -This policy will clearly indicate to users when an upgrade may affect their code. As a side effect, version numbers will climb more quickly than before. - - -## Release checklist - -Before each relese to Rubygems.org, the following steps will be taken: - -1. All driver tests will be run on Linux, OS X, and Windows via continuous integration system. - -2. Update the HISTORY file and document all significant commits. - -3. Update the version in lib/bson.rb, lib/mongo/version.rb, and ext/cbson/version.h. - -4. Commit: "RELEASE [VERSION]" - -5. git tag [version] - -6. Build gems. Ensure that they have the correct versions. - -7. Push tags and commit to GitHub (git push origin master, git push --tags). - -8. Build and push docs. (git: mongodb/apidocs) - -9. Push gems to Rubygems.org. - -10. Test that the gem is downloadable from Rubygems.org. - -11. Close out release in JIRA. - -12. Announce release on mongodb-user and mongodb-dev. - -## Rake Deploy Tasks -1. rake deploy:change_version[x.x.x] -2. rake deploy:git_prepare -3. rake deploy:git_push -4. rake deploy:gem_build -5. rake deploy:gem_push diff --git a/source/REPLICA_SETS.md b/source/REPLICA_SETS.md deleted file mode 100644 index 023919e36..000000000 --- a/source/REPLICA_SETS.md +++ /dev/null @@ -1,113 +0,0 @@ -# Replica Sets in Ruby - -Here follow a few considerations for those using the MongoDB Ruby driver with [replica sets](http://www.mongodb.org/display/DOCS/Replica+Sets). - -### Setup - -First, make sure that you've configured and initialized a replica set. - -Use `ReplSetConnection.new` to connect to a replica set. This method, which accepts a variable number of arguments, -takes a list of seed nodes followed by any connection options. You'll want to specify at least two seed nodes. This gives -the driver more chances to connect in the event that any one seed node is offline. Once the driver connects, it will -cache the replica set topology as reported by the given seed node and use that information if a failover is later required. - - @connection = ReplSetConnection.new(['n1.mydb.net:27017', 'n2.mydb.net:27017', 'n3.mydb.net:27017'], :safe => true) - -### Read slaves - -If you want to read from a secondary node, you can pass :read => :secondary to ReplSetConnection#new. - - @connection = ReplSetConnection.new(['n1.mydb.net:27017', 'n2.mydb.net:27017', 'n3.mydb.net:27017'], - :safe => true, :read => :secondary) - -A random secondary will be chosen to be read from. In a typical multi-process Ruby application, you'll have a good distribution of reads across secondary nodes. - -### Connection Failures - -Imagine that either the master node or one of the read nodes goes offline. How will the driver respond? - -If any read operation fails, the driver will raise a *ConnectionFailure* exception. It then becomes the client's responsibility to decide how to handle this. - -If the client decides to retry, it's not guaranteed that another member of the replica set will have been promoted to master right away, so it's still possible that the driver will raise another *ConnectionFailure*. However, once a member has been promoted to master, typically within a few seconds, subsequent operations will succeed. *Note that this does not prevent -exception in the event of a primary failover.* - -The driver will essentially cycle through all known seed addresses until a node identifies itself as master. - -### Refresh mode - -You can now specify a refresh mode and refresh interval for a replica set connection. This will help to ensure that -changes to a replica set's configuration are quickly reflected on the driver side. In particular, if you change -the state of any secondary node, the automated refresh will ensure that this state is recorded on the client side. - -There are two scenarios in which refresh is helpful and does not raise exceptions: - -1. You add a new secondary node to an existing replica set -2. You remove an unused secondary from an existing replica set - -If using MongoDB earlier than 2.0 any changes to replica set state will raise exceptions therefore refresh mode will not be useful. - -If you add a secondary that responds to pings much faster than the existing nodes, then the new secondary will -be used for reads if :read_preference is :secondary or :secondary_only - -Refresh mode is disabled by default. - -However, if you expect to make live changes to your secondaries, and you want this to be reflected without -having to manually restart your app server, then you should enable it. You can enable this mode -synchronously, which will refresh the replica set data in a synchronous fashion (which may -occasionally slow down your queries): - - @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :safe => true, :refresh_mode => :sync) - -If you want to change the default refresh interval of 90 seconds, you can do so like this: - - @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :safe => true, :refresh_mode => :sync, - :refresh_interval => 60) - -Do not set this value to anything lower than 30, or you may start to experience performance issues. - -You can also disable refresh mode altogether: - - @connection = ReplSetConnection.new(['n1.mydb.net:27017'], :safe => true, :refresh_mode => false) - -And you can call `refresh` manually on any replica set connection: - - @connection.refresh - -### Recovery - -Driver users may wish to wrap their database calls with failure recovery code. Here's one possibility, which will attempt to connection -every half second and time out after thirty seconds. - - # Ensure retry upon failure - def rescue_connection_failure(max_retries=60) - retries = 0 - begin - yield - rescue Mongo::ConnectionFailure => ex - retries += 1 - raise ex if retries > max_retries - sleep(0.5) - retry - end - end - - # Wrapping a call to #count() - rescue_connection_failure do - @db.collection('users').count() - end - -Of course, the proper way to handle connection failures will always depend on the individual application. We encourage object-mapper and application developers to publish any promising results. - -### Testing - -The Ruby driver (>= 1.1.5) includes unit tests for verifying replica set behavior. They reside in *tests/replica_sets*. You can run them as a group with the following rake task: - - rake test:rs - -The suite will set up a five-node replica set by itself and ensure that driver behaves correctly even in the face -of individual node failures. Note that the `mongod` executable must be in the search path for this to work. - -### Further Reading - -* [Replica Sets](http://www.mongodb.org/display/DOCS/Replica+Set+Configuration) -* [Replica Set Configuration](http://www.mongodb.org/display/DOCS/Replica+Set+Configuration) diff --git a/source/TAILABLE_CURSORS.md b/source/TAILABLE_CURSORS.md deleted file mode 100644 index 029f00e59..000000000 --- a/source/TAILABLE_CURSORS.md +++ /dev/null @@ -1,51 +0,0 @@ -# Tailable cursors in Ruby - -Tailable cursors are cursors that remain open even after they've returned -a final result. This way, if more documents are added to a collection (i.e., -to the cursor's result set), then you can continue to call `Cursor#next` to -retrieve those results. Here's a complete test case that demonstrates the use -of tailable cursors. - -Note that tailable cursors are for capped collections only. - - require 'mongo' - require 'test/unit' - - class TestTailable < Test::Unit::TestCase - include Mongo - - def test_tailable - - # Create a connection and capped collection. - @con = Connection.new('localhost', 27017, :safe => true) - @db = @con['test'] - @db.drop_collection('log') - @capped = @db.create_collection('log', :capped => true, :size => 1024) - - # Insert 10 documents. - 10.times do |n| - @capped.insert({:n => n}) - end - - # Create a tailable cursor that iterates the collection in natural order - @tail = Cursor.new(@capped, :tailable => true, :order => [['$natural', 1]]) - - # Call Cursor#next 10 times. Each call returns a document. - 10.times do - assert @tail.next - end - - # But the 11th time, the cursor returns nothing. - assert_nil @tail.next - - # Add a document to the capped collection. - @capped.insert({:n => 100}) - - # Now call Cursor#next again. This will return the just-inserted result. - assert @tail.next - - # Close the cursor. - @tail.close - end - - end diff --git a/source/TUTORIAL.md b/source/TUTORIAL.md deleted file mode 100644 index 3625a8de8..000000000 --- a/source/TUTORIAL.md +++ /dev/null @@ -1,364 +0,0 @@ -# MongoDB Ruby Driver Tutorial - -This tutorial gives many common examples of using MongoDB with the Ruby driver. If you're looking for information on data modeling, see [MongoDB Data Modeling and Rails](http://www.mongodb.org/display/DOCS/MongoDB+Data+Modeling+and+Rails). Links to the various object mappers are listed on our [object mappers page](http://www.mongodb.org/display/DOCS/Object+Mappers+for+Ruby+and+MongoDB). - -Interested in GridFS? See [GridFS in Ruby](file.GridFS.html). - -As always, the [latest source for the Ruby driver](http://github.com/mongodb/mongo-ruby-driver) can be found on [github](http://github.com/mongodb/mongo-ruby-driver/). - -## Installation - -The mongo-ruby-driver gem is served through Rubygems.org. To install, make sure you have the latest version of rubygems. - - gem update --system - -Next, install the mongo rubygem: - - gem install mongo - -The required `bson` gem will be installed automatically. - -For optimum performance, install the bson\_ext gem: - - gem install bson_ext - -After installing, you may want to look at the [examples](http://github.com/mongodb/mongo-ruby-driver/tree/master/examples) directory included in the source distribution. These examples walk through some of the basics of using the Ruby driver. - -## Getting started - -Note that the output in the following has been updated to Ruby 1.9, so if you are using Ruby 1.8, you will see some minor differences. To follow this tutorial interactively, at the command line, run the Interactive Ruby Shell. - - irb - -As you execute commands, irb will output the result using the `inspect` method. If you are editing and running a script for this tutorial, you can view output using the `puts` or `p` methods. - -### Using the gem - -Use the `mongo` gem via the `require` kernel method. - - require 'rubygems' # not necessary for Ruby 1.9 - require 'mongo' - -### Making a Connection - -An `Mongo::Connection` instance represents a connection to MongoDB. - - connection = Mongo::Connection.new("localhost", 27017, :safe => true) - -The option :safe => true is highly recommended. This specifies that the driver sends a getlasterror command after every update to ensure that the update succeeded. - -You can optionally specify the MongoDB server address and port when connecting. The following example shows three ways to connect to the local machine: - - connection = Mongo::Connection.new # (optional host/port args) - connection = Mongo::Connection.new("localhost") - connection = Mongo::Connection.new("localhost", 27017) - -In these cases, the :safe option defaults to false, and updates are fire-and-forget with higher performance. We do not recommend the default :safe => false unless your application can tolerate the potential loss of updates. - -### Listing All Databases - - connection.database_names - connection.database_info.each { |info| puts info.inspect } - -The `database_info` method returns a hash mapping database names to the size of the database in bytes. - -## Using a Database - -You use a Connection instance to obtain an Mongo::DB instance, which represents a named database. The database doesn't have to exist - if it doesn't, MongoDB will create it for you. The following examples use the database "mydb": - - db = connection.db("mydb") - db = Mongo::Connection.new("localhost", 27017, :safe => true).db("mydb") - -At this point, the `db` object will be a connection to a MongoDB server for the specified database. Each DB instance uses a separate socket connection to the server. - -If you're trying to connect to a replica set, see [Replica Sets in Ruby](http://www.mongodb.org/display/DOCS/Replica+Sets+in+Ruby). - -### Authentication - -MongoDB can be run in a secure mode where access to databases is controlled through name and password authentication. When run in this mode, any client application must provide a name and password before doing any operations. In the Ruby driver, you simply do the following with the connected mongo object: - - auth = db.authenticate(my_user_name, my_password) - -If the name and password are valid for the database, `auth` will be `true`. Otherwise, it will be `false`. You should look at the MongoDB log for further information if available. - -## Using a Collection - -You can get a collection to use using the `collection` method: - - coll = db.collection("testCollection") - -This is aliased to the \[\] method: - - coll = db["testCollection"] - -Once you have this collection object, you can now do create, read, update, and delete (CRUD) functions on persistent storage. - -### Creating Documents with `insert` - -Once you have the collection object, you can create or `insert` documents into the collection. For example, lets make a little document that in JSON would be represented as - - { - "name" : "MongoDB", - "type" : "database", - "count" : 1, - "info" : { - x : 203, - y : 102 - } - } - -Notice that the above has an "inner" document embedded within it. To do this, we can use a Hash or the driver's OrderedHash (which preserves key order) to create the document (including the inner document), and then just simply insert it into the collection using the `insert` method. - - doc = {"name" => "MongoDB", "type" => "database", "count" => 1, "info" => {"x" => 203, "y" => '102'}} - id = coll.insert(doc) - -We have saved the `id` for future use below. Now the collection has been created and you can list it. - -#### Getting a List Of Collections - -Each database has zero or more collections. You can retrieve a list of them from the db (and print out any that are there): - - db.collection_names - -You should see - - \["testCollection", "system.indexes"\] - -#### Adding Multiple Documents - -To demonstrate some more interesting queries, let's add multiple simple documents to the collection. These documents will have the following form: - - { - "i" : value - } - -Here's how to insert them: - - 100.times { |i| coll.insert("i" => i) } - -Notice that we can insert documents of different "shapes" into the same collection. These records are in the same collection as the complex record we inserted above. This aspect is what we mean when we say that MongoDB is "schema-free". - -### Reading Documents with `find_one` and `find` - -#### Reading the First Document in a Collection using `find_one` - -To retrieve the document that we inserted, we can do a simple `find_one` method to get the first document in the collection. This method returns a single document directly. - - coll.find_one - -and you should something like: - - {"_id"=>BSON::ObjectId('4f7b1ea6e4d30b35c9000001'), "name"=>"MongoDB", "type"=>"database", "count"=>1, "info"=>{"x"=>203, "y"=>"102"}} - -Note the `_id` element has been added automatically by MongoDB to your document. - -#### Reading All of the Documents with a Cursor using `find` - -To get all the documents from the collection, we use the `find` method. `find` returns a `Cursor` object, which allows us to iterate over the set of documents that matches our query. The Ruby driver's Cursor implemented Enumerable, which allows us to use `Enumerable#each`, `Enumerable#map}, etc. For instance: - - coll.find.each { |row| puts row.inspect } - -and that should print all 101 documents in the collection. You can take advantage of `Enumerable#to_a`. - - puts coll.find.to_a - -Important note - using `to_a` pulls all of the full result set into memory and is inefficient if you can process by each individual document. To process with more memory efficiency, use the `each` method with a code block for the cursor. - -#### Specific Queries - -We can create a _query_ hash to pass to the `find` method to get a subset of the documents in our collection. To check that our update worked, find the document by id: - - coll.find("_id" => id).to_a - -If we wanted to find the document for which the value of the "i" field is 71, we would do the following: - - coll.find("i" => 71).to_a - -and it should just print just one document: - - {"_id"=>BSON::ObjectId('4f7b20b4e4d30b35c9000049'), "i"=>71} - -#### Sorting Documents in a Collection - -To sort documents, simply use the `sort` method. The method can either take a key or an array of [key, direction] pairs to sort by. - -Direction defaults to ascending order but can be specified as Mongo::ASCENDING, :ascending, or :asc whereas descending order can be specified with Mongo::DESCENDING, :descending, or :desc. - - # Sort in ascending order by :i - coll.find.sort(:i) - - # Sort in descending order by :i - coll.find.sort(:i => :desc) - -#### Counting Documents in a Collection - -Now that we've inserted 101 documents (the 100 we did in the loop, plus the first one), we can check to see if we have them all using the `count` method. - - coll.count - -and it should print `101`. - -#### Getting a Set of Documents With a Query - -We can use the query to get a set of documents from our collection. For example, if we wanted to get all documents where "i" > 50, we could write: - - puts coll.find("i" => {"$gt" => 50}).to_a - -which should print the documents where i > 50. We could also get a range, say 20 < i <= 30: - - puts coll.find("i" => {"$gt" => 20, "$lte" => 30}).to_a - -#### Selecting a Subset of Fields for a Query - -Use the `:fields` option to specify fields to return. - - puts coll.find({"_id" => id}, :fields => ["name", "type"]).to_a - -#### Querying with Regular Expressions - -Regular expressions can be used to query MongoDB. To find all names that begin with 'M': - - puts coll.find({"name" => /^M/}).to_a - -You can also construct a regular expression dynamically. To match a given search string: - - params = {'search' => 'DB'} - search_string = params['search'] - - # Constructor syntax - puts coll.find({"name" => Regexp.new(search_string)}).to_a - - # Literal syntax - puts coll.find({"name" => /#{search_string}/}).to_a - -Although MongoDB isn't vulnerable to anything like SQL-injection, it may be worth checking the search string for anything malicious. - -### Updating Documents with `update` - -We can update the previous document using the `update` method. There are a couple ways to update a document. We can rewrite it: - - doc["name"] = "MongoDB Ruby" - coll.update({"_id" => id}, doc) - -Or we can use an atomic operator to change a single value: - - coll.update({"_id" => id}, {"$set" => {"name" => "MongoDB Ruby"}}) - -Verify the update. - - puts coll.find("_id" => id).to_a - -Read [more about updating documents|Updating]. - -### Deleting Documents with `remove` - -Use the `remove` method to delete documents. - - coll.count - coll.remove("i" => 71) - coll.count - puts coll.find("i" => 71).to_a - -The above shows that the count has been reduced and that the document can no longer be found. - -Without arguments, the `remove` method deletes all documents. - - coll.remove - coll.count - -Please program carefully. - -## Indexing - -### Creating An Index - -MongoDB supports indexes, and they are very easy to add on a collection. To create an index, you specify an index name and an array of field names to be indexed, or a single field name. The following creates an ascending index on the "i" field: - - # create_index assumes ascending order; see method docs - # for details - coll.create_index("i") -To specify complex indexes or a descending index you need to use a slightly more complex syntax - the index specifier must be an Array of [field name, direction] pairs. Directions should be specified as Mongo::ASCENDING or Mongo::DESCENDING: - - # Explicit "ascending" - coll.create_index([["i", Mongo::ASCENDING]]) - -Use the `explain` method on the cursor to show how MongoDB will run the query. - - coll.find("_id" => id).explain - coll.find("i" => 71).explain - coll.find("type" => "database").explain - -The above shows that the query by `_id` and `i` will use faster indexed BtreeCursor, while the query by `type` will use a slower BasicCursor. - -### Getting a List of Indexes on a Collection - -You can get a list of the indexes on a collection. - - coll.index_information - -### Creating and Querying on a Geospatial Index - -First, create the index on a field containing long-lat values: - - people.create_index([["loc", Mongo::GEO2D]]) - -Then get a list of the twenty locations nearest to the point 50, 50: - - people.find({"loc" => {"$near" => [50, 50]}}, {:limit => 20}).each do |p| - puts p.inspect - end - -## Dropping - -### Drop an Index - -To drop a secondary index, use the `drop_index` method on the collection. - - coll.drop_index("i_1") - coll.index_information - -The dropped index is no longer listed. - -### Drop All Indexes - -To drop all secondary indexes, use the `drop_indexes` method on the collection. - - coll.drop_indexes - coll.index_information - -Only the primary index "_id_" is listed. - -### Drop a Collection - -To drop a collection, use the `drop` method on the collection. - - coll.drop - db.collection_names - -The dropped collection is no longer listed. The `drop_collection` method can be used on the database as an alternative. - - db.drop_collection("testCollection") - -### Drop a Database - -To drop a database, use the `drop_database` method on the connection. - - connection.drop_database("mydb") - connection.database_names - -The dropped database is no longer listed. - -## Database Administration - -A database can have one of three profiling levels: off (:off), slow queries only (:slow_only), or all (:all). To see the database level: - - puts db.profiling_level # => off (the symbol :off printed as a string) - db.profiling_level = :slow_only - -Validating a collection will return an interesting hash if all is well or raise an exception if there is a problem. - p db.validate_collection('coll_name') - -## See Also - -* [MongoDB Koans](http://github.com/chicagoruby/MongoDB_Koans) A path to MongoDB enlightenment via the Ruby driver. -* [MongoDB Manual](http://www.mongodb.org/display/DOCS/Developer+Zone) diff --git a/source/WRITE_CONCERN.md b/source/WRITE_CONCERN.md deleted file mode 100644 index 16759dc08..000000000 --- a/source/WRITE_CONCERN.md +++ /dev/null @@ -1,31 +0,0 @@ -# Write Concern in Ruby - -## Setting the write concern - -Write concern is set using the `:safe` option. There are several possible options: - - @collection.save({:doc => 'foo'}, :safe => true) - @collection.save({:doc => 'foo'}, :safe => {:w => 2}) - @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200}) - @collection.save({:doc => 'foo'}, :safe => {:w => 2, :wtimeout => 200, :j => true}) - -The first, `true`, simply indicates that we should request a response from the server to ensure that no errors have occurred. The second, `{:w => 2}`, forces the server to wait until at least two servers have recorded the write. The third does the same but will time out if the replication can't be completed in 200 milliseconds. -Setting a value for `wtimeout` is encouraged. - -Finally, the fourth example forces the journal to sync to disk if journaling is enabled. - -## Write concern inheritance - -The Ruby driver allows you to set write concern on each of four levels: the connection, database, collection, and write operation. -Objects will inherit the default write concern from their parents. Thus, if you set a write concern of `{:w => 1}` when creating -a new connection, then all databases and collections created from that connection will inherit the same setting. See this code example: - - @con = Mongo::Connection.new('localhost', 27017, :safe => {:w => 2}) - @db = @con['test'] - @collection = @db['foo'] - @collection.save({:name => 'foo'}) - - @collection.save({:name => 'bar'}, :safe => false) - -Here, the first call to Collection#save will use the inherited write concern, `{:w => 2}`. But notice that the second call -to Collection#save overrides this setting. diff --git a/source/examples/admin.rb b/source/examples/admin.rb deleted file mode 100644 index 580f7bf6f..000000000 --- a/source/examples/admin.rb +++ /dev/null @@ -1,43 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) - -require 'mongo' -require 'pp' - -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts "Connecting to #{host}:#{port}" -con = Mongo::Connection.new(host, port, :safe => true) -db = con.db('ruby-mongo-examples') -coll = db.create_collection('test') - -# Erase all records from collection, if any -coll.remove - -admin = con['admin'] - -# Profiling level set/get -puts "Profiling level: #{admin.profiling_level}" - -# Start profiling everything -admin.profiling_level = :all - -# Read records, creating a profiling event -coll.find().to_a - -# Stop profiling -admin.profiling_level = :off - -# Print all profiling info -pp admin.profiling_info - -# Validate returns a hash if all is well and -# raises an exception if there is a problem. -info = db.validate_collection(coll.name) -puts "valid = #{info['ok']}" -puts info['result'] - -# Destroy the collection -coll.drop diff --git a/source/examples/capped.rb b/source/examples/capped.rb deleted file mode 100644 index ff63a5ebe..000000000 --- a/source/examples/capped.rb +++ /dev/null @@ -1,22 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) -require 'mongo' - -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') -db.drop_collection('test') - -# A capped collection has a max size and, optionally, a max number of records. -# Old records get pushed out by new ones once the size or max num records is reached. -coll = db.create_collection('test', :capped => true, :size => 1024, :max => 12) - -100.times { |i| coll.insert('a' => i+1) } - -# We will only see the last 12 records -coll.find().each { |row| p row } - -coll.drop diff --git a/source/examples/cursor.rb b/source/examples/cursor.rb deleted file mode 100644 index 1dae16757..000000000 --- a/source/examples/cursor.rb +++ /dev/null @@ -1,48 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) - -require 'mongo' -require 'pp' - -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') -coll = db.collection('test') - -# Erase all records from collection, if any -coll.remove - -# Insert 3 records -3.times { |i| coll.insert({'a' => i+1}) } - -# Cursors don't run their queries until you actually attempt to retrieve data -# from them. - -# Find returns a Cursor, which is Enumerable. You can iterate: -coll.find().each { |row| pp row } - -# You can turn it into an array: -array = coll.find().to_a - -# You can iterate after turning it into an array (the cursor will iterate over -# the copy of the array that it saves internally.) -cursor = coll.find() -array = cursor.to_a -cursor.each { |row| pp row } - -# You can get the next object -first_object = coll.find().next_document - -# next_document returns nil if there are no more objects that match -cursor = coll.find() -obj = cursor.next_document -while obj - pp obj - obj = cursor.next_document -end - -# Destroy the collection -coll.drop diff --git a/source/examples/gridfs.rb b/source/examples/gridfs.rb deleted file mode 100644 index 308eee658..000000000 --- a/source/examples/gridfs.rb +++ /dev/null @@ -1,44 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) -def assert - raise "Failed!" unless yield -end - -require 'mongo' -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') - -data = "hello, world!" - -grid = Grid.new(db) - -# Write a new file. data can be a string or an io object responding to #read. -id = grid.put(data, :filename => 'hello.txt') - -# Read it and print out the contents -file = grid.get(id) -puts file.read - -# Delete the file -grid.delete(id) - -begin -grid.get(id) -rescue => e - assert {e.class == Mongo::GridError} -end - -# Metadata -id = grid.put(data, :filename => 'hello.txt', :content_type => 'text/plain', :metadata => {'name' => 'hello'}) -file = grid.get(id) - -p file.content_type -p file.metadata.inspect -p file.chunk_size -p file.file_length -p file.filename -p file.data diff --git a/source/examples/index_test.rb b/source/examples/index_test.rb deleted file mode 100644 index eb77fe07c..000000000 --- a/source/examples/index_test.rb +++ /dev/null @@ -1,126 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) - -require 'mongo' - -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts ">> Connecting to #{host}:#{port}" -db = Connection.new(host, port, :safe => true).db('ruby-mongo-index_test') - -class Exception - def errmsg - "%s: %s\n%s" % [self.class, message, (backtrace || []).join("\n") << "\n"] - end -end - -puts ">> Dropping collection test" -begin - res = db.drop_collection('test') - puts "dropped : #{res.inspect}" -rescue => e - puts "Error: #{e.errmsg}" -end - -puts ">> Creating collection test" -begin - coll = db.collection('test') - puts "created : #{coll.inspect}" -rescue => e - puts "Error: #{e.errmsg}" -end - -OBJS_COUNT = 100 - -puts ">> Generating test data" -msgs = %w{hola hello aloha ciao} -arr = (0...OBJS_COUNT).collect {|x| { :number => x, :rndm => (rand(5)+1), :msg => msgs[rand(4)] }} -puts "generated" - -puts ">> Inserting data (#{arr.size})" -coll.insert(arr) -puts "inserted" - -puts ">> Creating index" -#res = coll.create_index "all", :_id => 1, :number => 1, :rndm => 1, :msg => 1 -res = coll.create_index [[:number, 1], [:rndm, 1], [:msg, 1]] -puts "created index: #{res.inspect}" -# ============================ Mongo Log ============================ -# Fri Dec 5 14:45:02 Adding all existing records for ruby-mongo-console.test to new index -# *** -# Bad data or size in BSONElement::size() -# bad type:30 -# totalsize:11 fieldnamesize:4 -# lastrec: -# Fri Dec 5 14:45:02 ruby-mongo-console.system.indexes Assertion failure false jsobj.cpp a0 -# Fri Dec 5 14:45:02 database: ruby-mongo-console op:7d2 0 -# Fri Dec 5 14:45:02 ns: ruby-mongo-console.system.indexes - -puts ">> Gathering index information" -begin - res = coll.index_information - puts "index_information : #{res.inspect}" -rescue => e - puts "Error: #{e.errmsg}" -end -# ============================ Console Output ============================ -# RuntimeError: Keys for index on return from db was nil. Coll = ruby-mongo-console.test -# from ./bin/../lib/mongo/db.rb:135:in `index_information' -# from (irb):11:in `collect' -# from ./bin/../lib/mongo/cursor.rb:47:in `each' -# from ./bin/../lib/mongo/db.rb:130:in `collect' -# from ./bin/../lib/mongo/db.rb:130:in `index_information' -# from ./bin/../lib/mongo/collection.rb:74:in `index_information' -# from (irb):11 - -puts ">> Dropping index" -begin - res = coll.drop_index "number_1_rndm_1_msg_1" - puts "dropped : #{res.inspect}" -rescue => e - puts "Error: #{e.errmsg}" -end - -# ============================ Console Output ============================ -# => {"nIndexesWas"=>2.0, "ok"=>1.0} -# ============================ Mongo Log ============================ -# 0x41802a 0x411549 0x42bac6 0x42c1f6 0x42c55b 0x42e6f7 0x41631e 0x41a89d 0x41ade2 0x41b448 0x4650d2 0x4695ad -# db/db(_Z12sayDbContextPKc+0x17a) [0x41802a] -# db/db(_Z8assertedPKcS0_j+0x9) [0x411549] -# db/db(_ZNK11BSONElement4sizeEv+0x1f6) [0x42bac6] -# db/db(_ZN7BSONObj8getFieldEPKc+0xa6) [0x42c1f6] -# db/db(_ZN7BSONObj14getFieldDottedEPKc+0x11b) [0x42c55b] -# db/db(_ZN7BSONObj19extractFieldsDottedES_R14BSONObjBuilder+0x87) [0x42e6f7] -# db/db(_ZN12IndexDetails17getKeysFromObjectER7BSONObjRSt3setIS0_St4lessIS0_ESaIS0_EE+0x24e) [0x41631e] -# db/db(_Z12_indexRecordR12IndexDetailsR7BSONObj7DiskLoc+0x5d) [0x41a89d] -# db/db(_Z18addExistingToIndexPKcR12IndexDetails+0xb2) [0x41ade2] -# db/db(_ZN11DataFileMgr6insertEPKcPKvib+0x508) [0x41b448] -# db/db(_Z14receivedInsertR7MessageRSt18basic_stringstreamIcSt11char_traitsIcESaIcEE+0x112) [0x4650d2] -# db/db(_Z10connThreadv+0xb4d) [0x4695ad] -# Fri Dec 5 14:45:02 ruby-mongo-console.system.indexes Caught Assertion insert, continuing -# Fri Dec 5 14:47:59 CMD: deleteIndexes ruby-mongo-console.test -# d->nIndexes was 2 -# alpha implementation, space not reclaimed - -puts ">> Gathering index information" -begin - res = coll.index_information - puts "index_information : #{res.inspect}" -rescue => e - puts "Error: #{e.errmsg}" -end -# ============================ Console Output ============================ -# RuntimeError: Keys for index on return from db was nil. Coll = ruby-mongo-console.test -# from ./bin/../lib/mongo/db.rb:135:in `index_information' -# from (irb):15:in `collect' -# from ./bin/../lib/mongo/cursor.rb:47:in `each' -# from ./bin/../lib/mongo/db.rb:130:in `collect' -# from ./bin/../lib/mongo/db.rb:130:in `index_information' -# from ./bin/../lib/mongo/collection.rb:74:in `index_information' -# from (irb):15 - -puts ">> Closing connection" -db.close -puts "closed" diff --git a/source/examples/info.rb b/source/examples/info.rb deleted file mode 100644 index 3737fc9e7..000000000 --- a/source/examples/info.rb +++ /dev/null @@ -1,31 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) - -require 'mongo' - -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') -coll = db.collection('test') - -# Erase all records from collection, if any -coll.remove - -# Insert 3 records -3.times { |i| coll.insert({'a' => i+1}) } - -# Collection names in database -p db.collection_names - -# More information about each collection -p db.collections_info - -# Index information -coll.create_index('a') -p db.index_information('test') - -# Destroy the collection -coll.drop diff --git a/source/examples/queries.rb b/source/examples/queries.rb deleted file mode 100644 index 79d9104bc..000000000 --- a/source/examples/queries.rb +++ /dev/null @@ -1,74 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) - -require 'mongo' -require 'pp' - -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') -coll = db.collection('test') - -# Remove all records, if any -coll.remove - -# Insert three records -coll.insert('a' => 1) -coll.insert('a' => 2) -coll.insert('b' => 3) -coll.insert('c' => 'foo') -coll.insert('c' => 'bar') - -# Count. -puts "There are #{coll.count} records." - -# Find all records. find() returns a Cursor. -puts "Find all records:" -pp cursor = coll.find.to_a - -# Print them. Note that all records have an _id automatically added by the -# database. See pk.rb for an example of how to use a primary key factory to -# generate your own values for _id. -puts "Print each document individually:" -pp cursor.each { |row| pp row } - -# See Collection#find. From now on in this file, we won't be printing the -# records we find. -puts "Find one record:" -pp coll.find('a' => 1).to_a - -# Find records sort by 'a', skip 1, limit 2 records. -# Sort can be single name, array, or hash. -puts "Skip 1, limit 2, sort by 'a':" -pp coll.find({}, {:skip => 1, :limit => 2, :sort => 'a'}).to_a - -# Find all records with 'a' > 1. There is also $lt, $gte, and $lte. -coll.find({'a' => {'$gt' => 1}}) -coll.find({'a' => {'$gt' => 1, '$lte' => 3}}) - -# Find all records with 'a' in a set of values. -puts "Find all records where a is $in [1, 2]:" -pp coll.find('a' => {'$in' => [1,2]}).to_a - -puts "Find by regex:" -pp coll.find({'c' => /f/}).to_a - -# Print query explanation -puts "Print an explain:" -pp coll.find({'c' => /f/}).explain - -# Use a hint with a query. Need an index. Hints can be stored with the -# collection, in which case they will be used with all queries, or they can be -# specified per query, in which case that hint overrides the hint associated -# with the collection if any. -coll.create_index('c') -coll.hint = 'c' - -puts "Print an explain with index:" -pp coll.find('c' => /[f|b]/).explain - -puts "Print an explain with natural order hint:" -pp coll.find({'c' => /[f|b]/}, :hint => '$natural').explain diff --git a/source/examples/replica_set.rb b/source/examples/replica_set.rb deleted file mode 100644 index 2cad06e8c..000000000 --- a/source/examples/replica_set.rb +++ /dev/null @@ -1,24 +0,0 @@ -# This code assumes a running replica set with at least one node at localhost:27017. -require 'mongo' - -cons = [] - -10.times do - cons << Mongo::ReplSetConnection(['localhost:27017'], :read => :secondary) -end - -ports = cons.map do |con| - con.read_pool.port -end - -puts "These ten connections will read from the following ports:" -p ports - -cons[rand(10)]['foo']['bar'].remove -100.times do |n| - cons[rand(10)]['foo']['bar'].insert({:a => n}) -end - -100.times do |n| - p cons[rand(10)]['foo']['bar'].find_one({:a => n}) -end diff --git a/source/examples/simple.rb b/source/examples/simple.rb deleted file mode 100644 index 907cda56b..000000000 --- a/source/examples/simple.rb +++ /dev/null @@ -1,25 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) - -require 'rubygems' -require 'mongo' - -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') -coll = db.collection('test') - -# Erase all records from collection, if any -coll.remove - -# Insert 3 records -3.times { |i| coll.insert({'a' => i+1}) } - -puts "There are #{coll.count()} records in the test collection. Here they are:" -coll.find().each { |doc| puts doc.inspect } - -# Destroy the collection -coll.drop diff --git a/source/examples/strict.rb b/source/examples/strict.rb deleted file mode 100644 index 9ad80231f..000000000 --- a/source/examples/strict.rb +++ /dev/null @@ -1,35 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) - -require 'mongo' - -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') - -db.drop_collection('does-not-exist') -db.create_collection('test') - -db.strict = true - -begin - # Can't reference collection that does not exist - db.collection('does-not-exist') - puts "error: expected exception" -rescue => ex - puts "expected exception: #{ex}" -end - -begin - # Can't create collection that already exists - db.create_collection('test') - puts "error: expected exception" -rescue => ex - puts "expected exception: #{ex}" -end - -db.strict = false -db.drop_collection('test') diff --git a/source/examples/types.rb b/source/examples/types.rb deleted file mode 100644 index 3ca19bd8c..000000000 --- a/source/examples/types.rb +++ /dev/null @@ -1,36 +0,0 @@ -$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')) - -require 'mongo' -require 'pp' - -include Mongo - -host = ENV['MONGO_RUBY_DRIVER_HOST'] || 'localhost' -port = ENV['MONGO_RUBY_DRIVER_PORT'] || Connection::DEFAULT_PORT - -puts "Connecting to #{host}:#{port}" -db = Connection.new(host, port, :safe => true).db('ruby-mongo-examples') -coll = db.collection('test') - -# Remove all records, if any -coll.remove - -# Insert record with all sorts of values -coll.insert('array' => [1, 2, 3], - 'string' => 'hello', - 'hash' => {'a' => 1, 'b' => 2}, - 'date' => Time.now, # milliseconds only; microseconds are not stored - 'oid' => ObjectID.new, - 'binary' => Binary.new([1, 2, 3]), - 'int' => 42, - 'float' => 33.33333, - 'regex' => /foobar/i, - 'boolean' => true, - 'where' => Code.new('this.x == 3'), - 'dbref' => DBRef.new(coll.name, ObjectID.new), - 'null' => nil, - 'symbol' => :zildjian) - -pp coll.find().next_document - -coll.remove diff --git a/source/load/thin/config.ru b/source/load/thin/config.ru deleted file mode 100644 index aa6ee91e7..000000000 --- a/source/load/thin/config.ru +++ /dev/null @@ -1,6 +0,0 @@ -require "rubygems" -require "sinatra" - -require File.join(File.dirname(__FILE__), 'load.rb') - -run Load diff --git a/source/load/thin/config.yml.template b/source/load/thin/config.yml.template deleted file mode 100644 index aa6ee91e7..000000000 --- a/source/load/thin/config.yml.template +++ /dev/null @@ -1,6 +0,0 @@ -require "rubygems" -require "sinatra" - -require File.join(File.dirname(__FILE__), 'load.rb') - -run Load diff --git a/source/load/thin/load.rb b/source/load/thin/load.rb deleted file mode 100644 index 9d471b658..000000000 --- a/source/load/thin/load.rb +++ /dev/null @@ -1,21 +0,0 @@ -require File.join(File.dirname(__FILE__), '..', '..', '..', 'lib', 'mongo') -require 'logger' - -$con = Mongo::ReplSetConnection.new(['localhost:30000', 'localhost:30001'], :safe => true, :read => :secondary, :refresh_mode => :sync, :refresh_interval => 30) -$db = $con['foo'] - -class Load < Sinatra::Base - - configure do - LOGGER = Logger.new("sinatra.log") - enable :logging, :dump_errors - set :raise_errors, true - end - - get '/' do - $db['test'].insert({:a => rand(1000)}) - $db['test'].find({:a => {'$gt' => rand(2)}}, :read => :secondary).limit(2).to_a - "ok" - end - -end diff --git a/source/load/unicorn/config.ru b/source/load/unicorn/config.ru deleted file mode 100644 index aa6ee91e7..000000000 --- a/source/load/unicorn/config.ru +++ /dev/null @@ -1,6 +0,0 @@ -require "rubygems" -require "sinatra" - -require File.join(File.dirname(__FILE__), 'load.rb') - -run Load diff --git a/source/load/unicorn/load.rb b/source/load/unicorn/load.rb deleted file mode 100644 index 83da34a26..000000000 --- a/source/load/unicorn/load.rb +++ /dev/null @@ -1,23 +0,0 @@ -require File.join(File.dirname(__FILE__), '..', '..', 'lib', 'mongo') - -$con = Mongo::Connection.new('localhost', 27017, :safe => true) -$db = $con['foo'] - -class Load < Sinatra::Base - - configure do - LOGGER = Logger.new("sinatra.log") - enable :logging, :dump_errors - set :raise_errors, true - end - - get '/' do - 3.times do |n| - if (v=$db.eval("1 + #{n}")) != 1 + n - STDERR << "#{1 + n} expected but got #{v}" - raise StandardError, "#{1 + n} expected but got #{v}" - end - end - end - -end diff --git a/source/load/unicorn/unicorn.rb.template b/source/load/unicorn/unicorn.rb.template deleted file mode 100644 index 333df2f6e..000000000 --- a/source/load/unicorn/unicorn.rb.template +++ /dev/null @@ -1,29 +0,0 @@ -# set path to app that will be used to configure unicorn, -# # note the trailing slash in this example -@dir = "/home/kyle/work/10gen/ruby-driver/test/load/" - -worker_processes 10 -working_directory @dir - -preload_app true - -timeout 30 - -# Specify path to socket unicorn listens to, -# we will use this in our nginx.conf later -listen "#{@dir}tmp/sockets/unicorn.sock", :backlog => 64 - -# Set process id path -pid "#{@dir}tmp/pids/unicorn.pid" - -# # Set log file paths -stderr_path "#{@dir}log/unicorn.stderr.log" -stdout_path "#{@dir}log/unicorn.stdout.log" - -# NOTE: You need this when using forking web servers! -after_fork do |server, worker| - $con.close if $con - $con = Mongo::Connection.new('localhost', 27017, :safe => true) - $db = $con['foo'] - STDERR << "FORKED #{server} #{worker}" -end From ad4659e44dea7786284697c7483a5a76cbfadbd7 Mon Sep 17 00:00:00 2001 From: kay Date: Thu, 28 Jul 2016 16:17:34 -0400 Subject: [PATCH 091/442] add bson docs for ruby driver manual --- docs/tutorials/bson-v4.txt | 265 +++++++++++++++++++++++++++++++++++++ 1 file changed, 265 insertions(+) create mode 100644 docs/tutorials/bson-v4.txt diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt new file mode 100644 index 000000000..e8df050f0 --- /dev/null +++ b/docs/tutorials/bson-v4.txt @@ -0,0 +1,265 @@ +.. http://docs.mongodb.com/ecosystem/tutorial/ruby-bson-tutorial-4-0/ + +.. _ruby-bson-tutorial-4-0: + +=================== +BSON 4.0.0 Tutorial +=================== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: twocols + +This tutorial discusses using the core Ruby BSON gem. + +Installation +------------ + +The BSON gem is hosted on `Rubygems `_ and can be installed +manually or with bundler. + +To install the gem manually: + +.. code-block:: sh + + gem install bson + +To install the gem with bundler, include the following in your Gemfile: + +.. code-block:: ruby + + gem 'bson', '~> 3.0' + +The BSON gem is compatible with MRI 1.9.3, 2.0.x, 2.1.x, 2.2.x, JRuby 1.7.x, and +Rubinius 2.5.x + +BSON Serialization +------------------ + +Getting a Ruby object's raw BSON representation is done by calling ``to_bson`` +on the Ruby object, which will return a ``ByteBuffer``. For example: + +.. code-block:: ruby + + "Shall I compare thee to a summer's day".to_bson + 1024.to_bson + +Generating an object from BSON is done via calling ``from_bson`` on the class +you wish to instantiate and passing it a ``ByteBuffer`` instance. + +.. code-block:: ruby + + String.from_bson(byte_buffer) + Int32.from_bson(byte_buffer) + +Byte Buffers +------------ + +In 4.0, BSON introduced the use of native byte buffers in both MRI and JRuby +instead of using ``StringIO``. This was done for performance improvements. + +API +``` + +A ``BSON::ByteBuffer`` can be instantiated with nothing (for write mode) or +with a string of raw bytes (for read mode). + +.. code-block:: ruby + + buffer = BSON::ByteBuffer.new # a write mode buffer. + buffer = BSON::ByteBuffer.new(string) # a read mode buffer. + +Writing to the buffer is done via the following API: + +.. code-block:: ruby + + buffer.put_byte(value) # Appends a single byte. + buffer.put_cstring(value) # Appends a null-terminated string. + buffer.put_double(value) # Appends a 64-bit floating point. + buffer.put_int32(value) # Appends a 32-bit integer (4 bytes). + buffer.put_int64(value) # Appends a 64-bit integer (8 bytes). + buffer.put_string(value) # Appends a UTF-8 string. + +Reading from the buffer is done via the following API: + +.. code-block:: ruby + + buffer.get_byte # Pulls a single byte from the buffer. + buffer.get_bytes(value) # Pulls n number of bytes from the buffer. + buffer.get_cstring # Pulls a null-terminted string from the buffer. + buffer.get_double # Pulls a 64-bit floating point from the buffer. + buffer.get_int32 # Pulls a 32-bit integer (4 bytes) from the buffer. + buffer.get_int64 # Pulls a 64-bit integer (8 bytes) from the buffer. + buffer.get_string # Pulls a UTF-8 string from the buffer. + +Convertig a buffer to it's raw bytes, for example to send over a socket, +is done by simply calling `to_s` on the buffer. + +.. code-block:: ruby + + buffer = BSON::ByteBuffer.new + buffer.put_string('testing') + socket.write(buffer.to_s) + +Supported Objects +----------------- + +Core Ruby object's that have representations in the BSON specification and +will have a ``to_bson`` method defined for them are: ``Object``, ``Array``, +``FalseClass``, ``Float``, ``Hash``, ``Integer``, ``NilClass``, ``Regexp``, +``String``, ``Symbol`` (deprecated), ``Time``, ``TrueClass``. + +In addition to the core Ruby objects, BSON also provides some special types +specific to the specification: + +``BSON::Binary`` +```````````````` + +This is a representation of binary data, and must provide the raw data and +a subtype when constructing. + +.. code-block:: ruby + + BSON::Binary.new(binary_data, :md5) + +Valid subtypes are: ``:generic``, ``:function``, ``:old``, ``:uuid_old``, ``:uuid``, +``:md5``, ``:user``. + +``BSON::Code`` +`````````````` + +Represents a string of Javascript code. + +.. code-block:: ruby + + BSON::Code.new("this.value = 5;") + +``BSON::CodeWithScope`` +``````````````````````` + +Represents a string of Javascript code with a hash of values. + +.. code-block:: ruby + + BSON::CodeWithScope.new("this.value = age;", age: 5) + +``BSON::Document`` +`````````````````` + +This is a subclass of a hash that stores all keys as strings but allows access to +them with symbol keys. + +.. code-block:: ruby + + BSON::Document[:key, "value"] + BSON::Document.new + +``BSON::MaxKey`` +```````````````` + +Represents a value in BSON that will always compare higher to another value. + +.. code-block:: ruby + + BSON::MaxKey.new + +``BSON::MinKey`` +```````````````` + +Represents a value in BSON that will always compare lower to another value. + +.. code-block:: ruby + + BSON::MinKey.new + +``BSON::ObjectId`` +`````````````````` + +Represents a 12 byte unique identifier for an object on a given machine. + +.. code-block:: ruby + + BSON::ObjectId.new + +``BSON::Timestamp`` +``````````````````` + +Represents a special time with a start and increment value. + +.. code-block:: ruby + + BSON::Timestamp.new(5, 30) + +``BSON::Undefined`` +``````````````````` + +Represents a placeholder for a value that was not provided. + +.. code-block:: ruby + + BSON::Undefined.new + +JSON Serialization +------------------ + +Some BSON types have special representations in JSON. These are as follows +and will be automatically serialized in the form when calling ``to_json`` on +them. + +.. list-table:: + :header-rows: 1 + :widths: 40 105 + + * - Object + - JSON + + * - ``BSON::Binary`` + - ``{ "$binary" : "\x01", "$type" : "md5" }`` + + * - ``BSON::Code`` + - ``{ "$code" : "this.v = 5 }`` + + * - ``BSON::CodeWithScope`` + - ``{ "$code" : "this.v = value", "$scope" : { v => 5 }}`` + + * - ``BSON::MaxKey`` + - ``{ "$maxKey" : 1 }`` + + * - ``BSON::MinKey`` + - ``{ "$minKey" : 1 }`` + + * - ``BSON::ObjectId`` + - ``{ "$oid" : "4e4d66343b39b68407000001" }`` + + * - ``BSON::Timestamp`` + - ``{ "t" : 5, "i" : 30 }`` + + * - ``Regexp`` + - ``{ "$regex" : "[abc]", "$options" : "i" }`` + + +Special Ruby Date Classes +------------------------- + +Ruby's ``Date`` and ``DateTime`` are able to be serialized, but when +they are deserialized they will always be returned as a ``Time`` since the BSON +specification only has a ``Time`` type and knows nothing about Ruby. + + +Compiling Regexes +----------------- + +When regular expressions are deserialized, they return a wrapper that holds the +raw regex string, but does not compile it. In order to get the Ruby ``Regexp`` +object, one must call ``compile`` on the returned object. + +.. code-block:: ruby + + regex = Regexp.from_bson(byte_buffer) + regex.pattern #=> Returns the pattern as a string. + regex.options #=> Returns the raw options as an int. + regex.compile #=> Returns the compiled Ruby Regexp object. From fa29c5235937aad6c7c214436c651e1048ccc92e Mon Sep 17 00:00:00 2001 From: kay Date: Thu, 28 Jul 2016 16:12:08 -0400 Subject: [PATCH 092/442] Add docs for ruby driver manual --- source/contribute.txt | 46 ++ .../older-server-versions-unsupported.rst | 1 + ...uby-driver-compatibility-full-language.rst | 2 + ...ruby-driver-compatibility-full-mongodb.rst | 2 + ...y-driver-compatibility-matrix-language.rst | 6 + ...by-driver-compatibility-matrix-mongodb.rst | 6 + source/includes/unicode-checkmark.rst | 1 + source/index.txt | 67 +++ source/installation.txt | 69 +++ source/quick-start.txt | 244 ++++++++++ source/reference/additional-resources.txt | 200 +++++++++ source/reference/driver-compatibility.txt | 126 ++++++ source/ruby-driver-tutorials.txt | 28 ++ source/tutorials/ruby-driver-admin-tasks.txt | 373 ++++++++++++++++ source/tutorials/ruby-driver-aggregation.txt | 116 +++++ .../tutorials/ruby-driver-bulk-operations.txt | 93 ++++ .../ruby-driver-collection-tasks.txt | 103 +++++ .../tutorials/ruby-driver-create-client.txt | 339 ++++++++++++++ .../tutorials/ruby-driver-crud-operations.txt | 417 ++++++++++++++++++ .../ruby-driver-geospatial-search.txt | 106 +++++ source/tutorials/ruby-driver-gridfs.txt | 261 +++++++++++ source/tutorials/ruby-driver-indexing.txt | 109 +++++ source/tutorials/ruby-driver-projections.txt | 67 +++ source/tutorials/ruby-driver-text-search.txt | 44 ++ 24 files changed, 2826 insertions(+) create mode 100644 source/contribute.txt create mode 100644 source/includes/older-server-versions-unsupported.rst create mode 100644 source/includes/ruby-driver-compatibility-full-language.rst create mode 100644 source/includes/ruby-driver-compatibility-full-mongodb.rst create mode 100644 source/includes/ruby-driver-compatibility-matrix-language.rst create mode 100644 source/includes/ruby-driver-compatibility-matrix-mongodb.rst create mode 100644 source/includes/unicode-checkmark.rst create mode 100644 source/index.txt create mode 100644 source/installation.txt create mode 100644 source/quick-start.txt create mode 100644 source/reference/additional-resources.txt create mode 100644 source/reference/driver-compatibility.txt create mode 100644 source/ruby-driver-tutorials.txt create mode 100644 source/tutorials/ruby-driver-admin-tasks.txt create mode 100644 source/tutorials/ruby-driver-aggregation.txt create mode 100644 source/tutorials/ruby-driver-bulk-operations.txt create mode 100644 source/tutorials/ruby-driver-collection-tasks.txt create mode 100644 source/tutorials/ruby-driver-create-client.txt create mode 100644 source/tutorials/ruby-driver-crud-operations.txt create mode 100644 source/tutorials/ruby-driver-geospatial-search.txt create mode 100644 source/tutorials/ruby-driver-gridfs.txt create mode 100644 source/tutorials/ruby-driver-indexing.txt create mode 100644 source/tutorials/ruby-driver-projections.txt create mode 100644 source/tutorials/ruby-driver-text-search.txt diff --git a/source/contribute.txt b/source/contribute.txt new file mode 100644 index 000000000..2e892342b --- /dev/null +++ b/source/contribute.txt @@ -0,0 +1,46 @@ +======================== +Contribute to the Driver +======================== + +.. default-domain:: mongodb + +For Bugs and Feature Requests +----------------------------- + +If you think you have found a bug or want to see a new feature in the +Ruby driver, please open an issue in `JIRA +`_: + +#. `Create a Jira account and login `_. + +#. Navigate to the `Ruby project + `_. + +#. Click :guilabel:`Create Issue`. Provide as much information as + possible about the issue, including: + + - Steps to reproduce it. + + - Version information for the driver and MongoDB. + + - Any applicable code snippets, stack traces and log data. Do not + include any sensitive data or server logs. + +.. note:: + + Bug reports in JIRA for the driver and the Core Server (i.e. **SERVER**) + projects are public. + +If you identify a security vulnerability in a driver or any other +MongoDB project, please report it according to the instructions found +in the :manual:`Create a Vulnerability Report +`. + +Contribute to the MongoDB Ruby Driver +------------------------------------- + +The MongoDB Ruby driver source is located at +``_. + +For instructions in contributing to the driver, see +``_. diff --git a/source/includes/older-server-versions-unsupported.rst b/source/includes/older-server-versions-unsupported.rst new file mode 100644 index 000000000..d99330fcf --- /dev/null +++ b/source/includes/older-server-versions-unsupported.rst @@ -0,0 +1 @@ +The driver does not support older versions of MongoDB. \ No newline at end of file diff --git a/source/includes/ruby-driver-compatibility-full-language.rst b/source/includes/ruby-driver-compatibility-full-language.rst new file mode 100644 index 000000000..945edf942 --- /dev/null +++ b/source/includes/ruby-driver-compatibility-full-language.rst @@ -0,0 +1,2 @@ +For additional driver versions, see :ref:`Ruby Driver Language Compatibility Reference `. + diff --git a/source/includes/ruby-driver-compatibility-full-mongodb.rst b/source/includes/ruby-driver-compatibility-full-mongodb.rst new file mode 100644 index 000000000..9cc1f5f6a --- /dev/null +++ b/source/includes/ruby-driver-compatibility-full-mongodb.rst @@ -0,0 +1,2 @@ +For additional driver versions, see :ref:`Ruby Driver MongoDB Compatibility Reference `. + diff --git a/source/includes/ruby-driver-compatibility-matrix-language.rst b/source/includes/ruby-driver-compatibility-matrix-language.rst new file mode 100644 index 000000000..a94fe600a --- /dev/null +++ b/source/includes/ruby-driver-compatibility-matrix-language.rst @@ -0,0 +1,6 @@ +The following compatibility table specifies the recommended +version(s) of the MongoDB Ruby driver for use with a specific version of +Ruby. + +The first column lists the driver version(s). + diff --git a/source/includes/ruby-driver-compatibility-matrix-mongodb.rst b/source/includes/ruby-driver-compatibility-matrix-mongodb.rst new file mode 100644 index 000000000..40003c7ab --- /dev/null +++ b/source/includes/ruby-driver-compatibility-matrix-mongodb.rst @@ -0,0 +1,6 @@ +The following compatibility table specifies the recommended +version(s) of the MongoDB Ruby driver for use with a specific version of +MongoDB. + +The first column lists the driver version(s). + diff --git a/source/includes/unicode-checkmark.rst b/source/includes/unicode-checkmark.rst new file mode 100644 index 000000000..16f4e9476 --- /dev/null +++ b/source/includes/unicode-checkmark.rst @@ -0,0 +1 @@ +.. |checkmark| unicode:: U+2713 diff --git a/source/index.txt b/source/index.txt new file mode 100644 index 000000000..1a4052857 --- /dev/null +++ b/source/index.txt @@ -0,0 +1,67 @@ +.. http://www.mongodb.org/display/DOCS/Ruby+Language+Center + +.. _ruby-language-center: + +=================== +Ruby MongoDB Driver +=================== + +.. default-domain:: mongodb + +The MongoDB Ruby driver is the officially supported Ruby driver for +MongoDB. It is written in pure Ruby and is optimized for simplicity. It +can be used on its own, but it also serves as the basis of several +object mapping libraries. + +Get Started +----------- + +To get started with the Ruby driver, see :doc:`/installation` and +:doc:`/quick-start`. + +BSON +---- + +The Ruby BSON implementation is packaged in a separate gem with C and +Java extensions for speed depending on the runtime enviroment. + +For reference on the Ruby BSON gem, see :doc:`/bson-tutorials`. + +Object Mappers +-------------- + +Because MongoDB is so easy to use, the basic Ruby driver can be the +best solution for many applications. But if you need validations, +associations, and other high-level data modeling functions, then you +may need Object Document Mapper may be needed. + +In the context of a Rails application, an Object Document Mapper +provides functionality equivalent to, but distinct from, ActiveRecord. +Because MongoDB is a document-based database, these mappers are called +Object Document Mappers (ODM) as opposed to Object Relational Mappers +(ORM). + +The ODM officially supported by MongoDB is Mongoid, originally written +by Durran Jordan. + +For tutorials on Mongoid, see :doc:`/mongoid-tutorials`. + +.. COMMENT For the actual build, see mongodb/docs-ruby repo which pulls the documentation source from: +.. mongo-ruby-driver, +.. bson-ruby, and +.. mongoid repos. + +.. class:: hidden + + .. toctree:: + :titlesonly: + + installation + quick-start + ruby-driver-tutorials + bson-tutorials + mongoid-tutorials + API + /reference/driver-compatibility + /reference/additional-resources + /contribute diff --git a/source/installation.txt b/source/installation.txt new file mode 100644 index 000000000..4dfc3ab68 --- /dev/null +++ b/source/installation.txt @@ -0,0 +1,69 @@ +============ +Installation +============ + +.. default-domain:: mongodb + +The Ruby driver is bundled as a gem and is hosted on `Rubygems +`_. + +.. _ruby-driver-install: + +Install the gem +--------------- + +The driver can be installed manually or with bundler. + +To install the gem manually: + +.. code-block:: sh + + gem install mongo + +To install the gem with bundler, include the following in your Gemfile: + +.. code-block:: ruby + + gem 'mongo', '~> 2.2' + +Note the following compatibility matrix to determine if the driver is +supported on your Ruby version and MongoDB server version. + +.. list-table:: + :header-rows: 1 + :widths: 28 30 30 30 + + * - Ruby Version + - MongoDB Server 2.4.x + - MongoDB Server 2.6.x + - MongoDB Server 3.0.x + + * - 1.8.x + - No + - No + - No + + * - 1.9.x + - Yes + - Yes + - Yes + + * - 2.0.x + - Yes + - Yes + - Yes + + * - 2.1.x + - Yes + - Yes + - Yes + + * - 2.2.x + - Yes + - Yes + - Yes + + * - JRuby 1.7.x + - Yes + - Yes + - Yes \ No newline at end of file diff --git a/source/quick-start.txt b/source/quick-start.txt new file mode 100644 index 000000000..a2389a99e --- /dev/null +++ b/source/quick-start.txt @@ -0,0 +1,244 @@ +=========== +Quick Start +=========== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Prerequisites +------------- + +- A running MongoDB instance on localhost using the default port, 27017. +- The Ruby MongoDB driver. See :ref:`installation ` + for instructions on how to install the MongoDB driver. +- The following statement at the top of your code: + +.. code-block:: ruby + + require 'mongo' + + +Make a Connection +----------------- + +Use ``Mongo::Client`` to establish a connection to a running MongoDB +instance. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + +You can also use a URI connection string: + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + +.. seealso:: + :ref:`Connect to a replica set `, + :ref:`Connect to a sharded cluster `, + :ref:`Client options ` + +Access a Database and a Collection +---------------------------------- + +The following examples demonstrate how to access a particular database +and show its collections: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + db = client.database + + db.collections # returns a list of collection objects + db.collection_names # returns a list of collection names + +To access a collection, refer to it by name. + +.. code-block:: ruby + + collection = client[:restaurants] + +If the collection does not exist, the server will create it the first +time you put data into it. + +Insert a Document +----------------- + +To insert a single document into a collection, use the +``insert_one`` method. + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + + collection = client[:people] + + doc = { name: 'Steve', hobbies: [ 'hiking', 'tennis', 'fly fishing' ] } + + result = collection.insert_one(doc) + result.n # returns 1, because one document was inserted + +To insert multiple documents into a collection, use the +``insert_many`` method. + +.. code-block:: ruby + + docs = [ { _id: 1, name: 'Steve', hobbies: [ 'hiking', 'tennis', 'fly fishing' ] }, + { _id: 2, name: 'Sally', hobbies: ['skiing', 'stamp collecting' ] } ] + + result = collection.insert_many(docs) + result.inserted_count # returns 2 because two documents were inserted + +Query the Collection +-------------------- + +Use the ``find`` method to create collection queries. + +An empty query filter returns all documents in the collection. + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:people] + + collection.find.each do |document| + #=> Yields a BSON::Document. + end + +Use a query filter to find only matching documents. + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:people] + + puts collection.find( { name: 'Sally' } ).first + +The example should print the following: + +.. code-block:: javascript + + {"_id" => 2, "name" => "Sally", "hobbies" => ["skiing", "stamp collecting"]} + +.. seealso:: + + :ref:`Query Options`, :ref:`Read Preference` + +Update Documents +---------------- + +There are several update methods, including ``update_one`` and +``update_many``. ``update_one`` updates a single document, while +``update_many`` updates multiple documents at once. + +Both methods take as arguments a query filter document and a second +document with the update data. Use ``$set`` to add or update a +particular field or fields. Without ``$set``, the entire existing +document is replaced with the update data. + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:people] + + result = collection.update_one( { 'name' => 'Sally' }, { '$set' => { 'phone_number' => "555-555-5555" } } ) + + puts collection.find( { 'name' => 'Sally' } ).first + +The example should print the following: + +.. code-block:: javascript + + {"_id" => 2, "name" => "Sally", "hobbies" => ["skiing", "stamp collecting"], "phone_number" => "555-555-5555"} + +The following example uses ``update_many`` with a blank query filter +to update all the documents in the collection. + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:people] + + result = collection.update_many( {}, { '$set' => { 'age' => 36 } } ) + + puts result.modified_count # returns 2 because 2 documents were updated + +.. seealso:: + + :ref:`Other update options` + +Delete Documents +---------------- + +Use the ``delete_one`` or ``delete_many`` methods to delete documents +from a collection (either singly or several at once). + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:people] + + result = collection.delete_one( name 'Steve' ) + + puts result.deleted_count # returns 1 because one document was deleted + +The following example inserts two more records into the collection, +then deletes all the documents with a ``name`` field which +matches a regular expression to find a string which begins with "S". + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:people] + + collection.insert_many([ { _id: 3, name: "Arnold" }, { _id: 4, name: "Susan" } ]) + + puts collection.count # counts all documents in collection + + result = collection.delete_many({ name: /$S*/ }) + + puts result.deleted_count # returns the number of documents deleted + +Create Indexes +-------------- + +Use the ``create_one`` or ``create_many`` methods to create indexes +singly or several at once. + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:people] + + collection.indexes.create_one({ name: 1 }, unique: true) + +Use the ``create_many`` method to create several indexes with one +statement. Note that when using ``create_many``, the syntax is +different from ``create_one``. + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:people] + + collection.indexes.create_many([ + { key: { name: 1 } , unique: true }, + { key: { hobbies: 1 } }, + ]) + +.. seealso:: + + :ref:`Index options ` + +Complete Sample App +------------------- + +A sample app using the Ruby driver for several common use cases +is available for download from +`GitHub `_. diff --git a/source/reference/additional-resources.txt b/source/reference/additional-resources.txt new file mode 100644 index 000000000..c2195590e --- /dev/null +++ b/source/reference/additional-resources.txt @@ -0,0 +1,200 @@ +.. _ruby-external-resources: + +==================== +Additional Resources +==================== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: twocols + +There are a number of good resources appearing all over the web for +learning about MongoDB and Ruby. A useful selection is listed below. If +you know of others, do let us know. + +Screencasts +----------- + +- `Introduction to MongoDB - Part I + `_ + + An introduction to MongoDB via the MongoDB shell. + +- `Introduction to MongoDB - Part II + `_ + + In this screencast, Joon You teaches how to use the Ruby driver to + build a simple Sinatra app. + +- `Introduction to MongoDB - Part III + `_ + + For the final screencast in the series, Joon You introduces + MongoMapper and Rails. + +- `RailsCasts: MongoDB & MongoMapper + `_ + + Ryan Bates' RailsCast introducing MongoDB and MongoMapper. + +- `RailsCasts: Mongoid `_ + + Ryan Bates' RailsCast introducing Mongoid. + +Presentations +------------- + +- `Introduction to MongoDB (Video) `_ + + Mike Dirolf's introduction to MongoDB at Pivotal Labs, SF. + +- `MongoDB: A Ruby Document Store that doesn't rhyme with 'Ouch' + (Slides) + `_ + + Wynn Netherland's introduction to MongoDB with some comparisons to + CouchDB. + +- `MongoDB (is) for Rubyists (Slides) + `_ + + Kyle Banker's presentation on why MongoDB is for Rubyists (and all + human-oriented programmers). + +Articles +-------- + +- `Why I Think Mongo is to Databases What Rails was to Frameworks + `_ + +- `What if a key-value store mated with a relational database system? + `_ + +- `Mongo Tips `_ + + John Nunemaker's articles on MongoDB and his Mongo Tips blog. + +- A series of articles on aggregation with MongoDB and Ruby: + + 1. `Part I: Introduction of Aggregation in MongoDB + `_ + + #. `Part II: MongoDB Grouping Elaborated + `_ + + #. `Part III: Introduction to Map-Reduce in MongoDB + `_ + +- `Does the MongoDB Driver Support Feature X? + `_ + + An explanation of how the MongoDB drivers usually automatically + support new database features. + +Projects +-------- + +- `Capistrano Mongo Sync `_ + + Sync your local development db with your remote production db using capistrano. + +- `Simple Pub/Sub `_ + + A very simple pub/sub system. + +- `Mongo Queue `_ + + An extensible thread safe job/message queueing system that uses + MongoDB as the persistent storage engine. + +- `Resque-mongo `_ + + A port of the Github's Resque to MongoDB. + +- `Mongo Admin `_ + + A Rails plugin for browsing and managing MongoDB data. See the `live + demo `_. + +- `Sinatra Resource `_ + + Resource Oriented Architecture (REST) for Sinatra and MongoMapper. + +- `NewsMonger `_ + + A simple social news application demonstrating MongoMapper and Rails. + +- `Data Catalog API `_ + + From `Sunlight Labs `_, a non-trivial + application using MongoMapper and Sinatra. + +- `Watchtower `_ + + An example application using Mustache, MongoDB, and Sinatra. + +- `Shapado `_ + + A question and answer site similar to Stack Overflow. Live version at + `shapado.com `_. + +.. Does not seem to exist +.. - `Shorty `_ +.. A URL-shortener written with Sinatra and the MongoDB Ruby driver. + +Libraries +--------- + +- `ActiveExpando `_ + + An extension to ActiveRecord to allow the storage of arbitrary + attributes in MongoDB. + +- `ActsAsTree (MongoMapper) + `_ + + ActsAsTree implementation for MongoMapper. + +- `Machinist adapter (MongoMapper) + `_ + + Machinist adapter using MongoMapper. + +- `Mongo-Delegate `_ + + A delegation library for experimenting with production data without + altering it. A quite useful pattern. + +- `Remarkable Matchers (MongoMapper) + `_ + + Testing / Matchers library using MongoMapper. + +- `OpenIdAuthentication, supporting MongoDB as the datastore + `_ + + Brandon Keepers' fork of OpenIdAuthentication supporting MongoDB. + +- `MongoTree (MongoRecord) + `_ + + MongoTree adds parent / child relationships to MongoRecord. + +- `Merb_MongoMapper + `_ + + A plugin for the Merb framework for supporting MongoMapper models. + +- `Mongolytics (MongoMapper) + `_ + + A web analytics tool. + +- `Rack-GridFS `_ + + A Rack middleware component that creates HTTP endpoints for files + stored in GridFS. diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt new file mode 100644 index 000000000..d9acd5edc --- /dev/null +++ b/source/reference/driver-compatibility.txt @@ -0,0 +1,126 @@ +.. _reference-compatibility-ruby: + +==================== +Driver Compatibility +==================== + +.. default-domain:: mongodb + +.. _reference-compatibility-mongodb-ruby: + +MongoDB Compatibility +~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/ruby-driver-compatibility-matrix-mongodb.rst + +.. list-table:: + :header-rows: 1 + :stub-columns: 1 + :widths: 20 20 20 20 20 + :class: compatibility + + * - Ruby Driver + - MongoDB 2.4 + - MongoDB 2.6 + - MongoDB 3.0 + - MongoDB 3.2 + + * - 2.2 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + + * - 2.0 + - |checkmark| + - |checkmark| + - |checkmark| + - + + * - 1.12 + - |checkmark| + - |checkmark| + - |checkmark| + - + + * - 1.11 + - |checkmark| + - |checkmark| + - + - + + * - 1.10 + - |checkmark| + - |checkmark| + - + - + + * - 1.9 + - |checkmark| + - + - + - + + * - 1.8 + - |checkmark| + - + - + - + +.. include:: /includes/older-server-versions-unsupported.rst + +.. _reference-compatibility-language-ruby: + +Language Compatibility +~~~~~~~~~~~~~~~~~~~~~~ + +.. include:: /includes/ruby-driver-compatibility-matrix-language.rst + +.. list-table:: + :header-rows: 1 + :stub-columns: 1 + :class: compatibility + + * - Ruby Driver + - Ruby 1.8.7 + - Ruby 1.9 + - Ruby 2.0 + - Ruby 2.1 + - JRuby + + * - 2.0 + - + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + + * - 1.12 - 1.9 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + + * - 1.8 + - |checkmark| + - |checkmark| + - |checkmark| + - + - |checkmark| + + * - 1.7 + - |checkmark| + - |checkmark| + - + - + - |checkmark| + + * - 1.6 + - |checkmark| + - |checkmark| + - + - + - |checkmark| + +.. include:: /includes/unicode-checkmark.rst diff --git a/source/ruby-driver-tutorials.txt b/source/ruby-driver-tutorials.txt new file mode 100644 index 000000000..1ee606660 --- /dev/null +++ b/source/ruby-driver-tutorials.txt @@ -0,0 +1,28 @@ +.. http://docs.mongodb.org/ecosystem/tutorial/ruby-driver-tutorial/ + +.. _ruby-driver-tutorial: + +========= +Tutorials +========= + +.. default-domain:: mongodb + +The tutorials in this section provide examples of some frequently used +operations. This section is not meant to be an exhaustive list of all +operations available in the Ruby driver. + +.. toctree:: + :titlesonly: + + /tutorials/ruby-driver-create-client + /tutorials/ruby-driver-crud-operations + /tutorials/ruby-driver-collection-tasks + /tutorials/ruby-driver-projections + /tutorials/ruby-driver-admin-tasks + /tutorials/ruby-driver-indexing + /tutorials/ruby-driver-aggregation + /tutorials/ruby-driver-bulk-operations + /tutorials/ruby-driver-text-search + /tutorials/ruby-driver-geospatial-search + /tutorials/ruby-driver-gridfs diff --git a/source/tutorials/ruby-driver-admin-tasks.txt b/source/tutorials/ruby-driver-admin-tasks.txt new file mode 100644 index 000000000..56919a7a5 --- /dev/null +++ b/source/tutorials/ruby-driver-admin-tasks.txt @@ -0,0 +1,373 @@ +============== +Administration +============== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Databases +--------- + +The driver provides various helpers on database objects for executing +commands, getting collection lists, and administrative tasks. + +List Collections +```````````````` + +To get a list of collections or collection names for a database, use +``collections`` and ``collection_names``, respectively. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + database = client.database + + database.collections # Returns an array of Collection objects. + database.collection_names # Returns an array of collection names as strings. + +To execute any command on the database, use the ``command`` method. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + database = client.database + + result = database.command(:ismaster => 1) + result.first # Returns the BSON::Document returned from the server. + +Drop Database +````````````` + +To drop a database, use the ``drop`` method. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client.database.drop + +Collections +----------- + +The driver provides some helpers for administrative tasks with +collections. + +To create a collection with options (such as creating a capped collection), +pass the options when getting the collection from the client, then call +``create``. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists, :capped => true, :size => 1024] + artists.create + artists.capped? # Returns true. + +Drop Collection +``````````````` + +To drop a collection, call ``drop`` on the collection object. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] + artists.drop + +Changing Read/Write Preferences +``````````````````````````````` + +To change the default read preference or write concern for specific operations, +use the ``with`` method on the collection. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] + artists.with(:read => { :mode => :primary_preferred }).find.to_a + artists.with(:write => { :w => :3 }).insert_one( { :name => 'Depeche Mode' } ) + +Authentication +-------------- + +MongoDB supports a variety of +:manual:`authentication mechanisms `. + +For more information about configuring your MongoDB server for each of +these authentication mechanisms see MongoDB's +:manual:`online documentation `. + +Creating a user +``````````````` + +To create a user in specific database, use the ``create`` method with the +username, password and roles parameters. + +.. code-block:: ruby + + client.database.users.create( + 'durran', + password: 'password', + roles: [ Mongo::Auth::Roles::READ_WRITE ]) + +.. seealso:: + :manual:`Built-in roles` + +Providing credentials +````````````````````` + +If authentication is enabled, provide credentials when creating a new +client. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], + user: 'test', + password: '123' ) + +For MongoDB 2.6 and later, ``:auth_source`` defaults to **admin**, +otherwise the current database is used. + +The current database can be changed with the client's ``use`` method. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ]) + music_client = client.use( 'music') + +A new client can be created with the authentication credentials. + +.. code-block:: ruby + + authenticated_client = client.with( user: 'test', + password: '123' ) + +Alternatively, setting the current database and credentials can be done in one step: + +.. code-block:: ruby + + authenticated_music_client = client.with( :database => 'music', + user:'test', + password:'123' ) + + +MONGODB-CR Mechanism +```````````````````` + +MONGODB-CR was the default authentication mechanism for MongoDB up through version 2.6. + +The mechanism can be explicitly set with the credentials: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], + :database => 'music', + user: 'test', + password: '123', + auth_mech: :mongodb_cr ) + + +Client Certificate (x509) +````````````````````````` +*Requires MongoDB v2.6 or greater.* + +The driver presents an X.509 certificate during SSL negotiation. +The Client Certificate (x509) mechanism authenticates a username +derived from the distinguished subject name of this certificate. + +This authentication method requires the use of SSL connections with +certificate validation. + +For more information about configuring X.509 authentication in MongoDB, +see the :manual:`X.509 tutorial in the MongoDB Manual +`. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], + auth_mech: :mongodb_x509, + ssl: true, + ssl_cert: '/path/to/client.pem', + ssl_ca_cert: '/path/to/ca.pem' ) + + +LDAP (SASL PLAIN) mechanism +``````````````````````````` +*Requires MongoDB Enterprise Edition v2.6 or greater.* + +MongoDB Enterprise Edition supports the LDAP authentication mechanism +which allows you to delegate authentication using a Lightweight Directory +Access Protocol `LDAP `_ server. + +.. warning:: + + When using LDAP, passwords are sent to the server in plain text. For this + reason, we strongly recommend enabling SSL when using LDAP as your + authentication mechanism. + +For more information about configuring LDAP authentication in +MongoDB, see the :manual:`SASL/LDAP tutorial in the MongoDB Manual +`. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], + auth_mech: :plain, + ssl: true, + ssl_verify: true, + ssl_cert: '/path/to/client.pem', + ssl_ca_cert: '/path/to/ca.pem' ) + +Kerberos (GSSAPI) mechanism +``````````````````````````` +*Requires MongoDB Enterprise Edition v2.4 or greater.* + +MongoDB Enterprise Edition v2.4+ supports Kerberos authentication. + +To use Kerberos in the Ruby driver with **JRuby**, do the following: + +1. Specify several system properties so that the underlying GSSAPI Java + libraries can acquire a Kerberos ticket. See the `MongoDB Java + Driver authentication documentation + `_ + for more information. + +2. Either provide a password OR set the 'java.security.auth.login.config' + system property to a config file that references a keytab file. + +To use Kerberos in the Ruby driver +with **Matz's Ruby Interpreter (MRI)**, create a +ticket-granting ticket using ``kinit``. See +`this documentation `_ for more +information. + +For more information about deploying MongoDB with Kerberos +authentication, see the :manual:`manual +`. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], + auth_mech: :gssapi, + user: 'test', + password: '123' ) + +Logger +------ + +You can either use the default global driver logger or set your own. To set your own: + +.. code-block:: ruby + + Mongo::Logger.logger = other_logger + +See the `Ruby Logger documentation `_ +for more information on the default logger API and available levels. + +Changing the Logger Level +````````````````````````` + +To change the logger level: + +.. code-block:: ruby + + Mongo::Logger.logger.level = Logger::WARN + +For more control, a logger can be passed to a client for per-client control over logging. + +.. code-block:: ruby + + my_logger = Logger.new($stdout) + Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :logger => my_logger ) + +Truncation +`````````` + +The default logging truncates logs at 250 characters by default. To turn this off pass an +option to the client instance. + +.. code-block:: ruby + + Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :truncate_logs => false ) + +Monitoring +---------- + +All user-initiated commands that are sent to the server publish events that can be +subscribed to for fine grained information. The monitoring API publishes a guaranteed +start event for each command, then either a succeeded or failed event. A subscriber +must implement 3 methods: ``started``, ``succeeded``, and ``failed``, each which takes +a single parameter for the event. An example is the default logging subscriber included +in the driver: + +.. code-block:: ruby + + module Mongo + class Monitoring + class CommandLogSubscriber + include Loggable + + attr_reader :options + + LOG_STRING_LIMIT = 250 + + def initialize(options = {}) + @options = options + end + + def started(event) + log_debug("#{prefix(event)} | STARTED | #{format_command(event.command)}") + end + + def succeeded(event) + log_debug("#{prefix(event)} | SUCCEEDED | #{event.duration}s") + end + + def failed(event) + log_debug("#{prefix(event)} | FAILED | #{event.message} | #{event.duration}s") + end + + private + + def format_command(args) + begin + truncating? ? truncate(args) : args.inspect + rescue Exception + '' + end + end + + def prefix(event) + "#{event.address.to_s} | #{event.database_name}.#{event.command_name}" + end + + def truncate(command) + ((s = command.inspect).length > LOG_STRING_LIMIT) ? "#{s[0..LOG_STRING_LIMIT]}..." : s + end + + def truncating? + @truncating ||= (options[:truncate_logs] != false) + end + end + end + end + +To register a custom subscriber, you can do so globally for +all clients or on a per-client basis: + +.. code-block:: ruby + + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::COMMAND, my_subscriber) + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test' ) + client.subscribe( Mongo::Monitoring::COMMAND, my_subscriber ) + +To turn off monitoring, set the client monitoring option to ``false``: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :monitoring => false ) diff --git a/source/tutorials/ruby-driver-aggregation.txt b/source/tutorials/ruby-driver-aggregation.txt new file mode 100644 index 000000000..f3819015d --- /dev/null +++ b/source/tutorials/ruby-driver-aggregation.txt @@ -0,0 +1,116 @@ +=========== +Aggregation +=========== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +:manual:`Aggregation framework` +operations process data records and return +computed results. Aggregation operations group values from +multiple documents together, and can perform a variety of +operations on the grouped data to return a single result. + +The Aggregation Pipeline +```````````````````````` + +The aggregation pipeline is a framework for data aggregation +modeled on the concept of data processing pipelines. Documents +enter a multi-stage pipeline that transforms the documents into +aggregated results. + +For a full explanation and a complete list of pipeline stages +and operators, see the +:manual:`manual`. + +The following example uses the aggregation pipeline on the +``restaurants`` sample dataset to find +a list of the total number of 5-star restaurants, grouped by restaurant +category. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + coll = client['restaurants'] + aggregation = coll.aggregate([ + { '$match'=> { 'stars'=> 5 } }, + { '$unwind'=> '$categories'}, + { '$group'=> { '_id'=> '$categories', 'fiveStars'=> { '$sum'=> 1 } } } + ]) + + aggregation.each do |doc| + #=> Yields a BSON::Document. + end + +Inside the ``aggregate`` method, the first pipeline stage filters out +all documents except those with ``5`` in the ``stars`` field. The +second stage unwinds the ``categories`` field, which is an array, and +treats each item in the array as a separate document. The third stage +groups the documents by category and adds up the number of matching +5-star results. + +Aggregation pipeline stages have a +:manual:`maximum memory use limit`. +To handle large datasets, set the ``allowDiskUse`` option to true to enable +writing data to temporary files. + +- You can call the ``allow_disk_use`` method the ``aggregation`` + object to get a new object with the option set: + +.. code-block:: ruby + + aggregation = coll.aggregate([ ]) + aggregation_with_disk_use = aggregation.allow_disk_use(true) + +- Or you can pass an option to the ``aggregate`` method: + +.. code-block:: ruby + + aggregation = coll.aggregate([ ], + :allow_disk_use => true) + +Single Purpose Aggregation Operations +````````````````````````````````````` + +MongoDB provides helper methods for some aggregation functions, +including :manual:`count` +and :manual:`distinct`. + +Count +~~~~~ + +The following example demonstrates how to use the ``count`` method to +find the total number of documents which have the exact array +``[ 'Chinese', 'Seafood' ]`` in the ``categories`` field. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + coll = client['restaurants'] + aggregation = coll.count({ 'categories': [ 'Chinese', 'Seafood' ] }) + + count = coll.count({ 'categories' => [ 'Chinese', 'Seafood' ] }) + +Distinct +~~~~~~~~ + +The ``distinct`` helper method eliminates results which contain +values and returns one record for each unique value. + +The following example returns a list of unique values for the +``categories`` field in the ``restaurants`` collection: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + coll = client['restaurants'] + aggregation = coll.distinct('categories') + + aggregation.each do |doc| + #=> Yields a BSON::Document. + end diff --git a/source/tutorials/ruby-driver-bulk-operations.txt b/source/tutorials/ruby-driver-bulk-operations.txt new file mode 100644 index 000000000..bc044388a --- /dev/null +++ b/source/tutorials/ruby-driver-bulk-operations.txt @@ -0,0 +1,93 @@ +=============== +Bulk Operations +=============== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +The bulk write API sends a list of write operations to the server in +one method call. Execution order of the operations is guaranteed if +you pass in the ``ordered`` option. + +The ``bulk_write`` method takes three arguments: + +- A list of operations. +- The ``ordered`` option with a boolean. Defaults to ``true``. +- A write concern option. Defaults to the collection's write concern. + +Valid bulk write operations are the following: + +insert_one +---------- + +.. code-block:: ruby + + { :insert_one => { :x => 1 } } + +delete_one +---------- + +.. code-block:: ruby + + { :delete_one => { :filter => { :x => 1 } } } + +delete_many +----------- + +.. code-block:: ruby + + { :delete_many => { :filter => { :x => 1 } } } + +replace_one +----------- + +.. code-block:: ruby + + { :replace_one => { :filter => { :x => 1 }, + :replacement => { :x => 2 }, + :upsert => true } # upsert is optional and defaults to false + } + +update_one +---------- + +.. code-block:: ruby + + { :update_one => { :filter => { :x => 1 }, + :update => { '$set' => { :x => 2 } }, + :upsert => true } # upsert is optional and defaults to false + } + +update_many +----------- + +.. code-block:: ruby + + { :update_many => { :filter => { :x => 1 }, + :update => { '$set' => { :x => 2 } }, + :upsert => true } # upsert is optional and defaults to false + } + +The following example shows how to pass operations +to the ``bulk_write`` method. + +.. code-block:: ruby + + coll = client['documents'] + coll.bulk_write([ { :insert_one => { :x => 1 } + }, + { :update_one => { :filter => { :x => 1 }, + :update => {'$set' => { :x => 2 } } + } + }, + { :replace_one => { :filter => { :x => 2 }, + :replacement => { :x => 3 } + } + } + ], + :ordered => true) diff --git a/source/tutorials/ruby-driver-collection-tasks.txt b/source/tutorials/ruby-driver-collection-tasks.txt new file mode 100644 index 000000000..89389c3c9 --- /dev/null +++ b/source/tutorials/ruby-driver-collection-tasks.txt @@ -0,0 +1,103 @@ +=========== +Collections +=========== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +MongoDB stores documents in collections. If a collection does not +exist, MongoDB creates the collection when you first insert a +document in that collection. + +You can also explicitly create a collection with various options, +such as setting the maximum size or the documentation validation rules. + +Capped Collection +````````````````` + +Capped collections have maximum size or document counts that prevent +them from growing beyond maximum thresholds. All capped collections must +specify a maximum size and may also specify a maximum document count. +MongoDB removes older documents if a collection reaches the maximum size +limit before it reaches the maximum document count. + +To create a :manual:`capped collection`, use +the ``capped: true`` option along with a ``size`` in bytes. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client[:artists, capped: true, size: 10000].create + +Convert an Existing Collection to Capped +```````````````````````````````````````` + +To convert an existing collection from non-capped to capped, use +the ``convertToCapped`` command. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + db = client.database + db.command({ 'convertToCapped' => 'contacts', 'size' => 8192 }) + + +Document Validation +``````````````````` + +If you're using MongoDB version 3.2 or later, you can use +:manual:`document validation`. +Collections with validations compare each inserted or updated +document against the criteria specified in the validator option. +Depending on the ``validationLevel`` and ``validationAction``, MongoDB +either returns a warning, or refuses to insert or update the document +if it fails to meet the specified criteria. + +The following example creates a ``contacts`` collection with a validator +that specifies that inserted or updated documents should match at +least one of three following conditions: + +- the ``phone`` field is a string +- the ``email`` field matches the regular expression +- the ``status`` field is either ``Unknown`` or ``Incomplete``. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + client[:contacts, + + { + 'validator' => { '$or' => + [ + { 'phone' => { '$type' => "string" } }, + { 'email' => { '$regex' => /@mongodb\.com$/ } }, + { 'status' => { '$in' => [ "Unknown", "Incomplete" ] } } + ] + } + } + + ].create + +Add Validation to an Existing Collection +```````````````````````````````````````` + +To add document validation criteria to an existing collection, use the +``collMod`` command. The example below demonstrates how to add a +validation to the ``contacts`` collection, ensuring that all new +documents must contain an ``age`` field which is a number. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + db = client.database + db.command({ 'collMod' => 'contacts', + 'validator' => + { 'age' => + { '$type' => "number" } + } + }) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt new file mode 100644 index 000000000..5113f4cd5 --- /dev/null +++ b/source/tutorials/ruby-driver-create-client.txt @@ -0,0 +1,339 @@ +================= +Creating a Client +================= + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Using ``Mongo::Client`` +----------------------- +To start a Ruby driver connection, create a ``Mongo::Client`` object. +Provide a list of hosts and options or a connection URI to the +``Mongo::Client`` constructor. The client's oselected database +defaults to ``admin``. + +To create a client to a standalone server, provide one host in the +seed list. Optionally, you can force the cluster topology to be +standalone without going through the auto-discovery steps. + +.. code-block:: ruby + + Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb') + Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb', :connect => :direct) + Mongo::Client.new('mongodb://127.0.0.1:27017/mydb') + +.. _ruby-driver-connect-replica-set: + +To connect to a :manual:`replica set`, +pass one or more hosts and the replica set name. +The driver's auto-discovery feature finds all members of the replica +set if they are not all provided. + +.. code-block:: ruby + + Mongo::Client.new([ '127.0.0.1:27017', '127.0.0.1:27018' ], :database => 'mydb', replica_set: 'myapp') + Mongo::Client.new('mongodb://127.0.0.1:27017,127.0.0.1:27018/mydb?replicaSet=myapp') + +.. _ruby-driver-connect-sharded-cluster: + +To create a client to a :manual:`sharded cluster`, +pass one or more :manual:`mongos` +hosts. The auto-discovery feature can determine that the +servers are ``mongos`` instances, but if you +would like to bypass the auto-discovery, pass the +``sharded`` option to the client. + +.. code-block:: ruby + + Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb') + Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb', :connect => :sharded) + Mongo::Client.new('mongodb://127.0.0.1:27017/mydb?connect=sharded') + +.. _ruby-driver-client-options: + +Client Options +-------------- + +A number of different options can be passed to a ``Mongo::Client`` to configure driver +behavior, either by providing them in the options hash to the constructor or by +providing them in the URI. + +Since the URI options are required in camel case, which is not the Ruby standard, the +following table shows the option in the URI and its corresponding option if passed +to the constructor in Ruby. + +.. note:: + + The options passed directly should be symbols. + +The options are explained in detail in the :manual:`Connection URI reference +`. + +.. note:: + Options that are set in **milliseconds** in the URI are + represented as a ``float`` in Ruby and the units are **seconds**. + +URI Options Conversions +----------------------- + +.. list-table:: + :header-rows: 1 + :widths: 40 105 + + * - URI Option + - Ruby Option + + * - replicaSet=String + - ``:replica_set => String`` + + * - connect=String + - ``:connect => Symbol`` + + * - ssl=Boolean + - ``:ssl => true|false`` + + * - connectTimeoutMS=Integer + - ``:connect_timeout => Float`` + + * - socketTimeoutMS=Integer + - ``:socket_timeout => Float`` + + * - serverSelectionTimeoutMS=Integer + - ``:server_selection_timeout => Float`` + + * - localThresholdMS=Integer + - ``:local_threshold => Float`` + + * - maxPoolSize=Integer + - ``:max_pool_size => Integer`` + + * - minPoolSize=Integer + - ``:min_pool_size => Integer`` + + * - waitQueueTimeoutMS=Integer + - ``:wait_queue_timeout => Float`` + + * - w=Integer|String + - ``{ :write => { :w => Integer|String }}`` + + * - wtimeoutMS=Integer + - ``{ :write => { :wtimeout => Float }}`` + + * - journal=Boolean + - ``{ :write => { :j => true|false }}`` + + * - fsync=Boolean + - ``{ :write => { :fsync => true|false }}`` + + * - readPreference=String + - ``{ :read => { :mode => Symbol }}`` + + * - readPreferenceTags=Strings + - ``{ :read => { :tag_sets => Array }}`` + + * - authSource=String + - ``:auth_source => String`` + + * - authMechanism=String + - ``:auth_mech => Symbol`` + + * - authMechanismProperties=Strings + - ``{ :auth_mech_properties => { :service_realm => String, :canonicalize_host_name => true|false, :service_name => String } }`` + + +Ruby Options +------------ + +.. list-table:: + :header-rows: 1 + :widths: 25 40 10 15 + + * - Option + - Description + - Type + - Default + + * - ``:replica_set`` + - When connecting to a replica set, this is the name of the set to + filter servers by. + - ``String`` + - none + + * - ``:ssl`` + - Tell the client to connect to the servers via SSL. + - ``Boolean`` + - false + + * - ``:2out`` + - The number of seconds to wait to establish a socket connection + before raising an exception. + - ``Float`` + - 10 seconds + + * - ``:socket_timeout`` + - The number of seconds to wait for an operation to execute on a + socket before raising an exception. + - ``Float`` + - 5 seconds + + * - ``:max_pool_size`` + - The maximum size of the connection pool for each server. + - ``Integer`` + - 5 + + * - ``:min_pool_size`` + - The minimum number of connections in the connection pool for each + server. + - ``Integer`` + - 1 + + * - ``:wait_queue_timeout`` + - The number of seconds to wait for a connection in the connection + pool to become available. + - ``Float`` + - 1 + + * - ``:write`` + - Specifies write concern options as a ``Hash``. + Keys in the hash can be ``:w``, ``:wtimeout``, ``:j``, ``:fsync``. + + .. code-block:: ruby + + { :write => { :w => 2 } } + - ``Hash`` + - ``{ :w => 1 }`` + + * - ``:read`` + - Specifies the read preference mode and tag sets for selecting servers as a ``Hash``. + Keys in the hash are ``:mode`` and ``:tag_sets``. + + .. code-block:: ruby + + { :read => + { :mode => :secondary, + :tag_sets => [ "berlin" ] + } + } + + - ``Hash`` + - ``{ :mode => :primary }`` + + * - ``:auth_source`` + - Specifies the authentication source. + - ``String`` + - For MongoDB 2.6 and later: **admin** if credentials are + supplied, otherwise the current database + + + * - ``:auth_mech`` + - Specifies the authenticaion mechanism to use. Can be one of: + ``:mongodb_cr``, ``:mongodb_x509``, ``:plain``, ``:scram``. + - ``Symbol`` + - MongoDB 3.0 and later: ``:scram`` if user credentials + are supplied but an ``:auth_mech`` is not. 2.6 and earlier: + ``:mongodb_cr`` + + * - ``:auth_mech_properties`` + - Provides additional authentication mechanism properties. + - ``Hash`` + - none + + * - ``:user`` + - The name of the user to authenticate with. + - ``String`` + - none + + * - ``:password`` + - The password of the user to authenticate with. + - ``String`` + - none + + * - ``:connect`` + - Overrides the auto-discovery feature of the driver and forces the cluster + topology to a specific type. Choices: ``:direct``, + ``:replica_set`` or ``:sharded``. + - ``Symbol`` + - none + + * - ``:heartbeat_frequency`` + - The number of seconds for the server monitors to refresh + server states asynchronously. + - ``Float`` + - 10 + + * - ``:database`` + - The name of the database to connect to. + - ``String`` + - admin + + * - ``:server_selection_timeout`` + - The number of seconds to wait for an appropriate server to + be selected for an operation to be executed before raising an exception. + - ``Float`` + - 30 + + * - ``:local_threshold`` + - Specifies the maximum latency in seconds between the nearest + server and the servers that can be available for selection to operate on. + - ``Float`` + - 0.015 + + + +Details on Timeout Options +-------------------------- + +``connect_timeout`` + On initialization of a connection to a server, this setting is the + number of seconds to wait to connect before raising an exception. + This timeout is also used when monitor threads ping their servers. + The default is 10 seconds. See the `socket timeout for monitoring + specification `_ for further explanation. + +``socket_timeout`` + The number of seconds to wait for an operation to + execute on a socket before raising an exception. It should take into + account network latency and operation duration. Defaults to 5 seconds. + See the `socket timeout for monitoring specification `_ + documentation for further information. + +``server_selection_timeout`` + The number of seconds to wait for the driver to find an appropriate server to + which an operation can be sent before raising an exception. Defaults to 30. + It should take the speed of :manual:`elections` + during a failover into account. See the + `serverSelectionTimeoutMS specification + `_ + for further information. + +``local_threshold`` + The maximum latency in seconds between the nearest server and the servers that can be considered available to send an + operation to. Defaults to 0.015. + +.. note:: + This is not the latency window between the driver and a server, but + rather the latency between the nearest server and other servers. See + `the localThresholdMS specification + `_. + +``wait_queue_timeout`` + The number of seconds to wait for a connection in the connection pool to + become available. You should consider increasing this + number if you are seeing many ``Timeout`` errors while using many threads + or when operations are long-running. Defaults to 1 second. + +``max_pool_size`` + Maximum size of the connection pool for each server. Defaults to 5 connections. + +``min_pool_size`` + Minimum number of connections in the connection pool for each server. + Increase this number to create connections when the pool is + initialized and to reduce the overhead of creating new connections + later on. Defaults to 1. diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt new file mode 100644 index 000000000..f42c34b25 --- /dev/null +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -0,0 +1,417 @@ +=============== +CRUD Operations +=============== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +CRUD operations are those which deal with creating, reading, updating, +and deleting documents. + +Key-value Pair Notation +----------------------- + +Key-value pairs appear in many different contexts in the MongoDB Ruby +driver, and there are some quirks of syntax with regard to how they can +be notated which depend on which version of Ruby you're using. + +When constructing a document, the following syntax is acceptable and +correct for Ruby version 1.9 and later: + +.. code-block:: javascript + + document = { name: "Harriet", age: 36 } + +If you're using Ruby version 2.2 or greater, you can optionally enclose +your keys in quotes. + +.. code-block:: javascript + + document = { "name": "Harriet", "age": 36 } + +If you need to use any MongoDB operator which begins with ``$``, +such as ``$set``, ``$gte``, or ``$near``, you must enclose it in +quotes. If you're using Ruby version 2.2 or greater, you can notate +it as follows: + +.. code-block:: ruby + + collection.update_one({ name: "Harriet" }, { "$set": { age: 42 } }) + +If you're using an earlier version of Ruby, use the hashrocket symbol: + +.. code-block:: ruby + + collection.update_one({ name: "Harriet" }, { "$set" => { age: 42 } }) + +Quoted strings and hashrockets for key-value pairs will work with any +version of Ruby: + +.. code-block:: ruby + + collection.update_one({ "name" => "Harriet" }, { "$set" => { age: 42 } }) + + +Creating Documents +------------------ + +To insert documents into a collection, select a +collection on the client and call ``insert_one`` or ``insert_many``. + +Insert operations return a ``Mongo::Operation::Result`` object which +gives you information about the insert itself. + +On MongoDB 2.6 and later, if the insert fails, an exception is +raised, because write commands are used. + +On MongoDB 2.4, an exception is only raised if the insert fails and the +:manual:`write concern` is 1 or higher. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + + result = client[:artists].insert_one( { :name => 'FKA Twigs' } ) + result.n # returns 1, because 1 document was inserted. + + result = client[:artists].insert_many([ + { :name => 'Flying Lotus' }, + { :name => 'Aphex Twin' } + ]) + result.inserted_count # returns 2, because 2 documents were inserted. + +Reading +------- + +The Ruby driver provides a fluent interface for queries using the ``find`` +method on the collection. Various options are available +to the ``find`` method. + +The query is lazily executed against the server only when iterating the +results - at that point the query is dispatched and a ``Mongo::Cursor`` is +returned. + +To find all documents for a given filter, call ``find`` with the +query: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + + client[:artists].find(:name => 'Flying Lotus').each do |document| + #=> Yields a BSON::Document. + end + +.. _ruby-driver-query-options: + +Query Options +````````````` + +To add options to a query, chain the appropriate methods after the +``find`` method. Note that the underlying object, the ``Mongo::Collection::View``, +is immutable and a new object will be returned after each method call. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + + documents = client[:artists].find(:name => 'Flying Lotus').skip(10).limit(10) + documents.each do |document| + #=> Yields a BSON::Document. + end + +The following is a full list of the available options that can be added +when querying and their corresponding methods as examples. + +.. list-table:: + :header-rows: 1 + :widths: 40 80 + + * - Option + - Description + * - ``allow_partial_results`` + - For use with sharded clusters. If a shard is down, allows the query + to return results from the shards that are up, potentially only getting + a portion of the results. + * - ``batch_size(Integer)`` + - Specifies the size of each batch of documents the cursor will return on + each ``GETMORE`` operation. + * - ``comment(String)`` + - Adds a comment to the query. + * - ``hint(Hash)`` + - Provides the query with an + :manual:`index hint` to use. + * - ``limit(Integer)`` + - Limits the number of returned documents to the provided value. + * - ``max_scan(Integer)`` + - Sets the maximum number of documents to scan if a full collection scan + would be performed. + * - ``no_cursor_timeout`` + - MongoDB automatically closes inactive cursors after a period of 10 + minutes. Call this for cursors to remain open indefinitely on the server. + * - ``projection(Hash)`` + - Specifies the fields to include or exclude from the results. + + .. code-block:: ruby + + client[:artists].find.projection(:name => 1) + + * - ``read(Hash)`` + - Changes the read preference for this query only. + + .. code-block:: ruby + + client[:artists].find.read(:mode => :secondary_preferred) + + * - ``show_disk_loc(Boolean)`` + - Tells the results to also include the location of the documents on disk. + * - ``skip(Integer)`` + - Skip the provided number of documents in the results. + * - ``snapshot`` + - Execute the query in snapshot mode. + * - ``sort(Hash)`` + - Specifies sort criteria for the query. + + .. code-block:: ruby + + client[:artists].find.sort(:name => -1) + +Additional Query Operations +``````````````````````````` + +``count`` + Get the total number of documents an operation returns. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + + client[:artists].find(:name => 'Flying Lotus').count + +``distinct`` + Filters out documents with duplicate values. Equivalent to the SQL + ``distinct`` clause. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + + client[:artists].find.distinct(:name ) + +Tailable Cursors +```````````````` + +For capped collections you may use a :manual:`tailable cursor +` that remains open +after the client exhausts the results in the initial cursor. The +following code example shows how a tailable cursor might be used: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client[:artists].drop + client[:artists, capped: true, size: 512].create + + result = client[:artists].insert_many([ + { :name => 'Flying Lotus' }, + { :name => 'Aphex Twin' } + ]) + + enum = client[:artists].find({}, cursor_type: :tailable_await).to_enum + + while true + doc = enum.next + # do something + sleep(1) + end + +.. _ruby-driver-read-preference: + +Read Preference +```````````````` + +Read preference determines the candidate :manual:`replica set` +members to which a query or command can be sent. They consist of a **mode** specified as +a symbol, an array of hashes known as **tag_sets**, and two timing options: +**local_threshold** and **server_selection_timeout**. + +``local_threshold`` + Defines the upper limit in seconds of the latency window + between the nearest server and suitable servers to which an operation may be sent. + The default is 15 milliseconds, or 0.015 seconds. + +``server_selection_timeout`` + Defines how long to block for server selection + before throwing an exception. The default is 30,000 milliseconds, or 30 seconds. + +For more information on the algorithm used to select a server, please +refer to the `Server Selection documentation, available on GitHub +`_. + +The read preference is set as an option on the client or passed an +option when a command is run on a database. + +.. code-block:: ruby + + # Set read preference on a client, used for all operations + client = Mongo::Client.new([ '127.0.0.1:27017' ], + read: { mode: :secondary, + tag_sets: [ { 'dc' => 'nyc' } ] + } ) + + # Set read preference for a given command + client.database.command( { collstats: 'test' }, read: { mode: secondary, + tag_sets: [ { 'dc' => 'nyc' } ] } ) + +Mode +```` + +There are five possible read preference modes. They are ``:primary``, +``:secondary``, ``:primary_preferred``, ``:secondary_preferred``, +``:nearest``. Please see the :manual:`read preference documentation in the MongoDB Manual +` for an explanation of the modes and tag sets. + +Tag sets +```````` + +The ``tag_sets`` parameter is an ordered list of tag sets used to +restrict the eligibility of servers for selection, such as for data +center awareness. + +A read preference tag set (T) matches a server tag set (S) – or +equivalently a server tag set (S) matches a read preference tag set +(T) — if T is a subset of S. + +For example, the read preference tag set ``{ dc: 'ny', rack: 2 }`` +matches a secondary server with tag set ``{ dc: 'ny', rack: 2, size: 'large' }``. + +A tag set that is an empty document matches any server, because +the empty tag set is a subset of any tag set. This means the default +``tag_sets`` parameter ``[{}]`` matches all servers. + +.. _ruby-driver-updating: + +Updating +-------- + +Updating documents is possible by executing a single or +multiple update, or by using the ``$findAndModify`` command. + +``update_one`` + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] + + result = artists.find(:name => 'Goldie').update_one("$inc" => { :plays => 1 } ) + result.n # Returns 1. + + result = artists.update_one( { :name => 'Goldie' }, { "$inc" => { :plays => 1 } } ) + result.n # Returns 1. + +``update_many`` + +.. code-block:: ruby + + result = artists.find(:label => 'Hospital').update_many( "$inc" => { :plays => 1 } ) + result.modified_count # Returns the number of documents that were updated. + + result = artists.update_many( { :label => 'Hospital' }, { "$inc" => { :plays => 1 } } ) + result.modified_count # Returns the number of documents that were updated. + +``replace_one`` + +.. code-block:: ruby + + result = artists.find(:name => 'Aphex Twin').replace_one(:name => 'Richard James') + result.modified_count # Returns 1. + + result = artists.replace_one( { :name => 'Aphex Twin' }, { :name => 'Richard James' } ) + result.modified_count # Returns 1. + +To update documents and return a document via ``$findAndModify``, use one of +the three provided helpers: ``find_one_and_delete``, ``find_one_and_replace``, +or ``find_one_and_update``. You can opt to return the document before or after +the modification occurs. + +``find_one_and_delete`` + +.. code-block:: ruby + + client = Mongo::Client.new( [ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] + + artists.find(:name => 'José James').find_one_and_delete # Returns the document. + +``find_one_and_replace`` + +.. code-block:: ruby + + doc = artists.find(:name => 'José James').find_one_and_replace(:name => 'José') + doc # Return the document before the update. + + doc = artists.find_one_and_replace({ :name => 'José James' }, { :name => 'José' }) + doc # Return the document before the update. + + doc = artists.find(:name => 'José James'). + find_one_and_replace( { :name => 'José' }, :return_document => :after ) + doc # Return the document after the update. + +``find_one_and_update`` + +.. code-block:: ruby + + doc = artists.find(:name => 'José James'). + find_one_and_update( '$set' => { :name => 'José' } ) + doc # Return the document before the update. + + doc = artists.find_one_and_update( { :name => 'José James' }, { '$set' => { :name => 'José' } } ) + doc # Return the document before the update. + +``find_one_and_replace`` + +.. code-block:: ruby + + doc = artists.find(:name => 'José James'). + find_one_and_replace( { '$set' => { :name => 'José' } }, :return_document => :after ) + doc # Return the document after the update. + + doc = artists.find_one_and_replace( + { :name => 'José James' }, + { '$set' => { :name => 'José' } }, + :return_document => :after + ) + doc # Return the document after the update. + +Deleting +-------- + +``delete_one`` + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] + + result = artists.find(:name => 'Björk').delete_one + result.deleted_count # Returns 1. + + result = artists.delete_one(:name => 'Björk') + result.deleted_count # Returns 1. + +``delete_many`` + +.. code-block:: ruby + + result = artists.find(:label => 'Mute').delete_many + result.deleted_count # Returns the number deleted. + + result = artists.delete_many(:label => 'Mute') + result.deleted_count # Returns the number deleted. diff --git a/source/tutorials/ruby-driver-geospatial-search.txt b/source/tutorials/ruby-driver-geospatial-search.txt new file mode 100644 index 000000000..bc6f6def6 --- /dev/null +++ b/source/tutorials/ruby-driver-geospatial-search.txt @@ -0,0 +1,106 @@ +================= +Geospatial Search +================= + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +MongoDB offers a number of indexes and query mechanisms to handle +geospatial information. This section demonstrates how to create +and use +:manual:`geospatial indexes` +with the Ruby driver. + +The examples on this page use a sample collection called +``restaurants`` in the ``test`` database. +A `sample dataset `_ +is available for download. + +The following is a sample document in the ``restaurants`` +collection: + +.. code-block:: javascript + + { + "address": { + "building": "1007", + "coord": [ -73.856077, 40.848447 ], + "street": "Morris Park Ave", + "zipcode": "10462" + }, + "borough": "Bronx", + "cuisine": "Bakery", + "grades": [ + { "date": { "$date": 1393804800000 }, "grade": "A", "score": 2 }, + { "date": { "$date": 1299715200000 }, "grade": "B", "score": 14 } + ], + "name": "Morris Park Bake Shop", + "restaurant_id": "30075445" + } + +The following example creates a ``2dsphere`` index on the +``address.coord`` field: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test' ) + client[:restaurants].indexes.create_one( { 'address.coord' => '2dsphere' }) + +Once the index is created, you can use several operators to query +against it, including the +:manual:`$near`, +:manual:`$geoWithin`, and +:manual:`$geoIntersects` +operators. The following example uses the ``$near`` operator to find +all restaurants within 500 meters of the given coordinates. + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:restaurants] + + collection.find( + { 'address.coord' => + { "$near" => + { "$geometry" => + { "type" => "Point", "coordinates" => [ -73.96, 40.78 ] }, + "$maxDistance" => 500 + } + } + } + ).each do |doc| + + #=> Yields a BSON::Document. + + end + +To find all documents with a location within the +perimeter of a given polygon, use the ``$geoWithin`` +operator: + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:restaurants] + + collection.find( + { "address.coord" => + { "$geoWithin" => + { "$geometry" => + { "type" => "Polygon" , + "coordinates" => [ [ [ -73, 40 ], [ -74, 41 ], [ -72, 39 ], [ -73, 40 ] ] ] + } + } + } + } + ).each do |doc| + + #=> Yields a BSON::Document. + + end + diff --git a/source/tutorials/ruby-driver-gridfs.txt b/source/tutorials/ruby-driver-gridfs.txt new file mode 100644 index 000000000..764677c3f --- /dev/null +++ b/source/tutorials/ruby-driver-gridfs.txt @@ -0,0 +1,261 @@ +====== +GridFS +====== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +The driver provides a clean and simple interface to work with storage of +chunked files in the database, also known as the pattern "GridFS". The API allows you to either +work with Grid::File objects or with read and write streams. + +Creating a GridFS object ("Grid::FSBucket") +------------------------------------------- + +You can create a GridFS object by calling ``fs`` on a database, with optional +arguments. ``fs`` returns a ``Grid::FSBucket`` object. + +The options that ``GridFS::Bucket`` supports are: + +.. list-table:: + :header-rows: 1 + :widths: 40 80 + + * - Option + - Description + * - ``:bucket_name`` + - The name of the GridFS Bucket. Default is ``fs``. + * - ``:fs_name`` + - The name of the GridFS Bucket. Takes precedence over ``bucket_name``. Default is ``fs``. + * - ``:chunk_size`` + - Specifies the size of each file chunk in the database. + * - ``:write`` + - The write concern used when uploading files. + * - ``:read`` + - The read preference used when downloading files. + + +For example, you can create a GridFS bucket object with a particular read preference: + +.. code-block:: ruby + + fs_bucket = database.fs( read: { mode: :secondary } ) + + +Working with write streams +-------------------------- + +To upload a file to GridFS using a write stream, you can either open a stream and write to it directly or write the +entire contents of an IO object to GridFS all at once. + +To open an upload stream and write to it: + +.. code-block:: ruby + + file = File.open('/path/to/my-file.txt', 'r') + fs_bucket.open_upload_stream('my-file.txt') do |stream| + stream.write(file) + end + file.close + +To upload the entire contents of an IO object in one call: + +.. code-block:: ruby + + file = File.open('/path/to/my-file.txt', 'r') + fs_bucket.upload_from_stream('my-file.txt', file) + file.close + + +Working with read streams +------------------------- + +To download a file from GridFS using a read stream, you can either open a read stream and read from it directly or +download the entire file all at once. + +To open a download stream and read from it: + +.. code-block:: ruby + + file = File.open('/path/to/my-output-file.txt', 'w') + fs_bucket.open_download_stream(file_id) do |stream| + file.write(stream.read) + end + file.close + +To download the file all at once and write it to an IO object: + +.. code-block:: ruby + + file = File.open('/path/to/my-output-file.txt', 'w') + fs_bucket.download_from_stream(file_id, file) + file.close + +You can also download a file specified by a name and (optionally) +revision number. Revision numbers are used to distinguish +between files sharing the same name, ordered by date of upload. The revision number passed to +``open_download_stream_by_name`` can be positive or negative. + +.. code-block:: ruby + + file = File.open('/path/to/my-output-file.txt', 'w') + fs_bucket.open_download_stream_by_name('my-file.txt', revision: -2) do |stream| + file.write(stream.read) + end + file.close + +To download the entire contents of the file specified by name and (optionally) +revision number: + +.. code-block:: ruby + + file = File.open('/path/to/my-output-file.txt', 'w') + fs_bucket.download_to_stream_by_name('my-file.txt', file, revision: -2) + file.close + + +Finding file metadata +--------------------- + +You can retrieve documents containing metadata about files in the GridFS files collection. + +.. code-block:: ruby + + fs_bucket.find(filename: 'my-file.txt') + +Deleting files +-------------- + +You can delete a file by id. + +.. code-block:: ruby + + fs_bucket.delete(file_id) + + +Working with Grid::File objects +------------------------------- + +This object can be used to wrap a file to be inserted into the database using GridFS and the object that is retrieved. + +To create a file with raw data: + +.. code-block:: ruby + + file = Mongo::Grid::File.new('I am a file', :filename => 'new-file.txt') + +To create a file from a Ruby ``File`` object: + +.. code-block:: ruby + + file = File.open('/path/to/my-file.txt') + grid_file = Mongo::Grid::File.new(file.read, :filename => File.basename(file.path)) + +To change file options such as chunk size, pass options to the constructor: + +.. code-block:: ruby + + file = File.open('/path/to/my-file.txt') + grid_file = Mongo::Grid::File.new( + file.read, + :filename => File.basename(file.path), + :chunk_size => 1024 + ) + +The following is a full list of the available options that files support. + +.. list-table:: + :header-rows: 1 + :widths: 40 80 + + * - Option + - Description + * - ``:chunk_size`` + - Sets the size of each file chunk in the database. + * - ``:content_type`` + - Set a content type for the file. + * - ``:filename`` (Required) + - The file name. + * - ``:upload_date`` + - The date the file was uploaded (stored). + + +Inserting Files +--------------- + +Files can be inserted into the database one at a time. File chunks are inserted +by default into the ``fs.chunks`` collection and file metadata is inserted into the +``fs.files`` collection. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + file = Mongo::Grid::File.new('I am a file', :filename => 'new-file.txt') + + client.database.fs.insert_one(file) + +To insert into collections with a name prefix other than `fs`, access the +filesystem with a ``:fs_name`` option. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + file = Mongo::Grid::File.new('I am a file', :filename => 'new-file.txt') + + client.database.fs(:fs_name => 'grid').insert_one(file) + +Note that the first time a file is inserted, it will create the required index +for you on the ``chunks`` collection. This index is a compound index: + +.. code-block:: ruby + + { :files_id => 1, :n => 1 } + +Files can also be streamed as an alternative to a direct insert. + +.. code-block:: ruby + + client.database.fs.open_upload_stream(filename) do |stream| + stream.write(file) + end + +Finding Files +------------- + +To retrieve a file from the database, call ``find_one`` with the appropriate filter. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client.database.fs.find_one(:filename => 'new-file.txt') # Returns a Mongo::Grid::File + +Files can also be streamed as an alternative to a direct find. + +.. code-block:: ruby + + client.database.fs.open_download_stream(file_id) do |stream| + io.write(stream.read) + end + + fs.download_to_stream(file_id, io) + + +Deleting Files +-------------- + +To delete a file, pass the file object to ``delete_one``. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + fs = client.database.fs + file = fs.find_one(:filename => 'new-file.txt') + fs.delete_one(file) + + + coll.find.to_a # { '_id' => ..., 'x' => 3 } diff --git a/source/tutorials/ruby-driver-indexing.txt b/source/tutorials/ruby-driver-indexing.txt new file mode 100644 index 000000000..dcc3575e5 --- /dev/null +++ b/source/tutorials/ruby-driver-indexing.txt @@ -0,0 +1,109 @@ +======== +Indexing +======== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +The driver provides the ability to create, drop and view +:manual:`indexes` on a collection. + +Creating Indexes +---------------- + +Indexes can be created one at a time, or several can be created in a single +operation. When creating multiple indexes on MongoDB 3.0 and later, the indexes +are created in parallel. On earlier versions they are created in order. + +To create a single index, use ``create_one``. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client[:bands].indexes.create_one( { :name => 1 }, unique: true ) + +To create multiple indexes, use ``create_many``. Note that when creating many, +the index keys must be passed as a ``key`` value in the provided spec. This is +due to the fact that options can be different for each index being created. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client[:bands].indexes.create_many([ + { :key => { name: 1 }, unique: true }, + { :key => { label: -1 } } + ]) + +.. _ruby-driver-index-options: + +The following is a full list of the available options that can be added +when creating indexes. + +.. list-table:: + :header-rows: 1 + :widths: 40 80 + + * - Option + - Description + * - ``:background`` + - Either ``true`` or ``false``. Tells the index to be created in the background. + * - ``:expire_after`` + - Number of seconds to expire documents in the collection after. + * - ``:name`` + - The name of the index. + * - ``:sparse`` + - Whether the index should be sparse or not, either ``true`` or ``false``. + * - ``:storage_engine`` + - The name of the storage engine for this particular index. + * - ``:version`` + - The index format version to use. + * - ``:default_language`` + - The default language of text indexes. + * - ``:language_override`` + - The field name to use when overriding the default language. + * - ``:text_version`` + - The version format for text index storage. + * - ``:weights`` + - A document specifying fields and weights in text search. + * - ``:sphere_version`` + - The 2d sphere index version. + * - ``:bits`` + - Sets the maximum boundary for latitude and longitude in the 2d index. + * - ``:max`` + - Maximum boundary for latitude and longitude in the 2d index. + * - ``:min`` + - Minimum boundary for latitude and longitude in the 2d index. + * - ``:bucket_size`` + - The number of units within which to group the location values in a geo haystack index. + * - ``:partial_filter_expression`` + - A filter for a partial index. + + +Dropping Indexes +---------------- + +To drop an index, call ``dropOne`` or ``dropAll``. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + + client[:bands].indexes.drop_one( 'name_1' ) # Drops the name_1 index. + client[:bands].indexes.drop_all # Drops all indexes in the collection. + + +Listing Indexes +--------------- + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + + client[:bands].indexes.each do |index| + p index + end diff --git a/source/tutorials/ruby-driver-projections.txt b/source/tutorials/ruby-driver-projections.txt new file mode 100644 index 000000000..23659a6da --- /dev/null +++ b/source/tutorials/ruby-driver-projections.txt @@ -0,0 +1,67 @@ +=========== +Projections +=========== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +By default, queries in MongoDB return all fields in matching +documents. To limit the amount of data that MongoDB sends to +applications, you can include a +:manual:`projection` +document in the query operation. + +Projection Document +``````````````````` + +The projection document limits the fields to return for all +matching documents. The projection document can specify the +inclusion of fields or the exclusion of field and has the +following form: + +.. code-block:: javascript + + { field1: , field2: ... } + +```` may be ``0`` (or ``false``) to exclude the field, or +``1`` (or ``true``) to include it. With the exception of the ``_id`` +field, you may not have both inclusions and exclusions in the same +projection document. + +Examples +```````` + +The following code example uses the ``restaurants`` sample dataset. + +To return only the ``name``, ``cuisine`` and ``_id`` fields for +documents that match the query filter, explicitly include the ``name`` +and ``cuisine`` fields in the projection document. The ``_id`` field is +included automatically unless specifically excluded. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + collection = client[:restaurants] + + collection.find({}, { 'name' => 1, 'cuisine' => 1 }).limit(5).each do |doc| + p doc + end + +To return ``name`` and ``cuisine`` but exclude all other fields, +including ``_id``, use the following projection document: + +.. code-block:: javascript + + { 'name' : 1, 'cuisine' : 1, '_id': 0 } + + +To return all fields *except* the address field, use the following: + +.. code-block:: javascript + + { 'address' : 0 } diff --git a/source/tutorials/ruby-driver-text-search.txt b/source/tutorials/ruby-driver-text-search.txt new file mode 100644 index 000000000..2cda531ff --- /dev/null +++ b/source/tutorials/ruby-driver-text-search.txt @@ -0,0 +1,44 @@ +=========== +Text Search +=========== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +MongoDB provides :manual:`text indexes ` +to support text search queries on string content. Text indexes +can include any field whose value is a string or an array of +string elements. + +To perform a text search with the Ruby driver, first create a text +index with ``indexes.create_one()``. The following command creates a +text index on the ``name`` field of the ``restaurants`` collection in +the ``test`` database. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + client['restaurants'].indexes.create_one( { :name => 'text' } ) + +Once the text index is created you can use it as part of a query. The +following code finds all documents in the ``restaurants`` collection +which contain the word ``garden``, without case sensitivity. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + client[:restaurants].find( + { '$text' => + { '$search' => 'garden', '$caseSensitive' => false } + } + ).each do |document| + + #=> Yields a BSON::Document. + + end + From 761a3fc14f91ad357ef04208e6bfb8e05d954a11 Mon Sep 17 00:00:00 2001 From: kay Date: Thu, 4 Aug 2016 17:58:35 -0400 Subject: [PATCH 093/442] DOCS-8466: fix duplicative phrase --- source/index.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/index.txt b/source/index.txt index 1a4052857..824ad74f1 100644 --- a/source/index.txt +++ b/source/index.txt @@ -33,7 +33,7 @@ Object Mappers Because MongoDB is so easy to use, the basic Ruby driver can be the best solution for many applications. But if you need validations, associations, and other high-level data modeling functions, then you -may need Object Document Mapper may be needed. +may need Object Document Mapper. In the context of a Rails application, an Object Document Mapper provides functionality equivalent to, but distinct from, ActiveRecord. From f79bca5d9328bb9eb7cfb7e456df0037011ff7f6 Mon Sep 17 00:00:00 2001 From: Emily Date: Fri, 12 Aug 2016 14:36:51 +0200 Subject: [PATCH 094/442] DOCS-7775 Update compatibility matrix for 2.3 --- source/reference/driver-compatibility.txt | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index d9acd5edc..4501f787d 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -25,6 +25,12 @@ MongoDB Compatibility - MongoDB 3.0 - MongoDB 3.2 + * - 2.3 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + * - 2.2 - |checkmark| - |checkmark| @@ -86,6 +92,8 @@ Language Compatibility - Ruby 1.9 - Ruby 2.0 - Ruby 2.1 + - Ruby 2.2 + - Ruby 2.3 - JRuby * - 2.0 @@ -94,12 +102,16 @@ Language Compatibility - |checkmark| - |checkmark| - |checkmark| + - |checkmark| + - |checkmark| * - 1.12 - 1.9 - |checkmark| - |checkmark| - |checkmark| - |checkmark| + - + - - |checkmark| * - 1.8 @@ -107,6 +119,8 @@ Language Compatibility - |checkmark| - |checkmark| - + - + - - |checkmark| * - 1.7 @@ -114,6 +128,8 @@ Language Compatibility - |checkmark| - - + - + - - |checkmark| * - 1.6 @@ -121,6 +137,8 @@ Language Compatibility - |checkmark| - - + - + - - |checkmark| .. include:: /includes/unicode-checkmark.rst From 23e00a4743f32ae0d808e0882f152344fa8d95ca Mon Sep 17 00:00:00 2001 From: Steve Renaker Date: Wed, 17 Aug 2016 11:11:11 -0700 Subject: [PATCH 095/442] DOCS-8592: added missing colon --- source/quick-start.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/quick-start.txt b/source/quick-start.txt index a2389a99e..c8c73eb64 100644 --- a/source/quick-start.txt +++ b/source/quick-start.txt @@ -184,7 +184,7 @@ from a collection (either singly or several at once). client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') collection = client[:people] - result = collection.delete_one( name 'Steve' ) + result = collection.delete_one( { name: 'Steve' } ) puts result.deleted_count # returns 1 because one document was deleted From 9620442211db8fc368891f42bed0be78e476195b Mon Sep 17 00:00:00 2001 From: kay Date: Wed, 21 Sep 2016 11:41:06 -0400 Subject: [PATCH 096/442] Docs update for mongoid section --- source/index.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/index.txt b/source/index.txt index 824ad74f1..9c345b3d0 100644 --- a/source/index.txt +++ b/source/index.txt @@ -44,7 +44,7 @@ Object Document Mappers (ODM) as opposed to Object Relational Mappers The ODM officially supported by MongoDB is Mongoid, originally written by Durran Jordan. -For tutorials on Mongoid, see :doc:`/mongoid-tutorials`. +For tutorials on Mongoid, see :doc:`/mongoid`. .. COMMENT For the actual build, see mongodb/docs-ruby repo which pulls the documentation source from: .. mongo-ruby-driver, @@ -60,7 +60,7 @@ For tutorials on Mongoid, see :doc:`/mongoid-tutorials`. quick-start ruby-driver-tutorials bson-tutorials - mongoid-tutorials + mongoid API /reference/driver-compatibility /reference/additional-resources From d67338f68e2be0832d3339980302fcf21ad3478f Mon Sep 17 00:00:00 2001 From: Emily Date: Mon, 3 Oct 2016 16:07:36 -0400 Subject: [PATCH 097/442] Fix docs typo --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 5113f4cd5..1d1ba83cf 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -169,7 +169,7 @@ Ruby Options - ``Boolean`` - false - * - ``:2out`` + * - ``:connect_timeout`` - The number of seconds to wait to establish a socket connection before raising an exception. - ``Float`` From f01683b641ae27a98f7a05b397c5132674857520 Mon Sep 17 00:00:00 2001 From: Emily Date: Tue, 4 Oct 2016 11:04:37 -0400 Subject: [PATCH 098/442] Add details in documentation on difference between max_time_ms and socket_timeout options --- source/tutorials/ruby-driver-create-client.txt | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 1d1ba83cf..836866ee7 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -299,10 +299,12 @@ Details on Timeout Options ``socket_timeout`` The number of seconds to wait for an operation to - execute on a socket before raising an exception. It should take into - account network latency and operation duration. Defaults to 5 seconds. + execute on a socket before raising a timeout exception. It should take into + account network latency and operation duration. The default is no value; the default is effectively infinity. + Please consider using ``max_time_ms`` per-operation instead, as the ``socket_timeout`` does not stop the operation + on the server; a long-running operation will continue to run on the server, beyond a socket timeout being reached. See the `socket timeout for monitoring specification `_ - documentation for further information. + documentation for further information relating to server discovery and monitoring. ``server_selection_timeout`` The number of seconds to wait for the driver to find an appropriate server to @@ -337,3 +339,10 @@ Details on Timeout Options Increase this number to create connections when the pool is initialized and to reduce the overhead of creating new connections later on. Defaults to 1. + +``max_time_ms`` + Specified as an option on a particular operation. It defines a cumulative time limit in milliseconds for processing + operations on a cursor. Consider using this option instead of a ``socket_timeout``, if the operation should be + interrupted on the server. See the + `CRUD specification `_ for details on + operations that support this option. From fedf2c8d01fb8ab97319f54e461ae1635e9ccaa3 Mon Sep 17 00:00:00 2001 From: Emily Date: Tue, 11 Oct 2016 12:57:29 +0200 Subject: [PATCH 099/442] RUBY-1142 Update client options documentation for new ssl options --- .../tutorials/ruby-driver-create-client.txt | 66 +++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 836866ee7..ab7edabb8 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -169,6 +169,72 @@ Ruby Options - ``Boolean`` - false + * - ``:ssl_cert`` + - The certificate file path used to identify the connection against MongoDB. This option, if present, + takes precedence over the values of :ssl_cert_string and :ssl_cert_object. + - ``String`` + - none + + * - ``:ssl_cert_string`` + - A string containing the PEM-encoded certificate used to identify the connection against MongoDB. + This option, if present, takes precedence over the value of :ssl_cert_object. + - ``String`` + - none + + * - ``:ssl_cert_object`` + - The OpenSSL::X509::Certificate used to identify the connection against MongoDB. + - ``OpenSSL::X509::Certificate`` + - none + + * - ``:ssl_key`` + - The private keyfile used to identify the connection against MongoDB. Note that even if the key is stored in + the same file as the certificate, both need to be explicitly specified. This option, if present, takes + precedence over the values of :ssl_key_string and :ssl_key_object. + - ``String`` + - none + + * - ``:ssl_key_string`` + - A string containing the PEM-encoded private key used to identify the connection against MongoDB. + This parameter, if present, takes precedence over the value of option :ssl_key_object. + - ``String`` + - none + + * - ``:ssl_key_object`` + - The private key used to identify the connection against MongoDB. + - ``OpenSSL::PKey`` + - none + + * - ``:ssl_key_pass_phrase`` + - A passphrase for the private key. + - ``String`` + - none + + * - ``:ssl_ca_cert`` + - The file path containing a set of concatenated certification authority certifications used to validate certs + passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object + (in order of priority) is required for :ssl_verify. + - ``String`` + - none + + * - ``:ssl_ca_cert_string`` + - A string containing a set of concatenated certification authority certifications used to validate certs + passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object + (in order of priority) is required for :ssl_verify. + - ``String`` + - none + + * - ``:ssl_ca_cert_object`` + - An array of OpenSSL::X509::Certificate representing the certification authority certifications used to + validate certs passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or + :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. + - ``Array`` + - none + + * - ``:ssl_verify`` + - Whether or not to do peer certification validation. + - ``Boolean`` + - false + * - ``:connect_timeout`` - The number of seconds to wait to establish a socket connection before raising an exception. From c7102c269e9b66000a68100b635cfed68fcf9a30 Mon Sep 17 00:00:00 2001 From: Steve Renaker Date: Thu, 15 Sep 2016 09:50:09 -0700 Subject: [PATCH 100/442] DOCS-8717: document Decimal128 in Ruby driver --- source/index.txt | 1 + .../tutorials/ruby-driver-crud-operations.txt | 57 +++++++++++++++++-- source/whats-new.txt | 16 ++++++ 3 files changed, 68 insertions(+), 6 deletions(-) create mode 100644 source/whats-new.txt diff --git a/source/index.txt b/source/index.txt index 9c345b3d0..14609bbc1 100644 --- a/source/index.txt +++ b/source/index.txt @@ -56,6 +56,7 @@ For tutorials on Mongoid, see :doc:`/mongoid`. .. toctree:: :titlesonly: + whats-new installation quick-start ruby-driver-tutorials diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index f42c34b25..7da7de519 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -85,6 +85,51 @@ On MongoDB 2.4, an exception is only raised if the insert fails and the ]) result.inserted_count # returns 2, because 2 documents were inserted. +.. _specify-decimal128: + +Specify a ``Decimal128`` number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. versionadded:: 3.4 + +:manual:`Decimal128` is a :doc:`BSON datatype ` +that employs 128-bit decimal-based floating-point values capable +of emulating decimal rounding with exact precision. This +functionality is intended for applications that handle +:manual:`monetary data `, such as financial and tax computations. + +.. COMMENT placeholder links for Decimal128 manual entries + +The following example inserts a value of type ``Decimal128`` into +the ``price`` field of a collection named ``inventory``: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + + price = BSON::Decimal128.new("428.79") + client[:inventory].insert_one({ "_id" => 1, + "item" => "26 inch monitor", + "price" => price }) + +The above operation produces the following document: + +.. code-block:: javascript + + { "_id" : 1, "item" : "26 inch monitor", "price" : NumberDecimal("428.79") } + +You can also create a ``Decimal128`` object from a Ruby ``BigDecimal`` +object, or with ``Decimal128.from_string()``. + +.. code-block:: ruby + + big_decimal = BigDecimal.new(428.79, 5) + price = BSON::Decimal128.new(big_decimal) + # => BSON::Decimal128('428.79') + + price = BSON::Decimal128.from_string("428.79") + # => BSON::Decimal128('428.79') + Reading ------- @@ -110,7 +155,7 @@ query: .. _ruby-driver-query-options: Query Options -````````````` +~~~~~~~~~~~~~ To add options to a query, chain the appropriate methods after the ``find`` method. Note that the underlying object, the ``Mongo::Collection::View``, @@ -182,7 +227,7 @@ when querying and their corresponding methods as examples. client[:artists].find.sort(:name => -1) Additional Query Operations -``````````````````````````` +~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``count`` Get the total number of documents an operation returns. @@ -204,7 +249,7 @@ Additional Query Operations client[:artists].find.distinct(:name ) Tailable Cursors -```````````````` +~~~~~~~~~~~~~~~~ For capped collections you may use a :manual:`tailable cursor ` that remains open @@ -233,7 +278,7 @@ following code example shows how a tailable cursor might be used: .. _ruby-driver-read-preference: Read Preference -```````````````` +~~~~~~~~~~~~~~~ Read preference determines the candidate :manual:`replica set` members to which a query or command can be sent. They consist of a **mode** specified as @@ -270,7 +315,7 @@ option when a command is run on a database. tag_sets: [ { 'dc' => 'nyc' } ] } ) Mode -```` +~~~~ There are five possible read preference modes. They are ``:primary``, ``:secondary``, ``:primary_preferred``, ``:secondary_preferred``, @@ -278,7 +323,7 @@ There are five possible read preference modes. They are ``:primary``, ` for an explanation of the modes and tag sets. Tag sets -```````` +~~~~~~~~ The ``tag_sets`` parameter is an ordered list of tag sets used to restrict the eligibility of servers for selection, such as for data diff --git a/source/whats-new.txt b/source/whats-new.txt new file mode 100644 index 000000000..0944f6e1e --- /dev/null +++ b/source/whats-new.txt @@ -0,0 +1,16 @@ +========== +What's New +========== + +.. default-domain:: mongodb + +What's New in the 2.4 Driver +---------------------------- + +- :doc:`Decimal128`, + a new :doc:`BSON datatype ` which employs 128-bit + decimal-based floating-point values capable of emulating decimal + rounding with exact precision. + +.. COMMENT need internal link to #specify-decimal128 in the CRUD page +.. COMMENT Collation and other items to come here \ No newline at end of file From 4a5813f82baf92801a4f50c4e580c109a32875c6 Mon Sep 17 00:00:00 2001 From: Steve Renaker Date: Thu, 15 Sep 2016 09:50:09 -0700 Subject: [PATCH 101/442] DOCS-8717: document Decimal128 in Ruby driver --- source/index.txt | 1 + .../tutorials/ruby-driver-crud-operations.txt | 57 +++++++++++++++++-- source/whats-new.txt | 16 ++++++ 3 files changed, 68 insertions(+), 6 deletions(-) create mode 100644 source/whats-new.txt diff --git a/source/index.txt b/source/index.txt index 9c345b3d0..14609bbc1 100644 --- a/source/index.txt +++ b/source/index.txt @@ -56,6 +56,7 @@ For tutorials on Mongoid, see :doc:`/mongoid`. .. toctree:: :titlesonly: + whats-new installation quick-start ruby-driver-tutorials diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index f42c34b25..4d6dec1b2 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -85,6 +85,51 @@ On MongoDB 2.4, an exception is only raised if the insert fails and the ]) result.inserted_count # returns 2, because 2 documents were inserted. +.. _specify-decimal128: + +Specify a ``Decimal128`` number +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. versionadded:: 3.4 + +:manual:`Decimal128` is a +:doc:`BSON datatype ` +that employs 128-bit decimal-based floating-point values capable +of emulating decimal rounding with exact precision. This +functionality is intended for applications that handle +:manual:`monetary data `, +such as financial and tax computations. + +The following example inserts a value of type ``Decimal128`` into +the ``price`` field of a collection named ``inventory``: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + + price = BSON::Decimal128.new("428.79") + client[:inventory].insert_one({ "_id" => 1, + "item" => "26 inch monitor", + "price" => price }) + +The above operation produces the following document: + +.. code-block:: javascript + + { "_id" : 1, "item" : "26 inch monitor", "price" : NumberDecimal("428.79") } + +You can also create a ``Decimal128`` object from a Ruby ``BigDecimal`` +object, or with ``Decimal128.from_string()``. + +.. code-block:: ruby + + big_decimal = BigDecimal.new(428.79, 5) + price = BSON::Decimal128.new(big_decimal) + # => BSON::Decimal128('428.79') + + price = BSON::Decimal128.from_string("428.79") + # => BSON::Decimal128('428.79') + Reading ------- @@ -110,7 +155,7 @@ query: .. _ruby-driver-query-options: Query Options -````````````` +~~~~~~~~~~~~~ To add options to a query, chain the appropriate methods after the ``find`` method. Note that the underlying object, the ``Mongo::Collection::View``, @@ -182,7 +227,7 @@ when querying and their corresponding methods as examples. client[:artists].find.sort(:name => -1) Additional Query Operations -``````````````````````````` +~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``count`` Get the total number of documents an operation returns. @@ -204,7 +249,7 @@ Additional Query Operations client[:artists].find.distinct(:name ) Tailable Cursors -```````````````` +~~~~~~~~~~~~~~~~ For capped collections you may use a :manual:`tailable cursor ` that remains open @@ -233,7 +278,7 @@ following code example shows how a tailable cursor might be used: .. _ruby-driver-read-preference: Read Preference -```````````````` +~~~~~~~~~~~~~~~ Read preference determines the candidate :manual:`replica set` members to which a query or command can be sent. They consist of a **mode** specified as @@ -270,7 +315,7 @@ option when a command is run on a database. tag_sets: [ { 'dc' => 'nyc' } ] } ) Mode -```` +~~~~ There are five possible read preference modes. They are ``:primary``, ``:secondary``, ``:primary_preferred``, ``:secondary_preferred``, @@ -278,7 +323,7 @@ There are five possible read preference modes. They are ``:primary``, ` for an explanation of the modes and tag sets. Tag sets -```````` +~~~~~~~~ The ``tag_sets`` parameter is an ordered list of tag sets used to restrict the eligibility of servers for selection, such as for data diff --git a/source/whats-new.txt b/source/whats-new.txt new file mode 100644 index 000000000..0944f6e1e --- /dev/null +++ b/source/whats-new.txt @@ -0,0 +1,16 @@ +========== +What's New +========== + +.. default-domain:: mongodb + +What's New in the 2.4 Driver +---------------------------- + +- :doc:`Decimal128`, + a new :doc:`BSON datatype ` which employs 128-bit + decimal-based floating-point values capable of emulating decimal + rounding with exact precision. + +.. COMMENT need internal link to #specify-decimal128 in the CRUD page +.. COMMENT Collation and other items to come here \ No newline at end of file From 104a120c4ff140a0c0f9bd2ffcd6ea43be4ac065 Mon Sep 17 00:00:00 2001 From: Steve Renaker Date: Mon, 12 Sep 2016 16:13:51 -0700 Subject: [PATCH 102/442] DOCS-8757: document collation in the Ruby driver docs --- source/ruby-driver-tutorials.txt | 1 + .../tutorials/ruby-driver-bulk-operations.txt | 4 +- source/tutorials/ruby-driver-collations.txt | 311 ++++++++++++++++++ 3 files changed, 315 insertions(+), 1 deletion(-) create mode 100644 source/tutorials/ruby-driver-collations.txt diff --git a/source/ruby-driver-tutorials.txt b/source/ruby-driver-tutorials.txt index 1ee606660..6ce883c3f 100644 --- a/source/ruby-driver-tutorials.txt +++ b/source/ruby-driver-tutorials.txt @@ -21,6 +21,7 @@ operations available in the Ruby driver. /tutorials/ruby-driver-projections /tutorials/ruby-driver-admin-tasks /tutorials/ruby-driver-indexing + /tutorials/ruby-driver-collations /tutorials/ruby-driver-aggregation /tutorials/ruby-driver-bulk-operations /tutorials/ruby-driver-text-search diff --git a/source/tutorials/ruby-driver-bulk-operations.txt b/source/tutorials/ruby-driver-bulk-operations.txt index bc044388a..c87175218 100644 --- a/source/tutorials/ruby-driver-bulk-operations.txt +++ b/source/tutorials/ruby-driver-bulk-operations.txt @@ -9,7 +9,9 @@ Bulk Operations :backlinks: none :depth: 1 :class: singlecol - + +.. _ruby-driver-bulk-ops: + The bulk write API sends a list of write operations to the server in one method call. Execution order of the operations is guaranteed if you pass in the ``ordered`` option. diff --git a/source/tutorials/ruby-driver-collations.txt b/source/tutorials/ruby-driver-collations.txt new file mode 100644 index 000000000..5a752b7ad --- /dev/null +++ b/source/tutorials/ruby-driver-collations.txt @@ -0,0 +1,311 @@ +========== +Collations +========== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Overview +-------- + +.. versionadded:: 3.4 + +Collations provide a set of rules which comply with the conventions +of a particular language when comparing strings. + +For example, in Canadian French, the last accent in a given word +determines the sorting order. + +Consider the following French words: + +.. code-block:: none + + cote < coté < côte < côté + +The sort order using the Canadian French collation would result in +the following: + +.. code-block:: none + + cote < côte < coté < côté + +If collation is unspecified, MongoDB uses the simple binary comparison for +strings. As such, the sort order of the words would be: + +.. code-block:: none + + cote < coté < côte < côté + +Usage +----- + +You can specify a default collation for collections and indexes when +they are created, or specify a collation for CRUD operations and +aggregations. For operations that support collation, MongoDB uses the +collection's default collation unless the operation specifies a +different collation. + +Collation Parameters +~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: ruby + + 'collation' => { + 'locale' => , + 'caseLevel' => , + 'caseFirst' => , + 'strength' => , + 'numericOrdering' => , + 'alternate' => , + 'maxVariable' => , + 'normalization' => , + 'backwards' => + } + +The only required parameter is ``locale``, which the server parses as +an `ICU format locale ID `_. +For example, set ``locale`` to ``en_US`` to represent US English +or ``fr_CA`` to represent Canadian French. + +For a complete description of the available parameters, see the +:manual:`MongoDB manual entry`. + +.. _collation-on-collection: + +Assign a Default Collation to a Collection +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following example creates a new collection +called ``contacts`` on the ``test`` database and assigns a default +collation with the ``fr_CA`` locale. Specifying a collation when you +create the collection ensures that all operations involving a query +that are run against the +``contacts`` collection use the ``fr_CA`` collation, unless the query +specifies another collation. Any indexes on the new collection also +inherit the default collation, unless the creation command specifies +another collation. + +.. code-block:: ruby + + client = Mongo::Client.new([ "127.0.0.1:27017" ], :database => "test") + client[:contacts, { "collation" => { "locale" => "fr_CA" } } ].create + +.. _collation-on-index: + +Assign a Collation to an Index +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To specify a collation for an index, use the ``collation`` +option when you create the index. + +The following example creates an index on the ``name`` +field of the ``address_book`` collection, with the ``unique`` parameter +enabled and a default collation with ``locale`` set to ``en_US``. + +.. code-block:: ruby + + client = Mongo::Client.new([ "127.0.0.1:27017" ], :database => "test") + client[:address_book].indexes.create_one( { "first_name" => 1 }, + "unique" => true, + "collation" => { "locale" => "en_US" } + ) + +To use this index, make sure your queries also specify the same +collation. The following query uses the above index: + +.. code-block:: ruby + + client[:address_book].find({"first_name" : "Adam" }, + "collation" => { "locale" => "en_US" }) + +The following queries do **NOT** use the index. The first query uses no +collation, and the second uses a collation with a different ``strength`` +value than the collation on the index. + +.. code-block:: ruby + + client[:address_book].find({"first_name" : "Adam" }) + + client[:address_book].find({"first_name" : "Adam" }, + "collation" => { "locale" => "en_US", "strength" => 2 }) + +Operations that Support Collation +--------------------------------- + +All reading, updating, and deleting methods support collation. Some +examples are listed below. + +``find()`` and ``sort()`` +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Individual queries can specify a collation to use when matching +and sorting results. The following query and sort operation uses +a German collation with the ``locale`` parameter set to ``de``. + +.. code-block:: ruby + + client = Mongo::Client.new([ "127.0.0.1:27017" ], :database => "test") + docs = client[:contacts].find({ "city" => "New York" }, + { "collation" => { "locale" => "de" } }).sort( "name" => 1 ) + +``find_one_and_update()`` +~~~~~~~~~~~~~~~~~~~~~~~~~ + +A collection called ``names`` contains the following documents: + +.. code-block:: javascript + + { "_id" : 1, "first_name" : "Hans" } + { "_id" : 2, "first_name" : "Gunter" } + { "_id" : 3, "first_name" : "Günter" } + { "_id" : 4, "first_name" : "Jürgen" } + +The following ``find_one_and_update`` operation on the collection +does not specify a collation. + +.. code-block:: ruby + + client = Mongo::Client.new([ "127.0.0.1:27017" ], :database => "test") + doc = client[:names].find_one_and_update( {"first_name" => { "$lt" => "Gunter" }}, + { "$set" => { "verified" => true } }) + +Because ``Gunter`` is lexically first in the collection, +the above operation returns no results and updates no documents. + +Consider the same ``find_one_and_update`` operation but with the +collation specified. The locale is set to ``de@collation=phonebook``. + +.. note:: + + Some locales have a ``collation=phonebook`` option available for + use with languages which sort proper nouns differently from other + words. According to the ``de@collation=phonebook`` collation, + characters with umlauts come before the same characters without + umlauts. + +.. code-block:: ruby + + client = Mongo::Client.new([ "127.0.0.1:27017" ], :database => "test") + doc = client[:names].find_one_and_update( { "first_name" => { "$lt" => "Gunter" } }, + { "$set" => { "verified" => true } }, { "collation" => { "locale" => "de@collation=phonebook" }, + :return_document => :after } ) + +The operation returns the following updated document: + +.. code-block:: javascript + + { "_id" => 3, "first_name" => "Günter", "verified" => true } + +``find_one_and_delete()`` +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Set the ``numericOrdering`` collation parameter to ``true`` +to compare numeric string by their numeric values. + +The collection ``numbers`` contains the following documents: + +.. code-block:: javascript + + { "_id" : 1, "a" : "16" } + { "_id" : 2, "a" : "84" } + { "_id" : 3, "a" : "179" } + +The following example matches the first document in which field ``a`` +has a numeric value greater than 100 and deletes it. + +.. code-block:: ruby + + docs = numbers.find_one_and_delete({ "a" => { "$gt" => "100" } }, + { "collation" => { "locale" => "en", "numericOrdering" => true } }) + +After the above operation, the following documents remain in the +collection: + +.. code-block:: javascript + + { "_id" : 1, "a" : "16" } + { "_id" : 2, "a" : "84" } + +If you perform the same operation without collation, the server deletes +the first document it finds in which the lexical value of ``a`` is +greater than ``"100"``. + +.. code-block:: ruby + + numbers = client[:numbers] + docs = numbers.find_one_and_delete({ "a" => { "$gt" => "100" } }) + +After the above operation the document in which ``a`` was equal to +``"16"`` has been deleted, and the following documents remain in the +collection: + +.. code-block:: javascript + + { "_id" : 2, "a" : "84" } + { "_id" : 3, "a" : "179" } + +``delete_many()`` +~~~~~~~~~~~~~~~~~ + +You can use collations with all the various bulk operations which +exist in the Ruby driver. + +The collection ``recipes`` contains the following documents: + +.. code-block:: javascript + + { "_id" : 1, "dish" : "veggie empanadas", "cuisine" : "Spanish" } + { "_id" : 2, "dish" : "beef bourgignon", "cuisine" : "French" } + { "_id" : 3, "dish" : "chicken molé", "cuisine" : "Mexican" } + { "_id" : 4, "dish" : "chicken paillard", "cuisine" : "french" } + { "_id" : 5, "dish" : "pozole verde", "cuisine" : "Mexican" } + +Setting the ``strength`` parameter of the collation document to ``1`` +or ``2`` causes the server to disregard case in the query filter. The +following example uses a case-insensitive query filter +to delete all records in which the ``cuisine`` field matches +``French``. + +.. code-block:: ruby + + client = Mongo::Client.new([ "127.0.0.1:27017" ], :database => "test") + recipes = client[:recipes] + docs = recipes.delete_many({ "cuisine" => "French" }, + "collation" => { "locale" => "en_US", "strength" => 1 }) + +After the above operation runs, the documents with ``_id`` values of +``2`` and ``4`` are deleted from the collection. + +Aggregation +~~~~~~~~~~~ + +To use collation with an aggregation operation, specify a collation in +the aggregation options. + +The following aggregation example uses a collection called ``names`` +and groups the ``first_name`` field together, counts the total +number of results in each group, and sorts the +results by German phonebook order. + +.. code-block:: ruby + + aggregation = names.aggregate( + [ + { + "$group" => { "_id" => "$first_name", "name_count" => { "$sum" => 1 } } + }, + { + "$sort" => { "_id" => 1 } + }, + + ], { "collation" => { "locale" => "de@collation=phonebook" } } + ) + + aggregation.each do |doc| + #=> Yields a BSON::Document. + end From 61d037f80b66e82f33dec7e4f96ae42a6baac86c Mon Sep 17 00:00:00 2001 From: Emily Date: Mon, 31 Oct 2016 11:39:36 +0100 Subject: [PATCH 103/442] Fix GridFS namespacing typos --- source/tutorials/ruby-driver-gridfs.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-gridfs.txt b/source/tutorials/ruby-driver-gridfs.txt index 764677c3f..db048d881 100644 --- a/source/tutorials/ruby-driver-gridfs.txt +++ b/source/tutorials/ruby-driver-gridfs.txt @@ -20,7 +20,7 @@ Creating a GridFS object ("Grid::FSBucket") You can create a GridFS object by calling ``fs`` on a database, with optional arguments. ``fs`` returns a ``Grid::FSBucket`` object. -The options that ``GridFS::Bucket`` supports are: +The options that ``Grid::FSBucket`` supports are: .. list-table:: :header-rows: 1 From 5ceb0a09152a70c6e481c941fe4cfe49ddc84684 Mon Sep 17 00:00:00 2001 From: Emily Date: Mon, 28 Nov 2016 10:39:25 +0100 Subject: [PATCH 104/442] RUBY-1161 Add Decimal128 example to docs --- docs/tutorials/bson-v4.txt | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index e8df050f0..5a5ef5dc8 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -108,7 +108,7 @@ is done by simply calling `to_s` on the buffer. Supported Objects ----------------- -Core Ruby object's that have representations in the BSON specification and +Core Ruby objects that have representations in the BSON specification and will have a ``to_bson`` method defined for them are: ``Object``, ``Array``, ``FalseClass``, ``Float``, ``Hash``, ``Integer``, ``NilClass``, ``Regexp``, ``String``, ``Symbol`` (deprecated), ``Time``, ``TrueClass``. @@ -203,6 +203,21 @@ Represents a placeholder for a value that was not provided. BSON::Undefined.new +``BSON::Decimal128`` +``````````````````` + +Represents a 128-bit decimal-based floating-point value capable of +emulating decimal rounding with exact precision.. + +.. code-block:: ruby + + # Instantiate with a String + BSON::Decimal128.new("1.28") + + # Instantiate with a BigDecimal + d = BigDecimal.new(1.28, 3) + BSON::Decimal128.new(d) + JSON Serialization ------------------ @@ -261,5 +276,5 @@ object, one must call ``compile`` on the returned object. regex = Regexp.from_bson(byte_buffer) regex.pattern #=> Returns the pattern as a string. - regex.options #=> Returns the raw options as an int. + regex.options #=> Returns the raw options as a String. regex.compile #=> Returns the compiled Ruby Regexp object. From b2f469b35a8ce1fe9d57dbd928c8b3d6c66a1540 Mon Sep 17 00:00:00 2001 From: Steve Renaker Date: Tue, 29 Nov 2016 15:42:29 -0800 Subject: [PATCH 105/442] DOCS-9370: insert_many in bulk operations page --- source/tutorials/ruby-driver-bulk-operations.txt | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/source/tutorials/ruby-driver-bulk-operations.txt b/source/tutorials/ruby-driver-bulk-operations.txt index c87175218..a7b5fc65c 100644 --- a/source/tutorials/ruby-driver-bulk-operations.txt +++ b/source/tutorials/ruby-driver-bulk-operations.txt @@ -31,6 +31,13 @@ insert_one { :insert_one => { :x => 1 } } +insert_many +----------- + +.. code-block:: ruby + + { :insert_many => [ { :x => 1 }, { :x => 2 } ] } + delete_one ---------- From bf3d0e2299da5d1bab2a9391e13fc43647f43656 Mon Sep 17 00:00:00 2001 From: Emily Date: Wed, 30 Nov 2016 12:37:18 +0100 Subject: [PATCH 106/442] Add Ruby driver 2.4 to compatibility matrices --- source/reference/driver-compatibility.txt | 54 ++++++++++++++++++++++- 1 file changed, 53 insertions(+), 1 deletion(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 4501f787d..1da4c315e 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -24,54 +24,70 @@ MongoDB Compatibility - MongoDB 2.6 - MongoDB 3.0 - MongoDB 3.2 + - MongoDB 3.4 + + * - 2.4 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| * - 2.3 - |checkmark| - |checkmark| - |checkmark| - |checkmark| + - * - 2.2 - |checkmark| - |checkmark| - |checkmark| - |checkmark| + - * - 2.0 - |checkmark| - |checkmark| - |checkmark| - + - * - 1.12 - |checkmark| - |checkmark| - |checkmark| - + - * - 1.11 - |checkmark| - |checkmark| - - + - * - 1.10 - |checkmark| - |checkmark| - - + - * - 1.9 - |checkmark| - - - + - * - 1.8 - |checkmark| - - - + - .. include:: /includes/older-server-versions-unsupported.rst @@ -96,6 +112,42 @@ Language Compatibility - Ruby 2.3 - JRuby + * - 2.4 + - + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + + * - 2.3 + - + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + + * - 2.2 + - + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + + * - 2.1 + - + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + * - 2.0 - - |checkmark| @@ -105,7 +157,7 @@ Language Compatibility - |checkmark| - |checkmark| - * - 1.12 - 1.9 + * - 1.9 - 1.12 - |checkmark| - |checkmark| - |checkmark| From d050bac3e705d76806d58042aa04e31b62069951 Mon Sep 17 00:00:00 2001 From: Emily Date: Thu, 1 Dec 2016 10:58:24 +0100 Subject: [PATCH 107/442] Remove compatibility matrix from Installation page as it has its own page --- source/installation.txt | 44 +---------------------------------------- 1 file changed, 1 insertion(+), 43 deletions(-) diff --git a/source/installation.txt b/source/installation.txt index 4dfc3ab68..e9fa2f30e 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -24,46 +24,4 @@ To install the gem with bundler, include the following in your Gemfile: .. code-block:: ruby - gem 'mongo', '~> 2.2' - -Note the following compatibility matrix to determine if the driver is -supported on your Ruby version and MongoDB server version. - -.. list-table:: - :header-rows: 1 - :widths: 28 30 30 30 - - * - Ruby Version - - MongoDB Server 2.4.x - - MongoDB Server 2.6.x - - MongoDB Server 3.0.x - - * - 1.8.x - - No - - No - - No - - * - 1.9.x - - Yes - - Yes - - Yes - - * - 2.0.x - - Yes - - Yes - - Yes - - * - 2.1.x - - Yes - - Yes - - Yes - - * - 2.2.x - - Yes - - Yes - - Yes - - * - JRuby 1.7.x - - Yes - - Yes - - Yes \ No newline at end of file + gem 'mongo', '~> 2.4' From b55cc386944923f10646141b4d34f54c7ea0d471 Mon Sep 17 00:00:00 2001 From: kay Date: Sun, 4 Dec 2016 20:04:51 -0500 Subject: [PATCH 108/442] Docs: update installation for bson gem 4.0 --- docs/tutorials/bson-v4.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 5a5ef5dc8..fb410af51 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -32,7 +32,7 @@ To install the gem with bundler, include the following in your Gemfile: .. code-block:: ruby - gem 'bson', '~> 3.0' + gem 'bson', '~> 4.0' The BSON gem is compatible with MRI 1.9.3, 2.0.x, 2.1.x, 2.2.x, JRuby 1.7.x, and Rubinius 2.5.x From 0e27e95e7f817ae5e7f04ce11054ecc193f71fe1 Mon Sep 17 00:00:00 2001 From: kay Date: Sun, 4 Dec 2016 20:10:05 -0500 Subject: [PATCH 109/442] Docs: Typo fix -- underscore too short --- docs/tutorials/bson-v4.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index fb410af51..225e0c19e 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -204,7 +204,7 @@ Represents a placeholder for a value that was not provided. BSON::Undefined.new ``BSON::Decimal128`` -``````````````````` +```````````````````` Represents a 128-bit decimal-based floating-point value capable of emulating decimal rounding with exact precision.. From d9e32417bacc02e43965988a986c2915dd3c3773 Mon Sep 17 00:00:00 2001 From: Steve Renaker Date: Tue, 6 Dec 2016 11:43:02 -0800 Subject: [PATCH 110/442] DOCS-9125: projections tutorial update --- source/tutorials/ruby-driver-projections.txt | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-projections.txt b/source/tutorials/ruby-driver-projections.txt index 23659a6da..eb572ba13 100644 --- a/source/tutorials/ruby-driver-projections.txt +++ b/source/tutorials/ruby-driver-projections.txt @@ -26,7 +26,7 @@ following form: .. code-block:: javascript - { field1: , field2: ... } + { 'projection': { field1: , field2: ... } } ```` may be ``0`` (or ``false``) to exclude the field, or ``1`` (or ``true``) to include it. With the exception of the ``_id`` @@ -48,7 +48,8 @@ included automatically unless specifically excluded. client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') collection = client[:restaurants] - collection.find({}, { 'name' => 1, 'cuisine' => 1 }).limit(5).each do |doc| + collection.find({}, { 'projection' => + { 'name' => 1, 'cuisine' => 1 } }).limit(5).each do |doc| p doc end @@ -57,11 +58,11 @@ including ``_id``, use the following projection document: .. code-block:: javascript - { 'name' : 1, 'cuisine' : 1, '_id': 0 } + { 'projection' => { 'name' => 1, 'cuisine' => 1, '_id' => 0 } } To return all fields *except* the address field, use the following: .. code-block:: javascript - { 'address' : 0 } + { 'projection' => { 'address' => 0 } } From 990557b56c05c070e0f5df49532647e6295a90ee Mon Sep 17 00:00:00 2001 From: kay Date: Sat, 24 Dec 2016 11:14:23 -0500 Subject: [PATCH 111/442] docs: fix table widths since column added --- source/reference/driver-compatibility.txt | 1 - 1 file changed, 1 deletion(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 1da4c315e..b491b323e 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -16,7 +16,6 @@ MongoDB Compatibility .. list-table:: :header-rows: 1 :stub-columns: 1 - :widths: 20 20 20 20 20 :class: compatibility * - Ruby Driver From ceee5b0d61b1cdec0a2afc77d416f3509a6c5e6f Mon Sep 17 00:00:00 2001 From: Emily Date: Tue, 31 Jan 2017 18:06:52 +0100 Subject: [PATCH 112/442] Enhance documentation for Ruby regular expressions --- docs/tutorials/bson-v4.txt | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 225e0c19e..c3c495690 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -265,9 +265,42 @@ they are deserialized they will always be returned as a ``Time`` since the BSON specification only has a ``Time`` type and knows nothing about Ruby. -Compiling Regexes +Using Regexes ----------------- +Ruby regular expressions always have BSON regular expression's equivalent of 'm' on. +In order for behavior to be preserved between the two, the 'm' option is always added +when a Ruby regular expression is serialized to BSON. + +There is a class provided by the bson gem, ``Regexp::Raw``, to allow Ruby users to get around this. +You can simply create a regular expression like this: + +.. code-block:: ruby + + Regexp::Raw.new("^b403158") + +This code example illustrates the difference between serializing a core Ruby ``Regexp`` versus a +``Regexp::Raw`` object: + +.. code-block:: ruby + + regexp_ruby = /^b403158/ + # => /^b403158/ + regexp_ruby.to_bson + # => # + _.to_s + # => "^b403158\x00m\x00" + regexp_raw = Regexp::Raw.new("^b403158") + # => # + regexp_raw.to_bson + # => # + _.to_s + # => "^b403158\x00\x00" + + +Please use the ``Regexp::Raw`` class to instantiate your BSON regular expressions to get the exact +pattern and options you want. + When regular expressions are deserialized, they return a wrapper that holds the raw regex string, but does not compile it. In order to get the Ruby ``Regexp`` object, one must call ``compile`` on the returned object. From 9a1af85443e5e24d2531e007160617edf580c496 Mon Sep 17 00:00:00 2001 From: kay Date: Mon, 22 May 2017 19:23:22 -0400 Subject: [PATCH 113/442] DOCS-9835 changes for separate mongoid docs site --- source/index.txt | 3 +-- source/meta/404.txt | 7 +++++++ 2 files changed, 8 insertions(+), 2 deletions(-) create mode 100644 source/meta/404.txt diff --git a/source/index.txt b/source/index.txt index 14609bbc1..6c4c67bf0 100644 --- a/source/index.txt +++ b/source/index.txt @@ -44,7 +44,7 @@ Object Document Mappers (ODM) as opposed to Object Relational Mappers The ODM officially supported by MongoDB is Mongoid, originally written by Durran Jordan. -For tutorials on Mongoid, see :doc:`/mongoid`. +For tutorials on Mongoid, see the `Mongoid Manual `_. .. COMMENT For the actual build, see mongodb/docs-ruby repo which pulls the documentation source from: .. mongo-ruby-driver, @@ -61,7 +61,6 @@ For tutorials on Mongoid, see :doc:`/mongoid`. quick-start ruby-driver-tutorials bson-tutorials - mongoid API /reference/driver-compatibility /reference/additional-resources diff --git a/source/meta/404.txt b/source/meta/404.txt new file mode 100644 index 000000000..0bdd725c2 --- /dev/null +++ b/source/meta/404.txt @@ -0,0 +1,7 @@ +:orphan: + +============== +File not found +============== + +The URL you requested does not exist or has been removed. From 0645c63f922df86a1dd09bbd3536ad73bf9db182 Mon Sep 17 00:00:00 2001 From: Emily Date: Thu, 8 Jun 2017 16:50:59 +0200 Subject: [PATCH 114/442] RUBY-1125 Document how connection pooling works for Ruby driver --- .../tutorials/ruby-driver-create-client.txt | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index ab7edabb8..6309577b4 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -412,3 +412,56 @@ Details on Timeout Options interrupted on the server. See the `CRUD specification `_ for details on operations that support this option. + +Details on Connection Pooling +----------------------------- + +``Mongo::Client`` instances have a connection pool per server in your MongoDB topology. The pool opens connections on +demand to support the number of concurrent MongoDB operations your application requires. There is no thread-affinity +for connections. + +The client instance opens one additional connection per server in your MongoDB topology for monitoring the server's +state. + +The size of each connection pool is capped at ``max_pool_size``, which defaults to 5. When a thread in your application +begins an operation on MongoDB, it tries to retrieve a connection from the pool to send that operation on. If there +are some connections available in the pool, it checks out a connection from the pool and uses it for the operation. +If there are no connections available and the size of the pool is less than the ``max_pool_size``, a new connection +will be created. If all connections are in use and the pool has reached its maximum, the thread pauses and waits for a +connection to be returned to the pool by another thread. + +You can also set the minimum number of connections initialized when the pool is created with the ``min_pool_size`` +setting. This is helpful if your application is prone to load spikes and you want to avoid a slow-down due to new +connections being created if a spike occurs. The default ``min_pool_size`` is 1. + +There is one more setting affecting the connection pool, called ``wait_queue_timeout``, defined in seconds. This setting +determines how long a thread will wait for a connection to be returned to the pool, when there are no connections +available. If this timeout is reached, a ``Timeout::Error`` will be raised. The default is 1 second. + +The default configuration for a ``Mongo::Client`` works for most applications: + +.. code-block:: ruby + + client = Mongo::Client.new(["localhost:27017"]) + +Create this client **once** for each process, and reuse it for all operations. It is a common mistake to create a new +client for each request, which is very inefficient and not what the client was designed for. + +To support extremely high numbers of concurrent MongoDB operations within one process, increase ``max_pool_size``: + +.. code-block:: ruby + + client = Mongo::Client.new(["localhost:27017"], max_pool_size: 200) + +Any number of threads are allowed to wait for connections to become available, and they can wait the default (1 second) +or the ``wait_queue_timeout`` setting: + +.. code-block:: ruby + + client = Mongo::Client.new(["localhost:27017"], wait_queue_timeout: 0.5) + +When ``#close`` is called on a client by any thread, all connections are closed: + +.. code-block:: ruby + + client.close From e34975103eacbc180694bf444b6223b26860cd80 Mon Sep 17 00:00:00 2001 From: Emily Date: Fri, 9 Jun 2017 15:10:23 +0200 Subject: [PATCH 115/442] RUBY-1125 Add an example --- .../tutorials/ruby-driver-create-client.txt | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 6309577b4..7af1c4a05 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -430,13 +430,20 @@ If there are no connections available and the size of the pool is less than the will be created. If all connections are in use and the pool has reached its maximum, the thread pauses and waits for a connection to be returned to the pool by another thread. -You can also set the minimum number of connections initialized when the pool is created with the ``min_pool_size`` -setting. This is helpful if your application is prone to load spikes and you want to avoid a slow-down due to new -connections being created if a spike occurs. The default ``min_pool_size`` is 1. +The number of seconds the thread will wait is configurable. This setting, called ``wait_queue_timeout``, is defined in +seconds. It determines how long a thread will wait for a connection to be returned to the pool, when there are no +connections available. If this timeout is reached, a ``Timeout::Error`` is raised. The default is 1 second. + +You can also set the number of connections initialized when the pool is created with the ``min_pool_size`` +setting. This is helpful if your application experiences load spikes and you want to avoid a slow-down when +connections are created at the time of the spike. The default ``min_pool_size`` is 1. + +For example, a client connected to a 3-node replica set opens 3 monitoring sockets. It also opens as many sockets as +needed to support a multi-threaded application's concurrent operations on each server, up to ``maxPoolSize``. If the +application only uses the primary (the default), then only the primary connection pool grows and the total connections +is at most 8 (5 connections for the primary pool + 3 monitoring connections). If the application uses a read +preference to query the secondaries, their pools also grow and the total connections can reach 18 (5 + 5 + 5 + 3). -There is one more setting affecting the connection pool, called ``wait_queue_timeout``, defined in seconds. This setting -determines how long a thread will wait for a connection to be returned to the pool, when there are no connections -available. If this timeout is reached, a ``Timeout::Error`` will be raised. The default is 1 second. The default configuration for a ``Mongo::Client`` works for most applications: From cfd65aec88e26aefb853a4b8b1966bd747b9e74d Mon Sep 17 00:00:00 2001 From: Emily Date: Mon, 12 Jun 2017 11:02:20 +0200 Subject: [PATCH 116/442] RUBY-1125 Updates after code review --- source/tutorials/ruby-driver-create-client.txt | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 7af1c4a05..b3b0375d5 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -427,7 +427,7 @@ The size of each connection pool is capped at ``max_pool_size``, which defaults begins an operation on MongoDB, it tries to retrieve a connection from the pool to send that operation on. If there are some connections available in the pool, it checks out a connection from the pool and uses it for the operation. If there are no connections available and the size of the pool is less than the ``max_pool_size``, a new connection -will be created. If all connections are in use and the pool has reached its maximum, the thread pauses and waits for a +will be created. If all connections are in use and the pool has reached its maximum, the thread waits for a connection to be returned to the pool by another thread. The number of seconds the thread will wait is configurable. This setting, called ``wait_queue_timeout``, is defined in @@ -435,10 +435,11 @@ seconds. It determines how long a thread will wait for a connection to be return connections available. If this timeout is reached, a ``Timeout::Error`` is raised. The default is 1 second. You can also set the number of connections initialized when the pool is created with the ``min_pool_size`` -setting. This is helpful if your application experiences load spikes and you want to avoid a slow-down when -connections are created at the time of the spike. The default ``min_pool_size`` is 1. +setting. This is helpful if your application experiences load spikes and you want to avoid the latency of creating new +connections at the beginning of a spike. The default ``min_pool_size`` is 1. -For example, a client connected to a 3-node replica set opens 3 monitoring sockets. It also opens as many sockets as +Here is an example of estimating the number of connections a multi-threaded application will open: +A client connected to a 3-node replica set opens 3 monitoring sockets. It also opens as many sockets as needed to support a multi-threaded application's concurrent operations on each server, up to ``maxPoolSize``. If the application only uses the primary (the default), then only the primary connection pool grows and the total connections is at most 8 (5 connections for the primary pool + 3 monitoring connections). If the application uses a read From b1292829957e8e2ffa0766b94dc9a619e88f66f9 Mon Sep 17 00:00:00 2001 From: Emily Date: Fri, 7 Jul 2017 15:32:03 +0200 Subject: [PATCH 117/442] Fix formatting of max pool size variable --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index b3b0375d5..7ab252878 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -440,7 +440,7 @@ connections at the beginning of a spike. The default ``min_pool_size`` is 1. Here is an example of estimating the number of connections a multi-threaded application will open: A client connected to a 3-node replica set opens 3 monitoring sockets. It also opens as many sockets as -needed to support a multi-threaded application's concurrent operations on each server, up to ``maxPoolSize``. If the +needed to support a multi-threaded application's concurrent operations on each server, up to ``max_pool_size``. If the application only uses the primary (the default), then only the primary connection pool grows and the total connections is at most 8 (5 connections for the primary pool + 3 monitoring connections). If the application uses a read preference to query the secondaries, their pools also grow and the total connections can reach 18 (5 + 5 + 5 + 3). From 78f613e8acb8ef0d0f1c75005bf26c0855bbbb89 Mon Sep 17 00:00:00 2001 From: Emily Date: Fri, 29 Sep 2017 15:57:25 +0200 Subject: [PATCH 118/442] RUBY-1228 Change stream tutorial --- .../ruby-driver-collection-tasks.txt | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/source/tutorials/ruby-driver-collection-tasks.txt b/source/tutorials/ruby-driver-collection-tasks.txt index 89389c3c9..8bde7ce98 100644 --- a/source/tutorials/ruby-driver-collection-tasks.txt +++ b/source/tutorials/ruby-driver-collection-tasks.txt @@ -101,3 +101,53 @@ documents must contain an ``age`` field which is a number. { '$type' => "number" } } }) + + +Change Streams +``````````````` + +As of version 3.6 of the MongoDB server, a new ```$changeStream`` pipeline stage is supported in the aggregation framework. +Specifying this stage first in an aggregation pipeline allows users to request that notifications are sent for all +changes to a particular collection. The helper method available for change streams on a collection object is preferred +to running a raw aggregation with a $changeStream stage, for the purpose of supporting resumability. + +The change stream will attempt to resume one time upon a network error or a server error indicating that a failover is +taking place. + +To start receiving change notifications, use the method ``#watch`` on a collection: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + stream = client[:test].watch + +Each document can be retrieved and processed by your application as in the following code example: + +.. code-block:: ruby + + enum = stream.to_enum + begin + while true + change = enum.next + # do something + sleep(1) + end + rescue => e + # we know for sure it's unrecoverable + log_error(e) + end + +The change documents will expose a resume token at the key '_id'. This token can be used to specify a logical starting +point for a new a ChangeStream: + +.. code-block:: ruby + + stream = client[:test].watch([], resume_token: token) + +A subset of aggregation pipeline stages are available to a change stream and can be passed into the ``#watch`` method: + +.. code-block:: ruby + + stream = client[:test].watch([ { '$match' => { 'operationType' => 'update' } }]) + + From 4af3b7a4a4c865a29673a594990057cf3d80ed7c Mon Sep 17 00:00:00 2001 From: Tony Ta Date: Mon, 15 Jan 2018 13:49:12 -0800 Subject: [PATCH 119/442] typos; copy fixes for ruby bson 4 tutorial --- docs/tutorials/bson-v4.txt | 43 +++++++++++++++++++------------------- 1 file changed, 21 insertions(+), 22 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index c3c495690..78916678a 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -26,7 +26,7 @@ To install the gem manually: .. code-block:: sh - gem install bson + gem install bson -v '~> 4.0' To install the gem with bundler, include the following in your Gemfile: @@ -34,14 +34,13 @@ To install the gem with bundler, include the following in your Gemfile: gem 'bson', '~> 4.0' -The BSON gem is compatible with MRI 1.9.3, 2.0.x, 2.1.x, 2.2.x, JRuby 1.7.x, and -Rubinius 2.5.x +The BSON gem is compatible with MRI >= 1.9.3, JRuby 1.7.x, and Rubinius 2.5.x BSON Serialization ------------------ Getting a Ruby object's raw BSON representation is done by calling ``to_bson`` -on the Ruby object, which will return a ``ByteBuffer``. For example: +on the Ruby object, which will return a ``BSON::ByteBuffer``. For example: .. code-block:: ruby @@ -49,12 +48,12 @@ on the Ruby object, which will return a ``ByteBuffer``. For example: 1024.to_bson Generating an object from BSON is done via calling ``from_bson`` on the class -you wish to instantiate and passing it a ``ByteBuffer`` instance. +you wish to instantiate and passing it a ``BSON::ByteBuffer`` instance. .. code-block:: ruby String.from_bson(byte_buffer) - Int32.from_bson(byte_buffer) + BSON::Int32.from_bson(byte_buffer) Byte Buffers ------------ @@ -90,14 +89,14 @@ Reading from the buffer is done via the following API: buffer.get_byte # Pulls a single byte from the buffer. buffer.get_bytes(value) # Pulls n number of bytes from the buffer. - buffer.get_cstring # Pulls a null-terminted string from the buffer. + buffer.get_cstring # Pulls a null-terminated string from the buffer. buffer.get_double # Pulls a 64-bit floating point from the buffer. buffer.get_int32 # Pulls a 32-bit integer (4 bytes) from the buffer. buffer.get_int64 # Pulls a 64-bit integer (8 bytes) from the buffer. buffer.get_string # Pulls a UTF-8 string from the buffer. -Convertig a buffer to it's raw bytes, for example to send over a socket, -is done by simply calling `to_s` on the buffer. +Converting a buffer to its raw bytes (for example, for sending over a socket) +is done by simply calling ``to_s`` on the buffer. .. code-block:: ruby @@ -119,20 +118,20 @@ specific to the specification: ``BSON::Binary`` ```````````````` -This is a representation of binary data, and must provide the raw data and -a subtype when constructing. +This is a representation of binary data. The raw data and a subtype must be +provided when constructing. .. code-block:: ruby BSON::Binary.new(binary_data, :md5) -Valid subtypes are: ``:generic``, ``:function``, ``:old``, ``:uuid_old``, ``:uuid``, -``:md5``, ``:user``. +Valid subtypes are: ``:generic``, ``:function``, ``:old``, ``:uuid_old``, +``:uuid``, ``:md5``, ``:user``. ``BSON::Code`` `````````````` -Represents a string of Javascript code. +Represents a string of JavaScript code. .. code-block:: ruby @@ -141,7 +140,7 @@ Represents a string of Javascript code. ``BSON::CodeWithScope`` ``````````````````````` -Represents a string of Javascript code with a hash of values. +Represents a string of JavaScript code with a hash of values. .. code-block:: ruby @@ -150,8 +149,8 @@ Represents a string of Javascript code with a hash of values. ``BSON::Document`` `````````````````` -This is a subclass of a hash that stores all keys as strings but allows access to -them with symbol keys. +This is a subclass of ``Hash`` that stores all keys as strings, but allows +access to them with symbol keys. .. code-block:: ruby @@ -206,8 +205,8 @@ Represents a placeholder for a value that was not provided. ``BSON::Decimal128`` ```````````````````` -Represents a 128-bit decimal-based floating-point value capable of -emulating decimal rounding with exact precision.. +Represents a 128-bit decimal-based floating-point value capable of emulating +decimal rounding with exact precision. .. code-block:: ruby @@ -260,8 +259,8 @@ them. Special Ruby Date Classes ------------------------- -Ruby's ``Date`` and ``DateTime`` are able to be serialized, but when -they are deserialized they will always be returned as a ``Time`` since the BSON +Ruby's ``Date`` and ``DateTime`` are able to be serialized, but when they are +deserialized, they will always be returned as a ``Time`` since the BSON specification only has a ``Time`` type and knows nothing about Ruby. @@ -302,7 +301,7 @@ Please use the ``Regexp::Raw`` class to instantiate your BSON regular expression pattern and options you want. When regular expressions are deserialized, they return a wrapper that holds the -raw regex string, but does not compile it. In order to get the Ruby ``Regexp`` +raw regex string, but do not compile it. In order to get the Ruby ``Regexp`` object, one must call ``compile`` on the returned object. .. code-block:: ruby From 2c3a062d0ac89fc165546e234c8bcf6405b95a79 Mon Sep 17 00:00:00 2001 From: Emily Date: Tue, 16 Jan 2018 15:08:26 +0100 Subject: [PATCH 120/442] Fix Mongo::Client options documentation --- .../tutorials/ruby-driver-create-client.txt | 57 +++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 7ab252878..f57c2afe2 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -349,6 +349,63 @@ Ruby Options - ``Float`` - 0.015 + * - ``:app_name`` + - Application name that is printed to the mongod logs upon establishing a connection + in server versions >= 3.4. + - ``String`` + - none + + * - ``:compressors`` + - A list of potential compressors to use, in order of preference. The driver chooses the first + compressor that is also supported by the server. Currently the driver only supports 'zlib'. + - ``Array`` + - none + + * - ``:id_generator`` + - A custom object to generate ids for documents. Must respond to #generate. + - ``Object`` + - none + + * - ``:logger`` + - A custom logger. + - ``Object`` + - ``Logger`` + + * - ``:max_idle_time`` + - The maximum seconds a socket can remain idle since it has been checked in to the pool. + - ``Integer`` + - none + + * - ``:max_read_retries`` + - The maximum number of read retries on mongos query failures. + - ``Integer`` + - 1 + + * - ``:monitoring`` + - The monitoring object. + - ``Object`` + - none + + * - ``:platform`` + - Platform information to include in the metadata printed to the mongod logs upon establishing a + connection in server versions >= 3.4. + - ``String`` + - none + + * - ``:read_retry_interval`` + - The interval, in seconds, in which reads on a mongos are retried. + - ``Integer`` + - 5 + + * - ``:truncate_logs`` + - Whether to truncate the logs at the default 250 characters. + - ``Boolean`` + - true + + * - ``:zlib_compression_level`` + - The Zlib compression level to use, if using compression. See Ruby's Zlib module for valid levels. + - ``Integer`` + - none Details on Timeout Options From d84194857d162b870bc9d9378bcdab58ccf0ddf2 Mon Sep 17 00:00:00 2001 From: Emily Date: Tue, 16 Jan 2018 15:30:38 +0100 Subject: [PATCH 121/442] Update compatibility matrices --- source/reference/driver-compatibility.txt | 38 +++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index b491b323e..227fdeccd 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -24,6 +24,15 @@ MongoDB Compatibility - MongoDB 3.0 - MongoDB 3.2 - MongoDB 3.4 + - MongoDB 3.6 + + * - 2.5 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| * - 2.4 - |checkmark| @@ -31,6 +40,7 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| + - * - 2.3 - |checkmark| @@ -38,6 +48,7 @@ MongoDB Compatibility - |checkmark| - |checkmark| - + - * - 2.2 - |checkmark| @@ -45,6 +56,7 @@ MongoDB Compatibility - |checkmark| - |checkmark| - + - * - 2.0 - |checkmark| @@ -52,6 +64,7 @@ MongoDB Compatibility - |checkmark| - - + - * - 1.12 - |checkmark| @@ -59,6 +72,7 @@ MongoDB Compatibility - |checkmark| - - + - * - 1.11 - |checkmark| @@ -66,6 +80,7 @@ MongoDB Compatibility - - - + - * - 1.10 - |checkmark| @@ -73,6 +88,7 @@ MongoDB Compatibility - - - + - * - 1.9 - |checkmark| @@ -80,6 +96,7 @@ MongoDB Compatibility - - - + - * - 1.8 - |checkmark| @@ -87,6 +104,7 @@ MongoDB Compatibility - - - + - .. include:: /includes/older-server-versions-unsupported.rst @@ -109,8 +127,19 @@ Language Compatibility - Ruby 2.1 - Ruby 2.2 - Ruby 2.3 + - Ruby 2.4 - JRuby + * - 2.5 + - + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + * - 2.4 - - |checkmark| @@ -118,6 +147,7 @@ Language Compatibility - |checkmark| - |checkmark| - |checkmark| + - - |checkmark| * - 2.3 @@ -127,6 +157,7 @@ Language Compatibility - |checkmark| - |checkmark| - |checkmark| + - - |checkmark| * - 2.2 @@ -136,6 +167,7 @@ Language Compatibility - |checkmark| - |checkmark| - |checkmark| + - - |checkmark| * - 2.1 @@ -145,6 +177,7 @@ Language Compatibility - |checkmark| - |checkmark| - |checkmark| + - - |checkmark| * - 2.0 @@ -154,6 +187,7 @@ Language Compatibility - |checkmark| - |checkmark| - |checkmark| + - - |checkmark| * - 1.9 - 1.12 @@ -163,6 +197,7 @@ Language Compatibility - |checkmark| - - + - - |checkmark| * - 1.8 @@ -172,6 +207,7 @@ Language Compatibility - - - + - - |checkmark| * - 1.7 @@ -181,6 +217,7 @@ Language Compatibility - - - + - - |checkmark| * - 1.6 @@ -190,6 +227,7 @@ Language Compatibility - - - + - - |checkmark| .. include:: /includes/unicode-checkmark.rst From 176351fe3d04c109bd173d20ac012ce9ac8f3b5d Mon Sep 17 00:00:00 2001 From: Emily Date: Tue, 16 Jan 2018 14:02:29 +0100 Subject: [PATCH 122/442] RUBY-1271 Sessions tutorial --- source/ruby-driver-tutorials.txt | 1 + source/tutorials/ruby-driver-sessions.txt | 115 ++++++++++++++++++++++ 2 files changed, 116 insertions(+) create mode 100644 source/tutorials/ruby-driver-sessions.txt diff --git a/source/ruby-driver-tutorials.txt b/source/ruby-driver-tutorials.txt index 6ce883c3f..855650ca0 100644 --- a/source/ruby-driver-tutorials.txt +++ b/source/ruby-driver-tutorials.txt @@ -19,6 +19,7 @@ operations available in the Ruby driver. /tutorials/ruby-driver-crud-operations /tutorials/ruby-driver-collection-tasks /tutorials/ruby-driver-projections + /tutorials/ruby-driver-sessions /tutorials/ruby-driver-admin-tasks /tutorials/ruby-driver-indexing /tutorials/ruby-driver-collations diff --git a/source/tutorials/ruby-driver-sessions.txt b/source/tutorials/ruby-driver-sessions.txt new file mode 100644 index 000000000..525e2c051 --- /dev/null +++ b/source/tutorials/ruby-driver-sessions.txt @@ -0,0 +1,115 @@ +============================ +Creating and using Sessions +============================ + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + + +Version 3.6 of the server introduces the concept of logical sessions for clients. +A session is an abstract concept that represents a set of sequential operations executed +by an application that are related in some way. A session object can be create via a ``Mongo::Client`` +and passed to operation methods that should be executed in the context of that session. + +Please note that sessions are not thread safe. They can only be used by one thread at a time. + +Creating a session from a ``Mongo::Client`` +------------------------------------------- + +A session can be created by calling the ``start_session`` method on a client: + +.. code-block:: ruby + + session = client.start_session + + +It is valid to call ``start_session`` with no options set. This will result in a +session that has no effect on the operations performed in the context of that session, +other than to include a session ID in commands sent to the server. Please see the API docs for all supported +session options. + +An error will be thrown if the driver is connected to a deployment that does not support sessions and the +``start_session`` method is called. + +Note that server sessions are discarded server-side if not used for a certain period of time. That said, +be aware that if the application calls ``#start_session`` on a client and waits more than 1 minute to use +the session, it risks getting errors due to the session going stale before it is used. + + + +Using a session +--------------- +A session can be passed to most operations performed via the driver so that the operation can be executed in the +context of that session. Please see the API docs for which methods support a session argument. + +Create a session and execute an insert, then a find using that session: + +.. code-block:: ruby + + session = client.start_session + client[:artists].insert_one({ :name => 'FKA Twigs' }, session: session) + client[:artists].find({ :name => 'FKA Twigs' }, session: session).first + +If you like to call methods on a ``Mongo::Collection::View`` that use a particular session, you can create the +``Mongo::Collection::View`` with the session and then call methods on it: + +.. code-block:: ruby + + session = client.start_session(causal_consistency: true) + view = client[:artists].find({ :name => 'FKA Twigs' }, session: session) + view.count # will use the session + +You can also pass the session option to the methods directly. This session will override any session associated with +the ``Mongo::Collection::View``: + +.. code-block:: ruby + + session = client.start_session + second_session = client.start_session + view = client[:artists].find({ :name => 'FKA Twigs' }, session: session) + view.count(session: second_session) # will use the second_session + +Causal Consistency +------------------ +A causally consistent session will let you read your writes and guarantee monotonically increasing +reads from secondaries. +To create a causally consistent session, set the ``causal_consistency`` option to true: + +.. code-block:: ruby + + session = client.start_session(causal_consistency: true) + + # The update message goes to the primary. + collection = client[:artists] + collection.update_one({ '_id' => 1 }, { '$set' => { 'x' => 0 } }, session: session) + + # Read your write, even when reading from a secondary! + collection.find({ '_id' => 1 }, limit: 1, session: session).first + + # This query returns data at least as new as the previous query, + # even if it chooses a different secondary. + collection.find({ '_id' => 2 }, limit: 1, session: session).first + +Since unacknowledged writes don't receive a response from the server (or don't wait for a response), the driver +has no way of keeping track of where the unacknowledged write is in logical time. Therefore, be aware that causally +consistent reads are not causally consistent with unacknowledged writes. + +Note that if you set the causal_consistency option to nil as in ``(causal_consistency: nil)``, it will be interpreted +as false. + +End a session +------------- +To end a session, call the ``end_session`` method: + +.. code-block:: ruby + + session.end_session + +The Ruby driver will then add the id for the corresponding server session to a pool for reuse. +When a client is closed, the driver will send a command to the server to end all sessions it has cached +in its server session pool. You may see this command in your logs when a client is closed. From 2f3442d318dc4221847a464e864dbc145fd2d1c9 Mon Sep 17 00:00:00 2001 From: Emily Date: Tue, 16 Jan 2018 14:32:22 +0100 Subject: [PATCH 123/442] RUBY-1271 Change Streams tutorial --- source/ruby-driver-tutorials.txt | 1 + .../tutorials/ruby-driver-change-streams.txt | 75 +++++++++++++++++++ 2 files changed, 76 insertions(+) create mode 100644 source/tutorials/ruby-driver-change-streams.txt diff --git a/source/ruby-driver-tutorials.txt b/source/ruby-driver-tutorials.txt index 855650ca0..19d16b157 100644 --- a/source/ruby-driver-tutorials.txt +++ b/source/ruby-driver-tutorials.txt @@ -20,6 +20,7 @@ operations available in the Ruby driver. /tutorials/ruby-driver-collection-tasks /tutorials/ruby-driver-projections /tutorials/ruby-driver-sessions + /tutorials/ruby-driver-change-streams /tutorials/ruby-driver-admin-tasks /tutorials/ruby-driver-indexing /tutorials/ruby-driver-collations diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt new file mode 100644 index 000000000..56e475154 --- /dev/null +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -0,0 +1,75 @@ +============== +Change Streams +============== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +As of version 3.6 of the MongoDB server, a new ``$changeStream`` pipeline stage is supported in the aggregation +framework. The Ruby driver provides an API for receiving notifications for changes to a particular collection using this +new pipeline stage. Although you can create a change stream using the pipeline operator and aggregation framework +directly, it is recommended to use the driver API described below as the driver resumes the change stream if there is +timeout or network error. + +Change streams on the server requires a ``"majority"`` read concern or no read concern. + +Change streams do not work properly with JRuby because of the issue documented here_. + +.. _here: https://github.com/jruby/jruby/issues/4212 + +Namely, JRuby eagerly evaluates ```#next`` on an Enumerator in a background green thread. +So calling ```#next`` on the change stream will cause getmores to be called in a loop in the background. + +Watching for changes on a particular collection +----------------------------------------------- + +A change stream is created by calling the ``#watch`` method on a collection: + +.. code-block:: ruby + + stream = collection.watch + collection.insert_one(a: 1) + doc = stream.to_enum.next + process(doc) + + +You can also receive the notifications as they are available: + +.. code-block:: ruby + + stream = client[:test].watch + enum = stream.to_enum + while doc = enum.next + process(doc) + end + + +The change stream can take filters in the aggregation framework pipeline operator format: + +.. code-block:: ruby + + stream = collection.watch([{'$match' => { 'operationType' => {'$in' => ['insert', 'replace'] } } }, + {'$match' => { 'fullDocument.n' => { '$gte' => 1 } } } + ]) + enum = stream.to_enum + while doc = enum.next + process(doc) + end + +Close a Change Stream +--------------------- + +You can close a Change Stream by calling the ``#close`` method: + +.. code-block:: ruby + + stream = collection.watch + collection.insert_one(a: 1) + doc = stream.to_enum.next + process(doc) + stream.close From 36c66e88de33c526d090a178794bd642bc1f8d1b Mon Sep 17 00:00:00 2001 From: Emily Date: Tue, 16 Jan 2018 14:45:26 +0100 Subject: [PATCH 124/442] RUBY-1271 Add retry_writes documentation for a client --- .../tutorials/ruby-driver-create-client.txt | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index f57c2afe2..fc962070a 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -407,6 +407,12 @@ Ruby Options - ``Integer`` - none + * - ``:retry_writes`` + - If a single-statement write operation fails from a network error, the driver automatically retries it once + when connected to server versions 3.6+. + - ``Boolean`` + - false + Details on Timeout Options -------------------------- @@ -530,3 +536,20 @@ When ``#close`` is called on a client by any thread, all connections are closed: .. code-block:: ruby client.close + +Details on Retryable Writes +--------------------------- + +If the ``retry_writes`` option is set to true, the driver will retry single-statement write operations that fail from +a network error. The driver automatically retries the operation once. + +Most of the write methods you use day-to-day on a collection will be retryable in 3.6. They are: + +- ``collection#insert_one`` +- ``collection#update_one`` +- ``collection#delete_one`` +- ``collection#replace_one`` +- ``collection#find_one_and_update`` +- ``collection#find_one_and_replace`` +- ``collection#find_one_and_delete`` +- ``collection#bulk_write`` # as long as there is no ``update_many`` or ``delete_many`` in the list of operations From 18dab72893a98f03e890dbf11793cdc95e1a81a0 Mon Sep 17 00:00:00 2001 From: Emily Date: Wed, 17 Jan 2018 11:16:49 +0100 Subject: [PATCH 125/442] RUBY-1271 Minor fixes to tutorial --- source/tutorials/ruby-driver-create-client.txt | 2 +- source/tutorials/ruby-driver-sessions.txt | 13 ++++++------- 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index fc962070a..73a72b72c 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -552,4 +552,4 @@ Most of the write methods you use day-to-day on a collection will be retryable i - ``collection#find_one_and_update`` - ``collection#find_one_and_replace`` - ``collection#find_one_and_delete`` -- ``collection#bulk_write`` # as long as there is no ``update_many`` or ``delete_many`` in the list of operations +- ``collection#bulk_write`` # for all single statement ops (i.e. not for ``update_many`` or ``delete_many``) diff --git a/source/tutorials/ruby-driver-sessions.txt b/source/tutorials/ruby-driver-sessions.txt index 525e2c051..e40bb5896 100644 --- a/source/tutorials/ruby-driver-sessions.txt +++ b/source/tutorials/ruby-driver-sessions.txt @@ -41,10 +41,9 @@ be aware that if the application calls ``#start_session`` on a client and waits the session, it risks getting errors due to the session going stale before it is used. - Using a session --------------- -A session can be passed to most operations performed via the driver so that the operation can be executed in the +A session object can be passed to most driver methods so that the operation can be executed in the context of that session. Please see the API docs for which methods support a session argument. Create a session and execute an insert, then a find using that session: @@ -53,9 +52,9 @@ Create a session and execute an insert, then a find using that session: session = client.start_session client[:artists].insert_one({ :name => 'FKA Twigs' }, session: session) - client[:artists].find({ :name => 'FKA Twigs' }, session: session).first + client[:artists].find({ :name => 'FKA Twigs' }, limit: 1, session: session).first -If you like to call methods on a ``Mongo::Collection::View`` that use a particular session, you can create the +If you like to call methods on a ``Mongo::Collection::View`` in the context of a particular session, you can create the ``Mongo::Collection::View`` with the session and then call methods on it: .. code-block:: ruby @@ -89,14 +88,14 @@ To create a causally consistent session, set the ``causal_consistency`` option t collection.update_one({ '_id' => 1 }, { '$set' => { 'x' => 0 } }, session: session) # Read your write, even when reading from a secondary! - collection.find({ '_id' => 1 }, limit: 1, session: session).first + collection.find({ '_id' => 1 }, session: session).first # This query returns data at least as new as the previous query, # even if it chooses a different secondary. - collection.find({ '_id' => 2 }, limit: 1, session: session).first + collection.find({ '_id' => 2 }, session: session).first Since unacknowledged writes don't receive a response from the server (or don't wait for a response), the driver -has no way of keeping track of where the unacknowledged write is in logical time. Therefore, be aware that causally +has no way of keeping track of where the unacknowledged write is in logical time. Therefore, causally consistent reads are not causally consistent with unacknowledged writes. Note that if you set the causal_consistency option to nil as in ``(causal_consistency: nil)``, it will be interpreted From c966e52103d9e2cbba04e00cee39fdf9426c6e0b Mon Sep 17 00:00:00 2001 From: kay Date: Wed, 17 Jan 2018 12:32:01 -0500 Subject: [PATCH 126/442] docs update: installation docs for v2.5 --- source/installation.txt | 9 ++------- source/whats-new.txt | 10 +++------- 2 files changed, 5 insertions(+), 14 deletions(-) diff --git a/source/installation.txt b/source/installation.txt index e9fa2f30e..ccd003a57 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -14,14 +14,9 @@ Install the gem The driver can be installed manually or with bundler. -To install the gem manually: +To install the mongo gem manually: .. code-block:: sh - gem install mongo + gem install mongo -v 2.5.0 -To install the gem with bundler, include the following in your Gemfile: - -.. code-block:: ruby - - gem 'mongo', '~> 2.4' diff --git a/source/whats-new.txt b/source/whats-new.txt index 0944f6e1e..f351001f7 100644 --- a/source/whats-new.txt +++ b/source/whats-new.txt @@ -4,13 +4,9 @@ What's New .. default-domain:: mongodb -What's New in the 2.4 Driver +What's New in the 2.5 Driver ---------------------------- -- :doc:`Decimal128`, - a new :doc:`BSON datatype ` which employs 128-bit - decimal-based floating-point values capable of emulating decimal - rounding with exact precision. +- Added support for MongoDB server version 3.6. -.. COMMENT need internal link to #specify-decimal128 in the CRUD page -.. COMMENT Collation and other items to come here \ No newline at end of file +- Removed support for MongoDB server version 2.4. From 8dd426c48081013021c17b0017c8409e518658bb Mon Sep 17 00:00:00 2001 From: Emily Date: Thu, 18 Jan 2018 12:05:34 +0100 Subject: [PATCH 127/442] Fix documentation typos --- source/reference/driver-compatibility.txt | 2 +- source/tutorials/ruby-driver-change-streams.txt | 4 ++-- source/tutorials/ruby-driver-create-client.txt | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 227fdeccd..a9edf1ae6 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -27,7 +27,7 @@ MongoDB Compatibility - MongoDB 3.6 * - 2.5 - - |checkmark| + - - |checkmark| - |checkmark| - |checkmark| diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 56e475154..4c5eb047f 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -22,8 +22,8 @@ Change streams do not work properly with JRuby because of the issue documented h .. _here: https://github.com/jruby/jruby/issues/4212 -Namely, JRuby eagerly evaluates ```#next`` on an Enumerator in a background green thread. -So calling ```#next`` on the change stream will cause getmores to be called in a loop in the background. +Namely, JRuby eagerly evaluates ``#next`` on an Enumerator in a background green thread. +So calling ``#next`` on the change stream will cause getmores to be called in a loop in the background. Watching for changes on a particular collection ----------------------------------------------- diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 73a72b72c..731a3045f 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -552,4 +552,4 @@ Most of the write methods you use day-to-day on a collection will be retryable i - ``collection#find_one_and_update`` - ``collection#find_one_and_replace`` - ``collection#find_one_and_delete`` -- ``collection#bulk_write`` # for all single statement ops (i.e. not for ``update_many`` or ``delete_many``) +- ``collection#bulk_write`` (for all single statement ops, i.e. not for ``update_many`` or ``delete_many``) From 3e070e7c59a7882fd28643a4dd1e33cb66c62f1f Mon Sep 17 00:00:00 2001 From: Emily Date: Wed, 24 Jan 2018 17:57:36 +0100 Subject: [PATCH 128/442] Add missing URI options to client tutorial --- source/tutorials/ruby-driver-create-client.txt | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 731a3045f..e9eb0bc99 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -145,6 +145,15 @@ URI Options Conversions * - authMechanismProperties=Strings - ``{ :auth_mech_properties => { :service_realm => String, :canonicalize_host_name => true|false, :service_name => String } }`` + * - appName=String + - ``:app_name => String`` + + * - compressors=Strings + - ``:compressors => Array`` + + * - zlibCompressionLevel=Integer + - ``:zlib_compression_level => Integer`` + Ruby Options ------------ From 3330deb8d09d9039ddc3c3f633dd51b7ab92a8f1 Mon Sep 17 00:00:00 2001 From: Emily Date: Wed, 24 Jan 2018 18:05:24 +0100 Subject: [PATCH 129/442] RUBY-1282 Add SRV record support to Tutorial documentation --- source/tutorials/ruby-driver-create-client.txt | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index e9eb0bc99..31aa25b4b 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -54,6 +54,14 @@ would like to bypass the auto-discovery, pass the Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb', :connect => :sharded) Mongo::Client.new('mongodb://127.0.0.1:27017/mydb?connect=sharded') +The URI parser in the driver can also accept the protocol ``mongodb+srv`` as a logical +pre-processing step before it considers the connection string and other options. +In this protocol, the comma separated list of host names is replaced with a single host name. The format is: + +.. code-block:: ruby + + Mongo::Client.new('mongodb+srv://test5.test.build.mongodb.cc') + .. _ruby-driver-client-options: Client Options @@ -65,7 +73,7 @@ providing them in the URI. Since the URI options are required in camel case, which is not the Ruby standard, the following table shows the option in the URI and its corresponding option if passed -to the constructor in Ruby. +to the constructor in Ruby. .. note:: From cfd1714f1e01476103e8e37b63e660e05b580092 Mon Sep 17 00:00:00 2001 From: Emily Date: Mon, 29 Jan 2018 18:03:51 +0100 Subject: [PATCH 130/442] RUBY-1283 Add documentation for TCP keepalive settings --- source/tutorials/ruby-driver-create-client.txt | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 31aa25b4b..9f18eda11 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -570,3 +570,18 @@ Most of the write methods you use day-to-day on a collection will be retryable i - ``collection#find_one_and_replace`` - ``collection#find_one_and_delete`` - ``collection#bulk_write`` (for all single statement ops, i.e. not for ``update_many`` or ``delete_many``) + + +TCP keepalive configuration +--------------------------- + +The driver sets TCP keepalive by default. The following default values are also set if the system +value can be determined and if the driver default value is less than the system value. + +- ``tcp_keepalive_time``: 300 seconds +- ``tcp_keepalive_intvl``: 10 seconds +- ``tcp_keepalive_cnt``: 9 probes + + +Please see the `MongoDB Diagnostics FAQ keepalive section `_ +for instructions on setting these values at the system level. From 1cc331449d7e510d42fd63ab0429d13271181583 Mon Sep 17 00:00:00 2001 From: Emily Date: Tue, 27 Feb 2018 15:59:39 +0100 Subject: [PATCH 131/442] RUBY-1302 Don't use sessions with unacknowledged writes --- source/tutorials/ruby-driver-sessions.txt | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/source/tutorials/ruby-driver-sessions.txt b/source/tutorials/ruby-driver-sessions.txt index e40bb5896..19849abe5 100644 --- a/source/tutorials/ruby-driver-sessions.txt +++ b/source/tutorials/ruby-driver-sessions.txt @@ -73,6 +73,14 @@ the ``Mongo::Collection::View``: view = client[:artists].find({ :name => 'FKA Twigs' }, session: session) view.count(session: second_session) # will use the second_session + +Unacknowledged Writes +--------------------- + +Unacknowledged writes are only allowed outside the session mechanism; if an explicit session is supplied for an +unacknowledged write, the driver will not send the session id with the operation. Similarly, the driver will not use +an implicit session for an unacknowledged write. + Causal Consistency ------------------ A causally consistent session will let you read your writes and guarantee monotonically increasing From f521e38d851ac84c63240596d0a273dbf8a758f8 Mon Sep 17 00:00:00 2001 From: Emily Date: Thu, 1 Mar 2018 12:05:22 +0100 Subject: [PATCH 132/442] RUBY-1303 Document how to use latest TLS protocols --- source/contribute.txt | 2 +- source/installation.txt | 37 ++++++++++++++++++++++++++++++++++++- 2 files changed, 37 insertions(+), 2 deletions(-) diff --git a/source/contribute.txt b/source/contribute.txt index 2e892342b..19163f4b5 100644 --- a/source/contribute.txt +++ b/source/contribute.txt @@ -42,5 +42,5 @@ Contribute to the MongoDB Ruby Driver The MongoDB Ruby driver source is located at ``_. -For instructions in contributing to the driver, see +For instructions on contributing to the driver, see ``_. diff --git a/source/installation.txt b/source/installation.txt index ccd003a57..8abcf1994 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -18,5 +18,40 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.5.0 + gem install mongo -v 2.5.1 +TLS/SSL and the Ruby driver +--------------------------- + +Industry best practices, and some regulations, require the use of TLS 1.1 or newer. Though no application changes are +required for the Ruby driver to make use of the newest protocols, some operating systems or versions may not provide +an OpenSSL version new enough to support them. + +Users of macOS older than 10.13 (High Sierra) will need to install Ruby from `rvm`_, `homebrew`_, `macports`_, or +another similar source. See `installation information on ruby-lang.org`_ for more options. + +Users of Linux or other non-macOS Unix can check their OpenSSL version like this: + +.. code-block:: sh + + $ openssl version + +If the version number is less than 1.0.1 support for TLS 1.1 or newer is not available. Contact your operating system +vendor for a solution or upgrade to a newer distribution. + +You can check your Ruby interpreter by executing the following command: + +.. code-block:: sh + + ruby -e "require 'net/http'; require 'json'; puts JSON.parse(Net::HTTP.get(URI('https://www.howsmyssl.com/a/check')))['tls_version']" + +You should see "TLS 1.X" where X is >= 1. + +You can read more about TLS versions and their security implications here: + +``_ + +.. _rvm: https://rvm.io/ +.. _homebrew: https://brew.sh/ +.. _macports: https://www.macports.org/ +.. _installation information on ruby-lang.org: https://www.ruby-lang.org/en/documentation/installation From 7693bdfd28f65afa8d53dbd18382e9b3213d96e5 Mon Sep 17 00:00:00 2001 From: Emily Date: Thu, 15 Mar 2018 17:16:24 +0100 Subject: [PATCH 133/442] RUBY-1313 Deprecate maxScan query option --- source/tutorials/ruby-driver-crud-operations.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 4d6dec1b2..88e66883d 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -195,7 +195,7 @@ when querying and their corresponding methods as examples. - Limits the number of returned documents to the provided value. * - ``max_scan(Integer)`` - Sets the maximum number of documents to scan if a full collection scan - would be performed. + would be performed. Deprecated as of MongoDB server version 4.0. * - ``no_cursor_timeout`` - MongoDB automatically closes inactive cursors after a period of 10 minutes. Call this for cursors to remain open indefinitely on the server. From 12a8809b5cfe6990c4b42898ccf60f048ed6e0a4 Mon Sep 17 00:00:00 2001 From: Emily Date: Thu, 15 Mar 2018 17:21:32 +0100 Subject: [PATCH 134/442] RUBY-1314 Deprecate cursor snapshot option --- source/tutorials/ruby-driver-crud-operations.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 88e66883d..c31152ecc 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -218,7 +218,7 @@ when querying and their corresponding methods as examples. * - ``skip(Integer)`` - Skip the provided number of documents in the results. * - ``snapshot`` - - Execute the query in snapshot mode. + - Execute the query in snapshot mode. Deprecated as of MongoDB server version 4.0. * - ``sort(Hash)`` - Specifies sort criteria for the query. From 5b9cee4e8ce366f67a0b6e864f38c527bf0fd348 Mon Sep 17 00:00:00 2001 From: Saghm Rossi Date: Fri, 13 Apr 2018 13:39:34 -0400 Subject: [PATCH 135/442] RUBY-1321 Update documentation to reflect Symbol deprecation --- .../tutorials/ruby-driver-crud-operations.txt | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index c31152ecc..16e1b2e7d 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -247,7 +247,7 @@ Additional Query Operations client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') client[:artists].find.distinct(:name ) - + Tailable Cursors ~~~~~~~~~~~~~~~~ @@ -338,10 +338,10 @@ matches a secondary server with tag set ``{ dc: 'ny', rack: 2, size: 'large' }`` A tag set that is an empty document matches any server, because the empty tag set is a subset of any tag set. This means the default -``tag_sets`` parameter ``[{}]`` matches all servers. +``tag_sets`` parameter ``[{}]`` matches all servers. .. _ruby-driver-updating: - + Updating -------- @@ -460,3 +460,14 @@ Deleting result = artists.delete_many(:label => 'Mute') result.deleted_count # Returns the number deleted. + +A Note about the BSON Symbol type +--------------------------------- + +Because the BSON specification deprecated the BSON symbol type, the `bson` gem +will serialize Ruby symbols into BSON strings when used on its own. However, in +order to maintain backwards compatibility with older datasets, the Ruby driver +overrides this behavior to serialize Ruby symbols as BSON symbols. This is +necessary to be able to specify queries for documents which contain BSON +symbols as fields. Despite this, new documents with symbol type fields should +*not* be stored in the database; instead, use string fields. From ecae272eb54e1b20ef9fc2f38d8e35e568958eec Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Tue, 29 May 2018 13:53:36 -0400 Subject: [PATCH 136/442] Link to documentation sections more prominently --- source/index.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/index.txt b/source/index.txt index 6c4c67bf0..0827170dc 100644 --- a/source/index.txt +++ b/source/index.txt @@ -17,7 +17,8 @@ Get Started ----------- To get started with the Ruby driver, see :doc:`/installation` and -:doc:`/quick-start`. +:doc:`/quick-start`. Continue to :doc:`/ruby-driver-tutorials` +for high level documentation for common operations. BSON ---- From 0b70f6cc15d08be11dec8bdd5525a92832c29f9d Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Tue, 29 May 2018 13:53:50 -0400 Subject: [PATCH 137/442] Minor grammar fixes --- source/tutorials/ruby-driver-change-streams.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 4c5eb047f..1662b153c 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -14,7 +14,7 @@ As of version 3.6 of the MongoDB server, a new ``$changeStream`` pipeline stage framework. The Ruby driver provides an API for receiving notifications for changes to a particular collection using this new pipeline stage. Although you can create a change stream using the pipeline operator and aggregation framework directly, it is recommended to use the driver API described below as the driver resumes the change stream if there is -timeout or network error. +a timeout or a network error. Change streams on the server requires a ``"majority"`` read concern or no read concern. @@ -38,7 +38,7 @@ A change stream is created by calling the ``#watch`` method on a collection: process(doc) -You can also receive the notifications as they are available: +You can also receive the notifications as they become available: .. code-block:: ruby @@ -64,7 +64,7 @@ The change stream can take filters in the aggregation framework pipeline operato Close a Change Stream --------------------- -You can close a Change Stream by calling the ``#close`` method: +You can close a change stream by calling the ``#close`` method: .. code-block:: ruby From 73d5602e35ce7f638913cbfcadc2f82d327f64ba Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 30 May 2018 22:02:55 -0400 Subject: [PATCH 138/442] RUBY-1288 Deprecate MONGODB-CR authentication mechanism --- source/tutorials/ruby-driver-admin-tasks.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/tutorials/ruby-driver-admin-tasks.txt b/source/tutorials/ruby-driver-admin-tasks.txt index 56919a7a5..a7bbe3c44 100644 --- a/source/tutorials/ruby-driver-admin-tasks.txt +++ b/source/tutorials/ruby-driver-admin-tasks.txt @@ -158,6 +158,9 @@ Alternatively, setting the current database and credentials can be done in one s MONGODB-CR Mechanism ```````````````````` +*Deprecated:* MONGODB-CR mechanism is deprecated as of MongoDB version 3.6. +Please use SCRAM authentication instead. + MONGODB-CR was the default authentication mechanism for MongoDB up through version 2.6. The mechanism can be explicitly set with the credentials: From c92b71e2e7e9b653139d68f3c611eeae73cca50b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Fri, 1 Jun 2018 13:05:33 -0400 Subject: [PATCH 139/442] RUBY-1333 Formatting fixes in the change stream doc and line length --- .../tutorials/ruby-driver-change-streams.txt | 22 +++++++++++-------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 1662b153c..713ced96e 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -10,21 +10,24 @@ Change Streams :depth: 1 :class: singlecol -As of version 3.6 of the MongoDB server, a new ``$changeStream`` pipeline stage is supported in the aggregation -framework. The Ruby driver provides an API for receiving notifications for changes to a particular collection using this -new pipeline stage. Although you can create a change stream using the pipeline operator and aggregation framework -directly, it is recommended to use the driver API described below as the driver resumes the change stream if there is +As of version 3.6 of the MongoDB server, a new ``$changeStream`` pipeline stage +is supported in the aggregation framework. The Ruby driver provides an API for +receiving notifications for changes to a particular collection using this +new pipeline stage. Although you can create a change stream using the pipeline +operator and aggregation framework directly, it is recommended to use the +driver API described below as the driver resumes the change stream if there is a timeout or a network error. -Change streams on the server requires a ``"majority"`` read concern or no read concern. +Change streams on the server require a ``"majority"`` read concern or no +read concern. Change streams do not work properly with JRuby because of the issue documented here_. +Namely, JRuby eagerly evaluates ``#next`` on an Enumerator in a background +green thread, therefore calling ``#next`` on the change stream will cause +getMores to be called in a loop in the background. .. _here: https://github.com/jruby/jruby/issues/4212 -Namely, JRuby eagerly evaluates ``#next`` on an Enumerator in a background green thread. -So calling ``#next`` on the change stream will cause getmores to be called in a loop in the background. - Watching for changes on a particular collection ----------------------------------------------- @@ -49,7 +52,8 @@ You can also receive the notifications as they become available: end -The change stream can take filters in the aggregation framework pipeline operator format: +The change stream can take filters in the aggregation framework pipeline +operator format: .. code-block:: ruby From 95d479a1c694508462f86a609e0cfba51935f4f6 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 13 Jun 2018 23:34:14 -0400 Subject: [PATCH 140/442] RUBY-1333 Document current retryable write behavior --- .../tutorials/ruby-driver-create-client.txt | 29 ++++++++++++++++--- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 9f18eda11..677489861 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -557,10 +557,31 @@ When ``#close`` is called on a client by any thread, all connections are closed: Details on Retryable Writes --------------------------- -If the ``retry_writes`` option is set to true, the driver will retry single-statement write operations that fail from -a network error. The driver automatically retries the operation once. - -Most of the write methods you use day-to-day on a collection will be retryable in 3.6. They are: +The driver implements two mechanisms for retrying writes: modern and legacy. +The modern mechanism is used when the driver connects to a MongoDB 3.6+ +replica set or a sharded cluster, and the ``retry_writes`` client option is +set to true. This mechanism retries writes once on network errors +(timeout, connection reset, etc.) as well as transient errors resulting +from servers being unavailable due to initialization, shutdown, replica +set elections and similar events (the corresponding errors are not master, +node is recovering, etc.). Prior to retrying the write the driver attempts +to select a new primary, since the server that was previously used is +likely to no longer be usable. + +The legacy mechanism is used when the driver connects to MongoDB 3.4 or +earlier clusters, or to MongoDB 3.6 or later standalone servers, or when +``retry_writes`` client option is not given. In this mode the driver will +retry write operations once due to transient errors resulting from +servers being unavailable due to initialization, shutdown, replica set +elections and similar events, but will not retry writes on network errors +like timeouts. + +Note that writes are always retried on transient server status-related +errors like "not master" and "node is recovering", regardless of what +``retry_writes`` option is set to. + +The following write methods used in day-to-day operations on collections will +be automatically retried: - ``collection#insert_one`` - ``collection#update_one`` From c47facdc0e594f228d6f262c94a19aba8c88f7ee Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 13 Jun 2018 23:35:13 -0400 Subject: [PATCH 141/442] RUBY-1333 Camelize the heading --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 677489861..fafbd4b5b 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -593,7 +593,7 @@ be automatically retried: - ``collection#bulk_write`` (for all single statement ops, i.e. not for ``update_many`` or ``delete_many``) -TCP keepalive configuration +TCP Keepalive Configuration --------------------------- The driver sets TCP keepalive by default. The following default values are also set if the system From 2d70cd4fc8b30ba090517dd4aaaa2df1ff9972ac Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 13 Jun 2018 23:40:37 -0400 Subject: [PATCH 142/442] RUBY-1333 Change streams are resumed on non-network errors as well --- source/tutorials/ruby-driver-change-streams.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 713ced96e..3f27963b8 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -16,7 +16,7 @@ receiving notifications for changes to a particular collection using this new pipeline stage. Although you can create a change stream using the pipeline operator and aggregation framework directly, it is recommended to use the driver API described below as the driver resumes the change stream if there is -a timeout or a network error. +a timeout, a network error or another type of a resumable error. Change streams on the server require a ``"majority"`` read concern or no read concern. From 1cfe1c2b9f6c6fab85486f9224cd00623948b6a1 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 14 Jun 2018 11:03:16 -0400 Subject: [PATCH 143/442] RUBY-1333 Spelling fix --- source/tutorials/ruby-driver-create-client.txt | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index fafbd4b5b..1e990643e 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -12,9 +12,10 @@ Creating a Client Using ``Mongo::Client`` ----------------------- -To start a Ruby driver connection, create a ``Mongo::Client`` object. + +To connect to a MongoDB cluster, create a ``Mongo::Client`` object. Provide a list of hosts and options or a connection URI to the -``Mongo::Client`` constructor. The client's oselected database +``Mongo::Client`` constructor. The client's selected database defaults to ``admin``. To create a client to a standalone server, provide one host in the From 49d76ebf3960d80944d2b443163d9399cf919ace Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 26 Jun 2018 17:53:41 -0400 Subject: [PATCH 144/442] Start at operation time implementation (#978) * RUBY-1369 Collection change stream integration test * RUBY-1369 try_next implementation * RUBY-1369 Move to spec/integration * RUBY-1369 Got try_next working when there are changes * RUBY-1369 Test try_next with no changes * RUBY-1369 Document try_next * RUBY-1369 Change change stream resume logic more to retry once only * RUBY-1369 Apply current resume behavior to try_next * RUBY-1369 Add docstrings for change_stream_resumable? * RUBY-1369 Document how to restart change streams and when they are resumed by the driver automatically * RUBY-1369 failCommand is 4.0+ only * RUBY-1369 These are now non-resumed * RUBY-1369 Repair the logic yet again. The concept of "retrying once" is apparently much more difficult than it first appears * RUBY-1369 Repair this test again and try another way to not have it be stuck in a getmore loop * RUBY-1369 Use timeout-interrupt to stop tests hanging forever on change stream reads * RUBY-1369 Force a successful getMore prior to failing them. I believe change stream drops documents upon a reset without startAtOperationTime, hence right now we can't reliably test initial getMores failing * RUBY-1369 Need to clear fail points only on tests using them if I want to run others on servers < 4.0 --- .../tutorials/ruby-driver-change-streams.txt | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 3f27963b8..3294a1b9d 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -77,3 +77,24 @@ You can close a change stream by calling the ``#close`` method: doc = stream.to_enum.next process(doc) stream.close + +Resuming a Change Stream +------------------------ + +The driver will automatically retry getMore operations on a change stream +once. Initial aggregation is never retried. In practical terms this means +that, for example: + +- Calling ``collection.watch`` will fail if the cluster does not have +enough available nodes to satisfy the ``"majority"`` read preference; +- Once ``collection.watch`` successfully returns, if the cluster subsequently +experiences an election or loses a node, but heals quickly enough, +change stream reads via ``next`` or ``each`` methods will continue +transparently to the application. + +If the cluster loses enough nodes to not be able to satisfy the ``"majority"`` +read preference and does not heal quickly enough, ``next`` and ``each`` +will raise an error. In these cases the application must track, via the +resume token, which documents from the change stream it has processed and +create a new change stream object via the ``watch`` call, passing an +appropriate ``:resume_after`` argument. From 7357621c56e3e18eacef3d02849ab568e833fa04 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 29 Jun 2018 15:40:41 -0400 Subject: [PATCH 145/442] =?UTF-8?q?The=20driver=20does=20perform=20certifi?= =?UTF-8?q?cate=20validation=20by=20default,=20update=20docs=20to=20?= =?UTF-8?q?=E2=80=A6=20(#980)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * The driver perform certificate validation by default, update docs to reflect this * certification -> certificate --- source/tutorials/ruby-driver-create-client.txt | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 1e990643e..7fbe22d2b 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -228,30 +228,30 @@ Ruby Options - none * - ``:ssl_ca_cert`` - - The file path containing a set of concatenated certification authority certifications used to validate certs + - The file path containing concatenated certificate authority certificates used to validate certs passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. - ``String`` - none * - ``:ssl_ca_cert_string`` - - A string containing a set of concatenated certification authority certifications used to validate certs + - A string containing concatenated certificate authority certificates used to validate certs passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. - ``String`` - none * - ``:ssl_ca_cert_object`` - - An array of OpenSSL::X509::Certificate representing the certification authority certifications used to + - An array of OpenSSL::X509::Certificate representing the certificate authority certificates used to validate certs passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. - ``Array`` - none * - ``:ssl_verify`` - - Whether or not to do peer certification validation. + - Whether to perform peer certificate validation. - ``Boolean`` - - false + - true * - ``:connect_timeout`` - The number of seconds to wait to establish a socket connection From bb169d2cc16811cb2fac119ac7c71d28254de076 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 23 Jul 2018 12:31:42 -0400 Subject: [PATCH 146/442] fix RUBY-1388 (#987) * fix RUBY-1388 * Streamline client construction * try this * recreate cluster upon reconnection to avoid using old mutexes in monitoring * Fix tests * review feedback * Integration test for reconnecting a client * Test monitor thread resurrection --- .../tutorials/ruby-driver-create-client.txt | 74 +++++++++++++++---- 1 file changed, 59 insertions(+), 15 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 7fbe22d2b..01c4e2a23 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -494,6 +494,22 @@ Details on Timeout Options `CRUD specification `_ for details on operations that support this option. + +TCP Keepalive Configuration +--------------------------- + +The driver sets TCP keepalive by default. The following default values are also set if the system +value can be determined and if the driver default value is less than the system value. + +- ``tcp_keepalive_time``: 300 seconds +- ``tcp_keepalive_intvl``: 10 seconds +- ``tcp_keepalive_cnt``: 9 probes + + +Please see the `MongoDB Diagnostics FAQ keepalive section `_ +for instructions on setting these values at the system level. + + Details on Connection Pooling ----------------------------- @@ -555,6 +571,49 @@ When ``#close`` is called on a client by any thread, all connections are closed: client.close + +Use with Forking Servers +------------------------ + +When using the Mongo Ruby driver with a forking web server such as Unicorn, +worker processes should generally each have their own Mongo::Client instances. +This is because: + +1. The background threads remain in the parent process and are not transfered + to the child process. +2. File descriptors like network sockets are shared between parent and + child processes. + +It is easiest to not create any Mongo::Client instances prior to the fork. +If the parent process needs to perform operations on the MongoDB database, +reset the client in an ``after_fork`` handler which is defined in +``unicorn.rb``:: + +.. code-block:: ruby + + after_fork do |server, worker| + $client.close + $client.reconnect + end + +The above code assumes your Mongo::Client instance is stored in the $client +global variable. It also keeps the MongoDB connection in the parent process +around, and this connection will continue to consume resources such as +a connection slot. If the parent process performs one time operation(s) on +MongoDB and does not need its MongoDB connection once workers are forked, +the following code will close the connection in the parent:: + +.. code-block:: ruby + + before_fork do |server, worker| + $client.close + end + + after_fork do |server, worker| + $client.reconnect + end + + Details on Retryable Writes --------------------------- @@ -592,18 +651,3 @@ be automatically retried: - ``collection#find_one_and_replace`` - ``collection#find_one_and_delete`` - ``collection#bulk_write`` (for all single statement ops, i.e. not for ``update_many`` or ``delete_many``) - - -TCP Keepalive Configuration ---------------------------- - -The driver sets TCP keepalive by default. The following default values are also set if the system -value can be determined and if the driver default value is less than the system value. - -- ``tcp_keepalive_time``: 300 seconds -- ``tcp_keepalive_intvl``: 10 seconds -- ``tcp_keepalive_cnt``: 9 probes - - -Please see the `MongoDB Diagnostics FAQ keepalive section `_ -for instructions on setting these values at the system level. From 597477cd44d01656dbf2b0b51ff62c753071675a Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 23 Jul 2018 14:49:18 -0400 Subject: [PATCH 147/442] Documentation formatting fix (#993) * Note that the reconnect pattern is only applicable to driver 2.7.0+ * Fix docs --- source/tutorials/ruby-driver-create-client.txt | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 01c4e2a23..587334340 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -587,7 +587,7 @@ This is because: It is easiest to not create any Mongo::Client instances prior to the fork. If the parent process needs to perform operations on the MongoDB database, reset the client in an ``after_fork`` handler which is defined in -``unicorn.rb``:: +``unicorn.rb``: .. code-block:: ruby @@ -601,7 +601,7 @@ global variable. It also keeps the MongoDB connection in the parent process around, and this connection will continue to consume resources such as a connection slot. If the parent process performs one time operation(s) on MongoDB and does not need its MongoDB connection once workers are forked, -the following code will close the connection in the parent:: +the following code will close the connection in the parent: .. code-block:: ruby @@ -613,6 +613,8 @@ the following code will close the connection in the parent:: $client.reconnect end +*Note:* This pattern should be used with Ruby driver version 2.7.0 or higher. +Previous driver versions did not recreate monitoring threads when reconnecting. Details on Retryable Writes --------------------------- From b3969942ac974398a8712e19a2637f6ba7663be7 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 26 Jul 2018 17:14:41 -0400 Subject: [PATCH 148/442] RUBY-1393 Update support matrix (#998) --- ...by-driver-compatibility-matrix-mongodb.rst | 3 +- source/reference/driver-compatibility.txt | 62 ++++++++++++------- 2 files changed, 42 insertions(+), 23 deletions(-) diff --git a/source/includes/ruby-driver-compatibility-matrix-mongodb.rst b/source/includes/ruby-driver-compatibility-matrix-mongodb.rst index 40003c7ab..bf3d8e572 100644 --- a/source/includes/ruby-driver-compatibility-matrix-mongodb.rst +++ b/source/includes/ruby-driver-compatibility-matrix-mongodb.rst @@ -2,5 +2,4 @@ The following compatibility table specifies the recommended version(s) of the MongoDB Ruby driver for use with a specific version of MongoDB. -The first column lists the driver version(s). - +The first column lists the driver versions. diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index a9edf1ae6..2626a29ca 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -19,22 +19,25 @@ MongoDB Compatibility :class: compatibility * - Ruby Driver - - MongoDB 2.4 - - MongoDB 2.6 - - MongoDB 3.0 - - MongoDB 3.2 - - MongoDB 3.4 + - MongoDB 4.0 - MongoDB 3.6 + - MongoDB 3.4 + - MongoDB 3.2 + - MongoDB 3.0 + - MongoDB 2.6 + - MongoDB 2.4 - * - 2.5 - - + * - 2.6 - |checkmark| - |checkmark| - |checkmark| - |checkmark| - |checkmark| + - |checkmark| + - - * - 2.4 + * - 2.5 + - - |checkmark| - |checkmark| - |checkmark| @@ -42,69 +45,86 @@ MongoDB Compatibility - |checkmark| - - * - 2.3 + * - 2.4 + - + - - |checkmark| - |checkmark| - |checkmark| - |checkmark| + - |checkmark| + + * - 2.3 + - - - - - * - 2.2 - |checkmark| - |checkmark| - |checkmark| - |checkmark| + + * - 2.2 + - - - - - * - 2.0 - |checkmark| - |checkmark| - |checkmark| + - |checkmark| + + * - 2.0 + - - - - - - * - 1.12 - |checkmark| - |checkmark| - |checkmark| + + * - 1.12 - - - - - * - 1.11 + - + - |checkmark| - |checkmark| - |checkmark| + + * - 1.11 + - - - - - - - * - 1.10 - |checkmark| - |checkmark| + + * - 1.10 - - - - + - + - |checkmark| + - |checkmark| * - 1.9 - - |checkmark| - - - - - + - + - |checkmark| * - 1.8 - - |checkmark| - - - - - + - + - |checkmark| .. include:: /includes/older-server-versions-unsupported.rst From 93c862bdb45415fd692c8addbd54e807477ab83f Mon Sep 17 00:00:00 2001 From: Saghm Rossi Date: Thu, 9 Aug 2018 18:21:46 -0400 Subject: [PATCH 149/442] minor: document that the new client creation pattern can be used on 2.6.2 and later --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 587334340..3aeee1999 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -613,7 +613,7 @@ the following code will close the connection in the parent: $client.reconnect end -*Note:* This pattern should be used with Ruby driver version 2.7.0 or higher. +*Note:* This pattern should be used with Ruby driver version 2.6.2 or higher. Previous driver versions did not recreate monitoring threads when reconnecting. Details on Retryable Writes From 56d09522c7d4db0633764740ba7ff3f4975fed84 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 13 Aug 2018 18:17:07 -0400 Subject: [PATCH 150/442] RUBY-1414 Transaction example in Ruby driver tutorial (#1005) * RUBY-1414 Transaction example in Ruby driver tutorial * Reword * The --- source/ruby-driver-tutorials.txt | 1 + source/tutorials/ruby-driver-sessions.txt | 10 ++- source/tutorials/ruby-driver-transactions.txt | 74 +++++++++++++++++++ 3 files changed, 81 insertions(+), 4 deletions(-) create mode 100644 source/tutorials/ruby-driver-transactions.txt diff --git a/source/ruby-driver-tutorials.txt b/source/ruby-driver-tutorials.txt index 19d16b157..c4f743c67 100644 --- a/source/ruby-driver-tutorials.txt +++ b/source/ruby-driver-tutorials.txt @@ -20,6 +20,7 @@ operations available in the Ruby driver. /tutorials/ruby-driver-collection-tasks /tutorials/ruby-driver-projections /tutorials/ruby-driver-sessions + /tutorials/ruby-driver-transactions /tutorials/ruby-driver-change-streams /tutorials/ruby-driver-admin-tasks /tutorials/ruby-driver-indexing diff --git a/source/tutorials/ruby-driver-sessions.txt b/source/tutorials/ruby-driver-sessions.txt index 19849abe5..8a8b1dfcc 100644 --- a/source/tutorials/ruby-driver-sessions.txt +++ b/source/tutorials/ruby-driver-sessions.txt @@ -1,6 +1,8 @@ -============================ -Creating and using Sessions -============================ +.. _sessions: + +======== +Sessions +======== .. default-domain:: mongodb @@ -11,7 +13,7 @@ Creating and using Sessions :class: singlecol -Version 3.6 of the server introduces the concept of logical sessions for clients. +Version 3.6 of the MongoDB server introduces the concept of logical sessions for clients. A session is an abstract concept that represents a set of sequential operations executed by an application that are related in some way. A session object can be create via a ``Mongo::Client`` and passed to operation methods that should be executed in the context of that session. diff --git a/source/tutorials/ruby-driver-transactions.txt b/source/tutorials/ruby-driver-transactions.txt new file mode 100644 index 000000000..b3e1f4354 --- /dev/null +++ b/source/tutorials/ruby-driver-transactions.txt @@ -0,0 +1,74 @@ +============ +Transactions +============ + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + + +Version 4.0 of the MongoDB server introduces +`multi-document transactions `_. +(Updates to multiple fields within a single document are atomic in all +versions of MongoDB.) Ruby driver version 2.6.0 adds support for transactions. + +Using Transactions +------------------ + +In order to start a transaction, the application must have a :ref:`session `. + +A transaction can be started by calling the ``start_transaction`` method on a session: + +.. code-block:: ruby + + session = client.start_session + session.start_transaction + +It is also possible to specify read concern, write concern and read preference +when starting a transaction: + +.. code-block:: ruby + + session = client.start_session + session.start_transaction( + read_concern: {level: :majority}, + write_concern: {w: 3}, + read: {mode: :primary}) + +To persist changes made in a transaction to the database, the transaction +must be explicitly committed. If a session ends with an open transaction, +`the transaction is aborted `_. +A transaction may also be aborted explicitly. + +To commit or abort a transaction, call ``commit_transaction`` or +``abort_transaction`` on the session instance: + +.. code-block:: ruby + + session.commit_transaction + + session.abort_transaction + + +Retrying Commits +---------------- + +The transaction commit `can be retried +`_ +if it fails. Here is the Ruby code to do so: + +.. code-block:: ruby + + begin + session.commit_transaction + rescue Mongo::Error => e + if e.label?(Mongo::Error::UNKNOWN_TRANSACTION_COMMIT_RESULT_LABEL) + retry + else + raise + end + end From 6d09534baee08e768f555d56c745a786989ff155 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Tue, 14 Aug 2018 14:58:57 -0400 Subject: [PATCH 151/442] Formatting fix --- source/tutorials/ruby-driver-change-streams.txt | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 3294a1b9d..1fdc1c0b3 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -86,11 +86,11 @@ once. Initial aggregation is never retried. In practical terms this means that, for example: - Calling ``collection.watch`` will fail if the cluster does not have -enough available nodes to satisfy the ``"majority"`` read preference; + enough available nodes to satisfy the ``"majority"`` read preference. - Once ``collection.watch`` successfully returns, if the cluster subsequently -experiences an election or loses a node, but heals quickly enough, -change stream reads via ``next`` or ``each`` methods will continue -transparently to the application. + experiences an election or loses a node, but heals quickly enough, + change stream reads via ``next`` or ``each`` methods will continue + transparently to the application. If the cluster loses enough nodes to not be able to satisfy the ``"majority"`` read preference and does not heal quickly enough, ``next`` and ``each`` From 023d867094aff99ce306af6dff156cae86c817d7 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 14 Aug 2018 16:44:24 -0400 Subject: [PATCH 152/442] Minor changes to fork guidance following mongoid (#1006) --- source/tutorials/ruby-driver-create-client.txt | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 3aeee1999..36d875dec 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -572,8 +572,8 @@ When ``#close`` is called on a client by any thread, all connections are closed: client.close -Use with Forking Servers ------------------------- +Usage with Forking Servers +-------------------------- When using the Mongo Ruby driver with a forking web server such as Unicorn, worker processes should generally each have their own Mongo::Client instances. @@ -584,7 +584,7 @@ This is because: 2. File descriptors like network sockets are shared between parent and child processes. -It is easiest to not create any Mongo::Client instances prior to the fork. +It is recommended to not create any Mongo::Client instances prior to the fork. If the parent process needs to perform operations on the MongoDB database, reset the client in an ``after_fork`` handler which is defined in ``unicorn.rb``: @@ -616,6 +616,11 @@ the following code will close the connection in the parent: *Note:* This pattern should be used with Ruby driver version 2.6.2 or higher. Previous driver versions did not recreate monitoring threads when reconnecting. +*Note:* If the parent process performs operations on the Mongo client and +does not close it, the parent process will continue consuming a connection slot +in the cluster and will continue monitoring the cluster for as long as the +parent remains alive. + Details on Retryable Writes --------------------------- From f24d93ccf7a7827b84f4f53c9086cac33423cda1 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Tue, 14 Aug 2018 17:12:52 -0400 Subject: [PATCH 153/442] jruby test suite changes --- ...y-driver-compatibility-matrix-language.rst | 2 +- source/reference/driver-compatibility.txt | 69 ++++++++++++++----- 2 files changed, 52 insertions(+), 19 deletions(-) diff --git a/source/includes/ruby-driver-compatibility-matrix-language.rst b/source/includes/ruby-driver-compatibility-matrix-language.rst index a94fe600a..a19182866 100644 --- a/source/includes/ruby-driver-compatibility-matrix-language.rst +++ b/source/includes/ruby-driver-compatibility-matrix-language.rst @@ -2,5 +2,5 @@ The following compatibility table specifies the recommended version(s) of the MongoDB Ruby driver for use with a specific version of Ruby. -The first column lists the driver version(s). +The first column lists the driver versions. diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 2626a29ca..5adb1ccce 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -141,15 +141,31 @@ Language Compatibility :class: compatibility * - Ruby Driver - - Ruby 1.8.7 - - Ruby 1.9 - - Ruby 2.0 - - Ruby 2.1 - - Ruby 2.2 - - Ruby 2.3 + - Ruby 2.5 - Ruby 2.4 + - Ruby 2.3 + - Ruby 2.2 + - Ruby 2.1 + - Ruby 2.0 + - Ruby 1.9 + - Ruby 1.8.7 + - JRuby 9.2 + - JRuby 9.1 - JRuby + * - 2.6 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + - |checkmark| + - |checkmark| + - + * - 2.5 - - |checkmark| @@ -159,8 +175,12 @@ Language Compatibility - |checkmark| - |checkmark| - + - + - + - |checkmark| * - 2.4 + - - - |checkmark| - |checkmark| @@ -168,9 +188,12 @@ Language Compatibility - |checkmark| - |checkmark| - + - + - - |checkmark| * - 2.3 + - - - |checkmark| - |checkmark| @@ -178,9 +201,12 @@ Language Compatibility - |checkmark| - |checkmark| - + - + - - |checkmark| * - 2.2 + - - - |checkmark| - |checkmark| @@ -188,9 +214,12 @@ Language Compatibility - |checkmark| - |checkmark| - + - + - - |checkmark| * - 2.1 + - - - |checkmark| - |checkmark| @@ -198,9 +227,12 @@ Language Compatibility - |checkmark| - |checkmark| - + - + - - |checkmark| * - 2.0 + - - - |checkmark| - |checkmark| @@ -208,44 +240,45 @@ Language Compatibility - |checkmark| - |checkmark| - + - + - - |checkmark| - * - 1.9 - 1.12 + * - 1.12 - 1.9 + - + - + - + - - |checkmark| - |checkmark| - |checkmark| - |checkmark| - - - - - |checkmark| * - 1.8 - - |checkmark| - - |checkmark| - - |checkmark| + - - - - - - |checkmark| - - * - 1.7 - |checkmark| - |checkmark| - - + - |checkmark| + + * - 1.7 - 1.6 - - - - - |checkmark| - - * - 1.6 - - |checkmark| - - |checkmark| - - - + - |checkmark| + - |checkmark| - - - |checkmark| From 26f66d46b2b46535f7410faed73757df2279fbe1 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 15 Aug 2018 10:50:41 -0400 Subject: [PATCH 154/442] ci From 5c5ce7f7d3edf217bd4eafaa628098a4334e1298 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 14 Sep 2018 16:04:40 -0400 Subject: [PATCH 155/442] Separate authentication and monitoring docs & reorganize admin tasks (#1028) * Separate authentication and monitoring docs * Move collections and preferences documentation * Rename admin tasks to database tasks --- source/ruby-driver-tutorials.txt | 4 +- ...sks.txt => ruby-driver-authentication.txt} | 202 +----------------- .../ruby-driver-collection-tasks.txt | 65 ++---- .../tutorials/ruby-driver-create-client.txt | 38 ++++ .../tutorials/ruby-driver-crud-operations.txt | 25 ++- .../tutorials/ruby-driver-database-tasks.txt | 54 +++++ source/tutorials/ruby-driver-monitoring.txt | 86 ++++++++ 7 files changed, 226 insertions(+), 248 deletions(-) rename source/tutorials/{ruby-driver-admin-tasks.txt => ruby-driver-authentication.txt} (50%) create mode 100644 source/tutorials/ruby-driver-database-tasks.txt create mode 100644 source/tutorials/ruby-driver-monitoring.txt diff --git a/source/ruby-driver-tutorials.txt b/source/ruby-driver-tutorials.txt index c4f743c67..70b017cd9 100644 --- a/source/ruby-driver-tutorials.txt +++ b/source/ruby-driver-tutorials.txt @@ -16,13 +16,15 @@ operations available in the Ruby driver. :titlesonly: /tutorials/ruby-driver-create-client + /tutorials/ruby-driver-authentication /tutorials/ruby-driver-crud-operations /tutorials/ruby-driver-collection-tasks /tutorials/ruby-driver-projections /tutorials/ruby-driver-sessions /tutorials/ruby-driver-transactions /tutorials/ruby-driver-change-streams - /tutorials/ruby-driver-admin-tasks + /tutorials/ruby-driver-database-tasks + /tutorials/ruby-driver-monitoring /tutorials/ruby-driver-indexing /tutorials/ruby-driver-collations /tutorials/ruby-driver-aggregation diff --git a/source/tutorials/ruby-driver-admin-tasks.txt b/source/tutorials/ruby-driver-authentication.txt similarity index 50% rename from source/tutorials/ruby-driver-admin-tasks.txt rename to source/tutorials/ruby-driver-authentication.txt index a7bbe3c44..ab20a3a21 100644 --- a/source/tutorials/ruby-driver-admin-tasks.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -1,5 +1,5 @@ ============== -Administration +Authentication ============== .. default-domain:: mongodb @@ -10,90 +10,6 @@ Administration :depth: 1 :class: singlecol -Databases ---------- - -The driver provides various helpers on database objects for executing -commands, getting collection lists, and administrative tasks. - -List Collections -```````````````` - -To get a list of collections or collection names for a database, use -``collections`` and ``collection_names``, respectively. - -.. code-block:: ruby - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - database = client.database - - database.collections # Returns an array of Collection objects. - database.collection_names # Returns an array of collection names as strings. - -To execute any command on the database, use the ``command`` method. - -.. code-block:: ruby - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - database = client.database - - result = database.command(:ismaster => 1) - result.first # Returns the BSON::Document returned from the server. - -Drop Database -````````````` - -To drop a database, use the ``drop`` method. - -.. code-block:: ruby - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - client.database.drop - -Collections ------------ - -The driver provides some helpers for administrative tasks with -collections. - -To create a collection with options (such as creating a capped collection), -pass the options when getting the collection from the client, then call -``create``. - -.. code-block:: ruby - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - artists = client[:artists, :capped => true, :size => 1024] - artists.create - artists.capped? # Returns true. - -Drop Collection -``````````````` - -To drop a collection, call ``drop`` on the collection object. - -.. code-block:: ruby - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - artists = client[:artists] - artists.drop - -Changing Read/Write Preferences -``````````````````````````````` - -To change the default read preference or write concern for specific operations, -use the ``with`` method on the collection. - -.. code-block:: ruby - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - artists = client[:artists] - artists.with(:read => { :mode => :primary_preferred }).find.to_a - artists.with(:write => { :w => :3 }).insert_one( { :name => 'Depeche Mode' } ) - -Authentication --------------- - MongoDB supports a variety of :manual:`authentication mechanisms `. @@ -258,119 +174,3 @@ authentication, see the :manual:`manual auth_mech: :gssapi, user: 'test', password: '123' ) - -Logger ------- - -You can either use the default global driver logger or set your own. To set your own: - -.. code-block:: ruby - - Mongo::Logger.logger = other_logger - -See the `Ruby Logger documentation `_ -for more information on the default logger API and available levels. - -Changing the Logger Level -````````````````````````` - -To change the logger level: - -.. code-block:: ruby - - Mongo::Logger.logger.level = Logger::WARN - -For more control, a logger can be passed to a client for per-client control over logging. - -.. code-block:: ruby - - my_logger = Logger.new($stdout) - Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :logger => my_logger ) - -Truncation -`````````` - -The default logging truncates logs at 250 characters by default. To turn this off pass an -option to the client instance. - -.. code-block:: ruby - - Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :truncate_logs => false ) - -Monitoring ----------- - -All user-initiated commands that are sent to the server publish events that can be -subscribed to for fine grained information. The monitoring API publishes a guaranteed -start event for each command, then either a succeeded or failed event. A subscriber -must implement 3 methods: ``started``, ``succeeded``, and ``failed``, each which takes -a single parameter for the event. An example is the default logging subscriber included -in the driver: - -.. code-block:: ruby - - module Mongo - class Monitoring - class CommandLogSubscriber - include Loggable - - attr_reader :options - - LOG_STRING_LIMIT = 250 - - def initialize(options = {}) - @options = options - end - - def started(event) - log_debug("#{prefix(event)} | STARTED | #{format_command(event.command)}") - end - - def succeeded(event) - log_debug("#{prefix(event)} | SUCCEEDED | #{event.duration}s") - end - - def failed(event) - log_debug("#{prefix(event)} | FAILED | #{event.message} | #{event.duration}s") - end - - private - - def format_command(args) - begin - truncating? ? truncate(args) : args.inspect - rescue Exception - '' - end - end - - def prefix(event) - "#{event.address.to_s} | #{event.database_name}.#{event.command_name}" - end - - def truncate(command) - ((s = command.inspect).length > LOG_STRING_LIMIT) ? "#{s[0..LOG_STRING_LIMIT]}..." : s - end - - def truncating? - @truncating ||= (options[:truncate_logs] != false) - end - end - end - end - -To register a custom subscriber, you can do so globally for -all clients or on a per-client basis: - -.. code-block:: ruby - - Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::COMMAND, my_subscriber) - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test' ) - client.subscribe( Mongo::Monitoring::COMMAND, my_subscriber ) - -To turn off monitoring, set the client monitoring option to ``false``: - -.. code-block:: ruby - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :monitoring => false ) diff --git a/source/tutorials/ruby-driver-collection-tasks.txt b/source/tutorials/ruby-driver-collection-tasks.txt index 8bde7ce98..f1ecc6f0d 100644 --- a/source/tutorials/ruby-driver-collection-tasks.txt +++ b/source/tutorials/ruby-driver-collection-tasks.txt @@ -17,8 +17,8 @@ document in that collection. You can also explicitly create a collection with various options, such as setting the maximum size or the documentation validation rules. -Capped Collection -````````````````` +Capped Collections +`````````````````` Capped collections have maximum size or document counts that prevent them from growing beyond maximum thresholds. All capped collections must @@ -32,7 +32,9 @@ the ``capped: true`` option along with a ``size`` in bytes. .. code-block:: ruby client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - client[:artists, capped: true, size: 10000].create + collection = client[:artists, capped: true, size: 10000] + collection.create + collection.capped? # => true Convert an Existing Collection to Capped ```````````````````````````````````````` @@ -42,9 +44,9 @@ the ``convertToCapped`` command. .. code-block:: ruby - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') db = client.database - db.command({ 'convertToCapped' => 'contacts', 'size' => 8192 }) + db.command({ 'convertToCapped' => 'artists', 'size' => 10000 }) Document Validation @@ -102,52 +104,27 @@ documents must contain an ``age`` field which is a number. } }) +Listing Collections +``````````````````` -Change Streams -``````````````` - -As of version 3.6 of the MongoDB server, a new ```$changeStream`` pipeline stage is supported in the aggregation framework. -Specifying this stage first in an aggregation pipeline allows users to request that notifications are sent for all -changes to a particular collection. The helper method available for change streams on a collection object is preferred -to running a raw aggregation with a $changeStream stage, for the purpose of supporting resumability. - -The change stream will attempt to resume one time upon a network error or a server error indicating that a failover is -taking place. - -To start receiving change notifications, use the method ``#watch`` on a collection: - -.. code-block:: ruby - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') - stream = client[:test].watch - -Each document can be retrieved and processed by your application as in the following code example: +Use ``collections`` or ``collection_names`` methods on a database +objects to list collections: .. code-block:: ruby - enum = stream.to_enum - begin - while true - change = enum.next - # do something - sleep(1) - end - rescue => e - # we know for sure it's unrecoverable - log_error(e) - end - -The change documents will expose a resume token at the key '_id'. This token can be used to specify a logical starting -point for a new a ChangeStream: + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + database = client.database -.. code-block:: ruby + database.collections # Returns an array of Collection objects. + database.collection_names # Returns an array of collection names as strings. - stream = client[:test].watch([], resume_token: token) +Dropping Collections +```````````````````` -A subset of aggregation pipeline stages are available to a change stream and can be passed into the ``#watch`` method: +To drop a collection, call ``drop`` on the collection object. .. code-block:: ruby - stream = client[:test].watch([ { '$match' => { 'operationType' => 'update' } }]) - - + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] + artists.drop diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 36d875dec..edb04bc20 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -658,3 +658,41 @@ be automatically retried: - ``collection#find_one_and_replace`` - ``collection#find_one_and_delete`` - ``collection#bulk_write`` (for all single statement ops, i.e. not for ``update_many`` or ``delete_many``) + +Logging +------- + +You can either use the default global driver logger or set your own. To set your own: + +.. code-block:: ruby + + Mongo::Logger.logger = other_logger + +See the `Ruby Logger documentation `_ +for more information on the default logger API and available levels. + +Changing the Logger Level +````````````````````````` + +To change the logger level: + +.. code-block:: ruby + + Mongo::Logger.logger.level = Logger::WARN + +For more control, a logger can be passed to a client for per-client control over logging. + +.. code-block:: ruby + + my_logger = Logger.new($stdout) + Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :logger => my_logger ) + +Truncation +`````````` + +The default logging truncates logs at 250 characters by default. To turn this off pass an +option to the client instance. + +.. code-block:: ruby + + Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :truncate_logs => false ) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 16e1b2e7d..fdc555dfc 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -299,8 +299,8 @@ refer to the `Server Selection documentation, available on GitHub `_. -The read preference is set as an option on the client or passed an -option when a command is run on a database. +Read preference can be set as an option on the client or passed an +option when a command is run on a database: .. code-block:: ruby @@ -314,6 +314,15 @@ option when a command is run on a database. client.database.command( { collstats: 'test' }, read: { mode: secondary, tag_sets: [ { 'dc' => 'nyc' } ] } ) +Read preference can also be set for specific operations on a collection +using the ``with`` method: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] + artists.with(:read => { :mode => :primary_preferred }).find.to_a + Mode ~~~~ @@ -435,6 +444,18 @@ the modification occurs. ) doc # Return the document after the update. +Write Concern +````````````` + +Write concern can be given for specific operations on a collection +using the ``with`` method, similarly to read preference: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] + artists.with(:write => { :w => :3 }).insert_one( { :name => 'Depeche Mode' } ) + Deleting -------- diff --git a/source/tutorials/ruby-driver-database-tasks.txt b/source/tutorials/ruby-driver-database-tasks.txt new file mode 100644 index 000000000..f7c8b90f4 --- /dev/null +++ b/source/tutorials/ruby-driver-database-tasks.txt @@ -0,0 +1,54 @@ +========= +Databases +========= + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +Databases +--------- + +The driver provides various helpers on database objects for executing +commands, getting collection lists, and administrative tasks. + +List Collections +```````````````` + +To get a list of collections or collection names for a database, use +``collections`` and ``collection_names``, respectively. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + database = client.database + + database.collections # Returns an array of Collection objects. + database.collection_names # Returns an array of collection names as strings. + +Arbitrary Comands +````````````````` + +To execute any command on the database, use the ``command`` method. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + database = client.database + + result = database.command(:ismaster => 1) + result.first # Returns the BSON::Document returned from the server. + +Drop Database +````````````` + +To drop a database, use the ``drop`` method. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client.database.drop diff --git a/source/tutorials/ruby-driver-monitoring.txt b/source/tutorials/ruby-driver-monitoring.txt new file mode 100644 index 000000000..e4a1d29a9 --- /dev/null +++ b/source/tutorials/ruby-driver-monitoring.txt @@ -0,0 +1,86 @@ +========== +Monitoring +========== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +All user-initiated commands that are sent to the server publish events that can be +subscribed to for fine grained information. The monitoring API publishes a guaranteed +start event for each command, then either a succeeded or failed event. A subscriber +must implement 3 methods: ``started``, ``succeeded``, and ``failed``, each which takes +a single parameter for the event. An example is the default logging subscriber included +in the driver: + +.. code-block:: ruby + + module Mongo + class Monitoring + class CommandLogSubscriber + include Loggable + + attr_reader :options + + LOG_STRING_LIMIT = 250 + + def initialize(options = {}) + @options = options + end + + def started(event) + log_debug("#{prefix(event)} | STARTED | #{format_command(event.command)}") + end + + def succeeded(event) + log_debug("#{prefix(event)} | SUCCEEDED | #{event.duration}s") + end + + def failed(event) + log_debug("#{prefix(event)} | FAILED | #{event.message} | #{event.duration}s") + end + + private + + def format_command(args) + begin + truncating? ? truncate(args) : args.inspect + rescue Exception + '' + end + end + + def prefix(event) + "#{event.address.to_s} | #{event.database_name}.#{event.command_name}" + end + + def truncate(command) + ((s = command.inspect).length > LOG_STRING_LIMIT) ? "#{s[0..LOG_STRING_LIMIT]}..." : s + end + + def truncating? + @truncating ||= (options[:truncate_logs] != false) + end + end + end + end + +To register a custom subscriber, you can do so globally for +all clients or on a per-client basis: + +.. code-block:: ruby + + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::COMMAND, my_subscriber) + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test' ) + client.subscribe( Mongo::Monitoring::COMMAND, my_subscriber ) + +To turn off monitoring, set the client monitoring option to ``false``: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :monitoring => false ) From ff331358a2f6d77955efb61c44ddbd613f743222 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Sat, 15 Sep 2018 00:18:34 -0400 Subject: [PATCH 156/442] Minor docs changes --- source/tutorials/ruby-driver-sessions.txt | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-sessions.txt b/source/tutorials/ruby-driver-sessions.txt index 8a8b1dfcc..df72ef7f4 100644 --- a/source/tutorials/ruby-driver-sessions.txt +++ b/source/tutorials/ruby-driver-sessions.txt @@ -15,10 +15,10 @@ Sessions Version 3.6 of the MongoDB server introduces the concept of logical sessions for clients. A session is an abstract concept that represents a set of sequential operations executed -by an application that are related in some way. A session object can be create via a ``Mongo::Client`` +by an application that are related in some way. A session object can be created via a ``Mongo::Client`` and passed to operation methods that should be executed in the context of that session. -Please note that sessions are not thread safe. They can only be used by one thread at a time. +Please note that session objects are not thread safe. They must only be used by one thread at a time. Creating a session from a ``Mongo::Client`` ------------------------------------------- @@ -38,8 +38,8 @@ session options. An error will be thrown if the driver is connected to a deployment that does not support sessions and the ``start_session`` method is called. -Note that server sessions are discarded server-side if not used for a certain period of time. That said, -be aware that if the application calls ``#start_session`` on a client and waits more than 1 minute to use +Note that server sessions are discarded server-side if not used for a certain period of time. +Be aware that if the application calls ``#start_session`` on a client and waits more than 1 minute to use the session, it risks getting errors due to the session going stale before it is used. From 2b7bc09ec5700b406023fa6d18b9d08f9a5141ca Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Fri, 14 Sep 2018 00:54:55 -0400 Subject: [PATCH 157/442] Combine and fix change stream docs --- .../tutorials/ruby-driver-change-streams.txt | 51 +++++++++++++++---- 1 file changed, 41 insertions(+), 10 deletions(-) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 1fdc1c0b3..8125508c7 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -11,12 +11,17 @@ Change Streams :class: singlecol As of version 3.6 of the MongoDB server, a new ``$changeStream`` pipeline stage -is supported in the aggregation framework. The Ruby driver provides an API for +is supported in the aggregation framework. Specifying this stage first in an +aggregation pipeline allows users to request that notifications are sent for all +changes to a particular collection. + +The Ruby driver provides an API for receiving notifications for changes to a particular collection using this new pipeline stage. Although you can create a change stream using the pipeline operator and aggregation framework directly, it is recommended to use the -driver API described below as the driver resumes the change stream if there is -a timeout, a network error or another type of a resumable error. +driver API described below as the driver resumes the change stream one time +if there is a timeout, a network error, a server error indicating that a +failover is taking place or another type of a resumable error. Change streams on the server require a ``"majority"`` read concern or no read concern. @@ -28,13 +33,15 @@ getMores to be called in a loop in the background. .. _here: https://github.com/jruby/jruby/issues/4212 -Watching for changes on a particular collection +Watching for Changes on a Particular Collection ----------------------------------------------- A change stream is created by calling the ``#watch`` method on a collection: .. code-block:: ruby + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + collection = client[:test] stream = collection.watch collection.insert_one(a: 1) doc = stream.to_enum.next @@ -45,12 +52,16 @@ You can also receive the notifications as they become available: .. code-block:: ruby - stream = client[:test].watch + stream = collection.watch enum = stream.to_enum while doc = enum.next process(doc) end +The ``next`` method blocks and polls the cluster until a change is available. +If there is a non-resumable error, ``next`` will raise an exception. +See Resuming a Change Stream section below for an example that reads +changes from a collection indefinitely. The change stream can take filters in the aggregation framework pipeline operator format: @@ -72,10 +83,6 @@ You can close a change stream by calling the ``#close`` method: .. code-block:: ruby - stream = collection.watch - collection.insert_one(a: 1) - doc = stream.to_enum.next - process(doc) stream.close Resuming a Change Stream @@ -97,4 +104,28 @@ read preference and does not heal quickly enough, ``next`` and ``each`` will raise an error. In these cases the application must track, via the resume token, which documents from the change stream it has processed and create a new change stream object via the ``watch`` call, passing an -appropriate ``:resume_after`` argument. +appropriate ``:resume_after`` argument. The resume token is exposed +at the key ``_id`` in each change document. + +.. code-block:: ruby + + token = doc._id + stream = collection.watch([], resume_after: token) + +To watch a collection indefinitely, retrying on all MongoDB errors: + +.. code-block:: ruby + + token = nil + while true + begin + stream = collection.watch([], resume_after: token) + enum = stream.to_enum + while doc = enum.next + process(doc) + token = doc['_id'] + end + rescue Mongo::Error + sleep 1 + end + end From fafc3e29c9061f583fc2165b2edd193e4b9cb81d Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Fri, 14 Sep 2018 01:03:32 -0400 Subject: [PATCH 158/442] Document database and cluster change streams --- .../tutorials/ruby-driver-change-streams.txt | 59 ++++++++++++++++--- 1 file changed, 50 insertions(+), 9 deletions(-) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 8125508c7..5fb5582fa 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -13,10 +13,12 @@ Change Streams As of version 3.6 of the MongoDB server, a new ``$changeStream`` pipeline stage is supported in the aggregation framework. Specifying this stage first in an aggregation pipeline allows users to request that notifications are sent for all -changes to a particular collection. +changes to a particular collection. As of MongoDB 4.0, change streams are +supported on databases and clusters in addition to collections. The Ruby driver provides an API for -receiving notifications for changes to a particular collection using this +receiving notifications for changes to a particular collection, database +or cluster using this new pipeline stage. Although you can create a change stream using the pipeline operator and aggregation framework directly, it is recommended to use the driver API described below as the driver resumes the change stream one time @@ -33,10 +35,11 @@ getMores to be called in a loop in the background. .. _here: https://github.com/jruby/jruby/issues/4212 -Watching for Changes on a Particular Collection ------------------------------------------------ +Watching for Changes on a Collection +------------------------------------ -A change stream is created by calling the ``#watch`` method on a collection: +A collection change stream is created by calling the ``#watch`` method on a +collection: .. code-block:: ruby @@ -76,15 +79,53 @@ operator format: process(doc) end -Close a Change Stream ---------------------- +Watching for Changes on a Database +---------------------------------- -You can close a change stream by calling the ``#close`` method: +A database change stream notifies on changes on any collection within the +database as well as database-wide events, such as the database being dropped. + +A database change stream is created by calling the ``#watch`` method on a +database object: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + database = client.database + stream = database.watch + client[:test].insert_one(a: 1) + doc = stream.to_enum.next + process(doc) + + +Watching for Changes on a Cluster +--------------------------------- + +A cluster change stream notifies on changes on any collection, any database +within the cluster as well as cluster-wide events. + +A cluster change stream is created by calling the ``#watch`` method on a +client object (not the cluster object): + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') + stream = client.watch + client[:test].insert_one(a: 1) + doc = stream.to_enum.next + process(doc) + + +Closing a Change Stream +----------------------- + +You can close a change stream by calling its ``#close`` method: .. code-block:: ruby stream.close + Resuming a Change Stream ------------------------ @@ -109,7 +150,7 @@ at the key ``_id`` in each change document. .. code-block:: ruby - token = doc._id + token = doc['_id'] stream = collection.watch([], resume_after: token) To watch a collection indefinitely, retrying on all MongoDB errors: From bade076b8c01ad4e25c3e37764c1c15323fb174c Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Tue, 18 Sep 2018 13:55:51 -0400 Subject: [PATCH 159/442] Link to change streams --- source/tutorials/ruby-driver-change-streams.txt | 2 ++ source/tutorials/ruby-driver-collection-tasks.txt | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 5fb5582fa..bae4df63b 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -1,3 +1,5 @@ +.. _change-streams: + ============== Change Streams ============== diff --git a/source/tutorials/ruby-driver-collection-tasks.txt b/source/tutorials/ruby-driver-collection-tasks.txt index f1ecc6f0d..02c311b02 100644 --- a/source/tutorials/ruby-driver-collection-tasks.txt +++ b/source/tutorials/ruby-driver-collection-tasks.txt @@ -128,3 +128,8 @@ To drop a collection, call ``drop`` on the collection object. client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') artists = client[:artists] artists.drop + +Change Streams +`````````````` + +See :ref:`Change Streams `. From 11f27c58c53e0c0d24989f597f111d3c8cfd603c Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 24 Sep 2018 12:50:38 -0400 Subject: [PATCH 160/442] RUBY-1470 Enable SNI on jruby where possible (#1037) --- source/reference/driver-compatibility.txt | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 5adb1ccce..3c3f147ba 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -283,4 +283,15 @@ Language Compatibility - - |checkmark| +Atlas Compatibility +~~~~~~~~~~~~~~~~~~~ + +`Driver version 2.6.1 `_ +or higher is recommended when using MongoDB Atlas, as this version has +significant SSL performance improvements. + +When running on JRuby and connecting to Atlas Free Tier, +`driver version 2.6.3 `_ +or higher and Java 8 or higher are required. + .. include:: /includes/unicode-checkmark.rst From 8bae0538ee5aee81b5b80091421cb7b72be94ff4 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 24 Sep 2018 14:21:26 -0400 Subject: [PATCH 161/442] RUBY-1417 Implement SDAM monitoring heartbeat events (#1039) * RUBY-1417 Implement SDAM monitoring heartbeat events * Fix ssl tests --- source/tutorials/ruby-driver-monitoring.txt | 167 ++++++++++++++------ 1 file changed, 122 insertions(+), 45 deletions(-) diff --git a/source/tutorials/ruby-driver-monitoring.txt b/source/tutorials/ruby-driver-monitoring.txt index e4a1d29a9..a1df1b941 100644 --- a/source/tutorials/ruby-driver-monitoring.txt +++ b/source/tutorials/ruby-driver-monitoring.txt @@ -10,63 +10,65 @@ Monitoring :depth: 1 :class: singlecol -All user-initiated commands that are sent to the server publish events that can be -subscribed to for fine grained information. The monitoring API publishes a guaranteed -start event for each command, then either a succeeded or failed event. A subscriber -must implement 3 methods: ``started``, ``succeeded``, and ``failed``, each which takes -a single parameter for the event. An example is the default logging subscriber included -in the driver: +The driver allows the application to be notified when certain events happen. +These events are organized into the following categories: -.. code-block:: ruby - - module Mongo - class Monitoring - class CommandLogSubscriber - include Loggable +- Command monitoring; +- Topology lifecycle; +- Server lifecycle; +- Server heartbeats. - attr_reader :options +Topology and server events are part of Server Discovery and Monitoring (SDAM). - LOG_STRING_LIMIT = 250 +Command Monitoring +------------------ - def initialize(options = {}) - @options = options - end +All user-initiated commands that are sent to the server publish events that +can be subscribed to for fine grained information. The monitoring API +publishes a guaranteed start event for each command, then either a succeeded +or a failed event. A subscriber must implement 3 methods: ``started``, +``succeeded``, and ``failed``, each which takes a single parameter for +the event. The following is an example logging subscriber based on a +logging subscriber used internally by the driver: - def started(event) - log_debug("#{prefix(event)} | STARTED | #{format_command(event.command)}") - end +.. code-block:: ruby - def succeeded(event) - log_debug("#{prefix(event)} | SUCCEEDED | #{event.duration}s") - end + class CommandLogSubscriber + include Mongo::Loggable - def failed(event) - log_debug("#{prefix(event)} | FAILED | #{event.message} | #{event.duration}s") - end + def started(event) + log_debug("#{prefix(event)} | STARTED | #{format_command(event.command)}") + end - private + def succeeded(event) + log_debug("#{prefix(event)} | SUCCEEDED | #{event.duration}s") + end - def format_command(args) - begin - truncating? ? truncate(args) : args.inspect - rescue Exception - '' - end - end + def failed(event) + log_debug("#{prefix(event)} | FAILED | #{event.message} | #{event.duration}s") + end - def prefix(event) - "#{event.address.to_s} | #{event.database_name}.#{event.command_name}" - end + private - def truncate(command) - ((s = command.inspect).length > LOG_STRING_LIMIT) ? "#{s[0..LOG_STRING_LIMIT]}..." : s - end + def logger + Mongo::Logger.logger + end - def truncating? - @truncating ||= (options[:truncate_logs] != false) - end + def format_command(args) + begin + args.inspect + rescue Exception + '' end end + + def format_message(message) + format("COMMAND | %s".freeze, message) + end + + def prefix(event) + "#{event.address.to_s} | #{event.database_name}.#{event.command_name}" + end end To register a custom subscriber, you can do so globally for @@ -74,10 +76,85 @@ all clients or on a per-client basis: .. code-block:: ruby - Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::COMMAND, my_subscriber) + subscriber = CommandLogSubscriber.new + + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::COMMAND, subscriber) client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test' ) - client.subscribe( Mongo::Monitoring::COMMAND, my_subscriber ) + client.subscribe( Mongo::Monitoring::COMMAND, subscriber ) + +Sample output: + + D, [2018-09-23T13:47:31.258020 #4692] DEBUG -- : COMMAND | 127.0.0.1:27027 | test.ismaster | STARTED | {"ismaster"=>1, "$readPreference"=>{"mode"=>"primary"}, "lsid"=>{"id"=>}} + D, [2018-09-23T13:47:31.259145 #4692] DEBUG -- : COMMAND | 127.0.0.1:27027 | test.ismaster | SUCCEEDED | 0.000791175s + + +Server Heartbeats +----------------- + +The application can be notified of each server heartbeat by subscribing +to SERVER_HEARTBEAT topic. A server heartbeat listener must implement +three methods: `started`, `succeeded` and `failed`. Each heartbeat invokes +the `started` method on the listener, and then either `succeeded` or `failed` +method depending on the outcome of the heartbeat. + +All heartbeat events contain the address of the server that the heartbeat +was sent to. Succeeded and failed events contain the round trip time for +the ismaster command. Failed event also contains the exception instance that +was raised during ismaster command execution. Please review the API +documentation for ServerHeartbeatStarted, ServerHeartbeatSucceeded and +ServerHeartbeatFailed for event attribute details. + +The following is an example logging heartbeat event subscriber: + +.. code-block:: ruby + + class HeartbeatLogSubscriber + include Mongo::Loggable + + def started(event) + log_debug("#{event.address} | STARTED") + end + + def succeeded(event) + log_debug("#{event.address} | SUCCEEDED | #{event.duration}s") + end + + def failed(event) + log_debug("#{event.address} | FAILED | #{event.error.class}: #{event.error.message} | #{event.duration}s") + end + + private + + def logger + Mongo::Logger.logger + end + + def format_message(message) + format("HEARTBEAT | %s".freeze, message) + end + end + +Similarly to command events, the application can subscribe to heartbeat +events globally or for a specific client: + +.. code-block:: ruby + + subscriber = HeartbeatLogSubscriber.new + + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::SERVER_HEARTBEAT, subscriber) + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test' ) + client.subscribe( Mongo::Monitoring::SERVER_HEARTBEAT, subscriber ) + +Sample output: + +D, [2018-09-23T13:44:10.707018 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | STARTED +D, [2018-09-23T13:44:10.707778 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | SUCCEEDED | 0.000772381s + + +Disabling Monitoring +-------------------- To turn off monitoring, set the client monitoring option to ``false``: From 861b57be9fdde89c2c05912722bd0e8637d886a9 Mon Sep 17 00:00:00 2001 From: Saghm Rossi Date: Fri, 5 Oct 2018 14:47:37 -0400 Subject: [PATCH 162/442] minor: add note about JRuby compatibility with the mongo_kerberos gem (#1062) --- source/reference/driver-compatibility.txt | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 3c3f147ba..42eecf686 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -295,3 +295,12 @@ When running on JRuby and connecting to Atlas Free Tier, or higher and Java 8 or higher are required. .. include:: /includes/unicode-checkmark.rst + +JRuby Kerberos +~~~~~~~~~~~~~~ + +If the ``mongo_kerberos`` gem is used for Kerberos authentication with JRuby, the the JVM system +property "sun.security.jgss.native" to will be set to "true" in order to facilitate the use of +the system cache of TGTs (e.g. TGTs obtained with ``kinit``). Any other use of the JGSS library +will also be affected by this setting, meaning any TGTs in the system cache will be available for +obtaining Kerberos credentials as well. From 4eef60da5d6f92a758b7e1fe816f685a3ce56fd4 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 9 Oct 2018 14:58:38 -0400 Subject: [PATCH 163/442] RUBY-1415 Add SDAM monitoring example to ruby driver tutorial (#1066) --- source/tutorials/ruby-driver-monitoring.txt | 183 +++++++++++++++++++- 1 file changed, 181 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-monitoring.txt b/source/tutorials/ruby-driver-monitoring.txt index a1df1b941..950487913 100644 --- a/source/tutorials/ruby-driver-monitoring.txt +++ b/source/tutorials/ruby-driver-monitoring.txt @@ -85,10 +85,187 @@ all clients or on a per-client basis: Sample output: +.. code-block:: none + D, [2018-09-23T13:47:31.258020 #4692] DEBUG -- : COMMAND | 127.0.0.1:27027 | test.ismaster | STARTED | {"ismaster"=>1, "$readPreference"=>{"mode"=>"primary"}, "lsid"=>{"id"=>}} D, [2018-09-23T13:47:31.259145 #4692] DEBUG -- : COMMAND | 127.0.0.1:27027 | test.ismaster | SUCCEEDED | 0.000791175s +Server Discovery And Monitoring +------------------------------- + +The Ruby driver implements `Server Discovery And Monitoring (SDAM) specification +`_. +and makes the following events available to the application: + +- Topology opening +- Server opening +- Server description changed +- Topology changed +- Server closed +- Topology closed +- Heartbeat events (covered below in a separate section) + +For all events other than the heartbeat events, the ``succeeded`` method +will be called on each event subscriber with the event as the sole argument. +Available data for events varies, therefore to log the events a separate +class is needed for each event type. A simple SDAM logging subscriber +can look like the following: + +.. code-block:: ruby + + class SDAMLogSubscriber + include Mongo::Loggable + + def succeeded(event) + log_debug(format_event(event)) + end + + private + + def logger + Mongo::Logger.logger + end + + def format_message(message) + format("SDAM | %s".freeze, message) + end + end + + class TopologyOpeningLogSubscriber < SDAMLogSubscriber + private + + def format_event(event) + "Topology type '#{event.topology.display_name}' initializing." + end + end + + class ServerOpeningLogSubscriber < SDAMLogSubscriber + private + + def format_event(event) + "Server #{event.address} initializing." + end + end + + class ServerDescriptionChangedLogSubscriber < SDAMLogSubscriber + private + + def format_event(event) + "Server description for #{event.address} changed from " + + "'#{event.previous_description.server_type}' to '#{event.new_description.server_type}'." + end + end + + class TopologyChangedLogSubscriber < SDAMLogSubscriber + private + + def format_event(event) + if event.previous_topology != event.new_topology + "Topology type '#{event.previous_topology.display_name}' changed to " + + "type '#{event.new_topology.display_name}'." + else + "There was a change in the members of the '#{event.new_topology.display_name}' " + + "topology." + end + end + end + + class ServerClosedLogSubscriber < SDAMLogSubscriber + private + + def format_event(event) + "Server #{event.address} connection closed." + end + end + + class TopologyClosedLogSubscriber < SDAMLogSubscriber + private + + def format_event(event) + "Topology type '#{event.topology.display_name}' closed." + end + end + +To subscribe to SDAM events globally: + +.. code-block:: ruby + + topology_opening_subscriber = TopologyOpeningLogSubscriber.new + server_opening_subscriber = ServerOpeningLogSubscriber.new + server_description_changed_subscriber = ServerDescriptionChangedLogSubscriber.new + topology_changed_subscriber = TopologyChangedLogSubscriber.new + server_closed_subscriber = ServerClosedLogSubscriber.new + topology_closed_subscriber = TopologyClosedLogSubscriber.new + + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::TOPOLOGY_OPENING, + topology_opening_subscriber) + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::SERVER_OPENING, + server_opening_subscriber) + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::SERVER_DESCRIPTION_CHANGED, + server_description_changed_subscriber) + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::TOPOLOGY_CHANGED, + topology_changed_subscriber) + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::SERVER_CLOSED, + server_closed_subscriber) + Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::TOPOLOGY_CLOSED, + topology_closed_subscriber) + +Subscribing to SDAM events for a single client is a little more involved +since the events may be published during the client's construction: + +.. code-block:: ruby + + topology_opening_subscriber = TopologyOpeningLogSubscriber.new + server_opening_subscriber = ServerOpeningLogSubscriber.new + server_description_changed_subscriber = ServerDescriptionChangedLogSubscriber.new + topology_changed_subscriber = TopologyChangedLogSubscriber.new + server_closed_subscriber = ServerClosedLogSubscriber.new + topology_closed_subscriber = TopologyClosedLogSubscriber.new + + sdam_proc = Proc.new do |client| + client.subscribe(Mongo::Monitoring::TOPOLOGY_OPENING, + topology_opening_subscriber) + client.subscribe(Mongo::Monitoring::SERVER_OPENING, + server_opening_subscriber) + client.subscribe(Mongo::Monitoring::SERVER_DESCRIPTION_CHANGED, + server_description_changed_subscriber) + client.subscribe(Mongo::Monitoring::TOPOLOGY_CHANGED, + topology_changed_subscriber) + client.subscribe(Mongo::Monitoring::SERVER_CLOSED, + server_closed_subscriber) + client.subscribe(Mongo::Monitoring::TOPOLOGY_CLOSED, + topology_closed_subscriber) + end + + client = Mongo::Client.new(['127.0.0.1:27017'], database: 'test', + sdam_proc: sdam_proc) + +Sample output: + +.. code-block:: none + + D, [2018-10-09T13:58:03.489461 #22079] DEBUG -- : SDAM | Topology type 'Unknown' initializing. + D, [2018-10-09T13:58:03.489699 #22079] DEBUG -- : SDAM | Server 127.0.0.1:27100 initializing. + D, [2018-10-09T13:58:03.491384 #22079] DEBUG -- : SDAM | Server description for 127.0.0.1:27100 changed from 'unknown' to 'unknown'. + D, [2018-10-09T13:58:03.491642 #22079] DEBUG -- : SDAM | Server localhost:27100 initializing. + D, [2018-10-09T13:58:03.493199 #22079] DEBUG -- : SDAM | Server description for localhost:27100 changed from 'unknown' to 'primary'. + D, [2018-10-09T13:58:03.493473 #22079] DEBUG -- : SDAM | Server localhost:27101 initializing. + D, [2018-10-09T13:58:03.494874 #22079] DEBUG -- : SDAM | Server description for localhost:27101 changed from 'unknown' to 'secondary'. + D, [2018-10-09T13:58:03.495139 #22079] DEBUG -- : SDAM | Server localhost:27102 initializing. + D, [2018-10-09T13:58:03.496504 #22079] DEBUG -- : SDAM | Server description for localhost:27102 changed from 'unknown' to 'secondary'. + D, [2018-10-09T13:58:03.496777 #22079] DEBUG -- : SDAM | Topology type 'Unknown' changed to type 'ReplicaSetNoPrimary'. + D, [2018-10-09T13:58:03.497306 #22079] DEBUG -- : SDAM | Server 127.0.0.1:27100 connection closed. + D, [2018-10-09T13:58:03.497606 #22079] DEBUG -- : SDAM | Topology type 'ReplicaSetNoPrimary' changed to type 'ReplicaSetWithPrimary'. + + # client.close + + D, [2018-10-09T13:58:05.342057 #22079] DEBUG -- : SDAM | Server localhost:27100 connection closed. + D, [2018-10-09T13:58:05.342299 #22079] DEBUG -- : SDAM | Server localhost:27101 connection closed. + D, [2018-10-09T13:58:05.342565 #22079] DEBUG -- : SDAM | Server localhost:27102 connection closed. + D, [2018-10-09T13:58:05.342693 #22079] DEBUG -- : SDAM | Topology type 'ReplicaSetWithPrimary' closed. + + Server Heartbeats ----------------- @@ -149,8 +326,10 @@ events globally or for a specific client: Sample output: -D, [2018-09-23T13:44:10.707018 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | STARTED -D, [2018-09-23T13:44:10.707778 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | SUCCEEDED | 0.000772381s +.. code-block:: none + + D, [2018-09-23T13:44:10.707018 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | STARTED + D, [2018-09-23T13:44:10.707778 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | SUCCEEDED | 0.000772381s Disabling Monitoring From 94b5c0a848114601f3f0937f254c975d9291ac0f Mon Sep 17 00:00:00 2001 From: Zach McCormick Date: Mon, 26 Nov 2018 13:28:18 -0500 Subject: [PATCH 164/442] RUBY-1529 Adding ActiveSupport::TimeWithZone patch (#104) * Adding ActiveSupport::TimeWithZone patch * Decouple bson-ruby from activesupport * Document ActiveSupport usage * Undepend shared examples on AS * Verify serialization correctness * Test AS in evergreen * Byebug is 2.0+ only --- docs/tutorials/bson-v4.txt | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 78916678a..6a4d3d2f7 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -34,7 +34,20 @@ To install the gem with bundler, include the following in your Gemfile: gem 'bson', '~> 4.0' -The BSON gem is compatible with MRI >= 1.9.3, JRuby 1.7.x, and Rubinius 2.5.x +The BSON gem is compatible with MRI >= 1.9.3, JRuby >= 9.1, and Rubinius 2.5.x. + +Use With ActiveSupport +---------------------- + +Serialization for ActiveSupport-defined classes, such as TimeWithZone, is +not loaded by default to avoid a hard dependency of BSON on ActiveSupport. +When using BSON in an application that also uses ActiveSupport, the +ActiveSupport-related code must be explicitly required: + +.. code-block:: ruby + + require 'bson' + require 'bson/active_support' BSON Serialization ------------------ From e71611f16f09781911862b90722d366664e5b8b6 Mon Sep 17 00:00:00 2001 From: Saghm Rossi Date: Thu, 13 Dec 2018 16:28:40 -0500 Subject: [PATCH 165/442] RUBY-1559 Implement uri option spec tests (#1170) --- .../tutorials/ruby-driver-create-client.txt | 156 ++++++++++++------ 1 file changed, 101 insertions(+), 55 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index edb04bc20..e405177d4 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -76,12 +76,12 @@ Since the URI options are required in camel case, which is not the Ruby standard following table shows the option in the URI and its corresponding option if passed to the constructor in Ruby. -.. note:: - +.. note:: + The options passed directly should be symbols. The options are explained in detail in the :manual:`Connection URI reference -`. +`. .. note:: Options that are set in **milliseconds** in the URI are @@ -97,8 +97,29 @@ URI Options Conversions * - URI Option - Ruby Option - * - replicaSet=String - - ``:replica_set => String`` + * - appName=String + - ``:app_name => String`` + + * - authMechanism=String + - ``:auth_mech => Symbol`` + + * - authMechanismProperties=Strings + - ``{ :auth_mech_properties => { :service_realm => String, :canonicalize_host_name => true|false, :service_name => String } }`` + + Specified as comma-separated key:value pairs, e.g. ``"SERVICE_REALM:foo,CANONICALIZE_HOSTNAME:TRUE"``. + + * - authSource=String + - ``:auth_source => String`` + + * - compressors=Strings + - ``:compressors => Array`` + + Specified as a comma-separated list. Note that the Ruby driver only supports zlib + compression; however, other drivers may support snappy. For maximum compatibility with + drivers, specify ``"snappy,zlib"``; if compatibility with other drivers is not a concern, + specify ``"zlib".`` Compression is not enabled by default and when using MongoDB 4.0 and + earlier, so zlib compression must be manually enabled on the server in order for the Ruby + driver to compress wire protocol data. * - connect=String - ``:connect => Symbol`` @@ -109,56 +130,81 @@ URI Options Conversions * - connectTimeoutMS=Integer - ``:connect_timeout => Float`` - * - socketTimeoutMS=Integer - - ``:socket_timeout => Float`` + * - fsync=Boolean + - ``{ :write => { :fsync => true|false }}`` - * - serverSelectionTimeoutMS=Integer - - ``:server_selection_timeout => Float`` + * - heartbeatFrequencyMS=Integer + - ``:heartbeat_frequency => Float`` + + * - journal=Boolean + - ``{ :write => { :j => true|false }}`` * - localThresholdMS=Integer - ``:local_threshold => Float`` + * - maxIdleTimeMS=Integer + - ``:max_idle_time => Float`` + * - maxPoolSize=Integer - ``:max_pool_size => Integer`` * - minPoolSize=Integer - ``:min_pool_size => Integer`` - * - waitQueueTimeoutMS=Integer - - ``:wait_queue_timeout => Float`` + * - readPreference=String + - ``{ :read => { :mode => Symbol }}`` - * - w=Integer|String - - ``{ :write => { :w => Integer|String }}`` + * - readPreferenceTags=Strings + - ``{ :read => { :tag_sets => Array }}`` - * - wtimeoutMS=Integer - - ``{ :write => { :wtimeout => Float }}`` + Each instance of the readPreferenceTags field is a comma-separated key:value pair which will appear in the :tag_sets array in the order they are specified. For instance, ``"readPreferenceTags=dc:ny,rack:1&readPreferenceTags=dc:ny"`` will be converted to ``[ { 'dc' => 'ny', 'rack' => '1' }, { 'dc' => 'ny' }]``. - * - journal=Boolean - - ``{ :write => { :j => true|false }}`` + * - replicaSet=String + - ``:replica_set => String`` - * - fsync=Boolean - - ``{ :write => { :fsync => true|false }}`` + * - serverSelectionTimeoutMS=Integer + - ``:server_selection_timeout => Float`` - * - readPreference=String - - ``{ :read => { :mode => Symbol }}`` + * - socketTimeoutMS=Integer + - ``:socket_timeout => Float`` - * - readPreferenceTags=Strings - - ``{ :read => { :tag_sets => Array }}`` + * - tls=Boolean + - ``:ssl => boolean`` - * - authSource=String - - ``:auth_source => String`` + * - tlsAllowInvalidCertificates=Boolean + - ``:ssl_verify => boolean`` - * - authMechanism=String - - ``:auth_mech => Symbol`` + Because ``tlsAllowInvalidCertificates`` uses ``true`` to signify that verification + should be disabled and ``ssl_verify`` uses ``false`` to signify that verification should be + disabled, the boolean is inverted before being used to set ``ssl_verify``. - * - authMechanismProperties=Strings - - ``{ :auth_mech_properties => { :service_realm => String, :canonicalize_host_name => true|false, :service_name => String } }`` + * - tlsCAFile=String + - ``:ssl_ca_cert => String`` - * - appName=String - - ``:app_name => String`` + * - tlsClientCertFile=String + - ``:ssl_cert => String`` - * - compressors=Strings - - ``:compressors => Array`` + * - tlsClientKeyFile=String + - ``:ssl_key => String`` + + * - tlsClientKeyPassword=String + - ``:ssl_key_pass_phrase => String`` + + * - tlsInsecure=Boolean + - ``:ssl_verify => boolean`` + + Because tlsInsecure uses ``true`` to signify that verification should be disabled and + ``ssl_verify`` uses ``false`` to signify that verification should be disabled, the boolean + is inverted before being used to set ``ssl_verify``. + + * - w=Integer|String + - ``{ :write => { :w => Integer|String }}`` + + * - waitQueueTimeoutMS=Integer + - ``:wait_queue_timeout => Float`` + + * - wtimeoutMS=Integer + - ``{ :write => { :wtimeout => Float }}`` * - zlibCompressionLevel=Integer - ``:zlib_compression_level => Integer`` @@ -178,12 +224,12 @@ Ruby Options * - ``:replica_set`` - When connecting to a replica set, this is the name of the set to - filter servers by. + filter servers by. - ``String`` - none * - ``:ssl`` - - Tell the client to connect to the servers via SSL. + - Tell the client to connect to the servers via SSL. - ``Boolean`` - false @@ -255,30 +301,30 @@ Ruby Options * - ``:connect_timeout`` - The number of seconds to wait to establish a socket connection - before raising an exception. + before raising an exception. - ``Float`` - 10 seconds * - ``:socket_timeout`` - The number of seconds to wait for an operation to execute on a - socket before raising an exception. + socket before raising an exception. - ``Float`` - 5 seconds * - ``:max_pool_size`` - - The maximum size of the connection pool for each server. + - The maximum size of the connection pool for each server. - ``Integer`` - 5 * - ``:min_pool_size`` - The minimum number of connections in the connection pool for each - server. + server. - ``Integer`` - 1 * - ``:wait_queue_timeout`` - The number of seconds to wait for a connection in the connection - pool to become available. + pool to become available. - ``Float`` - 1 @@ -291,22 +337,22 @@ Ruby Options { :write => { :w => 2 } } - ``Hash`` - ``{ :w => 1 }`` - + * - ``:read`` - Specifies the read preference mode and tag sets for selecting servers as a ``Hash``. Keys in the hash are ``:mode`` and ``:tag_sets``. .. code-block:: ruby - { :read => - { :mode => :secondary, - :tag_sets => [ "berlin" ] - } + { :read => + { :mode => :secondary, + :tag_sets => [ "berlin" ] + } } - ``Hash`` - ``{ :mode => :primary }`` - + * - ``:auth_source`` - Specifies the authentication source. - ``String`` @@ -316,7 +362,7 @@ Ruby Options * - ``:auth_mech`` - Specifies the authenticaion mechanism to use. Can be one of: - ``:mongodb_cr``, ``:mongodb_x509``, ``:plain``, ``:scram``. + ``:mongodb_cr``, ``:mongodb_x509``, ``:plain``, ``:scram``. - ``Symbol`` - MongoDB 3.0 and later: ``:scram`` if user credentials are supplied but an ``:auth_mech`` is not. 2.6 and earlier: @@ -343,7 +389,7 @@ Ruby Options ``:replica_set`` or ``:sharded``. - ``Symbol`` - none - + * - ``:heartbeat_frequency`` - The number of seconds for the server monitors to refresh server states asynchronously. @@ -437,9 +483,9 @@ Details on Timeout Options ``connect_timeout`` On initialization of a connection to a server, this setting is the - number of seconds to wait to connect before raising an exception. + number of seconds to wait to connect before raising an exception. This timeout is also used when monitor threads ping their servers. - The default is 10 seconds. See the `socket timeout for monitoring + The default is 10 seconds. See the `socket timeout for monitoring specification `_ for further explanation. @@ -457,7 +503,7 @@ Details on Timeout Options The number of seconds to wait for the driver to find an appropriate server to which an operation can be sent before raising an exception. Defaults to 30. It should take the speed of :manual:`elections` - during a failover into account. See the + during a failover into account. See the `serverSelectionTimeoutMS specification `_ for further information. @@ -466,10 +512,10 @@ Details on Timeout Options The maximum latency in seconds between the nearest server and the servers that can be considered available to send an operation to. Defaults to 0.015. -.. note:: +.. note:: This is not the latency window between the driver and a server, but rather the latency between the nearest server and other servers. See - `the localThresholdMS specification + `the localThresholdMS specification `_. ``wait_queue_timeout`` @@ -483,7 +529,7 @@ Details on Timeout Options ``min_pool_size`` Minimum number of connections in the connection pool for each server. - Increase this number to create connections when the pool is + Increase this number to create connections when the pool is initialized and to reduce the overhead of creating new connections later on. Defaults to 1. @@ -669,7 +715,7 @@ You can either use the default global driver logger or set your own. To set your Mongo::Logger.logger = other_logger See the `Ruby Logger documentation `_ -for more information on the default logger API and available levels. +for more information on the default logger API and available levels. Changing the Logger Level ````````````````````````` From c24c8368baf72718af37fef2a58c5b7007dc8856 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Mon, 3 Dec 2018 17:10:56 -0500 Subject: [PATCH 166/442] Reword doc text in third person --- source/tutorials/ruby-driver-create-client.txt | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index e405177d4..e99c2c73b 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -559,18 +559,19 @@ for instructions on setting these values at the system level. Details on Connection Pooling ----------------------------- -``Mongo::Client`` instances have a connection pool per server in your MongoDB topology. The pool opens connections on -demand to support the number of concurrent MongoDB operations your application requires. There is no thread-affinity +``Mongo::Client`` instances have a connection pool per server that the client +is connected to. The pool creates connections on demand to support concurrent +MongoDB operations issued by the application. There is no thread-affinity for connections. -The client instance opens one additional connection per server in your MongoDB topology for monitoring the server's -state. +The client instance opens one additional connection per known server +for monitoring the server's state. The size of each connection pool is capped at ``max_pool_size``, which defaults to 5. When a thread in your application begins an operation on MongoDB, it tries to retrieve a connection from the pool to send that operation on. If there are some connections available in the pool, it checks out a connection from the pool and uses it for the operation. If there are no connections available and the size of the pool is less than the ``max_pool_size``, a new connection -will be created. If all connections are in use and the pool has reached its maximum, the thread waits for a +will be created. If all connections are in use and the pool has reached its maximum size, the thread waits for a connection to be returned to the pool by another thread. The number of seconds the thread will wait is configurable. This setting, called ``wait_queue_timeout``, is defined in From 50d512521954dfc7c6af56f4b8c60c93faf8c1e2 Mon Sep 17 00:00:00 2001 From: Saghm Rossi Date: Fri, 18 Jan 2019 16:35:41 -0500 Subject: [PATCH 167/442] RUBY-1608 Add tlsAllowInvalidHostnames URI option (#1206) --- .../tutorials/ruby-driver-create-client.txt | 32 ++++++++++++++++--- 1 file changed, 28 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index e99c2c73b..8c338a4f2 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -172,11 +172,20 @@ URI Options Conversions - ``:ssl => boolean`` * - tlsAllowInvalidCertificates=Boolean - - ``:ssl_verify => boolean`` + - ``:ssl_verify_certificate => boolean`` Because ``tlsAllowInvalidCertificates`` uses ``true`` to signify that verification - should be disabled and ``ssl_verify`` uses ``false`` to signify that verification should be - disabled, the boolean is inverted before being used to set ``ssl_verify``. + should be disabled and ``ssl_verify_certificate`` uses ``false`` to signify that + verification should be disabled, the boolean is inverted before being used to set + ``ssl_verify_certificate``. + + * - tlsAllowInvalidHostnames=Boolean + - ``:ssl_verify_hostname => boolean`` + + Because ``tlsAllowInvalidHostnames`` uses ``true`` to signify that verification + should be disabled and ``ssl_verify_hostname`` uses ``false`` to signify that + verification should be disabled, the boolean is inverted before being used to set + ``ssl_verify_hostname``. * - tlsCAFile=String - ``:ssl_ca_cert => String`` @@ -295,7 +304,22 @@ Ruby Options - none * - ``:ssl_verify`` - - Whether to perform peer certificate validation. + - Whether to perform peer certificate validation and hostname verification. Note that + the decision of whether to validate certificates will be overriden if :ssl_verify_certificate + is set, and the decision of whether to validate hostnames will be overridden if + :ssl_verify_hostname is set. + - ``Boolean`` + - true + + * - ``:ssl_verify_certificate`` + - Whether to perform peer certificate validation. This setting overrides :ssl_verify with + respect to whether certificate validition is performed. + - ``Boolean`` + - true + + * - ``:ssl_verify_hostname`` + - Whether to perform peer hostname validation. This setting overrides :ssl_verify with + respect to whether hostname validition is performed. - ``Boolean`` - true From 58d8567a0afcc78a13235f0e25b2e5bf8a51ba1e Mon Sep 17 00:00:00 2001 From: Saghm Rossi Date: Fri, 18 Jan 2019 18:51:40 -0500 Subject: [PATCH 168/442] RUBY-1559 Update URI option spec tests and fix comparisons (#1213) --- source/tutorials/ruby-driver-create-client.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 8c338a4f2..366aa62d1 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -190,13 +190,13 @@ URI Options Conversions * - tlsCAFile=String - ``:ssl_ca_cert => String`` - * - tlsClientCertFile=String + * - tlsCertificateKeyFile=String - ``:ssl_cert => String`` * - tlsClientKeyFile=String - ``:ssl_key => String`` - * - tlsClientKeyPassword=String + * - tlsCertificateKeyFilePassword=String - ``:ssl_key_pass_phrase => String`` * - tlsInsecure=Boolean From 7495ecba078e2bc8e6d9671a46487eb341082187 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Sun, 20 Jan 2019 19:00:56 -0500 Subject: [PATCH 169/442] Fix RUBY-1464 Time offset ignored in aggregation queries when using ActiveSupport::TimeWithZone (#1214) * Fix RUBY-1464 Time offset ignored in aggregation queries when using ActiveSupport::TimeWithZone * Pass AS env var --- source/reference/driver-compatibility.txt | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 42eecf686..fd883cb36 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -130,8 +130,8 @@ MongoDB Compatibility .. _reference-compatibility-language-ruby: -Language Compatibility -~~~~~~~~~~~~~~~~~~~~~~ +Ruby Compatibility +~~~~~~~~~~~~~~~~~~ .. include:: /includes/ruby-driver-compatibility-matrix-language.rst @@ -283,6 +283,23 @@ Language Compatibility - - |checkmark| +Rails/ActiveSupport Compatibility +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Ruby driver does not depend on ActiveSupport. However, when an +application uses ActiveSupport or Ruby on Rails, +it must load the driver's ActiveSupport +compatibility code for behavior like time serialization to be correct: + +.. code-block:: ruby + + require 'mongo' + require 'mongo/active_support' + +Applications using Mongoid 7.0.3 or newer +do not need to explicitly load the driver's ActiveSupport code, since Mongoid +automatically does so. + Atlas Compatibility ~~~~~~~~~~~~~~~~~~~ From 51e5fe312c72cc2882c7891ec967f399dd6de21e Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 22 Jan 2019 12:58:41 -0500 Subject: [PATCH 170/442] RUBY-1663 Add ruby 2.6 to CI configurations (#1215) --- source/reference/driver-compatibility.txt | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index fd883cb36..d8d9bc729 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -141,6 +141,7 @@ Ruby Compatibility :class: compatibility * - Ruby Driver + - Ruby 2.6 - Ruby 2.5 - Ruby 2.4 - Ruby 2.3 @@ -161,12 +162,14 @@ Ruby Compatibility - |checkmark| - |checkmark| - |checkmark| + - |checkmark| - - |checkmark| - |checkmark| - * - 2.5 + - - - |checkmark| - |checkmark| @@ -180,6 +183,7 @@ Ruby Compatibility - |checkmark| * - 2.4 + - - - - |checkmark| @@ -193,6 +197,7 @@ Ruby Compatibility - |checkmark| * - 2.3 + - - - - |checkmark| @@ -206,6 +211,7 @@ Ruby Compatibility - |checkmark| * - 2.2 + - - - - |checkmark| @@ -219,6 +225,7 @@ Ruby Compatibility - |checkmark| * - 2.1 + - - - - |checkmark| @@ -232,6 +239,7 @@ Ruby Compatibility - |checkmark| * - 2.0 + - - - - |checkmark| @@ -249,6 +257,7 @@ Ruby Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -263,6 +272,7 @@ Ruby Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -277,6 +287,7 @@ Ruby Compatibility - - - + - - |checkmark| - |checkmark| - From 356e71ffac8e370fb4e4222fbbb7d5200cebb0bf Mon Sep 17 00:00:00 2001 From: Saghm Rossi Date: Tue, 22 Jan 2019 18:02:26 -0500 Subject: [PATCH 171/442] RUBY-1676 Add missing client option documentation (#1220) --- .../tutorials/ruby-driver-create-client.txt | 378 +++++++++--------- 1 file changed, 196 insertions(+), 182 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 366aa62d1..e20636621 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -124,9 +124,6 @@ URI Options Conversions * - connect=String - ``:connect => Symbol`` - * - ssl=Boolean - - ``:ssl => true|false`` - * - connectTimeoutMS=Integer - ``:connect_timeout => Float`` @@ -145,12 +142,18 @@ URI Options Conversions * - maxIdleTimeMS=Integer - ``:max_idle_time => Float`` + * - maxStalenessSeconds=Integer + - ``:max_staleness => Float`` + * - maxPoolSize=Integer - ``:max_pool_size => Integer`` * - minPoolSize=Integer - ``:min_pool_size => Integer`` + * - readConcernLevel=String + - ``:read_concern => Hash`` + * - readPreference=String - ``{ :read => { :mode => Symbol }}`` @@ -162,12 +165,18 @@ URI Options Conversions * - replicaSet=String - ``:replica_set => String`` + * - retryWrites=Boolean + - ``:retry_writes => boolean`` + * - serverSelectionTimeoutMS=Integer - ``:server_selection_timeout => Float`` * - socketTimeoutMS=Integer - ``:socket_timeout => Float`` + * - ssl=Boolean + - ``:ssl => true|false`` + * - tls=Boolean - ``:ssl => boolean`` @@ -193,7 +202,7 @@ URI Options Conversions * - tlsCertificateKeyFile=String - ``:ssl_cert => String`` - * - tlsClientKeyFile=String + * - tlsCertificateKeyFile=String - ``:ssl_key => String`` * - tlsCertificateKeyFilePassword=String @@ -231,136 +240,113 @@ Ruby Options - Type - Default - * - ``:replica_set`` - - When connecting to a replica set, this is the name of the set to - filter servers by. - - ``String`` - - none - - * - ``:ssl`` - - Tell the client to connect to the servers via SSL. - - ``Boolean`` - - false - - * - ``:ssl_cert`` - - The certificate file path used to identify the connection against MongoDB. This option, if present, - takes precedence over the values of :ssl_cert_string and :ssl_cert_object. + * - ``:app_name`` + - Application name that is printed to the mongod logs upon establishing a connection + in server versions >= 3.4. - ``String`` - none - * - ``:ssl_cert_string`` - - A string containing the PEM-encoded certificate used to identify the connection against MongoDB. - This option, if present, takes precedence over the value of :ssl_cert_object. - - ``String`` - - none + * - ``:auth_mech`` + - Specifies the authenticaion mechanism to use. Can be one of: + ``:mongodb_cr``, ``:mongodb_x509``, ``:plain``, ``:scram``. + - ``Symbol`` + - MongoDB 3.0 and later: ``:scram`` if user credentials + are supplied but an ``:auth_mech`` is not. 2.6 and earlier: + ``:mongodb_cr`` - * - ``:ssl_cert_object`` - - The OpenSSL::X509::Certificate used to identify the connection against MongoDB. - - ``OpenSSL::X509::Certificate`` + * - ``:auth_mech_properties`` + - Provides additional authentication mechanism properties. + - ``Hash`` - none - * - ``:ssl_key`` - - The private keyfile used to identify the connection against MongoDB. Note that even if the key is stored in - the same file as the certificate, both need to be explicitly specified. This option, if present, takes - precedence over the values of :ssl_key_string and :ssl_key_object. + * - ``:auth_source`` + - Specifies the authentication source. - ``String`` - - none + - For MongoDB 2.6 and later: **admin** if credentials are + supplied, otherwise the current database - * - ``:ssl_key_string`` - - A string containing the PEM-encoded private key used to identify the connection against MongoDB. - This parameter, if present, takes precedence over the value of option :ssl_key_object. - - ``String`` + * - ``:compressors`` + - A list of potential compressors to use, in order of preference. The driver chooses the first + compressor that is also supported by the server. Currently the driver only supports 'zlib'. + - ``Array`` - none - * - ``:ssl_key_object`` - - The private key used to identify the connection against MongoDB. - - ``OpenSSL::PKey`` + * - ``:connect`` + - Overrides the auto-discovery feature of the driver and forces the cluster + topology to a specific type. Choices: ``:direct``, + ``:replica_set`` or ``:sharded``. + - ``Symbol`` - none - * - ``:ssl_key_pass_phrase`` - - A passphrase for the private key. - - ``String`` - - none + * - ``:connect_timeout`` + - The number of seconds to wait to establish a socket connection + before raising an exception. + - ``Float`` + - 10 seconds - * - ``:ssl_ca_cert`` - - The file path containing concatenated certificate authority certificates used to validate certs - passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object - (in order of priority) is required for :ssl_verify. + * - ``:database`` + - The name of the database to connect to. - ``String`` - - none + - admin - * - ``:ssl_ca_cert_string`` - - A string containing concatenated certificate authority certificates used to validate certs - passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object - (in order of priority) is required for :ssl_verify. - - ``String`` - - none + * - ``:heartbeat_frequency`` + - The number of seconds for the server monitors to refresh + server states asynchronously. + - ``Float`` + - 10 - * - ``:ssl_ca_cert_object`` - - An array of OpenSSL::X509::Certificate representing the certificate authority certificates used to - validate certs passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or - :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. - - ``Array`` + * - ``:id_generator`` + - A custom object to generate ids for documents. Must respond to #generate. + - ``Object`` - none - * - ``:ssl_verify`` - - Whether to perform peer certificate validation and hostname verification. Note that - the decision of whether to validate certificates will be overriden if :ssl_verify_certificate - is set, and the decision of whether to validate hostnames will be overridden if - :ssl_verify_hostname is set. - - ``Boolean`` - - true - - * - ``:ssl_verify_certificate`` - - Whether to perform peer certificate validation. This setting overrides :ssl_verify with - respect to whether certificate validition is performed. - - ``Boolean`` - - true - - * - ``:ssl_verify_hostname`` - - Whether to perform peer hostname validation. This setting overrides :ssl_verify with - respect to whether hostname validition is performed. - - ``Boolean`` - - true - - * - ``:connect_timeout`` - - The number of seconds to wait to establish a socket connection - before raising an exception. + * - ``:local_threshold`` + - Specifies the maximum latency in seconds between the nearest + server and the servers that can be available for selection to operate on. - ``Float`` - - 10 seconds + - 0.015 - * - ``:socket_timeout`` - - The number of seconds to wait for an operation to execute on a - socket before raising an exception. - - ``Float`` - - 5 seconds + * - ``:logger`` + - A custom logger. + - ``Object`` + - ``Logger`` + + * - ``:max_idle_time`` + - The maximum seconds a socket can remain idle since it has been checked in to the pool. + - ``Integer`` + - none * - ``:max_pool_size`` - The maximum size of the connection pool for each server. - ``Integer`` - 5 + * - ``:max_read_retries`` + - The maximum number of read retries on mongos query failures. + - ``Integer`` + - 1 + * - ``:min_pool_size`` - The minimum number of connections in the connection pool for each server. - ``Integer`` - 1 - * - ``:wait_queue_timeout`` - - The number of seconds to wait for a connection in the connection - pool to become available. - - ``Float`` - - 1 - - * - ``:write`` - - Specifies write concern options as a ``Hash``. - Keys in the hash can be ``:w``, ``:wtimeout``, ``:j``, ``:fsync``. + * - ``:monitoring`` + - The monitoring object. + - ``Object`` + - none - .. code-block:: ruby + * - ``:password`` + - The password of the user to authenticate with. + - ``String`` + - none - { :write => { :w => 2 } } - - ``Hash`` - - ``{ :w => 1 }`` + * - ``:platform`` + - Platform information to include in the metadata printed to the mongod logs upon establishing a + connection in server versions >= 3.4. + - ``String`` + - none * - ``:read`` - Specifies the read preference mode and tag sets for selecting servers as a ``Hash``. @@ -377,53 +363,28 @@ Ruby Options - ``Hash`` - ``{ :mode => :primary }`` - * - ``:auth_source`` - - Specifies the authentication source. - - ``String`` - - For MongoDB 2.6 and later: **admin** if credentials are - supplied, otherwise the current database - - - * - ``:auth_mech`` - - Specifies the authenticaion mechanism to use. Can be one of: - ``:mongodb_cr``, ``:mongodb_x509``, ``:plain``, ``:scram``. - - ``Symbol`` - - MongoDB 3.0 and later: ``:scram`` if user credentials - are supplied but an ``:auth_mech`` is not. 2.6 and earlier: - ``:mongodb_cr`` - - * - ``:auth_mech_properties`` - - Provides additional authentication mechanism properties. + * - ``:read_concern`` + - Specifies the read concern options. The only valid key is ``level``, for which the valid + values are ``:local``, ``:majority``, and ``:snapshot``. - ``Hash`` - none - * - ``:user`` - - The name of the user to authenticate with. - - ``String`` - - none + * - ``:read_retry_interval`` + - The interval, in seconds, in which reads on a mongos are retried. + - ``Integer`` + - 5 - * - ``:password`` - - The password of the user to authenticate with. + * - ``:replica_set`` + - When connecting to a replica set, this is the name of the set to + filter servers by. - ``String`` - none - * - ``:connect`` - - Overrides the auto-discovery feature of the driver and forces the cluster - topology to a specific type. Choices: ``:direct``, - ``:replica_set`` or ``:sharded``. - - ``Symbol`` - - none - - * - ``:heartbeat_frequency`` - - The number of seconds for the server monitors to refresh - server states asynchronously. - - ``Float`` - - 10 - - * - ``:database`` - - The name of the database to connect to. - - ``String`` - - admin + * - ``:retry_writes`` + - If a single-statement write operation fails from a network error, the driver automatically retries it once + when connected to server versions 3.6+. + - ``Boolean`` + - false * - ``:server_selection_timeout`` - The number of seconds to wait for an appropriate server to @@ -431,77 +392,130 @@ Ruby Options - ``Float`` - 30 - * - ``:local_threshold`` - - Specifies the maximum latency in seconds between the nearest - server and the servers that can be available for selection to operate on. + * - ``:socket_timeout`` + - The number of seconds to wait for an operation to execute on a + socket before raising an exception. - ``Float`` - - 0.015 + - 5 seconds - * - ``:app_name`` - - Application name that is printed to the mongod logs upon establishing a connection - in server versions >= 3.4. + * - ``:ssl`` + - Tell the client to connect to the servers via SSL. + - ``Boolean`` + - false + + * - ``:ssl_ca_cert`` + - The file path containing concatenated certificate authority certificates used to validate certs + passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object + (in order of priority) is required for :ssl_verify. - ``String`` - none - * - ``:compressors`` - - A list of potential compressors to use, in order of preference. The driver chooses the first - compressor that is also supported by the server. Currently the driver only supports 'zlib'. - - ``Array`` + * - ``:ssl_ca_cert_object`` + - An array of OpenSSL::X509::Certificate representing the certificate authority certificates used to + validate certs passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or + :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. + - ``Array`` - none - * - ``:id_generator`` - - A custom object to generate ids for documents. Must respond to #generate. - - ``Object`` + * - ``:ssl_ca_cert_string`` + - A string containing concatenated certificate authority certificates used to validate certs + passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object + (in order of priority) is required for :ssl_verify. + - ``String`` - none - * - ``:logger`` - - A custom logger. - - ``Object`` - - ``Logger`` + * - ``:ssl_cert`` + - The certificate file path used to identify the connection against MongoDB. This option, if present, + takes precedence over the values of :ssl_cert_string and :ssl_cert_object. + - ``String`` + - none - * - ``:max_idle_time`` - - The maximum seconds a socket can remain idle since it has been checked in to the pool. - - ``Integer`` + * - ``:ssl_cert_object`` + - The OpenSSL::X509::Certificate used to identify the connection against MongoDB. + - ``OpenSSL::X509::Certificate`` - none - * - ``:max_read_retries`` - - The maximum number of read retries on mongos query failures. - - ``Integer`` - - 1 + * - ``:ssl_cert_string`` + - A string containing the PEM-encoded certificate used to identify the connection against MongoDB. + This option, if present, takes precedence over the value of :ssl_cert_object. + - ``String`` + - none - * - ``:monitoring`` - - The monitoring object. - - ``Object`` + * - ``:ssl_key`` + - The private keyfile used to identify the connection against MongoDB. Note that even if the key is stored in + the same file as the certificate, both need to be explicitly specified. This option, if present, takes + precedence over the values of :ssl_key_string and :ssl_key_object. + - ``String`` - none - * - ``:platform`` - - Platform information to include in the metadata printed to the mongod logs upon establishing a - connection in server versions >= 3.4. + * - ``:ssl_key_object`` + - The private key used to identify the connection against MongoDB. + - ``OpenSSL::PKey`` + - none + + * - ``:ssl_key_pass_phrase`` + - A passphrase for the private key. - ``String`` - none - * - ``:read_retry_interval`` - - The interval, in seconds, in which reads on a mongos are retried. - - ``Integer`` - - 5 + * - ``:ssl_key_string`` + - A string containing the PEM-encoded private key used to identify the connection against MongoDB. + This parameter, if present, takes precedence over the value of option :ssl_key_object. + - ``String`` + - none + + * - ``:ssl_verify`` + - Whether to perform peer certificate validation and hostname verification. Note that + the decision of whether to validate certificates will be overridden if + :ssl_verify_certificate is set, and the decision of whether to validate hostnames will be + overridden if :ssl_verify_hostname is set. + - ``Boolean`` + - true + + * - ``:ssl_verify_certificate`` + - Whether to perform peer certificate validation. This setting overrides :ssl_verify with + respect to whether certificate validation is performed. + - ``Boolean`` + - true + + * - ``:ssl_verify_hostname`` + - Whether to perform peer hostname validation. This setting overrides :ssl_verify with + respect to whether hostname validation is performed. + - ``Boolean`` + - true * - ``:truncate_logs`` - Whether to truncate the logs at the default 250 characters. - ``Boolean`` - true + * - ``:user`` + - The name of the user to authenticate with. + - ``String`` + - none + + * - ``:wait_queue_timeout`` + - The number of seconds to wait for a connection in the connection + pool to become available. + - ``Float`` + - 1 + + * - ``:write`` + - Specifies write concern options as a ``Hash``. + Keys in the hash can be ``:w``, ``:wtimeout``, ``:j``, ``:fsync``. + + .. code-block:: ruby + + { :write => { :w => 2 } } + + - ``Hash`` + - ``{ :w => 1 }`` + * - ``:zlib_compression_level`` - The Zlib compression level to use, if using compression. See Ruby's Zlib module for valid levels. - ``Integer`` - none - * - ``:retry_writes`` - - If a single-statement write operation fails from a network error, the driver automatically retries it once - when connected to server versions 3.6+. - - ``Boolean`` - - false - - Details on Timeout Options -------------------------- From 248eaa4ede8a03c9e98aa53fbf7c196ada505c64 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 31 Jan 2019 13:43:08 -0500 Subject: [PATCH 172/442] Release 2.7.0 --- source/reference/driver-compatibility.txt | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index d8d9bc729..31c769c93 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -27,6 +27,15 @@ MongoDB Compatibility - MongoDB 2.6 - MongoDB 2.4 + * - 2.7 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + * - 2.6 - |checkmark| - |checkmark| @@ -154,6 +163,20 @@ Ruby Compatibility - JRuby 9.1 - JRuby + * - 2.7 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + - |checkmark| + - |checkmark| + - + * - 2.6 - |checkmark| - |checkmark| From 18b3abb4596cff52bb5713736a922bbefb9d6604 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 1 Feb 2019 12:19:42 -0500 Subject: [PATCH 173/442] RUBY-1434 Document low server selection timeout in development mode (#1233) --- source/tutorials/ruby-driver-create-client.txt | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index e20636621..9adbc693d 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -781,3 +781,15 @@ option to the client instance. .. code-block:: ruby Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :truncate_logs => false ) + +Development Configuration +------------------------- + +Driver's default configuration is suitable for production deployment. +In development, some settings can be adjusted to provide a better developer +experience. + +- ``:server_selection_timeout``: set this to a low value (e.g., ``1``) +if your MongoDB server is running locally and you start it manually. A low +server selection timeout will cause the driver to fail quickly when there is +no server running. From 9b5c413d84734d8745f933cffe0b5920a599d507 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 4 Feb 2019 19:37:24 -0500 Subject: [PATCH 174/442] RUBY-1649 Tutorial documentation for convenient transaction api (#1232) --- source/tutorials/ruby-driver-transactions.txt | 65 +++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/source/tutorials/ruby-driver-transactions.txt b/source/tutorials/ruby-driver-transactions.txt index b3e1f4354..133da6bc5 100644 --- a/source/tutorials/ruby-driver-transactions.txt +++ b/source/tutorials/ruby-driver-transactions.txt @@ -21,6 +21,63 @@ Using Transactions In order to start a transaction, the application must have a :ref:`session `. +The recommended way to use transactions is to utilize the ``with_transaction`` +helper method: + +.. code-block:: ruby + + session = client.start_session + session.with_transaction do + collection.insert_one(hello: 'world') + end + +The ``with_transaction`` helper does the following: + +- It starts a transaction prior to calling the supplied block, and commits +the transaction when the block finishes. +- If any of the operations in the block, or the commit operation, result in +a transient transaction error, the block and/or the commit will be executed +again. + +The block should be idempotent, because it may be called multiple times. + +The block may explicitly commit or abort the transaction, by calling +``commit_transaction`` or ``abort_transaction``; in this case ``with_transaction`` +will not attempt to commit or abort (but may still retry the block on +transient transaction errors propagated out of the block). + +The block will also be retried if the transaction's commit result is unknown. +This may happen, for example, if the cluster undergoes an election during the +commit. In this case when the block is retried, the primary server of the +topology would likely have changed. + +Currently ``with_transaction`` will stop retrying the block and the commit once +120 seconds pass since the beginning of its execution. This time is not +configurable and may change in a future driver version. Note that this +does not guarantee the overall runtime of ``with_transactions`` will be 120 +seconds or less - just that once 120 seconds of wall clock time have elapsed, +further retry attempts will not be initiated. + +A low level API is also available if more control over transactions is desired. + +``with_transaction`` takes the same options as ``start_transaction`` does, +which are read concern, write concern and read preference: + +.. code-block:: ruby + + session = client.start_session + session.with_transaction( + read_concern: {level: :majority}, + write_concern: {w: 3}, + read: {mode: :primary} + ) do + collection.insert_one(hello: 'world') + end + + +Low Level API +------------- + A transaction can be started by calling the ``start_transaction`` method on a session: .. code-block:: ruby @@ -72,3 +129,11 @@ if it fails. Here is the Ruby code to do so: raise end end + + +Transaction Nesting +------------------- + +MongoDB does not support nesting transactions. Attempting to call +``start_transaction`` or ``with_transaction`` when a transaction is already +in progress will result in an error. From 63533738f2936444fe312d167a9b43f5d23af73e Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 14 Feb 2019 17:55:44 -0500 Subject: [PATCH 175/442] Bump version to 2.7.0 --- source/installation.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/installation.txt b/source/installation.txt index 8abcf1994..e495a4fa8 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -18,7 +18,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.5.1 + gem install mongo -v 2.7.0 TLS/SSL and the Ruby driver --------------------------- @@ -34,7 +34,7 @@ Users of Linux or other non-macOS Unix can check their OpenSSL version like this .. code-block:: sh - $ openssl version + $ openssl version If the version number is less than 1.0.1 support for TLS 1.1 or newer is not available. Contact your operating system vendor for a solution or upgrade to a newer distribution. @@ -43,7 +43,7 @@ You can check your Ruby interpreter by executing the following command: .. code-block:: sh - ruby -e "require 'net/http'; require 'json'; puts JSON.parse(Net::HTTP.get(URI('https://www.howsmyssl.com/a/check')))['tls_version']" + ruby -e "require 'net/http'; require 'json'; puts JSON.parse(Net::HTTP.get(URI('https://www.howsmyssl.com/a/check')))['tls_version']" You should see "TLS 1.X" where X is >= 1. From 418e23c3ff378efcbf3ec4aa467c4df49fd8f7ad Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 14 Feb 2019 18:09:52 -0500 Subject: [PATCH 176/442] Fix docs warnings --- source/tutorials/ruby-driver-create-client.txt | 6 +++--- source/tutorials/ruby-driver-crud-operations.txt | 2 +- source/tutorials/ruby-driver-transactions.txt | 6 +++--- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 9adbc693d..f1aee111a 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -790,6 +790,6 @@ In development, some settings can be adjusted to provide a better developer experience. - ``:server_selection_timeout``: set this to a low value (e.g., ``1``) -if your MongoDB server is running locally and you start it manually. A low -server selection timeout will cause the driver to fail quickly when there is -no server running. + if your MongoDB server is running locally and you start it manually. A low + server selection timeout will cause the driver to fail quickly when there is + no server running. diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index fdc555dfc..fbdeaa5da 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -445,7 +445,7 @@ the modification occurs. doc # Return the document after the update. Write Concern -````````````` +~~~~~~~~~~~~~ Write concern can be given for specific operations on a collection using the ``with`` method, similarly to read preference: diff --git a/source/tutorials/ruby-driver-transactions.txt b/source/tutorials/ruby-driver-transactions.txt index 133da6bc5..9135fb58d 100644 --- a/source/tutorials/ruby-driver-transactions.txt +++ b/source/tutorials/ruby-driver-transactions.txt @@ -34,10 +34,10 @@ helper method: The ``with_transaction`` helper does the following: - It starts a transaction prior to calling the supplied block, and commits -the transaction when the block finishes. + the transaction when the block finishes. - If any of the operations in the block, or the commit operation, result in -a transient transaction error, the block and/or the commit will be executed -again. + a transient transaction error, the block and/or the commit will be executed + again. The block should be idempotent, because it may be called multiple times. From 8d61b920f97b5b4b3b57391b0217bcb6e3f82f72 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 25 Feb 2019 12:39:20 -0500 Subject: [PATCH 177/442] Fix RUBY-1661 Non-mongo exception in abort_transaction does not change session state (#1207) --- source/tutorials/ruby-driver-transactions.txt | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/source/tutorials/ruby-driver-transactions.txt b/source/tutorials/ruby-driver-transactions.txt index 9135fb58d..d8ff27b35 100644 --- a/source/tutorials/ruby-driver-transactions.txt +++ b/source/tutorials/ruby-driver-transactions.txt @@ -110,6 +110,39 @@ To commit or abort a transaction, call ``commit_transaction`` or session.abort_transaction +Note: an outstanding transaction can hold locks to various objects in the +server, such as the database. For example, the drop call in the following +snippet will hang for `transactionLifetimeLimitSeconds +`_ +seconds (default 60) until the server expires and aborts the transaction: + +.. code-block:: ruby + + c1 = Mongo::Client.new(['127.0.0.1:27017']).use(:test_db) + session = c1.start_session + c1['foo'].insert_one(test: 1) + session.start_transaction + c1['foo'].insert_one({test: 2}, session: session) + + c2 = Mongo::Client.new(['127.0.0.1:27017']).use(:test_db) + # hangs + c2.database.drop + +Since transactions are associated with server-side sessions, closing the client +does not abort a transaction that this client initiated - the application must +either call ``abort_transaction`` or wait for the transaction to time out on +the server side. In addition to committing or aborting the transaction, an +application can also end the session which will abort a transaction on this +session if one is in progress: + +.. code-block:: ruby + + session.end_session + + c2 = Mongo::Client.new(['127.0.0.1:27017']).use(:test_db) + # ok + c2.database.drop + Retrying Commits ---------------- From 9f6ae6fd3c273a22dce9f034cb3e22437edc1dec Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 25 Feb 2019 12:39:50 -0500 Subject: [PATCH 178/442] RUBY-1734 Update accepted auth mechanisms in documentation (#1269) --- .../tutorials/ruby-driver-authentication.txt | 29 +++++++++++++++---- .../tutorials/ruby-driver-create-client.txt | 16 ++++++---- 2 files changed, 34 insertions(+), 11 deletions(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index ab20a3a21..4560f1d89 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -141,12 +141,35 @@ MongoDB, see the :manual:`SASL/LDAP tutorial in the MongoDB Manual ssl_cert: '/path/to/client.pem', ssl_ca_cert: '/path/to/ca.pem' ) +.. _kerberos: + Kerberos (GSSAPI) mechanism ``````````````````````````` *Requires MongoDB Enterprise Edition v2.4 or greater.* MongoDB Enterprise Edition v2.4+ supports Kerberos authentication. +To use Kerberos authentication from Ruby, +`mongo_kerberos `_ must be +installed. Add to your ``Gemfile``: + +.. code-block:: plain + + gem 'mongo' + gem 'mongo_kerberos' + +... and in your application: + +.. code-block:: ruby + + require 'mongo' + require 'mongo_kerberos' + +To use Kerberos in the Ruby driver with **MRI**, create a +ticket-granting ticket using ``kinit``. See +`this documentation `_ for more +information. + To use Kerberos in the Ruby driver with **JRuby**, do the following: 1. Specify several system properties so that the underlying GSSAPI Java @@ -158,12 +181,6 @@ To use Kerberos in the Ruby driver with **JRuby**, do the following: 2. Either provide a password OR set the 'java.security.auth.login.config' system property to a config file that references a keytab file. -To use Kerberos in the Ruby driver -with **Matz's Ruby Interpreter (MRI)**, create a -ticket-granting ticket using ``kinit``. See -`this documentation `_ for more -information. - For more information about deploying MongoDB with Kerberos authentication, see the :manual:`manual `. diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index f1aee111a..78bce4abe 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -248,11 +248,17 @@ Ruby Options * - ``:auth_mech`` - Specifies the authenticaion mechanism to use. Can be one of: - ``:mongodb_cr``, ``:mongodb_x509``, ``:plain``, ``:scram``. + ``:gssapi``, ``:mongodb_cr``, ``:mongodb_x509``, ``:plain``, + ``:scram``, ``:scram256``. GSSAPI (Kerberos) authentication + :ref:`requires additional dependencies `. - ``Symbol`` - - MongoDB 3.0 and later: ``:scram`` if user credentials - are supplied but an ``:auth_mech`` is not. 2.6 and earlier: - ``:mongodb_cr`` + - If user credentials are not supplied, ``nil``. If user credentials + are supplied, the default depends on server version. + MongoDB 4.0 and later: ``:scram256`` if user credentials correspond + to a user which supports SCRAM-SHA-256 authentication, otherwise + ``:scram``. + MongoDB 3.0-3.6: ``:scram``. + MongoDB 2.6: ``:mongodb_cr`` * - ``:auth_mech_properties`` - Provides additional authentication mechanism properties. @@ -414,7 +420,7 @@ Ruby Options - An array of OpenSSL::X509::Certificate representing the certificate authority certificates used to validate certs passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. - - ``Array`` + - ``Array< OpenSSL::X509::Certificate >`` - none * - ``:ssl_ca_cert_string`` From 784dc05aa0d9850e594d0f24e0cf0baed4c8f8e4 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Tue, 26 Feb 2019 17:25:27 -0500 Subject: [PATCH 179/442] Release 2.8.0.rc0 --- source/installation.txt | 2 +- source/reference/driver-compatibility.txt | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/source/installation.txt b/source/installation.txt index e495a4fa8..cc6f84d3b 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -18,7 +18,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.7.0 + gem install mongo -v 2.8.0.rc0 TLS/SSL and the Ruby driver --------------------------- diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 31c769c93..7a524c4a1 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -27,6 +27,15 @@ MongoDB Compatibility - MongoDB 2.6 - MongoDB 2.4 + * - 2.8 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + * - 2.7 - |checkmark| - |checkmark| @@ -163,6 +172,20 @@ Ruby Compatibility - JRuby 9.1 - JRuby + * - 2.8 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + - |checkmark| + - |checkmark| + - + * - 2.7 - |checkmark| - |checkmark| From 4fa16eb2a8d5e30816fc3ace7118494ffdb890d9 Mon Sep 17 00:00:00 2001 From: Jon Ekdahl Date: Wed, 27 Feb 2019 05:55:51 +0100 Subject: [PATCH 180/442] Remove call to deprecated BigDecimal.new (#119) RUBY-1653 Replace BigDecimal.new with BigDecimal() --- docs/tutorials/bson-v4.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 6a4d3d2f7..5ccc00519 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -227,7 +227,7 @@ decimal rounding with exact precision. BSON::Decimal128.new("1.28") # Instantiate with a BigDecimal - d = BigDecimal.new(1.28, 3) + d = BigDecimal(1.28, 3) BSON::Decimal128.new(d) JSON Serialization From 17e60a2aad1b2d33d788c209f68122c26b697e81 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Tue, 5 Mar 2019 15:50:53 -0500 Subject: [PATCH 181/442] Remove tutorials menu from TOC --- source/index.txt | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/source/index.txt b/source/index.txt index 0827170dc..cdc416a49 100644 --- a/source/index.txt +++ b/source/index.txt @@ -60,7 +60,23 @@ For tutorials on Mongoid, see the `Mongoid Manual /reference/driver-compatibility From 054ce7dbd6a0caf987589937f4c934d3a4855606 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 7 Mar 2019 12:47:10 -0500 Subject: [PATCH 182/442] RUBY-1757 CANONICALIZE_HOST_NAME is spelled wrong in auth mechanism URI option documentation --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 78bce4abe..6cae960f1 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -106,7 +106,7 @@ URI Options Conversions * - authMechanismProperties=Strings - ``{ :auth_mech_properties => { :service_realm => String, :canonicalize_host_name => true|false, :service_name => String } }`` - Specified as comma-separated key:value pairs, e.g. ``"SERVICE_REALM:foo,CANONICALIZE_HOSTNAME:TRUE"``. + Specified as comma-separated key:value pairs, e.g. ``"SERVICE_REALM:foo,CANONICALIZE_HOST_NAME:TRUE"``. * - authSource=String - ``:auth_source => String`` From 69be41df7c634b30e9e7b7e46dce8c85fd201725 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 7 Mar 2019 12:49:07 -0500 Subject: [PATCH 183/442] Fix RUBY-1766 Typo in tlsAllowInvalidCertificates and tlsAllowInvalidHostnames --- source/tutorials/ruby-driver-create-client.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 6cae960f1..de8343d6b 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -185,7 +185,7 @@ URI Options Conversions Because ``tlsAllowInvalidCertificates`` uses ``true`` to signify that verification should be disabled and ``ssl_verify_certificate`` uses ``false`` to signify that - verification should be disabled, the boolean is inverted before being used to set + verification should be enabled, the boolean is inverted before being used to set ``ssl_verify_certificate``. * - tlsAllowInvalidHostnames=Boolean @@ -193,7 +193,7 @@ URI Options Conversions Because ``tlsAllowInvalidHostnames`` uses ``true`` to signify that verification should be disabled and ``ssl_verify_hostname`` uses ``false`` to signify that - verification should be disabled, the boolean is inverted before being used to set + verification should be enabled, the boolean is inverted before being used to set ``ssl_verify_hostname``. * - tlsCAFile=String From 73ab8c8f943eefcc4cdd0fc3f6cdc3204d24ae54 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 7 Mar 2019 13:34:00 -0500 Subject: [PATCH 184/442] RUBY-1712 Document current connection pool min size behavior --- source/tutorials/ruby-driver-create-client.txt | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index de8343d6b..e1f1af255 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -788,6 +788,7 @@ option to the client instance. Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :truncate_logs => false ) + Development Configuration ------------------------- @@ -799,3 +800,18 @@ experience. if your MongoDB server is running locally and you start it manually. A low server selection timeout will cause the driver to fail quickly when there is no server running. + + +Production Configuration +------------------------ + +Please consider the following when deploying an application using the Ruby +driver in production: + +- The driver has a setting for minimum connection pool size. + This setting results in the driver creating the specified number of + connection objects but each connection object creates sockets and + actually connects to the server on demand. Setting a minimum connection + pool size will keep up to that many network connections to a server + alive once they are established. This behavior may change in a future + version of the driver (`RUBY-1605 `_). From 833c50aed7c987b692e0a14f057f7350ca56b723 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 11 Mar 2019 10:45:15 -0400 Subject: [PATCH 185/442] RUBY-1770 Create connection objects on demand only (#1304) --- .../tutorials/ruby-driver-create-client.txt | 82 ++++++++++--------- 1 file changed, 44 insertions(+), 38 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index e1f1af255..181fa860b 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -334,9 +334,15 @@ Ruby Options * - ``:min_pool_size`` - The minimum number of connections in the connection pool for each - server. + server. The driver does not create connections eagerly - a connection + is only created when a MongoDB operation is to be executed and there + are no available connections, and the total number of established + connections to the target server is below ``:max_pool_size``. + However, once connections to a particular server are created, up to + ``:min_pool_size`` connections will be kept in the connection pool + by the driver. - ``Integer`` - - 1 + - 0 * - ``:monitoring`` - The monitoring object. @@ -568,15 +574,6 @@ Details on Timeout Options number if you are seeing many ``Timeout`` errors while using many threads or when operations are long-running. Defaults to 1 second. -``max_pool_size`` - Maximum size of the connection pool for each server. Defaults to 5 connections. - -``min_pool_size`` - Minimum number of connections in the connection pool for each server. - Increase this number to create connections when the pool is - initialized and to reduce the overhead of creating new connections - later on. Defaults to 1. - ``max_time_ms`` Specified as an option on a particular operation. It defines a cumulative time limit in milliseconds for processing operations on a cursor. Consider using this option instead of a ``socket_timeout``, if the operation should be @@ -611,28 +608,35 @@ for connections. The client instance opens one additional connection per known server for monitoring the server's state. -The size of each connection pool is capped at ``max_pool_size``, which defaults to 5. When a thread in your application -begins an operation on MongoDB, it tries to retrieve a connection from the pool to send that operation on. If there -are some connections available in the pool, it checks out a connection from the pool and uses it for the operation. -If there are no connections available and the size of the pool is less than the ``max_pool_size``, a new connection -will be created. If all connections are in use and the pool has reached its maximum size, the thread waits for a -connection to be returned to the pool by another thread. - -The number of seconds the thread will wait is configurable. This setting, called ``wait_queue_timeout``, is defined in -seconds. It determines how long a thread will wait for a connection to be returned to the pool, when there are no -connections available. If this timeout is reached, a ``Timeout::Error`` is raised. The default is 1 second. - -You can also set the number of connections initialized when the pool is created with the ``min_pool_size`` -setting. This is helpful if your application experiences load spikes and you want to avoid the latency of creating new -connections at the beginning of a spike. The default ``min_pool_size`` is 1. - -Here is an example of estimating the number of connections a multi-threaded application will open: -A client connected to a 3-node replica set opens 3 monitoring sockets. It also opens as many sockets as -needed to support a multi-threaded application's concurrent operations on each server, up to ``max_pool_size``. If the -application only uses the primary (the default), then only the primary connection pool grows and the total connections -is at most 8 (5 connections for the primary pool + 3 monitoring connections). If the application uses a read -preference to query the secondaries, their pools also grow and the total connections can reach 18 (5 + 5 + 5 + 3). - +The size of each connection pool is capped at ``max_pool_size``, which defaults +to 5. When a thread in the application begins an operation on MongoDB, it tries +to retrieve a connection from the pool to send that operation on. If there +are some connections available in the pool, it checks out a connection from +the pool and uses it for the operation. If there are no connections available +and the size of the pool is less than the ``max_pool_size``, a new connection +will be created. If all connections are in use and the pool has reached its +maximum size, the thread waits for a connection to be returned to the pool by +another thread. + +The number of seconds the thread will wait for a connection to become available +is configurable. This setting, called ``wait_queue_timeout``, is defined in +seconds. If this timeout is reached, a ``Timeout::Error`` is raised. The +default is 1 second. + +The driver currently does not eagerly create connections, regardless of +``min_pool_size`` setting. However, once a connection is established, it will +be kept in the pool by the driver as long as the pool size does not exceed +``min_pool_size``. + +Here is an example of estimating the number of connections a multi-threaded +application will open: A client connected to a 3-node replica set opens 3 +monitoring sockets. It also opens as many sockets as needed to support a +multi-threaded application's concurrent operations on each server, up to +``max_pool_size``. If the application only uses the primary (the default), +then only the primary connection pool grows and the total connections is at +most 8 (5 connections for the primary pool + 3 monitoring connections). +If the application uses a read preference to query the secondaries, their +pools also grow and the total connections can reach 18 (5 + 5 + 5 + 3). The default configuration for a ``Mongo::Client`` works for most applications: @@ -640,17 +644,19 @@ The default configuration for a ``Mongo::Client`` works for most applications: client = Mongo::Client.new(["localhost:27017"]) -Create this client **once** for each process, and reuse it for all operations. It is a common mistake to create a new -client for each request, which is very inefficient and not what the client was designed for. +Create this client **once** for each process, and reuse it for all operations. +It is a common mistake to create a new client for each request, which is very +inefficient and not what the client was designed for. -To support extremely high numbers of concurrent MongoDB operations within one process, increase ``max_pool_size``: +To support extremely high numbers of concurrent MongoDB operations within one +process, increase ``max_pool_size``: .. code-block:: ruby client = Mongo::Client.new(["localhost:27017"], max_pool_size: 200) -Any number of threads are allowed to wait for connections to become available, and they can wait the default (1 second) -or the ``wait_queue_timeout`` setting: +Any number of threads are allowed to wait for connections to become available, +and they can wait the default (1 second) or the ``wait_queue_timeout`` setting: .. code-block:: ruby From b32af5b167a7bb8ce9c9fb577874471b52be23fa Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 19 Mar 2019 16:48:21 -0400 Subject: [PATCH 186/442] RUBY-1706 Support Retryable Writes on by Default (#1290) * RUBY-1706 Support Retryable Writes on by Default * Sync retryable writes spec tests * Sync transactions spec tests * Fix spec tests * Do not give retry writes option per the test label --- source/tutorials/ruby-driver-create-client.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 181fa860b..a2b742795 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -396,7 +396,7 @@ Ruby Options - If a single-statement write operation fails from a network error, the driver automatically retries it once when connected to server versions 3.6+. - ``Boolean`` - - false + - true * - ``:server_selection_timeout`` - The number of seconds to wait for an appropriate server to @@ -724,7 +724,7 @@ Details on Retryable Writes The driver implements two mechanisms for retrying writes: modern and legacy. The modern mechanism is used when the driver connects to a MongoDB 3.6+ replica set or a sharded cluster, and the ``retry_writes`` client option is -set to true. This mechanism retries writes once on network errors +not set to false. This mechanism retries writes once on network errors (timeout, connection reset, etc.) as well as transient errors resulting from servers being unavailable due to initialization, shutdown, replica set elections and similar events (the corresponding errors are not master, @@ -734,7 +734,7 @@ likely to no longer be usable. The legacy mechanism is used when the driver connects to MongoDB 3.4 or earlier clusters, or to MongoDB 3.6 or later standalone servers, or when -``retry_writes`` client option is not given. In this mode the driver will +``retry_writes`` client option is set to false. In this mode the driver will retry write operations once due to transient errors resulting from servers being unavailable due to initialization, shutdown, replica set elections and similar events, but will not retry writes on network errors From 8e0f500289fba5ef3ff1f3b2b0d69e67e5a7e6fd Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 21 Mar 2019 11:16:17 -0400 Subject: [PATCH 187/442] Revert "Fix RUBY-1766 Typo in tlsAllowInvalidCertificates and tlsAllowInvalidHostnames" This reverts commit 69be41df7c634b30e9e7b7e46dce8c85fd201725. --- source/tutorials/ruby-driver-create-client.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index a2b742795..201fc8b2c 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -185,7 +185,7 @@ URI Options Conversions Because ``tlsAllowInvalidCertificates`` uses ``true`` to signify that verification should be disabled and ``ssl_verify_certificate`` uses ``false`` to signify that - verification should be enabled, the boolean is inverted before being used to set + verification should be disabled, the boolean is inverted before being used to set ``ssl_verify_certificate``. * - tlsAllowInvalidHostnames=Boolean @@ -193,7 +193,7 @@ URI Options Conversions Because ``tlsAllowInvalidHostnames`` uses ``true`` to signify that verification should be disabled and ``ssl_verify_hostname`` uses ``false`` to signify that - verification should be enabled, the boolean is inverted before being used to set + verification should be disabled, the boolean is inverted before being used to set ``ssl_verify_hostname``. * - tlsCAFile=String From a6b3309af7c3fa8657dda66b044ad61819e076fd Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 9 Apr 2019 11:59:46 -0400 Subject: [PATCH 188/442] RUBY-1784 Document that retryable writes are now on by default (#1338) * RUBY-1784 Document that retryable writes are now on by default * Clarified language --- .../tutorials/ruby-driver-create-client.txt | 68 ++++++++++++------- 1 file changed, 44 insertions(+), 24 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 201fc8b2c..d1cc23699 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -722,30 +722,12 @@ Details on Retryable Writes --------------------------- The driver implements two mechanisms for retrying writes: modern and legacy. -The modern mechanism is used when the driver connects to a MongoDB 3.6+ -replica set or a sharded cluster, and the ``retry_writes`` client option is -not set to false. This mechanism retries writes once on network errors -(timeout, connection reset, etc.) as well as transient errors resulting -from servers being unavailable due to initialization, shutdown, replica -set elections and similar events (the corresponding errors are not master, -node is recovering, etc.). Prior to retrying the write the driver attempts -to select a new primary, since the server that was previously used is -likely to no longer be usable. - -The legacy mechanism is used when the driver connects to MongoDB 3.4 or -earlier clusters, or to MongoDB 3.6 or later standalone servers, or when -``retry_writes`` client option is set to false. In this mode the driver will -retry write operations once due to transient errors resulting from -servers being unavailable due to initialization, shutdown, replica set -elections and similar events, but will not retry writes on network errors -like timeouts. - -Note that writes are always retried on transient server status-related -errors like "not master" and "node is recovering", regardless of what -``retry_writes`` option is set to. - -The following write methods used in day-to-day operations on collections will -be automatically retried: +As of driver version 2.9.0, the modern mechanism is used by default on servers +that support it, and the legacy mechanism is deprecated and disabled by default +on all server versions. + +The following write methods used in day-to-day operations on collections +are subject to write retries: - ``collection#insert_one`` - ``collection#update_one`` @@ -756,6 +738,44 @@ be automatically retried: - ``collection#find_one_and_delete`` - ``collection#bulk_write`` (for all single statement ops, i.e. not for ``update_many`` or ``delete_many``) +Modern Retryable Writes +``````````````````````` + +The modern mechanism will retry failing writes once when the driver is +connected to a MongoDB 3.6 or higher replica set or a sharded cluster, +because they require an oplog on the serer. Modern mechanism will not retry +writes when the driver is connected to a standalone MongoDB server or +server versions 3.4 or older. + +The following errors will cause writes to be retried: + +- Network errors including timeouts +- "not master" errors +- "node is recovering" errors + +Prior to retrying the write the driver will perform server selection, +since the server that the original write was sent to is likely no longer +usable. + +Legacy Retryable Writes +``````````````````````` + +If modern retryable writes mechanism is disabled by setting the client +option ``retry_writes: false`` or by using the ``retryWrites=false`` +URI option, the driver will utilize the legacy retryable writes mechanism. +The legacy mechanism retries writes on the same operations as the modern +mechanism, and retries one time as the modern mechanism does. +The difference is that legacy mechanism retries writes for a different set +of errors compared to the modern mechanism, and specifically does not +retry writes when a network timeout is encountered. + +Disabling Retryable Writes +`````````````````````````` + +Currently the driver will always retry writes on transient server-related +errors like "not master" and "node is recovering". A future version of +the driver may provide a mechanism to disable write retries entirely. + Logging ------- From 6c38abb8c95e8ea99ac3ea2c042b0fbbd2158408 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 16 Apr 2019 16:45:27 -0400 Subject: [PATCH 189/442] RUBY-1793 Recommend lower server selection timeout by default (#1346) --- source/tutorials/ruby-driver-create-client.txt | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index d1cc23699..d754a602e 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -841,3 +841,9 @@ driver in production: pool size will keep up to that many network connections to a server alive once they are established. This behavior may change in a future version of the driver (`RUBY-1605 `_). +- If the application is reverse proxied to by another web server or a load + balancer, ``server_selection_timeout`` should generally be set to a lower + value than the reverse proxy's read timeout. For exampe, `Heroku request timeout + `_ is 30 seconds and + is not configurable; if deploying a Ruby application using MongoDB to Heroku, + consider lowering server selection timeout to 20 or 15 seconds. From 1843f91bf525d14586838ac46e4b54b22faf8339 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 16 Apr 2019 18:07:59 -0400 Subject: [PATCH 190/442] RUBY-1562 Retryable Reads implementation (#1335) * RUBY-1562 Retryable reads spec test runner * RUBY-1562 Retryable reads implementation * RUBY-1562 Fix tests broken by retryable reads change * RUBY-1562 Add a ci config with modern retryable reads off * RUBY-1562 Retryable reads user documentation * RUBY-1562 Fix fail points * Spell mapReduce in camel case to match the fail point * Update retryable docstrings * Docstring --- .../tutorials/ruby-driver-create-client.txt | 73 +++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index d754a602e..3db7f61d8 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -718,6 +718,79 @@ does not close it, the parent process will continue consuming a connection slot in the cluster and will continue monitoring the cluster for as long as the parent remains alive. + +Details on Retryable Reads +-------------------------- + +The driver implements two mechanisms for retrying reads: modern and legacy. +As of driver version 2.9.0, the modern mechanism is used by default, and the +legacy mechanism is deprecated. + +Modern Retryable Reads +`````````````````````` + +When the modern mechanism is used, read operations are retried once in the +event of a network error, a "not master" error, or a "node is recovering" error. +The following operations are covered: + +- `Collection#find `_ + and related methods +- `Collection#aggregate `_ +- `Collection#count `_, + `Collection#count_documents `_ +- Change stream helpers: `Collection#watch `_, + `Database#watch `_, + `Client#watch `_ +- Enumeration commands: `Client#list_mongo_databases `_, + `Client#list_databases `_, + `Client#database_names `_, + `Database#collection_names `_, + `Database#collections `_, + `Database#list_collections `_, + `Collection#indexes `_ + +When an operation returns a cursor, only the initial read command can be retried. +``getMore`` operations on cursors are not retried by driver version 2.9.0 or +newer. Additionally, when a read operation is retried, a new server for the +operation is selected; this may result in the retry being sent to a different +server from the one which received the first read. + +The behavior of modern retryable reads is covered in detail by the +`retryable reads specification +`_. + +Note that the modern retryable reads can only be used with MongoDB 3.6 and +higher servers. When used with MongoDB 3.4 and lower servers, Ruby driver +version 2.9.0 and higher will not retry reads by default - the application +must explicitly request legacy retryable reads by setting the +``retry_reads: false`` client option or using ``retryReads=false`` URI option. + +Legacy Retryable Reads +`````````````````````` + +The legacy read retry behavior of the Ruby driver is available by setting the +``retry_reads: false`` client option or passing the ``retryReads=false`` URI +option to the client. + +When using legacy read retry behavior, the number of retries can be set +by specifying the ``max_read_retries`` client option. When using driver version +2.9.0 or higher, the set of operations which would be retried with legacy +retryable reads is identical to the one described above for modern retryable +reads. In older driver versions the behavior of legacy retryable writes was +different in that some of the operations were not retried. + +As of driver version 2.9.0, legacy read retries perform server selection prior +to retrying the operation, as modern retriable writes do. In older driver +versions read retries would be sent to the same server which the initial read +was sent to. + +Disabling Retryable Reads +````````````````````````` + +To disable all read retries, set the following client options: +``retry_reads: false, max_read_retries: 0``. + + Details on Retryable Writes --------------------------- From 96aaf287f102cdc313a521d1ecfe21bccf83bcda Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 18 Apr 2019 13:42:30 -0400 Subject: [PATCH 191/442] RUBY-1626 Knob to turn off legacy retryable writes (#1342) --- .../tutorials/ruby-driver-create-client.txt | 22 ++++++++++++++----- 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 3db7f61d8..0a00be607 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -328,7 +328,14 @@ Ruby Options - 5 * - ``:max_read_retries`` - - The maximum number of read retries on mongos query failures. + - The maximum number of read retries, when legacy read retries are used. + Set to 0 to disable legacy read retries. + - ``Integer`` + - 1 + + * - ``:max_write_retries`` + - The maximum number of write retries, when legacy write retries are used. + Set to 0 to disable legacy write retries. - ``Integer`` - 1 @@ -837,17 +844,20 @@ If modern retryable writes mechanism is disabled by setting the client option ``retry_writes: false`` or by using the ``retryWrites=false`` URI option, the driver will utilize the legacy retryable writes mechanism. The legacy mechanism retries writes on the same operations as the modern -mechanism, and retries one time as the modern mechanism does. -The difference is that legacy mechanism retries writes for a different set +mechanism. By default the legacy mechanism retries once, like the modern +mechanism does; to change the number of retries, set ``:max_write_retries`` +client option. + +The difference between legacy and modern retry mechanisms is that the +legacy mechanism retries writes for a different set of errors compared to the modern mechanism, and specifically does not retry writes when a network timeout is encountered. Disabling Retryable Writes `````````````````````````` -Currently the driver will always retry writes on transient server-related -errors like "not master" and "node is recovering". A future version of -the driver may provide a mechanism to disable write retries entirely. +To disable all write retries, set the following client options: +``retry_writes: false, max_write_retries: 0``. Logging ------- From ce79dd610582c8c2a292a3235e525f0aad3e3cc5 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Mon, 29 Apr 2019 16:07:57 -0400 Subject: [PATCH 192/442] Driver release 2.6.3 was skipped, update the docs to point to 2.6.4 instead --- source/reference/driver-compatibility.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 7a524c4a1..766dd7872 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -365,7 +365,7 @@ or higher is recommended when using MongoDB Atlas, as this version has significant SSL performance improvements. When running on JRuby and connecting to Atlas Free Tier, -`driver version 2.6.3 `_ +`driver version 2.6.4 `_ or higher and Java 8 or higher are required. .. include:: /includes/unicode-checkmark.rst From 8735298d8060999ddf2c225c470dacad75160b06 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 21 May 2019 12:14:31 -0400 Subject: [PATCH 193/442] RUBY-1560 Finalize CMAP event API (#1310) This changes the method used for subscribing to CMAP events from succeeded to published, as some of the events are failed ones. User documentation for CMAP is also added. --- source/tutorials/ruby-driver-monitoring.txt | 63 +++++++++++++++++++-- 1 file changed, 59 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-monitoring.txt b/source/tutorials/ruby-driver-monitoring.txt index 950487913..925689a15 100644 --- a/source/tutorials/ruby-driver-monitoring.txt +++ b/source/tutorials/ruby-driver-monitoring.txt @@ -13,10 +13,11 @@ Monitoring The driver allows the application to be notified when certain events happen. These events are organized into the following categories: -- Command monitoring; -- Topology lifecycle; -- Server lifecycle; -- Server heartbeats. +- Command monitoring +- Topology lifecycle +- Server lifecycle +- Server heartbeats +- Connection pools and connections Topology and server events are part of Server Discovery and Monitoring (SDAM). @@ -331,6 +332,60 @@ Sample output: D, [2018-09-23T13:44:10.707018 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | STARTED D, [2018-09-23T13:44:10.707778 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | SUCCEEDED | 0.000772381s +Connection Pool And Connection Monitoring +----------------------------------------- + +Each client maintains a connection pool for each server in the deployment that +it is aware of, and publishes events for both connection pools and individual +connections. To subscribe to these events, define a subscriber class implementing +the method ``pubished`` which takes a single parameter for the event that +is being published. Note that future versions of the driver may introduce +additional events published through this mechanism. + +The following events are currently implemented by the driver, following +the `CMAP specification `_: + +- PoolCreated +- PoolCleared +- PoolClosed +- ConnectionCreated +- ConnectionReady +- ConnectionClosed +- ConnectionCheckOutStarted +- ConnectionCheckOutFailed +- ConnectionCheckOutSucceeded +- ConnectionCheckedIn + +The driver provides a logging subscriber which may be used to log all +connection pool and connection-related events. This subscriber is not enabled +by default because it will create log entries for each operation performed +by the application. To enable this subscriber globally or per client: + +.. code-block:: ruby + + Mongo::Monitoring::Global.subscribe( + Mongo::Monitoring::CONNECTION_POOL, + Mongo::Monitoring::CmapLogSubscriber.new) + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test' ) + subscriber = Mongo::Monitoring::CmapLogSubscriber.new + client.subscribe( Mongo::Monitoring::CONNECTION_POOL, subscriber ) + +Sample output: + +.. code-block:: none + + D, [2019-05-06T17:23:21.595412 #8576] DEBUG -- : MONGODB | EVENT: # + D, [2019-05-06T17:23:21.595584 #8576] DEBUG -- : MONGODB | EVENT: # + D, [2019-05-06T17:23:21.603549 #8576] DEBUG -- : MONGODB | EVENT: # + D, [2019-05-06T17:23:21.603616 #8576] DEBUG -- : MONGODB | EVENT: # + D, [2019-05-06T17:23:21.603684 #8576] DEBUG -- : MONGODB | EVENT: # + D, [2019-05-06T17:23:21.604079 #8576] DEBUG -- : MONGODB | EVENT: # + D, [2019-05-06T17:23:21.605759 #8576] DEBUG -- : MONGODB | EVENT: # + D, [2019-05-06T17:23:21.605784 #8576] DEBUG -- : MONGODB | EVENT: # + D, [2019-05-06T17:23:21.605817 #8576] DEBUG -- : MONGODB | EVENT: # + D, [2019-05-06T17:23:21.605852 #8576] DEBUG -- : MONGODB | EVENT: # + Disabling Monitoring -------------------- From c021679e6938c1cd45575ff33a8efa1bc1fd7bc4 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 23 May 2019 13:42:50 -0400 Subject: [PATCH 194/442] Release 2.9.0.rc0 --- source/installation.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/installation.txt b/source/installation.txt index cc6f84d3b..af6dbb95b 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -18,7 +18,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.8.0.rc0 + gem install mongo -v 2.9.0.rc0 TLS/SSL and the Ruby driver --------------------------- From 71222c78c49656cf20e6a4e7b3c90230f9de2651 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 23 May 2019 15:00:08 -0400 Subject: [PATCH 195/442] Reference Mongoid forking guidance from driver tutorial --- source/tutorials/ruby-driver-create-client.txt | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 0a00be607..b53a1c932 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -679,6 +679,10 @@ When ``#close`` is called on a client by any thread, all connections are closed: Usage with Forking Servers -------------------------- +*Note:* Applications using Mongoid should follow `Mongoid's forking guidance +`_. +The sample code below is provided for applications using the Ruby driver directly. + When using the Mongo Ruby driver with a forking web server such as Unicorn, worker processes should generally each have their own Mongo::Client instances. This is because: From f3c11a93080b3b7b1ba8855fa78d365b0b63d373 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 23 May 2019 15:01:25 -0400 Subject: [PATCH 196/442] Fix code block declaration --- source/tutorials/ruby-driver-authentication.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index 4560f1d89..1ee2bb0e4 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -153,7 +153,7 @@ To use Kerberos authentication from Ruby, `mongo_kerberos `_ must be installed. Add to your ``Gemfile``: -.. code-block:: plain +.. code-block:: ruby gem 'mongo' gem 'mongo_kerberos' From 4c7adbd7bf39d562d4b4616525e3235dcdaa0ef3 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 23 May 2019 15:14:15 -0400 Subject: [PATCH 197/442] Add driver version 2.9 to compatibility tables --- source/reference/driver-compatibility.txt | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 766dd7872..6033932a5 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -27,6 +27,15 @@ MongoDB Compatibility - MongoDB 2.6 - MongoDB 2.4 + * - 2.9 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + * - 2.8 - |checkmark| - |checkmark| @@ -172,6 +181,20 @@ Ruby Compatibility - JRuby 9.1 - JRuby + * - 2.9 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + - |checkmark| + - |checkmark| + - + * - 2.8 - |checkmark| - |checkmark| From 1da2a3aec5a1d0e7572d932b590f104486884c58 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Fri, 24 May 2019 14:20:32 -0400 Subject: [PATCH 198/442] Move unicode checkmark include to the bottom where it should have been --- source/reference/driver-compatibility.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 6033932a5..3fa797065 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -391,8 +391,6 @@ When running on JRuby and connecting to Atlas Free Tier, `driver version 2.6.4 `_ or higher and Java 8 or higher are required. -.. include:: /includes/unicode-checkmark.rst - JRuby Kerberos ~~~~~~~~~~~~~~ @@ -401,3 +399,5 @@ property "sun.security.jgss.native" to will be set to "true" in order to facilit the system cache of TGTs (e.g. TGTs obtained with ``kinit``). Any other use of the JGSS library will also be affected by this setting, meaning any TGTs in the system cache will be available for obtaining Kerberos credentials as well. + +.. include:: /includes/unicode-checkmark.rst From 2a38d611515bae9099f9eab7e35249f7f9bf458c Mon Sep 17 00:00:00 2001 From: HanaPearlman Date: Fri, 31 May 2019 12:51:27 -0400 Subject: [PATCH 199/442] RUBY-1790 Document connection string and URI option precedence rules (#1365) * RUBY-1790 Document connection string and URI option precedence rules * reference options in hash for clarity --- source/tutorials/ruby-driver-create-client.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index b53a1c932..cdd58ce20 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -70,7 +70,8 @@ Client Options A number of different options can be passed to a ``Mongo::Client`` to configure driver behavior, either by providing them in the options hash to the constructor or by -providing them in the URI. +providing them in the URI. If both a URI and an options hash are provided, the options +in the hash take precedence over any analogous options present in the URI. Since the URI options are required in camel case, which is not the Ruby standard, the following table shows the option in the URI and its corresponding option if passed From b61c49e8a85967a2c88acc75df969641f75235ed Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 31 May 2019 17:52:29 -0400 Subject: [PATCH 200/442] RUBY-1822 Support providing certificate chains as client certificates (#1366) --- .../tutorials/ruby-driver-create-client.txt | 92 ++++++++++++++++++- 1 file changed, 87 insertions(+), 5 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index cdd58ce20..d736dd520 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -445,19 +445,41 @@ Ruby Options - none * - ``:ssl_cert`` - - The certificate file path used to identify the connection against MongoDB. This option, if present, - takes precedence over the values of :ssl_cert_string and :ssl_cert_object. + - Path to the client certificate file used to identify the application to + the MongoDB servers. The file may also contain the certificate's private + key; if so, the private key is ignored by this option. The file may + also contain intermediate certificates forming the certificate chain + from the client certificate to the CA certificate; any intermediate + certificates will be parsed by the driver and provided to the OpenSSL + context in ``extra_chain_cert`` attribute. If intermediate certificates + are provided, they must follow the client certificate which must be + the first certificate in the file. + + This option, if present, takes precedence over ``:ssl_cert_string`` and + ``:ssl_cert_object`` options. - ``String`` - none * - ``:ssl_cert_object`` - - The OpenSSL::X509::Certificate used to identify the connection against MongoDB. + - The OpenSSL::X509::Certificate used to identify the application to + the MongoDB servers. Only one certificate may be passed through this + option. - ``OpenSSL::X509::Certificate`` - none * - ``:ssl_cert_string`` - - A string containing the PEM-encoded certificate used to identify the connection against MongoDB. - This option, if present, takes precedence over the value of :ssl_cert_object. + - A string containing the PEM-encoded certificate used to identify the + application to the MongoDB servers. The string may also contain the + certificate's private key; if so, the private key is ignored by this + option. The string may also contain intermediate certificates forming + the certificate chain from the client certificate to the CA certificate; + any intermediate certificates will be parsed by the driver and provided + to the OpenSSL context in ``extra_chain_cert`` attribute. If intermediate + certificates are provided, they must follow the client certificate which + must be the first certificatet in the string. + + This option, if present, takes precedence over the ``:ssl_cert_object`` + option. - ``String`` - none @@ -590,6 +612,66 @@ Details on Timeout Options operations that support this option. +Details on TLS Connections +-------------------------- + +To connect to the MongoDB cluster using TLS, pass ``tls`` Ruby or URI option +to the ``Mongo::Client`` constructor. TLS must be explicitly configured on the +client side when the cluster requires TLS connections - there is currently no +automatic detection of whether TLS is required. + +The driver will attempt to verify the server's TLS certificate by default, and +will abort the connection if this verification fails. In order for this +verification to succeed, you may need to specify the certificate authority +that the server certificate is signed with via ``tlsCAFile`` URI option or +``ssl_ca_cert`` Ruby option. If these options are not given, the driver will +use the default system root certificate store as the trust anchor; if these +options are given, the default system root certificate store will not be used. +To omit TLS certificate verification by the client, which is not +recommended, specify ``tlsInsecure=true`` URI option or ``ssl_verify: false`` +Ruby option. + +By default, MongoDB server will verify the connecting clients' TLS certificates, +which requires the client to specify a client certificate when connecting. +This can be accomplished through ``tlsCertificateKeyFile`` URI option which +takes as the argument the path to a single file containing both the certificate +and the corresponding private key, or through ``:ssl_cert`` and ``:ssl_key`` +Ruby options. The Ruby options allow passing separace certificate and key files +to the driver, but also can be pointed at the single PEM file containing both +the certificate and the corresponding private key. Further information on +configuring MongoDB server for TLS is available in the `MongoDB manual +`_. + +Using Intermediate Certificates +``````````````````````````````` + +It is possible to use certificate chains for both the client and the server +certificates. When using chains, the certificate authority parameter should +be configured to contain the trusted root certificates only; the intermediate +certificates, if any, should be provided in the server or client certificates +by concatenating them after the leaf server and client certificates, respectively. + +``:ssl_cert`` and ``:ssl_cert_string`` Ruby options, as well as +``tlsCertificateKeyFile`` URI option, support certificate chains. +``:ssl_cert_object`` Ruby option, which takes an instance of +``OpenSSL::X509::Certificate``, does not support certificate chains. + +The Ruby driver performs strict X509 certificate verification, which requires +that both of the following fields are set in the intermediate certificate(s): + +- X509v3 Basic Constraints: CA: TRUE -- Can sign certificates +- X509v3 Key Usage: Key Cert Sign -- Can sign certificates + +More information about these flags can be found `here +`. + +It is a common pitfall to concatenate intermediate certificates to the root +CA certificates passed in ``tlsCAFile`` / ``ssl_ca_cert`` options. By doing +so, the intermediate certificates are elevated to trusted status and are +themselves not verified against the actual CA root. More information on this +issue is available `here `. + + TCP Keepalive Configuration --------------------------- From d374ef48f6cb65d30e62abf3e797197fc58996b4 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Mon, 3 Jun 2019 12:04:38 -0400 Subject: [PATCH 201/442] Fix hyperlinks --- source/tutorials/ruby-driver-create-client.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index d736dd520..c71780f06 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -663,13 +663,13 @@ that both of the following fields are set in the intermediate certificate(s): - X509v3 Key Usage: Key Cert Sign -- Can sign certificates More information about these flags can be found `here -`. +`_. It is a common pitfall to concatenate intermediate certificates to the root CA certificates passed in ``tlsCAFile`` / ``ssl_ca_cert`` options. By doing so, the intermediate certificates are elevated to trusted status and are themselves not verified against the actual CA root. More information on this -issue is available `here `. +issue is available `here `_. TCP Keepalive Configuration From 25bb9be08530c4b380268bf79391915499ca98e1 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 3 Jun 2019 15:35:26 -0400 Subject: [PATCH 202/442] RUBY-1823 Support adding multiple CA certificates (#1368) --- source/tutorials/ruby-driver-create-client.txt | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index c71780f06..23cdcc0aa 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -671,6 +671,23 @@ so, the intermediate certificates are elevated to trusted status and are themselves not verified against the actual CA root. More information on this issue is available `here `_. +Specifying Multiple CA Certificates +``````````````````````````````````` + +``:ssl_ca_cert`` Ruby option and ``tlsCAFile`` URI option can be used with +a file containing multiple certificates. All certificates thus referenced +will become trust anchors. + +``:ssl_ca_cert_object`` option takes an array of certificates, and thus +can also be used to add multiple certificates as certificate authorities. + +``:ssl_ca_cert_string`` option supports passing only one CA certificate to the +driver. + +Intermediate certificates should not be specified in files given to any +of these options, because they would then not be verified against actual +trusted CA certificates. + TCP Keepalive Configuration --------------------------- From d472e36d6318111202f61203c5d28c7e1b546db0 Mon Sep 17 00:00:00 2001 From: Chris Bush Date: Wed, 5 Jun 2019 18:05:16 -0400 Subject: [PATCH 203/442] DOCS-12711 Fit compatibility tables in view by removing padding (#1378) This uses the new no-padding class added to docs-tools. The padding on the compatibility tables is interfering with the ability of the table to stretch to fit. --- source/reference/driver-compatibility.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 3fa797065..69dc0e5b2 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -16,7 +16,7 @@ MongoDB Compatibility .. list-table:: :header-rows: 1 :stub-columns: 1 - :class: compatibility + :class: compatibility-large no-padding * - Ruby Driver - MongoDB 4.0 @@ -165,7 +165,7 @@ Ruby Compatibility .. list-table:: :header-rows: 1 :stub-columns: 1 - :class: compatibility + :class: compatibility-large no-padding * - Ruby Driver - Ruby 2.6 From dc3196842a861f4a8e15594857ab1aeeb3a14d0c Mon Sep 17 00:00:00 2001 From: HanaPearlman Date: Thu, 13 Jun 2019 10:07:01 -0400 Subject: [PATCH 204/442] RUBY-1708 Support postBatchResumeToken in change streams (#1382) Also includes RUBY-1815 Clarify resume token used in resuming and getResumeToken --- source/tutorials/ruby-driver-change-streams.txt | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index bae4df63b..9d7138502 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -64,7 +64,10 @@ You can also receive the notifications as they become available: end The ``next`` method blocks and polls the cluster until a change is available. -If there is a non-resumable error, ``next`` will raise an exception. +Use the ``try_next`` method to iterate a change stream without blocking; this +method will wait up to max_await_time_ms milliseconds for changes from the server, +and if no changes are received it will return nil. If there is a non-resumable +error, both ``next`` and ``try_next`` will raise an exception. See Resuming a Change Stream section below for an example that reads changes from a collection indefinitely. @@ -147,15 +150,16 @@ read preference and does not heal quickly enough, ``next`` and ``each`` will raise an error. In these cases the application must track, via the resume token, which documents from the change stream it has processed and create a new change stream object via the ``watch`` call, passing an -appropriate ``:resume_after`` argument. The resume token is exposed -at the key ``_id`` in each change document. +appropriate ``:resume_after`` argument. The ``_id`` key in each change +document can be used as a resume token. However, to get the most up-to-date +resume token, use the ``resume_token`` method. .. code-block:: ruby token = doc['_id'] stream = collection.watch([], resume_after: token) -To watch a collection indefinitely, retrying on all MongoDB errors: +To watch a collection indefinitely, retrying on all MongoDB errors, using ``next``: .. code-block:: ruby @@ -166,9 +170,9 @@ To watch a collection indefinitely, retrying on all MongoDB errors: enum = stream.to_enum while doc = enum.next process(doc) - token = doc['_id'] + token = stream.resume_token end rescue Mongo::Error sleep 1 end - end + end \ No newline at end of file From c9f7e7249249127cf35c45b74360bf6ab17b4538 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 20 Jun 2019 13:37:30 -0400 Subject: [PATCH 205/442] RUBY-1848 deprecate support for Ruby 1.9-2.2 --- source/reference/driver-compatibility.txt | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 69dc0e5b2..ead86b092 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -186,10 +186,10 @@ Ruby Compatibility - |checkmark| - |checkmark| - |checkmark| - - |checkmark| - - |checkmark| - - |checkmark| - - |checkmark| + - D + - D + - D + - D - - |checkmark| - |checkmark| @@ -363,6 +363,12 @@ Ruby Compatibility - - |checkmark| +Support for Ruby versions less than 2.3 has been deprecated. Driver version +2.10 will be the last release to support Ruby 1.9-2.2. As of driver version +2.11.0, the minimum required Ruby version will be 2.3. Please note that, +as of January 1, 2019, Ruby versions 2.2 and older are `not supported by +the Ruby core team `_. + Rails/ActiveSupport Compatibility ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From cb0c1e5fd14768340a8503400bba698c307ee80e Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 24 Jun 2019 12:22:20 -0400 Subject: [PATCH 206/442] RUBY-1561 Move error labels into operations layer (#1392) * RUBY-1561 Use literal transaction error labels instead of constants * RUBY-1561 label exceptions in the operations layer rather than in sessions which is outside retryable writes processing --- source/tutorials/ruby-driver-transactions.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-transactions.txt b/source/tutorials/ruby-driver-transactions.txt index d8ff27b35..f1aeb8eba 100644 --- a/source/tutorials/ruby-driver-transactions.txt +++ b/source/tutorials/ruby-driver-transactions.txt @@ -156,7 +156,7 @@ if it fails. Here is the Ruby code to do so: begin session.commit_transaction rescue Mongo::Error => e - if e.label?(Mongo::Error::UNKNOWN_TRANSACTION_COMMIT_RESULT_LABEL) + if e.label?('UnknownTransactionCommitResult') retry else raise From 5501febab3b8833ca0dd874a9a9bdf7c9595666f Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Mon, 8 Jul 2019 13:55:01 -0400 Subject: [PATCH 207/442] Fix RUBY-1870 Transactions documentation does not pass session to operations --- source/tutorials/ruby-driver-transactions.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-transactions.txt b/source/tutorials/ruby-driver-transactions.txt index f1aeb8eba..d0fd2656c 100644 --- a/source/tutorials/ruby-driver-transactions.txt +++ b/source/tutorials/ruby-driver-transactions.txt @@ -28,7 +28,7 @@ helper method: session = client.start_session session.with_transaction do - collection.insert_one(hello: 'world') + collection.insert_one({hello: 'world'}, session: session) end The ``with_transaction`` helper does the following: @@ -71,7 +71,7 @@ which are read concern, write concern and read preference: write_concern: {w: 3}, read: {mode: :primary} ) do - collection.insert_one(hello: 'world') + collection.insert_one({hello: 'world'}, session: session) end From 2caeacdde0f27f7b4ea597bdabfede84c0dd7e0f Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 10 Jul 2019 15:26:19 -0400 Subject: [PATCH 208/442] RUBY-1860 Standardize on :write_concern for write concern options (#1413) * Change client and collection from :write to :write_concern * Use :write_concern internally * Switch gridfs code to :write_concern * Fix the docs --- .../tutorials/ruby-driver-create-client.txt | 19 +- .../tutorials/ruby-driver-crud-operations.txt | 183 ++++++++++++++++-- source/tutorials/ruby-driver-gridfs.txt | 118 +++++++---- 3 files changed, 268 insertions(+), 52 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 23cdcc0aa..e72241b42 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -129,13 +129,13 @@ URI Options Conversions - ``:connect_timeout => Float`` * - fsync=Boolean - - ``{ :write => { :fsync => true|false }}`` + - ``{ :write_concern => { :fsync => true|false }}`` * - heartbeatFrequencyMS=Integer - ``:heartbeat_frequency => Float`` * - journal=Boolean - - ``{ :write => { :j => true|false }}`` + - ``{ :write_concern => { :j => true|false }}`` * - localThresholdMS=Integer - ``:local_threshold => Float`` @@ -217,13 +217,13 @@ URI Options Conversions is inverted before being used to set ``ssl_verify``. * - w=Integer|String - - ``{ :write => { :w => Integer|String }}`` + - ``{ :write_concern => { :w => Integer|String }}`` * - waitQueueTimeoutMS=Integer - ``:wait_queue_timeout => Float`` * - wtimeoutMS=Integer - - ``{ :write => { :wtimeout => Float }}`` + - ``{ :write_concern => { :wtimeout => Float }}`` * - zlibCompressionLevel=Integer - ``:zlib_compression_level => Integer`` @@ -543,15 +543,22 @@ Ruby Options - 1 * - ``:write`` + - Deprecated. Equivalent to ``:write_concern`` option. If both ``:write`` + and ``:write_concern`` are specified, their values must be identical. + + - ``Hash`` + - ``{ w: 1 }`` + + * - ``:write_concern`` - Specifies write concern options as a ``Hash``. Keys in the hash can be ``:w``, ``:wtimeout``, ``:j``, ``:fsync``. .. code-block:: ruby - { :write => { :w => 2 } } + { write_concern: { w: 2 } } - ``Hash`` - - ``{ :w => 1 }`` + - ``{ w: 1 }`` * - ``:zlib_compression_level`` - The Zlib compression level to use, if using compression. See Ruby's Zlib module for valid levels. diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index fbdeaa5da..c18432ebd 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -444,18 +444,6 @@ the modification occurs. ) doc # Return the document after the update. -Write Concern -~~~~~~~~~~~~~ - -Write concern can be given for specific operations on a collection -using the ``with`` method, similarly to read preference: - -.. code-block:: ruby - - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - artists = client[:artists] - artists.with(:write => { :w => :3 }).insert_one( { :name => 'Depeche Mode' } ) - Deleting -------- @@ -482,6 +470,177 @@ Deleting result = artists.delete_many(:label => 'Mute') result.deleted_count # Returns the number deleted. +.. _write-concern: + +Write Concern +------------- + +All write operations in MongoDB are executed with a write concern which is +the level of acknowledgment requested from MongoDB for the particular write. +More information about write concerns in general is available in the +`MongoDB manual `_. + +The Ruby driver supports specifying write concern on client, collection, +session (for transactions on that session), transaction, GridFS bucket +and write stream levels, as well as when manually issuing commands via +``Database#command``. + +As of driver version 2.10, all driver objects accepting write concerns do so +through the ``:write_concern`` option, which should be given a hash with +the write concern options. Usage of the ``:write`` option is deprecated. +In driver versions 2.9 and below, client, collection and GridFS objects +took write concern options in the ``:write`` option with session and +transaction objects employing the ``:write_concern`` option. + +Below are some examples of passing write concerns to client and colection +objects. The ``:write_concern`` option can be provided when constructing +new client and collection objects, or to the ``#with`` methods. + +GridFS examples are provided on the :ref:`GridFS ` page. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', + write_concern: {w: 2}) + alt_client = client.with(write_concern: {w: :majority}) + + collection = client[:artists, write_concern: {w: 3}] + alt_collection = collection.with(write_concern: {w: :majority}) + + # Uses w: 3 + collection.insert_one({name: 'SUN Project'}) + # Uses w: :majority + alt_collection.insert_one({name: 'SUN Project'}) + +Driver versions 2.9 and earlier accepted write concerns on client and collection +level via the ``:write`` option. This usage continues to be supported for +backwards compatibility, but is deprecated: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', + write: {w: 2}) + alt_client = client.with(write: {w: :majority}) + + collection = client[:artists, write: {w: 3}] + alt_collection = collection.with(write: {w: :majority}) + +If both ``:write`` and ``:write_concern`` options are provided, their +values must be identical or an exception will be raised: + +.. code-block:: ruby + + # OK + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', + write_concern: {w: 3}, write: {w: 3}) + + # Error + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', + write_concern: {w: 3}, write: {w: :majority}) + +When ``#with`` methods are used to alter the options on a client or collection, +the last provided option wins in case of naming differences: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', + write_concern: {w: 2}) + alt_client = client.with(write: {w: 3}) + + alt_client.options[:write] + # => {"w"=>3} + + alt_client.options[:write_concern] + # => nil + +When using transactions, write concern is only sent to the server in +``commit_transaction`` and ``abort_transaction`` operations +per the `transactions specification +`_. +Write concern may be set via the ``:write_concern`` option in a +``with_transaction`` or ``start_transaction`` call, or via +``default_transaction_options`` option on a session object. +If neither of these is set, write concern of the client is used; note +that transactions ignore write concerns of collections that are involved +in their operations. Note that when setting the write concern as a +transaction option, the ``:write`` option is not recognized by any +driver version. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', + write_concern: {w: 2}) + collection = client[:artists, write_concern: {w: :majority}] + + + session = client.start_session + session.with_transaction do + collection.insert_one({test: 1}, session: session) + + # Uses w: 2 when committing + end + + + session = client.start_session(default_transaction_options: + {write_concern: {w: 3}) + ) + session.with_transaction do + collection.insert_one({test: 1}, session: session) + + # Uses w: 3 when committing + end + + + session = client.start_session + session.with_transaction(write_concern: {w: 3}) do + collection.insert_one({test: 1}, session: session) + + # Uses w: 3 when committing + end + +When write concerns are inherited, inheritance applies to the entire +write concern hash rather than individual elements. For example, ``j: true`` +is not inherited in the following case: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', + write_concern: {w: 1, j: true}) + collection = client[:artists, write_concern: {w: 2}] + + collection.write_concern.options + # => #2}> + +Although CRUD operations accept an options hash, they currently do not +recognize the ``:write_concern`` option: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', + write_concern: {w: 2}) + collection = client[:artists, write_concern: {w: :majority}] + + # Still uses w: :majority + collection.insert_one({name: 'SUN Project'}, write_concern: {w: 1}) + +The easiest workaround for this is to use ``#with`` to obtain a new collection +instance with the desired write concern: + +.. code-block:: ruby + + # Uses w: 1 + collection.with(write_concern: {w: 1}).insert_one(name: 'SUN Project') + +Write concern can also be manually specified in ``Database#command``: + +.. code-block:: ruby + + client.database.command(create: 'foo-collection', writeConcern: {w: :majority}) + +Note that writeConcern here is part of the operation rather than options, +and the syntax is the camel case one that MongoDB server recognizes, not the +underscore one that Ruby driver uses. + A Note about the BSON Symbol type --------------------------------- diff --git a/source/tutorials/ruby-driver-gridfs.txt b/source/tutorials/ruby-driver-gridfs.txt index db048d881..ceba2dc98 100644 --- a/source/tutorials/ruby-driver-gridfs.txt +++ b/source/tutorials/ruby-driver-gridfs.txt @@ -1,3 +1,5 @@ +.. _gridfs: + ====== GridFS ====== @@ -31,13 +33,18 @@ The options that ``Grid::FSBucket`` supports are: * - ``:bucket_name`` - The name of the GridFS Bucket. Default is ``fs``. * - ``:fs_name`` - - The name of the GridFS Bucket. Takes precedence over ``bucket_name``. Default is ``fs``. + - The name of the GridFS Bucket. Takes precedence over ``bucket_name``. + Default is ``fs``. * - ``:chunk_size`` - Specifies the size of each file chunk in the database. + * - ``:write_concern`` + - The write concern to use when uploading files. Please see the + :ref:`Write Concern ` section under CRUD operations + for how to work with write concerns. * - ``:write`` - - The write concern used when uploading files. + - Deprecated. Same as ``:write_concern``. * - ``:read`` - - The read preference used when downloading files. + - The read preference to use when downloading files. For example, you can create a GridFS bucket object with a particular read preference: @@ -50,74 +57,119 @@ For example, you can create a GridFS bucket object with a particular read prefer Working with write streams -------------------------- -To upload a file to GridFS using a write stream, you can either open a stream and write to it directly or write the -entire contents of an IO object to GridFS all at once. +To upload a file to GridFS using a write stream, you can either open a stream +and write to it directly or write the entire contents of an ``IO`` object to +GridFS all at once. To open an upload stream and write to it: .. code-block:: ruby - file = File.open('/path/to/my-file.txt', 'r') - fs_bucket.open_upload_stream('my-file.txt') do |stream| - stream.write(file) + File.open('/path/to/my-file.txt', 'r') do |file| + fs_bucket.open_upload_stream('my-file.txt') do |stream| + stream.write(file) + end end - file.close To upload the entire contents of an IO object in one call: .. code-block:: ruby - file = File.open('/path/to/my-file.txt', 'r') - fs_bucket.upload_from_stream('my-file.txt', file) - file.close + File.open('/path/to/my-file.txt', 'r') do |file| + fs_bucket.upload_from_stream('my-file.txt', file) + end + +Write streams support the following options: + +.. list-table:: + :header-rows: 1 + :widths: 40 80 + + * - Option + - Description + * - ``:chunk_size`` + - Specifies the size of each file chunk in the database. + * - ``:write_concern`` + - The write concern to use when uploading files. Please see the + :ref:`Write Concern ` section under CRUD operations + for how to work with write concerns. + * - ``:write`` + - Deprecated. Same as ``:write_concern``. + +The options can be provided as the last argument to the write stream methods: + +.. code-block:: ruby + + fs_bucket.open_upload_stream('my-file.txt', write_concern: {w: 2}) do |stream| + stream.write_concern + # => #2}> + + # ... + end + + fs_bucket.upload_from_stream('my-file.txt', file, write_concern: {w: 2}) Working with read streams ------------------------- -To download a file from GridFS using a read stream, you can either open a read stream and read from it directly or -download the entire file all at once. +To download a file from GridFS using a read stream, you can either open a +read stream and read from it directly or download the entire file all at once. To open a download stream and read from it: .. code-block:: ruby - file = File.open('/path/to/my-output-file.txt', 'w') - fs_bucket.open_download_stream(file_id) do |stream| - file.write(stream.read) + File.open('/path/to/my-output-file.txt', 'w') do |file| + fs_bucket.open_download_stream(file_id) do |stream| + file.write(stream.read) + end end - file.close To download the file all at once and write it to an IO object: .. code-block:: ruby - file = File.open('/path/to/my-output-file.txt', 'w') - fs_bucket.download_from_stream(file_id, file) - file.close + File.open('/path/to/my-output-file.txt', 'w') do |file| + fs_bucket.download_from_stream(file_id, file) + end You can also download a file specified by a name and (optionally) -revision number. Revision numbers are used to distinguish -between files sharing the same name, ordered by date of upload. The revision number passed to +revision number. Revision numbers are used to distinguish between files +sharing the same name, ordered by date of upload. The revision number passed to ``open_download_stream_by_name`` can be positive or negative. .. code-block:: ruby - file = File.open('/path/to/my-output-file.txt', 'w') - fs_bucket.open_download_stream_by_name('my-file.txt', revision: -2) do |stream| - file.write(stream.read) + File.open('/path/to/my-output-file.txt', 'w') do |file| + fs_bucket.open_download_stream_by_name('my-file.txt', revision: -2) do |stream| + file.write(stream.read) + end end - file.close To download the entire contents of the file specified by name and (optionally) revision number: .. code-block:: ruby - file = File.open('/path/to/my-output-file.txt', 'w') - fs_bucket.download_to_stream_by_name('my-file.txt', file, revision: -2) - file.close + File.open('/path/to/my-output-file.txt', 'w') do |file| + fs_bucket.download_to_stream_by_name('my-file.txt', file, revision: -2) + end + +Read streams support the following options: + +.. list-table:: + :header-rows: 1 + :widths: 40 80 + + * - Option + - Description + * - ``:read`` + - The read preference to use when downloading files. +Some, but not all, of the read methods listed above pass these options to +the underlying read streams. Please consult the API documentation for each +method to determine whether it supports a particular option. Finding file metadata --------------------- @@ -141,7 +193,8 @@ You can delete a file by id. Working with Grid::File objects ------------------------------- -This object can be used to wrap a file to be inserted into the database using GridFS and the object that is retrieved. +This object can be used to wrap a file to be inserted into the database using +GridFS and the object that is retrieved. To create a file with raw data: @@ -256,6 +309,3 @@ To delete a file, pass the file object to ``delete_one``. fs = client.database.fs file = fs.find_one(:filename => 'new-file.txt') fs.delete_one(file) - - - coll.find.to_a # { '_id' => ..., 'x' => 3 } From d8b659ca44de3d1ee6fd4c498b767b104ae77f7e Mon Sep 17 00:00:00 2001 From: Nathan Date: Tue, 16 Jul 2019 16:50:44 -0400 Subject: [PATCH 209/442] updates mongodb compatibility matrix (#1444) --- source/reference/driver-compatibility.txt | 25 +++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index ead86b092..cc110aaa6 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -19,6 +19,7 @@ MongoDB Compatibility :class: compatibility-large no-padding * - Ruby Driver + - MongoDB 4.2 - MongoDB 4.0 - MongoDB 3.6 - MongoDB 3.4 @@ -27,7 +28,18 @@ MongoDB Compatibility - MongoDB 2.6 - MongoDB 2.4 + * - 2.10 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + * - 2.9 + - - |checkmark| - |checkmark| - |checkmark| @@ -37,6 +49,7 @@ MongoDB Compatibility - * - 2.8 + - - |checkmark| - |checkmark| - |checkmark| @@ -46,6 +59,7 @@ MongoDB Compatibility - * - 2.7 + - - |checkmark| - |checkmark| - |checkmark| @@ -55,6 +69,7 @@ MongoDB Compatibility - * - 2.6 + - - |checkmark| - |checkmark| - |checkmark| @@ -64,6 +79,7 @@ MongoDB Compatibility - * - 2.5 + - - - |checkmark| - |checkmark| @@ -73,6 +89,7 @@ MongoDB Compatibility - * - 2.4 + - - - - |checkmark| @@ -85,6 +102,7 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -94,6 +112,7 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -104,6 +123,7 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -113,6 +133,7 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -123,6 +144,7 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| @@ -132,6 +154,7 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| @@ -142,6 +165,7 @@ MongoDB Compatibility - - - + - - |checkmark| * - 1.8 @@ -151,6 +175,7 @@ MongoDB Compatibility - - - + - - |checkmark| .. include:: /includes/older-server-versions-unsupported.rst From 501d749f06d170c259d587ab450ebf8873695de6 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 18 Jul 2019 12:16:18 -0400 Subject: [PATCH 210/442] RUBY-1498 Resync CRUD v1 tests for new bulkWrite() tests (#1450) * RUBY-1498 sync crud v1 write spec tests * revise language * Format remaining transforms consistently * RUBY-1498 mpass all arguments to bulk write operations --- source/tutorials/ruby-driver-collations.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-collations.txt b/source/tutorials/ruby-driver-collations.txt index 5a752b7ad..70d299241 100644 --- a/source/tutorials/ruby-driver-collations.txt +++ b/source/tutorials/ruby-driver-collations.txt @@ -15,8 +15,8 @@ Overview .. versionadded:: 3.4 -Collations provide a set of rules which comply with the conventions -of a particular language when comparing strings. +Collations are sets of rules for how to compare strings, typically in a +particular natural language. For example, in Canadian French, the last accent in a given word determines the sorting order. From 98bacd3193de610216417bb0f65790cf157ccefe Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 22 Jul 2019 16:52:19 -0400 Subject: [PATCH 211/442] RUBY-1761 URI option fixes (#1446) * RUBY-1881 fix double unescaping of replica set uri option * RUBY-1881 fix double unescaping of auth source uri option * RUBY-1763 pass through unknown values of read preference modes * RUBY-1761 accept all cases for canonicalize host name * RUBY-1882 use standard integer parser for pool size uri options * RUBY-1882 use standard bool parser * RUBY-1882 use one millisecond converter * RUBY-1882 use standard integer converter for wtimeout * RUBY-1882 extract a common inverse bool converter * RUBY-1761 reference local decode helper * RUBY-1881 fix double unescaping of read preference tags * RUBY-1761 reduce line length * RUBY-1882 dry tls/ssl option handling * RUBY-1764 do not warn on max staleness value of -1 * RUBY-1551 test empty authSource uri option * RUBY-1883 allow uri options string to begin with & * RUBY-1762 fix max_staleness documentation in URI options * RUBY-1756 document auth mechanism value conversions and pass unrecognized values to fail later * RUBY-1762 clarify max staleness URI option language --- .../tutorials/ruby-driver-create-client.txt | 39 ++++++++++++++++--- 1 file changed, 34 insertions(+), 5 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index e72241b42..bff7e48a6 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -103,6 +103,22 @@ URI Options Conversions * - authMechanism=String - ``:auth_mech => Symbol`` + + Auth mechanism values are converted as follows from URI options to + Ruby options: + + - ``GSSAPI`` => ``:gssapi`` + - ``MONGODB-CR`` => ``:mongodb_cr`` + - ``MONGODB-X509`` => ``:mongodb_x509`` + - ``PLAIN`` => ``:plain`` + - ``SCRAM-SHA-1`` => ``:scram`` + - ``SCRAM-SHA-256`` => ``:scram256`` + + If a different value is provided for auth mechanism, it is converted + to the Ruby option unmodified and retains its ``String`` type. + Note that, while currently the driver allows a ``Client`` instance + to be constructed with an unrecognized auth mechanism, this behavior + `may change in a future version of the driver `_. * - authMechanismProperties=Strings - ``{ :auth_mech_properties => { :service_realm => String, :canonicalize_host_name => true|false, :service_name => String } }`` @@ -144,7 +160,18 @@ URI Options Conversions - ``:max_idle_time => Float`` * - maxStalenessSeconds=Integer - - ``:max_staleness => Float`` + - ``{ :read => { :max_staleness => Integer }}`` + + If the maxStalenessSeconds URI option value is -1, the driver treats + this as if the option was not given at all. Otherwise, + if the option value is numeric, the Ruby option is set to the + specified value converted to an ``Integer``. + Note that numeric values greater than 0 but less than 90, or less than + -1, are accepted by the ``Client`` constructor but will cause server + selection to fail (unless the option is changed via, for example, the + ``with`` method prior to any operations being performed on the driver). + If the option value is non-numeric, it is ignored and the driver + treats this case as if the option was not given at all. * - maxPoolSize=Integer - ``:max_pool_size => Integer`` @@ -223,7 +250,7 @@ URI Options Conversions - ``:wait_queue_timeout => Float`` * - wtimeoutMS=Integer - - ``{ :write_concern => { :wtimeout => Float }}`` + - ``{ :write_concern => { :wtimeout => Integer }}`` * - zlibCompressionLevel=Integer - ``:zlib_compression_level => Integer`` @@ -369,14 +396,16 @@ Ruby Options - none * - ``:read`` - - Specifies the read preference mode and tag sets for selecting servers as a ``Hash``. - Keys in the hash are ``:mode`` and ``:tag_sets``. + - Specifies the read preference mode and tag sets for selecting servers + as a ``Hash``. Allowed Keys in the hash are ``:mode``, ``:tag_sets`` and + ``:max_staleness``. .. code-block:: ruby { :read => { :mode => :secondary, - :tag_sets => [ "berlin" ] + :tag_sets => [ "berlin" ], + :max_staleness => 5, } } From f73d0635fbfcd987ac4e9ffaf4a3e451748c5958 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Tue, 6 Aug 2019 17:46:47 -0400 Subject: [PATCH 212/442] Add 2.10 to driver compatibility Ruby table --- source/reference/driver-compatibility.txt | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index cc110aaa6..e524aae0b 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -206,6 +206,20 @@ Ruby Compatibility - JRuby 9.1 - JRuby + * - 2.10 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - D + - D + - D + - D + - + - |checkmark| + - |checkmark| + - + * - 2.9 - |checkmark| - |checkmark| From 38bace453246f75fef9c78f9e5c8fad1603bc4e2 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 15 Aug 2019 21:40:51 -0400 Subject: [PATCH 213/442] RUBY-1622 Require Ruby 2.3+ in driver (#1492) --- .../older-server-versions-unsupported.rst | 1 - ...y-driver-compatibility-matrix-language.rst | 6 - source/reference/driver-compatibility.txt | 142 ++++-------------- 3 files changed, 31 insertions(+), 118 deletions(-) delete mode 100644 source/includes/older-server-versions-unsupported.rst delete mode 100644 source/includes/ruby-driver-compatibility-matrix-language.rst diff --git a/source/includes/older-server-versions-unsupported.rst b/source/includes/older-server-versions-unsupported.rst deleted file mode 100644 index d99330fcf..000000000 --- a/source/includes/older-server-versions-unsupported.rst +++ /dev/null @@ -1 +0,0 @@ -The driver does not support older versions of MongoDB. \ No newline at end of file diff --git a/source/includes/ruby-driver-compatibility-matrix-language.rst b/source/includes/ruby-driver-compatibility-matrix-language.rst deleted file mode 100644 index a19182866..000000000 --- a/source/includes/ruby-driver-compatibility-matrix-language.rst +++ /dev/null @@ -1,6 +0,0 @@ -The following compatibility table specifies the recommended -version(s) of the MongoDB Ruby driver for use with a specific version of -Ruby. - -The first column lists the driver versions. - diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index e524aae0b..5859fd883 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -28,6 +28,16 @@ MongoDB Compatibility - MongoDB 2.6 - MongoDB 2.4 + * - 2.11 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + * - 2.10 - |checkmark| - |checkmark| @@ -128,64 +138,19 @@ MongoDB Compatibility - |checkmark| - |checkmark| - * - 1.12 - - - - - - - - - - - - |checkmark| - - |checkmark| - - |checkmark| - - * - 1.11 - - - - - - - - - - - - - - |checkmark| - - |checkmark| - - * - 1.10 - - - - - - - - - - - - - - |checkmark| - - |checkmark| - - * - 1.9 - - - - - - - - - - - - - - - - |checkmark| - - * - 1.8 - - - - - - - - - - - - - - - - |checkmark| - -.. include:: /includes/older-server-versions-unsupported.rst +The driver does not support older versions of MongoDB. .. _reference-compatibility-language-ruby: Ruby Compatibility ~~~~~~~~~~~~~~~~~~ -.. include:: /includes/ruby-driver-compatibility-matrix-language.rst +The following compatibility table specifies the recommended +version(s) of the MongoDB Ruby driver for use with a specific version of +Ruby. + +The first column lists the driver versions. "D" in a column means support +for that Ruby version is deprecated. .. list-table:: :header-rows: 1 @@ -201,11 +166,23 @@ Ruby Compatibility - Ruby 2.1 - Ruby 2.0 - Ruby 1.9 - - Ruby 1.8.7 - JRuby 9.2 - JRuby 9.1 - JRuby + * - 2.11 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + - + - + - + - |checkmark| + - |checkmark| + - + * - 2.10 - |checkmark| - |checkmark| @@ -215,7 +192,6 @@ Ruby Compatibility - D - D - D - - - |checkmark| - |checkmark| - @@ -229,7 +205,6 @@ Ruby Compatibility - D - D - D - - - |checkmark| - |checkmark| - @@ -243,7 +218,6 @@ Ruby Compatibility - |checkmark| - |checkmark| - |checkmark| - - - |checkmark| - |checkmark| - @@ -257,7 +231,6 @@ Ruby Compatibility - |checkmark| - |checkmark| - |checkmark| - - - |checkmark| - |checkmark| - @@ -271,7 +244,6 @@ Ruby Compatibility - |checkmark| - |checkmark| - |checkmark| - - - |checkmark| - |checkmark| - @@ -287,7 +259,6 @@ Ruby Compatibility - |checkmark| - - - - - |checkmark| * - 2.4 @@ -301,7 +272,6 @@ Ruby Compatibility - |checkmark| - - - - - |checkmark| * - 2.3 @@ -315,7 +285,6 @@ Ruby Compatibility - |checkmark| - - - - - |checkmark| * - 2.2 @@ -329,7 +298,6 @@ Ruby Compatibility - |checkmark| - - - - - |checkmark| * - 2.1 @@ -343,7 +311,6 @@ Ruby Compatibility - |checkmark| - - - - - |checkmark| * - 2.0 @@ -357,56 +324,9 @@ Ruby Compatibility - |checkmark| - - - - - - |checkmark| - - * - 1.12 - 1.9 - - - - - - - - - - - - |checkmark| - - |checkmark| - - |checkmark| - - |checkmark| - - - - - - |checkmark| - - * - 1.8 - - - - - - - - - - - - - - |checkmark| - - |checkmark| - - |checkmark| - - - - - - |checkmark| - - * - 1.7 - 1.6 - - - - - - - - - - - - - - - - |checkmark| - - |checkmark| - - - - - |checkmark| -Support for Ruby versions less than 2.3 has been deprecated. Driver version -2.10 will be the last release to support Ruby 1.9-2.2. As of driver version -2.11.0, the minimum required Ruby version will be 2.3. Please note that, -as of January 1, 2019, Ruby versions 2.2 and older are `not supported by -the Ruby core team `_. +The driver does not support older versions of Ruby. Rails/ActiveSupport Compatibility ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 0b373d91e41dd99ae7f35a143881d7cb8ecfdb3d Mon Sep 17 00:00:00 2001 From: Nathan Date: Tue, 20 Aug 2019 13:36:26 -0400 Subject: [PATCH 214/442] DOCS-10373 fix quotes (#122) --- docs/tutorials/bson-v4.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 5ccc00519..4ed550f64 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -248,7 +248,7 @@ them. - ``{ "$binary" : "\x01", "$type" : "md5" }`` * - ``BSON::Code`` - - ``{ "$code" : "this.v = 5 }`` + - ``{ "$code" : "this.v = 5" }`` * - ``BSON::CodeWithScope`` - ``{ "$code" : "this.v = value", "$scope" : { v => 5 }}`` From daa13784b0a11d59b8f2c297ddb99115601418a5 Mon Sep 17 00:00:00 2001 From: Nathan Date: Fri, 23 Aug 2019 19:05:51 -0400 Subject: [PATCH 215/442] adds examples for nested doc query (#1502) --- source/quick-start.txt | 111 +++++++++++------- .../tutorials/ruby-driver-crud-operations.txt | 11 ++ 2 files changed, 80 insertions(+), 42 deletions(-) diff --git a/source/quick-start.txt b/source/quick-start.txt index c8c73eb64..f9635b1ce 100644 --- a/source/quick-start.txt +++ b/source/quick-start.txt @@ -9,7 +9,7 @@ Quick Start :backlinks: none :depth: 1 :class: singlecol - + Prerequisites ------------- @@ -21,7 +21,7 @@ Prerequisites .. code-block:: ruby require 'mongo' - + Make a Connection ----------------- @@ -32,18 +32,18 @@ instance. .. code-block:: ruby client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') - + You can also use a URI connection string: .. code-block:: ruby client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') - + .. seealso:: :ref:`Connect to a replica set `, :ref:`Connect to a sharded cluster `, :ref:`Client options ` - + Access a Database and a Collection ---------------------------------- @@ -54,7 +54,7 @@ and show its collections: client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test') db = client.database - + db.collections # returns a list of collection objects db.collection_names # returns a list of collection names @@ -66,7 +66,7 @@ To access a collection, refer to it by name. If the collection does not exist, the server will create it the first time you put data into it. - + Insert a Document ----------------- @@ -78,23 +78,34 @@ To insert a single document into a collection, use the client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') collection = client[:people] - - doc = { name: 'Steve', hobbies: [ 'hiking', 'tennis', 'fly fishing' ] } - + + doc = { + name: 'Steve', + hobbies: [ 'hiking', 'tennis', 'fly fishing' ], + siblings: { + brothers: 0, + sisters: 1 + } + } + result = collection.insert_one(doc) result.n # returns 1, because one document was inserted - + To insert multiple documents into a collection, use the ``insert_many`` method. - + .. code-block:: ruby - - docs = [ { _id: 1, name: 'Steve', hobbies: [ 'hiking', 'tennis', 'fly fishing' ] }, - { _id: 2, name: 'Sally', hobbies: ['skiing', 'stamp collecting' ] } ] - + + docs = [ { _id: 1, name: 'Steve', + hobbies: [ 'hiking', 'tennis', 'fly fishing' ], + siblings: { brothers: 0, sisters: 1 } }, + { _id: 2, name: 'Sally', + hobbies: ['skiing', 'stamp collecting' ], + siblings: { brothers: 1, sisters: 0 } } ] + result = collection.insert_many(docs) result.inserted_count # returns 2 because two documents were inserted - + Query the Collection -------------------- @@ -106,26 +117,42 @@ An empty query filter returns all documents in the collection. client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') collection = client[:people] - + collection.find.each do |document| #=> Yields a BSON::Document. end - + Use a query filter to find only matching documents. .. code-block:: ruby client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') collection = client[:people] - + puts collection.find( { name: 'Sally' } ).first - + +The example should print the following: + +.. code-block:: javascript + + {"_id" => 2, "name" => "Sally", "hobbies" => ["skiing", "stamp collecting"], "siblings" => { "brothers": 1, "sisters": 0 } } + +Query nested documents by specifying the keys and values you want +to match. + +.. code-block:: ruby + + client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') + collection = client[:people] + + puts collection.find("siblings.sisters": 1 ).first + The example should print the following: .. code-block:: javascript - {"_id" => 2, "name" => "Sally", "hobbies" => ["skiing", "stamp collecting"]} - + {"_id"=>1, "name"=>"Steve", "hobbies"=>["hiking", "tennis", "fly fishing"], "siblings"=>{"brothers"=>0, "sisters"=>1}} + .. seealso:: :ref:`Query Options`, :ref:`Read Preference` @@ -146,17 +173,17 @@ document is replaced with the update data. client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') collection = client[:people] - + result = collection.update_one( { 'name' => 'Sally' }, { '$set' => { 'phone_number' => "555-555-5555" } } ) - + puts collection.find( { 'name' => 'Sally' } ).first - + The example should print the following: .. code-block:: javascript {"_id" => 2, "name" => "Sally", "hobbies" => ["skiing", "stamp collecting"], "phone_number" => "555-555-5555"} - + The following example uses ``update_many`` with a blank query filter to update all the documents in the collection. @@ -164,15 +191,15 @@ to update all the documents in the collection. client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') collection = client[:people] - + result = collection.update_many( {}, { '$set' => { 'age' => 36 } } ) - + puts result.modified_count # returns 2 because 2 documents were updated - + .. seealso:: - + :ref:`Other update options` - + Delete Documents ---------------- @@ -183,11 +210,11 @@ from a collection (either singly or several at once). client = Mongo::Client.new('mongodb://127.0.0.1:27017/test') collection = client[:people] - + result = collection.delete_one( { name: 'Steve' } ) - + puts result.deleted_count # returns 1 because one document was deleted - + The following example inserts two more records into the collection, then deletes all the documents with a ``name`` field which matches a regular expression to find a string which begins with "S". @@ -198,13 +225,13 @@ matches a regular expression to find a string which begins with "S". collection = client[:people] collection.insert_many([ { _id: 3, name: "Arnold" }, { _id: 4, name: "Susan" } ]) - + puts collection.count # counts all documents in collection - - result = collection.delete_many({ name: /$S*/ }) - + + result = collection.delete_many({ name: /$S*/ }) + puts result.deleted_count # returns the number of documents deleted - + Create Indexes -------------- @@ -217,7 +244,7 @@ singly or several at once. collection = client[:people] collection.indexes.create_one({ name: 1 }, unique: true) - + Use the ``create_many`` method to create several indexes with one statement. Note that when using ``create_many``, the syntax is different from ``create_one``. @@ -228,7 +255,7 @@ different from ``create_one``. collection = client[:people] collection.indexes.create_many([ - { key: { name: 1 } , unique: true }, + { key: { name: 1 } , unique: true }, { key: { hobbies: 1 } }, ]) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index c18432ebd..61a2e8516 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -152,6 +152,17 @@ query: #=> Yields a BSON::Document. end +To query nested documents, specify the keys in nested order using dot +notation. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client[:artists].find("records.releaseYear": 2008).each do |document| + #=> Yields a BSON::Document. + end + + .. _ruby-driver-query-options: Query Options From 869f6a6ed0c2a4e87222c7d6c0c6faa088431f6a Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 27 Aug 2019 16:53:10 -0400 Subject: [PATCH 216/442] RUBY-1899 Drop JRuby 9.1 support (#1504) --- source/reference/driver-compatibility.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 5859fd883..f703db4d4 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -180,7 +180,7 @@ for that Ruby version is deprecated. - - - |checkmark| - - |checkmark| + - - * - 2.10 From bad665e381ce69932774a265178146d2715cd5e0 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 29 Aug 2019 16:42:07 -0400 Subject: [PATCH 217/442] RUBY-1605 update tutorial for background connection pool implementation (#1511) --- .../tutorials/ruby-driver-create-client.txt | 29 ++++++++++++------- 1 file changed, 18 insertions(+), 11 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index bff7e48a6..101258115 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -766,10 +766,17 @@ is configurable. This setting, called ``wait_queue_timeout``, is defined in seconds. If this timeout is reached, a ``Timeout::Error`` is raised. The default is 1 second. -The driver currently does not eagerly create connections, regardless of -``min_pool_size`` setting. However, once a connection is established, it will -be kept in the pool by the driver as long as the pool size does not exceed -``min_pool_size``. +As of driver version 2.11, the driver eagerly creates connections up to +``min_pool_size`` setting. Prior to driver version 2.11, the driver always +created connections on demand. In all versions of the driver, once a connection +is established, it will be kept in the pool by the driver as long as the pool +size does not exceed ``min_pool_size``. + +Note that, if ``min_pool_size`` is set to a value greater than zero, the +driver will establish that many connections to secondaries in replica set +deployments even if the application does not perform secondary reads. The +purpose of these connections is to provide faster failover when the primary +changes. Here is an example of estimating the number of connections a multi-threaded application will open: A client connected to a 3-node replica set opens 3 @@ -1057,13 +1064,13 @@ Production Configuration Please consider the following when deploying an application using the Ruby driver in production: -- The driver has a setting for minimum connection pool size. - This setting results in the driver creating the specified number of - connection objects but each connection object creates sockets and - actually connects to the server on demand. Setting a minimum connection - pool size will keep up to that many network connections to a server - alive once they are established. This behavior may change in a future - version of the driver (`RUBY-1605 `_). +- As of driver version 2.11, the ``:min_pool_size`` client option is completely + respected - the driver will create that many connections to each server + identified as a standalone, primary or secondary. In previous driver versions + the driver created connections on demand. Applications using ``:min_pool_size`` + will see an increase in the number of idle connections to all servers as of + driver version 2.11, and especially to secondaries in replica set deployments + and to nodes in sharded clusters. - If the application is reverse proxied to by another web server or a load balancer, ``server_selection_timeout`` should generally be set to a lower value than the reverse proxy's read timeout. For exampe, `Heroku request timeout From fc850298147b999cab90568bc61b94576f220f31 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 3 Sep 2019 11:41:43 -0400 Subject: [PATCH 218/442] RUBY-1906 Update/revise driver timeout documentation (#1513) * Switch the order of Ruby and URI option sections * Note second is the default unit * Revise timeout options * Use headings and sections over blockquotes --- .../tutorials/ruby-driver-create-client.txt | 485 ++++++++++-------- 1 file changed, 260 insertions(+), 225 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 101258115..c438e073a 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -7,13 +7,13 @@ Creating a Client .. contents:: On this page :local: :backlinks: none - :depth: 1 + :depth: 2 :class: singlecol Using ``Mongo::Client`` ----------------------- -To connect to a MongoDB cluster, create a ``Mongo::Client`` object. +To connect to a MongoDB deployment, create a ``Mongo::Client`` object. Provide a list of hosts and options or a connection URI to the ``Mongo::Client`` constructor. The client's selected database defaults to ``admin``. @@ -65,199 +65,27 @@ In this protocol, the comma separated list of host names is replaced with a sing .. _ruby-driver-client-options: + Client Options -------------- -A number of different options can be passed to a ``Mongo::Client`` to configure driver -behavior, either by providing them in the options hash to the constructor or by -providing them in the URI. If both a URI and an options hash are provided, the options -in the hash take precedence over any analogous options present in the URI. +``Mongo::Client``'s constructor accepts a number of options configuring the +behavior of the driver. The options can be provided in the options hash as +Ruby options, in the URI as URI options, or both. If both a Ruby option and +the analogous URI option are provided, the Ruby option takes precedence. + -Since the URI options are required in camel case, which is not the Ruby standard, the -following table shows the option in the URI and its corresponding option if passed -to the constructor in Ruby. +Ruby Options +```````````` .. note:: The options passed directly should be symbols. -The options are explained in detail in the :manual:`Connection URI reference -`. - .. note:: - Options that are set in **milliseconds** in the URI are - represented as a ``float`` in Ruby and the units are **seconds**. - -URI Options Conversions ------------------------ - -.. list-table:: - :header-rows: 1 - :widths: 40 105 - - * - URI Option - - Ruby Option - - * - appName=String - - ``:app_name => String`` - - * - authMechanism=String - - ``:auth_mech => Symbol`` - - Auth mechanism values are converted as follows from URI options to - Ruby options: - - - ``GSSAPI`` => ``:gssapi`` - - ``MONGODB-CR`` => ``:mongodb_cr`` - - ``MONGODB-X509`` => ``:mongodb_x509`` - - ``PLAIN`` => ``:plain`` - - ``SCRAM-SHA-1`` => ``:scram`` - - ``SCRAM-SHA-256`` => ``:scram256`` - - If a different value is provided for auth mechanism, it is converted - to the Ruby option unmodified and retains its ``String`` type. - Note that, while currently the driver allows a ``Client`` instance - to be constructed with an unrecognized auth mechanism, this behavior - `may change in a future version of the driver `_. - - * - authMechanismProperties=Strings - - ``{ :auth_mech_properties => { :service_realm => String, :canonicalize_host_name => true|false, :service_name => String } }`` - - Specified as comma-separated key:value pairs, e.g. ``"SERVICE_REALM:foo,CANONICALIZE_HOST_NAME:TRUE"``. - - * - authSource=String - - ``:auth_source => String`` - - * - compressors=Strings - - ``:compressors => Array`` - - Specified as a comma-separated list. Note that the Ruby driver only supports zlib - compression; however, other drivers may support snappy. For maximum compatibility with - drivers, specify ``"snappy,zlib"``; if compatibility with other drivers is not a concern, - specify ``"zlib".`` Compression is not enabled by default and when using MongoDB 4.0 and - earlier, so zlib compression must be manually enabled on the server in order for the Ruby - driver to compress wire protocol data. - - * - connect=String - - ``:connect => Symbol`` - - * - connectTimeoutMS=Integer - - ``:connect_timeout => Float`` - - * - fsync=Boolean - - ``{ :write_concern => { :fsync => true|false }}`` - - * - heartbeatFrequencyMS=Integer - - ``:heartbeat_frequency => Float`` - - * - journal=Boolean - - ``{ :write_concern => { :j => true|false }}`` - - * - localThresholdMS=Integer - - ``:local_threshold => Float`` - - * - maxIdleTimeMS=Integer - - ``:max_idle_time => Float`` - - * - maxStalenessSeconds=Integer - - ``{ :read => { :max_staleness => Integer }}`` - - If the maxStalenessSeconds URI option value is -1, the driver treats - this as if the option was not given at all. Otherwise, - if the option value is numeric, the Ruby option is set to the - specified value converted to an ``Integer``. - Note that numeric values greater than 0 but less than 90, or less than - -1, are accepted by the ``Client`` constructor but will cause server - selection to fail (unless the option is changed via, for example, the - ``with`` method prior to any operations being performed on the driver). - If the option value is non-numeric, it is ignored and the driver - treats this case as if the option was not given at all. - - * - maxPoolSize=Integer - - ``:max_pool_size => Integer`` - - * - minPoolSize=Integer - - ``:min_pool_size => Integer`` - - * - readConcernLevel=String - - ``:read_concern => Hash`` - - * - readPreference=String - - ``{ :read => { :mode => Symbol }}`` - - * - readPreferenceTags=Strings - - ``{ :read => { :tag_sets => Array }}`` - - Each instance of the readPreferenceTags field is a comma-separated key:value pair which will appear in the :tag_sets array in the order they are specified. For instance, ``"readPreferenceTags=dc:ny,rack:1&readPreferenceTags=dc:ny"`` will be converted to ``[ { 'dc' => 'ny', 'rack' => '1' }, { 'dc' => 'ny' }]``. - - * - replicaSet=String - - ``:replica_set => String`` - - * - retryWrites=Boolean - - ``:retry_writes => boolean`` - - * - serverSelectionTimeoutMS=Integer - - ``:server_selection_timeout => Float`` - - * - socketTimeoutMS=Integer - - ``:socket_timeout => Float`` - - * - ssl=Boolean - - ``:ssl => true|false`` - - * - tls=Boolean - - ``:ssl => boolean`` - - * - tlsAllowInvalidCertificates=Boolean - - ``:ssl_verify_certificate => boolean`` - - Because ``tlsAllowInvalidCertificates`` uses ``true`` to signify that verification - should be disabled and ``ssl_verify_certificate`` uses ``false`` to signify that - verification should be disabled, the boolean is inverted before being used to set - ``ssl_verify_certificate``. - - * - tlsAllowInvalidHostnames=Boolean - - ``:ssl_verify_hostname => boolean`` - - Because ``tlsAllowInvalidHostnames`` uses ``true`` to signify that verification - should be disabled and ``ssl_verify_hostname`` uses ``false`` to signify that - verification should be disabled, the boolean is inverted before being used to set - ``ssl_verify_hostname``. - - * - tlsCAFile=String - - ``:ssl_ca_cert => String`` - - * - tlsCertificateKeyFile=String - - ``:ssl_cert => String`` - - * - tlsCertificateKeyFile=String - - ``:ssl_key => String`` - - * - tlsCertificateKeyFilePassword=String - - ``:ssl_key_pass_phrase => String`` - - * - tlsInsecure=Boolean - - ``:ssl_verify => boolean`` - - Because tlsInsecure uses ``true`` to signify that verification should be disabled and - ``ssl_verify`` uses ``false`` to signify that verification should be disabled, the boolean - is inverted before being used to set ``ssl_verify``. - - * - w=Integer|String - - ``{ :write_concern => { :w => Integer|String }}`` - - * - waitQueueTimeoutMS=Integer - - ``:wait_queue_timeout => Float`` - - * - wtimeoutMS=Integer - - ``{ :write_concern => { :wtimeout => Integer }}`` - - * - zlibCompressionLevel=Integer - - ``:zlib_compression_level => Integer`` - -Ruby Options ------------- + Unless otherwise specified, Ruby options that deal with times are given in + seconds. .. list-table:: :header-rows: 1 @@ -594,58 +422,265 @@ Ruby Options - ``Integer`` - none + +URI Options +``````````` + +Since the URI options are required to be in camel case, which is not the Ruby +standard, the following table shows URI options and their corresponding Ruby +options. + +URI options are explained in detail in the :manual:`Connection URI reference +`. + +.. note:: + Options that are set in **milliseconds** in the URI are + represented as a ``float`` in Ruby and the units are **seconds**. + +.. list-table:: + :header-rows: 1 + :widths: 40 105 + + * - URI Option + - Ruby Option + + * - appName=String + - ``:app_name => String`` + + * - authMechanism=String + - ``:auth_mech => Symbol`` + + Auth mechanism values are converted as follows from URI options to + Ruby options: + + - ``GSSAPI`` => ``:gssapi`` + - ``MONGODB-CR`` => ``:mongodb_cr`` + - ``MONGODB-X509`` => ``:mongodb_x509`` + - ``PLAIN`` => ``:plain`` + - ``SCRAM-SHA-1`` => ``:scram`` + - ``SCRAM-SHA-256`` => ``:scram256`` + + If a different value is provided for auth mechanism, it is converted + to the Ruby option unmodified and retains its ``String`` type. + Note that, while currently the driver allows a ``Client`` instance + to be constructed with an unrecognized auth mechanism, this behavior + `may change in a future version of the driver `_. + + * - authMechanismProperties=Strings + - ``{ :auth_mech_properties => { :service_realm => String, :canonicalize_host_name => true|false, :service_name => String } }`` + + Specified as comma-separated key:value pairs, e.g. ``"SERVICE_REALM:foo,CANONICALIZE_HOST_NAME:TRUE"``. + + * - authSource=String + - ``:auth_source => String`` + + * - compressors=Strings + - ``:compressors => Array`` + + Specified as a comma-separated list. Note that the Ruby driver only supports zlib + compression; however, other drivers may support snappy. For maximum compatibility with + drivers, specify ``"snappy,zlib"``; if compatibility with other drivers is not a concern, + specify ``"zlib".`` Compression is not enabled by default and when using MongoDB 4.0 and + earlier, so zlib compression must be manually enabled on the server in order for the Ruby + driver to compress wire protocol data. + + * - connect=String + - ``:connect => Symbol`` + + * - connectTimeoutMS=Integer + - ``:connect_timeout => Float`` + + * - fsync=Boolean + - ``{ :write_concern => { :fsync => true|false }}`` + + * - heartbeatFrequencyMS=Integer + - ``:heartbeat_frequency => Float`` + + * - journal=Boolean + - ``{ :write_concern => { :j => true|false }}`` + + * - localThresholdMS=Integer + - ``:local_threshold => Float`` + + * - maxIdleTimeMS=Integer + - ``:max_idle_time => Float`` + + * - maxStalenessSeconds=Integer + - ``{ :read => { :max_staleness => Integer }}`` + + If the maxStalenessSeconds URI option value is -1, the driver treats + this as if the option was not given at all. Otherwise, + if the option value is numeric, the Ruby option is set to the + specified value converted to an ``Integer``. + Note that numeric values greater than 0 but less than 90, or less than + -1, are accepted by the ``Client`` constructor but will cause server + selection to fail (unless the option is changed via, for example, the + ``with`` method prior to any operations being performed on the driver). + If the option value is non-numeric, it is ignored and the driver + treats this case as if the option was not given at all. + + * - maxPoolSize=Integer + - ``:max_pool_size => Integer`` + + * - minPoolSize=Integer + - ``:min_pool_size => Integer`` + + * - readConcernLevel=String + - ``:read_concern => Hash`` + + * - readPreference=String + - ``{ :read => { :mode => Symbol }}`` + + * - readPreferenceTags=Strings + - ``{ :read => { :tag_sets => Array }}`` + + Each instance of the readPreferenceTags field is a comma-separated key:value pair which will appear in the :tag_sets array in the order they are specified. For instance, ``"readPreferenceTags=dc:ny,rack:1&readPreferenceTags=dc:ny"`` will be converted to ``[ { 'dc' => 'ny', 'rack' => '1' }, { 'dc' => 'ny' }]``. + + * - replicaSet=String + - ``:replica_set => String`` + + * - retryWrites=Boolean + - ``:retry_writes => boolean`` + + * - serverSelectionTimeoutMS=Integer + - ``:server_selection_timeout => Float`` + + * - socketTimeoutMS=Integer + - ``:socket_timeout => Float`` + + * - ssl=Boolean + - ``:ssl => true|false`` + + * - tls=Boolean + - ``:ssl => boolean`` + + * - tlsAllowInvalidCertificates=Boolean + - ``:ssl_verify_certificate => boolean`` + + Because ``tlsAllowInvalidCertificates`` uses ``true`` to signify that verification + should be disabled and ``ssl_verify_certificate`` uses ``false`` to signify that + verification should be disabled, the boolean is inverted before being used to set + ``ssl_verify_certificate``. + + * - tlsAllowInvalidHostnames=Boolean + - ``:ssl_verify_hostname => boolean`` + + Because ``tlsAllowInvalidHostnames`` uses ``true`` to signify that verification + should be disabled and ``ssl_verify_hostname`` uses ``false`` to signify that + verification should be disabled, the boolean is inverted before being used to set + ``ssl_verify_hostname``. + + * - tlsCAFile=String + - ``:ssl_ca_cert => String`` + + * - tlsCertificateKeyFile=String + - ``:ssl_cert => String`` + + * - tlsCertificateKeyFile=String + - ``:ssl_key => String`` + + * - tlsCertificateKeyFilePassword=String + - ``:ssl_key_pass_phrase => String`` + + * - tlsInsecure=Boolean + - ``:ssl_verify => boolean`` + + Because tlsInsecure uses ``true`` to signify that verification should be disabled and + ``ssl_verify`` uses ``false`` to signify that verification should be disabled, the boolean + is inverted before being used to set ``ssl_verify``. + + * - w=Integer|String + - ``{ :write_concern => { :w => Integer|String }}`` + + * - waitQueueTimeoutMS=Integer + - ``:wait_queue_timeout => Float`` + + * - wtimeoutMS=Integer + - ``{ :write_concern => { :wtimeout => Integer }}`` + + * - zlibCompressionLevel=Integer + - ``:zlib_compression_level => Integer`` + Details on Timeout Options -------------------------- -``connect_timeout`` - On initialization of a connection to a server, this setting is the - number of seconds to wait to connect before raising an exception. - This timeout is also used when monitor threads ping their servers. - The default is 10 seconds. See the `socket timeout for monitoring - specification `_ for further explanation. +``server_selection_timeout`` +```````````````````````````` + +When executing an operation, the number of seconds to wait for the driver +to find an appropriate server to send an operation to. Defaults to 30. + +In replica set deployments, this timeout should be set to exceed the typical +:manual:`replica set election times ` +in order for the driver to transparently handle primary changes. This timeout +also allows the application and the database to be started simultaneously; +the application will wait up to this much time for the database to become +available. + +If the application server is behind a reverse proxy, server selection timeout +should be lower than the request timeout configured on the reverse proxy (for +example, this applies to deployments on Heroku which has a fixed 30 second +timeout in the routing layer). In development this value can be lowered to +provide quicker failure when the server is not running. ``socket_timeout`` - The number of seconds to wait for an operation to - execute on a socket before raising a timeout exception. It should take into - account network latency and operation duration. The default is no value; the default is effectively infinity. - Please consider using ``max_time_ms`` per-operation instead, as the ``socket_timeout`` does not stop the operation - on the server; a long-running operation will continue to run on the server, beyond a socket timeout being reached. - See the `socket timeout for monitoring specification `_ - documentation for further information relating to server discovery and monitoring. +`````````````````` -``server_selection_timeout`` - The number of seconds to wait for the driver to find an appropriate server to - which an operation can be sent before raising an exception. Defaults to 30. - It should take the speed of :manual:`elections` - during a failover into account. See the - `serverSelectionTimeoutMS specification - `_ - for further information. - -``local_threshold`` - The maximum latency in seconds between the nearest server and the servers that can be considered available to send an - operation to. Defaults to 0.015. +The number of seconds to wait for a socket read or write to complete on +regular (non-monitoring) connections. Default is no timeout. -.. note:: - This is not the latency window between the driver and a server, but - rather the latency between the nearest server and other servers. See - `the localThresholdMS specification - `_. +This timeout should take into account both network latency and operation +duration. For example, setting this timeout to 5 seconds will abort queries +taking more than 5 seconds to execute on the server with ``Mongo::Error::SocketTimeoutError``. + +Note that even though by default there is no socket timeout set, the +operating system may still time out read operations depending on its +configuration. The keepalive settings are intended to detect broken network +connections (as opposed to aborting operations simply because they take a +long time to execute). + +Note that if an operation is timed out by the driver due to exceeding the +``socket_timeout`` value, it is not aborted on the server. For this reason +it is recommended to use ``max_time_ms`` option for potentially long running +operations, as this will abort their execution on the server. + +This option does not apply to monitoring connections. + +``connect_timeout`` +``````````````````` + +The number of seconds to wait for a socket connection to be established to +a server. Defaults to 10. This timeout is also used as both connect timeout +and socket timeout for monitoring connections. ``wait_queue_timeout`` - The number of seconds to wait for a connection in the connection pool to - become available. You should consider increasing this - number if you are seeing many ``Timeout`` errors while using many threads - or when operations are long-running. Defaults to 1 second. +`````````````````````` + +The number of seconds to wait for a connection in the connection pool to +become available. Defaults to 10. + +As of driver version 2.11, this timeout should be set to a value at least +as large as ``connect_timeout`` because connection pool now fully establishes +connections prior to returning them, which may require several network +round trips. ``max_time_ms`` - Specified as an option on a particular operation. It defines a cumulative time limit in milliseconds for processing - operations on a cursor. Consider using this option instead of a ``socket_timeout``, if the operation should be - interrupted on the server. See the - `CRUD specification `_ for details on - operations that support this option. +``````````````` + +Specified as an option on a particular operation, the number of milliseconds +to allow the operation to execute for on the server. Not set by default. + +Consider using this option instead of a ``socket_timeout`` for potentially +long running operations to be interrupted on the server when they take too +long. + +``wtimeout`` +```````````` + +The number of milliseconds to wait for a write to be acknowledged by the +number of servers specified in the write concern. Not set by default, which +instructs the server to apply its default. This option can be set globally +on the client or passed to individual operations under ``:write_concern``. Details on TLS Connections From 3e0db18ac0e5686b772ecbb912350f973b620e89 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 16 Sep 2019 13:30:02 -0400 Subject: [PATCH 219/442] RUBY-1563 SRV polling documentation (#1525) --- .../tutorials/ruby-driver-create-client.txt | 22 +++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index c438e073a..980e138de 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -55,17 +55,31 @@ would like to bypass the auto-discovery, pass the Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb', :connect => :sharded) Mongo::Client.new('mongodb://127.0.0.1:27017/mydb?connect=sharded') -The URI parser in the driver can also accept the protocol ``mongodb+srv`` as a logical -pre-processing step before it considers the connection string and other options. -In this protocol, the comma separated list of host names is replaced with a single host name. The format is: +The URI parser in the driver also accepts the +:manual:`mongodb+srv protocol ` +URIs, for example: .. code-block:: ruby Mongo::Client.new('mongodb+srv://test5.test.build.mongodb.cc') -.. _ruby-driver-client-options: +When an SRV URI is given, the driver does the following: + +1. As part of ``Client`` object construction, the SRV URI is resolved to + the actual hosts comprising the deployment, as these are defined in the DNS + SRV records. +2. The driver looks up URI options in the DNS TXT records corresponding to the + SRV records. These options can be overridden by URI options specified in the + URI and by Ruby options, in this order. +3. If the topology of the constructed ``Client`` object is unknown or a + sharded cluster, the driver will begin monitoring the specified SRV DNS + records for changes and will automatically update the list of servers in the + cluster. The updates will stop if the topology becomes a single or a replica + set. +.. _ruby-driver-client-options: + Client Options -------------- From 11781955299df5e9fb4e691531454f40823ae553 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Mon, 16 Sep 2019 19:01:27 -0400 Subject: [PATCH 220/442] 2.11.0.rc0 release --- source/installation.txt | 2 +- source/reference/driver-compatibility.txt | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/source/installation.txt b/source/installation.txt index af6dbb95b..a581aee71 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -18,7 +18,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.9.0.rc0 + gem install mongo -v 2.11.0.rc0 TLS/SSL and the Ruby driver --------------------------- diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index f703db4d4..090a83386 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -39,7 +39,7 @@ MongoDB Compatibility - * - 2.10 - - |checkmark| + - |checkmark| [#srv-polling]_ - |checkmark| - |checkmark| - |checkmark| @@ -138,6 +138,9 @@ MongoDB Compatibility - |checkmark| - |checkmark| +.. [#srv-polling] Polling of SRV records in sharded topologies is + implemented as of driver version 2.11. + The driver does not support older versions of MongoDB. .. _reference-compatibility-language-ruby: From 42f0f46262b66f9daf73441a7457c50552c5fdd3 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 26 Sep 2019 14:53:21 -0400 Subject: [PATCH 221/442] Label bson tutorials as 3.x and 4.x to match the actual versions --- docs/tutorials/bson-v4.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 4ed550f64..f4776751f 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -2,9 +2,9 @@ .. _ruby-bson-tutorial-4-0: -=================== -BSON 4.0.0 Tutorial -=================== +================= +BSON 4.x Tutorial +================= .. default-domain:: mongodb From 2b2dd9d4f1819259eee50c07860b18d52ddbc9de Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Fri, 4 Oct 2019 09:25:08 -0400 Subject: [PATCH 222/442] Improve User Management Documentation (#1548) * add user documentation page * add general outline * add section on removing users * mal-formatted code blocks? * fixed small details with remove user section * add info section * fixed links within ruby docs * move some lines around * try to format links again * try something * try to get references correct * add section for user creation * add some information about roles * added section about updating user * added a short intro * remove info about creating users from the authentication section * some cleanup * fixed info documentation * respond to comments from pr review --- source/index.txt | 5 +- .../tutorials/ruby-driver-authentication.txt | 25 +-- source/tutorials/user-management.txt | 181 ++++++++++++++++++ 3 files changed, 190 insertions(+), 21 deletions(-) create mode 100644 source/tutorials/user-management.txt diff --git a/source/index.txt b/source/index.txt index cdc416a49..6db115fa7 100644 --- a/source/index.txt +++ b/source/index.txt @@ -48,8 +48,8 @@ by Durran Jordan. For tutorials on Mongoid, see the `Mongoid Manual `_. .. COMMENT For the actual build, see mongodb/docs-ruby repo which pulls the documentation source from: -.. mongo-ruby-driver, -.. bson-ruby, and +.. mongo-ruby-driver, +.. bson-ruby, and .. mongoid repos. .. class:: hidden @@ -62,6 +62,7 @@ For tutorials on Mongoid, see the `Mongoid Manual `. For more information about configuring your MongoDB server for each of -these authentication mechanisms see MongoDB's +these authentication mechanisms see MongoDB's :manual:`online documentation `. -Creating a user -``````````````` - -To create a user in specific database, use the ``create`` method with the -username, password and roles parameters. - -.. code-block:: ruby - - client.database.users.create( - 'durran', - password: 'password', - roles: [ Mongo::Auth::Roles::READ_WRITE ]) - -.. seealso:: - :manual:`Built-in roles` +For more information about users and user management, see the +:ref:`User Management tutorial`. Providing credentials ````````````````````` @@ -120,7 +107,7 @@ LDAP (SASL PLAIN) mechanism MongoDB Enterprise Edition supports the LDAP authentication mechanism which allows you to delegate authentication using a Lightweight Directory -Access Protocol `LDAP `_ server. +Access Protocol `LDAP `_ server. .. warning:: @@ -166,7 +153,7 @@ installed. Add to your ``Gemfile``: require 'mongo_kerberos' To use Kerberos in the Ruby driver with **MRI**, create a -ticket-granting ticket using ``kinit``. See +ticket-granting ticket using ``kinit``. See `this documentation `_ for more information. diff --git a/source/tutorials/user-management.txt b/source/tutorials/user-management.txt new file mode 100644 index 000000000..1483a3d9f --- /dev/null +++ b/source/tutorials/user-management.txt @@ -0,0 +1,181 @@ +.. _user-management: + +=============== +User Management +=============== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +The Mongo Ruby Driver provides a set of methods for managing users in a MongoDB deployment. +All of these methods are defined on the ``Mongo::Auth::User::View`` class, which defines the +behavior for performing user-related operations on a database. You can access a database's +user view by calling the ``users`` method on the correpsonding ``Mongo::Database`` object: + +.. code-block:: ruby + + client.database.users + +Note that this will open a view on the database to which the client is already connected. +To interact with the users defined on a different database, call the client's ``use`` method +and pass in the name of the database with which you want to connect: + +.. code-block:: ruby + + client.use(:users).database.users + +In this example, all operations would be performed on the ``users`` database. + +For more information about users and user management, see MongoDB's +:manual:`online documentation `. + +Creating a user +``````````````` +There are two ways to create a new database user with the Ruby Driver. + +The simplest way to create a new user is to use the ``create`` method, passing in a username, +password, and roles: + +.. code-block:: ruby + + client.database.users.create( + 'alanturing', + password: 'enigma', + roles: [ Mongo::Auth::Roles::READWRITE ] + ) + +Another way to create a user is to first create a ``Mongo::Auth::User`` object with all the +user information and then pass that object into the ``create`` method instead. + +.. code-block:: ruby + + user = Mongo::User.new({ + user: 'alanturing', + password: 'enigma', + roles: [ Mongo::Auth::Roles::READWRITE ] + }) + + client.database.users.create(user) + +The ``create`` method takes a ``Hash`` of options as an optional second argument. +The ``:roles`` option allows you to grant permissions to the new user. +For example, the ``Mongo::Auth::Roles::READ_WRITE`` role grants the user the ability to both +read from and write to the database in which they were created. Each role can be specified +as a ``String`` or as a ``Hash``. If you would like to grant permissions to a user on a database +other than the one on which they were created, you can pass that database name in the role ``Hash``. +To create a user ``alanturing`` with permission to read and write on the ``machines`` database, +you could execute the following code: + +.. code-block:: ruby + + client.database.users.create( + 'alanturing', + password: 'enigma', + roles: [{ role: Mongo::Auth::Roles::READWRITE, db: 'machines' }] + ) + +For more information about roles in MongoDB, see the :manual:`Built-in roles` +documentation. + +In addition to the ``:roles`` option, the ``create`` method supports a ``:session`` option, +which allows you to specify a ``Mongo::Session`` object to use for this operation, +as well as a ``:write_concern`` option, which specifies the write concern of this operation +when performed on a replica set. + +.. seealso:: + :manual:`Built-in roles` + :manual:`Write Concerns`, + :ref:`Sessions`, + +User Info +````````` +To view information about a user that already exists in the database, use the ``info`` method: + +.. code-block:: ruby + + client.database.users.info('alanturing') + +If the user exists, this method will return an ``Array`` object +containing a ``Hash`` with information about the user, such as +their id, username, the database they were created on, and their +roles. If the user doesn't exist, this method will return an +empty Array. + +The ``info`` method also takes an optional ``Hash`` of options +as a second argument. Currently, the only supported option is ``:session``, +which allows you to specify a ``Mongo::Session`` object to use for this +operation. + +The Ruby Driver does not have a method that lists all of the users +that currently exist in a database. + +.. seealso:: + :ref:`Sessions ` + +Updating a user +``````````````` +To update a user that already exists in the database, you can use the ``update`` method in +one of two ways. The first way is to specify the name of the user you wish to update, +along with a new set of options. + +.. warning:: + + You must include all user options in the options ``Hash``, even those options whose values + will remain the same. Omitting an option is the same as setting it to an empty value. + +.. code-block:: ruby + + client.database.users.update( + 'alanturing', + roles: [ Mongo::Auth::Roles::READ_WRITE ] + password: 'turing-test' + ) + +The second way to update a user is to pass an updated ``Mongo::Auth::User`` object +to the ``update`` method in lieu of a username. + +.. code-block:: ruby + + user = Mongo::Auth::User.new({ + user: 'alanturing', + roles: [ Mongo::Auth::Roles::READ_WRITE ], + password: 'turing-test' + }) + + client.database.users.update(user) + +Optionally, the ``update`` method takes a ``Hash`` of options as a second argument. +The two possible options for this method are ``:session``, which allows you to specify +a ``Mongo::Session`` object on which to perform this operation, and ``:write_concern``, +which sets a write concern if this operation is performed on a replica set. + +.. seealso:: + :ref:`Sessions` + :manual:`Write Concerns`, + +Removing users +`````````````` +To remove a user from the database, use the ``remove`` method: + +.. code-block:: ruby + + client.database.users.remove('alanturing') + +You may pass a ``Hash`` of options as a second argument. The two supported +options for the ``remove`` method are ``:session`` and ``:write_concern``. +``:session`` allows you to specify a ``Mongo::Session`` object to use for +this operation. ``:write_concern`` specifies the write concern +of the operation if you are running this command against a replica set. + +The Ruby Driver does not provide a method for removing all users +from a database. + +.. seealso:: + :ref:`Sessions` + :manual:`Write Concerns`, + From bc7f57d620a35ab03575a0253a2e907428b70373 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 11 Oct 2019 14:44:31 -0400 Subject: [PATCH 223/442] Fix RUBY-1948 Default authentication options are set on URI and not on client (#1550) * remove duplicitious uri_spec tests * dont change uri spec * get all client integration tests working with default client options * cleaned up some code * uncomment another test * for now add back ssl_key default * change some variable names * make changes to client options to differentiate ssl_key and ssl_cert * update documentation * add comments back to uri * tests failing because variable doesnt exist * revert auth_source to nil on connection_spec * fix tests by being explicit about authentication mechanism * include auth mech when auth is set up * use the right environment variables * fix mmap tests * only specify auth mech for versions that enable scram256 * whoops wrong env variable * use connection to server to determine whether scram256 is supported * make sure to close client * fix connection spec * make comments more specific * committed something by mistake * update the comment in client registry * update comment again * scram256 was introduced in 4.0 * respond to pr feedback * comment * App metadata needs to take server options into account * Remove scram256 workaround * Clarify why sometimes features are instantiated once and sometimes twice --- source/tutorials/ruby-driver-authentication.txt | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index e8ecabe0e..9889f285d 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -88,6 +88,13 @@ derived from the distinguished subject name of this certificate. This authentication method requires the use of SSL connections with certificate validation. +To authenticate the client, you will need a valid SSL certificate +and private encryption key. These can be stored in separate files, +or together in one file (in the PEM format). Even if the certificate +and private key are stored in the same file, you must specify the path to +that file by passing both the ``ssl_cert`` and ``ssl_key`` options +to the client. + For more information about configuring X.509 authentication in MongoDB, see the :manual:`X.509 tutorial in the MongoDB Manual `. @@ -98,6 +105,7 @@ see the :manual:`X.509 tutorial in the MongoDB Manual auth_mech: :mongodb_x509, ssl: true, ssl_cert: '/path/to/client.pem', + ssl_key: '/path/to/client.pem', ssl_ca_cert: '/path/to/ca.pem' ) From a79b32852a62f78ddb94a7243bedf748ee4437a7 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 16 Oct 2019 13:34:18 -0400 Subject: [PATCH 224/442] RUBY-1961 Require Ruby 2.3+ in bson-ruby (#129) * RUBY-1961 Require Ruby 2.3+ in bson-ruby * Remove Ruby 1.9 compatibility added in #120 --- docs/tutorials/bson-v4.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index f4776751f..4c86672c0 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -34,7 +34,7 @@ To install the gem with bundler, include the following in your Gemfile: gem 'bson', '~> 4.0' -The BSON gem is compatible with MRI >= 1.9.3, JRuby >= 9.1, and Rubinius 2.5.x. +The BSON gem is compatible with MRI >= 2.3 and JRuby >= 9.1. Use With ActiveSupport ---------------------- From c1f61a7f08db851cc66821dd7291db04e470b005 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 16 Oct 2019 14:10:17 -0400 Subject: [PATCH 225/442] RUBY-1949 Add X.509 authentication integration tests (#1553) * RUBY-1949 Add X.509 authentication integration tests * Ensure auth source for x509 auth is always $external * Handle x509 auth failing in x509 test when x509 auth is used globally * Remove X509_AUTH_REQUIRED as it is no longer used --- source/tutorials/ruby-driver-authentication.txt | 7 +++---- source/tutorials/ruby-driver-create-client.txt | 2 +- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index 9889f285d..15afb6148 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -77,12 +77,11 @@ The mechanism can be explicitly set with the credentials: auth_mech: :mongodb_cr ) -Client Certificate (x509) -````````````````````````` -*Requires MongoDB v2.6 or greater.* +Client Certificate (X.509) +`````````````````````````` The driver presents an X.509 certificate during SSL negotiation. -The Client Certificate (x509) mechanism authenticates a username +The MONGODB-X509 authentication mechanism authenticates a username derived from the distinguished subject name of this certificate. This authentication method requires the use of SSL connections with diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 980e138de..2af241fb2 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -741,7 +741,7 @@ by concatenating them after the leaf server and client certificates, respectivel ``:ssl_cert_object`` Ruby option, which takes an instance of ``OpenSSL::X509::Certificate``, does not support certificate chains. -The Ruby driver performs strict X509 certificate verification, which requires +The Ruby driver performs strict X.509 certificate verification, which requires that both of the following fields are set in the intermediate certificate(s): - X509v3 Basic Constraints: CA: TRUE -- Can sign certificates From ccda6e855fa468a664c633d5860b7e80c019d615 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 17 Oct 2019 12:22:28 -0400 Subject: [PATCH 226/442] Fix RUBY-1957 ByteBuffer#put_cstring truncates symbols at first null byte (#128) --- docs/tutorials/bson-v4.txt | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 4c86672c0..e46220cd3 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -90,11 +90,14 @@ Writing to the buffer is done via the following API: .. code-block:: ruby buffer.put_byte(value) # Appends a single byte. - buffer.put_cstring(value) # Appends a null-terminated string. buffer.put_double(value) # Appends a 64-bit floating point. buffer.put_int32(value) # Appends a 32-bit integer (4 bytes). buffer.put_int64(value) # Appends a 64-bit integer (8 bytes). buffer.put_string(value) # Appends a UTF-8 string. + + # Converts value to string, which must not contain any null bytes, and + # writes the string to the buffer. + buffer.put_cstring(value) Reading from the buffer is done via the following API: From 896cfdcf77a82f6b28b3b4c08f2f22afefcac82f Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Mon, 21 Oct 2019 20:12:56 -0400 Subject: [PATCH 227/442] Format tutorial section on regular expressions --- docs/tutorials/bson-v4.txt | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index e46220cd3..291e8e79e 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -280,22 +280,22 @@ deserialized, they will always be returned as a ``Time`` since the BSON specification only has a ``Time`` type and knows nothing about Ruby. -Using Regexes ------------------ +Regular Expressions +------------------- -Ruby regular expressions always have BSON regular expression's equivalent of 'm' on. -In order for behavior to be preserved between the two, the 'm' option is always added -when a Ruby regular expression is serialized to BSON. +Ruby regular expressions always have BSON regular expressions' equivalent of +'m' flag on. In order for behavior to be preserved between the two, the 'm' +option is always added when a Ruby regular expression is serialized to BSON. -There is a class provided by the bson gem, ``Regexp::Raw``, to allow Ruby users to get around this. -You can simply create a regular expression like this: +There is a class provided by the bson gem, ``Regexp::Raw``, to allow Ruby users +to get around this. You can simply create a regular expression like this: .. code-block:: ruby Regexp::Raw.new("^b403158") -This code example illustrates the difference between serializing a core Ruby ``Regexp`` versus a -``Regexp::Raw`` object: +This code example illustrates the difference between serializing a core Ruby +``Regexp`` versus a ``Regexp::Raw`` object: .. code-block:: ruby @@ -313,8 +313,8 @@ This code example illustrates the difference between serializing a core Ruby ``R # => "^b403158\x00\x00" -Please use the ``Regexp::Raw`` class to instantiate your BSON regular expressions to get the exact -pattern and options you want. +Please use the ``Regexp::Raw`` class to instantiate your BSON regular +expressions to get the exact pattern and options you want. When regular expressions are deserialized, they return a wrapper that holds the raw regex string, but do not compile it. In order to get the Ruby ``Regexp`` From 9b25ce8f30a333665b709427aca51e86334c3e85 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 23 Oct 2019 12:06:47 -0400 Subject: [PATCH 228/442] RUBY-1977 Document and repair edge cases in ByteBuffer (#138) * Revise ByteBuffer documentation * Reorder the methods * Document rewind in tutorial * Include C extension in documentation * C extension documentation * docs * Split byte buffer tests * Add empty buffer length test * Rewind test for write position * Move rewind test to general byte buffer file * put_bytes with null bytes test * put_bytes documentation * Move put_bytes tests next to put_byte * Document put_byte and prohibit strings of length != 1 * Rename BSON_TYPE_OBJECT to BSON_TYPE_DOCUMENT * Order types as they are listed in the bson spec * Putting partial strings is private helper * Bson string writing is also a private helper * put_string documentation * Move put_string tests * Add put_string with empty string test * Use pvt_put_byte * Number writing documentation * Do not duplicate docstrings * put_cstring docstring * Documentation for positions * Document to_s * put_decimal128 * Documentation and range checks in replace_int32 * More replace_int32 tests * put_symbol and null bytes * Fix signedness warning * Fix the length comparison again * put_hash and put_array docs * JRuby compatibility * JRuby implementation changes for replace_int32 * JRuby implementation changes for put_byte * JRuby 9.1 compatibility --- docs/tutorials/bson-v4.txt | 57 ++++++++++++++++++++++++++++---------- 1 file changed, 43 insertions(+), 14 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 291e8e79e..c4cfb9fa2 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -68,22 +68,22 @@ you wish to instantiate and passing it a ``BSON::ByteBuffer`` instance. String.from_bson(byte_buffer) BSON::Int32.from_bson(byte_buffer) + Byte Buffers ------------ -In 4.0, BSON introduced the use of native byte buffers in both MRI and JRuby -instead of using ``StringIO``. This was done for performance improvements. +BSON library 4.0 introduces the use of native byte buffers in MRI and JRuby +instead of using ``StringIO``, for improved performance. -API -``` +Writing +``````` -A ``BSON::ByteBuffer`` can be instantiated with nothing (for write mode) or -with a string of raw bytes (for read mode). +To create a ``ByteBuffer`` for writing (i.e. serializing to BSON), +instantiate ``BSON::ByteBuffer`` with no arguments: .. code-block:: ruby buffer = BSON::ByteBuffer.new # a write mode buffer. - buffer = BSON::ByteBuffer.new(string) # a read mode buffer. Writing to the buffer is done via the following API: @@ -99,6 +99,32 @@ Writing to the buffer is done via the following API: # writes the string to the buffer. buffer.put_cstring(value) +To obtain the serialized data as a byte string (for example, to send the data +over a socket), call ``to_s`` on the buffer: + +.. code-block:: ruby + + buffer = BSON::ByteBuffer.new + buffer.put_string('testing') + socket.write(buffer.to_s) + +.. note:: + + ``ByteBuffer`` keeps track of read and write positions separately. + There is no way to rewind the buffer for writing - ``rewind`` only affects + the read position. + + +Reading +``````` + +To create a ``ByteBuffer`` for reading (i.e. deserializing from BSON), +instantiate ``BSON::ByteBuffer`` with a byte string as the argument: + +.. code-block:: ruby + + buffer = BSON::ByteBuffer.new(string) # a read mode buffer. + Reading from the buffer is done via the following API: .. code-block:: ruby @@ -111,19 +137,22 @@ Reading from the buffer is done via the following API: buffer.get_int64 # Pulls a 64-bit integer (8 bytes) from the buffer. buffer.get_string # Pulls a UTF-8 string from the buffer. -Converting a buffer to its raw bytes (for example, for sending over a socket) -is done by simply calling ``to_s`` on the buffer. +To restart reading from the beginning of a buffer, use ``rewind``: .. code-block:: ruby - buffer = BSON::ByteBuffer.new - buffer.put_string('testing') - socket.write(buffer.to_s) + buffer.rewind + +.. note:: + + ``ByteBuffer`` keeps track of read and write positions separately. + ``rewind`` only affects the read position. + -Supported Objects +Supported Classes ----------------- -Core Ruby objects that have representations in the BSON specification and +Core Ruby classes that have representations in the BSON specification and will have a ``to_bson`` method defined for them are: ``Object``, ``Array``, ``FalseClass``, ``Float``, ``Hash``, ``Integer``, ``NilClass``, ``Regexp``, ``String``, ``Symbol`` (deprecated), ``Time``, ``TrueClass``. From c2b8e99f6075e6ac4885c6e16e9b1e0dba7f5879 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 30 Oct 2019 15:19:26 -0400 Subject: [PATCH 229/442] RUBY-1977 clarify more edge cases and update tutorial (#144) --- docs/tutorials/bson-v4.txt | 90 +++++++++++++++++++++++++++++++++----- 1 file changed, 79 insertions(+), 11 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index c4cfb9fa2..a5e121666 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -83,21 +83,89 @@ instantiate ``BSON::ByteBuffer`` with no arguments: .. code-block:: ruby - buffer = BSON::ByteBuffer.new # a write mode buffer. + buffer = BSON::ByteBuffer.new + +To write raw bytes to the byte buffer with no transformations, use +``put_byte`` and ``put_bytes`` methods. They take a byte string as the argument +and copy this string into the buffer. ``put_byte`` enforces that the argument +is a string of length 1; ``put_bytes`` accepts any length strings. +The strings can contain null bytes. + +.. code-block:: ruby + + buffer.put_byte("\x00") + + buffer.put_bytes("\xff\xfe\x00\xfd") + +.. note:: + + ``put_byte`` and ``put_bytes`` do not write a BSON type byte prior to + writing the argument to the byte buffer. + +Subsequent write methods write objects of particular types in the +`BSON spec `_. Note that the type indicated +by the method name takes precedence over the type of the argument - +for example, if a floating-point value is given to ``put_int32``, it is +coerced into an integer and the resulting integer is written to the byte +buffer. + +To write a UTF-8 string (BSON type 0x02) to the byte buffer, use ``put_string``: + +.. code-block:: ruby + + buffer.put_string("hello, world") + +Note that BSON strings are always encoded in UTF-8. Therefore, the +argument must be either in UTF-8 or in an encoding convertable to UTF-8 +(i.e. not binary). If the argument is in an encoding other than UTF-8, +the string is first converted to UTF-8 and the UTF-8 encoded version is +written to the buffer. The string must be valid in its claimed encoding, +including being valid UTF-8 if the encoding is UTF-8. +The string may contain null bytes. + +The BSON specification also defines a CString type, which is used for +example for document keys. To write CStrings to the buffer, use ``put_cstring``: + +.. code-block:: ruby + + buffer.put_cstring("hello, world") + +As with regular strings, CStrings in BSON must be UTF-8 encoded. If the +argument is not in UTF-8, it is converted to UTF-8 and the resulting string +is written to the buffer. Unlike ``put_string``, the UTF-8 encoding of +the argument given to ``put_cstring`` cannot have any null bytes, since the +CString serialization format in BSON is null terminated. + +Unlike ``put_string``, ``put_cstring`` also accepts symbols and integers. +In all cases the argument is stringified prior to being written: + +.. code-block:: ruby + + buffer.put_cstring(:hello) + buffer.put_cstring(42) + +To write a 32-bit or a 64-bit integer to the byte buffer, use +``put_int32`` and ``put_int64`` methods respectively. Note that Ruby +integers can be arbitrarily large; if the value being written exceeds the +range of a 32-bit or a 64-bit integer, ``put_int32`` and ``put_int64`` +raise ``RangeError``. + +.. code-block:: ruby + + buffer.put_int32(12345) + buffer.put_int64(123456789012345) + +.. note:: + + If ``put_int32`` or ``put_int64`` are given floating point arguments, + the arguments are first coerced into integers and the integers are + written to the byte buffer. -Writing to the buffer is done via the following API: +To write a 64-bit floating point value to the byte buffer, use ``put_double``: .. code-block:: ruby - buffer.put_byte(value) # Appends a single byte. - buffer.put_double(value) # Appends a 64-bit floating point. - buffer.put_int32(value) # Appends a 32-bit integer (4 bytes). - buffer.put_int64(value) # Appends a 64-bit integer (8 bytes). - buffer.put_string(value) # Appends a UTF-8 string. - - # Converts value to string, which must not contain any null bytes, and - # writes the string to the buffer. - buffer.put_cstring(value) + buffer.put_double(3.14159) To obtain the serialized data as a byte string (for example, to send the data over a socket), call ``to_s`` on the buffer: From b01708a59c8035671e5412ccbbadd89539d5d8ae Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 30 Oct 2019 15:56:49 -0400 Subject: [PATCH 230/442] RUBY-1961 remove JRuby 9.1 support in bson-ruby due to it becoming an undue burden (#145) --- docs/tutorials/bson-v4.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index a5e121666..056a47a10 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -34,7 +34,7 @@ To install the gem with bundler, include the following in your Gemfile: gem 'bson', '~> 4.0' -The BSON gem is compatible with MRI >= 2.3 and JRuby >= 9.1. +The BSON gem is compatible with MRI >= 2.3 and JRuby >= 9.2. Use With ActiveSupport ---------------------- From cd331f857f167943ae952965877308dd0e3bcacc Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 20 Nov 2019 11:31:43 -0500 Subject: [PATCH 231/442] Fold whats new page into installation and update it to point to github releases (#1578) --- source/index.txt | 1 - source/installation.txt | 14 ++++++++++++-- source/whats-new.txt | 12 ------------ 3 files changed, 12 insertions(+), 15 deletions(-) delete mode 100644 source/whats-new.txt diff --git a/source/index.txt b/source/index.txt index 6db115fa7..be3fdd20f 100644 --- a/source/index.txt +++ b/source/index.txt @@ -57,7 +57,6 @@ For tutorials on Mongoid, see the `Mongoid Manual `_. + .. _ruby-driver-install: -Install the gem +Install the Gem --------------- The driver can be installed manually or with bundler. @@ -20,7 +21,16 @@ To install the mongo gem manually: gem install mongo -v 2.11.0.rc0 -TLS/SSL and the Ruby driver + +What's New +---------- + +Please consult the `releases page on GitHub +`_ for the list +of improvements and changes in each version of the driver. + + +TLS/SSL and the Ruby Driver --------------------------- Industry best practices, and some regulations, require the use of TLS 1.1 or newer. Though no application changes are diff --git a/source/whats-new.txt b/source/whats-new.txt deleted file mode 100644 index f351001f7..000000000 --- a/source/whats-new.txt +++ /dev/null @@ -1,12 +0,0 @@ -========== -What's New -========== - -.. default-domain:: mongodb - -What's New in the 2.5 Driver ----------------------------- - -- Added support for MongoDB server version 3.6. - -- Removed support for MongoDB server version 2.4. From 9b7d7d9aaf9631879658e9a0d7aad865282fb7ac Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Wed, 27 Nov 2019 12:32:42 -0500 Subject: [PATCH 232/442] Add information about user auth_source to the Ruby Driver documentation (#1593) * provide info about the user auth source * add info about auth source to user-management docs --- .../tutorials/ruby-driver-authentication.txt | 26 +++++++++++++++++-- source/tutorials/user-management.txt | 5 ++++ 2 files changed, 29 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index 15afb6148..31052d672 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -32,8 +32,6 @@ client. user: 'test', password: '123' ) -For MongoDB 2.6 and later, ``:auth_source`` defaults to **admin**, -otherwise the current database is used. The current database can be changed with the client's ``use`` method. @@ -57,6 +55,30 @@ Alternatively, setting the current database and credentials can be done in one s user:'test', password:'123' ) +Auth Source +``````````` + +A user's auth source is the database where that user's authentication credentials are stored. + +When creating a client with authentication credentials, you may specify an auth source +for the user: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], + user: 'test', + password: '123', + auth_source: 'admin' ) + +If no auth source is specified, then a default will be assumed by the client. The default +auth source depends on the authentication mechanism that is being used to connect. + +For the ``MONGODB-CR``, ``SCRAM-SHA-1``, and ``SCRAM-SHA-256`` authentication mechanisms, the default +auth source is the database to which the client is connecting; if no database is specified, +`admin` becomes the default auth source. For the ``PLAIN`` mechanism (LDAP), the default auth source +is the database to which the client is connecting; if no database is specified, ``$external`` is +used as the auth source. For the ``GSSAPI`` and ``MONGODB_X509`` mechanisms, +the auth source is always ``$external``. MONGODB-CR Mechanism ```````````````````` diff --git a/source/tutorials/user-management.txt b/source/tutorials/user-management.txt index 1483a3d9f..f0a72faab 100644 --- a/source/tutorials/user-management.txt +++ b/source/tutorials/user-management.txt @@ -62,6 +62,11 @@ user information and then pass that object into the ``create`` method instead. client.database.users.create(user) +Note that your new user's credentials will be stored in whatever database your ``client`` +object is currently connected to. This will be your user's ``auth_source``, and you must +be connected to that same database in order to update, remove, or get information about +the user you just created in the future. + The ``create`` method takes a ``Hash`` of options as an optional second argument. The ``:roles`` option allows you to grant permissions to the new user. For example, the ``Mongo::Auth::Roles::READ_WRITE`` role grants the user the ability to both From 42f84580318f1d5aed541f5e95ec88f9a4d86cef Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 10 Dec 2019 19:51:34 -0500 Subject: [PATCH 233/442] RUBY-2039 UUID representation POC (#146) * RUBY-2039 UUID representation POC * Require that representation is a symbol for performance --- docs/tutorials/bson-v4.txt | 103 ++++++++++++++++++++++++++++++++++++- 1 file changed, 102 insertions(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 056a47a10..f629cd97f 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -228,10 +228,11 @@ will have a ``to_bson`` method defined for them are: ``Object``, ``Array``, In addition to the core Ruby objects, BSON also provides some special types specific to the specification: + ``BSON::Binary`` ```````````````` -This is a representation of binary data. The raw data and a subtype must be +This is a representation of binary data. The raw data and the subtype must be provided when constructing. .. code-block:: ruby @@ -241,6 +242,106 @@ provided when constructing. Valid subtypes are: ``:generic``, ``:function``, ``:old``, ``:uuid_old``, ``:uuid``, ``:md5``, ``:user``. +UUID Methods +~~~~~~~~~~~~ + +To create a UUID BSON::Binary (binary subtype 4) from its RFC 4122-compliant +string representation, use the ``from_uuid`` method: + +.. code-block:: ruby + + uuid_str = "00112233-4455-6677-8899-aabbccddeeff" + BSON::Binary.from_uuid(uuid_str) + # => + +To stringify a UUID BSON::Binary to an RFC 4122-compliant representation, +use the ``to_uuid`` method: + +.. code-block:: ruby + + binary = BSON::Binary.new("\x00\x11\x22\x33\x44\x55\x66\x77\x88\x99\xAA\xBB\xCC\xDD\xEE\xFF".force_encoding('BINARY'), :uuid) + => + binary.to_uuid + => "00112233-4455-6677-8899aabbccddeeff" + +The standard representation may be explicitly specified when invoking both +``from_uuid`` and ``to_uuid`` methods: + +.. code-block:: ruby + + binary = BSON::Binary.from_uuid(uuid_str, :standard) + binary.to_uuid(:standard) + +Note that the ``:standard`` representation can only be used with a Binary +of subtype ``:uuid`` (not ``:uuid_old``). + +Legacy UUIDs +~~~~~~~~~~~~ + +Data stored in BSON::Binary objects of subtype 3 (``:uuid_old``) may be +persisted in one of three different byte orders depending on which driver +created the data. The byte orders are CSharp legacy, Java legacy and Python +legacy. The Python legacy byte order is the same as the standard RFC 4122 +byte order; CSharp legacy and Java legacy byte orders have some of the bytes +swapped. + +The Binary object containing a legacy UUID does not encode *which* format +the UUID is stored in. Therefore, methods that convert to and from the legacy +UUID format take the desired format, or representation, as their argument. +An application may copy legacy UUID Binary objects without knowing which byte +order they store their data in. + +The following methods for working with legacy UUIDs are provided for +interoperability with existing deployments storing data in legacy UUID formats. +It is recommended that new applications use the ``:uuid`` (subtype 4) format +only, which is compliant with RFC 4122. + +To stringify a legacy UUID BSON::Binary, use the ``to_uuid`` method specifying +the desired representation. Accepted representations are ``:csharp_legacy``, +``:java_legacy`` and ``:python_legacy``. Note that a legacy UUID BSON::Binary +cannot be stringified without specifying a representation. + +.. code-block:: ruby + + binary = BSON::Binary.new("\x00\x11\x22\x33\x44\x55\x66\x77\x88\x99\xAA\xBB\xCC\xDD\xEE\xFF".force_encoding('BINARY'), :uuid_old) + => + + binary.to_uuid + # => ArgumentError (Representation must be specified for BSON::Binary objects of type :uuid_old) + + binary.to_uuid(:csharp_legacy) + # => "33221100-5544-7766-8899aabbccddeeff" + + binary.to_uuid(:java_legacy) + # => "77665544-3322-1100-ffeeddccbbaa9988" + + binary.to_uuid(:python_legacy) + # => "00112233-4455-6677-8899aabbccddeeff" + +To create a legacy UUID BSON::Binary from the string representation of the +UUID, use the ``from_uuid`` method specifying the desired representation: + +.. code-block:: ruby + + uuid_str = "00112233-4455-6677-8899-aabbccddeeff" + + BSON::Binary.from_uuid(uuid_str, :csharp_legacy) + # => + + BSON::Binary.from_uuid(uuid_str, :java_legacy) + # => + + BSON::Binary.from_uuid(uuid_str, :python_legacy) + # => + +These methods can be used to convert from one representation to another: + +.. code-block:: ruby + + BSON::Binary.from_uuid('77665544-3322-1100-ffeeddccbbaa9988',:java_legacy).to_uuid(:csharp_legacy) + # => "33221100-5544-7766-8899aabbccddeeff" + + ``BSON::Code`` `````````````` From a73d8d06e3e1a15b06488d9d860891b577a85541 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 12 Dec 2019 16:58:28 -0500 Subject: [PATCH 234/442] RUBY-2043 Document binary encoding on Binary objects in tutorial (#151) --- docs/tutorials/bson-v4.txt | 48 ++++++++++++++++++++++++++++++++++---- 1 file changed, 43 insertions(+), 5 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index f629cd97f..1d6c74c24 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -232,15 +232,53 @@ specific to the specification: ``BSON::Binary`` ```````````````` -This is a representation of binary data. The raw data and the subtype must be -provided when constructing. +Use ``BSON::Binary`` objects to store arbitrary binary data. The ``Binary`` +objects can be constructed from binary strings as follows: .. code-block:: ruby - BSON::Binary.new(binary_data, :md5) + BSON::Binary.new("binary_string") + # => -Valid subtypes are: ``:generic``, ``:function``, ``:old``, ``:uuid_old``, -``:uuid``, ``:md5``, ``:user``. +By default, ``Binary`` objects are created with BSON binary subtype 0 +(``:generic``). The subtype can be explicitly specified to indicate that +the bytes encode a particular type of data: + +.. code-block:: ruby + + BSON::Binary.new("binary_string", :user) + # => + +Valid subtypes are ``:generic``, ``:function``, ``:old``, ``:uuid_old``, +``:uuid``, ``:md5`` and ``:user``. + +The data and the subtype can be retrieved from ``Binary`` instances using +``data`` and ``type`` attributes, as follows: + +.. code-block:: ruby + + binary = BSON::Binary.new("binary_string", :user) + binary.data + => "binary_string" + binary.type + => :user + +.. note:: + + ``BSON::Binary`` objects always store the data in ``BINARY`` encoding, + regardless of the encoding that the string passed to the constructor + was in: + + .. code-block:: ruby + + str = "binary_string" + str.encoding + # => # + binary = BSON::Binary.new(str) + binary.data + # => "binary_string" + binary.data.encoding + # => # UUID Methods ~~~~~~~~~~~~ From 5c4279e3072cf718bad5d58c00940c32f1540f2b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 20 Dec 2019 18:12:17 -0500 Subject: [PATCH 235/442] Fix RUBY-2055 Driver sends null pwd field in createUser when password is not specified (#1609) * Shorten lines to 79 columns * RUBY-2055 fix user creation without password * Use dashes for second level headings * Give an example for creating an x.509 user Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-authentication.txt | 2 + source/tutorials/user-management.txt | 160 +++++++++++------- 2 files changed, 101 insertions(+), 61 deletions(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index 31052d672..82b35cda0 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -55,6 +55,8 @@ Alternatively, setting the current database and credentials can be done in one s user:'test', password:'123' ) +.. _auth-source: + Auth Source ``````````` diff --git a/source/tutorials/user-management.txt b/source/tutorials/user-management.txt index f0a72faab..b28eac582 100644 --- a/source/tutorials/user-management.txt +++ b/source/tutorials/user-management.txt @@ -12,18 +12,21 @@ User Management :depth: 1 :class: singlecol -The Mongo Ruby Driver provides a set of methods for managing users in a MongoDB deployment. -All of these methods are defined on the ``Mongo::Auth::User::View`` class, which defines the -behavior for performing user-related operations on a database. You can access a database's -user view by calling the ``users`` method on the correpsonding ``Mongo::Database`` object: +The Mongo Ruby Driver provides a set of methods for managing users in a +MongoDB deployment. All of these methods are defined on the +``Mongo::Auth::User::View`` class, which defines the behavior for +performing user-related operations on a database. You can access a database's +user view by calling the ``users`` method on the correpsonding +``Mongo::Database`` object: .. code-block:: ruby client.database.users -Note that this will open a view on the database to which the client is already connected. -To interact with the users defined on a different database, call the client's ``use`` method -and pass in the name of the database with which you want to connect: +Note that this will open a view on the database to which the client is already +connected. To interact with the users defined on a different database, call +the client's ``use`` method and pass in the name of the database with which +you want to connect: .. code-block:: ruby @@ -34,12 +37,39 @@ In this example, all operations would be performed on the ``users`` database. For more information about users and user management, see MongoDB's :manual:`online documentation `. -Creating a user -``````````````` + +Users and Databases +------------------- + +When a client connects to the server, MongoDB distinguishes the database +that the client will perform operations on from the `auth source `_ +which is the database storing the user that the client is authenticating as. + +In many cases, the auth source is the same as the database. When they differ, +user management operations must be done on the auth source database. For +example, to create a user authenticating with X.509 certifcate, which must be +defined on the ``$external`` database: + +.. code-block:: ruby + + client.use('$external').database.users.create( + 'C=US,ST=New York,L=New York City,O=MongoDB,OU=x509,CN=localhost', + roles: [{role: 'read', db: 'admin'}], + ) + +Note that the auth source is not specified for creating the user - auth source +is only used during the authentication process. If ``#create`` is invoked with +a ``User`` object with ``auth_source`` set, the auth source is ignored for +the purposes of user management. + + +Creating Users +-------------- + There are two ways to create a new database user with the Ruby Driver. -The simplest way to create a new user is to use the ``create`` method, passing in a username, -password, and roles: +The simplest way to create a new user is to use the ``create`` method, +passing in a username, password, and roles: .. code-block:: ruby @@ -49,32 +79,34 @@ password, and roles: roles: [ Mongo::Auth::Roles::READWRITE ] ) -Another way to create a user is to first create a ``Mongo::Auth::User`` object with all the -user information and then pass that object into the ``create`` method instead. +Another way to create a user is to first create a ``Mongo::Auth::User`` object +with all the user information and then pass that object into the ``create`` +method instead. .. code-block:: ruby - user = Mongo::User.new({ + user = Mongo::User.new( user: 'alanturing', password: 'enigma', roles: [ Mongo::Auth::Roles::READWRITE ] - }) + ) client.database.users.create(user) -Note that your new user's credentials will be stored in whatever database your ``client`` -object is currently connected to. This will be your user's ``auth_source``, and you must -be connected to that same database in order to update, remove, or get information about -the user you just created in the future. +Note that your new user's credentials will be stored in whatever database your +``client`` object is currently connected to. This will be your user's +``auth_source``, and you must be connected to that same database in order to +update, remove, or get information about the user you just created in the future. The ``create`` method takes a ``Hash`` of options as an optional second argument. The ``:roles`` option allows you to grant permissions to the new user. -For example, the ``Mongo::Auth::Roles::READ_WRITE`` role grants the user the ability to both -read from and write to the database in which they were created. Each role can be specified -as a ``String`` or as a ``Hash``. If you would like to grant permissions to a user on a database -other than the one on which they were created, you can pass that database name in the role ``Hash``. -To create a user ``alanturing`` with permission to read and write on the ``machines`` database, -you could execute the following code: +For example, the ``Mongo::Auth::Roles::READ_WRITE`` role grants the user the +ability to both read from and write to the database in which they were created. +Each role can be specified as a ``String`` or as a ``Hash``. If you would like +to grant permissions to a user on a database other than the one on which they +were created, you can pass that database name in the role ``Hash``. To create +a user ``alanturing`` with permission to read and write on the ``machines`` +database, you could execute the following code: .. code-block:: ruby @@ -84,54 +116,59 @@ you could execute the following code: roles: [{ role: Mongo::Auth::Roles::READWRITE, db: 'machines' }] ) -For more information about roles in MongoDB, see the :manual:`Built-in roles` -documentation. +For more information about roles in MongoDB, see the +:manual:`Built-in roles` documentation. -In addition to the ``:roles`` option, the ``create`` method supports a ``:session`` option, -which allows you to specify a ``Mongo::Session`` object to use for this operation, -as well as a ``:write_concern`` option, which specifies the write concern of this operation -when performed on a replica set. +In addition to the ``:roles`` option, the ``create`` method supports a +``:session`` option, which allows you to specify a ``Mongo::Session`` object +to use for this operation, as well as a ``:write_concern`` option, +which specifies the write concern of this operation when performed on a +replica set. .. seealso:: :manual:`Built-in roles` :manual:`Write Concerns`, :ref:`Sessions`, -User Info -````````` -To view information about a user that already exists in the database, use the ``info`` method: + +User Information +---------------- + +To view information about a user that already exists in the database, use the +``info`` method: .. code-block:: ruby client.database.users.info('alanturing') -If the user exists, this method will return an ``Array`` object -containing a ``Hash`` with information about the user, such as -their id, username, the database they were created on, and their -roles. If the user doesn't exist, this method will return an -empty Array. +If the user exists, this method will return an ``Array`` object containing a +``Hash`` with information about the user, such as their id, username, the +database they were created on, and their roles. If the user doesn't exist, +this method will return an empty Array. -The ``info`` method also takes an optional ``Hash`` of options -as a second argument. Currently, the only supported option is ``:session``, -which allows you to specify a ``Mongo::Session`` object to use for this -operation. +The ``info`` method also takes an optional ``Hash`` of options as a second +argument. Currently, the only supported option is ``:session``, which allows +you to specify a ``Mongo::Session`` object to use for this operation. -The Ruby Driver does not have a method that lists all of the users -that currently exist in a database. +The Ruby Driver does not have a method that lists all of the users that +currently exist in a database. .. seealso:: :ref:`Sessions ` -Updating a user -``````````````` -To update a user that already exists in the database, you can use the ``update`` method in -one of two ways. The first way is to specify the name of the user you wish to update, -along with a new set of options. + +Updating Users +-------------- + +To update a user that already exists in the database, you can use the +``update`` method in one of two ways. The first way is to specify the name of +the user you wish to update, along with a new set of options. .. warning:: - You must include all user options in the options ``Hash``, even those options whose values - will remain the same. Omitting an option is the same as setting it to an empty value. + You must include all user options in the options ``Hash``, even those options + whose values will remain the same. Omitting an option is the same as setting + it to an empty value. .. code-block:: ruby @@ -141,8 +178,8 @@ along with a new set of options. password: 'turing-test' ) -The second way to update a user is to pass an updated ``Mongo::Auth::User`` object -to the ``update`` method in lieu of a username. +The second way to update a user is to pass an updated ``Mongo::Auth::User`` +object to the ``update`` method in lieu of a username. .. code-block:: ruby @@ -154,17 +191,19 @@ to the ``update`` method in lieu of a username. client.database.users.update(user) -Optionally, the ``update`` method takes a ``Hash`` of options as a second argument. -The two possible options for this method are ``:session``, which allows you to specify -a ``Mongo::Session`` object on which to perform this operation, and ``:write_concern``, -which sets a write concern if this operation is performed on a replica set. +Optionally, the ``update`` method takes a ``Hash`` of options as a second +argument. The two possible options for this method are ``:session``, which +allows you to specify a ``Mongo::Session`` object on which to perform this +operation, and ``:write_concern``, which sets a write concern if this operation +is performed on a replica set. .. seealso:: :ref:`Sessions` :manual:`Write Concerns`, -Removing users -`````````````` +Removing Users +-------------- + To remove a user from the database, use the ``remove`` method: .. code-block:: ruby @@ -183,4 +222,3 @@ from a database. .. seealso:: :ref:`Sessions` :manual:`Write Concerns`, - From 6958dff1879ec79c8258b8fe9b5c76305aded62b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 6 Jan 2020 17:54:07 -0500 Subject: [PATCH 236/442] RUBY-2058 add Ruby 2.7 to compatibility tables (#1619) Co-authored-by: Oleg Pudeyev --- source/reference/driver-compatibility.txt | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 090a83386..224bde0df 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -161,6 +161,7 @@ for that Ruby version is deprecated. :class: compatibility-large no-padding * - Ruby Driver + - Ruby 2.7 - Ruby 2.6 - Ruby 2.5 - Ruby 2.4 @@ -178,6 +179,7 @@ for that Ruby version is deprecated. - |checkmark| - |checkmark| - |checkmark| + - |checkmark| - - - @@ -191,6 +193,7 @@ for that Ruby version is deprecated. - |checkmark| - |checkmark| - |checkmark| + - |checkmark| - D - D - D @@ -200,6 +203,7 @@ for that Ruby version is deprecated. - * - 2.9 + - - |checkmark| - |checkmark| - |checkmark| @@ -213,6 +217,7 @@ for that Ruby version is deprecated. - * - 2.8 + - - |checkmark| - |checkmark| - |checkmark| @@ -226,6 +231,7 @@ for that Ruby version is deprecated. - * - 2.7 + - - |checkmark| - |checkmark| - |checkmark| @@ -239,6 +245,7 @@ for that Ruby version is deprecated. - * - 2.6 + - - |checkmark| - |checkmark| - |checkmark| @@ -252,6 +259,7 @@ for that Ruby version is deprecated. - * - 2.5 + - - - - |checkmark| @@ -268,6 +276,7 @@ for that Ruby version is deprecated. - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -281,6 +290,7 @@ for that Ruby version is deprecated. - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -294,6 +304,7 @@ for that Ruby version is deprecated. - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -307,6 +318,7 @@ for that Ruby version is deprecated. - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -320,6 +332,7 @@ for that Ruby version is deprecated. - - - + - - |checkmark| - |checkmark| - |checkmark| From 8810c4237db309ad4fdb5f801b4501f4d110c60e Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 10 Feb 2020 11:32:24 -0500 Subject: [PATCH 237/442] RUBY-2122 Test Symbol de/serialization with its bson_type configured to both string and symbol (#187) Co-authored-by: Oleg Pudeyev --- docs/tutorials/bson-v4.txt | 55 +++++++++++++++++++++++++++++++++++--- 1 file changed, 51 insertions(+), 4 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 1d6c74c24..d072632f8 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -14,12 +14,12 @@ BSON 4.x Tutorial :depth: 1 :class: twocols -This tutorial discusses using the core Ruby BSON gem. +This tutorial discusses using the Ruby BSON library. Installation ------------ -The BSON gem is hosted on `Rubygems `_ and can be installed +The BSON library can be installed from `Rubygems `_ manually or with bundler. To install the gem manually: @@ -28,13 +28,13 @@ To install the gem manually: gem install bson -v '~> 4.0' -To install the gem with bundler, include the following in your Gemfile: +To install the gem with bundler, include the following in your ``Gemfile``: .. code-block:: ruby gem 'bson', '~> 4.0' -The BSON gem is compatible with MRI >= 2.3 and JRuby >= 9.2. +The BSON library is compatible with MRI >= 2.3 and JRuby >= 9.2. Use With ActiveSupport ---------------------- @@ -469,6 +469,53 @@ decimal rounding with exact precision. d = BigDecimal(1.28, 3) BSON::Decimal128.new(d) +``Symbol`` +`````````` + +The BSON specification defines a symbol type which allows round-tripping +Ruby ``Symbol`` values (i.e., a Ruby ``Symbol``is encoded into a BSON symbol +and a BSON symbol is decoded into a Ruby ``Symbol``). However, since most +programming langauges do not have a native symbol type, to promote +interoperabilty, MongoDB deprecated the BSON symbol type and encourages +strings to be used instead. + +.. note:: + + In BSON, hash *keys* are always strings. Non-string values will be + stringified when used as hash keys: + + .. code-block:: ruby + + Hash.from_bson({foo: 'bar'}.to_bson) + # => {"foo"=>"bar"} + + Hash.from_bson({1 => 2}.to_bson) + # => {"1"=>2} + +By default, the BSON library encodes ``Symbol`` hash values as strings and +decodes BSON symbols into Ruby ``Symbol`` values: + + .. code-block:: ruby + + {foo: :bar}.to_bson.to_s + # => "\x12\x00\x00\x00\x02foo\x00\x04\x00\x00\x00bar\x00\x00" + + # 0x02 is the string type + Hash.from_bson(BSON::ByteBuffer.new("\x12\x00\x00\x00\x02foo\x00\x04\x00\x00\x00bar\x00\x00".force_encoding('BINARY'))) + # => {"foo"=>"bar"} + + # 0x0E is the symbol type + Hash.from_bson(BSON::ByteBuffer.new("\x12\x00\x00\x00\x0Efoo\x00\x04\x00\x00\x00bar\x00\x00".force_encoding('BINARY'))) + # => {"foo"=>:bar} + +To force encoding of Ruby symbols to BSON symbols, wrap the Ruby symbols in +``BSON::Symbol::Raw``: + + .. code-block:: ruby + + {foo: BSON::Symbol::Raw.new(:bar)}.to_bson.to_s + # => "\x12\x00\x00\x00\x0Efoo\x00\x04\x00\x00\x00bar\x00\x00" + JSON Serialization ------------------ From ef357316a02db5c43ec37338010a5c96a88cd3a9 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Wed, 26 Feb 2020 16:22:26 -0500 Subject: [PATCH 238/442] RUBY-1974 Add a tutorial for Automatic Encryption (#1729) * set up client-side-encryption docs and added installation section * change * fix bullet points * try to fix bullet points again * one more attempt to fix bullet points * documentation for generating master keys * add a section about creating a data key * fix code blocks * worked on overall example * store libmongocrypt path in an env variable * a bit of clean up * fix links * incorporate PR feedback --- source/index.txt | 1 + source/tutorials/client-side-encryption.txt | 180 ++++++++++++++++++++ 2 files changed, 181 insertions(+) create mode 100644 source/tutorials/client-side-encryption.txt diff --git a/source/index.txt b/source/index.txt index be3fdd20f..d5ce3a96f 100644 --- a/source/index.txt +++ b/source/index.txt @@ -77,6 +77,7 @@ For tutorials on Mongoid, see the `Mongoid Manual /reference/driver-compatibility diff --git a/source/tutorials/client-side-encryption.txt b/source/tutorials/client-side-encryption.txt new file mode 100644 index 000000000..e418d38d3 --- /dev/null +++ b/source/tutorials/client-side-encryption.txt @@ -0,0 +1,180 @@ +.. _client-side-encryption: + +====================== +Client-Side Encryption +====================== + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + +New in MongoDB 4.2, client-side encryption allows administrators and developers +to encrypt specific fields in MongoDB documents before inserting them into the +database. + +With client-side encryption, developers can encrypt fields client-side without +any server-side configuration or directives. Client-side field-level encryption +supports workloads where applications must guarantee that unauthorized parties, +including server administrators, cannot read the encrypted data. + +Installation +------------ + +Client-side encryption requires the installation of additional packages. + +libmongocrypt +~~~~~~~~~~~~~ + +Libmongocrypt is a C library used by the driver for client-side encryption. +To use client-side encryption, you must install the libmongocrypt binary +on the machine running your Ruby program. + +To download a pre-built binary: + +- Download a tarball of all libmongocrypt variations `here `_. + +- Extract the file you downloaded. You will see a list of directories, each + corresponding to an operating system. Find the directory that matches your + operating system and open it. + +- Inside that folder, open the folder called "nocrypto." In either the + lib or lb64 folder, you will find the libmongocrypt.so or + libmongocrypt.dylib or libmongocrypt.dll file, depending on your OS. + +- Move that file to wherever you want to keep it on your machine. You may delete + the other files included in the tarball. + +To build the binary from source: + +- Follow the instructions in the README in the `libmongocrypt GitHub repo `_. + +Once you have the libmongocrypt binary on your machine, specify the path to the +binary using the LIBMONGOCRYPT_PATH environment variable. It is recommended that +you add this variable to your rc files. For example: + +.. code-block:: bash + + export LIBMONGOCRYPT_PATH=/path/to/your/libmongocrypt.so + +mongocryptd +~~~~~~~~~~~ + +Mongocryptd is a daemon that tells the driver which fields to encrypt in a +given operation. It is only required for automatic encryption, which is an +enterprise-only feature. If you only intend to use explicit encryption, you may +skip this step. + +Mongocryptd comes pre-packaged with enterprise builds of the MongoDB server +(versions 4.2 and newer). If you must install mongocryptd separately, follow +the `installation instructions in the MongoDB manual `_. + +Automatic Encryption +-------------------- + +Automatic encryption is a feature that allows users to configure a +``Mongo::Client`` instance to always encrypt specific document fields when +performing database operations. Once the ``Mongo::Client`` is configured, it +will automatically encrypt any field that requires encryption before writing +it to the database, and it will automatically decrypt those fields when reading +them. Automatic encryption is an enterprise-only feature. + +The following example provides a demonstration of auto-encryption using a local +master key. + +.. code-block:: ruby + + require 'mongo' + + # Generate a local encryption master key + # To reuse this master key, persist it to a file or environment variable + # on your machine. + local_master_key = Base64.encode64(SecureRandom.random_bytes(96)) + + kms_providers = { + local: { + key: local_master_key + } + } + + # Create an encryption data key and insert it into the key vault collection + key_vault_client = Mongo::Client.new(['localhost:27017']) + + client_encryption = Mongo::ClientEncryption.new( + key_vault_client, + { + key_vault_namespace: 'admin.datakeys', + kms_providers: kms_providers + } + ) + + data_key_id = client_encryption.create_data_key('local') + + # Create a schema map + schema_map = { + 'encryption_db.encryption_coll': { + properties: { + encrypted_field: { + encrypt: { + keyId: [BSON::Binary.new(data_key_id, :uuid)], + bsonType: "string", + algorithm: "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic" + } + } + }, + bsonType: "object" + } + } + + # Configure the client for automatic encryption + client = Mongo::Client.new( + ['localhost:27017'], + auto_encryption_options: { + key_vault_namespace: 'admin.datakeys', + kms_providers: kms_providers, + schema_map: schema_map + } + ) + + collection = client.use(:encryption_db)[:encryption_coll] + collection.drop # Make sure there is no data in the collection + + # The string "sensitive data" will be encrypted and stored in the database + # as ciphertext + collection.insert_one(encrypted_field: 'sensitive data') + + # The data is decrypted before being returned to the user + collection.find(encrypted_field: 'sensitive data').first['encrypted_field'] + # => "sensitive data" + + # A client with no auto_encryption_options is unable to decrypt the data + client_no_encryption = Mongo::Client.new(['localhost:27017']) + client_no_encryption.use(:encryption_db)[:encryption_coll].find.first['encrypted_field'] + # => + +For more information about creating an encryption master key, creating a data key, +or creating a schema map, see later sections of this tutorial. + +.. seealso:: + `Creating A Master Key`_, + `Creating A Data Key`_, + `Creating A Schema Map`_, + +Explicit Encryption +------------------- +Documentation for explicit encryption will be provided soon. + +Creating a Master Key +--------------------- +Documentation coming soon. + +Creating a Data Key +------------------- +Documentation coming soon. + +Creating a Schema Map +--------------------- +Documentation coming soon. From c41aefe8304b5c3385969bd03bcfe3a1ebd5dd55 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Wed, 26 Feb 2020 20:41:22 -0500 Subject: [PATCH 239/442] add explicit encryption example (#1730) --- source/tutorials/client-side-encryption.txt | 74 ++++++++++++++++++++- 1 file changed, 73 insertions(+), 1 deletion(-) diff --git a/source/tutorials/client-side-encryption.txt b/source/tutorials/client-side-encryption.txt index e418d38d3..c2484652b 100644 --- a/source/tutorials/client-side-encryption.txt +++ b/source/tutorials/client-side-encryption.txt @@ -165,7 +165,79 @@ or creating a schema map, see later sections of this tutorial. Explicit Encryption ------------------- -Documentation for explicit encryption will be provided soon. +Explicit encryption is a feature that allows users to encrypt and decrypt +individual pieces of data such as strings, integers, or symbols. Explicit +encryption is a community feature and does not require an enterprise build +of the MongoDB server to use. To perform all explicit encryption and decryption +operations, use an instance of the ClientEncryption class. + +The following is an example of using explicit encryption with a local encryption +master key to encrypt a piece of data before inserting it into the database, +and then decrypting it after reading it from the database. + +.. code-block:: ruby + + require 'mongo' + + # Generate a local encryption master key + # To reuse this master key, persist it to a file or environment variable + # on your machine. + local_master_key = Base64.encode64(SecureRandom.random_bytes(96)) + + kms_providers = { + local: { + key: local_master_key + } + } + + # Create an encryption data key and insert it into the key vault collection + key_vault_client = Mongo::Client.new(['localhost:27017']) + + client_encryption = Mongo::ClientEncryption.new( + key_vault_client, + { + key_vault_namespace: 'admin.datakeys', + kms_providers: kms_providers + } + ) + + data_key_id = client_encryption.create_data_key('local') + + # The value to encrypt + value = 'sensitive data' + + # Encrypt the value + encrypted_value = client_encryption.encrypt( + 'sensitive data', + { + key_id: data_key_id, + algorithm: "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic" + } + ) + + # Create the client you will use to read and write the data to MongoDB + client = Mongo::Client.new(['localhost:27017']) + collection = client.use(:encryption_db)[:encryption_coll] + collection.drop # Make sure there is no data in the collection + + # Insert the encrypted value into the collection + collection.insert_one(encrypted_field: encrypted_value) + + # Use the client to read the encrypted value from the database, then + # use the ClientEncryption object to decrypt it + find_result = collection.find(encrypted_field: encrypted_value).first['encrypted_field'] + # => (the find result is encrypted) + + unencrypted_result = client_encryption.decrypt(find_result) + # => "sensitive data" + +For more information about creating an encryption master key, creating a data key, +or creating a schema map, see later sections of this tutorial. + +.. seealso:: + `Creating A Master Key`_, + `Creating A Data Key`_, + `Creating A Schema Map`_, Creating a Master Key --------------------- From 8341562cebbfdd19516ffb72c1f761691f5495f1 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Thu, 27 Feb 2020 14:30:10 -0500 Subject: [PATCH 240/442] RUBY-2042 Additions to FLE Documentation (#1732) * RUBY-2042 add information about creating master keys, data keys, and json schemas * try to fix link * fixed all the links * fix bullet point indentation * one small formatting change * some cleanup --- source/tutorials/client-side-encryption.txt | 199 +++++++++++++++++++- 1 file changed, 196 insertions(+), 3 deletions(-) diff --git a/source/tutorials/client-side-encryption.txt b/source/tutorials/client-side-encryption.txt index c2484652b..3d78493ef 100644 --- a/source/tutorials/client-side-encryption.txt +++ b/source/tutorials/client-side-encryption.txt @@ -241,12 +241,205 @@ or creating a schema map, see later sections of this tutorial. Creating a Master Key --------------------- -Documentation coming soon. +Both automatic encryption and explicit encryption require an encryption master key. +This master key is used to encrypt data keys, which are in turn used to encrypt +user data. The master key can be generated in one of two ways: by creating a +local key, or by creating a key in the Amazon Web Services Key Management +Service (AWS KMS). + +Local Master Key +~~~~~~~~~~~~~~~~ +A local master key is a Base64-encoded, 96-byte string. It should be persisted +on your machine as an environment variable or in a text file. + +.. warning:: + + Using a local master key is insecure and not recommended if you plan + to use client-side encryption in production. + +Run the following code to generate a local master key using Ruby: + +.. code-block:: ruby + + require 'base64' + require 'securerandom' + + local_master_key = Base64.encode64(SecureRandom.random_bytes(96)) + +AWS Master Key +~~~~~~~~~~~~~~ +It is recommended that you use Amazon's Key Management Service to create and +store your master key. To do so, follow steps 1 and 2 of the +:manual:`"Convert to a Remote Master Key" section` +in the MongoDB Client-Side Encryption documentation. + +For more information about creating a master key, see the MongoDB manual. + +.. seealso:: + + :manual:`Create a Master Key ` Creating a Data Key ------------------- -Documentation coming soon. +Once you have created a master key, create a data key by calling the +``#create_data_key`` method on an instance of the ``Mongo::ClientEncryption`` +class. This method generates a new data key and inserts it into the key vault +collection, which is the MongoDB collection in which you choose to store your +data keys. The ``#create_data_key`` method returns the UUID of the newly-created data key. + +Create a Data Key Using a Local Master Key +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you have created a local master key, you may use it to generate a new data +key with the following code snippet: + +.. code-block:: ruby + + # A Mongo::Client instance that will be used to connect to the key vault + # collection. Replace the server address with the address of the MongoDB + # server where you would like to store your key vault collection. + key_vault_client = Mongo::Client.new(['localhost:27017']) + + client_encryption = Mongo::ClientEncryption.new( + key_vault_client, + { + # Replace with the database and collection names for your key vault collection + key_vault_namespace: 'admin.datakeys', + kms_providers: { + local: { + key: local_master_key + } + } + } + ) + + data_key_id = client_encryption.create_data_key('local') + +See the `Local Master Key`_ section for more information about generating a new +local master key. + +Create a Data Key Using an AWS Master Key +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you have created an AWS master key, note the access key ID and the secret access +key of the IAM user that has permissions to use the key. Additionally, note +the AWS region and the Amazon Resource Number (ARN) of your master key. You will +use that information to generate a data key. + +.. code-block:: ruby + + # A Mongo::Client instance that will be used to connect to the key vault + # collection. Replace the server address with the address of the MongoDB + # server where you would like to store your key vault collection. + key_vault_client = Mongo::Client.new(['localhost:27017']) + + client_encryption = Mongo::ClientEncryption.new( + key_vault_client, + { + # Replace with the database and collection names for your key vault collection + key_vault_namespace: 'admin.datakeys', + kms_providers: { + aws: { + access_key_id: 'IAM-ACCESS-KEY-ID', + secret_access_key: 'IAM-SECRET-ACCESS-KEY' + } + } + } + ) + + data_key_id = client_encryption.create_data_key( + 'aws', + { + master_key: { + region: 'REGION-OF-YOUR-MASTER-KEY', + key: 'ARN-OF-YOUR-MASTER-KEY' + } + + } + ) + +See the `AWS Master Key`_ section of this tutorial for more information about +generating a new master key on AWS and finding the information you need to +create data keys. + +For more information about creating a data key, see the MongoDB manual. + +.. seealso:: + + :manual:`Create a Data Encryption Key ` Creating a Schema Map --------------------- -Documentation coming soon. + +.. note:: + + Schema maps are only used in automatic encryption. + +Once you have created a data key, you can use it to encrypt and decrypt data +during automatic encryption by referencing it in a schema map. A schema map +is a Hash that provides the ``Mongo::Client`` with information about which +fields to automatically encrypt and decrypt. + +The code snippet at the top of this tutorial demonstrates creating a schema +map using a Ruby ``Hash``. While this will work, schema maps can grow quite +large and it could be unweildy to include them in your Ruby code. Instead, it is +recommended that you store them in a separate JSON (JavaScript Object Notation) +file. + +Before creating the JSON file, Base64-encode the UUID of the your data key. + +.. code-block:: ruby + + require 'base64' + + # Take note of the Base64-encoded uuid + Base64.encode64(data_key_id) + +Then, create a new JSON file containing your schema map in the extended JSON +format defined by the :manual:`Extended JSON v2 Documentation``. + +Note that: + +* ``encryption_db`` and ``encryption_coll`` should be replaced with the + names of the database and collection where you plan to store encrypted data. +* ``encrypted_field`` should be replaced with the name of the field you want to encrypt. +* ``"bsonType": "string"`` should be replaced with the data type you intend to + encrypt, such as ``"bsonType": "integer"`` or ``"bsonType": "symbol"``. + +.. code-block:: json + + { + "encryption_db.encryption_coll": { + "properties": { + "encrypted_field": { + "encrypt": { + "keyId": [{ + "$binary": { + "base64": "YOUR-BASE64-ENCODED-DATA-KEY-ID", + "subType": "04" + } + }], + "bsonType": "string", + "algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic" + } + } + }, + "bsonType": "object" + } + } + +When you intend to use your schema map, convert it to a Ruby ``Hash`` using the +``BSON::ExtJSON`` module in the ``bson`` Ruby gem. + +.. code-block:: ruby + + require 'bson' + + # schema_map is a Ruby Hash + schema_map = BSON::ExtJSON.parse(File.open('/path/to/your/file.json')) + +For more information about schema maps, see the MongoDB manual. + +.. seealso:: + + :manual:`Specify Encrypted Fields Using JSON Schema`` From 5cc639b798055c3425540f2651ecdc7f36cebdc6 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Mon, 2 Mar 2020 12:12:45 -0500 Subject: [PATCH 241/442] RUBY-1926 Don't Base64 encode local master keys (#1746) * RUBY-1926 dont base64 encode local master key * fix one test --- source/tutorials/client-side-encryption.txt | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/source/tutorials/client-side-encryption.txt b/source/tutorials/client-side-encryption.txt index 3d78493ef..a9e8e67c6 100644 --- a/source/tutorials/client-side-encryption.txt +++ b/source/tutorials/client-side-encryption.txt @@ -92,7 +92,7 @@ master key. # Generate a local encryption master key # To reuse this master key, persist it to a file or environment variable # on your machine. - local_master_key = Base64.encode64(SecureRandom.random_bytes(96)) + local_master_key = SecureRandom.random_bytes(96) kms_providers = { local: { @@ -182,7 +182,7 @@ and then decrypting it after reading it from the database. # Generate a local encryption master key # To reuse this master key, persist it to a file or environment variable # on your machine. - local_master_key = Base64.encode64(SecureRandom.random_bytes(96)) + local_master_key = SecureRandom.random_bytes(96) kms_providers = { local: { @@ -249,7 +249,7 @@ Service (AWS KMS). Local Master Key ~~~~~~~~~~~~~~~~ -A local master key is a Base64-encoded, 96-byte string. It should be persisted +A local master key is a 96-byte binary string. It should be persisted on your machine as an environment variable or in a text file. .. warning:: @@ -261,10 +261,9 @@ Run the following code to generate a local master key using Ruby: .. code-block:: ruby - require 'base64' require 'securerandom' - local_master_key = Base64.encode64(SecureRandom.random_bytes(96)) + local_master_key = SecureRandom.random_bytes(96) AWS Master Key ~~~~~~~~~~~~~~ From 4d52faab1e258fada79d32378c0ff9f693f9d6cf Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 3 Mar 2020 13:45:24 -0500 Subject: [PATCH 242/442] Fix RUBY-2145 bson 4.8.0 rounds Time during serialization to bson/extended json (#190) * Fix RUBY-2145 bson 4.8.0 rounds Time during serialization to bson/extended json * fix for older ruby versions * compatibility with ruby 2.3 * add jruby debugging tools * test times that precede unix epoch * work around jruby rounding times up Co-authored-by: Oleg Pudeyev --- docs/tutorials/bson-v4.txt | 44 +++++++++++++++++++++++++++++++++----- 1 file changed, 39 insertions(+), 5 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index d072632f8..275dedc18 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -555,12 +555,46 @@ them. - ``{ "$regex" : "[abc]", "$options" : "i" }`` -Special Ruby Date Classes -------------------------- +Time Instances +-------------- -Ruby's ``Date`` and ``DateTime`` are able to be serialized, but when they are -deserialized, they will always be returned as a ``Time`` since the BSON -specification only has a ``Time`` type and knows nothing about Ruby. +Times in Ruby can have nanosecond precision. Times in BSON (and MongoDB) +can only have microsecond precision. When Ruby ``Time`` instances are +serialized to BSON or Extended JSON, the times are floored to the nearest +millisecond. + +.. note:: + + The time as always rounded down. If the time precedes the Unix epoch + (January 1, 1970 00:00:00 UTC), the absolute value of the time would + increase: + + .. code-block:: ruby + + time = Time.utc(1960, 1, 1, 0, 0, 0, 999_999) + time.to_f + # => -315619199.000001 + time.floor(3).to_f + # => -315619199.001 + +.. note:: + + JRuby as of version 9.2.11.0 `rounds pre-Unix epoch times up rather than + down `_. bson-ruby works around + this and correctly floors the times when serializing on JRuby. + +Because of this flooring, applications are strongly recommended to perform +all time calculations using integer math, as inexactness of floating point +calculations may produce unexpected results. + + +Date And DateTime Instances +--------------------------- + +BSON only supports storing the time as the number of seconds since the +Unix epoch. Ruby's ``Date`` and ``DateTime`` instances can be serialized +to BSON, but when the BSON is deserialized the times will be returned as +``Time`` instances. Regular Expressions From 1b915b9b803792d378c8819e82af55bb933038de Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Tue, 3 Mar 2020 22:47:20 -0500 Subject: [PATCH 243/442] RUBY-2152 Implemented "data key and double encryption" prose test (#1752) * RUBY-2152 implemented "data key and double encryption" prose test * updated documentation * change docstrings * limit tests to 4.2 enterprise --- source/tutorials/client-side-encryption.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/tutorials/client-side-encryption.txt b/source/tutorials/client-side-encryption.txt index a9e8e67c6..a51000d59 100644 --- a/source/tutorials/client-side-encryption.txt +++ b/source/tutorials/client-side-encryption.txt @@ -284,7 +284,8 @@ Once you have created a master key, create a data key by calling the ``#create_data_key`` method on an instance of the ``Mongo::ClientEncryption`` class. This method generates a new data key and inserts it into the key vault collection, which is the MongoDB collection in which you choose to store your -data keys. The ``#create_data_key`` method returns the UUID of the newly-created data key. +data keys. The ``#create_data_key`` method returns id of the newly-created +data key in the form of a BSON::Binary object. Create a Data Key Using a Local Master Key ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 4ec3e1f01c6a5e78a8de7367a76ead46c2ed280f Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Wed, 4 Mar 2020 10:31:58 -0500 Subject: [PATCH 244/442] RUBY-2152 fix inaccurate line in client-side encryption docs (#1754) --- source/tutorials/client-side-encryption.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/client-side-encryption.txt b/source/tutorials/client-side-encryption.txt index a51000d59..9c30f71a5 100644 --- a/source/tutorials/client-side-encryption.txt +++ b/source/tutorials/client-side-encryption.txt @@ -119,7 +119,7 @@ master key. properties: { encrypted_field: { encrypt: { - keyId: [BSON::Binary.new(data_key_id, :uuid)], + keyId: [data_key_id], bsonType: "string", algorithm: "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic" } From 64931fa953917371c3f66f9def9a2fd7ff1d3575 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 9 Mar 2020 12:20:37 -0400 Subject: [PATCH 245/442] RUBY-2162 Close sockets when connections fail (#1770) * explicitly close tcp sockets * explicitly close ssl sockets * docs for ipv4/ipv6 connections Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 25 ++++++++++++++++--- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 2af241fb2..1c0bf61ee 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -615,8 +615,8 @@ URI options are explained in detail in the :manual:`Connection URI reference * - zlibCompressionLevel=Integer - ``:zlib_compression_level => Integer`` -Details on Timeout Options --------------------------- +Timeout Options +--------------- ``server_selection_timeout`` ```````````````````````````` @@ -697,8 +697,8 @@ instructs the server to apply its default. This option can be set globally on the client or passed to individual operations under ``:write_concern``. -Details on TLS Connections --------------------------- +TLS Connections +--------------- To connect to the MongoDB cluster using TLS, pass ``tls`` Ruby or URI option to the ``Mongo::Client`` constructor. TLS must be explicitly configured on the @@ -774,6 +774,23 @@ of these options, because they would then not be verified against actual trusted CA certificates. +IPv4/IPv6 Connections +--------------------- + +When a client is constructed with ``localhost`` as the host name, it will +attempt an IPv4 connection only (i.e. if ``localhost`` resolves to +``127.0.0.1`` and ``::1``, the driver will only try to connect to +``127.0.0.1``). + +When a client is constructed with hostnames other than ``localhost``, it will +attempt both IPv4 and IPv6 connections depending on the addresses that the +hostnames resolve to. The driver respects the order in which ``getaddrinfo`` +returns the addresses, and will attempt to connect to them sequentially. +The first successful connection will be used. + +The driver does not currently implement the Happy Eyeballs algorithm. + + TCP Keepalive Configuration --------------------------- From 3940e99402a19c7ba9b7ef4be66a153a01b5552b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 10 Mar 2020 18:36:58 -0400 Subject: [PATCH 246/442] RUBY-1997 Handle absence of 'ns' field in index specifications returned from listIndexes (#1774) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-indexing.txt | 68 ++++++++++++++++------- 1 file changed, 47 insertions(+), 21 deletions(-) diff --git a/source/tutorials/ruby-driver-indexing.txt b/source/tutorials/ruby-driver-indexing.txt index dcc3575e5..be120ea2d 100644 --- a/source/tutorials/ruby-driver-indexing.txt +++ b/source/tutorials/ruby-driver-indexing.txt @@ -11,38 +11,51 @@ Indexing :class: singlecol The driver provides the ability to create, drop and view -:manual:`indexes` on a collection. +:manual:`indexes` on a collection through the ``indexes`` attribute: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music') + client[:bands].indexes + # => #, @batch_size=nil, @options={}> + Creating Indexes ---------------- Indexes can be created one at a time, or several can be created in a single operation. When creating multiple indexes on MongoDB 3.0 and later, the indexes -are created in parallel. On earlier versions they are created in order. +are created in parallel; on earlier versions they are created sequentially. -To create a single index, use ``create_one``. +To create a single index, use ``indexes#create_one``, passing the key +specification as the first argument and options as the second argument: .. code-block:: ruby - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - client[:bands].indexes.create_one( { :name => 1 }, unique: true ) + client[:bands].indexes.create_one(genre: 1) + + client[:bands].indexes.create_one( + { name: 1 }, + unique: true, expire_after: 120, + ) -To create multiple indexes, use ``create_many``. Note that when creating many, -the index keys must be passed as a ``key`` value in the provided spec. This is -due to the fact that options can be different for each index being created. +To create multiple indexes, use ``indexes#create_many`` which accepts an array +of index specifications. Unlike ``create_one``, each index specification +is a hash with the ``key`` key mapped to the key specification and the +options being specified on the top level. .. code-block:: ruby - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') client[:bands].indexes.create_many([ - { :key => { name: 1 }, unique: true }, - { :key => { label: -1 } } + { key: { genre: 1 } }, + { key: { name: 1 }, unique: true, expire_after: 120 }, ]) .. _ruby-driver-index-options: The following is a full list of the available options that can be added -when creating indexes. +when creating indexes. These options mirror the options supported by the +:manual:`createIndex command`. .. list-table:: :header-rows: 1 @@ -87,23 +100,36 @@ when creating indexes. Dropping Indexes ---------------- -To drop an index, call ``dropOne`` or ``dropAll``. +To drop an index, call ``indexes#drop_one`` or ``indexes#drop_all``. .. code-block:: ruby - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') - - client[:bands].indexes.drop_one( 'name_1' ) # Drops the name_1 index. - client[:bands].indexes.drop_all # Drops all indexes in the collection. + # Drops the name_1 index. + client[:bands].indexes.drop_one( 'name_1' ) + + # Drops all indexes in the collection. + client[:bands].indexes.drop_all Listing Indexes --------------- -.. code-block:: ruby +To list the indexes, iterate the ``indexes`` object: - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') +.. code-block:: ruby - client[:bands].indexes.each do |index| - p index + client[:bands].indexes.each do |index_spec| + p index_spec + # {"v"=>2, "key"=>{"_id"=>1}, "name"=>"_id_"} + # {"v"=>2, "key"=>{"genre"=>1}, "name"=>"genre_1"} + # {"v"=>2, "unique"=>true, "key"=>{"name"=>1}, "name"=>"name_1", + # "expireAfterSeconds"=>120} end + +Each iteration returns an index specification as returned by the +:manual:`listIndexes` command. + +.. note:: + + The shape and contents of the index specifications returned by this method + may change from one version of MongoDB to another. From 9294bb2740ef1c8d03373a391b18e33fe90bbcad Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 11 Mar 2020 17:36:57 -0400 Subject: [PATCH 247/442] SPEC-1248 RUBY-2126 Unify behavior around configuration for replica set discovery (#1701) * direct connection * discovered tests * implement direct connection option * discovered mongos test * organize rs files * rs discovery * organize rs discovery spec tests * use discover in favor of discovered * add uri option spec tests * spec test updates following review * add missing warnings around server removal * spec tests for replicaSet with directConnection * add wire versions * remove bogus primary field * handle directconnection and connect uri option conflicts * organize client construction test * test for ruby option conflicts * update tutorial * more tutorial documentation * dns seed list test only requires lite helper * prohibit directconnection=true on srv uris * typo fix * reject directconnection=true with multiple seeds * handle client option modification via #with * add test for directconnection=false with multiple seeds Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 165 +++++++++++++----- 1 file changed, 124 insertions(+), 41 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 1c0bf61ee..b648ab703 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -14,68 +14,142 @@ Using ``Mongo::Client`` ----------------------- To connect to a MongoDB deployment, create a ``Mongo::Client`` object. -Provide a list of hosts and options or a connection URI to the -``Mongo::Client`` constructor. The client's selected database -defaults to ``admin``. +Provide a list of hosts and options or a :manual:`connection string URI +` to the``Mongo::Client`` constructor. +The client's selected database defaults to ``admin``. -To create a client to a standalone server, provide one host in the -seed list. Optionally, you can force the cluster topology to be -standalone without going through the auto-discovery steps. +By default, the driver will automatically detect the topology used by the +deployment and connect appropriately. + +To connect to a local standalone MongoDB deployment, specify the host and +port as follows: .. code-block:: ruby - Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb') - Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb', :connect => :direct) - Mongo::Client.new('mongodb://127.0.0.1:27017/mydb') + Mongo::Client.new([ '127.0.0.1:27017' ], database: 'mydb') + + # Or using the URI syntax: + Mongo::Client.new("mongodb://127.0.0.1:27017/mydb") + +.. note:: + + The hostname ``localhost`` is treated specially by the driver and will + be resolved to IPv4 addresses only. + +To `connect to MongoDB Atlas `, +specify the Atlas deployment URI: + +.. code-block:: ruby + + Mongo::Client.new("mongodb+srv://username:myRealPassword@cluster0.mongodb.net/test?w=majority") + +The driver will discover all nodes in the cluster and connect to them as +needed. + + +Direct Connection +````````````````` + +If the deployment is a replica set, the driver will by default discover all +members of the replica set given the address of any one member, and will +dispatch operations to the appropriate member (for example, writes would be +dispatched to the primary). + +To force all operations to be performed on the designated server, specify the +``direct_connection`` option: + +.. code-block:: ruby + + Mongo::Client.new([ '1.2.3.4:27017' ], database: 'mydb', direct_connection: true) + + # Or using the URI syntax: + Mongo::Client.new("mongodb://1.2.3.4:27017/mydb?directConnection=true") + .. _ruby-driver-connect-replica-set: -To connect to a :manual:`replica set`, -pass one or more hosts and the replica set name. -The driver's auto-discovery feature finds all members of the replica -set if they are not all provided. +Replica Set Connection +`````````````````````` + +To connect to a :manual:`replica set` deployment managed by a +service providing SRV URIs (such as MongoDB Atlas), connect to the URI: .. code-block:: ruby - Mongo::Client.new([ '127.0.0.1:27017', '127.0.0.1:27018' ], :database => 'mydb', replica_set: 'myapp') - Mongo::Client.new('mongodb://127.0.0.1:27017,127.0.0.1:27018/mydb?replicaSet=myapp') + Mongo::Client.new("mongodb+srv://username:myRealPassword@cluster0.mongodb.net/test?w=majority") + +If not using SRV URIs, it is sufficient to pass the address of any node in the +replica set to the driver; the driver will then automatically discover the +remaining nodes. However, it is recommended to specify all nodes that are part +of the replica set, so that in the event of one or more nodes being unavailable +(for example, due to maintenance or reconfiguration) the driver can still +connect to the replica set. + +.. code-block:: ruby + + Mongo::Client.new([ '127.0.0.1:27017', '127.0.0.1:27018' ], + :database => 'mydb', replica_set: 'myapp') + + # Or using the URI syntax: + Mongo::Client.new("mongodb://127.0.0.1:27017,127.0.0.1:27018/mydb?replicaSet=myapp") + .. _ruby-driver-connect-sharded-cluster: -To create a client to a :manual:`sharded cluster`, -pass one or more :manual:`mongos` -hosts. The auto-discovery feature can determine that the -servers are ``mongos`` instances, but if you -would like to bypass the auto-discovery, pass the -``sharded`` option to the client. +Sharded Cluster Connection +`````````````````````````` + +To connect to a :manual:`sharded cluster` deployment managed by a +service providing SRV URIs (such as MongoDB Atlas), connect to the URI: .. code-block:: ruby - Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb') - Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'mydb', :connect => :sharded) - Mongo::Client.new('mongodb://127.0.0.1:27017/mydb?connect=sharded') + Mongo::Client.new("mongodb+srv://username:myRealPassword@cluster0.mongodb.net/test?w=majority") -The URI parser in the driver also accepts the -:manual:`mongodb+srv protocol ` -URIs, for example: +When the driver connects to a sharded cluster via an SRV URI, it will monitor +the SRV records of the address specified in the URI for changes and will +automatically add and remove ``mongos`` hosts to/from its list of servers as +they are added and removed to/from the sharded cluster. + +If not using SRV URIs, pass the addresses of one or more +:manual:`mongos` hosts. Unlike with +replica set deployments, the driver is not able to discover all ``mongos`` +nodes in a sharded cluster when not using SRV URIs. It is not required that all +``mongos`` node addresses are given to the driver - the driver will balance +the operation load among the nodes it is given. Specifying more nodes will +spread the operation load accordingly. .. code-block:: ruby - Mongo::Client.new('mongodb+srv://test5.test.build.mongodb.cc') + Mongo::Client.new([ '1.2.3.4:27017', '1.2.3.5:27017' ], :database => 'mydb') + + Mongo::Client.new("mongodb://1.2.3.4:27017,1.2.3.5:27017/mydb") -When an SRV URI is given, the driver does the following: -1. As part of ``Client`` object construction, the SRV URI is resolved to - the actual hosts comprising the deployment, as these are defined in the DNS - SRV records. +SRV URIs +```````` + +When the driver connects to a +:manual:`mongodb+srv protocol ` +URI, keep in mind the following: + +1. SRV URI lookup is performed synchronously when the client is constructed. + If this lookup fails for any reason, client construction will fail with an + exception. When a client is constructed with a list of hosts, the driver + will attempt to contact and monitor those hosts for as long as the client + object exists. If one of these hosts does not resolve initially but becomes + resolvable later, the driver will be able to establish a connection to such + a host when it becomes available. The initial SRV URI lookup must succeed + on the first attempt; subsequent host lookups will be retried by the driver + as needed. 2. The driver looks up URI options in the DNS TXT records corresponding to the - SRV records. These options can be overridden by URI options specified in the - URI and by Ruby options, in this order. + SRV records. These options can be overridden by URI options specified in the + URI and by Ruby options, in this order. 3. If the topology of the constructed ``Client`` object is unknown or a - sharded cluster, the driver will begin monitoring the specified SRV DNS - records for changes and will automatically update the list of servers in the - cluster. The updates will stop if the topology becomes a single or a replica - set. + sharded cluster, the driver will begin monitoring the specified SRV DNS + records for changes and will automatically update the list of servers in the + cluster. The updates will stop if the topology becomes a single or a replica + set. .. _ruby-driver-client-options: @@ -148,9 +222,9 @@ Ruby Options - none * - ``:connect`` - - Overrides the auto-discovery feature of the driver and forces the cluster - topology to a specific type. Choices: ``:direct``, - ``:replica_set`` or ``:sharded``. + - **Deprecated.** Disables deployment topology discovery normally + performed by the dirver and forces the cluster topology to a specific + type. Choices: ``:direct``, ``:replica_set`` or ``:sharded``. - ``Symbol`` - none @@ -165,6 +239,12 @@ Ruby Options - ``String`` - admin + * - ``:direct_connection`` + - Connect directly to the specified host, do not discover deployment + topology. + - ``Boolean`` + - false + * - ``:heartbeat_frequency`` - The number of seconds for the server monitors to refresh server states asynchronously. @@ -504,6 +584,9 @@ URI options are explained in detail in the :manual:`Connection URI reference * - connectTimeoutMS=Integer - ``:connect_timeout => Float`` + * - directConnection=Boolean + - ``:direct_connection => Boolean`` + * - fsync=Boolean - ``{ :write_concern => { :fsync => true|false }}`` From 021a7971e3b73d260d6355bfd4dbd4d25036926e Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 12 Mar 2020 12:13:34 -0400 Subject: [PATCH 248/442] RUBY-1999 tweak existing auth documentation in preparation for adding the aws mechanism (#1780) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-authentication.txt | 156 +++++++++++++----- .../tutorials/ruby-driver-create-client.txt | 22 ++- 2 files changed, 132 insertions(+), 46 deletions(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index 82b35cda0..86cd2d01e 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -7,7 +7,7 @@ Authentication .. contents:: On this page :local: :backlinks: none - :depth: 1 + :depth: 2 :class: singlecol MongoDB supports a variety of @@ -17,14 +17,15 @@ For more information about configuring your MongoDB server for each of these authentication mechanisms see MongoDB's :manual:`online documentation `. -For more information about users and user management, see the -:ref:`User Management tutorial`. +For more information about users and the Ruby driver's helpers for +user management, see the :ref:`User Management tutorial`. + Providing credentials -````````````````````` +--------------------- If authentication is enabled, provide credentials when creating a new -client. +client: .. code-block:: ruby @@ -32,73 +33,117 @@ client. user: 'test', password: '123' ) + # If using a URI: + client = Mongo::Client.new("mongodb://test:123@127.0.0.1:27017") -The current database can be changed with the client's ``use`` method. +Authentication credentials can be changed on a client instance to obtain +a new client using the ``Client#with`` method: .. code-block:: ruby - client = Mongo::Client.new([ '127.0.0.1:27017' ]) - music_client = client.use( 'music') + authenticated_client = client.with( user: 'another-user', + password: '123' ) -A new client can be created with the authentication credentials. +It is also possible to change the client's database and credentials in +one step: .. code-block:: ruby - authenticated_client = client.with( user: 'test', - password: '123' ) - -Alternatively, setting the current database and credentials can be done in one step: + authenticated_music_client = client.with( database: 'music', + user:'test', + password:'123' ) -.. code-block:: ruby - - authenticated_music_client = client.with( :database => 'music', - user:'test', - password:'123' ) .. _auth-source: Auth Source -``````````` +----------- -A user's auth source is the database where that user's authentication credentials are stored. +A user's auth source is the database where that user's authentication +credentials are stored. -When creating a client with authentication credentials, you may specify an auth source -for the user: +The user's auth source may be specified whenever the credentials are specified: .. code-block:: ruby client = Mongo::Client.new([ '127.0.0.1:27017' ], + database: 'mydb', user: 'test', password: '123', auth_source: 'admin' ) -If no auth source is specified, then a default will be assumed by the client. The default -auth source depends on the authentication mechanism that is being used to connect. + # If using a URI: + client = Mongo::Client.new("mongodb://test:123@127.0.0.1:27017/mydb?authSource=admin") -For the ``MONGODB-CR``, ``SCRAM-SHA-1``, and ``SCRAM-SHA-256`` authentication mechanisms, the default -auth source is the database to which the client is connecting; if no database is specified, -`admin` becomes the default auth source. For the ``PLAIN`` mechanism (LDAP), the default auth source -is the database to which the client is connecting; if no database is specified, ``$external`` is -used as the auth source. For the ``GSSAPI`` and ``MONGODB_X509`` mechanisms, -the auth source is always ``$external``. +If no auth source is specified, then a default will be assumed by the client. +The default auth source depends on the authentication mechanism that is being +used to connect. -MONGODB-CR Mechanism -```````````````````` +For the ``MONGODB-CR``, ``SCRAM-SHA-1``, and ``SCRAM-SHA-256`` authentication +mechanisms, the default auth source is the database to which the client is +connecting; if no database is specified, ``admin`` database is the default +database and hence the default auth source. For the ``PLAIN`` mechanism (LDAP), +the default auth source is the database to which the client is connecting; +if no database is specified, the ``$external`` database is used as the +auth source. For the ``GSSAPI`` and ``MONGODB_X509`` mechanisms, the +auth source is always ``$external``. -*Deprecated:* MONGODB-CR mechanism is deprecated as of MongoDB version 3.6. -Please use SCRAM authentication instead. +.. note:: -MONGODB-CR was the default authentication mechanism for MongoDB up through version 2.6. + When changing the database using the ``with`` method, the auth source is + determined in the new ``Client`` instance using the full set of options + that applies to it. For example, if the original client had an auth source + specified, this auth source would take precedence over the database + given in the ``with`` call. If the original client did not have an auth + source specified, the new database would be the new auth source, subject + to the rules of the authentication mechanism used. -The mechanism can be explicitly set with the credentials: + +Authentication Mechanisms +------------------------- + +.. _scram: + +SCRAM +````` + +:manual:`SCRAM authentication ` is the default +authentication mechanism for MongoDB. There are two SCRAM mechanisms in +MongoDB: SCRAM-SHA-1 (available as of MongoDB 3.0) and SCRAM-SHA-256 +(available as of MongoDB 4.0). If an authentication mechanism is not +specified but user credentials are, the driver will attempt to use SCRAM +authentication on server 3.0 or newer and will negotiate the mechanism +to use based on the server version and the mechanisms defined for a +particular user (it is possible to configure a user in the server to only +allow SCRAM-SHA-1 mechanism, only SCRAM-SHA-256 mechanism or both). + +To explicitly specify SCRAM-SHA-1 as the authentication mechanism, use the +``auth_mech: :scram`` Ruby client option or the ``SCRAM-SHA-1`` as the value +for the ``authMechanism`` URI option, as follows: .. code-block:: ruby client = Mongo::Client.new([ '127.0.0.1:27017' ], - :database => 'music', + database: 'mydb', user: 'test', password: '123', - auth_mech: :mongodb_cr ) + auth_mech: :scram ) + + client = Mongo::Client.new("mongodb://test:123@127.0.0.1:27017/mydb?authMechanism=SCRAM-SHA-1") + +To explicitly specify SCRAM-SHA-256 as the authentication mechanism, use the +``auth_mech: :scram256`` Ruby client option or the ``SCRAM-SHA-256`` as the +value for the ``authMechanism`` URI option, as follows: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], + database: 'mydb', + user: 'test', + password: '123', + auth_mech: :scram256 ) + + client = Mongo::Client.new("mongodb://test:123@127.0.0.1:27017/mydb?authMechanism=SCRAM-SHA-256") Client Certificate (X.509) @@ -132,8 +177,9 @@ see the :manual:`X.509 tutorial in the MongoDB Manual ssl_ca_cert: '/path/to/ca.pem' ) -LDAP (SASL PLAIN) mechanism -``````````````````````````` +LDAP (SASL PLAIN) +````````````````` + *Requires MongoDB Enterprise Edition v2.6 or greater.* MongoDB Enterprise Edition supports the LDAP authentication mechanism @@ -159,10 +205,12 @@ MongoDB, see the :manual:`SASL/LDAP tutorial in the MongoDB Manual ssl_cert: '/path/to/client.pem', ssl_ca_cert: '/path/to/ca.pem' ) + .. _kerberos: -Kerberos (GSSAPI) mechanism -``````````````````````````` +Kerberos (GSSAPI) +````````````````` + *Requires MongoDB Enterprise Edition v2.4 or greater.* MongoDB Enterprise Edition v2.4+ supports Kerberos authentication. @@ -183,8 +231,8 @@ installed. Add to your ``Gemfile``: require 'mongo' require 'mongo_kerberos' -To use Kerberos in the Ruby driver with **MRI**, create a -ticket-granting ticket using ``kinit``. See +To use Kerberos in the Ruby driver with **MRI**, create a ticket-granting +ticket using ``kinit``. See `this documentation `_ for more information. @@ -209,3 +257,23 @@ authentication, see the :manual:`manual auth_mech: :gssapi, user: 'test', password: '123' ) + + +MONGODB-CR +`````````` + +*Deprecated:* MONGODB-CR mechanism is deprecated as of MongoDB 3.6 and +removed as of MongoDB 4.0. Please use `SCRAM authentication `_ instead. + +MONGODB-CR was the default authentication mechanism for MongoDB up through +version 2.6. + +The mechanism can be explicitly set with the credentials: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], + database: 'music', + user: 'test', + password: '123', + auth_mech: :mongodb_cr ) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index b648ab703..2760a3995 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -22,7 +22,9 @@ By default, the driver will automatically detect the topology used by the deployment and connect appropriately. To connect to a local standalone MongoDB deployment, specify the host and -port as follows: +port of the server. In most cases you would also specify the database name +to connect to; if no database name is specified, the client will use the +``admin`` database: .. code-block:: ruby @@ -41,12 +43,28 @@ specify the Atlas deployment URI: .. code-block:: ruby - Mongo::Client.new("mongodb+srv://username:myRealPassword@cluster0.mongodb.net/test?w=majority") + Mongo::Client.new("mongodb+srv://username:myRealPassword@cluster0.mongodb.net/mydb?w=majority") The driver will discover all nodes in the cluster and connect to them as needed. +Database +```````` + +Usually the client will connect to a specific database. It is possible to +change the active database by calling ``Client#use`` to obtain a new ``Client`` +instance configured to use the new database. For example, it is common to +perform operations on the ``admin`` database: + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'mydb') + admin_client = client.use('admin') + # Issue an administrative command + admin_client.database.command(replSetGetConfig: 1).documents.first + + Direct Connection ````````````````` From f5f37a210d7384d076dbe3b102a1c0b93017806b Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Thu, 12 Mar 2020 16:29:55 -0400 Subject: [PATCH 249/442] RUBY-2173 Improvements to Client-Side Encryption documentation (#1782) * started adding auto encryption options to the client documentation * add the rest of the options to client options * change symbols to strings * remove unnecessary braces * added required documentation items from the spec * add some expected outcomes as comments in code * added a lot of info about client options * link from create client * properly connect cse tutorial to client tutorial * fix link * some cleanup * add a link in mongocryptd installations ection * fix bold code --- source/tutorials/client-side-encryption.txt | 354 +++++++++++++----- .../tutorials/ruby-driver-create-client.txt | 37 +- 2 files changed, 292 insertions(+), 99 deletions(-) diff --git a/source/tutorials/client-side-encryption.txt b/source/tutorials/client-side-encryption.txt index 9c30f71a5..76357d7ef 100644 --- a/source/tutorials/client-side-encryption.txt +++ b/source/tutorials/client-side-encryption.txt @@ -21,6 +21,11 @@ any server-side configuration or directives. Client-side field-level encryption supports workloads where applications must guarantee that unauthorized parties, including server administrators, cannot read the encrypted data. +.. warning:: + + Enabling Client Side Encryption reduces the maximum write batch size and may + have a negative performance impact. + Installation ------------ @@ -69,8 +74,13 @@ enterprise-only feature. If you only intend to use explicit encryption, you may skip this step. Mongocryptd comes pre-packaged with enterprise builds of the MongoDB server -(versions 4.2 and newer). If you must install mongocryptd separately, follow -the `installation instructions in the MongoDB manual `_. +(versions 4.2 and newer). For installation instructions, see +`the MongoDB manual `_. + +In order to configure mongocryptd (for example, which port it listens on or the +path used to spawn the daemon), it is necessary to pass different options to the +``Mongo::Client`` performing automatic encryption. See the `:extra_options`_ +section of this tutorial for more information. Automatic Encryption -------------------- @@ -80,19 +90,47 @@ Automatic encryption is a feature that allows users to configure a performing database operations. Once the ``Mongo::Client`` is configured, it will automatically encrypt any field that requires encryption before writing it to the database, and it will automatically decrypt those fields when reading -them. Automatic encryption is an enterprise-only feature. +them. + +Client-side encryption implements envelope encryption, which is the practice of +encrypting data with a data key, which is in turn encrypted using a master key. +Thus, using client-side encryption with MongoDB involves three main steps: + +1. Create a master key +2. Create a data key (and encrypt it using the master key) +3. Encrypt data using the data key + +The example below demonstrates how to follow these steps with a local master key +in order to perform automatic encryption. + +.. note:: + + Automatic encryption is an enterprise only feature that only applies to + operations on a collection. Automatic encryption is not supported for operations + on a database or view, and operations that are not bypassed will result in + error (see `Auto Encryption Whitelist `_ + ). To bypass automatic encryption for all operations, set ``bypass_auto_encryption`` + to true in ``auto_encryption_options``. + +.. note:: -The following example provides a demonstration of auto-encryption using a local -master key. + Automatic encryption requires the authenticated user to have the listCollections privilege action. .. code-block:: ruby require 'mongo' - # Generate a local encryption master key - # To reuse this master key, persist it to a file or environment variable - # on your machine. + ##################################### + # Step 1: Create a local master key # + ##################################### + + # A local master key is a 96-byte binary blob. local_master_key = SecureRandom.random_bytes(96) + # => "\xB2\xBE\x8EN\xD4\x14\xC2\x13\xC3..." + + ############################# + # Step 2: Create a data key # + ############################# kms_providers = { local: { @@ -100,20 +138,25 @@ master key. } } - # Create an encryption data key and insert it into the key vault collection + # The key vault client is a Mongo::Client instance connected to the collection + # that will store your data keys. key_vault_client = Mongo::Client.new(['localhost:27017']) + # Use an instance of Mongo::ClientEncryption to create a new data key client_encryption = Mongo::ClientEncryption.new( key_vault_client, - { - key_vault_namespace: 'admin.datakeys', - kms_providers: kms_providers - } + key_vault_namespace: 'admin.datakeys', + kms_providers: kms_providers ) data_key_id = client_encryption.create_data_key('local') + # => + + ####################################################### + # Step 3: Configure Mongo::Client for auto-encryption # + ####################################################### - # Create a schema map + # Create a schema map, which tells the Mongo::Client which fields to encrypt schema_map = { 'encryption_db.encryption_coll': { properties: { @@ -139,7 +182,7 @@ master key. } ) - collection = client.use(:encryption_db)[:encryption_coll] + collection = client.use('encryption_db')['encryption_coll'] collection.drop # Make sure there is no data in the collection # The string "sensitive data" will be encrypted and stored in the database @@ -152,16 +195,15 @@ master key. # A client with no auto_encryption_options is unable to decrypt the data client_no_encryption = Mongo::Client.new(['localhost:27017']) - client_no_encryption.use(:encryption_db)[:encryption_coll].find.first['encrypted_field'] - # => + client_no_encryption.use('encryption_db')['encryption_coll'].find.first['encrypted_field'] + # => -For more information about creating an encryption master key, creating a data key, -or creating a schema map, see later sections of this tutorial. +The example above demonstrates using automatic encryption with a local master key. +For more information about using the AWS Key Management Service to create a +master key and create data keys, see the following sections of this tutorial: -.. seealso:: - `Creating A Master Key`_, - `Creating A Data Key`_, - `Creating A Schema Map`_, +- `Creating A Master Key`_ +- `Creating A Data Key`_ Explicit Encryption ------------------- @@ -171,18 +213,32 @@ encryption is a community feature and does not require an enterprise build of the MongoDB server to use. To perform all explicit encryption and decryption operations, use an instance of the ClientEncryption class. -The following is an example of using explicit encryption with a local encryption -master key to encrypt a piece of data before inserting it into the database, -and then decrypting it after reading it from the database. +Client-side encryption implements envelope encryption, which is the practice of +encrypting data with a data key, which is in turn encrypted using a master key. +Thus, using client-side encryption with MongoDB involves three main steps: + +1. Create a master key +2. Create a data key (and encrypt it using the master key) +3. Encrypt data using the data key + +The example below demonstrates how to follow these steps with a local master key +in order to perform explicit encryption. .. code-block:: ruby require 'mongo' - # Generate a local encryption master key - # To reuse this master key, persist it to a file or environment variable - # on your machine. + ##################################### + # Step 1: Create a local master key # + ##################################### + + # A local master key is a 96-byte binary blob. local_master_key = SecureRandom.random_bytes(96) + # => "\xB2\xBE\x8EN\xD4\x14\xC2\x13\xC3..." + + ############################# + # Step 2: Create a data key # + ############################# kms_providers = { local: { @@ -190,18 +246,23 @@ and then decrypting it after reading it from the database. } } - # Create an encryption data key and insert it into the key vault collection + # The key vault client is a Mongo::Client instance connected to the collection + # that will store your data keys. key_vault_client = Mongo::Client.new(['localhost:27017']) + # Use an instance of Mongo::ClientEncryption to create a new data key client_encryption = Mongo::ClientEncryption.new( key_vault_client, - { - key_vault_namespace: 'admin.datakeys', - kms_providers: kms_providers - } + key_vault_namespace: 'admin.datakeys', + kms_providers: kms_providers ) data_key_id = client_encryption.create_data_key('local') + # => + + ##################################################### + # Step 3: Encrypt a string with explicit encryption # + ##################################################### # The value to encrypt value = 'sensitive data' @@ -217,7 +278,7 @@ and then decrypting it after reading it from the database. # Create the client you will use to read and write the data to MongoDB client = Mongo::Client.new(['localhost:27017']) - collection = client.use(:encryption_db)[:encryption_coll] + collection = client.use('encryption_db')['encryption_coll'] collection.drop # Make sure there is no data in the collection # Insert the encrypted value into the collection @@ -231,13 +292,12 @@ and then decrypting it after reading it from the database. unencrypted_result = client_encryption.decrypt(find_result) # => "sensitive data" -For more information about creating an encryption master key, creating a data key, -or creating a schema map, see later sections of this tutorial. +The example above demonstrates using explicit encryption with a local master key. +For more information about using the AWS Key Management Service to create a +master key and create data keys, see the following sections of this tutorial: -.. seealso:: - `Creating A Master Key`_, - `Creating A Data Key`_, - `Creating A Schema Map`_, +- `Creating A Master Key`_, +- `Creating A Data Key`_, Creating a Master Key --------------------- @@ -261,9 +321,8 @@ Run the following code to generate a local master key using Ruby: .. code-block:: ruby - require 'securerandom' - local_master_key = SecureRandom.random_bytes(96) + # => "\xB2\xBE\x8EN\xD4\x14\xC2\x13\xC3..." (a binary blob) AWS Master Key ~~~~~~~~~~~~~~ @@ -272,11 +331,9 @@ store your master key. To do so, follow steps 1 and 2 of the :manual:`"Convert to a Remote Master Key" section` in the MongoDB Client-Side Encryption documentation. -For more information about creating a master key, see the MongoDB manual. - -.. seealso:: - - :manual:`Create a Master Key ` +For more information about creating a master key, see the +:manual:`Create a Master Key ` +section of the MongoDB manual. Creating a Data Key ------------------- @@ -293,6 +350,11 @@ Create a Data Key Using a Local Master Key If you have created a local master key, you may use it to generate a new data key with the following code snippet: +.. warning:: + + Using a local master key is insecure and not recommended if you plan + to use client-side encryption in production. + .. code-block:: ruby # A Mongo::Client instance that will be used to connect to the key vault @@ -302,18 +364,17 @@ key with the following code snippet: client_encryption = Mongo::ClientEncryption.new( key_vault_client, - { - # Replace with the database and collection names for your key vault collection - key_vault_namespace: 'admin.datakeys', - kms_providers: { - local: { - key: local_master_key - } + # Replace with the database and collection names for your key vault collection + key_vault_namespace: 'admin.datakeys', + kms_providers: { + local: { + key: local_master_key } } ) data_key_id = client_encryption.create_data_key('local') + # => See the `Local Master Key`_ section for more information about generating a new local master key. @@ -335,14 +396,12 @@ use that information to generate a data key. client_encryption = Mongo::ClientEncryption.new( key_vault_client, - { - # Replace with the database and collection names for your key vault collection - key_vault_namespace: 'admin.datakeys', - kms_providers: { - aws: { - access_key_id: 'IAM-ACCESS-KEY-ID', - secret_access_key: 'IAM-SECRET-ACCESS-KEY' - } + # Replace with the database and collection names for your key vault collection + key_vault_namespace: 'admin.datakeys', + kms_providers: { + aws: { + access_key_id: 'IAM-ACCESS-KEY-ID', + secret_access_key: 'IAM-SECRET-ACCESS-KEY' } } ) @@ -357,28 +416,66 @@ use that information to generate a data key. } ) + # => See the `AWS Master Key`_ section of this tutorial for more information about generating a new master key on AWS and finding the information you need to create data keys. -For more information about creating a data key, see the MongoDB manual. +For more information about creating a data key, see the +:manual:`Create a Data Encryption Key ` +section of the MongoDB manual. -.. seealso:: +Auto-Encryption Options +----------------------- +Automatic encryption can be configured on a ``Mongo::Client`` using the +``auto_encryption_options`` option ``Hash``. This section provides an overview +of the fields inside ``auto_encryption_options`` and explains how to choose their +values. - :manual:`Create a Data Encryption Key ` +``:key_vault_client`` +~~~~~~~~~~~~~~~~~~~~~ +The key vault client is a ``Mongo::Client`` instance that will be used to connect +to the MongoDB collection containing your encryption data keys. For example, if +your key vault was hosted on a MongoDB instance at ``localhost:30000``: -Creating a Schema Map ---------------------- +.. code-block:: ruby -.. note:: + key_vault_client = Mongo::Client.new(['localhost:30000']) - Schema maps are only used in automatic encryption. + Mongo::Client.new(['localhost:27017], + auto_encryption_options: { + key_vault_client: key_vault_client, + # ... (Fill in other options here) + } + ) + +If your data keys are stored in the same MongoDB instance that stores your encrypted +data, you may leave this option blank, and the top-level client will be used +to insert and fetch data keys. + +``:key_vault_namespace`` +~~~~~~~~~~~~~~~~~~~~~~~~ +The key vault namespace is a ``String`` in the format ``"database_name.collection_name"``, +where ``database_name`` and ``collection_name`` are the name of the database and +collection in which you would like to store your data keys. For example, if your data +keys are stored in the ``admin`` database in the ``datakeys`` collection: + +.. code-block:: ruby + + Mongo::Client.new(['localhost:27017], + auto_encryption_options: { + key_vault_namespace: 'admin.datakeys', + # ... (Fill in other options here) + } + ) -Once you have created a data key, you can use it to encrypt and decrypt data -during automatic encryption by referencing it in a schema map. A schema map -is a Hash that provides the ``Mongo::Client`` with information about which -fields to automatically encrypt and decrypt. +There is no default key vault namespace, and this option must be provided. + +``:schema_map`` +~~~~~~~~~~~~~~~ +A schema map is a Hash with information about which fields to automatically +encrypt and decrypt. The code snippet at the top of this tutorial demonstrates creating a schema map using a Ruby ``Hash``. While this will work, schema maps can grow quite @@ -390,21 +487,13 @@ Before creating the JSON file, Base64-encode the UUID of the your data key. .. code-block:: ruby - require 'base64' - - # Take note of the Base64-encoded uuid - Base64.encode64(data_key_id) - -Then, create a new JSON file containing your schema map in the extended JSON -format defined by the :manual:`Extended JSON v2 Documentation``. + Base64.encode64(data_key_id.data) + # => "sr6OTtQUwhPD..." (a base64-encoded string) -Note that: - -* ``encryption_db`` and ``encryption_coll`` should be replaced with the - names of the database and collection where you plan to store encrypted data. -* ``encrypted_field`` should be replaced with the name of the field you want to encrypt. -* ``"bsonType": "string"`` should be replaced with the data type you intend to - encrypt, such as ``"bsonType": "integer"`` or ``"bsonType": "symbol"``. +Then, create a new JSON file containing your schema map in the format defined by +the JSON Schema Draft 4 standard syntax. You can read more about formatting +your schema map in the :manual:`Automatic Encryption Rules` +section of the MongoDB manual. .. code-block:: json @@ -433,13 +522,92 @@ When you intend to use your schema map, convert it to a Ruby ``Hash`` using the .. code-block:: ruby - require 'bson' + schema_map = BSON::ExtJSON.parse(File.read('/path/to/your/file.json')) + # => { 'encryption_db.encryption_coll' => { ... } } + + Mongo::Client.new(['localhost:27017], + auto_encryption_options: { + schema_map: schema_map, + # ... (Fill in other options here) + } + ) + +.. note:: + + It is also possible to supply a schema map as a validator on a MongoDB collection. + This is referred to as a "remote schema map," while providing the schema map as + an option on the ``Mongo::Client`` is called a "local schema map." - # schema_map is a Ruby Hash - schema_map = BSON::ExtJSON.parse(File.open('/path/to/your/file.json')) + Supplying a local schema map provides more security than relying on JSON schemas + obtained from the server. It protects against a malicious server advertising + a false JSON schema, which could trick the client into sending unencrypted + data that should be encrypted. -For more information about schema maps, see the MongoDB manual. + See :manual:`Server-Side Field Level Encryption Enforcement` + in the MongoDB manual for more information about using the schema map to + create a JSON schema validator on your collection. .. seealso:: - :manual:`Specify Encrypted Fields Using JSON Schema`` + :manual:`Specify Encrypted Fields Using JSON Schema`, + :manual:`Automatic Encryption Rules` + +``:bypass_auto_encryption`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The ``:bypass_auto_encryption`` option is a ``Boolean`` that specifies whether the +``Mongo::Client`` should skip encryption when writing to the database. If +``:bypass_auto_encryption`` is ``true``, the client will still perform automatic +decryption of any previously-encrypted data. + +.. code-block:: ruby + + Mongo::Client.new(['localhost:27017], + auto_encryption_options: { + bypass_auto_encryption: true, + # ... (Fill in other options here) + } + ) + +``:extra_options`` +~~~~~~~~~~~~~~~~~~ +``:extra_options`` is a ``Hash`` of options related to spawning mongocryptd. +Every option in this ``Hash`` has a default value, so it is only necessary to +provide the options whose defaults you want to override. + +- ``:mongocryptd_spawn_args`` - This is an ``Array`` containing arguments + for spawning mongocryptd. The Ruby driver will pass these arguments to + mongocryptd on spawning the daemon. Possible arguments are: + + - ``"--idleShutdownTimeoutSecs"`` - The number of seconds mongocryptd must remain + idle before it shuts itself down. The default value is 60. + - ``"--port"`` - The port at which mongocryptd will listen for connections. The + default is 27020. + +- ``:mongocryptd_uri`` - The URI that the driver will use to connect to mongocryptd. + By default, this is ``"mongodb://localhost:27020"``. + +- ``:mongocryptd_spawn_path`` - The path to the mongocryptd executable. The default + is ``"mongocryptd"``. + +- ``:mongocryptd_bypass_spawn`` - A ``Boolean`` indicating whether the driver should + skip spawning mongocryptd. + +For example, if you would like to run mongocryptd on port 30000, provide +``extra_options`` as follows: + +.. code-block:: ruby + + Mongo::Client.new(['localhost:27017], + auto_encryption_options: { + extra_options: { + mongocryptd_spawn_args: ['--port=30000'], + mongocryptd_uri: 'mongodb://localhost:30000', + } + # ... (Fill in other options here) + } + ) + +.. warning:: + + The contents of ``:extra_options`` is subject to change in future versions + of the client-side encryption API. diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 2760a3995..0d746fad5 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -233,6 +233,31 @@ Ruby Options - For MongoDB 2.6 and later: **admin** if credentials are supplied, otherwise the current database + * - ``:auto_encryption_options`` + - A ``Hash`` of options for configuring automatic encryption. + + - ``:key_vault_client`` - A client connected to the MongoDB instance + storing the encryption data keys (``Mongo::Client``, defaults to the + top-level client instance). + - ``:key_vault_namespace`` - The namespace of the key vault collection + in the format ``"database.collection"`` (``String``, required). + - ``:kms_providers`` - Key management service configuration information. + One or both of the keys ``:local`` and ``:aws`` must be specified + (``Hash``, required). See the "The ``kms_providers`` option`` section of the + :ref:`Client-Side Encryption tutorial` for more + information about this option. + - ``:schema_map`` - The JSONSchema for one or more collections specifying + which fields should be encrypted (``Hash``, optional, defaults to ``nil``). + - ``:bypass_auto_encryption`` - Whether to skip automatic encryption when + performing database operations (``Boolean``, defaults to ``false``). + - ``:extra_options`` - Options related to spawning mongocryptd (``Hash``, + optional, defaults to ``nil``). + + For more information about formatting these options, see the + "Auto-Encryption Options" section of the :ref:`Client-Side Encryption tutorial`. + - ``Hash`` + - none + * - ``:compressors`` - A list of potential compressors to use, in order of preference. The driver chooses the first compressor that is also supported by the server. Currently the driver only supports 'zlib'. @@ -423,7 +448,7 @@ Ruby Options context in ``extra_chain_cert`` attribute. If intermediate certificates are provided, they must follow the client certificate which must be the first certificate in the file. - + This option, if present, takes precedence over ``:ssl_cert_string`` and ``:ssl_cert_object`` options. - ``String`` @@ -446,7 +471,7 @@ Ruby Options to the OpenSSL context in ``extra_chain_cert`` attribute. If intermediate certificates are provided, they must follow the client certificate which must be the first certificatet in the string. - + This option, if present, takes precedence over the ``:ssl_cert_object`` option. - ``String`` @@ -561,17 +586,17 @@ URI options are explained in detail in the :manual:`Connection URI reference * - authMechanism=String - ``:auth_mech => Symbol`` - + Auth mechanism values are converted as follows from URI options to Ruby options: - + - ``GSSAPI`` => ``:gssapi`` - ``MONGODB-CR`` => ``:mongodb_cr`` - ``MONGODB-X509`` => ``:mongodb_x509`` - ``PLAIN`` => ``:plain`` - ``SCRAM-SHA-1`` => ``:scram`` - ``SCRAM-SHA-256`` => ``:scram256`` - + If a different value is provided for auth mechanism, it is converted to the Ruby option unmodified and retains its ``String`` type. Note that, while currently the driver allows a ``Client`` instance @@ -622,7 +647,7 @@ URI options are explained in detail in the :manual:`Connection URI reference * - maxStalenessSeconds=Integer - ``{ :read => { :max_staleness => Integer }}`` - + If the maxStalenessSeconds URI option value is -1, the driver treats this as if the option was not given at all. Otherwise, if the option value is numeric, the Ruby option is set to the From 9ca9950282a1646c568c0e7a4034ab391ab19f15 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Fri, 13 Mar 2020 15:04:26 -0400 Subject: [PATCH 250/442] Automatic loading of AS support for driver was added in Mongoid 7.0.6, per MONGOID-4770 --- source/reference/driver-compatibility.txt | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 224bde0df..d525e2cb3 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -357,9 +357,8 @@ compatibility code for behavior like time serialization to be correct: require 'mongo' require 'mongo/active_support' -Applications using Mongoid 7.0.3 or newer -do not need to explicitly load the driver's ActiveSupport code, since Mongoid -automatically does so. +Applications using Mongoid 7.0.6 or newer do not need to explicitly load +the driver's ActiveSupport code, since Mongoid automatically does so. Atlas Compatibility ~~~~~~~~~~~~~~~~~~~ From cccbfea0feec1341d7127b2128341ffaec51c7cf Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Fri, 13 Mar 2020 15:05:38 -0400 Subject: [PATCH 251/442] Update driver compatibility matrix for 2.12 release (#1786) --- source/reference/driver-compatibility.txt | 39 +++++++++++++++++++---- 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index d525e2cb3..a726f754f 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -28,8 +28,18 @@ MongoDB Compatibility - MongoDB 2.6 - MongoDB 2.4 - * - 2.11 + * - 2.12 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| - |checkmark| + - + + * - 2.11 + - |checkmark| [#client-side-encryption]_ - |checkmark| - |checkmark| - |checkmark| @@ -39,7 +49,7 @@ MongoDB Compatibility - * - 2.10 - - |checkmark| [#srv-polling]_ + - |checkmark| [#srv-polling]_ [#client-side-encryption]_ - |checkmark| - |checkmark| - |checkmark| @@ -141,6 +151,9 @@ MongoDB Compatibility .. [#srv-polling] Polling of SRV records in sharded topologies is implemented as of driver version 2.11. +.. [#client-side-encryption] Client-side encryption is implemented as of + driver version 2.12. + The driver does not support older versions of MongoDB. .. _reference-compatibility-language-ruby: @@ -174,16 +187,30 @@ for that Ruby version is deprecated. - JRuby 9.1 - JRuby + * - 2.12 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + - + - + - + - |checkmark| + - + - + * - 2.11 - |checkmark| - |checkmark| - |checkmark| - |checkmark| - |checkmark| - - - - - - - - + - + - + - + - - |checkmark| - - From ffddbc927ea8e8555ecdee6d571b87bae19495f7 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 13 Mar 2020 15:24:44 -0400 Subject: [PATCH 252/442] RUBY-2175 Change all instances of "FLE/Field-Level Encryption" to say "Client-Side Encryption" (#1785) Co-authored-by: Oleg Pudeyev --- source/tutorials/client-side-encryption.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/client-side-encryption.txt b/source/tutorials/client-side-encryption.txt index 76357d7ef..8d921eca9 100644 --- a/source/tutorials/client-side-encryption.txt +++ b/source/tutorials/client-side-encryption.txt @@ -17,8 +17,8 @@ to encrypt specific fields in MongoDB documents before inserting them into the database. With client-side encryption, developers can encrypt fields client-side without -any server-side configuration or directives. Client-side field-level encryption -supports workloads where applications must guarantee that unauthorized parties, +any server-side configuration or directives. Client-side encryption supports +workloads where applications must guarantee that unauthorized parties, including server administrators, cannot read the encrypted data. .. warning:: From 15c51bfc1defa2b1b540800b7af1dd3c53b9ef9d Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Fri, 13 Mar 2020 15:32:38 -0400 Subject: [PATCH 253/442] release 2.12.0.rc0 --- source/installation.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/installation.txt b/source/installation.txt index 086f14c8d..ee0116a12 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -19,7 +19,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.11.0.rc0 + gem install mongo -v 2.12.0.rc0 What's New From 3cdeb7ad64f7194b7bcf99f05288ce7af10bd6e4 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 20 Mar 2020 16:09:02 -0400 Subject: [PATCH 254/442] Fix sdam_proc test and expand user documentation for sdam_proc option (#1801) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 21 +++++++++++++++++++ source/tutorials/ruby-driver-monitoring.txt | 10 +++++++++ 2 files changed, 31 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 0d746fad5..876827b22 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -400,6 +400,27 @@ Ruby Options - ``Boolean`` - true + * - ``:sdam_proc`` + - Since the client begins monitoring the deployment in background as + soon as it is constructed, constructing a client and then subscribing + to `SDAM `_ events in a separate statement may result in the + subscriber not receiving some of the SDAM events. The ``:sdam_proc`` + option permits adding event subscribers on the client being constructed + before any SDAM events are published. + + Pass a ``Proc`` which will be called with the ``Client`` as the argument + after the client's event subscription mechanism has been initialized + but before any of the servers are added to the client. Use this + ``Proc`` to set up SDAM event subscribers on the client. + + Note: the client is not fully constructed when the ``Proc`` provided in + ``:sdam_proc is invoked, in particular the cluster is nil at this time. + ``:sdam_proc`` procedure should limit itself to calling + ``Client#subscribe`` and ``Client#unsubscribe`` methods on on the + passed client only. + - ``Proc`` + - none + * - ``:server_selection_timeout`` - The number of seconds to wait for an appropriate server to be selected for an operation to be executed before raising an exception. diff --git a/source/tutorials/ruby-driver-monitoring.txt b/source/tutorials/ruby-driver-monitoring.txt index 925689a15..ffaa13a1a 100644 --- a/source/tutorials/ruby-driver-monitoring.txt +++ b/source/tutorials/ruby-driver-monitoring.txt @@ -92,6 +92,8 @@ Sample output: D, [2018-09-23T13:47:31.259145 #4692] DEBUG -- : COMMAND | 127.0.0.1:27027 | test.ismaster | SUCCEEDED | 0.000791175s +.. _sdam: + Server Discovery And Monitoring ------------------------------- @@ -266,6 +268,14 @@ Sample output: D, [2018-10-09T13:58:05.342565 #22079] DEBUG -- : SDAM | Server localhost:27102 connection closed. D, [2018-10-09T13:58:05.342693 #22079] DEBUG -- : SDAM | Topology type 'ReplicaSetWithPrimary' closed. +.. note:: + + ``:sdam_proc`` client option applies only to the client during whose + construction it is given. When certain client options are changed via the + ``Client#with`` call, a new cluster may be created by the driver with + a default set of event subscribers. If this happens, the provided + ``:sdam_proc`` is not called and the application may miss events. + Server Heartbeats ----------------- From d791107dae941bffe6fbc6069bbd1e8143f20c8a Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 24 Mar 2020 12:13:54 -0400 Subject: [PATCH 255/442] RUBY-2145 fix typo in documentation - MongoDB time resolution is milliseconds (#193) Co-authored-by: Oleg Pudeyev --- docs/tutorials/bson-v4.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 275dedc18..d4d7878ad 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -559,7 +559,7 @@ Time Instances -------------- Times in Ruby can have nanosecond precision. Times in BSON (and MongoDB) -can only have microsecond precision. When Ruby ``Time`` instances are +can only have millisecond precision. When Ruby ``Time`` instances are serialized to BSON or Extended JSON, the times are floored to the nearest millisecond. From d0ef57c4723cc1bd9b72b93052af7379aaefb3e1 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 6 Apr 2020 13:54:17 -0400 Subject: [PATCH 256/442] RUBY-1552 Document Ruby Kerberos usage (#1845) * RUBY-1552 remove pre-2.6 server language The driver only supports 2.6+ servers, therefore there is no need to have a minimum server requirement of 2.6 or lower. * RUBY-1552 add mongo_kerberos compatibility to the compat tables * RUBY-1552 revise kerberos auth instructions Co-authored-by: Oleg Pudeyev --- source/includes/unicode-nbsp.rst | 2 + source/reference/driver-compatibility.txt | 26 +++++++ .../tutorials/ruby-driver-authentication.txt | 74 ++++++++++++------- 3 files changed, 75 insertions(+), 27 deletions(-) create mode 100644 source/includes/unicode-nbsp.rst diff --git a/source/includes/unicode-nbsp.rst b/source/includes/unicode-nbsp.rst new file mode 100644 index 000000000..aa0e52ea7 --- /dev/null +++ b/source/includes/unicode-nbsp.rst @@ -0,0 +1,2 @@ +.. |nbsp| unicode:: 0xA0 + :trim: diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index a726f754f..627150c04 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -6,6 +6,12 @@ Driver Compatibility .. default-domain:: mongodb +.. contents:: On this page + :local: + :backlinks: none + :depth: 1 + :class: singlecol + .. _reference-compatibility-mongodb-ruby: MongoDB Compatibility @@ -398,6 +404,25 @@ When running on JRuby and connecting to Atlas Free Tier, `driver version 2.6.4 `_ or higher and Java 8 or higher are required. +``mongo_kerberos`` Compatibility +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following compatibility table specifies the version(s) of the +:ref:`mongo_kerberos library ` to use with a specific version of +the driver. + +.. list-table:: + :header-rows: 1 + :stub-columns: 1 + :class: compatibility-large no-padding + + * - Ruby Driver + - mongo_kerberos |nbsp| 2.1 + + * - 2.7 - 2.12 + - |checkmark| + + JRuby Kerberos ~~~~~~~~~~~~~~ @@ -408,3 +433,4 @@ will also be affected by this setting, meaning any TGTs in the system cache will obtaining Kerberos credentials as well. .. include:: /includes/unicode-checkmark.rst +.. include:: /includes/unicode-nbsp.rst diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index 86cd2d01e..1c106cc24 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -180,7 +180,7 @@ see the :manual:`X.509 tutorial in the MongoDB Manual LDAP (SASL PLAIN) ````````````````` -*Requires MongoDB Enterprise Edition v2.6 or greater.* +*Requires MongoDB Enterprise Edition.* MongoDB Enterprise Edition supports the LDAP authentication mechanism which allows you to delegate authentication using a Lightweight Directory @@ -211,52 +211,72 @@ MongoDB, see the :manual:`SASL/LDAP tutorial in the MongoDB Manual Kerberos (GSSAPI) ````````````````` -*Requires MongoDB Enterprise Edition v2.4 or greater.* +*Requires MongoDB Enterprise Edition.* -MongoDB Enterprise Edition v2.4+ supports Kerberos authentication. +To configure the MongoDB server to use Kerberos, please refer to the +:manual:`server Kerberos documentation +`. -To use Kerberos authentication from Ruby, -`mongo_kerberos `_ must be -installed. Add to your ``Gemfile``: +To use the Kerberos authentication mechanism with the Ruby MongoDB driver, +an additional library implementing the Kerberos authenticator - +`mongo_kerberos `_ - must be +installed and loaded. To do so, add to your ``Gemfile``: .. code-block:: ruby - gem 'mongo' - gem 'mongo_kerberos' + gem 'mongo', '~> 2' + gem 'mongo_kerberos', '~> 2' -... and in your application: +... and add to your application code: .. code-block:: ruby require 'mongo' require 'mongo_kerberos' -To use Kerberos in the Ruby driver with **MRI**, create a ticket-granting -ticket using ``kinit``. See -`this documentation `_ for more -information. +If using Kerberos authentication with **MRI**, the password is not specified +in driver configuration and it is not sent to the MongoDB server by the driver. +Instead a Kerberos session must be established externally to the driver +and this session is used by the driver to prove the user's identity to +the server. Establishing this session requires that the host system is +configured for Kerberos authentication; refer to the `Kerberos documentation +`_ +or your operating system documentation for details. Use the `kinit utility +`_ +to establish a Kerberos session. + +If using Kerberos authentication with **JRuby**, the Kerberos session may +be estabished externally to the driver using the process described above +for MRI; alternatively, the password may be provided directly to the driver +via client configuration, or the path to a keytab file may be provided via +configuration stored in the ``java.security.auth.login.config`` system property. +Additionally, the Java runtime environment must be configured for Kerberos; +please refer to the `MongoDB Java Driver Kerberos documentation +`_ +for more information. -To use Kerberos in the Ruby driver with **JRuby**, do the following: +.. note:: -1. Specify several system properties so that the underlying GSSAPI Java - libraries can acquire a Kerberos ticket. See the `MongoDB Java - Driver authentication documentation - `_ - for more information. + As per the server Kerberos documentation, the FQDN of the host + running MongoDB must be specified when using Kerberos authentication. -2. Either provide a password OR set the 'java.security.auth.login.config' - system property to a config file that references a keytab file. +.. note:: -For more information about deploying MongoDB with Kerberos -authentication, see the :manual:`manual -`. + If using MongoDB URIs, be sure to percent-escape special characters like + ``/`` and ``@`` when they appear in the username. .. code-block:: ruby - client = Mongo::Client.new([ '127.0.0.1:27017' ], + # Authenticate as appuser@MYREALM: + client = Mongo::Client.new("mongodb://appuser%40MYREALM@myserver.mycompany.com:27017/mydb?authMechanism=GSSAPI") + + # Authenticate as myapp/appuser@MYREALM: + client = Mongo::Client.new("mongodb://myapp%2Fappuser%40MYREALM@myserver.mycompany.com:27017/mydb?authMechanism=GSSAPI") + + # Authenticate using Ruby options: + client = Mongo::Client.new(['myserver.mycompany.com:27017'], auth_mech: :gssapi, - user: 'test', - password: '123' ) + user: 'myapp/appuser@MYREALM') MONGODB-CR From d3d1c781800dafef87ada478ac5271fe18a71f5b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 10 Apr 2020 16:26:32 -0400 Subject: [PATCH 257/442] Revise database documentation, recommend against using admin db (#1857) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-authentication.txt | 4 ++ .../tutorials/ruby-driver-create-client.txt | 41 ++++++++++++++++--- 2 files changed, 40 insertions(+), 5 deletions(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index 1c106cc24..ab345811a 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -146,6 +146,8 @@ value for the ``authMechanism`` URI option, as follows: client = Mongo::Client.new("mongodb://test:123@127.0.0.1:27017/mydb?authMechanism=SCRAM-SHA-256") +.. _x.509: + Client Certificate (X.509) `````````````````````````` @@ -177,6 +179,8 @@ see the :manual:`X.509 tutorial in the MongoDB Manual ssl_ca_cert: '/path/to/ca.pem' ) +.. _plain: + LDAP (SASL PLAIN) ````````````````` diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 876827b22..53919f868 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -52,18 +52,49 @@ needed. Database ```````` -Usually the client will connect to a specific database. It is possible to -change the active database by calling ``Client#use`` to obtain a new ``Client`` -instance configured to use the new database. For example, it is common to -perform operations on the ``admin`` database: +By default, the client will connect to the ``admin`` database. + +The ``admin`` database is a special database in MongoDB often used for +administrative tasks and storing administrative data such as users and +roles (although users and roles may also be defined in other databases). +In a sharded cluster, the ``admin`` database +:manual:`exists on the config servers ` +rather than the shard servers. Although it is possible to use the ``admin`` +database for ordinary operations (such as storing application data), this +is not recommended and the application should explicitly specify the +database it wishes to use. + +The database can be specified during ``Client`` construction: .. code-block:: ruby - client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'mydb') + # Using Ruby client options: + client = Mongo::Client.new(['localhost'], database: 'mydb') + + # Using a MongoDB URI: + client = Mongo::Client.new('mongodb://localhost/mydb') + +Given a ``Client`` instance, the ``use`` method can be invoked to obtain a +new ``Client`` instance configured with the specified database: + +.. code-block:: ruby + + client = Mongo::Client.new(['localhost'], database: 'mydb') + admin_client = client.use('admin') + # Issue an administrative command admin_client.database.command(replSetGetConfig: 1).documents.first +There are other special databases in MongoDB which should be only used for +their stated purposes: + +- The :manual:`config ` database. +- The :manual:`local ` database. +- The ``$external`` database, which is used with :ref:`PLAIN `, + :ref:`Kerberos ` and :ref:`X.509 ` authentication + mechanisms. + Direct Connection ````````````````` From ebb859dcf45fab2aae379a93b7ec1996529cd59f Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 17 Apr 2020 12:52:06 -0400 Subject: [PATCH 258/442] RUBY-1999 user documentation for AWS auth mechanism (#1871) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-authentication.txt | 169 +++++++++++++++++- 1 file changed, 165 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index ab345811a..eda1d3040 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -31,10 +31,11 @@ client: client = Mongo::Client.new([ '127.0.0.1:27017' ], user: 'test', - password: '123' ) + password: '123', + database: 'mydb' ) # If using a URI: - client = Mongo::Client.new("mongodb://test:123@127.0.0.1:27017") + client = Mongo::Client.new("mongodb://test:123@127.0.0.1:27017/mydb") Authentication credentials can be changed on a client instance to obtain a new client using the ``Client#with`` method: @@ -85,7 +86,7 @@ connecting; if no database is specified, ``admin`` database is the default database and hence the default auth source. For the ``PLAIN`` mechanism (LDAP), the default auth source is the database to which the client is connecting; if no database is specified, the ``$external`` database is used as the -auth source. For the ``GSSAPI`` and ``MONGODB_X509`` mechanisms, the +auth source. For the ``AWS``, ``GSSAPI`` and ``MONGODB_X509`` mechanisms, the auth source is always ``$external``. .. note:: @@ -179,6 +180,166 @@ see the :manual:`X.509 tutorial in the MongoDB Manual ssl_ca_cert: '/path/to/ca.pem' ) +AWS +``` + +*Requires MongoDB Enterprise Edition and server version 4.4 or later.* + +The AWS authentication mechanism uses AWS `Identity and Access Management (IAM) +`_ +and AWS `Security Token Service (STS) +`_ +to prove the client's identity to a MongoDB server. Briefly, AWS authentication +works as follows: + +1. The client uses AWS IAM credentials to create a signature that is sent to + the MongoDB server. +2. The server sends a request to AWS STS using the client's signature. +3. A successful STS request returns the username (technically, the ARN of + the IAM user or role) corresponding to the credentials that the client used. + The IAM user ARN is used by the server to look up a defined user, and the + client is considered to have authenticated as this user. + +.. note:: + + Unlike other authentication mechanisms, the username that the application + provides when creating a client and the username of the server user are + different: the username on the client is the AWS access key ID, but the + username on the server is the ARN of the IAM user or role corresponding + to the access key ID. + +AWS credentials are comprised of: + +- The access key ID. +- The secret access key. +- The optional session token. + +Authentication with `AWS IAM credentials +`_, +uses the access key ID and the secret access key. Authentication with +`temporary AWS IAM credentials +`_ +uses all three components. + +.. note:: + + The driver never sends the secret access key or the session token over + the network. + +Temporary credentials are used with: + +- STS `Assume Role `_ + requests. +- `EC2 instance roles `_. +- `ECS task roles `_. +- `AWS Lambda environment `_. + +The Ruby driver allows providing both regular and temporary credentials +explicitly as Ruby options or URI options. If credentials are not explicitly +provided, the driver will attempt to retrieve them from environment variables +described below and from EC2 instance and ECS task metadata endpoints. + +Providing Credentials Explicitly +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Regular (non-temporary) IAM credentials can be provided as Ruby options, +as follows: + +.. code-block:: ruby + + client = Mongo::Client.new(['mongodb.example.com'], + auth_mech: :aws, + user: '', + password: '', + database: 'mydb', + ) + +They can also be provided via a URI: + +.. code-block:: ruby + + client = Mongo::Client.new( + 'mongodb://:@mongodb.example.com/mydb?authMechanism=MONGODB-AWS') + +.. note:: + + When credentials are provided via a URI, they must be percent-escaped. + +To provide temporary credentials, specify the session token in the +authentication mechanism properties as follows: + +.. code-block:: ruby + + client = Mongo::Client.new(['mongodb.example.com'], + auth_mech: :aws, + user: '', + password: '', + auth_mech_properties: { + aws_session_token: '', + }, + database: 'mydb', + ) + +The temporary credentials can also be provided via a URI: + +.. code-block:: ruby + + client = Mongo::Client.new( + 'mongodb://:@mongodb.example.com/mydb?authMechanism=MONGODB-AWS&authMechanismProperties=AWS_SESSION_TOKEN:') + +Automatically Retrieving Credentials +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The client can retrieve credentials from the environment or from EC2 or ECS +metadata endpoints. To retrieve credentials automatically, specify the +AWS authentication mechanism but do not specify a username nor a password: + +.. code-block:: ruby + + client = Mongo::Client.new(['mongodb.example.com'], + auth_mech: :aws, + database: 'mydb', + ) + + # Using a URI: + client = Mongo::Client.new( + 'mongodb://mongodb.example.com/mydb?authMechanism=MONGODB-AWS') + +The driver will try to obtain credentials from the following sources, in +the specified order: + +- ``AWS_ACCESS_KEY_ID``, ``AWS_SECRET_ACCESS_KEY`` and ``AWS_SESSION_TOKEN`` + environment variables. These environment variables are recognized by + a variety of AWS-related libraries and tools such as the official + AWS Ruby SDK and the AWS CLI. They are also defined when running in an + AWS Lambda environment. +- The AWS `ECS task metadata endpoint + `_. + This returns credentials associated with the ECS task role assigned to + the container. +- The AWS `EC2 instance metadata endpoint + `_. + This returns credentials associated with the EC2 instance role assigned to + the instance. + +.. note:: + + A credentials source that provides any credentials must provide a complete + set of credentials. For example, the driver will raise an error if only + one of ``AWS_ACCESS_KEY_ID`` or ``AWS_SECRET_ACCESS_KEY`` environment + variables is populated but not the other. + +.. note:: + + If an application is running in an ECS container on an EC2 instance and + `the container is allowed access to the instance metadata + `_, + the driver will attempt to retrieve credentials for the AWS authentication + mechanism from the EC2 instance metadata endpoint, thus potentially + authenticating as the IAM role assigned to the EC2 instance, if it was not + able to retrieve ECS task role credentials from the ECS task endpoint. + + .. _plain: LDAP (SASL PLAIN) @@ -297,7 +458,7 @@ The mechanism can be explicitly set with the credentials: .. code-block:: ruby client = Mongo::Client.new([ '127.0.0.1:27017' ], - database: 'music', + database: 'mydb', user: 'test', password: '123', auth_mech: :mongodb_cr ) From 852fd9207bc34d6d3be24a9b3b15316d4bca1a43 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 17 Apr 2020 15:50:08 -0400 Subject: [PATCH 259/442] RUBY-2219 Force read preference primaryPreferred in Single topology (#1868) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-crud-operations.txt | 34 ++++++++++++++----- 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 61a2e8516..45cefa1cb 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -292,9 +292,9 @@ Read Preference ~~~~~~~~~~~~~~~ Read preference determines the candidate :manual:`replica set` -members to which a query or command can be sent. They consist of a **mode** specified as -a symbol, an array of hashes known as **tag_sets**, and two timing options: -**local_threshold** and **server_selection_timeout**. +members to which a query or command can be sent. They consist of a **mode** +specified as a symbol, an array of hashes known as **tag_sets**, and two +timing options: **local_threshold** and **server_selection_timeout**. ``local_threshold`` Defines the upper limit in seconds of the latency window @@ -305,6 +305,12 @@ a symbol, an array of hashes known as **tag_sets**, and two timing options: Defines how long to block for server selection before throwing an exception. The default is 30,000 milliseconds, or 30 seconds. +.. note:: + + Read preference does not apply to Standalone deployments. When a client + is connected to a Standalone deployment, any application-specified read + preference is ignored. + For more information on the algorithm used to select a server, please refer to the `Server Selection documentation, available on GitHub ` for an explanation of the modes and tag sets. +There are five possible read preference modes: ``:primary``, ``:secondary``, +``:primary_preferred``, ``:secondary_preferred`` and``:nearest``. +Please see the :manual:`read preference documentation in the MongoDB Manual +` for an explanation of the modes. + +.. note:: + + When a client is directly connected to a server using the ``:direct_connection`` + Ruby option or the ``directConnection`` URI option, read preference mode + is automatically set to ``:primary_preferred`` to permit read operations + against secondaries. If the application specified a ``:primary`` read + preference mode, the mode is automatically converted to ``:primary_preferred``. + If another read preference mode is specified, it is passed to the server + unchanged. Tag sets ~~~~~~~~ The ``tag_sets`` parameter is an ordered list of tag sets used to restrict the eligibility of servers for selection, such as for data -center awareness. +center awareness. Please see the :manual:`read preference documentation in +the MongoDB Manual ` for an explanation of tag sets. + A read preference tag set (T) matches a server tag set (S) – or equivalently a server tag set (S) matches a read preference tag set From f4f39f40fc8f1aeb7adfd330ab9b2971f4f0dec8 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 20 Apr 2020 12:43:40 -0400 Subject: [PATCH 260/442] RUBY-1908 Provide transaction example using new withTransaction API (#1880) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-create-client.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 53919f868..ed7e07398 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -598,6 +598,7 @@ Ruby Options * - ``:write_concern`` - Specifies write concern options as a ``Hash``. Keys in the hash can be ``:w``, ``:wtimeout``, ``:j``, ``:fsync``. + Note that ``:wtimeout`` is specified in milliseconds, not seconds. .. code-block:: ruby From ea8630b26cb788e5ec5ebf56e110af251366a504 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 20 Apr 2020 12:43:57 -0400 Subject: [PATCH 261/442] Fix RUBY-2032 Tutorial references nonexistent tls Ruby option (#1881) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-create-client.txt | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index ed7e07398..2259fd7ba 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -879,10 +879,11 @@ on the client or passed to individual operations under ``:write_concern``. TLS Connections --------------- -To connect to the MongoDB cluster using TLS, pass ``tls`` Ruby or URI option -to the ``Mongo::Client`` constructor. TLS must be explicitly configured on the -client side when the cluster requires TLS connections - there is currently no -automatic detection of whether TLS is required. +To connect to the MongoDB cluster using TLS, pass the ``ssl`` Ruby option or +the ``tls`` URI option to the ``Mongo::Client`` constructor. TLS must be +explicitly requested on the client side when the deployment requires TLS +connections - there is currently no automatic detection of whether the +deployment requires TLS. The driver will attempt to verify the server's TLS certificate by default, and will abort the connection if this verification fails. In order for this From 5a9316ca01df1d1760697483cf0f2868b54d2d66 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Wed, 22 Apr 2020 18:08:01 -0400 Subject: [PATCH 262/442] release 2.12.1 --- source/installation.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/installation.txt b/source/installation.txt index ee0116a12..358d8c331 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -19,7 +19,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.12.0.rc0 + gem install mongo -v 2.12.1 What's New From 57b53e6c9b719eaee729af9517bfc284df9a7c39 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 29 Apr 2020 12:20:58 -0400 Subject: [PATCH 263/442] RUBY-1737 Describe default auth mechanism selection and necessity to specify mongodb_cr explicitly (#1900) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-authentication.txt | 32 ++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index eda1d3040..da35825c1 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -103,6 +103,28 @@ auth source is always ``$external``. Authentication Mechanisms ------------------------- +MongoDB supports several authentication mechanisms, as detailed in this section. +Authentication mechanism to use can be explicitly specified when a Client is +created; if authentication mechanism is not provided by the application, it is +selected as follows: + +- For MongoDB 4.0 and higher, the client performs SCRAM mechanism negotiation + with the server. If the user specified in client configuration permits + authentication with SCRAM-SHA-256, then SCRAM-SHA-256 is used for + authentication. Otherwise SCRAM-SHA-1 is used. +- For MongoDB 3.0 through 3.6, SCRAM-SHA-1 is used. +- For MongoDB 2.6, MONGODB-CR is used. + +Note that: + +- X.509, AWS, LDAP and Kerberos authentication mechanisms must always be + explicitly requested. +- If the MongoDB server that the client is connecting to supports SCRAM, + the client will attempt to authenticate using SCRAM if no authentication + mechanism is explicitly specified. To authenticate to MongoDB 3.0 and + higher servers using MONGODB-CR, the MONGODB-CR mechanism must be + explicitly requested. + .. _scram: SCRAM @@ -450,7 +472,7 @@ MONGODB-CR *Deprecated:* MONGODB-CR mechanism is deprecated as of MongoDB 3.6 and removed as of MongoDB 4.0. Please use `SCRAM authentication `_ instead. -MONGODB-CR was the default authentication mechanism for MongoDB up through +MONGODB-CR was the default authentication mechanism for MongoDB through version 2.6. The mechanism can be explicitly set with the credentials: @@ -462,3 +484,11 @@ The mechanism can be explicitly set with the credentials: user: 'test', password: '123', auth_mech: :mongodb_cr ) + +.. note:: + + If the MongoDB server that the client is connecting to supports SCRAM, + the client will attempt to authenticate using SCRAM if no authentication + mechanism is explicitly specified. To authenticate to MongoDB 3.0 and + higher servers using MONGODB-CR, the MONGODB-CR mechanism must be + explicitly requested. From cd79aaa9c7077a782e8556ef5b2ec433c3265961 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Mon, 11 May 2020 16:40:16 -0400 Subject: [PATCH 264/442] Fix list syntax --- .../tutorials/ruby-driver-create-client.txt | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 2259fd7ba..db7799e59 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -183,22 +183,22 @@ When the driver connects to a URI, keep in mind the following: 1. SRV URI lookup is performed synchronously when the client is constructed. - If this lookup fails for any reason, client construction will fail with an - exception. When a client is constructed with a list of hosts, the driver - will attempt to contact and monitor those hosts for as long as the client - object exists. If one of these hosts does not resolve initially but becomes - resolvable later, the driver will be able to establish a connection to such - a host when it becomes available. The initial SRV URI lookup must succeed - on the first attempt; subsequent host lookups will be retried by the driver - as needed. + If this lookup fails for any reason, client construction will fail with an + exception. When a client is constructed with a list of hosts, the driver + will attempt to contact and monitor those hosts for as long as the client + object exists. If one of these hosts does not resolve initially but becomes + resolvable later, the driver will be able to establish a connection to such + a host when it becomes available. The initial SRV URI lookup must succeed + on the first attempt; subsequent host lookups will be retried by the driver + as needed. 2. The driver looks up URI options in the DNS TXT records corresponding to the - SRV records. These options can be overridden by URI options specified in the - URI and by Ruby options, in this order. + SRV records. These options can be overridden by URI options specified in the + URI and by Ruby options, in this order. 3. If the topology of the constructed ``Client`` object is unknown or a - sharded cluster, the driver will begin monitoring the specified SRV DNS - records for changes and will automatically update the list of servers in the - cluster. The updates will stop if the topology becomes a single or a replica - set. + sharded cluster, the driver will begin monitoring the specified SRV DNS + records for changes and will automatically update the list of servers in the + cluster. The updates will stop if the topology becomes a single or a replica + set. .. _ruby-driver-client-options: From 0d06efd11b82b813946dbf643b1694c4433d3ac2 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 26 May 2020 19:55:29 -0400 Subject: [PATCH 265/442] RUBY-2229 Link to current support/contribution resources (#1925) * RUBY-2229 Link to current support/contribution resources * fix hyperlink Co-authored-by: Oleg Pudeyev --- source/contribute.txt | 81 +++++++++++++++++++++++++++++++------------ source/index.txt | 3 +- source/support.txt | 15 ++++++++ 3 files changed, 75 insertions(+), 24 deletions(-) create mode 100644 source/support.txt diff --git a/source/contribute.txt b/source/contribute.txt index 19163f4b5..f536b734b 100644 --- a/source/contribute.txt +++ b/source/contribute.txt @@ -4,43 +4,78 @@ Contribute to the Driver .. default-domain:: mongodb -For Bugs and Feature Requests ------------------------------ +Report Bugs and Request Ruby Driver-Specific Features +----------------------------------------------------- -If you think you have found a bug or want to see a new feature in the -Ruby driver, please open an issue in `JIRA -`_: +To report a bug in the driver or request a feature specific to the Ruby driver: -#. `Create a Jira account and login `_. +1. Visit `our issue tracker `_ and login + (or create an account if you do not have one already). +2. Navigate to the `RUBY project `_. +3. Click :guilabel:`Create Issue` and fill out all of the applicable form + fields. -#. Navigate to the `Ruby project - `_. +When creating an issue, please keep in mind that all information in JIRA +for the RUBY project, as well as the core server (the SERVER project), +is publicly visible. -#. Click :guilabel:`Create Issue`. Provide as much information as - possible about the issue, including: +**PLEASE DO:** - - Steps to reproduce it. +- Provide as much information as possible about the issue. +- Provide detailed steps for reproducing the issue. +- Provide any applicable code snippets, stack traces and log data. + Do not include any sensitive data or server logs. +- Specify version numbers of the driver and MongoDB server. - - Version information for the driver and MongoDB. +**PLEASE DO NOT:** - - Any applicable code snippets, stack traces and log data. Do not - include any sensitive data or server logs. +- Provide any sensitive data or server logs. +- Report potential security issues publicly (see 'Security Issues' below). .. note:: - Bug reports in JIRA for the driver and the Core Server (i.e. **SERVER**) + Bug reports in JIRA for the Ruby driver and the core server (the **SERVER**) projects are public. -If you identify a security vulnerability in a driver or any other -MongoDB project, please report it according to the instructions found +If you identified a potential security vulnerability in the Ruby driver or +any other MongoDB product, please report it according to the instructions found in the :manual:`Create a Vulnerability Report `. -Contribute to the MongoDB Ruby Driver -------------------------------------- -The MongoDB Ruby driver source is located at -``_. +Request Product Features +------------------------ -For instructions on contributing to the driver, see -``_. +To request a feature which is not specific to the Ruby driver, or which +affects more than the driver alone (for example, a feature which requires +MongoDB server support), please submit your idea through the +`MongoDB Feedback Forum `_. + + +Contribute Code +--------------- + +The MongoDB Ruby driver source is located +`at GitHub `_. + +The list of known issues in the driver is available +`in JIRA `_. + +We recommend creating a JIRA ticket before starting work on a bug fix or +an improvement to the driver, to obtain feedback from the Ruby driver team +as to the proposed changes. A JIRA ticket is not required to submit +a pull request but it is appreciated, especially for non-trivial changes. + +Pull requests should be made against the ``master`` branch and +include relevant tests, if applicable. The Ruby driver team will backport +the changes to the stable branches, if needed. + +A MongoDB deployment is required to run the tests. Setup procedures and +recommendations for various deployments, as well as how to configure the +driver's test suite for the deployments, are covered in the `spec +readme `. + +The driver is tested on `Evergreen `_, +MongoDB's in-house continuous integration platform. After a pull request +is created, one of the Ruby driver team engineers will schedule an Evergreen +build. diff --git a/source/index.txt b/source/index.txt index d5ce3a96f..65b60f9f5 100644 --- a/source/index.txt +++ b/source/index.txt @@ -58,6 +58,7 @@ For tutorials on Mongoid, see the `Mongoid Manual /reference/driver-compatibility /reference/additional-resources - /contribute + contribute diff --git a/source/support.txt b/source/support.txt new file mode 100644 index 000000000..3a6f0ff00 --- /dev/null +++ b/source/support.txt @@ -0,0 +1,15 @@ +======= +Support +======= + +.. default-domain:: mongodb + +Commercial support for the Ruby driver is available through the +`MongoDB Support Portal `_. + +For questions, discussions or general technical support, please visit the +`MongoDB Community Forum +`_. + +Please see :manual:`Technical Support ` page +in the documentation for other support resources. From 739c97b114ebb91ab131f197449f29f8716a6651 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 27 May 2020 15:34:30 -0400 Subject: [PATCH 266/442] RUBY-2240 Use consistent heading symbols in tutorial (#1931) Co-authored-by: Oleg Pudeyev --- source/contribute.txt | 10 +++---- source/index.txt | 10 +++---- source/installation.txt | 10 +++---- source/meta/404.txt | 4 +-- source/nesting-levels.txt | 17 +++++++++++ source/quick-start.txt | 22 +++++++------- source/reference/additional-resources.txt | 14 ++++----- source/reference/driver-compatibility.txt | 16 +++++----- source/ruby-driver-tutorials.txt | 4 +-- source/support.txt | 4 +-- source/tutorials/client-side-encryption.txt | 16 +++++----- source/tutorials/ruby-driver-aggregation.txt | 4 +-- .../tutorials/ruby-driver-authentication.txt | 10 +++---- .../tutorials/ruby-driver-bulk-operations.txt | 18 +++++------ .../tutorials/ruby-driver-change-streams.txt | 14 ++++----- source/tutorials/ruby-driver-collations.txt | 10 +++---- .../ruby-driver-collection-tasks.txt | 4 +-- .../tutorials/ruby-driver-create-client.txt | 30 +++++++++---------- .../tutorials/ruby-driver-crud-operations.txt | 18 +++++------ .../tutorials/ruby-driver-database-tasks.txt | 6 ++-- .../ruby-driver-geospatial-search.txt | 4 +-- source/tutorials/ruby-driver-gridfs.txt | 22 +++++++------- source/tutorials/ruby-driver-indexing.txt | 10 +++---- source/tutorials/ruby-driver-monitoring.txt | 14 ++++----- source/tutorials/ruby-driver-projections.txt | 4 +-- source/tutorials/ruby-driver-sessions.txt | 14 ++++----- source/tutorials/ruby-driver-text-search.txt | 4 +-- source/tutorials/ruby-driver-transactions.txt | 12 ++++---- source/tutorials/user-management.txt | 14 ++++----- 29 files changed, 178 insertions(+), 161 deletions(-) create mode 100644 source/nesting-levels.txt diff --git a/source/contribute.txt b/source/contribute.txt index f536b734b..d25dda4e8 100644 --- a/source/contribute.txt +++ b/source/contribute.txt @@ -1,11 +1,11 @@ -======================== +************************ Contribute to the Driver -======================== +************************ .. default-domain:: mongodb Report Bugs and Request Ruby Driver-Specific Features ------------------------------------------------------ +===================================================== To report a bug in the driver or request a feature specific to the Ruby driver: @@ -44,7 +44,7 @@ in the :manual:`Create a Vulnerability Report Request Product Features ------------------------- +======================== To request a feature which is not specific to the Ruby driver, or which affects more than the driver alone (for example, a feature which requires @@ -53,7 +53,7 @@ MongoDB server support), please submit your idea through the Contribute Code ---------------- +=============== The MongoDB Ruby driver source is located `at GitHub `_. diff --git a/source/index.txt b/source/index.txt index 65b60f9f5..d7e881efc 100644 --- a/source/index.txt +++ b/source/index.txt @@ -2,9 +2,9 @@ .. _ruby-language-center: -=================== +******************* Ruby MongoDB Driver -=================== +******************* .. default-domain:: mongodb @@ -14,14 +14,14 @@ can be used on its own, but it also serves as the basis of several object mapping libraries. Get Started ------------ +=========== To get started with the Ruby driver, see :doc:`/installation` and :doc:`/quick-start`. Continue to :doc:`/ruby-driver-tutorials` for high level documentation for common operations. BSON ----- +==== The Ruby BSON implementation is packaged in a separate gem with C and Java extensions for speed depending on the runtime enviroment. @@ -29,7 +29,7 @@ Java extensions for speed depending on the runtime enviroment. For reference on the Ruby BSON gem, see :doc:`/bson-tutorials`. Object Mappers --------------- +============== Because MongoDB is so easy to use, the basic Ruby driver can be the best solution for many applications. But if you need validations, diff --git a/source/installation.txt b/source/installation.txt index 358d8c331..bb4c2a50e 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -1,6 +1,6 @@ -============ +************ Installation -============ +************ .. default-domain:: mongodb @@ -11,7 +11,7 @@ The Ruby driver is bundled as a gem and is hosted on `Rubygems .. _ruby-driver-install: Install the Gem ---------------- +=============== The driver can be installed manually or with bundler. @@ -23,7 +23,7 @@ To install the mongo gem manually: What's New ----------- +========== Please consult the `releases page on GitHub `_ for the list @@ -31,7 +31,7 @@ of improvements and changes in each version of the driver. TLS/SSL and the Ruby Driver ---------------------------- +=========================== Industry best practices, and some regulations, require the use of TLS 1.1 or newer. Though no application changes are required for the Ruby driver to make use of the newest protocols, some operating systems or versions may not provide diff --git a/source/meta/404.txt b/source/meta/404.txt index 0bdd725c2..4143ded8d 100644 --- a/source/meta/404.txt +++ b/source/meta/404.txt @@ -1,7 +1,7 @@ :orphan: -============== +************** File not found -============== +************** The URL you requested does not exist or has been removed. diff --git a/source/nesting-levels.txt b/source/nesting-levels.txt new file mode 100644 index 000000000..6df05d1cb --- /dev/null +++ b/source/nesting-levels.txt @@ -0,0 +1,17 @@ +This file is not part of Ruby driver documentation proper, it is an internal +reference for the nesting levels that other files should be using. + +Ruby driver documentation nesting levels: + +********** +Page Title +********** + +First Level Heading +=================== + +Second Level Heading +-------------------- + +Third Level Heading +``````````````````` diff --git a/source/quick-start.txt b/source/quick-start.txt index f9635b1ce..3b1835223 100644 --- a/source/quick-start.txt +++ b/source/quick-start.txt @@ -1,6 +1,6 @@ -=========== +*********** Quick Start -=========== +*********** .. default-domain:: mongodb @@ -11,7 +11,7 @@ Quick Start :class: singlecol Prerequisites -------------- +============= - A running MongoDB instance on localhost using the default port, 27017. - The Ruby MongoDB driver. See :ref:`installation ` @@ -24,7 +24,7 @@ Prerequisites Make a Connection ------------------ +================= Use ``Mongo::Client`` to establish a connection to a running MongoDB instance. @@ -45,7 +45,7 @@ You can also use a URI connection string: :ref:`Client options ` Access a Database and a Collection ----------------------------------- +================================== The following examples demonstrate how to access a particular database and show its collections: @@ -68,7 +68,7 @@ If the collection does not exist, the server will create it the first time you put data into it. Insert a Document ------------------ +================= To insert a single document into a collection, use the ``insert_one`` method. @@ -107,7 +107,7 @@ To insert multiple documents into a collection, use the result.inserted_count # returns 2 because two documents were inserted Query the Collection --------------------- +==================== Use the ``find`` method to create collection queries. @@ -158,7 +158,7 @@ The example should print the following: :ref:`Query Options`, :ref:`Read Preference` Update Documents ----------------- +================ There are several update methods, including ``update_one`` and ``update_many``. ``update_one`` updates a single document, while @@ -201,7 +201,7 @@ to update all the documents in the collection. :ref:`Other update options` Delete Documents ----------------- +================ Use the ``delete_one`` or ``delete_many`` methods to delete documents from a collection (either singly or several at once). @@ -233,7 +233,7 @@ matches a regular expression to find a string which begins with "S". puts result.deleted_count # returns the number of documents deleted Create Indexes --------------- +============== Use the ``create_one`` or ``create_many`` methods to create indexes singly or several at once. @@ -264,7 +264,7 @@ different from ``create_one``. :ref:`Index options ` Complete Sample App -------------------- +=================== A sample app using the Ruby driver for several common use cases is available for download from diff --git a/source/reference/additional-resources.txt b/source/reference/additional-resources.txt index c2195590e..218218f1c 100644 --- a/source/reference/additional-resources.txt +++ b/source/reference/additional-resources.txt @@ -1,8 +1,8 @@ .. _ruby-external-resources: -==================== +******************** Additional Resources -==================== +******************** .. default-domain:: mongodb @@ -17,7 +17,7 @@ learning about MongoDB and Ruby. A useful selection is listed below. If you know of others, do let us know. Screencasts ------------ +=========== - `Introduction to MongoDB - Part I `_ @@ -46,7 +46,7 @@ Screencasts Ryan Bates' RailsCast introducing Mongoid. Presentations -------------- +============= - `Introduction to MongoDB (Video) `_ @@ -66,7 +66,7 @@ Presentations human-oriented programmers). Articles --------- +======== - `Why I Think Mongo is to Databases What Rails was to Frameworks `_ @@ -96,7 +96,7 @@ Articles support new database features. Projects --------- +======== - `Capistrano Mongo Sync `_ @@ -147,7 +147,7 @@ Projects .. A URL-shortener written with Sinatra and the MongoDB Ruby driver. Libraries ---------- +========= - `ActiveExpando `_ diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 627150c04..cd88d9c42 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -1,8 +1,8 @@ .. _reference-compatibility-ruby: -==================== +******************** Driver Compatibility -==================== +******************** .. default-domain:: mongodb @@ -15,7 +15,7 @@ Driver Compatibility .. _reference-compatibility-mongodb-ruby: MongoDB Compatibility -~~~~~~~~~~~~~~~~~~~~~ +===================== .. include:: /includes/ruby-driver-compatibility-matrix-mongodb.rst @@ -165,7 +165,7 @@ The driver does not support older versions of MongoDB. .. _reference-compatibility-language-ruby: Ruby Compatibility -~~~~~~~~~~~~~~~~~~ +================== The following compatibility table specifies the recommended version(s) of the MongoDB Ruby driver for use with a specific version of @@ -378,7 +378,7 @@ for that Ruby version is deprecated. The driver does not support older versions of Ruby. Rails/ActiveSupport Compatibility -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +================================= The Ruby driver does not depend on ActiveSupport. However, when an application uses ActiveSupport or Ruby on Rails, @@ -394,7 +394,7 @@ Applications using Mongoid 7.0.6 or newer do not need to explicitly load the driver's ActiveSupport code, since Mongoid automatically does so. Atlas Compatibility -~~~~~~~~~~~~~~~~~~~ +=================== `Driver version 2.6.1 `_ or higher is recommended when using MongoDB Atlas, as this version has @@ -405,7 +405,7 @@ When running on JRuby and connecting to Atlas Free Tier, or higher and Java 8 or higher are required. ``mongo_kerberos`` Compatibility -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +================================ The following compatibility table specifies the version(s) of the :ref:`mongo_kerberos library ` to use with a specific version of @@ -424,7 +424,7 @@ the driver. JRuby Kerberos -~~~~~~~~~~~~~~ +============== If the ``mongo_kerberos`` gem is used for Kerberos authentication with JRuby, the the JVM system property "sun.security.jgss.native" to will be set to "true" in order to facilitate the use of diff --git a/source/ruby-driver-tutorials.txt b/source/ruby-driver-tutorials.txt index 70b017cd9..5145573ee 100644 --- a/source/ruby-driver-tutorials.txt +++ b/source/ruby-driver-tutorials.txt @@ -2,9 +2,9 @@ .. _ruby-driver-tutorial: -========= +********* Tutorials -========= +********* .. default-domain:: mongodb diff --git a/source/support.txt b/source/support.txt index 3a6f0ff00..b06675889 100644 --- a/source/support.txt +++ b/source/support.txt @@ -1,6 +1,6 @@ -======= +******* Support -======= +******* .. default-domain:: mongodb diff --git a/source/tutorials/client-side-encryption.txt b/source/tutorials/client-side-encryption.txt index 8d921eca9..db8907c2c 100644 --- a/source/tutorials/client-side-encryption.txt +++ b/source/tutorials/client-side-encryption.txt @@ -1,8 +1,8 @@ .. _client-side-encryption: -====================== +********************** Client-Side Encryption -====================== +********************** .. default-domain:: mongodb @@ -27,7 +27,7 @@ including server administrators, cannot read the encrypted data. have a negative performance impact. Installation ------------- +============ Client-side encryption requires the installation of additional packages. @@ -83,7 +83,7 @@ path used to spawn the daemon), it is necessary to pass different options to the section of this tutorial for more information. Automatic Encryption --------------------- +==================== Automatic encryption is a feature that allows users to configure a ``Mongo::Client`` instance to always encrypt specific document fields when @@ -206,7 +206,7 @@ master key and create data keys, see the following sections of this tutorial: - `Creating A Data Key`_ Explicit Encryption -------------------- +=================== Explicit encryption is a feature that allows users to encrypt and decrypt individual pieces of data such as strings, integers, or symbols. Explicit encryption is a community feature and does not require an enterprise build @@ -300,7 +300,7 @@ master key and create data keys, see the following sections of this tutorial: - `Creating A Data Key`_, Creating a Master Key ---------------------- +===================== Both automatic encryption and explicit encryption require an encryption master key. This master key is used to encrypt data keys, which are in turn used to encrypt user data. The master key can be generated in one of two ways: by creating a @@ -336,7 +336,7 @@ For more information about creating a master key, see the section of the MongoDB manual. Creating a Data Key -------------------- +=================== Once you have created a master key, create a data key by calling the ``#create_data_key`` method on an instance of the ``Mongo::ClientEncryption`` class. This method generates a new data key and inserts it into the key vault @@ -427,7 +427,7 @@ For more information about creating a data key, see the section of the MongoDB manual. Auto-Encryption Options ------------------------ +======================= Automatic encryption can be configured on a ``Mongo::Client`` using the ``auto_encryption_options`` option ``Hash``. This section provides an overview of the fields inside ``auto_encryption_options`` and explains how to choose their diff --git a/source/tutorials/ruby-driver-aggregation.txt b/source/tutorials/ruby-driver-aggregation.txt index f3819015d..0342adcd2 100644 --- a/source/tutorials/ruby-driver-aggregation.txt +++ b/source/tutorials/ruby-driver-aggregation.txt @@ -1,6 +1,6 @@ -=========== +*********** Aggregation -=========== +*********** .. default-domain:: mongodb diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index da35825c1..ab2f7f403 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -1,6 +1,6 @@ -============== +************** Authentication -============== +************** .. default-domain:: mongodb @@ -22,7 +22,7 @@ user management, see the :ref:`User Management tutorial`. Providing credentials ---------------------- +===================== If authentication is enabled, provide credentials when creating a new client: @@ -58,7 +58,7 @@ one step: .. _auth-source: Auth Source ------------ +=========== A user's auth source is the database where that user's authentication credentials are stored. @@ -101,7 +101,7 @@ auth source is always ``$external``. Authentication Mechanisms -------------------------- +========================= MongoDB supports several authentication mechanisms, as detailed in this section. Authentication mechanism to use can be explicitly specified when a Client is diff --git a/source/tutorials/ruby-driver-bulk-operations.txt b/source/tutorials/ruby-driver-bulk-operations.txt index a7b5fc65c..945361e21 100644 --- a/source/tutorials/ruby-driver-bulk-operations.txt +++ b/source/tutorials/ruby-driver-bulk-operations.txt @@ -1,6 +1,6 @@ -=============== +*************** Bulk Operations -=============== +*************** .. default-domain:: mongodb @@ -25,35 +25,35 @@ The ``bulk_write`` method takes three arguments: Valid bulk write operations are the following: insert_one ----------- +========== .. code-block:: ruby { :insert_one => { :x => 1 } } insert_many ------------ +=========== .. code-block:: ruby { :insert_many => [ { :x => 1 }, { :x => 2 } ] } delete_one ----------- +========== .. code-block:: ruby { :delete_one => { :filter => { :x => 1 } } } delete_many ------------ +=========== .. code-block:: ruby { :delete_many => { :filter => { :x => 1 } } } replace_one ------------ +=========== .. code-block:: ruby @@ -63,7 +63,7 @@ replace_one } update_one ----------- +========== .. code-block:: ruby @@ -73,7 +73,7 @@ update_one } update_many ------------ +=========== .. code-block:: ruby diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 9d7138502..114db9aa9 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -1,8 +1,8 @@ .. _change-streams: -============== +************** Change Streams -============== +************** .. default-domain:: mongodb @@ -38,7 +38,7 @@ getMores to be called in a loop in the background. .. _here: https://github.com/jruby/jruby/issues/4212 Watching for Changes on a Collection ------------------------------------- +==================================== A collection change stream is created by calling the ``#watch`` method on a collection: @@ -85,7 +85,7 @@ operator format: end Watching for Changes on a Database ----------------------------------- +================================== A database change stream notifies on changes on any collection within the database as well as database-wide events, such as the database being dropped. @@ -104,7 +104,7 @@ database object: Watching for Changes on a Cluster ---------------------------------- +================================= A cluster change stream notifies on changes on any collection, any database within the cluster as well as cluster-wide events. @@ -122,7 +122,7 @@ client object (not the cluster object): Closing a Change Stream ------------------------ +======================= You can close a change stream by calling its ``#close`` method: @@ -132,7 +132,7 @@ You can close a change stream by calling its ``#close`` method: Resuming a Change Stream ------------------------- +======================== The driver will automatically retry getMore operations on a change stream once. Initial aggregation is never retried. In practical terms this means diff --git a/source/tutorials/ruby-driver-collations.txt b/source/tutorials/ruby-driver-collations.txt index 70d299241..f58b63f0c 100644 --- a/source/tutorials/ruby-driver-collations.txt +++ b/source/tutorials/ruby-driver-collations.txt @@ -1,6 +1,6 @@ -========== +********** Collations -========== +********** .. default-domain:: mongodb @@ -11,7 +11,7 @@ Collations :class: singlecol Overview --------- +======== .. versionadded:: 3.4 @@ -42,7 +42,7 @@ strings. As such, the sort order of the words would be: cote < coté < côte < côté Usage ------ +===== You can specify a default collation for collections and indexes when they are created, or specify a collation for CRUD operations and @@ -135,7 +135,7 @@ value than the collation on the index. "collation" => { "locale" => "en_US", "strength" => 2 }) Operations that Support Collation ---------------------------------- +================================= All reading, updating, and deleting methods support collation. Some examples are listed below. diff --git a/source/tutorials/ruby-driver-collection-tasks.txt b/source/tutorials/ruby-driver-collection-tasks.txt index 02c311b02..aaeeb3c1d 100644 --- a/source/tutorials/ruby-driver-collection-tasks.txt +++ b/source/tutorials/ruby-driver-collection-tasks.txt @@ -1,6 +1,6 @@ -=========== +*********** Collections -=========== +*********** .. default-domain:: mongodb diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index db7799e59..02a26dee8 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -1,6 +1,6 @@ -================= +***************** Creating a Client -================= +***************** .. default-domain:: mongodb @@ -11,7 +11,7 @@ Creating a Client :class: singlecol Using ``Mongo::Client`` ------------------------ +======================= To connect to a MongoDB deployment, create a ``Mongo::Client`` object. Provide a list of hosts and options or a :manual:`connection string URI @@ -204,7 +204,7 @@ URI, keep in mind the following: .. _ruby-driver-client-options: Client Options --------------- +============== ``Mongo::Client``'s constructor accepts a number of options configuring the behavior of the driver. The options can be provided in the options hash as @@ -795,7 +795,7 @@ URI options are explained in detail in the :manual:`Connection URI reference - ``:zlib_compression_level => Integer`` Timeout Options ---------------- +=============== ``server_selection_timeout`` ```````````````````````````` @@ -877,7 +877,7 @@ on the client or passed to individual operations under ``:write_concern``. TLS Connections ---------------- +=============== To connect to the MongoDB cluster using TLS, pass the ``ssl`` Ruby option or the ``tls`` URI option to the ``Mongo::Client`` constructor. TLS must be @@ -955,7 +955,7 @@ trusted CA certificates. IPv4/IPv6 Connections ---------------------- +===================== When a client is constructed with ``localhost`` as the host name, it will attempt an IPv4 connection only (i.e. if ``localhost`` resolves to @@ -972,7 +972,7 @@ The driver does not currently implement the Happy Eyeballs algorithm. TCP Keepalive Configuration ---------------------------- +=========================== The driver sets TCP keepalive by default. The following default values are also set if the system value can be determined and if the driver default value is less than the system value. @@ -987,7 +987,7 @@ for instructions on setting these values at the system level. Details on Connection Pooling ------------------------------ +============================= ``Mongo::Client`` instances have a connection pool per server that the client is connected to. The pool creates connections on demand to support concurrent @@ -1066,7 +1066,7 @@ When ``#close`` is called on a client by any thread, all connections are closed: Usage with Forking Servers --------------------------- +========================== *Note:* Applications using Mongoid should follow `Mongoid's forking guidance `_. @@ -1120,7 +1120,7 @@ parent remains alive. Details on Retryable Reads --------------------------- +========================== The driver implements two mechanisms for retrying reads: modern and legacy. As of driver version 2.9.0, the modern mechanism is used by default, and the @@ -1192,7 +1192,7 @@ To disable all read retries, set the following client options: Details on Retryable Writes ---------------------------- +=========================== The driver implements two mechanisms for retrying writes: modern and legacy. As of driver version 2.9.0, the modern mechanism is used by default on servers @@ -1253,7 +1253,7 @@ To disable all write retries, set the following client options: ``retry_writes: false, max_write_retries: 0``. Logging -------- +======= You can either use the default global driver logger or set your own. To set your own: @@ -1292,7 +1292,7 @@ option to the client instance. Development Configuration -------------------------- +========================= Driver's default configuration is suitable for production deployment. In development, some settings can be adjusted to provide a better developer @@ -1305,7 +1305,7 @@ experience. Production Configuration ------------------------- +======================== Please consider the following when deploying an application using the Ruby driver in production: diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 45cefa1cb..eb8d61797 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -1,6 +1,6 @@ -=============== +*************** CRUD Operations -=============== +*************** .. default-domain:: mongodb @@ -14,7 +14,7 @@ CRUD operations are those which deal with creating, reading, updating, and deleting documents. Key-value Pair Notation ------------------------ +======================= Key-value pairs appear in many different contexts in the MongoDB Ruby driver, and there are some quirks of syntax with regard to how they can @@ -58,7 +58,7 @@ version of Ruby: Creating Documents ------------------- +================== To insert documents into a collection, select a collection on the client and call ``insert_one`` or ``insert_many``. @@ -131,7 +131,7 @@ object, or with ``Decimal128.from_string()``. # => BSON::Decimal128('428.79') Reading -------- +======= The Ruby driver provides a fluent interface for queries using the ``find`` method on the collection. Various options are available @@ -381,7 +381,7 @@ the empty tag set is a subset of any tag set. This means the default .. _ruby-driver-updating: Updating --------- +======== Updating documents is possible by executing a single or multiple update, or by using the ``$findAndModify`` command. @@ -474,7 +474,7 @@ the modification occurs. doc # Return the document after the update. Deleting --------- +======== ``delete_one`` @@ -502,7 +502,7 @@ Deleting .. _write-concern: Write Concern -------------- +============= All write operations in MongoDB are executed with a write concern which is the level of acknowledgment requested from MongoDB for the particular write. @@ -671,7 +671,7 @@ and the syntax is the camel case one that MongoDB server recognizes, not the underscore one that Ruby driver uses. A Note about the BSON Symbol type ---------------------------------- +================================= Because the BSON specification deprecated the BSON symbol type, the `bson` gem will serialize Ruby symbols into BSON strings when used on its own. However, in diff --git a/source/tutorials/ruby-driver-database-tasks.txt b/source/tutorials/ruby-driver-database-tasks.txt index f7c8b90f4..9e40328f7 100644 --- a/source/tutorials/ruby-driver-database-tasks.txt +++ b/source/tutorials/ruby-driver-database-tasks.txt @@ -1,6 +1,6 @@ -========= +********* Databases -========= +********* .. default-domain:: mongodb @@ -11,7 +11,7 @@ Databases :class: singlecol Databases ---------- +========= The driver provides various helpers on database objects for executing commands, getting collection lists, and administrative tasks. diff --git a/source/tutorials/ruby-driver-geospatial-search.txt b/source/tutorials/ruby-driver-geospatial-search.txt index bc6f6def6..b1626d6c0 100644 --- a/source/tutorials/ruby-driver-geospatial-search.txt +++ b/source/tutorials/ruby-driver-geospatial-search.txt @@ -1,6 +1,6 @@ -================= +***************** Geospatial Search -================= +***************** .. default-domain:: mongodb diff --git a/source/tutorials/ruby-driver-gridfs.txt b/source/tutorials/ruby-driver-gridfs.txt index ceba2dc98..66087bd9a 100644 --- a/source/tutorials/ruby-driver-gridfs.txt +++ b/source/tutorials/ruby-driver-gridfs.txt @@ -1,8 +1,8 @@ .. _gridfs: -====== +****** GridFS -====== +****** .. default-domain:: mongodb @@ -17,7 +17,7 @@ chunked files in the database, also known as the pattern "GridFS". The API allow work with Grid::File objects or with read and write streams. Creating a GridFS object ("Grid::FSBucket") -------------------------------------------- +=========================================== You can create a GridFS object by calling ``fs`` on a database, with optional arguments. ``fs`` returns a ``Grid::FSBucket`` object. @@ -55,7 +55,7 @@ For example, you can create a GridFS bucket object with a particular read prefer Working with write streams --------------------------- +========================== To upload a file to GridFS using a write stream, you can either open a stream and write to it directly or write the entire contents of an ``IO`` object to @@ -111,7 +111,7 @@ The options can be provided as the last argument to the write stream methods: Working with read streams -------------------------- +========================= To download a file from GridFS using a read stream, you can either open a read stream and read from it directly or download the entire file all at once. @@ -172,7 +172,7 @@ the underlying read streams. Please consult the API documentation for each method to determine whether it supports a particular option. Finding file metadata ---------------------- +===================== You can retrieve documents containing metadata about files in the GridFS files collection. @@ -181,7 +181,7 @@ You can retrieve documents containing metadata about files in the GridFS files c fs_bucket.find(filename: 'my-file.txt') Deleting files --------------- +============== You can delete a file by id. @@ -191,7 +191,7 @@ You can delete a file by id. Working with Grid::File objects -------------------------------- +=============================== This object can be used to wrap a file to be inserted into the database using GridFS and the object that is retrieved. @@ -239,7 +239,7 @@ The following is a full list of the available options that files support. Inserting Files ---------------- +=============== Files can be inserted into the database one at a time. File chunks are inserted by default into the ``fs.chunks`` collection and file metadata is inserted into the @@ -278,7 +278,7 @@ Files can also be streamed as an alternative to a direct insert. end Finding Files -------------- +============= To retrieve a file from the database, call ``find_one`` with the appropriate filter. @@ -299,7 +299,7 @@ Files can also be streamed as an alternative to a direct find. Deleting Files --------------- +============== To delete a file, pass the file object to ``delete_one``. diff --git a/source/tutorials/ruby-driver-indexing.txt b/source/tutorials/ruby-driver-indexing.txt index be120ea2d..4cb4c78f1 100644 --- a/source/tutorials/ruby-driver-indexing.txt +++ b/source/tutorials/ruby-driver-indexing.txt @@ -1,6 +1,6 @@ -======== +******** Indexing -======== +******** .. default-domain:: mongodb @@ -21,7 +21,7 @@ The driver provides the ability to create, drop and view Creating Indexes ----------------- +================ Indexes can be created one at a time, or several can be created in a single operation. When creating multiple indexes on MongoDB 3.0 and later, the indexes @@ -98,7 +98,7 @@ when creating indexes. These options mirror the options supported by the Dropping Indexes ----------------- +================ To drop an index, call ``indexes#drop_one`` or ``indexes#drop_all``. @@ -112,7 +112,7 @@ To drop an index, call ``indexes#drop_one`` or ``indexes#drop_all``. Listing Indexes ---------------- +=============== To list the indexes, iterate the ``indexes`` object: diff --git a/source/tutorials/ruby-driver-monitoring.txt b/source/tutorials/ruby-driver-monitoring.txt index ffaa13a1a..4cc955ca2 100644 --- a/source/tutorials/ruby-driver-monitoring.txt +++ b/source/tutorials/ruby-driver-monitoring.txt @@ -1,6 +1,6 @@ -========== +********** Monitoring -========== +********** .. default-domain:: mongodb @@ -22,7 +22,7 @@ These events are organized into the following categories: Topology and server events are part of Server Discovery and Monitoring (SDAM). Command Monitoring ------------------- +================== All user-initiated commands that are sent to the server publish events that can be subscribed to for fine grained information. The monitoring API @@ -95,7 +95,7 @@ Sample output: .. _sdam: Server Discovery And Monitoring -------------------------------- +=============================== The Ruby driver implements `Server Discovery And Monitoring (SDAM) specification `_. @@ -278,7 +278,7 @@ Sample output: Server Heartbeats ------------------ +================= The application can be notified of each server heartbeat by subscribing to SERVER_HEARTBEAT topic. A server heartbeat listener must implement @@ -343,7 +343,7 @@ Sample output: D, [2018-09-23T13:44:10.707778 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | SUCCEEDED | 0.000772381s Connection Pool And Connection Monitoring ------------------------------------------ +========================================= Each client maintains a connection pool for each server in the deployment that it is aware of, and publishes events for both connection pools and individual @@ -398,7 +398,7 @@ Sample output: Disabling Monitoring --------------------- +==================== To turn off monitoring, set the client monitoring option to ``false``: diff --git a/source/tutorials/ruby-driver-projections.txt b/source/tutorials/ruby-driver-projections.txt index eb572ba13..5b70220e0 100644 --- a/source/tutorials/ruby-driver-projections.txt +++ b/source/tutorials/ruby-driver-projections.txt @@ -1,6 +1,6 @@ -=========== +*********** Projections -=========== +*********** .. default-domain:: mongodb diff --git a/source/tutorials/ruby-driver-sessions.txt b/source/tutorials/ruby-driver-sessions.txt index df72ef7f4..c0418e176 100644 --- a/source/tutorials/ruby-driver-sessions.txt +++ b/source/tutorials/ruby-driver-sessions.txt @@ -1,8 +1,8 @@ .. _sessions: -======== +******** Sessions -======== +******** .. default-domain:: mongodb @@ -21,7 +21,7 @@ and passed to operation methods that should be executed in the context of that s Please note that session objects are not thread safe. They must only be used by one thread at a time. Creating a session from a ``Mongo::Client`` -------------------------------------------- +=========================================== A session can be created by calling the ``start_session`` method on a client: @@ -44,7 +44,7 @@ the session, it risks getting errors due to the session going stale before it is Using a session ---------------- +=============== A session object can be passed to most driver methods so that the operation can be executed in the context of that session. Please see the API docs for which methods support a session argument. @@ -77,14 +77,14 @@ the ``Mongo::Collection::View``: Unacknowledged Writes ---------------------- +===================== Unacknowledged writes are only allowed outside the session mechanism; if an explicit session is supplied for an unacknowledged write, the driver will not send the session id with the operation. Similarly, the driver will not use an implicit session for an unacknowledged write. Causal Consistency ------------------- +================== A causally consistent session will let you read your writes and guarantee monotonically increasing reads from secondaries. To create a causally consistent session, set the ``causal_consistency`` option to true: @@ -112,7 +112,7 @@ Note that if you set the causal_consistency option to nil as in ``(causal_consis as false. End a session -------------- +============= To end a session, call the ``end_session`` method: .. code-block:: ruby diff --git a/source/tutorials/ruby-driver-text-search.txt b/source/tutorials/ruby-driver-text-search.txt index 2cda531ff..634f8a36c 100644 --- a/source/tutorials/ruby-driver-text-search.txt +++ b/source/tutorials/ruby-driver-text-search.txt @@ -1,6 +1,6 @@ -=========== +*********** Text Search -=========== +*********** .. default-domain:: mongodb diff --git a/source/tutorials/ruby-driver-transactions.txt b/source/tutorials/ruby-driver-transactions.txt index d0fd2656c..3023925a4 100644 --- a/source/tutorials/ruby-driver-transactions.txt +++ b/source/tutorials/ruby-driver-transactions.txt @@ -1,6 +1,6 @@ -============ +************ Transactions -============ +************ .. default-domain:: mongodb @@ -17,7 +17,7 @@ Version 4.0 of the MongoDB server introduces versions of MongoDB.) Ruby driver version 2.6.0 adds support for transactions. Using Transactions ------------------- +================== In order to start a transaction, the application must have a :ref:`session `. @@ -76,7 +76,7 @@ which are read concern, write concern and read preference: Low Level API -------------- +============= A transaction can be started by calling the ``start_transaction`` method on a session: @@ -145,7 +145,7 @@ session if one is in progress: Retrying Commits ----------------- +================ The transaction commit `can be retried `_ @@ -165,7 +165,7 @@ if it fails. Here is the Ruby code to do so: Transaction Nesting -------------------- +=================== MongoDB does not support nesting transactions. Attempting to call ``start_transaction`` or ``with_transaction`` when a transaction is already diff --git a/source/tutorials/user-management.txt b/source/tutorials/user-management.txt index b28eac582..7d0f527cc 100644 --- a/source/tutorials/user-management.txt +++ b/source/tutorials/user-management.txt @@ -1,8 +1,8 @@ .. _user-management: -=============== +*************** User Management -=============== +*************** .. default-domain:: mongodb @@ -39,7 +39,7 @@ For more information about users and user management, see MongoDB's Users and Databases -------------------- +=================== When a client connects to the server, MongoDB distinguishes the database that the client will perform operations on from the `auth source `_ @@ -64,7 +64,7 @@ the purposes of user management. Creating Users --------------- +============== There are two ways to create a new database user with the Ruby Driver. @@ -132,7 +132,7 @@ replica set. User Information ----------------- +================ To view information about a user that already exists in the database, use the ``info`` method: @@ -158,7 +158,7 @@ currently exist in a database. Updating Users --------------- +============== To update a user that already exists in the database, you can use the ``update`` method in one of two ways. The first way is to specify the name of @@ -202,7 +202,7 @@ is performed on a replica set. :manual:`Write Concerns`, Removing Users --------------- +============== To remove a user from the database, use the ``remove`` method: From e785e97386731e4821ba545e393442fc0a82ea0d Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Tue, 2 Jun 2020 17:52:31 -0400 Subject: [PATCH 267/442] RUBY-2261 reduce default keepalive time to align with Azure defaults (#1936) --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 02a26dee8..5575f0083 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -977,7 +977,7 @@ TCP Keepalive Configuration The driver sets TCP keepalive by default. The following default values are also set if the system value can be determined and if the driver default value is less than the system value. -- ``tcp_keepalive_time``: 300 seconds +- ``tcp_keepalive_time``: 120 seconds - ``tcp_keepalive_intvl``: 10 seconds - ``tcp_keepalive_cnt``: 9 probes From 3e6b00aef70de0808b7cd0bbb1016c43a35b0388 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Fri, 5 Jun 2020 10:40:48 -0400 Subject: [PATCH 268/442] RUBY-2221 Add commitQuorum option to createIndexes (#1935) * RUBY-2221 support commit_quorum option on createIndexes * only send commitQuorum when value is provided * introduce new method with options * some cleanup * add comment to new method * add pull options from array of indexes * write some documentation * update formatting of documentation * made documentation clearer * suggested changes -- dont duplicate arguments before mutating! --- source/tutorials/ruby-driver-indexing.txt | 38 +++++++++++++++++++++-- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-indexing.txt b/source/tutorials/ruby-driver-indexing.txt index 4cb4c78f1..d187c3501 100644 --- a/source/tutorials/ruby-driver-indexing.txt +++ b/source/tutorials/ruby-driver-indexing.txt @@ -33,7 +33,7 @@ specification as the first argument and options as the second argument: .. code-block:: ruby client[:bands].indexes.create_one(genre: 1) - + client[:bands].indexes.create_one( { name: 1 }, unique: true, expire_after: 120, @@ -96,6 +96,40 @@ when creating indexes. These options mirror the options supported by the * - ``:partial_filter_expression`` - A filter for a partial index. +The :commit_quorum option +------------------------- +On MongoDB server versions 4.4 and newer, the ``:commit_quorum`` option may be +specified on index creation. This option differs from other index options in that +it determines server behavior during index creation, rather than determining +the behavior of an individual index. + +The ``:commit_quorum`` option specifies how many voting, data-bearing members +of a replica set must complete the index build before the index is ready. +Possible values are integers (0 to the number of voting, data-bearing members +of the replica set), "majority", or "votingMembers". + +To specify ``:commit_quorum`` when creating one index, add another option +to the second argument of the ``indexes#create_one`` method: + +.. code-block:: ruby + + client[:bands].indexes.create_one( + { name: 1 }, + unique: true, expire_after: 120, commit_quorum: 'majority' + ) + +To specify create options when creating multiple indexes, add a Hash specifying +``:commit_quorum`` as a final element to the Array of indexes passed to +``indexes#create_many``. Note that this Hash MUST be the final element in the +Array. + +.. code-block:: ruby + + client[:bands].indexes.create_many([ + { key: { genre: 1 } }, + { key: { name: 1 }, unique: true, expire_after: 120 }, + { commit_quorum: 'majority' }, + ]) Dropping Indexes ================ @@ -106,7 +140,7 @@ To drop an index, call ``indexes#drop_one`` or ``indexes#drop_all``. # Drops the name_1 index. client[:bands].indexes.drop_one( 'name_1' ) - + # Drops all indexes in the collection. client[:bands].indexes.drop_all From 075531ad24e21be9f9552456903fd80a3dc6afb8 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Mon, 8 Jun 2020 17:04:09 -0400 Subject: [PATCH 269/442] RUBY-1972 RUBY-2001 RUBY-2148 Hint documentation (#1948) * fix typo * add hint option to API docs * add a documentation section for delete options * added hint to API docs in collection methods * write hint documentation * fix update section * try to fix headings * actually fix headings * add information about which server versions support hints * add return_document info and key value --> key-value * make hint examples clearer --- .../tutorials/ruby-driver-crud-operations.txt | 162 ++++++++++++++---- 1 file changed, 129 insertions(+), 33 deletions(-) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index eb8d61797..77f661522 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -458,20 +458,77 @@ the modification occurs. doc = artists.find_one_and_update( { :name => 'José James' }, { '$set' => { :name => 'José' } } ) doc # Return the document before the update. -``find_one_and_replace`` +Update Options +~~~~~~~~~~~~~~ + +To add options to an update command, specify them as key-value pairs in the options +Hash argument. .. code-block:: ruby - doc = artists.find(:name => 'José James'). - find_one_and_replace( { '$set' => { :name => 'José' } }, :return_document => :after ) - doc # Return the document after the update. + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] - doc = artists.find_one_and_replace( - { :name => 'José James' }, - { '$set' => { :name => 'José' } }, - :return_document => :after + artists.indexes.create_one(name: 1) + + # Force the server to use the name index to perform this operation + result = artists.update_one( + { :name => 'Goldie' }, + { "$inc" => { :plays => 1 } }, + { hint: { name: 1 } } ) - doc # Return the document after the update. + result.n # Returns 1. + +The following is a list of the options that can be added to update operations, +including ``update_one``, ``update_many``, ``replace_one``, +``find_one_and_delete``, ``find_one_and_update``, and ``find_one_and_replace``. + +.. list-table:: + :header-rows: 1 + :widths: 40 80 + + * - Option + - Description + * - ``array_filters`` + - An Array of filter documents that determine which array elements to modify + for an update operation on an array field. + * - ``bypass_document_validation`` + - Whether to skip document-level validation before writing the document. + * - ``collation`` + - Specifies a set of rules to use when comparing strings complying with the + conventions of a particular language. + * - ``hint`` + - The index to use for this operation. May be specified as a Hash + (e.g. { _id: 1 }) or as a String (e.g. "_id_"). Supported on MongoDB + server versions 4.2 and newer for ``update_one``, ``update_many``, and + ``replace_one`` commands, and on server versions 4.4 and newer for + ``find_one_and_delete``, ``find_one_and_update``, and ``find_one_and_replace`` + commands. + * - ``projection`` + - The fields to exclude or include in the operation result (only available + on ``find_one_and_delete``, ``find_one_and_replace``, and + ``find_one_and_update`` commands). + * - ``return_document`` + - A symbol specifying whether to return the updated document as it was before or + after the update. Potential values are ``:before`` or ``:after``. + (Only available on ``find_one_and_update`` and ``find_one_and_replace`` commands). + * - ``sort`` + - How to sort the results of a find and modify command. Specified as a Hash + key-value pair, where the key is the name of the field to sort by, and + the value is either 1 or -1, specifying a sort in ascending or descending + order (only available on ``find_one_and_delete``, ``find_one_and_replace``, + and ``find_one_and_update`` commands). + * - ``session`` + - The session to use for this operation. + * - ``upsert`` + - Whether to upsert if the document doesn't exist. Cannot be used on + ``find_one_and_delete`` operation. + +For more information about update options, see the MongoDB server documentation +on the following commands: + +- :manual:`update ` +- :manual:`findAndModify ` Deleting ======== @@ -499,6 +556,45 @@ Deleting result = artists.delete_many(:label => 'Mute') result.deleted_count # Returns the number deleted. +Delete Options +~~~~~~~~~~~~~~ + +To add options to a delete command, specify them as key-value pairs in the +options Hash argument. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + artists = client[:artists] + + artists.indexes.create_one(name: 1) + + # Force the server to use the name index to perform this operation + result = artists.find(:name => 'Björk').delete_one(hint: { name: 1 }) + result.deleted_count # Returns 1. + +The following is a full list of the available options that can be added +to ``delete_one`` and ``delete_many`` operations. + +.. list-table:: + :header-rows: 1 + :widths: 40 80 + + * - Option + - Description + * - ``collation`` + - Specifies a set of rules to use when comparing strings complying with the + conventions of a particular language. + * - ``hint`` + - The index to use for this operation. May be specified as a Hash + (e.g. { _id: 1 }) or as a String (e.g. "_id_"). Supported on MongoDB + server versions 4.4 and newer. + * - ``session`` + - The session to use for this operation. + +For more information about update options, see the MongoDB server documentation +on the :manual:`delete command. ` + .. _write-concern: Write Concern @@ -521,7 +617,7 @@ In driver versions 2.9 and below, client, collection and GridFS objects took write concern options in the ``:write`` option with session and transaction objects employing the ``:write_concern`` option. -Below are some examples of passing write concerns to client and colection +Below are some examples of passing write concerns to client and collection objects. The ``:write_concern`` option can be provided when constructing new client and collection objects, or to the ``#with`` methods. @@ -532,10 +628,10 @@ GridFS examples are provided on the :ref:`GridFS ` page. client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', write_concern: {w: 2}) alt_client = client.with(write_concern: {w: :majority}) - + collection = client[:artists, write_concern: {w: 3}] alt_collection = collection.with(write_concern: {w: :majority}) - + # Uses w: 3 collection.insert_one({name: 'SUN Project'}) # Uses w: :majority @@ -550,7 +646,7 @@ backwards compatibility, but is deprecated: client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', write: {w: 2}) alt_client = client.with(write: {w: :majority}) - + collection = client[:artists, write: {w: 3}] alt_collection = collection.with(write: {w: :majority}) @@ -562,7 +658,7 @@ values must be identical or an exception will be raised: # OK client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', write_concern: {w: 3}, write: {w: 3}) - + # Error client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', write_concern: {w: 3}, write: {w: :majority}) @@ -575,10 +671,10 @@ the last provided option wins in case of naming differences: client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', write_concern: {w: 2}) alt_client = client.with(write: {w: 3}) - + alt_client.options[:write] # => {"w"=>3} - + alt_client.options[:write_concern] # => nil @@ -596,34 +692,34 @@ transaction option, the ``:write`` option is not recognized by any driver version. .. code-block:: ruby - + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', write_concern: {w: 2}) collection = client[:artists, write_concern: {w: :majority}] - - + + session = client.start_session session.with_transaction do collection.insert_one({test: 1}, session: session) - + # Uses w: 2 when committing end - - + + session = client.start_session(default_transaction_options: {write_concern: {w: 3}) ) session.with_transaction do collection.insert_one({test: 1}, session: session) - + # Uses w: 3 when committing end - - + + session = client.start_session session.with_transaction(write_concern: {w: 3}) do collection.insert_one({test: 1}, session: session) - + # Uses w: 3 when committing end @@ -632,36 +728,36 @@ write concern hash rather than individual elements. For example, ``j: true`` is not inherited in the following case: .. code-block:: ruby - + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', write_concern: {w: 1, j: true}) collection = client[:artists, write_concern: {w: 2}] - + collection.write_concern.options # => #2}> Although CRUD operations accept an options hash, they currently do not recognize the ``:write_concern`` option: - + .. code-block:: ruby client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music', write_concern: {w: 2}) collection = client[:artists, write_concern: {w: :majority}] - + # Still uses w: :majority collection.insert_one({name: 'SUN Project'}, write_concern: {w: 1}) The easiest workaround for this is to use ``#with`` to obtain a new collection instance with the desired write concern: - + .. code-block:: ruby # Uses w: 1 collection.with(write_concern: {w: 1}).insert_one(name: 'SUN Project') Write concern can also be manually specified in ``Database#command``: - + .. code-block:: ruby client.database.command(create: 'foo-collection', writeConcern: {w: :majority}) @@ -673,7 +769,7 @@ underscore one that Ruby driver uses. A Note about the BSON Symbol type ================================= -Because the BSON specification deprecated the BSON symbol type, the `bson` gem +Because the BSON specification deprecated the BSON symbol type, the `bson` gem will serialize Ruby symbols into BSON strings when used on its own. However, in order to maintain backwards compatibility with older datasets, the Ruby driver overrides this behavior to serialize Ruby symbols as BSON symbols. This is From 2ae6274f3057ab835569c072fb57b9f60f55bc6a Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Mon, 8 Jun 2020 17:06:22 -0400 Subject: [PATCH 270/442] RUBY-2184 add docs about hedge option (#1949) --- .../tutorials/ruby-driver-crud-operations.txt | 28 ++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 77f661522..27ebdc8ea 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -293,7 +293,8 @@ Read Preference Read preference determines the candidate :manual:`replica set` members to which a query or command can be sent. They consist of a **mode** -specified as a symbol, an array of hashes known as **tag_sets**, and two +specified as a symbol, an array of hashes known as **tag_sets**, +the ``hedge`` option, which is a Hash specifying hedged read behavior, and two timing options: **local_threshold** and **server_selection_timeout**. ``local_threshold`` @@ -378,6 +379,31 @@ A tag set that is an empty document matches any server, because the empty tag set is a subset of any tag set. This means the default ``tag_sets`` parameter ``[{}]`` matches all servers. +Hedge +~~~~~ + +The ``hedge`` parameter is a Hash that specifies whether the server should use +hedged reads. With hedged reads, sharded clusters can route read operations to +two replica set members and return results from the first respondent. + +The ``hedge`` option may only be specified on non-primary read preferences. It +must be provided as Hash with the key ``enabled`` set to ``true`` or ``false``. + +.. code-block:: ruby + + client = Mongo::Client.new( + [ '127.0.0.1:27017' ], + read: { mode: :secondary, hedge: { enabled: true } }, + ) + +See the :manual:`MongoDB Manual ` for +more information about hedged reads. + +.. note:: + + The ``hedge`` option is only available on MongoDB server versions 4.4 and newer. + Attempting to use this option on older server versions will result in an error. + .. _ruby-driver-updating: Updating From c396d002de0360512b3fb601cd4548d5b187deba Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Mon, 15 Jun 2020 17:28:47 -0400 Subject: [PATCH 271/442] Release 2.13.0.beta1 --- source/installation.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/installation.txt b/source/installation.txt index bb4c2a50e..fb4b0fa6c 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -19,7 +19,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.12.1 + gem install mongo -v 2.13.0.beta1 What's New From 8544d856afd693cfa4c6ab4dc747675b26e0bac0 Mon Sep 17 00:00:00 2001 From: Neilshweky Date: Wed, 24 Jun 2020 12:37:38 -0400 Subject: [PATCH 272/442] RUBY-1262 added tests for closing session after block (#1968) * RUBY-1262 added tests for closing session after block * RUBY-1262 changed start_session to accept block and modified tests to test start_session * RUBY-1262 test block return value * RUBY-1262 updated documentation to use block sessions; updated a test to make sure session is valid in block * RUBY-1262 updated documentation * RUBY-1262 moved the end-session instructions to the end * RUBY-1262 added note about block syntax end session * RUBY-1262 updated docs, added hyperlinks, fixed typos * RUBY-1262 fixed hyperlinks * RUBY-1262 fixed hyperlinks by adding underscore * RUBY-1262 fixed hyperlinks new strategy * RUBY-1262 revert to old method * RUBY-1262 fix broken scram link --- .../tutorials/ruby-driver-authentication.txt | 2 +- source/tutorials/ruby-driver-sessions.txt | 47 ++++++++++++++----- 2 files changed, 37 insertions(+), 12 deletions(-) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index ab2f7f403..bbbea4b56 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -470,7 +470,7 @@ MONGODB-CR `````````` *Deprecated:* MONGODB-CR mechanism is deprecated as of MongoDB 3.6 and -removed as of MongoDB 4.0. Please use `SCRAM authentication `_ instead. +removed as of MongoDB 4.0. Please use `SCRAM authentication <#scram>`_ instead. MONGODB-CR was the default authentication mechanism for MongoDB through version 2.6. diff --git a/source/tutorials/ruby-driver-sessions.txt b/source/tutorials/ruby-driver-sessions.txt index c0418e176..216a2bf7c 100644 --- a/source/tutorials/ruby-driver-sessions.txt +++ b/source/tutorials/ruby-driver-sessions.txt @@ -20,15 +20,20 @@ and passed to operation methods that should be executed in the context of that s Please note that session objects are not thread safe. They must only be used by one thread at a time. +.. _create-session: + Creating a session from a ``Mongo::Client`` =========================================== -A session can be created by calling the ``start_session`` method on a client: +A session can be created by calling the ``start_session`` method on a client and passing it a block: .. code-block:: ruby - session = client.start_session + client.start_session do |session| + # work with the session + end +When using the block form, the session will be automatically ended by the driver after the block finishes executing. It is valid to call ``start_session`` with no options set. This will result in a session that has no effect on the operations performed in the context of that session, @@ -43,6 +48,7 @@ Be aware that if the application calls ``#start_session`` on a client and waits the session, it risks getting errors due to the session going stale before it is used. + Using a session =============== A session object can be passed to most driver methods so that the operation can be executed in the @@ -52,29 +58,43 @@ Create a session and execute an insert, then a find using that session: .. code-block:: ruby - session = client.start_session - client[:artists].insert_one({ :name => 'FKA Twigs' }, session: session) - client[:artists].find({ :name => 'FKA Twigs' }, limit: 1, session: session).first + client.start_session do |session| + client[:artists].insert_one({ :name => 'FKA Twigs' }, session: session) + client[:artists].find({ :name => 'FKA Twigs' }, limit: 1, session: session).first + end If you like to call methods on a ``Mongo::Collection::View`` in the context of a particular session, you can create the ``Mongo::Collection::View`` with the session and then call methods on it: .. code-block:: ruby - session = client.start_session(causal_consistency: true) - view = client[:artists].find({ :name => 'FKA Twigs' }, session: session) - view.count # will use the session + client.start_session(causal_consistency: true) do |session| + view = client[:artists].find({ :name => 'FKA Twigs' }, session: session) + view.count # will use the session + end You can also pass the session option to the methods directly. This session will override any session associated with the ``Mongo::Collection::View``: +.. code-block:: ruby + + client.start_session do |session| + client.start_session do |second_session| + view = client[:artists].find({ :name => 'FKA Twigs' }, session: session) + view.count(session: second_session) # will use the second_session + end + end + +Alternative way to create a session +=================================== + +A session can be created by calling the ``start_session`` method on a client: + .. code-block:: ruby session = client.start_session - second_session = client.start_session - view = client[:artists].find({ :name => 'FKA Twigs' }, session: session) - view.count(session: second_session) # will use the second_session +When ``start_session`` is used without passing a block to it, the driver does not automatically clean up the session which can result in an accumulation of sessions on the server. Use `end_session <#end-a-session>`_ to manually end the session created. The server will automatically clean up old sessions after a timeout but the application should end sessions when the sessions are no longer needed. Unacknowledged Writes ===================== @@ -111,6 +131,8 @@ consistent reads are not causally consistent with unacknowledged writes. Note that if you set the causal_consistency option to nil as in ``(causal_consistency: nil)``, it will be interpreted as false. +.. _end-session: + End a session ============= To end a session, call the ``end_session`` method: @@ -122,3 +144,6 @@ To end a session, call the ``end_session`` method: The Ruby driver will then add the id for the corresponding server session to a pool for reuse. When a client is closed, the driver will send a command to the server to end all sessions it has cached in its server session pool. You may see this command in your logs when a client is closed. + +Note that when using the `block syntax <#creating-a-session-from-a-mongo-client>`_ for ``start_session`` the session is automatically ended after +the block finishes executing. \ No newline at end of file From 706bcc8991b24c3f54070916a214da3354c2b4b1 Mon Sep 17 00:00:00 2001 From: Neilshweky Date: Wed, 24 Jun 2020 14:11:13 -0400 Subject: [PATCH 273/442] RUBY-2165 added documentation to Create Client to explain block syntax (#1974) * RUBY-2165 added documentation for block syntax * RUBY-2165 fixed some typos --- source/tutorials/ruby-driver-create-client.txt | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 5575f0083..2c73428b4 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -48,6 +48,18 @@ specify the Atlas deployment URI: The driver will discover all nodes in the cluster and connect to them as needed. +Block Syntax +```````````` + +Another way to create a Mongo::Client object is to use the block syntax: + +.. code-block:: ruby + + Mongo::Client.new(...) do |client| + # work with the client + end + +Note that when creating a client using this syntax, the client is automatically closed after the block finishes executing. Database ```````` @@ -1064,6 +1076,8 @@ When ``#close`` is called on a client by any thread, all connections are closed: client.close +Note that when creating a client using the `block syntax <#block-syntax>`_ described above, the client is automatically closed after the block finishes executing. + Usage with Forking Servers ========================== From 32149faf01412bfd979dc87d0fcf1d2a939e1d34 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 13 Jul 2020 11:23:09 -0400 Subject: [PATCH 274/442] RUBY-2320 Configure how many backtrace lines are shown in monitor exceptions (#2004) * RUBY-2320 Configure how many backtrace lines are shown in monitor exceptions * fix tests * need to go through the Loggable module to maintain formatting * validate option values Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-create-client.txt | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 2c73428b4..967135b5f 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -301,6 +301,15 @@ Ruby Options - ``Hash`` - none + * - ``:bg_error_backtrace`` + - Experimental. Controls whether and how backtraces are logged when + errors occur in background threads. If ``true``, the driver will log + complete backtraces. If set to a positive integer, the driver will + log up to that many backtrace lines. If set to ``false`` or ``nil``, + no backtraces will be logged. Other values are an error. + - ``true``, ``false``, ``nil``, ``Integer`` + - none + * - ``:compressors`` - A list of potential compressors to use, in order of preference. The driver chooses the first compressor that is also supported by the server. Currently the driver only supports 'zlib'. From 309e9250ec6e3796abae165e094d4da86ea65da4 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 13 Jul 2020 15:21:05 -0400 Subject: [PATCH 275/442] Fix RUBY-2321 Timeout of 0 should mean no timeout (#2013) * Fix RUBY-2321 Timeout of 0 should mean no timeout * validate timeout option values * describe zero and invalid timeout behavior Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 30 +++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 967135b5f..c86b35ad2 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -325,7 +325,9 @@ Ruby Options * - ``:connect_timeout`` - The number of seconds to wait to establish a socket connection - before raising an exception. + before raising an exception. The value of ``0`` means no timeout. + Client creation will fail with an error if an invalid timeout value + is passed (such as a negative value or a non-numeric value). - ``Float`` - 10 seconds @@ -481,7 +483,9 @@ Ruby Options * - ``:socket_timeout`` - The number of seconds to wait for an operation to execute on a - socket before raising an exception. + socket before raising an exception. The value of ``0`` means no timeout. + Client creation will fail with an error if an invalid timeout value + is passed (such as a negative value or a non-numeric value). - ``Float`` - 5 seconds @@ -701,6 +705,10 @@ URI options are explained in detail in the :manual:`Connection URI reference * - connectTimeoutMS=Integer - ``:connect_timeout => Float`` + Unlike the corresponding Ruby option which fails client creation on + invalid values (e.g. negative and non-numeric values), invalid values + provided via this URI option are ignored with a warning. + * - directConnection=Boolean - ``:direct_connection => Boolean`` @@ -762,6 +770,10 @@ URI options are explained in detail in the :manual:`Connection URI reference * - socketTimeoutMS=Integer - ``:socket_timeout => Float`` + Unlike the corresponding Ruby option which fails client creation on + invalid values (e.g. negative and non-numeric values), invalid values + provided via this URI option are ignored with a warning. + * - ssl=Boolean - ``:ssl => true|false`` @@ -824,6 +836,13 @@ Timeout Options When executing an operation, the number of seconds to wait for the driver to find an appropriate server to send an operation to. Defaults to 30. +A value of 0 means no timeout. + +When an invalid value (e.g. a negative value or a non-numeric value) is passed +via the URI option, the invalid input is ignored with a warning. When an +invalid value is passed directly to Client via a Ruby option, Client +construction fails with an error. + In replica set deployments, this timeout should be set to exceed the typical :manual:`replica set election times ` in order for the driver to transparently handle primary changes. This timeout @@ -843,6 +862,13 @@ provide quicker failure when the server is not running. The number of seconds to wait for a socket read or write to complete on regular (non-monitoring) connections. Default is no timeout. +A value of 0 means no timeout. + +When an invalid value (e.g. a negative value or a non-numeric value) is passed +via the URI option, the invalid input is ignored with a warning. When an +invalid value is passed directly to Client via a Ruby option, Client +construction fails with an error. + This timeout should take into account both network latency and operation duration. For example, setting this timeout to 5 seconds will abort queries taking more than 5 seconds to execute on the server with ``Mongo::Error::SocketTimeoutError``. From 8a0748fda47bd5fed4ebad6763c84aaa0301b424 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 14 Jul 2020 11:47:28 -0400 Subject: [PATCH 276/442] RUBY-2132 publish heartbeat events from push monitor (#2017) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-monitoring.txt | 22 +++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/source/tutorials/ruby-driver-monitoring.txt b/source/tutorials/ruby-driver-monitoring.txt index 4cc955ca2..e822dc1ce 100644 --- a/source/tutorials/ruby-driver-monitoring.txt +++ b/source/tutorials/ruby-driver-monitoring.txt @@ -342,6 +342,28 @@ Sample output: D, [2018-09-23T13:44:10.707018 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | STARTED D, [2018-09-23T13:44:10.707778 #1739] DEBUG -- : HEARTBEAT | 127.0.0.1:27027 | SUCCEEDED | 0.000772381s +Heartbeat Event Intervals +------------------------- + +When connected to MongoDB 4.2 and earlier servers, Ruby driver by default +issues heartbeats every ``:heartbeat_frequency`` (Ruby client option) seconds, +and heartbeats are non-overlapping (the succeeded event for a heartbeat is +guaranteed to be published before the started event for the next heartbeat is +published). When connected to MongoDB 4.4 and later servers, the driver uses +multiple monitoring threads and a more complex heartbeat protocol designed +to detect changes in server state quicker; as a result, heartbeat event +intervals can be more irregular and heartbeat events can overlap. Specifically, +an *awaited heartbeat* can start or finish while a *non-awaited heartbeat* +is in progress, and vice versa. Use the ``ServerHeartbeatStarted#awaited?``, +``ServerHeartbeatSucceeded#awaited?`` and ``ServerHeartbeatFailed#awaited?`` +methods to distinguish between non-awaited and awaited heartbeats. + +When a client is attempting to perform an operation and it does not have a +suitable server, the deployment is scanned more frequently - each server can +be polled up to every 500 milliseconds. It is also possible for the application +to request a manual scan of a particular server; the driver enforces the +500 millisecond minimum interval between scans. + Connection Pool And Connection Monitoring ========================================= From 6ccdd1cc2fb4fb14d4fdb761886d55b7f404bd8b Mon Sep 17 00:00:00 2001 From: gjchong25 Date: Tue, 14 Jul 2020 14:07:31 -0400 Subject: [PATCH 277/442] Fix RUBY-2297 Broken formatting in BSON documentation (#211) * test changes * changes * fix whitespace issues * fix extra grey bar Co-authored-by: Grace Chong --- docs/tutorials/bson-v4.txt | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index d4d7878ad..644b110c4 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -270,7 +270,7 @@ The data and the subtype can be retrieved from ``Binary`` instances using was in: .. code-block:: ruby - + str = "binary_string" str.encoding # => # @@ -343,10 +343,10 @@ cannot be stringified without specifying a representation. binary = BSON::Binary.new("\x00\x11\x22\x33\x44\x55\x66\x77\x88\x99\xAA\xBB\xCC\xDD\xEE\xFF".force_encoding('BINARY'), :uuid_old) => - + binary.to_uuid # => ArgumentError (Representation must be specified for BSON::Binary objects of type :uuid_old) - + binary.to_uuid(:csharp_legacy) # => "33221100-5544-7766-8899aabbccddeeff" @@ -362,13 +362,13 @@ UUID, use the ``from_uuid`` method specifying the desired representation: .. code-block:: ruby uuid_str = "00112233-4455-6677-8899-aabbccddeeff" - + BSON::Binary.from_uuid(uuid_str, :csharp_legacy) # => BSON::Binary.from_uuid(uuid_str, :java_legacy) # => - + BSON::Binary.from_uuid(uuid_str, :python_legacy) # => @@ -485,17 +485,17 @@ strings to be used instead. stringified when used as hash keys: .. code-block:: ruby - - Hash.from_bson({foo: 'bar'}.to_bson) - # => {"foo"=>"bar"} - - Hash.from_bson({1 => 2}.to_bson) - # => {"1"=>2} + + Hash.from_bson({foo: 'bar'}.to_bson) + # => {"foo"=>"bar"} + + Hash.from_bson({1 => 2}.to_bson) + # => {"1"=>2} By default, the BSON library encodes ``Symbol`` hash values as strings and decodes BSON symbols into Ruby ``Symbol`` values: - .. code-block:: ruby +.. code-block:: ruby {foo: :bar}.to_bson.to_s # => "\x12\x00\x00\x00\x02foo\x00\x04\x00\x00\x00bar\x00\x00" @@ -511,7 +511,7 @@ decodes BSON symbols into Ruby ``Symbol`` values: To force encoding of Ruby symbols to BSON symbols, wrap the Ruby symbols in ``BSON::Symbol::Raw``: - .. code-block:: ruby +.. code-block:: ruby {foo: BSON::Symbol::Raw.new(:bar)}.to_bson.to_s # => "\x12\x00\x00\x00\x0Efoo\x00\x04\x00\x00\x00bar\x00\x00" @@ -571,11 +571,11 @@ millisecond. .. code-block:: ruby - time = Time.utc(1960, 1, 1, 0, 0, 0, 999_999) - time.to_f - # => -315619199.000001 - time.floor(3).to_f - # => -315619199.001 + time = Time.utc(1960, 1, 1, 0, 0, 0, 999_999) + time.to_f + # => -315619199.000001 + time.floor(3).to_f + # => -315619199.001 .. note:: From d9d0cbc0cb8be8a5164a131dee8a1d74731ffb2b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 15 Jul 2020 13:04:42 -0400 Subject: [PATCH 278/442] RUBY-2323 there is no default socket timeout (#2019) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-create-client.txt | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index c86b35ad2..5608ea70f 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -325,11 +325,11 @@ Ruby Options * - ``:connect_timeout`` - The number of seconds to wait to establish a socket connection - before raising an exception. The value of ``0`` means no timeout. + before raising an exception. ``nil`` and ``0`` mean no timeout. Client creation will fail with an error if an invalid timeout value is passed (such as a negative value or a non-numeric value). - ``Float`` - - 10 seconds + - 10 * - ``:database`` - The name of the database to connect to. @@ -483,11 +483,11 @@ Ruby Options * - ``:socket_timeout`` - The number of seconds to wait for an operation to execute on a - socket before raising an exception. The value of ``0`` means no timeout. + socket before raising an exception. ``nil`` and ``0`` mean no timeout. Client creation will fail with an error if an invalid timeout value is passed (such as a negative value or a non-numeric value). - ``Float`` - - 5 seconds + - none * - ``:ssl`` - Tell the client to connect to the servers via SSL. @@ -611,7 +611,7 @@ Ruby Options - The number of seconds to wait for a connection in the connection pool to become available. - ``Float`` - - 1 + - 10 * - ``:write`` - Deprecated. Equivalent to ``:write_concern`` option. If both ``:write`` From 6d34a68f37b97e44b672e47170e313abca34c0b3 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 17 Jul 2020 12:59:24 -0400 Subject: [PATCH 279/442] RUBY-2324 Update driver compatibility tables for 2.13 / server 4.4 (#2023) Co-authored-by: Oleg Pudeyev --- source/reference/driver-compatibility.txt | 42 ++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index cd88d9c42..676ed7452 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -25,6 +25,7 @@ MongoDB Compatibility :class: compatibility-large no-padding * - Ruby Driver + - MongoDB 4.4 - MongoDB 4.2 - MongoDB 4.0 - MongoDB 3.6 @@ -34,7 +35,19 @@ MongoDB Compatibility - MongoDB 2.6 - MongoDB 2.4 + * - 2.13 + - |checkmark| [#ocsp]_ + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + * - 2.12 + - - |checkmark| - |checkmark| - |checkmark| @@ -45,6 +58,7 @@ MongoDB Compatibility - * - 2.11 + - - |checkmark| [#client-side-encryption]_ - |checkmark| - |checkmark| @@ -55,6 +69,7 @@ MongoDB Compatibility - * - 2.10 + - - |checkmark| [#srv-polling]_ [#client-side-encryption]_ - |checkmark| - |checkmark| @@ -65,6 +80,7 @@ MongoDB Compatibility - * - 2.9 + - - - |checkmark| - |checkmark| @@ -75,6 +91,7 @@ MongoDB Compatibility - * - 2.8 + - - - |checkmark| - |checkmark| @@ -85,6 +102,7 @@ MongoDB Compatibility - * - 2.7 + - - - |checkmark| - |checkmark| @@ -95,6 +113,7 @@ MongoDB Compatibility - * - 2.6 + - - - |checkmark| - |checkmark| @@ -105,6 +124,7 @@ MongoDB Compatibility - * - 2.5 + - - - - |checkmark| @@ -118,6 +138,7 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -129,6 +150,7 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -139,6 +161,7 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| @@ -150,10 +173,13 @@ MongoDB Compatibility - - - + - - |checkmark| - |checkmark| - |checkmark| +.. [#ocsp] OCSP verification is not yet implemented. + .. [#srv-polling] Polling of SRV records in sharded topologies is implemented as of driver version 2.11. @@ -193,6 +219,20 @@ for that Ruby version is deprecated. - JRuby 9.1 - JRuby + * - 2.13 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + - + - + - + - |checkmark| + - + - + * - 2.12 - |checkmark| - |checkmark| @@ -419,7 +459,7 @@ the driver. * - Ruby Driver - mongo_kerberos |nbsp| 2.1 - * - 2.7 - 2.12 + * - 2.7 - 2.13 - |checkmark| From 4bf55c8dfd3cbf533d3e38aeb0ce38e8f48e7457 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Fri, 17 Jul 2020 14:43:20 -0400 Subject: [PATCH 280/442] Release 2.13.0.rc1 --- source/installation.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/installation.txt b/source/installation.txt index fb4b0fa6c..517f7a3fa 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -19,7 +19,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.13.0.beta1 + gem install mongo -v 2.13.0.rc1 What's New From 986d9863ddde55474d6338f9969402a256561fb4 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 20 Jul 2020 13:21:14 -0400 Subject: [PATCH 281/442] RUBY-2018 Add client metadata support for wrapping libraries (#2020) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-create-client.txt | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 5608ea70f..e07bddf15 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -613,6 +613,13 @@ Ruby Options - ``Float`` - 10 + * - ``:wrapping_libraries`` + - Information about libraries such as ODMs that are wrapping the driver. + Specify the lower level libraries first. Allowed hash keys: :name, + :version, :platform. Example: ``[name: 'Mongoid', version: '7.1.2']`` + - ``Array`` + - none + * - ``:write`` - Deprecated. Equivalent to ``:write_concern`` option. If both ``:write`` and ``:write_concern`` are specified, their values must be identical. From 33a0833ed1a42f3810d2463f659ef5d422d2770a Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 28 Jul 2020 10:17:03 -0400 Subject: [PATCH 282/442] RUBY-2244 Reference Atlas search in text search documentation (#2028) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-text-search.txt | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/source/tutorials/ruby-driver-text-search.txt b/source/tutorials/ruby-driver-text-search.txt index 634f8a36c..de2e09837 100644 --- a/source/tutorials/ruby-driver-text-search.txt +++ b/source/tutorials/ruby-driver-text-search.txt @@ -15,6 +15,13 @@ to support text search queries on string content. Text indexes can include any field whose value is a string or an array of string elements. +.. note:: + + MongoDB Atlas also provides + `Atlas Search `_ + which is a more powerful and flexible text search solution. + The rest of this page discusses text indexes and not Atlas Search. + To perform a text search with the Ruby driver, first create a text index with ``indexes.create_one()``. The following command creates a text index on the ``name`` field of the ``restaurants`` collection in From 32ff24d87e40cd9fbee5eff98442922eada44b1c Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Wed, 29 Jul 2020 11:48:41 -0400 Subject: [PATCH 283/442] RUBY-2243 Support hidden option on index creation (#2030) --- source/tutorials/ruby-driver-indexing.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/tutorials/ruby-driver-indexing.txt b/source/tutorials/ruby-driver-indexing.txt index d187c3501..0a29b768c 100644 --- a/source/tutorials/ruby-driver-indexing.txt +++ b/source/tutorials/ruby-driver-indexing.txt @@ -95,6 +95,9 @@ when creating indexes. These options mirror the options supported by the - The number of units within which to group the location values in a geo haystack index. * - ``:partial_filter_expression`` - A filter for a partial index. + * - ``:hidden`` + - A Boolean specifying whether the index should be hidden; a hidden index + is one that exists on the collection but will not be used by the query planner. The :commit_quorum option ------------------------- From d152ab6ca4c17e53a7bef9f50a7703d0175edf56 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Wed, 29 Jul 2020 11:53:16 -0400 Subject: [PATCH 284/442] RUBY-2274: fix CRUD headers (#2031) --- .../tutorials/ruby-driver-crud-operations.txt | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 27ebdc8ea..5d0b27347 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -88,7 +88,7 @@ On MongoDB 2.4, an exception is only raised if the insert fails and the .. _specify-decimal128: Specify a ``Decimal128`` number -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------------- .. versionadded:: 3.4 @@ -166,7 +166,7 @@ notation. .. _ruby-driver-query-options: Query Options -~~~~~~~~~~~~~ +------------- To add options to a query, chain the appropriate methods after the ``find`` method. Note that the underlying object, the ``Mongo::Collection::View``, @@ -238,7 +238,7 @@ when querying and their corresponding methods as examples. client[:artists].find.sort(:name => -1) Additional Query Operations -~~~~~~~~~~~~~~~~~~~~~~~~~~~ +--------------------------- ``count`` Get the total number of documents an operation returns. @@ -260,7 +260,7 @@ Additional Query Operations client[:artists].find.distinct(:name ) Tailable Cursors -~~~~~~~~~~~~~~~~ +---------------- For capped collections you may use a :manual:`tailable cursor ` that remains open @@ -289,11 +289,11 @@ following code example shows how a tailable cursor might be used: .. _ruby-driver-read-preference: Read Preference -~~~~~~~~~~~~~~~ +--------------- Read preference determines the candidate :manual:`replica set` members to which a query or command can be sent. They consist of a **mode** -specified as a symbol, an array of hashes known as **tag_sets**, +specified as a symbol, an array of hashes known as **tag_sets**, the ``hedge`` option, which is a Hash specifying hedged read behavior, and two timing options: **local_threshold** and **server_selection_timeout**. @@ -342,7 +342,7 @@ using the ``with`` method: artists.with(:read => { :mode => :primary_preferred }).find.to_a Mode -~~~~ +---- There are five possible read preference modes: ``:primary``, ``:secondary``, ``:primary_preferred``, ``:secondary_preferred`` and``:nearest``. @@ -360,7 +360,7 @@ Please see the :manual:`read preference documentation in the MongoDB Manual unchanged. Tag sets -~~~~~~~~ +-------- The ``tag_sets`` parameter is an ordered list of tag sets used to restrict the eligibility of servers for selection, such as for data @@ -380,7 +380,7 @@ the empty tag set is a subset of any tag set. This means the default ``tag_sets`` parameter ``[{}]`` matches all servers. Hedge -~~~~~ +----- The ``hedge`` parameter is a Hash that specifies whether the server should use hedged reads. With hedged reads, sharded clusters can route read operations to @@ -485,7 +485,7 @@ the modification occurs. doc # Return the document before the update. Update Options -~~~~~~~~~~~~~~ +-------------- To add options to an update command, specify them as key-value pairs in the options Hash argument. @@ -583,7 +583,7 @@ Deleting result.deleted_count # Returns the number deleted. Delete Options -~~~~~~~~~~~~~~ +-------------- To add options to a delete command, specify them as key-value pairs in the options Hash argument. From cd229bc74cb71bf63743d2f7818a329a9db12d69 Mon Sep 17 00:00:00 2001 From: gjchong25 Date: Wed, 29 Jul 2020 16:46:45 -0400 Subject: [PATCH 285/442] Ruby-2339 Write Driver Query Cache documentation (#2029) * RUBY-2339 query cache documentation * RUBY-2339 fix wording * RUBY-2339 fix title * RUBY-2339 style * RUBY-2339 formatting * RUBY-2339 PR comment fixes * RUBY-2339 style * RUBY-2339 fix grammar and formatting Co-authored-by: Grace Chong --- .../tutorials/ruby-driver-crud-operations.txt | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 5d0b27347..d9a7cf50c 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -130,6 +130,44 @@ object, or with ``Decimal128.from_string()``. price = BSON::Decimal128.from_string("428.79") # => BSON::Decimal128('428.79') +Query Cache +=========== + +If the Ruby driver's query cache is enabled, it will cache find queries on +the currently executed thread and avoid sending requests to the database for +identical queries. The QueryCache class stores CachingCursor objects which attempt +to load documents from memory before retrieving them from the database. Performing +any write operations such as insert, update, or delete clears the query cache. Note that +if the number of results is too large to be returned in a single batch, +the query cache will not be used, even if ``Mongo::QueryCache.enabled`` is true, and +an error will be returned. + +Similar to Mongoid's query cache, the driver's query cache implementation +does not support using CachingCursors for queries with different limit sizes if +the original query has a specified limit. For example, if a query with a limit of +100 were executed, followed by a query with a limit of 10, the second query +would still run against the database and a new CachingCursor object would be stored, +rather than using the existing one. However, if a query does not specify a limit +initially, then any query that is run after it with a specified limit will use +the original CachingCursor rather than going back to the database, and will return +the correct number of documents accordingly. + +To enable the query cache on a global scope: + +.. code-block:: ruby + + Mongo::QueryCache.enabled = true + +The following code shows how to enable the query cache within the context of a +specific block: + +.. code-block:: ruby + + Mongo::QueryCache.cache do + # all queries within this block are cached + end + + Reading ======= From 3a7f64135e4125209641489ff56a1db1f9074cb9 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Fri, 31 Jul 2020 11:52:38 -0400 Subject: [PATCH 286/442] RUBY-2021 include workaround for always serializing symbols as strings (#2034) --- source/tutorials/ruby-driver-crud-operations.txt | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index d9a7cf50c..2e53bbe8e 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -840,3 +840,14 @@ overrides this behavior to serialize Ruby symbols as BSON symbols. This is necessary to be able to specify queries for documents which contain BSON symbols as fields. Despite this, new documents with symbol type fields should *not* be stored in the database; instead, use string fields. + +To override default behavior and configure the driver to encode symbol values +as strings, include the following code snippet in your project: + +.. code-block:: ruby + + class Symbol + def bson_type + BSON::String::BSON_TYPE + end + end From d6eea01614f19227637b63f73dfacbad4cefba4b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 6 Aug 2020 10:07:33 -0400 Subject: [PATCH 287/442] RUBY-2179 Document that dates are serialized as midnight in UTC (#214) * RUBY-2179 Document that dates are serialized as midnight in UTC * RUBY-2187 document DateTime conversion to Gregorian calendar prior to serialization * remove the Co-authored-by: Oleg Pudeyev --- docs/tutorials/bson-v4.txt | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 644b110c4..7a0fba636 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -588,14 +588,30 @@ all time calculations using integer math, as inexactness of floating point calculations may produce unexpected results. -Date And DateTime Instances ---------------------------- +DateTime Instances +------------------ BSON only supports storing the time as the number of seconds since the -Unix epoch. Ruby's ``Date`` and ``DateTime`` instances can be serialized -to BSON, but when the BSON is deserialized the times will be returned as +Unix epoch. Ruby's ``DateTime`` instances can be serialized to BSON, +but when the BSON is deserialized the times will be returned as ``Time`` instances. +``DateTime`` class in Ruby supports non-Gregorian calendars. When non-Gregorian +``DateTime`` instances are serialized, they are first converted to Gregorian +calendar, and the respective date in the Gregorian calendar is stored in the +database. + + +Date Instances +-------------- + +BSON only supports storing the time as the number of seconds since the +Unix epoch. Ruby's ``Date`` instances can be serialized to BSON, but when +the BSON is deserialized the times will be returned as ``Time`` instances. + +When ``Date`` instances are serialized, the time value used is midnight +of the day that the ``Date`` refers to in UTC. + Regular Expressions ------------------- From 36fc4d5b494874077638da732d3d3b3359c74db1 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 6 Aug 2020 11:06:37 -0400 Subject: [PATCH 288/442] RUBY-2293 Document retrieving auth source from TXT records with SRV URIs (#2043) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-authentication.txt | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index bbbea4b56..f87552218 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -89,6 +89,23 @@ if no database is specified, the ``$external`` database is used as the auth source. For the ``AWS``, ``GSSAPI`` and ``MONGODB_X509`` mechanisms, the auth source is always ``$external``. +When a client is constructed using an SRV URI, the driver will look for URI +options in a TXT DNS record that corresponds to the SRV record. Thus, for +example, MongoDB Atlas generally uses the ``admin`` database as its auth +source, but this is not specified in SRV URIs because the database is given +as a URI option on the TXT records. + +Note that when using SRV URIs, the SRV query and the TXT query are performed +separately. On systems where DNS resolution is not 100% reliable, the +failure to look up TXT records can cause authentication errors, as the driver +may end up using an incorrect auth source. If reliable DNS resolution cannot +be guaranteed, the auth source can be specified explicitly in SRV URIs as +a URI option: + +.. code-block:: ruby + + Mongo::Client.new("mongodb+srv://username:myRealPassword@cluster0.mongodb.net/test?w=majority&authSource=admin") + .. note:: When changing the database using the ``with`` method, the auth source is From c1245ea305fe95942eec2adc66890af85e22f4f1 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 6 Aug 2020 11:13:11 -0400 Subject: [PATCH 289/442] Fix RUBY-2346 Tag names are downcased when given in URI options (#2038) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-create-client.txt | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index e07bddf15..f785b9ca5 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -267,8 +267,13 @@ Ruby Options * - ``:auth_mech_properties`` - Provides additional authentication mechanism properties. + + The keys in properties are interpreted case-insensitively. + When the client is created, keys are lowercased. + - ``Hash`` - - none + - When using the GSSAPI authentication mechanism, the default properties + are ``{service_name: "mongodb"}``. Otherwise the default is nil. * - ``:auth_source`` - Specifies the authentication source. From 650dbc97a944ff509087892040f9ad7c6fedd107 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 7 Aug 2020 13:39:25 -0400 Subject: [PATCH 290/442] RUBY-1817 Link countDocuments <-> estimatedDocumentCount documentation (#2044) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-crud-operations.txt | 30 ++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 2e53bbe8e..ed9e92dcc 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -278,8 +278,36 @@ when querying and their corresponding methods as examples. Additional Query Operations --------------------------- +``count_documents`` + Get the total number of documents matching a filter, or the total number + of documents in a collection. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + + client[:artists].find(:name => 'Flying Lotus').count_documents + +``estimated_document_count`` + Get an approximate number of documents in the collection. + + Note that unlike ``count_documents``, ``estimated_document_count`` does not + accept a filter. + +.. code-block:: ruby + + client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + + client[:artists].estimated_document_count + ``count`` - Get the total number of documents an operation returns. + Get an approximate number of documents matching a filter, or an approximate + number of documents in the collection. + + *Deprecated:* The ``count`` method is deprecated and does not work in + transactions. Please use ``count_documents`` to obtain an exact count of + documents potentially matching a filter or ``estimated_document_count`` + to obtain an approximate number of documents in the collection. .. code-block:: ruby From 7cd6e31d31f5c470c946076718d58722688241f0 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 7 Aug 2020 16:15:02 -0400 Subject: [PATCH 291/442] RUBY-1672 Document passing read concern to find operations in tutorial (#2046) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-crud-operations.txt | 34 +++++++++++++++++++ source/tutorials/ruby-driver-transactions.txt | 2 ++ 2 files changed, 36 insertions(+) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index ed9e92dcc..9a55ecec5 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -352,6 +352,40 @@ following code example shows how a tailable cursor might be used: sleep(1) end + +Read Concern +------------ + +Read concern can be :ref:`set on the client ` +or on the collection: + +.. code-block:: ruby + + client = Mongo::Client.new(['localhost:14420'], database: 'music', + read_concern: {level: :local}) + + client['collection'].find.to_a + + collection = client['collection', read_concern: {level: :majority}] + + collection.find.to_a + +The driver does not currently support setting read concern on an individual +query. + +Read concern can be specified when :ref:`starting a transaction +`. When a transaction is active, :manual:`any read concern +specified on the client or on the collection is ignored +`. + +When using the generic command helper, the read concern can be specified as +part of the command: + +.. code-block:: ruby + + client.database.command(collstats: 'test', readConcern: {level: :majority}) + + .. _ruby-driver-read-preference: Read Preference diff --git a/source/tutorials/ruby-driver-transactions.txt b/source/tutorials/ruby-driver-transactions.txt index 3023925a4..da15f14c1 100644 --- a/source/tutorials/ruby-driver-transactions.txt +++ b/source/tutorials/ruby-driver-transactions.txt @@ -16,6 +16,8 @@ Version 4.0 of the MongoDB server introduces (Updates to multiple fields within a single document are atomic in all versions of MongoDB.) Ruby driver version 2.6.0 adds support for transactions. +.. _using-transactions: + Using Transactions ================== From f4735612c5f20142326e9afc32ceba16f6a1f150 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 10 Aug 2020 11:52:46 -0400 Subject: [PATCH 292/442] Fix RUBY-2252 Bulk write documentation references nonexistent insert_many operation (#2048) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-bulk-operations.txt | 77 +++++++++++-------- 1 file changed, 43 insertions(+), 34 deletions(-) diff --git a/source/tutorials/ruby-driver-bulk-operations.txt b/source/tutorials/ruby-driver-bulk-operations.txt index 945361e21..986f7bfc2 100644 --- a/source/tutorials/ruby-driver-bulk-operations.txt +++ b/source/tutorials/ruby-driver-bulk-operations.txt @@ -12,18 +12,24 @@ Bulk Operations .. _ruby-driver-bulk-ops: -The bulk write API sends a list of write operations to the server in -one method call. Execution order of the operations is guaranteed if -you pass in the ``ordered`` option. +The bulk write API sends several write operations to the server in a single +command. Use the bulk write API to reduce the number of network round-trips +when performing several writes at a time. + +If the ``ordered`` option is set to ``true`` (which is the default), +the operations are applied in order and if any operation fails, subsequent +operations are not attempted. If the ``ordered`` option is set to ``false``, +all specified operations are attempted. The ``bulk_write`` method takes three arguments: -- A list of operations. -- The ``ordered`` option with a boolean. Defaults to ``true``. -- A write concern option. Defaults to the collection's write concern. +- A list of operations to execute. +- The ``ordered`` option taking a boolean value. Defaults to ``true``. +- The write concern option. Defaults to the collection's write concern. Valid bulk write operations are the following: + insert_one ========== @@ -31,36 +37,11 @@ insert_one { :insert_one => { :x => 1 } } -insert_many -=========== - -.. code-block:: ruby - - { :insert_many => [ { :x => 1 }, { :x => 2 } ] } - -delete_one -========== - -.. code-block:: ruby - - { :delete_one => { :filter => { :x => 1 } } } - -delete_many -=========== - -.. code-block:: ruby - - { :delete_many => { :filter => { :x => 1 } } } - -replace_one -=========== +.. note:: -.. code-block:: ruby + There is no ``insert_many`` operation. To insert multiple documents, + specify multiple ``insert_one`` operations. - { :replace_one => { :filter => { :x => 1 }, - :replacement => { :x => 2 }, - :upsert => true } # upsert is optional and defaults to false - } update_one ========== @@ -72,6 +53,7 @@ update_one :upsert => true } # upsert is optional and defaults to false } + update_many =========== @@ -100,3 +82,30 @@ to the ``bulk_write`` method. } ], :ordered => true) + + +replace_one +=========== + +.. code-block:: ruby + + { :replace_one => { :filter => { :x => 1 }, + :replacement => { :x => 2 }, + :upsert => true } # upsert is optional and defaults to false + } + + +delete_one +========== + +.. code-block:: ruby + + { :delete_one => { :filter => { :x => 1 } } } + + +delete_many +=========== + +.. code-block:: ruby + + { :delete_many => { :filter => { :x => 1 } } } From 502149d35f33285101fdb0ec3cb7270a0a3d67d5 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 10 Aug 2020 11:59:38 -0400 Subject: [PATCH 293/442] RUBY-2194 Provide forking guidance relevant to the driver (#2049) * RUBY-2194 Provide forking guidance relevant to the driver * misc fixes * link to server docs Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 113 +++++++++++++----- 1 file changed, 81 insertions(+), 32 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index f785b9ca5..6dcc2b92b 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -1129,12 +1129,16 @@ Note that when creating a client using the `block syntax <#block-syntax>`_ descr Usage with Forking Servers ========================== -*Note:* Applications using Mongoid should follow `Mongoid's forking guidance -`_. -The sample code below is provided for applications using the Ruby driver directly. - -When using the Mongo Ruby driver with a forking web server such as Unicorn, -worker processes should generally each have their own Mongo::Client instances. +.. note:: + + Applications using Mongoid should follow `Mongoid's forking guidance + `_. + The guidance and sample code below is provided for applications using the + Ruby driver directly. + +When using the Mongo Ruby driver in a Web application with a forking web server +such as Unicorn, Puma or Passenger, or when the application otherwise forks, +each process should generally each have their own ``Mongo::Client`` instances. This is because: 1. The background threads remain in the parent process and are not transfered @@ -1142,42 +1146,87 @@ This is because: 2. File descriptors like network sockets are shared between parent and child processes. -It is recommended to not create any Mongo::Client instances prior to the fork. -If the parent process needs to perform operations on the MongoDB database, -reset the client in an ``after_fork`` handler which is defined in -``unicorn.rb``: +The driver attempts to detect client use from forked processes and +reestablish network connections when such use is detected, alleviating +the issue of file descriptor sharing. + +If both parent and child processes need to perform MongoDB operations, +it is recommended for each of the processes to create their own +``Mongo::Client`` instances. Specifically, the child process should create +its own client instance and not use any of the instances that were created +in the parent. + +If the parent continues to perform MongoDB operations using an already +established client instance after forking children, this client instance will +continue to operate normally as long as no child uses it in any way. +The child processes will not inherit any of the monitoring threads, and +will not perform background operations on the client instance. + +If the parent does not need to perform MongoDB operation after forking +children (which is what typically happens in web applications), the parent +should close all of the client instances it created to free up connections +and cease background monitoring: .. code-block:: ruby - after_fork do |server, worker| - $client.close - $client.reconnect - end + client.reconnect + +.. note:: + + If the parent process performs operations on the Mongo client and does not + close it, the parent process will continue consuming a connection slot + in the cluster and will continue monitoring the cluster for as long as the + parent remains alive. + +Reconnecting Client Instances +````````````````````````````` + +When the Ruby driver is used in a web application, it is recommended to not +create any ``Mongo::Client`` instances in the management processes (prior to +the workers being forked), and instead only create client instances in the +workers. -The above code assumes your Mongo::Client instance is stored in the $client -global variable. It also keeps the MongoDB connection in the parent process -around, and this connection will continue to consume resources such as -a connection slot. If the parent process performs one time operation(s) on -MongoDB and does not need its MongoDB connection once workers are forked, -the following code will close the connection in the parent: +It is possible, although not recommended, to use the same ``Mongo::Client`` +instances in parent and child processes. In order to do so, the instance +must be closed and reconnected in the child process so that the background +threads can be recreated: .. code-block:: ruby - before_fork do |server, worker| - $client.close - end + client.close + client.reconnect - after_fork do |server, worker| - $client.reconnect - end +.. note:: + + This pattern should be used with Ruby driver version 2.6.2 or higher. + Previous driver versions did not recreate monitoring threads when + reconnecting. -*Note:* This pattern should be used with Ruby driver version 2.6.2 or higher. -Previous driver versions did not recreate monitoring threads when reconnecting. +.. note:: -*Note:* If the parent process performs operations on the Mongo client and -does not close it, the parent process will continue consuming a connection slot -in the cluster and will continue monitoring the cluster for as long as the -parent remains alive. + When closing and reconnecting the client instance in the child, + due to file descriptor sharing, the parent process may experience network + and monitoring errors. + +Web servers generally provide hooks that can be used by applications to +perform actions when the worker processes are forked. The recommended hooks +to use are: + +- For `Puma `_, ``before_fork`` to close clients in the + parent process and ``on_worker_boot`` to reconnect in the child processes. +- For `Unicorn `_, + ``before_fork`` to close clients in the parent process and + ``after_fork`` to reconnect clients in the child processes. +- For `Passenger `_, + ``starting_worker_process`` to reconnect clients in the child processes + (Passenger does not appear to have a pre-fork hook). + +This documentation does not provide example code for using the aforementioned +hooks, because there is no standard for client instance management when +using the Ruby driver directly. `Mongoid documentation +`_ +however provides examples for closing clients in the parent process and +reconnecting clients in the child processes. Details on Retryable Reads From 03b976cf2a4d3003cd6752d6ef6d62c8bcb94dd0 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 13 Aug 2020 12:36:32 -0400 Subject: [PATCH 294/442] RUBY-2357 standardize on STDOUT over $stdout (#2052) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 6dcc2b92b..ba9610324 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -1387,7 +1387,7 @@ For more control, a logger can be passed to a client for per-client control over .. code-block:: ruby - my_logger = Logger.new($stdout) + my_logger = Logger.new(STDOUT) Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :logger => my_logger ) Truncation From 5119adaa98072757d2a87e0e06d676287dcfb32b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Sat, 15 Aug 2020 23:42:04 -0400 Subject: [PATCH 295/442] RUBY-2359 Implement OCSP URI options (#2058) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 44 ++++++++++++++++--- 1 file changed, 38 insertions(+), 6 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index ba9610324..55595ae33 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -583,10 +583,12 @@ Ruby Options - none * - ``:ssl_verify`` - - Whether to perform peer certificate validation and hostname verification. Note that - the decision of whether to validate certificates will be overridden if - :ssl_verify_certificate is set, and the decision of whether to validate hostnames will be - overridden if :ssl_verify_hostname is set. + - Whether to perform peer certificate, hostname and OCSP endpoint + validation. Note that the decision of whether to validate certificates + will be overridden if ``:ssl_verify_certificate`` is set, the decision + of whether to validate hostnames will be overridden if + ``:ssl_verify_hostname`` is set and the decision of whether to validate + OCSP endpoint will be overridden if ``:ssl_verify_ocsp_endpoint`` is set. - ``Boolean`` - true @@ -597,8 +599,16 @@ Ruby Options - true * - ``:ssl_verify_hostname`` - - Whether to perform peer hostname validation. This setting overrides :ssl_verify with - respect to whether hostname validation is performed. + - Whether to perform peer hostname validation. This setting overrides + :ssl_verify with respect to whether hostname validation is performed. + - ``Boolean`` + - true + + * - ``:ssl_verify_ocsp_endpoint`` + - Whether to validate server-supplied certificate against the OCSP + endpoint specified in the certificate, if the OCSP endpoint is specified + in the certificate. This setting overrides :ssl_verify with respect to + whether OCSP endpoint validation is performed. - ``Boolean`` - true @@ -649,6 +659,11 @@ Ruby Options - ``Integer`` - none +.. note:: + + The Ruby driver does not implement certificate revocation list (CRL) + checking. + URI Options ``````````` @@ -820,6 +835,14 @@ URI options are explained in detail in the :manual:`Connection URI reference * - tlsCertificateKeyFilePassword=String - ``:ssl_key_pass_phrase => String`` + * - tlsDisableOCSPEndpointCheck=Boolean + - ``:ssl_verify_ocsp_endpoint => boolean`` + + Because ``tlsDisableOCSPEndpointCheck`` uses ``true`` to signify that + verification should be disabled and ``ssl_verify_ocsp_endpoint`` uses + ``false`` to signify that verification should be disabled, the boolean + is inverted before being used to set ``ssl_verify_ocsp_endpoint``. + * - tlsInsecure=Boolean - ``:ssl_verify => boolean`` @@ -839,6 +862,15 @@ URI options are explained in detail in the :manual:`Connection URI reference * - zlibCompressionLevel=Integer - ``:zlib_compression_level => Integer`` +.. note:: + + The Ruby driver only fails connections when it receives a definitive signed + response indicating that the server's certificate has been revoked. + Because of this, the driver does not recognize the + ``tlsDisableCertificateRevocationCheck`` URI option. If this option is + provided in a URI, it will be ignored. + + Timeout Options =============== From 94d710a70a0eaa32d157c3a2dcdf14469824db9b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 17 Aug 2020 10:36:05 -0400 Subject: [PATCH 296/442] RUBY-2353 Deprecate Ruby 2.3 and 2.4 support (#2063) Co-authored-by: Oleg Pudeyev --- ...by-driver-compatibility-matrix-mongodb.rst | 8 ++++- source/reference/driver-compatibility.txt | 34 ++++++++++++++++--- 2 files changed, 36 insertions(+), 6 deletions(-) diff --git a/source/includes/ruby-driver-compatibility-matrix-mongodb.rst b/source/includes/ruby-driver-compatibility-matrix-mongodb.rst index bf3d8e572..dd37364bc 100644 --- a/source/includes/ruby-driver-compatibility-matrix-mongodb.rst +++ b/source/includes/ruby-driver-compatibility-matrix-mongodb.rst @@ -1,5 +1,11 @@ The following compatibility table specifies the recommended version(s) of the MongoDB Ruby driver for use with a specific version of -MongoDB. +MongoDB. Except when indicated, the specified driver versions expose or +take advantage of the features added in the corresponding server versions. + +MongoDB server releases are generally backwards compatible, meaning a +particular version of the driver will generally work with newer versions of +the server but may not take advantage of the functionality released in the +newer version of the server. The first column lists the driver versions. diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 676ed7452..7ae0ec4a3 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -35,6 +35,17 @@ MongoDB Compatibility - MongoDB 2.6 - MongoDB 2.4 + * - 2.14 + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - |checkmark| + - + * - 2.13 - |checkmark| [#ocsp]_ - |checkmark| @@ -178,7 +189,7 @@ MongoDB Compatibility - |checkmark| - |checkmark| -.. [#ocsp] OCSP verification is not yet implemented. +.. [#ocsp] OCSP verification is implemented as of driver version 2.14. .. [#srv-polling] Polling of SRV records in sharded topologies is implemented as of driver version 2.11. @@ -193,9 +204,8 @@ The driver does not support older versions of MongoDB. Ruby Compatibility ================== -The following compatibility table specifies the recommended -version(s) of the MongoDB Ruby driver for use with a specific version of -Ruby. +The following compatibility table specifies the versions of Ruby supported +by the various versions of the MongoDB Ruby driver. The first column lists the driver versions. "D" in a column means support for that Ruby version is deprecated. @@ -219,6 +229,20 @@ for that Ruby version is deprecated. - JRuby 9.1 - JRuby + * - 2.14 + - |checkmark| + - |checkmark| + - |checkmark| + - D + - D + - + - + - + - + - |checkmark| + - + - + * - 2.13 - |checkmark| - |checkmark| @@ -459,7 +483,7 @@ the driver. * - Ruby Driver - mongo_kerberos |nbsp| 2.1 - * - 2.7 - 2.13 + * - 2.7 - 2.14 - |checkmark| From 8b4d7f1448b6cee01f51eb66e59010b22d039145 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 17 Aug 2020 15:53:09 -0400 Subject: [PATCH 297/442] RUBY-2289 Display monitoring state in client summary (#2053) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 55595ae33..04d70f85a 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -1260,6 +1260,39 @@ using the Ruby driver directly. `Mongoid documentation however provides examples for closing clients in the parent process and reconnecting clients in the child processes. +Troubleshooting +``````````````` + +The client's ``summary`` method returns the current state of the client, +including servers that the client is monitoring and their state. If any of +the servers are not being monitored, this is indicated by the ``NO-MONITORING`` +flag. + +A normally operating client will produce a summary similar to the following: + +.. code-block:: ruby + + client.summary + => "#>, + #>, + #>, + #]>>" + +A client that is missing background threads will produce a summary similar to +the following: + +.. code-block:: ruby + + client.summary + => "#>, + #>, + #>, + #]>>" + Details on Retryable Reads ========================== From 55da1f2d75fd06cd87d64028156b940c479fab5e Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 21 Aug 2020 11:55:54 -0400 Subject: [PATCH 298/442] RUBY-1932 Specify timeouts for DNS queries for SRV URIs (#2065) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-create-client.txt | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 04d70f85a..3b7886ba6 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -330,7 +330,8 @@ Ruby Options * - ``:connect_timeout`` - The number of seconds to wait to establish a socket connection - before raising an exception. ``nil`` and ``0`` mean no timeout. + before raising an exception. This timeout is also used for SRV DNS + record resolution. ``nil`` and ``0`` mean no timeout. Client creation will fail with an error if an invalid timeout value is passed (such as a negative value or a non-numeric value). - ``Float`` @@ -934,8 +935,14 @@ This option does not apply to monitoring connections. ``````````````````` The number of seconds to wait for a socket connection to be established to -a server. Defaults to 10. This timeout is also used as both connect timeout -and socket timeout for monitoring connections. +a server. Defaults to 10. + +This timeout is also used as both connect timeout and socket timeout for +monitoring connections. + +When using a ``mongodb+srv://` URI, this timeout is also used for SRV and TXT +DNS lookups. Note that the timeout applies per lookup; due to DNS suffix search +lists, multiple lookups may be performed as part of a single name resolution. ``wait_queue_timeout`` `````````````````````` From b631866bfb50ea1c85499f07a4410230f315a1dc Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Mon, 24 Aug 2020 13:56:58 -0400 Subject: [PATCH 299/442] Add query cache anchor --- source/tutorials/ruby-driver-crud-operations.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 9a55ecec5..d4e58d8a4 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -130,6 +130,9 @@ object, or with ``Decimal128.from_string()``. price = BSON::Decimal128.from_string("428.79") # => BSON::Decimal128('428.79') + +.. _query-cache: + Query Cache =========== From fb334313c24134765b665742b0627895751c5ea1 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 31 Aug 2020 10:42:19 -0400 Subject: [PATCH 300/442] RUBY-1748 Add documentation warning against the use of duplicate key names (#217) Co-authored-by: Oleg Pudeyev --- docs/tutorials/bson-v4.txt | 53 +++++++++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 7a0fba636..0a81e072e 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -11,7 +11,7 @@ BSON 4.x Tutorial .. contents:: On this page :local: :backlinks: none - :depth: 1 + :depth: 2 :class: twocols This tutorial discusses using the Ruby BSON library. @@ -409,6 +409,20 @@ access to them with symbol keys. BSON::Document[:key, "value"] BSON::Document.new +.. note:: + + All BSON documents are deserialized into instances of BSON::Document, + even when the invocation is made from the ``Hash`` class: + + .. code-block:: ruby + + bson = {test: 1}.to_bson.to_s + loaded = Hash.from_bson(BSON::ByteBuffer.new(bson)) + => {"test"=>1} + loaded.class + => BSON::Document + + ``BSON::MaxKey`` ```````````````` @@ -659,3 +673,40 @@ object, one must call ``compile`` on the returned object. regex.pattern #=> Returns the pattern as a string. regex.options #=> Returns the raw options as a String. regex.compile #=> Returns the compiled Ruby Regexp object. + + +Key Order +--------- + +BSON documents preserve the order of keys, because the documents are stored +as lists of key-value pairs. Hashes in Ruby also preserve key order; thus +the order of keys specified in Ruby will be respected when serializing a +hash to a BSON document, and when deserializing a BSON document into a hash +the order of keys in the document will match the order of keys in the hash. + + +Duplicate Keys +-------------- + +BSON specification allows BSON documents to have duplicate keys, because the +documents are stored as lists of key-value pairs. Applications should refrain +from generating such documents, because MongoDB server behavior is undefined +when a BSON document contains duplicate keys. + +Since in Ruby hashes cannot have duplicate keys, when serializing Ruby hashes +to BSON documents no duplicate keys will be generated. (It is still possible +to hand-craft a BSON document that would have duplicate keys in Ruby, and +some of the other MongoDB BSON libraries may permit creating BSON documents +with duplicate keys.) + +Note that, since keys in BSON documents are always stored as strings, +specifying the same key as as string and a symbol in Ruby only retains the +most recent specification: + +.. code-block:: ruby + + BSON::Document.new(test: 1, 'test' => 2) + => {"test"=>2} + +When loading a BSON document with duplicate keys, the last value for a +duplicated key overwrites previous values for the same key. From c1f069cf6284bd56de8db2fba60107a59bcfd22a Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 31 Aug 2020 10:45:04 -0400 Subject: [PATCH 301/442] Add CodeWithScope deprecation notice to documentation (#216) Co-authored-by: Nobody --- docs/tutorials/bson-v4.txt | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 0a81e072e..6d47fc74e 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -392,6 +392,13 @@ Represents a string of JavaScript code. ``BSON::CodeWithScope`` ``````````````````````` +.. note:: + + The ``CodeWithScope`` type is deprecated as of MongoDB 4.2.1. Starting + with MongoDB 4.4, support from ``CodeWithScope`` is being removed from + various server commands and operators such as ``$where``. Please use + other BSON types and operators when working with MongoDB 4.4 and newer. + Represents a string of JavaScript code with a hash of values. .. code-block:: ruby From 54f93fada241e900dbc4d305bbbe96e2dcbb4fdc Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 1 Sep 2020 11:04:27 -0400 Subject: [PATCH 302/442] Remove pure Ruby note from documentation (#2076) The driver depends on bson-ruby which requires its C extension, hence the pure Ruby note is misleading. The "simplicity" part is content-free, get rid of it also. --- source/index.txt | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/source/index.txt b/source/index.txt index d7e881efc..efd35b5f9 100644 --- a/source/index.txt +++ b/source/index.txt @@ -8,9 +8,8 @@ Ruby MongoDB Driver .. default-domain:: mongodb -The MongoDB Ruby driver is the officially supported Ruby driver for -MongoDB. It is written in pure Ruby and is optimized for simplicity. It -can be used on its own, but it also serves as the basis of several +The MongoDB Ruby driver is the officially supported Ruby driver for MongoDB. +It can be used on its own, but it also serves as the basis of several object mapping libraries. Get Started From 86a13d185aafb1cfb6ec7c371fec85ee5b65adff Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Tue, 1 Sep 2020 23:16:08 -0400 Subject: [PATCH 303/442] Fix Atlas link in docs --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 3b7886ba6..5ebf82506 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -38,7 +38,7 @@ to connect to; if no database name is specified, the client will use the The hostname ``localhost`` is treated specially by the driver and will be resolved to IPv4 addresses only. -To `connect to MongoDB Atlas `, +To `connect to MongoDB Atlas `_, specify the Atlas deployment URI: .. code-block:: ruby From c73e34db7a6fe548bba8ab7d29bdf279386b881d Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 2 Sep 2020 00:14:32 -0400 Subject: [PATCH 304/442] Add a newline at the end --- source/tutorials/ruby-driver-sessions.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-sessions.txt b/source/tutorials/ruby-driver-sessions.txt index 216a2bf7c..64e5db692 100644 --- a/source/tutorials/ruby-driver-sessions.txt +++ b/source/tutorials/ruby-driver-sessions.txt @@ -146,4 +146,4 @@ When a client is closed, the driver will send a command to the server to end all in its server session pool. You may see this command in your logs when a client is closed. Note that when using the `block syntax <#creating-a-session-from-a-mongo-client>`_ for ``start_session`` the session is automatically ended after -the block finishes executing. \ No newline at end of file +the block finishes executing. From 5c7be2770ea990f646932d4afe8be9c5f86eb2fd Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 2 Sep 2020 16:06:39 -0400 Subject: [PATCH 305/442] RUBY-2393 Clarify TLS configuration behavior (#2078) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 142 ++++++++++++++---- 1 file changed, 109 insertions(+), 33 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 5ebf82506..cb457a77e 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -977,33 +977,83 @@ on the client or passed to individual operations under ``:write_concern``. TLS Connections =============== -To connect to the MongoDB cluster using TLS, pass the ``ssl`` Ruby option or -the ``tls`` URI option to the ``Mongo::Client`` constructor. TLS must be -explicitly requested on the client side when the deployment requires TLS -connections - there is currently no automatic detection of whether the -deployment requires TLS. +To connect to the MongoDB deployment using TLS: -The driver will attempt to verify the server's TLS certificate by default, and -will abort the connection if this verification fails. In order for this -verification to succeed, you may need to specify the certificate authority -that the server certificate is signed with via ``tlsCAFile`` URI option or -``ssl_ca_cert`` Ruby option. If these options are not given, the driver will -use the default system root certificate store as the trust anchor; if these -options are given, the default system root certificate store will not be used. -To omit TLS certificate verification by the client, which is not -recommended, specify ``tlsInsecure=true`` URI option or ``ssl_verify: false`` -Ruby option. - -By default, MongoDB server will verify the connecting clients' TLS certificates, -which requires the client to specify a client certificate when connecting. -This can be accomplished through ``tlsCertificateKeyFile`` URI option which -takes as the argument the path to a single file containing both the certificate -and the corresponding private key, or through ``:ssl_cert`` and ``:ssl_key`` -Ruby options. The Ruby options allow passing separace certificate and key files -to the driver, but also can be pointed at the single PEM file containing both -the certificate and the corresponding private key. Further information on -configuring MongoDB server for TLS is available in the `MongoDB manual -`_. +- Enable TLS connections in ``Mongo::Client``. +- Specify the client TLS certificate. +- Specify the CA certificate to verify the server's TLS certificate. + +Enable TLS Connections +---------------------- + +TLS must be explicitly requested on the client side when the deployment +requires TLS connections - there is currently no automatic detection of +whether the deployment requires TLS. + +To request TLS connections, specify the following client options when +constructing a ``Mongo::Client``: + +- The ``:ssl`` Ruby option. +- The ``tls`` URI option. +- The ``ssl`` URI option (deprecated). + +Specify Client TLS Certificate +------------------------------ + +By default, MongoDB server will attempt to verify the connecting clients' +TLS certificates, which requires the clients to specify their TLS certificates +when connecting. This can be accomplished via: + +- The ``:ssl_cert``/``:ssl_cert_object``/``:ssl_cert_string` and + ``:ssl_key``/``:ssl_key_object``/``:ssl_key_string``/``:ssl_key_pass_phrase`` + Ruby options. +- The``tlsCertificateKeyFile`` URI option. + +When using the Ruby options, the client TLS certificate and the corresponding +private key may be provided separately. For example, if the certificate is +stored in ``client.crt`` and the private key is stored in ``client.key``, +a ``Mongo::Client`` may be constructed as follows: + +.. code-block:: ruby + + client = Mongo::Client.new(["localhost:27017"], + ssl: true, + ssl_cert: 'path/to/client.crt', + ssl_key: 'path/to/client.key', + ssl_ca_cert: 'path/to/ca.crt', + ) + +``ssl_cert``, ``ssl_cert_string``, ``ssl_key`` and ``ssl_key_string`` Ruby +options also permit the certificate and the key to be provided in the same +file or string, respectively. The files containing both certificate and +private key frequently have the ``.pem`` extension. When both certificate +and the private key are provided in the same file or string, both the +certifcate and the key options must be utilized, as follows: + +.. code-block:: ruby + + client = Mongo::Client.new(["localhost:27017"], + ssl: true, + ssl_cert: 'path/to/client.pem', + ssl_key: 'path/to/client.pem', + ssl_ca_cert: 'path/to/ca.crt', + ) + +When using the URI option, the certificate and the key must be stored in a +file and both must be stored in the same file. Example usage: + +.. code-block:: ruby + + client = Mongo::Client.new( + "mongodb://localhost:27017/?tls=true&tlsCertificateKeyFile=path%2fto%2fclient.pem&tlsCertificateKeyFile=path%2fto%2fca.crt") + +.. note:: + + URI option values must be properly URI escaped. This applies, for example, to + slashes in the paths. + +Further information on configuring MongoDB server for TLS is available in the +:manual:`MongoDB manual `. Using Intermediate Certificates ``````````````````````````````` @@ -1034,22 +1084,48 @@ so, the intermediate certificates are elevated to trusted status and are themselves not verified against the actual CA root. More information on this issue is available `here `_. +Specify CA Certificate +---------------------- + +The driver will attempt to verify the server's TLS certificate by default, and +will abort the connection if this verification fails. By default, the driver +will use the default system root certificate store as the trust anchor. +To specify the CA certificate that the server's certificate is signed with, +use: + +- The ``:ssl_ca_cert``/``:ssl_ca_cert_string``/``:ssl_ca_cert_object`` + Ruby options +- The ``tlsCAFile`` URI option. + +If any of these options are given, the server's certificate will be verified +only against the specified CA certificate and the default system root +certificate store will not be used. + +To not perform server TLS certificate verification, which is not +recommended, specify the ``ssl_verify: false`` Ruby option or the +``tlsInsecure=true`` URI option. + Specifying Multiple CA Certificates ``````````````````````````````````` -``:ssl_ca_cert`` Ruby option and ``tlsCAFile`` URI option can be used with +The ``:ssl_ca_cert`` Ruby option and ``tlsCAFile`` URI option can be used with a file containing multiple certificates. All certificates thus referenced will become trust anchors. -``:ssl_ca_cert_object`` option takes an array of certificates, and thus +The ``:ssl_ca_cert_object`` option takes an array of certificates, and thus can also be used to add multiple certificates as certificate authorities. -``:ssl_ca_cert_string`` option supports passing only one CA certificate to the -driver. +The ``:ssl_ca_cert_string`` option supports specifying only one CA certificate. -Intermediate certificates should not be specified in files given to any -of these options, because they would then not be verified against actual -trusted CA certificates. +.. warning:: + + Intermediate certificates must not be provided in files specified by the + CA certificate options. Doing so would elevate the intermediate certificates + to the status of root certificates, rather than verifying intermediate + certificates against the root certificates. + + If intermediate certificates need to be used, specify them as part of the + client or server TLS certificate files. IPv4/IPv6 Connections From c87f87f06316e3867b11b02af427f985f55820f2 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 8 Sep 2020 14:36:59 -0400 Subject: [PATCH 306/442] RUBY-2352 Add try_next example to resuming change stream user guide (#2079) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-change-streams.txt | 71 ++++++++++++++----- 1 file changed, 53 insertions(+), 18 deletions(-) diff --git a/source/tutorials/ruby-driver-change-streams.txt b/source/tutorials/ruby-driver-change-streams.txt index 114db9aa9..e8f896a8c 100644 --- a/source/tutorials/ruby-driver-change-streams.txt +++ b/source/tutorials/ruby-driver-change-streams.txt @@ -134,9 +134,15 @@ You can close a change stream by calling its ``#close`` method: Resuming a Change Stream ======================== -The driver will automatically retry getMore operations on a change stream -once. Initial aggregation is never retried. In practical terms this means -that, for example: +A change stream consists of two types of operations: the initial aggregation +and ``getMore`` requests to receive the next batch of changes. + +The driver will automatically retry each ``getMore`` operation once on +network errors and when the server returns an error indicating it changed +state (for example, it is no longer the primary). The driver does not retry +the initial aggregation. + +In practical terms this means that, for example: - Calling ``collection.watch`` will fail if the cluster does not have enough available nodes to satisfy the ``"majority"`` read preference. @@ -145,34 +151,63 @@ that, for example: change stream reads via ``next`` or ``each`` methods will continue transparently to the application. -If the cluster loses enough nodes to not be able to satisfy the ``"majority"`` -read preference and does not heal quickly enough, ``next`` and ``each`` -will raise an error. In these cases the application must track, via the -resume token, which documents from the change stream it has processed and -create a new change stream object via the ``watch`` call, passing an -appropriate ``:resume_after`` argument. The ``_id`` key in each change -document can be used as a resume token. However, to get the most up-to-date -resume token, use the ``resume_token`` method. +To indefinitely and reliably watch for changes without losing any changes or +processing a change more than once, the application must track the resume +token for the change stream and restart the change stream when it experiences +extended error conditions that cause the driver's automatic resume to also +fail. The following code snippet shows an example of iterating a change stream +indefinitely, retrieving the resume token using the ``resume_token`` change +stream method and restarting the change stream using the ``:resume_after`` +option on all MongoDB or network errors: .. code-block:: ruby - token = doc['_id'] - stream = collection.watch([], resume_after: token) + token = nil + loop do + begin + stream = collection.watch([], resume_after: token) + enum = stream.to_enum + while doc = enum.next + process(doc) + token = stream.resume_token + end + rescue Mongo::Error + sleep 1 + end + end -To watch a collection indefinitely, retrying on all MongoDB errors, using ``next``: +The above iteration is blocking at the ``enum.next`` call, and does not +permit resuming processing in the event the Ruby process running this code +is terminated. The driver also provides the ``try_next`` method which returns +``nil`` (after a small waiting period) instead of blocking indefinitely when +there are no changes in the change stream. Using the ``try_next`` method, +the resume token may be persisted after each ``getMore`` request, even when +a particular request does not return any changes, such that the resume token +remains at the top of the oplog and the application has an opportunity to +persist it should the process handling changes terminates: .. code-block:: ruby token = nil - while true + loop do begin stream = collection.watch([], resume_after: token) enum = stream.to_enum - while doc = enum.next + doc = enum.try_next + if doc process(doc) - token = stream.resume_token end + token = stream.resume_token + # Persist +token+ to support resuming processing upon process restart rescue Mongo::Error sleep 1 end - end \ No newline at end of file + end + +Note that the resume token should be retrieved from the change stream after +every ``try_next`` call, even if the call returned no document. + +The resume token is also provided in the ``_id`` field of each change stream +document. Reading the ``_id`` field is not recommended because it may be +projected out by the application, and because using only the ``_id`` field +would not advance the resume token when a ``getMore`` returns no documents. From 232d9ad696f124aa55dac78c1535e85d9f5713e2 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 11 Sep 2020 11:48:38 -0400 Subject: [PATCH 307/442] RUBY-2361 stub ocsp verifier tests on jruby due to lack of ocsp endpoint retrieval support (#2082) * RUBY-2361 stub ocsp verifier tests on jruby due to lack of ocsp endpoint retrieval support * RUBY-2362 omit revoked configurations with jruby * RUBY-2362 omit jruby+ecdsa configurations Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index cb457a77e..1d8b59963 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -983,6 +983,10 @@ To connect to the MongoDB deployment using TLS: - Specify the client TLS certificate. - Specify the CA certificate to verify the server's TLS certificate. +.. note:: + + When using JRuby, ECDSA certificates are not currently supported. + Enable TLS Connections ---------------------- @@ -1127,6 +1131,22 @@ The ``:ssl_ca_cert_string`` option supports specifying only one CA certificate. If intermediate certificates need to be used, specify them as part of the client or server TLS certificate files. +OCSP Verification +----------------- + +If the certificate provided by the server contains an OCSP endpoint URI, +the driver will issue an OCSP request to the specified endpoint to verify the +validity of the certificate. + +The OCSP endpoint check may be disabled by setting the +``:ssl_verify_ocsp_endpoint`` Ruby option to ``false`` or by setting the +``tlsDisableOCSPEndpointCheck`` URI option to ``true`` when creating a client. + +.. note:: + + OCSP endpoint checking is not currently performed when running on JRuby, + since JRuby does not correctly expose the OCSP endpoint URI. + IPv4/IPv6 Connections ===================== From 68b5bb8881fcb01a4bc47ade26a8ce50c2715ed9 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 14 Sep 2020 15:13:05 -0400 Subject: [PATCH 308/442] RUBY-2398 Use TLS instead of SSL in driver documentation (#2089) * RUBY-2398 Use TLS instead of SSL in driver documentation * change the docstrings also * chase message change in tests Co-authored-by: Oleg Pudeyev --- source/installation.txt | 17 +++-- source/reference/driver-compatibility.txt | 16 ++++- .../tutorials/ruby-driver-authentication.txt | 8 +-- .../tutorials/ruby-driver-create-client.txt | 67 ++++++++++++------- 4 files changed, 71 insertions(+), 37 deletions(-) diff --git a/source/installation.txt b/source/installation.txt index 517f7a3fa..632d24fef 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -33,12 +33,14 @@ of improvements and changes in each version of the driver. TLS/SSL and the Ruby Driver =========================== -Industry best practices, and some regulations, require the use of TLS 1.1 or newer. Though no application changes are -required for the Ruby driver to make use of the newest protocols, some operating systems or versions may not provide -an OpenSSL version new enough to support them. +Industry best practices, and some regulations, require the use of TLS 1.1 +or newer. Though no application changes are required for the Ruby driver to +make use of the newest protocols, some operating systems or versions may +not provide an OpenSSL version new enough to support them. -Users of macOS older than 10.13 (High Sierra) will need to install Ruby from `rvm`_, `homebrew`_, `macports`_, or -another similar source. See `installation information on ruby-lang.org`_ for more options. +Users of macOS older than 10.13 (High Sierra) will need to install Ruby from +`rvm`_, `homebrew`_, `macports`_, or another similar source. See +`installation information on ruby-lang.org`_ for more options. Users of Linux or other non-macOS Unix can check their OpenSSL version like this: @@ -46,8 +48,9 @@ Users of Linux or other non-macOS Unix can check their OpenSSL version like this $ openssl version -If the version number is less than 1.0.1 support for TLS 1.1 or newer is not available. Contact your operating system -vendor for a solution or upgrade to a newer distribution. +If the version number is less than 1.0.1 support for TLS 1.1 or newer is +not available. Contact your operating system vendor for a solution or upgrade +to a newer distribution. You can check your Ruby interpreter by executing the following command: diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 7ae0ec4a3..a1379c8f2 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -462,7 +462,8 @@ Atlas Compatibility `Driver version 2.6.1 `_ or higher is recommended when using MongoDB Atlas, as this version has -significant SSL performance improvements. +significant performance improvements when TLS connections are used, and all +Atlas connections use TLS. When running on JRuby and connecting to Atlas Free Tier, `driver version 2.6.4 `_ @@ -487,8 +488,8 @@ the driver. - |checkmark| -JRuby Kerberos -============== +JRuby and Kerberos Authentication +================================= If the ``mongo_kerberos`` gem is used for Kerberos authentication with JRuby, the the JVM system property "sun.security.jgss.native" to will be set to "true" in order to facilitate the use of @@ -498,3 +499,12 @@ obtaining Kerberos credentials as well. .. include:: /includes/unicode-checkmark.rst .. include:: /includes/unicode-nbsp.rst + + +JRuby and TLS Connections +========================= + +Due to JRuby limitations: + +- ECDSA server certificates are not supported. +- OCSP endpoint checking is not performed. diff --git a/source/tutorials/ruby-driver-authentication.txt b/source/tutorials/ruby-driver-authentication.txt index f87552218..c45a2e00f 100644 --- a/source/tutorials/ruby-driver-authentication.txt +++ b/source/tutorials/ruby-driver-authentication.txt @@ -191,14 +191,14 @@ value for the ``authMechanism`` URI option, as follows: Client Certificate (X.509) `````````````````````````` -The driver presents an X.509 certificate during SSL negotiation. +The driver presents an X.509 certificate during TLS negotiation. The MONGODB-X509 authentication mechanism authenticates a username derived from the distinguished subject name of this certificate. -This authentication method requires the use of SSL connections with +This authentication method requires the use of TLS connections with certificate validation. -To authenticate the client, you will need a valid SSL certificate +To authenticate the client, you will need a valid TLS certificate and private encryption key. These can be stored in separate files, or together in one file (in the PEM format). Even if the certificate and private key are stored in the same file, you must specify the path to @@ -393,7 +393,7 @@ Access Protocol `LDAP `_ server. .. warning:: When using LDAP, passwords are sent to the server in plain text. For this - reason, we strongly recommend enabling SSL when using LDAP as your + reason, we strongly recommend enabling TLS when using LDAP as your authentication mechanism. For more information about configuring LDAP authentication in diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 1d8b59963..cabc0a4ac 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -496,27 +496,30 @@ Ruby Options - none * - ``:ssl`` - - Tell the client to connect to the servers via SSL. + - Tell the client to connect to the servers via TLS. - ``Boolean`` - false * - ``:ssl_ca_cert`` - - The file path containing concatenated certificate authority certificates used to validate certs - passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object + - The file path containing concatenated certificate authority certificates + used to validate certs passed from the other end of the connection. + One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. - ``String`` - none * - ``:ssl_ca_cert_object`` - - An array of OpenSSL::X509::Certificate representing the certificate authority certificates used to - validate certs passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or + - An array of OpenSSL::X509::Certificate representing the certificate + authority certificates used to validate certs passed from the other end + of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. - ``Array< OpenSSL::X509::Certificate >`` - none * - ``:ssl_ca_cert_string`` - - A string containing concatenated certificate authority certificates used to validate certs - passed from the other end of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object + - A string containing concatenated certificate authority certificates + used to validate certs passed from the other end of the connection. + One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. - ``String`` - none @@ -578,8 +581,9 @@ Ruby Options - none * - ``:ssl_key_string`` - - A string containing the PEM-encoded private key used to identify the connection against MongoDB. - This parameter, if present, takes precedence over the value of option :ssl_key_object. + - A string containing the PEM-encoded private key used to identify the + connection against MongoDB. This parameter, if present, takes precedence + over the value of option :ssl_key_object. - ``String`` - none @@ -594,14 +598,16 @@ Ruby Options - true * - ``:ssl_verify_certificate`` - - Whether to perform peer certificate validation. This setting overrides :ssl_verify with - respect to whether certificate validation is performed. + - Whether to perform peer certificate validation. This setting overrides + the ``:ssl_verify`` setting with respect to whether certificate + validation is performed. - ``Boolean`` - true * - ``:ssl_verify_hostname`` - Whether to perform peer hostname validation. This setting overrides - :ssl_verify with respect to whether hostname validation is performed. + the ``:ssl_verify`` setting with respect to whether hostname validation + is performed. - ``Boolean`` - true @@ -811,18 +817,18 @@ URI options are explained in detail in the :manual:`Connection URI reference * - tlsAllowInvalidCertificates=Boolean - ``:ssl_verify_certificate => boolean`` - Because ``tlsAllowInvalidCertificates`` uses ``true`` to signify that verification - should be disabled and ``ssl_verify_certificate`` uses ``false`` to signify that - verification should be disabled, the boolean is inverted before being used to set - ``ssl_verify_certificate``. + Because ``tlsAllowInvalidCertificates`` uses ``true`` to signify that + verification should be disabled and ``ssl_verify_certificate`` uses + ``false`` to signify that verification should be disabled, the boolean + is inverted before being used to set ``ssl_verify_certificate``. * - tlsAllowInvalidHostnames=Boolean - ``:ssl_verify_hostname => boolean`` - Because ``tlsAllowInvalidHostnames`` uses ``true`` to signify that verification - should be disabled and ``ssl_verify_hostname`` uses ``false`` to signify that - verification should be disabled, the boolean is inverted before being used to set - ``ssl_verify_hostname``. + Because ``tlsAllowInvalidHostnames`` uses ``true`` to signify that + verification should be disabled and ``ssl_verify_hostname`` uses + ``false`` to signify that verification should be disabled, the boolean + is inverted before being used to set ``ssl_verify_hostname``. * - tlsCAFile=String - ``:ssl_ca_cert => String`` @@ -847,9 +853,10 @@ URI options are explained in detail in the :manual:`Connection URI reference * - tlsInsecure=Boolean - ``:ssl_verify => boolean`` - Because tlsInsecure uses ``true`` to signify that verification should be disabled and - ``ssl_verify`` uses ``false`` to signify that verification should be disabled, the boolean - is inverted before being used to set ``ssl_verify``. + Because tlsInsecure uses ``true`` to signify that verification should + be disabled and ``ssl_verify`` uses ``false`` to signify that + verification should be disabled, the boolean is inverted before being + used to set ``ssl_verify``. * - w=Integer|String - ``{ :write_concern => { :w => Integer|String }}`` @@ -987,6 +994,20 @@ To connect to the MongoDB deployment using TLS: When using JRuby, ECDSA certificates are not currently supported. +TLS vs SSL Option Names +----------------------- + +All MongoDB server versions supported by the Ruby driver (2.6 and higher) +only implement TLS. 2.6 and higher servers do not use SSL. + +For historical reasons, the Ruby option names pertaining to TLS configuration +use the ``ssl`` rather than the ``tls`` prefix. The next major version of +the Ruby driver (3.0) will use the ``tls`` prefix for Ruby option names. + +The URI option names use the ``tls`` prefix, with one exception: there is +a ``ssl`` URI option that is deprecated and equivalent to the ``tls`` URI +option. + Enable TLS Connections ---------------------- From 81ebaff0936f5b6e0de7a6a85c07fd09114f4b15 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Wed, 23 Sep 2020 18:09:12 -0400 Subject: [PATCH 309/442] RUBY-2376 Query cache documentation (#2101) --- source/index.txt | 1 + source/ruby-driver-tutorials.txt | 2 +- source/tutorials/query-cache.txt | 213 ++++++++++++++++++ .../tutorials/ruby-driver-crud-operations.txt | 51 +---- 4 files changed, 225 insertions(+), 42 deletions(-) create mode 100644 source/tutorials/query-cache.txt diff --git a/source/index.txt b/source/index.txt index efd35b5f9..f3dbe2da4 100644 --- a/source/index.txt +++ b/source/index.txt @@ -78,6 +78,7 @@ For tutorials on Mongoid, see the `Mongoid Manual /reference/driver-compatibility diff --git a/source/ruby-driver-tutorials.txt b/source/ruby-driver-tutorials.txt index 5145573ee..8acf73b19 100644 --- a/source/ruby-driver-tutorials.txt +++ b/source/ruby-driver-tutorials.txt @@ -11,7 +11,7 @@ Tutorials The tutorials in this section provide examples of some frequently used operations. This section is not meant to be an exhaustive list of all operations available in the Ruby driver. - + .. toctree:: :titlesonly: diff --git a/source/tutorials/query-cache.txt b/source/tutorials/query-cache.txt new file mode 100644 index 000000000..6e152147e --- /dev/null +++ b/source/tutorials/query-cache.txt @@ -0,0 +1,213 @@ +*********** +Query Cache +*********** + +.. default-domain:: mongodb + +.. contents:: On this page + :local: + :backlinks: none + :depth: 2 + :class: singlecol + +.. _query-cache: + +The MongoDB Ruby driver provides a built-in query cache. When enabled, the +query cache saves the results of previously-executed find and aggregation +queries. When those same queries are performed again, the driver returns +the cached reuslts to prevent unnecessary roundtrips to the database. + +Usage +===== + +The query cache is disabled by default. It cache can be enabled on the global +scope as well as within the context of a specific block. + +To enable the query cache globally: + +.. code-block:: ruby + + Mongo::QueryCache.enabled = true + +Similarly, to disable it globally: + +.. code-block:: ruby + + Mongo::QueryCache.enabled = false + +To enable the query cache within the context of a block: + +.. code-block:: ruby + + Mongo::QueryCache.cache do + Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music') do |client| + client['artists'].find(name: 'Flying Lotus').first + #=> Queries the database and caches the result + + client['artists'].find(name: 'Flying Lotus').first + #=> Returns the previously cached result + end + end + +And to disable the query cache in the context of a block: + +.. code-block:: ruby + + Mongo::QueryCache.uncached do + Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music') do |client| + client['artists'].find(name: 'Flying Lotus').first + #=> Sends the query to the database; does NOT cache the result + + client['artists'].find(name: 'Flying Lotus').first + #=> Queries the database again + end + end + +You may check whether the query cache is enabled at any time by calling +``Mongo::QueryCache.enabled?``, which will return ``true`` or ``false``. + +.. _query-cache-matching: + +Query Matching +============== + +A query is eligible to use cached results if it matches the original query +that produced the cached results. Two queries are considered matching if they +are identical in the following values: + +* Namespace (the database and collection on which the query was performed) +* Selector (for aggregations, the aggregation pipeline stages) +* Skip +* Sort +* Projection +* Collation +* Read Concern +* Read Preference + +For example, if you perform one query, and then perform a mostly identical query +with a different sort order, those queries will not be considered matching, +and the second query will not use the cached results of the first. + +Limits +====== + +When performing a query with a limit, the query cache will reuse an existing +cached query with a larger limit if one exists. For example: + +.. code-block:: ruby + + Mongo::QueryCache.cache do + Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music') do |client| + client['artists'].find(genre: 'Rock', limit: 10) + #=> Queries the database and caches the result + + client['artists'].find(genre: 'Rock', limit: 5) + #=> Returns the first 5 results from the cached query + + client['artists'].find(genre: 'Rock', limit: 20) + #=> Queries the database again and replaces the previously cached query results + end + end + +Cache Invalidation +================== + +The query cache is cleared in part or in full on every write operation. Most +write operations will clear the results of any queries were performed on the same +collection that is being written to. Some operations will clear the entire +query cache. + +The following operations will clear cached query results on the same database and +collection (including during bulk writes): + +* ``insert_one`` +* ``update_one`` +* ``replace_one`` +* ``update_many`` +* ``delete_one`` +* ``delete_many`` +* ``find_one_and_delete`` +* ``find_one_and_update`` +* ``find_one_and_replace`` + +The following operations will clear the entire query cache: + +* aggregation with ``$merge`` or ``$out`` pipeline stages +* ``commit_transaction`` +* ``abort_transaction`` + +Manual Cache Invalidation +========================= + +You may clear the query cache at any time with the following method: + +.. code-block:: ruby + + Mongo::QueryCache.clear + +This will remove all cached query results. + +Transactions +============ + +Queries are cached within the context of a transaction, but the entire +cache will be cleared when the transaction is committed or aborted. + +.. code-block:: ruby + + Mongo::QueryCache.cache do + Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music') do |client| + session = client.start_session + + session.with_transaction do + client['artists'].insert_one({ name: 'Fleet Foxes' }, session: session) + + client['artists'].find({}, session: session).first + #=> { name: 'Fleet Foxes' } + #=> Queries the database and caches the result + + client['artists'].find({}, session: session).first + #=> { name: 'Fleet Foxes' } + #=> Returns the previously cached result + + session.abort_transaction + end + + client['artists'].find.first + #=> nil + # The query cache was cleared on abort_transaction + end + end + +.. note:: + + Transactions are often performed with a "snapshot" read concern level. Keep + in mind that a query with a "snapshot" read concern cannot return cached + results from a query without the "snapshot" read concern, so it is possible + that a transaction may not use previously cached queries. + + To understand when a query will use a cached result, see the + :ref:`Query Matching ` section. + +Aggregations +============ + +The query cache also caches the results of aggregation pipelines. For example: + +.. code-block:: ruby + + Mongo::QueryCache.cache do + Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music') do |client| + client['artists'].aggregate([ { '$match' => { name: 'Fleet Foxes' } } ]).first + #=> Queries the database and caches the result + + client['artists'].aggregate([ { '$match' => { name: 'Fleet Foxes' } } ]).first + #=> Returns the previously cached result + end + end + +.. note:: + + Aggregation results are cleared from the cache during every write operation, + with no exceptions. + diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index d4e58d8a4..55e630199 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -130,46 +130,15 @@ object, or with ``Decimal128.from_string()``. price = BSON::Decimal128.from_string("428.79") # => BSON::Decimal128('428.79') - -.. _query-cache: - Query Cache =========== -If the Ruby driver's query cache is enabled, it will cache find queries on -the currently executed thread and avoid sending requests to the database for -identical queries. The QueryCache class stores CachingCursor objects which attempt -to load documents from memory before retrieving them from the database. Performing -any write operations such as insert, update, or delete clears the query cache. Note that -if the number of results is too large to be returned in a single batch, -the query cache will not be used, even if ``Mongo::QueryCache.enabled`` is true, and -an error will be returned. - -Similar to Mongoid's query cache, the driver's query cache implementation -does not support using CachingCursors for queries with different limit sizes if -the original query has a specified limit. For example, if a query with a limit of -100 were executed, followed by a query with a limit of 10, the second query -would still run against the database and a new CachingCursor object would be stored, -rather than using the existing one. However, if a query does not specify a limit -initially, then any query that is run after it with a specified limit will use -the original CachingCursor rather than going back to the database, and will return -the correct number of documents accordingly. - -To enable the query cache on a global scope: - -.. code-block:: ruby - - Mongo::QueryCache.enabled = true - -The following code shows how to enable the query cache within the context of a -specific block: - -.. code-block:: ruby - - Mongo::QueryCache.cache do - # all queries within this block are cached - end +The Ruby driver provides a query cache. When enabled, the query cache will +save the results of find and aggregation queries and return those saved results +when the same queries are performed again. +To read more about the query cache, visit the +:ref:`query cache tutorial `. Reading ======= @@ -293,7 +262,7 @@ Additional Query Operations ``estimated_document_count`` Get an approximate number of documents in the collection. - + Note that unlike ``count_documents``, ``estimated_document_count`` does not accept a filter. @@ -306,7 +275,7 @@ Additional Query Operations ``count`` Get an approximate number of documents matching a filter, or an approximate number of documents in the collection. - + *Deprecated:* The ``count`` method is deprecated and does not work in transactions. Please use ``count_documents`` to obtain an exact count of documents potentially matching a filter or ``estimated_document_count`` @@ -366,11 +335,11 @@ or on the collection: client = Mongo::Client.new(['localhost:14420'], database: 'music', read_concern: {level: :local}) - + client['collection'].find.to_a collection = client['collection', read_concern: {level: :majority}] - + collection.find.to_a The driver does not currently support setting read concern on an individual @@ -387,7 +356,7 @@ part of the command: .. code-block:: ruby client.database.command(collstats: 'test', readConcern: {level: :majority}) - + .. _ruby-driver-read-preference: From 73344845861bb4b726e555b0c5faebc7a9e4a7f0 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Thu, 1 Oct 2020 10:19:35 -0400 Subject: [PATCH 310/442] RUBY-2413 Add a test and documentation for system collections with query cache (#2105) --- source/tutorials/query-cache.txt | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/source/tutorials/query-cache.txt b/source/tutorials/query-cache.txt index 6e152147e..cd0ea8eb5 100644 --- a/source/tutorials/query-cache.txt +++ b/source/tutorials/query-cache.txt @@ -211,3 +211,24 @@ The query cache also caches the results of aggregation pipelines. For example: Aggregation results are cleared from the cache during every write operation, with no exceptions. +System Collections +================== + +MongoDB stores system information in collections that use the ``database.system.*`` +namespace pattern. + +When the query cache is enabled, it will cache all system collection results. +However, system collections are not subject to the same cache invalidation rules +as normal collections; if your app relies on up-to-date information from +system collections, please ensure that you are performing queries against the +system collections with the query cache disabled, as demonstrated below. + +.. code-block:: ruby + + Mongo::QueryCache.uncached do + Mongo::Client.new([ '127.0.0.1:27017' ], database: 'admin') do |client| + users = client['system.users'].find.to_a + #=> Does not return cached results + end + end + From d5dfbc25bb2db5e59788fe39a48deba1416b12c5 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Fri, 2 Oct 2020 14:58:29 -0400 Subject: [PATCH 311/442] RUBY-2413 Do not cache query results from system collections (#2106) --- source/tutorials/query-cache.txt | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/source/tutorials/query-cache.txt b/source/tutorials/query-cache.txt index cd0ea8eb5..0ab119106 100644 --- a/source/tutorials/query-cache.txt +++ b/source/tutorials/query-cache.txt @@ -215,20 +215,19 @@ System Collections ================== MongoDB stores system information in collections that use the ``database.system.*`` -namespace pattern. +namespace pattern. These are called system collections. -When the query cache is enabled, it will cache all system collection results. -However, system collections are not subject to the same cache invalidation rules -as normal collections; if your app relies on up-to-date information from -system collections, please ensure that you are performing queries against the -system collections with the query cache disabled, as demonstrated below. +Data in system collections can change due to activity not triggered by the +application (such as internal server processes) and as a result of a variety of +database commands issued by the application. Because of the difficulty of +determining when the cached results for system collections should be expired, +queries on system collections bypass the query cache. -.. code-block:: ruby +You may read more about system collections in the +:manual:`MongoDB documentation `. - Mongo::QueryCache.uncached do - Mongo::Client.new([ '127.0.0.1:27017' ], database: 'admin') do |client| - users = client['system.users'].find.to_a - #=> Does not return cached results - end - end +.. note :: + + Even when the query cache is enabled, query results from system collections + will not be cached. From daae440535c5daf7ff671b8ba8e92a25809fa98b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Sat, 26 Sep 2020 12:52:28 -0400 Subject: [PATCH 312/442] Link to local api docs --- source/index.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/index.txt b/source/index.txt index f3dbe2da4..2c71be98b 100644 --- a/source/index.txt +++ b/source/index.txt @@ -80,7 +80,7 @@ For tutorials on Mongoid, see the `Mongoid Manual + API /reference/driver-compatibility /reference/additional-resources contribute From de3fdb9dd0131ade8106c61aac8469579753f2d5 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Fri, 2 Oct 2020 19:55:43 -0400 Subject: [PATCH 313/442] Fix heading symbols --- .../tutorials/ruby-driver-create-client.txt | 43 ++++++++++--------- 1 file changed, 23 insertions(+), 20 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index cabc0a4ac..7ef1ef8d2 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -49,7 +49,7 @@ The driver will discover all nodes in the cluster and connect to them as needed. Block Syntax -```````````` +------------ Another way to create a Mongo::Client object is to use the block syntax: @@ -61,8 +61,8 @@ Another way to create a Mongo::Client object is to use the block syntax: Note that when creating a client using this syntax, the client is automatically closed after the block finishes executing. -Database -```````` +Database Selection +================== By default, the client will connect to the ``admin`` database. @@ -108,8 +108,11 @@ their stated purposes: mechanisms. +Connection Types +================ + Direct Connection -````````````````` +----------------- If the deployment is a replica set, the driver will by default discover all members of the replica set given the address of any one member, and will @@ -130,7 +133,7 @@ To force all operations to be performed on the designated server, specify the .. _ruby-driver-connect-replica-set: Replica Set Connection -`````````````````````` +---------------------- To connect to a :manual:`replica set` deployment managed by a service providing SRV URIs (such as MongoDB Atlas), connect to the URI: @@ -158,7 +161,7 @@ connect to the replica set. .. _ruby-driver-connect-sharded-cluster: Sharded Cluster Connection -`````````````````````````` +-------------------------- To connect to a :manual:`sharded cluster` deployment managed by a service providing SRV URIs (such as MongoDB Atlas), connect to the URI: @@ -188,7 +191,7 @@ spread the operation load accordingly. SRV URIs -```````` +======== When the driver connects to a :manual:`mongodb+srv protocol ` @@ -225,7 +228,7 @@ the analogous URI option are provided, the Ruby option takes precedence. Ruby Options -```````````` +------------ .. note:: @@ -673,7 +676,7 @@ Ruby Options URI Options -``````````` +----------- Since the URI options are required to be in camel case, which is not the Ruby standard, the following table shows URI options and their corresponding Ruby @@ -883,7 +886,7 @@ Timeout Options =============== ``server_selection_timeout`` -```````````````````````````` +---------------------------- When executing an operation, the number of seconds to wait for the driver to find an appropriate server to send an operation to. Defaults to 30. @@ -909,7 +912,7 @@ timeout in the routing layer). In development this value can be lowered to provide quicker failure when the server is not running. ``socket_timeout`` -`````````````````` +------------------ The number of seconds to wait for a socket read or write to complete on regular (non-monitoring) connections. Default is no timeout. @@ -939,7 +942,7 @@ operations, as this will abort their execution on the server. This option does not apply to monitoring connections. ``connect_timeout`` -``````````````````` +------------------- The number of seconds to wait for a socket connection to be established to a server. Defaults to 10. @@ -963,7 +966,7 @@ connections prior to returning them, which may require several network round trips. ``max_time_ms`` -``````````````` +--------------- Specified as an option on a particular operation, the number of milliseconds to allow the operation to execute for on the server. Not set by default. @@ -973,7 +976,7 @@ long running operations to be interrupted on the server when they take too long. ``wtimeout`` -```````````` +------------ The number of milliseconds to wait for a write to be acknowledged by the number of servers specified in the write concern. Not set by default, which @@ -1201,8 +1204,8 @@ Please see the `MongoDB Diagnostics FAQ keepalive section ]>>" -Details on Retryable Reads -========================== +Retryable Reads +=============== The driver implements two mechanisms for retrying reads: modern and legacy. As of driver version 2.9.0, the modern mechanism is used by default, and the @@ -1490,8 +1493,8 @@ To disable all read retries, set the following client options: ``retry_reads: false, max_read_retries: 0``. -Details on Retryable Writes -=========================== +Retryable Writes +================ The driver implements two mechanisms for retrying writes: modern and legacy. As of driver version 2.9.0, the modern mechanism is used by default on servers From a510a4a65bd93c97f755d5e28457b547ad102f68 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Sat, 3 Oct 2020 00:52:00 -0400 Subject: [PATCH 314/442] Bump version to 2.14.0.beta for query cache testing in Mongoid --- source/installation.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/installation.txt b/source/installation.txt index 632d24fef..00cbf8e26 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -19,7 +19,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.13.0.rc1 + gem install mongo -v 2.14.0.beta What's New From 07d225a523b622711083939f8aa0cd62c9c20782 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 5 Oct 2020 10:52:53 -0400 Subject: [PATCH 315/442] Change headings to be consistent with driver docs (#220) Co-authored-by: Oleg Pudeyev --- docs/tutorials/bson-v4.txt | 58 +++++++++++++++++++------------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 6d47fc74e..06f3c6c0a 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -2,9 +2,9 @@ .. _ruby-bson-tutorial-4-0: -================= +***************** BSON 4.x Tutorial -================= +***************** .. default-domain:: mongodb @@ -17,7 +17,7 @@ BSON 4.x Tutorial This tutorial discusses using the Ruby BSON library. Installation ------------- +============ The BSON library can be installed from `Rubygems `_ manually or with bundler. @@ -37,7 +37,7 @@ To install the gem with bundler, include the following in your ``Gemfile``: The BSON library is compatible with MRI >= 2.3 and JRuby >= 9.2. Use With ActiveSupport ----------------------- +====================== Serialization for ActiveSupport-defined classes, such as TimeWithZone, is not loaded by default to avoid a hard dependency of BSON on ActiveSupport. @@ -50,7 +50,7 @@ ActiveSupport-related code must be explicitly required: require 'bson/active_support' BSON Serialization ------------------- +================== Getting a Ruby object's raw BSON representation is done by calling ``to_bson`` on the Ruby object, which will return a ``BSON::ByteBuffer``. For example: @@ -70,13 +70,13 @@ you wish to instantiate and passing it a ``BSON::ByteBuffer`` instance. Byte Buffers ------------- +============ BSON library 4.0 introduces the use of native byte buffers in MRI and JRuby instead of using ``StringIO``, for improved performance. Writing -``````` +------- To create a ``ByteBuffer`` for writing (i.e. serializing to BSON), instantiate ``BSON::ByteBuffer`` with no arguments: @@ -184,7 +184,7 @@ over a socket), call ``to_s`` on the buffer: Reading -``````` +------- To create a ``ByteBuffer`` for reading (i.e. deserializing from BSON), instantiate ``BSON::ByteBuffer`` with a byte string as the argument: @@ -218,7 +218,7 @@ To restart reading from the beginning of a buffer, use ``rewind``: Supported Classes ------------------ +================= Core Ruby classes that have representations in the BSON specification and will have a ``to_bson`` method defined for them are: ``Object``, ``Array``, @@ -230,7 +230,7 @@ specific to the specification: ``BSON::Binary`` -```````````````` +---------------- Use ``BSON::Binary`` objects to store arbitrary binary data. The ``Binary`` objects can be constructed from binary strings as follows: @@ -281,7 +281,7 @@ The data and the subtype can be retrieved from ``Binary`` instances using # => # UUID Methods -~~~~~~~~~~~~ +```````````` To create a UUID BSON::Binary (binary subtype 4) from its RFC 4122-compliant string representation, use the ``from_uuid`` method: @@ -314,7 +314,7 @@ Note that the ``:standard`` representation can only be used with a Binary of subtype ``:uuid`` (not ``:uuid_old``). Legacy UUIDs -~~~~~~~~~~~~ +```````````` Data stored in BSON::Binary objects of subtype 3 (``:uuid_old``) may be persisted in one of three different byte orders depending on which driver @@ -381,7 +381,7 @@ These methods can be used to convert from one representation to another: ``BSON::Code`` -`````````````` +-------------- Represents a string of JavaScript code. @@ -390,7 +390,7 @@ Represents a string of JavaScript code. BSON::Code.new("this.value = 5;") ``BSON::CodeWithScope`` -``````````````````````` +----------------------- .. note:: @@ -406,7 +406,7 @@ Represents a string of JavaScript code with a hash of values. BSON::CodeWithScope.new("this.value = age;", age: 5) ``BSON::Document`` -`````````````````` +------------------ This is a subclass of ``Hash`` that stores all keys as strings, but allows access to them with symbol keys. @@ -431,7 +431,7 @@ access to them with symbol keys. ``BSON::MaxKey`` -```````````````` +---------------- Represents a value in BSON that will always compare higher to another value. @@ -440,7 +440,7 @@ Represents a value in BSON that will always compare higher to another value. BSON::MaxKey.new ``BSON::MinKey`` -```````````````` +---------------- Represents a value in BSON that will always compare lower to another value. @@ -449,7 +449,7 @@ Represents a value in BSON that will always compare lower to another value. BSON::MinKey.new ``BSON::ObjectId`` -`````````````````` +------------------ Represents a 12 byte unique identifier for an object on a given machine. @@ -458,7 +458,7 @@ Represents a 12 byte unique identifier for an object on a given machine. BSON::ObjectId.new ``BSON::Timestamp`` -``````````````````` +------------------- Represents a special time with a start and increment value. @@ -467,7 +467,7 @@ Represents a special time with a start and increment value. BSON::Timestamp.new(5, 30) ``BSON::Undefined`` -``````````````````` +------------------- Represents a placeholder for a value that was not provided. @@ -476,7 +476,7 @@ Represents a placeholder for a value that was not provided. BSON::Undefined.new ``BSON::Decimal128`` -```````````````````` +-------------------- Represents a 128-bit decimal-based floating-point value capable of emulating decimal rounding with exact precision. @@ -491,7 +491,7 @@ decimal rounding with exact precision. BSON::Decimal128.new(d) ``Symbol`` -`````````` +---------- The BSON specification defines a symbol type which allows round-tripping Ruby ``Symbol`` values (i.e., a Ruby ``Symbol``is encoded into a BSON symbol @@ -538,7 +538,7 @@ To force encoding of Ruby symbols to BSON symbols, wrap the Ruby symbols in # => "\x12\x00\x00\x00\x0Efoo\x00\x04\x00\x00\x00bar\x00\x00" JSON Serialization ------------------- +================== Some BSON types have special representations in JSON. These are as follows and will be automatically serialized in the form when calling ``to_json`` on @@ -577,7 +577,7 @@ them. Time Instances --------------- +============== Times in Ruby can have nanosecond precision. Times in BSON (and MongoDB) can only have millisecond precision. When Ruby ``Time`` instances are @@ -610,7 +610,7 @@ calculations may produce unexpected results. DateTime Instances ------------------- +================== BSON only supports storing the time as the number of seconds since the Unix epoch. Ruby's ``DateTime`` instances can be serialized to BSON, @@ -624,7 +624,7 @@ database. Date Instances --------------- +============== BSON only supports storing the time as the number of seconds since the Unix epoch. Ruby's ``Date`` instances can be serialized to BSON, but when @@ -635,7 +635,7 @@ of the day that the ``Date`` refers to in UTC. Regular Expressions -------------------- +=================== Ruby regular expressions always have BSON regular expressions' equivalent of 'm' flag on. In order for behavior to be preserved between the two, the 'm' @@ -683,7 +683,7 @@ object, one must call ``compile`` on the returned object. Key Order ---------- +========= BSON documents preserve the order of keys, because the documents are stored as lists of key-value pairs. Hashes in Ruby also preserve key order; thus @@ -693,7 +693,7 @@ the order of keys in the document will match the order of keys in the hash. Duplicate Keys --------------- +============== BSON specification allows BSON documents to have duplicate keys, because the documents are stored as lists of key-value pairs. Applications should refrain From 564d37338782b1e4e82ad416993c0109d747cf4c Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 6 Oct 2020 13:00:11 -0400 Subject: [PATCH 316/442] RUBY-2346 server selection test with mixed case tag sets (#2107) * add tags to data bearing nodes in replica sets * report requested tag sets in NoServerAvailable * server selection test with mixed case tag sets * fix tests and update documentation to match behavior * bundle exec * exclude 2.6 server Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-create-client.txt | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 7ef1ef8d2..184260191 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -430,12 +430,22 @@ Ruby Options .. code-block:: ruby - { :read => - { :mode => :secondary, - :tag_sets => [ "berlin" ], - :max_staleness => 5, + { read: + { mode: :secondary, + tag_sets: [ "data_center" => "berlin" ], + max_staleness: 5, } } + + If tag sets are provided, they must be an array of hashes. A server + satisfies the read preference if its tags match any one hash in the + provided tag sets. + + Each tag set must be a hash, and will be converted internally to + a ``BSON::Document`` instance prior to being used for server selection. + Hash keys can be strings or symbols. The keys are case sensitive. + Hash values must be strings, and are matched exactly against the values + in the replica set configuration. - ``Hash`` - ``{ :mode => :primary }`` From 16e8f19d12f4d93f362f08b92d34071528544f9c Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 7 Oct 2020 11:13:50 -0400 Subject: [PATCH 317/442] RUBY-2020 Set explain verbosity from Mongo::Collection::View::Explainable (#2108) * RUBY-2020 Set explain verbosity from Mongo::Collection::View::Explainable * typo fix Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-crud-operations.txt | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 55e630199..4e6fc1e0b 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -209,6 +209,39 @@ when querying and their corresponding methods as examples. each ``GETMORE`` operation. * - ``comment(String)`` - Adds a comment to the query. + + * - ``explain(**opts)`` + - Returns the query plan for the query. Pass the :manual:`explain options + ` via the keyword arguments using symbol + keys. + + .. code-block:: ruby + + # All server versions - default explain behavior + client[:artists].find.explain + + # MongoDB 3.0 and newer + client[:artists].find.explain(verbosity: :query_planner) + client[:artists].find.explain(verbosity: :execution_stats) + client[:artists].find.explain(verbosity: :all_plans_execution) + + # Alternative syntax using camel case + client[:artists].find.explain(verbosity: "queryPlanner") + client[:artists].find.explain(verbosity: "executionStats") + client[:artists].find.explain(verbosity: "allPlansExecution") + + # MongoDB 2.6 + client[:artists].find.explain(verbose: true) + + .. note:: + + The information returned by the server for the ``explain`` command + varies with server version and deployment topology. The driver's + ``explain`` method returns whatever the server provided. + + **The return value of ``explain`` method is not part of the driver's + public API and depends on the server version and deployment topology.** + * - ``hint(Hash)`` - Provides the query with an :manual:`index hint` to use. From 64c02d08173091e4fc47779fb0c771b51557d6b5 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 7 Oct 2020 18:19:14 -0400 Subject: [PATCH 318/442] Fix documentation issues called out by sphinx --- .../tutorials/ruby-driver-create-client.txt | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 184260191..260d123b0 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -960,7 +960,7 @@ a server. Defaults to 10. This timeout is also used as both connect timeout and socket timeout for monitoring connections. -When using a ``mongodb+srv://` URI, this timeout is also used for SRV and TXT +When using a ``mongodb+srv://`` URI, this timeout is also used for SRV and TXT DNS lookups. Note that the timeout applies per lookup; due to DNS suffix search lists, multiple lookups may be performed as part of a single name resolution. @@ -1113,14 +1113,15 @@ that both of the following fields are set in the intermediate certificate(s): - X509v3 Basic Constraints: CA: TRUE -- Can sign certificates - X509v3 Key Usage: Key Cert Sign -- Can sign certificates -More information about these flags can be found `here +More information about these flags can be found `in this Stack Overflow question `_. It is a common pitfall to concatenate intermediate certificates to the root CA certificates passed in ``tlsCAFile`` / ``ssl_ca_cert`` options. By doing so, the intermediate certificates are elevated to trusted status and are themselves not verified against the actual CA root. More information on this -issue is available `here `_. +issue is available `in this mailing list post +`_. Specify CA Certificate ---------------------- @@ -1348,7 +1349,7 @@ and cease background monitoring: parent remains alive. Reconnecting Client Instances -````````````````````````````` +----------------------------- When the Ruby driver is used in a web application, it is recommended to not create any ``Mongo::Client`` instances in the management processes (prior to @@ -1398,7 +1399,7 @@ however provides examples for closing clients in the parent process and reconnecting clients in the child processes. Troubleshooting -``````````````` +--------------- The client's ``summary`` method returns the current state of the client, including servers that the client is monitoring and their state. If any of @@ -1439,7 +1440,7 @@ As of driver version 2.9.0, the modern mechanism is used by default, and the legacy mechanism is deprecated. Modern Retryable Reads -`````````````````````` +---------------------- When the modern mechanism is used, read operations are retried once in the event of a network error, a "not master" error, or a "node is recovering" error. @@ -1478,7 +1479,7 @@ must explicitly request legacy retryable reads by setting the ``retry_reads: false`` client option or using ``retryReads=false`` URI option. Legacy Retryable Reads -`````````````````````` +---------------------- The legacy read retry behavior of the Ruby driver is available by setting the ``retry_reads: false`` client option or passing the ``retryReads=false`` URI @@ -1497,7 +1498,7 @@ versions read retries would be sent to the same server which the initial read was sent to. Disabling Retryable Reads -````````````````````````` +------------------------- To disable all read retries, set the following client options: ``retry_reads: false, max_read_retries: 0``. @@ -1524,7 +1525,7 @@ are subject to write retries: - ``collection#bulk_write`` (for all single statement ops, i.e. not for ``update_many`` or ``delete_many``) Modern Retryable Writes -``````````````````````` +----------------------- The modern mechanism will retry failing writes once when the driver is connected to a MongoDB 3.6 or higher replica set or a sharded cluster, @@ -1543,7 +1544,7 @@ since the server that the original write was sent to is likely no longer usable. Legacy Retryable Writes -``````````````````````` +----------------------- If modern retryable writes mechanism is disabled by setting the client option ``retry_writes: false`` or by using the ``retryWrites=false`` @@ -1559,7 +1560,7 @@ of errors compared to the modern mechanism, and specifically does not retry writes when a network timeout is encountered. Disabling Retryable Writes -`````````````````````````` +-------------------------- To disable all write retries, set the following client options: ``retry_writes: false, max_write_retries: 0``. @@ -1577,7 +1578,7 @@ See the `Ruby Logger documentation 'test', :logger => my_logger ) Truncation -`````````` +---------- The default logging truncates logs at 250 characters by default. To turn this off pass an option to the client instance. From 86690cab4b6f54b87c16fb00ba425f5a4aee93ee Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 7 Oct 2020 18:22:17 -0400 Subject: [PATCH 319/442] Quote ssl option symbols --- source/tutorials/ruby-driver-create-client.txt | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 260d123b0..a7bdb1694 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -516,24 +516,24 @@ Ruby Options * - ``:ssl_ca_cert`` - The file path containing concatenated certificate authority certificates used to validate certs passed from the other end of the connection. - One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object - (in order of priority) is required for :ssl_verify. + One of ``:ssl_ca_cert``, ``:ssl_ca_cert_string`` or ``:ssl_ca_cert_object`` + (in order of priority) is required for ``:ssl_verify``. - ``String`` - none * - ``:ssl_ca_cert_object`` - An array of OpenSSL::X509::Certificate representing the certificate authority certificates used to validate certs passed from the other end - of the connection. One of :ssl_ca_cert, :ssl_ca_cert_string or - :ssl_ca_cert_object (in order of priority) is required for :ssl_verify. + of the connection. One of ``:ssl_ca_cert``, ``:ssl_ca_cert_string`` or + ``:ssl_ca_cert_object`` (in order of priority) is required for ``:ssl_verify``. - ``Array< OpenSSL::X509::Certificate >`` - none * - ``:ssl_ca_cert_string`` - A string containing concatenated certificate authority certificates used to validate certs passed from the other end of the connection. - One of :ssl_ca_cert, :ssl_ca_cert_string or :ssl_ca_cert_object - (in order of priority) is required for :ssl_verify. + One of ``:ssl_ca_cert``, ``:ssl_ca_cert_string`` or ``:ssl_ca_cert_object`` + (in order of priority) is required for ``:ssl_verify``. - ``String`` - none From 6f31a68d1cbcf235ce088935b1dc7c0d4114301b Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 7 Oct 2020 18:22:24 -0400 Subject: [PATCH 320/442] Repair a missing quote --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index a7bdb1694..f974dfd02 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -1042,7 +1042,7 @@ By default, MongoDB server will attempt to verify the connecting clients' TLS certificates, which requires the clients to specify their TLS certificates when connecting. This can be accomplished via: -- The ``:ssl_cert``/``:ssl_cert_object``/``:ssl_cert_string` and +- The ``:ssl_cert``/``:ssl_cert_object``/``:ssl_cert_string`` and ``:ssl_key``/``:ssl_key_object``/``:ssl_key_string``/``:ssl_key_pass_phrase`` Ruby options. - The``tlsCertificateKeyFile`` URI option. From 83501521fe9cbc5475cfd7d17a98c2eaf52ceb42 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Wed, 7 Oct 2020 18:23:18 -0400 Subject: [PATCH 321/442] Add a missing space to fix formatting --- source/tutorials/ruby-driver-create-client.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index f974dfd02..5dbb0d514 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -1045,7 +1045,7 @@ when connecting. This can be accomplished via: - The ``:ssl_cert``/``:ssl_cert_object``/``:ssl_cert_string`` and ``:ssl_key``/``:ssl_key_object``/``:ssl_key_string``/``:ssl_key_pass_phrase`` Ruby options. -- The``tlsCertificateKeyFile`` URI option. +- The ``tlsCertificateKeyFile`` URI option. When using the Ruby options, the client TLS certificate and the corresponding private key may be provided separately. For example, if the certificate is From 4439b4adb10f4ec26cc634583af60d824a58b9a2 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 9 Oct 2020 14:55:30 -0400 Subject: [PATCH 322/442] RUBY-2236 Explain Ruby vs MongoDB/PCRE regular expressions better (#222) Co-authored-by: Oleg Pudeyev --- docs/tutorials/bson-v4.txt | 194 +++++++++++++++++++++++++++++++++---- 1 file changed, 177 insertions(+), 17 deletions(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index 06f3c6c0a..ed5636d10 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -637,19 +637,175 @@ of the day that the ``Date`` refers to in UTC. Regular Expressions =================== -Ruby regular expressions always have BSON regular expressions' equivalent of -'m' flag on. In order for behavior to be preserved between the two, the 'm' -option is always added when a Ruby regular expression is serialized to BSON. +Both MongoDB and Ruby provide facilities for working with regular expressions, +but they use regular expression engines. The following subsections detail the +differences between Ruby regular expressions and MongoDB regular expressions +and describe how to work with both. + +Ruby vs MongoDB Regular Expressions +----------------------------------- + +MongoDB server uses `Perl-compatible regular expressions implemented using +the PCRE library`_ and `Ruby regular expressions +`_ are implemented using the +`Onigmo regular expression engine `_, +which is a fork of `Oniguruma `_. +The two regular expression implementations generally provide equivalent +functionality but have several important syntax differences, as described +below. + +Unfortunately, there is no simple way to programmatically convert a PCRE +regular expression into the equivalent Ruby regular expression, +and there are currently no Ruby bindings for PCRE. + +Options / Flags / Modifiers +``````````````````````````` + +Both Ruby and PCRE regular expressions support modifiers. These are +also called "options" in Ruby parlance and "flags" in PCRE parlance. +The meaning of ``s`` and ``m`` modifiers differs in Ruby and PCRE: + +- Ruby does not have the ``s`` modifier, instead the Ruby ``m`` modifier + performs the same function as the PCRE ``s`` modifier which is to make the + period (``.``) match any character including newlines. Confusingly, the + Ruby documentation refers to the ``m`` modifier as "enabling multi-line mode". +- Ruby always operates in the equivalent of PCRE's multi-line mode, enabled by + the ``m`` modifier in PCRE regular expressions. In Ruby the ``^`` anchor + always refers to the beginning of line and the ``$`` anchor always refers + to the end of line. + +When writing regular expressions intended to be used in both Ruby and +PCRE environments (including MongoDB server and most other MongoDB drivers), +henceforth referred to as "portable regular expressions", avoid using +the ``^`` and ``$`` anchors. The following sections provide workarounds and +recommendations for authoring portable regular expressions. + +``^`` Anchor +```````````` + +In Ruby regular expressions, the ``^`` anchor always refers to the beginning +of line. In PCRE regular expressions, the ``^`` anchor refers to the beginning +of input by default and the ``m`` flag changes its meaning to the beginning +of line. + +Both Ruby and PCRE regular expressions support the ``\A`` anchor to refer to +the beginning of input, regardless of modifiers. + +When writing portable regular expressions: + +- Use the ``\A`` anchor to refer to the beginning of input. +- Use the ``^`` anchor to refer to the beginning of line (this requires + setting the ``m`` flag in PCRE regular expressions). Alternatively use + one of the following constructs which work regardless of modifiers: + - ``(?:\A|(?<=\n))`` (handles LF and CR+LF line ends) + - ``(?:\A|(?<=[\r\n]))`` (handles CR, LF and CR+LF line ends) + +``$`` Anchor +```````````` + +In Ruby regular expressions, the ``$`` anchor always refers to the end +of line. In PCRE regular expressions, the ``$`` anchor refers to the end +of input by default and the ``m`` flag changes its meaning to the end +of line. + +Both Ruby and PCRE regular expressions support the ``\z`` anchor to refer to +the end of input, regardless of modifiers. + +When writing portable regular expressions: + +- Use the ``\z`` anchor to refer to the end of input. +- Use the ``$`` anchor to refer to the beginning of line (this requires + setting the ``m`` flag in PCRE regular expressions). Alternatively use + one of the following constructs which work regardless of modifiers: + - ``(?:\z|(?=\n))`` (handles LF and CR+LF line ends) + - ``(?:\z|(?=[\n\n]))`` (handles CR, LF and CR+LF line ends) -There is a class provided by the bson gem, ``Regexp::Raw``, to allow Ruby users -to get around this. You can simply create a regular expression like this: +``BSON::Regexp::Raw`` Class +--------------------------- + +Since there is no simple way to programmatically convert a PCRE +regular expression into the equivalent Ruby regular expression, +bson-ruby provides the ``BSON::Regexp::Raw`` class for holding MongoDB/PCRE +regular expressions. Instances of this class are called "BSON regular +expressions" in this documentation. + +Instances of this class can be created using the regular expression text +as a string and optional PCRE modifiers: + +.. code-block:: ruby + + BSON::Regexp::Raw.new("^b403158") + # => # + + BSON::Regexp::Raw.new("^Hello.world$", "s") + # => # + +The ``BSON::Regexp`` module is included in the Ruby ``Regexp`` class, such that +the ``BSON::`` prefix may be omitted: .. code-block:: ruby Regexp::Raw.new("^b403158") + # => # + + Regexp::Raw.new("^Hello.world$", "s") + # => # + +Regular Expression Conversion +----------------------------- -This code example illustrates the difference between serializing a core Ruby -``Regexp`` versus a ``Regexp::Raw`` object: +To convert a Ruby regular expression to a BSON regular expression, +instantiate a ``BSON::Regexp::Raw`` object as follows: + +.. code-block:: ruby + + regexp = /^Hello.world/ + bson_regexp = BSON::Regexp::Raw.new(regexp.source, regexp.options) + # => # + +Note that the ``BSON::Regexp::Raw`` constructor accepts both the Ruby numeric +options and the PCRE modifier strings. + +To convert a BSON regular expression to a Ruby regular expression, call the +``compile`` method on the BSON regular expression: + +.. code-block:: ruby + + bson_regexp = BSON::Regexp::Raw.new("^hello.world", "s") + bson_regexp.compile + # => /^hello.world/m + + bson_regexp = BSON::Regexp::Raw.new("^hello", "") + bson_regexp.compile + # => /^hello.world/ + + bson_regexp = BSON::Regexp::Raw.new("^hello.world", "m") + bson_regexp.compile + # => /^hello.world/ + +Note that the ``s`` PCRE modifier was converted to the ``m`` Ruby modifier +in the first example, and the last two examples were converted to the same +regular expression even though the original BSON regular expressions had +different meanings. + +When a BSON regular expression uses the non-portable ``^`` and ``$`` +anchors, its conversion to a Ruby regular expression can change its meaning: + +.. code-block:: ruby + + BSON::Regexp::Raw.new("^hello.world", "").compile =~ "42\nhello world" + # => 3 + +When a Ruby regular expression is converted to a BSON regular expression +(for example, to send to the server as part of a query), the BSON regular +expression always has the ``m`` modifier set reflecting the behavior of +``^`` and ``$`` anchors in Ruby regular expressions. + +Reading and Writing +------------------- + +Both Ruby and BSON regular expressions implement the ``to_bson`` method +for serialization to BSON: .. code-block:: ruby @@ -659,6 +815,7 @@ This code example illustrates the difference between serializing a core Ruby # => # _.to_s # => "^b403158\x00m\x00" + regexp_raw = Regexp::Raw.new("^b403158") # => # regexp_raw.to_bson @@ -666,20 +823,23 @@ This code example illustrates the difference between serializing a core Ruby _.to_s # => "^b403158\x00\x00" - -Please use the ``Regexp::Raw`` class to instantiate your BSON regular -expressions to get the exact pattern and options you want. - -When regular expressions are deserialized, they return a wrapper that holds the -raw regex string, but do not compile it. In order to get the Ruby ``Regexp`` -object, one must call ``compile`` on the returned object. +Both ``Regexp`` and ``BSON::Regexp::Raw`` classes implement the ``from_bson`` +class method that deserializes a regular expression from a BSON byte buffer. +Methods of both classes return a ``BSON::Regexp::Raw`` instance that +must be converted to a Ruby regular expression using the ``compile`` method +as described above. .. code-block:: ruby + byte_buffer = BSON::ByteBuffer.new("^b403158\x00\x00") regex = Regexp.from_bson(byte_buffer) - regex.pattern #=> Returns the pattern as a string. - regex.options #=> Returns the raw options as a String. - regex.compile #=> Returns the compiled Ruby Regexp object. + # => # + regex.pattern + # => "^b403158" + regex.options + # => "" + regex.compile + # => /^b403158/ Key Order From b1b8cc6c42e1a007b8e64682e47d127479030e83 Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Fri, 9 Oct 2020 16:15:35 -0400 Subject: [PATCH 323/442] Release 2.14.0.rc1 --- source/installation.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/installation.txt b/source/installation.txt index 00cbf8e26..c30dd52f8 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -19,7 +19,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.14.0.beta + gem install mongo -v 2.14.0.rc1 What's New From ea7b70c02c5fbe69f20b5111666ed1e20eac3b3f Mon Sep 17 00:00:00 2001 From: Emily Giurleo Date: Fri, 6 Nov 2020 15:38:32 -0500 Subject: [PATCH 324/442] RUBY-1387 Add global TLS context hooks (#2123) --- .../tutorials/ruby-driver-create-client.txt | 74 +++++++++++++++---- 1 file changed, 59 insertions(+), 15 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 5dbb0d514..9768fd26f 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -29,7 +29,7 @@ to connect to; if no database name is specified, the client will use the .. code-block:: ruby Mongo::Client.new([ '127.0.0.1:27017' ], database: 'mydb') - + # Or using the URI syntax: Mongo::Client.new("mongodb://127.0.0.1:27017/mydb") @@ -82,7 +82,7 @@ The database can be specified during ``Client`` construction: # Using Ruby client options: client = Mongo::Client.new(['localhost'], database: 'mydb') - + # Using a MongoDB URI: client = Mongo::Client.new('mongodb://localhost/mydb') @@ -92,9 +92,9 @@ new ``Client`` instance configured with the specified database: .. code-block:: ruby client = Mongo::Client.new(['localhost'], database: 'mydb') - + admin_client = client.use('admin') - + # Issue an administrative command admin_client.database.command(replSetGetConfig: 1).documents.first @@ -125,7 +125,7 @@ To force all operations to be performed on the designated server, specify the .. code-block:: ruby Mongo::Client.new([ '1.2.3.4:27017' ], database: 'mydb', direct_connection: true) - + # Or using the URI syntax: Mongo::Client.new("mongodb://1.2.3.4:27017/mydb?directConnection=true") @@ -153,7 +153,7 @@ connect to the replica set. Mongo::Client.new([ '127.0.0.1:27017', '127.0.0.1:27018' ], :database => 'mydb', replica_set: 'myapp') - + # Or using the URI syntax: Mongo::Client.new("mongodb://127.0.0.1:27017,127.0.0.1:27018/mydb?replicaSet=myapp") @@ -186,7 +186,7 @@ spread the operation load accordingly. .. code-block:: ruby Mongo::Client.new([ '1.2.3.4:27017', '1.2.3.5:27017' ], :database => 'mydb') - + Mongo::Client.new("mongodb://1.2.3.4:27017,1.2.3.5:27017/mydb") @@ -270,7 +270,7 @@ Ruby Options * - ``:auth_mech_properties`` - Provides additional authentication mechanism properties. - + The keys in properties are interpreted case-insensitively. When the client is created, keys are lowercased. @@ -436,11 +436,11 @@ Ruby Options max_staleness: 5, } } - + If tag sets are provided, they must be an array of hashes. A server satisfies the read preference if its tags match any one hash in the provided tag sets. - + Each tag set must be a hash, and will be converted internally to a ``BSON::Document`` instance prior to being used for server selection. Hash keys can be strings or symbols. The keys are case sensitive. @@ -480,12 +480,12 @@ Ruby Options subscriber not receiving some of the SDAM events. The ``:sdam_proc`` option permits adding event subscribers on the client being constructed before any SDAM events are published. - + Pass a ``Proc`` which will be called with the ``Client`` as the argument after the client's event subscription mechanism has been initialized but before any of the servers are added to the client. Use this ``Proc`` to set up SDAM event subscribers on the client. - + Note: the client is not fully constructed when the ``Proc`` provided in ``:sdam_proc is invoked, in particular the cluster is nil at this time. ``:sdam_proc`` procedure should limit itself to calling @@ -1090,6 +1090,50 @@ file and both must be stored in the same file. Example usage: URI option values must be properly URI escaped. This applies, for example, to slashes in the paths. +Modifying ``SSLContext`` +------------------------ +It may be desirable to further configure TLS options in the driver, for example +by enabling or disabling certain ciphers. Currently, the Ruby driver does not +provide a way to do this when initializing a ``Mongo::Client``. + +However, the Ruby driver provides a way to set global "TLS context hooks" -- +these are user-provided ``Proc``s that will be invoked before any TLS socket +connection and can be used to modify the underlying ``OpenSSL::SSL::SSLContext`` +object used by the socket. + +To set the TLS context hooks, add ``Proc``s to the ``Mongo.tls_context_hooks`` +array. This should be done before creating any Mongo::Client instances. +For example, in a Rails application this code could be placed in an initializer. + +.. code-block:: ruby + + Mongo.tls_context_hooks.push( + Proc.new { |context| + context.ciphers = ["AES256-SHA"] + } + ) + + # Only the AES256-SHA cipher will be enabled from this point forward + +Every ``Proc`` in ``Mongo.tls_context_hooks`` will be passed an +``OpenSSL::SSL::SSLContext`` object as its sole argument. These ``Proc``s will +be executed sequentially during the creation of every ``Mongo::Socket::SSL`` object. + +It is possible to assign the entire array of hooks calling ``Mongo.tls_context_hooks=``, +but doing so will remove any previously assigned hooks. It is recommended to use +the ``Array#push`` or ``Array#unshift`` methods to add new hooks. + +It is also possible to remove hooks from ``Mongo.tls_context_hooks`` by storing +a reference to the ``Proc``s somewhere else in the application, and then using +``Array#delete_if`` to remove the desired hooks. + +..warning :: + + TLS context hooks are global and will affect every instance of ``Mongo::Client``. + Any library that allows applications to enable these hooks should expose methods to + modify the hooks (which can be called by the application) rather than + automatically enabling the hooks when the library is loaded. + Further information on configuring MongoDB server for TLS is available in the :manual:`MongoDB manual `. @@ -1162,7 +1206,7 @@ The ``:ssl_ca_cert_string`` option supports specifying only one CA certificate. CA certificate options. Doing so would elevate the intermediate certificates to the status of root certificates, rather than verifying intermediate certificates against the root certificates. - + If intermediate certificates need to be used, specify them as part of the client or server TLS certificate files. @@ -1300,7 +1344,7 @@ Usage with Forking Servers ========================== .. note:: - + Applications using Mongoid should follow `Mongoid's forking guidance `_. The guidance and sample code below is provided for applications using the @@ -1316,7 +1360,7 @@ This is because: 2. File descriptors like network sockets are shared between parent and child processes. -The driver attempts to detect client use from forked processes and +The driver attempts to detect client use from forked processes and reestablish network connections when such use is detected, alleviating the issue of file descriptor sharing. From b7037e21824798159fe0af73a51ab8dc9d35aba4 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 9 Nov 2020 10:50:34 -0500 Subject: [PATCH 325/442] RUBY-2439 Implement snappy compression support (#2125) * WIP * WIP 2 * Fix uri options spec tests * Ensure that snappy can install gem extensions * Add new test constraints * Require snappy compression for appropriate tests * Make snappy an optional dependency of the driver * Make changes suggested in PR review * Make sure that tests with snappy compression only run when snappy support is enabled * Add snappy to URI options * Add new UnmetDependency error and improve performance of Snappy check * tweak exception message * fix uri options spec tests * put jruby config into the template * fix zlib test * do not require that snappy is defined * redo snappy system dep handling * clean up Co-authored-by: Zach McCormick Co-authored-by: Emily Giurleo Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 20 ++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 9768fd26f..4d5c044da 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -320,7 +320,8 @@ Ruby Options * - ``:compressors`` - A list of potential compressors to use, in order of preference. The driver chooses the first - compressor that is also supported by the server. Currently the driver only supports 'zlib'. + compressor that is also supported by the server. Currently the driver only supports 'snappy' + and 'zlib'. - ``Array`` - none @@ -739,12 +740,17 @@ URI options are explained in detail in the :manual:`Connection URI reference * - compressors=Strings - ``:compressors => Array`` - Specified as a comma-separated list. Note that the Ruby driver only supports zlib - compression; however, other drivers may support snappy. For maximum compatibility with - drivers, specify ``"snappy,zlib"``; if compatibility with other drivers is not a concern, - specify ``"zlib".`` Compression is not enabled by default and when using MongoDB 4.0 and - earlier, so zlib compression must be manually enabled on the server in order for the Ruby - driver to compress wire protocol data. + Specified as a comma-separated list. The Ruby driver currently supports snappy + and zlib compression algorithms; the first algorithm in the list that the server + also supports will be used. Compression must be explicitly enabled by + specifying the desired compression algorithm; use "snappy,zlib" for maximum server + compatibility. Snappy compression requires separate installation of the + snappy Ruby library; if the snappy Ruby library is not available, + the driver raises an error during ``Mongo::Client`` creation. + When using MongoDB 4.0 and earlier, zlib compression is supported by the + server but must be manually enabled; if zlib is the only compression algorithm + requested (or available due to missing snappy Ruby library), 4.0 and + earlier servers must be configured to explicitly enable zlib compression. * - connect=String - ``:connect => Symbol`` From 16b064735770ffbb814dfc17c31e02033619b460 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Tue, 8 Dec 2020 10:33:30 -0500 Subject: [PATCH 326/442] expand bulk write documentation (#2154) Co-authored-by: Oleg Pudeyev --- source/index.txt | 2 +- .../tutorials/ruby-driver-bulk-operations.txt | 152 ++++++++++++------ 2 files changed, 106 insertions(+), 48 deletions(-) diff --git a/source/index.txt b/source/index.txt index 2c71be98b..48ac5ecd8 100644 --- a/source/index.txt +++ b/source/index.txt @@ -63,6 +63,7 @@ For tutorials on Mongoid, see the `Mongoid Manual {hex: 'ffff00'}}, + }, + }, + { + update_one: { + filter: {name: 'purple'}, + update: {'$set' => {hex: '800080'}}, + }, + }, + ], ordered: true, write_concern: {w: :majority}) + +The following example shows how to execute different types of operations +in the same request: -The ``bulk_write`` method takes three arguments: +.. code-block:: ruby -- A list of operations to execute. -- The ``ordered`` option taking a boolean value. Defaults to ``true``. -- The write concern option. Defaults to the collection's write concern. + collection.bulk_write([ + { insert_one: { x: 1 } }, + { update_one: { + filter: { x: 1 }, + update: {'$set' => { x: 2 } }, + } }, + { replace_one: { + filter: { x: 2 }, + replacement: { x: 3 }, + } }, + ], :ordered => true) + +The first argument to ``bulk_write`` is the list of operations to perform. +Each operation must be specified as a hash with exactly one key which is +the operation name and the operation specification as the corresponding +value. The supported operations are detailed below. The ``bulk_write`` method +also accepts the following options: + +.. list-table:: + :header-rows: 1 + :widths: 40 80 + + * - Option + - Description + * - ``bypass_document_validation`` + - ``true`` or ``false``. Whether to bypass document validation. + * - ``ordered`` + - If the ``ordered`` option is set to ``true`` (which is the default), + the operations are applied in order and if any operation fails, subsequent + operations are not attempted. If the ``ordered`` option is set to ``false``, + all specified operations are attempted. + * - ``write_concern`` + - The write concern for the operation, specified as a hash. Valid bulk write operations are the following: @@ -35,11 +82,11 @@ insert_one .. code-block:: ruby - { :insert_one => { :x => 1 } } + { insert_one: { x: 1 } } .. note:: - There is no ``insert_many`` operation. To insert multiple documents, + There is no ``insert_many`` bulk operation. To insert multiple documents, specify multiple ``insert_one`` operations. @@ -48,10 +95,12 @@ update_one .. code-block:: ruby - { :update_one => { :filter => { :x => 1 }, - :update => { '$set' => { :x => 2 } }, - :upsert => true } # upsert is optional and defaults to false - } + { update_one: { + filter: { x: 1 }, + update: { '$set' => { x: 2 } }, + # upsert is optional and defaults to false + upsert: true, + } } update_many @@ -59,29 +108,12 @@ update_many .. code-block:: ruby - { :update_many => { :filter => { :x => 1 }, - :update => { '$set' => { :x => 2 } }, - :upsert => true } # upsert is optional and defaults to false - } - -The following example shows how to pass operations -to the ``bulk_write`` method. - -.. code-block:: ruby - - coll = client['documents'] - coll.bulk_write([ { :insert_one => { :x => 1 } - }, - { :update_one => { :filter => { :x => 1 }, - :update => {'$set' => { :x => 2 } } - } - }, - { :replace_one => { :filter => { :x => 2 }, - :replacement => { :x => 3 } - } - } - ], - :ordered => true) + { update_many: { + filter: { x: 1 }, + update: { '$set' => { x: 2 } }, + # upsert is optional and defaults to false + :upsert => true, + } } replace_one @@ -89,10 +121,19 @@ replace_one .. code-block:: ruby - { :replace_one => { :filter => { :x => 1 }, - :replacement => { :x => 2 }, - :upsert => true } # upsert is optional and defaults to false - } + { replace_one: { + filter: { x: 1 }, + replacement: { x: 2 }, + # upsert is optional and defaults to false + upsert: true, + } } + +.. note:: + + The ``:replace_one`` operation requires that the replacement value is a + document. ``:replace_one`` does not recognize MongoDB update operators in + the replacement value. In a future release the driver is expected to + prohibit using keys beginning with ``$`` in the replacement document. delete_one @@ -100,7 +141,9 @@ delete_one .. code-block:: ruby - { :delete_one => { :filter => { :x => 1 } } } + { delete_one: { + filter: { x: 1 }, + } } delete_many @@ -108,4 +151,19 @@ delete_many .. code-block:: ruby - { :delete_many => { :filter => { :x => 1 } } } + { delete_many: { + filter: { x: 1 }, + } } + + +Bulk Write Splitting +==================== + +The driver allows the application to submit arbitrarily large bulk write +requests. However, since MongoDB server limits the size of command documents +(currently this limit is 48 MiB), bulk writes that exceed this limit will be +split into multiple requests. + +When :ref:`client-side encryption ` is used, the +threshold used for bulk write splitting is reduced to allow for overhead in +the ciphertext. From 32eb29dc04cad911531615853c58b5d2faca3e7d Mon Sep 17 00:00:00 2001 From: Chris Cho Date: Thu, 10 Dec 2020 09:21:41 -0500 Subject: [PATCH 327/442] DOCSP-13536: remove MongoDB 2.4 from compatibility table (#2156) --- source/reference/driver-compatibility.txt | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index a1379c8f2..11501bb79 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -33,7 +33,6 @@ MongoDB Compatibility - MongoDB 3.2 - MongoDB 3.0 - MongoDB 2.6 - - MongoDB 2.4 * - 2.14 - |checkmark| @@ -44,7 +43,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.13 - |checkmark| [#ocsp]_ @@ -55,7 +53,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.12 - @@ -66,7 +63,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.11 - @@ -77,7 +73,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.10 - @@ -88,7 +83,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.9 - @@ -99,7 +93,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.8 - @@ -110,7 +103,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.7 - @@ -121,7 +113,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.6 - @@ -132,7 +123,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.5 - @@ -143,7 +133,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - * - 2.4 - @@ -154,7 +143,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - |checkmark| * - 2.3 - @@ -165,7 +153,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - |checkmark| * - 2.2 - @@ -176,7 +163,6 @@ MongoDB Compatibility - |checkmark| - |checkmark| - |checkmark| - - |checkmark| * - 2.0 - @@ -187,7 +173,6 @@ MongoDB Compatibility - - |checkmark| - |checkmark| - - |checkmark| .. [#ocsp] OCSP verification is implemented as of driver version 2.14. From 5f52bdbedeebdfa3d5aa0cadbe1ec28bc673d8de Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 11 Jan 2021 12:31:19 -0500 Subject: [PATCH 328/442] RUBY-2488 use Ruby 3 toolchain (#2159) * use ubuntu1604 for ruby 2.3 testing * JRuby 9.2.14.0 added TCP_KEEPINTVL etc. but they are not working * document jruby keepalive limitation Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 4d5c044da..57700f6be 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -1253,16 +1253,27 @@ The driver does not currently implement the Happy Eyeballs algorithm. TCP Keepalive Configuration =========================== -The driver sets TCP keepalive by default. The following default values are also set if the system -value can be determined and if the driver default value is less than the system value. +Where allowed by system configuration and the Ruby language runtime, +the driver enables TCP keepalive and, for each of the keepalive parameters +listed below, sets the value of the respective parameter to the specified +value if the system value can be determined and is higher than the +listed driver value: - ``tcp_keepalive_time``: 120 seconds - ``tcp_keepalive_intvl``: 10 seconds - ``tcp_keepalive_cnt``: 9 probes +.. note:: + + As of JRuby 9.2.14.0, JRuby does not implement the APIs required to + set the keepalive parameters. When using JRuby, the driver will not be + able to set the keepalive parameters and the system configuration will + be in effect. -Please see the `MongoDB Diagnostics FAQ keepalive section `_ -for instructions on setting these values at the system level. +To use lower values, or to change the parameters in environments like JRuby +that do not expose the required APIs, please adjust the parameters at the +system level as described in the `MongoDB Diagnostics FAQ keepalive section +`_. Connection Pooling From 78c61b23047ebeb37ce26f6458c53ead9b863931 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 25 Feb 2021 23:40:07 -0500 Subject: [PATCH 329/442] RUBY-2411 Port query cache middleware to driver (#2165) * RUBY-2411 Port query cache middleware to driver * fix typos Co-authored-by: Oleg Pudeyev --- source/tutorials/query-cache.txt | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/source/tutorials/query-cache.txt b/source/tutorials/query-cache.txt index 0ab119106..27af0b815 100644 --- a/source/tutorials/query-cache.txt +++ b/source/tutorials/query-cache.txt @@ -20,8 +20,10 @@ the cached reuslts to prevent unnecessary roundtrips to the database. Usage ===== -The query cache is disabled by default. It cache can be enabled on the global -scope as well as within the context of a specific block. +The query cache is disabled by default. It can be enabled on the global +scope as well as within the context of a specific block. The driver also +provides a :ref:`Rack middleware ` to enable the +query cache automatically for each web request. To enable the query cache globally: @@ -231,3 +233,24 @@ You may read more about system collections in the Even when the query cache is enabled, query results from system collections will not be cached. + +.. _query-cache-middleware: + +Query Cache Middleware +====================== + +The driver provides a Rack middleware which enables the query cache for the +duration of each web request. Below is an example of how to enable the +query cache middleware in a Ruby on Rails application: + +.. code-block:: ruby + + # config/application.rb + + # Add Mongo::QueryCache::Middleware at the bottom of the middleware stack + # or before other middleware that queries MongoDB. + config.middleware.use Mongo::QueryCache::Middleware + +Please refer to the `Rails on Rack guide +`_ +for more information about using Rack middleware in Rails applications. From 4b15c81cd29a6a87eeb60039aa3b5e94bf9619ad Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Fri, 26 Feb 2021 18:08:03 -0500 Subject: [PATCH 330/442] Bump version to 2.15.0.alpha --- source/installation.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/installation.txt b/source/installation.txt index c30dd52f8..31909acd4 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -19,7 +19,7 @@ To install the mongo gem manually: .. code-block:: sh - gem install mongo -v 2.14.0.rc1 + gem install mongo -v 2.15.0.alpha What's New From cd0bee5322c513cf9b3b4aa94b2e24d4ede9109b Mon Sep 17 00:00:00 2001 From: Alex Bevilacqua Date: Sat, 6 Mar 2021 21:52:03 -0500 Subject: [PATCH 331/442] RUBY-1682: Add Zstandard (zstd) network compression support (#2166) * RUBY-1682: Add zstd network compression support * RUBY-1682: Update evergreen config for zstd tests * RUBY-1682: Reorder compressors so zstd is listed first * RUBY-1682: Update comment regarding default compression level * RUBY-1682: Revert zstd compression to zstd-ruby default * reword the compression documentation * RUBY-1682: Remove jruby test configurations * snappy works on 3.4 it seems * RUBY-1682: Fix zstd-noauth by targeting mongodb 4.2 * RUBY-1682: rebase and update reference to spec/shared * RUBY-1682: Add min_server_version requirement to zstd integration test * RUBY-1682: update evergreen configs Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 53 ++++++++++++++----- 1 file changed, 39 insertions(+), 14 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 57700f6be..c8e5b5a84 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -319,9 +319,8 @@ Ruby Options - none * - ``:compressors`` - - A list of potential compressors to use, in order of preference. The driver chooses the first - compressor that is also supported by the server. Currently the driver only supports 'snappy' - and 'zlib'. + - A list of potential compressors to use, in order of preference. + Please see below for details on how the driver implements compression. - ``Array`` - none @@ -740,17 +739,9 @@ URI options are explained in detail in the :manual:`Connection URI reference * - compressors=Strings - ``:compressors => Array`` - Specified as a comma-separated list. The Ruby driver currently supports snappy - and zlib compression algorithms; the first algorithm in the list that the server - also supports will be used. Compression must be explicitly enabled by - specifying the desired compression algorithm; use "snappy,zlib" for maximum server - compatibility. Snappy compression requires separate installation of the - snappy Ruby library; if the snappy Ruby library is not available, - the driver raises an error during ``Mongo::Client`` creation. - When using MongoDB 4.0 and earlier, zlib compression is supported by the - server but must be manually enabled; if zlib is the only compression algorithm - requested (or available due to missing snappy Ruby library), 4.0 and - earlier servers must be configured to explicitly enable zlib compression. + A comma-separated list of potential compressors to use, in order of + preference. Please see below for details on how the driver implements + compression. * - connect=String - ``:connect => Symbol`` @@ -1665,6 +1656,40 @@ option to the client instance. Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'test', :truncate_logs => false ) +Compression +=========== + +To use wire protocol compression, at least one compressor must be explicitly +requested using either the ``:compressors`` Ruby option or the ``compressors`` +URI option. If no compressors are explicitly requested, the driver will not +use compression, even if the required dependencies for one or more compressors +are present on the system. + +The driver chooses the first compressor of the ones requested that is also +supported by the server. The driver currently supports ``zstd``, ``snappy`` and +``zlib`` compressors. ``zstd`` compressor is recommended as it produces +the highest compression at the same CPU consumption compared to the other +compressors. For maximum server compatibility all three compressors can be +specified, e.g. as ``compressors: ["zstd", "snappy", "zlib"]``. + +``zstd`` compressor requires the +`zstd-ruby `_ library to be installed. +``snappy`` compressor requires the +`snappy `_ library to be installed. +If ``zstd`` or ``snappy`` compression is requested, and the respective +library is not loadable, the driver will raise an error during +``Mongo::Client`` creation. ``zlib`` compression requires the ``zlib`` +standard library extension to be present. + +The server support for various compressors is as follows: + +- ``zstd`` requires and is enabled by default in MongoDB 4.2 or higher. +- ``snappy`` requires MongoDB 3.4 or higher and is enabled by default in + MongoDB 3.6 or higher. +- ``zlib`` requires MongoDB 3.6 or higher and is enabled by default in + MongoDB 4.2 and higher. + + Development Configuration ========================= From d51ec2dffcd2cb0c80cedd16276e70a8c44d6701 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Thu, 11 Mar 2021 09:05:40 -0500 Subject: [PATCH 332/442] RUBY-2429 Implement base Versioned MongoDB API (#2185) * RUBY-2429 Implement base Versioned MongoDB API * always send strict value * always send deprecation errors value * update shared ref Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 19 ++++++++++++++++++- .../tutorials/ruby-driver-database-tasks.txt | 15 +++++++++++++-- 2 files changed, 31 insertions(+), 3 deletions(-) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index c8e5b5a84..6ddb442f1 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -494,6 +494,23 @@ Ruby Options - ``Proc`` - none + * - ``:server_api`` + - The server API version requested. + This is a hash with the following allowed items: + - ``:version`` (String) + - ``:strict`` (true or false) + - ``:deprecation_errors`` (true or false) + + Note that the server API version can only be specified as a Ruby option, + not as a URI option, and it cannot be overridden for database and + collection objects. + + If server API version is changed on a client (such as via the ``with`` + call), the entire API version hash is replaced with the new specification + (the old and the new individual fields are NOT merged). + - ``Hash`` + - none + * - ``:server_selection_timeout`` - The number of seconds to wait for an appropriate server to be selected for an operation to be executed before raising an exception. @@ -1121,7 +1138,7 @@ but doing so will remove any previously assigned hooks. It is recommended to use the ``Array#push`` or ``Array#unshift`` methods to add new hooks. It is also possible to remove hooks from ``Mongo.tls_context_hooks`` by storing -a reference to the ``Proc``s somewhere else in the application, and then using +a reference to the Procs somewhere else in the application, and then using ``Array#delete_if`` to remove the desired hooks. ..warning :: diff --git a/source/tutorials/ruby-driver-database-tasks.txt b/source/tutorials/ruby-driver-database-tasks.txt index 9e40328f7..ec01899bb 100644 --- a/source/tutorials/ruby-driver-database-tasks.txt +++ b/source/tutorials/ruby-driver-database-tasks.txt @@ -16,6 +16,7 @@ Databases The driver provides various helpers on database objects for executing commands, getting collection lists, and administrative tasks. + List Collections ```````````````` @@ -24,12 +25,13 @@ To get a list of collections or collection names for a database, use .. code-block:: ruby - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music') database = client.database database.collections # Returns an array of Collection objects. database.collection_names # Returns an array of collection names as strings. + Arbitrary Comands ````````````````` @@ -37,12 +39,21 @@ To execute any command on the database, use the ``command`` method. .. code-block:: ruby - client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'music') + client = Mongo::Client.new([ '127.0.0.1:27017' ], database: 'music') database = client.database result = database.command(:ismaster => 1) result.first # Returns the BSON::Document returned from the server. +.. note:: + + Specifying server API version as a client option and also specifying + any of the respective command parameters to the ``command`` method + (i.e. the ``apiVersion``, ``apiStrict`` and ``apiDeprecationErrors`` + command parameters) at the same time is not allowed, and the result + of attempting to do so is undefined. + + Drop Database ````````````` From a658a668f8eaed89f6636740ca17584897672797 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 22 Mar 2021 21:26:43 -0400 Subject: [PATCH 333/442] RUBY-2511 Document Versioned API Usage in tutorial (#2195) Co-authored-by: Oleg Pudeyev --- .../tutorials/ruby-driver-create-client.txt | 104 ++++++++++++++++++ .../tutorials/ruby-driver-database-tasks.txt | 2 + 2 files changed, 106 insertions(+) diff --git a/source/tutorials/ruby-driver-create-client.txt b/source/tutorials/ruby-driver-create-client.txt index 6ddb442f1..7fe4fc771 100644 --- a/source/tutorials/ruby-driver-create-client.txt +++ b/source/tutorials/ruby-driver-create-client.txt @@ -1707,6 +1707,110 @@ The server support for various compressors is as follows: MongoDB 4.2 and higher. +Server API Parameters +===================== + +Starting with MongoDB 5.0, applications can request that the server behaves +in accordance with a particular server API version. + +Server API parameters can be specified via the ``:server_api`` option to +``Client``. These parameters cannot be provided via a URI. + +Currently the only defined API version is ``"1"``. It can be requested +as follows: + +.. code-block:: ruby + + client = Mongo::Client.new(['localhost'], server_api: {version: "1"}) + +MongoDB server defines API versions as string values. For convenience, if the +API version is provided as an integer, the Ruby driver will stringify it and +send it to the server as a string: + +.. code-block:: ruby + + client = Mongo::Client.new(['localhost'], server_api: {version: 1}) + +Note that the server may define API versions that are not stringified integers. +Applications must not assume that all legal API versions can be expressed +as integers. + +When a particular API version is requested, operations which are part of that +API version behave as specified in that API version. Operations which are not +part of the specified API version behave as they would had the API version +not been specified at all. Operations whose behavior is subject to the +configured API version are commands including command arguments, queries, +aggregation pipeline stages and arguments. + +Applications may request that the server rejects all operations which are not +part of the specified API version by setting the ``:strict`` option: + +.. code-block:: ruby + + client = Mongo::Client.new(['localhost'], server_api: {version: "1", strict: true}) + +For example, since the ``:tailable`` option is not part of the server API +version 1, the following query would fail: + +.. code-block:: ruby + + client = Mongo::Client.new(['localhost'], server_api: {version: "1", strict: true}) + client['collection'].find({}, tailable: true) + # => Mongo::Error::OperationFailure (BSON field 'FindCommand.tailable' is not allowed with apiStrict:true. (323) (on localhost:27017, modern retry, attempt 1)) + +Applications may request that the server rejects all operations which are +deprecated in the specified API version by setting the ``:deprecation_errors`` +option: + +.. code-block:: ruby + + client = Mongo::Client.new(['localhost'], server_api: {version: "1", deprecation_errors: true}) + +Note that, as of this writing, there are no deprecated operations in API +version ``"1"``. + +If the server API parameters have been defined on a ``Client`` object, +they will be sent by the client as part of each [*]_ executed operation. + +.. [*] ``getMore`` commands and commands in transactions do not accept + API parameters, thus the driver will not send them in these cases. + +MongoDB servers prior to 5.0 do not recognize the API parameters, and will +produce a variety of errors should the application configure them. +The Ruby driver will send the API parameters to all MongoDB 3.6 and newer +servers, but the API parameters should only be configured when the application +is communicating with MongoDB 5.0 or newer servers. The API parameters +cannot be sent to MongoDB 3.4 and older servers that use the legacy wire +protocol; if an application configures the API parameters and connects to +MongoDB 3.4 or older servers, the driver will produce an error on every +operation. + +The :ref:`command helper ` permits the application to +send manually constructed commands to the server. If the client is not +configured with server API parameters, the command helper may be used to +issue commands with API parameters: + +.. code-block:: ruby + + client.database.command( + ping: 1, + apiVersion: "1", + apiStrict: false, + apiDeprecationErrors: false, + ) + +If the client is configured with server API parameters, the command helper +may not be used to issue commands with server API parameters. This includes the +case when the server API parameters provided to the client and to the +command helper are identical. If a client is constructed with server API +parameters, to send different API parameters (or none at all) a new client +must be constructed, either from scratch or using the ``with`` method. + +The server API parameters may only be specified on the client level. +They may not be specified on the database, collection, session, transaction +or individual operation level. + + Development Configuration ========================= diff --git a/source/tutorials/ruby-driver-database-tasks.txt b/source/tutorials/ruby-driver-database-tasks.txt index ec01899bb..d327470e0 100644 --- a/source/tutorials/ruby-driver-database-tasks.txt +++ b/source/tutorials/ruby-driver-database-tasks.txt @@ -32,6 +32,8 @@ To get a list of collections or collection names for a database, use database.collection_names # Returns an array of collection names as strings. +.. _arbitrary-commands: + Arbitrary Comands ````````````````` From f0039fac3a7043865ef2708cfb7df4ee878371d1 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Mon, 22 Mar 2021 22:20:54 -0400 Subject: [PATCH 334/442] RUBY-2550 Support connecting to servers that require API version to be specified (#2188) * RUBY-2550 Support connecting to servers that require API version to be specified * tests fail on replica sets and sharded clusters, skip those Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-database-tasks.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/tutorials/ruby-driver-database-tasks.txt b/source/tutorials/ruby-driver-database-tasks.txt index d327470e0..b9a3668bd 100644 --- a/source/tutorials/ruby-driver-database-tasks.txt +++ b/source/tutorials/ruby-driver-database-tasks.txt @@ -52,8 +52,7 @@ To execute any command on the database, use the ``command`` method. Specifying server API version as a client option and also specifying any of the respective command parameters to the ``command`` method (i.e. the ``apiVersion``, ``apiStrict`` and ``apiDeprecationErrors`` - command parameters) at the same time is not allowed, and the result - of attempting to do so is undefined. + command parameters) at the same time is not allowed and will produce an error. Drop Database From 24042c1258c94920f1c5839dcd90431bdb98f48e Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 28 Apr 2021 07:12:38 -0400 Subject: [PATCH 335/442] RUBY-2588 Unpin sessions after all abortTransaction attempts (#2218) Co-authored-by: Oleg Pudeyev --- source/tutorials/ruby-driver-crud-operations.txt | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/source/tutorials/ruby-driver-crud-operations.txt b/source/tutorials/ruby-driver-crud-operations.txt index 4e6fc1e0b..b43b26363 100644 --- a/source/tutorials/ruby-driver-crud-operations.txt +++ b/source/tutorials/ruby-driver-crud-operations.txt @@ -200,6 +200,10 @@ when querying and their corresponding methods as examples. * - Option - Description + * - ``allow_disk_use`` + - When set to true, the server can write temporary data to disk while + executing the find operation. This option is only available on MongoDB + server versions 4.4 and newer. * - ``allow_partial_results`` - For use with sharded clusters. If a shard is down, allows the query to return results from the shards that are up, potentially only getting @@ -250,6 +254,8 @@ when querying and their corresponding methods as examples. * - ``max_scan(Integer)`` - Sets the maximum number of documents to scan if a full collection scan would be performed. Deprecated as of MongoDB server version 4.0. + * - ``max_time_ms(Integer)`` + - The maximum amount of time to allow the query to run, in milliseconds. * - ``no_cursor_timeout`` - MongoDB automatically closes inactive cursors after a period of 10 minutes. Call this for cursors to remain open indefinitely on the server. @@ -267,6 +273,8 @@ when querying and their corresponding methods as examples. client[:artists].find.read(:mode => :secondary_preferred) + * - ``session(Session)`` + - The session to use. * - ``show_disk_loc(Boolean)`` - Tells the results to also include the location of the documents on disk. * - ``skip(Integer)`` @@ -280,6 +288,7 @@ when querying and their corresponding methods as examples. client[:artists].find.sort(:name => -1) + Additional Query Operations --------------------------- From 0486e282b25617f174dc522d05241332f1587196 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev Date: Thu, 6 May 2021 16:50:48 -0400 Subject: [PATCH 336/442] Fix RST syntax --- docs/tutorials/bson-v4.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/bson-v4.txt b/docs/tutorials/bson-v4.txt index ed5636d10..17abffe98 100644 --- a/docs/tutorials/bson-v4.txt +++ b/docs/tutorials/bson-v4.txt @@ -646,7 +646,7 @@ Ruby vs MongoDB Regular Expressions ----------------------------------- MongoDB server uses `Perl-compatible regular expressions implemented using -the PCRE library`_ and `Ruby regular expressions +the PCRE library `_ and `Ruby regular expressions `_ are implemented using the `Onigmo regular expression engine `_, which is a fork of `Oniguruma `_. From de043a6c28a956b530da84d12dd97c3d69187f1c Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Fri, 7 May 2021 08:52:25 -0400 Subject: [PATCH 337/442] Move TLS notes to compatibility page (#2225) Co-authored-by: Oleg Pudeyev --- ...uby-driver-compatibility-full-language.rst | 2 - ...ruby-driver-compatibility-full-mongodb.rst | 2 - ...by-driver-compatibility-matrix-mongodb.rst | 11 --- source/installation.txt | 73 ++++++++----------- source/reference/driver-compatibility.txt | 68 ++++++++++++++++- 5 files changed, 93 insertions(+), 63 deletions(-) delete mode 100644 source/includes/ruby-driver-compatibility-full-language.rst delete mode 100644 source/includes/ruby-driver-compatibility-full-mongodb.rst delete mode 100644 source/includes/ruby-driver-compatibility-matrix-mongodb.rst diff --git a/source/includes/ruby-driver-compatibility-full-language.rst b/source/includes/ruby-driver-compatibility-full-language.rst deleted file mode 100644 index 945edf942..000000000 --- a/source/includes/ruby-driver-compatibility-full-language.rst +++ /dev/null @@ -1,2 +0,0 @@ -For additional driver versions, see :ref:`Ruby Driver Language Compatibility Reference `. - diff --git a/source/includes/ruby-driver-compatibility-full-mongodb.rst b/source/includes/ruby-driver-compatibility-full-mongodb.rst deleted file mode 100644 index 9cc1f5f6a..000000000 --- a/source/includes/ruby-driver-compatibility-full-mongodb.rst +++ /dev/null @@ -1,2 +0,0 @@ -For additional driver versions, see :ref:`Ruby Driver MongoDB Compatibility Reference `. - diff --git a/source/includes/ruby-driver-compatibility-matrix-mongodb.rst b/source/includes/ruby-driver-compatibility-matrix-mongodb.rst deleted file mode 100644 index dd37364bc..000000000 --- a/source/includes/ruby-driver-compatibility-matrix-mongodb.rst +++ /dev/null @@ -1,11 +0,0 @@ -The following compatibility table specifies the recommended -version(s) of the MongoDB Ruby driver for use with a specific version of -MongoDB. Except when indicated, the specified driver versions expose or -take advantage of the features added in the corresponding server versions. - -MongoDB server releases are generally backwards compatible, meaning a -particular version of the driver will generally work with newer versions of -the server but may not take advantage of the functionality released in the -newer version of the server. - -The first column lists the driver versions. diff --git a/source/installation.txt b/source/installation.txt index 31909acd4..d9ff044eb 100644 --- a/source/installation.txt +++ b/source/installation.txt @@ -4,22 +4,47 @@ Installation .. default-domain:: mongodb -The Ruby driver is bundled as a gem and is hosted on `Rubygems +The Ruby driver is released as a gem hosted on `Rubygems `_. +Prerequisites +============= + +Please see the :ref:`compatibility ` page for the list of +Ruby versions and MongoDB server versions that this release of the Ruby +driver is compatible with. + +The driver itself is written entirely in Ruby, however it depends on the +`bson library `_ which includes a C extension +for MRI and a compiled Java extension for JRuby. A working C compiler and Ruby +development headers and libraries are required when installing on MRI. +When installing on JRuby, JRE is sufficient because the ``bson`` gem includes +the compiled extension. + +Connecting to TLS-enabled MongoDB servers, using SCRAM authentication +(both SCRAM-SHA-1 and SCRAM-SHA-256) and using X.509 authentication (which +is performed over a TLS connection) requires the Ruby ``openssl`` extension +to be present and working. The :ref:`TLS compatibility ` +section provides further details on usage of newer TLS protocols like TLS 1.1. + + .. _ruby-driver-install: Install the Gem =============== -The driver can be installed manually or with bundler. +Add ``mongo`` to your ``Gemfile``: + +.. code-block:: ruby -To install the mongo gem manually: + gem "mongo", "~> 2" + +To install the driver manually: .. code-block:: sh - gem install mongo -v 2.15.0.alpha + gem install mongo -v '~> 2' What's New @@ -28,43 +53,3 @@ What's New Please consult the `releases page on GitHub `_ for the list of improvements and changes in each version of the driver. - - -TLS/SSL and the Ruby Driver -=========================== - -Industry best practices, and some regulations, require the use of TLS 1.1 -or newer. Though no application changes are required for the Ruby driver to -make use of the newest protocols, some operating systems or versions may -not provide an OpenSSL version new enough to support them. - -Users of macOS older than 10.13 (High Sierra) will need to install Ruby from -`rvm`_, `homebrew`_, `macports`_, or another similar source. See -`installation information on ruby-lang.org`_ for more options. - -Users of Linux or other non-macOS Unix can check their OpenSSL version like this: - -.. code-block:: sh - - $ openssl version - -If the version number is less than 1.0.1 support for TLS 1.1 or newer is -not available. Contact your operating system vendor for a solution or upgrade -to a newer distribution. - -You can check your Ruby interpreter by executing the following command: - -.. code-block:: sh - - ruby -e "require 'net/http'; require 'json'; puts JSON.parse(Net::HTTP.get(URI('https://www.howsmyssl.com/a/check')))['tls_version']" - -You should see "TLS 1.X" where X is >= 1. - -You can read more about TLS versions and their security implications here: - -``_ - -.. _rvm: https://rvm.io/ -.. _homebrew: https://brew.sh/ -.. _macports: https://www.macports.org/ -.. _installation information on ruby-lang.org: https://www.ruby-lang.org/en/documentation/installation diff --git a/source/reference/driver-compatibility.txt b/source/reference/driver-compatibility.txt index 11501bb79..5bedc7613 100644 --- a/source/reference/driver-compatibility.txt +++ b/source/reference/driver-compatibility.txt @@ -1,4 +1,4 @@ -.. _reference-compatibility-ruby: +.. _compatibility: ******************** Driver Compatibility @@ -12,12 +12,23 @@ Driver Compatibility :depth: 1 :class: singlecol -.. _reference-compatibility-mongodb-ruby: + +.. _mongodb-compatibility: MongoDB Compatibility ===================== -.. include:: /includes/ruby-driver-compatibility-matrix-mongodb.rst +The following compatibility table specifies the recommended +version(s) of the MongoDB Ruby driver for use with a specific version of +MongoDB. Except when indicated, the specified driver versions expose or +take advantage of the features added in the corresponding server versions. + +MongoDB server releases are generally backwards compatible, meaning a +particular version of the driver will generally work with newer versions of +the server but may not take advantage of the functionality released in the +newer version of the server. + +The first column lists the driver versions. .. list-table:: :header-rows: 1 @@ -184,7 +195,8 @@ MongoDB Compatibility The driver does not support older versions of MongoDB. -.. _reference-compatibility-language-ruby: + +.. _ruby-compatibility: Ruby Compatibility ================== @@ -426,6 +438,7 @@ for that Ruby version is deprecated. The driver does not support older versions of Ruby. + Rails/ActiveSupport Compatibility ================================= @@ -442,6 +455,52 @@ compatibility code for behavior like time serialization to be correct: Applications using Mongoid 7.0.6 or newer do not need to explicitly load the driver's ActiveSupport code, since Mongoid automatically does so. + +.. _tls-compatibility: + +TLS/SSL Compatibility +===================== + +The driver will utilize the protocols supported by the underlying Ruby +``openssl`` extension. In turn, the ``openssl`` extension generally exposes +the functionality that exists in the operating system's OpenSSL library. + +Industry best practices, and some regulations, require the use of TLS 1.1 +or newer. Some operating systems or versions may not provide an OpenSSL version +new enough to support these TLS versions. + +Users of macOS older than 10.13 (High Sierra) will need to install Ruby from +`rvm`_, `homebrew`_, `macports`_, or another similar source. See +`installation information on ruby-lang.org`_ for more options. + +Users of Linux or other non-macOS Unix can check their OpenSSL version +as follows: + +.. code-block:: sh + + openssl version + +If the version number is less than 1.0.1 support for TLS 1.1 or newer is +not available. Contact your operating system vendor for a solution or upgrade +to a newer distribution. + +You can check your Ruby interpreter by executing the following command: + +.. code-block:: sh + + ruby -e "require 'net/http'; require 'json'; puts JSON.parse(Net::HTTP.get(URI('https://www.howsmyssl.com/a/check')))['tls_version']" + +You should see "TLS 1.X" where X is >= 1. + +You can read more about TLS versions and their security implications `here +`_. + +.. _rvm: https://rvm.io/ +.. _homebrew: https://brew.sh/ +.. _macports: https://www.macports.org/ +.. _installation information on ruby-lang.org: https://www.ruby-lang.org/en/documentation/installation + + Atlas Compatibility =================== @@ -454,6 +513,7 @@ When running on JRuby and connecting to Atlas Free Tier, `driver version 2.6.4 `_ or higher and Java 8 or higher are required. + ``mongo_kerberos`` Compatibility ================================ From d550d3881ad8e6efe245fe61d0e09d6e35229e19 Mon Sep 17 00:00:00 2001 From: Oleg Pudeyev <39304720+p-mongo@users.noreply.github.com> Date: Wed, 12 May 2021 06:29:30 -0400 Subject: [PATCH 338/442] Reorganize documentation navigation (#2226) * Remove needless ruby-driver- prefix from documentation * Reorganize navigation * Make projection singular Co-authored-by: Oleg Pudeyev --- source/getting-started.txt | 17 +++++++ source/index.txt | 45 ++++++------------- source/installation.txt | 2 +- .../aggregation.txt} | 0 .../authentication.txt} | 0 .../bulk-operations.txt} | 2 +- .../change-streams.txt} | 0 .../client-side-encryption.txt | 0 .../collations.txt} | 0 .../collection-tasks.txt} | 5 --- .../connection-and-configuration.txt | 18 ++++++++ .../create-client.txt} | 6 +-- .../crud-operations.txt} | 8 ++-- .../database-tasks.txt} | 9 ++-- .../geospatial-search.txt} | 0 .../gridfs.txt} | 0 .../indexing.txt} | 2 +- .../monitoring.txt} | 0 .../projection.txt} | 10 ++--- .../{tutorials => reference}/query-cache.txt | 0 source/reference/schema-operations.txt | 18 ++++++++ .../sessions.txt} | 0 .../text-search.txt} | 0 .../transactions.txt} | 0 .../user-management.txt | 0 source/reference/working-with-data.txt | 26 +++++++++++ source/ruby-driver-tutorials.txt | 34 -------------- source/tutorials.txt | 16 +++++++ source/{ => tutorials}/quick-start.txt | 14 +++--- 29 files changed, 133 insertions(+), 99 deletions(-) create mode 100644 source/getting-started.txt rename source/{tutorials/ruby-driver-aggregation.txt => reference/aggregation.txt} (100%) rename source/{tutorials/ruby-driver-authentication.txt => reference/authentication.txt} (100%) rename source/{tutorials/ruby-driver-bulk-operations.txt => reference/bulk-operations.txt} (99%) rename source/{tutorials/ruby-driver-change-streams.txt => reference/change-streams.txt} (100%) rename source/{tutorials => reference}/client-side-encryption.txt (100%) rename source/{tutorials/ruby-driver-collations.txt => reference/collations.txt} (100%) rename source/{tutorials/ruby-driver-collection-tasks.txt => reference/collection-tasks.txt} (98%) create mode 100644 source/reference/connection-and-configuration.txt rename source/{tutorials/ruby-driver-create-client.txt => reference/create-client.txt} (99%) rename source/{tutorials/ruby-driver-crud-operations.txt => reference/crud-operations.txt} (99%) rename source/{tutorials/ruby-driver-database-tasks.txt => reference/database-tasks.txt} (95%) rename source/{tutorials/ruby-driver-geospatial-search.txt => reference/geospatial-search.txt} (100%) rename source/{tutorials/ruby-driver-gridfs.txt => reference/gridfs.txt} (100%) rename source/{tutorials/ruby-driver-indexing.txt => reference/indexing.txt} (99%) rename source/{tutorials/ruby-driver-monitoring.txt => reference/monitoring.txt} (100%) rename source/{tutorials/ruby-driver-projections.txt => reference/projection.txt} (96%) rename source/{tutorials => reference}/query-cache.txt (100%) create mode 100644 source/reference/schema-operations.txt rename source/{tutorials/ruby-driver-sessions.txt => reference/sessions.txt} (100%) rename source/{tutorials/ruby-driver-text-search.txt => reference/text-search.txt} (100%) rename source/{tutorials/ruby-driver-transactions.txt => reference/transactions.txt} (100%) rename source/{tutorials => reference}/user-management.txt (100%) create mode 100644 source/reference/working-with-data.txt delete mode 100644 source/ruby-driver-tutorials.txt create mode 100644 source/tutorials.txt rename source/{ => tutorials}/quick-start.txt (93%) diff --git a/source/getting-started.txt b/source/getting-started.txt new file mode 100644 index 000000000..b92298a54 --- /dev/null +++ b/source/getting-started.txt @@ -0,0 +1,17 @@ +.. _getting-started: + +*************** +Getting Started +*************** + +.. default-domain:: mongodb + +This section describes how to install the driver, installation prerequisites +and compatibility considerations. + +.. toctree:: + :titlesonly: + + installation + reference/driver-compatibility + support diff --git a/source/index.txt b/source/index.txt index 48ac5ecd8..6e3f022eb 100644 --- a/source/index.txt +++ b/source/index.txt @@ -16,7 +16,7 @@ Get Started =========== To get started with the Ruby driver, see :doc:`/installation` and -:doc:`/quick-start`. Continue to :doc:`/ruby-driver-tutorials` +:doc:`/tutorials/quick-start`. Continue to :doc:`/tutorials` for high level documentation for common operations. BSON @@ -53,34 +53,15 @@ For tutorials on Mongoid, see the `Mongoid Manual