MongoDB 4.0
is just around the corner and with rc0 we can get a good idea of what we can expect in the GA version. MongoDB 4.0 will allow transactions to run in a replica set and, in a future release, the MongoDB transaction will work for sharded clusters. This is a really big change!
Multi-statement transactions are a big deal for a lot of companies. The transactions feature has been in development since MongoDB version 3.0; in version 3.6 logical sessions were added. In an earlier blog post we highlighted a few details from what was delivered in 3.6 that indicated that 4.0 would have transactions.
There are a few limitations for transactions and some operations are not allowed yet. A detailed list can be found in the MongoDB documentation of the Session.startTransaction() method.
One restriction that we must be aware of is that the collection MUST exist in order to use transactions.
A simple transaction will be declared in a very similar way to that we use for other databases. The caveat is that we need to start a session before starting a transaction. This means that multi-statement transactions are not the default behavior to write to the database.
How to use transactions in MongoDB® 4.0
Download MongoDB 4.0 RC (or you can install it from the repositories).
1 | wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1604-4.0.0-rc1.tgz |
Uncompress the files:
1 | tar -xvzf mongodb-linux-x86_64-ubuntu1604-4.0.0-rc1.tgz |
Rename the folder to mongo4.0 and create the data folder inside of the bin folder:
1 2 3 4 | mv mongodb-linux-x86_64-ubuntu1604-4.0.0-rc1 mongo4.0 cd mongo4.0 cd bin mkdir data |
Start the database process:
Important: in order to have multi-statement transactions replica-set must be enabled
1 | ./mongod --dbpath data --logpath data/log.log --fork --replSet foo |
Initialize the replica-set:
1 2 3 | > rs.initiate() foo:Primary> use percona foo:Primary> db.createCollection('test') |
Start a session and then a transaction:
1 2 3 4 | session = db.getMongo().startSession() session.startTransaction() session.getDatabase("percona").test.insert({today : new Date()}) session.getDatabase("percona").test.insert({some_value : "abc"}) |
Then you can decide whether to commit the transaction or abort it:
1 2 | session.commitTransaction() session.abortTransaction() |
If the startTransaction throws the IllegalOperation error, make sure the database is running with replica set.
Transaction isolation level in in MongoDB 4.0: Snapshot Isolation
MongoDB 4.0 implements snapshot isolation for the transactions. The pending uncommitted changes are only visible inside the session context (the session which has started the transaction) and are not visible outside. Here is an example:
Connection 1:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | foo:PRIMARY> use percona switched to db percona foo:PRIMARY> db.createCollection('test') { "ok" : 1, "operationTime" : Timestamp(1528903182, 1), "$clusterTime" : { "clusterTime" : Timestamp(1528903182, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } foo:PRIMARY> session = db.getMongo().startSession() session { "id" : UUID("bdd82af7-ab9d-4cd3-9238-f08ee928f31e") } foo:PRIMARY> session.startTransaction() foo:PRIMARY> session.getDatabase("percona").test.insert({today : new Date()}) WriteResult({ "nInserted" : 1 }) foo:PRIMARY> session.getDatabase("percona").test.insert({some_value : "abc"}) WriteResult({ "nInserted" : 1 }) |
Connection 2: starting second transaction in its own session:
1 2 3 4 5 6 7 8 9 | foo:PRIMARY> use percona switched to db percona foo:PRIMARY> db.test.find() foo:PRIMARY> db.test.find() foo:PRIMARY> session = db.getMongo().startSession() session { "id" : UUID("eb628bfd-425e-450c-a51b-733435474eaa") } foo:PRIMARY> session.startTransaction() foo:PRIMARY> session.getDatabase("percona").test.find() foo:PRIMARY> |
Connection 1: commit
1 | foo:PRIMARY> session.commitTransaction() |
Connection 2: after connection1 commits:
1 2 3 | foo:PRIMARY> db.test.find() { "_id" : ObjectId("5b21361252bbe6e5b9a70a4e"), "today" : ISODate("2018-06-13T15:19:46.645Z") } { "_id" : ObjectId("5b21361252bbe6e5b9a70a4f"), "some_value" : "abc" } |
Outside of the session it sees the new values, however inside the opened session it will not see the new values.
1 2 | foo:PRIMARY> session.getDatabase("percona").test.find() foo:PRIMARY> |
Now if we commit the transaction inside connection 2 it will commit as well, and we will have 2 rows now (as there are no conflicts).
Sometimes, however, we may see the transient transaction error when committing or even doing find() inside a session:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | foo:PRIMARY> session.commitTransaction() 2018-06-14T21:56:29.111+0000 E QUERY [js] Error: command failed: { "errorLabels" : [ "TransientTransactionError" ], "operationTime" : Timestamp(1529013385, 1), "ok" : 0, "errmsg" : "Transaction 0 has been aborted.", "code" : 251, "codeName" : "NoSuchTransaction", "$clusterTime" : { "clusterTime" : Timestamp(1529013385, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } } : _getErrorWithCode@src/mongo/shell/utils.js:25:13 doassert@src/mongo/shell/assert.js:18:14 _assertCommandWorked@src/mongo/shell/assert.js:520:17 assert.commandWorked@src/mongo/shell/assert.js:604:16 commitTransaction@src/mongo/shell/session.js:878:17 @(shell):1:1 |
From the MongoDB doc we can read that we could retry the transaction back when we have this error.
If an operation encounters an error, the returned error may have an errorLabels array field. If the error is a transient error, the errorLabels array field contains “TransientTransactionError” as an element and the transaction as a whole can be retried.
MongoDB transactions: conflict
What about transaction conflicts in MongoDB? Let’s say we are updating the same row. Here is the demo:
First we create a record, trx, in the collection:
1 2 | use percona db.test.insert({trx : 0}) |
Then we create session1 and update trx to change from 0 to 1:
1 2 3 4 5 | foo:PRIMARY> session = db.getMongo().startSession() session { "id" : UUID("0b7b8ce0-919a-401a-af01-69fe90876301") } foo:PRIMARY> session.startTransaction() foo:PRIMARY> session.getDatabase("percona").test.update({trx : 0}, {trx: 1}) WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 }) |
Then (before committing) create another session which will try to change from 0 to 2:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | foo:PRIMARY> session = db.getMongo().startSession() session { "id" : UUID("b312c662-247c-47c5-b0c9-23d77f4e9f6d") } foo:PRIMARY> session.startTransaction() foo:PRIMARY> session.getDatabase("percona").test.update({trx : 0}, {trx: 2}) WriteCommandError({ "errorLabels" : [ "TransientTransactionError" ], "operationTime" : Timestamp(1529675754, 1), "ok" : 0, "errmsg" : "WriteConflict", "code" : 112, "codeName" : "WriteConflict", "$clusterTime" : { "clusterTime" : Timestamp(1529675754, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } }) |
As we can see, MongoDB catches the conflict and return the error on the insert (even before the commit).
We hope this post, with its simple example how transactions will work, has been useful.
You May Also Like
For more information on how multi-document ACID transactions are supported in MongoDB 4.0, watch our webinar, MongoDB 4.0 Features – Transactions & More. Other topics and features covered include future transaction improvements, non-blocking secondary reads and security improvements. We know data integrity is crucial for your business. Our blog, MongoDB: how to use the JSON Schema Validator, explains how to introduce schema validation checks at the database level to enforce the integrity of your data.
Does snapshot isolation mean this is similar to repeatable read in Postgres and MyRocks?
Gap locks as used by InnoDB can be confusing, but make RR more useful. For linkbench I need to use RC with MyRocks, while RR works fine with InnoDB (thanks to gap locks). I have yet to run it for Postgres, but assume that would have to use RC as well (because PG RR doesn’t do gap locks).
This article is very nice. I implemented multi-document transaction. Thanks for your help. But I’ve a tweaked requirement from my client. In this article, at the end you are updating same document from 2 sessions. While first one has updated the document but not committed, in the meanwhile one more session has already accessed it and while updating its getting “writeConflict” error. Seems ok from our end but the client wants that the 2nd session should not be able to pull the same document which is already in the previous session. We dont want all the codes and at last facing writeConflict error. We should get the error on prior pulling the document. Any help would be appreciated.
Does mysql support snapshot isolation??