-
-
Notifications
You must be signed in to change notification settings - Fork 5.2k
[Lock] Re-add the Lock component in 3.4 #7866
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,258 @@ | ||
.. index:: | ||
single: Lock | ||
single: Components; Lock | ||
|
||
The Lock Component | ||
================== | ||
|
||
The Lock Component creates and manages `locks`_, a mechanism to provide | ||
exclusive access to a shared resource. | ||
|
||
.. versionadded:: 3.4 | ||
The Lock component was introduced in Symfony 3.4. | ||
|
||
Installation | ||
------------ | ||
|
||
You can install the component in 2 different ways: | ||
|
||
* :doc:`Install it via Composer </components/using_components>` (``symfony/lock`` on `Packagist`_); | ||
* Use the official Git repository (https://github.com/symfony/lock). | ||
|
||
.. include:: /components/require_autoload.rst.inc | ||
|
||
Usage | ||
----- | ||
|
||
Locks are used to guarantee exclusive access to some shared resource. In | ||
Symfony applications, you can use locks for example to ensure that a command is | ||
not executed more than once at the same time (on the same or different servers). | ||
|
||
In order to manage the state of locks, a ``Store`` needs to be created first | ||
and then use the :class:`Symfony\\Component\\Lock\\Factory` class to actually | ||
create the lock for some resource:: | ||
|
||
use Symfony\Component\Lock\Factory; | ||
use Symfony\Component\Lock\Store\SemaphoreStore; | ||
|
||
$store = new SemaphoreStore(); | ||
$factory = new Factory($store); | ||
|
||
Then, a call to the :method:`Symfony\\Component\\Lock\\LockInterface::acquire` | ||
method will try to acquire the lock. Its first argument is an arbitrary string | ||
that represents the locked resource:: | ||
|
||
// ... | ||
$lock = $factory->createLock('pdf-invoice-generation'); | ||
|
||
if ($lock->acquire()) { | ||
// The resource "pdf-invoice-generation" is locked. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since a lock is an unmanaged resource, there is an exeception safety issue right here. Despite this is not a bug in the lock component itself, you should promote exception safety in examples IMO. It can be fixed with a simple try / finally $lock = $factory->createLock('pdf-invoice-generation');
if ($lock->acquire()) {
try {
// ...
} finally {
$lock->release();
}
} You can even add an helper method to avoid this boilerplate $lock = $factory->createLock('pdf-invoice-generation');
$lock->with(function() {
// ...
}); With the following implementation public function with($callback)
{
if (!$this->acquire()) {
return false;
}
try {
$callback();
} finally {
$this->release();
}
return true;
} There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're right, this chapter was covered in |
||
// You can compute and generate invoice safely here. | ||
|
||
$lock->release(); | ||
} | ||
|
||
If the lock can not be acquired, the method returns ``false``. The ``acquire()`` | ||
method can be safely called repeatedly, even if the lock is already acquired. | ||
|
||
.. note:: | ||
|
||
Unlike other implementations, the Lock Component distinguishes locks | ||
instances even when they are created for the same resource. If a lock has | ||
to be used by several services, they should share the same ``Lock`` instance | ||
returned by the ``Factory::createLock`` method. | ||
|
||
Blocking Locks | ||
-------------- | ||
|
||
By default, when a lock cannot be acquired, the ``acquire`` method returns | ||
``false`` immediately. To wait (indefinitely) until the lock | ||
can be created, pass ``true`` as the argument of the ``acquire()`` method. This | ||
is called a **blocking lock** because the execution of your application stops | ||
until the lock is acquired. | ||
|
||
Some of the built-in ``Store`` classes support this feature. When they don't, | ||
they can be decorated with the ``RetryTillSaveStore`` class:: | ||
|
||
use Symfony\Component\Lock\Factory; | ||
use Symfony\Component\Lock\Store\RedisStore; | ||
use Symfony\Component\Lock\Store\RetryTillSaveStore; | ||
|
||
$store = new RedisStore(new \Predis\Client('tcp://localhost:6379')); | ||
$store = new RetryTillSaveStore($store); | ||
$factory = new Factory($store); | ||
|
||
$lock = $factory->createLock('notification-flush'); | ||
$lock->acquire(true); | ||
|
||
Expiring Locks | ||
-------------- | ||
|
||
Locks created remotely are difficult to manage because there is no way for the | ||
remote ``Store`` to know if the locker process is still alive. Due to bugs, | ||
fatal errors or segmentation faults, it cannot be guaranteed that ``release()`` | ||
method will be called, which would cause the resource to be locked infinitely. | ||
|
||
The best solution in those cases is to create **expiring locks**, which are | ||
released automatically after some amount of time has passed (called TTL for | ||
*Time To Live*). This time, in seconds, is configured as the second argument of | ||
the ``createLock()`` method. If needed, these locks can also be released early | ||
with the ``release()`` method. | ||
|
||
The trickiest part when working with expiring locks is choosing the right TTL. | ||
If it's too short, other processes could acquire the lock before finishing the | ||
job; it it's too long and the process crashes before calling the ``release()`` | ||
method, the resource will stay locked until the timeout:: | ||
|
||
// ... | ||
// create an expiring lock that lasts 30 seconds | ||
$lock = $factory->createLock('charts-generation', 30); | ||
|
||
$lock->acquire(); | ||
try { | ||
// perform a job during less than 30 seconds | ||
} finally { | ||
$lock->release(); | ||
} | ||
|
||
.. tip:: | ||
|
||
To avoid letting the lock in a locking state, it's recommended to wrap the | ||
job in a try/catch/finally block to always try to release the expiring lock. | ||
|
||
In case of long-running tasks, it's better to start with a not too long TTL and | ||
then use the :method:`Symfony\\Component\\Lock\\LockInterface::refresh` method | ||
to reset the TTL to its original value:: | ||
|
||
// ... | ||
$lock = $factory->createLock('charts-generation', 30); | ||
|
||
$lock->acquire(); | ||
try { | ||
while (!$finished) { | ||
// perform a small part of the job. | ||
|
||
// renew the lock for 30 more seconds. | ||
$lock->refresh(); | ||
} | ||
} finally { | ||
$lock->release(); | ||
} | ||
|
||
Available Stores | ||
---------------- | ||
|
||
Locks are created and managed in ``Stores``, which are classes that implement | ||
:class:`Symfony\\Component\\Lock\\StoreInterface`. The component includes the | ||
following built-in store types: | ||
|
||
|
||
============================================ ====== ======== ======== | ||
Store Scope Blocking Expiring | ||
============================================ ====== ======== ======== | ||
:ref:`FlockStore <lock-store-flock>` local yes no | ||
:ref:`MemcachedStore <lock-store-memcached>` remote no yes | ||
:ref:`RedisStore <lock-store-redis>` remote no yes | ||
:ref:`SemaphoreStore <lock-store-semaphore>` local yes no | ||
============================================ ====== ======== ======== | ||
|
||
.. _lock-store-flock: | ||
|
||
FlockStore | ||
~~~~~~~~~~ | ||
|
||
The FlockStore uses the file system on the local computer to create the locks. | ||
It does not support expiration, but the lock is automatically released when the | ||
PHP process is terminated:: | ||
|
||
use Symfony\Component\Lock\Store\FlockStore; | ||
|
||
// the argument is the path of the directory where the locks are created | ||
$store = new FlockStore(sys_get_temp_dir()); | ||
|
||
.. caution:: | ||
|
||
Beware that some file systems (such as some types of NFS) do not support | ||
locking. In those cases, it's better to use a directory on a local disk | ||
drive or a remote store based on Redis or Memcached. | ||
|
||
.. _lock-store-memcached: | ||
|
||
MemcachedStore | ||
~~~~~~~~~~~~~~ | ||
|
||
The MemcachedStore saves locks on a Memcached server, it requires a Memcached | ||
connection implementing the ``\Memcached`` class. This store does not | ||
support blocking, and expects a TTL to avoid stalled locks:: | ||
|
||
use Symfony\Component\Lock\Store\MemcachedStore; | ||
|
||
$memcached = new \Memcached(); | ||
$memcached->addServer('localhost', 11211); | ||
|
||
$store = new MemcachedStore($memcached); | ||
|
||
.. note:: | ||
|
||
Memcached does not support TTL lower than 1 second. | ||
|
||
.. _lock-store-redis: | ||
|
||
RedisStore | ||
~~~~~~~~~~ | ||
|
||
The RedisStore saves locks on a Redis server, it requires a Redis connection | ||
implementing the ``\Redis``, ``\RedisArray``, ``\RedisCluster`` or | ||
``\Predis`` classes. This store does not support blocking, and expects a TTL to | ||
avoid stalled locks:: | ||
|
||
use Symfony\Component\Lock\Store\RedisStore; | ||
|
||
$redis = new \Redis(); | ||
$redis->connect('localhost'); | ||
|
||
$store = new RedisStore($redis); | ||
|
||
.. _lock-store-semaphore: | ||
|
||
SemaphoreStore | ||
~~~~~~~~~~~~~~ | ||
|
||
The SemaphoreStore uses the `PHP semaphore functions`_ to create the locks:: | ||
|
||
use Symfony\Component\Lock\Store\SemaphoreStore; | ||
|
||
$store = new SemaphoreStore(); | ||
|
||
.. _lock-store-combined: | ||
|
||
CombinedStore | ||
~~~~~~~~~~~~~ | ||
|
||
The CombinedStore is designed for High Availability applications because it | ||
manages several stores in sync (for example, several Redis servers). When a lock | ||
is being acquired, it forwards the call to all the managed stores, and it | ||
collects their responses. If a simple majority of stores have acquired the lock, | ||
then the lock is considered as acquired; otherwise as not acquired:: | ||
|
||
use Symfony\Component\Lock\Strategy\ConsensusStrategy; | ||
use Symfony\Component\Lock\Store\CombinedStore; | ||
use Symfony\Component\Lock\Store\RedisStore; | ||
|
||
$stores = []; | ||
foreach (array('server1', 'server2', 'server3') as $server) { | ||
$redis= new \Redis(); | ||
$redis->connect($server); | ||
|
||
$stores[] = new RedisStore($redis); | ||
} | ||
|
||
$store = new CombinedStore($stores, new ConsensusStrategy()); | ||
|
||
Instead of the simple majority strategy (``ConsensusStrategy``) an | ||
``UnanimousStrategy`` can be used to require the lock to be acquired in all | ||
the stores. | ||
|
||
.. _`locks`: https://en.wikipedia.org/wiki/Lock_(computer_science) | ||
.. _Packagist: https://packagist.org/packages/symfony/lock | ||
.. _`PHP semaphore functions`: http://php.net/manual/en/book.sem.php |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on my experience in my company, we've spent lot of time and headaches using distributed locks for use cases where more reliable tools are best suited.
I mean:
For example, to avoid concurrent business transactions on the same object within a DBMS, Doctrine's OCC is the best solution.
So it would be fine to explain it right here IMO.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed, a chapter about other altenatives would be relevant. I will add it when this PR will be merged given this PR had already been reviewed.
But in my opinion, it really depend of the use case:
True for concurrency between read and write, but wrong for concurrency between writes.
True when computing the changset does not affect other systems:
BTW, I am interested by what you said:
we've spent lot of time and headaches
. Can you give me exemples/use case and how this component can be improved to fix such issue?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For both AFAIK. For example, when your business transaction simply consists in incrementing something in your data store, most DBMS (relational or not) does support this kind of atomic write. If several atomic writes are required, ACID-capable DBMS is required. This is applicable as long as the whole transaction can be executed on DBMS side.
Yes, not in all cases indeed, that's what I mean by saying "when available" (i.e. applicable).
This component seems to be quite close from our own internal locking implementation. The main issue I've spotted (I may be wrong) is the one present here https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Lock/Store/CombinedStore.php#L88 as described here symfony/symfony#22132 (review) (lack of onwership tracking to catch program flaws).
Issues can also arise at infrastructure level, and some hints may be added in the doc. For example, about the Redis setup, the user has to be aware of the HA vs consistency trade-off with this kind of backend. If consistency (lock must works as expected or not be working) is more important than availability (lock is always working, whatever its correctness) then a single Redis instance with maximum durability and without replication (even synchronous) should be preferred AFAIK.
Some other minor points:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm working on an other PR to add a lot of tips/warning about reliability and weakness of each store.
Combine several stores together to provide an implementation of the Redlock pattern and offer an highest availability.