Skip to content

[Lock] Re-add the Lock component in 3.4 #7866

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 7, 2017
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
258 changes: 258 additions & 0 deletions components/lock.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,258 @@
.. index::
single: Lock
single: Components; Lock

The Lock Component
==================

The Lock Component creates and manages `locks`_, a mechanism to provide
exclusive access to a shared resource.

.. versionadded:: 3.4
The Lock component was introduced in Symfony 3.4.

Installation
------------

You can install the component in 2 different ways:

* :doc:`Install it via Composer </components/using_components>` (``symfony/lock`` on `Packagist`_);
* Use the official Git repository (https://github.com/symfony/lock).

.. include:: /components/require_autoload.rst.inc

Usage
-----

Locks are used to guarantee exclusive access to some shared resource. In
Copy link

@NoiseByNorthwest NoiseByNorthwest Sep 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my experience in my company, we've spent lot of time and headaches using distributed locks for use cases where more reliable tools are best suited.
I mean:

  • for high concurrency use cases, atomic write operations are better when available.
  • for low concurrency use cases, OCC is better when available.

For example, to avoid concurrent business transactions on the same object within a DBMS, Doctrine's OCC is the best solution.

So it would be fine to explain it right here IMO.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, a chapter about other altenatives would be relevant. I will add it when this PR will be merged given this PR had already been reviewed.

But in my opinion, it really depend of the use case:

for high concurrency use cases, atomic write operations are better when available.

True for concurrency between read and write, but wrong for concurrency between writes.

for low concurrency use cases, OCC is better when available.

True when computing the changset does not affect other systems:

  • must be free (otherwise you pay twice to update the object) (ie. call to a billable API)
  • must be quick (in a scalable environment, it's close to the previous point given you pay for the CPU time)
  • other system can be rollbacked (ie. Don't call non-safe operation on third API, don't send email during the changset computation)

BTW, I am interested by what you said: we've spent lot of time and headaches. Can you give me exemples/use case and how this component can be improved to fix such issue?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True for concurrency between read and write, but wrong for concurrency between writes.

For both AFAIK. For example, when your business transaction simply consists in incrementing something in your data store, most DBMS (relational or not) does support this kind of atomic write. If several atomic writes are required, ACID-capable DBMS is required. This is applicable as long as the whole transaction can be executed on DBMS side.

True when computing the changset does not affect other systems:

Yes, not in all cases indeed, that's what I mean by saying "when available" (i.e. applicable).

BTW, I am interested by what you said: we've spent lot of time and headaches. Can you give me exemples/use case and how this component can be improved to fix such issue?

This component seems to be quite close from our own internal locking implementation. The main issue I've spotted (I may be wrong) is the one present here https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Lock/Store/CombinedStore.php#L88 as described here symfony/symfony#22132 (review) (lack of onwership tracking to catch program flaws).

Issues can also arise at infrastructure level, and some hints may be added in the doc. For example, about the Redis setup, the user has to be aware of the HA vs consistency trade-off with this kind of backend. If consistency (lock must works as expected or not be working) is more important than availability (lock is always working, whatever its correctness) then a single Redis instance with maximum durability and without replication (even synchronous) should be preferred AFAIK.

Some other minor points:

  • you provide a decorator which add blocking behavior to a non-blocking implementation, this looks great. But since this is implemented with a spinlock, the lack of fairness (no FIFO semantic) might be documented.
  • I don't get what kind of problem solves CombinedStore. Some examples would be great.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issues can also arise at infrastructure level, and some hints may be added in the doc

I'm working on an other PR to add a lot of tips/warning about reliability and weakness of each store.

I don't get what kind of problem solves CombinedStore

Combine several stores together to provide an implementation of the Redlock pattern and offer an highest availability.

Symfony applications, you can use locks for example to ensure that a command is
not executed more than once at the same time (on the same or different servers).

In order to manage the state of locks, a ``Store`` needs to be created first
and then use the :class:`Symfony\\Component\\Lock\\Factory` class to actually
create the lock for some resource::

use Symfony\Component\Lock\Factory;
use Symfony\Component\Lock\Store\SemaphoreStore;

$store = new SemaphoreStore();
$factory = new Factory($store);

Then, a call to the :method:`Symfony\\Component\\Lock\\LockInterface::acquire`
method will try to acquire the lock. Its first argument is an arbitrary string
that represents the locked resource::

// ...
$lock = $factory->createLock('pdf-invoice-generation');

if ($lock->acquire()) {
// The resource "pdf-invoice-generation" is locked.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since a lock is an unmanaged resource, there is an exeception safety issue right here. Despite this is not a bug in the lock component itself, you should promote exception safety in examples IMO.

It can be fixed with a simple try / finally

$lock = $factory->createLock('pdf-invoice-generation');
if ($lock->acquire()) {
    try {
        // ...
    } finally {
        $lock->release();
    }
}

You can even add an helper method to avoid this boilerplate

$lock = $factory->createLock('pdf-invoice-generation');

$lock->with(function() {
   // ... 
});

With the following implementation

public function with($callback)
{
    if (!$this->acquire()) {
        return false;
    }

    try {
        $callback();
    } finally {
        $this->release();
    }

    return true;
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, this chapter was covered in Blocking Locks but it's better to promote it everywhere.

// You can compute and generate invoice safely here.

$lock->release();
}

If the lock can not be acquired, the method returns ``false``. The ``acquire()``
method can be safely called repeatedly, even if the lock is already acquired.

.. note::

Unlike other implementations, the Lock Component distinguishes locks
instances even when they are created for the same resource. If a lock has
to be used by several services, they should share the same ``Lock`` instance
returned by the ``Factory::createLock`` method.

Blocking Locks
--------------

By default, when a lock cannot be acquired, the ``acquire`` method returns
``false`` immediately. To wait (indefinitely) until the lock
can be created, pass ``true`` as the argument of the ``acquire()`` method. This
is called a **blocking lock** because the execution of your application stops
until the lock is acquired.

Some of the built-in ``Store`` classes support this feature. When they don't,
they can be decorated with the ``RetryTillSaveStore`` class::

use Symfony\Component\Lock\Factory;
use Symfony\Component\Lock\Store\RedisStore;
use Symfony\Component\Lock\Store\RetryTillSaveStore;

$store = new RedisStore(new \Predis\Client('tcp://localhost:6379'));
$store = new RetryTillSaveStore($store);
$factory = new Factory($store);

$lock = $factory->createLock('notification-flush');
$lock->acquire(true);

Expiring Locks
--------------

Locks created remotely are difficult to manage because there is no way for the
remote ``Store`` to know if the locker process is still alive. Due to bugs,
fatal errors or segmentation faults, it cannot be guaranteed that ``release()``
method will be called, which would cause the resource to be locked infinitely.

The best solution in those cases is to create **expiring locks**, which are
released automatically after some amount of time has passed (called TTL for
*Time To Live*). This time, in seconds, is configured as the second argument of
the ``createLock()`` method. If needed, these locks can also be released early
with the ``release()`` method.

The trickiest part when working with expiring locks is choosing the right TTL.
If it's too short, other processes could acquire the lock before finishing the
job; it it's too long and the process crashes before calling the ``release()``
method, the resource will stay locked until the timeout::

// ...
// create an expiring lock that lasts 30 seconds
$lock = $factory->createLock('charts-generation', 30);

$lock->acquire();
try {
// perform a job during less than 30 seconds
} finally {
$lock->release();
}

.. tip::

To avoid letting the lock in a locking state, it's recommended to wrap the
job in a try/catch/finally block to always try to release the expiring lock.

In case of long-running tasks, it's better to start with a not too long TTL and
then use the :method:`Symfony\\Component\\Lock\\LockInterface::refresh` method
to reset the TTL to its original value::

// ...
$lock = $factory->createLock('charts-generation', 30);

$lock->acquire();
try {
while (!$finished) {
// perform a small part of the job.

// renew the lock for 30 more seconds.
$lock->refresh();
}
} finally {
$lock->release();
}

Available Stores
----------------

Locks are created and managed in ``Stores``, which are classes that implement
:class:`Symfony\\Component\\Lock\\StoreInterface`. The component includes the
following built-in store types:


============================================ ====== ======== ========
Store Scope Blocking Expiring
============================================ ====== ======== ========
:ref:`FlockStore <lock-store-flock>` local yes no
:ref:`MemcachedStore <lock-store-memcached>` remote no yes
:ref:`RedisStore <lock-store-redis>` remote no yes
:ref:`SemaphoreStore <lock-store-semaphore>` local yes no
============================================ ====== ======== ========

.. _lock-store-flock:

FlockStore
~~~~~~~~~~

The FlockStore uses the file system on the local computer to create the locks.
It does not support expiration, but the lock is automatically released when the
PHP process is terminated::

use Symfony\Component\Lock\Store\FlockStore;

// the argument is the path of the directory where the locks are created
$store = new FlockStore(sys_get_temp_dir());

.. caution::

Beware that some file systems (such as some types of NFS) do not support
locking. In those cases, it's better to use a directory on a local disk
drive or a remote store based on Redis or Memcached.

.. _lock-store-memcached:

MemcachedStore
~~~~~~~~~~~~~~

The MemcachedStore saves locks on a Memcached server, it requires a Memcached
connection implementing the ``\Memcached`` class. This store does not
support blocking, and expects a TTL to avoid stalled locks::

use Symfony\Component\Lock\Store\MemcachedStore;

$memcached = new \Memcached();
$memcached->addServer('localhost', 11211);

$store = new MemcachedStore($memcached);

.. note::

Memcached does not support TTL lower than 1 second.

.. _lock-store-redis:

RedisStore
~~~~~~~~~~

The RedisStore saves locks on a Redis server, it requires a Redis connection
implementing the ``\Redis``, ``\RedisArray``, ``\RedisCluster`` or
``\Predis`` classes. This store does not support blocking, and expects a TTL to
avoid stalled locks::

use Symfony\Component\Lock\Store\RedisStore;

$redis = new \Redis();
$redis->connect('localhost');

$store = new RedisStore($redis);

.. _lock-store-semaphore:

SemaphoreStore
~~~~~~~~~~~~~~

The SemaphoreStore uses the `PHP semaphore functions`_ to create the locks::

use Symfony\Component\Lock\Store\SemaphoreStore;

$store = new SemaphoreStore();

.. _lock-store-combined:

CombinedStore
~~~~~~~~~~~~~

The CombinedStore is designed for High Availability applications because it
manages several stores in sync (for example, several Redis servers). When a lock
is being acquired, it forwards the call to all the managed stores, and it
collects their responses. If a simple majority of stores have acquired the lock,
then the lock is considered as acquired; otherwise as not acquired::

use Symfony\Component\Lock\Strategy\ConsensusStrategy;
use Symfony\Component\Lock\Store\CombinedStore;
use Symfony\Component\Lock\Store\RedisStore;

$stores = [];
foreach (array('server1', 'server2', 'server3') as $server) {
$redis= new \Redis();
$redis->connect($server);

$stores[] = new RedisStore($redis);
}

$store = new CombinedStore($stores, new ConsensusStrategy());

Instead of the simple majority strategy (``ConsensusStrategy``) an
``UnanimousStrategy`` can be used to require the lock to be acquired in all
the stores.

.. _`locks`: https://en.wikipedia.org/wiki/Lock_(computer_science)
.. _Packagist: https://packagist.org/packages/symfony/lock
.. _`PHP semaphore functions`: http://php.net/manual/en/book.sem.php