1. Principle of distributed lock implemented by Redis

Why distributed lock is needed

Before talking about distributed lock, it is necessary to explain why it is neededDistributed lock.

Compared with distributed locks, stand-alone locks. When writing multi-threaded programs, we avoid data problems caused by operating a shared variable at the same time. We usually use a lock to mutually exclude each other to ensure the correctness of shared variables. property, its scope of use is within the same process. If there are multiple processes that need to operate a shared resource at the same time, how can they be mutually exclusive? Today's business applications are usually microservice architecture, which also means that one application will deploy multiple processes. If multiple processes need to modify the same row of records in MySQL, in order to avoid dirty data caused by out-of-order operations, distribution needs to be introduced at this time. The style is locked.

#If you want to implement distributed locks, you must use an external system. All processes go to this system to apply for locks. This external system must be mutually exclusive, that is, if two requests arrive at the same time, the system will only successfully lock one process, and the other process will fail. This external system can be a database, Redis or Zookeeper, but in order to pursue performance, we usually choose to use Redis or Zookeeper.

Redis can be used as a shared storage system, and multiple clients can share access, so it can be used to save distributed locks. Moreover, Redis has high read and write performance and can handle high-concurrency lock operation scenarios. The focus of this article is to introduce how to use Redis to implement distributed locks, and discuss the problems that may be encountered during the implementation process.

How to implement distributed locks

As a shared storage system in the implementation of distributed locks, Redis can use key-value pairs to save lock variables and receive and process them. Operation requests for locking and releasing locks sent by different clients. So, how are the key and value of the key-value pair determined? We need to give the lock variable a variable name and use this variable name as the key of the key-value pair, and the value of the lock variable is the value of the key-value pair. In this way, Redis can save the lock variable, and the client can Lock operations can be implemented through Redis command operations.

To implement distributed locks, Redis must have mutual exclusion capabilities. You can use the SETNX command, which means SET IF NOT EXIST, that is, if the key does not exist, its value will be set, otherwise nothing will be done. A distributed lock is implemented by having two client processes execute the command mutually exclusive.

The following shows the operation process of Redis using key/value pairs to save lock variables and two clients requesting locks at the same time.

#After the locking operation is completed, the client that has successfully locked can operate the shared resources, for example, modify a certain row of data in MySQL. After the operation is completed, the lock must be released in time to give latecomers the opportunity to operate the shared resources. How to release the lock? Just use the DEL command to delete this key. The logic is very simple. The overall process written in pseudo code is as follows.

// 加鎖
SETNX lock_key 1
// 業(yè)務(wù)邏輯
DO THINGS
// 釋放鎖
DEL lock_key

However, there is a big problem in the above implementation. When client 1 gets the lock, if the following scenario occurs, a deadlock will occur.

The program handles business logic exceptions and fails to release the lock in time. The process hangs and has no chance to release the lock.

The above situation will cause the client that has obtained the lock to occupy the lock forever, and other clients will never be able to obtain it. to the lock.

How to avoid deadlock

In order to solve the above deadlock problem, the easiest solution to think of is to set a lock for the lock when applying for a lock and implementing it in Redis. Expiration time, assuming that the time to operate the shared resource will not exceed 10 seconds, then when locking, just set the expiration time of 10 seconds for this key.

But there are still problems with the above operations. There are two commands to lock and set the expiration time. It is possible that only the first one is executed, but the second one fails to execute, for example:

1. SETNX was executed successfully, but EXPIRE failed due to network problems.
2. SETNX was executed successfully, but Redis crashed abnormally, and EXPIRE had no chance to execute.
3. SETNX was executed successfully, and the customer The terminal crashed abnormally, and EXPIRE had no chance to execute

In short, if these two commands cannot be guaranteed to be atomic operations, there is a potential risk that the expiration time setting will fail, and deadlock problems may still occur. . Fortunately, after Redis 2.6.12, Redis has expanded the parameters of the SET command. You can specify the EXPIRE time at the same time as SET. This operation is atomic. For example, the following command sets the lock expiration time to 10 seconds.

SET lock_key 1 EX 10 NX

So far, the deadlock problem has been solved, but there are still other problems. Imagine the following scenario:

1. Client 1 is locked successfully and starts operating shared resources

2. 客戶端1操作共享資源耗時太久，超過了鎖的過期時間，鎖失效（鎖被自動釋放）

3. 客戶端2加鎖成功，開始操作共享資源

4. 客戶端1操作共享資源完成，在finally塊中手動釋放鎖，但此時它釋放的是客戶端2的鎖。

這里存在兩個嚴(yán)重的問題：

• 鎖過期

• 釋放了別人的鎖

第1個問題是評估操作共享資源的時間不準(zhǔn)確導(dǎo)致的，如果只是一味增大過期時間，只能緩解問題降低出現(xiàn)問題的概率，依舊無法徹底解決問題。原因在于客戶端在拿到鎖之后，在操作共享資源時，遇到的場景是很復(fù)雜的，既然是預(yù)估的時間，也只能是大致的計算，不可能覆蓋所有導(dǎo)致耗時變長的場景。

第二個問題在于解鎖操作是不夠嚴(yán)謹(jǐn)?shù)?，因為它是一種不加區(qū)分地釋放鎖的操作，沒有對鎖的所有權(quán)進行檢查。如何解決呢？

鎖被別人給釋放了

解決辦法是，客戶端在加鎖時，設(shè)置一個只有自己知道的唯一標(biāo)識進去，例如可以是自己的線程ID，如果是redis實現(xiàn)，就是SET key unique_value EX 10 NX。之后在釋放鎖時，要先判斷這把鎖是否歸自己持有，只有是自己的才能釋放它。

//釋放鎖 比較unique_value是否相等，避免誤釋放
if redis.get("key") == unique_value then
    return redis.del("key")

這里釋放鎖使用的是GET + DEL兩條命令，這時又會遇到原子性問題了。

1. 客戶端1執(zhí)行GET，判斷鎖是自己的

2. 客戶端2執(zhí)行了SET命令，強制獲取到鎖（雖然發(fā)生概念很低，但要嚴(yán)謹(jǐn)考慮鎖的安全性）

3. 客戶端1執(zhí)行DEL，卻釋放了客戶端2的鎖

由此可見，以上GET + DEL兩個命令還是必須原子的執(zhí)行才行。怎樣原子執(zhí)行兩條命令呢？答案是Lua腳本，可以把以上邏輯寫成Lua腳本，讓Redis執(zhí)行。因為Redis處理每個請求是單線程執(zhí)行的，在執(zhí)行一個Lua腳本時其它請求必須等待，直到這個Lua腳本處理完成，這樣一來GET+DEL之間就不會有其他命令執(zhí)行了。

以下是使用Lua腳本（unlock.script）實現(xiàn)的釋放鎖操作的偽代碼，其中，KEYS[1]表示lock_key，ARGV[1]是當(dāng)前客戶端的唯一標(biāo)識，這兩個值都是我們在執(zhí)行 Lua腳本時作為參數(shù)傳入的。

//Lua腳本語言，釋放鎖 比較unique_value是否相等，避免誤釋放
if redis.call("get",KEYS[1]) == ARGV[1] then
    return redis.call("del",KEYS[1])
else
    return 0
end

最后我們執(zhí)行以下命令，即可

redis-cli  --eval  unlock.script lock_key , unique_value

這樣一路優(yōu)先下來，整個加鎖、解鎖流程就更嚴(yán)謹(jǐn)了，先小結(jié)一下，基于Redis實現(xiàn)的分布式鎖，一個嚴(yán)謹(jǐn)?shù)牧鞒倘缦拢?/p>

1. 加鎖時要設(shè)置過期時間SET lock_key unique_value EX expire_time NX

2. 操作共享資源

3. 釋放鎖：Lua腳本，先GET判斷鎖是否歸屬自己，再DEL釋放鎖

有了這個嚴(yán)謹(jǐn)?shù)逆i模型，我們還需要重新思考之前的那個問題，鎖的過期時間不好評估怎么辦。

如何確定鎖的過期時間

前面提到過，過期時間如果評估得不好，這個鎖就會有提前過期的風(fēng)險，一種妥協(xié)的解決方案是，盡量冗余過期時間，降低鎖提前過期的概率，但這個方案并不能完美解決問題。是否可以設(shè)置這樣的方案，加鎖時，先設(shè)置一個預(yù)估的過期時間，然后開啟一個守護線程，定時去檢測這個鎖的失效時間，如果鎖快要過期了，操作共享資源還未完成，那么就自動對鎖進行續(xù)期，重新設(shè)置過期時間。

Redisson是一個已封裝好這些工作的庫，可以說是一種非常優(yōu)秀的解決方案。Redisson是一個Java語言實現(xiàn)的Redis SDK客戶端，在使用分布式鎖時，它就采用了自動續(xù)期的方案來避免鎖過期，這個守護線程我們一般叫它看門狗線程。這個SDK提供的API非常友好，它可以像操作本地鎖一樣操作分布式鎖?？蛻舳艘坏┘渔i成功，就會啟動一個watch dog看門狗線程，它是一個后臺線程，會每隔一段時間（這段時間的長度與設(shè)置的鎖的過期時間有關(guān)）檢查一下，如果檢查時客戶端還持有鎖key（也就是說還在操作共享資源），那么就會延長鎖key的生存時間。

那如果客戶端在加鎖成功后就宕機了呢？宕機了那么看門狗任務(wù)就不存在了，也就無法為鎖續(xù)期了，鎖到期自動失效。

Redis的部署方式對鎖的影響

上面討論的情況，都是鎖在單個Redis 實例中可能產(chǎn)生的問題，并沒有涉及到Redis的部署架構(gòu)細(xì)節(jié)。

Redis發(fā)展到現(xiàn)在，幾種常見的部署架構(gòu)有：

• Single mode;

• Master-slave mode;

• Sentinel mode;

• Cluster mode;

When we use Redis,generally deploy it in the master-slave cluster sentinel mode. The role of the sentinel is to monitor the redis node. Operating status. In the ordinary master-slave mode, when the master crashes, you need to manually switch to make the slave the master. The advantage of using the master-slave sentinel combination is that when the master crashes abnormally, the sentinel can implement automatic failover and promote the slave to the new master. Availability is guaranteed by continuing to provide services. So when the master-slave switch occurs, is the distributed lock still safe?

Imagine this scenario:

1. Client 1 executes the SET command on the master and the lock is successful

2. At this time, the master is down abnormally, and the SET command has not yet been synchronized to the slave (master-slave replication is asynchronous)

3. The sentinel promotes the slave to a new one master, but the lock was lost on the new master, causing client 2 to successfully lock , distributed locks may still be affected. Even if Redis ensures high availability through sentinel, if the master node switches master-slave for some reason, the lock will be lost.

Cluster mode Redlock implements highly reliable distributed locks

In order to avoid the problem of lock failure caused by Redis instance failure, Redis developer Antirez proposed distribution Formula lock algorithm Redlock. The basic idea of ??the Redlock algorithm is to let the client and multiple independent Redis instances request locks in sequence. If the client can successfully complete the locking operation with more than half of the instances, then we consider that the client has successfully The distributed lock is obtained, otherwise the lock fails. In this way, even if a single Redis instance fails, because the lock variables are also saved on other instances, the client can still perform lock operations normally and the lock variables will not be lost.

Let’s take a closer look at the execution steps of the Redlock algorithm. The implementation of the Redlock algorithm requires Redis to adopt cluster deployment mode, without sentinel nodes, and N independent Redis instances (officially recommended at least 5 instances). Next, we can complete the locking operation in 3 steps.

The first step is for the client to obtain the current time.

The second step is for the client to perform locking operations on N Redis instances in sequence.

The locking operation here is the same as the locking operation performed on a single instance. Use the SET command with the NX, EX/PX options, and the unique identifier of the client. Of course, if a Redis instance fails, in order to ensure that the Redlock algorithm can continue to run in this case, we need to set a timeout for the locking operation. If the client fails to request a lock with a Redis instance until the timeout, then at this time, the client will continue to request a lock with the next Redis instance. Generally, it is necessary to set the timeout of the lock operation to a small part of the effective time of the lock, usually about tens of milliseconds.

The third step is that once the client completes the locking operation with all Redis instances, the client must calculate the total time spent on the entire locking process.

The client can only consider the lock to be successful when two conditions are met. Condition 1 is that the client has successfully obtained the lock from more than half (greater than or equal to N/2 1) of the Redis instances. ;The second condition is that the total time spent by the client in acquiring the lock does not exceed the effective time of the lock.

Why can the operation be considered successful only when most instances are successfully locked? In fact, multiple Redis instances are used together to form a distributed system. There will always be abnormal nodes in a distributed system, so when talking about a distributed system, you need to consider how many abnormal nodes there are without affecting the correct operation of the entire system. This is a fault tolerance problem in a distributed system. The conclusion of this problem is: if there are only faulty nodes, as long as most nodes are normal, the entire system can still provide correct services.

After meeting these two conditions, we need torecalculate the effective time of the lock. The result of the calculation is the initial effective time of the lock minus the total time spent by the client to obtain the lock. If the lock's validity time is too late to complete the shared data operation, we can release the lock to avoid the situation where the lock expires before the shared resource operation is completed

.

當(dāng)然，如果客戶端在和所有實例執(zhí)行完加鎖操作后，沒能同時滿足這兩個條件，那么，客戶端就要向所有Redis節(jié)點發(fā)起釋放鎖的操作。為什么釋放鎖，要操作所有的節(jié)點呢，不能只操作那些加鎖成功的節(jié)點嗎？因為在某一個Redis節(jié)點加鎖時，可能因為網(wǎng)絡(luò)原因?qū)е录渔i失敗，例如一個客戶端在一個Redis實例上加鎖成功，但在讀取響應(yīng)結(jié)果時由于網(wǎng)絡(luò)問題導(dǎo)致讀取失敗，那這把鎖其實已經(jīng)在Redis上加鎖成功了。所以釋放鎖時，不管之前有沒有加鎖成功，需要釋放所有節(jié)點上的鎖以保證清理節(jié)點上的殘留的鎖。

在Redlock算法中，釋放鎖的操作和在單實例上釋放鎖的操作一樣，只要執(zhí)行釋放鎖的 Lua腳本就可以了。如果N個Redis實例中超過一半的實例正常工作，就能確保分布式鎖正常運作。為了提高分布式鎖的可靠性，您可以在實際業(yè)務(wù)應(yīng)用中使用Redlock算法。

二、代碼實現(xiàn)Redis分布式鎖

1.SpringBoot整合redis用到最多的當(dāng)然屬于我們的老朋友RedisTemplate，pom依賴如下：

<!-- springboot整合redis -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

2.Redis配置類:

package com.example.redisdemo.config;

import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;

/**
 * @description: Redis配置類
 * @author Keson
 * @date 21:20 2022/11/14
 * @Param
 * @return
 * @version 1.0
 */
@Configuration
public class RedisConfig {

    @Bean
    public RedisTemplate<String, Object> redisTemplate(LettuceConnectionFactory lettuceConnectionFactory) {
        // 設(shè)置序列化
        Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<Object>(Object.class);
        ObjectMapper om = new ObjectMapper();
        om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
        om.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        jackson2JsonRedisSerializer.setObjectMapper(om);
        // 配置redisTemplate
        RedisTemplate<String, Object> redisTemplate = new RedisTemplate<String, Object>();
        redisTemplate.setConnectionFactory(lettuceConnectionFactory);
        RedisSerializer<?> stringSerializer = new StringRedisSerializer();
        redisTemplate.setKeySerializer(stringSerializer);// key序列化
        redisTemplate.setValueSerializer(jackson2JsonRedisSerializer);// value序列化
        redisTemplate.setHashKeySerializer(stringSerializer);// Hash key序列化
        redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer);// Hash value序列化
        redisTemplate.afterPropertiesSet();
        return redisTemplate;
    }
}

3.Service層面

package com.example.redisdemo.service;

import com.example.redisdemo.entity.CustomerBalance;
import java.util.concurrent.Callable;

/**
 * @author Keson
 * @version 1.0
 * @description: TODO
 * @date 2022/11/14 15:12
 */
public interface RedisService {

    <T> T callWithLock(CustomerBalance customerBalance, Callable<T> callable) throws Exception;
}

package com.example.redisdemo.service.impl;

import com.example.redisdemo.entity.CustomerBalance;
import com.example.redisdemo.service.RedisService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.connection.RedisStringCommands;
import org.springframework.data.redis.connection.ReturnType;
import org.springframework.data.redis.core.RedisCallback;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.types.Expiration;
import org.springframework.stereotype.Service;
import java.nio.charset.StandardCharsets;
import java.util.UUID;
import java.util.concurrent.Callable;
import java.util.concurrent.TimeUnit;

/**
 * @author Keson
 * @version 1.0
 * @description: TODO Redis實現(xiàn)分布式鎖
 * @date 2022/11/14 15:13
 */
@Service
@Slf4j
public class RedisServiceImpl implements RedisService {

    //設(shè)置默認(rèn)過期時間
    private final static int DEFAULT_LOCK_EXPIRY_TIME = 20;
    //自定義lock key前綴
    private final static String LOCK_PREFIX = "LOCK:CUSTOMER_BALANCE";

    @Autowired
    private RedisTemplate redisTemplate;

    @Override
    public <T> T callWithLock(CustomerBalance customerBalance, Callable<T> callable) throws Exception{
        //自定義lock key
        String lockKey = getLockKey(customerBalance.getCustomerNumber(), customerBalance.getSubAccountNumber(), customerBalance.getCurrencyCode());
        //將UUID當(dāng)做value，確保唯一性
        String lockReference = UUID.randomUUID().toString();

        try {
            if (!lock(lockKey, lockReference, DEFAULT_LOCK_EXPIRY_TIME, TimeUnit.SECONDS)) {
                throw new Exception("lock加鎖失敗");
            }
            return callable.call();
        } finally {
            unlock(lockKey, lockReference);
        }
    }

    //定義lock key
    String getLockKey(String customerNumber, String subAccountNumber, String currencyCode) {
        return String.format("%s:%s:%s:%s", LOCK_PREFIX, customerNumber, subAccountNumber, currencyCode);
    }

    //redis加鎖
    private boolean lock(String key, String value, long timeout, TimeUnit timeUnit) {
        Boolean locked;
        try {
            //SET_IF_ABSENT --> NX: Only set the key if it does not already exist.
            //SET_IF_PRESENT --> XX: Only set the key if it already exist.
            locked = (Boolean) redisTemplate.execute((RedisCallback<Boolean>) connection ->
                    connection.set(key.getBytes(StandardCharsets.UTF_8), value.getBytes(StandardCharsets.UTF_8),
                            Expiration.from(timeout, timeUnit), RedisStringCommands.SetOption.SET_IF_ABSENT));
        } catch (Exception e) {
            log.error("Lock failed for redis key: {}, value: {}", key, value);
            locked = false;
        }
        return locked != null && locked;
    }

    //redis解鎖
    private boolean unlock(String key, String value) {
        try {
            //使用lua腳本保證刪除的原子性，確保解鎖
            String script = "if redis.call(&#39;get&#39;, KEYS[1]) == ARGV[1] " +
                            "then return redis.call(&#39;del&#39;, KEYS[1]) " +
                            "else return 0 end";
            Boolean unlockState = (Boolean) redisTemplate.execute((RedisCallback<Boolean>) connection ->
                    connection.eval(script.getBytes(), ReturnType.BOOLEAN, 1,
                            key.getBytes(StandardCharsets.UTF_8), value.getBytes(StandardCharsets.UTF_8)));
            return unlockState == null || !unlockState;
        } catch (Exception e) {
            log.error("unLock failed for redis key: {}, value: {}", key, value);
            return false;
        }
    }
}

4.業(yè)務(wù)調(diào)用實現(xiàn)分布式鎖示例：

    @Override
    public int updateById(CustomerBalance customerBalance) throws Exception {
        return redisService.callWithLock(customerBalance, ()-> customerBalanceMapper.updateById(customerBalance));
    }

The above is the detailed content of How to use Redis to implement distributed locks in SpringBoot. For more information, please follow other related articles on the PHP Chinese website!