Therefore, there must be a mechanism to ensure that the data of Redis will not be lost because of failure. This mechanism is the persistence mechanism of Redis. (Recommended learning: Redis video tutorial)
Redis has two persistence mechanisms, the first is snapshot, and the second is AOF log. Snapshots are full backups and AOF logs are continuous incremental backups. Snapshot is a binary serialized form of memory data, which is very compact in storage, while AOF log records the instruction record text of memory data modification. In the long-term running process, AOF logs will become extremely large. When the database is restarted, it needs to be loaded for instruction replay, which will take a long time. Therefore, it is necessary to rewrite the AOF log regularly to slim down the AOF log.
Snapshot principle
We know that Redis is a single-threaded program, which is responsible for the simultaneous reading and writing of multiple client sockets and the logical reading and writing of memory data structures.
While serving online requests, Redis also needs to take a memory snapshot, which requires Redis to perform file IO operations, but file IO operations cannot use multiplexing API.
This means that a single thread must perform file IO operations while serving online requests, which will seriously drag down the performance of server requests.
There is another important question. In order not to block online business, Redis needs to respond to client requests while insisting. While persisting, the data structure in memory is still changing. For example, a large hash dictionary is being persisted, and a request is made to delete it, but it has not been persisted. What should I do?
Redis uses the multi-process COW (copy-on-write) mechanism of the operating system to achieve snapshot persistence, which is very interesting and little known.
AOF principle
AOF log stores the sequential instruction sequence of Redis server, and AOF log only records the instruction record of modifying memory.
Assuming that the AOF log records all modified instruction sequences since the creation of the Redis instance, the state of the memory data structure of the current instance of Redis can be restored by sequentially executing all instructions on an empty Redis instance, that is, "replay".
After receiving the client modification instruction, Redis will perform parameter verification and logic processing. If there is no problem, it will immediately store the instruction text in the AOF log, that is, the instruction will be executed before the log is saved. This is different from storage engines such as leveldb and hbase, which all store logs first and then do logical processing.
During the long-term operation of Redis, the log of AOF will become longer and longer. If the instance is down and restarted, it will be very time-consuming to replay the entire AOF log, which will cause Redis to be unable to provide services to the outside world for a long time. So we need to streamline the AOF log.
For more technical articles related to Redis, please visit the Redis database and use the introductory tutorial column to learn!