MicroStream Compared to Redis

Several people have mentioned to us during discussions that MicroStream is similar to Redis. When we explain the basic functionality of MicroStream, they see some parallels with Redis. And there are some similarities about the provided functionality but there are some important differences, mainly on the technical level. And no surprise if we claim that MicroStream is better.

Introduction

Redis, short for Remote Dictionary Server, is an in-memory data structure store that is widely used as a caching and storage solution. Redis supports various data structures such as strings, lists, sets, sorted sets, and hashes, allowing developers to store and manipulate data with ease.

One of the primary use cases of Redis is caching, which involves storing frequently accessed data in memory for faster retrieval. By caching data in Redis, applications can improve their performance and reduce the load on backend systems. Redis employs an advanced eviction mechanism, where it can automatically remove less frequently used items from memory to make room for new data. Additionally, Redis offers various features like time-to-live (TTL) settings, allowing cached data to expire after a certain period, ensuring that the cache remains fresh and up-to-date.

Another essential functionality of Redis is its ability to store and query JSON data. Redis provides native support for storing and retrieving JSON documents, enabling developers to work with more complex and hierarchical data structures. With Redis’ JSON commands, you can store JSON objects as values and perform operations like indexing, searching, and querying on specific fields within the JSON document. Thus, Redis is often used for scenarios where querying JSON data is needed.

Caching

One of the major use cases of Redis is caching data so that retrieving them from an external system like a database or NoSQL solution is not needed. Since it has a C codebase, it does not really know about Java and thus the Redis client you have in your application needs to make a conversion to a certain format before it can store the data. We can make use of the standard Java serialisation to create a binary representation of the objects. But many solutions are converting to JSON and storing it as a String. As JSON, it can be used by other applications also, including those written in another programming language.

But these conversions come with a cost. Data needs to be converted each time you write and read it from the Redis server. And you need some kind of unique identification of the piece of data that you want to retrieve. You store it in a key-value-based structure so the key, like the id of the customer, needs to be known to retrieve the customer information. Looping over all value entries to find a certain value is highly inefficient.

Also storing and retrieving many pieces of information can be quite time-consuming. As a test, you can try to write and retrieve 100,000 items in a Redis Cache and see how much time its takes.

MicroStream is much more efficient and easier to use as a cache than Redis. MicroStream stores the Java objects in a binary format to a storage target. You don’t need to specify how the data is stored. You don’t need to convert it to a JSON string yourself. So getting started with MicroStream is easier, less to configure, and also more efficient as it just stores the Java objects in a format that resembles the storage in the Java Heap.

It also supports any Java type and can be used to store circular data which is not possible in JSON.

One of the main challenges with caching in general is ensuring that cached data remains up to date. When the underlying data in changes, cached data have to be refreshed. Cache configuration adds significant complexity to your system, especially when dealing with cache eviction policies, cache partitioning, and cache coherency. You need to carefully design and maintain your caching layer. With MicroStream, your in-memory data is your single-source-of-truth. On a single node, MicroStream is a full-consistent solution without the need for complex cache configurations. MicroStream Cluster, where data is replicated through multiple JVM nodes, is an eventually consistant approach.

Database

Another use case of Redis is a database alike one where you can store JSON items and search through them after you have defined a Schema. This modus simulates the scenario where you have records like in a table that you can search through. And you can update, insert and delete items.

But this schema, index, and query feature comes with a learning cost. Just as you need to learn SQL to retrieve data from a relational database, you need to learn the query language of Redis JSON. As an example, the following statements define a Schema on a JSON structure that has a firstname and lastname property, apply it to all keys that are prefixed with student value, and retrieve the result of all key ids where the JSON item have a first name that starts with ‘maya’. Once we have those indexes, we can retrieve the actual JSON from Redis.

Schema schema = new Schema().addTextField("$.firstName", 1.0)
                            .addTextField("$.lastName", 1.0);

IndexDefinition rule = new IndexDefinition(IndexDefinition.Type.JSON)
	                        .setPrefixes("student:");

client.ftCreate("student-index", IndexOptions.defaultOptions()
.setDefinition(rule), schema);

Query q = new Query("@\\$\\.firstName:maya*");
SearchResult mayaSearch = client.ftSearch("student-index", q);

In contrast, with MicroStream you can just operate on the Java objects. When developing in Java, we all know how we can handle Lists, Maps, use the Streams API, etc. MicroStream has no query language, other than plain Java. So you can use MicroStream efficiently from day 1, without the need to learn new idioms and syntax.

Conclusion

Yes, Redis and MicroStream have some common high-level functionality as you can use it as a caching and data storage solution. Redis is widely used as a distributed cache but the setup, configuration to serialise your data, and steep learning curve to query JSON documents is much higher.

With MicroStream, once you have configured where the data will be stored, all you need to do is know how you need to work with Java objects. And that is easy, for us Java developers.