New Features of MicroStream version 8.0

Earlier this week, we made the new version of MicroStream available on Maven Central. It contains new functionality within the Core of the framework, we changed the minimum JDK level, added and improved the integrations with frameworks, and added a new tool.

This blog gives you a short overview of all these changes. More detailed blogs will be available in the coming weeks, for the impatient ones, you can always check the documentation of course, in follow-up blogs and videos.

Also important to note is the section about the Eclipse Projects. This is the last major version under the name of MicroStream and all major new development will happen under the wings of the Eclipse Foundation. More information about migration options will be made available when a first release of the Eclipse Projects are performed.

Do you want to update your version of MicroStream within your project, update the artifact version number to 08.00.00-MS-GA.

All changes can be found in the change log.

JDK 11

Java 8 is still officially supported for a few years by the JVM vendors. However, JDK 11 has become the most widely used version, so updating MicroStream’s minimum version makes sense. So version 8.0 will no longer run on JDK 8.

Core additions

• The Lazy Collections are experimental implementations of List, Map, and Set that can handle large amounts of items in a more memory-efficient way. The data is stored in several Segments which are lazy-loaded. These allow you to store large collections without the need to worry about loading them all at once into the JVM heap. [documentation][video]
• The Reloader class exposes the already available functionality of loading a specific object or object graph from the data storage in an easier way. This can be used if you need to perform a kind of ‘rollback’ functionality so that you discard the current data in an object or sub object graph and load the data from the data storage.
• We have now a read-only mode so that you can have multiple processes accessing the same storage, as long as only one process makes changes and the other only access the data in read-only mode. This read-only process doesn’t receive updates out of the box.

Framework Integrations

We have again some new integrations and improved the functionality of the already available ones.  We will continue this path in the future to improve the useability of MiroStream within these and other frameworks.

• For the Spring Boot integration, we have now the possibility to define multiple Storage Managers in case you want to access multiple ‘databases’ from within one application. These can be accessed using Spring qualifiers. [documentation]
• There is now an integration available for SpringBoot 3.x. The functionality is the same as the one for Spring Boot 2.x but is based on version 3 and thus requires JDK 17. Besides the artifact name change, no changes should be needed in using this integration. [documentation]
• Another addition is an integration for Quarkus 2.x.  It provides you the same functionality as the Spring Boot and Jakarta EE integration. More detailed information will follow in another blog but you can get already started by following the documentation.
• Also for the Jakarta EE integration, an updated version is provided.  This integration is now using CDI 3+ and MicroProfile Config 3.0.  This allows you to have the integration with MicroStream when you are running on a Jakarta EE 10 certified runtime. [documentation]
• And lastly, I want to mention that the dev-mode of Spring Boot can be used now with the MicroStream integration without the need for any additional configuration of the restart classloader. Thanks to Mitchel Skaggs that made the code contribution to support this.

Tools

We have created a tool that can update your data storage to a different channel count, to improve performance if you need this due to the popularity of your program. But it can also be used to migrate storage from one type of storage to another type. Handy if you are migrating to the cloud, for example.

SBOM

I want to mention the support for the SBOM in a separate section. The Software Bill of Material got a lot of attention lately due to some high-risk vulnerabilities that were discovered in some Java frameworks. With the SBOM functionality in place, it becomes easier to verify which other frameworks are used and makes it possible to have a better view of the supply chain security as data is available in a digital format.

For MicroStream, we are not using many external dependencies. Only the SLFJ API dependency is added to the MicroStream Artifacts. The integrations are depending on the frameworks for which they are created and thus more dependencies appear in the list.

We are using the CycloneDX format, maintained and developed by the OWASP foundation, and the data s available within the META-INF directory of our Jar files in a file called MicroStream-Sbom. The data is provided in JSON and XML formats.

Updates and Fixes

Besides the new features described in the above lists, there are also various smaller fixes and updates.  The most important ones are

• The specific Type Handlers we have for the immutable collections added in JDK 12 are improved. You can use these handlers by adding the JDK 17 specific artefact to your project.
• When using the Serializer only, we have now the possibility to include the Type Dictionary information within the binary output. This increases its output size but makes it easier to support loading binary data from an old version of your class.

Cleanup @Deprecated

I also want to mention that we removed all classes and methods that were marked as @Deprecated in version 7. So if you still make use of any of them in your project, you will need to update your code before it will compile with this new version 8.

Eclipse Projects

The development of MicroStream started already 10 years ago. In 2019 we have launched the very first official MicroStream release version 1. Until 2021, MicroStream was closed-source. In August 2021, we have published MicroStream version 5 as open source. Now, MicroStream becomes an Eclipse project. To be more precise, MicroStream is divided into two Eclipse projects.

At its core, MicroStream contains a custom Serializer which is very secure since it works with a white list of class names and there is no code execution, not even the code within the constructor. The code for this serialise functionality and dependent classes that are the foundation of the MicroStream functionality can be found in the Eclipse Serializer project.

The code related to the StorageManager functionality, including the support for the different storage targets, will be part of another Eclipse project. The current codename for that is Eclipse Store. At this moment (April 2023), the project is not yet visible in the list of Eclipse Projects but should appear soon.

Once the paperwork and projects are in place, the plan is to make an initial release of the code which is based on MicroStream version 7.1. We then continue porting the newly added features of version 8 to the Eclipse projects and provide you with a first release once we are out of the incubation period.

The integrations with SpringBoot, Quarkus, and CDI will not be part of an Eclipse Project. They also will be hosted in a MicroStream repository but they stay open source.

Also regarding the MicroStream cluster and enterprise versions, nothing will change. They are still available and will be developed further as closed-source products.

Watch the video: Lazy Collections: Search Million Records with Minimal RAM