Skip to content
  • Products
    • Persistence for Java / JVM
    • Persistence for Android
    • MicroStream Cluster
    • Serializer
  • Community
    • GitHub
    • Discussion
    • Contribute
    • Issues
    • StackOverflow
  • Resources
    • Get Started
    • Docs
    • Examples
    • Videos
    • Blog
    • Release & Update Plan
  • Services
    • Support
    • Training
  • Company
    • About us
    • Contact
  • Subscribe to our Newsletter
  • User icon Sign in

Cookie

We use cookies to make it easier to use and to further improve our service.

If you have given us your consent, you can revoke it at any time in the data protection declaration.

Use all cookies Manage cookies Reject cookies

Cookie

?
?
?
Use all cookiesSave settingsReject cookiesView Privacy Policy

Necessary

Necessary cookies help make a website usable by enabling basic functions such as page navigation and access to secure areas of the website. The website cannot function properly without these cookies.

Name Anbieter Zweck Ablauf Typ
c3kie_removeCookieWindow microstream.one Saves the consent status of the user whether the cookie window should be displayed. 1 Jahr HTML Local Storage
c3kie_googleAnalytics microstream.one Saves the consent status of the user as to whether Google Analytics is allowed to run. 1 Jahr HTML Local Storage
c3kie_googleAnalytics microstream.one Saves the consent status of the user as to whether Google Analytics is allowed to run. 1 Jahr HTML Local Storage
c3kie_tagManager microstream.one Saves the consent status of the user as to whether Google Tag Manager is allowed to run. 1 Jahr HTML Local Storage
c3kie_facebook microstream.one Saves the consent status of the user as to whether Facebook is allowed to run. 1 Jahr HTML Local Storage
c3kie_matomo microstream.one Saves the consent status of the user as to whether Matomo is allowed to run. 1 Jahr HTML Local Storage
c3kie_youtube microstream.one Saves the consent status of the user as to whether YouTube is allowed to run. 1 Jahr HTML Local Storage

Statistics

Statistics cookies help website owners understand how visitors interact with websites by collecting and reporting information anonymously.

Name Anbieter Zweck Ablauf Typ
_dc_gtm_ Google Used by Google Analytics to limit the request rate. 1 Jahr HTTP Cookie
_gid_ Google Registers a unique ID that is used to generate statistical data on how the visitor uses the website. 2 Jahre HTTP Cookie
_gcl_au Google Used to send data to Google Analytics about the device and visitor behavior. Captures the visitor across devices and marketing channels. Session Pixel Tracker
_gat_ Google Used to store a unique user ID. 1 Tag HTTP Cookie
_gat_gtag_UA_ Google Used to store a unique user ID. 1 Tag HTTP Cookie
yt-player-headers-readable YouTube Used to determine the optimal video quality based on the visitor's device and network settings. Persistent HTML Local Storage

Marketing

Marketing cookies are used to track visitors across websites. The intent is to show ads that are relevant and engaging to the individual user, making them more valuable to publishers and third party advertising providers.

Name Anbieter Zweck Ablauf Typ
VISITOR_INFO1_LIVE YouTube Tries to estimate the range of users on pages with built-in YouTube videos. 179 Tage HTTP Cookie
YSC YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Session HTTP Cookie
yt.innertube::nextId YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Persistent HTML Local Storage
yt.innertube::requests YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Persistent HTML Local Storage
ytidb::LAST_RESULT_ENTRY_KEY YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-cast-available YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-cast-installed YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-connected-devices YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-device-id YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-fast-check-period YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-session-app YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-session-name YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage

Data Model Evolution with Legacy Type Mapping

Aug102022
Blog

With the Legacy Type Mapping functionality of MicroStream, it becomes possible to evolve the data model of your application as your application matures.
In almost all projects, the data model changes. You need additional fields, you perform a restructuring of the classes to improve your data structure, and so on.

MiroStream helps you to convert the data that is stored to your new format. With small changes, this can be performed automatically, for class renames you can define the mapping, and for a large refactoring of the structure, you need to write a Legacy Type Handler.

This blog is accompanied by a video that showcases the scenarios that are described in this text : https://youtu.be/yYw-DzUdOHQ

The storage

As you might know, MicroStream stores the data in a completely different format than the Standard Java Serialisation. You can read more about the Serialisation engine in this FooJay article.

Since only data is stored together with some field identification, MicroStream can perform some transformations and conversions when the data is read from the storage. If it determines that the class structure between the JVM loaded class and the recorder structure in the storage doesn’t match, it tries to find a match itself. It might make the ‘wrong guesses’ or even fail when classes are renamed. In that case, you must provide the refactoring mapping describing how the old situation should be used with the new classes.

Automatic conversion

When the field names of the classes don’t match, it tries to determine the renamed fields based on the Levenshtein distance.

When the ‘probability’ is high enough that it is the same property that the developer renamed, it uses it as a mapping from the old to the new situation. When ‘probability’ is too low, it considers it as a new or deleted property. The mapping is written out in the log and performed when the instance is recreated in the Java memory.

The new structure is only used when an instance is saved as a result of the store() method for example. This means that the storage can hold a mixture of data of a certain type in the old and the new structure. The MicroStream code can handle this perfectly and performs the mapping if needed.

An example of such an automatic mapping is demonstrated in this Github project https://github.com/rdebusscher/microstream-legacy-type-mapping/tree/main/automatic.

Manual Mapping

The automatic mapping is of course based on guesses and makes mistakes. You as a developer of course know very well how the old situation relates to the new structure. You can define the mapping in the format of Foo#field. Where Foo is the fully qualified name of the class. The pairs can be defined within your code within a Map like structure.

Or you can define the mapping within a CSV file. The first column indicates the old reference, and the second column the new one. If you want to indicate a removed field, place it in the first column and leave the second column empty. For a new field, the first column must be empty.

Based on this mapping, you can instruct how MicroStream must map the fields if you restructured them in a class. Or you can indicate a class rename in this way also by specifying different names.

As mentioned, you can use the CSV format to specify the mapping or a Map-Based structure in code. But just as with many other aspects of MicroStream, you can extend its capabilities by implementing the PersistenceRefactoringMapping interface so that you can have to implement your ideal mapping provider mechanism.

A code example can be found in this repository https://github.com/rdebusscher/microstream-legacy-type-mapping/tree/main/manual.

Legacy Type Handler

The mapping capabilities we have covered until now cannot handle large restructurings of the code. You can rename a class but you cannot split the fields from one class into multiple classes. So it is not possible to extract the address information into a separate instance from the other user information for example.

You can either provide this conversion yourself in code when you start up a helper program that performs these changes where mappings as we described can be combined with some Java code to restructure the data.

You can also make use of the Legacy Type Handler. A normal Type Handler determines how the data within a Java instance is stored and retrieved from the storage in a binary format. MicroStream has a generic type handler but also some specialised ones for certain classes so that it can handle any class in your application.

The Legacy Type Handler can be used to manipulate the information from the storage when creating the Java instance. The developer can perform refactoring in this handler as I mentioned at the beginning of this section, to extract address information in a separate instance.

This approach is more complex as you need to use your information from the old class structure (order of pointers and primitives and their byte length, to access the data from the old structure.

An example can be seen in the repository https://github.com/rdebusscher/microstream-legacy-type-mapping/tree/main/complex.

Legacy Type Mapping

Because MicroStream stores only data and references, and not the class structure of Java itself, some conversion can be applied when data is read into memory to handle changes to your data model.

When MicroStream detects changes, it automatically tries to map the new situation to the old one. When you have renamed or added a field within a class, MicroStream can handle this change without the interaction of the developer.

You can always specify the mapping yourself to handle the case that MicroStream is not able to figure it out by itself. Or you can use this explicit manual mapping to handle class renames for example.

The more complex refactoring like extracting some fields in a separate instance requires the Legacy Handler that reads the data in the old structure and you as the developer can perform the required changes.

Category: BlogAugust 10, 2022Leave a comment
Tags: Data PersistenceHow to

Author: Rudy De Busscher

Post navigation

PreviousPrevious post:Using a Database Target with MicroStream CDI IntegrationNextNext post:MicroStream Sessions at the JCON Conference

Related Posts

Two important updates in MicroStream 8.1
June 1, 2023
Quarkus Extension for MicroStream
May 22, 2023
New Features of MicroStream version 8.0
April 28, 2023
MicroStream 8.0 is Now Available
April 26, 2023
MicroStream Becomes an Eclipse Project
April 10, 2023
Blog title MicroStream vs JPA - the ultimate speed test
MicroStream vs JPA and SQL: The Ultimate Performance Test
April 5, 2023

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Post comment

Search
Categories
  • Blog(49)
  • CloudNotification(1)
  • Conferences(13)
  • General(3)
  • News(21)
  • Performance(2)
  • Releases(15)
  • Training(4)
Related Posts
  • Two important updates in MicroStream 8.1
    June 1, 2023
  • Quarkus Extension for MicroStream
    May 22, 2023
  • New Features of MicroStream version 8.0
    April 28, 2023
  • MicroStream 8.0 is Now Available
    April 26, 2023
  • MicroStream Becomes an Eclipse Project
    April 10, 2023
  • Blog title MicroStream vs JPA - the ultimate speed test
    MicroStream vs JPA and SQL: The Ultimate Performance Test
    April 5, 2023
  • MicroStream training in 2023
    March 28, 2023
  • May 16 – 17:00 – 21:00 CEST MicroStream Advanced Course Part 2
    March 13, 2023
  • May 9 – 17:00 – 21:00 CEST MicroStream Advanced Course Part 1
    March 13, 2023
  • April 26 – 17:00 – 21:00 CEST MicroStream Fundamentals Course
    March 13, 2023
Tags
Android AWS Cache Cloud connector Cluster Communication Conference Data Persistence eclipse event feature framework free fundamentals Hackathon Heldion How to InfoQ Integration Java JCON logging LTS Micronaut MicroProfile Microservices MicroStream MicroStream 2 MicroStream 4 MicroStream Day New Features online Open Liberty Open Source Oracle Performance persistence Quarkus release Serialization serializer Spring Boot Subscription Support Training
MicroStream

Store Java Object Graphs natively, relieved of heavy-weight DBMS Dependencies. Create ultra-fast In- Memory Database Applications & Microservices with Pure Java. The Pure Java Paradigm Shift in Database Development.

Upcoming Event

April 19, 2023 | 17:00 – 21:00 CEST
Read more

Platforms

  • MicroStream for Java / JVM
  • MicroStream for Android

Community

  • GitHub
  • Discussion
  • Contribute
  • Issues
  • StackOverflow
  • MeetUp

Resources

  • Get Started
  • Docs
  • Examples
  • Videos
  • Blog
  • Release plan

Services

  • Support
  • Training

Company

  • About us
  • Contact

Stay Connected

  • Twitter
  • LinkedIn
  • YouTube
  • GitHub
  • StackOverflow
  • MeetUp

Get the latest MicroStream news:

Subscribe

© 2023 MicroStream Software. All rights reserved.
  • Terms of use
  • Privacy
  • Legal notice

Cookie

We use cookies to make it easier to use and to further improve our service.

If you have given us your consent, you can revoke it at any time in the data protection declaration.

Use all cookies Manage cookies Reject cookies

Cookie

?
?
?
Use all cookiesSave settingsReject cookiesView Privacy Policy

Necessary

Necessary cookies help make a website usable by enabling basic functions such as page navigation and access to secure areas of the website. The website cannot function properly without these cookies.

Name Anbieter Zweck Ablauf Typ
c3kie_removeCookieWindow microstream.one Saves the consent status of the user whether the cookie window should be displayed. 1 Jahr HTML Local Storage
c3kie_googleAnalytics microstream.one Saves the consent status of the user as to whether Google Analytics is allowed to run. 1 Jahr HTML Local Storage
c3kie_googleAnalytics microstream.one Saves the consent status of the user as to whether Google Analytics is allowed to run. 1 Jahr HTML Local Storage
c3kie_tagManager microstream.one Saves the consent status of the user as to whether Google Tag Manager is allowed to run. 1 Jahr HTML Local Storage
c3kie_facebook microstream.one Saves the consent status of the user as to whether Facebook is allowed to run. 1 Jahr HTML Local Storage
c3kie_matomo microstream.one Saves the consent status of the user as to whether Matomo is allowed to run. 1 Jahr HTML Local Storage
c3kie_youtube microstream.one Saves the consent status of the user as to whether YouTube is allowed to run. 1 Jahr HTML Local Storage

Statistics

Statistics cookies help website owners understand how visitors interact with websites by collecting and reporting information anonymously.

Name Anbieter Zweck Ablauf Typ
_dc_gtm_ Google Used by Google Analytics to limit the request rate. 1 Jahr HTTP Cookie
_gid_ Google Registers a unique ID that is used to generate statistical data on how the visitor uses the website. 2 Jahre HTTP Cookie
_gcl_au Google Used to send data to Google Analytics about the device and visitor behavior. Captures the visitor across devices and marketing channels. Session Pixel Tracker
_gat_ Google Used to store a unique user ID. 1 Tag HTTP Cookie
_gat_gtag_UA_ Google Used to store a unique user ID. 1 Tag HTTP Cookie
yt-player-headers-readable YouTube Used to determine the optimal video quality based on the visitor's device and network settings. Persistent HTML Local Storage

Marketing

Marketing cookies are used to track visitors across websites. The intent is to show ads that are relevant and engaging to the individual user, making them more valuable to publishers and third party advertising providers.

Name Anbieter Zweck Ablauf Typ
VISITOR_INFO1_LIVE YouTube Tries to estimate the range of users on pages with built-in YouTube videos. 179 Tage HTTP Cookie
YSC YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Session HTTP Cookie
yt.innertube::nextId YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Persistent HTML Local Storage
yt.innertube::requests YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Persistent HTML Local Storage
ytidb::LAST_RESULT_ENTRY_KEY YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-cast-available YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-cast-installed YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-connected-devices YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-device-id YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-fast-check-period YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-session-app YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-session-name YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage