Skip to content
  • Products
    • Persistence for Java / JVM
    • Persistence for Android
    • MicroStream Cluster
    • Serializer
  • Community
    • GitHub
    • Discussion
    • Contribute
    • Issues
    • StackOverflow
  • Resources
    • Get Started
    • Docs
    • Examples
    • Videos
    • Blog
    • Release & Update Plan
  • Services
    • Support
    • Training
  • Company
    • About us
    • Contact
  • Subscribe to our Newsletter
  • User icon Sign in

Cookie

We use cookies to make it easier to use and to further improve our service.

If you have given us your consent, you can revoke it at any time in the data protection declaration.

Use all cookies Manage cookies Reject cookies

Cookie

?
?
?
Use all cookiesSave settingsReject cookiesView Privacy Policy

Necessary

Necessary cookies help make a website usable by enabling basic functions such as page navigation and access to secure areas of the website. The website cannot function properly without these cookies.

Name Anbieter Zweck Ablauf Typ
c3kie_removeCookieWindow microstream.one Saves the consent status of the user whether the cookie window should be displayed. 1 Jahr HTML Local Storage
c3kie_googleAnalytics microstream.one Saves the consent status of the user as to whether Google Analytics is allowed to run. 1 Jahr HTML Local Storage
c3kie_googleAnalytics microstream.one Saves the consent status of the user as to whether Google Analytics is allowed to run. 1 Jahr HTML Local Storage
c3kie_tagManager microstream.one Saves the consent status of the user as to whether Google Tag Manager is allowed to run. 1 Jahr HTML Local Storage
c3kie_facebook microstream.one Saves the consent status of the user as to whether Facebook is allowed to run. 1 Jahr HTML Local Storage
c3kie_matomo microstream.one Saves the consent status of the user as to whether Matomo is allowed to run. 1 Jahr HTML Local Storage
c3kie_youtube microstream.one Saves the consent status of the user as to whether YouTube is allowed to run. 1 Jahr HTML Local Storage

Statistics

Statistics cookies help website owners understand how visitors interact with websites by collecting and reporting information anonymously.

Name Anbieter Zweck Ablauf Typ
_dc_gtm_ Google Used by Google Analytics to limit the request rate. 1 Jahr HTTP Cookie
_gid_ Google Registers a unique ID that is used to generate statistical data on how the visitor uses the website. 2 Jahre HTTP Cookie
_gcl_au Google Used to send data to Google Analytics about the device and visitor behavior. Captures the visitor across devices and marketing channels. Session Pixel Tracker
_gat_ Google Used to store a unique user ID. 1 Tag HTTP Cookie
_gat_gtag_UA_ Google Used to store a unique user ID. 1 Tag HTTP Cookie
yt-player-headers-readable YouTube Used to determine the optimal video quality based on the visitor's device and network settings. Persistent HTML Local Storage

Marketing

Marketing cookies are used to track visitors across websites. The intent is to show ads that are relevant and engaging to the individual user, making them more valuable to publishers and third party advertising providers.

Name Anbieter Zweck Ablauf Typ
VISITOR_INFO1_LIVE YouTube Tries to estimate the range of users on pages with built-in YouTube videos. 179 Tage HTTP Cookie
YSC YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Session HTTP Cookie
yt.innertube::nextId YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Persistent HTML Local Storage
yt.innertube::requests YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Persistent HTML Local Storage
ytidb::LAST_RESULT_ENTRY_KEY YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-cast-available YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-cast-installed YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-connected-devices YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-device-id YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-fast-check-period YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-session-app YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-session-name YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage

Store State of a Background process with MicroStream

Oct312022
Blog

With the MicroStream framework, you can treat an object graph within the JVM heap as your database. This means you can perform queries by executing any java method against those objects. The Stream API is very well-suited to be used here.
The Java instances are serialised in a binary format to an external storage so that the state of the object graph can be restored when your process starts up the next time.

We have developed a new serialisation engine from scratch, as we want to overcome a few shortcomings of the standard Java one.

Serialise any object

One of the requirements we had, was that any kind of Object could be part of the Object graph that consists of your database. And thus we should be able to serialise any object, not only those that implement Serializable.

Therefore we developed a new engine that can just store any object. There is no need to have a certain (marker) interface implemented, has some annotations, or a schema that generates code. Any Java instance can be stored and loaded again and we don’t even use the constructor or any other method for this, like the readObject() method. This way, we make sure that no vulnerability exploit can be made.

Any object?

In theory, yes, any object can be handled by MicroStream. We have excluded a few classes as serialising them doesn’t make much sense or are related to some OS resources so a serialisation would not work. An example is the Thread class.

When you try to serialise a Thread instance, you will receive a PersistenceExceptionTypeNotPersistable exception.

Other classes that will throw this Exception are the InputStream and OutputStream ones, Socket, and Iterator and Enumeration instances. Of course, the collections themselves can be used, but references to an Iterator that might be stored in a field, are not.

Also, lambdas cannot be serialised due to the special format they are stored in by the compiler. So when this field f is part of the object graph that you want to store, you will receive an Exception.

Function<Integer, String> f = x -> String.format("Function result for %s", x);

 

Of course, the above lambda doesn’t make sense to store, since it doesn’t hold any data. If you want to store a ‘lambda‘, define it as an actual class, an anonymous inner class is also supported. This doesn’t change how you can make use of it in your code.

  Function<Integer, String> f = new Function<>() {

      @Override
      public String apply(Integer x) {
          return String.format("Anonymous inner class result for %s", x);
      }
  };

Background process with state

Since we have indicated that a Thread cannot be stored, you cannot store for example a Thread that runs in the background of your application and execute every period some tasks and keep the last run time to filter the next time, for example.

Also, a TimerTask cannot be stored since it has a state field that gets set once the task is scheduled by a Timer instance so that the same task cannot be started again later on. And since this state field is also serialised by MicroStream, a deserialised instance cannot be used anymore.
But a Runnable instance can be serialised, so with the help of ScheduledExecutorService, you can achieve your requirement.

For example, the Runnable implementation can look like this. Here we just keep the last run date as an example.

public class ProcessRunnable implements Runnable {

    transient Persister persister;

    private Date lastRun;

    public ProcessRunnable(Persister storageManager) {
        this.persister = storageManager;
    }

    @Override
    public void run() {
        System.out.println("Last run was at " + lastRun);
        lastRun = new Date();
        System.out.println("Now running at " + lastRun);
        persister.store(this);

    }

}

 

So every time the Runnable is executed, it stores its state in the external storage.

And a simple Java SE program to demonstrate how it can be used looks like

   public static void main(String[] args) {
        BackgroundRoot root = new BackgroundRoot();

        StorageManager storageManager = EmbeddedStorage.start(root, Paths.get("data/background"));

        ScheduledExecutorService ses = Executors.newScheduledThreadPool(1);

        if (root.getRunnableTask() == null) {
            root.setRunnableTask(new ProcessRunnable(storageManager));
            storageManager.store(root);
        }

        ses.scheduleAtFixedRate(root.getRunnableTask(), 0, 10, TimeUnit.SECONDS);


    }

    public static class BackgroundRoot {
        private ProcessRunnable runnableTask;

        public ProcessRunnable getRunnableTask() {
            return runnableTask;
        }

        public void setRunnableTask(ProcessRunnable runnableTask) {
            this.runnableTask = runnableTask;
        }
    }

When there is no ProcessRunnable yet, first it creates one. Otherwise, there is already an instance defined with the data from the last run and we just start it through ScheduledExecutorService instance.

Background process with state

The MicroStream framework has no limitation on which object can be stored. There is no interface required like serialisable, annotations, or any other special construct. However, some classes are excluded like Thread, InputStream, and OutputStream instances.

Since also a TimerTask cannot be reused due to the restriction within the JDK implementation (related to the state field).

But you can implement your requirements for a background task that keeps state using a Runnable and the ScheduledExecutorService. Be aware that you cannot define the Runnable as a lambda when you want to make use of MicroStream but since you also need fields for your state, that is not a problem here.

Category: BlogOctober 31, 2022Leave a comment
Tags: Data Persistence

Author: Rudy De Busscher

Post navigation

PreviousPrevious post:Updates to the Spring Boot integration in version 7.1NextNext post:MicroStream with Helidon MP

Related Posts

Two important updates in MicroStream 8.1
June 1, 2023
Quarkus Extension for MicroStream
May 22, 2023
New Features of MicroStream version 8.0
April 28, 2023
MicroStream 8.0 is Now Available
April 26, 2023
MicroStream Becomes an Eclipse Project
April 10, 2023
Blog title MicroStream vs JPA - the ultimate speed test
MicroStream vs JPA and SQL: The Ultimate Performance Test
April 5, 2023

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Post comment

Search
Categories
  • Blog(49)
  • CloudNotification(1)
  • Conferences(13)
  • General(3)
  • News(21)
  • Performance(2)
  • Releases(15)
  • Training(4)
Related Posts
  • Two important updates in MicroStream 8.1
    June 1, 2023
  • Quarkus Extension for MicroStream
    May 22, 2023
  • New Features of MicroStream version 8.0
    April 28, 2023
  • MicroStream 8.0 is Now Available
    April 26, 2023
  • MicroStream Becomes an Eclipse Project
    April 10, 2023
  • Blog title MicroStream vs JPA - the ultimate speed test
    MicroStream vs JPA and SQL: The Ultimate Performance Test
    April 5, 2023
  • MicroStream training in 2023
    March 28, 2023
  • May 16 – 17:00 – 21:00 CEST MicroStream Advanced Course Part 2
    March 13, 2023
  • May 9 – 17:00 – 21:00 CEST MicroStream Advanced Course Part 1
    March 13, 2023
  • April 26 – 17:00 – 21:00 CEST MicroStream Fundamentals Course
    March 13, 2023
Tags
Android AWS Cache Cloud connector Cluster Communication Conference Data Persistence eclipse event feature framework free fundamentals Hackathon Heldion How to InfoQ Integration Java JCON logging LTS Micronaut MicroProfile Microservices MicroStream MicroStream 2 MicroStream 4 MicroStream Day New Features online Open Liberty Open Source Oracle Performance persistence Quarkus release Serialization serializer Spring Boot Subscription Support Training
MicroStream

Store Java Object Graphs natively, relieved of heavy-weight DBMS Dependencies. Create ultra-fast In- Memory Database Applications & Microservices with Pure Java. The Pure Java Paradigm Shift in Database Development.

Upcoming Event

April 19, 2023 | 17:00 – 21:00 CEST
Read more

Platforms

  • MicroStream for Java / JVM
  • MicroStream for Android

Community

  • GitHub
  • Discussion
  • Contribute
  • Issues
  • StackOverflow
  • MeetUp

Resources

  • Get Started
  • Docs
  • Examples
  • Videos
  • Blog
  • Release plan

Services

  • Support
  • Training

Company

  • About us
  • Contact

Stay Connected

  • Twitter
  • LinkedIn
  • YouTube
  • GitHub
  • StackOverflow
  • MeetUp

Get the latest MicroStream news:

Subscribe

© 2023 MicroStream Software. All rights reserved.
  • Terms of use
  • Privacy
  • Legal notice

Cookie

We use cookies to make it easier to use and to further improve our service.

If you have given us your consent, you can revoke it at any time in the data protection declaration.

Use all cookies Manage cookies Reject cookies

Cookie

?
?
?
Use all cookiesSave settingsReject cookiesView Privacy Policy

Necessary

Necessary cookies help make a website usable by enabling basic functions such as page navigation and access to secure areas of the website. The website cannot function properly without these cookies.

Name Anbieter Zweck Ablauf Typ
c3kie_removeCookieWindow microstream.one Saves the consent status of the user whether the cookie window should be displayed. 1 Jahr HTML Local Storage
c3kie_googleAnalytics microstream.one Saves the consent status of the user as to whether Google Analytics is allowed to run. 1 Jahr HTML Local Storage
c3kie_googleAnalytics microstream.one Saves the consent status of the user as to whether Google Analytics is allowed to run. 1 Jahr HTML Local Storage
c3kie_tagManager microstream.one Saves the consent status of the user as to whether Google Tag Manager is allowed to run. 1 Jahr HTML Local Storage
c3kie_facebook microstream.one Saves the consent status of the user as to whether Facebook is allowed to run. 1 Jahr HTML Local Storage
c3kie_matomo microstream.one Saves the consent status of the user as to whether Matomo is allowed to run. 1 Jahr HTML Local Storage
c3kie_youtube microstream.one Saves the consent status of the user as to whether YouTube is allowed to run. 1 Jahr HTML Local Storage

Statistics

Statistics cookies help website owners understand how visitors interact with websites by collecting and reporting information anonymously.

Name Anbieter Zweck Ablauf Typ
_dc_gtm_ Google Used by Google Analytics to limit the request rate. 1 Jahr HTTP Cookie
_gid_ Google Registers a unique ID that is used to generate statistical data on how the visitor uses the website. 2 Jahre HTTP Cookie
_gcl_au Google Used to send data to Google Analytics about the device and visitor behavior. Captures the visitor across devices and marketing channels. Session Pixel Tracker
_gat_ Google Used to store a unique user ID. 1 Tag HTTP Cookie
_gat_gtag_UA_ Google Used to store a unique user ID. 1 Tag HTTP Cookie
yt-player-headers-readable YouTube Used to determine the optimal video quality based on the visitor's device and network settings. Persistent HTML Local Storage

Marketing

Marketing cookies are used to track visitors across websites. The intent is to show ads that are relevant and engaging to the individual user, making them more valuable to publishers and third party advertising providers.

Name Anbieter Zweck Ablauf Typ
VISITOR_INFO1_LIVE YouTube Tries to estimate the range of users on pages with built-in YouTube videos. 179 Tage HTTP Cookie
YSC YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Session HTTP Cookie
yt.innertube::nextId YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Persistent HTML Local Storage
yt.innertube::requests YouTube Registers a unique ID to keep statistics on which videos from YouTube the user has seen. Persistent HTML Local Storage
ytidb::LAST_RESULT_ENTRY_KEY YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-cast-available YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-cast-installed YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-connected-devices YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-device-id YouTube Saves the user's video player settings with embedded YouTube video. Persistent HTML Local Storage
yt-remote-fast-check-period YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-session-app YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage
yt-remote-session-name YouTube Saves the user's video player settings with embedded YouTube video. Session HTML Local Storage