vlcj-pro Out of Process Technical Background

For most applications, using the standard component framework provided by core vlcj is usually the right thing to do. Such applications will embed a native media player inside a Java application in the same operating-system process. This is an "in-process" approach.

The main downside to using an in-process approach is that a failure in the native process can immediately terminate the Java Virtual Machine under which the Java application is executing.

A further disadvantage of the in-process approach is that it is not always completely reliable for applications that require multiple native media players to be embedded in the same application at the same time. This lack of reliability is due to the fact that one or more of the native libraries that LibVLC uses may not be re-entrant and/or may not be thread-safe with regards to concurrent execution.

These problems usually manifest themselves at best as strange application behaviour or more commonly an immediate fatal Java Virtual Machine termination. With a small number of embedded media players, it is possible to manage the risk of fatal crashes, but as the number of embedded media players in the same application increases, so does the risk of instability and fatal crashes. It is often inevitable that an application with multiple in-process media players will crash eventually.

There are also limitations inherent to LibVLC, the native library that vlcj depends upon, when dealing with multiple media players in the same process - in particular there are no independent audio controls, so muting or changing the audio volume will affect all native media player instances.

So for applications that require the use of more than one native media player, an out-of-process solution is recommended. For applications that embed a single media player, there may still be benefits to using an out-of-process approach instead - mainly to mitigate the risk of a crashing media player taking down the entire Java Virtual Machine without warning.

The only way to reliably address these shortcomings is to move each native media player instance to it's own separate operating system process. However, this reliability comes at a cost - implementing and dealing with out-of-process media players is not a simple task and involves advanced concepts.

The benefits of using an out-of-process approach are that all of the previously described issues go away. Multiple out-of-process media players generally perform better than multiple in-process media players.

Out of Process Basics

Usually when a Java application is executed, there is one operating system process created to execute that application. When the Java Virtual Machine is started an operating system process runs, loads your application main class and executes the main method. Applications may create their own background threads, but they rarely create new operating system processes.

The approach is in essence:

• Main application creates a user interface;
• Main application creates a video surface component for each required media player and adds each video surface to its user interface (each video surface has a unique native component identifier, an integer);
• For each media player the main application creates a new process (for example using the Java ProcessBuilder class) with a different main class;
• The main class used by the new process(es) creates a native media player and associates the unique component identifier of the video surface with the media player;
• The main application and the new process(es) co-ordinate their operation and communicate via some sort of remote procedure call or message-passing protocol.

Video Surface

The main application must communicate the unique component identifier of the video surface to the child process. The simplest approach is to pass the identifier as one of the main class arguments when the process is created. An alternate approach is to somehow send the process a message telling it to associate its own media player instance with a particular component identifier.

Process Communication

The immediate difficulty is how to co-ordinate the communications required between the main application and the out-of-process media players. The user interacts with the main application, so for example a request to play a video, or pause, or whatever, must be propagated to the corresponding out-of-process media player. A further complication is how to propagate media player events from the external process back to the main application - an application still needs to know when a video finishes, or may still like to know the current position/time of the play-back.

There are many possible solutions for this, but all require some form of remote procedure call.

Some examples:

• Use process input/output streams, implement some sort of text protocol to send commands to the external process and receive responses;
• Embed an HTTP server in the external process and implement some sort of RESTful (or other) web-services to handle media player commands;
• Embed a message queue and send messages via JMS;
• Use Java's Remote Method Invocation or some other RPC mechanism.

These examples are just a small sub-set of the technologies that could be used to solve this problem.

Whichever technology is chosen, a component layer will be required in the client application to encapsulate the required protocol for communication with the external processes. In basic terms, a call to mediaPlayer.play() must somehow be translated into a remote procedure call and dispatched to the external process.

Other Considerations

Applications that use out-of-process media players also need to consider:

• Error handling in the external process;
• Unexpected termination of the external process;
• Detecting a broken process and replacing it, and the associated media player, with a new one;
• Not leaving orphaned media player process if the main application terminates;
• Security/integrity of the communications between the main application and the external processes.

What Can Be Achieved

The extent of what can be achieved with out-of-process media players will depend largely on the capabilities of the computer those media players are running on.

On a decent quad-core machine, with a reasonable graphics card, vlcj applications have been created that embed, in the same application, up to twelve completely isolated reliable media players for movie playback - including independent audio controls. More than twelve may well be possible with a computer with improved specifications. What can be achieved also depends on the nature of the media being played of course.

The screen-shots on this page show examples of real applications using out-of-process media players.

Conclusion

Using out-of-process media players is without doubt the most reliable way to implement native media players - especially for applications that require multiple media players, but also for applications that only use a single media player.

The cost of this reliability is in the complexity of the required solution.

A proper solution for out-of-process media players must reliably handle remote communications, including propagation of remote media player events, and must tolerate and recover from external process failures.

For applications that absolutely can not tolerate the total failure of a Java Virtual Machine (which is of course every application), out-of-process media players are the only solution.

Support for Out of Process Media Players in vlcj

There is no free open source out-of-the-box solution for out-of-process media players provided with vlcj.

vlcj-pro is a new product that provides robust and reliable support for out-of-process media players using vlcj.