Check it out, pictures I took of my Eternity-Puzzle ended up in a french boardgame-magazine! :D




Do you want to make your own games with lots of cutscenes like in Command & Conquer, Wing Commander, or the newer Roundabout? In this introduction to the Unity Video Player you’ll learn how to do that, and how to implement videos in a variety of ways of your own into your game. Unity comes with its own Video Player component, which means adding videos and creating games around it is not at all complicated.
You will learn how to:
You will need the newest version of Unity, and the starter-files which you can download here. This tutorial will build on them. In that download you can also find the finished project!
You will also need to know the basics of Unity, and have already created something with it.
Load the starter-project, Land of Full Motion Video Starter, and take a look at the folder-structure inside the Assets folder.
Here is what each folder contains:
Now open up the scene 01FullScreenCutscene. It should look something like this:
Welcome to the magical Land of Full Motion Video, where 3d-environments intersect with real-life video-recordings in completely plausible ways.
Right now you can see a small environment you can explore in first-person, and a console with buttons on it. Try it out! Walk around using the wasd-keys and try to press the buttons by pointing at them and left-clicking.
Nothing will happen though. That’s what you’ll fix next. :)
First you will need to open the VideoController.cs file, which barely has anything in it! Of course nothing is playing.
Start by adding these two methods to the VideoController class:
//1 public void PlayFullScreenOfflineVideo() { StartFullScreenVideo(videoFilePath); } private void StartFullScreenVideo(string path) { //2 if (fullScreenVideoPlayer) { Destroy(fullScreenVideoPlayer); } //3 fullScreenVideoPlayer = Camera.main.gameObject.AddComponent(); //4 firstPersonInteractor.SetImmobile(true); //5 fullScreenVideoPlayer.playOnAwake = false; fullScreenVideoPlayer.renderMode = UnityEngine.Video.VideoRenderMode.CameraNearPlane; fullScreenVideoPlayer.targetCameraAlpha = 1F; fullScreenVideoPlayer.url = path; fullScreenVideoPlayer.frame = 0; fullScreenVideoPlayer.isLooping = false; //6 fullScreenVideoPlayer.Play(); }
Here is what happens in the code of PlayFullScreenOfflineVideo()
:
PlayFullScreenOfflineVideo
executes when you press the button. It then calls StartFullScreenVideo()
with a saved filepath as parameter.playOnAwake = false;
makes sure the video will not start immediately. The renderMode
is set to cameraNearPlane
for fullscreen-videos. The isLooping
variable sets if the video should loop endlessly. frame
sets the start-frame of the video. With a larger number it would start at that point (although the number should be within the actual length of the videoclip). targetCameraAlpha
sets how “transparent” the fullScreen-video is going to be. 1F
means it will not fully visible, 0.5F
would be 50% visible.But you are not done yet, as these functions will not call themselves.
Open the WorldUIElement.cs file. It is assigned to the buttons you see in the level, and can be set to be a certain type of button (in this case ButtonTypes.playFullScreenOfflineVideo
). In order to call the VideoController class and start the video add these lines to the start of Interact()
.
if (buttonType == ButtonTypes.playFullScreenOfflineVideo) { videoController.PlayFullScreenOfflineVideo(); }
The entire function should look like this:
public void Interact() { if (!isActivated) { if (buttonType == ButtonTypes.playFullScreenOfflineVideo) { videoController.PlayFullScreenOfflineVideo(); } isActivated = true; displayMesh.transform.localPosition = buttonPressedPosition; audioSource.PlayOneShot(buttonSound); thisRenderer.material = materialNormal; StartCoroutine(DeactivateButton()); } }
The important part is that now that PlayFullScreenOfflineVideo()
is called, and a video will play when you press the Offline Video button on the console! The rest of the code handles the button animating, and does not need to be modified by you.
Try it out and press the Offline Video button! The result should look something like this for about six seconds:
You cannot quit the cutscene yet. To do that you’ll need this code inside Update()
of the VideoController class:
void Update() { //1 if (fullScreenVideoPlayer && fullScreenVideoPlayer.isPlaying) { //2 if (Input.GetKey(KeyCode.Escape)) { //3 fullScreenVideoPlayer.Stop(); firstPersonInteractor.SetImmobile(false); } } }
Here is what the code does:
Once the video has finished press the escape-key to quit the video-view.
Well done, you created your very own first full-screen cutscene! Now you can get some actors together and create your own Command & Conquer-esque masterpiece. ;]
Note: When using a local file you need to adapt its filepath to the current save-location. The editor-filepath will no longer work when deployed. Luckily you can solve this by prefixing Application.dataPath
to the filepath. This has already been happening in Start()
:
private string videoFilePath = "/RW/Videos/ForestFlyover.webm"; void Start() { videoFilePath = Application.dataPath + videoFilePath; }
After watching the video pause the gameplay and take a look at the main camera in the inspector. This is what you should see:
Here are all the variables that you set via code. It is also possible to manually add the Video Player component by selecting the needed GameObject, and then selecting Component ▸ Video ▸ Video Player. Afterwards only a call to the Video Player and Play()
is necessary to start a video.
Note: There are some limitations on what kind of files you can use. Depending on the system you use Unity’s file import will not work for certain formats. More information and details can are available here. For example a wmv-file will only work on windows, and no other platform. Unity can import the webm-format on the other hand without issue on any platoform.
The next thing to keep in mind is that not every video file (even if the import works) will run on every mobile device, as only certain codecs run on Android devices or iPhones. More details on that are on this page.
Be sure to chose a format and codec that works on the platforms you intend to deploy too!
Your astute eyes probably noticed the second, still not functional button labelled Online Video.
Right now the Video Player uses a locally saved file, which has been hard-coded. Alternatively you can also use a hyperlink and use an online source! To make it work add this method to the VideoController class:
public void PlayFullScreenOnlineVideo() { StartFullScreenVideo(videoURL); }
What this does is call StartFullScreenVideo
, which you used before, but with the video-url as parameter.
You will also need to adapt Interact()
in the WorldUIElement class with this code:
if (buttonType == ButtonTypes.playFullScreenOnlineVideo) { videoController.PlayFullScreenOnlineVideo(); }
When you press the Online Video button inside the game this will call the VideoController, and PlayFullScreenOnlineVideo()
, which you just added.
The video will play the same, try it out!
Right now an online video has already been selected for you to use:
private string videoURL = "https://www.videvo.net/videvo_files/converted/2016_01/preview/Forest_15_3b_Videvo.mov47209.webm";
You can find this variable and swap out the link for your own favorite YouTube-video, and that video (assuming you have a working internet-connection) will play once you hit the Online Video button in the gameworld.
This offers lots of possibilities in regards to game-design, where you could possibly create a game which has a new video every time, or you could host your tutorial-video on the web, and then change the video without having to update the game itself.
Open the scene 02VideoPlayer. it is very similar to the first one, only that this one has a an actual video-screen inside the world, and new buttons.
Like before the buttons don’t do anything right now. In order to have a video displayed on the big screen first add this variable to the VideoController class:
public VideoPlayer displayVideoPlayer;
This is where the script will access the Video Player. Next assign the DisplayMesh of the VideoDisplayScreen object to this variable. If you look the the DisplayMesh in the inspector you’ll see that it already has a Video Player component attached to it.
Next add this method to the VideoController:
public void PlayInWorldVideo() { //1 if (!displayVideoIsPaused) { //2 displayVideoPlayer.playOnAwake = false; displayVideoPlayer.renderMode = UnityEngine.Video.VideoRenderMode.MaterialOverride; displayVideoPlayer.url = videoFilePath; displayVideoPlayer.frame = 0; displayVideoPlayer.isLooping = true; } //3 displayVideoPlayer.Play(); //4 displayVideoIsPaused = false; }
As before you will need this check in Interact()
in the WorldUIElement class:
if (buttonType == ButtonTypes.playInWorldVideo) { videoController.PlayInWorldVideo(); }
This is what happens when the code runs:
displayVideoPlayer.renderMode = UnityEngine.Video.VideoRenderMode.MaterialOverride;
, which makes the video play on the object it’s on, and not in full-screen.displayVideoIsPaused
variable is set to false. This will be more important for using the pause-feature.Try it out! When you press the Play button on the console the video will now run on the screen.
Next up is the Pause Button, which will pause the video, if it is playing. Add this method to the script:
public void PauseInWorldVideo() { //1 if (displayVideoIsPaused) { //2 displayVideoPlayer.Play(); displayVideoIsPaused = false; } else { //3 displayVideoPlayer.Pause(); displayVideoIsPaused = true; } }
displayVideoIsPaused
variable is set to false. This affects the usage of the Play and Pause buttons.displayVideoIsPaused
variable is set to true.Next add this method to stop video-playback:
public void StopInWorldVideo() { //1 displayVideoPlayer.Stop(); //2 displayVideoIsPaused = false; }
Adapt the Interact
method of the WorldUIElement class with these checks to make it all work again:
if (buttonType == ButtonTypes.pauseInWorldVideo) { videoController.PauseInWorldVideo(); } else if (buttonType == ButtonTypes.stopInWorldVideo) { videoController.StopInWorldVideo(); }
The Interact
method should have all five checks now. You can also turn these into a single if-else block, instead of multiple ones:
if (buttonType == ButtonTypes.playFullScreenOnlineVideo) { videoController.PlayFullScreenOnlineVideo(); } else if (buttonType == ButtonTypes.playFullScreenOfflineVideo) { videoController.PlayFullScreenOfflineVideo(); } else if (buttonType == ButtonTypes.playInWorldVideo) { videoController.PlayInWorldVideo(); } else if (buttonType == ButtonTypes.pauseInWorldVideo) { videoController.PauseInWorldVideo(); } else if (buttonType == ButtonTypes.stopInWorldVideo) { videoController.StopInWorldVideo(); }
Here is what happens in the new functions:
displayVideoIsPaused
variable is set to false.Now you have a simple and working in-game video player! You can start, pause, and stop the playback of videos. But you can go one step further.
See the text-display over the video-screen?
This is where you can add a timer-display. Start by adding this variable to the scrip, which will hold the 3d text mesh:
public TextMesh timeDisplay;
And assign the TimeDisplay
object from the hierarchy onto it.
Then add DisplayTime
:
private void DisplayTime() { //1 string minutes = Mathf.Floor ((int)displayVideoPlayer.time / 60).ToString ("00"); string seconds = ((int)displayVideoPlayer.time % 60).ToString ("00"); string lengthMinutes = Mathf.Floor ((int)displayVideoPlayer.clip.length / 60). ToString ("00"); string lengthSeconds = ((int)displayVideoPlayer.clip.length % 60). ToString ("00"); //2 timeDisplay.text = minutes + ":" + seconds + " / " + lengthMinutes + ":" + lengthSeconds; }
This is what each part does:
Also add this code to Update()
:
//1 if (displayVideoPlayer && timeDisplay) { DisplayTime(); }
DisplayTime()
only executes when both an in-game video display object and a time display exist.Try it out! Your video-screen will now display the elapsed time.
You can also use videos with a built-in alpha channel! A lot of stock-video sites like Videvo offer these. Sadly only certain video-types and codecs are supported. On Mac you use .mov files, and on both platforms .webm. The full details can be read on the page for Video Transparency in the Unity Reference Manual.
Note: It’s possible to transform videos with transparency that aren’t .mov or .webm files, but it can be a bit tricky, but here is the highly abbreviated version:
First you need the tool FFmpeg. Navigate to the folder where the file ffmpeg is stored (ffmpeg ▸ bin on Windows, on Mac you just get the file without any others) in you command-line-terminal. Also make sure the file you want to transform is in there too. Then run this command: ./ffmpeg -i ORIGINAL_FILE_NAME.mov -c:v libvpx -pix_fmt yuva420p -auto-alt-ref 0 NEW_FILE_NAME.webm
.
This should create a new file with the necessary encoding.
Luckily there is already a video with transparency available in your project-files, and it’s called ColorHeartsTrasparent.
In addition to the Video Player you also need a custom Render Texture on the object that will be made transparent. Take the TransparentVideo material and assign it to the DisplayMesh object.
Then set the videoFilePath
in the VideoController from
private string videoFilePath = "/RW/Videos/ForestFlyover.webm";
to
private string videoFilePath = "/RW/Videos/ColorHeartsTransparent.webm";
so that the correct (transparent) video will play, and not the previous Forest Flyover.
This is how the Video Player should look like in the inspector:
And if you try it out it should look like this! :)
Note: There are also some videos with a green background that could be potentially be chroma-keyed out, like the wonderful, free-to-use Van Damme Action Video Set.
That way you could replace the green parts with transparency. There are some advanced shaders out there on the web, but they are a bit tricky to use. If you find a good solution post it in the comments below!
Another cool thing you can do is implement 360°-videos in your games. A 360°-video is a video created with a special camera (often a series of cameras attached to each other to record an entire sphere of action). You can then watch it by turning your VR-headset (or even your cellphone!) to see all different parts of it. You possibly have seen these already on YouTube and Facebook, which already integrate 360°-videos. If you haven’t yet check out the VR category on Youtube!
These videos play like the in-game videos you saw before, but on an inverted sphere instead of a flat plane. You can imagine it like a regular ball that you can already create in Unity by clicking Game Object ▸ 3D Object ▸ Sphere, but with the outside facing inwards. If you now add a Video Player and a 360°-video it will get displayed on it, and if your point of view is inside it will be as if you are inside it!
Luckily there is already an inverted sphere present in the last scene, 03SphericalVideo. Open it and check it out!
If you press play you can see the sphere from the inside, without a video playing. You cannot move outside the sphere, something that is important to not break immersion in this scenario.
If you look the sphere from inside the editor you can see that the surface is inside, i.e. you can see the sphere from being within it, but not the outside. The same thing happens when you glitch into a wall in a game and you cannot see the wall you are in from the backside.
In order to show a video on the sphere create a new script and name it EnvironmentalVideo.cs. Add it to the inverted sphere, and add this code to it:
public class EnvironmentalVideo : MonoBehaviour { //1 public GameObject displayMesh; //2 public string videoFilePath = "/RW/Videos/HundraKnektarsMarsch.webm"; void Start() { //3 videoFilePath = Application.dataPath + videoFilePath; //4 var videoPlayer = displayMesh.AddComponent(); videoPlayer.url = videoFilePath; videoPlayer.isLooping = true; videoPlayer.renderMode = UnityEngine.Video.VideoRenderMode.MaterialOverride; videoPlayer.targetMaterialRenderer = GetComponent(); videoPlayer.frame = 0; //5 videoPlayer.Play(); } }
Application.dataPath
so that it also works in a deployed buildThen do not forget to assign the sphere-mesh to the displayMesh
variable.
Note: Sadly implementing VR-videos directly from YouTube does not work, which is why you need to use a local file here. If you can find a good 360° file you can still implement it. The stock video site Videvo and Wikipedia have a few good ones you can experiment with!
Try it out! If you press play you should be inside the video-sphere.
Note: You can create you own Virtual-Reality game or app by looking into the Google Cardboard SDK for Unity, which comes with its own premade first-person VR-Controller! If you add it to this scene and deploy it to your phone (or even better if you have your own inexpensive Google Cardboard VR Headset) you can point and look in any direction.
Note: It is technically possible to sync events in your game to the framerate of a video, but this can have unforeseen repercussions. Synchronized events that work with regular videos might possibly not work in 360°-videos.
Enjoyment in Virtual Reality especially is tied to a smooth framerate, otherwise lags and skips will destroy the presence of the player, and cause VR-sickness. In these cases make sure beforehand that synchronization would work.
You can download the final project here, or at the top of this tutorial.
And you learned how to implement videos into Unity, nice work!
There are multiple ways to now use these features in a game. You could go the established way and use it for cutscenes and other videos, like credits. Or you could implement in-game video-players, which display what you recorded before. If you implement video-recording you can show the last ten seconds in an instant-replay, which is perfect for local multiplayer games like Worms.
Flying Over Forest 3 (Royalty Free Licence, Videvo)
Hundra knektars marsch på Forum Vulgaris (Creative Commons Attribution-Share Alike 4.0 International Licence, Jan Ainali)