What I Learned from Creating an Unreal Engine Cinematic
Unreal Engine is the video game engine behind Fortnite that has been used by many game developers as the building blocks for their creative ideas and interactive experiences. However, it’s becoming increasingly popular to see game engines be used outside of the game industry due the (seemingly) endless possibilities that are provided by it’s real-time 3D rendering capabilities. One noteable mention is Disney’s Mandalorian, for this series the team built entire 3D environments using massive screens that rendered real-time digital scenes through Unreal Engine which proved a great asset to the actors on-set, providing them context during their performances. Having a game engine be used as an asset in both the game and film industry excites me as it provides a way for both industries to dip their toes into each others fields. Unreal Engine provides a platform for game developers to create visually impressive cinematics while also allowing film-makers the digital freedom that game devs innately get when working within virtual worlds.
Now you have a bit of knowledge behind how talented groups of people have used Unreal Engine, I’m going to share with you what a less talented lone ranger learned when exploring Unreal Engine cinematics for the first time. Before developing this cinematic I had been working as a technical designer within the engine itself for around 2 years, so I was confident with the fundamentals of the engine and the visual scripting language of blueprints. However one thing I’d never had the pleasure of delving into was Unreal Engine’s sequencer to create in-game animations and cinematics. I had a lot to learn…
So, I wrote a short horror story based around a nightmare I’d recently had and set to work transforming this from dream to reality inside of a digital world.
10 Tips I Learnt The Hard Way When Creating A Cinematic Through UE4
1. Set up your projects exposure settings
I spent an awful long time setting up my scenes lighting, models and artstyle using post processing (which I’ll talk more about later) only to frustratingly realise that my cameras would render my scene out to be too light or dark. I eventually realised that my project was automatically adjusting the maps exposure levels by default! To stop this make sure you open your Project settings and look for the ‘Auto Exposure’ tick box and untick it. To apply this change you will have to reboot your project. Now you can be sure that the exposure will be set only within the cameras that you use in your scene.
2. Test the size of your map
When you’re not developing a level with a player in mind it’s easy to cut corners and only think about the layout being visually pleasing through the camera lens, not necessarily geometrically perfectly aligned or to scale as you have full control over what the player will view. However, I’d advise you to AT LEAST check the size of the assets you are using and how they react with lights and particles. You will no doubt want to be using these to spruce up your scene. I made a big mistake of dragging in all of my 3D models, maticulously placing them and then realised that the assets were so big that the lights and particles I had started to place within the scene were barely noticeable even at their largest size and brightness.
If you do end up making this mistake though then have no fear, to fix this I added an empty actor to the level and made this a parent to everything within my scene. I did this by highlighting everything within the World Outliner and then right-clicking and choosing ‘Attach to’, choosing my empty actor. This then kept everything into place while I scaled the scene actor down.
3. Make use of the assets on the marketplace
When you’re a lonely dev it’s very easy to become overwhelmed with the amount of work that is to be done to make a half-decent digital creation. You need 3D models, textures, normal maps, lighting, particles, audio, animations and not to mention a cohesive artstyle that will hopefully scream out that the work is uniquely your own. It’s exhausting just to think about.
But lucky for us, and certainly for me, the Unreal marketplace exists! This was a life saver for me as I hadn’t modelled in years and have never animated in my life, so I couldn’t rely on any skill in those areas. I also managed to find a post processing package called Chameleon which I fell in
love with. It offers a tremendous amount of visual customisations, from cartoonish styles to horrific decayed overlays and even camera effects such as shaking, kaleidescope and so much more. You can even blend these effects based on the camera depth distance. I spent a long time tweaking these values to get a look that reminded me of a storybook, with paint-like washed out edges that blurred in the distance and etchings within the shadows. I’m very pleased with the end result.
4. Set up your cameras and sequencer
When setting up your cameras make sure to use ‘Cine Camera Actor’. These cameras give you more options to control such as focus or ‘look at tracking’ that enables the camera to stay facing an actor even when moving (I imagine this would be great for in-game dialogue scenes if you don’t necessarily want to manually create the camera cuts). When using the focus settings I found it very useful to use the ‘Draw Debug Focus Plane’ to make sure that I was focusing on the correct parts of my level as sometimes it can be hard to see in editor (especially on my laptop). Bare in mind though that this focus plane will be visible during the cinematic render so be sure to turn it back off once you’re happy with your focus settings!
The sequencer can be found under ‘Cinematics’ in the toolbar. To add your cameras into the timeline you can use the + icon and bind an existing camera to that viewport. You’ll then have access to the actor and be able to track certain aspects of it such as it’s transform or visibility settings, manually updating these throughout the timeline using keyframes. Another warning though, location keyframes DO NOT update manually, so if you move a camera within the editor then do make sure to manually key in these changes in the sequencer (annoying, I know…) At least you can also add an audio track :) More on Sequencer in point 8.
To view the timeline in real-time click the camera icon next to your camera cuts, this will lock the viewport to these cuts, allowing you to play through the squence and watch it back. When you have a camera selected you can also pin its view to the bottom right corner of your viewport, this way even when you click onto a different actor it will still persist. I found this extremely useful for lining up subjects within my camera view.
5. Including video is harder than it seems…
Seeing as I’m not much of a 3D modeller or animater I had the bright idea of including videos within this cinematic. This opened up more possibilities but also many more problems…
Firstly, I needed a way to load in certain videos at certain points throughout the sequencer because the tech designer in me screamed at the idea of having multiple videos running on a loop throughout the entire render, not to mention, sometimes I would need to reuse scenes without the videos present or with a different video. My solution: A blueprint actor that would act as a host for multiple videos to be streamed onto it’s plane mesh through a playlist. The videos would be updated through events called within the sequencer at specific points (more on this in point 8).
When inserting a video into Unreal Engine a File Media Source is created. I then added a Media Playlist and collated all of these sources. This playlist is then used within a Media Player. After setting up this Media Player I could then create a Media Texture for the Media Player to output to. Right-clicking this texture gives you the option to create a Media Material to use on any object. A more in depth explanation can be found on Unreal’s website.
I ran into a few issues while using videos in my scene however…
I’m not sure if it’s a bug but I noticed that often the Media Player would lose reference to the playlist, meaning I couldn’t see any previews in editor. To fix this I’d make sure to always have the Media Player up and loaded in the background. This way I could also play through each video and see it playing in editor so I could preview how the videos looked in the level.
When rendering out my sequencer as an mp4 I realised that the playback for the videos was playing far too fast, unable to match that of the sequencers rendering speed. Upon googling it appeared it might not have anything to do with matching the framerate as the video will be playing at an in-game framerate which could, in theory, be set to the same as the sequencer, but when creating a high quality render the sequencer seems to record at a slower rate. Or that’s just my limited understanding of what’s going on at least… One way to get around this would be to convert the mp4 into an image sequence and then create a file image source for them all. The playback should use these images instead of an actual video and this will fix the playback speed issue. However, if you were to use this in a game package build then it would be huge! I personally work on a laptop which already sounded like it was going to take off half of the time while I ran UE4 so I decided against this workaround. Instead, I exported my videos again at half speed and used these slow-mo version of the videos as the references to be rendered throughout the project.
Once I had created a blueprint actor to output these videos to I placed multiple actors around my scene. I was under the impression that I could call the same functions on individual instances of the blueprint. E.g. if I had two camera planes, plane 1 and plane 2, playing video 1 and I then told plane 2 to show video 6, then I would expect plane 1 to continue to play video 1 while plane 2 ran through the function and started to play video 6. However, this was not the case as the rendering of the video media player would not get instanced, meaning that it would output the same video for every actor using it.
In summary, after all of this, I was able to create a blueprint actor that had a Static Mesh component of a Plane. On this plane I would use my Media Material and create functions that could be called from the Sequencer to update the playlist track to be played on a loop as well as the Media Material that should be used as some videos, like the one you can see in the preview, had green screen that I removed within the media material itself. I’ll take you through this in detail in point 8.
6. Keying Green Screen Through Materials
Luckily not everything turned out to be as difficult as I though. Keying green screen is surprisingly simple! There are many tutorials online so I would advice you to go and seek out an expert for this, as I have taken a look and there are many more ways to key green screen within unreal since I set up my material in 2019. But, this set up worked for me so I shall leave a screenshot below for anyone who might find it useful.
Once I had created this material I then made material instances of it for the individual videos. I have used Scalar Parameters (Power, Emmisive, EdgeOffset, ColourGain) that can be tweaked throughout every instance of the material to better key each individual video.
The only pain about keying the green screen was realising that I really need to get myself a better green screen set up! I lacked consistent lighting throughout the green screen videos so it was hard to cleaning key them without clipping into other parts of the video as a lot of them had dark edges on the video due to the poor lighting but I made it work as best as I could.
7. Using Sprite Animation
When showing movement throughout the 3D environment I decided to use hand drawn sprites to keep to the ‘storybook’ feel. Using Sprite animations in Unreal proved to be very simple and effective. I created these sprites in Photoshop first, using the videos I had taken of the momevent that I wanted to include and drawing over them every few frames. To do this I would import the videos frames to layers and then create a new layer over every 6 or so frames and trace over the top. Once I created as many of these as I wanted I could then delete the video layers and keep the drawn ones in the animation frame timeline, previewing how the animation would look on a loop. This effect of tracing over video frames to create animation is called rotoscope animation. Using a layers to sprite Photoshop script (developed by JesseFreeman) I then turned all of these layers into sprite sheets which I could use in Unreal Engine.
Creating the material for these sprite sheets meant using the Flipbook node and Scalar Parameters to control the Row and Column numbers as well as the Speed for each sprite. This way I could create instances of this material for each of my sprites quickly and easily, adjusting the speed and column numbers accordingly. (If you’re wondering why I multiplied the texture for the final color output it’s only because this made the outlines bolder which I felt helped the sprites stand out better among the 3D backgrounds)
8. Editing Animation Sequences
Remember when I told you I’m no animator? Well, I’m not. But I found a way to fake it (sorry to all talented animators out there, look away now). Originally I was going to rely on my videos and sprites but once I saw that Unreal Engine marketplace had a Scanned 3D people pack for free then I just had to try it out!(in hindsight I probably went a bit over the top with adding things to this project…)
This pack came with free rigged models which each had a handful of animations, however none of these really fit the vibe of the horror story I was trying to tell. In hindsight I should have downloaded fbx versions of these models from their website and used these to get some aimations from Mixamo, but at the time I did not know this option was available to me and was only given Unreal Engine ready versions of the models (derp). So instead I opened up the animation sequence for these models and added new keys to the animation timeline. To do so I selected the + key icon and chose which rig bones I wanted to manipulate using the ‘Additive Layer Tracks’ drop down. I could then move parts of the model into the position I wanted and key this onto the animation track. This certainly did not produce clean and intuitive-looking animations but it did the job. I just told myself, it’s a horror, a bit of janky movement will add some disturbing, uncanney valley ambience, right? It’s a design choice, not a flaw ;)
9. Sequencer Set Up and Events
I can’t talk about making an Unreal Cinematic without delving into Sequencer itself! As stated previously Sequencer can be found under ‘Cinematics’ in the toolbar. There are two main types: Level Sequence, used to create a sequence for the level you have open, and Master Sequence, which should be used to contain multiple level sequences. To keep it simple I worked in one level and only used a level sequence for this project.
What I love about sequencer is how much it can manipulate within the scene. Once you have a sequencer open you can either manually add actors to track (by selecting the green +track button up the top left) or you can easily click on an actor within the editor and view a small icon next to customisable values (such as its mesh, visibility, etc). Once an actor is added to the Sequencer you can click the plus next to it to track everything related to that actor such as any inherited components such as its mesh. From here you can even do things like switch out materials on each individual static mesh component during runtime. You can also add a fade in / out track which works handy for transitions between camera scenes.
One neat thing is that you can also trigger scripted functionality within sequencer using Sequencer Events! With the event track I was able to have control over the exact frame where I wanted my functionality to begin. This was especially useful when switching out which video I wanted playing within my video blueprint that I had set up. You can read a more in detail explanation of Sequencer Events on Unreal Engine’s website, however I will take you through my blueprint set up for this.
When creating these events I bound them to functions within the video blueprint that I had already set up. There were individual functions for each video. All of these functions ran the same functionality through another ‘Switch Media’ function, however the ‘New Media Source’ variable would update, matching the Media Player index that I wanted to be played.
The Switch Media function would then update the Main Media Material which was a public variable that I could update in the Sequencer timeline itself. Then I would set the Current Playlist variable integer to have a reference to it for when I wanted the video to loop. Then all I had to call was Open Playlist Index at the number I had called Switch Media with.
The next function isn’t my proudest piece of work (heck, none if this really is) but I had to make a hack to loop each individual track. Within the Media Player there is a ‘Loop’ tick box, however this will only loop the source that it’s outputting and if that source is a full playlist then it will loop the whole playlist, not the individual videos of that playlist. So in this case I used ‘Bind Event to On End Reached’ to recall the SwitchMedia function once again when the video had played through to the end. I had to add a delay before calling ‘SwitchMedia’ again to stop the playlist loading in the next video before replaying. Hacky I know, in my day job I stay away from Delays, but it worked for what I needed so don’t judge me too hard if you can…
To preview these events and see if they were triggering correctly / at the right time I had to see this in-game. To do so I edited the level blueprint to play my level sequence at BeginPlay. This mean that whichever part of the Sequencer I had highlighted would play within the editor as soon as I hit the play button. All I needed was a to set up a variable for the level sequence that I wanted to play and create a level sequence player to render this. Nice and easy!
10. Export Settings
Below are my export settings for this cinematic. I would advise for you to take a few test exports before running your final render just to review first as this can take a long time depending on your project size and length. To do these test renders I would always tick ‘Use Compression’ to speed up the process. Adjust the compression quality to your needs, with 1 being the most compressed and 100 being the best quality. I also like the fact that you can account for warm up frame count and a delay before that warm up to start running particle effects to make sure your scene looks it’s best by the time the sequence starts to render.
Sneaky 11. Tying Up Loose Ends in Premiere Pro
You thought we were over? Not quite! Sequencer is great, however nothing quite scratches that video editor’s itch like good ol’ Premiere Pro and I couldn’t help but drag in the exported AVI into premiere to polish off the final video. This helped with including titles and more professional looking transitions between scenes as Unreal Engine only offers a black fade in / out transition and sometimes that just doesn’t cut it. Working in Unreal Engine also meant that the video render would be slightly different each time, with particles being rendered uniquely each time. Polishing the final video in Premiere meant that I could render out certain parts of the cinematic that I wasn’t 100% happy with multiple times and edit in the best render. I was also able to have more control over the video speed, as I explained earlier, I was having trouble with the videos playing faster than what the cameras were rendering.
Editing the final version in Premiere Pro also meant that I could add the audio track in here too, meaning that I didn’t have to render the audio through UE4 which would have slowed down the render time (which was already a few hours).
What I Would Change For Next Time…
Ultimately I had a blast making this cinematic and, as you can probably tell, I learnt so much during the process! However, I still feel I have lots yet to learn and when opening back up the UE4 project to get the snapshots for this article I still cringe at a lot to do with this project. So here’s what I’m going to make sure NOT to do in my next cinematic…
1. Have a Storyboard and Stick To It
I feel like I did a good job at creating a distinct artstyle through using the post process, however I do feel like the overall vision and delivery was messy and too busy for a horror piece. Even from the audio itself, I felt there needed to be more moments of silence, pauses and tension building scenes. I definitely got carried away with what I could learn and add to the project instead of making a storyboard and sticking to it, which I feel impacted the visual narrative of the video greatly.
2. Simplify Ideas
This goes hand in hand with point one. For my next cinematic I want to focus in on using one form of animation, either sprite OR 3D OR video. I feel that including all three added to the busy feeling and could have distracted the viewer from the story too much. The visuals need to be an addition to the narrative, not a distraction from it.
3. Get a PC
As soon as I finished this project I just knew that I would need an upgrade from my gaming laptop. The closer I got to finishing the cinematic within unreal, the more it sounded like it was about to take off at any moment! I had to reduce the quality settings within unreal itself and only move around in editor in unlit mode just to keep it running semi-smoothly. If I ever want to play around in UE5 with all it’s wonderful triangles then it’s about time I trade in this ol’ gal!
4. Tidy Folder Structure and Naming Conventions!
I am bad at this with personal projects. When I’m working with other people it’s built in me to keep things tidy but when I’m working on my own it’s too easy to throw these values out of the window in favour of a speedy workflow, but this always comes back around to bite you in the ass.
Keep things neat: Comment your blueprints! Use folders and group actors within the World Outliner! Create clean and understandable folder structures and stick to them! Name your assets accordingly to their type! Name variables appropriately! Just do it…
If you reached this far, thank you for reading and sticking through my rambles. I commend you! I hope this was somewhat interesting if not educational. Overall Unreal Engine is a great way to create atmospheric scenes without having to leave the comfort of your bedroom (which is perfect in our current climate).
If you’d like to check out more of my work or what I’m up to then you can follow me on YouTube, Twitter or Instagram @YagmanX . I also have a free horror game ‘Perfection’ on itch.io and a music EP ‘Sweater’ available on all major music streaming platforms :)
