Introduction
Aleph provides powerful video editing capabilities, and you can achieve even more precise control by combining it with edited first frames for exact object placement and introducing consistent subjects.
This workflow enhances your results by editing a single frame with Gen-4 Image References, then using that image to guide Aleph's video generation with frame-level precision.
This technique works best when you need to:
- Put objects in exact locations
- Add a particular subject or object to the video
- Keep your video consistent while adding specific elements
After completing this guide, you will have created a controlled video edit with precise object placement and visual consistency throughout.
Prerequisites
Before starting, ensure you have:
- Experience with Creating with Aleph
- Familiarity with Creating with References
- A video file that you want to edit using Aleph
- Access to both Gen-4 Image and Aleph tools in Runway
Step 1 — Extracting the first frame from your video
Begin by extracting the first frame from your video to use as the foundation for your Reference-based edits. The extraction process depends on where your video is located within Runway:
For videos generated in Runway
- Navigate to the session containing your video
- Position the scrubber at the first frame of your video
- Right-click on the video player
- Select Download Video Frame from the context menu
- Enter a descriptive filename for the frame
- Save the frame to your device
For videos uploaded to assets
- Open your Assets library
- Locate and double-click your video to open it
- Position the scrubber at the first frame of the video
- Right-click on the video player
- Select Save Video Frame As… from the context menu
- Enter a descriptive filename for the frame
- Save the frame to your device
In this tutorial, we will be working with the following first frame and video as an example:
| Extracted first frame | Video |
After downloading the frame, you can proceed to modify it using Gen-4 References to create the specific edits you need.
Step 2 — Editing the frame with Gen-4 Image References
Navigate to Gen-4 Image within a Runway session. Create a new session if needed and confirm that Image is selected as your generation mode.
Upload the frame you extracted in Step 1 to use as your Gen-4 Image Reference. This frame serves as the base image that References will modify according to your specifications.
The goal of this step is to generate a result that is near-identical to the first frame of your video, but with the new elements added in. In example, an optimal result would maintain the camera angle, structure, and details of the scene.
Choosing your editing approach
Select the appropriate method based on your editing needs. To learn more about using Sketch, please see the Workflow Overview section from Creating with Gen-4 Image References.
Click each approach to learn more about when one may be preferred and review examples:
Sketch-only approach
Use this method when placement and scale matter more than the specific appearance of objects or characters. This approach works well for adding generic elements in precise locations.
| Prompt | IMG_1 | Output |
add a car to the outlined area in IMG_1. keep the rest of the scene in IMG_1 exactly the same.
|
|
|
References-only approach
Choose this method when consistency of introduced objects or characters takes priority over their exact placement or scale. This approach maintains visual coherence across your edits.
| Prompt | IMG_1 | IMG_2 | Output |
add the vehicle from IMG_2 to IMG_1. keep the rest of
the scene
in IMG_1 exactly the same.
|
|
|
|
Sketch + References combined approach
Apply this method when you need both precise placement and consistent appearance of the elements you're adding. This approach provides the highest level of control over your edits.
| Prompt | IMG_1 | IMG_2 | Output |
add IMG_2 to the scene of IMG_1. keep the rest of the scene in IMG_1 exactly the same. use the lighting and color grade from IMG_1
|
|
|
|
If using Sketch to annotate the area of the video that should be modified, we recommend using a bold color that contrasts against the image. You can also add brief text annotations within Sketch to reduce prompting, as seen in the example below:
When crafting your text prompt for any of these approaches, specify that the original scene should remain unchanged while describing the modifications you want to make. Include details about how new elements should integrate with the existing lighting and color grading.
Tip: If you receive Sketch artifacts in the results, try
adding phrasing like Remove the red outline to your prompt.
Save the edited image once you achieve the desired result. This image will guide your Aleph generation in the next step.
Step 3 — Applying the edited frame with Aleph
Switch to Aleph by selecting the Video tab in your session:
Upload your original video file that you want to edit with Aleph. Next, upload the edited frame you created in Step 2 as the accompanying input image.
Crafting Your Aleph Prompt
Structure your prompt to specify that the edited input image should serve as the first frame of your video.
Include behavioral instructions for any new elements you added to ensure they animate appropriately throughout the video sequence. Describing how the added elements behave will help reinforce permanence in the generated results.
| Prompt | Input Video | Input Image | Output |
| edit the video to start on the input image as the first frame. add motion so that the car floats weightlessly, as if in zero gravity, throughout the video |
Your prompt should explicitly state that the video should start with the input image as the first frame, then describe how the added elements should move or behave across the duration of the video. Please note that asking for intense motion of the added element(s) may require multiple iterations to perfect.
If the results don't match your expectations, adjust your Aleph prompt or return to Step 2 to refine your reference image before regenerating.