AI Generated Video to Animation Sprites

This image technically features two characters, but for demonstration purposes, we will focus on the girl with red hair.

I will prompt the AI platform to generate a character sheet for the red-haired character, including front, side, and back views. The AI will then return the following image:

Once you have the character sheet, use a photo editor to generate a single view of the character. Clean or fix any artifacts in the image, or make any necessary design changes during this step.

Although it is a nicely generated and fixed image that would work well for the process, it is recommended that you change the background to a chroma green or blue color. This allows you to remove the background and create a transparent set of sprites later. Below is an example:

Once you have your character image ready, you can upload it and generate a video using a prompt that specifies the required animation or action. In the example below, I am using Google Vids, which utilizes the Veo model to generate videos.

The image below shows that I have selected the “Animate an image” option and uploaded the reference image into the image slot. I can then specify the animation prompt in the designated area and press “Generate.”

For this example, I provided a prompt to generate a video of this character jumping for joy. Below are some frames of the result I received:

Upload the sprite / image

 

Once finished, and if you are happy with the resulting video, you can download it to your computer. Ensure that you obtain the highest resolution possible.

Step 04 : Transparency and Background Removal

Import the video into your favorite video editing application; in this case, I am using DaVinci Resolve to remove the background. I used the 3D Keyer filter effect to remove the green chroma background pixels, rendering the entire video with a transparent background. Once finished, I can test it by placing an image layer behind my main character footage layer.

Step 05 : Export final frames

Once finished, you can export the final frames in either TIFF or PNG format from your video editor. Ensure the format you select for export supports transparency. For this particular example, I have selected the TIFF format with the RGB 16-bit codec and checked the “Export Alpha” setting in the tab.

Once exported, you will have all the animation frames in the form of sprites that can be used in sprite sheets.

Bonus

If you have a character animation video and want to generate sprites or a sprite sheet directly from it without using video editing software, here is a website that will help you do exactly that in just a few clicks.

This tool will let you upload your video directly and then convert it into a sprite.

You can specify the number of frames required and the output format for the sprite, which can be either individual frames or a compiled sprite sheet.

The second bonus tool is an online sprite generator that can also be used to transfer animation from one 2D character to another. This process aids in retargeting already created animations from one character to another using AI, eventually saving time.

Upload the sprite / image

 

Upload the reference animation

Transfer the animations