Bit Byte Bulletin

Start Your Learning Journey in AI and Machine Learning For Free

OpenAI: Say Goodbye to Green Screens: This AI Creates Realistic Videos in Seconds

by · Published · Updated

Sora is a an AI model developed by OpenAI team that can create realistic imaginative scenes from text prompts. It can create videos up to a minute long while maintaining visual quality and adherence to the user’s prompt.

Here’s an example from the OpenAI site:

Prompt: Animated scene features a close-up of a short fluffy monster kneeling beside a melting red candle. The art style is 3D and realistic, with a focus on lighting and texture. The mood of the painting is one of wonder and curiosity, as the monster gazes at the flame with wide eyes and open mouth. Its pose and expression convey a sense of innocence and playfulness, as if it is exploring the world around it for the first time. The use of warm colors and dramatic lighting further enhances the cozy atmosphere of the image.

Few sample videos generated by OpenAI Sora

Prompt: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.
Prompt: A movie trailer featuring the adventures of the 30 year old space man wearing a red wool knitted motorcycle helmet, blue sky, salt desert, cinematic style, shot on 35mm film, vivid colors.
Prompt: Several giant wooly mammoths approach treading through a snowy meadow, their long wooly fur lightly blows in the wind as they walk, snow covered trees and dramatic snow capped mountains in the distance, mid afternoon light with wispy clouds and a sun high in the distance creates a warm glow, the low camera view is stunning capturing the large furry mammal with beautiful photography, depth of field.

How does OpenAI Sora work ?

Sora is a diffusion model, it generates a video by starting off with one that looks like static noise and gradually transforms it by removing the noise over many steps. The videos can be up to 60 seconds long.

Sora builds on past research in DALL·E and GPT models. It uses the recaptioning technique from DALL·E 3, which involves generating highly descriptive captions for the visual training data.

Limitations of OpenAI Sora

While Sora excels at bringing text to life with video, it’s still under development and has areas to improve. In intricate scenes, it might not perfectly capture real-world physics, leading to occasional glitches like a cookie bite magically disappearing.

Similarly, it’s learning to navigate spatial relationships and might sometimes mix up left and right. Additionally, precise descriptions of events unfolding over time, like a specific camera movement, can pose a challenge.

What are the risks of OpenAI Sora ?

  • Generation of harmful content
  • Misinformation
  • Biases and stereotypes

How will OpenAI ensure safety of the Product usage ?

OpenAI is building tools to help detect misleading content such as a detection classifier that can tell when a video was generated by Sora. 

How Can I Access Sora?

Sora currently available to only “red teams“(experts researchers).They will try to assess critical areas for harms or risks. We are also granting access to a number of visual artists, designers, and filmmakers to gain feedback on how to advance the model to be most helpful for creative professionals.

Tags:

You may also like...

Leave a Reply