It uses GLM-5.2 fast via Fireworks to generate the scripts and image prompts and, like I said, Nano Banana 2 Lite for the images, gpt-4o-mini-tts for the narration, and ffmpeg to string it all together and add the Ken Burns zoom effect (which still has a shake I haven't been able to get rid of). The video compilation proved to be the blocker once the rest was in place, but I was able to speed that up by putting it on a 64 vCPU EC2.
The cost might be the most interesting aspect as the short form videos tend to be about 25 cents. Almost 90% of that is the images, which are 3.336 cents a piece. Of course, running the big 64 core EC2 to allow for the creation isn't cheap.
It seems like on-demand AI video is coming, and I thought this was an interesting demo of how close it might be in at least one narrow video domain.
For example, I have several posts that explain concepts in detail, with supporting images. They're written in Markdown, and I keep refining them until I'm confident that the average person can read them and immediately understand the concept.
The problem is that text is becoming a less popular medium for learning, while short form videos are increasingly being used to explain the same ideas.
From what I can tell, this tool only lets you provide a prompt. Can it turn my existing Markdown posts and images into videos, or is it mainly intended for generating videos from simple prompts? I'm looking for something that preserves the explanations I've already written rather than creating something entirely new.
Style 1: https://www.youtube.com/shorts/vGjrv2zjfsM
Style 2: https://www.youtube.com/shorts/6KoRHpAZ4zk
These are the two styles that consistently keep me watching. I can't really describe them well in words, so I linked the videos directly to avoid losing important details in the translation from visual format to text.
Maybe someone with more experience in video production can explain what techniques are being used here and why they're so effective at holding viewers' attention.
https://www.youtube.com/shorts/IhGR8AC3a_g
Uploaded on sep 24, 2024.
I wasn't happy with how it turned out, so after making 3–4 videos I dropped the idea.
Later, I came across several different video styles that consistently kept me watching. That's when I realized what my videos were missing they simply didn't have the same ability to capture and hold attention.
But i never managed to generate those styles after dozens of attempts and gave up.
One small quirk I noticed with the generated short video: for each new slide/photo, there is a very gradual "zoom in" effect that occurs, which causes text to jitter a bit and is a bit disorienting. Not sure if that is specific to the "The Great Molasses Flood of 1919 in Boston" prompt I used or not.
Also, is Claude involved in the video generation itself? I noticed the generated video has "By Claude" in the top left corner, but the logs seem to indicate that only GLM-5.2, Gemini Flash, and GPT-4o are involved (just curious!)
Quotations from Chairman Mao Tse-tung (Short form)
The audio narration is great and the documentary is accurate. Improvements could include more pictures or animations.
This is a very great tool and framework! Thank you so much.
---
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, *self.kwargs) File "/home/ubuntu/byclaude/video/harness/pipeline/pipeline.py", line 829, in build_clip clip = round(dur + TAIL, 3) TypeError: unsupported operand type(s) for +: 'NoneType' and 'float' Video build failed (pipeline exit 1) — please try again. Powered by AI · by Claude · ~
I think that simplicity is its main selling point at the moment as its otherwise a pretty crowded marketplace.
Best of luck with it, however you decide to progress it.