Modfy is a purely browser-based privacy-first video tool capable of performing tasks like converting, compression, etc without uploading your files.
I thought it would be cool to process video in the browser using Web Assembly and FFmpeg
The project is also open source and based on https://github.com/ffmpegwasm/ffmpeg.wasm
Let me know what you think!
Just wanted to add I really like the "no-upload" programming paradigm a lot. I use browser native webgl rendering internally for example to automate generation of static digital art assets. All samples and finished images are transferred via http. And its easy to run on any device: just fire up chrome and enter url.
I'd like to do the same for the video asset generation pipeline as well. For which I do most ffmpeg processing on host. WASM build allows for browser based video generation on the fly ;)
modfy.video - 258 seconds
native ffmpeg - 23 seconds
So it looks like performance definitely takes a hit due to WASM, but I'm sure performance will improve, and this is technically a very impressive achievement. The UI is really nice, and the privacy benefits of remaining client side, whilst delivered through the Web are great (i.e. nothing to install locally).
I'm one of the co-founders @ Zamzar so really interested in this space. Congrats to @cryogenicplanet on this project - you've done a great job with it!
Yeah this version doesn't have hardware acceleration, but if I am not wrong WASM has hardware acceleration in development or coming soon.
Regarding the slowness, can you comment on why you think WebAssembly is slower than native? I'm sure some remarks would be very helpful to the Wasm team. Thanks!
On the other hand SIMD support in WASM is in progress: https://www.chromestatus.com/feature/6533147810332672 (preview available on chromium)
I'd say the overall performance of the tool itself is a bit mixed. There are some task where it performances really well (trimming, converting photos to montage) but some where it is horrible(transcoding to VP9)
The notes you mentioned sound like fundamental ffmpeg issues rather than problems with the implementation of this tool. It can usually trim without transcoding (much of) the file and encoding VP9 is notoriously slow.
How does running a transcode job in a browser provide "security and convenience" vs a native app?
Basic things like not being able to get stuff from CPU to screen without multiple memcpy's are all problematic too...
Giving it has to be universal, i imagine where a LLVM bitcode op might translate to 1, 2 or 3 native opcodes, with WebAssembly you might with much more opcodes in the native output.
I mean, of course they will be able to make it better with time and effort, but i dont think we can spect a much bigger speedup in comparison with the native equivalent.
Note: this is all very speculative, but by looking at it, it looked great as an accelerator, specially for math stuff, but i imagine that it will be hard even to beat the Javascript code for generic stuff giving how much it was already optimized for it.
If any compiler expert here that is familiar with it want to correct in my assumptions, please feel free to do it.
(Not event touching in issues like the ones you mention, like memory copy, where the compiler cannot translate the thing in a more optmized manner giving it cannot speculate about the environment and optimize for it)
- By default, it's set to All Features instead of Basic Features
- The app repeatedly says it doesn't upload your files, yet there's a big button in the app saying Upload File. Perhaps "Your files are never uploaded" should be reworded to something like "Your files never leave your device"?
- It also was not obvious how to get started with the app. Perhaps some text could be added to the play button?
Someone opened an issue related to this too, so will update it on that issue https://github.com/Etwas-Builders/modfy.video/issues/88
If you have control over the left file picker, it might make sense to change the label to "Click or Drag to pick a video" or similar.
Really great to see this -- I once worked on a cloud-based video editor and the cost of running ffmpeg in the cloud was surprisingly prohibitive.
Already tracking the label issue here https://github.com/Etwas-Builders/modfy.video/issues/88 will resolve that tonight
I think that's another great thing for me with this product, is that it almost costs nothing to run it (Thanks netlify)
Question: Are you compiling FFmpeg to WASM, or FFMpeg's libav and interfacing with that directly?
Also for anyone curious, the author of ffmpeg.wasm[1] has an excellent guide on compiling FFmpeg to WASM:
https://itnext.io/build-ffmpeg-webassembly-version-ffmpeg-js...