Dynamic Panels Test for Video Rendering

🪟 Beyond the window

I’ve been growing a bit impatient with the “Loop Windows” at the top of the video edits that are generated when you finish recording.  The motion-blurred animation upgrade helped for sure, but there is still so much more we could try.

I wanted to test something more dynamic where video panels would render as animations, repositioning and resizing themselves so that the whole frame would evolve to fit all the loop videos.  With this in mind here’s a prototype of what I had in mind that you can play with.

🔬 Why, and how? Let’s get into it

I have been thinking about how Muser Studio arranges things on screen when you have more than one video going at once.  Today the main video fills the frame and the loops are pinned on top as smaller windows.

How about a layout where every clip gets a proper rectangle of its own and the whole frame is used, like a mosaic that rearranges itself as you add and remove parts?

The little prototype above was to see what this could look and feel like.  The approach is an well-established graphics idea called BSP or Binary Space Partitioning. Take the frame and cut it in two. Take each half and, if you still have clips to place, cut it in two again - until every clip has its own cell. Each cut is a left/right or top/bottom split, and it need not be down the middle, so an off-centre cut gives you cells of different sizes - exactly what you want when clips are different shapes. The magical bit?  No overlaps, no gaps.

🤦‍♂️ The catch

Most ways of doing those cuts look bad - a wide clip squeezed into a tall sliver gets cropped to nothing. So the prototype searches rather than splitting blindly: for each split it tries both axes and a range of positions, scores how badly each clip’s shape is squashed by its cell, and keeps the best. With a few clips it tries every grouping; with more it falls back to sensible candidates under a time budget.

The “hybrid” part was a reaction to a sad truth I found: pure tiling is not always nicest!  On a tall phone-shaped frame with one, two or three clips, strict tiling looks awkward, so those cases may benefit from hand-tuned placements instead. Everything else goes through the BSP search. The layout picks whichever suits the shape of the problem.

🧪 Welcome to my Work in Progress

Right now this is only a layout study. We’ve got coloured panels wearing a their details, like nervous hopefuls at a speed-dating event.  No videos yet. There are sliders for corner radius and border, a control to switch the host frame between portrait, landscape and square, and buttons to change the panel count so we can watch it reflow. And it does reflow nicely - add a panel and the others slide and resize into place, remove one and it fades while the rest close the gap. That animated rearrangement is what I most wanted to confirm felt good, and it does.

There is plenty still to figure out: the panels need to display real video not solid colours, the main video should probably carry more weight than the loops, I would like per-panel colour and some motion as clips appear, and none of this is connected to Muser Studio yet.  But the core question, “can a self-arranging BSP mosaic look interesting and move well?” - now has a promising answer.

Muser Studio - available now for iOS

Photo credit: Julio Wolf - Shape