I’ve pulled together some drafts of my ideas for the design of the timeline portion of the Lumiera non-linear video editor (hopefully, the successor to Cinelerra).
The annotated version (explaining some of the finer points):
The un-annotated version (to show the sketch better):
cehteh | 22-May-08 at 6:09 am | Permalink
First of all: this looks very good
Now some annotations:
- the curves need to be *way* bigger, this could be done with some kind of zoom or a dedicated curve view of course (having the small curves in the effect is good too)
- The ‘renderpipe’ in lumiera is a graph, effects, decoders, encoders, .. everything. We already (?) agreed to make tracks like trees, a video track with 2 sound channel could be represented this way, but different layouts are possible, yet lumiera will be able to load other projects as subproject into this hierarchical tracks. Thumbnails (for video, waveforms for audio) and compositor/viewer windows are then attached to nodes in this graph (for example you can have two viewers, one showing a track before an effect got applied, another after it got applied). You might notice this ‘tree’ is not sufficient to represent a graph, this needs further tools, I am thinking about some ‘wiring’ over the timeline, kind of patch panel, others want a node editor, finally we like need both :).
- You showing that effects and tracks are grouped by dashed boxes, which is rather limiting to the layout how you can organize things on screen and how things get wired together, we need some more flexible way for this, also sometimes one wants to see only video or only audio and such.
- For ergonomic working the (+) and arrows you suggested for dragging are somewhat small, aiming to small screen areas for clicking is no fun, I already find that very inconvenient in cinelerra (hitting and dragging frame borders). One solution would be to make this areas bigger and/or use keyboard modifiers, then the whole effect/track area acts like you nudge the start/end point when dragged together with shift or ctrl or so.
- You mention ‘composite’ effects, these will be really essential for Lumiera, we plan to implement very basic effects only and more complicated effects will be compiled by stacking some simple effects together. I think using the ‘tree’ feature as mentioned above will be more convenient and orthogonal than make a dedicated composite track.
- You show that some things (audio) tracks can be hidden, for tree like organization this would be just a tree collapse.
Christian
ichthyo | 22-May-08 at 8:06 pm | Permalink
agreed. looks very good and moreover is well in line with the capabilities of the middle layer I am currently implementing.
Of course there are some possible more complicated features on top of what is proposed here, but, Robert, if I understand you right, you care for creating a UI design which makes handling the basic cases very steamlined and accessible.
some random interesting points
- you create sort-of “sub-tracks” within each track for each of the contained media types (here video and audio). To me this seems a good idea, as I am myself pushing the Idea (and implementing the necessary infrastructure) of the clips being typically multi-channel. So, rather than having a Audio and Video section, separate Video and audio tracks, and then having to solve the problem how to “link” audio to video, I favour to treat each clip as a compound of related media streams. Of course then you can split off (or merge in) individual streams with an separate command, exactly as you proposed
- btw., note this could include the handling of different but related tracks within one clip. E.g various separate sound pickups and overdubbed dialogue, or several video streams with different viewangle. Of course, then you’d need some additional mechanism to control the mixing or the selection of the current viewangle, probably by attaching some local lables/tags directly to the individual streams within a clip — and we’d have to consider how to handle these conveniently.
- as cehteh already pointed out, besides the simple effect stack attached to a individual clip, we are planning to have more elaborate wiring and routing capabilities. But this is no argument against your proposed design. Personally I think it’s a good idea to keep these effect stacks (as attached to an individual clip) rather simple in structure. I’d rather keep them mostly linear (with the exception of “send nodes” which can be wired to every other input port or attached to a probe monitor). Also I want to limit the media type /within/ such a stack to be the same as provided by the clip’s source, thus disallowing any non-trivial conversion.
- but outside such a linear effect stack, the output of any clip can be wired rather freely, and my preference is to bring in a rule based approach here. Specifically, you can send it up the track-tree to use the next-reachable output connection, directly (hard) wire it to the entry port of a “virtual clip” on another track, or plug it to a global pipe (master bus). Any of those non-local connections can include conversions (e.g. from mono sound to stereo)
- regarding the problem of the small handles: Richard Spindler recently showed a rubber curve widget with the interesting feature to display enlarged handles (here of the bezier ctrl points) only when the mouse is near by
- another feature closely related is to provide inline handles for fading in/out. This is not to replace a global fade automation or the transitions between overlapping clips, but in practice it’s a real simplification. Ardour provides such fade handles btw. I’d propose to extend these fade handles to the effects too. So you could “fade into” an applied effect, and consequently you could avoid to make an explicit transition to another track/clip in many cases. To make it more clear, assume you applied a colour correction to some range. Now, if you’d drag the fade-in handle of this effect, internally the Proc-Layer would create a local crossfade of the data with and without colour correction, so the effect is introduced softly and without jump.
- you mention ‘composite’ effects; if I understood you correct in our recent IRC discussion, the idea is to be able to create sort of “macro effect stacks” out of the simple effects. I am looking ahead at your ideas how to handle the more intricate HCI problems in relation with such a feature, basically because I’d consider this rather important and helpful (but non-trivial from an interface and workflow POV)
Hermann
sakalli | 27-May-08 at 4:08 am | Permalink
hi, bit of a rush, but fwiw a few quick thoughts. apologies if im missing/misunderstaning something explained or stated above already.
first off, great job clay. nice to see something concrete to think about. well done.
from a user perspective i would like to propose a guiding principle which wouol state that everything should be as minimal as possible. as little graphical clutter on the page as possible.
stemming from that idea and usability it would be preferrable if for example two clips with a crossfade can stay on the same track with a superimposed clip that refers to the crossfade. by chosing the fade you could open a dialog with specs and by dragging the edges you could make it smaller or bigger.
also would prefer if effects etc would not need its own track to make the graphical representation calmer. maybe a slight shift in hue of the track or similar solutions could be considered.
while i like the colors estethically that you have chosen, it would be useful if as much neutral gray as possible would be used because that disturbs less when judging color, even though you judge it in a broadcast monitor next to the computer monitor. the way we see colour is very relative to colour around us. but perhaps this would be up to the user to configure.
while editing i very rarely need/want to see thumbnails on the tracks itself so while useful it would be great if this would be optional. simply a way to toggle them on and off. same goes for the sound waves.
ok. will try to study the proposal in more detail and come back if i think of something else. but anyway, thanks for the work you put in this!
cheers,
/sami