Feature Digital video guru and author of The MPEG Handbook, John Watkinson, examines the next generation of TV and film and reveals it shouldn't be anything like what we're being sold today.
The next time you watch TV or go to the movies, bear in mind that you are not actually going to see any moving pictures at all. The movement is an illusion and it’s entirely in your own mind.
What is actually on the screen doesn’t move. It is a series of still pictures that are each held there for a short while before being replaced.
Currently, the obsession is for ever higher pixel counts, an approach that disregards how we actually see moving images. If broadcasters have their way, we could be on course for some ridiculous format decisions.
Intuitively, you would think that the frame rate - the number of pictures per second - would have quite a large bearing on the quality of the illusion, and you would be right. Equally, you might think that the film and TV companies had done a lot of research into human vision in order to choose those rates that have been in use unchanged for decades. Unfortunately, you would be wrong.
Let’s see how we arrived at the current frame rate of movie film: 24 frames per second. Silent movies ran even slower, at about 16fps, but when the optical soundtrack was invented, it was found that the sound was too muffled at the old film speed because the optics couldn’t resolve the higher pitches in the soundtrack. So the film was sped up in order to get enough sound quality. The use of 24fps in movies has nothing whatsoever to do with any study of human vision.
American television runs at 60 fields per second, whereas in the old world we get along with 50. Do Americans have better vision than Europeans so they need a higher rate? As many Americans are descendents of émigrés from the Old World that’s not very likely. Fact is, the rates used in television were chosen to be the same as the frequency of the local electricity supply because of fears that different rates might interfere or beat with electric light. Once more, the decision had nothing to do with human vision. The entire edifice of film and TV picture rates has no foundation and it is going to have to change.
Fields of view
Any attempt to project a film at its native 24fps risked giving the audience headaches or triggering epilepsy. In practice, every film frame was flashed twice on the screen. The projector had a two-bladed shutter and a new frame was pulled down only when one of the blades was blocking the light. That gives the illusion of 48fps, provided nothing moves.
The TV industry must have taken note that the movies were short-changing their audiences by showing every frame twice, so they thought up a similar cost-saving mechanism, called interlacing. Pictures are associated into pairs, called odd and even. In one, the even-numbered lines are simply discarded to make a field, whereas in the other, the field is made by discarding odd-numbered lines. When the two types of field are presented in rapid succession, all of the lines in the frame become visible again provided nothing moves.
The proponents of frame repeat in cinema and interlace in television had made the classic mistake of assuming all that matters is the time axis. Along the time axis, frame repeat doubles the effective frame rate, cutting flicker, and two interlaced fields correctly mesh together again.
There’s a whopping and tragic flaw in these assumptions, and it’s due to the simple fact that the human eye can move. Everyone who has tried to photograph moving objects knows that if the camera is held still the image is smeared by the motion and if the camera follows or pans the motion the moving object is sharp.
The eye evolved to track moving objects of interest as a simple survival tool. In real life, the tracking eye renders the image stationary on the retina and detail can be seen despite motion. Self-evidently, the eye can only track one object at a time and everything else will be in relative motion.
However, when the eye tracks a “moving” object on a screen, the same level of detail won’t be seen, not least because the object in each frame isn’t actually moving. This can be seen in Figure 1. There are several more reasons for loss of detail, which we can examine here.
a) If the eye were fixed, then it would effectively scan the detail in a moving object such that the detail would turn into a signal changing rapidly with time that the eye can’t handle, so it would look blurred.
b) In real life the eye tracks an object of interest, so it becomes stationary on the retina, allowing detail to be seen.
In the case of cinema, the static resolution of the film is quite good in 35mm and startling in 70mm, but the audience can’t enjoy it. One reason is the frame repeat. The two versions of the same frame are identical, but to an eye tracking across the screen they appear in two different places. Thus the moving object of interest that is being tracked is blurred at low eye-tracking speeds, and at higher speeds a second image is seen displaced from the first.
The static resolution of moving imaging systems is never reached; figures like the number of lines in the TV picture or the number of pixels in a frame are for marketing to suckers who think bigger numbers must be better.
A more useful metric for a moving picture is dynamic resolution: the apparent resolution presented to a tracking eye. The present fixation with static resolution leads to completely incorrect decisions being made. It’s the reason why today’s HDTV couldn’t be dramatically better than SDTV. How could it be when the frame rates remained the same?