A comparison among watermarking schemes used in digital cinema.
Watermarking is used in several areas such as CDNs (Content Delivery Networks), as part of the rights management system for counterfeit prevention. Watermarking schemes need some additional features in order to be used in digital cinema. In fact, extra watermarks are added to movies by cinema projectors in projection time, which help identify the cinema hall in which the illegal copy has been recorded. But distortions caused by hand vibrations and the point of view angle make it difficult to recover the watermark. This makes it necessary to be distortion-resistant for the watermarking schemes used in digital cinema. On the other hand, theatre owners would like to locate the camcorder that has recorded the pirate copy. This requires watermarking schemes to be able to estimate the distance and angle using the distributed pirate copy. In this chapter, we present a review on watermarking techniques specifically designed to attack the aforementioned problems.
- digital cinema
- digital distribution
Digital distribution has been considered by researchers since four decades ago [1, 2, 3]. With the growth of available bandwidth in the telecommunications infrastructures, it gradually appeared as one of the main purposes of communication during two decades. This encouraged a lot of research on different aspects of digital distribution over internet [4, 5, 6]. Digital distribution brings about several challenges among which one can refer to performance, but in this chapter, we are more interested in security. Security of digital distribution has been investigated from different aspects [7, 8, 9]. One of the most challenging aspects in digital distribution security is digital rights management, which has been the topic for several research works [10, 11]. With the emergence of internet, online digital distribution found its applications in a variety of environments [12, 13, 14]. Meanwhile, the public access to internet made security more and more challenging .
Content delivery is a branch of digital distribution, which is growing very fast in different aspects [16, 17]. The more content delivery systems grow, the more complex security their security management becomes . Among advances in content delivery, one may mention CDNs [19, 20, 21]. These networks serve to high-performance delivery of digital contents as well as streaming of audio and video in collaboration with other technologies such as cloud computing [22, 23]. Different aspects of security in CDNs such as privacy , digital property management , intrusion detection , trust , and eavesdropping protection  have been studied by researchers.
Digital cinema is a service that can be provided using CDNs. It is an online video streaming and projection service gaining a research focus in recent years [29, 30, 31]. Security is a serious challenge in digital cinema [32, 33, 34] like other services provided over CDNs. One of the most important aspects of digital cinema security is digital rights management [35, 36], which is about preserving the publication rights for the generator of contents. Counterfeit protection is one of the most challenging problems in the area of digital cinema rights management.
Digital property management and especially counterfeit protection mostly depend on watermarking in a variety of environments [37, 38, 39]. Digital watermarking has been finding its applications in digital cinema in recent years [40, 41]. But, there is a delicate challenge faced by watermarking systems used in digital cinema. In the absence of proper protective measures, counterfeit copies of movies can be recorded by handheld cameras in the cinema hall. Movie theater holders are obliged to take preventive measures in this regard. This creates two problems, which need to be taken into consideration while designing watermarking schemes for digital cinema. These problems are explained in the following.
The first problem is faced by movie providers. When a pirate copy is distributed, providers would like to identify the theatre where the copy has been recorded. Then they can sue the identified theatres as they have not fulfilled their legal obligations. One way to approach this problem is to add watermarks to the movie by the projector in projection time. This watermark carries some identification information about the cinema hall as well as the projection time. This makes it possible to identify the hall in which a counterfeit copy has been recorded. But, the distortions made by hand vibrations and point of view angle, makes it difficult to recover the watermark from the movie. Thus, watermarking schemes designed for digital cinema should be specifically resistant against these distortions.
The second problem is to locate the seat on which the camcorder has been recording. This problem is to be solved by cinema holders as they are legally responsible for illegally recorded copies. Solving this problem requires digital cinema watermarking schemes to be able to calculate distance and angle.
There are some surveys on techniques used to solve the two problems mentioned above. But, they are outdated for such a fast-growing area [42, 43]. This motivates our work in this chapter, which is a comparative review on watermarking schemes used in digital cinema and the methods used by these schemes in order to achieve the mentioned goals.
The rest of this chapter is organized as follows. Section 2 presents an introduction to digital cinema. Section 3 discusses watermarking and related concepts. Section 4 presents a survey on watermarking in digital cinema. Section 5 concludes the chapter and suggests further research.
2. Digital cinema
Digital cinema refers to the set of processes, tools, and components which aim at the preparation, transmission, and projection of digital movies in cinema. This system is taking the place of traditional cinema which depends on reels of film being projected by film projectors.
In digital cinema, 2K (2048 × 1080 or 2.2 megapixels) or 4K (4096 × 2160 or 8.8 megapixels) resolutions are commonly used.
Among the advantages of digital cinema, we can point out reduction in time to market, reduction in cost per print, greater reach, and piracy control.
A digital cinema system consists of three main processes. In the first process, the film is converted to a digital print called DCP (Digital Cinema Package). The DCP is then delivered to the theatres and ingested (stored) by the related servers. This can be done through physical transportation of hard disks or through a digital distribution system. A content delivery network can serve to this purpose . The DCP is often encrypted, and the keys required to decrypt the file are ingested in the screening tools to prevent theater owners from illegal screen extensions. In the third process, the movie is played back using a digital video projector. The aforementioned three processes are shown in Figure 1.
The first system specification for digital cinema was published in 2005 by a consortium of six major studios. This specification is referred to as DCI (Digital Cinema Initiatives) and uses JPG 2000 video encoding with XML playback standard.
3. Digital watermarking
Watermarking is a copyright ownership identification method used for noise-tolerant media such as video, image, and audio. In this technique, an intentional noise is added to the signal in the form of covertly embedded information. In most cases, the hidden information is normally not perceptible. Rather it requires some revealing process to be extracted from the carrying content file. However, in some cases, it can be perceptible. For example, it can be visibly added to a film or image in order to encourage the user to buy the original version without the mark.
The hidden information should be robust (resistant) against simple modifications normally used to the media type. For example, a marker hidden in an image should not be removed when the image is converted from Bitmap to JPG.
Steganographic techniques such as watermarking and steganography are evaluated using measures like robustness, imperceptibility, and capacity. In watermarking, the main objective is robustness. It should not be confused with steganography where imperceptibility is the main goal .
Figure 2 shows how the character “a” can be watermarked in a video using a simple watermarking scheme.
In the example shown in Figure 2, the video is first decomposed into segments. Then two different variants of the video represented by and are prepared which are imperceptibly different in each segment. Each segment can be chosen from the variant or . The choice of for a segment watermarks the bit value “,” and choosing means watermarking “1.” In this example, the ASCII code “” assigned to the character “” has been watermarked to the video by choosing segments and from the variant and the others from .
4. Survey on watermarking in digital cinema
Illegal recording from the cinema screen using handheld cameras is a serious problem encountered by film makers. Watermarking, as a fingerprinting method aiming at piracy protection is a promising method for attacking this problem in the context of digital cinema. The watermarking process can be performed in show time and embed information regarding the projector and the playback time into the video. Since theatre owners are obliged to prevent handheld cameras in their premises, they are responsible for the pirate copies, and the watermark extracted from the pirate copies can specify the theatre where the copy has been recorded.
One of the most serious problems in the application of watermarking in digital cinema is that perspective, zoom, and some other parameters can be changed by the handheld camera, which leads to geometrical distortions. These distortions make it difficult to recover the watermark. This problem is faced by movie providers as they need to identify the theatre from which the pirate copy has been recorded. A watermarking system should guarantee distortion resistance in order to be capable of being used in digital cinema. In fact, old correlation-based watermark schemes are unable to prevent piracy because they are not robust to geometric distortions created by the handheld camera [46, 47].
The second challenge, which is faced by theatre owners, is locating the pirate in the theatre. Since theatre owners are sued when a pirate copy is distributed, they need mechanisms to identify the person who has recorded the illegal copy. To do this, they need to locate the camcorder. This way, they can identify the criminal using their ticket databases. This raises the need for watermarking methods capable of locating the camcorder.
The two aforementioned challenges have been part of the topic in several research works since early 2000s [48, 49], and they are still considered as research concerns in this area [42, 50]. In the following, we separately review the methods used by different watermarking schemes in order to attack each of the problems.
4.1 Identifying the theatre
In this section, we study the methods used in order to identify the theatres wherein hand camera copies are recorded. These methods mainly depend on geometric process while storing or detecting the watermark.
4.1.1 Temporal watermarking
One solution to the aforementioned problem is to design methods which use only the temporal access to embed the watermark. A watermarking scheme of this kind was proposed in [51, 52], which modifies the mean luminance value in each frame of the video according to the samples of the watermark. In this research report, it has been taken into consideration that the human visual system is sensitive to flickers in low spatial frequencies. Thus, the proposed scheme attempts at avoiding this kind of flickers. To do this, the same watermark is applied to a certain number of consequent frames.
In the aforementioned scheme, the watermark is considered as a pseudo-random sequence of length consisting of and values. Embedding the watermark causes the luminance of each pixel to increase or decrease. But the change depends on a local scaling factor which has higher values for moving textual areas and lower values for non-moving flat areas. This scaling factor is defined as the minimum of a spatial scaling factor and a motion scaling factor. Moreover, there is a maximum allowable change that cannot be violated.
The motion scaling factor for each pixel is calculated as the difference from the corresponding pixel in the previous frame. On the other hand, the spatial scaling factor is calculated using a Laplacian filter. Figure 3 shows this watermarking scheme.
In Figure 3, and are predefined thresholds and represents the absolute operator. Moreover, ,is the number of the frame among the selected set of consecutive frames for embedding the watermark, and is the number of consecutive frames.
The authors of this report successfully tested the robustness of their watermark under several conditions such as changing the zoom and angle of the camera in addition to handheld camera recording and people walking between the camera and the screen.
Compensation is another solution to the geometrical distortion problem. In this method, a pre-process is used to restore the geometrical distortions before watermark extraction. A method based on compensation was proposed in . The authors of this report made a simplifying assumption in order to make it easier to model the nonlinear distortions. They assumed that the handheld camera is located far from the screen at the back of the movie theatre but not so far from the center line of theatre.
The authors of  present a parameterized mathematical model that compares the pirate copy with its original in a general case. For each specific pair of pirate-original pair, the compensation method tries to find the values for the parameters such that the model best matches the case. They modeled the distortions with four mathematical transformations as follows.
Affine transformations: there is assumed to be six degrees of parameters for this kind of transformations, which model rescaling (zooming), translation (shifting caused by shakes of the camera), rotation (caused by the angle of the camera), and shearing (caused by cropping).
Bilinear transform: this kind of transform can convert a square to an ordinary quadrilateral with eight degrees of freedom.
Curved transform: tt is a simple mathematical transform that models the impacts of the lens properties such as short focal length.
Curved-bilinear transform: it has been designed to compensate the bilinear transform as well as the curved transform.
They tested their method via simulating a handheld camera. The simulation process applies a curved-bilinear transformation along with converting from 1920 × 1980 resolution to 1024 × 576 and then cropping to 720 × 576. Figure 4 shows this transformation. Their method successfully recovered a 64-bit watermark from the transformed video.
4.1.3 Hardware-based additional watermarking
A hardware-based watermarking scheme named “Additional Watermarking” was presented in [54, 55, 56], which does not require the original movie for the recovery of the watermark. This method uses DWT (Discrete Wavelet Transform) instead of the Laplacian transforms. In this scheme, watermarks are added to the video in both the encoding and the decoding processes. This method uses CRC32 as a hash function for selecting random places in the video to embed the watermark. This method places the mark in the lowest frequency components of the video. It was shown in the research that the proposed method does not make any perceptible change in the original video and it is robust to a variety of signal processing transforms.
4.1.4 Spread transform dither modulation
The problem of tradeoff between fidelity and robustness was posed in . A watermarking scheme based on STDM (Spread Transform Dither Modulation) was proposed in this research which uses the HVS (Human Visual System) properties of wavelet in order to resolve the tradeoff. These properties include the following facts.
Noise in high resolution bands is not clearly perceptible to human eyes.
HVS is less sensitive to changes to very high or very low brightness portions of the video.
HVS is not very sensitive to changes in highly textured portions unless they are close to the borders.
In this scheme, the watermark is embedded during the JPEG2000 decoding phase. Figure 5 shows a screen shot from Big Buck Bunny film before and after watermarking with this method.
4.1.5 Spread spectrum watermarking
In  another application for watermarking in digital cinema was proposed. This report proposes a watermarking method based on spread spectrum. This method is not only robust to geometric distortions, but also able to estimate the position of the handheld camera that has recorded the pirate copy. This may help identifying the ticket number and the credit card number of the person.
The method proposed in  embeds the watermark in the payload in a way that the geometric transformation applied by the handheld camera can be identified in addition to the position of the camera. This method requires the watermark payload to follow a periodic pattern. A basic pattern is generated using a secret key in a way that it follows a periodic Gaussian distribution with the mean equal to zero and the variance equal to one. Then the payload, which contains the serial number of the projector as well as a time stamp is embedded into the video after the modulation process using an additive spread-spectrum method.
The authors of  tested the accuracy of their method in a small seminar room as well as a large auditorium. The mean absolute error was at maximum 6.87 cm for the small-scale test and 50.38 cm for the large-scale test.
Another watermarking scheme based on spread spectrum techniques was introduced in . This method stores the watermark in the YCbCr color space using translation, rotation, scaling or composite operations.
Table 1 compares the methods used to identify the theatre. The parameters used in this comparison are as follows.
Resistance against hand vibrations
Resistance against perspective drifts
Resistance against obstacles such as people’s heads or people walking between the handheld camera and the screen
Resistance against zoom changes
One point to note here is that the distortion factors studied in Table 1 are not the only factors that make it complex to recover the projection time watermark from pirate copies. For example, it was shown in  that the lamination flickers, induced by the interplay between an LCD and a camcorder can cause some difficulties in the recovery of the watermark. In addition to temporal luminance variations studied in , spatial luminance variations induced by camcorder recording have been studied in .
4.2 Locating the camcorder
In this section, we study the methods that can be used by cinema owners in order to locate the pirate camera.
4.2.1 Audio watermarking
An approach to estimating the camcorder location has been proposed in . In this method, unlike previous methods, audio is watermarked in the movie soundtrack instead of images, numbers or text messages being watermarked in the video itself. Most of the methods introduced in the literature avoid audio watermarking because of its complexity. In fact, it is difficult to hide any audio in soundtracks as they contain different types of audio such as voices, sound effects, and music.
The method proposed in  depends on a stochastic analysis in the watermark detection process. This method uses the audio watermarking technique introduced in . The idea behind this method is that the attenuation of an audio signal depends on the distance. Thus, a few watermarked audio signals played from different locations can provide adequate information for locating the pirate. The advantage of this method is that audio is not prone to geometric distortion.
4.2.2 Hybrid audio-video watermarking
Some recent research works such as the one reported in  have combined video watermarking and audio watermarking in order to achieve the advantages of both methods. Audio watermarking is not affected by geometric distortions resulting from perspective, zoom, and vibrations. On the other hand, video watermarking is not affected by distance. Thus, it looks pertinent to combine them to achieve better results.
In the method proposed in , audio is watermarked in the video itself instead of the soundtrack. This way, there is no need for the watermarked audio to be played from different sources.
Another hybrid audio-video watermarking scheme was introduced in . In this method unlike the one presented in , audio and video are separately watermarked in the soundtrack and in the movie itself.
In this chapter, we introduced the challenges faced by watermarking schemes in digital cinema. Then, we presented a review on watermarking methods specifically designed to confront these challenges. We focused on two main problems, first of which is the geometric distortion problem which and the second is camcorder location estimation. Our work in this chapter can be continued by studying other security and performance aspects of digital cinema.