[media] v4l: Common documentation for selection targets
[deliverable/linux.git] / Documentation / DocBook / media / v4l / selection-api.xml
CommitLineData
8af4922f
TS
1<section id="selection-api">
2
3 <title>Experimental API for cropping, composing and scaling</title>
4
5 <note>
6 <title>Experimental</title>
7
8 <para>This is an <link linkend="experimental">experimental</link>
9interface and may change in the future.</para>
10 </note>
11
12 <section>
13 <title>Introduction</title>
14
15<para>Some video capture devices can sample a subsection of a picture and
16shrink or enlarge it to an image of arbitrary size. Next, the devices can
17insert the image into larger one. Some video output devices can crop part of an
18input image, scale it up or down and insert it at an arbitrary scan line and
19horizontal offset into a video signal. We call these abilities cropping,
20scaling and composing.</para>
21
22<para>On a video <emphasis>capture</emphasis> device the source is a video
23signal, and the cropping target determine the area actually sampled. The sink
24is an image stored in a memory buffer. The composing area specifies which part
25of the buffer is actually written to by the hardware. </para>
26
27<para>On a video <emphasis>output</emphasis> device the source is an image in a
28memory buffer, and the cropping target is a part of an image to be shown on a
29display. The sink is the display or the graphics screen. The application may
30select the part of display where the image should be displayed. The size and
31position of such a window is controlled by the compose target.</para>
32
33<para>Rectangles for all cropping and composing targets are defined even if the
34device does supports neither cropping nor composing. Their size and position
35will be fixed in such a case. If the device does not support scaling then the
36cropping and composing rectangles have the same size.</para>
37
38 </section>
39
40 <section>
41 <title>Selection targets</title>
42
43 <figure id="sel-targets-capture">
44 <title>Cropping and composing targets</title>
45 <mediaobject>
46 <imageobject>
47 <imagedata fileref="selection.png" format="PNG" />
48 </imageobject>
49 <textobject>
50 <phrase>Targets used by a cropping, composing and scaling
51 process</phrase>
52 </textobject>
53 </mediaobject>
54 </figure>
9080d5d3 55
8af4922f
TS
56 </section>
57
64b9ce83
SA
58 See <xref linkend="v4l2-selection-targets-table" /> for more
59 information.
60
8af4922f
TS
61 <section>
62
63 <title>Configuration</title>
64
65<para>Applications can use the <link linkend="vidioc-g-selection">selection
66API</link> to select an area in a video signal or a buffer, and to query for
67default settings and hardware limits.</para>
68
69<para>Video hardware can have various cropping, composing and scaling
70limitations. It may only scale up or down, support only discrete scaling
71factors, or have different scaling abilities in the horizontal and vertical
72directions. Also it may not support scaling at all. At the same time the
73cropping/composing rectangles may have to be aligned, and both the source and
74the sink may have arbitrary upper and lower size limits. Therefore, as usual,
75drivers are expected to adjust the requested parameters and return the actual
76values selected. An application can control the rounding behaviour using <link
77linkend="v4l2-sel-flags"> constraint flags </link>.</para>
78
79 <section>
80
81 <title>Configuration of video capture</title>
82
83<para>See figure <xref linkend="sel-targets-capture" /> for examples of the
84selection targets available for a video capture device. It is recommended to
85configure the cropping targets before to the composing targets.</para>
86
87<para>The range of coordinates of the top left corner, width and height of
88areas that can be sampled is given by the <constant> V4L2_SEL_TGT_CROP_BOUNDS
89</constant> target. It is recommended for the driver developers to put the
90top/left corner at position <constant> (0,0) </constant>. The rectangle's
aa73ab96 91coordinates are expressed in pixels.</para>
8af4922f
TS
92
93<para>The top left corner, width and height of the source rectangle, that is
c1334823 94the area actually sampled, is given by the <constant> V4L2_SEL_TGT_CROP
8af4922f
TS
95</constant> target. It uses the same coordinate system as <constant>
96V4L2_SEL_TGT_CROP_BOUNDS </constant>. The active cropping area must lie
97completely inside the capture boundaries. The driver may further adjust the
98requested size and/or position according to hardware limitations.</para>
99
100<para>Each capture device has a default source rectangle, given by the
101<constant> V4L2_SEL_TGT_CROP_DEFAULT </constant> target. This rectangle shall
102over what the driver writer considers the complete picture. Drivers shall set
103the active crop rectangle to the default when the driver is first loaded, but
104not later.</para>
105
106<para>The composing targets refer to a memory buffer. The limits of composing
107coordinates are obtained using <constant> V4L2_SEL_TGT_COMPOSE_BOUNDS
aa73ab96
TS
108</constant>. All coordinates are expressed in pixels. The rectangle's top/left
109corner must be located at position <constant> (0,0) </constant>. The width and
110height are equal to the image size set by <constant> VIDIOC_S_FMT </constant>.
111</para>
8af4922f
TS
112
113<para>The part of a buffer into which the image is inserted by the hardware is
c1334823 114controlled by the <constant> V4L2_SEL_TGT_COMPOSE </constant> target.
8af4922f
TS
115The rectangle's coordinates are also expressed in the same coordinate system as
116the bounds rectangle. The composing rectangle must lie completely inside bounds
117rectangle. The driver must adjust the composing rectangle to fit to the
118bounding limits. Moreover, the driver can perform other adjustments according
119to hardware limitations. The application can control rounding behaviour using
120<link linkend="v4l2-sel-flags"> constraint flags </link>.</para>
121
122<para>For capture devices the default composing rectangle is queried using
123<constant> V4L2_SEL_TGT_COMPOSE_DEFAULT </constant>. It is usually equal to the
124bounding rectangle.</para>
125
126<para>The part of a buffer that is modified by the hardware is given by
127<constant> V4L2_SEL_TGT_COMPOSE_PADDED </constant>. It contains all pixels
c1334823 128defined using <constant> V4L2_SEL_TGT_COMPOSE </constant> plus all
8af4922f
TS
129padding data modified by hardware during insertion process. All pixels outside
130this rectangle <emphasis>must not</emphasis> be changed by the hardware. The
131content of pixels that lie inside the padded area but outside active area is
132undefined. The application can use the padded and active rectangles to detect
133where the rubbish pixels are located and remove them if needed.</para>
134
135 </section>
136
137 <section>
138
139 <title>Configuration of video output</title>
140
141<para>For output devices targets and ioctls are used similarly to the video
142capture case. The <emphasis> composing </emphasis> rectangle refers to the
143insertion of an image into a video signal. The cropping rectangles refer to a
144memory buffer. It is recommended to configure the composing targets before to
145the cropping targets.</para>
146
147<para>The cropping targets refer to the memory buffer that contains an image to
148be inserted into a video signal or graphical screen. The limits of cropping
149coordinates are obtained using <constant> V4L2_SEL_TGT_CROP_BOUNDS </constant>.
aa73ab96
TS
150All coordinates are expressed in pixels. The top/left corner is always point
151<constant> (0,0) </constant>. The width and height is equal to the image size
152specified using <constant> VIDIOC_S_FMT </constant> ioctl.</para>
8af4922f
TS
153
154<para>The top left corner, width and height of the source rectangle, that is
155the area from which image date are processed by the hardware, is given by the
c1334823 156<constant> V4L2_SEL_TGT_CROP </constant>. Its coordinates are expressed
8af4922f
TS
157in in the same coordinate system as the bounds rectangle. The active cropping
158area must lie completely inside the crop boundaries and the driver may further
159adjust the requested size and/or position according to hardware
160limitations.</para>
161
162<para>For output devices the default cropping rectangle is queried using
163<constant> V4L2_SEL_TGT_CROP_DEFAULT </constant>. It is usually equal to the
164bounding rectangle.</para>
165
166<para>The part of a video signal or graphics display where the image is
aa73ab96 167inserted by the hardware is controlled by <constant>
c1334823 168V4L2_SEL_TGT_COMPOSE </constant> target. The rectangle's coordinates
aa73ab96
TS
169are expressed in pixels. The composing rectangle must lie completely inside the
170bounds rectangle. The driver must adjust the area to fit to the bounding
8af4922f
TS
171limits. Moreover, the driver can perform other adjustments according to
172hardware limitations. </para>
173
174<para>The device has a default composing rectangle, given by the <constant>
175V4L2_SEL_TGT_COMPOSE_DEFAULT </constant> target. This rectangle shall cover what
176the driver writer considers the complete picture. It is recommended for the
177driver developers to put the top/left corner at position <constant> (0,0)
178</constant>. Drivers shall set the active composing rectangle to the default
179one when the driver is first loaded.</para>
180
181<para>The devices may introduce additional content to video signal other than
182an image from memory buffers. It includes borders around an image. However,
183such a padded area is driver-dependent feature not covered by this document.
184Driver developers are encouraged to keep padded rectangle equal to active one.
185The padded target is accessed by the <constant> V4L2_SEL_TGT_COMPOSE_PADDED
186</constant> identifier. It must contain all pixels from the <constant>
c1334823 187V4L2_SEL_TGT_COMPOSE </constant> target.</para>
8af4922f
TS
188
189 </section>
190
191 <section>
192
9080d5d3 193 <title>Scaling control</title>
8af4922f
TS
194
195<para>An application can detect if scaling is performed by comparing the width
c1334823
SN
196and the height of rectangles obtained using <constant> V4L2_SEL_TGT_CROP
197</constant> and <constant> V4L2_SEL_TGT_COMPOSE </constant> targets. If
8af4922f
TS
198these are not equal then the scaling is applied. The application can compute
199the scaling ratios using these values.</para>
200
201 </section>
202
203 </section>
204
205 <section>
206
9080d5d3 207 <title>Comparison with old cropping API</title>
8af4922f
TS
208
209<para>The selection API was introduced to cope with deficiencies of previous
210<link linkend="crop"> API </link>, that was designed to control simple capture
211devices. Later the cropping API was adopted by video output drivers. The ioctls
212are used to select a part of the display were the video signal is inserted. It
213should be considered as an API abuse because the described operation is
214actually the composing. The selection API makes a clear distinction between
215composing and cropping operations by setting the appropriate targets. The V4L2
216API lacks any support for composing to and cropping from an image inside a
217memory buffer. The application could configure a capture device to fill only a
218part of an image by abusing V4L2 API. Cropping a smaller image from a larger
219one is achieved by setting the field <structfield>
220&v4l2-pix-format;::bytesperline </structfield>. Introducing an image offsets
221could be done by modifying field <structfield> &v4l2-buffer;::m:userptr
222</structfield> before calling <constant> VIDIOC_QBUF </constant>. Those
223operations should be avoided because they are not portable (endianness), and do
224not work for macroblock and Bayer formats and mmap buffers. The selection API
225deals with configuration of buffer cropping/composing in a clear, intuitive and
226portable way. Next, with the selection API the concepts of the padded target
227and constraints flags are introduced. Finally, <structname> &v4l2-crop;
228</structname> and <structname> &v4l2-cropcap; </structname> have no reserved
229fields. Therefore there is no way to extend their functionality. The new
230<structname> &v4l2-selection; </structname> provides a lot of place for future
231extensions. Driver developers are encouraged to implement only selection API.
232The former cropping API would be simulated using the new one. </para>
233
234 </section>
235
236 <section>
237 <title>Examples</title>
238 <example>
239 <title>Resetting the cropping parameters</title>
240
241 <para>(A video capture device is assumed; change <constant>
242V4L2_BUF_TYPE_VIDEO_CAPTURE </constant> for other devices; change target to
243<constant> V4L2_SEL_TGT_COMPOSE_* </constant> family to configure composing
244area)</para>
245
246 <programlisting>
247
248 &v4l2-selection; sel = {
249 .type = V4L2_BUF_TYPE_VIDEO_CAPTURE,
250 .target = V4L2_SEL_TGT_CROP_DEFAULT,
251 };
252 ret = ioctl(fd, &VIDIOC-G-SELECTION;, &amp;sel);
253 if (ret)
254 exit(-1);
c1334823 255 sel.target = V4L2_SEL_TGT_CROP;
8af4922f
TS
256 ret = ioctl(fd, &VIDIOC-S-SELECTION;, &amp;sel);
257 if (ret)
258 exit(-1);
259
260 </programlisting>
261 </example>
262
263 <example>
264 <title>Simple downscaling</title>
265 <para>Setting a composing area on output of size of <emphasis> at most
266</emphasis> half of limit placed at a center of a display.</para>
267 <programlisting>
268
269 &v4l2-selection; sel = {
270 .type = V4L2_BUF_TYPE_VIDEO_OUTPUT,
271 .target = V4L2_SEL_TGT_COMPOSE_BOUNDS,
272 };
273 struct v4l2_rect r;
274
275 ret = ioctl(fd, &VIDIOC-G-SELECTION;, &amp;sel);
276 if (ret)
277 exit(-1);
278 /* setting smaller compose rectangle */
279 r.width = sel.r.width / 2;
280 r.height = sel.r.height / 2;
281 r.left = sel.r.width / 4;
282 r.top = sel.r.height / 4;
283 sel.r = r;
c1334823 284 sel.target = V4L2_SEL_TGT_COMPOSE;
8af4922f
TS
285 sel.flags = V4L2_SEL_FLAG_LE;
286 ret = ioctl(fd, &VIDIOC-S-SELECTION;, &amp;sel);
287 if (ret)
288 exit(-1);
289
290 </programlisting>
291 </example>
292
293 <example>
294 <title>Querying for scaling factors</title>
295 <para>A video output device is assumed; change <constant>
296V4L2_BUF_TYPE_VIDEO_OUTPUT </constant> for other devices</para>
297 <programlisting>
298
299 &v4l2-selection; compose = {
300 .type = V4L2_BUF_TYPE_VIDEO_OUTPUT,
c1334823 301 .target = V4L2_SEL_TGT_COMPOSE,
8af4922f
TS
302 };
303 &v4l2-selection; crop = {
304 .type = V4L2_BUF_TYPE_VIDEO_OUTPUT,
c1334823 305 .target = V4L2_SEL_TGT_CROP,
8af4922f
TS
306 };
307 double hscale, vscale;
308
309 ret = ioctl(fd, &VIDIOC-G-SELECTION;, &amp;compose);
310 if (ret)
311 exit(-1);
312 ret = ioctl(fd, &VIDIOC-G-SELECTION;, &amp;crop);
313 if (ret)
314 exit(-1);
315
316 /* computing scaling factors */
317 hscale = (double)compose.r.width / crop.r.width;
318 vscale = (double)compose.r.height / crop.r.height;
319
320 </programlisting>
321 </example>
322
323 </section>
324
325</section>
This page took 0.055859 seconds and 5 git commands to generate.