Commit | Line | Data |
---|---|---|
8af4922f TS |
1 | <section id="selection-api"> |
2 | ||
3 | <title>Experimental API for cropping, composing and scaling</title> | |
4 | ||
5 | <note> | |
6 | <title>Experimental</title> | |
7 | ||
8 | <para>This is an <link linkend="experimental">experimental</link> | |
9 | interface and may change in the future.</para> | |
10 | </note> | |
11 | ||
12 | <section> | |
13 | <title>Introduction</title> | |
14 | ||
15 | <para>Some video capture devices can sample a subsection of a picture and | |
16 | shrink or enlarge it to an image of arbitrary size. Next, the devices can | |
17 | insert the image into larger one. Some video output devices can crop part of an | |
18 | input image, scale it up or down and insert it at an arbitrary scan line and | |
19 | horizontal offset into a video signal. We call these abilities cropping, | |
20 | scaling and composing.</para> | |
21 | ||
22 | <para>On a video <emphasis>capture</emphasis> device the source is a video | |
23 | signal, and the cropping target determine the area actually sampled. The sink | |
24 | is an image stored in a memory buffer. The composing area specifies which part | |
25 | of the buffer is actually written to by the hardware. </para> | |
26 | ||
27 | <para>On a video <emphasis>output</emphasis> device the source is an image in a | |
28 | memory buffer, and the cropping target is a part of an image to be shown on a | |
29 | display. The sink is the display or the graphics screen. The application may | |
30 | select the part of display where the image should be displayed. The size and | |
31 | position of such a window is controlled by the compose target.</para> | |
32 | ||
33 | <para>Rectangles for all cropping and composing targets are defined even if the | |
34 | device does supports neither cropping nor composing. Their size and position | |
35 | will be fixed in such a case. If the device does not support scaling then the | |
36 | cropping and composing rectangles have the same size.</para> | |
37 | ||
38 | </section> | |
39 | ||
40 | <section> | |
41 | <title>Selection targets</title> | |
42 | ||
43 | <figure id="sel-targets-capture"> | |
44 | <title>Cropping and composing targets</title> | |
45 | <mediaobject> | |
46 | <imageobject> | |
47 | <imagedata fileref="selection.png" format="PNG" /> | |
48 | </imageobject> | |
49 | <textobject> | |
50 | <phrase>Targets used by a cropping, composing and scaling | |
51 | process</phrase> | |
52 | </textobject> | |
53 | </mediaobject> | |
54 | </figure> | |
9080d5d3 | 55 | |
8af4922f TS |
56 | </section> |
57 | ||
64b9ce83 SA |
58 | See <xref linkend="v4l2-selection-targets-table" /> for more |
59 | information. | |
60 | ||
8af4922f TS |
61 | <section> |
62 | ||
63 | <title>Configuration</title> | |
64 | ||
65 | <para>Applications can use the <link linkend="vidioc-g-selection">selection | |
66 | API</link> to select an area in a video signal or a buffer, and to query for | |
67 | default settings and hardware limits.</para> | |
68 | ||
69 | <para>Video hardware can have various cropping, composing and scaling | |
70 | limitations. It may only scale up or down, support only discrete scaling | |
71 | factors, or have different scaling abilities in the horizontal and vertical | |
72 | directions. Also it may not support scaling at all. At the same time the | |
73 | cropping/composing rectangles may have to be aligned, and both the source and | |
74 | the sink may have arbitrary upper and lower size limits. Therefore, as usual, | |
75 | drivers are expected to adjust the requested parameters and return the actual | |
76 | values selected. An application can control the rounding behaviour using <link | |
77 | linkend="v4l2-sel-flags"> constraint flags </link>.</para> | |
78 | ||
79 | <section> | |
80 | ||
81 | <title>Configuration of video capture</title> | |
82 | ||
83 | <para>See figure <xref linkend="sel-targets-capture" /> for examples of the | |
84 | selection targets available for a video capture device. It is recommended to | |
85 | configure the cropping targets before to the composing targets.</para> | |
86 | ||
87 | <para>The range of coordinates of the top left corner, width and height of | |
88 | areas that can be sampled is given by the <constant> V4L2_SEL_TGT_CROP_BOUNDS | |
89 | </constant> target. It is recommended for the driver developers to put the | |
90 | top/left corner at position <constant> (0,0) </constant>. The rectangle's | |
aa73ab96 | 91 | coordinates are expressed in pixels.</para> |
8af4922f TS |
92 | |
93 | <para>The top left corner, width and height of the source rectangle, that is | |
c1334823 | 94 | the area actually sampled, is given by the <constant> V4L2_SEL_TGT_CROP |
8af4922f TS |
95 | </constant> target. It uses the same coordinate system as <constant> |
96 | V4L2_SEL_TGT_CROP_BOUNDS </constant>. The active cropping area must lie | |
97 | completely inside the capture boundaries. The driver may further adjust the | |
98 | requested size and/or position according to hardware limitations.</para> | |
99 | ||
100 | <para>Each capture device has a default source rectangle, given by the | |
101 | <constant> V4L2_SEL_TGT_CROP_DEFAULT </constant> target. This rectangle shall | |
102 | over what the driver writer considers the complete picture. Drivers shall set | |
103 | the active crop rectangle to the default when the driver is first loaded, but | |
104 | not later.</para> | |
105 | ||
106 | <para>The composing targets refer to a memory buffer. The limits of composing | |
107 | coordinates are obtained using <constant> V4L2_SEL_TGT_COMPOSE_BOUNDS | |
aa73ab96 TS |
108 | </constant>. All coordinates are expressed in pixels. The rectangle's top/left |
109 | corner must be located at position <constant> (0,0) </constant>. The width and | |
110 | height are equal to the image size set by <constant> VIDIOC_S_FMT </constant>. | |
111 | </para> | |
8af4922f TS |
112 | |
113 | <para>The part of a buffer into which the image is inserted by the hardware is | |
c1334823 | 114 | controlled by the <constant> V4L2_SEL_TGT_COMPOSE </constant> target. |
8af4922f TS |
115 | The rectangle's coordinates are also expressed in the same coordinate system as |
116 | the bounds rectangle. The composing rectangle must lie completely inside bounds | |
117 | rectangle. The driver must adjust the composing rectangle to fit to the | |
118 | bounding limits. Moreover, the driver can perform other adjustments according | |
119 | to hardware limitations. The application can control rounding behaviour using | |
120 | <link linkend="v4l2-sel-flags"> constraint flags </link>.</para> | |
121 | ||
122 | <para>For capture devices the default composing rectangle is queried using | |
123 | <constant> V4L2_SEL_TGT_COMPOSE_DEFAULT </constant>. It is usually equal to the | |
124 | bounding rectangle.</para> | |
125 | ||
126 | <para>The part of a buffer that is modified by the hardware is given by | |
127 | <constant> V4L2_SEL_TGT_COMPOSE_PADDED </constant>. It contains all pixels | |
c1334823 | 128 | defined using <constant> V4L2_SEL_TGT_COMPOSE </constant> plus all |
8af4922f TS |
129 | padding data modified by hardware during insertion process. All pixels outside |
130 | this rectangle <emphasis>must not</emphasis> be changed by the hardware. The | |
131 | content of pixels that lie inside the padded area but outside active area is | |
132 | undefined. The application can use the padded and active rectangles to detect | |
133 | where the rubbish pixels are located and remove them if needed.</para> | |
134 | ||
135 | </section> | |
136 | ||
137 | <section> | |
138 | ||
139 | <title>Configuration of video output</title> | |
140 | ||
141 | <para>For output devices targets and ioctls are used similarly to the video | |
142 | capture case. The <emphasis> composing </emphasis> rectangle refers to the | |
143 | insertion of an image into a video signal. The cropping rectangles refer to a | |
144 | memory buffer. It is recommended to configure the composing targets before to | |
145 | the cropping targets.</para> | |
146 | ||
147 | <para>The cropping targets refer to the memory buffer that contains an image to | |
148 | be inserted into a video signal or graphical screen. The limits of cropping | |
149 | coordinates are obtained using <constant> V4L2_SEL_TGT_CROP_BOUNDS </constant>. | |
aa73ab96 TS |
150 | All coordinates are expressed in pixels. The top/left corner is always point |
151 | <constant> (0,0) </constant>. The width and height is equal to the image size | |
152 | specified using <constant> VIDIOC_S_FMT </constant> ioctl.</para> | |
8af4922f TS |
153 | |
154 | <para>The top left corner, width and height of the source rectangle, that is | |
155 | the area from which image date are processed by the hardware, is given by the | |
c1334823 | 156 | <constant> V4L2_SEL_TGT_CROP </constant>. Its coordinates are expressed |
8af4922f TS |
157 | in in the same coordinate system as the bounds rectangle. The active cropping |
158 | area must lie completely inside the crop boundaries and the driver may further | |
159 | adjust the requested size and/or position according to hardware | |
160 | limitations.</para> | |
161 | ||
162 | <para>For output devices the default cropping rectangle is queried using | |
163 | <constant> V4L2_SEL_TGT_CROP_DEFAULT </constant>. It is usually equal to the | |
164 | bounding rectangle.</para> | |
165 | ||
166 | <para>The part of a video signal or graphics display where the image is | |
aa73ab96 | 167 | inserted by the hardware is controlled by <constant> |
c1334823 | 168 | V4L2_SEL_TGT_COMPOSE </constant> target. The rectangle's coordinates |
aa73ab96 TS |
169 | are expressed in pixels. The composing rectangle must lie completely inside the |
170 | bounds rectangle. The driver must adjust the area to fit to the bounding | |
8af4922f TS |
171 | limits. Moreover, the driver can perform other adjustments according to |
172 | hardware limitations. </para> | |
173 | ||
174 | <para>The device has a default composing rectangle, given by the <constant> | |
175 | V4L2_SEL_TGT_COMPOSE_DEFAULT </constant> target. This rectangle shall cover what | |
176 | the driver writer considers the complete picture. It is recommended for the | |
177 | driver developers to put the top/left corner at position <constant> (0,0) | |
178 | </constant>. Drivers shall set the active composing rectangle to the default | |
179 | one when the driver is first loaded.</para> | |
180 | ||
181 | <para>The devices may introduce additional content to video signal other than | |
182 | an image from memory buffers. It includes borders around an image. However, | |
183 | such a padded area is driver-dependent feature not covered by this document. | |
184 | Driver developers are encouraged to keep padded rectangle equal to active one. | |
185 | The padded target is accessed by the <constant> V4L2_SEL_TGT_COMPOSE_PADDED | |
186 | </constant> identifier. It must contain all pixels from the <constant> | |
c1334823 | 187 | V4L2_SEL_TGT_COMPOSE </constant> target.</para> |
8af4922f TS |
188 | |
189 | </section> | |
190 | ||
191 | <section> | |
192 | ||
9080d5d3 | 193 | <title>Scaling control</title> |
8af4922f TS |
194 | |
195 | <para>An application can detect if scaling is performed by comparing the width | |
c1334823 SN |
196 | and the height of rectangles obtained using <constant> V4L2_SEL_TGT_CROP |
197 | </constant> and <constant> V4L2_SEL_TGT_COMPOSE </constant> targets. If | |
8af4922f TS |
198 | these are not equal then the scaling is applied. The application can compute |
199 | the scaling ratios using these values.</para> | |
200 | ||
201 | </section> | |
202 | ||
203 | </section> | |
204 | ||
205 | <section> | |
206 | ||
9080d5d3 | 207 | <title>Comparison with old cropping API</title> |
8af4922f TS |
208 | |
209 | <para>The selection API was introduced to cope with deficiencies of previous | |
210 | <link linkend="crop"> API </link>, that was designed to control simple capture | |
211 | devices. Later the cropping API was adopted by video output drivers. The ioctls | |
212 | are used to select a part of the display were the video signal is inserted. It | |
213 | should be considered as an API abuse because the described operation is | |
214 | actually the composing. The selection API makes a clear distinction between | |
215 | composing and cropping operations by setting the appropriate targets. The V4L2 | |
216 | API lacks any support for composing to and cropping from an image inside a | |
217 | memory buffer. The application could configure a capture device to fill only a | |
218 | part of an image by abusing V4L2 API. Cropping a smaller image from a larger | |
219 | one is achieved by setting the field <structfield> | |
220 | &v4l2-pix-format;::bytesperline </structfield>. Introducing an image offsets | |
221 | could be done by modifying field <structfield> &v4l2-buffer;::m:userptr | |
222 | </structfield> before calling <constant> VIDIOC_QBUF </constant>. Those | |
223 | operations should be avoided because they are not portable (endianness), and do | |
224 | not work for macroblock and Bayer formats and mmap buffers. The selection API | |
225 | deals with configuration of buffer cropping/composing in a clear, intuitive and | |
226 | portable way. Next, with the selection API the concepts of the padded target | |
227 | and constraints flags are introduced. Finally, <structname> &v4l2-crop; | |
228 | </structname> and <structname> &v4l2-cropcap; </structname> have no reserved | |
229 | fields. Therefore there is no way to extend their functionality. The new | |
230 | <structname> &v4l2-selection; </structname> provides a lot of place for future | |
231 | extensions. Driver developers are encouraged to implement only selection API. | |
232 | The former cropping API would be simulated using the new one. </para> | |
233 | ||
234 | </section> | |
235 | ||
236 | <section> | |
237 | <title>Examples</title> | |
238 | <example> | |
239 | <title>Resetting the cropping parameters</title> | |
240 | ||
241 | <para>(A video capture device is assumed; change <constant> | |
242 | V4L2_BUF_TYPE_VIDEO_CAPTURE </constant> for other devices; change target to | |
243 | <constant> V4L2_SEL_TGT_COMPOSE_* </constant> family to configure composing | |
244 | area)</para> | |
245 | ||
246 | <programlisting> | |
247 | ||
248 | &v4l2-selection; sel = { | |
249 | .type = V4L2_BUF_TYPE_VIDEO_CAPTURE, | |
250 | .target = V4L2_SEL_TGT_CROP_DEFAULT, | |
251 | }; | |
252 | ret = ioctl(fd, &VIDIOC-G-SELECTION;, &sel); | |
253 | if (ret) | |
254 | exit(-1); | |
c1334823 | 255 | sel.target = V4L2_SEL_TGT_CROP; |
8af4922f TS |
256 | ret = ioctl(fd, &VIDIOC-S-SELECTION;, &sel); |
257 | if (ret) | |
258 | exit(-1); | |
259 | ||
260 | </programlisting> | |
261 | </example> | |
262 | ||
263 | <example> | |
264 | <title>Simple downscaling</title> | |
265 | <para>Setting a composing area on output of size of <emphasis> at most | |
266 | </emphasis> half of limit placed at a center of a display.</para> | |
267 | <programlisting> | |
268 | ||
269 | &v4l2-selection; sel = { | |
270 | .type = V4L2_BUF_TYPE_VIDEO_OUTPUT, | |
271 | .target = V4L2_SEL_TGT_COMPOSE_BOUNDS, | |
272 | }; | |
273 | struct v4l2_rect r; | |
274 | ||
275 | ret = ioctl(fd, &VIDIOC-G-SELECTION;, &sel); | |
276 | if (ret) | |
277 | exit(-1); | |
278 | /* setting smaller compose rectangle */ | |
279 | r.width = sel.r.width / 2; | |
280 | r.height = sel.r.height / 2; | |
281 | r.left = sel.r.width / 4; | |
282 | r.top = sel.r.height / 4; | |
283 | sel.r = r; | |
c1334823 | 284 | sel.target = V4L2_SEL_TGT_COMPOSE; |
8af4922f TS |
285 | sel.flags = V4L2_SEL_FLAG_LE; |
286 | ret = ioctl(fd, &VIDIOC-S-SELECTION;, &sel); | |
287 | if (ret) | |
288 | exit(-1); | |
289 | ||
290 | </programlisting> | |
291 | </example> | |
292 | ||
293 | <example> | |
294 | <title>Querying for scaling factors</title> | |
295 | <para>A video output device is assumed; change <constant> | |
296 | V4L2_BUF_TYPE_VIDEO_OUTPUT </constant> for other devices</para> | |
297 | <programlisting> | |
298 | ||
299 | &v4l2-selection; compose = { | |
300 | .type = V4L2_BUF_TYPE_VIDEO_OUTPUT, | |
c1334823 | 301 | .target = V4L2_SEL_TGT_COMPOSE, |
8af4922f TS |
302 | }; |
303 | &v4l2-selection; crop = { | |
304 | .type = V4L2_BUF_TYPE_VIDEO_OUTPUT, | |
c1334823 | 305 | .target = V4L2_SEL_TGT_CROP, |
8af4922f TS |
306 | }; |
307 | double hscale, vscale; | |
308 | ||
309 | ret = ioctl(fd, &VIDIOC-G-SELECTION;, &compose); | |
310 | if (ret) | |
311 | exit(-1); | |
312 | ret = ioctl(fd, &VIDIOC-G-SELECTION;, &crop); | |
313 | if (ret) | |
314 | exit(-1); | |
315 | ||
316 | /* computing scaling factors */ | |
317 | hscale = (double)compose.r.width / crop.r.width; | |
318 | vscale = (double)compose.r.height / crop.r.height; | |
319 | ||
320 | </programlisting> | |
321 | </example> | |
322 | ||
323 | </section> | |
324 | ||
325 | </section> |