Commit 72c8a94a585afea1f45aa8c4f6938ed6d05be57a
Committed by
Dmitry Torokhov
1 parent
40d007e7df
Exists in
master
and in
4 other branches
Input: document the MT event slot protocol
This patch adds documentation for the ABS_MT_SLOT event and gives examples of how to use the event slot protocol. Reviewed-by: Ping Cheng <pingc@wacom.com> Signed-off-by: Henrik Rydberg <rydberg@euromail.se> Signed-off-by: Dmitry Torokhov <dtor@mail.ru>
Showing 1 changed file with 149 additions and 69 deletions Side-by-side Diff
Documentation/input/multi-touch-protocol.txt
... | ... | @@ -6,31 +6,149 @@ |
6 | 6 | Introduction |
7 | 7 | ------------ |
8 | 8 | |
9 | -In order to utilize the full power of the new multi-touch devices, a way to | |
10 | -report detailed finger data to user space is needed. This document | |
11 | -describes the multi-touch (MT) protocol which allows kernel drivers to | |
12 | -report details for an arbitrary number of fingers. | |
9 | +In order to utilize the full power of the new multi-touch and multi-user | |
10 | +devices, a way to report detailed data from multiple contacts, i.e., | |
11 | +objects in direct contact with the device surface, is needed. This | |
12 | +document describes the multi-touch (MT) protocol which allows kernel | |
13 | +drivers to report details for an arbitrary number of contacts. | |
13 | 14 | |
15 | +The protocol is divided into two types, depending on the capabilities of the | |
16 | +hardware. For devices handling anonymous contacts (type A), the protocol | |
17 | +describes how to send the raw data for all contacts to the receiver. For | |
18 | +devices capable of tracking identifiable contacts (type B), the protocol | |
19 | +describes how to send updates for individual contacts via event slots. | |
14 | 20 | |
15 | -Usage | |
16 | ------ | |
17 | 21 | |
18 | -Anonymous finger details are sent sequentially as separate packets of ABS | |
19 | -events. Only the ABS_MT events are recognized as part of a finger | |
20 | -packet. The end of a packet is marked by calling the input_mt_sync() | |
21 | -function, which generates a SYN_MT_REPORT event. This instructs the | |
22 | -receiver to accept the data for the current finger and prepare to receive | |
23 | -another. The end of a multi-touch transfer is marked by calling the usual | |
22 | +Protocol Usage | |
23 | +-------------- | |
24 | + | |
25 | +Contact details are sent sequentially as separate packets of ABS_MT | |
26 | +events. Only the ABS_MT events are recognized as part of a contact | |
27 | +packet. Since these events are ignored by current single-touch (ST) | |
28 | +applications, the MT protocol can be implemented on top of the ST protocol | |
29 | +in an existing driver. | |
30 | + | |
31 | +Drivers for type A devices separate contact packets by calling | |
32 | +input_mt_sync() at the end of each packet. This generates a SYN_MT_REPORT | |
33 | +event, which instructs the receiver to accept the data for the current | |
34 | +contact and prepare to receive another. | |
35 | + | |
36 | +Drivers for type B devices separate contact packets by calling | |
37 | +input_mt_slot(), with a slot as argument, at the beginning of each packet. | |
38 | +This generates an ABS_MT_SLOT event, which instructs the receiver to | |
39 | +prepare for updates of the given slot. | |
40 | + | |
41 | +All drivers mark the end of a multi-touch transfer by calling the usual | |
24 | 42 | input_sync() function. This instructs the receiver to act upon events |
25 | -accumulated since last EV_SYN/SYN_REPORT and prepare to receive a new | |
26 | -set of events/packets. | |
43 | +accumulated since last EV_SYN/SYN_REPORT and prepare to receive a new set | |
44 | +of events/packets. | |
27 | 45 | |
46 | +The main difference between the stateless type A protocol and the stateful | |
47 | +type B slot protocol lies in the usage of identifiable contacts to reduce | |
48 | +the amount of data sent to userspace. The slot protocol requires the use of | |
49 | +the ABS_MT_TRACKING_ID, either provided by the hardware or computed from | |
50 | +the raw data [5]. | |
51 | + | |
52 | +For type A devices, the kernel driver should generate an arbitrary | |
53 | +enumeration of the full set of anonymous contacts currently on the | |
54 | +surface. The order in which the packets appear in the event stream is not | |
55 | +important. Event filtering and finger tracking is left to user space [3]. | |
56 | + | |
57 | +For type B devices, the kernel driver should associate a slot with each | |
58 | +identified contact, and use that slot to propagate changes for the contact. | |
59 | +Creation, replacement and destruction of contacts is achieved by modifying | |
60 | +the ABS_MT_TRACKING_ID of the associated slot. A non-negative tracking id | |
61 | +is interpreted as a contact, and the value -1 denotes an unused slot. A | |
62 | +tracking id not previously present is considered new, and a tracking id no | |
63 | +longer present is considered removed. Since only changes are propagated, | |
64 | +the full state of each initiated contact has to reside in the receiving | |
65 | +end. Upon receiving an MT event, one simply updates the appropriate | |
66 | +attribute of the current slot. | |
67 | + | |
68 | + | |
69 | +Protocol Example A | |
70 | +------------------ | |
71 | + | |
72 | +Here is what a minimal event sequence for a two-contact touch would look | |
73 | +like for a type A device: | |
74 | + | |
75 | + ABS_MT_POSITION_X x[0] | |
76 | + ABS_MT_POSITION_Y y[0] | |
77 | + SYN_MT_REPORT | |
78 | + ABS_MT_POSITION_X x[1] | |
79 | + ABS_MT_POSITION_Y y[1] | |
80 | + SYN_MT_REPORT | |
81 | + SYN_REPORT | |
82 | + | |
83 | +The sequence after moving one of the contacts looks exactly the same; the | |
84 | +raw data for all present contacts are sent between every synchronization | |
85 | +with SYN_REPORT. | |
86 | + | |
87 | +Here is the sequence after lifting the first contact: | |
88 | + | |
89 | + ABS_MT_POSITION_X x[1] | |
90 | + ABS_MT_POSITION_Y y[1] | |
91 | + SYN_MT_REPORT | |
92 | + SYN_REPORT | |
93 | + | |
94 | +And here is the sequence after lifting the second contact: | |
95 | + | |
96 | + SYN_MT_REPORT | |
97 | + SYN_REPORT | |
98 | + | |
99 | +If the driver reports one of BTN_TOUCH or ABS_PRESSURE in addition to the | |
100 | +ABS_MT events, the last SYN_MT_REPORT event may be omitted. Otherwise, the | |
101 | +last SYN_REPORT will be dropped by the input core, resulting in no | |
102 | +zero-contact event reaching userland. | |
103 | + | |
104 | + | |
105 | +Protocol Example B | |
106 | +------------------ | |
107 | + | |
108 | +Here is what a minimal event sequence for a two-contact touch would look | |
109 | +like for a type B device: | |
110 | + | |
111 | + ABS_MT_SLOT 0 | |
112 | + ABS_MT_TRACKING_ID 45 | |
113 | + ABS_MT_POSITION_X x[0] | |
114 | + ABS_MT_POSITION_Y y[0] | |
115 | + ABS_MT_SLOT 1 | |
116 | + ABS_MT_TRACKING_ID 46 | |
117 | + ABS_MT_POSITION_X x[1] | |
118 | + ABS_MT_POSITION_Y y[1] | |
119 | + SYN_REPORT | |
120 | + | |
121 | +Here is the sequence after moving contact 45 in the x direction: | |
122 | + | |
123 | + ABS_MT_SLOT 0 | |
124 | + ABS_MT_POSITION_X x[0] | |
125 | + SYN_REPORT | |
126 | + | |
127 | +Here is the sequence after lifting the contact in slot 0: | |
128 | + | |
129 | + ABS_MT_TRACKING_ID -1 | |
130 | + SYN_REPORT | |
131 | + | |
132 | +The slot being modified is already 0, so the ABS_MT_SLOT is omitted. The | |
133 | +message removes the association of slot 0 with contact 45, thereby | |
134 | +destroying contact 45 and freeing slot 0 to be reused for another contact. | |
135 | + | |
136 | +Finally, here is the sequence after lifting the second contact: | |
137 | + | |
138 | + ABS_MT_SLOT 1 | |
139 | + ABS_MT_TRACKING_ID -1 | |
140 | + SYN_REPORT | |
141 | + | |
142 | + | |
143 | +Event Usage | |
144 | +----------- | |
145 | + | |
28 | 146 | A set of ABS_MT events with the desired properties is defined. The events |
29 | 147 | are divided into categories, to allow for partial implementation. The |
30 | 148 | minimum set consists of ABS_MT_POSITION_X and ABS_MT_POSITION_Y, which |
31 | -allows for multiple fingers to be tracked. If the device supports it, the | |
149 | +allows for multiple contacts to be tracked. If the device supports it, the | |
32 | 150 | ABS_MT_TOUCH_MAJOR and ABS_MT_WIDTH_MAJOR may be used to provide the size |
33 | -of the contact area and approaching finger, respectively. | |
151 | +of the contact area and approaching contact, respectively. | |
34 | 152 | |
35 | 153 | The TOUCH and WIDTH parameters have a geometrical interpretation; imagine |
36 | 154 | looking through a window at someone gently holding a finger against the |
37 | 155 | |
38 | 156 | |
39 | 157 | |
40 | 158 | |
41 | 159 | |
42 | 160 | |
... | ... | @@ -41,56 +159,26 @@ |
41 | 159 | ABS_MT_WIDTH_MAJOR. Now imagine the person pressing the finger harder |
42 | 160 | against the glass. The inner region will increase, and in general, the |
43 | 161 | ratio ABS_MT_TOUCH_MAJOR / ABS_MT_WIDTH_MAJOR, which is always smaller than |
44 | -unity, is related to the finger pressure. For pressure-based devices, | |
162 | +unity, is related to the contact pressure. For pressure-based devices, | |
45 | 163 | ABS_MT_PRESSURE may be used to provide the pressure on the contact area |
46 | 164 | instead. |
47 | 165 | |
48 | -In addition to the MAJOR parameters, the oval shape of the finger can be | |
166 | +In addition to the MAJOR parameters, the oval shape of the contact can be | |
49 | 167 | described by adding the MINOR parameters, such that MAJOR and MINOR are the |
50 | 168 | major and minor axis of an ellipse. Finally, the orientation of the oval |
51 | 169 | shape can be describe with the ORIENTATION parameter. |
52 | 170 | |
53 | 171 | The ABS_MT_TOOL_TYPE may be used to specify whether the touching tool is a |
54 | -finger or a pen or something else. Devices with more granular information | |
172 | +contact or a pen or something else. Devices with more granular information | |
55 | 173 | may specify general shapes as blobs, i.e., as a sequence of rectangular |
56 | 174 | shapes grouped together by an ABS_MT_BLOB_ID. Finally, for the few devices |
57 | 175 | that currently support it, the ABS_MT_TRACKING_ID event may be used to |
58 | -report finger tracking from hardware [5]. | |
176 | +report contact tracking from hardware [5]. | |
59 | 177 | |
60 | -Here is what a minimal event sequence for a two-finger touch would look | |
61 | -like: | |
62 | 178 | |
63 | - ABS_MT_POSITION_X | |
64 | - ABS_MT_POSITION_Y | |
65 | - SYN_MT_REPORT | |
66 | - ABS_MT_POSITION_X | |
67 | - ABS_MT_POSITION_Y | |
68 | - SYN_MT_REPORT | |
69 | - SYN_REPORT | |
70 | - | |
71 | -Here is the sequence after lifting one of the fingers: | |
72 | - | |
73 | - ABS_MT_POSITION_X | |
74 | - ABS_MT_POSITION_Y | |
75 | - SYN_MT_REPORT | |
76 | - SYN_REPORT | |
77 | - | |
78 | -And here is the sequence after lifting the remaining finger: | |
79 | - | |
80 | - SYN_MT_REPORT | |
81 | - SYN_REPORT | |
82 | - | |
83 | -If the driver reports one of BTN_TOUCH or ABS_PRESSURE in addition to the | |
84 | -ABS_MT events, the last SYN_MT_REPORT event may be omitted. Otherwise, the | |
85 | -last SYN_REPORT will be dropped by the input core, resulting in no | |
86 | -zero-finger event reaching userland. | |
87 | - | |
88 | 179 | Event Semantics |
89 | 180 | --------------- |
90 | 181 | |
91 | -The word "contact" is used to describe a tool which is in direct contact | |
92 | -with the surface. A finger, a pen or a rubber all classify as contacts. | |
93 | - | |
94 | 182 | ABS_MT_TOUCH_MAJOR |
95 | 183 | |
96 | 184 | The length of the major axis of the contact. The length should be given in |
97 | 185 | |
... | ... | @@ -157,15 +245,16 @@ |
157 | 245 | ABS_MT_BLOB_ID |
158 | 246 | |
159 | 247 | The BLOB_ID groups several packets together into one arbitrarily shaped |
160 | -contact. This is a low-level anonymous grouping, and should not be confused | |
161 | -with the high-level trackingID [5]. Most kernel drivers will not have blob | |
162 | -capability, and can safely omit the event. | |
248 | +contact. This is a low-level anonymous grouping for type A devices, and | |
249 | +should not be confused with the high-level trackingID [5]. Most type A | |
250 | +devices do not have blob capability, so drivers can safely omit this event. | |
163 | 251 | |
164 | 252 | ABS_MT_TRACKING_ID |
165 | 253 | |
166 | 254 | The TRACKING_ID identifies an initiated contact throughout its life cycle |
167 | -[5]. There are currently only a few devices that support it, so this event | |
168 | -should normally be omitted. | |
255 | +[5]. This event is mandatory for type B devices. The value range of the | |
256 | +TRACKING_ID should be large enough to ensure unique identification of a | |
257 | +contact maintained over an extended period of time. | |
169 | 258 | |
170 | 259 | |
171 | 260 | Event Computation |
172 | 261 | |
... | ... | @@ -192,20 +281,11 @@ |
192 | 281 | Finger Tracking |
193 | 282 | --------------- |
194 | 283 | |
195 | -The kernel driver should generate an arbitrary enumeration of the set of | |
196 | -anonymous contacts currently on the surface. The order in which the packets | |
197 | -appear in the event stream is not important. | |
198 | - | |
199 | 284 | The process of finger tracking, i.e., to assign a unique trackingID to each |
200 | -initiated contact on the surface, is left to user space; preferably the | |
201 | -multi-touch X driver [3]. In that driver, the trackingID stays the same and | |
202 | -unique until the contact vanishes (when the finger leaves the surface). The | |
203 | -problem of assigning a set of anonymous fingers to a set of identified | |
204 | -fingers is a euclidian bipartite matching problem at each event update, and | |
205 | -relies on a sufficiently rapid update rate. | |
206 | - | |
207 | -There are a few devices that support trackingID in hardware. User space can | |
208 | -make use of these native identifiers to reduce bandwidth and cpu usage. | |
285 | +initiated contact on the surface, is a Euclidian Bipartite Matching | |
286 | +problem. At each event synchronization, the set of actual contacts is | |
287 | +matched to the set of contacts from the previous synchronization. A full | |
288 | +implementation can be found in [3]. | |
209 | 289 | |
210 | 290 | |
211 | 291 | Gestures |