Doug / smarc-fsl-linux-kernel

Download as
Browse Code »

Commit 06e3041164a1c52d6e8a369af8be3827f428272b

Authored by Joe Perches
Committed by David S. Miller
1 parent 3fbca4a2fe
Exists in smarc-l5.0.0_1.0.0-ga and in 5 other branches imx_3.10.17_1.0.1_ga, imx_3.10.53_1.1.0_ga, imx_3.14.28_1.0.0_ga, smarc-imx_3.10.53_1.1.0_ga, smarc-imx_3.14.28_1.0.0_ga

pktgen: Use ipv6_addr_any 

Use the standard test for a non-zero ipv6 address.

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

Showing 1 changed file with 1 additions and 5 deletions Inline Diff

1 /* 1 /*
2 * Authors: 2 * Authors:
3 * Copyright 2001, 2002 by Robert Olsson <robert.olsson@its.uu.se> 3 * Copyright 2001, 2002 by Robert Olsson <robert.olsson@its.uu.se>
4 * Uppsala University and 4 * Uppsala University and
5 * Swedish University of Agricultural Sciences 5 * Swedish University of Agricultural Sciences
6 * 6 *
7 * Alexey Kuznetsov <kuznet@ms2.inr.ac.ru> 7 * Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>
8 * Ben Greear <greearb@candelatech.com> 8 * Ben Greear <greearb@candelatech.com>
9 * Jens Låås <jens.laas@data.slu.se> 9 * Jens Låås <jens.laas@data.slu.se>
10 * 10 *
11 * This program is free software; you can redistribute it and/or 11 * This program is free software; you can redistribute it and/or
12 * modify it under the terms of the GNU General Public License 12 * modify it under the terms of the GNU General Public License
13 * as published by the Free Software Foundation; either version 13 * as published by the Free Software Foundation; either version
14 * 2 of the License, or (at your option) any later version. 14 * 2 of the License, or (at your option) any later version.
15 * 15 *
16 * 16 *
17 * A tool for loading the network with preconfigurated packets. 17 * A tool for loading the network with preconfigurated packets.
18 * The tool is implemented as a linux module. Parameters are output 18 * The tool is implemented as a linux module. Parameters are output
19 * device, delay (to hard_xmit), number of packets, and whether 19 * device, delay (to hard_xmit), number of packets, and whether
20 * to use multiple SKBs or just the same one. 20 * to use multiple SKBs or just the same one.
21 * pktgen uses the installed interface's output routine. 21 * pktgen uses the installed interface's output routine.
22 * 22 *
23 * Additional hacking by: 23 * Additional hacking by:
24 * 24 *
25 * Jens.Laas@data.slu.se 25 * Jens.Laas@data.slu.se
26 * Improved by ANK. 010120. 26 * Improved by ANK. 010120.
27 * Improved by ANK even more. 010212. 27 * Improved by ANK even more. 010212.
28 * MAC address typo fixed. 010417 --ro 28 * MAC address typo fixed. 010417 --ro
29 * Integrated. 020301 --DaveM 29 * Integrated. 020301 --DaveM
30 * Added multiskb option 020301 --DaveM 30 * Added multiskb option 020301 --DaveM
31 * Scaling of results. 020417--sigurdur@linpro.no 31 * Scaling of results. 020417--sigurdur@linpro.no
32 * Significant re-work of the module: 32 * Significant re-work of the module:
33 * * Convert to threaded model to more efficiently be able to transmit 33 * * Convert to threaded model to more efficiently be able to transmit
34 * and receive on multiple interfaces at once. 34 * and receive on multiple interfaces at once.
35 * * Converted many counters to __u64 to allow longer runs. 35 * * Converted many counters to __u64 to allow longer runs.
36 * * Allow configuration of ranges, like min/max IP address, MACs, 36 * * Allow configuration of ranges, like min/max IP address, MACs,
37 * and UDP-ports, for both source and destination, and can 37 * and UDP-ports, for both source and destination, and can
38 * set to use a random distribution or sequentially walk the range. 38 * set to use a random distribution or sequentially walk the range.
39 * * Can now change most values after starting. 39 * * Can now change most values after starting.
40 * * Place 12-byte packet in UDP payload with magic number, 40 * * Place 12-byte packet in UDP payload with magic number,
41 * sequence number, and timestamp. 41 * sequence number, and timestamp.
42 * * Add receiver code that detects dropped pkts, re-ordered pkts, and 42 * * Add receiver code that detects dropped pkts, re-ordered pkts, and
43 * latencies (with micro-second) precision. 43 * latencies (with micro-second) precision.
44 * * Add IOCTL interface to easily get counters & configuration. 44 * * Add IOCTL interface to easily get counters & configuration.
45 * --Ben Greear <greearb@candelatech.com> 45 * --Ben Greear <greearb@candelatech.com>
46 * 46 *
47 * Renamed multiskb to clone_skb and cleaned up sending core for two distinct 47 * Renamed multiskb to clone_skb and cleaned up sending core for two distinct
48 * skb modes. A clone_skb=0 mode for Ben "ranges" work and a clone_skb != 0 48 * skb modes. A clone_skb=0 mode for Ben "ranges" work and a clone_skb != 0
49 * as a "fastpath" with a configurable number of clones after alloc's. 49 * as a "fastpath" with a configurable number of clones after alloc's.
50 * clone_skb=0 means all packets are allocated this also means ranges time 50 * clone_skb=0 means all packets are allocated this also means ranges time
51 * stamps etc can be used. clone_skb=100 means 1 malloc is followed by 100 51 * stamps etc can be used. clone_skb=100 means 1 malloc is followed by 100
52 * clones. 52 * clones.
53 * 53 *
54 * Also moved to /proc/net/pktgen/ 54 * Also moved to /proc/net/pktgen/
55 * --ro 55 * --ro
56 * 56 *
57 * Sept 10: Fixed threading/locking. Lots of bone-headed and more clever 57 * Sept 10: Fixed threading/locking. Lots of bone-headed and more clever
58 * mistakes. Also merged in DaveM's patch in the -pre6 patch. 58 * mistakes. Also merged in DaveM's patch in the -pre6 patch.
59 * --Ben Greear <greearb@candelatech.com> 59 * --Ben Greear <greearb@candelatech.com>
60 * 60 *
61 * Integrated to 2.5.x 021029 --Lucio Maciel (luciomaciel@zipmail.com.br) 61 * Integrated to 2.5.x 021029 --Lucio Maciel (luciomaciel@zipmail.com.br)
62 * 62 *
63 * 63 *
64 * 021124 Finished major redesign and rewrite for new functionality. 64 * 021124 Finished major redesign and rewrite for new functionality.
65 * See Documentation/networking/pktgen.txt for how to use this. 65 * See Documentation/networking/pktgen.txt for how to use this.
66 * 66 *
67 * The new operation: 67 * The new operation:
68 * For each CPU one thread/process is created at start. This process checks 68 * For each CPU one thread/process is created at start. This process checks
69 * for running devices in the if_list and sends packets until count is 0 it 69 * for running devices in the if_list and sends packets until count is 0 it
70 * also the thread checks the thread->control which is used for inter-process 70 * also the thread checks the thread->control which is used for inter-process
71 * communication. controlling process "posts" operations to the threads this 71 * communication. controlling process "posts" operations to the threads this
72 * way. The if_lock should be possible to remove when add/rem_device is merged 72 * way. The if_lock should be possible to remove when add/rem_device is merged
73 * into this too. 73 * into this too.
74 * 74 *
75 * By design there should only be *one* "controlling" process. In practice 75 * By design there should only be *one* "controlling" process. In practice
76 * multiple write accesses gives unpredictable result. Understood by "write" 76 * multiple write accesses gives unpredictable result. Understood by "write"
77 * to /proc gives result code thats should be read be the "writer". 77 * to /proc gives result code thats should be read be the "writer".
78 * For practical use this should be no problem. 78 * For practical use this should be no problem.
79 * 79 *
80 * Note when adding devices to a specific CPU there good idea to also assign 80 * Note when adding devices to a specific CPU there good idea to also assign
81 * /proc/irq/XX/smp_affinity so TX-interrupts gets bound to the same CPU. 81 * /proc/irq/XX/smp_affinity so TX-interrupts gets bound to the same CPU.
82 * --ro 82 * --ro
83 * 83 *
84 * Fix refcount off by one if first packet fails, potential null deref, 84 * Fix refcount off by one if first packet fails, potential null deref,
85 * memleak 030710- KJP 85 * memleak 030710- KJP
86 * 86 *
87 * First "ranges" functionality for ipv6 030726 --ro 87 * First "ranges" functionality for ipv6 030726 --ro
88 * 88 *
89 * Included flow support. 030802 ANK. 89 * Included flow support. 030802 ANK.
90 * 90 *
91 * Fixed unaligned access on IA-64 Grant Grundler <grundler@parisc-linux.org> 91 * Fixed unaligned access on IA-64 Grant Grundler <grundler@parisc-linux.org>
92 * 92 *
93 * Remove if fix from added Harald Welte <laforge@netfilter.org> 040419 93 * Remove if fix from added Harald Welte <laforge@netfilter.org> 040419
94 * ia64 compilation fix from Aron Griffis <aron@hp.com> 040604 94 * ia64 compilation fix from Aron Griffis <aron@hp.com> 040604
95 * 95 *
96 * New xmit() return, do_div and misc clean up by Stephen Hemminger 96 * New xmit() return, do_div and misc clean up by Stephen Hemminger
97 * <shemminger@osdl.org> 040923 97 * <shemminger@osdl.org> 040923
98 * 98 *
99 * Randy Dunlap fixed u64 printk compiler waring 99 * Randy Dunlap fixed u64 printk compiler waring
100 * 100 *
101 * Remove FCS from BW calculation. Lennert Buytenhek <buytenh@wantstofly.org> 101 * Remove FCS from BW calculation. Lennert Buytenhek <buytenh@wantstofly.org>
102 * New time handling. Lennert Buytenhek <buytenh@wantstofly.org> 041213 102 * New time handling. Lennert Buytenhek <buytenh@wantstofly.org> 041213
103 * 103 *
104 * Corrections from Nikolai Malykh (nmalykh@bilim.com) 104 * Corrections from Nikolai Malykh (nmalykh@bilim.com)
105 * Removed unused flags F_SET_SRCMAC & F_SET_SRCIP 041230 105 * Removed unused flags F_SET_SRCMAC & F_SET_SRCIP 041230
106 * 106 *
107 * interruptible_sleep_on_timeout() replaced Nishanth Aravamudan <nacc@us.ibm.com> 107 * interruptible_sleep_on_timeout() replaced Nishanth Aravamudan <nacc@us.ibm.com>
108 * 050103 108 * 050103
109 * 109 *
110 * MPLS support by Steven Whitehouse <steve@chygwyn.com> 110 * MPLS support by Steven Whitehouse <steve@chygwyn.com>
111 * 111 *
112 * 802.1Q/Q-in-Q support by Francesco Fondelli (FF) <francesco.fondelli@gmail.com> 112 * 802.1Q/Q-in-Q support by Francesco Fondelli (FF) <francesco.fondelli@gmail.com>
113 * 113 *
114 * Fixed src_mac command to set source mac of packet to value specified in 114 * Fixed src_mac command to set source mac of packet to value specified in
115 * command by Adit Ranadive <adit.262@gmail.com> 115 * command by Adit Ranadive <adit.262@gmail.com>
116 * 116 *
117 */ 117 */
118 118
119 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt 119 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
120 120
121 #include <linux/sys.h> 121 #include <linux/sys.h>
122 #include <linux/types.h> 122 #include <linux/types.h>
123 #include <linux/module.h> 123 #include <linux/module.h>
124 #include <linux/moduleparam.h> 124 #include <linux/moduleparam.h>
125 #include <linux/kernel.h> 125 #include <linux/kernel.h>
126 #include <linux/mutex.h> 126 #include <linux/mutex.h>
127 #include <linux/sched.h> 127 #include <linux/sched.h>
128 #include <linux/slab.h> 128 #include <linux/slab.h>
129 #include <linux/vmalloc.h> 129 #include <linux/vmalloc.h>
130 #include <linux/unistd.h> 130 #include <linux/unistd.h>
131 #include <linux/string.h> 131 #include <linux/string.h>
132 #include <linux/ptrace.h> 132 #include <linux/ptrace.h>
133 #include <linux/errno.h> 133 #include <linux/errno.h>
134 #include <linux/ioport.h> 134 #include <linux/ioport.h>
135 #include <linux/interrupt.h> 135 #include <linux/interrupt.h>
136 #include <linux/capability.h> 136 #include <linux/capability.h>
137 #include <linux/hrtimer.h> 137 #include <linux/hrtimer.h>
138 #include <linux/freezer.h> 138 #include <linux/freezer.h>
139 #include <linux/delay.h> 139 #include <linux/delay.h>
140 #include <linux/timer.h> 140 #include <linux/timer.h>
141 #include <linux/list.h> 141 #include <linux/list.h>
142 #include <linux/init.h> 142 #include <linux/init.h>
143 #include <linux/skbuff.h> 143 #include <linux/skbuff.h>
144 #include <linux/netdevice.h> 144 #include <linux/netdevice.h>
145 #include <linux/inet.h> 145 #include <linux/inet.h>
146 #include <linux/inetdevice.h> 146 #include <linux/inetdevice.h>
147 #include <linux/rtnetlink.h> 147 #include <linux/rtnetlink.h>
148 #include <linux/if_arp.h> 148 #include <linux/if_arp.h>
149 #include <linux/if_vlan.h> 149 #include <linux/if_vlan.h>
150 #include <linux/in.h> 150 #include <linux/in.h>
151 #include <linux/ip.h> 151 #include <linux/ip.h>
152 #include <linux/ipv6.h> 152 #include <linux/ipv6.h>
153 #include <linux/udp.h> 153 #include <linux/udp.h>
154 #include <linux/proc_fs.h> 154 #include <linux/proc_fs.h>
155 #include <linux/seq_file.h> 155 #include <linux/seq_file.h>
156 #include <linux/wait.h> 156 #include <linux/wait.h>
157 #include <linux/etherdevice.h> 157 #include <linux/etherdevice.h>
158 #include <linux/kthread.h> 158 #include <linux/kthread.h>
159 #include <linux/prefetch.h> 159 #include <linux/prefetch.h>
160 #include <net/net_namespace.h> 160 #include <net/net_namespace.h>
161 #include <net/checksum.h> 161 #include <net/checksum.h>
162 #include <net/ipv6.h> 162 #include <net/ipv6.h>
163 #include <net/addrconf.h> 163 #include <net/addrconf.h>
164 #ifdef CONFIG_XFRM 164 #ifdef CONFIG_XFRM
165 #include <net/xfrm.h> 165 #include <net/xfrm.h>
166 #endif 166 #endif
167 #include <asm/byteorder.h> 167 #include <asm/byteorder.h>
168 #include <linux/rcupdate.h> 168 #include <linux/rcupdate.h>
169 #include <linux/bitops.h> 169 #include <linux/bitops.h>
170 #include <linux/io.h> 170 #include <linux/io.h>
171 #include <linux/timex.h> 171 #include <linux/timex.h>
172 #include <linux/uaccess.h> 172 #include <linux/uaccess.h>
173 #include <asm/dma.h> 173 #include <asm/dma.h>
174 #include <asm/div64.h> /* do_div */ 174 #include <asm/div64.h> /* do_div */
175 175
176 #define VERSION "2.74" 176 #define VERSION "2.74"
177 #define IP_NAME_SZ 32 177 #define IP_NAME_SZ 32
178 #define MAX_MPLS_LABELS 16 /* This is the max label stack depth */ 178 #define MAX_MPLS_LABELS 16 /* This is the max label stack depth */
179 #define MPLS_STACK_BOTTOM htonl(0x00000100) 179 #define MPLS_STACK_BOTTOM htonl(0x00000100)
180 180
181 #define func_enter() pr_debug("entering %s\n", __func__); 181 #define func_enter() pr_debug("entering %s\n", __func__);
182 182
183 /* Device flag bits */ 183 /* Device flag bits */
184 #define F_IPSRC_RND (1<<0) /* IP-Src Random */ 184 #define F_IPSRC_RND (1<<0) /* IP-Src Random */
185 #define F_IPDST_RND (1<<1) /* IP-Dst Random */ 185 #define F_IPDST_RND (1<<1) /* IP-Dst Random */
186 #define F_UDPSRC_RND (1<<2) /* UDP-Src Random */ 186 #define F_UDPSRC_RND (1<<2) /* UDP-Src Random */
187 #define F_UDPDST_RND (1<<3) /* UDP-Dst Random */ 187 #define F_UDPDST_RND (1<<3) /* UDP-Dst Random */
188 #define F_MACSRC_RND (1<<4) /* MAC-Src Random */ 188 #define F_MACSRC_RND (1<<4) /* MAC-Src Random */
189 #define F_MACDST_RND (1<<5) /* MAC-Dst Random */ 189 #define F_MACDST_RND (1<<5) /* MAC-Dst Random */
190 #define F_TXSIZE_RND (1<<6) /* Transmit size is random */ 190 #define F_TXSIZE_RND (1<<6) /* Transmit size is random */
191 #define F_IPV6 (1<<7) /* Interface in IPV6 Mode */ 191 #define F_IPV6 (1<<7) /* Interface in IPV6 Mode */
192 #define F_MPLS_RND (1<<8) /* Random MPLS labels */ 192 #define F_MPLS_RND (1<<8) /* Random MPLS labels */
193 #define F_VID_RND (1<<9) /* Random VLAN ID */ 193 #define F_VID_RND (1<<9) /* Random VLAN ID */
194 #define F_SVID_RND (1<<10) /* Random SVLAN ID */ 194 #define F_SVID_RND (1<<10) /* Random SVLAN ID */
195 #define F_FLOW_SEQ (1<<11) /* Sequential flows */ 195 #define F_FLOW_SEQ (1<<11) /* Sequential flows */
196 #define F_IPSEC_ON (1<<12) /* ipsec on for flows */ 196 #define F_IPSEC_ON (1<<12) /* ipsec on for flows */
197 #define F_QUEUE_MAP_RND (1<<13) /* queue map Random */ 197 #define F_QUEUE_MAP_RND (1<<13) /* queue map Random */
198 #define F_QUEUE_MAP_CPU (1<<14) /* queue map mirrors smp_processor_id() */ 198 #define F_QUEUE_MAP_CPU (1<<14) /* queue map mirrors smp_processor_id() */
199 #define F_NODE (1<<15) /* Node memory alloc*/ 199 #define F_NODE (1<<15) /* Node memory alloc*/
200 200
201 /* Thread control flag bits */ 201 /* Thread control flag bits */
202 #define T_STOP (1<<0) /* Stop run */ 202 #define T_STOP (1<<0) /* Stop run */
203 #define T_RUN (1<<1) /* Start run */ 203 #define T_RUN (1<<1) /* Start run */
204 #define T_REMDEVALL (1<<2) /* Remove all devs */ 204 #define T_REMDEVALL (1<<2) /* Remove all devs */
205 #define T_REMDEV (1<<3) /* Remove one dev */ 205 #define T_REMDEV (1<<3) /* Remove one dev */
206 206
207 /* If lock -- can be removed after some work */ 207 /* If lock -- can be removed after some work */
208 #define if_lock(t) spin_lock(&(t->if_lock)); 208 #define if_lock(t) spin_lock(&(t->if_lock));
209 #define if_unlock(t) spin_unlock(&(t->if_lock)); 209 #define if_unlock(t) spin_unlock(&(t->if_lock));
210 210
211 /* Used to help with determining the pkts on receive */ 211 /* Used to help with determining the pkts on receive */
212 #define PKTGEN_MAGIC 0xbe9be955 212 #define PKTGEN_MAGIC 0xbe9be955
213 #define PG_PROC_DIR "pktgen" 213 #define PG_PROC_DIR "pktgen"
214 #define PGCTRL "pgctrl" 214 #define PGCTRL "pgctrl"
215 static struct proc_dir_entry *pg_proc_dir; 215 static struct proc_dir_entry *pg_proc_dir;
216 216
217 #define MAX_CFLOWS 65536 217 #define MAX_CFLOWS 65536
218 218
219 #define VLAN_TAG_SIZE(x) ((x)->vlan_id == 0xffff ? 0 : 4) 219 #define VLAN_TAG_SIZE(x) ((x)->vlan_id == 0xffff ? 0 : 4)
220 #define SVLAN_TAG_SIZE(x) ((x)->svlan_id == 0xffff ? 0 : 4) 220 #define SVLAN_TAG_SIZE(x) ((x)->svlan_id == 0xffff ? 0 : 4)
221 221
222 struct flow_state { 222 struct flow_state {
223 __be32 cur_daddr; 223 __be32 cur_daddr;
224 int count; 224 int count;
225 #ifdef CONFIG_XFRM 225 #ifdef CONFIG_XFRM
226 struct xfrm_state *x; 226 struct xfrm_state *x;
227 #endif 227 #endif
228 __u32 flags; 228 __u32 flags;
229 }; 229 };
230 230
231 /* flow flag bits */ 231 /* flow flag bits */
232 #define F_INIT (1<<0) /* flow has been initialized */ 232 #define F_INIT (1<<0) /* flow has been initialized */
233 233
234 struct pktgen_dev { 234 struct pktgen_dev {
235 /* 235 /*
236 * Try to keep frequent/infrequent used vars. separated. 236 * Try to keep frequent/infrequent used vars. separated.
237 */ 237 */
238 struct proc_dir_entry *entry; /* proc file */ 238 struct proc_dir_entry *entry; /* proc file */
239 struct pktgen_thread *pg_thread;/* the owner */ 239 struct pktgen_thread *pg_thread;/* the owner */
240 struct list_head list; /* chaining in the thread's run-queue */ 240 struct list_head list; /* chaining in the thread's run-queue */
241 241
242 int running; /* if false, the test will stop */ 242 int running; /* if false, the test will stop */
243 243
244 /* If min != max, then we will either do a linear iteration, or 244 /* If min != max, then we will either do a linear iteration, or
245 * we will do a random selection from within the range. 245 * we will do a random selection from within the range.
246 */ 246 */
247 __u32 flags; 247 __u32 flags;
248 int removal_mark; /* non-zero => the device is marked for 248 int removal_mark; /* non-zero => the device is marked for
249 * removal by worker thread */ 249 * removal by worker thread */
250 250
251 int min_pkt_size; 251 int min_pkt_size;
252 int max_pkt_size; 252 int max_pkt_size;
253 int pkt_overhead; /* overhead for MPLS, VLANs, IPSEC etc */ 253 int pkt_overhead; /* overhead for MPLS, VLANs, IPSEC etc */
254 int nfrags; 254 int nfrags;
255 struct page *page; 255 struct page *page;
256 u64 delay; /* nano-seconds */ 256 u64 delay; /* nano-seconds */
257 257
258 __u64 count; /* Default No packets to send */ 258 __u64 count; /* Default No packets to send */
259 __u64 sofar; /* How many pkts we've sent so far */ 259 __u64 sofar; /* How many pkts we've sent so far */
260 __u64 tx_bytes; /* How many bytes we've transmitted */ 260 __u64 tx_bytes; /* How many bytes we've transmitted */
261 __u64 errors; /* Errors when trying to transmit, */ 261 __u64 errors; /* Errors when trying to transmit, */
262 262
263 /* runtime counters relating to clone_skb */ 263 /* runtime counters relating to clone_skb */
264 264
265 __u64 allocated_skbs; 265 __u64 allocated_skbs;
266 __u32 clone_count; 266 __u32 clone_count;
267 int last_ok; /* Was last skb sent? 267 int last_ok; /* Was last skb sent?
268 * Or a failed transmit of some sort? 268 * Or a failed transmit of some sort?
269 * This will keep sequence numbers in order 269 * This will keep sequence numbers in order
270 */ 270 */
271 ktime_t next_tx; 271 ktime_t next_tx;
272 ktime_t started_at; 272 ktime_t started_at;
273 ktime_t stopped_at; 273 ktime_t stopped_at;
274 u64 idle_acc; /* nano-seconds */ 274 u64 idle_acc; /* nano-seconds */
275 275
276 __u32 seq_num; 276 __u32 seq_num;
277 277
278 int clone_skb; /* 278 int clone_skb; /*
279 * Use multiple SKBs during packet gen. 279 * Use multiple SKBs during packet gen.
280 * If this number is greater than 1, then 280 * If this number is greater than 1, then
281 * that many copies of the same packet will be 281 * that many copies of the same packet will be
282 * sent before a new packet is allocated. 282 * sent before a new packet is allocated.
283 * If you want to send 1024 identical packets 283 * If you want to send 1024 identical packets
284 * before creating a new packet, 284 * before creating a new packet,
285 * set clone_skb to 1024. 285 * set clone_skb to 1024.
286 */ 286 */
287 287
288 char dst_min[IP_NAME_SZ]; /* IP, ie 1.2.3.4 */ 288 char dst_min[IP_NAME_SZ]; /* IP, ie 1.2.3.4 */
289 char dst_max[IP_NAME_SZ]; /* IP, ie 1.2.3.4 */ 289 char dst_max[IP_NAME_SZ]; /* IP, ie 1.2.3.4 */
290 char src_min[IP_NAME_SZ]; /* IP, ie 1.2.3.4 */ 290 char src_min[IP_NAME_SZ]; /* IP, ie 1.2.3.4 */
291 char src_max[IP_NAME_SZ]; /* IP, ie 1.2.3.4 */ 291 char src_max[IP_NAME_SZ]; /* IP, ie 1.2.3.4 */
292 292
293 struct in6_addr in6_saddr; 293 struct in6_addr in6_saddr;
294 struct in6_addr in6_daddr; 294 struct in6_addr in6_daddr;
295 struct in6_addr cur_in6_daddr; 295 struct in6_addr cur_in6_daddr;
296 struct in6_addr cur_in6_saddr; 296 struct in6_addr cur_in6_saddr;
297 /* For ranges */ 297 /* For ranges */
298 struct in6_addr min_in6_daddr; 298 struct in6_addr min_in6_daddr;
299 struct in6_addr max_in6_daddr; 299 struct in6_addr max_in6_daddr;
300 struct in6_addr min_in6_saddr; 300 struct in6_addr min_in6_saddr;
301 struct in6_addr max_in6_saddr; 301 struct in6_addr max_in6_saddr;
302 302
303 /* If we're doing ranges, random or incremental, then this 303 /* If we're doing ranges, random or incremental, then this
304 * defines the min/max for those ranges. 304 * defines the min/max for those ranges.
305 */ 305 */
306 __be32 saddr_min; /* inclusive, source IP address */ 306 __be32 saddr_min; /* inclusive, source IP address */
307 __be32 saddr_max; /* exclusive, source IP address */ 307 __be32 saddr_max; /* exclusive, source IP address */
308 __be32 daddr_min; /* inclusive, dest IP address */ 308 __be32 daddr_min; /* inclusive, dest IP address */
309 __be32 daddr_max; /* exclusive, dest IP address */ 309 __be32 daddr_max; /* exclusive, dest IP address */
310 310
311 __u16 udp_src_min; /* inclusive, source UDP port */ 311 __u16 udp_src_min; /* inclusive, source UDP port */
312 __u16 udp_src_max; /* exclusive, source UDP port */ 312 __u16 udp_src_max; /* exclusive, source UDP port */
313 __u16 udp_dst_min; /* inclusive, dest UDP port */ 313 __u16 udp_dst_min; /* inclusive, dest UDP port */
314 __u16 udp_dst_max; /* exclusive, dest UDP port */ 314 __u16 udp_dst_max; /* exclusive, dest UDP port */
315 315
316 /* DSCP + ECN */ 316 /* DSCP + ECN */
317 __u8 tos; /* six MSB of (former) IPv4 TOS 317 __u8 tos; /* six MSB of (former) IPv4 TOS
318 are for dscp codepoint */ 318 are for dscp codepoint */
319 __u8 traffic_class; /* ditto for the (former) Traffic Class in IPv6 319 __u8 traffic_class; /* ditto for the (former) Traffic Class in IPv6
320 (see RFC 3260, sec. 4) */ 320 (see RFC 3260, sec. 4) */
321 321
322 /* MPLS */ 322 /* MPLS */
323 unsigned int nr_labels; /* Depth of stack, 0 = no MPLS */ 323 unsigned int nr_labels; /* Depth of stack, 0 = no MPLS */
324 __be32 labels[MAX_MPLS_LABELS]; 324 __be32 labels[MAX_MPLS_LABELS];
325 325
326 /* VLAN/SVLAN (802.1Q/Q-in-Q) */ 326 /* VLAN/SVLAN (802.1Q/Q-in-Q) */
327 __u8 vlan_p; 327 __u8 vlan_p;
328 __u8 vlan_cfi; 328 __u8 vlan_cfi;
329 __u16 vlan_id; /* 0xffff means no vlan tag */ 329 __u16 vlan_id; /* 0xffff means no vlan tag */
330 330
331 __u8 svlan_p; 331 __u8 svlan_p;
332 __u8 svlan_cfi; 332 __u8 svlan_cfi;
333 __u16 svlan_id; /* 0xffff means no svlan tag */ 333 __u16 svlan_id; /* 0xffff means no svlan tag */
334 334
335 __u32 src_mac_count; /* How many MACs to iterate through */ 335 __u32 src_mac_count; /* How many MACs to iterate through */
336 __u32 dst_mac_count; /* How many MACs to iterate through */ 336 __u32 dst_mac_count; /* How many MACs to iterate through */
337 337
338 unsigned char dst_mac[ETH_ALEN]; 338 unsigned char dst_mac[ETH_ALEN];
339 unsigned char src_mac[ETH_ALEN]; 339 unsigned char src_mac[ETH_ALEN];
340 340
341 __u32 cur_dst_mac_offset; 341 __u32 cur_dst_mac_offset;
342 __u32 cur_src_mac_offset; 342 __u32 cur_src_mac_offset;
343 __be32 cur_saddr; 343 __be32 cur_saddr;
344 __be32 cur_daddr; 344 __be32 cur_daddr;
345 __u16 ip_id; 345 __u16 ip_id;
346 __u16 cur_udp_dst; 346 __u16 cur_udp_dst;
347 __u16 cur_udp_src; 347 __u16 cur_udp_src;
348 __u16 cur_queue_map; 348 __u16 cur_queue_map;
349 __u32 cur_pkt_size; 349 __u32 cur_pkt_size;
350 __u32 last_pkt_size; 350 __u32 last_pkt_size;
351 351
352 __u8 hh[14]; 352 __u8 hh[14];
353 /* = { 353 /* = {
354 0x00, 0x80, 0xC8, 0x79, 0xB3, 0xCB, 354 0x00, 0x80, 0xC8, 0x79, 0xB3, 0xCB,
355 355
356 We fill in SRC address later 356 We fill in SRC address later
357 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 357 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
358 0x08, 0x00 358 0x08, 0x00
359 }; 359 };
360 */ 360 */
361 __u16 pad; /* pad out the hh struct to an even 16 bytes */ 361 __u16 pad; /* pad out the hh struct to an even 16 bytes */
362 362
363 struct sk_buff *skb; /* skb we are to transmit next, used for when we 363 struct sk_buff *skb; /* skb we are to transmit next, used for when we
364 * are transmitting the same one multiple times 364 * are transmitting the same one multiple times
365 */ 365 */
366 struct net_device *odev; /* The out-going device. 366 struct net_device *odev; /* The out-going device.
367 * Note that the device should have it's 367 * Note that the device should have it's
368 * pg_info pointer pointing back to this 368 * pg_info pointer pointing back to this
369 * device. 369 * device.
370 * Set when the user specifies the out-going 370 * Set when the user specifies the out-going
371 * device name (not when the inject is 371 * device name (not when the inject is
372 * started as it used to do.) 372 * started as it used to do.)
373 */ 373 */
374 char odevname[32]; 374 char odevname[32];
375 struct flow_state *flows; 375 struct flow_state *flows;
376 unsigned int cflows; /* Concurrent flows (config) */ 376 unsigned int cflows; /* Concurrent flows (config) */
377 unsigned int lflow; /* Flow length (config) */ 377 unsigned int lflow; /* Flow length (config) */
378 unsigned int nflows; /* accumulated flows (stats) */ 378 unsigned int nflows; /* accumulated flows (stats) */
379 unsigned int curfl; /* current sequenced flow (state)*/ 379 unsigned int curfl; /* current sequenced flow (state)*/
380 380
381 u16 queue_map_min; 381 u16 queue_map_min;
382 u16 queue_map_max; 382 u16 queue_map_max;
383 __u32 skb_priority; /* skb priority field */ 383 __u32 skb_priority; /* skb priority field */
384 int node; /* Memory node */ 384 int node; /* Memory node */
385 385
386 #ifdef CONFIG_XFRM 386 #ifdef CONFIG_XFRM
387 __u8 ipsmode; /* IPSEC mode (config) */ 387 __u8 ipsmode; /* IPSEC mode (config) */
388 __u8 ipsproto; /* IPSEC type (config) */ 388 __u8 ipsproto; /* IPSEC type (config) */
389 #endif 389 #endif
390 char result[512]; 390 char result[512];
391 }; 391 };
392 392
393 struct pktgen_hdr { 393 struct pktgen_hdr {
394 __be32 pgh_magic; 394 __be32 pgh_magic;
395 __be32 seq_num; 395 __be32 seq_num;
396 __be32 tv_sec; 396 __be32 tv_sec;
397 __be32 tv_usec; 397 __be32 tv_usec;
398 }; 398 };
399 399
400 static bool pktgen_exiting __read_mostly; 400 static bool pktgen_exiting __read_mostly;
401 401
402 struct pktgen_thread { 402 struct pktgen_thread {
403 spinlock_t if_lock; /* for list of devices */ 403 spinlock_t if_lock; /* for list of devices */
404 struct list_head if_list; /* All device here */ 404 struct list_head if_list; /* All device here */
405 struct list_head th_list; 405 struct list_head th_list;
406 struct task_struct *tsk; 406 struct task_struct *tsk;
407 char result[512]; 407 char result[512];
408 408
409 /* Field for thread to receive "posted" events terminate, 409 /* Field for thread to receive "posted" events terminate,
410 stop ifs etc. */ 410 stop ifs etc. */
411 411
412 u32 control; 412 u32 control;
413 int cpu; 413 int cpu;
414 414
415 wait_queue_head_t queue; 415 wait_queue_head_t queue;
416 struct completion start_done; 416 struct completion start_done;
417 }; 417 };
418 418
419 #define REMOVE 1 419 #define REMOVE 1
420 #define FIND 0 420 #define FIND 0
421 421
422 static inline ktime_t ktime_now(void) 422 static inline ktime_t ktime_now(void)
423 { 423 {
424 struct timespec ts; 424 struct timespec ts;
425 ktime_get_ts(&ts); 425 ktime_get_ts(&ts);
426 426
427 return timespec_to_ktime(ts); 427 return timespec_to_ktime(ts);
428 } 428 }
429 429
430 /* This works even if 32 bit because of careful byte order choice */ 430 /* This works even if 32 bit because of careful byte order choice */
431 static inline int ktime_lt(const ktime_t cmp1, const ktime_t cmp2) 431 static inline int ktime_lt(const ktime_t cmp1, const ktime_t cmp2)
432 { 432 {
433 return cmp1.tv64 < cmp2.tv64; 433 return cmp1.tv64 < cmp2.tv64;
434 } 434 }
435 435
436 static const char version[] = 436 static const char version[] =
437 "Packet Generator for packet performance testing. " 437 "Packet Generator for packet performance testing. "
438 "Version: " VERSION "\n"; 438 "Version: " VERSION "\n";
439 439
440 static int pktgen_remove_device(struct pktgen_thread *t, struct pktgen_dev *i); 440 static int pktgen_remove_device(struct pktgen_thread *t, struct pktgen_dev *i);
441 static int pktgen_add_device(struct pktgen_thread *t, const char *ifname); 441 static int pktgen_add_device(struct pktgen_thread *t, const char *ifname);
442 static struct pktgen_dev *pktgen_find_dev(struct pktgen_thread *t, 442 static struct pktgen_dev *pktgen_find_dev(struct pktgen_thread *t,
443 const char *ifname, bool exact); 443 const char *ifname, bool exact);
444 static int pktgen_device_event(struct notifier_block *, unsigned long, void *); 444 static int pktgen_device_event(struct notifier_block *, unsigned long, void *);
445 static void pktgen_run_all_threads(void); 445 static void pktgen_run_all_threads(void);
446 static void pktgen_reset_all_threads(void); 446 static void pktgen_reset_all_threads(void);
447 static void pktgen_stop_all_threads_ifs(void); 447 static void pktgen_stop_all_threads_ifs(void);
448 448
449 static void pktgen_stop(struct pktgen_thread *t); 449 static void pktgen_stop(struct pktgen_thread *t);
450 static void pktgen_clear_counters(struct pktgen_dev *pkt_dev); 450 static void pktgen_clear_counters(struct pktgen_dev *pkt_dev);
451 451
452 /* Module parameters, defaults. */ 452 /* Module parameters, defaults. */
453 static int pg_count_d __read_mostly = 1000; 453 static int pg_count_d __read_mostly = 1000;
454 static int pg_delay_d __read_mostly; 454 static int pg_delay_d __read_mostly;
455 static int pg_clone_skb_d __read_mostly; 455 static int pg_clone_skb_d __read_mostly;
456 static int debug __read_mostly; 456 static int debug __read_mostly;
457 457
458 static DEFINE_MUTEX(pktgen_thread_lock); 458 static DEFINE_MUTEX(pktgen_thread_lock);
459 static LIST_HEAD(pktgen_threads); 459 static LIST_HEAD(pktgen_threads);
460 460
461 static struct notifier_block pktgen_notifier_block = { 461 static struct notifier_block pktgen_notifier_block = {
462 .notifier_call = pktgen_device_event, 462 .notifier_call = pktgen_device_event,
463 }; 463 };
464 464
465 /* 465 /*
466 * /proc handling functions 466 * /proc handling functions
467 * 467 *
468 */ 468 */
469 469
470 static int pgctrl_show(struct seq_file *seq, void *v) 470 static int pgctrl_show(struct seq_file *seq, void *v)
471 { 471 {
472 seq_puts(seq, version); 472 seq_puts(seq, version);
473 return 0; 473 return 0;
474 } 474 }
475 475
476 static ssize_t pgctrl_write(struct file *file, const char __user *buf, 476 static ssize_t pgctrl_write(struct file *file, const char __user *buf,
477 size_t count, loff_t *ppos) 477 size_t count, loff_t *ppos)
478 { 478 {
479 int err = 0; 479 int err = 0;
480 char data[128]; 480 char data[128];
481 481
482 if (!capable(CAP_NET_ADMIN)) { 482 if (!capable(CAP_NET_ADMIN)) {
483 err = -EPERM; 483 err = -EPERM;
484 goto out; 484 goto out;
485 } 485 }
486 486
487 if (count > sizeof(data)) 487 if (count > sizeof(data))
488 count = sizeof(data); 488 count = sizeof(data);
489 489
490 if (copy_from_user(data, buf, count)) { 490 if (copy_from_user(data, buf, count)) {
491 err = -EFAULT; 491 err = -EFAULT;
492 goto out; 492 goto out;
493 } 493 }
494 data[count - 1] = 0; /* Make string */ 494 data[count - 1] = 0; /* Make string */
495 495
496 if (!strcmp(data, "stop")) 496 if (!strcmp(data, "stop"))
497 pktgen_stop_all_threads_ifs(); 497 pktgen_stop_all_threads_ifs();
498 498
499 else if (!strcmp(data, "start")) 499 else if (!strcmp(data, "start"))
500 pktgen_run_all_threads(); 500 pktgen_run_all_threads();
501 501
502 else if (!strcmp(data, "reset")) 502 else if (!strcmp(data, "reset"))
503 pktgen_reset_all_threads(); 503 pktgen_reset_all_threads();
504 504
505 else 505 else
506 pr_warning("Unknown command: %s\n", data); 506 pr_warning("Unknown command: %s\n", data);
507 507
508 err = count; 508 err = count;
509 509
510 out: 510 out:
511 return err; 511 return err;
512 } 512 }
513 513
514 static int pgctrl_open(struct inode *inode, struct file *file) 514 static int pgctrl_open(struct inode *inode, struct file *file)
515 { 515 {
516 return single_open(file, pgctrl_show, PDE(inode)->data); 516 return single_open(file, pgctrl_show, PDE(inode)->data);
517 } 517 }
518 518
519 static const struct file_operations pktgen_fops = { 519 static const struct file_operations pktgen_fops = {
520 .owner = THIS_MODULE, 520 .owner = THIS_MODULE,
521 .open = pgctrl_open, 521 .open = pgctrl_open,
522 .read = seq_read, 522 .read = seq_read,
523 .llseek = seq_lseek, 523 .llseek = seq_lseek,
524 .write = pgctrl_write, 524 .write = pgctrl_write,
525 .release = single_release, 525 .release = single_release,
526 }; 526 };
527 527
528 static int pktgen_if_show(struct seq_file *seq, void *v) 528 static int pktgen_if_show(struct seq_file *seq, void *v)
529 { 529 {
530 const struct pktgen_dev *pkt_dev = seq->private; 530 const struct pktgen_dev *pkt_dev = seq->private;
531 ktime_t stopped; 531 ktime_t stopped;
532 u64 idle; 532 u64 idle;
533 533
534 seq_printf(seq, 534 seq_printf(seq,
535 "Params: count %llu min_pkt_size: %u max_pkt_size: %u\n", 535 "Params: count %llu min_pkt_size: %u max_pkt_size: %u\n",
536 (unsigned long long)pkt_dev->count, pkt_dev->min_pkt_size, 536 (unsigned long long)pkt_dev->count, pkt_dev->min_pkt_size,
537 pkt_dev->max_pkt_size); 537 pkt_dev->max_pkt_size);
538 538
539 seq_printf(seq, 539 seq_printf(seq,
540 " frags: %d delay: %llu clone_skb: %d ifname: %s\n", 540 " frags: %d delay: %llu clone_skb: %d ifname: %s\n",
541 pkt_dev->nfrags, (unsigned long long) pkt_dev->delay, 541 pkt_dev->nfrags, (unsigned long long) pkt_dev->delay,
542 pkt_dev->clone_skb, pkt_dev->odevname); 542 pkt_dev->clone_skb, pkt_dev->odevname);
543 543
544 seq_printf(seq, " flows: %u flowlen: %u\n", pkt_dev->cflows, 544 seq_printf(seq, " flows: %u flowlen: %u\n", pkt_dev->cflows,
545 pkt_dev->lflow); 545 pkt_dev->lflow);
546 546
547 seq_printf(seq, 547 seq_printf(seq,
548 " queue_map_min: %u queue_map_max: %u\n", 548 " queue_map_min: %u queue_map_max: %u\n",
549 pkt_dev->queue_map_min, 549 pkt_dev->queue_map_min,
550 pkt_dev->queue_map_max); 550 pkt_dev->queue_map_max);
551 551
552 if (pkt_dev->skb_priority) 552 if (pkt_dev->skb_priority)
553 seq_printf(seq, " skb_priority: %u\n", 553 seq_printf(seq, " skb_priority: %u\n",
554 pkt_dev->skb_priority); 554 pkt_dev->skb_priority);
555 555
556 if (pkt_dev->flags & F_IPV6) { 556 if (pkt_dev->flags & F_IPV6) {
557 seq_printf(seq, 557 seq_printf(seq,
558 " saddr: %pI6c min_saddr: %pI6c max_saddr: %pI6c\n" 558 " saddr: %pI6c min_saddr: %pI6c max_saddr: %pI6c\n"
559 " daddr: %pI6c min_daddr: %pI6c max_daddr: %pI6c\n", 559 " daddr: %pI6c min_daddr: %pI6c max_daddr: %pI6c\n",
560 &pkt_dev->in6_saddr, 560 &pkt_dev->in6_saddr,
561 &pkt_dev->min_in6_saddr, &pkt_dev->max_in6_saddr, 561 &pkt_dev->min_in6_saddr, &pkt_dev->max_in6_saddr,
562 &pkt_dev->in6_daddr, 562 &pkt_dev->in6_daddr,
563 &pkt_dev->min_in6_daddr, &pkt_dev->max_in6_daddr); 563 &pkt_dev->min_in6_daddr, &pkt_dev->max_in6_daddr);
564 } else { 564 } else {
565 seq_printf(seq, 565 seq_printf(seq,
566 " dst_min: %s dst_max: %s\n", 566 " dst_min: %s dst_max: %s\n",
567 pkt_dev->dst_min, pkt_dev->dst_max); 567 pkt_dev->dst_min, pkt_dev->dst_max);
568 seq_printf(seq, 568 seq_printf(seq,
569 " src_min: %s src_max: %s\n", 569 " src_min: %s src_max: %s\n",
570 pkt_dev->src_min, pkt_dev->src_max); 570 pkt_dev->src_min, pkt_dev->src_max);
571 } 571 }
572 572
573 seq_puts(seq, " src_mac: "); 573 seq_puts(seq, " src_mac: ");
574 574
575 seq_printf(seq, "%pM ", 575 seq_printf(seq, "%pM ",
576 is_zero_ether_addr(pkt_dev->src_mac) ? 576 is_zero_ether_addr(pkt_dev->src_mac) ?
577 pkt_dev->odev->dev_addr : pkt_dev->src_mac); 577 pkt_dev->odev->dev_addr : pkt_dev->src_mac);
578 578
579 seq_printf(seq, "dst_mac: "); 579 seq_printf(seq, "dst_mac: ");
580 seq_printf(seq, "%pM\n", pkt_dev->dst_mac); 580 seq_printf(seq, "%pM\n", pkt_dev->dst_mac);
581 581
582 seq_printf(seq, 582 seq_printf(seq,
583 " udp_src_min: %d udp_src_max: %d" 583 " udp_src_min: %d udp_src_max: %d"
584 " udp_dst_min: %d udp_dst_max: %d\n", 584 " udp_dst_min: %d udp_dst_max: %d\n",
585 pkt_dev->udp_src_min, pkt_dev->udp_src_max, 585 pkt_dev->udp_src_min, pkt_dev->udp_src_max,
586 pkt_dev->udp_dst_min, pkt_dev->udp_dst_max); 586 pkt_dev->udp_dst_min, pkt_dev->udp_dst_max);
587 587
588 seq_printf(seq, 588 seq_printf(seq,
589 " src_mac_count: %d dst_mac_count: %d\n", 589 " src_mac_count: %d dst_mac_count: %d\n",
590 pkt_dev->src_mac_count, pkt_dev->dst_mac_count); 590 pkt_dev->src_mac_count, pkt_dev->dst_mac_count);
591 591
592 if (pkt_dev->nr_labels) { 592 if (pkt_dev->nr_labels) {
593 unsigned int i; 593 unsigned int i;
594 seq_printf(seq, " mpls: "); 594 seq_printf(seq, " mpls: ");
595 for (i = 0; i < pkt_dev->nr_labels; i++) 595 for (i = 0; i < pkt_dev->nr_labels; i++)
596 seq_printf(seq, "%08x%s", ntohl(pkt_dev->labels[i]), 596 seq_printf(seq, "%08x%s", ntohl(pkt_dev->labels[i]),
597 i == pkt_dev->nr_labels-1 ? "\n" : ", "); 597 i == pkt_dev->nr_labels-1 ? "\n" : ", ");
598 } 598 }
599 599
600 if (pkt_dev->vlan_id != 0xffff) 600 if (pkt_dev->vlan_id != 0xffff)
601 seq_printf(seq, " vlan_id: %u vlan_p: %u vlan_cfi: %u\n", 601 seq_printf(seq, " vlan_id: %u vlan_p: %u vlan_cfi: %u\n",
602 pkt_dev->vlan_id, pkt_dev->vlan_p, 602 pkt_dev->vlan_id, pkt_dev->vlan_p,
603 pkt_dev->vlan_cfi); 603 pkt_dev->vlan_cfi);
604 604
605 if (pkt_dev->svlan_id != 0xffff) 605 if (pkt_dev->svlan_id != 0xffff)
606 seq_printf(seq, " svlan_id: %u vlan_p: %u vlan_cfi: %u\n", 606 seq_printf(seq, " svlan_id: %u vlan_p: %u vlan_cfi: %u\n",
607 pkt_dev->svlan_id, pkt_dev->svlan_p, 607 pkt_dev->svlan_id, pkt_dev->svlan_p,
608 pkt_dev->svlan_cfi); 608 pkt_dev->svlan_cfi);
609 609
610 if (pkt_dev->tos) 610 if (pkt_dev->tos)
611 seq_printf(seq, " tos: 0x%02x\n", pkt_dev->tos); 611 seq_printf(seq, " tos: 0x%02x\n", pkt_dev->tos);
612 612
613 if (pkt_dev->traffic_class) 613 if (pkt_dev->traffic_class)
614 seq_printf(seq, " traffic_class: 0x%02x\n", pkt_dev->traffic_class); 614 seq_printf(seq, " traffic_class: 0x%02x\n", pkt_dev->traffic_class);
615 615
616 if (pkt_dev->node >= 0) 616 if (pkt_dev->node >= 0)
617 seq_printf(seq, " node: %d\n", pkt_dev->node); 617 seq_printf(seq, " node: %d\n", pkt_dev->node);
618 618
619 seq_printf(seq, " Flags: "); 619 seq_printf(seq, " Flags: ");
620 620
621 if (pkt_dev->flags & F_IPV6) 621 if (pkt_dev->flags & F_IPV6)
622 seq_printf(seq, "IPV6 "); 622 seq_printf(seq, "IPV6 ");
623 623
624 if (pkt_dev->flags & F_IPSRC_RND) 624 if (pkt_dev->flags & F_IPSRC_RND)
625 seq_printf(seq, "IPSRC_RND "); 625 seq_printf(seq, "IPSRC_RND ");
626 626
627 if (pkt_dev->flags & F_IPDST_RND) 627 if (pkt_dev->flags & F_IPDST_RND)
628 seq_printf(seq, "IPDST_RND "); 628 seq_printf(seq, "IPDST_RND ");
629 629
630 if (pkt_dev->flags & F_TXSIZE_RND) 630 if (pkt_dev->flags & F_TXSIZE_RND)
631 seq_printf(seq, "TXSIZE_RND "); 631 seq_printf(seq, "TXSIZE_RND ");
632 632
633 if (pkt_dev->flags & F_UDPSRC_RND) 633 if (pkt_dev->flags & F_UDPSRC_RND)
634 seq_printf(seq, "UDPSRC_RND "); 634 seq_printf(seq, "UDPSRC_RND ");
635 635
636 if (pkt_dev->flags & F_UDPDST_RND) 636 if (pkt_dev->flags & F_UDPDST_RND)
637 seq_printf(seq, "UDPDST_RND "); 637 seq_printf(seq, "UDPDST_RND ");
638 638
639 if (pkt_dev->flags & F_MPLS_RND) 639 if (pkt_dev->flags & F_MPLS_RND)
640 seq_printf(seq, "MPLS_RND "); 640 seq_printf(seq, "MPLS_RND ");
641 641
642 if (pkt_dev->flags & F_QUEUE_MAP_RND) 642 if (pkt_dev->flags & F_QUEUE_MAP_RND)
643 seq_printf(seq, "QUEUE_MAP_RND "); 643 seq_printf(seq, "QUEUE_MAP_RND ");
644 644
645 if (pkt_dev->flags & F_QUEUE_MAP_CPU) 645 if (pkt_dev->flags & F_QUEUE_MAP_CPU)
646 seq_printf(seq, "QUEUE_MAP_CPU "); 646 seq_printf(seq, "QUEUE_MAP_CPU ");
647 647
648 if (pkt_dev->cflows) { 648 if (pkt_dev->cflows) {
649 if (pkt_dev->flags & F_FLOW_SEQ) 649 if (pkt_dev->flags & F_FLOW_SEQ)
650 seq_printf(seq, "FLOW_SEQ "); /*in sequence flows*/ 650 seq_printf(seq, "FLOW_SEQ "); /*in sequence flows*/
651 else 651 else
652 seq_printf(seq, "FLOW_RND "); 652 seq_printf(seq, "FLOW_RND ");
653 } 653 }
654 654
655 #ifdef CONFIG_XFRM 655 #ifdef CONFIG_XFRM
656 if (pkt_dev->flags & F_IPSEC_ON) 656 if (pkt_dev->flags & F_IPSEC_ON)
657 seq_printf(seq, "IPSEC "); 657 seq_printf(seq, "IPSEC ");
658 #endif 658 #endif
659 659
660 if (pkt_dev->flags & F_MACSRC_RND) 660 if (pkt_dev->flags & F_MACSRC_RND)
661 seq_printf(seq, "MACSRC_RND "); 661 seq_printf(seq, "MACSRC_RND ");
662 662
663 if (pkt_dev->flags & F_MACDST_RND) 663 if (pkt_dev->flags & F_MACDST_RND)
664 seq_printf(seq, "MACDST_RND "); 664 seq_printf(seq, "MACDST_RND ");
665 665
666 if (pkt_dev->flags & F_VID_RND) 666 if (pkt_dev->flags & F_VID_RND)
667 seq_printf(seq, "VID_RND "); 667 seq_printf(seq, "VID_RND ");
668 668
669 if (pkt_dev->flags & F_SVID_RND) 669 if (pkt_dev->flags & F_SVID_RND)
670 seq_printf(seq, "SVID_RND "); 670 seq_printf(seq, "SVID_RND ");
671 671
672 if (pkt_dev->flags & F_NODE) 672 if (pkt_dev->flags & F_NODE)
673 seq_printf(seq, "NODE_ALLOC "); 673 seq_printf(seq, "NODE_ALLOC ");
674 674
675 seq_puts(seq, "\n"); 675 seq_puts(seq, "\n");
676 676
677 /* not really stopped, more like last-running-at */ 677 /* not really stopped, more like last-running-at */
678 stopped = pkt_dev->running ? ktime_now() : pkt_dev->stopped_at; 678 stopped = pkt_dev->running ? ktime_now() : pkt_dev->stopped_at;
679 idle = pkt_dev->idle_acc; 679 idle = pkt_dev->idle_acc;
680 do_div(idle, NSEC_PER_USEC); 680 do_div(idle, NSEC_PER_USEC);
681 681
682 seq_printf(seq, 682 seq_printf(seq,
683 "Current:\n pkts-sofar: %llu errors: %llu\n", 683 "Current:\n pkts-sofar: %llu errors: %llu\n",
684 (unsigned long long)pkt_dev->sofar, 684 (unsigned long long)pkt_dev->sofar,
685 (unsigned long long)pkt_dev->errors); 685 (unsigned long long)pkt_dev->errors);
686 686
687 seq_printf(seq, 687 seq_printf(seq,
688 " started: %lluus stopped: %lluus idle: %lluus\n", 688 " started: %lluus stopped: %lluus idle: %lluus\n",
689 (unsigned long long) ktime_to_us(pkt_dev->started_at), 689 (unsigned long long) ktime_to_us(pkt_dev->started_at),
690 (unsigned long long) ktime_to_us(stopped), 690 (unsigned long long) ktime_to_us(stopped),
691 (unsigned long long) idle); 691 (unsigned long long) idle);
692 692
693 seq_printf(seq, 693 seq_printf(seq,
694 " seq_num: %d cur_dst_mac_offset: %d cur_src_mac_offset: %d\n", 694 " seq_num: %d cur_dst_mac_offset: %d cur_src_mac_offset: %d\n",
695 pkt_dev->seq_num, pkt_dev->cur_dst_mac_offset, 695 pkt_dev->seq_num, pkt_dev->cur_dst_mac_offset,
696 pkt_dev->cur_src_mac_offset); 696 pkt_dev->cur_src_mac_offset);
697 697
698 if (pkt_dev->flags & F_IPV6) { 698 if (pkt_dev->flags & F_IPV6) {
699 seq_printf(seq, " cur_saddr: %pI6c cur_daddr: %pI6c\n", 699 seq_printf(seq, " cur_saddr: %pI6c cur_daddr: %pI6c\n",
700 &pkt_dev->cur_in6_saddr, 700 &pkt_dev->cur_in6_saddr,
701 &pkt_dev->cur_in6_daddr); 701 &pkt_dev->cur_in6_daddr);
702 } else 702 } else
703 seq_printf(seq, " cur_saddr: %pI4 cur_daddr: %pI4\n", 703 seq_printf(seq, " cur_saddr: %pI4 cur_daddr: %pI4\n",
704 &pkt_dev->cur_saddr, &pkt_dev->cur_daddr); 704 &pkt_dev->cur_saddr, &pkt_dev->cur_daddr);
705 705
706 seq_printf(seq, " cur_udp_dst: %d cur_udp_src: %d\n", 706 seq_printf(seq, " cur_udp_dst: %d cur_udp_src: %d\n",
707 pkt_dev->cur_udp_dst, pkt_dev->cur_udp_src); 707 pkt_dev->cur_udp_dst, pkt_dev->cur_udp_src);
708 708
709 seq_printf(seq, " cur_queue_map: %u\n", pkt_dev->cur_queue_map); 709 seq_printf(seq, " cur_queue_map: %u\n", pkt_dev->cur_queue_map);
710 710
711 seq_printf(seq, " flows: %u\n", pkt_dev->nflows); 711 seq_printf(seq, " flows: %u\n", pkt_dev->nflows);
712 712
713 if (pkt_dev->result[0]) 713 if (pkt_dev->result[0])
714 seq_printf(seq, "Result: %s\n", pkt_dev->result); 714 seq_printf(seq, "Result: %s\n", pkt_dev->result);
715 else 715 else
716 seq_printf(seq, "Result: Idle\n"); 716 seq_printf(seq, "Result: Idle\n");
717 717
718 return 0; 718 return 0;
719 } 719 }
720 720
721 721
722 static int hex32_arg(const char __user *user_buffer, unsigned long maxlen, 722 static int hex32_arg(const char __user *user_buffer, unsigned long maxlen,
723 __u32 *num) 723 __u32 *num)
724 { 724 {
725 int i = 0; 725 int i = 0;
726 *num = 0; 726 *num = 0;
727 727
728 for (; i < maxlen; i++) { 728 for (; i < maxlen; i++) {
729 int value; 729 int value;
730 char c; 730 char c;
731 *num <<= 4; 731 *num <<= 4;
732 if (get_user(c, &user_buffer[i])) 732 if (get_user(c, &user_buffer[i]))
733 return -EFAULT; 733 return -EFAULT;
734 value = hex_to_bin(c); 734 value = hex_to_bin(c);
735 if (value >= 0) 735 if (value >= 0)
736 *num |= value; 736 *num |= value;
737 else 737 else
738 break; 738 break;
739 } 739 }
740 return i; 740 return i;
741 } 741 }
742 742
743 static int count_trail_chars(const char __user * user_buffer, 743 static int count_trail_chars(const char __user * user_buffer,
744 unsigned int maxlen) 744 unsigned int maxlen)
745 { 745 {
746 int i; 746 int i;
747 747
748 for (i = 0; i < maxlen; i++) { 748 for (i = 0; i < maxlen; i++) {
749 char c; 749 char c;
750 if (get_user(c, &user_buffer[i])) 750 if (get_user(c, &user_buffer[i]))
751 return -EFAULT; 751 return -EFAULT;
752 switch (c) { 752 switch (c) {
753 case '\"': 753 case '\"':
754 case '\n': 754 case '\n':
755 case '\r': 755 case '\r':
756 case '\t': 756 case '\t':
757 case ' ': 757 case ' ':
758 case '=': 758 case '=':
759 break; 759 break;
760 default: 760 default:
761 goto done; 761 goto done;
762 } 762 }
763 } 763 }
764 done: 764 done:
765 return i; 765 return i;
766 } 766 }
767 767
768 static long num_arg(const char __user *user_buffer, unsigned long maxlen, 768 static long num_arg(const char __user *user_buffer, unsigned long maxlen,
769 unsigned long *num) 769 unsigned long *num)
770 { 770 {
771 int i; 771 int i;
772 *num = 0; 772 *num = 0;
773 773
774 for (i = 0; i < maxlen; i++) { 774 for (i = 0; i < maxlen; i++) {
775 char c; 775 char c;
776 if (get_user(c, &user_buffer[i])) 776 if (get_user(c, &user_buffer[i]))
777 return -EFAULT; 777 return -EFAULT;
778 if ((c >= '0') && (c <= '9')) { 778 if ((c >= '0') && (c <= '9')) {
779 *num *= 10; 779 *num *= 10;
780 *num += c - '0'; 780 *num += c - '0';
781 } else 781 } else
782 break; 782 break;
783 } 783 }
784 return i; 784 return i;
785 } 785 }
786 786
787 static int strn_len(const char __user * user_buffer, unsigned int maxlen) 787 static int strn_len(const char __user * user_buffer, unsigned int maxlen)
788 { 788 {
789 int i; 789 int i;
790 790
791 for (i = 0; i < maxlen; i++) { 791 for (i = 0; i < maxlen; i++) {
792 char c; 792 char c;
793 if (get_user(c, &user_buffer[i])) 793 if (get_user(c, &user_buffer[i]))
794 return -EFAULT; 794 return -EFAULT;
795 switch (c) { 795 switch (c) {
796 case '\"': 796 case '\"':
797 case '\n': 797 case '\n':
798 case '\r': 798 case '\r':
799 case '\t': 799 case '\t':
800 case ' ': 800 case ' ':
801 goto done_str; 801 goto done_str;
802 break; 802 break;
803 default: 803 default:
804 break; 804 break;
805 } 805 }
806 } 806 }
807 done_str: 807 done_str:
808 return i; 808 return i;
809 } 809 }
810 810
811 static ssize_t get_labels(const char __user *buffer, struct pktgen_dev *pkt_dev) 811 static ssize_t get_labels(const char __user *buffer, struct pktgen_dev *pkt_dev)
812 { 812 {
813 unsigned int n = 0; 813 unsigned int n = 0;
814 char c; 814 char c;
815 ssize_t i = 0; 815 ssize_t i = 0;
816 int len; 816 int len;
817 817
818 pkt_dev->nr_labels = 0; 818 pkt_dev->nr_labels = 0;
819 do { 819 do {
820 __u32 tmp; 820 __u32 tmp;
821 len = hex32_arg(&buffer[i], 8, &tmp); 821 len = hex32_arg(&buffer[i], 8, &tmp);
822 if (len <= 0) 822 if (len <= 0)
823 return len; 823 return len;
824 pkt_dev->labels[n] = htonl(tmp); 824 pkt_dev->labels[n] = htonl(tmp);
825 if (pkt_dev->labels[n] & MPLS_STACK_BOTTOM) 825 if (pkt_dev->labels[n] & MPLS_STACK_BOTTOM)
826 pkt_dev->flags |= F_MPLS_RND; 826 pkt_dev->flags |= F_MPLS_RND;
827 i += len; 827 i += len;
828 if (get_user(c, &buffer[i])) 828 if (get_user(c, &buffer[i]))
829 return -EFAULT; 829 return -EFAULT;
830 i++; 830 i++;
831 n++; 831 n++;
832 if (n >= MAX_MPLS_LABELS) 832 if (n >= MAX_MPLS_LABELS)
833 return -E2BIG; 833 return -E2BIG;
834 } while (c == ','); 834 } while (c == ',');
835 835
836 pkt_dev->nr_labels = n; 836 pkt_dev->nr_labels = n;
837 return i; 837 return i;
838 } 838 }
839 839
840 static ssize_t pktgen_if_write(struct file *file, 840 static ssize_t pktgen_if_write(struct file *file,
841 const char __user * user_buffer, size_t count, 841 const char __user * user_buffer, size_t count,
842 loff_t * offset) 842 loff_t * offset)
843 { 843 {
844 struct seq_file *seq = file->private_data; 844 struct seq_file *seq = file->private_data;
845 struct pktgen_dev *pkt_dev = seq->private; 845 struct pktgen_dev *pkt_dev = seq->private;
846 int i, max, len; 846 int i, max, len;
847 char name[16], valstr[32]; 847 char name[16], valstr[32];
848 unsigned long value = 0; 848 unsigned long value = 0;
849 char *pg_result = NULL; 849 char *pg_result = NULL;
850 int tmp = 0; 850 int tmp = 0;
851 char buf[128]; 851 char buf[128];
852 852
853 pg_result = &(pkt_dev->result[0]); 853 pg_result = &(pkt_dev->result[0]);
854 854
855 if (count < 1) { 855 if (count < 1) {
856 pr_warning("wrong command format\n"); 856 pr_warning("wrong command format\n");
857 return -EINVAL; 857 return -EINVAL;
858 } 858 }
859 859
860 max = count; 860 max = count;
861 tmp = count_trail_chars(user_buffer, max); 861 tmp = count_trail_chars(user_buffer, max);
862 if (tmp < 0) { 862 if (tmp < 0) {
863 pr_warning("illegal format\n"); 863 pr_warning("illegal format\n");
864 return tmp; 864 return tmp;
865 } 865 }
866 i = tmp; 866 i = tmp;
867 867
868 /* Read variable name */ 868 /* Read variable name */
869 869
870 len = strn_len(&user_buffer[i], sizeof(name) - 1); 870 len = strn_len(&user_buffer[i], sizeof(name) - 1);
871 if (len < 0) 871 if (len < 0)
872 return len; 872 return len;
873 873
874 memset(name, 0, sizeof(name)); 874 memset(name, 0, sizeof(name));
875 if (copy_from_user(name, &user_buffer[i], len)) 875 if (copy_from_user(name, &user_buffer[i], len))
876 return -EFAULT; 876 return -EFAULT;
877 i += len; 877 i += len;
878 878
879 max = count - i; 879 max = count - i;
880 len = count_trail_chars(&user_buffer[i], max); 880 len = count_trail_chars(&user_buffer[i], max);
881 if (len < 0) 881 if (len < 0)
882 return len; 882 return len;
883 883
884 i += len; 884 i += len;
885 885
886 if (debug) { 886 if (debug) {
887 size_t copy = min_t(size_t, count, 1023); 887 size_t copy = min_t(size_t, count, 1023);
888 char tb[copy + 1]; 888 char tb[copy + 1];
889 if (copy_from_user(tb, user_buffer, copy)) 889 if (copy_from_user(tb, user_buffer, copy))
890 return -EFAULT; 890 return -EFAULT;
891 tb[copy] = 0; 891 tb[copy] = 0;
892 pr_debug("%s,%lu buffer -:%s:-\n", 892 pr_debug("%s,%lu buffer -:%s:-\n",
893 name, (unsigned long)count, tb); 893 name, (unsigned long)count, tb);
894 } 894 }
895 895
896 if (!strcmp(name, "min_pkt_size")) { 896 if (!strcmp(name, "min_pkt_size")) {
897 len = num_arg(&user_buffer[i], 10, &value); 897 len = num_arg(&user_buffer[i], 10, &value);
898 if (len < 0) 898 if (len < 0)
899 return len; 899 return len;
900 900
901 i += len; 901 i += len;
902 if (value < 14 + 20 + 8) 902 if (value < 14 + 20 + 8)
903 value = 14 + 20 + 8; 903 value = 14 + 20 + 8;
904 if (value != pkt_dev->min_pkt_size) { 904 if (value != pkt_dev->min_pkt_size) {
905 pkt_dev->min_pkt_size = value; 905 pkt_dev->min_pkt_size = value;
906 pkt_dev->cur_pkt_size = value; 906 pkt_dev->cur_pkt_size = value;
907 } 907 }
908 sprintf(pg_result, "OK: min_pkt_size=%u", 908 sprintf(pg_result, "OK: min_pkt_size=%u",
909 pkt_dev->min_pkt_size); 909 pkt_dev->min_pkt_size);
910 return count; 910 return count;
911 } 911 }
912 912
913 if (!strcmp(name, "max_pkt_size")) { 913 if (!strcmp(name, "max_pkt_size")) {
914 len = num_arg(&user_buffer[i], 10, &value); 914 len = num_arg(&user_buffer[i], 10, &value);
915 if (len < 0) 915 if (len < 0)
916 return len; 916 return len;
917 917
918 i += len; 918 i += len;
919 if (value < 14 + 20 + 8) 919 if (value < 14 + 20 + 8)
920 value = 14 + 20 + 8; 920 value = 14 + 20 + 8;
921 if (value != pkt_dev->max_pkt_size) { 921 if (value != pkt_dev->max_pkt_size) {
922 pkt_dev->max_pkt_size = value; 922 pkt_dev->max_pkt_size = value;
923 pkt_dev->cur_pkt_size = value; 923 pkt_dev->cur_pkt_size = value;
924 } 924 }
925 sprintf(pg_result, "OK: max_pkt_size=%u", 925 sprintf(pg_result, "OK: max_pkt_size=%u",
926 pkt_dev->max_pkt_size); 926 pkt_dev->max_pkt_size);
927 return count; 927 return count;
928 } 928 }
929 929
930 /* Shortcut for min = max */ 930 /* Shortcut for min = max */
931 931
932 if (!strcmp(name, "pkt_size")) { 932 if (!strcmp(name, "pkt_size")) {
933 len = num_arg(&user_buffer[i], 10, &value); 933 len = num_arg(&user_buffer[i], 10, &value);
934 if (len < 0) 934 if (len < 0)
935 return len; 935 return len;
936 936
937 i += len; 937 i += len;
938 if (value < 14 + 20 + 8) 938 if (value < 14 + 20 + 8)
939 value = 14 + 20 + 8; 939 value = 14 + 20 + 8;
940 if (value != pkt_dev->min_pkt_size) { 940 if (value != pkt_dev->min_pkt_size) {
941 pkt_dev->min_pkt_size = value; 941 pkt_dev->min_pkt_size = value;
942 pkt_dev->max_pkt_size = value; 942 pkt_dev->max_pkt_size = value;
943 pkt_dev->cur_pkt_size = value; 943 pkt_dev->cur_pkt_size = value;
944 } 944 }
945 sprintf(pg_result, "OK: pkt_size=%u", pkt_dev->min_pkt_size); 945 sprintf(pg_result, "OK: pkt_size=%u", pkt_dev->min_pkt_size);
946 return count; 946 return count;
947 } 947 }
948 948
949 if (!strcmp(name, "debug")) { 949 if (!strcmp(name, "debug")) {
950 len = num_arg(&user_buffer[i], 10, &value); 950 len = num_arg(&user_buffer[i], 10, &value);
951 if (len < 0) 951 if (len < 0)
952 return len; 952 return len;
953 953
954 i += len; 954 i += len;
955 debug = value; 955 debug = value;
956 sprintf(pg_result, "OK: debug=%u", debug); 956 sprintf(pg_result, "OK: debug=%u", debug);
957 return count; 957 return count;
958 } 958 }
959 959
960 if (!strcmp(name, "frags")) { 960 if (!strcmp(name, "frags")) {
961 len = num_arg(&user_buffer[i], 10, &value); 961 len = num_arg(&user_buffer[i], 10, &value);
962 if (len < 0) 962 if (len < 0)
963 return len; 963 return len;
964 964
965 i += len; 965 i += len;
966 pkt_dev->nfrags = value; 966 pkt_dev->nfrags = value;
967 sprintf(pg_result, "OK: frags=%u", pkt_dev->nfrags); 967 sprintf(pg_result, "OK: frags=%u", pkt_dev->nfrags);
968 return count; 968 return count;
969 } 969 }
970 if (!strcmp(name, "delay")) { 970 if (!strcmp(name, "delay")) {
971 len = num_arg(&user_buffer[i], 10, &value); 971 len = num_arg(&user_buffer[i], 10, &value);
972 if (len < 0) 972 if (len < 0)
973 return len; 973 return len;
974 974
975 i += len; 975 i += len;
976 if (value == 0x7FFFFFFF) 976 if (value == 0x7FFFFFFF)
977 pkt_dev->delay = ULLONG_MAX; 977 pkt_dev->delay = ULLONG_MAX;
978 else 978 else
979 pkt_dev->delay = (u64)value; 979 pkt_dev->delay = (u64)value;
980 980
981 sprintf(pg_result, "OK: delay=%llu", 981 sprintf(pg_result, "OK: delay=%llu",
982 (unsigned long long) pkt_dev->delay); 982 (unsigned long long) pkt_dev->delay);
983 return count; 983 return count;
984 } 984 }
985 if (!strcmp(name, "rate")) { 985 if (!strcmp(name, "rate")) {
986 len = num_arg(&user_buffer[i], 10, &value); 986 len = num_arg(&user_buffer[i], 10, &value);
987 if (len < 0) 987 if (len < 0)
988 return len; 988 return len;
989 989
990 i += len; 990 i += len;
991 if (!value) 991 if (!value)
992 return len; 992 return len;
993 pkt_dev->delay = pkt_dev->min_pkt_size*8*NSEC_PER_USEC/value; 993 pkt_dev->delay = pkt_dev->min_pkt_size*8*NSEC_PER_USEC/value;
994 if (debug) 994 if (debug)
995 pr_info("Delay set at: %llu ns\n", pkt_dev->delay); 995 pr_info("Delay set at: %llu ns\n", pkt_dev->delay);
996 996
997 sprintf(pg_result, "OK: rate=%lu", value); 997 sprintf(pg_result, "OK: rate=%lu", value);
998 return count; 998 return count;
999 } 999 }
1000 if (!strcmp(name, "ratep")) { 1000 if (!strcmp(name, "ratep")) {
1001 len = num_arg(&user_buffer[i], 10, &value); 1001 len = num_arg(&user_buffer[i], 10, &value);
1002 if (len < 0) 1002 if (len < 0)
1003 return len; 1003 return len;
1004 1004
1005 i += len; 1005 i += len;
1006 if (!value) 1006 if (!value)
1007 return len; 1007 return len;
1008 pkt_dev->delay = NSEC_PER_SEC/value; 1008 pkt_dev->delay = NSEC_PER_SEC/value;
1009 if (debug) 1009 if (debug)
1010 pr_info("Delay set at: %llu ns\n", pkt_dev->delay); 1010 pr_info("Delay set at: %llu ns\n", pkt_dev->delay);
1011 1011
1012 sprintf(pg_result, "OK: rate=%lu", value); 1012 sprintf(pg_result, "OK: rate=%lu", value);
1013 return count; 1013 return count;
1014 } 1014 }
1015 if (!strcmp(name, "udp_src_min")) { 1015 if (!strcmp(name, "udp_src_min")) {
1016 len = num_arg(&user_buffer[i], 10, &value); 1016 len = num_arg(&user_buffer[i], 10, &value);
1017 if (len < 0) 1017 if (len < 0)
1018 return len; 1018 return len;
1019 1019
1020 i += len; 1020 i += len;
1021 if (value != pkt_dev->udp_src_min) { 1021 if (value != pkt_dev->udp_src_min) {
1022 pkt_dev->udp_src_min = value; 1022 pkt_dev->udp_src_min = value;
1023 pkt_dev->cur_udp_src = value; 1023 pkt_dev->cur_udp_src = value;
1024 } 1024 }
1025 sprintf(pg_result, "OK: udp_src_min=%u", pkt_dev->udp_src_min); 1025 sprintf(pg_result, "OK: udp_src_min=%u", pkt_dev->udp_src_min);
1026 return count; 1026 return count;
1027 } 1027 }
1028 if (!strcmp(name, "udp_dst_min")) { 1028 if (!strcmp(name, "udp_dst_min")) {
1029 len = num_arg(&user_buffer[i], 10, &value); 1029 len = num_arg(&user_buffer[i], 10, &value);
1030 if (len < 0) 1030 if (len < 0)
1031 return len; 1031 return len;
1032 1032
1033 i += len; 1033 i += len;
1034 if (value != pkt_dev->udp_dst_min) { 1034 if (value != pkt_dev->udp_dst_min) {
1035 pkt_dev->udp_dst_min = value; 1035 pkt_dev->udp_dst_min = value;
1036 pkt_dev->cur_udp_dst = value; 1036 pkt_dev->cur_udp_dst = value;
1037 } 1037 }
1038 sprintf(pg_result, "OK: udp_dst_min=%u", pkt_dev->udp_dst_min); 1038 sprintf(pg_result, "OK: udp_dst_min=%u", pkt_dev->udp_dst_min);
1039 return count; 1039 return count;
1040 } 1040 }
1041 if (!strcmp(name, "udp_src_max")) { 1041 if (!strcmp(name, "udp_src_max")) {
1042 len = num_arg(&user_buffer[i], 10, &value); 1042 len = num_arg(&user_buffer[i], 10, &value);
1043 if (len < 0) 1043 if (len < 0)
1044 return len; 1044 return len;
1045 1045
1046 i += len; 1046 i += len;
1047 if (value != pkt_dev->udp_src_max) { 1047 if (value != pkt_dev->udp_src_max) {
1048 pkt_dev->udp_src_max = value; 1048 pkt_dev->udp_src_max = value;
1049 pkt_dev->cur_udp_src = value; 1049 pkt_dev->cur_udp_src = value;
1050 } 1050 }
1051 sprintf(pg_result, "OK: udp_src_max=%u", pkt_dev->udp_src_max); 1051 sprintf(pg_result, "OK: udp_src_max=%u", pkt_dev->udp_src_max);
1052 return count; 1052 return count;
1053 } 1053 }
1054 if (!strcmp(name, "udp_dst_max")) { 1054 if (!strcmp(name, "udp_dst_max")) {
1055 len = num_arg(&user_buffer[i], 10, &value); 1055 len = num_arg(&user_buffer[i], 10, &value);
1056 if (len < 0) 1056 if (len < 0)
1057 return len; 1057 return len;
1058 1058
1059 i += len; 1059 i += len;
1060 if (value != pkt_dev->udp_dst_max) { 1060 if (value != pkt_dev->udp_dst_max) {
1061 pkt_dev->udp_dst_max = value; 1061 pkt_dev->udp_dst_max = value;
1062 pkt_dev->cur_udp_dst = value; 1062 pkt_dev->cur_udp_dst = value;
1063 } 1063 }
1064 sprintf(pg_result, "OK: udp_dst_max=%u", pkt_dev->udp_dst_max); 1064 sprintf(pg_result, "OK: udp_dst_max=%u", pkt_dev->udp_dst_max);
1065 return count; 1065 return count;
1066 } 1066 }
1067 if (!strcmp(name, "clone_skb")) { 1067 if (!strcmp(name, "clone_skb")) {
1068 len = num_arg(&user_buffer[i], 10, &value); 1068 len = num_arg(&user_buffer[i], 10, &value);
1069 if (len < 0) 1069 if (len < 0)
1070 return len; 1070 return len;
1071 if ((value > 0) && 1071 if ((value > 0) &&
1072 (!(pkt_dev->odev->priv_flags & IFF_TX_SKB_SHARING))) 1072 (!(pkt_dev->odev->priv_flags & IFF_TX_SKB_SHARING)))
1073 return -ENOTSUPP; 1073 return -ENOTSUPP;
1074 i += len; 1074 i += len;
1075 pkt_dev->clone_skb = value; 1075 pkt_dev->clone_skb = value;
1076 1076
1077 sprintf(pg_result, "OK: clone_skb=%d", pkt_dev->clone_skb); 1077 sprintf(pg_result, "OK: clone_skb=%d", pkt_dev->clone_skb);
1078 return count; 1078 return count;
1079 } 1079 }
1080 if (!strcmp(name, "count")) { 1080 if (!strcmp(name, "count")) {
1081 len = num_arg(&user_buffer[i], 10, &value); 1081 len = num_arg(&user_buffer[i], 10, &value);
1082 if (len < 0) 1082 if (len < 0)
1083 return len; 1083 return len;
1084 1084
1085 i += len; 1085 i += len;
1086 pkt_dev->count = value; 1086 pkt_dev->count = value;
1087 sprintf(pg_result, "OK: count=%llu", 1087 sprintf(pg_result, "OK: count=%llu",
1088 (unsigned long long)pkt_dev->count); 1088 (unsigned long long)pkt_dev->count);
1089 return count; 1089 return count;
1090 } 1090 }
1091 if (!strcmp(name, "src_mac_count")) { 1091 if (!strcmp(name, "src_mac_count")) {
1092 len = num_arg(&user_buffer[i], 10, &value); 1092 len = num_arg(&user_buffer[i], 10, &value);
1093 if (len < 0) 1093 if (len < 0)
1094 return len; 1094 return len;
1095 1095
1096 i += len; 1096 i += len;
1097 if (pkt_dev->src_mac_count != value) { 1097 if (pkt_dev->src_mac_count != value) {
1098 pkt_dev->src_mac_count = value; 1098 pkt_dev->src_mac_count = value;
1099 pkt_dev->cur_src_mac_offset = 0; 1099 pkt_dev->cur_src_mac_offset = 0;
1100 } 1100 }
1101 sprintf(pg_result, "OK: src_mac_count=%d", 1101 sprintf(pg_result, "OK: src_mac_count=%d",
1102 pkt_dev->src_mac_count); 1102 pkt_dev->src_mac_count);
1103 return count; 1103 return count;
1104 } 1104 }
1105 if (!strcmp(name, "dst_mac_count")) { 1105 if (!strcmp(name, "dst_mac_count")) {
1106 len = num_arg(&user_buffer[i], 10, &value); 1106 len = num_arg(&user_buffer[i], 10, &value);
1107 if (len < 0) 1107 if (len < 0)
1108 return len; 1108 return len;
1109 1109
1110 i += len; 1110 i += len;
1111 if (pkt_dev->dst_mac_count != value) { 1111 if (pkt_dev->dst_mac_count != value) {
1112 pkt_dev->dst_mac_count = value; 1112 pkt_dev->dst_mac_count = value;
1113 pkt_dev->cur_dst_mac_offset = 0; 1113 pkt_dev->cur_dst_mac_offset = 0;
1114 } 1114 }
1115 sprintf(pg_result, "OK: dst_mac_count=%d", 1115 sprintf(pg_result, "OK: dst_mac_count=%d",
1116 pkt_dev->dst_mac_count); 1116 pkt_dev->dst_mac_count);
1117 return count; 1117 return count;
1118 } 1118 }
1119 if (!strcmp(name, "node")) { 1119 if (!strcmp(name, "node")) {
1120 len = num_arg(&user_buffer[i], 10, &value); 1120 len = num_arg(&user_buffer[i], 10, &value);
1121 if (len < 0) 1121 if (len < 0)
1122 return len; 1122 return len;
1123 1123
1124 i += len; 1124 i += len;
1125 1125
1126 if (node_possible(value)) { 1126 if (node_possible(value)) {
1127 pkt_dev->node = value; 1127 pkt_dev->node = value;
1128 sprintf(pg_result, "OK: node=%d", pkt_dev->node); 1128 sprintf(pg_result, "OK: node=%d", pkt_dev->node);
1129 if (pkt_dev->page) { 1129 if (pkt_dev->page) {
1130 put_page(pkt_dev->page); 1130 put_page(pkt_dev->page);
1131 pkt_dev->page = NULL; 1131 pkt_dev->page = NULL;
1132 } 1132 }
1133 } 1133 }
1134 else 1134 else
1135 sprintf(pg_result, "ERROR: node not possible"); 1135 sprintf(pg_result, "ERROR: node not possible");
1136 return count; 1136 return count;
1137 } 1137 }
1138 if (!strcmp(name, "flag")) { 1138 if (!strcmp(name, "flag")) {
1139 char f[32]; 1139 char f[32];
1140 memset(f, 0, 32); 1140 memset(f, 0, 32);
1141 len = strn_len(&user_buffer[i], sizeof(f) - 1); 1141 len = strn_len(&user_buffer[i], sizeof(f) - 1);
1142 if (len < 0) 1142 if (len < 0)
1143 return len; 1143 return len;
1144 1144
1145 if (copy_from_user(f, &user_buffer[i], len)) 1145 if (copy_from_user(f, &user_buffer[i], len))
1146 return -EFAULT; 1146 return -EFAULT;
1147 i += len; 1147 i += len;
1148 if (strcmp(f, "IPSRC_RND") == 0) 1148 if (strcmp(f, "IPSRC_RND") == 0)
1149 pkt_dev->flags |= F_IPSRC_RND; 1149 pkt_dev->flags |= F_IPSRC_RND;
1150 1150
1151 else if (strcmp(f, "!IPSRC_RND") == 0) 1151 else if (strcmp(f, "!IPSRC_RND") == 0)
1152 pkt_dev->flags &= ~F_IPSRC_RND; 1152 pkt_dev->flags &= ~F_IPSRC_RND;
1153 1153
1154 else if (strcmp(f, "TXSIZE_RND") == 0) 1154 else if (strcmp(f, "TXSIZE_RND") == 0)
1155 pkt_dev->flags |= F_TXSIZE_RND; 1155 pkt_dev->flags |= F_TXSIZE_RND;
1156 1156
1157 else if (strcmp(f, "!TXSIZE_RND") == 0) 1157 else if (strcmp(f, "!TXSIZE_RND") == 0)
1158 pkt_dev->flags &= ~F_TXSIZE_RND; 1158 pkt_dev->flags &= ~F_TXSIZE_RND;
1159 1159
1160 else if (strcmp(f, "IPDST_RND") == 0) 1160 else if (strcmp(f, "IPDST_RND") == 0)
1161 pkt_dev->flags |= F_IPDST_RND; 1161 pkt_dev->flags |= F_IPDST_RND;
1162 1162
1163 else if (strcmp(f, "!IPDST_RND") == 0) 1163 else if (strcmp(f, "!IPDST_RND") == 0)
1164 pkt_dev->flags &= ~F_IPDST_RND; 1164 pkt_dev->flags &= ~F_IPDST_RND;
1165 1165
1166 else if (strcmp(f, "UDPSRC_RND") == 0) 1166 else if (strcmp(f, "UDPSRC_RND") == 0)
1167 pkt_dev->flags |= F_UDPSRC_RND; 1167 pkt_dev->flags |= F_UDPSRC_RND;
1168 1168
1169 else if (strcmp(f, "!UDPSRC_RND") == 0) 1169 else if (strcmp(f, "!UDPSRC_RND") == 0)
1170 pkt_dev->flags &= ~F_UDPSRC_RND; 1170 pkt_dev->flags &= ~F_UDPSRC_RND;
1171 1171
1172 else if (strcmp(f, "UDPDST_RND") == 0) 1172 else if (strcmp(f, "UDPDST_RND") == 0)
1173 pkt_dev->flags |= F_UDPDST_RND; 1173 pkt_dev->flags |= F_UDPDST_RND;
1174 1174
1175 else if (strcmp(f, "!UDPDST_RND") == 0) 1175 else if (strcmp(f, "!UDPDST_RND") == 0)
1176 pkt_dev->flags &= ~F_UDPDST_RND; 1176 pkt_dev->flags &= ~F_UDPDST_RND;
1177 1177
1178 else if (strcmp(f, "MACSRC_RND") == 0) 1178 else if (strcmp(f, "MACSRC_RND") == 0)
1179 pkt_dev->flags |= F_MACSRC_RND; 1179 pkt_dev->flags |= F_MACSRC_RND;
1180 1180
1181 else if (strcmp(f, "!MACSRC_RND") == 0) 1181 else if (strcmp(f, "!MACSRC_RND") == 0)
1182 pkt_dev->flags &= ~F_MACSRC_RND; 1182 pkt_dev->flags &= ~F_MACSRC_RND;
1183 1183
1184 else if (strcmp(f, "MACDST_RND") == 0) 1184 else if (strcmp(f, "MACDST_RND") == 0)
1185 pkt_dev->flags |= F_MACDST_RND; 1185 pkt_dev->flags |= F_MACDST_RND;
1186 1186
1187 else if (strcmp(f, "!MACDST_RND") == 0) 1187 else if (strcmp(f, "!MACDST_RND") == 0)
1188 pkt_dev->flags &= ~F_MACDST_RND; 1188 pkt_dev->flags &= ~F_MACDST_RND;
1189 1189
1190 else if (strcmp(f, "MPLS_RND") == 0) 1190 else if (strcmp(f, "MPLS_RND") == 0)
1191 pkt_dev->flags |= F_MPLS_RND; 1191 pkt_dev->flags |= F_MPLS_RND;
1192 1192
1193 else if (strcmp(f, "!MPLS_RND") == 0) 1193 else if (strcmp(f, "!MPLS_RND") == 0)
1194 pkt_dev->flags &= ~F_MPLS_RND; 1194 pkt_dev->flags &= ~F_MPLS_RND;
1195 1195
1196 else if (strcmp(f, "VID_RND") == 0) 1196 else if (strcmp(f, "VID_RND") == 0)
1197 pkt_dev->flags |= F_VID_RND; 1197 pkt_dev->flags |= F_VID_RND;
1198 1198
1199 else if (strcmp(f, "!VID_RND") == 0) 1199 else if (strcmp(f, "!VID_RND") == 0)
1200 pkt_dev->flags &= ~F_VID_RND; 1200 pkt_dev->flags &= ~F_VID_RND;
1201 1201
1202 else if (strcmp(f, "SVID_RND") == 0) 1202 else if (strcmp(f, "SVID_RND") == 0)
1203 pkt_dev->flags |= F_SVID_RND; 1203 pkt_dev->flags |= F_SVID_RND;
1204 1204
1205 else if (strcmp(f, "!SVID_RND") == 0) 1205 else if (strcmp(f, "!SVID_RND") == 0)
1206 pkt_dev->flags &= ~F_SVID_RND; 1206 pkt_dev->flags &= ~F_SVID_RND;
1207 1207
1208 else if (strcmp(f, "FLOW_SEQ") == 0) 1208 else if (strcmp(f, "FLOW_SEQ") == 0)
1209 pkt_dev->flags |= F_FLOW_SEQ; 1209 pkt_dev->flags |= F_FLOW_SEQ;
1210 1210
1211 else if (strcmp(f, "QUEUE_MAP_RND") == 0) 1211 else if (strcmp(f, "QUEUE_MAP_RND") == 0)
1212 pkt_dev->flags |= F_QUEUE_MAP_RND; 1212 pkt_dev->flags |= F_QUEUE_MAP_RND;
1213 1213
1214 else if (strcmp(f, "!QUEUE_MAP_RND") == 0) 1214 else if (strcmp(f, "!QUEUE_MAP_RND") == 0)
1215 pkt_dev->flags &= ~F_QUEUE_MAP_RND; 1215 pkt_dev->flags &= ~F_QUEUE_MAP_RND;
1216 1216
1217 else if (strcmp(f, "QUEUE_MAP_CPU") == 0) 1217 else if (strcmp(f, "QUEUE_MAP_CPU") == 0)
1218 pkt_dev->flags |= F_QUEUE_MAP_CPU; 1218 pkt_dev->flags |= F_QUEUE_MAP_CPU;
1219 1219
1220 else if (strcmp(f, "!QUEUE_MAP_CPU") == 0) 1220 else if (strcmp(f, "!QUEUE_MAP_CPU") == 0)
1221 pkt_dev->flags &= ~F_QUEUE_MAP_CPU; 1221 pkt_dev->flags &= ~F_QUEUE_MAP_CPU;
1222 #ifdef CONFIG_XFRM 1222 #ifdef CONFIG_XFRM
1223 else if (strcmp(f, "IPSEC") == 0) 1223 else if (strcmp(f, "IPSEC") == 0)
1224 pkt_dev->flags |= F_IPSEC_ON; 1224 pkt_dev->flags |= F_IPSEC_ON;
1225 #endif 1225 #endif
1226 1226
1227 else if (strcmp(f, "!IPV6") == 0) 1227 else if (strcmp(f, "!IPV6") == 0)
1228 pkt_dev->flags &= ~F_IPV6; 1228 pkt_dev->flags &= ~F_IPV6;
1229 1229
1230 else if (strcmp(f, "NODE_ALLOC") == 0) 1230 else if (strcmp(f, "NODE_ALLOC") == 0)
1231 pkt_dev->flags |= F_NODE; 1231 pkt_dev->flags |= F_NODE;
1232 1232
1233 else if (strcmp(f, "!NODE_ALLOC") == 0) 1233 else if (strcmp(f, "!NODE_ALLOC") == 0)
1234 pkt_dev->flags &= ~F_NODE; 1234 pkt_dev->flags &= ~F_NODE;
1235 1235
1236 else { 1236 else {
1237 sprintf(pg_result, 1237 sprintf(pg_result,
1238 "Flag -:%s:- unknown\nAvailable flags, (prepend ! to un-set flag):\n%s", 1238 "Flag -:%s:- unknown\nAvailable flags, (prepend ! to un-set flag):\n%s",
1239 f, 1239 f,
1240 "IPSRC_RND, IPDST_RND, UDPSRC_RND, UDPDST_RND, " 1240 "IPSRC_RND, IPDST_RND, UDPSRC_RND, UDPDST_RND, "
1241 "MACSRC_RND, MACDST_RND, TXSIZE_RND, IPV6, MPLS_RND, VID_RND, SVID_RND, FLOW_SEQ, IPSEC, NODE_ALLOC\n"); 1241 "MACSRC_RND, MACDST_RND, TXSIZE_RND, IPV6, MPLS_RND, VID_RND, SVID_RND, FLOW_SEQ, IPSEC, NODE_ALLOC\n");
1242 return count; 1242 return count;
1243 } 1243 }
1244 sprintf(pg_result, "OK: flags=0x%x", pkt_dev->flags); 1244 sprintf(pg_result, "OK: flags=0x%x", pkt_dev->flags);
1245 return count; 1245 return count;
1246 } 1246 }
1247 if (!strcmp(name, "dst_min") || !strcmp(name, "dst")) { 1247 if (!strcmp(name, "dst_min") || !strcmp(name, "dst")) {
1248 len = strn_len(&user_buffer[i], sizeof(pkt_dev->dst_min) - 1); 1248 len = strn_len(&user_buffer[i], sizeof(pkt_dev->dst_min) - 1);
1249 if (len < 0) 1249 if (len < 0)
1250 return len; 1250 return len;
1251 1251
1252 if (copy_from_user(buf, &user_buffer[i], len)) 1252 if (copy_from_user(buf, &user_buffer[i], len))
1253 return -EFAULT; 1253 return -EFAULT;
1254 buf[len] = 0; 1254 buf[len] = 0;
1255 if (strcmp(buf, pkt_dev->dst_min) != 0) { 1255 if (strcmp(buf, pkt_dev->dst_min) != 0) {
1256 memset(pkt_dev->dst_min, 0, sizeof(pkt_dev->dst_min)); 1256 memset(pkt_dev->dst_min, 0, sizeof(pkt_dev->dst_min));
1257 strncpy(pkt_dev->dst_min, buf, len); 1257 strncpy(pkt_dev->dst_min, buf, len);
1258 pkt_dev->daddr_min = in_aton(pkt_dev->dst_min); 1258 pkt_dev->daddr_min = in_aton(pkt_dev->dst_min);
1259 pkt_dev->cur_daddr = pkt_dev->daddr_min; 1259 pkt_dev->cur_daddr = pkt_dev->daddr_min;
1260 } 1260 }
1261 if (debug) 1261 if (debug)
1262 pr_debug("dst_min set to: %s\n", pkt_dev->dst_min); 1262 pr_debug("dst_min set to: %s\n", pkt_dev->dst_min);
1263 i += len; 1263 i += len;
1264 sprintf(pg_result, "OK: dst_min=%s", pkt_dev->dst_min); 1264 sprintf(pg_result, "OK: dst_min=%s", pkt_dev->dst_min);
1265 return count; 1265 return count;
1266 } 1266 }
1267 if (!strcmp(name, "dst_max")) { 1267 if (!strcmp(name, "dst_max")) {
1268 len = strn_len(&user_buffer[i], sizeof(pkt_dev->dst_max) - 1); 1268 len = strn_len(&user_buffer[i], sizeof(pkt_dev->dst_max) - 1);
1269 if (len < 0) 1269 if (len < 0)
1270 return len; 1270 return len;
1271 1271
1272 1272
1273 if (copy_from_user(buf, &user_buffer[i], len)) 1273 if (copy_from_user(buf, &user_buffer[i], len))
1274 return -EFAULT; 1274 return -EFAULT;
1275 1275
1276 buf[len] = 0; 1276 buf[len] = 0;
1277 if (strcmp(buf, pkt_dev->dst_max) != 0) { 1277 if (strcmp(buf, pkt_dev->dst_max) != 0) {
1278 memset(pkt_dev->dst_max, 0, sizeof(pkt_dev->dst_max)); 1278 memset(pkt_dev->dst_max, 0, sizeof(pkt_dev->dst_max));
1279 strncpy(pkt_dev->dst_max, buf, len); 1279 strncpy(pkt_dev->dst_max, buf, len);
1280 pkt_dev->daddr_max = in_aton(pkt_dev->dst_max); 1280 pkt_dev->daddr_max = in_aton(pkt_dev->dst_max);
1281 pkt_dev->cur_daddr = pkt_dev->daddr_max; 1281 pkt_dev->cur_daddr = pkt_dev->daddr_max;
1282 } 1282 }
1283 if (debug) 1283 if (debug)
1284 pr_debug("dst_max set to: %s\n", pkt_dev->dst_max); 1284 pr_debug("dst_max set to: %s\n", pkt_dev->dst_max);
1285 i += len; 1285 i += len;
1286 sprintf(pg_result, "OK: dst_max=%s", pkt_dev->dst_max); 1286 sprintf(pg_result, "OK: dst_max=%s", pkt_dev->dst_max);
1287 return count; 1287 return count;
1288 } 1288 }
1289 if (!strcmp(name, "dst6")) { 1289 if (!strcmp(name, "dst6")) {
1290 len = strn_len(&user_buffer[i], sizeof(buf) - 1); 1290 len = strn_len(&user_buffer[i], sizeof(buf) - 1);
1291 if (len < 0) 1291 if (len < 0)
1292 return len; 1292 return len;
1293 1293
1294 pkt_dev->flags |= F_IPV6; 1294 pkt_dev->flags |= F_IPV6;
1295 1295
1296 if (copy_from_user(buf, &user_buffer[i], len)) 1296 if (copy_from_user(buf, &user_buffer[i], len))
1297 return -EFAULT; 1297 return -EFAULT;
1298 buf[len] = 0; 1298 buf[len] = 0;
1299 1299
1300 in6_pton(buf, -1, pkt_dev->in6_daddr.s6_addr, -1, NULL); 1300 in6_pton(buf, -1, pkt_dev->in6_daddr.s6_addr, -1, NULL);
1301 snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->in6_daddr); 1301 snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->in6_daddr);
1302 1302
1303 pkt_dev->cur_in6_daddr = pkt_dev->in6_daddr; 1303 pkt_dev->cur_in6_daddr = pkt_dev->in6_daddr;
1304 1304
1305 if (debug) 1305 if (debug)
1306 pr_debug("dst6 set to: %s\n", buf); 1306 pr_debug("dst6 set to: %s\n", buf);
1307 1307
1308 i += len; 1308 i += len;
1309 sprintf(pg_result, "OK: dst6=%s", buf); 1309 sprintf(pg_result, "OK: dst6=%s", buf);
1310 return count; 1310 return count;
1311 } 1311 }
1312 if (!strcmp(name, "dst6_min")) { 1312 if (!strcmp(name, "dst6_min")) {
1313 len = strn_len(&user_buffer[i], sizeof(buf) - 1); 1313 len = strn_len(&user_buffer[i], sizeof(buf) - 1);
1314 if (len < 0) 1314 if (len < 0)
1315 return len; 1315 return len;
1316 1316
1317 pkt_dev->flags |= F_IPV6; 1317 pkt_dev->flags |= F_IPV6;
1318 1318
1319 if (copy_from_user(buf, &user_buffer[i], len)) 1319 if (copy_from_user(buf, &user_buffer[i], len))
1320 return -EFAULT; 1320 return -EFAULT;
1321 buf[len] = 0; 1321 buf[len] = 0;
1322 1322
1323 in6_pton(buf, -1, pkt_dev->min_in6_daddr.s6_addr, -1, NULL); 1323 in6_pton(buf, -1, pkt_dev->min_in6_daddr.s6_addr, -1, NULL);
1324 snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->min_in6_daddr); 1324 snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->min_in6_daddr);
1325 1325
1326 pkt_dev->cur_in6_daddr = pkt_dev->min_in6_daddr; 1326 pkt_dev->cur_in6_daddr = pkt_dev->min_in6_daddr;
1327 if (debug) 1327 if (debug)
1328 pr_debug("dst6_min set to: %s\n", buf); 1328 pr_debug("dst6_min set to: %s\n", buf);
1329 1329
1330 i += len; 1330 i += len;
1331 sprintf(pg_result, "OK: dst6_min=%s", buf); 1331 sprintf(pg_result, "OK: dst6_min=%s", buf);
1332 return count; 1332 return count;
1333 } 1333 }
1334 if (!strcmp(name, "dst6_max")) { 1334 if (!strcmp(name, "dst6_max")) {
1335 len = strn_len(&user_buffer[i], sizeof(buf) - 1); 1335 len = strn_len(&user_buffer[i], sizeof(buf) - 1);
1336 if (len < 0) 1336 if (len < 0)
1337 return len; 1337 return len;
1338 1338
1339 pkt_dev->flags |= F_IPV6; 1339 pkt_dev->flags |= F_IPV6;
1340 1340
1341 if (copy_from_user(buf, &user_buffer[i], len)) 1341 if (copy_from_user(buf, &user_buffer[i], len))
1342 return -EFAULT; 1342 return -EFAULT;
1343 buf[len] = 0; 1343 buf[len] = 0;
1344 1344
1345 in6_pton(buf, -1, pkt_dev->max_in6_daddr.s6_addr, -1, NULL); 1345 in6_pton(buf, -1, pkt_dev->max_in6_daddr.s6_addr, -1, NULL);
1346 snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->max_in6_daddr); 1346 snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->max_in6_daddr);
1347 1347
1348 if (debug) 1348 if (debug)
1349 pr_debug("dst6_max set to: %s\n", buf); 1349 pr_debug("dst6_max set to: %s\n", buf);
1350 1350
1351 i += len; 1351 i += len;
1352 sprintf(pg_result, "OK: dst6_max=%s", buf); 1352 sprintf(pg_result, "OK: dst6_max=%s", buf);
1353 return count; 1353 return count;
1354 } 1354 }
1355 if (!strcmp(name, "src6")) { 1355 if (!strcmp(name, "src6")) {
1356 len = strn_len(&user_buffer[i], sizeof(buf) - 1); 1356 len = strn_len(&user_buffer[i], sizeof(buf) - 1);
1357 if (len < 0) 1357 if (len < 0)
1358 return len; 1358 return len;
1359 1359
1360 pkt_dev->flags |= F_IPV6; 1360 pkt_dev->flags |= F_IPV6;
1361 1361
1362 if (copy_from_user(buf, &user_buffer[i], len)) 1362 if (copy_from_user(buf, &user_buffer[i], len))
1363 return -EFAULT; 1363 return -EFAULT;
1364 buf[len] = 0; 1364 buf[len] = 0;
1365 1365
1366 in6_pton(buf, -1, pkt_dev->in6_saddr.s6_addr, -1, NULL); 1366 in6_pton(buf, -1, pkt_dev->in6_saddr.s6_addr, -1, NULL);
1367 snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->in6_saddr); 1367 snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->in6_saddr);
1368 1368
1369 pkt_dev->cur_in6_saddr = pkt_dev->in6_saddr; 1369 pkt_dev->cur_in6_saddr = pkt_dev->in6_saddr;
1370 1370
1371 if (debug) 1371 if (debug)
1372 pr_debug("src6 set to: %s\n", buf); 1372 pr_debug("src6 set to: %s\n", buf);
1373 1373
1374 i += len; 1374 i += len;
1375 sprintf(pg_result, "OK: src6=%s", buf); 1375 sprintf(pg_result, "OK: src6=%s", buf);
1376 return count; 1376 return count;
1377 } 1377 }
1378 if (!strcmp(name, "src_min")) { 1378 if (!strcmp(name, "src_min")) {
1379 len = strn_len(&user_buffer[i], sizeof(pkt_dev->src_min) - 1); 1379 len = strn_len(&user_buffer[i], sizeof(pkt_dev->src_min) - 1);
1380 if (len < 0) 1380 if (len < 0)
1381 return len; 1381 return len;
1382 1382
1383 if (copy_from_user(buf, &user_buffer[i], len)) 1383 if (copy_from_user(buf, &user_buffer[i], len))
1384 return -EFAULT; 1384 return -EFAULT;
1385 buf[len] = 0; 1385 buf[len] = 0;
1386 if (strcmp(buf, pkt_dev->src_min) != 0) { 1386 if (strcmp(buf, pkt_dev->src_min) != 0) {
1387 memset(pkt_dev->src_min, 0, sizeof(pkt_dev->src_min)); 1387 memset(pkt_dev->src_min, 0, sizeof(pkt_dev->src_min));
1388 strncpy(pkt_dev->src_min, buf, len); 1388 strncpy(pkt_dev->src_min, buf, len);
1389 pkt_dev->saddr_min = in_aton(pkt_dev->src_min); 1389 pkt_dev->saddr_min = in_aton(pkt_dev->src_min);
1390 pkt_dev->cur_saddr = pkt_dev->saddr_min; 1390 pkt_dev->cur_saddr = pkt_dev->saddr_min;
1391 } 1391 }
1392 if (debug) 1392 if (debug)
1393 pr_debug("src_min set to: %s\n", pkt_dev->src_min); 1393 pr_debug("src_min set to: %s\n", pkt_dev->src_min);
1394 i += len; 1394 i += len;
1395 sprintf(pg_result, "OK: src_min=%s", pkt_dev->src_min); 1395 sprintf(pg_result, "OK: src_min=%s", pkt_dev->src_min);
1396 return count; 1396 return count;
1397 } 1397 }
1398 if (!strcmp(name, "src_max")) { 1398 if (!strcmp(name, "src_max")) {
1399 len = strn_len(&user_buffer[i], sizeof(pkt_dev->src_max) - 1); 1399 len = strn_len(&user_buffer[i], sizeof(pkt_dev->src_max) - 1);
1400 if (len < 0) 1400 if (len < 0)
1401 return len; 1401 return len;
1402 1402
1403 if (copy_from_user(buf, &user_buffer[i], len)) 1403 if (copy_from_user(buf, &user_buffer[i], len))
1404 return -EFAULT; 1404 return -EFAULT;
1405 buf[len] = 0; 1405 buf[len] = 0;
1406 if (strcmp(buf, pkt_dev->src_max) != 0) { 1406 if (strcmp(buf, pkt_dev->src_max) != 0) {
1407 memset(pkt_dev->src_max, 0, sizeof(pkt_dev->src_max)); 1407 memset(pkt_dev->src_max, 0, sizeof(pkt_dev->src_max));
1408 strncpy(pkt_dev->src_max, buf, len); 1408 strncpy(pkt_dev->src_max, buf, len);
1409 pkt_dev->saddr_max = in_aton(pkt_dev->src_max); 1409 pkt_dev->saddr_max = in_aton(pkt_dev->src_max);
1410 pkt_dev->cur_saddr = pkt_dev->saddr_max; 1410 pkt_dev->cur_saddr = pkt_dev->saddr_max;
1411 } 1411 }
1412 if (debug) 1412 if (debug)
1413 pr_debug("src_max set to: %s\n", pkt_dev->src_max); 1413 pr_debug("src_max set to: %s\n", pkt_dev->src_max);
1414 i += len; 1414 i += len;
1415 sprintf(pg_result, "OK: src_max=%s", pkt_dev->src_max); 1415 sprintf(pg_result, "OK: src_max=%s", pkt_dev->src_max);
1416 return count; 1416 return count;
1417 } 1417 }
1418 if (!strcmp(name, "dst_mac")) { 1418 if (!strcmp(name, "dst_mac")) {
1419 len = strn_len(&user_buffer[i], sizeof(valstr) - 1); 1419 len = strn_len(&user_buffer[i], sizeof(valstr) - 1);
1420 if (len < 0) 1420 if (len < 0)
1421 return len; 1421 return len;
1422 1422
1423 memset(valstr, 0, sizeof(valstr)); 1423 memset(valstr, 0, sizeof(valstr));
1424 if (copy_from_user(valstr, &user_buffer[i], len)) 1424 if (copy_from_user(valstr, &user_buffer[i], len))
1425 return -EFAULT; 1425 return -EFAULT;
1426 1426
1427 if (!mac_pton(valstr, pkt_dev->dst_mac)) 1427 if (!mac_pton(valstr, pkt_dev->dst_mac))
1428 return -EINVAL; 1428 return -EINVAL;
1429 /* Set up Dest MAC */ 1429 /* Set up Dest MAC */
1430 memcpy(&pkt_dev->hh[0], pkt_dev->dst_mac, ETH_ALEN); 1430 memcpy(&pkt_dev->hh[0], pkt_dev->dst_mac, ETH_ALEN);
1431 1431
1432 sprintf(pg_result, "OK: dstmac %pM", pkt_dev->dst_mac); 1432 sprintf(pg_result, "OK: dstmac %pM", pkt_dev->dst_mac);
1433 return count; 1433 return count;
1434 } 1434 }
1435 if (!strcmp(name, "src_mac")) { 1435 if (!strcmp(name, "src_mac")) {
1436 len = strn_len(&user_buffer[i], sizeof(valstr) - 1); 1436 len = strn_len(&user_buffer[i], sizeof(valstr) - 1);
1437 if (len < 0) 1437 if (len < 0)
1438 return len; 1438 return len;
1439 1439
1440 memset(valstr, 0, sizeof(valstr)); 1440 memset(valstr, 0, sizeof(valstr));
1441 if (copy_from_user(valstr, &user_buffer[i], len)) 1441 if (copy_from_user(valstr, &user_buffer[i], len))
1442 return -EFAULT; 1442 return -EFAULT;
1443 1443
1444 if (!mac_pton(valstr, pkt_dev->src_mac)) 1444 if (!mac_pton(valstr, pkt_dev->src_mac))
1445 return -EINVAL; 1445 return -EINVAL;
1446 /* Set up Src MAC */ 1446 /* Set up Src MAC */
1447 memcpy(&pkt_dev->hh[6], pkt_dev->src_mac, ETH_ALEN); 1447 memcpy(&pkt_dev->hh[6], pkt_dev->src_mac, ETH_ALEN);
1448 1448
1449 sprintf(pg_result, "OK: srcmac %pM", pkt_dev->src_mac); 1449 sprintf(pg_result, "OK: srcmac %pM", pkt_dev->src_mac);
1450 return count; 1450 return count;
1451 } 1451 }
1452 1452
1453 if (!strcmp(name, "clear_counters")) { 1453 if (!strcmp(name, "clear_counters")) {
1454 pktgen_clear_counters(pkt_dev); 1454 pktgen_clear_counters(pkt_dev);
1455 sprintf(pg_result, "OK: Clearing counters.\n"); 1455 sprintf(pg_result, "OK: Clearing counters.\n");
1456 return count; 1456 return count;
1457 } 1457 }
1458 1458
1459 if (!strcmp(name, "flows")) { 1459 if (!strcmp(name, "flows")) {
1460 len = num_arg(&user_buffer[i], 10, &value); 1460 len = num_arg(&user_buffer[i], 10, &value);
1461 if (len < 0) 1461 if (len < 0)
1462 return len; 1462 return len;
1463 1463
1464 i += len; 1464 i += len;
1465 if (value > MAX_CFLOWS) 1465 if (value > MAX_CFLOWS)
1466 value = MAX_CFLOWS; 1466 value = MAX_CFLOWS;
1467 1467
1468 pkt_dev->cflows = value; 1468 pkt_dev->cflows = value;
1469 sprintf(pg_result, "OK: flows=%u", pkt_dev->cflows); 1469 sprintf(pg_result, "OK: flows=%u", pkt_dev->cflows);
1470 return count; 1470 return count;
1471 } 1471 }
1472 1472
1473 if (!strcmp(name, "flowlen")) { 1473 if (!strcmp(name, "flowlen")) {
1474 len = num_arg(&user_buffer[i], 10, &value); 1474 len = num_arg(&user_buffer[i], 10, &value);
1475 if (len < 0) 1475 if (len < 0)
1476 return len; 1476 return len;
1477 1477
1478 i += len; 1478 i += len;
1479 pkt_dev->lflow = value; 1479 pkt_dev->lflow = value;
1480 sprintf(pg_result, "OK: flowlen=%u", pkt_dev->lflow); 1480 sprintf(pg_result, "OK: flowlen=%u", pkt_dev->lflow);
1481 return count; 1481 return count;
1482 } 1482 }
1483 1483
1484 if (!strcmp(name, "queue_map_min")) { 1484 if (!strcmp(name, "queue_map_min")) {
1485 len = num_arg(&user_buffer[i], 5, &value); 1485 len = num_arg(&user_buffer[i], 5, &value);
1486 if (len < 0) 1486 if (len < 0)
1487 return len; 1487 return len;
1488 1488
1489 i += len; 1489 i += len;
1490 pkt_dev->queue_map_min = value; 1490 pkt_dev->queue_map_min = value;
1491 sprintf(pg_result, "OK: queue_map_min=%u", pkt_dev->queue_map_min); 1491 sprintf(pg_result, "OK: queue_map_min=%u", pkt_dev->queue_map_min);
1492 return count; 1492 return count;
1493 } 1493 }
1494 1494
1495 if (!strcmp(name, "queue_map_max")) { 1495 if (!strcmp(name, "queue_map_max")) {
1496 len = num_arg(&user_buffer[i], 5, &value); 1496 len = num_arg(&user_buffer[i], 5, &value);
1497 if (len < 0) 1497 if (len < 0)
1498 return len; 1498 return len;
1499 1499
1500 i += len; 1500 i += len;
1501 pkt_dev->queue_map_max = value; 1501 pkt_dev->queue_map_max = value;
1502 sprintf(pg_result, "OK: queue_map_max=%u", pkt_dev->queue_map_max); 1502 sprintf(pg_result, "OK: queue_map_max=%u", pkt_dev->queue_map_max);
1503 return count; 1503 return count;
1504 } 1504 }
1505 1505
1506 if (!strcmp(name, "mpls")) { 1506 if (!strcmp(name, "mpls")) {
1507 unsigned int n, cnt; 1507 unsigned int n, cnt;
1508 1508
1509 len = get_labels(&user_buffer[i], pkt_dev); 1509 len = get_labels(&user_buffer[i], pkt_dev);
1510 if (len < 0) 1510 if (len < 0)
1511 return len; 1511 return len;
1512 i += len; 1512 i += len;
1513 cnt = sprintf(pg_result, "OK: mpls="); 1513 cnt = sprintf(pg_result, "OK: mpls=");
1514 for (n = 0; n < pkt_dev->nr_labels; n++) 1514 for (n = 0; n < pkt_dev->nr_labels; n++)
1515 cnt += sprintf(pg_result + cnt, 1515 cnt += sprintf(pg_result + cnt,
1516 "%08x%s", ntohl(pkt_dev->labels[n]), 1516 "%08x%s", ntohl(pkt_dev->labels[n]),
1517 n == pkt_dev->nr_labels-1 ? "" : ","); 1517 n == pkt_dev->nr_labels-1 ? "" : ",");
1518 1518
1519 if (pkt_dev->nr_labels && pkt_dev->vlan_id != 0xffff) { 1519 if (pkt_dev->nr_labels && pkt_dev->vlan_id != 0xffff) {
1520 pkt_dev->vlan_id = 0xffff; /* turn off VLAN/SVLAN */ 1520 pkt_dev->vlan_id = 0xffff; /* turn off VLAN/SVLAN */
1521 pkt_dev->svlan_id = 0xffff; 1521 pkt_dev->svlan_id = 0xffff;
1522 1522
1523 if (debug) 1523 if (debug)
1524 pr_debug("VLAN/SVLAN auto turned off\n"); 1524 pr_debug("VLAN/SVLAN auto turned off\n");
1525 } 1525 }
1526 return count; 1526 return count;
1527 } 1527 }
1528 1528
1529 if (!strcmp(name, "vlan_id")) { 1529 if (!strcmp(name, "vlan_id")) {
1530 len = num_arg(&user_buffer[i], 4, &value); 1530 len = num_arg(&user_buffer[i], 4, &value);
1531 if (len < 0) 1531 if (len < 0)
1532 return len; 1532 return len;
1533 1533
1534 i += len; 1534 i += len;
1535 if (value <= 4095) { 1535 if (value <= 4095) {
1536 pkt_dev->vlan_id = value; /* turn on VLAN */ 1536 pkt_dev->vlan_id = value; /* turn on VLAN */
1537 1537
1538 if (debug) 1538 if (debug)
1539 pr_debug("VLAN turned on\n"); 1539 pr_debug("VLAN turned on\n");
1540 1540
1541 if (debug && pkt_dev->nr_labels) 1541 if (debug && pkt_dev->nr_labels)
1542 pr_debug("MPLS auto turned off\n"); 1542 pr_debug("MPLS auto turned off\n");
1543 1543
1544 pkt_dev->nr_labels = 0; /* turn off MPLS */ 1544 pkt_dev->nr_labels = 0; /* turn off MPLS */
1545 sprintf(pg_result, "OK: vlan_id=%u", pkt_dev->vlan_id); 1545 sprintf(pg_result, "OK: vlan_id=%u", pkt_dev->vlan_id);
1546 } else { 1546 } else {
1547 pkt_dev->vlan_id = 0xffff; /* turn off VLAN/SVLAN */ 1547 pkt_dev->vlan_id = 0xffff; /* turn off VLAN/SVLAN */
1548 pkt_dev->svlan_id = 0xffff; 1548 pkt_dev->svlan_id = 0xffff;
1549 1549
1550 if (debug) 1550 if (debug)
1551 pr_debug("VLAN/SVLAN turned off\n"); 1551 pr_debug("VLAN/SVLAN turned off\n");
1552 } 1552 }
1553 return count; 1553 return count;
1554 } 1554 }
1555 1555
1556 if (!strcmp(name, "vlan_p")) { 1556 if (!strcmp(name, "vlan_p")) {
1557 len = num_arg(&user_buffer[i], 1, &value); 1557 len = num_arg(&user_buffer[i], 1, &value);
1558 if (len < 0) 1558 if (len < 0)
1559 return len; 1559 return len;
1560 1560
1561 i += len; 1561 i += len;
1562 if ((value <= 7) && (pkt_dev->vlan_id != 0xffff)) { 1562 if ((value <= 7) && (pkt_dev->vlan_id != 0xffff)) {
1563 pkt_dev->vlan_p = value; 1563 pkt_dev->vlan_p = value;
1564 sprintf(pg_result, "OK: vlan_p=%u", pkt_dev->vlan_p); 1564 sprintf(pg_result, "OK: vlan_p=%u", pkt_dev->vlan_p);
1565 } else { 1565 } else {
1566 sprintf(pg_result, "ERROR: vlan_p must be 0-7"); 1566 sprintf(pg_result, "ERROR: vlan_p must be 0-7");
1567 } 1567 }
1568 return count; 1568 return count;
1569 } 1569 }
1570 1570
1571 if (!strcmp(name, "vlan_cfi")) { 1571 if (!strcmp(name, "vlan_cfi")) {
1572 len = num_arg(&user_buffer[i], 1, &value); 1572 len = num_arg(&user_buffer[i], 1, &value);
1573 if (len < 0) 1573 if (len < 0)
1574 return len; 1574 return len;
1575 1575
1576 i += len; 1576 i += len;
1577 if ((value <= 1) && (pkt_dev->vlan_id != 0xffff)) { 1577 if ((value <= 1) && (pkt_dev->vlan_id != 0xffff)) {
1578 pkt_dev->vlan_cfi = value; 1578 pkt_dev->vlan_cfi = value;
1579 sprintf(pg_result, "OK: vlan_cfi=%u", pkt_dev->vlan_cfi); 1579 sprintf(pg_result, "OK: vlan_cfi=%u", pkt_dev->vlan_cfi);
1580 } else { 1580 } else {
1581 sprintf(pg_result, "ERROR: vlan_cfi must be 0-1"); 1581 sprintf(pg_result, "ERROR: vlan_cfi must be 0-1");
1582 } 1582 }
1583 return count; 1583 return count;
1584 } 1584 }
1585 1585
1586 if (!strcmp(name, "svlan_id")) { 1586 if (!strcmp(name, "svlan_id")) {
1587 len = num_arg(&user_buffer[i], 4, &value); 1587 len = num_arg(&user_buffer[i], 4, &value);
1588 if (len < 0) 1588 if (len < 0)
1589 return len; 1589 return len;
1590 1590
1591 i += len; 1591 i += len;
1592 if ((value <= 4095) && ((pkt_dev->vlan_id != 0xffff))) { 1592 if ((value <= 4095) && ((pkt_dev->vlan_id != 0xffff))) {
1593 pkt_dev->svlan_id = value; /* turn on SVLAN */ 1593 pkt_dev->svlan_id = value; /* turn on SVLAN */
1594 1594
1595 if (debug) 1595 if (debug)
1596 pr_debug("SVLAN turned on\n"); 1596 pr_debug("SVLAN turned on\n");
1597 1597
1598 if (debug && pkt_dev->nr_labels) 1598 if (debug && pkt_dev->nr_labels)
1599 pr_debug("MPLS auto turned off\n"); 1599 pr_debug("MPLS auto turned off\n");
1600 1600
1601 pkt_dev->nr_labels = 0; /* turn off MPLS */ 1601 pkt_dev->nr_labels = 0; /* turn off MPLS */
1602 sprintf(pg_result, "OK: svlan_id=%u", pkt_dev->svlan_id); 1602 sprintf(pg_result, "OK: svlan_id=%u", pkt_dev->svlan_id);
1603 } else { 1603 } else {
1604 pkt_dev->vlan_id = 0xffff; /* turn off VLAN/SVLAN */ 1604 pkt_dev->vlan_id = 0xffff; /* turn off VLAN/SVLAN */
1605 pkt_dev->svlan_id = 0xffff; 1605 pkt_dev->svlan_id = 0xffff;
1606 1606
1607 if (debug) 1607 if (debug)
1608 pr_debug("VLAN/SVLAN turned off\n"); 1608 pr_debug("VLAN/SVLAN turned off\n");
1609 } 1609 }
1610 return count; 1610 return count;
1611 } 1611 }
1612 1612
1613 if (!strcmp(name, "svlan_p")) { 1613 if (!strcmp(name, "svlan_p")) {
1614 len = num_arg(&user_buffer[i], 1, &value); 1614 len = num_arg(&user_buffer[i], 1, &value);
1615 if (len < 0) 1615 if (len < 0)
1616 return len; 1616 return len;
1617 1617
1618 i += len; 1618 i += len;
1619 if ((value <= 7) && (pkt_dev->svlan_id != 0xffff)) { 1619 if ((value <= 7) && (pkt_dev->svlan_id != 0xffff)) {
1620 pkt_dev->svlan_p = value; 1620 pkt_dev->svlan_p = value;
1621 sprintf(pg_result, "OK: svlan_p=%u", pkt_dev->svlan_p); 1621 sprintf(pg_result, "OK: svlan_p=%u", pkt_dev->svlan_p);
1622 } else { 1622 } else {
1623 sprintf(pg_result, "ERROR: svlan_p must be 0-7"); 1623 sprintf(pg_result, "ERROR: svlan_p must be 0-7");
1624 } 1624 }
1625 return count; 1625 return count;
1626 } 1626 }
1627 1627
1628 if (!strcmp(name, "svlan_cfi")) { 1628 if (!strcmp(name, "svlan_cfi")) {
1629 len = num_arg(&user_buffer[i], 1, &value); 1629 len = num_arg(&user_buffer[i], 1, &value);
1630 if (len < 0) 1630 if (len < 0)
1631 return len; 1631 return len;
1632 1632
1633 i += len; 1633 i += len;
1634 if ((value <= 1) && (pkt_dev->svlan_id != 0xffff)) { 1634 if ((value <= 1) && (pkt_dev->svlan_id != 0xffff)) {
1635 pkt_dev->svlan_cfi = value; 1635 pkt_dev->svlan_cfi = value;
1636 sprintf(pg_result, "OK: svlan_cfi=%u", pkt_dev->svlan_cfi); 1636 sprintf(pg_result, "OK: svlan_cfi=%u", pkt_dev->svlan_cfi);
1637 } else { 1637 } else {
1638 sprintf(pg_result, "ERROR: svlan_cfi must be 0-1"); 1638 sprintf(pg_result, "ERROR: svlan_cfi must be 0-1");
1639 } 1639 }
1640 return count; 1640 return count;
1641 } 1641 }
1642 1642
1643 if (!strcmp(name, "tos")) { 1643 if (!strcmp(name, "tos")) {
1644 __u32 tmp_value = 0; 1644 __u32 tmp_value = 0;
1645 len = hex32_arg(&user_buffer[i], 2, &tmp_value); 1645 len = hex32_arg(&user_buffer[i], 2, &tmp_value);
1646 if (len < 0) 1646 if (len < 0)
1647 return len; 1647 return len;
1648 1648
1649 i += len; 1649 i += len;
1650 if (len == 2) { 1650 if (len == 2) {
1651 pkt_dev->tos = tmp_value; 1651 pkt_dev->tos = tmp_value;
1652 sprintf(pg_result, "OK: tos=0x%02x", pkt_dev->tos); 1652 sprintf(pg_result, "OK: tos=0x%02x", pkt_dev->tos);
1653 } else { 1653 } else {
1654 sprintf(pg_result, "ERROR: tos must be 00-ff"); 1654 sprintf(pg_result, "ERROR: tos must be 00-ff");
1655 } 1655 }
1656 return count; 1656 return count;
1657 } 1657 }
1658 1658
1659 if (!strcmp(name, "traffic_class")) { 1659 if (!strcmp(name, "traffic_class")) {
1660 __u32 tmp_value = 0; 1660 __u32 tmp_value = 0;
1661 len = hex32_arg(&user_buffer[i], 2, &tmp_value); 1661 len = hex32_arg(&user_buffer[i], 2, &tmp_value);
1662 if (len < 0) 1662 if (len < 0)
1663 return len; 1663 return len;
1664 1664
1665 i += len; 1665 i += len;
1666 if (len == 2) { 1666 if (len == 2) {
1667 pkt_dev->traffic_class = tmp_value; 1667 pkt_dev->traffic_class = tmp_value;
1668 sprintf(pg_result, "OK: traffic_class=0x%02x", pkt_dev->traffic_class); 1668 sprintf(pg_result, "OK: traffic_class=0x%02x", pkt_dev->traffic_class);
1669 } else { 1669 } else {
1670 sprintf(pg_result, "ERROR: traffic_class must be 00-ff"); 1670 sprintf(pg_result, "ERROR: traffic_class must be 00-ff");
1671 } 1671 }
1672 return count; 1672 return count;
1673 } 1673 }
1674 1674
1675 if (!strcmp(name, "skb_priority")) { 1675 if (!strcmp(name, "skb_priority")) {
1676 len = num_arg(&user_buffer[i], 9, &value); 1676 len = num_arg(&user_buffer[i], 9, &value);
1677 if (len < 0) 1677 if (len < 0)
1678 return len; 1678 return len;
1679 1679
1680 i += len; 1680 i += len;
1681 pkt_dev->skb_priority = value; 1681 pkt_dev->skb_priority = value;
1682 sprintf(pg_result, "OK: skb_priority=%i", 1682 sprintf(pg_result, "OK: skb_priority=%i",
1683 pkt_dev->skb_priority); 1683 pkt_dev->skb_priority);
1684 return count; 1684 return count;
1685 } 1685 }
1686 1686
1687 sprintf(pkt_dev->result, "No such parameter \"%s\"", name); 1687 sprintf(pkt_dev->result, "No such parameter \"%s\"", name);
1688 return -EINVAL; 1688 return -EINVAL;
1689 } 1689 }
1690 1690
1691 static int pktgen_if_open(struct inode *inode, struct file *file) 1691 static int pktgen_if_open(struct inode *inode, struct file *file)
1692 { 1692 {
1693 return single_open(file, pktgen_if_show, PDE(inode)->data); 1693 return single_open(file, pktgen_if_show, PDE(inode)->data);
1694 } 1694 }
1695 1695
1696 static const struct file_operations pktgen_if_fops = { 1696 static const struct file_operations pktgen_if_fops = {
1697 .owner = THIS_MODULE, 1697 .owner = THIS_MODULE,
1698 .open = pktgen_if_open, 1698 .open = pktgen_if_open,
1699 .read = seq_read, 1699 .read = seq_read,
1700 .llseek = seq_lseek, 1700 .llseek = seq_lseek,
1701 .write = pktgen_if_write, 1701 .write = pktgen_if_write,
1702 .release = single_release, 1702 .release = single_release,
1703 }; 1703 };
1704 1704
1705 static int pktgen_thread_show(struct seq_file *seq, void *v) 1705 static int pktgen_thread_show(struct seq_file *seq, void *v)
1706 { 1706 {
1707 struct pktgen_thread *t = seq->private; 1707 struct pktgen_thread *t = seq->private;
1708 const struct pktgen_dev *pkt_dev; 1708 const struct pktgen_dev *pkt_dev;
1709 1709
1710 BUG_ON(!t); 1710 BUG_ON(!t);
1711 1711
1712 seq_printf(seq, "Running: "); 1712 seq_printf(seq, "Running: ");
1713 1713
1714 if_lock(t); 1714 if_lock(t);
1715 list_for_each_entry(pkt_dev, &t->if_list, list) 1715 list_for_each_entry(pkt_dev, &t->if_list, list)
1716 if (pkt_dev->running) 1716 if (pkt_dev->running)
1717 seq_printf(seq, "%s ", pkt_dev->odevname); 1717 seq_printf(seq, "%s ", pkt_dev->odevname);
1718 1718
1719 seq_printf(seq, "\nStopped: "); 1719 seq_printf(seq, "\nStopped: ");
1720 1720
1721 list_for_each_entry(pkt_dev, &t->if_list, list) 1721 list_for_each_entry(pkt_dev, &t->if_list, list)
1722 if (!pkt_dev->running) 1722 if (!pkt_dev->running)
1723 seq_printf(seq, "%s ", pkt_dev->odevname); 1723 seq_printf(seq, "%s ", pkt_dev->odevname);
1724 1724
1725 if (t->result[0]) 1725 if (t->result[0])
1726 seq_printf(seq, "\nResult: %s\n", t->result); 1726 seq_printf(seq, "\nResult: %s\n", t->result);
1727 else 1727 else
1728 seq_printf(seq, "\nResult: NA\n"); 1728 seq_printf(seq, "\nResult: NA\n");
1729 1729
1730 if_unlock(t); 1730 if_unlock(t);
1731 1731
1732 return 0; 1732 return 0;
1733 } 1733 }
1734 1734
1735 static ssize_t pktgen_thread_write(struct file *file, 1735 static ssize_t pktgen_thread_write(struct file *file,
1736 const char __user * user_buffer, 1736 const char __user * user_buffer,
1737 size_t count, loff_t * offset) 1737 size_t count, loff_t * offset)
1738 { 1738 {
1739 struct seq_file *seq = file->private_data; 1739 struct seq_file *seq = file->private_data;
1740 struct pktgen_thread *t = seq->private; 1740 struct pktgen_thread *t = seq->private;
1741 int i, max, len, ret; 1741 int i, max, len, ret;
1742 char name[40]; 1742 char name[40];
1743 char *pg_result; 1743 char *pg_result;
1744 1744
1745 if (count < 1) { 1745 if (count < 1) {
1746 // sprintf(pg_result, "Wrong command format"); 1746 // sprintf(pg_result, "Wrong command format");
1747 return -EINVAL; 1747 return -EINVAL;
1748 } 1748 }
1749 1749
1750 max = count; 1750 max = count;
1751 len = count_trail_chars(user_buffer, max); 1751 len = count_trail_chars(user_buffer, max);
1752 if (len < 0) 1752 if (len < 0)
1753 return len; 1753 return len;
1754 1754
1755 i = len; 1755 i = len;
1756 1756
1757 /* Read variable name */ 1757 /* Read variable name */
1758 1758
1759 len = strn_len(&user_buffer[i], sizeof(name) - 1); 1759 len = strn_len(&user_buffer[i], sizeof(name) - 1);
1760 if (len < 0) 1760 if (len < 0)
1761 return len; 1761 return len;
1762 1762
1763 memset(name, 0, sizeof(name)); 1763 memset(name, 0, sizeof(name));
1764 if (copy_from_user(name, &user_buffer[i], len)) 1764 if (copy_from_user(name, &user_buffer[i], len))
1765 return -EFAULT; 1765 return -EFAULT;
1766 i += len; 1766 i += len;
1767 1767
1768 max = count - i; 1768 max = count - i;
1769 len = count_trail_chars(&user_buffer[i], max); 1769 len = count_trail_chars(&user_buffer[i], max);
1770 if (len < 0) 1770 if (len < 0)
1771 return len; 1771 return len;
1772 1772
1773 i += len; 1773 i += len;
1774 1774
1775 if (debug) 1775 if (debug)
1776 pr_debug("t=%s, count=%lu\n", name, (unsigned long)count); 1776 pr_debug("t=%s, count=%lu\n", name, (unsigned long)count);
1777 1777
1778 if (!t) { 1778 if (!t) {
1779 pr_err("ERROR: No thread\n"); 1779 pr_err("ERROR: No thread\n");
1780 ret = -EINVAL; 1780 ret = -EINVAL;
1781 goto out; 1781 goto out;
1782 } 1782 }
1783 1783
1784 pg_result = &(t->result[0]); 1784 pg_result = &(t->result[0]);
1785 1785
1786 if (!strcmp(name, "add_device")) { 1786 if (!strcmp(name, "add_device")) {
1787 char f[32]; 1787 char f[32];
1788 memset(f, 0, 32); 1788 memset(f, 0, 32);
1789 len = strn_len(&user_buffer[i], sizeof(f) - 1); 1789 len = strn_len(&user_buffer[i], sizeof(f) - 1);
1790 if (len < 0) { 1790 if (len < 0) {
1791 ret = len; 1791 ret = len;
1792 goto out; 1792 goto out;
1793 } 1793 }
1794 if (copy_from_user(f, &user_buffer[i], len)) 1794 if (copy_from_user(f, &user_buffer[i], len))
1795 return -EFAULT; 1795 return -EFAULT;
1796 i += len; 1796 i += len;
1797 mutex_lock(&pktgen_thread_lock); 1797 mutex_lock(&pktgen_thread_lock);
1798 pktgen_add_device(t, f); 1798 pktgen_add_device(t, f);
1799 mutex_unlock(&pktgen_thread_lock); 1799 mutex_unlock(&pktgen_thread_lock);
1800 ret = count; 1800 ret = count;
1801 sprintf(pg_result, "OK: add_device=%s", f); 1801 sprintf(pg_result, "OK: add_device=%s", f);
1802 goto out; 1802 goto out;
1803 } 1803 }
1804 1804
1805 if (!strcmp(name, "rem_device_all")) { 1805 if (!strcmp(name, "rem_device_all")) {
1806 mutex_lock(&pktgen_thread_lock); 1806 mutex_lock(&pktgen_thread_lock);
1807 t->control |= T_REMDEVALL; 1807 t->control |= T_REMDEVALL;
1808 mutex_unlock(&pktgen_thread_lock); 1808 mutex_unlock(&pktgen_thread_lock);
1809 schedule_timeout_interruptible(msecs_to_jiffies(125)); /* Propagate thread->control */ 1809 schedule_timeout_interruptible(msecs_to_jiffies(125)); /* Propagate thread->control */
1810 ret = count; 1810 ret = count;
1811 sprintf(pg_result, "OK: rem_device_all"); 1811 sprintf(pg_result, "OK: rem_device_all");
1812 goto out; 1812 goto out;
1813 } 1813 }
1814 1814
1815 if (!strcmp(name, "max_before_softirq")) { 1815 if (!strcmp(name, "max_before_softirq")) {
1816 sprintf(pg_result, "OK: Note! max_before_softirq is obsoleted -- Do not use"); 1816 sprintf(pg_result, "OK: Note! max_before_softirq is obsoleted -- Do not use");
1817 ret = count; 1817 ret = count;
1818 goto out; 1818 goto out;
1819 } 1819 }
1820 1820
1821 ret = -EINVAL; 1821 ret = -EINVAL;
1822 out: 1822 out:
1823 return ret; 1823 return ret;
1824 } 1824 }
1825 1825
1826 static int pktgen_thread_open(struct inode *inode, struct file *file) 1826 static int pktgen_thread_open(struct inode *inode, struct file *file)
1827 { 1827 {
1828 return single_open(file, pktgen_thread_show, PDE(inode)->data); 1828 return single_open(file, pktgen_thread_show, PDE(inode)->data);
1829 } 1829 }
1830 1830
1831 static const struct file_operations pktgen_thread_fops = { 1831 static const struct file_operations pktgen_thread_fops = {
1832 .owner = THIS_MODULE, 1832 .owner = THIS_MODULE,
1833 .open = pktgen_thread_open, 1833 .open = pktgen_thread_open,
1834 .read = seq_read, 1834 .read = seq_read,
1835 .llseek = seq_lseek, 1835 .llseek = seq_lseek,
1836 .write = pktgen_thread_write, 1836 .write = pktgen_thread_write,
1837 .release = single_release, 1837 .release = single_release,
1838 }; 1838 };
1839 1839
1840 /* Think find or remove for NN */ 1840 /* Think find or remove for NN */
1841 static struct pktgen_dev *__pktgen_NN_threads(const char *ifname, int remove) 1841 static struct pktgen_dev *__pktgen_NN_threads(const char *ifname, int remove)
1842 { 1842 {
1843 struct pktgen_thread *t; 1843 struct pktgen_thread *t;
1844 struct pktgen_dev *pkt_dev = NULL; 1844 struct pktgen_dev *pkt_dev = NULL;
1845 bool exact = (remove == FIND); 1845 bool exact = (remove == FIND);
1846 1846
1847 list_for_each_entry(t, &pktgen_threads, th_list) { 1847 list_for_each_entry(t, &pktgen_threads, th_list) {
1848 pkt_dev = pktgen_find_dev(t, ifname, exact); 1848 pkt_dev = pktgen_find_dev(t, ifname, exact);
1849 if (pkt_dev) { 1849 if (pkt_dev) {
1850 if (remove) { 1850 if (remove) {
1851 if_lock(t); 1851 if_lock(t);
1852 pkt_dev->removal_mark = 1; 1852 pkt_dev->removal_mark = 1;
1853 t->control |= T_REMDEV; 1853 t->control |= T_REMDEV;
1854 if_unlock(t); 1854 if_unlock(t);
1855 } 1855 }
1856 break; 1856 break;
1857 } 1857 }
1858 } 1858 }
1859 return pkt_dev; 1859 return pkt_dev;
1860 } 1860 }
1861 1861
1862 /* 1862 /*
1863 * mark a device for removal 1863 * mark a device for removal
1864 */ 1864 */
1865 static void pktgen_mark_device(const char *ifname) 1865 static void pktgen_mark_device(const char *ifname)
1866 { 1866 {
1867 struct pktgen_dev *pkt_dev = NULL; 1867 struct pktgen_dev *pkt_dev = NULL;
1868 const int max_tries = 10, msec_per_try = 125; 1868 const int max_tries = 10, msec_per_try = 125;
1869 int i = 0; 1869 int i = 0;
1870 1870
1871 mutex_lock(&pktgen_thread_lock); 1871 mutex_lock(&pktgen_thread_lock);
1872 pr_debug("%s: marking %s for removal\n", __func__, ifname); 1872 pr_debug("%s: marking %s for removal\n", __func__, ifname);
1873 1873
1874 while (1) { 1874 while (1) {
1875 1875
1876 pkt_dev = __pktgen_NN_threads(ifname, REMOVE); 1876 pkt_dev = __pktgen_NN_threads(ifname, REMOVE);
1877 if (pkt_dev == NULL) 1877 if (pkt_dev == NULL)
1878 break; /* success */ 1878 break; /* success */
1879 1879
1880 mutex_unlock(&pktgen_thread_lock); 1880 mutex_unlock(&pktgen_thread_lock);
1881 pr_debug("%s: waiting for %s to disappear....\n", 1881 pr_debug("%s: waiting for %s to disappear....\n",
1882 __func__, ifname); 1882 __func__, ifname);
1883 schedule_timeout_interruptible(msecs_to_jiffies(msec_per_try)); 1883 schedule_timeout_interruptible(msecs_to_jiffies(msec_per_try));
1884 mutex_lock(&pktgen_thread_lock); 1884 mutex_lock(&pktgen_thread_lock);
1885 1885
1886 if (++i >= max_tries) { 1886 if (++i >= max_tries) {
1887 pr_err("%s: timed out after waiting %d msec for device %s to be removed\n", 1887 pr_err("%s: timed out after waiting %d msec for device %s to be removed\n",
1888 __func__, msec_per_try * i, ifname); 1888 __func__, msec_per_try * i, ifname);
1889 break; 1889 break;
1890 } 1890 }
1891 1891
1892 } 1892 }
1893 1893
1894 mutex_unlock(&pktgen_thread_lock); 1894 mutex_unlock(&pktgen_thread_lock);
1895 } 1895 }
1896 1896
1897 static void pktgen_change_name(struct net_device *dev) 1897 static void pktgen_change_name(struct net_device *dev)
1898 { 1898 {
1899 struct pktgen_thread *t; 1899 struct pktgen_thread *t;
1900 1900
1901 list_for_each_entry(t, &pktgen_threads, th_list) { 1901 list_for_each_entry(t, &pktgen_threads, th_list) {
1902 struct pktgen_dev *pkt_dev; 1902 struct pktgen_dev *pkt_dev;
1903 1903
1904 list_for_each_entry(pkt_dev, &t->if_list, list) { 1904 list_for_each_entry(pkt_dev, &t->if_list, list) {
1905 if (pkt_dev->odev != dev) 1905 if (pkt_dev->odev != dev)
1906 continue; 1906 continue;
1907 1907
1908 remove_proc_entry(pkt_dev->entry->name, pg_proc_dir); 1908 remove_proc_entry(pkt_dev->entry->name, pg_proc_dir);
1909 1909
1910 pkt_dev->entry = proc_create_data(dev->name, 0600, 1910 pkt_dev->entry = proc_create_data(dev->name, 0600,
1911 pg_proc_dir, 1911 pg_proc_dir,
1912 &pktgen_if_fops, 1912 &pktgen_if_fops,
1913 pkt_dev); 1913 pkt_dev);
1914 if (!pkt_dev->entry) 1914 if (!pkt_dev->entry)
1915 pr_err("can't move proc entry for '%s'\n", 1915 pr_err("can't move proc entry for '%s'\n",
1916 dev->name); 1916 dev->name);
1917 break; 1917 break;
1918 } 1918 }
1919 } 1919 }
1920 } 1920 }
1921 1921
1922 static int pktgen_device_event(struct notifier_block *unused, 1922 static int pktgen_device_event(struct notifier_block *unused,
1923 unsigned long event, void *ptr) 1923 unsigned long event, void *ptr)
1924 { 1924 {
1925 struct net_device *dev = ptr; 1925 struct net_device *dev = ptr;
1926 1926
1927 if (!net_eq(dev_net(dev), &init_net) || pktgen_exiting) 1927 if (!net_eq(dev_net(dev), &init_net) || pktgen_exiting)
1928 return NOTIFY_DONE; 1928 return NOTIFY_DONE;
1929 1929
1930 /* It is OK that we do not hold the group lock right now, 1930 /* It is OK that we do not hold the group lock right now,
1931 * as we run under the RTNL lock. 1931 * as we run under the RTNL lock.
1932 */ 1932 */
1933 1933
1934 switch (event) { 1934 switch (event) {
1935 case NETDEV_CHANGENAME: 1935 case NETDEV_CHANGENAME:
1936 pktgen_change_name(dev); 1936 pktgen_change_name(dev);
1937 break; 1937 break;
1938 1938
1939 case NETDEV_UNREGISTER: 1939 case NETDEV_UNREGISTER:
1940 pktgen_mark_device(dev->name); 1940 pktgen_mark_device(dev->name);
1941 break; 1941 break;
1942 } 1942 }
1943 1943
1944 return NOTIFY_DONE; 1944 return NOTIFY_DONE;
1945 } 1945 }
1946 1946
1947 static struct net_device *pktgen_dev_get_by_name(struct pktgen_dev *pkt_dev, 1947 static struct net_device *pktgen_dev_get_by_name(struct pktgen_dev *pkt_dev,
1948 const char *ifname) 1948 const char *ifname)
1949 { 1949 {
1950 char b[IFNAMSIZ+5]; 1950 char b[IFNAMSIZ+5];
1951 int i; 1951 int i;
1952 1952
1953 for (i = 0; ifname[i] != '@'; i++) { 1953 for (i = 0; ifname[i] != '@'; i++) {
1954 if (i == IFNAMSIZ) 1954 if (i == IFNAMSIZ)
1955 break; 1955 break;
1956 1956
1957 b[i] = ifname[i]; 1957 b[i] = ifname[i];
1958 } 1958 }
1959 b[i] = 0; 1959 b[i] = 0;
1960 1960
1961 return dev_get_by_name(&init_net, b); 1961 return dev_get_by_name(&init_net, b);
1962 } 1962 }
1963 1963
1964 1964
1965 /* Associate pktgen_dev with a device. */ 1965 /* Associate pktgen_dev with a device. */
1966 1966
1967 static int pktgen_setup_dev(struct pktgen_dev *pkt_dev, const char *ifname) 1967 static int pktgen_setup_dev(struct pktgen_dev *pkt_dev, const char *ifname)
1968 { 1968 {
1969 struct net_device *odev; 1969 struct net_device *odev;
1970 int err; 1970 int err;
1971 1971
1972 /* Clean old setups */ 1972 /* Clean old setups */
1973 if (pkt_dev->odev) { 1973 if (pkt_dev->odev) {
1974 dev_put(pkt_dev->odev); 1974 dev_put(pkt_dev->odev);
1975 pkt_dev->odev = NULL; 1975 pkt_dev->odev = NULL;
1976 } 1976 }
1977 1977
1978 odev = pktgen_dev_get_by_name(pkt_dev, ifname); 1978 odev = pktgen_dev_get_by_name(pkt_dev, ifname);
1979 if (!odev) { 1979 if (!odev) {
1980 pr_err("no such netdevice: \"%s\"\n", ifname); 1980 pr_err("no such netdevice: \"%s\"\n", ifname);
1981 return -ENODEV; 1981 return -ENODEV;
1982 } 1982 }
1983 1983
1984 if (odev->type != ARPHRD_ETHER) { 1984 if (odev->type != ARPHRD_ETHER) {
1985 pr_err("not an ethernet device: \"%s\"\n", ifname); 1985 pr_err("not an ethernet device: \"%s\"\n", ifname);
1986 err = -EINVAL; 1986 err = -EINVAL;
1987 } else if (!netif_running(odev)) { 1987 } else if (!netif_running(odev)) {
1988 pr_err("device is down: \"%s\"\n", ifname); 1988 pr_err("device is down: \"%s\"\n", ifname);
1989 err = -ENETDOWN; 1989 err = -ENETDOWN;
1990 } else { 1990 } else {
1991 pkt_dev->odev = odev; 1991 pkt_dev->odev = odev;
1992 return 0; 1992 return 0;
1993 } 1993 }
1994 1994
1995 dev_put(odev); 1995 dev_put(odev);
1996 return err; 1996 return err;
1997 } 1997 }
1998 1998
1999 /* Read pkt_dev from the interface and set up internal pktgen_dev 1999 /* Read pkt_dev from the interface and set up internal pktgen_dev
2000 * structure to have the right information to create/send packets 2000 * structure to have the right information to create/send packets
2001 */ 2001 */
2002 static void pktgen_setup_inject(struct pktgen_dev *pkt_dev) 2002 static void pktgen_setup_inject(struct pktgen_dev *pkt_dev)
2003 { 2003 {
2004 int ntxq; 2004 int ntxq;
2005 2005
2006 if (!pkt_dev->odev) { 2006 if (!pkt_dev->odev) {
2007 pr_err("ERROR: pkt_dev->odev == NULL in setup_inject\n"); 2007 pr_err("ERROR: pkt_dev->odev == NULL in setup_inject\n");
2008 sprintf(pkt_dev->result, 2008 sprintf(pkt_dev->result,
2009 "ERROR: pkt_dev->odev == NULL in setup_inject.\n"); 2009 "ERROR: pkt_dev->odev == NULL in setup_inject.\n");
2010 return; 2010 return;
2011 } 2011 }
2012 2012
2013 /* make sure that we don't pick a non-existing transmit queue */ 2013 /* make sure that we don't pick a non-existing transmit queue */
2014 ntxq = pkt_dev->odev->real_num_tx_queues; 2014 ntxq = pkt_dev->odev->real_num_tx_queues;
2015 2015
2016 if (ntxq <= pkt_dev->queue_map_min) { 2016 if (ntxq <= pkt_dev->queue_map_min) {
2017 pr_warning("WARNING: Requested queue_map_min (zero-based) (%d) exceeds valid range [0 - %d] for (%d) queues on %s, resetting\n", 2017 pr_warning("WARNING: Requested queue_map_min (zero-based) (%d) exceeds valid range [0 - %d] for (%d) queues on %s, resetting\n",
2018 pkt_dev->queue_map_min, (ntxq ?: 1) - 1, ntxq, 2018 pkt_dev->queue_map_min, (ntxq ?: 1) - 1, ntxq,
2019 pkt_dev->odevname); 2019 pkt_dev->odevname);
2020 pkt_dev->queue_map_min = (ntxq ?: 1) - 1; 2020 pkt_dev->queue_map_min = (ntxq ?: 1) - 1;
2021 } 2021 }
2022 if (pkt_dev->queue_map_max >= ntxq) { 2022 if (pkt_dev->queue_map_max >= ntxq) {
2023 pr_warning("WARNING: Requested queue_map_max (zero-based) (%d) exceeds valid range [0 - %d] for (%d) queues on %s, resetting\n", 2023 pr_warning("WARNING: Requested queue_map_max (zero-based) (%d) exceeds valid range [0 - %d] for (%d) queues on %s, resetting\n",
2024 pkt_dev->queue_map_max, (ntxq ?: 1) - 1, ntxq, 2024 pkt_dev->queue_map_max, (ntxq ?: 1) - 1, ntxq,
2025 pkt_dev->odevname); 2025 pkt_dev->odevname);
2026 pkt_dev->queue_map_max = (ntxq ?: 1) - 1; 2026 pkt_dev->queue_map_max = (ntxq ?: 1) - 1;
2027 } 2027 }
2028 2028
2029 /* Default to the interface's mac if not explicitly set. */ 2029 /* Default to the interface's mac if not explicitly set. */
2030 2030
2031 if (is_zero_ether_addr(pkt_dev->src_mac)) 2031 if (is_zero_ether_addr(pkt_dev->src_mac))
2032 memcpy(&(pkt_dev->hh[6]), pkt_dev->odev->dev_addr, ETH_ALEN); 2032 memcpy(&(pkt_dev->hh[6]), pkt_dev->odev->dev_addr, ETH_ALEN);
2033 2033
2034 /* Set up Dest MAC */ 2034 /* Set up Dest MAC */
2035 memcpy(&(pkt_dev->hh[0]), pkt_dev->dst_mac, ETH_ALEN); 2035 memcpy(&(pkt_dev->hh[0]), pkt_dev->dst_mac, ETH_ALEN);
2036 2036
2037 if (pkt_dev->flags & F_IPV6) { 2037 if (pkt_dev->flags & F_IPV6) {
2038 int i, set = 0, err = 1; 2038 int i, set = 0, err = 1;
2039 struct inet6_dev *idev; 2039 struct inet6_dev *idev;
2040 2040
2041 if (pkt_dev->min_pkt_size == 0) { 2041 if (pkt_dev->min_pkt_size == 0) {
2042 pkt_dev->min_pkt_size = 14 + sizeof(struct ipv6hdr) 2042 pkt_dev->min_pkt_size = 14 + sizeof(struct ipv6hdr)
2043 + sizeof(struct udphdr) 2043 + sizeof(struct udphdr)
2044 + sizeof(struct pktgen_hdr) 2044 + sizeof(struct pktgen_hdr)
2045 + pkt_dev->pkt_overhead; 2045 + pkt_dev->pkt_overhead;
2046 } 2046 }
2047 2047
2048 for (i = 0; i < IN6_ADDR_HSIZE; i++) 2048 for (i = 0; i < IN6_ADDR_HSIZE; i++)
2049 if (pkt_dev->cur_in6_saddr.s6_addr[i]) { 2049 if (pkt_dev->cur_in6_saddr.s6_addr[i]) {
2050 set = 1; 2050 set = 1;
2051 break; 2051 break;
2052 } 2052 }
2053 2053
2054 if (!set) { 2054 if (!set) {
2055 2055
2056 /* 2056 /*
2057 * Use linklevel address if unconfigured. 2057 * Use linklevel address if unconfigured.
2058 * 2058 *
2059 * use ipv6_get_lladdr if/when it's get exported 2059 * use ipv6_get_lladdr if/when it's get exported
2060 */ 2060 */
2061 2061
2062 rcu_read_lock(); 2062 rcu_read_lock();
2063 idev = __in6_dev_get(pkt_dev->odev); 2063 idev = __in6_dev_get(pkt_dev->odev);
2064 if (idev) { 2064 if (idev) {
2065 struct inet6_ifaddr *ifp; 2065 struct inet6_ifaddr *ifp;
2066 2066
2067 read_lock_bh(&idev->lock); 2067 read_lock_bh(&idev->lock);
2068 list_for_each_entry(ifp, &idev->addr_list, if_list) { 2068 list_for_each_entry(ifp, &idev->addr_list, if_list) {
2069 if ((ifp->scope & IFA_LINK) && 2069 if ((ifp->scope & IFA_LINK) &&
2070 !(ifp->flags & IFA_F_TENTATIVE)) { 2070 !(ifp->flags & IFA_F_TENTATIVE)) {
2071 pkt_dev->cur_in6_saddr = ifp->addr; 2071 pkt_dev->cur_in6_saddr = ifp->addr;
2072 err = 0; 2072 err = 0;
2073 break; 2073 break;
2074 } 2074 }
2075 } 2075 }
2076 read_unlock_bh(&idev->lock); 2076 read_unlock_bh(&idev->lock);
2077 } 2077 }
2078 rcu_read_unlock(); 2078 rcu_read_unlock();
2079 if (err) 2079 if (err)
2080 pr_err("ERROR: IPv6 link address not available\n"); 2080 pr_err("ERROR: IPv6 link address not available\n");
2081 } 2081 }
2082 } else { 2082 } else {
2083 if (pkt_dev->min_pkt_size == 0) { 2083 if (pkt_dev->min_pkt_size == 0) {
2084 pkt_dev->min_pkt_size = 14 + sizeof(struct iphdr) 2084 pkt_dev->min_pkt_size = 14 + sizeof(struct iphdr)
2085 + sizeof(struct udphdr) 2085 + sizeof(struct udphdr)
2086 + sizeof(struct pktgen_hdr) 2086 + sizeof(struct pktgen_hdr)
2087 + pkt_dev->pkt_overhead; 2087 + pkt_dev->pkt_overhead;
2088 } 2088 }
2089 2089
2090 pkt_dev->saddr_min = 0; 2090 pkt_dev->saddr_min = 0;
2091 pkt_dev->saddr_max = 0; 2091 pkt_dev->saddr_max = 0;
2092 if (strlen(pkt_dev->src_min) == 0) { 2092 if (strlen(pkt_dev->src_min) == 0) {
2093 2093
2094 struct in_device *in_dev; 2094 struct in_device *in_dev;
2095 2095
2096 rcu_read_lock(); 2096 rcu_read_lock();
2097 in_dev = __in_dev_get_rcu(pkt_dev->odev); 2097 in_dev = __in_dev_get_rcu(pkt_dev->odev);
2098 if (in_dev) { 2098 if (in_dev) {
2099 if (in_dev->ifa_list) { 2099 if (in_dev->ifa_list) {
2100 pkt_dev->saddr_min = 2100 pkt_dev->saddr_min =
2101 in_dev->ifa_list->ifa_address; 2101 in_dev->ifa_list->ifa_address;
2102 pkt_dev->saddr_max = pkt_dev->saddr_min; 2102 pkt_dev->saddr_max = pkt_dev->saddr_min;
2103 } 2103 }
2104 } 2104 }
2105 rcu_read_unlock(); 2105 rcu_read_unlock();
2106 } else { 2106 } else {
2107 pkt_dev->saddr_min = in_aton(pkt_dev->src_min); 2107 pkt_dev->saddr_min = in_aton(pkt_dev->src_min);
2108 pkt_dev->saddr_max = in_aton(pkt_dev->src_max); 2108 pkt_dev->saddr_max = in_aton(pkt_dev->src_max);
2109 } 2109 }
2110 2110
2111 pkt_dev->daddr_min = in_aton(pkt_dev->dst_min); 2111 pkt_dev->daddr_min = in_aton(pkt_dev->dst_min);
2112 pkt_dev->daddr_max = in_aton(pkt_dev->dst_max); 2112 pkt_dev->daddr_max = in_aton(pkt_dev->dst_max);
2113 } 2113 }
2114 /* Initialize current values. */ 2114 /* Initialize current values. */
2115 pkt_dev->cur_pkt_size = pkt_dev->min_pkt_size; 2115 pkt_dev->cur_pkt_size = pkt_dev->min_pkt_size;
2116 if (pkt_dev->min_pkt_size > pkt_dev->max_pkt_size) 2116 if (pkt_dev->min_pkt_size > pkt_dev->max_pkt_size)
2117 pkt_dev->max_pkt_size = pkt_dev->min_pkt_size; 2117 pkt_dev->max_pkt_size = pkt_dev->min_pkt_size;
2118 2118
2119 pkt_dev->cur_dst_mac_offset = 0; 2119 pkt_dev->cur_dst_mac_offset = 0;
2120 pkt_dev->cur_src_mac_offset = 0; 2120 pkt_dev->cur_src_mac_offset = 0;
2121 pkt_dev->cur_saddr = pkt_dev->saddr_min; 2121 pkt_dev->cur_saddr = pkt_dev->saddr_min;
2122 pkt_dev->cur_daddr = pkt_dev->daddr_min; 2122 pkt_dev->cur_daddr = pkt_dev->daddr_min;
2123 pkt_dev->cur_udp_dst = pkt_dev->udp_dst_min; 2123 pkt_dev->cur_udp_dst = pkt_dev->udp_dst_min;
2124 pkt_dev->cur_udp_src = pkt_dev->udp_src_min; 2124 pkt_dev->cur_udp_src = pkt_dev->udp_src_min;
2125 pkt_dev->nflows = 0; 2125 pkt_dev->nflows = 0;
2126 } 2126 }
2127 2127
2128 2128
2129 static void spin(struct pktgen_dev *pkt_dev, ktime_t spin_until) 2129 static void spin(struct pktgen_dev *pkt_dev, ktime_t spin_until)
2130 { 2130 {
2131 ktime_t start_time, end_time; 2131 ktime_t start_time, end_time;
2132 s64 remaining; 2132 s64 remaining;
2133 struct hrtimer_sleeper t; 2133 struct hrtimer_sleeper t;
2134 2134
2135 hrtimer_init_on_stack(&t.timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS); 2135 hrtimer_init_on_stack(&t.timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
2136 hrtimer_set_expires(&t.timer, spin_until); 2136 hrtimer_set_expires(&t.timer, spin_until);
2137 2137
2138 remaining = ktime_to_ns(hrtimer_expires_remaining(&t.timer)); 2138 remaining = ktime_to_ns(hrtimer_expires_remaining(&t.timer));
2139 if (remaining <= 0) { 2139 if (remaining <= 0) {
2140 pkt_dev->next_tx = ktime_add_ns(spin_until, pkt_dev->delay); 2140 pkt_dev->next_tx = ktime_add_ns(spin_until, pkt_dev->delay);
2141 return; 2141 return;
2142 } 2142 }
2143 2143
2144 start_time = ktime_now(); 2144 start_time = ktime_now();
2145 if (remaining < 100000) { 2145 if (remaining < 100000) {
2146 /* for small delays (<100us), just loop until limit is reached */ 2146 /* for small delays (<100us), just loop until limit is reached */
2147 do { 2147 do {
2148 end_time = ktime_now(); 2148 end_time = ktime_now();
2149 } while (ktime_lt(end_time, spin_until)); 2149 } while (ktime_lt(end_time, spin_until));
2150 } else { 2150 } else {
2151 /* see do_nanosleep */ 2151 /* see do_nanosleep */
2152 hrtimer_init_sleeper(&t, current); 2152 hrtimer_init_sleeper(&t, current);
2153 do { 2153 do {
2154 set_current_state(TASK_INTERRUPTIBLE); 2154 set_current_state(TASK_INTERRUPTIBLE);
2155 hrtimer_start_expires(&t.timer, HRTIMER_MODE_ABS); 2155 hrtimer_start_expires(&t.timer, HRTIMER_MODE_ABS);
2156 if (!hrtimer_active(&t.timer)) 2156 if (!hrtimer_active(&t.timer))
2157 t.task = NULL; 2157 t.task = NULL;
2158 2158
2159 if (likely(t.task)) 2159 if (likely(t.task))
2160 schedule(); 2160 schedule();
2161 2161
2162 hrtimer_cancel(&t.timer); 2162 hrtimer_cancel(&t.timer);
2163 } while (t.task && pkt_dev->running && !signal_pending(current)); 2163 } while (t.task && pkt_dev->running && !signal_pending(current));
2164 __set_current_state(TASK_RUNNING); 2164 __set_current_state(TASK_RUNNING);
2165 end_time = ktime_now(); 2165 end_time = ktime_now();
2166 } 2166 }
2167 2167
2168 pkt_dev->idle_acc += ktime_to_ns(ktime_sub(end_time, start_time)); 2168 pkt_dev->idle_acc += ktime_to_ns(ktime_sub(end_time, start_time));
2169 pkt_dev->next_tx = ktime_add_ns(spin_until, pkt_dev->delay); 2169 pkt_dev->next_tx = ktime_add_ns(spin_until, pkt_dev->delay);
2170 } 2170 }
2171 2171
2172 static inline void set_pkt_overhead(struct pktgen_dev *pkt_dev) 2172 static inline void set_pkt_overhead(struct pktgen_dev *pkt_dev)
2173 { 2173 {
2174 pkt_dev->pkt_overhead = 0; 2174 pkt_dev->pkt_overhead = 0;
2175 pkt_dev->pkt_overhead += pkt_dev->nr_labels*sizeof(u32); 2175 pkt_dev->pkt_overhead += pkt_dev->nr_labels*sizeof(u32);
2176 pkt_dev->pkt_overhead += VLAN_TAG_SIZE(pkt_dev); 2176 pkt_dev->pkt_overhead += VLAN_TAG_SIZE(pkt_dev);
2177 pkt_dev->pkt_overhead += SVLAN_TAG_SIZE(pkt_dev); 2177 pkt_dev->pkt_overhead += SVLAN_TAG_SIZE(pkt_dev);
2178 } 2178 }
2179 2179
2180 static inline int f_seen(const struct pktgen_dev *pkt_dev, int flow) 2180 static inline int f_seen(const struct pktgen_dev *pkt_dev, int flow)
2181 { 2181 {
2182 return !!(pkt_dev->flows[flow].flags & F_INIT); 2182 return !!(pkt_dev->flows[flow].flags & F_INIT);
2183 } 2183 }
2184 2184
2185 static inline int f_pick(struct pktgen_dev *pkt_dev) 2185 static inline int f_pick(struct pktgen_dev *pkt_dev)
2186 { 2186 {
2187 int flow = pkt_dev->curfl; 2187 int flow = pkt_dev->curfl;
2188 2188
2189 if (pkt_dev->flags & F_FLOW_SEQ) { 2189 if (pkt_dev->flags & F_FLOW_SEQ) {
2190 if (pkt_dev->flows[flow].count >= pkt_dev->lflow) { 2190 if (pkt_dev->flows[flow].count >= pkt_dev->lflow) {
2191 /* reset time */ 2191 /* reset time */
2192 pkt_dev->flows[flow].count = 0; 2192 pkt_dev->flows[flow].count = 0;
2193 pkt_dev->flows[flow].flags = 0; 2193 pkt_dev->flows[flow].flags = 0;
2194 pkt_dev->curfl += 1; 2194 pkt_dev->curfl += 1;
2195 if (pkt_dev->curfl >= pkt_dev->cflows) 2195 if (pkt_dev->curfl >= pkt_dev->cflows)
2196 pkt_dev->curfl = 0; /*reset */ 2196 pkt_dev->curfl = 0; /*reset */
2197 } 2197 }
2198 } else { 2198 } else {
2199 flow = random32() % pkt_dev->cflows; 2199 flow = random32() % pkt_dev->cflows;
2200 pkt_dev->curfl = flow; 2200 pkt_dev->curfl = flow;
2201 2201
2202 if (pkt_dev->flows[flow].count > pkt_dev->lflow) { 2202 if (pkt_dev->flows[flow].count > pkt_dev->lflow) {
2203 pkt_dev->flows[flow].count = 0; 2203 pkt_dev->flows[flow].count = 0;
2204 pkt_dev->flows[flow].flags = 0; 2204 pkt_dev->flows[flow].flags = 0;
2205 } 2205 }
2206 } 2206 }
2207 2207
2208 return pkt_dev->curfl; 2208 return pkt_dev->curfl;
2209 } 2209 }
2210 2210
2211 2211
2212 #ifdef CONFIG_XFRM 2212 #ifdef CONFIG_XFRM
2213 /* If there was already an IPSEC SA, we keep it as is, else 2213 /* If there was already an IPSEC SA, we keep it as is, else
2214 * we go look for it ... 2214 * we go look for it ...
2215 */ 2215 */
2216 #define DUMMY_MARK 0 2216 #define DUMMY_MARK 0
2217 static void get_ipsec_sa(struct pktgen_dev *pkt_dev, int flow) 2217 static void get_ipsec_sa(struct pktgen_dev *pkt_dev, int flow)
2218 { 2218 {
2219 struct xfrm_state *x = pkt_dev->flows[flow].x; 2219 struct xfrm_state *x = pkt_dev->flows[flow].x;
2220 if (!x) { 2220 if (!x) {
2221 /*slow path: we dont already have xfrm_state*/ 2221 /*slow path: we dont already have xfrm_state*/
2222 x = xfrm_stateonly_find(&init_net, DUMMY_MARK, 2222 x = xfrm_stateonly_find(&init_net, DUMMY_MARK,
2223 (xfrm_address_t *)&pkt_dev->cur_daddr, 2223 (xfrm_address_t *)&pkt_dev->cur_daddr,
2224 (xfrm_address_t *)&pkt_dev->cur_saddr, 2224 (xfrm_address_t *)&pkt_dev->cur_saddr,
2225 AF_INET, 2225 AF_INET,
2226 pkt_dev->ipsmode, 2226 pkt_dev->ipsmode,
2227 pkt_dev->ipsproto, 0); 2227 pkt_dev->ipsproto, 0);
2228 if (x) { 2228 if (x) {
2229 pkt_dev->flows[flow].x = x; 2229 pkt_dev->flows[flow].x = x;
2230 set_pkt_overhead(pkt_dev); 2230 set_pkt_overhead(pkt_dev);
2231 pkt_dev->pkt_overhead += x->props.header_len; 2231 pkt_dev->pkt_overhead += x->props.header_len;
2232 } 2232 }
2233 2233
2234 } 2234 }
2235 } 2235 }
2236 #endif 2236 #endif
2237 static void set_cur_queue_map(struct pktgen_dev *pkt_dev) 2237 static void set_cur_queue_map(struct pktgen_dev *pkt_dev)
2238 { 2238 {
2239 2239
2240 if (pkt_dev->flags & F_QUEUE_MAP_CPU) 2240 if (pkt_dev->flags & F_QUEUE_MAP_CPU)
2241 pkt_dev->cur_queue_map = smp_processor_id(); 2241 pkt_dev->cur_queue_map = smp_processor_id();
2242 2242
2243 else if (pkt_dev->queue_map_min <= pkt_dev->queue_map_max) { 2243 else if (pkt_dev->queue_map_min <= pkt_dev->queue_map_max) {
2244 __u16 t; 2244 __u16 t;
2245 if (pkt_dev->flags & F_QUEUE_MAP_RND) { 2245 if (pkt_dev->flags & F_QUEUE_MAP_RND) {
2246 t = random32() % 2246 t = random32() %
2247 (pkt_dev->queue_map_max - 2247 (pkt_dev->queue_map_max -
2248 pkt_dev->queue_map_min + 1) 2248 pkt_dev->queue_map_min + 1)
2249 + pkt_dev->queue_map_min; 2249 + pkt_dev->queue_map_min;
2250 } else { 2250 } else {
2251 t = pkt_dev->cur_queue_map + 1; 2251 t = pkt_dev->cur_queue_map + 1;
2252 if (t > pkt_dev->queue_map_max) 2252 if (t > pkt_dev->queue_map_max)
2253 t = pkt_dev->queue_map_min; 2253 t = pkt_dev->queue_map_min;
2254 } 2254 }
2255 pkt_dev->cur_queue_map = t; 2255 pkt_dev->cur_queue_map = t;
2256 } 2256 }
2257 pkt_dev->cur_queue_map = pkt_dev->cur_queue_map % pkt_dev->odev->real_num_tx_queues; 2257 pkt_dev->cur_queue_map = pkt_dev->cur_queue_map % pkt_dev->odev->real_num_tx_queues;
2258 } 2258 }
2259 2259
2260 /* Increment/randomize headers according to flags and current values 2260 /* Increment/randomize headers according to flags and current values
2261 * for IP src/dest, UDP src/dst port, MAC-Addr src/dst 2261 * for IP src/dest, UDP src/dst port, MAC-Addr src/dst
2262 */ 2262 */
2263 static void mod_cur_headers(struct pktgen_dev *pkt_dev) 2263 static void mod_cur_headers(struct pktgen_dev *pkt_dev)
2264 { 2264 {
2265 __u32 imn; 2265 __u32 imn;
2266 __u32 imx; 2266 __u32 imx;
2267 int flow = 0; 2267 int flow = 0;
2268 2268
2269 if (pkt_dev->cflows) 2269 if (pkt_dev->cflows)
2270 flow = f_pick(pkt_dev); 2270 flow = f_pick(pkt_dev);
2271 2271
2272 /* Deal with source MAC */ 2272 /* Deal with source MAC */
2273 if (pkt_dev->src_mac_count > 1) { 2273 if (pkt_dev->src_mac_count > 1) {
2274 __u32 mc; 2274 __u32 mc;
2275 __u32 tmp; 2275 __u32 tmp;
2276 2276
2277 if (pkt_dev->flags & F_MACSRC_RND) 2277 if (pkt_dev->flags & F_MACSRC_RND)
2278 mc = random32() % pkt_dev->src_mac_count; 2278 mc = random32() % pkt_dev->src_mac_count;
2279 else { 2279 else {
2280 mc = pkt_dev->cur_src_mac_offset++; 2280 mc = pkt_dev->cur_src_mac_offset++;
2281 if (pkt_dev->cur_src_mac_offset >= 2281 if (pkt_dev->cur_src_mac_offset >=
2282 pkt_dev->src_mac_count) 2282 pkt_dev->src_mac_count)
2283 pkt_dev->cur_src_mac_offset = 0; 2283 pkt_dev->cur_src_mac_offset = 0;
2284 } 2284 }
2285 2285
2286 tmp = pkt_dev->src_mac[5] + (mc & 0xFF); 2286 tmp = pkt_dev->src_mac[5] + (mc & 0xFF);
2287 pkt_dev->hh[11] = tmp; 2287 pkt_dev->hh[11] = tmp;
2288 tmp = (pkt_dev->src_mac[4] + ((mc >> 8) & 0xFF) + (tmp >> 8)); 2288 tmp = (pkt_dev->src_mac[4] + ((mc >> 8) & 0xFF) + (tmp >> 8));
2289 pkt_dev->hh[10] = tmp; 2289 pkt_dev->hh[10] = tmp;
2290 tmp = (pkt_dev->src_mac[3] + ((mc >> 16) & 0xFF) + (tmp >> 8)); 2290 tmp = (pkt_dev->src_mac[3] + ((mc >> 16) & 0xFF) + (tmp >> 8));
2291 pkt_dev->hh[9] = tmp; 2291 pkt_dev->hh[9] = tmp;
2292 tmp = (pkt_dev->src_mac[2] + ((mc >> 24) & 0xFF) + (tmp >> 8)); 2292 tmp = (pkt_dev->src_mac[2] + ((mc >> 24) & 0xFF) + (tmp >> 8));
2293 pkt_dev->hh[8] = tmp; 2293 pkt_dev->hh[8] = tmp;
2294 tmp = (pkt_dev->src_mac[1] + (tmp >> 8)); 2294 tmp = (pkt_dev->src_mac[1] + (tmp >> 8));
2295 pkt_dev->hh[7] = tmp; 2295 pkt_dev->hh[7] = tmp;
2296 } 2296 }
2297 2297
2298 /* Deal with Destination MAC */ 2298 /* Deal with Destination MAC */
2299 if (pkt_dev->dst_mac_count > 1) { 2299 if (pkt_dev->dst_mac_count > 1) {
2300 __u32 mc; 2300 __u32 mc;
2301 __u32 tmp; 2301 __u32 tmp;
2302 2302
2303 if (pkt_dev->flags & F_MACDST_RND) 2303 if (pkt_dev->flags & F_MACDST_RND)
2304 mc = random32() % pkt_dev->dst_mac_count; 2304 mc = random32() % pkt_dev->dst_mac_count;
2305 2305
2306 else { 2306 else {
2307 mc = pkt_dev->cur_dst_mac_offset++; 2307 mc = pkt_dev->cur_dst_mac_offset++;
2308 if (pkt_dev->cur_dst_mac_offset >= 2308 if (pkt_dev->cur_dst_mac_offset >=
2309 pkt_dev->dst_mac_count) { 2309 pkt_dev->dst_mac_count) {
2310 pkt_dev->cur_dst_mac_offset = 0; 2310 pkt_dev->cur_dst_mac_offset = 0;
2311 } 2311 }
2312 } 2312 }
2313 2313
2314 tmp = pkt_dev->dst_mac[5] + (mc & 0xFF); 2314 tmp = pkt_dev->dst_mac[5] + (mc & 0xFF);
2315 pkt_dev->hh[5] = tmp; 2315 pkt_dev->hh[5] = tmp;
2316 tmp = (pkt_dev->dst_mac[4] + ((mc >> 8) & 0xFF) + (tmp >> 8)); 2316 tmp = (pkt_dev->dst_mac[4] + ((mc >> 8) & 0xFF) + (tmp >> 8));
2317 pkt_dev->hh[4] = tmp; 2317 pkt_dev->hh[4] = tmp;
2318 tmp = (pkt_dev->dst_mac[3] + ((mc >> 16) & 0xFF) + (tmp >> 8)); 2318 tmp = (pkt_dev->dst_mac[3] + ((mc >> 16) & 0xFF) + (tmp >> 8));
2319 pkt_dev->hh[3] = tmp; 2319 pkt_dev->hh[3] = tmp;
2320 tmp = (pkt_dev->dst_mac[2] + ((mc >> 24) & 0xFF) + (tmp >> 8)); 2320 tmp = (pkt_dev->dst_mac[2] + ((mc >> 24) & 0xFF) + (tmp >> 8));
2321 pkt_dev->hh[2] = tmp; 2321 pkt_dev->hh[2] = tmp;
2322 tmp = (pkt_dev->dst_mac[1] + (tmp >> 8)); 2322 tmp = (pkt_dev->dst_mac[1] + (tmp >> 8));
2323 pkt_dev->hh[1] = tmp; 2323 pkt_dev->hh[1] = tmp;
2324 } 2324 }
2325 2325
2326 if (pkt_dev->flags & F_MPLS_RND) { 2326 if (pkt_dev->flags & F_MPLS_RND) {
2327 unsigned int i; 2327 unsigned int i;
2328 for (i = 0; i < pkt_dev->nr_labels; i++) 2328 for (i = 0; i < pkt_dev->nr_labels; i++)
2329 if (pkt_dev->labels[i] & MPLS_STACK_BOTTOM) 2329 if (pkt_dev->labels[i] & MPLS_STACK_BOTTOM)
2330 pkt_dev->labels[i] = MPLS_STACK_BOTTOM | 2330 pkt_dev->labels[i] = MPLS_STACK_BOTTOM |
2331 ((__force __be32)random32() & 2331 ((__force __be32)random32() &
2332 htonl(0x000fffff)); 2332 htonl(0x000fffff));
2333 } 2333 }
2334 2334
2335 if ((pkt_dev->flags & F_VID_RND) && (pkt_dev->vlan_id != 0xffff)) { 2335 if ((pkt_dev->flags & F_VID_RND) && (pkt_dev->vlan_id != 0xffff)) {
2336 pkt_dev->vlan_id = random32() & (4096-1); 2336 pkt_dev->vlan_id = random32() & (4096-1);
2337 } 2337 }
2338 2338
2339 if ((pkt_dev->flags & F_SVID_RND) && (pkt_dev->svlan_id != 0xffff)) { 2339 if ((pkt_dev->flags & F_SVID_RND) && (pkt_dev->svlan_id != 0xffff)) {
2340 pkt_dev->svlan_id = random32() & (4096 - 1); 2340 pkt_dev->svlan_id = random32() & (4096 - 1);
2341 } 2341 }
2342 2342
2343 if (pkt_dev->udp_src_min < pkt_dev->udp_src_max) { 2343 if (pkt_dev->udp_src_min < pkt_dev->udp_src_max) {
2344 if (pkt_dev->flags & F_UDPSRC_RND) 2344 if (pkt_dev->flags & F_UDPSRC_RND)
2345 pkt_dev->cur_udp_src = random32() % 2345 pkt_dev->cur_udp_src = random32() %
2346 (pkt_dev->udp_src_max - pkt_dev->udp_src_min) 2346 (pkt_dev->udp_src_max - pkt_dev->udp_src_min)
2347 + pkt_dev->udp_src_min; 2347 + pkt_dev->udp_src_min;
2348 2348
2349 else { 2349 else {
2350 pkt_dev->cur_udp_src++; 2350 pkt_dev->cur_udp_src++;
2351 if (pkt_dev->cur_udp_src >= pkt_dev->udp_src_max) 2351 if (pkt_dev->cur_udp_src >= pkt_dev->udp_src_max)
2352 pkt_dev->cur_udp_src = pkt_dev->udp_src_min; 2352 pkt_dev->cur_udp_src = pkt_dev->udp_src_min;
2353 } 2353 }
2354 } 2354 }
2355 2355
2356 if (pkt_dev->udp_dst_min < pkt_dev->udp_dst_max) { 2356 if (pkt_dev->udp_dst_min < pkt_dev->udp_dst_max) {
2357 if (pkt_dev->flags & F_UDPDST_RND) { 2357 if (pkt_dev->flags & F_UDPDST_RND) {
2358 pkt_dev->cur_udp_dst = random32() % 2358 pkt_dev->cur_udp_dst = random32() %
2359 (pkt_dev->udp_dst_max - pkt_dev->udp_dst_min) 2359 (pkt_dev->udp_dst_max - pkt_dev->udp_dst_min)
2360 + pkt_dev->udp_dst_min; 2360 + pkt_dev->udp_dst_min;
2361 } else { 2361 } else {
2362 pkt_dev->cur_udp_dst++; 2362 pkt_dev->cur_udp_dst++;
2363 if (pkt_dev->cur_udp_dst >= pkt_dev->udp_dst_max) 2363 if (pkt_dev->cur_udp_dst >= pkt_dev->udp_dst_max)
2364 pkt_dev->cur_udp_dst = pkt_dev->udp_dst_min; 2364 pkt_dev->cur_udp_dst = pkt_dev->udp_dst_min;
2365 } 2365 }
2366 } 2366 }
2367 2367
2368 if (!(pkt_dev->flags & F_IPV6)) { 2368 if (!(pkt_dev->flags & F_IPV6)) {
2369 2369
2370 imn = ntohl(pkt_dev->saddr_min); 2370 imn = ntohl(pkt_dev->saddr_min);
2371 imx = ntohl(pkt_dev->saddr_max); 2371 imx = ntohl(pkt_dev->saddr_max);
2372 if (imn < imx) { 2372 if (imn < imx) {
2373 __u32 t; 2373 __u32 t;
2374 if (pkt_dev->flags & F_IPSRC_RND) 2374 if (pkt_dev->flags & F_IPSRC_RND)
2375 t = random32() % (imx - imn) + imn; 2375 t = random32() % (imx - imn) + imn;
2376 else { 2376 else {
2377 t = ntohl(pkt_dev->cur_saddr); 2377 t = ntohl(pkt_dev->cur_saddr);
2378 t++; 2378 t++;
2379 if (t > imx) 2379 if (t > imx)
2380 t = imn; 2380 t = imn;
2381 2381
2382 } 2382 }
2383 pkt_dev->cur_saddr = htonl(t); 2383 pkt_dev->cur_saddr = htonl(t);
2384 } 2384 }
2385 2385
2386 if (pkt_dev->cflows && f_seen(pkt_dev, flow)) { 2386 if (pkt_dev->cflows && f_seen(pkt_dev, flow)) {
2387 pkt_dev->cur_daddr = pkt_dev->flows[flow].cur_daddr; 2387 pkt_dev->cur_daddr = pkt_dev->flows[flow].cur_daddr;
2388 } else { 2388 } else {
2389 imn = ntohl(pkt_dev->daddr_min); 2389 imn = ntohl(pkt_dev->daddr_min);
2390 imx = ntohl(pkt_dev->daddr_max); 2390 imx = ntohl(pkt_dev->daddr_max);
2391 if (imn < imx) { 2391 if (imn < imx) {
2392 __u32 t; 2392 __u32 t;
2393 __be32 s; 2393 __be32 s;
2394 if (pkt_dev->flags & F_IPDST_RND) { 2394 if (pkt_dev->flags & F_IPDST_RND) {
2395 2395
2396 t = random32() % (imx - imn) + imn; 2396 t = random32() % (imx - imn) + imn;
2397 s = htonl(t); 2397 s = htonl(t);
2398 2398
2399 while (ipv4_is_loopback(s) || 2399 while (ipv4_is_loopback(s) ||
2400 ipv4_is_multicast(s) || 2400 ipv4_is_multicast(s) ||
2401 ipv4_is_lbcast(s) || 2401 ipv4_is_lbcast(s) ||
2402 ipv4_is_zeronet(s) || 2402 ipv4_is_zeronet(s) ||
2403 ipv4_is_local_multicast(s)) { 2403 ipv4_is_local_multicast(s)) {
2404 t = random32() % (imx - imn) + imn; 2404 t = random32() % (imx - imn) + imn;
2405 s = htonl(t); 2405 s = htonl(t);
2406 } 2406 }
2407 pkt_dev->cur_daddr = s; 2407 pkt_dev->cur_daddr = s;
2408 } else { 2408 } else {
2409 t = ntohl(pkt_dev->cur_daddr); 2409 t = ntohl(pkt_dev->cur_daddr);
2410 t++; 2410 t++;
2411 if (t > imx) { 2411 if (t > imx) {
2412 t = imn; 2412 t = imn;
2413 } 2413 }
2414 pkt_dev->cur_daddr = htonl(t); 2414 pkt_dev->cur_daddr = htonl(t);
2415 } 2415 }
2416 } 2416 }
2417 if (pkt_dev->cflows) { 2417 if (pkt_dev->cflows) {
2418 pkt_dev->flows[flow].flags |= F_INIT; 2418 pkt_dev->flows[flow].flags |= F_INIT;
2419 pkt_dev->flows[flow].cur_daddr = 2419 pkt_dev->flows[flow].cur_daddr =
2420 pkt_dev->cur_daddr; 2420 pkt_dev->cur_daddr;
2421 #ifdef CONFIG_XFRM 2421 #ifdef CONFIG_XFRM
2422 if (pkt_dev->flags & F_IPSEC_ON) 2422 if (pkt_dev->flags & F_IPSEC_ON)
2423 get_ipsec_sa(pkt_dev, flow); 2423 get_ipsec_sa(pkt_dev, flow);
2424 #endif 2424 #endif
2425 pkt_dev->nflows++; 2425 pkt_dev->nflows++;
2426 } 2426 }
2427 } 2427 }
2428 } else { /* IPV6 * */ 2428 } else { /* IPV6 * */
2429 2429
2430 if (pkt_dev->min_in6_daddr.s6_addr32[0] == 0 && 2430 if (!ipv6_addr_any(&pkt_dev->min_in6_daddr)) {
2431 pkt_dev->min_in6_daddr.s6_addr32[1] == 0 &&
2432 pkt_dev->min_in6_daddr.s6_addr32[2] == 0 &&
2433 pkt_dev->min_in6_daddr.s6_addr32[3] == 0) ;
2434 else {
2435 int i; 2431 int i;
2436 2432
2437 /* Only random destinations yet */ 2433 /* Only random destinations yet */
2438 2434
2439 for (i = 0; i < 4; i++) { 2435 for (i = 0; i < 4; i++) {
2440 pkt_dev->cur_in6_daddr.s6_addr32[i] = 2436 pkt_dev->cur_in6_daddr.s6_addr32[i] =
2441 (((__force __be32)random32() | 2437 (((__force __be32)random32() |
2442 pkt_dev->min_in6_daddr.s6_addr32[i]) & 2438 pkt_dev->min_in6_daddr.s6_addr32[i]) &
2443 pkt_dev->max_in6_daddr.s6_addr32[i]); 2439 pkt_dev->max_in6_daddr.s6_addr32[i]);
2444 } 2440 }
2445 } 2441 }
2446 } 2442 }
2447 2443
2448 if (pkt_dev->min_pkt_size < pkt_dev->max_pkt_size) { 2444 if (pkt_dev->min_pkt_size < pkt_dev->max_pkt_size) {
2449 __u32 t; 2445 __u32 t;
2450 if (pkt_dev->flags & F_TXSIZE_RND) { 2446 if (pkt_dev->flags & F_TXSIZE_RND) {
2451 t = random32() % 2447 t = random32() %
2452 (pkt_dev->max_pkt_size - pkt_dev->min_pkt_size) 2448 (pkt_dev->max_pkt_size - pkt_dev->min_pkt_size)
2453 + pkt_dev->min_pkt_size; 2449 + pkt_dev->min_pkt_size;
2454 } else { 2450 } else {
2455 t = pkt_dev->cur_pkt_size + 1; 2451 t = pkt_dev->cur_pkt_size + 1;
2456 if (t > pkt_dev->max_pkt_size) 2452 if (t > pkt_dev->max_pkt_size)
2457 t = pkt_dev->min_pkt_size; 2453 t = pkt_dev->min_pkt_size;
2458 } 2454 }
2459 pkt_dev->cur_pkt_size = t; 2455 pkt_dev->cur_pkt_size = t;
2460 } 2456 }
2461 2457
2462 set_cur_queue_map(pkt_dev); 2458 set_cur_queue_map(pkt_dev);
2463 2459
2464 pkt_dev->flows[flow].count++; 2460 pkt_dev->flows[flow].count++;
2465 } 2461 }
2466 2462
2467 2463
2468 #ifdef CONFIG_XFRM 2464 #ifdef CONFIG_XFRM
2469 static int pktgen_output_ipsec(struct sk_buff *skb, struct pktgen_dev *pkt_dev) 2465 static int pktgen_output_ipsec(struct sk_buff *skb, struct pktgen_dev *pkt_dev)
2470 { 2466 {
2471 struct xfrm_state *x = pkt_dev->flows[pkt_dev->curfl].x; 2467 struct xfrm_state *x = pkt_dev->flows[pkt_dev->curfl].x;
2472 int err = 0; 2468 int err = 0;
2473 2469
2474 if (!x) 2470 if (!x)
2475 return 0; 2471 return 0;
2476 /* XXX: we dont support tunnel mode for now until 2472 /* XXX: we dont support tunnel mode for now until
2477 * we resolve the dst issue */ 2473 * we resolve the dst issue */
2478 if (x->props.mode != XFRM_MODE_TRANSPORT) 2474 if (x->props.mode != XFRM_MODE_TRANSPORT)
2479 return 0; 2475 return 0;
2480 2476
2481 spin_lock(&x->lock); 2477 spin_lock(&x->lock);
2482 2478
2483 err = x->outer_mode->output(x, skb); 2479 err = x->outer_mode->output(x, skb);
2484 if (err) 2480 if (err)
2485 goto error; 2481 goto error;
2486 err = x->type->output(x, skb); 2482 err = x->type->output(x, skb);
2487 if (err) 2483 if (err)
2488 goto error; 2484 goto error;
2489 2485
2490 x->curlft.bytes += skb->len; 2486 x->curlft.bytes += skb->len;
2491 x->curlft.packets++; 2487 x->curlft.packets++;
2492 error: 2488 error:
2493 spin_unlock(&x->lock); 2489 spin_unlock(&x->lock);
2494 return err; 2490 return err;
2495 } 2491 }
2496 2492
2497 static void free_SAs(struct pktgen_dev *pkt_dev) 2493 static void free_SAs(struct pktgen_dev *pkt_dev)
2498 { 2494 {
2499 if (pkt_dev->cflows) { 2495 if (pkt_dev->cflows) {
2500 /* let go of the SAs if we have them */ 2496 /* let go of the SAs if we have them */
2501 int i; 2497 int i;
2502 for (i = 0; i < pkt_dev->cflows; i++) { 2498 for (i = 0; i < pkt_dev->cflows; i++) {
2503 struct xfrm_state *x = pkt_dev->flows[i].x; 2499 struct xfrm_state *x = pkt_dev->flows[i].x;
2504 if (x) { 2500 if (x) {
2505 xfrm_state_put(x); 2501 xfrm_state_put(x);
2506 pkt_dev->flows[i].x = NULL; 2502 pkt_dev->flows[i].x = NULL;
2507 } 2503 }
2508 } 2504 }
2509 } 2505 }
2510 } 2506 }
2511 2507
2512 static int process_ipsec(struct pktgen_dev *pkt_dev, 2508 static int process_ipsec(struct pktgen_dev *pkt_dev,
2513 struct sk_buff *skb, __be16 protocol) 2509 struct sk_buff *skb, __be16 protocol)
2514 { 2510 {
2515 if (pkt_dev->flags & F_IPSEC_ON) { 2511 if (pkt_dev->flags & F_IPSEC_ON) {
2516 struct xfrm_state *x = pkt_dev->flows[pkt_dev->curfl].x; 2512 struct xfrm_state *x = pkt_dev->flows[pkt_dev->curfl].x;
2517 int nhead = 0; 2513 int nhead = 0;
2518 if (x) { 2514 if (x) {
2519 int ret; 2515 int ret;
2520 __u8 *eth; 2516 __u8 *eth;
2521 nhead = x->props.header_len - skb_headroom(skb); 2517 nhead = x->props.header_len - skb_headroom(skb);
2522 if (nhead > 0) { 2518 if (nhead > 0) {
2523 ret = pskb_expand_head(skb, nhead, 0, GFP_ATOMIC); 2519 ret = pskb_expand_head(skb, nhead, 0, GFP_ATOMIC);
2524 if (ret < 0) { 2520 if (ret < 0) {
2525 pr_err("Error expanding ipsec packet %d\n", 2521 pr_err("Error expanding ipsec packet %d\n",
2526 ret); 2522 ret);
2527 goto err; 2523 goto err;
2528 } 2524 }
2529 } 2525 }
2530 2526
2531 /* ipsec is not expecting ll header */ 2527 /* ipsec is not expecting ll header */
2532 skb_pull(skb, ETH_HLEN); 2528 skb_pull(skb, ETH_HLEN);
2533 ret = pktgen_output_ipsec(skb, pkt_dev); 2529 ret = pktgen_output_ipsec(skb, pkt_dev);
2534 if (ret) { 2530 if (ret) {
2535 pr_err("Error creating ipsec packet %d\n", ret); 2531 pr_err("Error creating ipsec packet %d\n", ret);
2536 goto err; 2532 goto err;
2537 } 2533 }
2538 /* restore ll */ 2534 /* restore ll */
2539 eth = (__u8 *) skb_push(skb, ETH_HLEN); 2535 eth = (__u8 *) skb_push(skb, ETH_HLEN);
2540 memcpy(eth, pkt_dev->hh, 12); 2536 memcpy(eth, pkt_dev->hh, 12);
2541 *(u16 *) &eth[12] = protocol; 2537 *(u16 *) &eth[12] = protocol;
2542 } 2538 }
2543 } 2539 }
2544 return 1; 2540 return 1;
2545 err: 2541 err:
2546 kfree_skb(skb); 2542 kfree_skb(skb);
2547 return 0; 2543 return 0;
2548 } 2544 }
2549 #endif 2545 #endif
2550 2546
2551 static void mpls_push(__be32 *mpls, struct pktgen_dev *pkt_dev) 2547 static void mpls_push(__be32 *mpls, struct pktgen_dev *pkt_dev)
2552 { 2548 {
2553 unsigned int i; 2549 unsigned int i;
2554 for (i = 0; i < pkt_dev->nr_labels; i++) 2550 for (i = 0; i < pkt_dev->nr_labels; i++)
2555 *mpls++ = pkt_dev->labels[i] & ~MPLS_STACK_BOTTOM; 2551 *mpls++ = pkt_dev->labels[i] & ~MPLS_STACK_BOTTOM;
2556 2552
2557 mpls--; 2553 mpls--;
2558 *mpls |= MPLS_STACK_BOTTOM; 2554 *mpls |= MPLS_STACK_BOTTOM;
2559 } 2555 }
2560 2556
2561 static inline __be16 build_tci(unsigned int id, unsigned int cfi, 2557 static inline __be16 build_tci(unsigned int id, unsigned int cfi,
2562 unsigned int prio) 2558 unsigned int prio)
2563 { 2559 {
2564 return htons(id | (cfi << 12) | (prio << 13)); 2560 return htons(id | (cfi << 12) | (prio << 13));
2565 } 2561 }
2566 2562
2567 static void pktgen_finalize_skb(struct pktgen_dev *pkt_dev, struct sk_buff *skb, 2563 static void pktgen_finalize_skb(struct pktgen_dev *pkt_dev, struct sk_buff *skb,
2568 int datalen) 2564 int datalen)
2569 { 2565 {
2570 struct timeval timestamp; 2566 struct timeval timestamp;
2571 struct pktgen_hdr *pgh; 2567 struct pktgen_hdr *pgh;
2572 2568
2573 pgh = (struct pktgen_hdr *)skb_put(skb, sizeof(*pgh)); 2569 pgh = (struct pktgen_hdr *)skb_put(skb, sizeof(*pgh));
2574 datalen -= sizeof(*pgh); 2570 datalen -= sizeof(*pgh);
2575 2571
2576 if (pkt_dev->nfrags <= 0) { 2572 if (pkt_dev->nfrags <= 0) {
2577 memset(skb_put(skb, datalen), 0, datalen); 2573 memset(skb_put(skb, datalen), 0, datalen);
2578 } else { 2574 } else {
2579 int frags = pkt_dev->nfrags; 2575 int frags = pkt_dev->nfrags;
2580 int i, len; 2576 int i, len;
2581 int frag_len; 2577 int frag_len;
2582 2578
2583 2579
2584 if (frags > MAX_SKB_FRAGS) 2580 if (frags > MAX_SKB_FRAGS)
2585 frags = MAX_SKB_FRAGS; 2581 frags = MAX_SKB_FRAGS;
2586 len = datalen - frags * PAGE_SIZE; 2582 len = datalen - frags * PAGE_SIZE;
2587 if (len > 0) { 2583 if (len > 0) {
2588 memset(skb_put(skb, len), 0, len); 2584 memset(skb_put(skb, len), 0, len);
2589 datalen = frags * PAGE_SIZE; 2585 datalen = frags * PAGE_SIZE;
2590 } 2586 }
2591 2587
2592 i = 0; 2588 i = 0;
2593 frag_len = (datalen/frags) < PAGE_SIZE ? 2589 frag_len = (datalen/frags) < PAGE_SIZE ?
2594 (datalen/frags) : PAGE_SIZE; 2590 (datalen/frags) : PAGE_SIZE;
2595 while (datalen > 0) { 2591 while (datalen > 0) {
2596 if (unlikely(!pkt_dev->page)) { 2592 if (unlikely(!pkt_dev->page)) {
2597 int node = numa_node_id(); 2593 int node = numa_node_id();
2598 2594
2599 if (pkt_dev->node >= 0 && (pkt_dev->flags & F_NODE)) 2595 if (pkt_dev->node >= 0 && (pkt_dev->flags & F_NODE))
2600 node = pkt_dev->node; 2596 node = pkt_dev->node;
2601 pkt_dev->page = alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0); 2597 pkt_dev->page = alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0);
2602 if (!pkt_dev->page) 2598 if (!pkt_dev->page)
2603 break; 2599 break;
2604 } 2600 }
2605 get_page(pkt_dev->page); 2601 get_page(pkt_dev->page);
2606 skb_frag_set_page(skb, i, pkt_dev->page); 2602 skb_frag_set_page(skb, i, pkt_dev->page);
2607 skb_shinfo(skb)->frags[i].page_offset = 0; 2603 skb_shinfo(skb)->frags[i].page_offset = 0;
2608 /*last fragment, fill rest of data*/ 2604 /*last fragment, fill rest of data*/
2609 if (i == (frags - 1)) 2605 if (i == (frags - 1))
2610 skb_frag_size_set(&skb_shinfo(skb)->frags[i], 2606 skb_frag_size_set(&skb_shinfo(skb)->frags[i],
2611 (datalen < PAGE_SIZE ? datalen : PAGE_SIZE)); 2607 (datalen < PAGE_SIZE ? datalen : PAGE_SIZE));
2612 else 2608 else
2613 skb_frag_size_set(&skb_shinfo(skb)->frags[i], frag_len); 2609 skb_frag_size_set(&skb_shinfo(skb)->frags[i], frag_len);
2614 datalen -= skb_frag_size(&skb_shinfo(skb)->frags[i]); 2610 datalen -= skb_frag_size(&skb_shinfo(skb)->frags[i]);
2615 skb->len += skb_frag_size(&skb_shinfo(skb)->frags[i]); 2611 skb->len += skb_frag_size(&skb_shinfo(skb)->frags[i]);
2616 skb->data_len += skb_frag_size(&skb_shinfo(skb)->frags[i]); 2612 skb->data_len += skb_frag_size(&skb_shinfo(skb)->frags[i]);
2617 i++; 2613 i++;
2618 skb_shinfo(skb)->nr_frags = i; 2614 skb_shinfo(skb)->nr_frags = i;
2619 } 2615 }
2620 } 2616 }
2621 2617
2622 /* Stamp the time, and sequence number, 2618 /* Stamp the time, and sequence number,
2623 * convert them to network byte order 2619 * convert them to network byte order
2624 */ 2620 */
2625 pgh->pgh_magic = htonl(PKTGEN_MAGIC); 2621 pgh->pgh_magic = htonl(PKTGEN_MAGIC);
2626 pgh->seq_num = htonl(pkt_dev->seq_num); 2622 pgh->seq_num = htonl(pkt_dev->seq_num);
2627 2623
2628 do_gettimeofday(&timestamp); 2624 do_gettimeofday(&timestamp);
2629 pgh->tv_sec = htonl(timestamp.tv_sec); 2625 pgh->tv_sec = htonl(timestamp.tv_sec);
2630 pgh->tv_usec = htonl(timestamp.tv_usec); 2626 pgh->tv_usec = htonl(timestamp.tv_usec);
2631 } 2627 }
2632 2628
2633 static struct sk_buff *fill_packet_ipv4(struct net_device *odev, 2629 static struct sk_buff *fill_packet_ipv4(struct net_device *odev,
2634 struct pktgen_dev *pkt_dev) 2630 struct pktgen_dev *pkt_dev)
2635 { 2631 {
2636 struct sk_buff *skb = NULL; 2632 struct sk_buff *skb = NULL;
2637 __u8 *eth; 2633 __u8 *eth;
2638 struct udphdr *udph; 2634 struct udphdr *udph;
2639 int datalen, iplen; 2635 int datalen, iplen;
2640 struct iphdr *iph; 2636 struct iphdr *iph;
2641 __be16 protocol = htons(ETH_P_IP); 2637 __be16 protocol = htons(ETH_P_IP);
2642 __be32 *mpls; 2638 __be32 *mpls;
2643 __be16 *vlan_tci = NULL; /* Encapsulates priority and VLAN ID */ 2639 __be16 *vlan_tci = NULL; /* Encapsulates priority and VLAN ID */
2644 __be16 *vlan_encapsulated_proto = NULL; /* packet type ID field (or len) for VLAN tag */ 2640 __be16 *vlan_encapsulated_proto = NULL; /* packet type ID field (or len) for VLAN tag */
2645 __be16 *svlan_tci = NULL; /* Encapsulates priority and SVLAN ID */ 2641 __be16 *svlan_tci = NULL; /* Encapsulates priority and SVLAN ID */
2646 __be16 *svlan_encapsulated_proto = NULL; /* packet type ID field (or len) for SVLAN tag */ 2642 __be16 *svlan_encapsulated_proto = NULL; /* packet type ID field (or len) for SVLAN tag */
2647 u16 queue_map; 2643 u16 queue_map;
2648 2644
2649 if (pkt_dev->nr_labels) 2645 if (pkt_dev->nr_labels)
2650 protocol = htons(ETH_P_MPLS_UC); 2646 protocol = htons(ETH_P_MPLS_UC);
2651 2647
2652 if (pkt_dev->vlan_id != 0xffff) 2648 if (pkt_dev->vlan_id != 0xffff)
2653 protocol = htons(ETH_P_8021Q); 2649 protocol = htons(ETH_P_8021Q);
2654 2650
2655 /* Update any of the values, used when we're incrementing various 2651 /* Update any of the values, used when we're incrementing various
2656 * fields. 2652 * fields.
2657 */ 2653 */
2658 mod_cur_headers(pkt_dev); 2654 mod_cur_headers(pkt_dev);
2659 queue_map = pkt_dev->cur_queue_map; 2655 queue_map = pkt_dev->cur_queue_map;
2660 2656
2661 datalen = (odev->hard_header_len + 16) & ~0xf; 2657 datalen = (odev->hard_header_len + 16) & ~0xf;
2662 2658
2663 if (pkt_dev->flags & F_NODE) { 2659 if (pkt_dev->flags & F_NODE) {
2664 int node; 2660 int node;
2665 2661
2666 if (pkt_dev->node >= 0) 2662 if (pkt_dev->node >= 0)
2667 node = pkt_dev->node; 2663 node = pkt_dev->node;
2668 else 2664 else
2669 node = numa_node_id(); 2665 node = numa_node_id();
2670 2666
2671 skb = __alloc_skb(NET_SKB_PAD + pkt_dev->cur_pkt_size + 64 2667 skb = __alloc_skb(NET_SKB_PAD + pkt_dev->cur_pkt_size + 64
2672 + datalen + pkt_dev->pkt_overhead, GFP_NOWAIT, 0, node); 2668 + datalen + pkt_dev->pkt_overhead, GFP_NOWAIT, 0, node);
2673 if (likely(skb)) { 2669 if (likely(skb)) {
2674 skb_reserve(skb, NET_SKB_PAD); 2670 skb_reserve(skb, NET_SKB_PAD);
2675 skb->dev = odev; 2671 skb->dev = odev;
2676 } 2672 }
2677 } 2673 }
2678 else 2674 else
2679 skb = __netdev_alloc_skb(odev, 2675 skb = __netdev_alloc_skb(odev,
2680 pkt_dev->cur_pkt_size + 64 2676 pkt_dev->cur_pkt_size + 64
2681 + datalen + pkt_dev->pkt_overhead, GFP_NOWAIT); 2677 + datalen + pkt_dev->pkt_overhead, GFP_NOWAIT);
2682 2678
2683 if (!skb) { 2679 if (!skb) {
2684 sprintf(pkt_dev->result, "No memory"); 2680 sprintf(pkt_dev->result, "No memory");
2685 return NULL; 2681 return NULL;
2686 } 2682 }
2687 prefetchw(skb->data); 2683 prefetchw(skb->data);
2688 2684
2689 skb_reserve(skb, datalen); 2685 skb_reserve(skb, datalen);
2690 2686
2691 /* Reserve for ethernet and IP header */ 2687 /* Reserve for ethernet and IP header */
2692 eth = (__u8 *) skb_push(skb, 14); 2688 eth = (__u8 *) skb_push(skb, 14);
2693 mpls = (__be32 *)skb_put(skb, pkt_dev->nr_labels*sizeof(__u32)); 2689 mpls = (__be32 *)skb_put(skb, pkt_dev->nr_labels*sizeof(__u32));
2694 if (pkt_dev->nr_labels) 2690 if (pkt_dev->nr_labels)
2695 mpls_push(mpls, pkt_dev); 2691 mpls_push(mpls, pkt_dev);
2696 2692
2697 if (pkt_dev->vlan_id != 0xffff) { 2693 if (pkt_dev->vlan_id != 0xffff) {
2698 if (pkt_dev->svlan_id != 0xffff) { 2694 if (pkt_dev->svlan_id != 0xffff) {
2699 svlan_tci = (__be16 *)skb_put(skb, sizeof(__be16)); 2695 svlan_tci = (__be16 *)skb_put(skb, sizeof(__be16));
2700 *svlan_tci = build_tci(pkt_dev->svlan_id, 2696 *svlan_tci = build_tci(pkt_dev->svlan_id,
2701 pkt_dev->svlan_cfi, 2697 pkt_dev->svlan_cfi,
2702 pkt_dev->svlan_p); 2698 pkt_dev->svlan_p);
2703 svlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16)); 2699 svlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16));
2704 *svlan_encapsulated_proto = htons(ETH_P_8021Q); 2700 *svlan_encapsulated_proto = htons(ETH_P_8021Q);
2705 } 2701 }
2706 vlan_tci = (__be16 *)skb_put(skb, sizeof(__be16)); 2702 vlan_tci = (__be16 *)skb_put(skb, sizeof(__be16));
2707 *vlan_tci = build_tci(pkt_dev->vlan_id, 2703 *vlan_tci = build_tci(pkt_dev->vlan_id,
2708 pkt_dev->vlan_cfi, 2704 pkt_dev->vlan_cfi,
2709 pkt_dev->vlan_p); 2705 pkt_dev->vlan_p);
2710 vlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16)); 2706 vlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16));
2711 *vlan_encapsulated_proto = htons(ETH_P_IP); 2707 *vlan_encapsulated_proto = htons(ETH_P_IP);
2712 } 2708 }
2713 2709
2714 skb->network_header = skb->tail; 2710 skb->network_header = skb->tail;
2715 skb->transport_header = skb->network_header + sizeof(struct iphdr); 2711 skb->transport_header = skb->network_header + sizeof(struct iphdr);
2716 skb_put(skb, sizeof(struct iphdr) + sizeof(struct udphdr)); 2712 skb_put(skb, sizeof(struct iphdr) + sizeof(struct udphdr));
2717 skb_set_queue_mapping(skb, queue_map); 2713 skb_set_queue_mapping(skb, queue_map);
2718 skb->priority = pkt_dev->skb_priority; 2714 skb->priority = pkt_dev->skb_priority;
2719 2715
2720 iph = ip_hdr(skb); 2716 iph = ip_hdr(skb);
2721 udph = udp_hdr(skb); 2717 udph = udp_hdr(skb);
2722 2718
2723 memcpy(eth, pkt_dev->hh, 12); 2719 memcpy(eth, pkt_dev->hh, 12);
2724 *(__be16 *) & eth[12] = protocol; 2720 *(__be16 *) & eth[12] = protocol;
2725 2721
2726 /* Eth + IPh + UDPh + mpls */ 2722 /* Eth + IPh + UDPh + mpls */
2727 datalen = pkt_dev->cur_pkt_size - 14 - 20 - 8 - 2723 datalen = pkt_dev->cur_pkt_size - 14 - 20 - 8 -
2728 pkt_dev->pkt_overhead; 2724 pkt_dev->pkt_overhead;
2729 if (datalen < 0 || datalen < sizeof(struct pktgen_hdr)) 2725 if (datalen < 0 || datalen < sizeof(struct pktgen_hdr))
2730 datalen = sizeof(struct pktgen_hdr); 2726 datalen = sizeof(struct pktgen_hdr);
2731 2727
2732 udph->source = htons(pkt_dev->cur_udp_src); 2728 udph->source = htons(pkt_dev->cur_udp_src);
2733 udph->dest = htons(pkt_dev->cur_udp_dst); 2729 udph->dest = htons(pkt_dev->cur_udp_dst);
2734 udph->len = htons(datalen + 8); /* DATA + udphdr */ 2730 udph->len = htons(datalen + 8); /* DATA + udphdr */
2735 udph->check = 0; /* No checksum */ 2731 udph->check = 0; /* No checksum */
2736 2732
2737 iph->ihl = 5; 2733 iph->ihl = 5;
2738 iph->version = 4; 2734 iph->version = 4;
2739 iph->ttl = 32; 2735 iph->ttl = 32;
2740 iph->tos = pkt_dev->tos; 2736 iph->tos = pkt_dev->tos;
2741 iph->protocol = IPPROTO_UDP; /* UDP */ 2737 iph->protocol = IPPROTO_UDP; /* UDP */
2742 iph->saddr = pkt_dev->cur_saddr; 2738 iph->saddr = pkt_dev->cur_saddr;
2743 iph->daddr = pkt_dev->cur_daddr; 2739 iph->daddr = pkt_dev->cur_daddr;
2744 iph->id = htons(pkt_dev->ip_id); 2740 iph->id = htons(pkt_dev->ip_id);
2745 pkt_dev->ip_id++; 2741 pkt_dev->ip_id++;
2746 iph->frag_off = 0; 2742 iph->frag_off = 0;
2747 iplen = 20 + 8 + datalen; 2743 iplen = 20 + 8 + datalen;
2748 iph->tot_len = htons(iplen); 2744 iph->tot_len = htons(iplen);
2749 iph->check = 0; 2745 iph->check = 0;
2750 iph->check = ip_fast_csum((void *)iph, iph->ihl); 2746 iph->check = ip_fast_csum((void *)iph, iph->ihl);
2751 skb->protocol = protocol; 2747 skb->protocol = protocol;
2752 skb->mac_header = (skb->network_header - ETH_HLEN - 2748 skb->mac_header = (skb->network_header - ETH_HLEN -
2753 pkt_dev->pkt_overhead); 2749 pkt_dev->pkt_overhead);
2754 skb->dev = odev; 2750 skb->dev = odev;
2755 skb->pkt_type = PACKET_HOST; 2751 skb->pkt_type = PACKET_HOST;
2756 pktgen_finalize_skb(pkt_dev, skb, datalen); 2752 pktgen_finalize_skb(pkt_dev, skb, datalen);
2757 2753
2758 #ifdef CONFIG_XFRM 2754 #ifdef CONFIG_XFRM
2759 if (!process_ipsec(pkt_dev, skb, protocol)) 2755 if (!process_ipsec(pkt_dev, skb, protocol))
2760 return NULL; 2756 return NULL;
2761 #endif 2757 #endif
2762 2758
2763 return skb; 2759 return skb;
2764 } 2760 }
2765 2761
2766 static struct sk_buff *fill_packet_ipv6(struct net_device *odev, 2762 static struct sk_buff *fill_packet_ipv6(struct net_device *odev,
2767 struct pktgen_dev *pkt_dev) 2763 struct pktgen_dev *pkt_dev)
2768 { 2764 {
2769 struct sk_buff *skb = NULL; 2765 struct sk_buff *skb = NULL;
2770 __u8 *eth; 2766 __u8 *eth;
2771 struct udphdr *udph; 2767 struct udphdr *udph;
2772 int datalen; 2768 int datalen;
2773 struct ipv6hdr *iph; 2769 struct ipv6hdr *iph;
2774 __be16 protocol = htons(ETH_P_IPV6); 2770 __be16 protocol = htons(ETH_P_IPV6);
2775 __be32 *mpls; 2771 __be32 *mpls;
2776 __be16 *vlan_tci = NULL; /* Encapsulates priority and VLAN ID */ 2772 __be16 *vlan_tci = NULL; /* Encapsulates priority and VLAN ID */
2777 __be16 *vlan_encapsulated_proto = NULL; /* packet type ID field (or len) for VLAN tag */ 2773 __be16 *vlan_encapsulated_proto = NULL; /* packet type ID field (or len) for VLAN tag */
2778 __be16 *svlan_tci = NULL; /* Encapsulates priority and SVLAN ID */ 2774 __be16 *svlan_tci = NULL; /* Encapsulates priority and SVLAN ID */
2779 __be16 *svlan_encapsulated_proto = NULL; /* packet type ID field (or len) for SVLAN tag */ 2775 __be16 *svlan_encapsulated_proto = NULL; /* packet type ID field (or len) for SVLAN tag */
2780 u16 queue_map; 2776 u16 queue_map;
2781 2777
2782 if (pkt_dev->nr_labels) 2778 if (pkt_dev->nr_labels)
2783 protocol = htons(ETH_P_MPLS_UC); 2779 protocol = htons(ETH_P_MPLS_UC);
2784 2780
2785 if (pkt_dev->vlan_id != 0xffff) 2781 if (pkt_dev->vlan_id != 0xffff)
2786 protocol = htons(ETH_P_8021Q); 2782 protocol = htons(ETH_P_8021Q);
2787 2783
2788 /* Update any of the values, used when we're incrementing various 2784 /* Update any of the values, used when we're incrementing various
2789 * fields. 2785 * fields.
2790 */ 2786 */
2791 mod_cur_headers(pkt_dev); 2787 mod_cur_headers(pkt_dev);
2792 queue_map = pkt_dev->cur_queue_map; 2788 queue_map = pkt_dev->cur_queue_map;
2793 2789
2794 skb = __netdev_alloc_skb(odev, 2790 skb = __netdev_alloc_skb(odev,
2795 pkt_dev->cur_pkt_size + 64 2791 pkt_dev->cur_pkt_size + 64
2796 + 16 + pkt_dev->pkt_overhead, GFP_NOWAIT); 2792 + 16 + pkt_dev->pkt_overhead, GFP_NOWAIT);
2797 if (!skb) { 2793 if (!skb) {
2798 sprintf(pkt_dev->result, "No memory"); 2794 sprintf(pkt_dev->result, "No memory");
2799 return NULL; 2795 return NULL;
2800 } 2796 }
2801 prefetchw(skb->data); 2797 prefetchw(skb->data);
2802 2798
2803 skb_reserve(skb, 16); 2799 skb_reserve(skb, 16);
2804 2800
2805 /* Reserve for ethernet and IP header */ 2801 /* Reserve for ethernet and IP header */
2806 eth = (__u8 *) skb_push(skb, 14); 2802 eth = (__u8 *) skb_push(skb, 14);
2807 mpls = (__be32 *)skb_put(skb, pkt_dev->nr_labels*sizeof(__u32)); 2803 mpls = (__be32 *)skb_put(skb, pkt_dev->nr_labels*sizeof(__u32));
2808 if (pkt_dev->nr_labels) 2804 if (pkt_dev->nr_labels)
2809 mpls_push(mpls, pkt_dev); 2805 mpls_push(mpls, pkt_dev);
2810 2806
2811 if (pkt_dev->vlan_id != 0xffff) { 2807 if (pkt_dev->vlan_id != 0xffff) {
2812 if (pkt_dev->svlan_id != 0xffff) { 2808 if (pkt_dev->svlan_id != 0xffff) {
2813 svlan_tci = (__be16 *)skb_put(skb, sizeof(__be16)); 2809 svlan_tci = (__be16 *)skb_put(skb, sizeof(__be16));
2814 *svlan_tci = build_tci(pkt_dev->svlan_id, 2810 *svlan_tci = build_tci(pkt_dev->svlan_id,
2815 pkt_dev->svlan_cfi, 2811 pkt_dev->svlan_cfi,
2816 pkt_dev->svlan_p); 2812 pkt_dev->svlan_p);
2817 svlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16)); 2813 svlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16));
2818 *svlan_encapsulated_proto = htons(ETH_P_8021Q); 2814 *svlan_encapsulated_proto = htons(ETH_P_8021Q);
2819 } 2815 }
2820 vlan_tci = (__be16 *)skb_put(skb, sizeof(__be16)); 2816 vlan_tci = (__be16 *)skb_put(skb, sizeof(__be16));
2821 *vlan_tci = build_tci(pkt_dev->vlan_id, 2817 *vlan_tci = build_tci(pkt_dev->vlan_id,
2822 pkt_dev->vlan_cfi, 2818 pkt_dev->vlan_cfi,
2823 pkt_dev->vlan_p); 2819 pkt_dev->vlan_p);
2824 vlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16)); 2820 vlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16));
2825 *vlan_encapsulated_proto = htons(ETH_P_IPV6); 2821 *vlan_encapsulated_proto = htons(ETH_P_IPV6);
2826 } 2822 }
2827 2823
2828 skb->network_header = skb->tail; 2824 skb->network_header = skb->tail;
2829 skb->transport_header = skb->network_header + sizeof(struct ipv6hdr); 2825 skb->transport_header = skb->network_header + sizeof(struct ipv6hdr);
2830 skb_put(skb, sizeof(struct ipv6hdr) + sizeof(struct udphdr)); 2826 skb_put(skb, sizeof(struct ipv6hdr) + sizeof(struct udphdr));
2831 skb_set_queue_mapping(skb, queue_map); 2827 skb_set_queue_mapping(skb, queue_map);
2832 skb->priority = pkt_dev->skb_priority; 2828 skb->priority = pkt_dev->skb_priority;
2833 iph = ipv6_hdr(skb); 2829 iph = ipv6_hdr(skb);
2834 udph = udp_hdr(skb); 2830 udph = udp_hdr(skb);
2835 2831
2836 memcpy(eth, pkt_dev->hh, 12); 2832 memcpy(eth, pkt_dev->hh, 12);
2837 *(__be16 *) &eth[12] = protocol; 2833 *(__be16 *) &eth[12] = protocol;
2838 2834
2839 /* Eth + IPh + UDPh + mpls */ 2835 /* Eth + IPh + UDPh + mpls */
2840 datalen = pkt_dev->cur_pkt_size - 14 - 2836 datalen = pkt_dev->cur_pkt_size - 14 -
2841 sizeof(struct ipv6hdr) - sizeof(struct udphdr) - 2837 sizeof(struct ipv6hdr) - sizeof(struct udphdr) -
2842 pkt_dev->pkt_overhead; 2838 pkt_dev->pkt_overhead;
2843 2839
2844 if (datalen < 0 || datalen < sizeof(struct pktgen_hdr)) { 2840 if (datalen < 0 || datalen < sizeof(struct pktgen_hdr)) {
2845 datalen = sizeof(struct pktgen_hdr); 2841 datalen = sizeof(struct pktgen_hdr);
2846 net_info_ratelimited("increased datalen to %d\n", datalen); 2842 net_info_ratelimited("increased datalen to %d\n", datalen);
2847 } 2843 }
2848 2844
2849 udph->source = htons(pkt_dev->cur_udp_src); 2845 udph->source = htons(pkt_dev->cur_udp_src);
2850 udph->dest = htons(pkt_dev->cur_udp_dst); 2846 udph->dest = htons(pkt_dev->cur_udp_dst);
2851 udph->len = htons(datalen + sizeof(struct udphdr)); 2847 udph->len = htons(datalen + sizeof(struct udphdr));
2852 udph->check = 0; /* No checksum */ 2848 udph->check = 0; /* No checksum */
2853 2849
2854 *(__be32 *) iph = htonl(0x60000000); /* Version + flow */ 2850 *(__be32 *) iph = htonl(0x60000000); /* Version + flow */
2855 2851
2856 if (pkt_dev->traffic_class) { 2852 if (pkt_dev->traffic_class) {
2857 /* Version + traffic class + flow (0) */ 2853 /* Version + traffic class + flow (0) */
2858 *(__be32 *)iph |= htonl(0x60000000 | (pkt_dev->traffic_class << 20)); 2854 *(__be32 *)iph |= htonl(0x60000000 | (pkt_dev->traffic_class << 20));
2859 } 2855 }
2860 2856
2861 iph->hop_limit = 32; 2857 iph->hop_limit = 32;
2862 2858
2863 iph->payload_len = htons(sizeof(struct udphdr) + datalen); 2859 iph->payload_len = htons(sizeof(struct udphdr) + datalen);
2864 iph->nexthdr = IPPROTO_UDP; 2860 iph->nexthdr = IPPROTO_UDP;
2865 2861
2866 iph->daddr = pkt_dev->cur_in6_daddr; 2862 iph->daddr = pkt_dev->cur_in6_daddr;
2867 iph->saddr = pkt_dev->cur_in6_saddr; 2863 iph->saddr = pkt_dev->cur_in6_saddr;
2868 2864
2869 skb->mac_header = (skb->network_header - ETH_HLEN - 2865 skb->mac_header = (skb->network_header - ETH_HLEN -
2870 pkt_dev->pkt_overhead); 2866 pkt_dev->pkt_overhead);
2871 skb->protocol = protocol; 2867 skb->protocol = protocol;
2872 skb->dev = odev; 2868 skb->dev = odev;
2873 skb->pkt_type = PACKET_HOST; 2869 skb->pkt_type = PACKET_HOST;
2874 2870
2875 pktgen_finalize_skb(pkt_dev, skb, datalen); 2871 pktgen_finalize_skb(pkt_dev, skb, datalen);
2876 2872
2877 return skb; 2873 return skb;
2878 } 2874 }
2879 2875
2880 static struct sk_buff *fill_packet(struct net_device *odev, 2876 static struct sk_buff *fill_packet(struct net_device *odev,
2881 struct pktgen_dev *pkt_dev) 2877 struct pktgen_dev *pkt_dev)
2882 { 2878 {
2883 if (pkt_dev->flags & F_IPV6) 2879 if (pkt_dev->flags & F_IPV6)
2884 return fill_packet_ipv6(odev, pkt_dev); 2880 return fill_packet_ipv6(odev, pkt_dev);
2885 else 2881 else
2886 return fill_packet_ipv4(odev, pkt_dev); 2882 return fill_packet_ipv4(odev, pkt_dev);
2887 } 2883 }
2888 2884
2889 static void pktgen_clear_counters(struct pktgen_dev *pkt_dev) 2885 static void pktgen_clear_counters(struct pktgen_dev *pkt_dev)
2890 { 2886 {
2891 pkt_dev->seq_num = 1; 2887 pkt_dev->seq_num = 1;
2892 pkt_dev->idle_acc = 0; 2888 pkt_dev->idle_acc = 0;
2893 pkt_dev->sofar = 0; 2889 pkt_dev->sofar = 0;
2894 pkt_dev->tx_bytes = 0; 2890 pkt_dev->tx_bytes = 0;
2895 pkt_dev->errors = 0; 2891 pkt_dev->errors = 0;
2896 } 2892 }
2897 2893
2898 /* Set up structure for sending pkts, clear counters */ 2894 /* Set up structure for sending pkts, clear counters */
2899 2895
2900 static void pktgen_run(struct pktgen_thread *t) 2896 static void pktgen_run(struct pktgen_thread *t)
2901 { 2897 {
2902 struct pktgen_dev *pkt_dev; 2898 struct pktgen_dev *pkt_dev;
2903 int started = 0; 2899 int started = 0;
2904 2900
2905 func_enter(); 2901 func_enter();
2906 2902
2907 if_lock(t); 2903 if_lock(t);
2908 list_for_each_entry(pkt_dev, &t->if_list, list) { 2904 list_for_each_entry(pkt_dev, &t->if_list, list) {
2909 2905
2910 /* 2906 /*
2911 * setup odev and create initial packet. 2907 * setup odev and create initial packet.
2912 */ 2908 */
2913 pktgen_setup_inject(pkt_dev); 2909 pktgen_setup_inject(pkt_dev);
2914 2910
2915 if (pkt_dev->odev) { 2911 if (pkt_dev->odev) {
2916 pktgen_clear_counters(pkt_dev); 2912 pktgen_clear_counters(pkt_dev);
2917 pkt_dev->running = 1; /* Cranke yeself! */ 2913 pkt_dev->running = 1; /* Cranke yeself! */
2918 pkt_dev->skb = NULL; 2914 pkt_dev->skb = NULL;
2919 pkt_dev->started_at = 2915 pkt_dev->started_at =
2920 pkt_dev->next_tx = ktime_now(); 2916 pkt_dev->next_tx = ktime_now();
2921 2917
2922 set_pkt_overhead(pkt_dev); 2918 set_pkt_overhead(pkt_dev);
2923 2919
2924 strcpy(pkt_dev->result, "Starting"); 2920 strcpy(pkt_dev->result, "Starting");
2925 started++; 2921 started++;
2926 } else 2922 } else
2927 strcpy(pkt_dev->result, "Error starting"); 2923 strcpy(pkt_dev->result, "Error starting");
2928 } 2924 }
2929 if_unlock(t); 2925 if_unlock(t);
2930 if (started) 2926 if (started)
2931 t->control &= ~(T_STOP); 2927 t->control &= ~(T_STOP);
2932 } 2928 }
2933 2929
2934 static void pktgen_stop_all_threads_ifs(void) 2930 static void pktgen_stop_all_threads_ifs(void)
2935 { 2931 {
2936 struct pktgen_thread *t; 2932 struct pktgen_thread *t;
2937 2933
2938 func_enter(); 2934 func_enter();
2939 2935
2940 mutex_lock(&pktgen_thread_lock); 2936 mutex_lock(&pktgen_thread_lock);
2941 2937
2942 list_for_each_entry(t, &pktgen_threads, th_list) 2938 list_for_each_entry(t, &pktgen_threads, th_list)
2943 t->control |= T_STOP; 2939 t->control |= T_STOP;
2944 2940
2945 mutex_unlock(&pktgen_thread_lock); 2941 mutex_unlock(&pktgen_thread_lock);
2946 } 2942 }
2947 2943
2948 static int thread_is_running(const struct pktgen_thread *t) 2944 static int thread_is_running(const struct pktgen_thread *t)
2949 { 2945 {
2950 const struct pktgen_dev *pkt_dev; 2946 const struct pktgen_dev *pkt_dev;
2951 2947
2952 list_for_each_entry(pkt_dev, &t->if_list, list) 2948 list_for_each_entry(pkt_dev, &t->if_list, list)
2953 if (pkt_dev->running) 2949 if (pkt_dev->running)
2954 return 1; 2950 return 1;
2955 return 0; 2951 return 0;
2956 } 2952 }
2957 2953
2958 static int pktgen_wait_thread_run(struct pktgen_thread *t) 2954 static int pktgen_wait_thread_run(struct pktgen_thread *t)
2959 { 2955 {
2960 if_lock(t); 2956 if_lock(t);
2961 2957
2962 while (thread_is_running(t)) { 2958 while (thread_is_running(t)) {
2963 2959
2964 if_unlock(t); 2960 if_unlock(t);
2965 2961
2966 msleep_interruptible(100); 2962 msleep_interruptible(100);
2967 2963
2968 if (signal_pending(current)) 2964 if (signal_pending(current))
2969 goto signal; 2965 goto signal;
2970 if_lock(t); 2966 if_lock(t);
2971 } 2967 }
2972 if_unlock(t); 2968 if_unlock(t);
2973 return 1; 2969 return 1;
2974 signal: 2970 signal:
2975 return 0; 2971 return 0;
2976 } 2972 }
2977 2973
2978 static int pktgen_wait_all_threads_run(void) 2974 static int pktgen_wait_all_threads_run(void)
2979 { 2975 {
2980 struct pktgen_thread *t; 2976 struct pktgen_thread *t;
2981 int sig = 1; 2977 int sig = 1;
2982 2978
2983 mutex_lock(&pktgen_thread_lock); 2979 mutex_lock(&pktgen_thread_lock);
2984 2980
2985 list_for_each_entry(t, &pktgen_threads, th_list) { 2981 list_for_each_entry(t, &pktgen_threads, th_list) {
2986 sig = pktgen_wait_thread_run(t); 2982 sig = pktgen_wait_thread_run(t);
2987 if (sig == 0) 2983 if (sig == 0)
2988 break; 2984 break;
2989 } 2985 }
2990 2986
2991 if (sig == 0) 2987 if (sig == 0)
2992 list_for_each_entry(t, &pktgen_threads, th_list) 2988 list_for_each_entry(t, &pktgen_threads, th_list)
2993 t->control |= (T_STOP); 2989 t->control |= (T_STOP);
2994 2990
2995 mutex_unlock(&pktgen_thread_lock); 2991 mutex_unlock(&pktgen_thread_lock);
2996 return sig; 2992 return sig;
2997 } 2993 }
2998 2994
2999 static void pktgen_run_all_threads(void) 2995 static void pktgen_run_all_threads(void)
3000 { 2996 {
3001 struct pktgen_thread *t; 2997 struct pktgen_thread *t;
3002 2998
3003 func_enter(); 2999 func_enter();
3004 3000
3005 mutex_lock(&pktgen_thread_lock); 3001 mutex_lock(&pktgen_thread_lock);
3006 3002
3007 list_for_each_entry(t, &pktgen_threads, th_list) 3003 list_for_each_entry(t, &pktgen_threads, th_list)
3008 t->control |= (T_RUN); 3004 t->control |= (T_RUN);
3009 3005
3010 mutex_unlock(&pktgen_thread_lock); 3006 mutex_unlock(&pktgen_thread_lock);
3011 3007
3012 /* Propagate thread->control */ 3008 /* Propagate thread->control */
3013 schedule_timeout_interruptible(msecs_to_jiffies(125)); 3009 schedule_timeout_interruptible(msecs_to_jiffies(125));
3014 3010
3015 pktgen_wait_all_threads_run(); 3011 pktgen_wait_all_threads_run();
3016 } 3012 }
3017 3013
3018 static void pktgen_reset_all_threads(void) 3014 static void pktgen_reset_all_threads(void)
3019 { 3015 {
3020 struct pktgen_thread *t; 3016 struct pktgen_thread *t;
3021 3017
3022 func_enter(); 3018 func_enter();
3023 3019
3024 mutex_lock(&pktgen_thread_lock); 3020 mutex_lock(&pktgen_thread_lock);
3025 3021
3026 list_for_each_entry(t, &pktgen_threads, th_list) 3022 list_for_each_entry(t, &pktgen_threads, th_list)
3027 t->control |= (T_REMDEVALL); 3023 t->control |= (T_REMDEVALL);
3028 3024
3029 mutex_unlock(&pktgen_thread_lock); 3025 mutex_unlock(&pktgen_thread_lock);
3030 3026
3031 /* Propagate thread->control */ 3027 /* Propagate thread->control */
3032 schedule_timeout_interruptible(msecs_to_jiffies(125)); 3028 schedule_timeout_interruptible(msecs_to_jiffies(125));
3033 3029
3034 pktgen_wait_all_threads_run(); 3030 pktgen_wait_all_threads_run();
3035 } 3031 }
3036 3032
3037 static void show_results(struct pktgen_dev *pkt_dev, int nr_frags) 3033 static void show_results(struct pktgen_dev *pkt_dev, int nr_frags)
3038 { 3034 {
3039 __u64 bps, mbps, pps; 3035 __u64 bps, mbps, pps;
3040 char *p = pkt_dev->result; 3036 char *p = pkt_dev->result;
3041 ktime_t elapsed = ktime_sub(pkt_dev->stopped_at, 3037 ktime_t elapsed = ktime_sub(pkt_dev->stopped_at,
3042 pkt_dev->started_at); 3038 pkt_dev->started_at);
3043 ktime_t idle = ns_to_ktime(pkt_dev->idle_acc); 3039 ktime_t idle = ns_to_ktime(pkt_dev->idle_acc);
3044 3040
3045 p += sprintf(p, "OK: %llu(c%llu+d%llu) usec, %llu (%dbyte,%dfrags)\n", 3041 p += sprintf(p, "OK: %llu(c%llu+d%llu) usec, %llu (%dbyte,%dfrags)\n",
3046 (unsigned long long)ktime_to_us(elapsed), 3042 (unsigned long long)ktime_to_us(elapsed),
3047 (unsigned long long)ktime_to_us(ktime_sub(elapsed, idle)), 3043 (unsigned long long)ktime_to_us(ktime_sub(elapsed, idle)),
3048 (unsigned long long)ktime_to_us(idle), 3044 (unsigned long long)ktime_to_us(idle),
3049 (unsigned long long)pkt_dev->sofar, 3045 (unsigned long long)pkt_dev->sofar,
3050 pkt_dev->cur_pkt_size, nr_frags); 3046 pkt_dev->cur_pkt_size, nr_frags);
3051 3047
3052 pps = div64_u64(pkt_dev->sofar * NSEC_PER_SEC, 3048 pps = div64_u64(pkt_dev->sofar * NSEC_PER_SEC,
3053 ktime_to_ns(elapsed)); 3049 ktime_to_ns(elapsed));
3054 3050
3055 bps = pps * 8 * pkt_dev->cur_pkt_size; 3051 bps = pps * 8 * pkt_dev->cur_pkt_size;
3056 3052
3057 mbps = bps; 3053 mbps = bps;
3058 do_div(mbps, 1000000); 3054 do_div(mbps, 1000000);
3059 p += sprintf(p, " %llupps %lluMb/sec (%llubps) errors: %llu", 3055 p += sprintf(p, " %llupps %lluMb/sec (%llubps) errors: %llu",
3060 (unsigned long long)pps, 3056 (unsigned long long)pps,
3061 (unsigned long long)mbps, 3057 (unsigned long long)mbps,
3062 (unsigned long long)bps, 3058 (unsigned long long)bps,
3063 (unsigned long long)pkt_dev->errors); 3059 (unsigned long long)pkt_dev->errors);
3064 } 3060 }
3065 3061
3066 /* Set stopped-at timer, remove from running list, do counters & statistics */ 3062 /* Set stopped-at timer, remove from running list, do counters & statistics */
3067 static int pktgen_stop_device(struct pktgen_dev *pkt_dev) 3063 static int pktgen_stop_device(struct pktgen_dev *pkt_dev)
3068 { 3064 {
3069 int nr_frags = pkt_dev->skb ? skb_shinfo(pkt_dev->skb)->nr_frags : -1; 3065 int nr_frags = pkt_dev->skb ? skb_shinfo(pkt_dev->skb)->nr_frags : -1;
3070 3066
3071 if (!pkt_dev->running) { 3067 if (!pkt_dev->running) {
3072 pr_warning("interface: %s is already stopped\n", 3068 pr_warning("interface: %s is already stopped\n",
3073 pkt_dev->odevname); 3069 pkt_dev->odevname);
3074 return -EINVAL; 3070 return -EINVAL;
3075 } 3071 }
3076 3072
3077 kfree_skb(pkt_dev->skb); 3073 kfree_skb(pkt_dev->skb);
3078 pkt_dev->skb = NULL; 3074 pkt_dev->skb = NULL;
3079 pkt_dev->stopped_at = ktime_now(); 3075 pkt_dev->stopped_at = ktime_now();
3080 pkt_dev->running = 0; 3076 pkt_dev->running = 0;
3081 3077
3082 show_results(pkt_dev, nr_frags); 3078 show_results(pkt_dev, nr_frags);
3083 3079
3084 return 0; 3080 return 0;
3085 } 3081 }
3086 3082
3087 static struct pktgen_dev *next_to_run(struct pktgen_thread *t) 3083 static struct pktgen_dev *next_to_run(struct pktgen_thread *t)
3088 { 3084 {
3089 struct pktgen_dev *pkt_dev, *best = NULL; 3085 struct pktgen_dev *pkt_dev, *best = NULL;
3090 3086
3091 if_lock(t); 3087 if_lock(t);
3092 3088
3093 list_for_each_entry(pkt_dev, &t->if_list, list) { 3089 list_for_each_entry(pkt_dev, &t->if_list, list) {
3094 if (!pkt_dev->running) 3090 if (!pkt_dev->running)
3095 continue; 3091 continue;
3096 if (best == NULL) 3092 if (best == NULL)
3097 best = pkt_dev; 3093 best = pkt_dev;
3098 else if (ktime_lt(pkt_dev->next_tx, best->next_tx)) 3094 else if (ktime_lt(pkt_dev->next_tx, best->next_tx))
3099 best = pkt_dev; 3095 best = pkt_dev;
3100 } 3096 }
3101 if_unlock(t); 3097 if_unlock(t);
3102 return best; 3098 return best;
3103 } 3099 }
3104 3100
3105 static void pktgen_stop(struct pktgen_thread *t) 3101 static void pktgen_stop(struct pktgen_thread *t)
3106 { 3102 {
3107 struct pktgen_dev *pkt_dev; 3103 struct pktgen_dev *pkt_dev;
3108 3104
3109 func_enter(); 3105 func_enter();
3110 3106
3111 if_lock(t); 3107 if_lock(t);
3112 3108
3113 list_for_each_entry(pkt_dev, &t->if_list, list) { 3109 list_for_each_entry(pkt_dev, &t->if_list, list) {
3114 pktgen_stop_device(pkt_dev); 3110 pktgen_stop_device(pkt_dev);
3115 } 3111 }
3116 3112
3117 if_unlock(t); 3113 if_unlock(t);
3118 } 3114 }
3119 3115
3120 /* 3116 /*
3121 * one of our devices needs to be removed - find it 3117 * one of our devices needs to be removed - find it
3122 * and remove it 3118 * and remove it
3123 */ 3119 */
3124 static void pktgen_rem_one_if(struct pktgen_thread *t) 3120 static void pktgen_rem_one_if(struct pktgen_thread *t)
3125 { 3121 {
3126 struct list_head *q, *n; 3122 struct list_head *q, *n;
3127 struct pktgen_dev *cur; 3123 struct pktgen_dev *cur;
3128 3124
3129 func_enter(); 3125 func_enter();
3130 3126
3131 if_lock(t); 3127 if_lock(t);
3132 3128
3133 list_for_each_safe(q, n, &t->if_list) { 3129 list_for_each_safe(q, n, &t->if_list) {
3134 cur = list_entry(q, struct pktgen_dev, list); 3130 cur = list_entry(q, struct pktgen_dev, list);
3135 3131
3136 if (!cur->removal_mark) 3132 if (!cur->removal_mark)
3137 continue; 3133 continue;
3138 3134
3139 kfree_skb(cur->skb); 3135 kfree_skb(cur->skb);
3140 cur->skb = NULL; 3136 cur->skb = NULL;
3141 3137
3142 pktgen_remove_device(t, cur); 3138 pktgen_remove_device(t, cur);
3143 3139
3144 break; 3140 break;
3145 } 3141 }
3146 3142
3147 if_unlock(t); 3143 if_unlock(t);
3148 } 3144 }
3149 3145
3150 static void pktgen_rem_all_ifs(struct pktgen_thread *t) 3146 static void pktgen_rem_all_ifs(struct pktgen_thread *t)
3151 { 3147 {
3152 struct list_head *q, *n; 3148 struct list_head *q, *n;
3153 struct pktgen_dev *cur; 3149 struct pktgen_dev *cur;
3154 3150
3155 func_enter(); 3151 func_enter();
3156 3152
3157 /* Remove all devices, free mem */ 3153 /* Remove all devices, free mem */
3158 3154
3159 if_lock(t); 3155 if_lock(t);
3160 3156
3161 list_for_each_safe(q, n, &t->if_list) { 3157 list_for_each_safe(q, n, &t->if_list) {
3162 cur = list_entry(q, struct pktgen_dev, list); 3158 cur = list_entry(q, struct pktgen_dev, list);
3163 3159
3164 kfree_skb(cur->skb); 3160 kfree_skb(cur->skb);
3165 cur->skb = NULL; 3161 cur->skb = NULL;
3166 3162
3167 pktgen_remove_device(t, cur); 3163 pktgen_remove_device(t, cur);
3168 } 3164 }
3169 3165
3170 if_unlock(t); 3166 if_unlock(t);
3171 } 3167 }
3172 3168
3173 static void pktgen_rem_thread(struct pktgen_thread *t) 3169 static void pktgen_rem_thread(struct pktgen_thread *t)
3174 { 3170 {
3175 /* Remove from the thread list */ 3171 /* Remove from the thread list */
3176 3172
3177 remove_proc_entry(t->tsk->comm, pg_proc_dir); 3173 remove_proc_entry(t->tsk->comm, pg_proc_dir);
3178 3174
3179 } 3175 }
3180 3176
3181 static void pktgen_resched(struct pktgen_dev *pkt_dev) 3177 static void pktgen_resched(struct pktgen_dev *pkt_dev)
3182 { 3178 {
3183 ktime_t idle_start = ktime_now(); 3179 ktime_t idle_start = ktime_now();
3184 schedule(); 3180 schedule();
3185 pkt_dev->idle_acc += ktime_to_ns(ktime_sub(ktime_now(), idle_start)); 3181 pkt_dev->idle_acc += ktime_to_ns(ktime_sub(ktime_now(), idle_start));
3186 } 3182 }
3187 3183
3188 static void pktgen_wait_for_skb(struct pktgen_dev *pkt_dev) 3184 static void pktgen_wait_for_skb(struct pktgen_dev *pkt_dev)
3189 { 3185 {
3190 ktime_t idle_start = ktime_now(); 3186 ktime_t idle_start = ktime_now();
3191 3187
3192 while (atomic_read(&(pkt_dev->skb->users)) != 1) { 3188 while (atomic_read(&(pkt_dev->skb->users)) != 1) {
3193 if (signal_pending(current)) 3189 if (signal_pending(current))
3194 break; 3190 break;
3195 3191
3196 if (need_resched()) 3192 if (need_resched())
3197 pktgen_resched(pkt_dev); 3193 pktgen_resched(pkt_dev);
3198 else 3194 else
3199 cpu_relax(); 3195 cpu_relax();
3200 } 3196 }
3201 pkt_dev->idle_acc += ktime_to_ns(ktime_sub(ktime_now(), idle_start)); 3197 pkt_dev->idle_acc += ktime_to_ns(ktime_sub(ktime_now(), idle_start));
3202 } 3198 }
3203 3199
3204 static void pktgen_xmit(struct pktgen_dev *pkt_dev) 3200 static void pktgen_xmit(struct pktgen_dev *pkt_dev)
3205 { 3201 {
3206 struct net_device *odev = pkt_dev->odev; 3202 struct net_device *odev = pkt_dev->odev;
3207 netdev_tx_t (*xmit)(struct sk_buff *, struct net_device *) 3203 netdev_tx_t (*xmit)(struct sk_buff *, struct net_device *)
3208 = odev->netdev_ops->ndo_start_xmit; 3204 = odev->netdev_ops->ndo_start_xmit;
3209 struct netdev_queue *txq; 3205 struct netdev_queue *txq;
3210 u16 queue_map; 3206 u16 queue_map;
3211 int ret; 3207 int ret;
3212 3208
3213 /* If device is offline, then don't send */ 3209 /* If device is offline, then don't send */
3214 if (unlikely(!netif_running(odev) || !netif_carrier_ok(odev))) { 3210 if (unlikely(!netif_running(odev) || !netif_carrier_ok(odev))) {
3215 pktgen_stop_device(pkt_dev); 3211 pktgen_stop_device(pkt_dev);
3216 return; 3212 return;
3217 } 3213 }
3218 3214
3219 /* This is max DELAY, this has special meaning of 3215 /* This is max DELAY, this has special meaning of
3220 * "never transmit" 3216 * "never transmit"
3221 */ 3217 */
3222 if (unlikely(pkt_dev->delay == ULLONG_MAX)) { 3218 if (unlikely(pkt_dev->delay == ULLONG_MAX)) {
3223 pkt_dev->next_tx = ktime_add_ns(ktime_now(), ULONG_MAX); 3219 pkt_dev->next_tx = ktime_add_ns(ktime_now(), ULONG_MAX);
3224 return; 3220 return;
3225 } 3221 }
3226 3222
3227 /* If no skb or clone count exhausted then get new one */ 3223 /* If no skb or clone count exhausted then get new one */
3228 if (!pkt_dev->skb || (pkt_dev->last_ok && 3224 if (!pkt_dev->skb || (pkt_dev->last_ok &&
3229 ++pkt_dev->clone_count >= pkt_dev->clone_skb)) { 3225 ++pkt_dev->clone_count >= pkt_dev->clone_skb)) {
3230 /* build a new pkt */ 3226 /* build a new pkt */
3231 kfree_skb(pkt_dev->skb); 3227 kfree_skb(pkt_dev->skb);
3232 3228
3233 pkt_dev->skb = fill_packet(odev, pkt_dev); 3229 pkt_dev->skb = fill_packet(odev, pkt_dev);
3234 if (pkt_dev->skb == NULL) { 3230 if (pkt_dev->skb == NULL) {
3235 pr_err("ERROR: couldn't allocate skb in fill_packet\n"); 3231 pr_err("ERROR: couldn't allocate skb in fill_packet\n");
3236 schedule(); 3232 schedule();
3237 pkt_dev->clone_count--; /* back out increment, OOM */ 3233 pkt_dev->clone_count--; /* back out increment, OOM */
3238 return; 3234 return;
3239 } 3235 }
3240 pkt_dev->last_pkt_size = pkt_dev->skb->len; 3236 pkt_dev->last_pkt_size = pkt_dev->skb->len;
3241 pkt_dev->allocated_skbs++; 3237 pkt_dev->allocated_skbs++;
3242 pkt_dev->clone_count = 0; /* reset counter */ 3238 pkt_dev->clone_count = 0; /* reset counter */
3243 } 3239 }
3244 3240
3245 if (pkt_dev->delay && pkt_dev->last_ok) 3241 if (pkt_dev->delay && pkt_dev->last_ok)
3246 spin(pkt_dev, pkt_dev->next_tx); 3242 spin(pkt_dev, pkt_dev->next_tx);
3247 3243
3248 queue_map = skb_get_queue_mapping(pkt_dev->skb); 3244 queue_map = skb_get_queue_mapping(pkt_dev->skb);
3249 txq = netdev_get_tx_queue(odev, queue_map); 3245 txq = netdev_get_tx_queue(odev, queue_map);
3250 3246
3251 __netif_tx_lock_bh(txq); 3247 __netif_tx_lock_bh(txq);
3252 3248
3253 if (unlikely(netif_xmit_frozen_or_stopped(txq))) { 3249 if (unlikely(netif_xmit_frozen_or_stopped(txq))) {
3254 ret = NETDEV_TX_BUSY; 3250 ret = NETDEV_TX_BUSY;
3255 pkt_dev->last_ok = 0; 3251 pkt_dev->last_ok = 0;
3256 goto unlock; 3252 goto unlock;
3257 } 3253 }
3258 atomic_inc(&(pkt_dev->skb->users)); 3254 atomic_inc(&(pkt_dev->skb->users));
3259 ret = (*xmit)(pkt_dev->skb, odev); 3255 ret = (*xmit)(pkt_dev->skb, odev);
3260 3256
3261 switch (ret) { 3257 switch (ret) {
3262 case NETDEV_TX_OK: 3258 case NETDEV_TX_OK:
3263 txq_trans_update(txq); 3259 txq_trans_update(txq);
3264 pkt_dev->last_ok = 1; 3260 pkt_dev->last_ok = 1;
3265 pkt_dev->sofar++; 3261 pkt_dev->sofar++;
3266 pkt_dev->seq_num++; 3262 pkt_dev->seq_num++;
3267 pkt_dev->tx_bytes += pkt_dev->last_pkt_size; 3263 pkt_dev->tx_bytes += pkt_dev->last_pkt_size;
3268 break; 3264 break;
3269 case NET_XMIT_DROP: 3265 case NET_XMIT_DROP:
3270 case NET_XMIT_CN: 3266 case NET_XMIT_CN:
3271 case NET_XMIT_POLICED: 3267 case NET_XMIT_POLICED:
3272 /* skb has been consumed */ 3268 /* skb has been consumed */
3273 pkt_dev->errors++; 3269 pkt_dev->errors++;
3274 break; 3270 break;
3275 default: /* Drivers are not supposed to return other values! */ 3271 default: /* Drivers are not supposed to return other values! */
3276 net_info_ratelimited("%s xmit error: %d\n", 3272 net_info_ratelimited("%s xmit error: %d\n",
3277 pkt_dev->odevname, ret); 3273 pkt_dev->odevname, ret);
3278 pkt_dev->errors++; 3274 pkt_dev->errors++;
3279 /* fallthru */ 3275 /* fallthru */
3280 case NETDEV_TX_LOCKED: 3276 case NETDEV_TX_LOCKED:
3281 case NETDEV_TX_BUSY: 3277 case NETDEV_TX_BUSY:
3282 /* Retry it next time */ 3278 /* Retry it next time */
3283 atomic_dec(&(pkt_dev->skb->users)); 3279 atomic_dec(&(pkt_dev->skb->users));
3284 pkt_dev->last_ok = 0; 3280 pkt_dev->last_ok = 0;
3285 } 3281 }
3286 unlock: 3282 unlock:
3287 __netif_tx_unlock_bh(txq); 3283 __netif_tx_unlock_bh(txq);
3288 3284
3289 /* If pkt_dev->count is zero, then run forever */ 3285 /* If pkt_dev->count is zero, then run forever */
3290 if ((pkt_dev->count != 0) && (pkt_dev->sofar >= pkt_dev->count)) { 3286 if ((pkt_dev->count != 0) && (pkt_dev->sofar >= pkt_dev->count)) {
3291 pktgen_wait_for_skb(pkt_dev); 3287 pktgen_wait_for_skb(pkt_dev);
3292 3288
3293 /* Done with this */ 3289 /* Done with this */
3294 pktgen_stop_device(pkt_dev); 3290 pktgen_stop_device(pkt_dev);
3295 } 3291 }
3296 } 3292 }
3297 3293
3298 /* 3294 /*
3299 * Main loop of the thread goes here 3295 * Main loop of the thread goes here
3300 */ 3296 */
3301 3297
3302 static int pktgen_thread_worker(void *arg) 3298 static int pktgen_thread_worker(void *arg)
3303 { 3299 {
3304 DEFINE_WAIT(wait); 3300 DEFINE_WAIT(wait);
3305 struct pktgen_thread *t = arg; 3301 struct pktgen_thread *t = arg;
3306 struct pktgen_dev *pkt_dev = NULL; 3302 struct pktgen_dev *pkt_dev = NULL;
3307 int cpu = t->cpu; 3303 int cpu = t->cpu;
3308 3304
3309 BUG_ON(smp_processor_id() != cpu); 3305 BUG_ON(smp_processor_id() != cpu);
3310 3306
3311 init_waitqueue_head(&t->queue); 3307 init_waitqueue_head(&t->queue);
3312 complete(&t->start_done); 3308 complete(&t->start_done);
3313 3309
3314 pr_debug("starting pktgen/%d: pid=%d\n", cpu, task_pid_nr(current)); 3310 pr_debug("starting pktgen/%d: pid=%d\n", cpu, task_pid_nr(current));
3315 3311
3316 set_current_state(TASK_INTERRUPTIBLE); 3312 set_current_state(TASK_INTERRUPTIBLE);
3317 3313
3318 set_freezable(); 3314 set_freezable();
3319 3315
3320 while (!kthread_should_stop()) { 3316 while (!kthread_should_stop()) {
3321 pkt_dev = next_to_run(t); 3317 pkt_dev = next_to_run(t);
3322 3318
3323 if (unlikely(!pkt_dev && t->control == 0)) { 3319 if (unlikely(!pkt_dev && t->control == 0)) {
3324 if (pktgen_exiting) 3320 if (pktgen_exiting)
3325 break; 3321 break;
3326 wait_event_interruptible_timeout(t->queue, 3322 wait_event_interruptible_timeout(t->queue,
3327 t->control != 0, 3323 t->control != 0,
3328 HZ/10); 3324 HZ/10);
3329 try_to_freeze(); 3325 try_to_freeze();
3330 continue; 3326 continue;
3331 } 3327 }
3332 3328
3333 __set_current_state(TASK_RUNNING); 3329 __set_current_state(TASK_RUNNING);
3334 3330
3335 if (likely(pkt_dev)) { 3331 if (likely(pkt_dev)) {
3336 pktgen_xmit(pkt_dev); 3332 pktgen_xmit(pkt_dev);
3337 3333
3338 if (need_resched()) 3334 if (need_resched())
3339 pktgen_resched(pkt_dev); 3335 pktgen_resched(pkt_dev);
3340 else 3336 else
3341 cpu_relax(); 3337 cpu_relax();
3342 } 3338 }
3343 3339
3344 if (t->control & T_STOP) { 3340 if (t->control & T_STOP) {
3345 pktgen_stop(t); 3341 pktgen_stop(t);
3346 t->control &= ~(T_STOP); 3342 t->control &= ~(T_STOP);
3347 } 3343 }
3348 3344
3349 if (t->control & T_RUN) { 3345 if (t->control & T_RUN) {
3350 pktgen_run(t); 3346 pktgen_run(t);
3351 t->control &= ~(T_RUN); 3347 t->control &= ~(T_RUN);
3352 } 3348 }
3353 3349
3354 if (t->control & T_REMDEVALL) { 3350 if (t->control & T_REMDEVALL) {
3355 pktgen_rem_all_ifs(t); 3351 pktgen_rem_all_ifs(t);
3356 t->control &= ~(T_REMDEVALL); 3352 t->control &= ~(T_REMDEVALL);
3357 } 3353 }
3358 3354
3359 if (t->control & T_REMDEV) { 3355 if (t->control & T_REMDEV) {
3360 pktgen_rem_one_if(t); 3356 pktgen_rem_one_if(t);
3361 t->control &= ~(T_REMDEV); 3357 t->control &= ~(T_REMDEV);
3362 } 3358 }
3363 3359
3364 try_to_freeze(); 3360 try_to_freeze();
3365 3361
3366 set_current_state(TASK_INTERRUPTIBLE); 3362 set_current_state(TASK_INTERRUPTIBLE);
3367 } 3363 }
3368 3364
3369 pr_debug("%s stopping all device\n", t->tsk->comm); 3365 pr_debug("%s stopping all device\n", t->tsk->comm);
3370 pktgen_stop(t); 3366 pktgen_stop(t);
3371 3367
3372 pr_debug("%s removing all device\n", t->tsk->comm); 3368 pr_debug("%s removing all device\n", t->tsk->comm);
3373 pktgen_rem_all_ifs(t); 3369 pktgen_rem_all_ifs(t);
3374 3370
3375 pr_debug("%s removing thread\n", t->tsk->comm); 3371 pr_debug("%s removing thread\n", t->tsk->comm);
3376 pktgen_rem_thread(t); 3372 pktgen_rem_thread(t);
3377 3373
3378 /* Wait for kthread_stop */ 3374 /* Wait for kthread_stop */
3379 while (!kthread_should_stop()) { 3375 while (!kthread_should_stop()) {
3380 set_current_state(TASK_INTERRUPTIBLE); 3376 set_current_state(TASK_INTERRUPTIBLE);
3381 schedule(); 3377 schedule();
3382 } 3378 }
3383 __set_current_state(TASK_RUNNING); 3379 __set_current_state(TASK_RUNNING);
3384 3380
3385 return 0; 3381 return 0;
3386 } 3382 }
3387 3383
3388 static struct pktgen_dev *pktgen_find_dev(struct pktgen_thread *t, 3384 static struct pktgen_dev *pktgen_find_dev(struct pktgen_thread *t,
3389 const char *ifname, bool exact) 3385 const char *ifname, bool exact)
3390 { 3386 {
3391 struct pktgen_dev *p, *pkt_dev = NULL; 3387 struct pktgen_dev *p, *pkt_dev = NULL;
3392 size_t len = strlen(ifname); 3388 size_t len = strlen(ifname);
3393 3389
3394 if_lock(t); 3390 if_lock(t);
3395 list_for_each_entry(p, &t->if_list, list) 3391 list_for_each_entry(p, &t->if_list, list)
3396 if (strncmp(p->odevname, ifname, len) == 0) { 3392 if (strncmp(p->odevname, ifname, len) == 0) {
3397 if (p->odevname[len]) { 3393 if (p->odevname[len]) {
3398 if (exact || p->odevname[len] != '@') 3394 if (exact || p->odevname[len] != '@')
3399 continue; 3395 continue;
3400 } 3396 }
3401 pkt_dev = p; 3397 pkt_dev = p;
3402 break; 3398 break;
3403 } 3399 }
3404 3400
3405 if_unlock(t); 3401 if_unlock(t);
3406 pr_debug("find_dev(%s) returning %p\n", ifname, pkt_dev); 3402 pr_debug("find_dev(%s) returning %p\n", ifname, pkt_dev);
3407 return pkt_dev; 3403 return pkt_dev;
3408 } 3404 }
3409 3405
3410 /* 3406 /*
3411 * Adds a dev at front of if_list. 3407 * Adds a dev at front of if_list.
3412 */ 3408 */
3413 3409
3414 static int add_dev_to_thread(struct pktgen_thread *t, 3410 static int add_dev_to_thread(struct pktgen_thread *t,
3415 struct pktgen_dev *pkt_dev) 3411 struct pktgen_dev *pkt_dev)
3416 { 3412 {
3417 int rv = 0; 3413 int rv = 0;
3418 3414
3419 if_lock(t); 3415 if_lock(t);
3420 3416
3421 if (pkt_dev->pg_thread) { 3417 if (pkt_dev->pg_thread) {
3422 pr_err("ERROR: already assigned to a thread\n"); 3418 pr_err("ERROR: already assigned to a thread\n");
3423 rv = -EBUSY; 3419 rv = -EBUSY;
3424 goto out; 3420 goto out;
3425 } 3421 }
3426 3422
3427 list_add(&pkt_dev->list, &t->if_list); 3423 list_add(&pkt_dev->list, &t->if_list);
3428 pkt_dev->pg_thread = t; 3424 pkt_dev->pg_thread = t;
3429 pkt_dev->running = 0; 3425 pkt_dev->running = 0;
3430 3426
3431 out: 3427 out:
3432 if_unlock(t); 3428 if_unlock(t);
3433 return rv; 3429 return rv;
3434 } 3430 }
3435 3431
3436 /* Called under thread lock */ 3432 /* Called under thread lock */
3437 3433
3438 static int pktgen_add_device(struct pktgen_thread *t, const char *ifname) 3434 static int pktgen_add_device(struct pktgen_thread *t, const char *ifname)
3439 { 3435 {
3440 struct pktgen_dev *pkt_dev; 3436 struct pktgen_dev *pkt_dev;
3441 int err; 3437 int err;
3442 int node = cpu_to_node(t->cpu); 3438 int node = cpu_to_node(t->cpu);
3443 3439
3444 /* We don't allow a device to be on several threads */ 3440 /* We don't allow a device to be on several threads */
3445 3441
3446 pkt_dev = __pktgen_NN_threads(ifname, FIND); 3442 pkt_dev = __pktgen_NN_threads(ifname, FIND);
3447 if (pkt_dev) { 3443 if (pkt_dev) {
3448 pr_err("ERROR: interface already used\n"); 3444 pr_err("ERROR: interface already used\n");
3449 return -EBUSY; 3445 return -EBUSY;
3450 } 3446 }
3451 3447
3452 pkt_dev = kzalloc_node(sizeof(struct pktgen_dev), GFP_KERNEL, node); 3448 pkt_dev = kzalloc_node(sizeof(struct pktgen_dev), GFP_KERNEL, node);
3453 if (!pkt_dev) 3449 if (!pkt_dev)
3454 return -ENOMEM; 3450 return -ENOMEM;
3455 3451
3456 strcpy(pkt_dev->odevname, ifname); 3452 strcpy(pkt_dev->odevname, ifname);
3457 pkt_dev->flows = vzalloc_node(MAX_CFLOWS * sizeof(struct flow_state), 3453 pkt_dev->flows = vzalloc_node(MAX_CFLOWS * sizeof(struct flow_state),
3458 node); 3454 node);
3459 if (pkt_dev->flows == NULL) { 3455 if (pkt_dev->flows == NULL) {
3460 kfree(pkt_dev); 3456 kfree(pkt_dev);
3461 return -ENOMEM; 3457 return -ENOMEM;
3462 } 3458 }
3463 3459
3464 pkt_dev->removal_mark = 0; 3460 pkt_dev->removal_mark = 0;
3465 pkt_dev->nfrags = 0; 3461 pkt_dev->nfrags = 0;
3466 pkt_dev->delay = pg_delay_d; 3462 pkt_dev->delay = pg_delay_d;
3467 pkt_dev->count = pg_count_d; 3463 pkt_dev->count = pg_count_d;
3468 pkt_dev->sofar = 0; 3464 pkt_dev->sofar = 0;
3469 pkt_dev->udp_src_min = 9; /* sink port */ 3465 pkt_dev->udp_src_min = 9; /* sink port */
3470 pkt_dev->udp_src_max = 9; 3466 pkt_dev->udp_src_max = 9;
3471 pkt_dev->udp_dst_min = 9; 3467 pkt_dev->udp_dst_min = 9;
3472 pkt_dev->udp_dst_max = 9; 3468 pkt_dev->udp_dst_max = 9;
3473 pkt_dev->vlan_p = 0; 3469 pkt_dev->vlan_p = 0;
3474 pkt_dev->vlan_cfi = 0; 3470 pkt_dev->vlan_cfi = 0;
3475 pkt_dev->vlan_id = 0xffff; 3471 pkt_dev->vlan_id = 0xffff;
3476 pkt_dev->svlan_p = 0; 3472 pkt_dev->svlan_p = 0;
3477 pkt_dev->svlan_cfi = 0; 3473 pkt_dev->svlan_cfi = 0;
3478 pkt_dev->svlan_id = 0xffff; 3474 pkt_dev->svlan_id = 0xffff;
3479 pkt_dev->node = -1; 3475 pkt_dev->node = -1;
3480 3476
3481 err = pktgen_setup_dev(pkt_dev, ifname); 3477 err = pktgen_setup_dev(pkt_dev, ifname);
3482 if (err) 3478 if (err)
3483 goto out1; 3479 goto out1;
3484 if (pkt_dev->odev->priv_flags & IFF_TX_SKB_SHARING) 3480 if (pkt_dev->odev->priv_flags & IFF_TX_SKB_SHARING)
3485 pkt_dev->clone_skb = pg_clone_skb_d; 3481 pkt_dev->clone_skb = pg_clone_skb_d;
3486 3482
3487 pkt_dev->entry = proc_create_data(ifname, 0600, pg_proc_dir, 3483 pkt_dev->entry = proc_create_data(ifname, 0600, pg_proc_dir,
3488 &pktgen_if_fops, pkt_dev); 3484 &pktgen_if_fops, pkt_dev);
3489 if (!pkt_dev->entry) { 3485 if (!pkt_dev->entry) {
3490 pr_err("cannot create %s/%s procfs entry\n", 3486 pr_err("cannot create %s/%s procfs entry\n",
3491 PG_PROC_DIR, ifname); 3487 PG_PROC_DIR, ifname);
3492 err = -EINVAL; 3488 err = -EINVAL;
3493 goto out2; 3489 goto out2;
3494 } 3490 }
3495 #ifdef CONFIG_XFRM 3491 #ifdef CONFIG_XFRM
3496 pkt_dev->ipsmode = XFRM_MODE_TRANSPORT; 3492 pkt_dev->ipsmode = XFRM_MODE_TRANSPORT;
3497 pkt_dev->ipsproto = IPPROTO_ESP; 3493 pkt_dev->ipsproto = IPPROTO_ESP;
3498 #endif 3494 #endif
3499 3495
3500 return add_dev_to_thread(t, pkt_dev); 3496 return add_dev_to_thread(t, pkt_dev);
3501 out2: 3497 out2:
3502 dev_put(pkt_dev->odev); 3498 dev_put(pkt_dev->odev);
3503 out1: 3499 out1:
3504 #ifdef CONFIG_XFRM 3500 #ifdef CONFIG_XFRM
3505 free_SAs(pkt_dev); 3501 free_SAs(pkt_dev);
3506 #endif 3502 #endif
3507 vfree(pkt_dev->flows); 3503 vfree(pkt_dev->flows);
3508 kfree(pkt_dev); 3504 kfree(pkt_dev);
3509 return err; 3505 return err;
3510 } 3506 }
3511 3507
3512 static int __init pktgen_create_thread(int cpu) 3508 static int __init pktgen_create_thread(int cpu)
3513 { 3509 {
3514 struct pktgen_thread *t; 3510 struct pktgen_thread *t;
3515 struct proc_dir_entry *pe; 3511 struct proc_dir_entry *pe;
3516 struct task_struct *p; 3512 struct task_struct *p;
3517 3513
3518 t = kzalloc_node(sizeof(struct pktgen_thread), GFP_KERNEL, 3514 t = kzalloc_node(sizeof(struct pktgen_thread), GFP_KERNEL,
3519 cpu_to_node(cpu)); 3515 cpu_to_node(cpu));
3520 if (!t) { 3516 if (!t) {
3521 pr_err("ERROR: out of memory, can't create new thread\n"); 3517 pr_err("ERROR: out of memory, can't create new thread\n");
3522 return -ENOMEM; 3518 return -ENOMEM;
3523 } 3519 }
3524 3520
3525 spin_lock_init(&t->if_lock); 3521 spin_lock_init(&t->if_lock);
3526 t->cpu = cpu; 3522 t->cpu = cpu;
3527 3523
3528 INIT_LIST_HEAD(&t->if_list); 3524 INIT_LIST_HEAD(&t->if_list);
3529 3525
3530 list_add_tail(&t->th_list, &pktgen_threads); 3526 list_add_tail(&t->th_list, &pktgen_threads);
3531 init_completion(&t->start_done); 3527 init_completion(&t->start_done);
3532 3528
3533 p = kthread_create_on_node(pktgen_thread_worker, 3529 p = kthread_create_on_node(pktgen_thread_worker,
3534 t, 3530 t,
3535 cpu_to_node(cpu), 3531 cpu_to_node(cpu),
3536 "kpktgend_%d", cpu); 3532 "kpktgend_%d", cpu);
3537 if (IS_ERR(p)) { 3533 if (IS_ERR(p)) {
3538 pr_err("kernel_thread() failed for cpu %d\n", t->cpu); 3534 pr_err("kernel_thread() failed for cpu %d\n", t->cpu);
3539 list_del(&t->th_list); 3535 list_del(&t->th_list);
3540 kfree(t); 3536 kfree(t);
3541 return PTR_ERR(p); 3537 return PTR_ERR(p);
3542 } 3538 }
3543 kthread_bind(p, cpu); 3539 kthread_bind(p, cpu);
3544 t->tsk = p; 3540 t->tsk = p;
3545 3541
3546 pe = proc_create_data(t->tsk->comm, 0600, pg_proc_dir, 3542 pe = proc_create_data(t->tsk->comm, 0600, pg_proc_dir,
3547 &pktgen_thread_fops, t); 3543 &pktgen_thread_fops, t);
3548 if (!pe) { 3544 if (!pe) {
3549 pr_err("cannot create %s/%s procfs entry\n", 3545 pr_err("cannot create %s/%s procfs entry\n",
3550 PG_PROC_DIR, t->tsk->comm); 3546 PG_PROC_DIR, t->tsk->comm);
3551 kthread_stop(p); 3547 kthread_stop(p);
3552 list_del(&t->th_list); 3548 list_del(&t->th_list);
3553 kfree(t); 3549 kfree(t);
3554 return -EINVAL; 3550 return -EINVAL;
3555 } 3551 }
3556 3552
3557 wake_up_process(p); 3553 wake_up_process(p);
3558 wait_for_completion(&t->start_done); 3554 wait_for_completion(&t->start_done);
3559 3555
3560 return 0; 3556 return 0;
3561 } 3557 }
3562 3558
3563 /* 3559 /*
3564 * Removes a device from the thread if_list. 3560 * Removes a device from the thread if_list.
3565 */ 3561 */
3566 static void _rem_dev_from_if_list(struct pktgen_thread *t, 3562 static void _rem_dev_from_if_list(struct pktgen_thread *t,
3567 struct pktgen_dev *pkt_dev) 3563 struct pktgen_dev *pkt_dev)
3568 { 3564 {
3569 struct list_head *q, *n; 3565 struct list_head *q, *n;
3570 struct pktgen_dev *p; 3566 struct pktgen_dev *p;
3571 3567
3572 list_for_each_safe(q, n, &t->if_list) { 3568 list_for_each_safe(q, n, &t->if_list) {
3573 p = list_entry(q, struct pktgen_dev, list); 3569 p = list_entry(q, struct pktgen_dev, list);
3574 if (p == pkt_dev) 3570 if (p == pkt_dev)
3575 list_del(&p->list); 3571 list_del(&p->list);
3576 } 3572 }
3577 } 3573 }
3578 3574
3579 static int pktgen_remove_device(struct pktgen_thread *t, 3575 static int pktgen_remove_device(struct pktgen_thread *t,
3580 struct pktgen_dev *pkt_dev) 3576 struct pktgen_dev *pkt_dev)
3581 { 3577 {
3582 3578
3583 pr_debug("remove_device pkt_dev=%p\n", pkt_dev); 3579 pr_debug("remove_device pkt_dev=%p\n", pkt_dev);
3584 3580
3585 if (pkt_dev->running) { 3581 if (pkt_dev->running) {
3586 pr_warning("WARNING: trying to remove a running interface, stopping it now\n"); 3582 pr_warning("WARNING: trying to remove a running interface, stopping it now\n");
3587 pktgen_stop_device(pkt_dev); 3583 pktgen_stop_device(pkt_dev);
3588 } 3584 }
3589 3585
3590 /* Dis-associate from the interface */ 3586 /* Dis-associate from the interface */
3591 3587
3592 if (pkt_dev->odev) { 3588 if (pkt_dev->odev) {
3593 dev_put(pkt_dev->odev); 3589 dev_put(pkt_dev->odev);
3594 pkt_dev->odev = NULL; 3590 pkt_dev->odev = NULL;
3595 } 3591 }
3596 3592
3597 /* And update the thread if_list */ 3593 /* And update the thread if_list */
3598 3594
3599 _rem_dev_from_if_list(t, pkt_dev); 3595 _rem_dev_from_if_list(t, pkt_dev);
3600 3596
3601 if (pkt_dev->entry) 3597 if (pkt_dev->entry)
3602 remove_proc_entry(pkt_dev->entry->name, pg_proc_dir); 3598 remove_proc_entry(pkt_dev->entry->name, pg_proc_dir);
3603 3599
3604 #ifdef CONFIG_XFRM 3600 #ifdef CONFIG_XFRM
3605 free_SAs(pkt_dev); 3601 free_SAs(pkt_dev);
3606 #endif 3602 #endif
3607 vfree(pkt_dev->flows); 3603 vfree(pkt_dev->flows);
3608 if (pkt_dev->page) 3604 if (pkt_dev->page)
3609 put_page(pkt_dev->page); 3605 put_page(pkt_dev->page);
3610 kfree(pkt_dev); 3606 kfree(pkt_dev);
3611 return 0; 3607 return 0;
3612 } 3608 }
3613 3609
3614 static int __init pg_init(void) 3610 static int __init pg_init(void)
3615 { 3611 {
3616 int cpu; 3612 int cpu;
3617 struct proc_dir_entry *pe; 3613 struct proc_dir_entry *pe;
3618 int ret = 0; 3614 int ret = 0;
3619 3615
3620 pr_info("%s", version); 3616 pr_info("%s", version);
3621 3617
3622 pg_proc_dir = proc_mkdir(PG_PROC_DIR, init_net.proc_net); 3618 pg_proc_dir = proc_mkdir(PG_PROC_DIR, init_net.proc_net);
3623 if (!pg_proc_dir) 3619 if (!pg_proc_dir)
3624 return -ENODEV; 3620 return -ENODEV;
3625 3621
3626 pe = proc_create(PGCTRL, 0600, pg_proc_dir, &pktgen_fops); 3622 pe = proc_create(PGCTRL, 0600, pg_proc_dir, &pktgen_fops);
3627 if (pe == NULL) { 3623 if (pe == NULL) {
3628 pr_err("ERROR: cannot create %s procfs entry\n", PGCTRL); 3624 pr_err("ERROR: cannot create %s procfs entry\n", PGCTRL);
3629 ret = -EINVAL; 3625 ret = -EINVAL;
3630 goto remove_dir; 3626 goto remove_dir;
3631 } 3627 }
3632 3628
3633 register_netdevice_notifier(&pktgen_notifier_block); 3629 register_netdevice_notifier(&pktgen_notifier_block);
3634 3630
3635 for_each_online_cpu(cpu) { 3631 for_each_online_cpu(cpu) {
3636 int err; 3632 int err;
3637 3633
3638 err = pktgen_create_thread(cpu); 3634 err = pktgen_create_thread(cpu);
3639 if (err) 3635 if (err)
3640 pr_warning("WARNING: Cannot create thread for cpu %d (%d)\n", 3636 pr_warning("WARNING: Cannot create thread for cpu %d (%d)\n",
3641 cpu, err); 3637 cpu, err);
3642 } 3638 }
3643 3639
3644 if (list_empty(&pktgen_threads)) { 3640 if (list_empty(&pktgen_threads)) {
3645 pr_err("ERROR: Initialization failed for all threads\n"); 3641 pr_err("ERROR: Initialization failed for all threads\n");
3646 ret = -ENODEV; 3642 ret = -ENODEV;
3647 goto unregister; 3643 goto unregister;
3648 } 3644 }
3649 3645
3650 return 0; 3646 return 0;
3651 3647
3652 unregister: 3648 unregister:
3653 unregister_netdevice_notifier(&pktgen_notifier_block); 3649 unregister_netdevice_notifier(&pktgen_notifier_block);
3654 remove_proc_entry(PGCTRL, pg_proc_dir); 3650 remove_proc_entry(PGCTRL, pg_proc_dir);
3655 remove_dir: 3651 remove_dir:
3656 proc_net_remove(&init_net, PG_PROC_DIR); 3652 proc_net_remove(&init_net, PG_PROC_DIR);
3657 return ret; 3653 return ret;
3658 } 3654 }
3659 3655
3660 static void __exit pg_cleanup(void) 3656 static void __exit pg_cleanup(void)
3661 { 3657 {
3662 struct pktgen_thread *t; 3658 struct pktgen_thread *t;
3663 struct list_head *q, *n; 3659 struct list_head *q, *n;
3664 LIST_HEAD(list); 3660 LIST_HEAD(list);
3665 3661
3666 /* Stop all interfaces & threads */ 3662 /* Stop all interfaces & threads */
3667 pktgen_exiting = true; 3663 pktgen_exiting = true;
3668 3664
3669 mutex_lock(&pktgen_thread_lock); 3665 mutex_lock(&pktgen_thread_lock);
3670 list_splice_init(&pktgen_threads, &list); 3666 list_splice_init(&pktgen_threads, &list);
3671 mutex_unlock(&pktgen_thread_lock); 3667 mutex_unlock(&pktgen_thread_lock);
3672 3668
3673 list_for_each_safe(q, n, &list) { 3669 list_for_each_safe(q, n, &list) {
3674 t = list_entry(q, struct pktgen_thread, th_list); 3670 t = list_entry(q, struct pktgen_thread, th_list);
3675 list_del(&t->th_list); 3671 list_del(&t->th_list);
3676 kthread_stop(t->tsk); 3672 kthread_stop(t->tsk);
3677 kfree(t); 3673 kfree(t);
3678 } 3674 }
3679 3675
3680 /* Un-register us from receiving netdevice events */ 3676 /* Un-register us from receiving netdevice events */
3681 unregister_netdevice_notifier(&pktgen_notifier_block); 3677 unregister_netdevice_notifier(&pktgen_notifier_block);
3682 3678
3683 /* Clean up proc file system */ 3679 /* Clean up proc file system */
3684 remove_proc_entry(PGCTRL, pg_proc_dir); 3680 remove_proc_entry(PGCTRL, pg_proc_dir);
3685 proc_net_remove(&init_net, PG_PROC_DIR); 3681 proc_net_remove(&init_net, PG_PROC_DIR);
3686 } 3682 }
3687 3683
3688 module_init(pg_init); 3684 module_init(pg_init);
3689 module_exit(pg_cleanup); 3685 module_exit(pg_cleanup);
3690 3686
3691 MODULE_AUTHOR("Robert Olsson <robert.olsson@its.uu.se>"); 3687 MODULE_AUTHOR("Robert Olsson <robert.olsson@its.uu.se>");
3692 MODULE_DESCRIPTION("Packet Generator tool"); 3688 MODULE_DESCRIPTION("Packet Generator tool");
3693 MODULE_LICENSE("GPL"); 3689 MODULE_LICENSE("GPL");
3694 MODULE_VERSION(VERSION); 3690 MODULE_VERSION(VERSION);
3695 module_param(pg_count_d, int, 0); 3691 module_param(pg_count_d, int, 0);
3696 MODULE_PARM_DESC(pg_count_d, "Default number of packets to inject"); 3692 MODULE_PARM_DESC(pg_count_d, "Default number of packets to inject");
3697 module_param(pg_delay_d, int, 0); 3693 module_param(pg_delay_d, int, 0);
3698 MODULE_PARM_DESC(pg_delay_d, "Default delay between packets (nanoseconds)"); 3694 MODULE_PARM_DESC(pg_delay_d, "Default delay between packets (nanoseconds)");
3699 module_param(pg_clone_skb_d, int, 0); 3695 module_param(pg_clone_skb_d, int, 0);
3700 MODULE_PARM_DESC(pg_clone_skb_d, "Default number of copies of the same packet"); 3696 MODULE_PARM_DESC(pg_clone_skb_d, "Default number of copies of the same packet");
3701 module_param(debug, int, 0); 3697 module_param(debug, int, 0);
3702 MODULE_PARM_DESC(debug, "Enable debugging of pktgen module"); 3698 MODULE_PARM_DESC(debug, "Enable debugging of pktgen module");
3703 3699