Blame view

net/core/pktgen.c 90.8 KB
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1
2
3
4
5
6
7
8
  /*
   * Authors:
   * Copyright 2001, 2002 by Robert Olsson <robert.olsson@its.uu.se>
   *                             Uppsala University and
   *                             Swedish University of Agricultural Sciences
   *
   * Alexey Kuznetsov  <kuznet@ms2.inr.ac.ru>
   * Ben Greear <greearb@candelatech.com>
96de0e252   Jan Engelhardt   Convert files to ...
9
   * Jens Låås <jens.laas@data.slu.se>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
10
11
12
13
14
15
16
17
   *
   * This program is free software; you can redistribute it and/or
   * modify it under the terms of the GNU General Public License
   * as published by the Free Software Foundation; either version
   * 2 of the License, or (at your option) any later version.
   *
   *
   * A tool for loading the network with preconfigurated packets.
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
18
   * The tool is implemented as a linux module.  Parameters are output
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
   * device, delay (to hard_xmit), number of packets, and whether
   * to use multiple SKBs or just the same one.
   * pktgen uses the installed interface's output routine.
   *
   * Additional hacking by:
   *
   * Jens.Laas@data.slu.se
   * Improved by ANK. 010120.
   * Improved by ANK even more. 010212.
   * MAC address typo fixed. 010417 --ro
   * Integrated.  020301 --DaveM
   * Added multiskb option 020301 --DaveM
   * Scaling of results. 020417--sigurdur@linpro.no
   * Significant re-work of the module:
   *   *  Convert to threaded model to more efficiently be able to transmit
   *       and receive on multiple interfaces at once.
   *   *  Converted many counters to __u64 to allow longer runs.
   *   *  Allow configuration of ranges, like min/max IP address, MACs,
   *       and UDP-ports, for both source and destination, and can
   *       set to use a random distribution or sequentially walk the range.
   *   *  Can now change most values after starting.
   *   *  Place 12-byte packet in UDP payload with magic number,
   *       sequence number, and timestamp.
   *   *  Add receiver code that detects dropped pkts, re-ordered pkts, and
   *       latencies (with micro-second) precision.
   *   *  Add IOCTL interface to easily get counters & configuration.
   *   --Ben Greear <greearb@candelatech.com>
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
47
48
   * Renamed multiskb to clone_skb and cleaned up sending core for two distinct
   * skb modes. A clone_skb=0 mode for Ben "ranges" work and a clone_skb != 0
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
49
   * as a "fastpath" with a configurable number of clones after alloc's.
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
50
51
   * clone_skb=0 means all packets are allocated this also means ranges time
   * stamps etc can be used. clone_skb=100 means 1 malloc is followed by 100
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
52
53
   * clones.
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
54
   * Also moved to /proc/net/pktgen/
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
55
56
57
58
59
60
61
62
   * --ro
   *
   * Sept 10:  Fixed threading/locking.  Lots of bone-headed and more clever
   *    mistakes.  Also merged in DaveM's patch in the -pre6 patch.
   * --Ben Greear <greearb@candelatech.com>
   *
   * Integrated to 2.5.x 021029 --Lucio Maciel (luciomaciel@zipmail.com.br)
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
63
   *
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
64
65
66
67
   * 021124 Finished major redesign and rewrite for new functionality.
   * See Documentation/networking/pktgen.txt for how to use this.
   *
   * The new operation:
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
68
69
70
71
   * For each CPU one thread/process is created at start. This process checks
   * for running devices in the if_list and sends packets until count is 0 it
   * also the thread checks the thread->control which is used for inter-process
   * communication. controlling process "posts" operations to the threads this
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
72
73
74
   * way. The if_lock should be possible to remove when add/rem_device is merged
   * into this too.
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
75
76
   * By design there should only be *one* "controlling" process. In practice
   * multiple write accesses gives unpredictable result. Understood by "write"
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
77
   * to /proc gives result code thats should be read be the "writer".
b4099fab7   Stephen Hemminger   [PKTGEN]: Spellin...
78
   * For practical use this should be no problem.
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
79
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
80
81
   * Note when adding devices to a specific CPU there good idea to also assign
   * /proc/irq/XX/smp_affinity so TX-interrupts gets bound to the same CPU.
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
82
83
   * --ro
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
84
   * Fix refcount off by one if first packet fails, potential null deref,
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
85
86
87
88
89
90
91
   * memleak 030710- KJP
   *
   * First "ranges" functionality for ipv6 030726 --ro
   *
   * Included flow support. 030802 ANK.
   *
   * Fixed unaligned access on IA-64 Grant Grundler <grundler@parisc-linux.org>
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
92
   *
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
93
94
95
   * Remove if fix from added Harald Welte <laforge@netfilter.org> 040419
   * ia64 compilation fix from  Aron Griffis <aron@hp.com> 040604
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
96
   * New xmit() return, do_div and misc clean up by Stephen Hemminger
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
97
98
   * <shemminger@osdl.org> 040923
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
99
   * Randy Dunlap fixed u64 printk compiler waring
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
100
101
102
103
   *
   * Remove FCS from BW calculation.  Lennert Buytenhek <buytenh@wantstofly.org>
   * New time handling. Lennert Buytenhek <buytenh@wantstofly.org> 041213
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
104
   * Corrections from Nikolai Malykh (nmalykh@bilim.com)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
105
106
   * Removed unused flags F_SET_SRCMAC & F_SET_SRCIP 041230
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
107
   * interruptible_sleep_on_timeout() replaced Nishanth Aravamudan <nacc@us.ibm.com>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
108
   * 050103
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
109
110
111
   *
   * MPLS support by Steven Whitehouse <steve@chygwyn.com>
   *
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
112
113
   * 802.1Q/Q-in-Q support by Francesco Fondelli (FF) <francesco.fondelli@gmail.com>
   *
ce5d0b47f   Adit Ranadive   [PKTGEN]: srcmac fix
114
115
116
   * Fixed src_mac command to set source mac of packet to value specified in
   * command by Adit Ranadive <adit.262@gmail.com>
   *
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
117
   */
f9467eaec   Joe Perches   net/core/pktgen.c...
118
119
  
  #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
120
121
122
123
124
  #include <linux/sys.h>
  #include <linux/types.h>
  #include <linux/module.h>
  #include <linux/moduleparam.h>
  #include <linux/kernel.h>
222fa0766   Luiz Capitulino   [PKTGEN]: Convert...
125
  #include <linux/mutex.h>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
126
127
128
  #include <linux/sched.h>
  #include <linux/slab.h>
  #include <linux/vmalloc.h>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
129
130
131
132
133
134
  #include <linux/unistd.h>
  #include <linux/string.h>
  #include <linux/ptrace.h>
  #include <linux/errno.h>
  #include <linux/ioport.h>
  #include <linux/interrupt.h>
4fc268d24   Randy Dunlap   [PATCH] capable/c...
135
  #include <linux/capability.h>
2bc481cf4   Stephen Hemminger   pktgen: spin usin...
136
  #include <linux/hrtimer.h>
09fe3ef46   Andrew Morton   [PKTGEN]: Add try...
137
  #include <linux/freezer.h>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
138
139
  #include <linux/delay.h>
  #include <linux/timer.h>
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
140
  #include <linux/list.h>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
141
142
143
144
145
146
147
  #include <linux/init.h>
  #include <linux/skbuff.h>
  #include <linux/netdevice.h>
  #include <linux/inet.h>
  #include <linux/inetdevice.h>
  #include <linux/rtnetlink.h>
  #include <linux/if_arp.h>
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
148
  #include <linux/if_vlan.h>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
149
150
151
152
153
  #include <linux/in.h>
  #include <linux/ip.h>
  #include <linux/ipv6.h>
  #include <linux/udp.h>
  #include <linux/proc_fs.h>
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
154
  #include <linux/seq_file.h>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
155
  #include <linux/wait.h>
f404e9a67   Kris Katterjohn   [PKTGEN]: Replaci...
156
  #include <linux/etherdevice.h>
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
157
  #include <linux/kthread.h>
268bb0ce3   Linus Torvalds   sanitize <linux/p...
158
  #include <linux/prefetch.h>
457c4cbc5   Eric W. Biederman   [NET]: Make /proc...
159
  #include <net/net_namespace.h>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
160
161
162
  #include <net/checksum.h>
  #include <net/ipv6.h>
  #include <net/addrconf.h>
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
163
164
165
  #ifdef CONFIG_XFRM
  #include <net/xfrm.h>
  #endif
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
166
167
  #include <asm/byteorder.h>
  #include <linux/rcupdate.h>
1977f0327   Jiri Slaby   remove asm/bitops...
168
  #include <linux/bitops.h>
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
169
170
171
  #include <linux/io.h>
  #include <linux/timex.h>
  #include <linux/uaccess.h>
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
172
  #include <asm/dma.h>
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
173
  #include <asm/div64.h>		/* do_div */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
174

43d28b651   Daniel Turull   pktgen: increasin...
175
  #define VERSION	"2.74"
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
176
  #define IP_NAME_SZ 32
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
177
  #define MAX_MPLS_LABELS 16 /* This is the max label stack depth */
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
178
  #define MPLS_STACK_BOTTOM htonl(0x00000100)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
179

f9467eaec   Joe Perches   net/core/pktgen.c...
180
181
  #define func_enter() pr_debug("entering %s
  ", __func__);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
182
  /* Device flag bits */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
183
184
185
186
187
188
189
190
  #define F_IPSRC_RND   (1<<0)	/* IP-Src Random  */
  #define F_IPDST_RND   (1<<1)	/* IP-Dst Random  */
  #define F_UDPSRC_RND  (1<<2)	/* UDP-Src Random */
  #define F_UDPDST_RND  (1<<3)	/* UDP-Dst Random */
  #define F_MACSRC_RND  (1<<4)	/* MAC-Src Random */
  #define F_MACDST_RND  (1<<5)	/* MAC-Dst Random */
  #define F_TXSIZE_RND  (1<<6)	/* Transmit size is random */
  #define F_IPV6        (1<<7)	/* Interface in IPV6 Mode */
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
191
  #define F_MPLS_RND    (1<<8)	/* Random MPLS labels */
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
192
193
  #define F_VID_RND     (1<<9)	/* Random VLAN ID */
  #define F_SVID_RND    (1<<10)	/* Random SVLAN ID */
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
194
  #define F_FLOW_SEQ    (1<<11)	/* Sequential flows */
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
195
  #define F_IPSEC_ON    (1<<12)	/* ipsec on for flows */
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
196
  #define F_QUEUE_MAP_RND (1<<13)	/* queue map Random */
e6fce5b91   Robert Olsson   pktgen: multiqueu...
197
  #define F_QUEUE_MAP_CPU (1<<14)	/* queue map mirrors smp_processor_id() */
e99b99b47   Robert Olsson   pktgen node alloc...
198
  #define F_NODE          (1<<15)	/* Node memory alloc*/
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
199
200
  
  /* Thread control flag bits */
6b80d6a6b   Stephen Hemminger   pktgen: T_TERMINA...
201
202
203
204
  #define T_STOP        (1<<0)	/* Stop run */
  #define T_RUN         (1<<1)	/* Start run */
  #define T_REMDEVALL   (1<<2)	/* Remove all devs */
  #define T_REMDEV      (1<<3)	/* Remove one dev */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
205

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
206
207
208
209
210
211
  /* If lock -- can be removed after some work */
  #define   if_lock(t)           spin_lock(&(t->if_lock));
  #define   if_unlock(t)           spin_unlock(&(t->if_lock));
  
  /* Used to help with determining the pkts on receive */
  #define PKTGEN_MAGIC 0xbe9be955
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
212
213
  #define PG_PROC_DIR "pktgen"
  #define PGCTRL	    "pgctrl"
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
214
  static struct proc_dir_entry *pg_proc_dir;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
215
216
  
  #define MAX_CFLOWS  65536
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
217
218
  #define VLAN_TAG_SIZE(x) ((x)->vlan_id == 0xffff ? 0 : 4)
  #define SVLAN_TAG_SIZE(x) ((x)->svlan_id == 0xffff ? 0 : 4)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
219
  struct flow_state {
252e33467   Al Viro   [NET] net/core: A...
220
  	__be32 cur_daddr;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
221
  	int count;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
222
223
224
  #ifdef CONFIG_XFRM
  	struct xfrm_state *x;
  #endif
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
225
  	__u32 flags;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
226
  };
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
227
228
  /* flow flag bits */
  #define F_INIT   (1<<0)		/* flow has been initialized */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
229
  struct pktgen_dev {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
230
231
232
  	/*
  	 * Try to keep frequent/infrequent used vars. separated.
  	 */
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
233
234
  	struct proc_dir_entry *entry;	/* proc file */
  	struct pktgen_thread *pg_thread;/* the owner */
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
235
  	struct list_head list;		/* chaining in the thread's run-queue */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
236

63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
237
  	int running;		/* if false, the test will stop */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
238

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
239
240
241
242
  	/* If min != max, then we will either do a linear iteration, or
  	 * we will do a random selection from within the range.
  	 */
  	__u32 flags;
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
243
244
  	int removal_mark;	/* non-zero => the device is marked for
  				 * removal by worker thread */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
245

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
246
247
  	int min_pkt_size;	/* = ETH_ZLEN; */
  	int max_pkt_size;	/* = ETH_ZLEN; */
16dab72f6   Jamal Hadi Salim   [PKTGEN]: Central...
248
  	int pkt_overhead;	/* overhead for MPLS, VLANs, IPSEC etc */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
249
  	int nfrags;
26ad78796   Eric Dumazet   pktgen: speedup f...
250
  	struct page *page;
fd29cf726   Stephen Hemminger   pktgen: convert t...
251
  	u64 delay;		/* nano-seconds */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
252
253
254
  	__u64 count;		/* Default No packets to send */
  	__u64 sofar;		/* How many pkts we've sent so far */
  	__u64 tx_bytes;		/* How many bytes we've transmitted */
f466dba18   John Fastabend   pktgen: ndo_start...
255
  	__u64 errors;		/* Errors when trying to transmit, */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
256
257
  
  	/* runtime counters relating to clone_skb */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
258
259
260
261
  
  	__u64 allocated_skbs;
  	__u32 clone_count;
  	int last_ok;		/* Was last skb sent?
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
262
263
  				 * Or a failed transmit of some sort?
  				 * This will keep sequence numbers in order
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
264
  				 */
fd29cf726   Stephen Hemminger   pktgen: convert t...
265
266
267
268
  	ktime_t next_tx;
  	ktime_t started_at;
  	ktime_t stopped_at;
  	u64	idle_acc;	/* nano-seconds */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
269
  	__u32 seq_num;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
270
271
272
273
274
275
276
277
  	int clone_skb;		/*
  				 * Use multiple SKBs during packet gen.
  				 * If this number is greater than 1, then
  				 * that many copies of the same packet will be
  				 * sent before a new packet is allocated.
  				 * If you want to send 1024 identical packets
  				 * before creating a new packet,
  				 * set clone_skb to 1024.
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
278
279
280
281
282
283
284
285
286
287
288
  				 */
  
  	char dst_min[IP_NAME_SZ];	/* IP, ie 1.2.3.4 */
  	char dst_max[IP_NAME_SZ];	/* IP, ie 1.2.3.4 */
  	char src_min[IP_NAME_SZ];	/* IP, ie 1.2.3.4 */
  	char src_max[IP_NAME_SZ];	/* IP, ie 1.2.3.4 */
  
  	struct in6_addr in6_saddr;
  	struct in6_addr in6_daddr;
  	struct in6_addr cur_in6_daddr;
  	struct in6_addr cur_in6_saddr;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
289
  	/* For ranges */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
290
291
292
293
294
295
296
297
  	struct in6_addr min_in6_daddr;
  	struct in6_addr max_in6_daddr;
  	struct in6_addr min_in6_saddr;
  	struct in6_addr max_in6_saddr;
  
  	/* If we're doing ranges, random or incremental, then this
  	 * defines the min/max for those ranges.
  	 */
252e33467   Al Viro   [NET] net/core: A...
298
299
300
301
  	__be32 saddr_min;	/* inclusive, source IP address */
  	__be32 saddr_max;	/* exclusive, source IP address */
  	__be32 daddr_min;	/* inclusive, dest IP address */
  	__be32 daddr_max;	/* exclusive, dest IP address */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
302
303
304
305
306
  
  	__u16 udp_src_min;	/* inclusive, source UDP port */
  	__u16 udp_src_max;	/* exclusive, source UDP port */
  	__u16 udp_dst_min;	/* inclusive, dest UDP port */
  	__u16 udp_dst_max;	/* exclusive, dest UDP port */
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
307
  	/* DSCP + ECN */
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
308
309
310
311
  	__u8 tos;            /* six MSB of (former) IPv4 TOS
  				are for dscp codepoint */
  	__u8 traffic_class;  /* ditto for the (former) Traffic Class in IPv6
  				(see RFC 3260, sec. 4) */
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
312

ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
313
314
315
  	/* MPLS */
  	unsigned nr_labels;	/* Depth of stack, 0 = no MPLS */
  	__be32 labels[MAX_MPLS_LABELS];
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
316
317
318
319
320
321
322
323
  	/* VLAN/SVLAN (802.1Q/Q-in-Q) */
  	__u8  vlan_p;
  	__u8  vlan_cfi;
  	__u16 vlan_id;  /* 0xffff means no vlan tag */
  
  	__u8  svlan_p;
  	__u8  svlan_cfi;
  	__u16 svlan_id; /* 0xffff means no svlan tag */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
324
325
326
327
328
329
330
331
  	__u32 src_mac_count;	/* How many MACs to iterate through */
  	__u32 dst_mac_count;	/* How many MACs to iterate through */
  
  	unsigned char dst_mac[ETH_ALEN];
  	unsigned char src_mac[ETH_ALEN];
  
  	__u32 cur_dst_mac_offset;
  	__u32 cur_src_mac_offset;
252e33467   Al Viro   [NET] net/core: A...
332
333
  	__be32 cur_saddr;
  	__be32 cur_daddr;
66ed1e5ec   Eric Dumazet   pktgen: Dont leak...
334
  	__u16 ip_id;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
335
336
  	__u16 cur_udp_dst;
  	__u16 cur_udp_src;
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
337
  	__u16 cur_queue_map;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
338
  	__u32 cur_pkt_size;
baac85645   Eric Dumazet   pktgen: tx_bytes ...
339
  	__u32 last_pkt_size;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
340
341
342
343
344
345
346
347
348
349
350
  
  	__u8 hh[14];
  	/* = {
  	   0x00, 0x80, 0xC8, 0x79, 0xB3, 0xCB,
  
  	   We fill in SRC address later
  	   0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  	   0x08, 0x00
  	   };
  	 */
  	__u16 pad;		/* pad out the hh struct to an even 16 bytes */
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
351
  	struct sk_buff *skb;	/* skb we are to transmit next, used for when we
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
352
353
  				 * are transmitting the same one multiple times
  				 */
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
354
355
356
357
358
359
360
361
  	struct net_device *odev; /* The out-going device.
  				  * Note that the device should have it's
  				  * pg_info pointer pointing back to this
  				  * device.
  				  * Set when the user specifies the out-going
  				  * device name (not when the inject is
  				  * started as it used to do.)
  				  */
593f63b0b   Eric Dumazet   pktgen: Fix devic...
362
  	char odevname[32];
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
363
  	struct flow_state *flows;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
364
365
366
  	unsigned cflows;	/* Concurrent flows (config) */
  	unsigned lflow;		/* Flow length  (config) */
  	unsigned nflows;	/* accumulated flows (stats) */
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
367
  	unsigned curfl;		/* current sequenced flow (state)*/
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
368
369
370
  
  	u16 queue_map_min;
  	u16 queue_map_max;
9e50e3ac5   John Fastabend   net: add priority...
371
  	__u32 skb_priority;	/* skb priority field */
e99b99b47   Robert Olsson   pktgen node alloc...
372
  	int node;               /* Memory node */
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
373

a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
374
375
376
377
  #ifdef CONFIG_XFRM
  	__u8	ipsmode;		/* IPSEC mode (config) */
  	__u8	ipsproto;		/* IPSEC type (config) */
  #endif
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
378
  	char result[512];
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
379
380
381
  };
  
  struct pktgen_hdr {
252e33467   Al Viro   [NET] net/core: A...
382
383
384
385
  	__be32 pgh_magic;
  	__be32 seq_num;
  	__be32 tv_sec;
  	__be32 tv_usec;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
386
  };
551eaff1b   Eric Dumazet   pktgen: allow fas...
387
  static bool pktgen_exiting __read_mostly;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
388
  struct pktgen_thread {
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
389
  	spinlock_t if_lock;		/* for list of devices */
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
390
  	struct list_head if_list;	/* All device here */
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
391
  	struct list_head th_list;
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
392
  	struct task_struct *tsk;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
393
  	char result[512];
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
394

63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
395
396
  	/* Field for thread to receive "posted" events terminate,
  	   stop ifs etc. */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
397
398
  
  	u32 control;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
399
  	int cpu;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
400
  	wait_queue_head_t queue;
d3ede327e   Denis V. Lunev   pktgen: make sure...
401
  	struct completion start_done;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
402
403
404
405
  };
  
  #define REMOVE 1
  #define FIND   0
fd29cf726   Stephen Hemminger   pktgen: convert t...
406
  static inline ktime_t ktime_now(void)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
407
  {
fd29cf726   Stephen Hemminger   pktgen: convert t...
408
409
410
411
  	struct timespec ts;
  	ktime_get_ts(&ts);
  
  	return timespec_to_ktime(ts);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
412
  }
fd29cf726   Stephen Hemminger   pktgen: convert t...
413
414
  /* This works even if 32 bit because of careful byte order choice */
  static inline int ktime_lt(const ktime_t cmp1, const ktime_t cmp2)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
415
  {
fd29cf726   Stephen Hemminger   pktgen: convert t...
416
  	return cmp1.tv64 < cmp2.tv64;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
417
  }
c3d2f52dd   Stephen Hemminger   pktgen: increase ...
418
  static const char version[] =
f9467eaec   Joe Perches   net/core/pktgen.c...
419
420
421
  	"Packet Generator for packet performance testing. "
  	"Version: " VERSION "
  ";
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
422

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
423
424
425
  static int pktgen_remove_device(struct pktgen_thread *t, struct pktgen_dev *i);
  static int pktgen_add_device(struct pktgen_thread *t, const char *ifname);
  static struct pktgen_dev *pktgen_find_dev(struct pktgen_thread *t,
3e9848403   Eric Dumazet   pktgen: Fix netde...
426
  					  const char *ifname, bool exact);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
427
428
  static int pktgen_device_event(struct notifier_block *, unsigned long, void *);
  static void pktgen_run_all_threads(void);
eb37b41cc   Jesse Brandeburg   pktgen: add full ...
429
  static void pktgen_reset_all_threads(void);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
430
  static void pktgen_stop_all_threads_ifs(void);
3bda06a3d   Stephen Hemminger   pktgen: stop_devi...
431

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
432
  static void pktgen_stop(struct pktgen_thread *t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
433
  static void pktgen_clear_counters(struct pktgen_dev *pkt_dev);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
434

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
435
  static unsigned int scan_ip6(const char *s, char ip[16]);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
436
437
  
  /* Module parameters, defaults. */
65c5b786a   Stephen Hemminger   pktgen: mark read...
438
439
440
441
  static int pg_count_d __read_mostly = 1000;
  static int pg_delay_d __read_mostly;
  static int pg_clone_skb_d  __read_mostly;
  static int debug  __read_mostly;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
442

222fa0766   Luiz Capitulino   [PKTGEN]: Convert...
443
  static DEFINE_MUTEX(pktgen_thread_lock);
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
444
  static LIST_HEAD(pktgen_threads);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
445

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
446
447
448
  static struct notifier_block pktgen_notifier_block = {
  	.notifier_call = pktgen_device_event,
  };
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
449
  /*
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
450
   * /proc handling functions
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
451
452
   *
   */
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
453
  static int pgctrl_show(struct seq_file *seq, void *v)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
454
  {
c3d2f52dd   Stephen Hemminger   pktgen: increase ...
455
  	seq_puts(seq, version);
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
456
  	return 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
457
  }
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
458
459
  static ssize_t pgctrl_write(struct file *file, const char __user *buf,
  			    size_t count, loff_t *ppos)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
460
  {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
461
  	int err = 0;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
462
  	char data[128];
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
463

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
464
465
  	if (!capable(CAP_NET_ADMIN)) {
  		err = -EPERM;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
466
  		goto out;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
467
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
468

d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
469
470
  	if (count > sizeof(data))
  		count = sizeof(data);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
471

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
472
  	if (copy_from_user(data, buf, count)) {
b4099fab7   Stephen Hemminger   [PKTGEN]: Spellin...
473
  		err = -EFAULT;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
474
  		goto out;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
475
476
  	}
  	data[count - 1] = 0;	/* Make string */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
477

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
478
  	if (!strcmp(data, "stop"))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
479
  		pktgen_stop_all_threads_ifs();
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
480
  	else if (!strcmp(data, "start"))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
481
  		pktgen_run_all_threads();
eb37b41cc   Jesse Brandeburg   pktgen: add full ...
482
483
  	else if (!strcmp(data, "reset"))
  		pktgen_reset_all_threads();
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
484
  	else
f9467eaec   Joe Perches   net/core/pktgen.c...
485
486
  		pr_warning("Unknown command: %s
  ", data);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
487
488
  
  	err = count;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
489
490
  out:
  	return err;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
491
  }
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
492
493
494
495
  static int pgctrl_open(struct inode *inode, struct file *file)
  {
  	return single_open(file, pgctrl_show, PDE(inode)->data);
  }
9a32144e9   Arjan van de Ven   [PATCH] mark stru...
496
  static const struct file_operations pktgen_fops = {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
497
498
499
500
501
502
  	.owner   = THIS_MODULE,
  	.open    = pgctrl_open,
  	.read    = seq_read,
  	.llseek  = seq_lseek,
  	.write   = pgctrl_write,
  	.release = single_release,
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
503
504
505
  };
  
  static int pktgen_if_show(struct seq_file *seq, void *v)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
506
  {
648fda740   Stephen Hemminger   pktgen: minor cle...
507
  	const struct pktgen_dev *pkt_dev = seq->private;
fd29cf726   Stephen Hemminger   pktgen: convert t...
508
509
  	ktime_t stopped;
  	u64 idle;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
510
511
512
513
514
515
516
517
  
  	seq_printf(seq,
  		   "Params: count %llu  min_pkt_size: %u  max_pkt_size: %u
  ",
  		   (unsigned long long)pkt_dev->count, pkt_dev->min_pkt_size,
  		   pkt_dev->max_pkt_size);
  
  	seq_printf(seq,
fd29cf726   Stephen Hemminger   pktgen: convert t...
518
519
520
  		   "     frags: %d  delay: %llu  clone_skb: %d  ifname: %s
  ",
  		   pkt_dev->nfrags, (unsigned long long) pkt_dev->delay,
593f63b0b   Eric Dumazet   pktgen: Fix devic...
521
  		   pkt_dev->clone_skb, pkt_dev->odevname);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
522
523
524
525
  
  	seq_printf(seq, "     flows: %u flowlen: %u
  ", pkt_dev->cflows,
  		   pkt_dev->lflow);
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
526
527
528
529
530
  	seq_printf(seq,
  		   "     queue_map_min: %u  queue_map_max: %u
  ",
  		   pkt_dev->queue_map_min,
  		   pkt_dev->queue_map_max);
9e50e3ac5   John Fastabend   net: add priority...
531
532
533
534
  	if (pkt_dev->skb_priority)
  		seq_printf(seq, "     skb_priority: %u
  ",
  			   pkt_dev->skb_priority);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
535
  	if (pkt_dev->flags & F_IPV6) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
536
  		seq_printf(seq,
47a0200d5   Alexey Dobriyan   pktgen: use %pI6c...
537
538
539
540
541
542
543
544
  			   "     saddr: %pI6c  min_saddr: %pI6c  max_saddr: %pI6c
  "
  			   "     daddr: %pI6c  min_daddr: %pI6c  max_daddr: %pI6c
  ",
  			   &pkt_dev->in6_saddr,
  			   &pkt_dev->min_in6_saddr, &pkt_dev->max_in6_saddr,
  			   &pkt_dev->in6_daddr,
  			   &pkt_dev->min_in6_daddr, &pkt_dev->max_in6_daddr);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
545
546
547
548
549
  	} else {
  		seq_printf(seq,
  			   "     dst_min: %s  dst_max: %s
  ",
  			   pkt_dev->dst_min, pkt_dev->dst_max);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
550
  		seq_printf(seq,
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
551
552
553
554
  			   "        src_min: %s  src_max: %s
  ",
  			   pkt_dev->src_min, pkt_dev->src_max);
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
555

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
556
  	seq_puts(seq, "     src_mac: ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
557

e174961ca   Johannes Berg   net: convert prin...
558
559
560
  	seq_printf(seq, "%pM ",
  		   is_zero_ether_addr(pkt_dev->src_mac) ?
  			     pkt_dev->odev->dev_addr : pkt_dev->src_mac);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
561
562
  
  	seq_printf(seq, "dst_mac: ");
e174961ca   Johannes Berg   net: convert prin...
563
564
  	seq_printf(seq, "%pM
  ", pkt_dev->dst_mac);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
565
566
  
  	seq_printf(seq,
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
567
568
569
  		   "     udp_src_min: %d  udp_src_max: %d"
  		   "  udp_dst_min: %d  udp_dst_max: %d
  ",
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
570
571
572
573
  		   pkt_dev->udp_src_min, pkt_dev->udp_src_max,
  		   pkt_dev->udp_dst_min, pkt_dev->udp_dst_max);
  
  	seq_printf(seq,
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
574
575
  		   "     src_mac_count: %d  dst_mac_count: %d
  ",
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
576
  		   pkt_dev->src_mac_count, pkt_dev->dst_mac_count);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
577

ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
578
579
580
  	if (pkt_dev->nr_labels) {
  		unsigned i;
  		seq_printf(seq, "     mpls: ");
e71a4783a   Stephen Hemminger   [NET] core: white...
581
  		for (i = 0; i < pkt_dev->nr_labels; i++)
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
582
583
584
585
  			seq_printf(seq, "%08x%s", ntohl(pkt_dev->labels[i]),
  				   i == pkt_dev->nr_labels-1 ? "
  " : ", ");
  	}
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
586
  	if (pkt_dev->vlan_id != 0xffff)
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
587
588
  		seq_printf(seq, "     vlan_id: %u  vlan_p: %u  vlan_cfi: %u
  ",
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
589
590
  			   pkt_dev->vlan_id, pkt_dev->vlan_p,
  			   pkt_dev->vlan_cfi);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
591

63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
592
  	if (pkt_dev->svlan_id != 0xffff)
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
593
594
  		seq_printf(seq, "     svlan_id: %u  vlan_p: %u  vlan_cfi: %u
  ",
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
595
596
  			   pkt_dev->svlan_id, pkt_dev->svlan_p,
  			   pkt_dev->svlan_cfi);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
597

63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
598
  	if (pkt_dev->tos)
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
599
600
  		seq_printf(seq, "     tos: 0x%02x
  ", pkt_dev->tos);
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
601

63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
602
  	if (pkt_dev->traffic_class)
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
603
604
  		seq_printf(seq, "     traffic_class: 0x%02x
  ", pkt_dev->traffic_class);
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
605

e99b99b47   Robert Olsson   pktgen node alloc...
606
607
608
  	if (pkt_dev->node >= 0)
  		seq_printf(seq, "     node: %d
  ", pkt_dev->node);
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
609
  	seq_printf(seq, "     Flags: ");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
610
611
  	if (pkt_dev->flags & F_IPV6)
  		seq_printf(seq, "IPV6  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
612

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
613
614
  	if (pkt_dev->flags & F_IPSRC_RND)
  		seq_printf(seq, "IPSRC_RND  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
615

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
616
617
  	if (pkt_dev->flags & F_IPDST_RND)
  		seq_printf(seq, "IPDST_RND  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
618

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
619
620
  	if (pkt_dev->flags & F_TXSIZE_RND)
  		seq_printf(seq, "TXSIZE_RND  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
621

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
622
623
  	if (pkt_dev->flags & F_UDPSRC_RND)
  		seq_printf(seq, "UDPSRC_RND  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
624

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
625
626
  	if (pkt_dev->flags & F_UDPDST_RND)
  		seq_printf(seq, "UDPDST_RND  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
627

ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
628
629
  	if (pkt_dev->flags & F_MPLS_RND)
  		seq_printf(seq,  "MPLS_RND  ");
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
630
631
  	if (pkt_dev->flags & F_QUEUE_MAP_RND)
  		seq_printf(seq,  "QUEUE_MAP_RND  ");
e6fce5b91   Robert Olsson   pktgen: multiqueu...
632
633
  	if (pkt_dev->flags & F_QUEUE_MAP_CPU)
  		seq_printf(seq,  "QUEUE_MAP_CPU  ");
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
634
635
636
637
638
639
  	if (pkt_dev->cflows) {
  		if (pkt_dev->flags & F_FLOW_SEQ)
  			seq_printf(seq,  "FLOW_SEQ  "); /*in sequence flows*/
  		else
  			seq_printf(seq,  "FLOW_RND  ");
  	}
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
640
641
642
643
  #ifdef CONFIG_XFRM
  	if (pkt_dev->flags & F_IPSEC_ON)
  		seq_printf(seq,  "IPSEC  ");
  #endif
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
644
645
  	if (pkt_dev->flags & F_MACSRC_RND)
  		seq_printf(seq, "MACSRC_RND  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
646

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
647
648
  	if (pkt_dev->flags & F_MACDST_RND)
  		seq_printf(seq, "MACDST_RND  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
649

34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
650
651
652
653
654
  	if (pkt_dev->flags & F_VID_RND)
  		seq_printf(seq, "VID_RND  ");
  
  	if (pkt_dev->flags & F_SVID_RND)
  		seq_printf(seq, "SVID_RND  ");
e99b99b47   Robert Olsson   pktgen node alloc...
655
656
  	if (pkt_dev->flags & F_NODE)
  		seq_printf(seq, "NODE_ALLOC  ");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
657
658
  	seq_puts(seq, "
  ");
fd29cf726   Stephen Hemminger   pktgen: convert t...
659
660
661
662
  	/* not really stopped, more like last-running-at */
  	stopped = pkt_dev->running ? ktime_now() : pkt_dev->stopped_at;
  	idle = pkt_dev->idle_acc;
  	do_div(idle, NSEC_PER_USEC);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
663

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
664
  	seq_printf(seq,
fd29cf726   Stephen Hemminger   pktgen: convert t...
665
666
667
  		   "Current:
       pkts-sofar: %llu  errors: %llu
  ",
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
668
  		   (unsigned long long)pkt_dev->sofar,
fd29cf726   Stephen Hemminger   pktgen: convert t...
669
670
671
672
673
674
675
676
  		   (unsigned long long)pkt_dev->errors);
  
  	seq_printf(seq,
  		   "     started: %lluus  stopped: %lluus idle: %lluus
  ",
  		   (unsigned long long) ktime_to_us(pkt_dev->started_at),
  		   (unsigned long long) ktime_to_us(stopped),
  		   (unsigned long long) idle);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
677

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
678
679
680
  	seq_printf(seq,
  		   "     seq_num: %d  cur_dst_mac_offset: %d  cur_src_mac_offset: %d
  ",
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
681
682
  		   pkt_dev->seq_num, pkt_dev->cur_dst_mac_offset,
  		   pkt_dev->cur_src_mac_offset);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
683

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
684
  	if (pkt_dev->flags & F_IPV6) {
47a0200d5   Alexey Dobriyan   pktgen: use %pI6c...
685
686
687
688
  		seq_printf(seq, "     cur_saddr: %pI6c  cur_daddr: %pI6c
  ",
  				&pkt_dev->cur_in6_saddr,
  				&pkt_dev->cur_in6_daddr);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
689
690
691
  	} else
  		seq_printf(seq, "     cur_saddr: 0x%x  cur_daddr: 0x%x
  ",
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
692
  			   pkt_dev->cur_saddr, pkt_dev->cur_daddr);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
693

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
694
695
  	seq_printf(seq, "     cur_udp_dst: %d  cur_udp_src: %d
  ",
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
696
  		   pkt_dev->cur_udp_dst, pkt_dev->cur_udp_src);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
697

45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
698
699
  	seq_printf(seq, "     cur_queue_map: %u
  ", pkt_dev->cur_queue_map);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
700
701
  	seq_printf(seq, "     flows: %u
  ", pkt_dev->nflows);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
702
703
  
  	if (pkt_dev->result[0])
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
704
705
  		seq_printf(seq, "Result: %s
  ", pkt_dev->result);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
706
  	else
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
707
708
  		seq_printf(seq, "Result: Idle
  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
709

d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
710
  	return 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
711
  }
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
712

63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
713
714
  static int hex32_arg(const char __user *user_buffer, unsigned long maxlen,
  		     __u32 *num)
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
715
716
717
  {
  	int i = 0;
  	*num = 0;
e71a4783a   Stephen Hemminger   [NET] core: white...
718
  	for (; i < maxlen; i++) {
82fd5b5d1   Andy Shevchenko   net: core: use ke...
719
  		int value;
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
720
721
722
723
  		char c;
  		*num <<= 4;
  		if (get_user(c, &user_buffer[i]))
  			return -EFAULT;
82fd5b5d1   Andy Shevchenko   net: core: use ke...
724
725
726
  		value = hex_to_bin(c);
  		if (value >= 0)
  			*num |= value;
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
727
728
729
730
731
  		else
  			break;
  	}
  	return i;
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
732
733
  static int count_trail_chars(const char __user * user_buffer,
  			     unsigned int maxlen)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
734
735
736
737
  {
  	int i;
  
  	for (i = 0; i < maxlen; i++) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
738
739
740
741
  		char c;
  		if (get_user(c, &user_buffer[i]))
  			return -EFAULT;
  		switch (c) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
742
743
744
745
746
747
748
749
750
751
  		case '\"':
  		case '
  ':
  		case '\r':
  		case '\t':
  		case ' ':
  		case '=':
  			break;
  		default:
  			goto done;
3ff50b799   Stephen Hemminger   [NET]: cleanup ex...
752
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
753
754
755
756
  	}
  done:
  	return i;
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
757
758
  static unsigned long num_arg(const char __user * user_buffer,
  			     unsigned long maxlen, unsigned long *num)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
759
  {
d61822235   Paul Gortmaker   pktgen: clean up ...
760
  	int i;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
761
  	*num = 0;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
762

d61822235   Paul Gortmaker   pktgen: clean up ...
763
  	for (i = 0; i < maxlen; i++) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
764
765
766
767
  		char c;
  		if (get_user(c, &user_buffer[i]))
  			return -EFAULT;
  		if ((c >= '0') && (c <= '9')) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
768
  			*num *= 10;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
769
  			*num += c - '0';
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
770
771
772
773
774
  		} else
  			break;
  	}
  	return i;
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
775
  static int strn_len(const char __user * user_buffer, unsigned int maxlen)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
776
  {
d61822235   Paul Gortmaker   pktgen: clean up ...
777
  	int i;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
778

d61822235   Paul Gortmaker   pktgen: clean up ...
779
  	for (i = 0; i < maxlen; i++) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
780
781
782
783
  		char c;
  		if (get_user(c, &user_buffer[i]))
  			return -EFAULT;
  		switch (c) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
784
785
786
787
788
789
790
791
792
793
  		case '\"':
  		case '
  ':
  		case '\r':
  		case '\t':
  		case ' ':
  			goto done_str;
  			break;
  		default:
  			break;
3ff50b799   Stephen Hemminger   [NET]: cleanup ex...
794
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
795
796
  	}
  done_str:
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
797
798
  	return i;
  }
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
799
800
801
802
803
804
805
806
807
808
  static ssize_t get_labels(const char __user *buffer, struct pktgen_dev *pkt_dev)
  {
  	unsigned n = 0;
  	char c;
  	ssize_t i = 0;
  	int len;
  
  	pkt_dev->nr_labels = 0;
  	do {
  		__u32 tmp;
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
809
  		len = hex32_arg(&buffer[i], 8, &tmp);
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
810
811
812
813
814
815
816
817
818
819
820
821
  		if (len <= 0)
  			return len;
  		pkt_dev->labels[n] = htonl(tmp);
  		if (pkt_dev->labels[n] & MPLS_STACK_BOTTOM)
  			pkt_dev->flags |= F_MPLS_RND;
  		i += len;
  		if (get_user(c, &buffer[i]))
  			return -EFAULT;
  		i++;
  		n++;
  		if (n >= MAX_MPLS_LABELS)
  			return -E2BIG;
e71a4783a   Stephen Hemminger   [NET] core: white...
822
  	} while (c == ',');
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
823
824
825
826
  
  	pkt_dev->nr_labels = n;
  	return i;
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
827
828
829
  static ssize_t pktgen_if_write(struct file *file,
  			       const char __user * user_buffer, size_t count,
  			       loff_t * offset)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
830
  {
8a994a718   Joe Perches   net/core: Remove ...
831
  	struct seq_file *seq = file->private_data;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
832
  	struct pktgen_dev *pkt_dev = seq->private;
d61822235   Paul Gortmaker   pktgen: clean up ...
833
  	int i, max, len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
834
835
  	char name[16], valstr[32];
  	unsigned long value = 0;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
836
837
  	char *pg_result = NULL;
  	int tmp = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
838
  	char buf[128];
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
839
840
  
  	pg_result = &(pkt_dev->result[0]);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
841
  	if (count < 1) {
f9467eaec   Joe Perches   net/core/pktgen.c...
842
843
  		pr_warning("wrong command format
  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
844
845
  		return -EINVAL;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
846

d61822235   Paul Gortmaker   pktgen: clean up ...
847
848
  	max = count;
  	tmp = count_trail_chars(user_buffer, max);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
849
  	if (tmp < 0) {
f9467eaec   Joe Perches   net/core/pktgen.c...
850
851
  		pr_warning("illegal format
  ");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
852
  		return tmp;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
853
  	}
d61822235   Paul Gortmaker   pktgen: clean up ...
854
  	i = tmp;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
855

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
856
857
858
  	/* Read variable name */
  
  	len = strn_len(&user_buffer[i], sizeof(name) - 1);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
859
  	if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
860
  		return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
861

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
862
  	memset(name, 0, sizeof(name));
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
863
  	if (copy_from_user(name, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
864
865
  		return -EFAULT;
  	i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
866
867
  
  	max = count - i;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
868
  	len = count_trail_chars(&user_buffer[i], max);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
869
870
  	if (len < 0)
  		return len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
871
872
873
  	i += len;
  
  	if (debug) {
86c2c0a8a   Dmitry Torokhov   NET: pktgen - fix...
874
  		size_t copy = min_t(size_t, count, 1023);
448d7b5da   Nelson Elhage   pktgen: Limit how...
875
876
  		char tb[copy + 1];
  		if (copy_from_user(tb, user_buffer, copy))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
877
  			return -EFAULT;
448d7b5da   Nelson Elhage   pktgen: Limit how...
878
  		tb[copy] = 0;
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
879
880
  		printk(KERN_DEBUG "pktgen: %s,%lu  buffer -:%s:-
  ", name,
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
881
882
  		       (unsigned long)count, tb);
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
883
884
885
  
  	if (!strcmp(name, "min_pkt_size")) {
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
886
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
887
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
888

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
889
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
890
891
892
893
894
895
896
897
  		if (value < 14 + 20 + 8)
  			value = 14 + 20 + 8;
  		if (value != pkt_dev->min_pkt_size) {
  			pkt_dev->min_pkt_size = value;
  			pkt_dev->cur_pkt_size = value;
  		}
  		sprintf(pg_result, "OK: min_pkt_size=%u",
  			pkt_dev->min_pkt_size);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
898
899
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
900
  	if (!strcmp(name, "max_pkt_size")) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
901
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
902
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
903
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
904

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
905
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
906
907
908
909
910
911
912
913
  		if (value < 14 + 20 + 8)
  			value = 14 + 20 + 8;
  		if (value != pkt_dev->max_pkt_size) {
  			pkt_dev->max_pkt_size = value;
  			pkt_dev->cur_pkt_size = value;
  		}
  		sprintf(pg_result, "OK: max_pkt_size=%u",
  			pkt_dev->max_pkt_size);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
914
915
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
916
  	/* Shortcut for min = max */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
917
918
919
  
  	if (!strcmp(name, "pkt_size")) {
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
920
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
921
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
922

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
923
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
924
925
926
927
928
929
930
  		if (value < 14 + 20 + 8)
  			value = 14 + 20 + 8;
  		if (value != pkt_dev->min_pkt_size) {
  			pkt_dev->min_pkt_size = value;
  			pkt_dev->max_pkt_size = value;
  			pkt_dev->cur_pkt_size = value;
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
931
932
933
  		sprintf(pg_result, "OK: pkt_size=%u", pkt_dev->min_pkt_size);
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
934
  	if (!strcmp(name, "debug")) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
935
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
936
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
937
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
938

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
939
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
940
  		debug = value;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
941
942
943
  		sprintf(pg_result, "OK: debug=%u", debug);
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
944
  	if (!strcmp(name, "frags")) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
945
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
946
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
947
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
948

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
949
950
951
952
953
954
955
  		i += len;
  		pkt_dev->nfrags = value;
  		sprintf(pg_result, "OK: frags=%u", pkt_dev->nfrags);
  		return count;
  	}
  	if (!strcmp(name, "delay")) {
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
956
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
957
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
958

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
959
  		i += len;
fd29cf726   Stephen Hemminger   pktgen: convert t...
960
961
962
  		if (value == 0x7FFFFFFF)
  			pkt_dev->delay = ULLONG_MAX;
  		else
9240d7154   Eric Dumazet   pktgen: restore n...
963
  			pkt_dev->delay = (u64)value;
fd29cf726   Stephen Hemminger   pktgen: convert t...
964
965
966
  
  		sprintf(pg_result, "OK: delay=%llu",
  			(unsigned long long) pkt_dev->delay);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
967
968
  		return count;
  	}
43d28b651   Daniel Turull   pktgen: increasin...
969
970
971
972
973
974
975
976
977
978
  	if (!strcmp(name, "rate")) {
  		len = num_arg(&user_buffer[i], 10, &value);
  		if (len < 0)
  			return len;
  
  		i += len;
  		if (!value)
  			return len;
  		pkt_dev->delay = pkt_dev->min_pkt_size*8*NSEC_PER_USEC/value;
  		if (debug)
f9467eaec   Joe Perches   net/core/pktgen.c...
979
980
  			pr_info("Delay set at: %llu ns
  ", pkt_dev->delay);
43d28b651   Daniel Turull   pktgen: increasin...
981
982
983
984
985
986
987
988
989
990
991
992
993
994
  
  		sprintf(pg_result, "OK: rate=%lu", value);
  		return count;
  	}
  	if (!strcmp(name, "ratep")) {
  		len = num_arg(&user_buffer[i], 10, &value);
  		if (len < 0)
  			return len;
  
  		i += len;
  		if (!value)
  			return len;
  		pkt_dev->delay = NSEC_PER_SEC/value;
  		if (debug)
f9467eaec   Joe Perches   net/core/pktgen.c...
995
996
  			pr_info("Delay set at: %llu ns
  ", pkt_dev->delay);
43d28b651   Daniel Turull   pktgen: increasin...
997
998
999
1000
  
  		sprintf(pg_result, "OK: rate=%lu", value);
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1001
  	if (!strcmp(name, "udp_src_min")) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1002
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1003
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1004
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1005

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1006
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1007
1008
1009
1010
  		if (value != pkt_dev->udp_src_min) {
  			pkt_dev->udp_src_min = value;
  			pkt_dev->cur_udp_src = value;
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1011
1012
1013
  		sprintf(pg_result, "OK: udp_src_min=%u", pkt_dev->udp_src_min);
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1014
  	if (!strcmp(name, "udp_dst_min")) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1015
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1016
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1017
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1018

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1019
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1020
1021
1022
1023
  		if (value != pkt_dev->udp_dst_min) {
  			pkt_dev->udp_dst_min = value;
  			pkt_dev->cur_udp_dst = value;
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1024
1025
1026
  		sprintf(pg_result, "OK: udp_dst_min=%u", pkt_dev->udp_dst_min);
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1027
  	if (!strcmp(name, "udp_src_max")) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1028
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1029
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1030
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1031

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1032
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1033
1034
1035
1036
  		if (value != pkt_dev->udp_src_max) {
  			pkt_dev->udp_src_max = value;
  			pkt_dev->cur_udp_src = value;
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1037
1038
1039
  		sprintf(pg_result, "OK: udp_src_max=%u", pkt_dev->udp_src_max);
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1040
  	if (!strcmp(name, "udp_dst_max")) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1041
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1042
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1043
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1044

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1045
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1046
1047
1048
1049
  		if (value != pkt_dev->udp_dst_max) {
  			pkt_dev->udp_dst_max = value;
  			pkt_dev->cur_udp_dst = value;
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1050
1051
1052
1053
1054
  		sprintf(pg_result, "OK: udp_dst_max=%u", pkt_dev->udp_dst_max);
  		return count;
  	}
  	if (!strcmp(name, "clone_skb")) {
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1055
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1056
  			return len;
d88733150   Neil Horman   net: add IFF_SKB_...
1057
1058
1059
  		if ((value > 0) &&
  		    (!(pkt_dev->odev->priv_flags & IFF_TX_SKB_SHARING)))
  			return -ENOTSUPP;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1060
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1061
  		pkt_dev->clone_skb = value;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1062
1063
1064
1065
1066
  		sprintf(pg_result, "OK: clone_skb=%d", pkt_dev->clone_skb);
  		return count;
  	}
  	if (!strcmp(name, "count")) {
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1067
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1068
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1069

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1070
1071
1072
  		i += len;
  		pkt_dev->count = value;
  		sprintf(pg_result, "OK: count=%llu",
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1073
  			(unsigned long long)pkt_dev->count);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1074
1075
1076
1077
  		return count;
  	}
  	if (!strcmp(name, "src_mac_count")) {
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1078
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1079
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1080

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1081
1082
  		i += len;
  		if (pkt_dev->src_mac_count != value) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1083
1084
1085
1086
1087
  			pkt_dev->src_mac_count = value;
  			pkt_dev->cur_src_mac_offset = 0;
  		}
  		sprintf(pg_result, "OK: src_mac_count=%d",
  			pkt_dev->src_mac_count);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1088
1089
1090
1091
  		return count;
  	}
  	if (!strcmp(name, "dst_mac_count")) {
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1092
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1093
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1094

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1095
1096
  		i += len;
  		if (pkt_dev->dst_mac_count != value) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1097
1098
1099
1100
1101
  			pkt_dev->dst_mac_count = value;
  			pkt_dev->cur_dst_mac_offset = 0;
  		}
  		sprintf(pg_result, "OK: dst_mac_count=%d",
  			pkt_dev->dst_mac_count);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1102
1103
  		return count;
  	}
e99b99b47   Robert Olsson   pktgen node alloc...
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
  	if (!strcmp(name, "node")) {
  		len = num_arg(&user_buffer[i], 10, &value);
  		if (len < 0)
  			return len;
  
  		i += len;
  
  		if (node_possible(value)) {
  			pkt_dev->node = value;
  			sprintf(pg_result, "OK: node=%d", pkt_dev->node);
26ad78796   Eric Dumazet   pktgen: speedup f...
1114
1115
1116
1117
  			if (pkt_dev->page) {
  				put_page(pkt_dev->page);
  				pkt_dev->page = NULL;
  			}
e99b99b47   Robert Olsson   pktgen node alloc...
1118
1119
1120
1121
1122
  		}
  		else
  			sprintf(pg_result, "ERROR: node not possible");
  		return count;
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1123
  	if (!strcmp(name, "flag")) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1124
1125
  		char f[32];
  		memset(f, 0, 32);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1126
  		len = strn_len(&user_buffer[i], sizeof(f) - 1);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1127
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1128
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1129

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1130
1131
1132
  		if (copy_from_user(f, &user_buffer[i], len))
  			return -EFAULT;
  		i += len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
  		if (strcmp(f, "IPSRC_RND") == 0)
  			pkt_dev->flags |= F_IPSRC_RND;
  
  		else if (strcmp(f, "!IPSRC_RND") == 0)
  			pkt_dev->flags &= ~F_IPSRC_RND;
  
  		else if (strcmp(f, "TXSIZE_RND") == 0)
  			pkt_dev->flags |= F_TXSIZE_RND;
  
  		else if (strcmp(f, "!TXSIZE_RND") == 0)
  			pkt_dev->flags &= ~F_TXSIZE_RND;
  
  		else if (strcmp(f, "IPDST_RND") == 0)
  			pkt_dev->flags |= F_IPDST_RND;
  
  		else if (strcmp(f, "!IPDST_RND") == 0)
  			pkt_dev->flags &= ~F_IPDST_RND;
  
  		else if (strcmp(f, "UDPSRC_RND") == 0)
  			pkt_dev->flags |= F_UDPSRC_RND;
  
  		else if (strcmp(f, "!UDPSRC_RND") == 0)
  			pkt_dev->flags &= ~F_UDPSRC_RND;
  
  		else if (strcmp(f, "UDPDST_RND") == 0)
  			pkt_dev->flags |= F_UDPDST_RND;
  
  		else if (strcmp(f, "!UDPDST_RND") == 0)
  			pkt_dev->flags &= ~F_UDPDST_RND;
  
  		else if (strcmp(f, "MACSRC_RND") == 0)
  			pkt_dev->flags |= F_MACSRC_RND;
  
  		else if (strcmp(f, "!MACSRC_RND") == 0)
  			pkt_dev->flags &= ~F_MACSRC_RND;
  
  		else if (strcmp(f, "MACDST_RND") == 0)
  			pkt_dev->flags |= F_MACDST_RND;
  
  		else if (strcmp(f, "!MACDST_RND") == 0)
  			pkt_dev->flags &= ~F_MACDST_RND;
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
1174
1175
1176
1177
1178
  		else if (strcmp(f, "MPLS_RND") == 0)
  			pkt_dev->flags |= F_MPLS_RND;
  
  		else if (strcmp(f, "!MPLS_RND") == 0)
  			pkt_dev->flags &= ~F_MPLS_RND;
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
  		else if (strcmp(f, "VID_RND") == 0)
  			pkt_dev->flags |= F_VID_RND;
  
  		else if (strcmp(f, "!VID_RND") == 0)
  			pkt_dev->flags &= ~F_VID_RND;
  
  		else if (strcmp(f, "SVID_RND") == 0)
  			pkt_dev->flags |= F_SVID_RND;
  
  		else if (strcmp(f, "!SVID_RND") == 0)
  			pkt_dev->flags &= ~F_SVID_RND;
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
1190
1191
  		else if (strcmp(f, "FLOW_SEQ") == 0)
  			pkt_dev->flags |= F_FLOW_SEQ;
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
1192
1193
1194
1195
1196
  		else if (strcmp(f, "QUEUE_MAP_RND") == 0)
  			pkt_dev->flags |= F_QUEUE_MAP_RND;
  
  		else if (strcmp(f, "!QUEUE_MAP_RND") == 0)
  			pkt_dev->flags &= ~F_QUEUE_MAP_RND;
e6fce5b91   Robert Olsson   pktgen: multiqueu...
1197
1198
1199
1200
1201
1202
  
  		else if (strcmp(f, "QUEUE_MAP_CPU") == 0)
  			pkt_dev->flags |= F_QUEUE_MAP_CPU;
  
  		else if (strcmp(f, "!QUEUE_MAP_CPU") == 0)
  			pkt_dev->flags &= ~F_QUEUE_MAP_CPU;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
1203
1204
1205
1206
  #ifdef CONFIG_XFRM
  		else if (strcmp(f, "IPSEC") == 0)
  			pkt_dev->flags |= F_IPSEC_ON;
  #endif
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
1207
1208
  		else if (strcmp(f, "!IPV6") == 0)
  			pkt_dev->flags &= ~F_IPV6;
e99b99b47   Robert Olsson   pktgen node alloc...
1209
1210
1211
1212
1213
  		else if (strcmp(f, "NODE_ALLOC") == 0)
  			pkt_dev->flags |= F_NODE;
  
  		else if (strcmp(f, "!NODE_ALLOC") == 0)
  			pkt_dev->flags &= ~F_NODE;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1214
1215
1216
1217
1218
1219
  		else {
  			sprintf(pg_result,
  				"Flag -:%s:- unknown
  Available flags, (prepend ! to un-set flag):
  %s",
  				f,
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
1220
  				"IPSRC_RND, IPDST_RND, UDPSRC_RND, UDPDST_RND, "
e99b99b47   Robert Olsson   pktgen node alloc...
1221
1222
  				"MACSRC_RND, MACDST_RND, TXSIZE_RND, IPV6, MPLS_RND, VID_RND, SVID_RND, FLOW_SEQ, IPSEC, NODE_ALLOC
  ");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1223
1224
  			return count;
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1225
1226
1227
1228
1229
  		sprintf(pg_result, "OK: flags=0x%x", pkt_dev->flags);
  		return count;
  	}
  	if (!strcmp(name, "dst_min") || !strcmp(name, "dst")) {
  		len = strn_len(&user_buffer[i], sizeof(pkt_dev->dst_min) - 1);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1230
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1231
  			return len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1232

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1233
  		if (copy_from_user(buf, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1234
  			return -EFAULT;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1235
1236
1237
1238
1239
1240
1241
1242
  		buf[len] = 0;
  		if (strcmp(buf, pkt_dev->dst_min) != 0) {
  			memset(pkt_dev->dst_min, 0, sizeof(pkt_dev->dst_min));
  			strncpy(pkt_dev->dst_min, buf, len);
  			pkt_dev->daddr_min = in_aton(pkt_dev->dst_min);
  			pkt_dev->cur_daddr = pkt_dev->daddr_min;
  		}
  		if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1243
1244
  			printk(KERN_DEBUG "pktgen: dst_min set to: %s
  ",
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1245
1246
  			       pkt_dev->dst_min);
  		i += len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1247
1248
1249
1250
1251
  		sprintf(pg_result, "OK: dst_min=%s", pkt_dev->dst_min);
  		return count;
  	}
  	if (!strcmp(name, "dst_max")) {
  		len = strn_len(&user_buffer[i], sizeof(pkt_dev->dst_max) - 1);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1252
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1253
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1254

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1255

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1256
  		if (copy_from_user(buf, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1257
  			return -EFAULT;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1258
1259
1260
1261
1262
1263
1264
1265
  		buf[len] = 0;
  		if (strcmp(buf, pkt_dev->dst_max) != 0) {
  			memset(pkt_dev->dst_max, 0, sizeof(pkt_dev->dst_max));
  			strncpy(pkt_dev->dst_max, buf, len);
  			pkt_dev->daddr_max = in_aton(pkt_dev->dst_max);
  			pkt_dev->cur_daddr = pkt_dev->daddr_max;
  		}
  		if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1266
1267
  			printk(KERN_DEBUG "pktgen: dst_max set to: %s
  ",
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1268
  			       pkt_dev->dst_max);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1269
1270
1271
1272
1273
1274
  		i += len;
  		sprintf(pg_result, "OK: dst_max=%s", pkt_dev->dst_max);
  		return count;
  	}
  	if (!strcmp(name, "dst6")) {
  		len = strn_len(&user_buffer[i], sizeof(buf) - 1);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1275
1276
  		if (len < 0)
  			return len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1277
1278
  
  		pkt_dev->flags |= F_IPV6;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1279
  		if (copy_from_user(buf, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1280
  			return -EFAULT;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1281
  		buf[len] = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1282
1283
  
  		scan_ip6(buf, pkt_dev->in6_daddr.s6_addr);
47a0200d5   Alexey Dobriyan   pktgen: use %pI6c...
1284
  		snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->in6_daddr);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1285

4e3fd7a06   Alexey Dobriyan   net: remove ipv6_...
1286
  		pkt_dev->cur_in6_daddr = pkt_dev->in6_daddr;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1287

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1288
  		if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1289
1290
  			printk(KERN_DEBUG "pktgen: dst6 set to: %s
  ", buf);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1291

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1292
  		i += len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1293
1294
1295
1296
1297
  		sprintf(pg_result, "OK: dst6=%s", buf);
  		return count;
  	}
  	if (!strcmp(name, "dst6_min")) {
  		len = strn_len(&user_buffer[i], sizeof(buf) - 1);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1298
1299
  		if (len < 0)
  			return len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1300
1301
  
  		pkt_dev->flags |= F_IPV6;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1302
  		if (copy_from_user(buf, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1303
  			return -EFAULT;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1304
  		buf[len] = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1305
1306
  
  		scan_ip6(buf, pkt_dev->min_in6_daddr.s6_addr);
47a0200d5   Alexey Dobriyan   pktgen: use %pI6c...
1307
  		snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->min_in6_daddr);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1308

4e3fd7a06   Alexey Dobriyan   net: remove ipv6_...
1309
  		pkt_dev->cur_in6_daddr = pkt_dev->min_in6_daddr;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1310
  		if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1311
1312
  			printk(KERN_DEBUG "pktgen: dst6_min set to: %s
  ", buf);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1313

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1314
  		i += len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1315
1316
1317
1318
1319
  		sprintf(pg_result, "OK: dst6_min=%s", buf);
  		return count;
  	}
  	if (!strcmp(name, "dst6_max")) {
  		len = strn_len(&user_buffer[i], sizeof(buf) - 1);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1320
1321
  		if (len < 0)
  			return len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1322
1323
  
  		pkt_dev->flags |= F_IPV6;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1324
  		if (copy_from_user(buf, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1325
  			return -EFAULT;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1326
  		buf[len] = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1327
1328
  
  		scan_ip6(buf, pkt_dev->max_in6_daddr.s6_addr);
47a0200d5   Alexey Dobriyan   pktgen: use %pI6c...
1329
  		snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->max_in6_daddr);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1330

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1331
  		if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1332
1333
  			printk(KERN_DEBUG "pktgen: dst6_max set to: %s
  ", buf);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1334

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1335
  		i += len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1336
1337
1338
1339
1340
  		sprintf(pg_result, "OK: dst6_max=%s", buf);
  		return count;
  	}
  	if (!strcmp(name, "src6")) {
  		len = strn_len(&user_buffer[i], sizeof(buf) - 1);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1341
1342
  		if (len < 0)
  			return len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1343
1344
  
  		pkt_dev->flags |= F_IPV6;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1345
  		if (copy_from_user(buf, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1346
  			return -EFAULT;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1347
  		buf[len] = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1348
1349
  
  		scan_ip6(buf, pkt_dev->in6_saddr.s6_addr);
47a0200d5   Alexey Dobriyan   pktgen: use %pI6c...
1350
  		snprintf(buf, sizeof(buf), "%pI6c", &pkt_dev->in6_saddr);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1351

4e3fd7a06   Alexey Dobriyan   net: remove ipv6_...
1352
  		pkt_dev->cur_in6_saddr = pkt_dev->in6_saddr;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1353

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1354
  		if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1355
1356
  			printk(KERN_DEBUG "pktgen: src6 set to: %s
  ", buf);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1357
1358
  
  		i += len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1359
1360
1361
1362
1363
  		sprintf(pg_result, "OK: src6=%s", buf);
  		return count;
  	}
  	if (!strcmp(name, "src_min")) {
  		len = strn_len(&user_buffer[i], sizeof(pkt_dev->src_min) - 1);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1364
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1365
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1366

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1367
  		if (copy_from_user(buf, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1368
  			return -EFAULT;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1369
1370
1371
1372
1373
1374
1375
1376
  		buf[len] = 0;
  		if (strcmp(buf, pkt_dev->src_min) != 0) {
  			memset(pkt_dev->src_min, 0, sizeof(pkt_dev->src_min));
  			strncpy(pkt_dev->src_min, buf, len);
  			pkt_dev->saddr_min = in_aton(pkt_dev->src_min);
  			pkt_dev->cur_saddr = pkt_dev->saddr_min;
  		}
  		if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1377
1378
  			printk(KERN_DEBUG "pktgen: src_min set to: %s
  ",
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1379
  			       pkt_dev->src_min);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1380
1381
1382
1383
1384
1385
  		i += len;
  		sprintf(pg_result, "OK: src_min=%s", pkt_dev->src_min);
  		return count;
  	}
  	if (!strcmp(name, "src_max")) {
  		len = strn_len(&user_buffer[i], sizeof(pkt_dev->src_max) - 1);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1386
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1387
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1388

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1389
  		if (copy_from_user(buf, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1390
  			return -EFAULT;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1391
1392
1393
1394
1395
1396
1397
1398
  		buf[len] = 0;
  		if (strcmp(buf, pkt_dev->src_max) != 0) {
  			memset(pkt_dev->src_max, 0, sizeof(pkt_dev->src_max));
  			strncpy(pkt_dev->src_max, buf, len);
  			pkt_dev->saddr_max = in_aton(pkt_dev->src_max);
  			pkt_dev->cur_saddr = pkt_dev->saddr_max;
  		}
  		if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1399
1400
  			printk(KERN_DEBUG "pktgen: src_max set to: %s
  ",
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1401
  			       pkt_dev->src_max);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1402
1403
1404
1405
1406
  		i += len;
  		sprintf(pg_result, "OK: src_max=%s", pkt_dev->src_max);
  		return count;
  	}
  	if (!strcmp(name, "dst_mac")) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1407
  		len = strn_len(&user_buffer[i], sizeof(valstr) - 1);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1408
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1409
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1410

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1411
  		memset(valstr, 0, sizeof(valstr));
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1412
  		if (copy_from_user(valstr, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1413
  			return -EFAULT;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1414

4940fc889   Alexey Dobriyan   net: add mac_pton...
1415
1416
  		if (!mac_pton(valstr, pkt_dev->dst_mac))
  			return -EINVAL;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1417
  		/* Set up Dest MAC */
4940fc889   Alexey Dobriyan   net: add mac_pton...
1418
  		memcpy(&pkt_dev->hh[0], pkt_dev->dst_mac, ETH_ALEN);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1419

4940fc889   Alexey Dobriyan   net: add mac_pton...
1420
  		sprintf(pg_result, "OK: dstmac %pM", pkt_dev->dst_mac);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1421
1422
1423
  		return count;
  	}
  	if (!strcmp(name, "src_mac")) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1424
  		len = strn_len(&user_buffer[i], sizeof(valstr) - 1);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1425
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1426
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1427

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1428
  		memset(valstr, 0, sizeof(valstr));
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1429
  		if (copy_from_user(valstr, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1430
  			return -EFAULT;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1431

4940fc889   Alexey Dobriyan   net: add mac_pton...
1432
1433
  		if (!mac_pton(valstr, pkt_dev->src_mac))
  			return -EINVAL;
ce5d0b47f   Adit Ranadive   [PKTGEN]: srcmac fix
1434
  		/* Set up Src MAC */
4940fc889   Alexey Dobriyan   net: add mac_pton...
1435
  		memcpy(&pkt_dev->hh[6], pkt_dev->src_mac, ETH_ALEN);
ce5d0b47f   Adit Ranadive   [PKTGEN]: srcmac fix
1436

4940fc889   Alexey Dobriyan   net: add mac_pton...
1437
  		sprintf(pg_result, "OK: srcmac %pM", pkt_dev->src_mac);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1438
1439
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1440
1441
1442
1443
1444
1445
  	if (!strcmp(name, "clear_counters")) {
  		pktgen_clear_counters(pkt_dev);
  		sprintf(pg_result, "OK: Clearing counters.
  ");
  		return count;
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1446
1447
1448
  
  	if (!strcmp(name, "flows")) {
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1449
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1450
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1451

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
  		i += len;
  		if (value > MAX_CFLOWS)
  			value = MAX_CFLOWS;
  
  		pkt_dev->cflows = value;
  		sprintf(pg_result, "OK: flows=%u", pkt_dev->cflows);
  		return count;
  	}
  
  	if (!strcmp(name, "flowlen")) {
  		len = num_arg(&user_buffer[i], 10, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1463
  		if (len < 0)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1464
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1465

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1466
1467
1468
1469
1470
  		i += len;
  		pkt_dev->lflow = value;
  		sprintf(pg_result, "OK: flowlen=%u", pkt_dev->lflow);
  		return count;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1471

45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
1472
1473
  	if (!strcmp(name, "queue_map_min")) {
  		len = num_arg(&user_buffer[i], 5, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1474
  		if (len < 0)
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
1475
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1476

45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
1477
1478
1479
1480
1481
1482
1483
1484
  		i += len;
  		pkt_dev->queue_map_min = value;
  		sprintf(pg_result, "OK: queue_map_min=%u", pkt_dev->queue_map_min);
  		return count;
  	}
  
  	if (!strcmp(name, "queue_map_max")) {
  		len = num_arg(&user_buffer[i], 5, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1485
  		if (len < 0)
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
1486
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1487

45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
1488
1489
1490
1491
1492
  		i += len;
  		pkt_dev->queue_map_max = value;
  		sprintf(pg_result, "OK: queue_map_max=%u", pkt_dev->queue_map_max);
  		return count;
  	}
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
1493
  	if (!strcmp(name, "mpls")) {
cfcabdcc2   Stephen Hemminger   [NET]: sparse war...
1494
  		unsigned n, cnt;
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
1495
  		len = get_labels(&user_buffer[i], pkt_dev);
cfcabdcc2   Stephen Hemminger   [NET]: sparse war...
1496
1497
  		if (len < 0)
  			return len;
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
1498
  		i += len;
cfcabdcc2   Stephen Hemminger   [NET]: sparse war...
1499
  		cnt = sprintf(pg_result, "OK: mpls=");
e71a4783a   Stephen Hemminger   [NET] core: white...
1500
  		for (n = 0; n < pkt_dev->nr_labels; n++)
cfcabdcc2   Stephen Hemminger   [NET]: sparse war...
1501
1502
1503
  			cnt += sprintf(pg_result + cnt,
  				       "%08x%s", ntohl(pkt_dev->labels[n]),
  				       n == pkt_dev->nr_labels-1 ? "" : ",");
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1504
1505
1506
1507
1508
1509
  
  		if (pkt_dev->nr_labels && pkt_dev->vlan_id != 0xffff) {
  			pkt_dev->vlan_id = 0xffff; /* turn off VLAN/SVLAN */
  			pkt_dev->svlan_id = 0xffff;
  
  			if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1510
1511
  				printk(KERN_DEBUG "pktgen: VLAN/SVLAN auto turned off
  ");
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1512
1513
1514
1515
1516
1517
  		}
  		return count;
  	}
  
  	if (!strcmp(name, "vlan_id")) {
  		len = num_arg(&user_buffer[i], 4, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1518
  		if (len < 0)
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1519
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1520

34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1521
1522
1523
1524
1525
  		i += len;
  		if (value <= 4095) {
  			pkt_dev->vlan_id = value;  /* turn on VLAN */
  
  			if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1526
1527
  				printk(KERN_DEBUG "pktgen: VLAN turned on
  ");
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1528
1529
  
  			if (debug && pkt_dev->nr_labels)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1530
1531
  				printk(KERN_DEBUG "pktgen: MPLS auto turned off
  ");
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1532
1533
1534
1535
1536
1537
1538
1539
  
  			pkt_dev->nr_labels = 0;    /* turn off MPLS */
  			sprintf(pg_result, "OK: vlan_id=%u", pkt_dev->vlan_id);
  		} else {
  			pkt_dev->vlan_id = 0xffff; /* turn off VLAN/SVLAN */
  			pkt_dev->svlan_id = 0xffff;
  
  			if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1540
1541
  				printk(KERN_DEBUG "pktgen: VLAN/SVLAN turned off
  ");
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1542
1543
1544
1545
1546
1547
  		}
  		return count;
  	}
  
  	if (!strcmp(name, "vlan_p")) {
  		len = num_arg(&user_buffer[i], 1, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1548
  		if (len < 0)
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1549
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1550

34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
  		i += len;
  		if ((value <= 7) && (pkt_dev->vlan_id != 0xffff)) {
  			pkt_dev->vlan_p = value;
  			sprintf(pg_result, "OK: vlan_p=%u", pkt_dev->vlan_p);
  		} else {
  			sprintf(pg_result, "ERROR: vlan_p must be 0-7");
  		}
  		return count;
  	}
  
  	if (!strcmp(name, "vlan_cfi")) {
  		len = num_arg(&user_buffer[i], 1, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1563
  		if (len < 0)
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1564
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1565

34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
  		i += len;
  		if ((value <= 1) && (pkt_dev->vlan_id != 0xffff)) {
  			pkt_dev->vlan_cfi = value;
  			sprintf(pg_result, "OK: vlan_cfi=%u", pkt_dev->vlan_cfi);
  		} else {
  			sprintf(pg_result, "ERROR: vlan_cfi must be 0-1");
  		}
  		return count;
  	}
  
  	if (!strcmp(name, "svlan_id")) {
  		len = num_arg(&user_buffer[i], 4, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1578
  		if (len < 0)
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1579
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1580

34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1581
1582
1583
1584
1585
  		i += len;
  		if ((value <= 4095) && ((pkt_dev->vlan_id != 0xffff))) {
  			pkt_dev->svlan_id = value;  /* turn on SVLAN */
  
  			if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1586
1587
  				printk(KERN_DEBUG "pktgen: SVLAN turned on
  ");
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1588
1589
  
  			if (debug && pkt_dev->nr_labels)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1590
1591
  				printk(KERN_DEBUG "pktgen: MPLS auto turned off
  ");
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1592
1593
1594
1595
1596
1597
1598
1599
  
  			pkt_dev->nr_labels = 0;    /* turn off MPLS */
  			sprintf(pg_result, "OK: svlan_id=%u", pkt_dev->svlan_id);
  		} else {
  			pkt_dev->vlan_id = 0xffff; /* turn off VLAN/SVLAN */
  			pkt_dev->svlan_id = 0xffff;
  
  			if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1600
1601
  				printk(KERN_DEBUG "pktgen: VLAN/SVLAN turned off
  ");
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1602
1603
1604
1605
1606
1607
  		}
  		return count;
  	}
  
  	if (!strcmp(name, "svlan_p")) {
  		len = num_arg(&user_buffer[i], 1, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1608
  		if (len < 0)
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1609
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1610

34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
  		i += len;
  		if ((value <= 7) && (pkt_dev->svlan_id != 0xffff)) {
  			pkt_dev->svlan_p = value;
  			sprintf(pg_result, "OK: svlan_p=%u", pkt_dev->svlan_p);
  		} else {
  			sprintf(pg_result, "ERROR: svlan_p must be 0-7");
  		}
  		return count;
  	}
  
  	if (!strcmp(name, "svlan_cfi")) {
  		len = num_arg(&user_buffer[i], 1, &value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1623
  		if (len < 0)
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1624
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1625

34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
1626
1627
1628
1629
1630
1631
1632
  		i += len;
  		if ((value <= 1) && (pkt_dev->svlan_id != 0xffff)) {
  			pkt_dev->svlan_cfi = value;
  			sprintf(pg_result, "OK: svlan_cfi=%u", pkt_dev->svlan_cfi);
  		} else {
  			sprintf(pg_result, "ERROR: svlan_cfi must be 0-1");
  		}
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
1633
1634
  		return count;
  	}
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
1635
1636
1637
  	if (!strcmp(name, "tos")) {
  		__u32 tmp_value = 0;
  		len = hex32_arg(&user_buffer[i], 2, &tmp_value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1638
  		if (len < 0)
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
1639
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1640

1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
  		i += len;
  		if (len == 2) {
  			pkt_dev->tos = tmp_value;
  			sprintf(pg_result, "OK: tos=0x%02x", pkt_dev->tos);
  		} else {
  			sprintf(pg_result, "ERROR: tos must be 00-ff");
  		}
  		return count;
  	}
  
  	if (!strcmp(name, "traffic_class")) {
  		__u32 tmp_value = 0;
  		len = hex32_arg(&user_buffer[i], 2, &tmp_value);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1654
  		if (len < 0)
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
1655
  			return len;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1656

1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
1657
1658
1659
1660
1661
1662
1663
1664
1665
  		i += len;
  		if (len == 2) {
  			pkt_dev->traffic_class = tmp_value;
  			sprintf(pg_result, "OK: traffic_class=0x%02x", pkt_dev->traffic_class);
  		} else {
  			sprintf(pg_result, "ERROR: traffic_class must be 00-ff");
  		}
  		return count;
  	}
9e50e3ac5   John Fastabend   net: add priority...
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
  	if (!strcmp(name, "skb_priority")) {
  		len = num_arg(&user_buffer[i], 9, &value);
  		if (len < 0)
  			return len;
  
  		i += len;
  		pkt_dev->skb_priority = value;
  		sprintf(pg_result, "OK: skb_priority=%i",
  			pkt_dev->skb_priority);
  		return count;
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1677
1678
1679
  	sprintf(pkt_dev->result, "No such parameter \"%s\"", name);
  	return -EINVAL;
  }
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1680
  static int pktgen_if_open(struct inode *inode, struct file *file)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1681
  {
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1682
1683
  	return single_open(file, pktgen_if_show, PDE(inode)->data);
  }
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1684

9a32144e9   Arjan van de Ven   [PATCH] mark stru...
1685
  static const struct file_operations pktgen_if_fops = {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1686
1687
1688
1689
1690
1691
  	.owner   = THIS_MODULE,
  	.open    = pktgen_if_open,
  	.read    = seq_read,
  	.llseek  = seq_lseek,
  	.write   = pktgen_if_write,
  	.release = single_release,
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1692
  };
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1693

d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1694
1695
  static int pktgen_thread_show(struct seq_file *seq, void *v)
  {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1696
  	struct pktgen_thread *t = seq->private;
648fda740   Stephen Hemminger   pktgen: minor cle...
1697
  	const struct pktgen_dev *pkt_dev;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1698
1699
  
  	BUG_ON(!t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1700

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1701
  	seq_printf(seq, "Running: ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1702

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1703
  	if_lock(t);
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
1704
  	list_for_each_entry(pkt_dev, &t->if_list, list)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1705
  		if (pkt_dev->running)
593f63b0b   Eric Dumazet   pktgen: Fix devic...
1706
  			seq_printf(seq, "%s ", pkt_dev->odevname);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1707

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1708
1709
  	seq_printf(seq, "
  Stopped: ");
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
1710
  	list_for_each_entry(pkt_dev, &t->if_list, list)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1711
  		if (!pkt_dev->running)
593f63b0b   Eric Dumazet   pktgen: Fix devic...
1712
  			seq_printf(seq, "%s ", pkt_dev->odevname);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1713
1714
  
  	if (t->result[0])
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1715
1716
1717
  		seq_printf(seq, "
  Result: %s
  ", t->result);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1718
  	else
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1719
1720
1721
  		seq_printf(seq, "
  Result: NA
  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1722

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1723
  	if_unlock(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1724

d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1725
  	return 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1726
  }
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1727
  static ssize_t pktgen_thread_write(struct file *file,
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1728
1729
  				   const char __user * user_buffer,
  				   size_t count, loff_t * offset)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1730
  {
8a994a718   Joe Perches   net/core: Remove ...
1731
  	struct seq_file *seq = file->private_data;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1732
  	struct pktgen_thread *t = seq->private;
d61822235   Paul Gortmaker   pktgen: clean up ...
1733
  	int i, max, len, ret;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1734
  	char name[40];
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1735
  	char *pg_result;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1736

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1737
  	if (count < 1) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1738
  		//      sprintf(pg_result, "Wrong command format");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1739
1740
  		return -EINVAL;
  	}
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1741

d61822235   Paul Gortmaker   pktgen: clean up ...
1742
1743
  	max = count;
  	len = count_trail_chars(user_buffer, max);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1744
  	if (len < 0)
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1745
  		return len;
d61822235   Paul Gortmaker   pktgen: clean up ...
1746
  	i = len;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1747

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1748
1749
1750
  	/* Read variable name */
  
  	len = strn_len(&user_buffer[i], sizeof(name) - 1);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1751
  	if (len < 0)
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1752
  		return len;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1753

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1754
1755
1756
1757
  	memset(name, 0, sizeof(name));
  	if (copy_from_user(name, &user_buffer[i], len))
  		return -EFAULT;
  	i += len;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1758

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1759
  	max = count - i;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1760
  	len = count_trail_chars(&user_buffer[i], max);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1761
  	if (len < 0)
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1762
  		return len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1763
  	i += len;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1764
  	if (debug)
25a8b2545   David S. Miller   [PKTGEN]: Add mis...
1765
1766
1767
  		printk(KERN_DEBUG "pktgen: t=%s, count=%lu
  ",
  		       name, (unsigned long)count);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1768

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1769
  	if (!t) {
f9467eaec   Joe Perches   net/core/pktgen.c...
1770
1771
  		pr_err("ERROR: No thread
  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1772
1773
1774
1775
1776
  		ret = -EINVAL;
  		goto out;
  	}
  
  	pg_result = &(t->result[0]);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1777
1778
1779
  	if (!strcmp(name, "add_device")) {
  		char f[32];
  		memset(f, 0, 32);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1780
  		len = strn_len(&user_buffer[i], sizeof(f) - 1);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1781
1782
  		if (len < 0) {
  			ret = len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1783
1784
  			goto out;
  		}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1785
  		if (copy_from_user(f, &user_buffer[i], len))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1786
1787
  			return -EFAULT;
  		i += len;
6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
1788
  		mutex_lock(&pktgen_thread_lock);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1789
  		pktgen_add_device(t, f);
6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
1790
  		mutex_unlock(&pktgen_thread_lock);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1791
1792
  		ret = count;
  		sprintf(pg_result, "OK: add_device=%s", f);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1793
1794
  		goto out;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1795
  	if (!strcmp(name, "rem_device_all")) {
6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
1796
  		mutex_lock(&pktgen_thread_lock);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1797
  		t->control |= T_REMDEVALL;
6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
1798
  		mutex_unlock(&pktgen_thread_lock);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1799
  		schedule_timeout_interruptible(msecs_to_jiffies(125));	/* Propagate thread->control  */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1800
  		ret = count;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1801
  		sprintf(pg_result, "OK: rem_device_all");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1802
1803
  		goto out;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1804
  	if (!strcmp(name, "max_before_softirq")) {
b163911f8   Robert Olsson   [PKTGEN]: Remove ...
1805
  		sprintf(pg_result, "OK: Note! max_before_softirq is obsoleted -- Do not use");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1806
  		ret = count;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1807
1808
1809
1810
  		goto out;
  	}
  
  	ret = -EINVAL;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1811
  out:
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1812
1813
  	return ret;
  }
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1814
  static int pktgen_thread_open(struct inode *inode, struct file *file)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1815
  {
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1816
  	return single_open(file, pktgen_thread_show, PDE(inode)->data);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1817
  }
9a32144e9   Arjan van de Ven   [PATCH] mark stru...
1818
  static const struct file_operations pktgen_thread_fops = {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1819
1820
1821
1822
1823
1824
  	.owner   = THIS_MODULE,
  	.open    = pktgen_thread_open,
  	.read    = seq_read,
  	.llseek  = seq_lseek,
  	.write   = pktgen_thread_write,
  	.release = single_release,
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
1825
  };
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1826
1827
  
  /* Think find or remove for NN */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1828
  static struct pktgen_dev *__pktgen_NN_threads(const char *ifname, int remove)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1829
1830
1831
  {
  	struct pktgen_thread *t;
  	struct pktgen_dev *pkt_dev = NULL;
3e9848403   Eric Dumazet   pktgen: Fix netde...
1832
  	bool exact = (remove == FIND);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1833

cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
1834
  	list_for_each_entry(t, &pktgen_threads, th_list) {
3e9848403   Eric Dumazet   pktgen: Fix netde...
1835
  		pkt_dev = pktgen_find_dev(t, ifname, exact);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1836
  		if (pkt_dev) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1837
1838
1839
1840
1841
1842
  			if (remove) {
  				if_lock(t);
  				pkt_dev->removal_mark = 1;
  				t->control |= T_REMDEV;
  				if_unlock(t);
  			}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1843
1844
  			break;
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1845
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1846
  	return pkt_dev;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1847
  }
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1848
1849
1850
  /*
   * mark a device for removal
   */
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1851
  static void pktgen_mark_device(const char *ifname)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1852
1853
  {
  	struct pktgen_dev *pkt_dev = NULL;
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1854
1855
  	const int max_tries = 10, msec_per_try = 125;
  	int i = 0;
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1856

6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
1857
  	mutex_lock(&pktgen_thread_lock);
f9467eaec   Joe Perches   net/core/pktgen.c...
1858
1859
  	pr_debug("%s: marking %s for removal
  ", __func__, ifname);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1860

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1861
  	while (1) {
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1862
1863
  
  		pkt_dev = __pktgen_NN_threads(ifname, REMOVE);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1864
1865
  		if (pkt_dev == NULL)
  			break;	/* success */
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1866

6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
1867
  		mutex_unlock(&pktgen_thread_lock);
f9467eaec   Joe Perches   net/core/pktgen.c...
1868
1869
1870
  		pr_debug("%s: waiting for %s to disappear....
  ",
  			 __func__, ifname);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1871
  		schedule_timeout_interruptible(msecs_to_jiffies(msec_per_try));
6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
1872
  		mutex_lock(&pktgen_thread_lock);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1873
1874
  
  		if (++i >= max_tries) {
f9467eaec   Joe Perches   net/core/pktgen.c...
1875
1876
1877
  			pr_err("%s: timed out after waiting %d msec for device %s to be removed
  ",
  			       __func__, msec_per_try * i, ifname);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1878
1879
1880
1881
  			break;
  		}
  
  	}
6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
1882
  	mutex_unlock(&pktgen_thread_lock);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1883
  }
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
1884

39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
  static void pktgen_change_name(struct net_device *dev)
  {
  	struct pktgen_thread *t;
  
  	list_for_each_entry(t, &pktgen_threads, th_list) {
  		struct pktgen_dev *pkt_dev;
  
  		list_for_each_entry(pkt_dev, &t->if_list, list) {
  			if (pkt_dev->odev != dev)
  				continue;
  
  			remove_proc_entry(pkt_dev->entry->name, pg_proc_dir);
2975315b7   Alexey Dobriyan   pktgen: use proc_...
1897
1898
1899
1900
  			pkt_dev->entry = proc_create_data(dev->name, 0600,
  							  pg_proc_dir,
  							  &pktgen_if_fops,
  							  pkt_dev);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1901
  			if (!pkt_dev->entry)
f9467eaec   Joe Perches   net/core/pktgen.c...
1902
1903
1904
  				pr_err("can't move proc entry for '%s'
  ",
  				       dev->name);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1905
1906
1907
  			break;
  		}
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1908
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1909
1910
  static int pktgen_device_event(struct notifier_block *unused,
  			       unsigned long event, void *ptr)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1911
  {
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1912
  	struct net_device *dev = ptr;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1913

721499e89   YOSHIFUJI Hideaki   netns: Use net_eq...
1914
  	if (!net_eq(dev_net(dev), &init_net))
e9dc86534   Eric W. Biederman   [NET]: Make devic...
1915
  		return NOTIFY_DONE;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1916
1917
1918
1919
1920
  	/* It is OK that we do not hold the group lock right now,
  	 * as we run under the RTNL lock.
  	 */
  
  	switch (event) {
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1921
1922
  	case NETDEV_CHANGENAME:
  		pktgen_change_name(dev);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1923
  		break;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1924

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1925
  	case NETDEV_UNREGISTER:
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1926
  		pktgen_mark_device(dev->name);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1927
  		break;
3ff50b799   Stephen Hemminger   [NET]: cleanup ex...
1928
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1929
1930
1931
  
  	return NOTIFY_DONE;
  }
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1932
1933
  static struct net_device *pktgen_dev_get_by_name(struct pktgen_dev *pkt_dev,
  						 const char *ifname)
e6fce5b91   Robert Olsson   pktgen: multiqueu...
1934
1935
  {
  	char b[IFNAMSIZ+5];
d61822235   Paul Gortmaker   pktgen: clean up ...
1936
  	int i;
e6fce5b91   Robert Olsson   pktgen: multiqueu...
1937

63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
1938
1939
  	for (i = 0; ifname[i] != '@'; i++) {
  		if (i == IFNAMSIZ)
e6fce5b91   Robert Olsson   pktgen: multiqueu...
1940
1941
1942
1943
1944
1945
1946
1947
  			break;
  
  		b[i] = ifname[i];
  	}
  	b[i] = 0;
  
  	return dev_get_by_name(&init_net, b);
  }
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1948
  /* Associate pktgen_dev with a device. */
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1949
  static int pktgen_setup_dev(struct pktgen_dev *pkt_dev, const char *ifname)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1950
  {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1951
  	struct net_device *odev;
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1952
  	int err;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1953
1954
  
  	/* Clean old setups */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1955
1956
  	if (pkt_dev->odev) {
  		dev_put(pkt_dev->odev);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1957
1958
  		pkt_dev->odev = NULL;
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1959

e6fce5b91   Robert Olsson   pktgen: multiqueu...
1960
  	odev = pktgen_dev_get_by_name(pkt_dev, ifname);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1961
  	if (!odev) {
f9467eaec   Joe Perches   net/core/pktgen.c...
1962
1963
  		pr_err("no such netdevice: \"%s\"
  ", ifname);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1964
  		return -ENODEV;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1965
  	}
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1966

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1967
  	if (odev->type != ARPHRD_ETHER) {
f9467eaec   Joe Perches   net/core/pktgen.c...
1968
1969
  		pr_err("not an ethernet device: \"%s\"
  ", ifname);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1970
1971
  		err = -EINVAL;
  	} else if (!netif_running(odev)) {
f9467eaec   Joe Perches   net/core/pktgen.c...
1972
1973
  		pr_err("device is down: \"%s\"
  ", ifname);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1974
1975
1976
1977
  		err = -ENETDOWN;
  	} else {
  		pkt_dev->odev = odev;
  		return 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1978
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1979

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1980
  	dev_put(odev);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
1981
  	return err;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
1982
1983
1984
1985
1986
1987
1988
  }
  
  /* Read pkt_dev from the interface and set up internal pktgen_dev
   * structure to have the right information to create/send packets
   */
  static void pktgen_setup_inject(struct pktgen_dev *pkt_dev)
  {
64c00d81b   Andrew Gallatin   pktgen: prevent p...
1989
  	int ntxq;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1990
  	if (!pkt_dev->odev) {
f9467eaec   Joe Perches   net/core/pktgen.c...
1991
1992
  		pr_err("ERROR: pkt_dev->odev == NULL in setup_inject
  ");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
1993
1994
1995
1996
1997
  		sprintf(pkt_dev->result,
  			"ERROR: pkt_dev->odev == NULL in setup_inject.
  ");
  		return;
  	}
64c00d81b   Andrew Gallatin   pktgen: prevent p...
1998
1999
  	/* make sure that we don't pick a non-existing transmit queue */
  	ntxq = pkt_dev->odev->real_num_tx_queues;
bfdbc0aca   Robert Olsson   pktgen: fix multi...
2000

64c00d81b   Andrew Gallatin   pktgen: prevent p...
2001
  	if (ntxq <= pkt_dev->queue_map_min) {
f9467eaec   Joe Perches   net/core/pktgen.c...
2002
2003
2004
2005
  		pr_warning("WARNING: Requested queue_map_min (zero-based) (%d) exceeds valid range [0 - %d] for (%d) queues on %s, resetting
  ",
  			   pkt_dev->queue_map_min, (ntxq ?: 1) - 1, ntxq,
  			   pkt_dev->odevname);
26e29eeda   Dan Carpenter   pktgen: set corre...
2006
  		pkt_dev->queue_map_min = (ntxq ?: 1) - 1;
64c00d81b   Andrew Gallatin   pktgen: prevent p...
2007
  	}
882716604   Jesse Brandeburg   pktgen: fix multi...
2008
  	if (pkt_dev->queue_map_max >= ntxq) {
f9467eaec   Joe Perches   net/core/pktgen.c...
2009
2010
2011
2012
  		pr_warning("WARNING: Requested queue_map_max (zero-based) (%d) exceeds valid range [0 - %d] for (%d) queues on %s, resetting
  ",
  			   pkt_dev->queue_map_max, (ntxq ?: 1) - 1, ntxq,
  			   pkt_dev->odevname);
26e29eeda   Dan Carpenter   pktgen: set corre...
2013
  		pkt_dev->queue_map_max = (ntxq ?: 1) - 1;
64c00d81b   Andrew Gallatin   pktgen: prevent p...
2014
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2015
  	/* Default to the interface's mac if not explicitly set. */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2016

f404e9a67   Kris Katterjohn   [PKTGEN]: Replaci...
2017
  	if (is_zero_ether_addr(pkt_dev->src_mac))
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2018
  		memcpy(&(pkt_dev->hh[6]), pkt_dev->odev->dev_addr, ETH_ALEN);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2019

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2020
  	/* Set up Dest MAC */
f404e9a67   Kris Katterjohn   [PKTGEN]: Replaci...
2021
  	memcpy(&(pkt_dev->hh[0]), pkt_dev->dst_mac, ETH_ALEN);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2022

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2023
2024
2025
2026
  	/* Set up pkt size */
  	pkt_dev->cur_pkt_size = pkt_dev->min_pkt_size;
  
  	if (pkt_dev->flags & F_IPV6) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2027
  		/*
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
2028
  		 * Skip this automatic address setting until locks or functions
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2029
2030
2031
2032
  		 * gets exported
  		 */
  
  #ifdef NOTNOW
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2033
  		int i, set = 0, err = 1;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2034
  		struct inet6_dev *idev;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2035
2036
  		for (i = 0; i < IN6_ADDR_HSIZE; i++)
  			if (pkt_dev->cur_in6_saddr.s6_addr[i]) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2037
2038
2039
  				set = 1;
  				break;
  			}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2040
  		if (!set) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2041
2042
2043
2044
2045
  			/*
  			 * Use linklevel address if unconfigured.
  			 *
  			 * use ipv6_get_lladdr if/when it's get exported
  			 */
8814c4b53   YOSHIFUJI Hideaki   [IPV6] ADDRCONF: ...
2046
  			rcu_read_lock();
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2047
2048
  			idev = __in6_dev_get(pkt_dev->odev);
  			if (idev) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2049
2050
2051
  				struct inet6_ifaddr *ifp;
  
  				read_lock_bh(&idev->lock);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2052
2053
  				for (ifp = idev->addr_list; ifp;
  				     ifp = ifp->if_next) {
f64f9e719   Joe Perches   net: Move && and ...
2054
2055
  					if (ifp->scope == IFA_LINK &&
  					    !(ifp->flags & IFA_F_TENTATIVE)) {
4e3fd7a06   Alexey Dobriyan   net: remove ipv6_...
2056
  						pkt_dev->cur_in6_saddr = ifp->addr;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2057
2058
2059
2060
2061
2062
  						err = 0;
  						break;
  					}
  				}
  				read_unlock_bh(&idev->lock);
  			}
8814c4b53   YOSHIFUJI Hideaki   [IPV6] ADDRCONF: ...
2063
  			rcu_read_unlock();
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2064
  			if (err)
f9467eaec   Joe Perches   net/core/pktgen.c...
2065
2066
  				pr_err("ERROR: IPv6 link address not available
  ");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2067
2068
  		}
  #endif
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2069
  	} else {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2070
2071
2072
  		pkt_dev->saddr_min = 0;
  		pkt_dev->saddr_max = 0;
  		if (strlen(pkt_dev->src_min) == 0) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2073
2074
  
  			struct in_device *in_dev;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2075
2076
  
  			rcu_read_lock();
e5ed63991   Herbert Xu   [IPV4]: Replace _...
2077
  			in_dev = __in_dev_get_rcu(pkt_dev->odev);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2078
2079
  			if (in_dev) {
  				if (in_dev->ifa_list) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2080
2081
  					pkt_dev->saddr_min =
  					    in_dev->ifa_list->ifa_address;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2082
2083
  					pkt_dev->saddr_max = pkt_dev->saddr_min;
  				}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2084
2085
  			}
  			rcu_read_unlock();
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2086
  		} else {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2087
2088
2089
2090
2091
2092
2093
  			pkt_dev->saddr_min = in_aton(pkt_dev->src_min);
  			pkt_dev->saddr_max = in_aton(pkt_dev->src_max);
  		}
  
  		pkt_dev->daddr_min = in_aton(pkt_dev->dst_min);
  		pkt_dev->daddr_max = in_aton(pkt_dev->dst_max);
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2094
2095
2096
2097
2098
2099
2100
  	/* Initialize current values. */
  	pkt_dev->cur_dst_mac_offset = 0;
  	pkt_dev->cur_src_mac_offset = 0;
  	pkt_dev->cur_saddr = pkt_dev->saddr_min;
  	pkt_dev->cur_daddr = pkt_dev->daddr_min;
  	pkt_dev->cur_udp_dst = pkt_dev->udp_dst_min;
  	pkt_dev->cur_udp_src = pkt_dev->udp_src_min;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2101
2102
  	pkt_dev->nflows = 0;
  }
fd29cf726   Stephen Hemminger   pktgen: convert t...
2103
2104
2105
  
  static void spin(struct pktgen_dev *pkt_dev, ktime_t spin_until)
  {
ef87979c2   Stephen Hemminger   pktgen: better sc...
2106
  	ktime_t start_time, end_time;
417bc4b85   Eric Dumazet   pktgen: Fix delay...
2107
  	s64 remaining;
2bc481cf4   Stephen Hemminger   pktgen: spin usin...
2108
2109
2110
2111
  	struct hrtimer_sleeper t;
  
  	hrtimer_init_on_stack(&t.timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
  	hrtimer_set_expires(&t.timer, spin_until);
43d28b651   Daniel Turull   pktgen: increasin...
2112
  	remaining = ktime_to_ns(hrtimer_expires_remaining(&t.timer));
417bc4b85   Eric Dumazet   pktgen: Fix delay...
2113
2114
  	if (remaining <= 0) {
  		pkt_dev->next_tx = ktime_add_ns(spin_until, pkt_dev->delay);
2bc481cf4   Stephen Hemminger   pktgen: spin usin...
2115
  		return;
417bc4b85   Eric Dumazet   pktgen: Fix delay...
2116
  	}
2bc481cf4   Stephen Hemminger   pktgen: spin usin...
2117

ef87979c2   Stephen Hemminger   pktgen: better sc...
2118
  	start_time = ktime_now();
33136d12b   Eric Dumazet   pktgen: remove nd...
2119
2120
2121
2122
2123
2124
  	if (remaining < 100000) {
  		/* for small delays (<100us), just loop until limit is reached */
  		do {
  			end_time = ktime_now();
  		} while (ktime_lt(end_time, spin_until));
  	} else {
2bc481cf4   Stephen Hemminger   pktgen: spin usin...
2125
2126
2127
2128
2129
2130
2131
2132
2133
  		/* see do_nanosleep */
  		hrtimer_init_sleeper(&t, current);
  		do {
  			set_current_state(TASK_INTERRUPTIBLE);
  			hrtimer_start_expires(&t.timer, HRTIMER_MODE_ABS);
  			if (!hrtimer_active(&t.timer))
  				t.task = NULL;
  
  			if (likely(t.task))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2134
  				schedule();
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2135

2bc481cf4   Stephen Hemminger   pktgen: spin usin...
2136
2137
2138
  			hrtimer_cancel(&t.timer);
  		} while (t.task && pkt_dev->running && !signal_pending(current));
  		__set_current_state(TASK_RUNNING);
33136d12b   Eric Dumazet   pktgen: remove nd...
2139
  		end_time = ktime_now();
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2140
  	}
ef87979c2   Stephen Hemminger   pktgen: better sc...
2141
2142
  
  	pkt_dev->idle_acc += ktime_to_ns(ktime_sub(end_time, start_time));
07a0f0f07   Daniel Turull   pktgen: Fix accur...
2143
  	pkt_dev->next_tx = ktime_add_ns(spin_until, pkt_dev->delay);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2144
  }
16dab72f6   Jamal Hadi Salim   [PKTGEN]: Central...
2145
2146
  static inline void set_pkt_overhead(struct pktgen_dev *pkt_dev)
  {
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2147
  	pkt_dev->pkt_overhead = 0;
16dab72f6   Jamal Hadi Salim   [PKTGEN]: Central...
2148
2149
2150
2151
  	pkt_dev->pkt_overhead += pkt_dev->nr_labels*sizeof(u32);
  	pkt_dev->pkt_overhead += VLAN_TAG_SIZE(pkt_dev);
  	pkt_dev->pkt_overhead += SVLAN_TAG_SIZE(pkt_dev);
  }
648fda740   Stephen Hemminger   pktgen: minor cle...
2152
  static inline int f_seen(const struct pktgen_dev *pkt_dev, int flow)
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
2153
  {
648fda740   Stephen Hemminger   pktgen: minor cle...
2154
  	return !!(pkt_dev->flows[flow].flags & F_INIT);
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
  }
  
  static inline int f_pick(struct pktgen_dev *pkt_dev)
  {
  	int flow = pkt_dev->curfl;
  
  	if (pkt_dev->flags & F_FLOW_SEQ) {
  		if (pkt_dev->flows[flow].count >= pkt_dev->lflow) {
  			/* reset time */
  			pkt_dev->flows[flow].count = 0;
1211a6455   Robert Olsson   pktgen: random flow
2165
  			pkt_dev->flows[flow].flags = 0;
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
2166
2167
2168
2169
2170
2171
  			pkt_dev->curfl += 1;
  			if (pkt_dev->curfl >= pkt_dev->cflows)
  				pkt_dev->curfl = 0; /*reset */
  		}
  	} else {
  		flow = random32() % pkt_dev->cflows;
1211a6455   Robert Olsson   pktgen: random flow
2172
  		pkt_dev->curfl = flow;
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
2173

1211a6455   Robert Olsson   pktgen: random flow
2174
  		if (pkt_dev->flows[flow].count > pkt_dev->lflow) {
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
2175
  			pkt_dev->flows[flow].count = 0;
1211a6455   Robert Olsson   pktgen: random flow
2176
2177
  			pkt_dev->flows[flow].flags = 0;
  		}
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
2178
2179
2180
2181
  	}
  
  	return pkt_dev->curfl;
  }
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2182
2183
2184
2185
2186
  
  #ifdef CONFIG_XFRM
  /* If there was already an IPSEC SA, we keep it as is, else
   * we go look for it ...
  */
bd55775c8   Jamal Hadi Salim   xfrm: SA lookups ...
2187
  #define DUMMY_MARK 0
fea1ab0fc   Adrian Bunk   [PKTGEN]: make ge...
2188
  static void get_ipsec_sa(struct pktgen_dev *pkt_dev, int flow)
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2189
2190
2191
2192
  {
  	struct xfrm_state *x = pkt_dev->flows[flow].x;
  	if (!x) {
  		/*slow path: we dont already have xfrm_state*/
bd55775c8   Jamal Hadi Salim   xfrm: SA lookups ...
2193
  		x = xfrm_stateonly_find(&init_net, DUMMY_MARK,
5447c5e40   Alexey Dobriyan   netns xfrm: findi...
2194
  					(xfrm_address_t *)&pkt_dev->cur_daddr,
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2195
2196
2197
2198
2199
2200
2201
  					(xfrm_address_t *)&pkt_dev->cur_saddr,
  					AF_INET,
  					pkt_dev->ipsmode,
  					pkt_dev->ipsproto, 0);
  		if (x) {
  			pkt_dev->flows[flow].x = x;
  			set_pkt_overhead(pkt_dev);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2202
  			pkt_dev->pkt_overhead += x->props.header_len;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2203
2204
2205
2206
2207
  		}
  
  	}
  }
  #endif
fd2ea0a79   David S. Miller   net: Use queue aw...
2208
2209
  static void set_cur_queue_map(struct pktgen_dev *pkt_dev)
  {
e6fce5b91   Robert Olsson   pktgen: multiqueu...
2210
2211
2212
  
  	if (pkt_dev->flags & F_QUEUE_MAP_CPU)
  		pkt_dev->cur_queue_map = smp_processor_id();
896a7cf8d   Eric Dumazet   pktgen: Fix multi...
2213
  	else if (pkt_dev->queue_map_min <= pkt_dev->queue_map_max) {
fd2ea0a79   David S. Miller   net: Use queue aw...
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
  		__u16 t;
  		if (pkt_dev->flags & F_QUEUE_MAP_RND) {
  			t = random32() %
  				(pkt_dev->queue_map_max -
  				 pkt_dev->queue_map_min + 1)
  				+ pkt_dev->queue_map_min;
  		} else {
  			t = pkt_dev->cur_queue_map + 1;
  			if (t > pkt_dev->queue_map_max)
  				t = pkt_dev->queue_map_min;
  		}
  		pkt_dev->cur_queue_map = t;
  	}
bfdbc0aca   Robert Olsson   pktgen: fix multi...
2227
  	pkt_dev->cur_queue_map  = pkt_dev->cur_queue_map % pkt_dev->odev->real_num_tx_queues;
fd2ea0a79   David S. Miller   net: Use queue aw...
2228
  }
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2229
2230
2231
  /* Increment/randomize headers according to flags and current values
   * for IP src/dest, UDP src/dst port, MAC-Addr src/dst
   */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2232
2233
2234
2235
2236
  static void mod_cur_headers(struct pktgen_dev *pkt_dev)
  {
  	__u32 imn;
  	__u32 imx;
  	int flow = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2237

007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
2238
2239
  	if (pkt_dev->cflows)
  		flow = f_pick(pkt_dev);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2240
2241
  
  	/*  Deal with source MAC */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2242
2243
2244
2245
2246
  	if (pkt_dev->src_mac_count > 1) {
  		__u32 mc;
  		__u32 tmp;
  
  		if (pkt_dev->flags & F_MACSRC_RND)
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2247
  			mc = random32() % pkt_dev->src_mac_count;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2248
2249
  		else {
  			mc = pkt_dev->cur_src_mac_offset++;
ff2a79a5a   Robert Olsson   pktgen: mac count
2250
  			if (pkt_dev->cur_src_mac_offset >=
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
  			    pkt_dev->src_mac_count)
  				pkt_dev->cur_src_mac_offset = 0;
  		}
  
  		tmp = pkt_dev->src_mac[5] + (mc & 0xFF);
  		pkt_dev->hh[11] = tmp;
  		tmp = (pkt_dev->src_mac[4] + ((mc >> 8) & 0xFF) + (tmp >> 8));
  		pkt_dev->hh[10] = tmp;
  		tmp = (pkt_dev->src_mac[3] + ((mc >> 16) & 0xFF) + (tmp >> 8));
  		pkt_dev->hh[9] = tmp;
  		tmp = (pkt_dev->src_mac[2] + ((mc >> 24) & 0xFF) + (tmp >> 8));
  		pkt_dev->hh[8] = tmp;
  		tmp = (pkt_dev->src_mac[1] + (tmp >> 8));
  		pkt_dev->hh[7] = tmp;
  	}
  
  	/*  Deal with Destination MAC */
  	if (pkt_dev->dst_mac_count > 1) {
  		__u32 mc;
  		__u32 tmp;
  
  		if (pkt_dev->flags & F_MACDST_RND)
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2273
  			mc = random32() % pkt_dev->dst_mac_count;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2274
2275
2276
  
  		else {
  			mc = pkt_dev->cur_dst_mac_offset++;
ff2a79a5a   Robert Olsson   pktgen: mac count
2277
  			if (pkt_dev->cur_dst_mac_offset >=
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
  			    pkt_dev->dst_mac_count) {
  				pkt_dev->cur_dst_mac_offset = 0;
  			}
  		}
  
  		tmp = pkt_dev->dst_mac[5] + (mc & 0xFF);
  		pkt_dev->hh[5] = tmp;
  		tmp = (pkt_dev->dst_mac[4] + ((mc >> 8) & 0xFF) + (tmp >> 8));
  		pkt_dev->hh[4] = tmp;
  		tmp = (pkt_dev->dst_mac[3] + ((mc >> 16) & 0xFF) + (tmp >> 8));
  		pkt_dev->hh[3] = tmp;
  		tmp = (pkt_dev->dst_mac[2] + ((mc >> 24) & 0xFF) + (tmp >> 8));
  		pkt_dev->hh[2] = tmp;
  		tmp = (pkt_dev->dst_mac[1] + (tmp >> 8));
  		pkt_dev->hh[1] = tmp;
  	}
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2294
2295
  	if (pkt_dev->flags & F_MPLS_RND) {
  		unsigned i;
e71a4783a   Stephen Hemminger   [NET] core: white...
2296
  		for (i = 0; i < pkt_dev->nr_labels; i++)
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2297
2298
  			if (pkt_dev->labels[i] & MPLS_STACK_BOTTOM)
  				pkt_dev->labels[i] = MPLS_STACK_BOTTOM |
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2299
  					     ((__force __be32)random32() &
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2300
2301
  						      htonl(0x000fffff));
  	}
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2302
  	if ((pkt_dev->flags & F_VID_RND) && (pkt_dev->vlan_id != 0xffff)) {
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2303
  		pkt_dev->vlan_id = random32() & (4096-1);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2304
2305
2306
  	}
  
  	if ((pkt_dev->flags & F_SVID_RND) && (pkt_dev->svlan_id != 0xffff)) {
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2307
  		pkt_dev->svlan_id = random32() & (4096 - 1);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2308
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2309
2310
  	if (pkt_dev->udp_src_min < pkt_dev->udp_src_max) {
  		if (pkt_dev->flags & F_UDPSRC_RND)
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2311
2312
2313
  			pkt_dev->cur_udp_src = random32() %
  				(pkt_dev->udp_src_max - pkt_dev->udp_src_min)
  				+ pkt_dev->udp_src_min;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2314
2315
  
  		else {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2316
2317
2318
  			pkt_dev->cur_udp_src++;
  			if (pkt_dev->cur_udp_src >= pkt_dev->udp_src_max)
  				pkt_dev->cur_udp_src = pkt_dev->udp_src_min;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2319
2320
2321
2322
2323
  		}
  	}
  
  	if (pkt_dev->udp_dst_min < pkt_dev->udp_dst_max) {
  		if (pkt_dev->flags & F_UDPDST_RND) {
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2324
2325
2326
  			pkt_dev->cur_udp_dst = random32() %
  				(pkt_dev->udp_dst_max - pkt_dev->udp_dst_min)
  				+ pkt_dev->udp_dst_min;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2327
  		} else {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2328
  			pkt_dev->cur_udp_dst++;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2329
  			if (pkt_dev->cur_udp_dst >= pkt_dev->udp_dst_max)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2330
  				pkt_dev->cur_udp_dst = pkt_dev->udp_dst_min;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2331
2332
  		}
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2333
2334
  
  	if (!(pkt_dev->flags & F_IPV6)) {
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2335
2336
2337
  		imn = ntohl(pkt_dev->saddr_min);
  		imx = ntohl(pkt_dev->saddr_max);
  		if (imn < imx) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2338
  			__u32 t;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2339
  			if (pkt_dev->flags & F_IPSRC_RND)
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2340
  				t = random32() % (imx - imn) + imn;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2341
2342
2343
  			else {
  				t = ntohl(pkt_dev->cur_saddr);
  				t++;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2344
  				if (t > imx)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2345
  					t = imn;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2346

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2347
2348
2349
  			}
  			pkt_dev->cur_saddr = htonl(t);
  		}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2350

007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
2351
  		if (pkt_dev->cflows && f_seen(pkt_dev, flow)) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2352
2353
  			pkt_dev->cur_daddr = pkt_dev->flows[flow].cur_daddr;
  		} else {
252e33467   Al Viro   [NET] net/core: A...
2354
2355
2356
  			imn = ntohl(pkt_dev->daddr_min);
  			imx = ntohl(pkt_dev->daddr_max);
  			if (imn < imx) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2357
  				__u32 t;
252e33467   Al Viro   [NET] net/core: A...
2358
  				__be32 s;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2359
  				if (pkt_dev->flags & F_IPDST_RND) {
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2360
  					t = random32() % (imx - imn) + imn;
252e33467   Al Viro   [NET] net/core: A...
2361
  					s = htonl(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2362

21cf2253e   Joe Perches   [IPV4] net/core: ...
2363
2364
  					while (ipv4_is_loopback(s) ||
  					       ipv4_is_multicast(s) ||
1e637c74b   Jan Engelhardt   [IPV4]: Enable us...
2365
  					       ipv4_is_lbcast(s) ||
21cf2253e   Joe Perches   [IPV4] net/core: ...
2366
2367
  					       ipv4_is_zeronet(s) ||
  					       ipv4_is_local_multicast(s)) {
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2368
  						t = random32() % (imx - imn) + imn;
252e33467   Al Viro   [NET] net/core: A...
2369
  						s = htonl(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2370
  					}
252e33467   Al Viro   [NET] net/core: A...
2371
2372
  					pkt_dev->cur_daddr = s;
  				} else {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2373
2374
2375
2376
2377
2378
2379
2380
  					t = ntohl(pkt_dev->cur_daddr);
  					t++;
  					if (t > imx) {
  						t = imn;
  					}
  					pkt_dev->cur_daddr = htonl(t);
  				}
  			}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2381
  			if (pkt_dev->cflows) {
007a531b0   Jamal Hadi Salim   [PKTGEN]: Introdu...
2382
  				pkt_dev->flows[flow].flags |= F_INIT;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2383
2384
  				pkt_dev->flows[flow].cur_daddr =
  				    pkt_dev->cur_daddr;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2385
2386
2387
2388
  #ifdef CONFIG_XFRM
  				if (pkt_dev->flags & F_IPSEC_ON)
  					get_ipsec_sa(pkt_dev, flow);
  #endif
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2389
2390
2391
  				pkt_dev->nflows++;
  			}
  		}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2392
2393
2394
2395
2396
2397
  	} else {		/* IPV6 * */
  
  		if (pkt_dev->min_in6_daddr.s6_addr32[0] == 0 &&
  		    pkt_dev->min_in6_daddr.s6_addr32[1] == 0 &&
  		    pkt_dev->min_in6_daddr.s6_addr32[2] == 0 &&
  		    pkt_dev->min_in6_daddr.s6_addr32[3] == 0) ;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2398
2399
2400
2401
  		else {
  			int i;
  
  			/* Only random destinations yet */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2402
  			for (i = 0; i < 4; i++) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2403
  				pkt_dev->cur_in6_daddr.s6_addr32[i] =
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2404
  				    (((__force __be32)random32() |
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2405
2406
  				      pkt_dev->min_in6_daddr.s6_addr32[i]) &
  				     pkt_dev->max_in6_daddr.s6_addr32[i]);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2407
  			}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2408
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2409
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2410
2411
2412
  	if (pkt_dev->min_pkt_size < pkt_dev->max_pkt_size) {
  		__u32 t;
  		if (pkt_dev->flags & F_TXSIZE_RND) {
5fa6fc76f   Stephen Hemminger   [PKTGEN]: use ran...
2413
2414
2415
  			t = random32() %
  				(pkt_dev->max_pkt_size - pkt_dev->min_pkt_size)
  				+ pkt_dev->min_pkt_size;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2416
  		} else {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2417
  			t = pkt_dev->cur_pkt_size + 1;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2418
  			if (t > pkt_dev->max_pkt_size)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2419
  				t = pkt_dev->min_pkt_size;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2420
2421
2422
  		}
  		pkt_dev->cur_pkt_size = t;
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2423

fd2ea0a79   David S. Miller   net: Use queue aw...
2424
  	set_cur_queue_map(pkt_dev);
45b270f88   Robert Olsson   [PKTGEN]: Multiqu...
2425

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2426
2427
  	pkt_dev->flows[flow].count++;
  }
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2428
2429
2430
2431
2432
2433
  
  #ifdef CONFIG_XFRM
  static int pktgen_output_ipsec(struct sk_buff *skb, struct pktgen_dev *pkt_dev)
  {
  	struct xfrm_state *x = pkt_dev->flows[pkt_dev->curfl].x;
  	int err = 0;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2434
2435
2436
2437
2438
2439
2440
2441
2442
  
  	if (!x)
  		return 0;
  	/* XXX: we dont support tunnel mode for now until
  	 * we resolve the dst issue */
  	if (x->props.mode != XFRM_MODE_TRANSPORT)
  		return 0;
  
  	spin_lock(&x->lock);
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2443

13996378e   Herbert Xu   [IPSEC]: Rename m...
2444
  	err = x->outer_mode->output(x, skb);
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2445
2446
2447
2448
2449
  	if (err)
  		goto error;
  	err = x->type->output(x, skb);
  	if (err)
  		goto error;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2450
  	x->curlft.bytes += skb->len;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2451
  	x->curlft.packets++;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2452
2453
2454
2455
  error:
  	spin_unlock(&x->lock);
  	return err;
  }
475ac1e40   Stephen Hemminger   pktgen: change in...
2456
  static void free_SAs(struct pktgen_dev *pkt_dev)
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2457
2458
2459
  {
  	if (pkt_dev->cflows) {
  		/* let go of the SAs if we have them */
d61822235   Paul Gortmaker   pktgen: clean up ...
2460
2461
  		int i;
  		for (i = 0; i < pkt_dev->cflows; i++) {
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2462
2463
2464
2465
2466
2467
2468
2469
  			struct xfrm_state *x = pkt_dev->flows[i].x;
  			if (x) {
  				xfrm_state_put(x);
  				pkt_dev->flows[i].x = NULL;
  			}
  		}
  	}
  }
475ac1e40   Stephen Hemminger   pktgen: change in...
2470
  static int process_ipsec(struct pktgen_dev *pkt_dev,
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2471
2472
2473
2474
2475
2476
2477
2478
2479
  			      struct sk_buff *skb, __be16 protocol)
  {
  	if (pkt_dev->flags & F_IPSEC_ON) {
  		struct xfrm_state *x = pkt_dev->flows[pkt_dev->curfl].x;
  		int nhead = 0;
  		if (x) {
  			int ret;
  			__u8 *eth;
  			nhead = x->props.header_len - skb_headroom(skb);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2480
  			if (nhead > 0) {
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2481
2482
  				ret = pskb_expand_head(skb, nhead, 0, GFP_ATOMIC);
  				if (ret < 0) {
f9467eaec   Joe Perches   net/core/pktgen.c...
2483
2484
2485
  					pr_err("Error expanding ipsec packet %d
  ",
  					       ret);
b4bb4ac8c   Ilpo Järvinen   pktgen: fix skb l...
2486
  					goto err;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2487
2488
2489
2490
2491
2492
2493
  				}
  			}
  
  			/* ipsec is not expecting ll header */
  			skb_pull(skb, ETH_HLEN);
  			ret = pktgen_output_ipsec(skb, pkt_dev);
  			if (ret) {
f9467eaec   Joe Perches   net/core/pktgen.c...
2494
2495
  				pr_err("Error creating ipsec packet %d
  ", ret);
b4bb4ac8c   Ilpo Järvinen   pktgen: fix skb l...
2496
  				goto err;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2497
2498
2499
2500
  			}
  			/* restore ll */
  			eth = (__u8 *) skb_push(skb, ETH_HLEN);
  			memcpy(eth, pkt_dev->hh, 12);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2501
  			*(u16 *) &eth[12] = protocol;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2502
2503
2504
  		}
  	}
  	return 1;
b4bb4ac8c   Ilpo Järvinen   pktgen: fix skb l...
2505
2506
2507
  err:
  	kfree_skb(skb);
  	return 0;
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2508
2509
  }
  #endif
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2510
2511
2512
  static void mpls_push(__be32 *mpls, struct pktgen_dev *pkt_dev)
  {
  	unsigned i;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2513
  	for (i = 0; i < pkt_dev->nr_labels; i++)
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2514
  		*mpls++ = pkt_dev->labels[i] & ~MPLS_STACK_BOTTOM;
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2515

ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2516
2517
2518
  	mpls--;
  	*mpls |= MPLS_STACK_BOTTOM;
  }
0f37c6057   Al Viro   [PKTGEN]: TCI end...
2519
2520
2521
2522
2523
  static inline __be16 build_tci(unsigned int id, unsigned int cfi,
  			       unsigned int prio)
  {
  	return htons(id | (cfi << 12) | (prio << 13));
  }
26ad78796   Eric Dumazet   pktgen: speedup f...
2524
2525
2526
2527
2528
2529
2530
2531
2532
2533
  static void pktgen_finalize_skb(struct pktgen_dev *pkt_dev, struct sk_buff *skb,
  				int datalen)
  {
  	struct timeval timestamp;
  	struct pktgen_hdr *pgh;
  
  	pgh = (struct pktgen_hdr *)skb_put(skb, sizeof(*pgh));
  	datalen -= sizeof(*pgh);
  
  	if (pkt_dev->nfrags <= 0) {
05aebe2e5   Daniel Turull   pktgen: bug fix i...
2534
  		memset(skb_put(skb, datalen), 0, datalen);
26ad78796   Eric Dumazet   pktgen: speedup f...
2535
2536
2537
  	} else {
  		int frags = pkt_dev->nfrags;
  		int i, len;
7d36a991e   amit salecha   pktgen: create nu...
2538
  		int frag_len;
26ad78796   Eric Dumazet   pktgen: speedup f...
2539
2540
2541
2542
2543
2544
2545
2546
2547
2548
2549
  
  
  		if (frags > MAX_SKB_FRAGS)
  			frags = MAX_SKB_FRAGS;
  		len = datalen - frags * PAGE_SIZE;
  		if (len > 0) {
  			memset(skb_put(skb, len), 0, len);
  			datalen = frags * PAGE_SIZE;
  		}
  
  		i = 0;
7d36a991e   amit salecha   pktgen: create nu...
2550
2551
  		frag_len = (datalen/frags) < PAGE_SIZE ?
  			   (datalen/frags) : PAGE_SIZE;
26ad78796   Eric Dumazet   pktgen: speedup f...
2552
2553
2554
2555
2556
2557
2558
2559
2560
2561
  		while (datalen > 0) {
  			if (unlikely(!pkt_dev->page)) {
  				int node = numa_node_id();
  
  				if (pkt_dev->node >= 0 && (pkt_dev->flags & F_NODE))
  					node = pkt_dev->node;
  				pkt_dev->page = alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0);
  				if (!pkt_dev->page)
  					break;
  			}
a0bec1cd8   Ian Campbell   net: do not take ...
2562
  			get_page(pkt_dev->page);
ea2ab6937   Ian Campbell   net: convert core...
2563
  			skb_frag_set_page(skb, i, pkt_dev->page);
26ad78796   Eric Dumazet   pktgen: speedup f...
2564
  			skb_shinfo(skb)->frags[i].page_offset = 0;
7d36a991e   amit salecha   pktgen: create nu...
2565
2566
  			/*last fragment, fill rest of data*/
  			if (i == (frags - 1))
9e903e085   Eric Dumazet   net: add skb frag...
2567
2568
  				skb_frag_size_set(&skb_shinfo(skb)->frags[i],
  				    (datalen < PAGE_SIZE ? datalen : PAGE_SIZE));
7d36a991e   amit salecha   pktgen: create nu...
2569
  			else
9e903e085   Eric Dumazet   net: add skb frag...
2570
2571
2572
2573
  				skb_frag_size_set(&skb_shinfo(skb)->frags[i], frag_len);
  			datalen -= skb_frag_size(&skb_shinfo(skb)->frags[i]);
  			skb->len += skb_frag_size(&skb_shinfo(skb)->frags[i]);
  			skb->data_len += skb_frag_size(&skb_shinfo(skb)->frags[i]);
26ad78796   Eric Dumazet   pktgen: speedup f...
2574
2575
2576
  			i++;
  			skb_shinfo(skb)->nr_frags = i;
  		}
26ad78796   Eric Dumazet   pktgen: speedup f...
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
  	}
  
  	/* Stamp the time, and sequence number,
  	 * convert them to network byte order
  	 */
  	pgh->pgh_magic = htonl(PKTGEN_MAGIC);
  	pgh->seq_num = htonl(pkt_dev->seq_num);
  
  	do_gettimeofday(&timestamp);
  	pgh->tv_sec = htonl(timestamp.tv_sec);
  	pgh->tv_usec = htonl(timestamp.tv_usec);
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2589
2590
  static struct sk_buff *fill_packet_ipv4(struct net_device *odev,
  					struct pktgen_dev *pkt_dev)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2591
2592
2593
2594
2595
2596
  {
  	struct sk_buff *skb = NULL;
  	__u8 *eth;
  	struct udphdr *udph;
  	int datalen, iplen;
  	struct iphdr *iph;
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2597
  	__be16 protocol = htons(ETH_P_IP);
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2598
  	__be32 *mpls;
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2599
2600
2601
2602
  	__be16 *vlan_tci = NULL;                 /* Encapsulates priority and VLAN ID */
  	__be16 *vlan_encapsulated_proto = NULL;  /* packet type ID field (or len) for VLAN tag */
  	__be16 *svlan_tci = NULL;                /* Encapsulates priority and SVLAN ID */
  	__be16 *svlan_encapsulated_proto = NULL; /* packet type ID field (or len) for SVLAN tag */
fd2ea0a79   David S. Miller   net: Use queue aw...
2603
  	u16 queue_map;
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2604
2605
  
  	if (pkt_dev->nr_labels)
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2606
  		protocol = htons(ETH_P_MPLS_UC);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2607

34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2608
  	if (pkt_dev->vlan_id != 0xffff)
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2609
  		protocol = htons(ETH_P_8021Q);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2610

64053beeb   Robert Olsson   [PKTGEN]: Fix ran...
2611
2612
2613
2614
  	/* Update any of the values, used when we're incrementing various
  	 * fields.
  	 */
  	mod_cur_headers(pkt_dev);
eb589063e   Junchang Wang   pktgen: correct u...
2615
  	queue_map = pkt_dev->cur_queue_map;
64053beeb   Robert Olsson   [PKTGEN]: Fix ran...
2616

7ac5459ec   David S. Miller   [PKTGEN]: Respect...
2617
  	datalen = (odev->hard_header_len + 16) & ~0xf;
e99b99b47   Robert Olsson   pktgen node alloc...
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
  
  	if (pkt_dev->flags & F_NODE) {
  		int node;
  
  		if (pkt_dev->node >= 0)
  			node = pkt_dev->node;
  		else
  			node =  numa_node_id();
  
  		skb = __alloc_skb(NET_SKB_PAD + pkt_dev->cur_pkt_size + 64
  				  + datalen + pkt_dev->pkt_overhead, GFP_NOWAIT, 0, node);
  		if (likely(skb)) {
  			skb_reserve(skb, NET_SKB_PAD);
  			skb->dev = odev;
  		}
  	}
  	else
  	  skb = __netdev_alloc_skb(odev,
  				   pkt_dev->cur_pkt_size + 64
  				   + datalen + pkt_dev->pkt_overhead, GFP_NOWAIT);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2638
2639
2640
2641
  	if (!skb) {
  		sprintf(pkt_dev->result, "No memory");
  		return NULL;
  	}
a8d764b98   Junchang Wang   pktgen: adding pr...
2642
  	prefetchw(skb->data);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2643

7ac5459ec   David S. Miller   [PKTGEN]: Respect...
2644
  	skb_reserve(skb, datalen);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2645
2646
2647
  
  	/*  Reserve for ethernet and IP header  */
  	eth = (__u8 *) skb_push(skb, 14);
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2648
2649
2650
  	mpls = (__be32 *)skb_put(skb, pkt_dev->nr_labels*sizeof(__u32));
  	if (pkt_dev->nr_labels)
  		mpls_push(mpls, pkt_dev);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2651
2652
  
  	if (pkt_dev->vlan_id != 0xffff) {
e71a4783a   Stephen Hemminger   [NET] core: white...
2653
  		if (pkt_dev->svlan_id != 0xffff) {
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2654
  			svlan_tci = (__be16 *)skb_put(skb, sizeof(__be16));
0f37c6057   Al Viro   [PKTGEN]: TCI end...
2655
2656
2657
  			*svlan_tci = build_tci(pkt_dev->svlan_id,
  					       pkt_dev->svlan_cfi,
  					       pkt_dev->svlan_p);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2658
  			svlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16));
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2659
  			*svlan_encapsulated_proto = htons(ETH_P_8021Q);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2660
2661
  		}
  		vlan_tci = (__be16 *)skb_put(skb, sizeof(__be16));
0f37c6057   Al Viro   [PKTGEN]: TCI end...
2662
2663
2664
  		*vlan_tci = build_tci(pkt_dev->vlan_id,
  				      pkt_dev->vlan_cfi,
  				      pkt_dev->vlan_p);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2665
  		vlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16));
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2666
  		*vlan_encapsulated_proto = htons(ETH_P_IP);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2667
  	}
27a884dc3   Arnaldo Carvalho de Melo   [SK_BUFF]: Conver...
2668
  	skb->network_header = skb->tail;
b0e380b1d   Arnaldo Carvalho de Melo   [SK_BUFF]: unions...
2669
  	skb->transport_header = skb->network_header + sizeof(struct iphdr);
ddc7b8e32   Arnaldo Carvalho de Melo   [SK_BUFF]: Some m...
2670
  	skb_put(skb, sizeof(struct iphdr) + sizeof(struct udphdr));
fd2ea0a79   David S. Miller   net: Use queue aw...
2671
  	skb_set_queue_mapping(skb, queue_map);
9e50e3ac5   John Fastabend   net: add priority...
2672
  	skb->priority = pkt_dev->skb_priority;
ddc7b8e32   Arnaldo Carvalho de Melo   [SK_BUFF]: Some m...
2673
2674
  	iph = ip_hdr(skb);
  	udph = udp_hdr(skb);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2675

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2676
  	memcpy(eth, pkt_dev->hh, 12);
252e33467   Al Viro   [NET] net/core: A...
2677
  	*(__be16 *) & eth[12] = protocol;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2678

ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2679
2680
  	/* Eth + IPh + UDPh + mpls */
  	datalen = pkt_dev->cur_pkt_size - 14 - 20 - 8 -
16dab72f6   Jamal Hadi Salim   [PKTGEN]: Central...
2681
  		  pkt_dev->pkt_overhead;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2682
  	if (datalen < sizeof(struct pktgen_hdr))
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2683
  		datalen = sizeof(struct pktgen_hdr);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2684

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2685
2686
  	udph->source = htons(pkt_dev->cur_udp_src);
  	udph->dest = htons(pkt_dev->cur_udp_dst);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2687
2688
  	udph->len = htons(datalen + 8);	/* DATA + udphdr */
  	udph->check = 0;	/* No checksum */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2689
2690
2691
2692
  
  	iph->ihl = 5;
  	iph->version = 4;
  	iph->ttl = 32;
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
2693
  	iph->tos = pkt_dev->tos;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2694
  	iph->protocol = IPPROTO_UDP;	/* UDP */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2695
2696
  	iph->saddr = pkt_dev->cur_saddr;
  	iph->daddr = pkt_dev->cur_daddr;
66ed1e5ec   Eric Dumazet   pktgen: Dont leak...
2697
2698
  	iph->id = htons(pkt_dev->ip_id);
  	pkt_dev->ip_id++;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2699
2700
2701
2702
  	iph->frag_off = 0;
  	iplen = 20 + 8 + datalen;
  	iph->tot_len = htons(iplen);
  	iph->check = 0;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2703
  	iph->check = ip_fast_csum((void *)iph, iph->ihl);
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2704
  	skb->protocol = protocol;
b0e380b1d   Arnaldo Carvalho de Melo   [SK_BUFF]: unions...
2705
  	skb->mac_header = (skb->network_header - ETH_HLEN -
16dab72f6   Jamal Hadi Salim   [PKTGEN]: Central...
2706
  			   pkt_dev->pkt_overhead);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2707
2708
  	skb->dev = odev;
  	skb->pkt_type = PACKET_HOST;
26ad78796   Eric Dumazet   pktgen: speedup f...
2709
  	pktgen_finalize_skb(pkt_dev, skb, datalen);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2710

a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
2711
2712
2713
2714
  #ifdef CONFIG_XFRM
  	if (!process_ipsec(pkt_dev, skb, protocol))
  		return NULL;
  #endif
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2715
2716
2717
2718
  	return skb;
  }
  
  /*
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
2719
   * scan_ip6, fmt_ip taken from dietlibc-0.21
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2720
2721
   * Author Felix von Leitner <felix-dietlibc@fefe.de>
   *
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
2722
   * Slightly modified for kernel.
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2723
2724
2725
   * Should be candidate for net/ipv4/utils.c
   * --ro
   */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2726
  static unsigned int scan_ip6(const char *s, char ip[16])
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2727
2728
  {
  	unsigned int i;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2729
  	unsigned int len = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2730
2731
  	unsigned long u;
  	char suffix[16];
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2732
2733
  	unsigned int prefixlen = 0;
  	unsigned int suffixlen = 0;
252e33467   Al Viro   [NET] net/core: A...
2734
  	__be32 tmp;
cfcabdcc2   Stephen Hemminger   [NET]: sparse war...
2735
  	char *pos;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2736

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2737
2738
  	for (i = 0; i < 16; i++)
  		ip[i] = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2739
2740
2741
2742
  
  	for (;;) {
  		if (*s == ':') {
  			len++;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2743
2744
  			if (s[1] == ':') {	/* Found "::", skip to part 2 */
  				s += 2;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2745
2746
2747
2748
2749
  				len++;
  				break;
  			}
  			s++;
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2750

cfcabdcc2   Stephen Hemminger   [NET]: sparse war...
2751
2752
  		u = simple_strtoul(s, &pos, 16);
  		i = pos - s;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2753
2754
2755
  		if (!i)
  			return 0;
  		if (prefixlen == 12 && s[i] == '.') {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2756
2757
2758
2759
  
  			/* the last 4 bytes may be written as IPv4 address */
  
  			tmp = in_aton(s);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2760
2761
  			memcpy((struct in_addr *)(ip + 12), &tmp, sizeof(tmp));
  			return i + len;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2762
2763
2764
  		}
  		ip[prefixlen++] = (u >> 8);
  		ip[prefixlen++] = (u & 255);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2765
2766
2767
  		s += i;
  		len += i;
  		if (prefixlen == 16)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2768
2769
2770
2771
2772
2773
  			return len;
  	}
  
  /* part 2, after "::" */
  	for (;;) {
  		if (*s == ':') {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2774
  			if (suffixlen == 0)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2775
2776
2777
  				break;
  			s++;
  			len++;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2778
  		} else if (suffixlen != 0)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2779
  			break;
cfcabdcc2   Stephen Hemminger   [NET]: sparse war...
2780
2781
2782
  
  		u = simple_strtol(s, &pos, 16);
  		i = pos - s;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2783
  		if (!i) {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2784
2785
  			if (*s)
  				len--;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2786
2787
  			break;
  		}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2788
  		if (suffixlen + prefixlen <= 12 && s[i] == '.') {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2789
  			tmp = in_aton(s);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2790
2791
2792
2793
  			memcpy((struct in_addr *)(suffix + suffixlen), &tmp,
  			       sizeof(tmp));
  			suffixlen += 4;
  			len += strlen(s);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2794
2795
2796
2797
  			break;
  		}
  		suffix[suffixlen++] = (u >> 8);
  		suffix[suffixlen++] = (u & 255);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2798
2799
2800
  		s += i;
  		len += i;
  		if (prefixlen + suffixlen == 16)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2801
2802
  			break;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2803
2804
  	for (i = 0; i < suffixlen; i++)
  		ip[16 - suffixlen + i] = suffix[i];
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2805
2806
  	return len;
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2807
2808
  static struct sk_buff *fill_packet_ipv6(struct net_device *odev,
  					struct pktgen_dev *pkt_dev)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2809
2810
2811
2812
2813
2814
  {
  	struct sk_buff *skb = NULL;
  	__u8 *eth;
  	struct udphdr *udph;
  	int datalen;
  	struct ipv6hdr *iph;
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2815
  	__be16 protocol = htons(ETH_P_IPV6);
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2816
  	__be32 *mpls;
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2817
2818
2819
2820
  	__be16 *vlan_tci = NULL;                 /* Encapsulates priority and VLAN ID */
  	__be16 *vlan_encapsulated_proto = NULL;  /* packet type ID field (or len) for VLAN tag */
  	__be16 *svlan_tci = NULL;                /* Encapsulates priority and SVLAN ID */
  	__be16 *svlan_encapsulated_proto = NULL; /* packet type ID field (or len) for SVLAN tag */
fd2ea0a79   David S. Miller   net: Use queue aw...
2821
  	u16 queue_map;
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2822
2823
  
  	if (pkt_dev->nr_labels)
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2824
  		protocol = htons(ETH_P_MPLS_UC);
64053beeb   Robert Olsson   [PKTGEN]: Fix ran...
2825

34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2826
  	if (pkt_dev->vlan_id != 0xffff)
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2827
  		protocol = htons(ETH_P_8021Q);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2828

64053beeb   Robert Olsson   [PKTGEN]: Fix ran...
2829
2830
2831
2832
  	/* Update any of the values, used when we're incrementing various
  	 * fields.
  	 */
  	mod_cur_headers(pkt_dev);
eb589063e   Junchang Wang   pktgen: correct u...
2833
  	queue_map = pkt_dev->cur_queue_map;
64053beeb   Robert Olsson   [PKTGEN]: Fix ran...
2834

e470757d6   Stephen Hemminger   pktgen: use netde...
2835
2836
2837
  	skb = __netdev_alloc_skb(odev,
  				 pkt_dev->cur_pkt_size + 64
  				 + 16 + pkt_dev->pkt_overhead, GFP_NOWAIT);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2838
2839
2840
2841
  	if (!skb) {
  		sprintf(pkt_dev->result, "No memory");
  		return NULL;
  	}
a8d764b98   Junchang Wang   pktgen: adding pr...
2842
  	prefetchw(skb->data);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2843
2844
2845
2846
2847
  
  	skb_reserve(skb, 16);
  
  	/*  Reserve for ethernet and IP header  */
  	eth = (__u8 *) skb_push(skb, 14);
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2848
2849
2850
  	mpls = (__be32 *)skb_put(skb, pkt_dev->nr_labels*sizeof(__u32));
  	if (pkt_dev->nr_labels)
  		mpls_push(mpls, pkt_dev);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2851
2852
  
  	if (pkt_dev->vlan_id != 0xffff) {
e71a4783a   Stephen Hemminger   [NET] core: white...
2853
  		if (pkt_dev->svlan_id != 0xffff) {
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2854
  			svlan_tci = (__be16 *)skb_put(skb, sizeof(__be16));
0f37c6057   Al Viro   [PKTGEN]: TCI end...
2855
2856
2857
  			*svlan_tci = build_tci(pkt_dev->svlan_id,
  					       pkt_dev->svlan_cfi,
  					       pkt_dev->svlan_p);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2858
  			svlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16));
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2859
  			*svlan_encapsulated_proto = htons(ETH_P_8021Q);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2860
2861
  		}
  		vlan_tci = (__be16 *)skb_put(skb, sizeof(__be16));
0f37c6057   Al Viro   [PKTGEN]: TCI end...
2862
2863
2864
  		*vlan_tci = build_tci(pkt_dev->vlan_id,
  				      pkt_dev->vlan_cfi,
  				      pkt_dev->vlan_p);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2865
  		vlan_encapsulated_proto = (__be16 *)skb_put(skb, sizeof(__be16));
d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2866
  		*vlan_encapsulated_proto = htons(ETH_P_IPV6);
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
2867
  	}
27a884dc3   Arnaldo Carvalho de Melo   [SK_BUFF]: Conver...
2868
  	skb->network_header = skb->tail;
b0e380b1d   Arnaldo Carvalho de Melo   [SK_BUFF]: unions...
2869
  	skb->transport_header = skb->network_header + sizeof(struct ipv6hdr);
ddc7b8e32   Arnaldo Carvalho de Melo   [SK_BUFF]: Some m...
2870
  	skb_put(skb, sizeof(struct ipv6hdr) + sizeof(struct udphdr));
fd2ea0a79   David S. Miller   net: Use queue aw...
2871
  	skb_set_queue_mapping(skb, queue_map);
9e50e3ac5   John Fastabend   net: add priority...
2872
  	skb->priority = pkt_dev->skb_priority;
ddc7b8e32   Arnaldo Carvalho de Melo   [SK_BUFF]: Some m...
2873
2874
  	iph = ipv6_hdr(skb);
  	udph = udp_hdr(skb);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2875

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2876
  	memcpy(eth, pkt_dev->hh, 12);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
2877
  	*(__be16 *) &eth[12] = protocol;
64053beeb   Robert Olsson   [PKTGEN]: Fix ran...
2878

ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2879
2880
2881
  	/* Eth + IPh + UDPh + mpls */
  	datalen = pkt_dev->cur_pkt_size - 14 -
  		  sizeof(struct ipv6hdr) - sizeof(struct udphdr) -
16dab72f6   Jamal Hadi Salim   [PKTGEN]: Central...
2882
  		  pkt_dev->pkt_overhead;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2883

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2884
  	if (datalen < sizeof(struct pktgen_hdr)) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2885
2886
  		datalen = sizeof(struct pktgen_hdr);
  		if (net_ratelimit())
f9467eaec   Joe Perches   net/core/pktgen.c...
2887
2888
  			pr_info("increased datalen to %d
  ", datalen);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2889
2890
2891
2892
  	}
  
  	udph->source = htons(pkt_dev->cur_udp_src);
  	udph->dest = htons(pkt_dev->cur_udp_dst);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2893
2894
  	udph->len = htons(datalen + sizeof(struct udphdr));
  	udph->check = 0;	/* No checksum */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2895

d5f1ce9a5   Stephen Hemminger   [PKTGEN]: don't u...
2896
  	*(__be32 *) iph = htonl(0x60000000);	/* Version + flow */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2897

1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
2898
2899
  	if (pkt_dev->traffic_class) {
  		/* Version + traffic class + flow (0) */
252e33467   Al Viro   [NET] net/core: A...
2900
  		*(__be32 *)iph |= htonl(0x60000000 | (pkt_dev->traffic_class << 20));
1ca7768c8   Francesco Fondelli   [PKTGEN]: DSCP su...
2901
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2902
2903
2904
2905
  	iph->hop_limit = 32;
  
  	iph->payload_len = htons(sizeof(struct udphdr) + datalen);
  	iph->nexthdr = IPPROTO_UDP;
4e3fd7a06   Alexey Dobriyan   net: remove ipv6_...
2906
2907
  	iph->daddr = pkt_dev->cur_in6_daddr;
  	iph->saddr = pkt_dev->cur_in6_saddr;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2908

b0e380b1d   Arnaldo Carvalho de Melo   [SK_BUFF]: unions...
2909
  	skb->mac_header = (skb->network_header - ETH_HLEN -
16dab72f6   Jamal Hadi Salim   [PKTGEN]: Central...
2910
  			   pkt_dev->pkt_overhead);
ca6549af7   Steven Whitehouse   [PKTGEN]: Add MPL...
2911
  	skb->protocol = protocol;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2912
2913
  	skb->dev = odev;
  	skb->pkt_type = PACKET_HOST;
26ad78796   Eric Dumazet   pktgen: speedup f...
2914
  	pktgen_finalize_skb(pkt_dev, skb, datalen);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2915

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2916
2917
  	return skb;
  }
475ac1e40   Stephen Hemminger   pktgen: change in...
2918
2919
  static struct sk_buff *fill_packet(struct net_device *odev,
  				   struct pktgen_dev *pkt_dev)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2920
  {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2921
  	if (pkt_dev->flags & F_IPV6)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2922
2923
2924
2925
  		return fill_packet_ipv6(odev, pkt_dev);
  	else
  		return fill_packet_ipv4(odev, pkt_dev);
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2926
  static void pktgen_clear_counters(struct pktgen_dev *pkt_dev)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2927
  {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2928
2929
  	pkt_dev->seq_num = 1;
  	pkt_dev->idle_acc = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2930
  	pkt_dev->sofar = 0;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2931
2932
  	pkt_dev->tx_bytes = 0;
  	pkt_dev->errors = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2933
2934
2935
2936
2937
2938
  }
  
  /* Set up structure for sending pkts, clear counters */
  
  static void pktgen_run(struct pktgen_thread *t)
  {
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
2939
  	struct pktgen_dev *pkt_dev;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2940
  	int started = 0;
f9467eaec   Joe Perches   net/core/pktgen.c...
2941
  	func_enter();
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2942
2943
  
  	if_lock(t);
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
2944
  	list_for_each_entry(pkt_dev, &t->if_list, list) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2945
2946
2947
2948
2949
  
  		/*
  		 * setup odev and create initial packet.
  		 */
  		pktgen_setup_inject(pkt_dev);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2950
  		if (pkt_dev->odev) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2951
  			pktgen_clear_counters(pkt_dev);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2952
  			pkt_dev->running = 1;	/* Cranke yeself! */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2953
  			pkt_dev->skb = NULL;
fd29cf726   Stephen Hemminger   pktgen: convert t...
2954
2955
  			pkt_dev->started_at =
  				pkt_dev->next_tx = ktime_now();
16dab72f6   Jamal Hadi Salim   [PKTGEN]: Central...
2956
  			set_pkt_overhead(pkt_dev);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2957

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2958
2959
  			strcpy(pkt_dev->result, "Starting");
  			started++;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2960
  		} else
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2961
2962
2963
  			strcpy(pkt_dev->result, "Error starting");
  	}
  	if_unlock(t);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2964
2965
  	if (started)
  		t->control &= ~(T_STOP);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2966
2967
2968
2969
  }
  
  static void pktgen_stop_all_threads_ifs(void)
  {
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
2970
  	struct pktgen_thread *t;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2971

f9467eaec   Joe Perches   net/core/pktgen.c...
2972
  	func_enter();
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2973

6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
2974
  	mutex_lock(&pktgen_thread_lock);
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
2975
2976
  
  	list_for_each_entry(t, &pktgen_threads, th_list)
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
2977
  		t->control |= T_STOP;
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
2978

6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
2979
  	mutex_unlock(&pktgen_thread_lock);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2980
  }
648fda740   Stephen Hemminger   pktgen: minor cle...
2981
  static int thread_is_running(const struct pktgen_thread *t)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2982
  {
648fda740   Stephen Hemminger   pktgen: minor cle...
2983
  	const struct pktgen_dev *pkt_dev;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2984

c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
2985
  	list_for_each_entry(pkt_dev, &t->if_list, list)
648fda740   Stephen Hemminger   pktgen: minor cle...
2986
2987
2988
  		if (pkt_dev->running)
  			return 1;
  	return 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2989
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2990
  static int pktgen_wait_thread_run(struct pktgen_thread *t)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2991
  {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2992
  	if_lock(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2993

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2994
  	while (thread_is_running(t)) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2995

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2996
  		if_unlock(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2997

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
2998
  		msleep_interruptible(100);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
2999

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3000
3001
3002
3003
3004
3005
3006
3007
  		if (signal_pending(current))
  			goto signal;
  		if_lock(t);
  	}
  	if_unlock(t);
  	return 1;
  signal:
  	return 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3008
3009
3010
3011
  }
  
  static int pktgen_wait_all_threads_run(void)
  {
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3012
  	struct pktgen_thread *t;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3013
  	int sig = 1;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3014

6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
3015
  	mutex_lock(&pktgen_thread_lock);
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3016
3017
  
  	list_for_each_entry(t, &pktgen_threads, th_list) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3018
  		sig = pktgen_wait_thread_run(t);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3019
3020
  		if (sig == 0)
  			break;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3021
  	}
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3022
3023
3024
  
  	if (sig == 0)
  		list_for_each_entry(t, &pktgen_threads, th_list)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3025
  			t->control |= (T_STOP);
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3026

6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
3027
  	mutex_unlock(&pktgen_thread_lock);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3028
3029
3030
3031
3032
  	return sig;
  }
  
  static void pktgen_run_all_threads(void)
  {
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3033
  	struct pktgen_thread *t;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3034

f9467eaec   Joe Perches   net/core/pktgen.c...
3035
  	func_enter();
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3036

6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
3037
  	mutex_lock(&pktgen_thread_lock);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3038

cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3039
  	list_for_each_entry(t, &pktgen_threads, th_list)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3040
  		t->control |= (T_RUN);
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3041

6146e6a43   Luiz Capitulino   [PKTGEN]: Removes...
3042
  	mutex_unlock(&pktgen_thread_lock);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3043

63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
3044
3045
  	/* Propagate thread->control  */
  	schedule_timeout_interruptible(msecs_to_jiffies(125));
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3046

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3047
3048
  	pktgen_wait_all_threads_run();
  }
eb37b41cc   Jesse Brandeburg   pktgen: add full ...
3049
3050
3051
  static void pktgen_reset_all_threads(void)
  {
  	struct pktgen_thread *t;
f9467eaec   Joe Perches   net/core/pktgen.c...
3052
  	func_enter();
eb37b41cc   Jesse Brandeburg   pktgen: add full ...
3053
3054
3055
3056
3057
3058
3059
  
  	mutex_lock(&pktgen_thread_lock);
  
  	list_for_each_entry(t, &pktgen_threads, th_list)
  		t->control |= (T_REMDEVALL);
  
  	mutex_unlock(&pktgen_thread_lock);
63adc6fb8   Stephen Hemminger   pktgen: cleanup c...
3060
3061
  	/* Propagate thread->control  */
  	schedule_timeout_interruptible(msecs_to_jiffies(125));
eb37b41cc   Jesse Brandeburg   pktgen: add full ...
3062
3063
3064
  
  	pktgen_wait_all_threads_run();
  }
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3065
3066
  static void show_results(struct pktgen_dev *pkt_dev, int nr_frags)
  {
fd29cf726   Stephen Hemminger   pktgen: convert t...
3067
  	__u64 bps, mbps, pps;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3068
  	char *p = pkt_dev->result;
fd29cf726   Stephen Hemminger   pktgen: convert t...
3069
3070
3071
  	ktime_t elapsed = ktime_sub(pkt_dev->stopped_at,
  				    pkt_dev->started_at);
  	ktime_t idle = ns_to_ktime(pkt_dev->idle_acc);
03a14ab13   Daniel Turull   pktgen: fix errat...
3072
3073
  	p += sprintf(p, "OK: %llu(c%llu+d%llu) usec, %llu (%dbyte,%dfrags)
  ",
fd29cf726   Stephen Hemminger   pktgen: convert t...
3074
3075
3076
  		     (unsigned long long)ktime_to_us(elapsed),
  		     (unsigned long long)ktime_to_us(ktime_sub(elapsed, idle)),
  		     (unsigned long long)ktime_to_us(idle),
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3077
3078
  		     (unsigned long long)pkt_dev->sofar,
  		     pkt_dev->cur_pkt_size, nr_frags);
fd29cf726   Stephen Hemminger   pktgen: convert t...
3079
3080
  	pps = div64_u64(pkt_dev->sofar * NSEC_PER_SEC,
  			ktime_to_ns(elapsed));
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3081
3082
3083
3084
3085
3086
3087
3088
3089
3090
  
  	bps = pps * 8 * pkt_dev->cur_pkt_size;
  
  	mbps = bps;
  	do_div(mbps, 1000000);
  	p += sprintf(p, "  %llupps %lluMb/sec (%llubps) errors: %llu",
  		     (unsigned long long)pps,
  		     (unsigned long long)mbps,
  		     (unsigned long long)bps,
  		     (unsigned long long)pkt_dev->errors);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3091
  }
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3092
3093
  
  /* Set stopped-at timer, remove from running list, do counters & statistics */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3094
  static int pktgen_stop_device(struct pktgen_dev *pkt_dev)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3095
  {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3096
  	int nr_frags = pkt_dev->skb ? skb_shinfo(pkt_dev->skb)->nr_frags : -1;
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3097

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3098
  	if (!pkt_dev->running) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3099
3100
3101
  		pr_warning("interface: %s is already stopped
  ",
  			   pkt_dev->odevname);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3102
3103
  		return -EINVAL;
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3104

3bda06a3d   Stephen Hemminger   pktgen: stop_devi...
3105
3106
  	kfree_skb(pkt_dev->skb);
  	pkt_dev->skb = NULL;
fd29cf726   Stephen Hemminger   pktgen: convert t...
3107
  	pkt_dev->stopped_at = ktime_now();
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3108
  	pkt_dev->running = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3109

95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3110
  	show_results(pkt_dev, nr_frags);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3111

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3112
  	return 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3113
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3114
  static struct pktgen_dev *next_to_run(struct pktgen_thread *t)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3115
  {
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3116
  	struct pktgen_dev *pkt_dev, *best = NULL;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3117

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3118
  	if_lock(t);
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3119
3120
  	list_for_each_entry(pkt_dev, &t->if_list, list) {
  		if (!pkt_dev->running)
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3121
3122
  			continue;
  		if (best == NULL)
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3123
  			best = pkt_dev;
fd29cf726   Stephen Hemminger   pktgen: convert t...
3124
  		else if (ktime_lt(pkt_dev->next_tx, best->next_tx))
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3125
  			best = pkt_dev;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3126
3127
  	}
  	if_unlock(t);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3128
  	return best;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3129
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3130
3131
  static void pktgen_stop(struct pktgen_thread *t)
  {
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3132
  	struct pktgen_dev *pkt_dev;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3133

f9467eaec   Joe Perches   net/core/pktgen.c...
3134
  	func_enter();
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3135

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3136
  	if_lock(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3137

c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3138
3139
  	list_for_each_entry(pkt_dev, &t->if_list, list) {
  		pktgen_stop_device(pkt_dev);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3140
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3141

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3142
  	if_unlock(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3143
  }
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3144
3145
3146
3147
3148
3149
  /*
   * one of our devices needs to be removed - find it
   * and remove it
   */
  static void pktgen_rem_one_if(struct pktgen_thread *t)
  {
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3150
3151
  	struct list_head *q, *n;
  	struct pktgen_dev *cur;
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3152

f9467eaec   Joe Perches   net/core/pktgen.c...
3153
  	func_enter();
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3154
3155
  
  	if_lock(t);
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3156
3157
  	list_for_each_safe(q, n, &t->if_list) {
  		cur = list_entry(q, struct pktgen_dev, list);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3158

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3159
3160
  		if (!cur->removal_mark)
  			continue;
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3161

86dc1ad2b   Wei Yongjun   pktgen: remove so...
3162
  		kfree_skb(cur->skb);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3163
3164
3165
3166
3167
3168
3169
3170
3171
  		cur->skb = NULL;
  
  		pktgen_remove_device(t, cur);
  
  		break;
  	}
  
  	if_unlock(t);
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3172
  static void pktgen_rem_all_ifs(struct pktgen_thread *t)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3173
  {
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3174
3175
  	struct list_head *q, *n;
  	struct pktgen_dev *cur;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3176

f9467eaec   Joe Perches   net/core/pktgen.c...
3177
  	func_enter();
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3178
  	/* Remove all devices, free mem */
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3179

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3180
  	if_lock(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3181

c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3182
3183
  	list_for_each_safe(q, n, &t->if_list) {
  		cur = list_entry(q, struct pktgen_dev, list);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3184

86dc1ad2b   Wei Yongjun   pktgen: remove so...
3185
  		kfree_skb(cur->skb);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3186
  		cur->skb = NULL;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3187
3188
  		pktgen_remove_device(t, cur);
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3189
  	if_unlock(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3190
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3191
  static void pktgen_rem_thread(struct pktgen_thread *t)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3192
  {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3193
  	/* Remove from the thread list */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3194

ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3195
  	remove_proc_entry(t->tsk->comm, pg_proc_dir);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3196

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3197
  }
ef87979c2   Stephen Hemminger   pktgen: better sc...
3198
  static void pktgen_resched(struct pktgen_dev *pkt_dev)
3791decb5   Stephen Hemminger   pktgen: xmit logi...
3199
  {
fd29cf726   Stephen Hemminger   pktgen: convert t...
3200
  	ktime_t idle_start = ktime_now();
ef87979c2   Stephen Hemminger   pktgen: better sc...
3201
3202
3203
  	schedule();
  	pkt_dev->idle_acc += ktime_to_ns(ktime_sub(ktime_now(), idle_start));
  }
3791decb5   Stephen Hemminger   pktgen: xmit logi...
3204

ef87979c2   Stephen Hemminger   pktgen: better sc...
3205
3206
3207
  static void pktgen_wait_for_skb(struct pktgen_dev *pkt_dev)
  {
  	ktime_t idle_start = ktime_now();
3791decb5   Stephen Hemminger   pktgen: xmit logi...
3208

ef87979c2   Stephen Hemminger   pktgen: better sc...
3209
3210
3211
3212
3213
3214
3215
3216
3217
  	while (atomic_read(&(pkt_dev->skb->users)) != 1) {
  		if (signal_pending(current))
  			break;
  
  		if (need_resched())
  			pktgen_resched(pkt_dev);
  		else
  			cpu_relax();
  	}
fd29cf726   Stephen Hemminger   pktgen: convert t...
3218
  	pkt_dev->idle_acc += ktime_to_ns(ktime_sub(ktime_now(), idle_start));
3791decb5   Stephen Hemminger   pktgen: xmit logi...
3219
  }
475ac1e40   Stephen Hemminger   pktgen: change in...
3220
  static void pktgen_xmit(struct pktgen_dev *pkt_dev)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3221
  {
008298231   Stephen Hemminger   netdev: add more ...
3222
  	struct net_device *odev = pkt_dev->odev;
6fef4c0c8   Stephen Hemminger   netdev: convert p...
3223
  	netdev_tx_t (*xmit)(struct sk_buff *, struct net_device *)
008298231   Stephen Hemminger   netdev: add more ...
3224
  		= odev->netdev_ops->ndo_start_xmit;
fd2ea0a79   David S. Miller   net: Use queue aw...
3225
  	struct netdev_queue *txq;
fd2ea0a79   David S. Miller   net: Use queue aw...
3226
  	u16 queue_map;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3227
  	int ret;
ef87979c2   Stephen Hemminger   pktgen: better sc...
3228
3229
3230
3231
  	/* If device is offline, then don't send */
  	if (unlikely(!netif_running(odev) || !netif_carrier_ok(odev))) {
  		pktgen_stop_device(pkt_dev);
  		return;
fd2ea0a79   David S. Miller   net: Use queue aw...
3232
  	}
ef87979c2   Stephen Hemminger   pktgen: better sc...
3233
3234
3235
3236
3237
  	/* This is max DELAY, this has special meaning of
  	 * "never transmit"
  	 */
  	if (unlikely(pkt_dev->delay == ULLONG_MAX)) {
  		pkt_dev->next_tx = ktime_add_ns(ktime_now(), ULONG_MAX);
3791decb5   Stephen Hemminger   pktgen: xmit logi...
3238
  		return;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3239
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3240

ef87979c2   Stephen Hemminger   pktgen: better sc...
3241
  	/* If no skb or clone count exhausted then get new one */
7d7bb1cf0   Stephen Hemminger   pktgen: cleanup c...
3242
3243
3244
3245
3246
3247
3248
  	if (!pkt_dev->skb || (pkt_dev->last_ok &&
  			      ++pkt_dev->clone_count >= pkt_dev->clone_skb)) {
  		/* build a new pkt */
  		kfree_skb(pkt_dev->skb);
  
  		pkt_dev->skb = fill_packet(odev, pkt_dev);
  		if (pkt_dev->skb == NULL) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3249
3250
  			pr_err("ERROR: couldn't allocate skb in fill_packet
  ");
7d7bb1cf0   Stephen Hemminger   pktgen: cleanup c...
3251
3252
3253
  			schedule();
  			pkt_dev->clone_count--;	/* back out increment, OOM */
  			return;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3254
  		}
baac85645   Eric Dumazet   pktgen: tx_bytes ...
3255
  		pkt_dev->last_pkt_size = pkt_dev->skb->len;
7d7bb1cf0   Stephen Hemminger   pktgen: cleanup c...
3256
3257
  		pkt_dev->allocated_skbs++;
  		pkt_dev->clone_count = 0;	/* reset counter */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3258
  	}
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3259

ef87979c2   Stephen Hemminger   pktgen: better sc...
3260
3261
  	if (pkt_dev->delay && pkt_dev->last_ok)
  		spin(pkt_dev, pkt_dev->next_tx);
fd2ea0a79   David S. Miller   net: Use queue aw...
3262
3263
3264
3265
  	queue_map = skb_get_queue_mapping(pkt_dev->skb);
  	txq = netdev_get_tx_queue(odev, queue_map);
  
  	__netif_tx_lock_bh(txq);
5b8db2f56   Stephen Hemminger   pktgen: reorganiz...
3266

734664982   Tom Herbert   net: Add queue st...
3267
  	if (unlikely(netif_xmit_frozen_or_stopped(txq))) {
ef87979c2   Stephen Hemminger   pktgen: better sc...
3268
  		ret = NETDEV_TX_BUSY;
0835acfe7   Eric Dumazet   pktgen: Avoid dir...
3269
3270
3271
3272
3273
  		pkt_dev->last_ok = 0;
  		goto unlock;
  	}
  	atomic_inc(&(pkt_dev->skb->users));
  	ret = (*xmit)(pkt_dev->skb, odev);
ef87979c2   Stephen Hemminger   pktgen: better sc...
3274
3275
3276
3277
3278
3279
3280
  
  	switch (ret) {
  	case NETDEV_TX_OK:
  		txq_trans_update(txq);
  		pkt_dev->last_ok = 1;
  		pkt_dev->sofar++;
  		pkt_dev->seq_num++;
baac85645   Eric Dumazet   pktgen: tx_bytes ...
3281
  		pkt_dev->tx_bytes += pkt_dev->last_pkt_size;
ef87979c2   Stephen Hemminger   pktgen: better sc...
3282
  		break;
f466dba18   John Fastabend   pktgen: ndo_start...
3283
3284
3285
3286
3287
3288
  	case NET_XMIT_DROP:
  	case NET_XMIT_CN:
  	case NET_XMIT_POLICED:
  		/* skb has been consumed */
  		pkt_dev->errors++;
  		break;
ef87979c2   Stephen Hemminger   pktgen: better sc...
3289
3290
  	default: /* Drivers are not supposed to return other values! */
  		if (net_ratelimit())
58231186c   Joe Perches   pktgen: Remove un...
3291
3292
  			pr_info("%s xmit error: %d
  ", pkt_dev->odevname, ret);
ef87979c2   Stephen Hemminger   pktgen: better sc...
3293
3294
3295
3296
3297
3298
3299
  		pkt_dev->errors++;
  		/* fallthru */
  	case NETDEV_TX_LOCKED:
  	case NETDEV_TX_BUSY:
  		/* Retry it next time */
  		atomic_dec(&(pkt_dev->skb->users));
  		pkt_dev->last_ok = 0;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3300
  	}
0835acfe7   Eric Dumazet   pktgen: Avoid dir...
3301
  unlock:
fd2ea0a79   David S. Miller   net: Use queue aw...
3302
  	__netif_tx_unlock_bh(txq);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3303

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3304
3305
  	/* If pkt_dev->count is zero, then run forever */
  	if ((pkt_dev->count != 0) && (pkt_dev->sofar >= pkt_dev->count)) {
ef87979c2   Stephen Hemminger   pktgen: better sc...
3306
  		pktgen_wait_for_skb(pkt_dev);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3307

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3308
3309
  		/* Done with this */
  		pktgen_stop_device(pkt_dev);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3310
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3311
  }
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3312

4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
3313
  /*
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3314
3315
   * Main loop of the thread goes here
   */
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3316
  static int pktgen_thread_worker(void *arg)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3317
3318
  {
  	DEFINE_WAIT(wait);
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3319
  	struct pktgen_thread *t = arg;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3320
  	struct pktgen_dev *pkt_dev = NULL;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3321
  	int cpu = t->cpu;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3322

ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3323
  	BUG_ON(smp_processor_id() != cpu);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3324
3325
  
  	init_waitqueue_head(&t->queue);
d3ede327e   Denis V. Lunev   pktgen: make sure...
3326
  	complete(&t->start_done);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3327

f9467eaec   Joe Perches   net/core/pktgen.c...
3328
3329
  	pr_debug("starting pktgen/%d:  pid=%d
  ", cpu, task_pid_nr(current));
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3330

ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3331
  	set_current_state(TASK_INTERRUPTIBLE);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3332

831441862   Rafael J. Wysocki   Freezer: make ker...
3333
  	set_freezable();
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3334
3335
  	while (!kthread_should_stop()) {
  		pkt_dev = next_to_run(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3336

ef87979c2   Stephen Hemminger   pktgen: better sc...
3337
  		if (unlikely(!pkt_dev && t->control == 0)) {
551eaff1b   Eric Dumazet   pktgen: allow fas...
3338
3339
  			if (pktgen_exiting)
  				break;
ef87979c2   Stephen Hemminger   pktgen: better sc...
3340
3341
3342
  			wait_event_interruptible_timeout(t->queue,
  							 t->control != 0,
  							 HZ/10);
1b3f720bf   Rafael J. Wysocki   pktgen: Fix freez...
3343
  			try_to_freeze();
ef87979c2   Stephen Hemminger   pktgen: better sc...
3344
  			continue;
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3345
  		}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3346

ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3347
  		__set_current_state(TASK_RUNNING);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3348

ef87979c2   Stephen Hemminger   pktgen: better sc...
3349
  		if (likely(pkt_dev)) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3350
  			pktgen_xmit(pkt_dev);
ef87979c2   Stephen Hemminger   pktgen: better sc...
3351
3352
3353
3354
3355
  			if (need_resched())
  				pktgen_resched(pkt_dev);
  			else
  				cpu_relax();
  		}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3356
  		if (t->control & T_STOP) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3357
3358
3359
  			pktgen_stop(t);
  			t->control &= ~(T_STOP);
  		}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3360
  		if (t->control & T_RUN) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3361
3362
3363
  			pktgen_run(t);
  			t->control &= ~(T_RUN);
  		}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3364
  		if (t->control & T_REMDEVALL) {
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3365
  			pktgen_rem_all_ifs(t);
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3366
3367
  			t->control &= ~(T_REMDEVALL);
  		}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3368
  		if (t->control & T_REMDEV) {
95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3369
  			pktgen_rem_one_if(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3370
3371
  			t->control &= ~(T_REMDEV);
  		}
09fe3ef46   Andrew Morton   [PKTGEN]: Add try...
3372
  		try_to_freeze();
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3373
  		set_current_state(TASK_INTERRUPTIBLE);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3374
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3375

f9467eaec   Joe Perches   net/core/pktgen.c...
3376
3377
  	pr_debug("%s stopping all device
  ", t->tsk->comm);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3378
  	pktgen_stop(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3379

f9467eaec   Joe Perches   net/core/pktgen.c...
3380
3381
  	pr_debug("%s removing all device
  ", t->tsk->comm);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3382
  	pktgen_rem_all_ifs(t);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3383

f9467eaec   Joe Perches   net/core/pktgen.c...
3384
3385
  	pr_debug("%s removing thread
  ", t->tsk->comm);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3386
  	pktgen_rem_thread(t);
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3387

551eaff1b   Eric Dumazet   pktgen: allow fas...
3388
3389
3390
3391
3392
3393
  	/* Wait for kthread_stop */
  	while (!kthread_should_stop()) {
  		set_current_state(TASK_INTERRUPTIBLE);
  		schedule();
  	}
  	__set_current_state(TASK_RUNNING);
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3394
  	return 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3395
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3396
  static struct pktgen_dev *pktgen_find_dev(struct pktgen_thread *t,
3e9848403   Eric Dumazet   pktgen: Fix netde...
3397
  					  const char *ifname, bool exact)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3398
  {
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3399
  	struct pktgen_dev *p, *pkt_dev = NULL;
3e9848403   Eric Dumazet   pktgen: Fix netde...
3400
  	size_t len = strlen(ifname);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3401

3e9848403   Eric Dumazet   pktgen: Fix netde...
3402
  	if_lock(t);
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3403
  	list_for_each_entry(p, &t->if_list, list)
3e9848403   Eric Dumazet   pktgen: Fix netde...
3404
3405
3406
3407
3408
  		if (strncmp(p->odevname, ifname, len) == 0) {
  			if (p->odevname[len]) {
  				if (exact || p->odevname[len] != '@')
  					continue;
  			}
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3409
  			pkt_dev = p;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3410
3411
  			break;
  		}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3412
3413
  
  	if_unlock(t);
f9467eaec   Joe Perches   net/core/pktgen.c...
3414
3415
  	pr_debug("find_dev(%s) returning %p
  ", ifname, pkt_dev);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3416
  	return pkt_dev;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3417
  }
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
3418
3419
  /*
   * Adds a dev at front of if_list.
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3420
   */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3421
3422
  static int add_dev_to_thread(struct pktgen_thread *t,
  			     struct pktgen_dev *pkt_dev)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3423
3424
  {
  	int rv = 0;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3425
3426
3427
3428
  
  	if_lock(t);
  
  	if (pkt_dev->pg_thread) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3429
3430
  		pr_err("ERROR: already assigned to a thread
  ");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3431
3432
3433
  		rv = -EBUSY;
  		goto out;
  	}
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3434
3435
  
  	list_add(&pkt_dev->list, &t->if_list);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3436
  	pkt_dev->pg_thread = t;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3437
  	pkt_dev->running = 0;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3438
3439
3440
  out:
  	if_unlock(t);
  	return rv;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3441
3442
3443
  }
  
  /* Called under thread lock */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3444
  static int pktgen_add_device(struct pktgen_thread *t, const char *ifname)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3445
  {
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3446
  	struct pktgen_dev *pkt_dev;
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
3447
  	int err;
3291b9db5   Eric Dumazet   pktgen: NUMA aware
3448
  	int node = cpu_to_node(t->cpu);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3449

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3450
  	/* We don't allow a device to be on several threads */
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3451
3452
  	pkt_dev = __pktgen_NN_threads(ifname, FIND);
  	if (pkt_dev) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3453
3454
  		pr_err("ERROR: interface already used
  ");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3455
3456
  		return -EBUSY;
  	}
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3457

3291b9db5   Eric Dumazet   pktgen: NUMA aware
3458
  	pkt_dev = kzalloc_node(sizeof(struct pktgen_dev), GFP_KERNEL, node);
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3459
3460
  	if (!pkt_dev)
  		return -ENOMEM;
593f63b0b   Eric Dumazet   pktgen: Fix devic...
3461
  	strcpy(pkt_dev->odevname, ifname);
68d5ac2ed   WANG Cong   pktgen: use vzall...
3462
  	pkt_dev->flows = vzalloc_node(MAX_CFLOWS * sizeof(struct flow_state),
3291b9db5   Eric Dumazet   pktgen: NUMA aware
3463
  				      node);
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3464
3465
3466
3467
  	if (pkt_dev->flows == NULL) {
  		kfree(pkt_dev);
  		return -ENOMEM;
  	}
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3468

95ed63f79   Arthur Kepner   [NET] pktgen: Fix...
3469
  	pkt_dev->removal_mark = 0;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3470
3471
3472
  	pkt_dev->min_pkt_size = ETH_ZLEN;
  	pkt_dev->max_pkt_size = ETH_ZLEN;
  	pkt_dev->nfrags = 0;
fd29cf726   Stephen Hemminger   pktgen: convert t...
3473
  	pkt_dev->delay = pg_delay_d;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3474
3475
  	pkt_dev->count = pg_count_d;
  	pkt_dev->sofar = 0;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3476
  	pkt_dev->udp_src_min = 9;	/* sink port */
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3477
3478
3479
  	pkt_dev->udp_src_max = 9;
  	pkt_dev->udp_dst_min = 9;
  	pkt_dev->udp_dst_max = 9;
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
3480
3481
3482
3483
3484
3485
  	pkt_dev->vlan_p = 0;
  	pkt_dev->vlan_cfi = 0;
  	pkt_dev->vlan_id = 0xffff;
  	pkt_dev->svlan_p = 0;
  	pkt_dev->svlan_cfi = 0;
  	pkt_dev->svlan_id = 0xffff;
e99b99b47   Robert Olsson   pktgen node alloc...
3486
  	pkt_dev->node = -1;
34954ddc4   Francesco Fondelli   [PKTGEN]: vlan su...
3487

39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
3488
3489
3490
  	err = pktgen_setup_dev(pkt_dev, ifname);
  	if (err)
  		goto out1;
d88733150   Neil Horman   net: add IFF_SKB_...
3491
3492
  	if (pkt_dev->odev->priv_flags & IFF_TX_SKB_SHARING)
  		pkt_dev->clone_skb = pg_clone_skb_d;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3493

5efdccbcd   Denis V. Lunev   net: assign PDE->...
3494
3495
  	pkt_dev->entry = proc_create_data(ifname, 0600, pg_proc_dir,
  					  &pktgen_if_fops, pkt_dev);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
3496
  	if (!pkt_dev->entry) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3497
3498
  		pr_err("cannot create %s/%s procfs entry
  ",
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3499
  		       PG_PROC_DIR, ifname);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
3500
3501
  		err = -EINVAL;
  		goto out2;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3502
  	}
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
3503
3504
3505
3506
  #ifdef CONFIG_XFRM
  	pkt_dev->ipsmode = XFRM_MODE_TRANSPORT;
  	pkt_dev->ipsproto = IPPROTO_ESP;
  #endif
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3507
3508
  
  	return add_dev_to_thread(t, pkt_dev);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
3509
3510
3511
  out2:
  	dev_put(pkt_dev->odev);
  out1:
a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
3512
3513
3514
  #ifdef CONFIG_XFRM
  	free_SAs(pkt_dev);
  #endif
1d0ebfe7c   Figo.zhang   net pkgen.c:fix n...
3515
  	vfree(pkt_dev->flows);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
3516
3517
  	kfree(pkt_dev);
  	return err;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3518
  }
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3519
  static int __init pktgen_create_thread(int cpu)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3520
  {
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3521
  	struct pktgen_thread *t;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3522
  	struct proc_dir_entry *pe;
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3523
  	struct task_struct *p;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3524

3291b9db5   Eric Dumazet   pktgen: NUMA aware
3525
3526
  	t = kzalloc_node(sizeof(struct pktgen_thread), GFP_KERNEL,
  			 cpu_to_node(cpu));
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3527
  	if (!t) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3528
3529
  		pr_err("ERROR: out of memory, can't create new thread
  ");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3530
3531
  		return -ENOMEM;
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3532
  	spin_lock_init(&t->if_lock);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3533
  	t->cpu = cpu;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3534

ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3535
3536
3537
  	INIT_LIST_HEAD(&t->if_list);
  
  	list_add_tail(&t->th_list, &pktgen_threads);
d3ede327e   Denis V. Lunev   pktgen: make sure...
3538
  	init_completion(&t->start_done);
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3539

94dcf29a1   Eric Dumazet   kthread: use kthr...
3540
3541
3542
3543
  	p = kthread_create_on_node(pktgen_thread_worker,
  				   t,
  				   cpu_to_node(cpu),
  				   "kpktgend_%d", cpu);
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3544
  	if (IS_ERR(p)) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3545
3546
  		pr_err("kernel_thread() failed for cpu %d
  ", t->cpu);
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3547
3548
3549
3550
3551
3552
  		list_del(&t->th_list);
  		kfree(t);
  		return PTR_ERR(p);
  	}
  	kthread_bind(p, cpu);
  	t->tsk = p;
5efdccbcd   Denis V. Lunev   net: assign PDE->...
3553
3554
  	pe = proc_create_data(t->tsk->comm, 0600, pg_proc_dir,
  			      &pktgen_thread_fops, t);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3555
  	if (!pe) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3556
3557
  		pr_err("cannot create %s/%s procfs entry
  ",
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3558
3559
3560
  		       PG_PROC_DIR, t->tsk->comm);
  		kthread_stop(p);
  		list_del(&t->th_list);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3561
3562
3563
  		kfree(t);
  		return -EINVAL;
  	}
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3564

ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3565
  	wake_up_process(p);
d3ede327e   Denis V. Lunev   pktgen: make sure...
3566
  	wait_for_completion(&t->start_done);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3567
3568
3569
  
  	return 0;
  }
4ec93edb1   YOSHIFUJI Hideaki   [NET] CORE: Fix w...
3570
3571
  /*
   * Removes a device from the thread if_list.
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3572
   */
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3573
3574
  static void _rem_dev_from_if_list(struct pktgen_thread *t,
  				  struct pktgen_dev *pkt_dev)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3575
  {
c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3576
3577
  	struct list_head *q, *n;
  	struct pktgen_dev *p;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3578

c26a80168   Luiz Capitulino   [PKTGEN]: Ports i...
3579
3580
3581
3582
  	list_for_each_safe(q, n, &t->if_list) {
  		p = list_entry(q, struct pktgen_dev, list);
  		if (p == pkt_dev)
  			list_del(&p->list);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3583
3584
  	}
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3585
3586
  static int pktgen_remove_device(struct pktgen_thread *t,
  				struct pktgen_dev *pkt_dev)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3587
  {
f9467eaec   Joe Perches   net/core/pktgen.c...
3588
3589
  	pr_debug("remove_device pkt_dev=%p
  ", pkt_dev);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3590

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3591
  	if (pkt_dev->running) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3592
3593
  		pr_warning("WARNING: trying to remove a running interface, stopping it now
  ");
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3594
3595
3596
3597
  		pktgen_stop_device(pkt_dev);
  	}
  
  	/* Dis-associate from the interface */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3598
3599
3600
  
  	if (pkt_dev->odev) {
  		dev_put(pkt_dev->odev);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3601
3602
  		pkt_dev->odev = NULL;
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3603
3604
3605
  	/* And update the thread if_list */
  
  	_rem_dev_from_if_list(t, pkt_dev);
39df232f1   Stephen Hemminger   [PKTGEN]: fix dev...
3606
3607
  	if (pkt_dev->entry)
  		remove_proc_entry(pkt_dev->entry->name, pg_proc_dir);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3608

a553e4a63   Jamal Hadi Salim   [PKTGEN]: IPSEC s...
3609
3610
3611
  #ifdef CONFIG_XFRM
  	free_SAs(pkt_dev);
  #endif
1d0ebfe7c   Figo.zhang   net pkgen.c:fix n...
3612
  	vfree(pkt_dev->flows);
26ad78796   Eric Dumazet   pktgen: speedup f...
3613
3614
  	if (pkt_dev->page)
  		put_page(pkt_dev->page);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3615
  	kfree(pkt_dev);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3616
  	return 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3617
  }
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3618
  static int __init pg_init(void)
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3619
3620
  {
  	int cpu;
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3621
  	struct proc_dir_entry *pe;
ce14f8946   WANG Cong   pktgen: refactor ...
3622
  	int ret = 0;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3623

f9467eaec   Joe Perches   net/core/pktgen.c...
3624
  	pr_info("%s", version);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3625

457c4cbc5   Eric W. Biederman   [NET]: Make /proc...
3626
  	pg_proc_dir = proc_mkdir(PG_PROC_DIR, init_net.proc_net);
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3627
3628
  	if (!pg_proc_dir)
  		return -ENODEV;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3629

25296d599   Wang Chen   [PKTGEN]: Use pro...
3630
  	pe = proc_create(PGCTRL, 0600, pg_proc_dir, &pktgen_fops);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3631
  	if (pe == NULL) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3632
3633
  		pr_err("ERROR: cannot create %s procfs entry
  ", PGCTRL);
ce14f8946   WANG Cong   pktgen: refactor ...
3634
3635
  		ret = -EINVAL;
  		goto remove_dir;
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3636
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3637

1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3638
  	register_netdevice_notifier(&pktgen_notifier_block);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3639

670c02c2b   John Hawkes   [NET]: Wider use ...
3640
  	for_each_online_cpu(cpu) {
8024bb245   Luiz Capitulino   [PKTGEN]: Fix Ini...
3641
  		int err;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3642

ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3643
  		err = pktgen_create_thread(cpu);
8024bb245   Luiz Capitulino   [PKTGEN]: Fix Ini...
3644
  		if (err)
f9467eaec   Joe Perches   net/core/pktgen.c...
3645
3646
3647
  			pr_warning("WARNING: Cannot create thread for cpu %d (%d)
  ",
  				   cpu, err);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3648
  	}
8024bb245   Luiz Capitulino   [PKTGEN]: Fix Ini...
3649
3650
  
  	if (list_empty(&pktgen_threads)) {
f9467eaec   Joe Perches   net/core/pktgen.c...
3651
3652
  		pr_err("ERROR: Initialization failed for all threads
  ");
ce14f8946   WANG Cong   pktgen: refactor ...
3653
3654
  		ret = -ENODEV;
  		goto unregister;
8024bb245   Luiz Capitulino   [PKTGEN]: Fix Ini...
3655
  	}
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3656
  	return 0;
ce14f8946   WANG Cong   pktgen: refactor ...
3657
3658
3659
3660
3661
3662
3663
  
   unregister:
  	unregister_netdevice_notifier(&pktgen_notifier_block);
  	remove_proc_entry(PGCTRL, pg_proc_dir);
   remove_dir:
  	proc_net_remove(&init_net, PG_PROC_DIR);
  	return ret;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3664
3665
3666
3667
  }
  
  static void __exit pg_cleanup(void)
  {
cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3668
3669
  	struct pktgen_thread *t;
  	struct list_head *q, *n;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3670

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3671
  	/* Stop all interfaces & threads */
551eaff1b   Eric Dumazet   pktgen: allow fas...
3672
  	pktgen_exiting = true;
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3673

cdcdbe0b1   Luiz Capitulino   [PKTGEN]: Ports t...
3674
3675
  	list_for_each_safe(q, n, &pktgen_threads) {
  		t = list_entry(q, struct pktgen_thread, th_list);
ee74baa7d   David S. Miller   [PKTGEN]: Convert...
3676
3677
  		kthread_stop(t->tsk);
  		kfree(t);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3678
  	}
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3679

222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3680
  	/* Un-register us from receiving netdevice events */
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3681
  	unregister_netdevice_notifier(&pktgen_notifier_block);
222f18065   Luiz Capitulino   [PKTGEN]: Lindent...
3682
  	/* Clean up proc file system */
d50a6b56f   Stephen Hemminger   [PKTGEN]: proc in...
3683
  	remove_proc_entry(PGCTRL, pg_proc_dir);
457c4cbc5   Eric W. Biederman   [NET]: Make /proc...
3684
  	proc_net_remove(&init_net, PG_PROC_DIR);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3685
  }
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3686
3687
  module_init(pg_init);
  module_exit(pg_cleanup);
c3d2f52dd   Stephen Hemminger   pktgen: increase ...
3688
  MODULE_AUTHOR("Robert Olsson <robert.olsson@its.uu.se>");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3689
3690
  MODULE_DESCRIPTION("Packet Generator tool");
  MODULE_LICENSE("GPL");
c3d2f52dd   Stephen Hemminger   pktgen: increase ...
3691
  MODULE_VERSION(VERSION);
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3692
  module_param(pg_count_d, int, 0);
c3d2f52dd   Stephen Hemminger   pktgen: increase ...
3693
  MODULE_PARM_DESC(pg_count_d, "Default number of packets to inject");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3694
  module_param(pg_delay_d, int, 0);
c3d2f52dd   Stephen Hemminger   pktgen: increase ...
3695
  MODULE_PARM_DESC(pg_delay_d, "Default delay between packets (nanoseconds)");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3696
  module_param(pg_clone_skb_d, int, 0);
c3d2f52dd   Stephen Hemminger   pktgen: increase ...
3697
  MODULE_PARM_DESC(pg_clone_skb_d, "Default number of copies of the same packet");
1da177e4c   Linus Torvalds   Linux-2.6.12-rc2
3698
  module_param(debug, int, 0);
c3d2f52dd   Stephen Hemminger   pktgen: increase ...
3699
  MODULE_PARM_DESC(debug, "Enable debugging of pktgen module");