09 Sep, 2010
1 commit
-
commit 30fff923 introduced in linux-2.6.33 (udp: bind() optimisation)
added a secondary hash on UDP, hashed on (local addr, local port).Problem is that following sequence :
fd = socket(...)
connect(fd, &remote, ...)not only selects remote end point (address and port), but also sets
local address, while UDP stack stored in secondary hash table the socket
while its local address was INADDR_ANY (or ipv6 equivalent)Sequence is :
- autobind() : choose a random local port, insert socket in hash tables
[while local address is INADDR_ANY]
- connect() : set remote address and port, change local address to IP
given by a route lookup.When an incoming UDP frame comes, if more than 10 sockets are found in
primary hash table, we switch to secondary table, and fail to find
socket because its local address changed.One solution to this problem is to rehash datagram socket if needed.
We add a new rehash(struct socket *) method in "struct proto", and
implement this method for UDP v4 & v6, using a common helper.This rehashing only takes care of secondary hash table, since primary
hash (based on local port only) is not changed.Reported-by: Krzysztof Piotr Oledzki
Signed-off-by: Eric Dumazet
Tested-by: Krzysztof Piotr Oledzki
Signed-off-by: David S. Miller
07 Jun, 2010
1 commit
-
Conflicts:
drivers/net/sfc/net_driver.h
drivers/net/sfc/siena.c
02 Jun, 2010
1 commit
-
There are more than a dozen occurrences of following code in the
IPv6 stack:if (opt && opt->srcrt) {
struct rt0_hdr *rt0 = (struct rt0_hdr *) opt->srcrt;
ipv6_addr_copy(&final, &fl.fl6_dst);
ipv6_addr_copy(&fl.fl6_dst, rt0->addr);
final_p = &final;
}Replace those with a helper. Note that the helper overrides final_p
in all cases. This is ok as final_p was previously initialized to
NULL when declared.Signed-off-by: Arnaud Ebalard
Signed-off-by: David S. Miller
01 Jun, 2010
1 commit
-
Correct sk_forward_alloc handling for error_queue would need to use a
backlog of frames that softirq handler could not deliver because socket
is owned by user thread. Or extend backlog processing to be able to
process normal and error packets.Another possibility is to not use mem charge for error queue, this is
what I implemented in this patch.Note: this reverts commit 29030374
(net: fix sk_forward_alloc corruptions), since we dont need to lock
socket anymore.Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
29 May, 2010
1 commit
-
As David found out, sock_queue_err_skb() should be called with socket
lock hold, or we risk sk_forward_alloc corruption, since we use non
atomic operations to update this field.This patch adds bh_lock_sock()/bh_unlock_sock() pair to three spots.
(BH already disabled)1) skb_tstamp_tx()
2) Before calling ip_icmp_error(), in __udp4_lib_err()
3) Before calling ipv6_icmp_error(), in __udp6_lib_err()Reported-by: Anton Blanchard
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
27 May, 2010
1 commit
-
This new sock lock primitive was introduced to speedup some user context
socket manipulation. But it is unsafe to protect two threads, one using
regular lock_sock/release_sock, one using lock_sock_bh/unlock_sock_bhThis patch changes lock_sock_bh to be careful against 'owned' state.
If owned is found to be set, we must take the slow path.
lock_sock_bh() now returns a boolean to say if the slow path was taken,
and this boolean is used at unlock_sock_bh time to call the appropriate
unlock function.After this change, BH are either disabled or enabled during the
lock_sock_bh/unlock_sock_bh protected section. This might be misleading,
so we rename these functions to lock_sock_fast()/unlock_sock_fast().Reported-by: Anton Blanchard
Signed-off-by: Eric Dumazet
Tested-by: Anton Blanchard
Signed-off-by: David S. Miller
07 May, 2010
1 commit
-
Adding addresses and ports to the short packet log message,
like ipv4/udp.c does it, makes these messages a lot more useful:[ 822.182450] UDPv6: short packet: From [2001:db8:ffb4:3::1]:47839 23715/178 to [2001:db8:ffb4:3:5054:ff:feff:200]:1234
This requires us to drop logging in case pskb_may_pull() fails,
which also is consistent with ipv4/udp.cSigned-off-by: Bjørn Mork
Acked-by: Eric Dumazet
Signed-off-by: David S. Miller
29 Apr, 2010
2 commits
-
When queueing a skb to socket, we can immediately release its dst if
target socket do not use IP_CMSG_PKTINFO.tcp_data_queue() can drop dst too.
This to benefit from a hot cache line and avoid the receiver, possibly
on another cpu, to dirty this cache line himself.Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller -
Since commit 95766fff ([UDP]: Add memory accounting.),
each received packet needs one extra sock_lock()/sock_release() pair.This added latency because of possible backlog handling. Then later,
ticket spinlocks added yet another latency source in case of DDOS.This patch introduces lock_sock_bh() and unlock_sock_bh()
synchronization primitives, avoiding one atomic operation and backlog
processing.skb_free_datagram_locked() uses them instead of full blown
lock_sock()/release_sock(). skb is orphaned inside locked section for
proper socket memory reclaim, and finally freed outside of it.UDP receive path now take the socket spinlock only once.
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
28 Apr, 2010
1 commit
-
Current socket backlog limit is not enough to really stop DDOS attacks,
because user thread spend many time to process a full backlog each
round, and user might crazy spin on socket lock.We should add backlog size and receive_queue size (aka rmem_alloc) to
pace writers, and let user run without being slow down too much.Introduce a sk_rcvqueues_full() helper, to avoid taking socket lock in
stress situations.Under huge stress from a multiqueue/RPS enabled NIC, a single flow udp
receiver can now process ~200.000 pps (instead of ~100 pps before the
patch) on a 8 core machine.Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
24 Apr, 2010
2 commits
-
Finally add support to detect a local IPV6_DONTFRAG event
and return the relevant data to the user if they've enabled
IPV6_RECVPATHMTU on the socket. The next recvmsg() will
return no data, but have an IPV6_PATHMTU as ancillary data.Signed-off-by: Brian Haley
Signed-off-by: David S. Miller -
Add dontfrag argument to relevant functions for
IPV6_DONTFRAG support, as well as allowing the value
to be passed-in via ancillary cmsg data.Signed-off-by: Brian Haley
Signed-off-by: David S. Miller
21 Apr, 2010
1 commit
-
Sparse can help us find endianness bugs, but we need to make some
cleanups to be able to more easily spot real bugs.Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
11 Apr, 2010
1 commit
09 Apr, 2010
1 commit
-
Commits 5051ebd275de672b807c28d93002c2fb0514a3c9 and
5051ebd275de672b807c28d93002c2fb0514a3c9 ("ipv[46]: udp: optimize unicast RX
path") broke some programs.After upgrading a L2TP server to 2.6.33 it started to fail, tunnels going up an
down, after the 10th tunnel came up. My modified rp-l2tp uses a global
unconnected socket bound to (INADDR_ANY, 1701) and one connected socket per
tunnel after parameter negotiation.After ten sockets were open and due to mixed parameters to
udp[46]_lib_lookup2() kernel started to drop packets.Signed-off-by: Jorge Boncompte [DTI2]
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
30 Mar, 2010
1 commit
-
…it slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
06 Mar, 2010
2 commits
-
sk_add_backlog -> __sk_add_backlog
sk_add_backlog_limited -> sk_add_backlogSigned-off-by: Zhu Yi
Acked-by: Eric Dumazet
Signed-off-by: David S. Miller -
Make udp adapt to the limited socket backlog change.
Cc: "David S. Miller"
Cc: Alexey Kuznetsov
Cc: "Pekka Savola (ipv6)"
Cc: Patrick McHardy
Signed-off-by: Zhu Yi
Acked-by: Eric Dumazet
Signed-off-by: David S. Miller
19 Feb, 2010
1 commit
-
Dunno, what was the idea, it wasn't used for a long time.
Signed-off-by: Alexey Dobriyan
Signed-off-by: David S. Miller
13 Feb, 2010
1 commit
-
The variable 'copied' is used in udp_recvmsg() to emphasize that the passed
'len' is adjusted to fit the actual datagram length. But the same can be
done by adjusting 'len' directly. This patch thus removes the indirection.Signed-off-by: Gerrit Renker
Signed-off-by: David S. Miller
18 Jan, 2010
1 commit
-
__net_init/__net_exit are apparently not going away, so use them
to full extent.In some cases __net_init was removed, because it was called from
__net_exit code.Signed-off-by: Alexey Dobriyan
Signed-off-by: David S. Miller
11 Nov, 2009
2 commits
-
Change udp6_portaddr_hash() to use ipv6_addr_v4mapped()
inline instead of ipv6_addr_type().Signed-off-by: Brian Haley
Acked-by: Eric Dumazet
Signed-off-by: David S. Miller -
UDP bind() can be O(N^2) in some pathological cases.
Thanks to secondary hash tables, we can make it O(N)
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
09 Nov, 2009
4 commits
-
When skb_clone() fails, we should increment sk_drops and SNMP counters.
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller -
IPV6 UDP multicast rx path is a bit complex and can hold a spinlock
for a long time.Using a small (32 or 64 entries) stack of socket pointers can help
to perform expensive operations (skb_clone(), udp_queue_rcv_skb())
outside of the lock, in most cases.Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller -
We first locate the (local port) hash chain head
If few sockets are in this chain, we proceed with previous lookup algo.If too many sockets are listed, we take a look at the secondary
(port, address) hash chain.We choose the shortest chain and proceed with a RCU lookup on the elected chain.
But, if we chose (port, address) chain, and fail to find a socket on given address,
we must try another lookup on (port, in6addr_any) chain to find sockets not bound
to a particular IP.-> No extra cost for typical setups, where the first lookup will probabbly
be performed.RCU lookups everywhere, we dont acquire spinlock.
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller -
Union sk_hash with two u16 hashes for udp (no extra memory taken)
One 16 bits hash on (local port) value (the previous udp 'hash')
One 16 bits hash on (local address, local port) values, initialized
but not yet used. This second hash is using jenkin hash for better
distribution.Because the 'port' is xored later, a partial hash is performed
on local address + net_hash_mix(net)Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
06 Nov, 2009
2 commits
-
Conflicts:
drivers/net/usb/cdc_ether.cAll CDC ethernet devices of type USB_CLASS_COMM need to use
'&mbm_info'.Signed-off-by: David S. Miller
-
struct can_proto had a capability field which wasn't ever used. It is
dropped entirely.struct inet_protosw had a capability field which can be more clearly
expressed in the code by just checking if sock->type = SOCK_RAW.Signed-off-by: Eric Paris
Acked-by: Arnaldo Carvalho de Melo
Signed-off-by: David S. Miller
31 Oct, 2009
1 commit
-
On UDP sockets, we must call skb_free_datagram() with socket locked,
or risk sk_forward_alloc corruption. This requirement is not respected
in SUNRPC.Add a convenient helper, skb_free_datagram_locked() and use it in SUNRPC
Reported-by: Francis Moreau
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
19 Oct, 2009
2 commits
-
- skb_kill_datagram() can increment sk->sk_drops itself, not callers.
- UDP on IPV4 & IPV6 dropped frames (because of bad checksum or policy checks) increment sk_drops
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller -
In order to have better cache layouts of struct sock (separate zones
for rx/tx paths), we need this preliminary patch.Goal is to transfert fields used at lookup time in the first
read-mostly cache line (inside struct sock_common) and move sk_refcnt
to a separate cache line (only written by rx path)This patch adds inet_ prefix to daddr, rcv_saddr, dport, num, saddr,
sport and id fields. This allows a future patch to define these
fields as macros, like sk_refcnt, without name clashes.Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
15 Oct, 2009
1 commit
-
sock_queue_rcv_skb() can update sk_drops itself, removing need for
callers to take care of it. This is more consistent since
sock_queue_rcv_skb() also reads sk_drops when queueing a skb.This adds sk_drops managment to many protocols that not cared yet.
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
13 Oct, 2009
1 commit
-
Create a new socket level option to report number of queue overflows
Recently I augmented the AF_PACKET protocol to report the number of frames lost
on the socket receive queue between any two enqueued frames. This value was
exported via a SOL_PACKET level cmsg. AFter I completed that work it was
requested that this feature be generalized so that any datagram oriented socket
could make use of this option. As such I've created this patch, It creates a
new SOL_SOCKET level option called SO_RXQ_OVFL, which when enabled exports a
SOL_SOCKET level cmsg that reports the nubmer of times the sk_receive_queue
overflowed between any two given frames. It also augments the AF_PACKET
protocol to take advantage of this new feature (as it previously did not touch
sk->sk_drops, which this patch uses to record the overflow count). Tested
successfully by me.Notes:
1) Unlike my previous patch, this patch simply records the sk_drops value, which
is not a number of drops between packets, but rather a total number of drops.
Deltas must be computed in user space.2) While this patch currently works with datagram oriented protocols, it will
also be accepted by non-datagram oriented protocols. I'm not sure if thats
agreeable to everyone, but my argument in favor of doing so is that, for those
protocols which aren't applicable to this option, sk_drops will always be zero,
and reporting no drops on a receive queue that isn't used for those
non-participating protocols seems reasonable to me. This also saves us having
to code in a per-protocol opt in mechanism.3) This applies cleanly to net-next assuming that commit
977750076d98c7ff6cbda51858bb5a5894a9d9ab (my af packet cmsg patch) is revertedSigned-off-by: Neil Horman
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller
08 Oct, 2009
2 commits
-
UDP_HTABLE_SIZE was initialy defined to 128, which is a bit small for
several setups.4000 active UDP sockets -> 32 sockets per chain in average. An
incoming frame has to lookup all sockets to find best match, so long
chains hurt latency.Instead of a fixed size hash table that cant be perfect for every
needs, let UDP stack choose its table size at boot time like tcp/ip
route, using alloc_large_system_hash() helperAdd an optional boot parameter, uhash_entries=x so that an admin can
force a size between 256 and 65536 if needed, like thash_entries and
rhash_entries.dmesg logs two new lines :
[ 0.647039] UDP hash table entries: 512 (order: 0, 4096 bytes)
[ 0.647099] UDP Lite hash table entries: 512 (order: 0, 4096 bytes)Maximal size on 64bit arches would be 65536 slots, ie 1 MBytes for non
debugging spinlocks.Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller -
Might as well use the ipv6_addr_set_v4mapped() inline we created last
year.Signed-off-by: Brian Haley
Signed-off-by: David S. Miller
07 Oct, 2009
1 commit
-
Atis Elsts wrote:
> Not sure if there is need to fill the mark from skb in tunnel xmit functions. In any case, it's not done for GRE or IPIP tunnels at the moment.Ok, I'll just drop that part, I'm not sure what should be done in this case.
> Also, in this patch you are doing that for SIT (v6-in-v4) tunnels only, and not doing it for v4-in-v6 or v6-in-v6 tunnels. Any reason for that?
I just sent that patch out too quickly, here's a better one with the updates.
Add support for IPv6 route lookups using sk_mark.
Signed-off-by: Brian Haley
Signed-off-by: David S. Miller
01 Oct, 2009
1 commit
-
This provides safety against negative optlen at the type
level instead of depending upon (sometimes non-trivial)
checks against this sprinkled all over the the place, in
each and every implementation.Based upon work done by Arjan van de Ven and feedback
from Linus Torvalds.Signed-off-by: David S. Miller
15 Sep, 2009
1 commit
-
Signed-off-by: Alexey Dobriyan
Signed-off-by: David S. Miller
03 Sep, 2009
1 commit
-
Christoph Lameter pointed out that packet drops at qdisc level where not
accounted in SNMP counters. Only if application sets IP_RECVERR, drops
are reported to user (-ENOBUFS errors) and SNMP counters updated.IP_RECVERR is used to enable extended reliable error message passing,
but these are not needed to update system wide SNMP stats.This patch changes things a bit to allow SNMP counters to be updated,
regardless of IP_RECVERR being set or not on the socket.Example after an UDP tx flood
# netstat -s
...
IP:
1487048 outgoing packets dropped
...
Udp:
...
SndbufErrors: 1487048send() syscalls, do however still return an OK status, to not
break applications.Note : send() manual page explicitly says for -ENOBUFS error :
"The output queue for a network interface was full.
This generally indicates that the interface has stopped sending,
but may be caused by transient congestion.
(Normally, this does not occur in Linux. Packets are just silently
dropped when a device queue overflows.) "This is not true for IP_RECVERR enabled sockets : a send() syscall
that hit a qdisc drop returns an ENOBUFS error.Many thanks to Christoph, David, and last but not least, Alexey !
Signed-off-by: Eric Dumazet
Signed-off-by: David S. Miller