09 Dec, 2015
1 commit
-
Receipt of CM MAD with other than the Send method for an attribute
other than the ClassPortInfo attribute is invalid.CM attributes other than ClassPortInfo only use the send method.
The SRP initiator does not maintain a timeout policy for CM connect
requests relies on the CM layer to do that. The result was that
the SRP initiator hung as the connect request never completed.A new SRP target has been observed to respond to Send CM REQ
with GetResp of CM REQ with bad status. This is non conformant
with IBA spec but exposes a vulnerability in the current MAD/CM
code which will respond to the incoming GetResp of CM REQ as if
it was a valid incoming Send of CM REQ rather than tossing
this on the floor. It also causes the MAD layer not to
retransmit the original REQ even though it has not received a REP.Reviewed-by: Sagi Grimberg
Signed-off-by: Hal Rosenstock
Reviewed-by: Ira Weiny
Signed-off-by: Doug Ledford
08 Dec, 2015
1 commit
-
The current implementation gets a spin_lock, and at any scale with
qib and hfi1 post send, the lock contention grows exponentially
with the number of QPs.idr_find() is RCU compatibile, so read doesn't need the lock.
Change to use rcu_read_lock() and rcu_read_unlock() in
__idr_get_uobj().kfree_rcu() is used to insure a grace period between the
idr removal and actual free.Reviewed-by: Ira Weiny
Signed-off-by: Mike Marciniszyn
Reviewed-By: Jason Gunthorpe
Signed-off-by: Doug Ledford
31 Oct, 2015
1 commit
-
Move the __attribute_const__ declarations such that sparse understands
that these apply to the function itself and not to the return type.
This avoids that sparse reports error messages like the following:drivers/infiniband/core/verbs.c:73:12: error: symbol 'ib_event_msg' redeclared with different type (originally declared at include/rdma/ib_verbs.h:470) - different modifiers
Fixes: 2b1b5b601230 ("IB/core, cma: Nice log-friendly string helpers")
Signed-off-by: Bart Van Assche
Cc: Sagi Grimberg
Reviewed-by: Sagi Grimberg
Signed-off-by: Doug Ledford
29 Oct, 2015
5 commits
-
No callers and no providers left, go ahead and remove it.
Signed-off-by: Sagi Grimberg
Acked-by: Christoph Hellwig
Signed-off-by: Doug Ledford -
The new fast registration verb ib_map_mr_sg receives a scatterlist
and converts it to a page list under the verbs API thus hiding
the specific HW mapping details away from the consumer.The provider drivers are provided with a generic helper ib_sg_to_pages
that converts a scatterlist into a vector of page addresses. The
drivers can still perform any HW specific page address setting
by passing a set_page function pointer which will be invoked for
each page address. This allows drivers to avoid keeping a shadow
page vectors and convert them to HW specific translations by doing
extra copies.This API will allow ULPs to remove the duplicated code of constructing
a page vector from a given sg list.The send work request ib_reg_wr also shrinks as it will contain only
mr, key and access flags in addition.Signed-off-by: Sagi Grimberg
Tested-by: Christoph Hellwig
Signed-off-by: Doug Ledford -
Add support for network namespaces in the ib_cma module. This is
accomplished by:1. Adding network namespace parameter for rdma_create_id. This parameter is
used to populate the network namespace field in rdma_id_private.
rdma_create_id keeps a reference on the network namespace.
2. Using the network namespace from the rdma_id instead of init_net inside
of ib_cma, when listening on an ID and when looking for an ID for an
incoming request.
3. Decrementing the reference count for the appropriate network namespace
when calling rdma_destroy_id.In order to preserve the current behavior init_net is passed when calling
from other modules.Signed-off-by: Guy Shapiro
Signed-off-by: Haggai Eran
Signed-off-by: Yotam Kenneth
Signed-off-by: Shachar Raindel
Signed-off-by: Doug Ledford -
Add network namespace support to the ib_addr module. For that, all the
address resolution and matching should be done using the appropriate
namespace instead of init_net.This is achieved by:
1. Adding an explicit network namespace argument to exported function that
require a namespace.
2. Saving the namespace in the rdma_addr_client structure.
3. Using it when calling networking functions.In order to preserve the behavior of calling modules, &init_net is
passed as the parameter in calls from other modules. This is modified as
namespace support is added on more levels.Signed-off-by: Haggai Eran
Signed-off-by: Yotam Kenneth
Signed-off-by: Shachar Raindel
Signed-off-by: Guy Shapiro
Signed-off-by: Doug Ledford
22 Oct, 2015
8 commits
-
The GID cache accompanies every GID with attributes.
The GID attributes link the GID with its netdevice, which could be
resolved to smac and vlan id easily. Since we've added the netdevice
(ifindex and net) to the path record, storing the L2 attributes is
duplicated data and hence these attributes are removed.Signed-off-by: Matan Barak
Reviewed-By: Devesh Sharma
Signed-off-by: Doug Ledford -
Smac and vlan id could be resolved from the GID attribute, and thus
these attributes aren't needed anymore. Removing them.Signed-off-by: Matan Barak
Reviewed-By: Devesh Sharma
Signed-off-by: Doug Ledford -
Previously, vlan id and source MAC were used from QP attributes. Since
the net device is now stored in the GID attributes, they could be used
instead of getting this information from the QP attributes.IB_QP_SMAC, IB_QP_ALT_SMAC, IB_QP_VID and IB_QP_ALT_VID were removed
because there is no known libibverbs that uses them.This commit also modifies the vendors (mlx4, ocrdma) drivers in order
to use the new approach.ocrdma driver changes were done by Somnath Kotur
Signed-off-by: Matan Barak
Signed-off-by: Doug Ledford -
GID cache API users might want to search for GIDs with specific
attributes rather than just specifying GID, net device and port.
This is used in a later patch, where we find the sgid index by
L2 Ethernet attributes.Signed-off-by: Matan Barak
Reviewed-By: Devesh Sharma
Signed-off-by: Doug Ledford -
In order to find the sgid_index, one could just query the IB cache
with the correct GID and netdevice. Therefore, instead of storing
the L2 attributes directly in the path, we only store the
ifindex and net and use them later to get the sgid_index.
The vlan_id and smac L2 attributes are removed in a later patch.Signed-off-by: Matan Barak
Reviewed-By: Devesh Sharma
Signed-off-by: Doug Ledford -
Sometime consumers might want to search for a GID in a specific port.
For example, when a WC arrives and we want to search the GID
that matches that port - it's better to search only the relevant
port.
Exposing and renaming ib_cache_gid_find_by_port in order to match
the naming convention of the module.Signed-off-by: Matan Barak
Signed-off-by: Doug Ledford -
Adding an ability to query the IB cache by a netdev and get the
attributes of a GID. These parameters are necessary in order to
successfully resolve the required GID (when the netdevice is known)
and get the Ethernet L2 attributes from a GID.Signed-off-by: Matan Barak
Reviewed-By: Devesh Sharma
Signed-off-by: Doug Ledford -
IBA spec is now 1.3 not 3.1 and vol 1 should be mentioned
as there is also vol 2.Signed-off-by: Hal Rosenstock
Signed-off-by: Doug Ledford
08 Oct, 2015
2 commits
-
The field is only initialized in mlx, but never used.
If we want to add proper XRC support it should be done with a new
struct ib_xrc_wr.This shrinks the various WR structures by another 4 bytes.
Signed-off-by: Christoph Hellwig
Reviewed-by: Sagi Grimberg
Reviewed-by: Jason Gunthorpe
Tested-by: Haggai Eran -
This patch split up struct ib_send_wr so that all non-trivial verbs
use their own structure which embedds struct ib_send_wr. This dramaticly
shrinks the size of a WR for most common operations:sizeof(struct ib_send_wr) (old): 96
sizeof(struct ib_send_wr): 48
sizeof(struct ib_rdma_wr): 64
sizeof(struct ib_atomic_wr): 96
sizeof(struct ib_ud_wr): 88
sizeof(struct ib_fast_reg_wr): 88
sizeof(struct ib_bind_mw_wr): 96
sizeof(struct ib_sig_handover_wr): 80And with Sagi's pending MR rework the fast registration WR will also be
down to a reasonable size:sizeof(struct ib_fastreg_wr): 64
Signed-off-by: Christoph Hellwig
Reviewed-by: Bart Van Assche [srp, srpt]
Reviewed-by: Chuck Lever [sunrpc]
Tested-by: Haggai Eran
Tested-by: Sagi Grimberg
Tested-by: Steve Wise
29 Sep, 2015
1 commit
-
Two enum members IB_DEVICE_RC_IP_CSUM and IB_DEVICE_RAW_IP_CSUM are
added to ib_device_cap_flags. Device should set these two flags if they support
insertion of UDP and TCP checksum on outgoing IPv4 messages and can verify the
validity of checksum for incoming IPv4 messages, for RC IPoIB and RAW over
Ethernet respectively. They are similar to IB_DEVICE_UD_IP_CSUM.Signed-off-by: Bodong Wang
Signed-off-by: Doug Ledford
20 Sep, 2015
1 commit
-
Pull rdma fixes from Doug Ledford:
"The new hfi1 driver in staging/rdma has had a number of fixup patches
since being added to the tree. This is the first batch of those fixes"* tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dledford/rdma:
IB/hfi: Properly set permissions for user device files
IB/hfi1: mask vs shift confusion
IB/hfi1: clean up some defines
IB/hfi1: info leak in get_ctxt_info()
IB/hfi1: fix a locking bug
IB/hfi1: checking for NULL instead of IS_ERR
IB/hfi1: fix sdma_descq_cnt parameter parsing
IB/hfi1: fix copy_to/from_user() error handling
IB/hfi1: fix pstateinfo from returning improperly byteswapped value
18 Sep, 2015
1 commit
-
Byteswap link_width_downgrade_*_active values before sending on the wire. In
addition properly define the Port State Info structure.Reviewed-by: Dennis Dalessandro
Reviewed-by: Christian Gomez
Signed-off-by: Rimmer, Todd
Signed-off-by: Ira Weiny
Acked-by: Mike Marciniszyn
Signed-off-by: Doug Ledford
09 Sep, 2015
1 commit
-
Pull inifiniband/rdma updates from Doug Ledford:
"This is a fairly sizeable set of changes. I've put them through a
decent amount of testing prior to sending the pull request due to
that.There are still a few fixups that I know are coming, but I wanted to
go ahead and get the big, sizable chunk into your hands sooner rather
than waiting for those last few fixups.Of note is the fact that this creates what is intended to be a
temporary area in the drivers/staging tree specifically for some
cleanups and additions that are coming for the RDMA stack. We
deprecated two drivers (ipath and amso1100) and are waiting to hear
back if we can deprecate another one (ehca). We also put Intel's new
hfi1 driver into this area because it needs to be refactored and a
transfer library created out of the factored out code, and then it and
the qib driver and the soft-roce driver should all be modified to use
that library.I expect drivers/staging/rdma to be around for three or four kernel
releases and then to go away as all of the work is completed and final
deletions of deprecated drivers are done.Summary of changes for 4.3:
- Create drivers/staging/rdma
- Move amso1100 driver to staging/rdma and schedule for deletion
- Move ipath driver to staging/rdma and schedule for deletion
- Add hfi1 driver to staging/rdma and set TODO for move to regular
tree
- Initial support for namespaces to be used on RDMA devices
- Add RoCE GID table handling to the RDMA core caching code
- Infrastructure to support handling of devices with differing read
and write scatter gather capabilities
- Various iSER updates
- Kill off unsafe usage of global mr registrations
- Update SRP driver
- Misc mlx4 driver updates
- Support for the mr_alloc verb
- Support for a netlink interface between kernel and user space cache
daemon to speed path record queries and route resolution
- Ininitial support for safe hot removal of verbs devices"* tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dledford/rdma: (136 commits)
IB/ipoib: Suppress warning for send only join failures
IB/ipoib: Clean up send-only multicast joins
IB/srp: Fix possible protection fault
IB/core: Move SM class defines from ib_mad.h to ib_smi.h
IB/core: Remove unnecessary defines from ib_mad.h
IB/hfi1: Add PSM2 user space header to header_install
IB/hfi1: Add CSRs for CONFIG_SDMA_VERBOSITY
mlx5: Fix incorrect wc pkey_index assignment for GSI messages
IB/mlx5: avoid destroying a NULL mr in reg_user_mr error flow
IB/uverbs: reject invalid or unknown opcodes
IB/cxgb4: Fix if statement in pick_local_ip6adddrs
IB/sa: Fix rdma netlink message flags
IB/ucma: HW Device hot-removal support
IB/mlx4_ib: Disassociate support
IB/uverbs: Enable device removal when there are active user space applications
IB/uverbs: Explicitly pass ib_dev to uverbs commands
IB/uverbs: Fix race between ib_uverbs_open and remove_one
IB/uverbs: Fix reference counting usage of event files
IB/core: Make ib_dealloc_pd return void
IB/srp: Create an insecure all physical rkey only if needed
...
04 Sep, 2015
2 commits
-
When the hfi1 driver was added these definitions were moved from the qib driver
to ib_mad.h to be used by both qib and hfi1. They should have been moved to
ib_smi.h instead.Fixes: d4ab347005fb ("IB/core: Add core header changes needed for OPA")
Reviewed-by: Hal Rosenstock
Signed-off-by: Ira Weiny
Signed-off-by: Doug Ledford -
Remove the unused IB_NOTICE_REPRESS_* defines.
When the hfi1 driver was added these definitions were moved from the qib driver
to ib_mad.h. They should have been removed instead.Fixes: d4ab347005fb ("IB/core: Add core header changes needed for OPA")
Signed-off-by: Ira Weiny
Reviewed-by: Hal Rosenstock
Signed-off-by: Doug Ledford
31 Aug, 2015
16 commits
-
Enables the uverbs_remove_one to succeed despite the fact that there are
running IB applications working with the given ib device. This
functionality enables a HW device to be unbind/reset despite the fact that
there are running user space applications using it.It exposes a new IB kernel API named 'disassociate_ucontext' which lets
a driver detaching its HW resources from a given user context without
crashing/terminating the application. In case a driver implemented the
above API and registered with ib_uverb there will be no dependency between its
device to its uverbs_device. Upon calling remove_one of ib_uverbs the call
should return after disassociating the open HW resources without waiting to
clients disconnecting. In case driver didn't implement this API there will be no
change to current behaviour and uverbs_remove_one will return only when last
client has disconnected and reference count on uverbs device became 0.In case the lower driver device was removed any application will
continue working over some zombie HCA, further calls will ended with an
immediate error.Signed-off-by: Yishai Hadas
Signed-off-by: Shachar Raindel
Reviewed-by: Jason Gunthorpe
Signed-off-by: Doug Ledford -
The majority of callers never check the return value, and even if they
did, they can't do anything about a failure.All possible failure cases represent a bug in the caller, so just
WARN_ON inside the function instead.This fixes a few random errors:
net/rd/iw.c infinite loops while it fails. (racing with EBUSY?)This also lays the ground work to get rid of error return from the
drivers. Most drivers do not error, the few that do are broken since
it cannot be handled.Since uverbs can legitimately make use of EBUSY, open code the
check.Signed-off-by: Jason Gunthorpe
Reviewed-by: Chuck Lever
Signed-off-by: Doug Ledford -
The pd now has a local_dma_lkey member which completely replaces
ib_get_dma_mr, use it instead.Signed-off-by: Jason Gunthorpe
Signed-off-by: Doug Ledford -
Every single ULP requires a local_dma_lkey to do anything with
a QP, so let us ensure one exists for every PD created.If the driver can supply a global local_dma_lkey then use that, otherwise
ask the driver to create a local use all physical memory MR associated
with the new PD.Signed-off-by: Jason Gunthorpe
Reviewed-by: Sagi Grimberg
Acked-by: Christoph Hellwig
Reviewed-by: Steve Wise
Reviewed-by: Ira Weiny
Tested-by: Ira Weiny
Signed-off-by: Doug Ledford -
This patch adds a function to check if listeners for a netlink multicast
group are present. It also adds a function to receive netlink response
messages.Signed-off-by: Kaike Wan
Signed-off-by: John Fleck
Signed-off-by: Ira Weiny
Signed-off-by: Doug Ledford -
get_netdev: get the net_device on the physical port of the IB transport port. In
port aggregation mode it is required to return the netdev of the active port.modify_gid: note for a change in the RoCE gid cache. Handle this by writing to
the harsware GID table. It is possible that indexes in cahce and hardware tables
won't match so a translation is required when modifying a QP or creating an
address handle.Signed-off-by: Moni Shoua
Signed-off-by: Doug Ledford -
RoCE GIDs are based on IP addresses configured on Ethernet net-devices
which relate to the RDMA (RoCE) device port.Currently, each of the low-level drivers that support RoCE (ocrdma,
mlx4) manages its own RoCE port GID table. As there's nothing which is
essentially vendor specific, we generalize that, and enhance the RDMA
core GID cache to do this job.In order to populate the GID table, we listen for events:
(a) netdev up/down/change_addr events - if a netdev is built onto
our RoCE device, we need to add/delete its IPs. This involves
adding all GIDs related to this ndev, add default GIDs, etc.(b) inet events - add new GIDs (according to the IP addresses)
to the table.For programming the port RoCE GID table, providers must implement
the add_gid and del_gid callbacks.RoCE GID management requires us to state the associated net_device
alongside the GID. This information is necessary in order to manage
the GID table. For example, when a net_device is removed, its
associated GIDs need to be removed as well.RoCE mandates generating a default GID for each port, based on the
related net-device's IPv6 link local. In contrast to the GID based on
the regular IPv6 link-local (as we generate GID per IP address),
the default GID is also available when the net device is down (in
order to support loopback).Locking is done as follows:
The patch modify the GID table code both for new RoCE drivers
implementing the add_gid/del_gid callbacks and for current RoCE and
IB drivers that do not. The flows for updating the table are
different, so the locking requirements are too.While updating RoCE GID table, protection against multiple writers is
achieved via mutex_lock(&table->lock). Since writing to a table
requires us to find an entry (possible a free entry) in the table and
then modify it, this mutex protects both the find_gid and write_gid
ensuring the atomicity of the action.
Each entry in the GID cache is protected by rwlock. In RoCE, writing
(usually results from netdev notifier) involves invoking the vendor's
add_gid and del_gid callbacks, which could sleep.
Therefore, an invalid flag is added for each entry. Updates for RoCE are
done via a workqueue, thus sleeping is permitted.In IB, updates are done in write_lock_irq(&device->cache.lock), thus
write_gid isn't allowed to sleep and add_gid/del_gid are not called.When passing net-device into/out-of the GID cache, the device
is always passed held (dev_hold).The code uses a single work item for updating all RDMA devices,
following a netdev or inet notifier.The patch moves the cache from being a client (which was incorrect,
as the cache is part of the IB infrastructure) to being explicitly
initialized/freed when a device is registered/removed.Signed-off-by: Matan Barak
Signed-off-by: Doug Ledford -
Fully replaced by a more generic and suitable
ib_alloc_mr.Signed-off-by: Sagi Grimberg
Signed-off-by: Doug Ledford -
Use ib_alloc_mr with specific parameters.
Change the existing callers.Signed-off-by: Sagi Grimberg
Signed-off-by: Doug Ledford -
This was added in a thought of uniting all mr allocation
and deallocation routines but the fact is we have a single
deallocation routine already, ib_dereg_mr.And, move mlx5_ib_destroy_mr specific logic into mlx5_ib_dereg_mr
(includes only signature stuff for now).And, fixup the only callers (iser/isert) accordingly.
Signed-off-by: Sagi Grimberg
Signed-off-by: Doug Ledford -
Now that there are no ib_cm clients using the compare_data feature for
matching IB CM requests' private data, remove the compare_data parameter of
ib_cm_listen and remove the code implementing the feature.Signed-off-by: Haggai Eran
Signed-off-by: Doug Ledford -
The rdma_cm module will later use the P_Key from the BTH to de-mux
requests.See discussion at:
http://www.spinics.net/lists/netdev/msg336067.htmlCc: Jason Gunthorpe
Cc: Liran Liss
Signed-off-by: Haggai Eran
Signed-off-by: Doug Ledford -
Enabling network namespaces for RDMA CM will allow processes on different
namespaces to listen on the same port. In order to leave namespace support
out of the CM layer, this requires that multiple RDMA CM IDs will be able
to share a single CM ID.This patch adds infrastructure to retrieve an existing listening ib_cm_id,
based on its device and service ID, or create a new one if one does not
already exist. It also adds a reference count for such instances
(cm_id_private.listen_sharecount), and prevents cm_destroy_id from
destroying a CM if it is still shared. See the relevant discussion [1].[1] Re: [PATCH v3 for-next 05/13] IB/cm: Reference count ib_cm_ids
http://www.spinics.net/lists/netdev/msg328860.htmlReviewed-by: Jason Gunthorpe
Signed-off-by: Haggai Eran
Signed-off-by: Doug Ledford -
Expose the service ID on an incoming CM or SIDR request to the event
handler. This will allow the RDMA CM module to de-multiplex connection
requests based on the information encoded in the service ID.Acked-by: Sean Hefty
Signed-off-by: Haggai Eran
Signed-off-by: Doug Ledford -
In the case of IPoIB, and maybe in other cases, the network device is
managed by an upper-layer protocol (ULP). In order to expose this
network device to other users of the IB device, let ULPs implement
a callback that returns network device according to connection parameters.The IB device and port, together with the P_Key and the GID should
be enough to uniquely identify the ULP net device. However, in current
kernels there can be multiple IPoIB interfaces created with the same GID.
Furthermore, such configuration may be desireable to support ipvlan-like
configurations for RDMA CM with IPoIB. To resolve the device in these
cases the code will also take the IP address as an additional input.Reviewed-by: Jason Gunthorpe
Signed-off-by: Haggai Eran
Signed-off-by: Yotam Kenneth
Signed-off-by: Shachar Raindel
Signed-off-by: Guy Shapiro
Signed-off-by: Doug Ledford -
An ib_client callback that is called with the lists_rwsem locked only for
read is protected from changes to the IB client lists, but not from
ib_unregister_device() freeing its client data. This is because
ib_unregister_device() will remove the device from the device list with
lists_rwsem locked for write, but perform the rest of the cleanup,
including the call to remove() without that lock.Mark client data that is undergoing de-registration with a new going_down
flag in the client data context. Lock the client data list with lists_rwsem
for write in addition to using the spinlock, so that functions calling the
callback would be able to lock only lists_rwsem for read and let callbacks
sleep.Since ib_unregister_client() now marks the client data context, no need for
remove() to search the context again, so pass the client data directly to
remove() callbacks.Reviewed-by: Jason Gunthorpe
Signed-off-by: Haggai Eran
Signed-off-by: Doug Ledford
