29 Sep, 2020
5 commits
-
SMC Version 2 defines a first contact extension for CLC accept
and CLC confirm. This patch covers sending and receiving of the
CLC first contact extension.Signed-off-by: Ursula Braun
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller -
SMCD Version 2 allows to propose up to 8 additional ISM devices
offered to the peer as candidates for SMCD communication.
This patch covers the server side, i.e. selection of an ISM device
matching one of the proposed ISM devices, that will be used for
CLC acceptSigned-off-by: Ursula Braun
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller -
SMCD Version 2 allows to propose up to 8 additional ISM devices
offered to the peer as candidates for SMCD communication.
This patch covers determination of the ISM devices to be proposed.
ISM devices without PNETID are preferred, since ISM devices with
PNETID are a V1 leftover and will disappear over the time.Signed-off-by: Ursula Braun
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller -
With SMCD version 2 the CHIDs of ISM devices are needed for the
CLC handshake.
This patch provides the new callback to retrieve the CHID of an
ISM device.Signed-off-by: Ursula Braun
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller -
SMCD Version 2 allows proposing of up to 8 ISM devices in addition
to the native ISM device of SMCD Version 1.
This patch prepares the struct smc_init_info to deal with these
additional 8 ISM devices.Signed-off-by: Ursula Braun
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller
11 Sep, 2020
3 commits
-
There are 6 types of workers which exist per smc connection. 3 of them
are used for listen and handshake processing, another 2 are used for
close and abort processing and 1 is the tx worker that moves calls to
sleeping functions into a worker.
To prevent flooding of the system work queue when many connections are
opened or closed at the same time (some pattern uperf implements), move
those workers to one of 3 smc-specific work queues. Two work queues are
module-global and used for handshake and close workers. The third work
queue is defined per link group and used by the tx workers that may
sleep waiting for resources of this link group.
And in smc_llc_enqueue() queue the llc_event_work work to the system
prio work queue because its critical that this work is started fast.Reviewed-by: Ursula Braun
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller -
smc_lgr_cleanup_early() schedules the free worker with delay. DMB
unregistering occurs in this delayed worker increasing the risk
to reach the SMCD SBA limit without need. Terminate the
linkgroup immediately, since termination means early DMB unregistering.For SMCD the global smc_server_lgr_pending lock is given up early.
A linkgroup to be given up with smc_lgr_cleanup_early() may already
contain more than one connection. Using __smc_lgr_terminate() in
smc_lgr_cleanup_early() covers this.And consolidate smc_ism_put_vlan() and smc_put_device() into smc_lgr_free()
only.Signed-off-by: Ursula Braun
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller -
Field names "srv_first_contact" and "cln_first_contact" are misleading,
since they apply to both, server and client. Rename them to
"first_contact_peer" and "first_contact_local".
Rename "ism_gid" by the more precise name "ism_peer_gid".
Rename version constant "SMC_CLC_V1" into "SMC_V1".
No functional change.Signed-off-by: Ursula Braun
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller
20 Jul, 2020
1 commit
-
The dma related ...sync_sg... functions check the link state before the
dma function is actually called. But the check in smc_link_usable()
allows links in ACTIVATING state which are not yet mapped to dma memory.
Under high load it may happen that the sync_sg functions are called for
such a link which results in an debug output like
DMA-API: mlx5_core 0002:00:00.0: device driver tries to sync
DMA memory it has not allocated [device address=0x0000000103370000]
[size=65536 bytes]
To fix that introduce a helper to check for the link state ACTIVE and
use it where appropriate. And move the link state update to ACTIVATING
to the end of smcr_link_init() when most initial setup is done.Reviewed-by: Ursula Braun
Fixes: d854fcbfaeda ("net/smc: add new link state and related helpers")
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller
09 Jul, 2020
1 commit
-
There might be races in scenarios where both SMC link groups are on the
same system. Prevent that by creating separate wait queues for LLC flows
and messages. Switch to non-interruptable versions of wait_event() and
wake_up() for the llc flow waiter to make sure the waiters get control
sequentially. Fine tune the llc_flow_lock to include the assignment of
the message. Write to system log when an unexpected message was
dropped. And remove an extra indirection and use the existing local
variable lgr in smc_llc_enqueue().Fixes: 555da9af827d ("net/smc: add event-based llc_flow framework")
Reviewed-by: Ursula Braun
Signed-off-by: Karsten Graul
Signed-off-by: David S. Miller
06 May, 2020
1 commit
-
Print to system log when SMC links are available or go down, link group
state changes or pnetids are applied to and removed from devices.
The log entries are triggered by either user configuration actions or
adapter activation/deactivation events and are not expected to happen
often. The entries help SMC users to keep track of the SMC link group
status and to detect when actions are needed (like to add replacements
for failed adapters).Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller
05 May, 2020
6 commits
-
During SMC-R link establishment the peers exchange the link_uid that
is used for debugging purposes. Save the peer link_uid in smc_link so it
can be retrieved by the smc_diag netlink interface.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
The link_uid of an SMC-R link is exchanged between SMC peers and its
value can be used for debugging purposes. Create a unique link_uid
during link initialization and use it in communication with SMC-R peers.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Allow to set the reason code for the link group termination, and set
meaningful values before termination processing is triggered. This
reason code is sent to the peer in the final delete link message.
When the LLC request or response layer receives a message type that was
not handled, drop a warning and terminate the link group.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
New connections must not be assigned to asymmetric links. Add asymmetric
link tagging using new link variable link_is_asym. The new helpers
smcr_lgr_set_type() and smcr_lgr_set_type_asym() are called to set the
state of the link group, and tag all links accordingly.
smcr_lgr_conn_assign_link() respects the link tagging and will not
assign new connections to links tagged as asymmetric link.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Introduce smc_wr_tx_send_wait() to send an IB message and wait for the
tx completion event of the message. This makes sure that the message is
no longer in-flight when the function returns.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Add smc_switch_conns() to switch all connections from a link that is
going down. Find an other link to switch the connections to, and
switch each connection to the new link. smc_switch_cursor() updates the
cursors of a connection to the state of the last successfully sent CDC
message. When there is no link to switch to, terminate the link group.
Call smc_switch_conns() when a link is going down.
And with the possibility that links of connections can switch adapt CDC
and TX functions to detect and handle link switches.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller
04 May, 2020
2 commits
-
Introduce a work that is scheduled when a new DELETE_LINK LLC request is
received. The work will call either the SMC client or SMC server
DELETE_LINK processing.
And use the LLC flow framework to process incoming DELETE_LINK LLC
messages, scheduling the llc_del_link_work for those events.
With these changes smc_lgr_forget() is only called by one function and
can be migrated into smc_lgr_cleanup_early().Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
First set of functions to process an ADD_LINK LLC request as an SMC
client. Find an alternate IB device, determine the new link group type
and get the index for the new link. Then ready the link, map the buffers
and send an ADD_LINK LLC response. If any error occurs, send a reject
LLC message and terminate the processing.
Add smc_llc_alloc_alt_link() to find a free link index for a new link,
depending on the new link group type.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller
02 May, 2020
9 commits
-
Introduce a work that is scheduled when a new ADD_LINK LLC request is
received. The work will call either the SMC client or SMC server
ADD_LINK processing.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Add smc_llc_alloc_alt_link() to find a free link index for a new link,
depending on the new link group type. And update constants for the
maximum number of links to 3 (2 symmetric and 1 dangling asymmetric link).
These maximum numbers are the same as used by other implementations of the
SMC-R protocol.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Call smcr_port_err() when an IB event reports an inactive IB device.
smcr_port_err() calls smcr_link_down() for all affected links.
smcr_link_down() either triggers the local DELETE_LINK processing, or
sends an DELETE_LINK LLC message to the SMC server to initiate the
processing.
The old handler function smc_port_terminate() is removed.
Add helper smcr_link_down_cond() to take a link down conditionally, and
smcr_link_down_cond_sched() to schedule the link_down processing to a
work.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Call smcr_port_add() when an IB event reports a new active IB device.
smcr_port_add() will start a work which either triggers the local
ADD_LINK processing, or send an ADD_LINK LLC message to the SMC server
to initiate the processing.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
The PNETID is needed to find an alternate link for a link group.
Save the PNETID of the link that is used to create the link group for
later device matching.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Introduce llc_conf_mutex in the link group which is used to protect the
buffers and lgr states against parallel link reconfiguration.
This ensures that new connections do not start to register buffers with
the links of a link group when link creation or termination is running.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Introduce support to map and register all current buffers for a new
link. smcr_buf_map_lgr() will map used buffers for a new link and
smcr_buf_reg_lgr() can be called to register used buffers on the
IB device of the new link.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
With the support of multiple links that are created and cleared there
is a need to unmap one link from all current buffers. Add unmapping by
link and by rmb. And make smcr_link_clear() available to be called from
the LLC layer.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
The CONFIRM_RKEY LLC processing handles all links in one LLC message.
Move the call to this processing out of smcr_link_reg_rmb() which does
processing per link, into smcr_lgr_reg_rmbs() which is responsible for
link group level processing. Move smcr_link_reg_rmb() into module
smc_core.c.
>From af_smc.c now call smcr_lgr_reg_rmbs() to register new rmbs on all
available links.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller
01 May, 2020
8 commits
-
The connection layer in af_smc.c is now using the new LLC flow
framework, which made the link state DELETING obsolete. Remove the state
and the respective helpers.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Introduce smc_rtoken_set() to set the rtoken for a new link to an
existing rmb whose rtoken is given, and smc_rtoken_set2() to set an
rtoken for a new link whose link_id is given.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Adapt smc_llc_do_delete_rkey() to use the LLC flow and support multiple
links when deleting the rkeys for rmb buffers at the peer.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Adapt smc_llc_do_confirm_rkey() to use the LLC flow and support the
rkeys of multiple links when the CONFIRM_RKEY LLC message is build.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Change the code that processes the SMC client part of connection
establishment to use the LLC flow framework (CONFIRM_LINK request
messages).Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Change the code that processes the SMC server part of connection
establishment to use the LLC flow framework (CONFIRM_LINK response
messages).Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Add a type field to the link group which reflects the current link group
redundancy state.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
The new framework allows to start specific types of LLC control flows,
protects active flows and makes it possible to wait for flows to finish
before starting a new flow.
This mechanism is used for the LLC control layer to model flows like
'add link' or 'delete link' which need to send/receive several LLC
messages and are not allowed to get interrupted by the wrong type of
messages.
'Add link' or 'Delete link' messages arriving in the middle of a flow
are delayed and processed when the current flow finished.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller
30 Apr, 2020
4 commits
-
Introduce smc_llc_lgr_init() and smc_llc_lgr_clear() to implement all
llc layer specific initialization and cleanup in module smc_llc.c.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
The locks for sndbufs and rmbs are never used from atomic context. Using
a mutex for these locks will allow to nest locks with other mutexes.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
When llc responses are received then possible waiters for this response
are to be notified. This can be done in tasklet context, without to
use a work in the llc work queue. Move all code that handles llc
responses into smc_llc_rx_response().Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller -
Incoming llc messages are processed in irq tasklet context, and
a worker is used to send outgoing messages. The worker is needed
because getting a send buffer could result in a wait for a free buffer.To make sure all incoming llc messages are processed in a serialized way
introduce an event queue and create a new queue entry for each message
which is queued to this event queue. A new worker processes the event
queue entries in order.
And remove the use of a separate worker to send outgoing llc messages
because the messages are processed in worker context already.
With this event queue the serialized llc_wq work queue is obsolete,
remove it.Signed-off-by: Karsten Graul
Reviewed-by: Ursula Braun
Signed-off-by: David S. Miller