Blame view
Documentation/cgroups/freezer-subsystem.txt
4.79 KB
3b1b3f6e5 freezer_cg: disab... |
1 |
The cgroup freezer is useful to batch job management system which start |
bde5ab655 container freezer... |
2 3 4 5 6 7 |
and stop sets of tasks in order to schedule the resources of a machine according to the desires of a system administrator. This sort of program is often used on HPC clusters to schedule access to the cluster as a whole. The cgroup freezer uses cgroups to describe the set of tasks to be started/stopped by the batch job management system. It also provides a means to start and stop the tasks composing the job. |
3b1b3f6e5 freezer_cg: disab... |
8 |
The cgroup freezer will also be useful for checkpointing running groups |
bde5ab655 container freezer... |
9 10 11 12 13 14 15 16 |
of tasks. The freezer allows the checkpoint code to obtain a consistent image of the tasks by attempting to force the tasks in a cgroup into a quiescent state. Once the tasks are quiescent another task can walk /proc or invoke a kernel interface to gather information about the quiesced tasks. Checkpointed tasks can be restarted later should a recoverable error occur. This also allows the checkpointed tasks to be migrated between nodes in a cluster by copying the gathered information to another node and restarting the tasks there. |
3b1b3f6e5 freezer_cg: disab... |
17 |
Sequences of SIGSTOP and SIGCONT are not always sufficient for stopping |
bde5ab655 container freezer... |
18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
and resuming tasks in userspace. Both of these signals are observable from within the tasks we wish to freeze. While SIGSTOP cannot be caught, blocked, or ignored it can be seen by waiting or ptracing parent tasks. SIGCONT is especially unsuitable since it can be caught by the task. Any programs designed to watch for SIGSTOP and SIGCONT could be broken by attempting to use SIGSTOP and SIGCONT to stop and resume tasks. We can demonstrate this problem using nested bash shells: $ echo $$ 16644 $ bash $ echo $$ 16690 From a second, unrelated bash shell: $ kill -SIGSTOP 16690 |
5f1116167 Documentation: Fi... |
34 |
$ kill -SIGCONT 16690 |
bde5ab655 container freezer... |
35 |
|
5f1116167 Documentation: Fi... |
36 |
<at this point 16690 exits and causes 16644 to exit too> |
bde5ab655 container freezer... |
37 |
|
3b1b3f6e5 freezer_cg: disab... |
38 |
This happens because bash can observe both signals and choose how it |
bde5ab655 container freezer... |
39 |
responds to them. |
3b1b3f6e5 freezer_cg: disab... |
40 |
Another example of a program which catches and responds to these |
bde5ab655 container freezer... |
41 42 |
signals is gdb. In fact any program designed to use ptrace is likely to have a problem with this method of stopping and resuming tasks. |
3b1b3f6e5 freezer_cg: disab... |
43 |
In contrast, the cgroup freezer uses the kernel freezer code to |
bde5ab655 container freezer... |
44 45 46 |
prevent the freeze/unfreeze cycle from becoming visible to the tasks being frozen. This allows the bash example above and gdb to run as expected. |
ef9fe980c cgroup_freezer: i... |
47 48 49 50 51 |
The cgroup freezer is hierarchical. Freezing a cgroup freezes all tasks beloning to the cgroup and all its descendant cgroups. Each cgroup has its own state (self-state) and the state inherited from the parent (parent-state). Iff both states are THAWED, the cgroup is THAWED. |
bde5ab655 container freezer... |
52 |
|
ef9fe980c cgroup_freezer: i... |
53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
The following cgroupfs files are created by cgroup freezer. * freezer.state: Read-write. When read, returns the effective state of the cgroup - "THAWED", "FREEZING" or "FROZEN". This is the combined self and parent-states. If any is freezing, the cgroup is freezing (FREEZING or FROZEN). FREEZING cgroup transitions into FROZEN state when all tasks belonging to the cgroup and its descendants become frozen. Note that a cgroup reverts to FREEZING from FROZEN after a new task is added to the cgroup or one of its descendant cgroups until the new task is frozen. When written, sets the self-state of the cgroup. Two values are allowed - "FROZEN" and "THAWED". If FROZEN is written, the cgroup, if not already freezing, enters FREEZING state along with all its descendant cgroups. If THAWED is written, the self-state of the cgroup is changed to THAWED. Note that the effective state may not change to THAWED if the parent-state is still freezing. If a cgroup's effective state becomes THAWED, all its descendants which are freezing because of the cgroup also leave the freezing state. * freezer.self_freezing: Read only. Shows the self-state. 0 if the self-state is THAWED; otherwise, 1. This value is 1 iff the last write to freezer.state was "FROZEN". * freezer.parent_freezing: Read only. Shows the parent-state. 0 if none of the cgroup's ancestors is frozen; otherwise, 1. The root cgroup is non-freezable and the above interface files don't exist. |
3b1b3f6e5 freezer_cg: disab... |
90 |
|
bde5ab655 container freezer... |
91 |
* Examples of usage : |
f6e07d380 Documentation: up... |
92 93 94 95 |
# mkdir /sys/fs/cgroup/freezer # mount -t cgroup -ofreezer freezer /sys/fs/cgroup/freezer # mkdir /sys/fs/cgroup/freezer/0 # echo $some_pid > /sys/fs/cgroup/freezer/0/tasks |
bde5ab655 container freezer... |
96 97 |
to get status of the freezer subsystem : |
f6e07d380 Documentation: up... |
98 |
# cat /sys/fs/cgroup/freezer/0/freezer.state |
bde5ab655 container freezer... |
99 100 101 |
THAWED to freeze all tasks in the container : |
f6e07d380 Documentation: up... |
102 103 |
# echo FROZEN > /sys/fs/cgroup/freezer/0/freezer.state # cat /sys/fs/cgroup/freezer/0/freezer.state |
bde5ab655 container freezer... |
104 |
FREEZING |
f6e07d380 Documentation: up... |
105 |
# cat /sys/fs/cgroup/freezer/0/freezer.state |
bde5ab655 container freezer... |
106 107 108 |
FROZEN to unfreeze all tasks in the container : |
f6e07d380 Documentation: up... |
109 110 |
# echo THAWED > /sys/fs/cgroup/freezer/0/freezer.state # cat /sys/fs/cgroup/freezer/0/freezer.state |
bde5ab655 container freezer... |
111 112 113 114 |
THAWED This is the basic mechanism which should do the right thing for user space task in a simple scenario. |