Blame view

Documentation/cgroups/freezer-subsystem.txt 4.79 KB
3b1b3f6e5   Li Zefan   freezer_cg: disab...
1
  The cgroup freezer is useful to batch job management system which start
bde5ab655   Matt Helsley   container freezer...
2
3
4
5
6
7
  and stop sets of tasks in order to schedule the resources of a machine
  according to the desires of a system administrator. This sort of program
  is often used on HPC clusters to schedule access to the cluster as a
  whole. The cgroup freezer uses cgroups to describe the set of tasks to
  be started/stopped by the batch job management system. It also provides
  a means to start and stop the tasks composing the job.
3b1b3f6e5   Li Zefan   freezer_cg: disab...
8
  The cgroup freezer will also be useful for checkpointing running groups
bde5ab655   Matt Helsley   container freezer...
9
10
11
12
13
14
15
16
  of tasks. The freezer allows the checkpoint code to obtain a consistent
  image of the tasks by attempting to force the tasks in a cgroup into a
  quiescent state. Once the tasks are quiescent another task can
  walk /proc or invoke a kernel interface to gather information about the
  quiesced tasks. Checkpointed tasks can be restarted later should a
  recoverable error occur. This also allows the checkpointed tasks to be
  migrated between nodes in a cluster by copying the gathered information
  to another node and restarting the tasks there.
3b1b3f6e5   Li Zefan   freezer_cg: disab...
17
  Sequences of SIGSTOP and SIGCONT are not always sufficient for stopping
bde5ab655   Matt Helsley   container freezer...
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
  and resuming tasks in userspace. Both of these signals are observable
  from within the tasks we wish to freeze. While SIGSTOP cannot be caught,
  blocked, or ignored it can be seen by waiting or ptracing parent tasks.
  SIGCONT is especially unsuitable since it can be caught by the task. Any
  programs designed to watch for SIGSTOP and SIGCONT could be broken by
  attempting to use SIGSTOP and SIGCONT to stop and resume tasks. We can
  demonstrate this problem using nested bash shells:
  
  	$ echo $$
  	16644
  	$ bash
  	$ echo $$
  	16690
  
  	From a second, unrelated bash shell:
  	$ kill -SIGSTOP 16690
5f1116167   Rafael J. Wysocki   Documentation: Fi...
34
  	$ kill -SIGCONT 16690
bde5ab655   Matt Helsley   container freezer...
35

5f1116167   Rafael J. Wysocki   Documentation: Fi...
36
  	<at this point 16690 exits and causes 16644 to exit too>
bde5ab655   Matt Helsley   container freezer...
37

3b1b3f6e5   Li Zefan   freezer_cg: disab...
38
  This happens because bash can observe both signals and choose how it
bde5ab655   Matt Helsley   container freezer...
39
  responds to them.
3b1b3f6e5   Li Zefan   freezer_cg: disab...
40
  Another example of a program which catches and responds to these
bde5ab655   Matt Helsley   container freezer...
41
42
  signals is gdb. In fact any program designed to use ptrace is likely to
  have a problem with this method of stopping and resuming tasks.
3b1b3f6e5   Li Zefan   freezer_cg: disab...
43
  In contrast, the cgroup freezer uses the kernel freezer code to
bde5ab655   Matt Helsley   container freezer...
44
45
46
  prevent the freeze/unfreeze cycle from becoming visible to the tasks
  being frozen. This allows the bash example above and gdb to run as
  expected.
ef9fe980c   Tejun Heo   cgroup_freezer: i...
47
48
49
50
51
  The cgroup freezer is hierarchical. Freezing a cgroup freezes all
  tasks beloning to the cgroup and all its descendant cgroups. Each
  cgroup has its own state (self-state) and the state inherited from the
  parent (parent-state). Iff both states are THAWED, the cgroup is
  THAWED.
bde5ab655   Matt Helsley   container freezer...
52

ef9fe980c   Tejun Heo   cgroup_freezer: i...
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
  The following cgroupfs files are created by cgroup freezer.
  
  * freezer.state: Read-write.
  
    When read, returns the effective state of the cgroup - "THAWED",
    "FREEZING" or "FROZEN". This is the combined self and parent-states.
    If any is freezing, the cgroup is freezing (FREEZING or FROZEN).
  
    FREEZING cgroup transitions into FROZEN state when all tasks
    belonging to the cgroup and its descendants become frozen. Note that
    a cgroup reverts to FREEZING from FROZEN after a new task is added
    to the cgroup or one of its descendant cgroups until the new task is
    frozen.
  
    When written, sets the self-state of the cgroup. Two values are
    allowed - "FROZEN" and "THAWED". If FROZEN is written, the cgroup,
    if not already freezing, enters FREEZING state along with all its
    descendant cgroups.
  
    If THAWED is written, the self-state of the cgroup is changed to
    THAWED.  Note that the effective state may not change to THAWED if
    the parent-state is still freezing. If a cgroup's effective state
    becomes THAWED, all its descendants which are freezing because of
    the cgroup also leave the freezing state.
  
  * freezer.self_freezing: Read only.
  
    Shows the self-state. 0 if the self-state is THAWED; otherwise, 1.
    This value is 1 iff the last write to freezer.state was "FROZEN".
  
  * freezer.parent_freezing: Read only.
  
    Shows the parent-state.  0 if none of the cgroup's ancestors is
    frozen; otherwise, 1.
  
  The root cgroup is non-freezable and the above interface files don't
  exist.
3b1b3f6e5   Li Zefan   freezer_cg: disab...
90

bde5ab655   Matt Helsley   container freezer...
91
  * Examples of usage :
f6e07d380   Jörg Sommer   Documentation: up...
92
93
94
95
     # mkdir /sys/fs/cgroup/freezer
     # mount -t cgroup -ofreezer freezer /sys/fs/cgroup/freezer
     # mkdir /sys/fs/cgroup/freezer/0
     # echo $some_pid > /sys/fs/cgroup/freezer/0/tasks
bde5ab655   Matt Helsley   container freezer...
96
97
  
  to get status of the freezer subsystem :
f6e07d380   Jörg Sommer   Documentation: up...
98
     # cat /sys/fs/cgroup/freezer/0/freezer.state
bde5ab655   Matt Helsley   container freezer...
99
100
101
     THAWED
  
  to freeze all tasks in the container :
f6e07d380   Jörg Sommer   Documentation: up...
102
103
     # echo FROZEN > /sys/fs/cgroup/freezer/0/freezer.state
     # cat /sys/fs/cgroup/freezer/0/freezer.state
bde5ab655   Matt Helsley   container freezer...
104
     FREEZING
f6e07d380   Jörg Sommer   Documentation: up...
105
     # cat /sys/fs/cgroup/freezer/0/freezer.state
bde5ab655   Matt Helsley   container freezer...
106
107
108
     FROZEN
  
  to unfreeze all tasks in the container :
f6e07d380   Jörg Sommer   Documentation: up...
109
110
     # echo THAWED > /sys/fs/cgroup/freezer/0/freezer.state
     # cat /sys/fs/cgroup/freezer/0/freezer.state
bde5ab655   Matt Helsley   container freezer...
111
112
113
114
     THAWED
  
  This is the basic mechanism which should do the right thing for user space task
  in a simple scenario.