27 Oct, 2010

4 commits

  • There have been numerous reports of stalls that pointed at the problem
    being somewhere in the VM. There are multiple roots to the problems which
    means dealing with any of the root problems in isolation is tricky to
    justify on their own and they would still need integration testing. This
    patch series puts together two different patch sets which in combination
    should tackle some of the root causes of latency problems being reported.

    Patch 1 adds a tracepoint for shrink_inactive_list. For this series, the
    most important results is being able to calculate the scanning/reclaim
    ratio as a measure of the amount of work being done by page reclaim.

    Patch 2 accounts for time spent in congestion_wait.

    Patches 3-6 were originally developed by Kosaki Motohiro but reworked for
    this series. It has been noted that lumpy reclaim is far too aggressive
    and trashes the system somewhat. As SLUB uses high-order allocations, a
    large cost incurred by lumpy reclaim will be noticeable. It was also
    reported during transparent hugepage support testing that lumpy reclaim
    was trashing the system and these patches should mitigate that problem
    without disabling lumpy reclaim.

    Patch 7 adds wait_iff_congested() and replaces some callers of
    congestion_wait(). wait_iff_congested() only sleeps if there is a BDI
    that is currently congested. Patch 8 notes that any BDI being congested
    is not necessarily a problem because there could be multiple BDIs of
    varying speeds and numberous zones. It attempts to track when a zone
    being reclaimed contains many pages backed by a congested BDI and if so,
    reclaimers wait on the congestion queue.

    I ran a number of tests with monitoring on X86, X86-64 and PPC64. Each
    machine had 3G of RAM and the CPUs were

    X86: Intel P4 2-core
    X86-64: AMD Phenom 4-core
    PPC64: PPC970MP

    Each used a single disk and the onboard IO controller. Dirty ratio was
    left at 20. I'm just going to report for X86-64 and PPC64 in a vague
    attempt to keep this report short. Four kernels were tested each based on
    v2.6.36-rc4

    traceonly-v2r2: Patches 1 and 2 to instrument vmscan reclaims and congestion_wait
    lowlumpy-v2r3: Patches 1-6 to test if lumpy reclaim is better
    waitcongest-v2r3: Patches 1-7 to only wait on congestion
    waitwriteback-v2r4: Patches 1-8 to detect when a zone is congested

    nocongest-v1r5: Patches 1-3 for testing wait_iff_congestion
    nodirect-v1r5: Patches 1-10 to disable filesystem writeback for better IO

    The tests run were as follows

    kernbench
    compile-based benchmark. Smoke test performance

    sysbench
    OLTP read-only benchmark. Will be re-run in the future as read-write

    micro-mapped-file-stream
    This is a micro-benchmark from Johannes Weiner that accesses a
    large sparse-file through mmap(). It was configured to run in only
    single-CPU mode but can be indicative of how well page reclaim
    identifies suitable pages.

    stress-highalloc
    Tries to allocate huge pages under heavy load.

    kernbench, iozone and sysbench did not report any performance regression
    on any machine. sysbench did pressure the system lightly and there was
    reclaim activity but there were no difference of major interest between
    the kernels.

    X86-64 micro-mapped-file-stream

    traceonly-v2r2 lowlumpy-v2r3 waitcongest-v2r3 waitwriteback-v2r4
    pgalloc_dma 1639.00 ( 0.00%) 667.00 (-145.73%) 1167.00 ( -40.45%) 578.00 (-183.56%)
    pgalloc_dma32 2842410.00 ( 0.00%) 2842626.00 ( 0.01%) 2843043.00 ( 0.02%) 2843014.00 ( 0.02%)
    pgalloc_normal 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%)
    pgsteal_dma 729.00 ( 0.00%) 85.00 (-757.65%) 609.00 ( -19.70%) 125.00 (-483.20%)
    pgsteal_dma32 2338721.00 ( 0.00%) 2447354.00 ( 4.44%) 2429536.00 ( 3.74%) 2436772.00 ( 4.02%)
    pgsteal_normal 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%)
    pgscan_kswapd_dma 1469.00 ( 0.00%) 532.00 (-176.13%) 1078.00 ( -36.27%) 220.00 (-567.73%)
    pgscan_kswapd_dma32 4597713.00 ( 0.00%) 4503597.00 ( -2.09%) 4295673.00 ( -7.03%) 3891686.00 ( -18.14%)
    pgscan_kswapd_normal 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%)
    pgscan_direct_dma 71.00 ( 0.00%) 134.00 ( 47.01%) 243.00 ( 70.78%) 352.00 ( 79.83%)
    pgscan_direct_dma32 305820.00 ( 0.00%) 280204.00 ( -9.14%) 600518.00 ( 49.07%) 957485.00 ( 68.06%)
    pgscan_direct_normal 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%)
    pageoutrun 16296.00 ( 0.00%) 21254.00 ( 23.33%) 18447.00 ( 11.66%) 20067.00 ( 18.79%)
    allocstall 443.00 ( 0.00%) 273.00 ( -62.27%) 513.00 ( 13.65%) 1568.00 ( 71.75%)

    These are based on the raw figures taken from /proc/vmstat. It's a rough
    measure of reclaim activity. Note that allocstall counts are higher
    because we are entering direct reclaim more often as a result of not
    sleeping in congestion. In itself, it's not necessarily a bad thing.
    It's easier to get a view of what happened from the vmscan tracepoint
    report.

    FTrace Reclaim Statistics: vmscan

    traceonly-v2r2 lowlumpy-v2r3 waitcongest-v2r3 waitwriteback-v2r4
    Direct reclaims 443 273 513 1568
    Direct reclaim pages scanned 305968 280402 600825 957933
    Direct reclaim pages reclaimed 43503 19005 30327 117191
    Direct reclaim write file async I/O 0 0 0 0
    Direct reclaim write anon async I/O 0 3 4 12
    Direct reclaim write file sync I/O 0 0 0 0
    Direct reclaim write anon sync I/O 0 0 0 0
    Wake kswapd requests 187649 132338 191695 267701
    Kswapd wakeups 3 1 4 1
    Kswapd pages scanned 4599269 4454162 4296815 3891906
    Kswapd pages reclaimed 2295947 2428434 2399818 2319706
    Kswapd reclaim write file async I/O 1 0 1 1
    Kswapd reclaim write anon async I/O 59 187 41 222
    Kswapd reclaim write file sync I/O 0 0 0 0
    Kswapd reclaim write anon sync I/O 0 0 0 0
    Time stalled direct reclaim (seconds) 4.34 2.52 6.63 2.96
    Time kswapd awake (seconds) 11.15 10.25 11.01 10.19

    Total pages scanned 4905237 4734564 4897640 4849839
    Total pages reclaimed 2339450 2447439 2430145 2436897
    %age total pages scanned/reclaimed 47.69% 51.69% 49.62% 50.25%
    %age total pages scanned/written 0.00% 0.00% 0.00% 0.00%
    %age file pages scanned/written 0.00% 0.00% 0.00% 0.00%
    Percentage Time Spent Direct Reclaim 29.23% 19.02% 38.48% 20.25%
    Percentage Time kswapd Awake 78.58% 78.85% 76.83% 79.86%

    What is interesting here for nocongest in particular is that while direct
    reclaim scans more pages, the overall number of pages scanned remains the
    same and the ratio of pages scanned to pages reclaimed is more or less the
    same. In other words, while we are sleeping less, reclaim is not doing
    more work and as direct reclaim and kswapd is awake for less time, it
    would appear to be doing less work.

    FTrace Reclaim Statistics: congestion_wait
    Direct number congest waited 87 196 64 0
    Direct time congest waited 4604ms 4732ms 5420ms 0ms
    Direct full congest waited 72 145 53 0
    Direct number conditional waited 0 0 324 1315
    Direct time conditional waited 0ms 0ms 0ms 0ms
    Direct full conditional waited 0 0 0 0
    KSwapd number congest waited 20 10 15 7
    KSwapd time congest waited 1264ms 536ms 884ms 284ms
    KSwapd full congest waited 10 4 6 2
    KSwapd number conditional waited 0 0 0 0
    KSwapd time conditional waited 0ms 0ms 0ms 0ms
    KSwapd full conditional waited 0 0 0 0

    The vanilla kernel spent 8 seconds asleep in direct reclaim and no time at
    all asleep with the patches.

    MMTests Statistics: duration
    User/Sys Time Running Test (seconds) 10.51 10.73 10.6 11.66
    Total Elapsed Time (seconds) 14.19 13.00 14.33 12.76

    Overall, the tests completed faster. It is interesting to note that backing off further
    when a zone is congested and not just a BDI was more efficient overall.

    PPC64 micro-mapped-file-stream
    pgalloc_dma 3024660.00 ( 0.00%) 3027185.00 ( 0.08%) 3025845.00 ( 0.04%) 3026281.00 ( 0.05%)
    pgalloc_normal 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%)
    pgsteal_dma 2508073.00 ( 0.00%) 2565351.00 ( 2.23%) 2463577.00 ( -1.81%) 2532263.00 ( 0.96%)
    pgsteal_normal 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%)
    pgscan_kswapd_dma 4601307.00 ( 0.00%) 4128076.00 ( -11.46%) 3912317.00 ( -17.61%) 3377165.00 ( -36.25%)
    pgscan_kswapd_normal 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%)
    pgscan_direct_dma 629825.00 ( 0.00%) 971622.00 ( 35.18%) 1063938.00 ( 40.80%) 1711935.00 ( 63.21%)
    pgscan_direct_normal 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%)
    pageoutrun 27776.00 ( 0.00%) 20458.00 ( -35.77%) 18763.00 ( -48.04%) 18157.00 ( -52.98%)
    allocstall 977.00 ( 0.00%) 2751.00 ( 64.49%) 2098.00 ( 53.43%) 5136.00 ( 80.98%)

    Similar trends to x86-64. allocstalls are up but it's not necessarily bad.

    FTrace Reclaim Statistics: vmscan
    Direct reclaims 977 2709 2098 5136
    Direct reclaim pages scanned 629825 963814 1063938 1711935
    Direct reclaim pages reclaimed 75550 242538 150904 387647
    Direct reclaim write file async I/O 0 0 0 2
    Direct reclaim write anon async I/O 0 10 0 4
    Direct reclaim write file sync I/O 0 0 0 0
    Direct reclaim write anon sync I/O 0 0 0 0
    Wake kswapd requests 392119 1201712 571935 571921
    Kswapd wakeups 3 2 3 3
    Kswapd pages scanned 4601307 4128076 3912317 3377165
    Kswapd pages reclaimed 2432523 2318797 2312673 2144616
    Kswapd reclaim write file async I/O 20 1 1 1
    Kswapd reclaim write anon async I/O 57 132 11 121
    Kswapd reclaim write file sync I/O 0 0 0 0
    Kswapd reclaim write anon sync I/O 0 0 0 0
    Time stalled direct reclaim (seconds) 6.19 7.30 13.04 10.88
    Time kswapd awake (seconds) 21.73 26.51 25.55 23.90

    Total pages scanned 5231132 5091890 4976255 5089100
    Total pages reclaimed 2508073 2561335 2463577 2532263
    %age total pages scanned/reclaimed 47.95% 50.30% 49.51% 49.76%
    %age total pages scanned/written 0.00% 0.00% 0.00% 0.00%
    %age file pages scanned/written 0.00% 0.00% 0.00% 0.00%
    Percentage Time Spent Direct Reclaim 18.89% 20.65% 32.65% 27.65%
    Percentage Time kswapd Awake 72.39% 80.68% 78.21% 77.40%

    Again, a similar trend that the congestion_wait changes mean that direct
    reclaim scans more pages but the overall number of pages scanned while
    slightly reduced, are very similar. The ratio of scanning/reclaimed
    remains roughly similar. The downside is that kswapd and direct reclaim
    was awake longer and for a larger percentage of the overall workload.
    It's possible there were big differences in the amount of time spent
    reclaiming slab pages between the different kernels which is plausible
    considering that the micro tests runs after fsmark and sysbench.

    Trace Reclaim Statistics: congestion_wait
    Direct number congest waited 845 1312 104 0
    Direct time congest waited 19416ms 26560ms 7544ms 0ms
    Direct full congest waited 745 1105 72 0
    Direct number conditional waited 0 0 1322 2935
    Direct time conditional waited 0ms 0ms 12ms 312ms
    Direct full conditional waited 0 0 0 3
    KSwapd number congest waited 39 102 75 63
    KSwapd time congest waited 2484ms 6760ms 5756ms 3716ms
    KSwapd full congest waited 20 48 46 25
    KSwapd number conditional waited 0 0 0 0
    KSwapd time conditional waited 0ms 0ms 0ms 0ms
    KSwapd full conditional waited 0 0 0 0

    The vanilla kernel spent 20 seconds asleep in direct reclaim and only
    312ms asleep with the patches. The time kswapd spent congest waited was
    also reduced by a large factor.

    MMTests Statistics: duration
    ser/Sys Time Running Test (seconds) 26.58 28.05 26.9 28.47
    Total Elapsed Time (seconds) 30.02 32.86 32.67 30.88

    With all patches applies, the completion times are very similar.

    X86-64 STRESS-HIGHALLOC
    traceonly-v2r2 lowlumpy-v2r3 waitcongest-v2r3waitwriteback-v2r4
    Pass 1 82.00 ( 0.00%) 84.00 ( 2.00%) 85.00 ( 3.00%) 85.00 ( 3.00%)
    Pass 2 90.00 ( 0.00%) 87.00 (-3.00%) 88.00 (-2.00%) 89.00 (-1.00%)
    At Rest 92.00 ( 0.00%) 90.00 (-2.00%) 90.00 (-2.00%) 91.00 (-1.00%)

    Success figures across the board are broadly similar.

    traceonly-v2r2 lowlumpy-v2r3 waitcongest-v2r3waitwriteback-v2r4
    Direct reclaims 1045 944 886 887
    Direct reclaim pages scanned 135091 119604 109382 101019
    Direct reclaim pages reclaimed 88599 47535 47863 46671
    Direct reclaim write file async I/O 494 283 465 280
    Direct reclaim write anon async I/O 29357 13710 16656 13462
    Direct reclaim write file sync I/O 154 2 2 3
    Direct reclaim write anon sync I/O 14594 571 509 561
    Wake kswapd requests 7491 933 872 892
    Kswapd wakeups 814 778 731 780
    Kswapd pages scanned 7290822 15341158 11916436 13703442
    Kswapd pages reclaimed 3587336 3142496 3094392 3187151
    Kswapd reclaim write file async I/O 91975 32317 28022 29628
    Kswapd reclaim write anon async I/O 1992022 789307 829745 849769
    Kswapd reclaim write file sync I/O 0 0 0 0
    Kswapd reclaim write anon sync I/O 0 0 0 0
    Time stalled direct reclaim (seconds) 4588.93 2467.16 2495.41 2547.07
    Time kswapd awake (seconds) 2497.66 1020.16 1098.06 1176.82

    Total pages scanned 7425913 15460762 12025818 13804461
    Total pages reclaimed 3675935 3190031 3142255 3233822
    %age total pages scanned/reclaimed 49.50% 20.63% 26.13% 23.43%
    %age total pages scanned/written 28.66% 5.41% 7.28% 6.47%
    %age file pages scanned/written 1.25% 0.21% 0.24% 0.22%
    Percentage Time Spent Direct Reclaim 57.33% 42.15% 42.41% 42.99%
    Percentage Time kswapd Awake 43.56% 27.87% 29.76% 31.25%

    Scanned/reclaimed ratios again look good with big improvements in
    efficiency. The Scanned/written ratios also look much improved. With a
    better scanned/written ration, there is an expectation that IO would be
    more efficient and indeed, the time spent in direct reclaim is much
    reduced by the full series and kswapd spends a little less time awake.

    Overall, indications here are that allocations were happening much faster
    and this can be seen with a graph of the latency figures as the
    allocations were taking place
    http://www.csn.ul.ie/~mel/postings/vmscanreduce-20101509/highalloc-interlatency-hydra-mean.ps

    FTrace Reclaim Statistics: congestion_wait
    Direct number congest waited 1333 204 169 4
    Direct time congest waited 78896ms 8288ms 7260ms 200ms
    Direct full congest waited 756 92 69 2
    Direct number conditional waited 0 0 26 186
    Direct time conditional waited 0ms 0ms 0ms 2504ms
    Direct full conditional waited 0 0 0 25
    KSwapd number congest waited 4 395 227 282
    KSwapd time congest waited 384ms 25136ms 10508ms 18380ms
    KSwapd full congest waited 3 232 98 176
    KSwapd number conditional waited 0 0 0 0
    KSwapd time conditional waited 0ms 0ms 0ms 0ms
    KSwapd full conditional waited 0 0 0 0
    KSwapd full conditional waited 318 0 312 9

    Overall, the time spent speeping is reduced. kswapd is still hitting
    congestion_wait() but that is because there are callers remaining where it
    wasn't clear in advance if they should be changed to wait_iff_congested()
    or not. Overall the sleep imes are reduced though - from 79ish seconds to
    about 19.

    MMTests Statistics: duration
    User/Sys Time Running Test (seconds) 3415.43 3386.65 3388.39 3377.5
    Total Elapsed Time (seconds) 5733.48 3660.33 3689.41 3765.39

    With the full series, the time to complete the tests are reduced by 30%

    PPC64 STRESS-HIGHALLOC
    traceonly-v2r2 lowlumpy-v2r3 waitcongest-v2r3waitwriteback-v2r4
    Pass 1 17.00 ( 0.00%) 34.00 (17.00%) 38.00 (21.00%) 43.00 (26.00%)
    Pass 2 25.00 ( 0.00%) 37.00 (12.00%) 42.00 (17.00%) 46.00 (21.00%)
    At Rest 49.00 ( 0.00%) 43.00 (-6.00%) 45.00 (-4.00%) 51.00 ( 2.00%)

    Success rates there are *way* up particularly considering that the 16MB
    huge pages on PPC64 mean that it's always much harder to allocate them.

    FTrace Reclaim Statistics: vmscan
    stress-highalloc stress-highalloc stress-highalloc stress-highalloc
    traceonly-v2r2 lowlumpy-v2r3 waitcongest-v2r3waitwriteback-v2r4
    Direct reclaims 499 505 564 509
    Direct reclaim pages scanned 223478 41898 51818 45605
    Direct reclaim pages reclaimed 137730 21148 27161 23455
    Direct reclaim write file async I/O 399 136 162 136
    Direct reclaim write anon async I/O 46977 2865 4686 3998
    Direct reclaim write file sync I/O 29 0 1 3
    Direct reclaim write anon sync I/O 31023 159 237 239
    Wake kswapd requests 420 351 360 326
    Kswapd wakeups 185 294 249 277
    Kswapd pages scanned 15703488 16392500 17821724 17598737
    Kswapd pages reclaimed 5808466 2908858 3139386 3145435
    Kswapd reclaim write file async I/O 159938 18400 18717 13473
    Kswapd reclaim write anon async I/O 3467554 228957 322799 234278
    Kswapd reclaim write file sync I/O 0 0 0 0
    Kswapd reclaim write anon sync I/O 0 0 0 0
    Time stalled direct reclaim (seconds) 9665.35 1707.81 2374.32 1871.23
    Time kswapd awake (seconds) 9401.21 1367.86 1951.75 1328.88

    Total pages scanned 15926966 16434398 17873542 17644342
    Total pages reclaimed 5946196 2930006 3166547 3168890
    %age total pages scanned/reclaimed 37.33% 17.83% 17.72% 17.96%
    %age total pages scanned/written 23.27% 1.52% 1.94% 1.43%
    %age file pages scanned/written 1.01% 0.11% 0.11% 0.08%
    Percentage Time Spent Direct Reclaim 44.55% 35.10% 41.42% 36.91%
    Percentage Time kswapd Awake 86.71% 43.58% 52.67% 41.14%

    While the scanning rates are slightly up, the scanned/reclaimed and
    scanned/written figures are much improved. The time spent in direct
    reclaim and with kswapd are massively reduced, mostly by the lowlumpy
    patches.

    FTrace Reclaim Statistics: congestion_wait
    Direct number congest waited 725 303 126 3
    Direct time congest waited 45524ms 9180ms 5936ms 300ms
    Direct full congest waited 487 190 52 3
    Direct number conditional waited 0 0 200 301
    Direct time conditional waited 0ms 0ms 0ms 1904ms
    Direct full conditional waited 0 0 0 19
    KSwapd number congest waited 0 2 23 4
    KSwapd time congest waited 0ms 200ms 420ms 404ms
    KSwapd full congest waited 0 2 2 4
    KSwapd number conditional waited 0 0 0 0
    KSwapd time conditional waited 0ms 0ms 0ms 0ms
    KSwapd full conditional waited 0 0 0 0

    Not as dramatic a story here but the time spent asleep is reduced and we
    can still see what wait_iff_congested is going to sleep when necessary.

    MMTests Statistics: duration
    User/Sys Time Running Test (seconds) 12028.09 3157.17 3357.79 3199.16
    Total Elapsed Time (seconds) 10842.07 3138.72 3705.54 3229.85

    The time to complete this test goes way down. With the full series, we
    are allocating over twice the number of huge pages in 30% of the time and
    there is a corresponding impact on the allocation latency graph available
    at.

    http://www.csn.ul.ie/~mel/postings/vmscanreduce-20101509/highalloc-interlatency-powyah-mean.ps

    This patch:

    Add a trace event for shrink_inactive_list() and updates the sample
    postprocessing script appropriately. It can be used to determine how many
    pages were reclaimed and for non-lumpy reclaim where exactly the pages
    were reclaimed from.

    Signed-off-by: Mel Gorman
    Cc: Johannes Weiner
    Cc: Minchan Kim
    Cc: Wu Fengguang
    Cc: KAMEZAWA Hiroyuki
    Cc: KOSAKI Motohiro
    Cc: Rik van Riel
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Mel Gorman
     
  • Signed-off-by: Matt Mackall
    Cc: Nikanth Karthikesan
    Cc: Balbir Singh
    Cc: Wu Fengguang
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Matt Mackall
     
  • * 'for-2.6.37' of git://linux-nfs.org/~bfields/linux: (99 commits)
    svcrpc: svc_tcp_sendto XPT_DEAD check is redundant
    svcrpc: no need for XPT_DEAD check in svc_xprt_enqueue
    svcrpc: assume svc_delete_xprt() called only once
    svcrpc: never clear XPT_BUSY on dead xprt
    nfsd4: fix connection allocation in sequence()
    nfsd4: only require krb5 principal for NFSv4.0 callbacks
    nfsd4: move minorversion to client
    nfsd4: delay session removal till free_client
    nfsd4: separate callback change and callback probe
    nfsd4: callback program number is per-session
    nfsd4: track backchannel connections
    nfsd4: confirm only on succesful create_session
    nfsd4: make backchannel sequence number per-session
    nfsd4: use client pointer to backchannel session
    nfsd4: move callback setup into session init code
    nfsd4: don't cache seq_misordered replies
    SUNRPC: Properly initialize sock_xprt.srcaddr in all cases
    SUNRPC: Use conventional switch statement when reclassifying sockets
    sunrpc/xprtrdma: clean up workqueue usage
    sunrpc: Turn list_for_each-s into the ..._entry-s
    ...

    Fix up trivial conflicts (two different deprecation notices added in
    separate branches) in Documentation/feature-removal-schedule.txt

    Linus Torvalds
     
  • * 'nfs-for-2.6.37' of git://git.linux-nfs.org/projects/trondmy/nfs-2.6:
    net/sunrpc: Use static const char arrays
    nfs4: fix channel attribute sanity-checks
    NFSv4.1: Use more sensible names for 'initialize_mountpoint'
    NFSv4.1: pnfs: filelayout: add driver's LAYOUTGET and GETDEVICEINFO infrastructure
    NFSv4.1: pnfs: add LAYOUTGET and GETDEVICEINFO infrastructure
    NFS: client needs to maintain list of inodes with active layouts
    NFS: create and destroy inode's layout cache
    NFSv4.1: pnfs: filelayout: introduce minimal file layout driver
    NFSv4.1: pnfs: full mount/umount infrastructure
    NFS: set layout driver
    NFS: ask for layouttypes during v4 fsinfo call
    NFS: change stateid to be a union
    NFSv4.1: pnfsd, pnfs: protocol level pnfs constants
    SUNRPC: define xdr_decode_opaque_fixed
    NFSD: remove duplicate NFS4_STATEID_SIZE

    Linus Torvalds
     

26 Oct, 2010

3 commits

  • * 'hwmon-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/groeck/staging: (24 commits)
    hwmon: lis3: Release resources in case of failure
    hwmon: lis3: Short explanations of platform data fields
    hwmon: lis3: Enhance lis3 selftest with IRQ line test
    hwmon: lis3: use block read to access data registers
    hwmon: lis3: Adjust fuzziness for 8 bit device
    hwmon: lis3: New parameters to platform data
    hwmon: lis3: restore axis enabled bits
    hwmon: lis3: Power on corrections
    hwmon: lis3: Update coordinates at polled device open
    hwmon: lis3: Cleanup interrupt handling
    hwmon: lis3: regulator control
    hwmon: lis3: pm_runtime support
    Kirkwood: add fan support for Network Space Max v2
    hwmon: add generic GPIO fan driver
    hwmon: (coretemp) fix reading of microcode revision (v2)
    hwmon: ({core, pkg, via-cpu}temp) remove unnecessary CONFIG_HOTPLUG_CPU ifdefs
    hwmon: (pkgtemp) align driver initialization style with coretemp
    hwmon: LTC4261 Hardware monitoring driver
    hwmon: (lis3) add axes module parameter for custom axis-mapping
    hwmon: (hp_accel) Add HP Mini 510x family support
    ...

    Linus Torvalds
     
  • This driver adds support for Linear Technology LTC4261 I2C Negative
    Voltage Hot Swap Controller.

    Reviewed-by: Ira W. Snyder
    Reviewed-by: Tom Grennan
    Signed-off-by: Guenter Roeck

    Guenter Roeck
     
  • * 'nfs-for-2.6.37' of git://git.linux-nfs.org/projects/trondmy/nfs-2.6: (67 commits)
    SUNRPC: Cleanup duplicate assignment in rpcauth_refreshcred
    nfs: fix unchecked value
    Ask for time_delta during fsinfo probe
    Revalidate caches on lock
    SUNRPC: After calling xprt_release(), we must restart from call_reserve
    NFSv4: Fix up the 'dircount' hint in encode_readdir
    NFSv4: Clean up nfs4_decode_dirent
    NFSv4: nfs4_decode_dirent must clear entry->fattr->valid
    NFSv4: Fix a regression in decode_getfattr
    NFSv4: Fix up decode_attr_filehandle() to handle the case of empty fh pointer
    NFS: Ensure we check all allocation return values in new readdir code
    NFS: Readdir plus in v4
    NFS: introduce generic decode_getattr function
    NFS: check xdr_decode for errors
    NFS: nfs_readdir_filler catch all errors
    NFS: readdir with vmapped pages
    NFS: remove page size checking code
    NFS: decode_dirent should use an xdr_stream
    SUNRPC: Add a helper function xdr_inline_peek
    NFS: remove readdir plus limit
    ...

    Linus Torvalds
     

25 Oct, 2010

9 commits

  • * 'for-linus' of git://git390.marist.edu/pub/scm/linux-2.6: (48 commits)
    [S390] topology: export cpu topology via proc/sysinfo
    [S390] topology: move topology sysinfo code
    [S390] topology: clean up facility detection
    [S390] cleanup facility list handling
    [S390] enable ARCH_DMA_ADDR_T_64BIT with 64BIT
    [S390] dasd: ignore unsolicited interrupts for DIAG
    [S390] kvm: Enable z196 instruction facilities
    [S390] dasd: fix unsolicited interrupt recognition
    [S390] dasd: fix use after free in dbf
    [S390] kvm: Fix badness at include/asm/mmu_context.h:83
    [S390] cio: fix I/O cancel function
    [S390] topology: change default
    [S390] smp: use correct cpu address in print_cpu_info()
    [S390] remove ieee_instruction_pointer from thread_struct
    [S390] cleanup system call parameter setup
    [S390] correct alignment of cpuid structure
    [S390] cleanup lowcore access from external interrupts
    [S390] cleanup lowcore access from program checks
    [S390] pgtable: move pte_mkhuge() from hugetlb.h to pgtable.h
    [S390] fix SIGBUS handling
    ...

    Linus Torvalds
     
  • * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound-2.6: (365 commits)
    ALSA: hda - Disable sticky PCM stream assignment for AD codecs
    ALSA: usb - Creative USB X-Fi volume knob support
    ALSA: ca0106: Use card specific dac id for mute controls.
    ALSA: ca0106: Allow different sound cards to use different SPI channel mappings.
    ALSA: ca0106: Create a nice spot for mapping channels to dacs.
    ALSA: ca0106: Move enabling of front dac out of hardcoded setup sequence.
    ALSA: ca0106: Pull out dac powering routine into separate function.
    ALSA: ca0106 - add Sound Blaster 5.1vx info.
    ASoC: tlv320dac33: Use usleep_range for delays
    ALSA: usb-audio: add Novation Launchpad support
    ALSA: hda - Add workarounds for CT-IBG controllers
    ALSA: hda - Fix wrong TLV mute bit for STAC/IDT codecs
    ASoC: tpa6130a2: Error handling for broken chip
    ASoC: max98088: Staticise m98088_eq_band
    ASoC: soc-core: Fix codec->name memory leak
    ALSA: hda - Apply ideapad quirk to Acer laptops with Cxt5066
    ALSA: hda - Add some workarounds for Creative IBG
    ALSA: hda - Fix wrong SPDIF NID assignment for CA0110
    ALSA: hda - Fix codec rename rules for ALC662-compatible codecs
    ALSA: hda - Add alc_init_jacks() call to other codecs
    ...

    Linus Torvalds
     
  • * 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mjg59/platform-drivers-x86: (44 commits)
    eeepc-wmi: Add cpufv sysfs interface
    eeepc-wmi: add additional hotkeys
    panasonic-laptop: Simplify calls to acpi_pcc_retrieve_biosdata
    panasonic-laptop: Handle errors properly if they happen
    intel_pmic_gpio: fix off-by-one value range checking
    IBM Real-Time "SMI Free" mode driver -v7
    Add OLPC XO-1 rfkill driver
    Move hdaps driver to platform/x86
    ideapad-laptop: Fix Makefile
    intel_pmic_gpio: swap the bits and mask args for intel_scu_ipc_update_register
    ideapad: Add param: no_bt_rfkill
    ideapad: Change the driver name to ideapad-laptop
    ideapad: rewrite the sw rfkill set
    ideapad: rewrite the hw rfkill notify
    ideapad: use EC command to control camera
    ideapad: use return value of _CFG to tell if device exist or not
    ideapad: make sure we bind on the correct device
    ideapad: check VPC bit before sync rfkill hw status
    ideapad: add ACPI helpers
    dell-laptop: Add debugfs support
    ...

    Linus Torvalds
     
  • …/ieee1394/linux1394-2.6

    * 'ieee1394-removal' of git://git.kernel.org/pub/scm/linux/kernel/git/ieee1394/linux1394-2.6:
    ieee1394: remove the old IEEE 1394 driver stack
    ieee1394: move init_ohci1394_dma to drivers/firewire/

    Fix trivial change/delete conflict: drivers/ieee1394/eth1394.c is
    getting removed, but was modified by the networking merge.

    Linus Torvalds
     
  • Switch default value of the kernel parameter 'topology' from off to on.
    Various performance measurements have finally shown that there are no
    (known) regressions anywhere.

    Signed-off-by: Heiko Carstens
    Signed-off-by: Martin Schwidefsky

    Heiko Carstens
     
  • Takashi Iwai
     
  • Allow a module implementing a layout type to register, and
    have its mount/umount routines called for filesystems that
    the server declares support it.

    Signed-off-by: Fred Isaman
    Signed-off-by: Marc Eshel
    Signed-off-by: Andy Adamson
    Signed-off-by: Bian Naimeng
    Signed-off-by: Benny Halevy
    Signed-off-by: Trond Myklebust

    Fred Isaman
     
  • * 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial: (39 commits)
    Update broken web addresses in arch directory.
    Update broken web addresses in the kernel.
    Revert "drivers/usb: Remove unnecessary return's from void functions" for musb gadget
    Revert "Fix typo: configuation => configuration" partially
    ida: document IDA_BITMAP_LONGS calculation
    ext2: fix a typo on comment in ext2/inode.c
    drivers/scsi: Remove unnecessary casts of private_data
    drivers/s390: Remove unnecessary casts of private_data
    net/sunrpc/rpc_pipe.c: Remove unnecessary casts of private_data
    drivers/infiniband: Remove unnecessary casts of private_data
    drivers/gpu/drm: Remove unnecessary casts of private_data
    kernel/pm_qos_params.c: Remove unnecessary casts of private_data
    fs/ecryptfs: Remove unnecessary casts of private_data
    fs/seq_file.c: Remove unnecessary casts of private_data
    arm: uengine.c: remove C99 comments
    arm: scoop.c: remove C99 comments
    Fix typo configue => configure in comments
    Fix typo: configuation => configuration
    Fix typo interrest[ing|ed] => interest[ing|ed]
    Fix various typos of valid in comments
    ...

    Fix up trivial conflicts in:
    drivers/char/ipmi/ipmi_si_intf.c
    drivers/usb/gadget/rndis.c
    net/irda/irnet/irnet_ppp.c

    Linus Torvalds
     
  • * 'kvm-updates/2.6.37' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (321 commits)
    KVM: Drop CONFIG_DMAR dependency around kvm_iommu_map_pages
    KVM: Fix signature of kvm_iommu_map_pages stub
    KVM: MCE: Send SRAR SIGBUS directly
    KVM: MCE: Add MCG_SER_P into KVM_MCE_CAP_SUPPORTED
    KVM: fix typo in copyright notice
    KVM: Disable interrupts around get_kernel_ns()
    KVM: MMU: Avoid sign extension in mmu_alloc_direct_roots() pae root address
    KVM: MMU: move access code parsing to FNAME(walk_addr) function
    KVM: MMU: audit: check whether have unsync sps after root sync
    KVM: MMU: audit: introduce audit_printk to cleanup audit code
    KVM: MMU: audit: unregister audit tracepoints before module unloaded
    KVM: MMU: audit: fix vcpu's spte walking
    KVM: MMU: set access bit for direct mapping
    KVM: MMU: cleanup for error mask set while walk guest page table
    KVM: MMU: update 'root_hpa' out of loop in PAE shadow path
    KVM: x86 emulator: Eliminate compilation warning in x86_decode_insn()
    KVM: x86: Fix constant type in kvm_get_time_scale
    KVM: VMX: Add AX to list of registers clobbered by guest switch
    KVM guest: Move a printk that's using the clock before it's ready
    KVM: x86: TSC catchup mode
    ...

    Linus Torvalds
     

24 Oct, 2010

12 commits


23 Oct, 2010

12 commits

  • This reverts commit f4a3e0bceb57466c31757f25e4e0ed108d1299ec. Jiri
    Sladby points out that the tty structure we're using may already be
    gone, and Al Viro doesn't hold back in complaining about the random
    loading of 'filp->private_data' which doesn't have to be a pointer at
    all, nor does checking the magic field for TTY_MAGIC prove anything.

    Belated review by Al:

    "a) global variable depending on stdin of the last opener? Affecting
    output of read(2)? Really?

    b) iterator is broken; list should be locked in ->start(), unlocked in
    ->stop() and *NOT* unlocked/relocked in ->next()

    c) ->show() ought to do nothing in case of ->device == NULL, instead
    of skipping those in ->next()/->start()

    d) regardless of the merits of the bright idea about asterisk at that
    line in output *and* regardless of (a), the implementation is not
    only atrociously ugly, it's actually very likely to be a roothole.
    Verifying that Cthulhu knows what number happens to be address of a
    tty_struct by blindly dereferencing memory at that address...
    Ouch.

    Please revert that crap."

    And Christoph pipes in and NAK's the approach of walking fd tables etc
    too. So it's pretty unanimous.

    Noticed-by: Jri Slaby
    Requested-by: Al Viro
    Cc: Greg Kroah-Hartman
    Cc: Werner Fink
    Cc: Alan Cox
    Cc: Christoph Hellwig
    Signed-off-by: Linus Torvalds

    Linus Torvalds
     
  • * git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb-2.6: (141 commits)
    USB: mct_u232: fix broken close
    USB: gadget: amd5536udc.c: fix error path
    USB: imx21-hcd - fix off by one resource size calculation
    usb: gadget: fix Kconfig warning
    usb: r8a66597-udc: Add processing when USB was removed.
    mxc_udc: add workaround for ENGcm09152 for i.MX35
    USB: ftdi_sio: add device ids for ScienceScope
    USB: musb: AM35x: Workaround for fifo read issue
    USB: musb: add musb support for AM35x
    USB: AM35x: Add musb support
    usb: Fix linker errors with CONFIG_PM=n
    USB: ohci-sh - use resource_size instead of defining its own resource_len macro
    USB: isp1362-hcd - use resource_size instead of defining its own resource_len macro
    USB: isp116x-hcd - use resource_size instead of defining its own resource_len macro
    USB: xhci: Fix compile error when CONFIG_PM=n
    USB: accept some invalid ep0-maxpacket values
    USB: xHCI: PCI power management implementation
    USB: xHCI: bus power management implementation
    USB: xHCI: port remote wakeup implementation
    USB: xHCI: port power management implementation
    ...

    Manually fix up (non-data) conflict: the SCSI merge gad renamed the
    'hw_sector_size' member to 'physical_block_size', and the USB tree
    brought a new use of it.

    Linus Torvalds
     
  • * git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty-2.6: (49 commits)
    serial8250: ratelimit "too much work" error
    serial: bfin_sport_uart: speed up sport RX sample rate to be 3% faster
    serial: abstraction for 8250 legacy ports
    serial/imx: check that the buffer is non-empty before sending it out
    serial: mfd: add more baud rates support
    jsm: Remove the uart port on errors
    Alchemy: Add UART PM methods.
    8250: allow platforms to override PM hook.
    altera_uart: Don't use plain integer as NULL pointer
    altera_uart: Fix missing prototype for registering an early console
    altera_uart: Fixup type usage of port flags
    altera_uart: Make it possible to use Altera UART and 8250 ports together
    altera_uart: Add support for different address strides
    altera_uart: Add support for getting mapbase and IRQ from resources
    altera_uart: Add support for polling mode (IRQ-less)
    serial: Factor out uart_poll_timeout() from 8250 driver
    serial: mark the 8250 driver as maintained
    serial: 8250: Don't delay after transmitter is ready.
    tty: MAINTAINERS: add drivers/serial/jsm/ as maintained driver
    vcs: invoke the vt update callback when /dev/vcs* is written to
    ...

    Linus Torvalds
     
  • * git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core-2.6: (31 commits)
    driver core: Display error codes when class suspend fails
    Driver core: Add section count to memory_block struct
    Driver core: Add mutex for adding/removing memory blocks
    Driver core: Move find_memory_block routine
    hpilo: Despecificate driver from iLO generation
    driver core: Convert link_mem_sections to use find_memory_block_hinted.
    driver core: Introduce find_memory_block_hinted which utilizes kset_find_obj_hinted.
    kobject: Introduce kset_find_obj_hinted.
    driver core: fix build for CONFIG_BLOCK not enabled
    driver-core: base: change to new flag variable
    sysfs: only access bin file vm_ops with the active lock
    sysfs: Fail bin file mmap if vma close is implemented.
    FW_LOADER: fix kconfig dependency warning on HOTPLUG
    uio: Statically allocate uio_class and use class .dev_attrs.
    uio: Support 2^MINOR_BITS minors
    uio: Cleanup irq handling.
    uio: Don't clear driver data
    uio: Fix lack of locking in init_uio_class
    SYSFS: Allow boot time switching between deprecated and modern sysfs layout
    driver core: remove CONFIG_SYSFS_DEPRECATED_V2 but keep it for block devices
    ...

    Linus Torvalds
     
  • * git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi-misc-2.6: (84 commits)
    [SCSI] be2iscsi: SGE Len == 64K
    [SCSI] be2iscsi: Remove premature free of cid
    [SCSI] be2iscsi: More time for FW
    [SCSI] libsas: fix bug for vacant phy
    [SCSI] sd: Fix overflow with big physical blocks
    [SCSI] st: add MTWEOFI to write filemarks without flushing drive buffer
    [SCSI] libsas: Don't issue commands to devices that have been hot-removed
    [SCSI] megaraid_sas: Add Online Controller Reset to MegaRAID SAS drive
    [SCSI] lpfc 8.3.17: Update lpfc driver version to 8.3.17
    [SCSI] lpfc 8.3.17: Replace function reset methodology
    [SCSI] lpfc 8.3.17: SCSI fixes
    [SCSI] lpfc 8.3.17: BSG fixes
    [SCSI] lpfc 8.3.17: SLI Additions and Fixes
    [SCSI] lpfc 8.3.17: Code Cleanup and Locking fixes
    [SCSI] zfcp: Remove scsi_cmnd->serial_number from debug traces
    [SCSI] ipr: fix array error logging
    [SCSI] aha152x: enable PCMCIA on 64bit
    [SCSI] scsi_dh_alua: Handle all states correctly
    [SCSI] cxgb4i: connection and ddp setting update
    [SCSI] cxgb3i: fixed connection over vlan
    ...

    Linus Torvalds
     
  • * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq:
    workqueue: remove in_workqueue_context()
    workqueue: Clarify that schedule_on_each_cpu is synchronous
    memory_hotplug: drop spurious calls to flush_scheduled_work()
    shpchp: update workqueue usage
    pciehp: update workqueue usage
    isdn/eicon: don't call flush_scheduled_work() from diva_os_remove_soft_isr()
    workqueue: add and use WQ_MEM_RECLAIM flag
    workqueue: fix HIGHPRI handling in keep_working()
    workqueue: add queue_work and activate_work trace points
    workqueue: prepare for more tracepoints
    workqueue: implement flush[_delayed]_work_sync()
    workqueue: factor out start_flush_work()
    workqueue: cleanup flush/cancel functions
    workqueue: implement alloc_ordered_workqueue()

    Fix up trivial conflict in fs/gfs2/main.c as per Tejun

    Linus Torvalds
     
  • * 'for-2.6.37/barrier' of git://git.kernel.dk/linux-2.6-block: (46 commits)
    xen-blkfront: disable barrier/flush write support
    Added blk-lib.c and blk-barrier.c was renamed to blk-flush.c
    block: remove BLKDEV_IFL_WAIT
    aic7xxx_old: removed unused 'req' variable
    block: remove the BH_Eopnotsupp flag
    block: remove the BLKDEV_IFL_BARRIER flag
    block: remove the WRITE_BARRIER flag
    swap: do not send discards as barriers
    fat: do not send discards as barriers
    ext4: do not send discards as barriers
    jbd2: replace barriers with explicit flush / FUA usage
    jbd2: Modify ASYNC_COMMIT code to not rely on queue draining on barrier
    jbd: replace barriers with explicit flush / FUA usage
    nilfs2: replace barriers with explicit flush / FUA usage
    reiserfs: replace barriers with explicit flush / FUA usage
    gfs2: replace barriers with explicit flush / FUA usage
    btrfs: replace barriers with explicit flush / FUA usage
    xfs: replace barriers with explicit flush / FUA usage
    block: pass gfp_mask and flags to sb_issue_discard
    dm: convey that all flushes are processed as empty
    ...

    Linus Torvalds
     
  • * 'for-2.6.37/core' of git://git.kernel.dk/linux-2.6-block: (39 commits)
    cfq-iosched: Fix a gcc 4.5 warning and put some comments
    block: Turn bvec_k{un,}map_irq() into static inline functions
    block: fix accounting bug on cross partition merges
    block: Make the integrity mapped property a bio flag
    block: Fix double free in blk_integrity_unregister
    block: Ensure physical block size is unsigned int
    blkio-throttle: Fix possible multiplication overflow in iops calculations
    blkio-throttle: limit max iops value to UINT_MAX
    blkio-throttle: There is no need to convert jiffies to milli seconds
    blkio-throttle: Fix link failure failure on i386
    blkio: Recalculate the throttled bio dispatch time upon throttle limit change
    blkio: Add root group to td->tg_list
    blkio: deletion of a cgroup was causes oops
    blkio: Do not export throttle files if CONFIG_BLK_DEV_THROTTLING=n
    block: set the bounce_pfn to the actual DMA limit rather than to max memory
    block: revert bad fix for memory hotplug causing bounces
    Fix compile error in blk-exec.c for !CONFIG_DETECT_HUNG_TASK
    block: set the bounce_pfn to the actual DMA limit rather than to max memory
    block: Prevent hang_check firing during long I/O
    cfq: improve fsync performance for small files
    ...

    Fix up trivial conflicts due to __rcu sparse annotation in include/linux/genhd.h

    Linus Torvalds
     
  • * 'llseek' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl:
    vfs: make no_llseek the default
    vfs: don't use BKL in default_llseek
    llseek: automatically add .llseek fop
    libfs: use generic_file_llseek for simple_attr
    mac80211: disallow seeks in minstrel debug code
    lirc: make chardev nonseekable
    viotape: use noop_llseek
    raw: use explicit llseek file operations
    ibmasmfs: use generic_file_llseek
    spufs: use llseek in all file operations
    arm/omap: use generic_file_llseek in iommu_debug
    lkdtm: use generic_file_llseek in debugfs
    net/wireless: use generic_file_llseek in debugfs
    drm: use noop_llseek

    Linus Torvalds
     
  • Extends FSL EHCI platform driver glue layer to support
    MPC5121 USB controllers. MPC5121 Rev 2.0 silicon EHCI
    registers are in big endian format. The appropriate flags
    are set using the information in the platform data structure.
    MPC83xx system interface registers are not available on
    MPC512x, so the access to these registers is isolated in
    MPC512x case. Furthermore the USB controller clocks
    must be enabled before 512x register accesses which is
    done by providing platform specific init callback.

    The MPC512x internal USB PHY doesn't provide supply voltage.
    For boards using different power switches allow specifying
    DRVVBUS and PWR_FAULT signal polarity of the MPC5121 internal
    PHY using "fsl,invert-drvvbus" and "fsl,invert-pwr-fault"
    properties in the device tree USB nodes. Adds documentation
    for this new device tree bindings.

    Signed-off-by: Anatolij Gustschin
    Cc: Grant Likely
    Signed-off-by: Greg Kroah-Hartman

    Anatolij Gustschin
     
  • This assigns the minor number 192 to the yurex driver.

    We also fix up the previous usb minor number entry, it was wrong.

    Cc: Tomoki Sekiyama
    Signed-off-by: Greg Kroah-Hartman

    Greg Kroah-Hartman
     
  • The /sys/kernel/debug/usb/devices file doesn't know about Wireless or
    SuperSpeed USB. This patch (as1416b) teaches it, and updates the
    Documentation/usb/proc_sub_info.txt file accordingly.

    Signed-off-by: Alan Stern
    CC: David Vrabel
    CC: Sarah Sharp
    Signed-off-by: Greg Kroah-Hartman

    Alan Stern