Commit 774fea1a38c6a5a8ccc10969db84da24565f276f

Authored by Stewart Smith
Committed by Benjamin Herrenschmidt
1 parent 60b962239a

powerpc/powernv: Read OPAL error log and export it through sysfs

Based on a patch by: Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>

This patch adds support to read error logs from OPAL and export
them to userspace through a sysfs interface.

We export each log entry as a directory in /sys/firmware/opal/elog/

Currently, OPAL will buffer up to 128 error log records, we don't
need to have any knowledge of this limit on the Linux side as that
is actually largely transparent to us.

Each error log entry has the following files: id, type, acknowledge, raw.
Currently we just export the raw binary error log in the 'raw' attribute.
In a future patch, we may parse more of the error log to make it a bit
easier for userspace (e.g. to be able to display a brief summary in
petitboot without having to have a full parser).

If we have >128 logs from OPAL, we'll only be notified of 128 until
userspace starts acknowledging them. This limitation may be lifted in
the future and with this patch, that should "just work" from the linux side.

A userspace daemon should:
- wait for error log entries using normal mechanisms (we announce creation)
- read error log entry
- save error log entry safely to disk
- acknowledge the error log entry
- rinse, repeat.

On the Linux side, we read the error log when we're notified of it. This
possibly isn't ideal as it would be better to only read them on-demand.
However, this doesn't really work with current OPAL interface, so we
read the error log immediately when notified at the moment.

I've tested this pretty extensively and am rather confident that the
linux side of things works rather well. There is currently an issue with
the service processor side of things for >128 error logs though.

Signed-off-by: Stewart Smith <stewart@linux.vnet.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

Showing 6 changed files with 394 additions and 1 deletions Side-by-side Diff

Documentation/ABI/stable/sysfs-firmware-opal-elog
  1 +What: /sys/firmware/opal/elog
  2 +Date: Feb 2014
  3 +Contact: Stewart Smith <stewart@linux.vnet.ibm.com>
  4 +Description:
  5 + This directory exposes error log entries retrieved
  6 + through the OPAL firmware interface.
  7 +
  8 + Each error log is identified by a unique ID and will
  9 + exist until explicitly acknowledged to firmware.
  10 +
  11 + Each log entry has a directory in /sys/firmware/opal/elog.
  12 +
  13 + Log entries may be purged by the service processor
  14 + before retrieved by firmware or retrieved/acknowledged by
  15 + Linux if there is no room for more log entries.
  16 +
  17 + In the event that Linux has retrieved the log entries
  18 + but not explicitly acknowledged them to firmware and
  19 + the service processor needs more room for log entries,
  20 + the only remaining copy of a log message may be in
  21 + Linux.
  22 +
  23 + Typically, a user space daemon will monitor for new
  24 + entries, read them out and acknowledge them.
  25 +
  26 + The service processor may be able to store more log
  27 + entries than firmware can, so after you acknowledge
  28 + an event from Linux you may instantly get another one
  29 + from the queue that was generated some time in the past.
  30 +
  31 + The raw log format is a binary format. We currently
  32 + do not parse this at all in kernel, leaving it up to
  33 + user space to solve the problem. In future, we may
  34 + do more parsing in kernel and add more files to make
  35 + it easier for simple user space processes to extract
  36 + more information.
  37 +
  38 + For each log entry (directory), there are the following
  39 + files:
  40 +
  41 + id: An ASCII representation of the ID of the
  42 + error log, in hex - e.g. "0x01".
  43 +
  44 + type: An ASCII representation of the type id and
  45 + description of the type of error log.
  46 + Currently just "0x00 PEL" - platform error log.
  47 + In the future there may be additional types.
  48 +
  49 + raw: A read-only binary file that can be read
  50 + to get the raw log entry. These are
  51 + <16kb, often just hundreds of bytes and
  52 + "average" 2kb.
  53 +
  54 + acknowledge: Writing 'ack' to this file will acknowledge
  55 + the error log to firmware (and in turn
  56 + the service processor, if applicable).
  57 + Shortly after acknowledging it, the log
  58 + entry will be removed from sysfs.
  59 + Reading this file will list the supported
  60 + operations (curently just acknowledge).
arch/powerpc/include/asm/opal.h
... ... @@ -151,6 +151,11 @@
151 151 #define OPAL_LPC_READ 67
152 152 #define OPAL_LPC_WRITE 68
153 153 #define OPAL_RETURN_CPU 69
  154 +#define OPAL_ELOG_READ 71
  155 +#define OPAL_ELOG_WRITE 72
  156 +#define OPAL_ELOG_ACK 73
  157 +#define OPAL_ELOG_RESEND 74
  158 +#define OPAL_ELOG_SIZE 75
154 159 #define OPAL_FLASH_VALIDATE 76
155 160 #define OPAL_FLASH_MANAGE 77
156 161 #define OPAL_FLASH_UPDATE 78
... ... @@ -823,6 +828,13 @@
823 828 uint32_t addr, uint32_t data, uint32_t sz);
824 829 int64_t opal_lpc_read(uint32_t chip_id, enum OpalLPCAddressType addr_type,
825 830 uint32_t addr, __be32 *data, uint32_t sz);
  831 +
  832 +int64_t opal_read_elog(uint64_t buffer, size_t size, uint64_t log_id);
  833 +int64_t opal_get_elog_size(uint64_t *log_id, size_t *size, uint64_t *elog_type);
  834 +int64_t opal_write_elog(uint64_t buffer, uint64_t size, uint64_t offset);
  835 +int64_t opal_send_ack_elog(uint64_t log_id);
  836 +void opal_resend_pending_logs(void);
  837 +
826 838 int64_t opal_validate_flash(uint64_t buffer, uint32_t *size, uint32_t *result);
827 839 int64_t opal_manage_flash(uint8_t op);
828 840 int64_t opal_update_flash(uint64_t blk_list);
... ... @@ -863,6 +875,7 @@
863 875 extern unsigned long opal_get_boot_time(void);
864 876 extern void opal_nvram_init(void);
865 877 extern void opal_flash_init(void);
  878 +extern int opal_elog_init(void);
866 879  
867 880 extern int opal_machine_check(struct pt_regs *regs);
868 881 extern bool opal_mce_check_early_recovery(struct pt_regs *regs);
arch/powerpc/platforms/powernv/Makefile
1 1 obj-y += setup.o opal-takeover.o opal-wrappers.o opal.o
2 2 obj-y += opal-rtc.o opal-nvram.o opal-lpc.o opal-flash.o
3   -obj-y += rng.o
  3 +obj-y += rng.o opal-elog.o
4 4  
5 5 obj-$(CONFIG_SMP) += smp.o
6 6 obj-$(CONFIG_PCI) += pci.o pci-p5ioc2.o pci-ioda.o
arch/powerpc/platforms/powernv/opal-elog.c
  1 +/*
  2 + * Error log support on PowerNV.
  3 + *
  4 + * Copyright 2013,2014 IBM Corp.
  5 + *
  6 + * This program is free software; you can redistribute it and/or
  7 + * modify it under the terms of the GNU General Public License
  8 + * as published by the Free Software Foundation; either version
  9 + * 2 of the License, or (at your option) any later version.
  10 + */
  11 +#include <linux/kernel.h>
  12 +#include <linux/init.h>
  13 +#include <linux/of.h>
  14 +#include <linux/slab.h>
  15 +#include <linux/sysfs.h>
  16 +#include <linux/fs.h>
  17 +#include <linux/vmalloc.h>
  18 +#include <linux/fcntl.h>
  19 +#include <linux/kobject.h>
  20 +#include <asm/uaccess.h>
  21 +#include <asm/opal.h>
  22 +
  23 +struct elog_obj {
  24 + struct kobject kobj;
  25 + struct bin_attribute raw_attr;
  26 + uint64_t id;
  27 + uint64_t type;
  28 + size_t size;
  29 + char *buffer;
  30 +};
  31 +#define to_elog_obj(x) container_of(x, struct elog_obj, kobj)
  32 +
  33 +struct elog_attribute {
  34 + struct attribute attr;
  35 + ssize_t (*show)(struct elog_obj *elog, struct elog_attribute *attr,
  36 + char *buf);
  37 + ssize_t (*store)(struct elog_obj *elog, struct elog_attribute *attr,
  38 + const char *buf, size_t count);
  39 +};
  40 +#define to_elog_attr(x) container_of(x, struct elog_attribute, attr)
  41 +
  42 +static ssize_t elog_id_show(struct elog_obj *elog_obj,
  43 + struct elog_attribute *attr,
  44 + char *buf)
  45 +{
  46 + return sprintf(buf, "0x%llx\n", elog_obj->id);
  47 +}
  48 +
  49 +static const char *elog_type_to_string(uint64_t type)
  50 +{
  51 + switch (type) {
  52 + case 0: return "PEL";
  53 + default: return "unknown";
  54 + }
  55 +}
  56 +
  57 +static ssize_t elog_type_show(struct elog_obj *elog_obj,
  58 + struct elog_attribute *attr,
  59 + char *buf)
  60 +{
  61 + return sprintf(buf, "0x%llx %s\n",
  62 + elog_obj->type,
  63 + elog_type_to_string(elog_obj->type));
  64 +}
  65 +
  66 +static ssize_t elog_ack_show(struct elog_obj *elog_obj,
  67 + struct elog_attribute *attr,
  68 + char *buf)
  69 +{
  70 + return sprintf(buf, "ack - acknowledge log message\n");
  71 +}
  72 +
  73 +static void delay_release_kobj(void *kobj)
  74 +{
  75 + kobject_put((struct kobject *)kobj);
  76 +}
  77 +
  78 +static ssize_t elog_ack_store(struct elog_obj *elog_obj,
  79 + struct elog_attribute *attr,
  80 + const char *buf,
  81 + size_t count)
  82 +{
  83 + opal_send_ack_elog(elog_obj->id);
  84 + sysfs_schedule_callback(&elog_obj->kobj, delay_release_kobj,
  85 + &elog_obj->kobj, THIS_MODULE);
  86 + return count;
  87 +}
  88 +
  89 +static struct elog_attribute id_attribute =
  90 + __ATTR(id, 0666, elog_id_show, NULL);
  91 +static struct elog_attribute type_attribute =
  92 + __ATTR(type, 0666, elog_type_show, NULL);
  93 +static struct elog_attribute ack_attribute =
  94 + __ATTR(acknowledge, 0660, elog_ack_show, elog_ack_store);
  95 +
  96 +static struct kset *elog_kset;
  97 +
  98 +static ssize_t elog_attr_show(struct kobject *kobj,
  99 + struct attribute *attr,
  100 + char *buf)
  101 +{
  102 + struct elog_attribute *attribute;
  103 + struct elog_obj *elog;
  104 +
  105 + attribute = to_elog_attr(attr);
  106 + elog = to_elog_obj(kobj);
  107 +
  108 + if (!attribute->show)
  109 + return -EIO;
  110 +
  111 + return attribute->show(elog, attribute, buf);
  112 +}
  113 +
  114 +static ssize_t elog_attr_store(struct kobject *kobj,
  115 + struct attribute *attr,
  116 + const char *buf, size_t len)
  117 +{
  118 + struct elog_attribute *attribute;
  119 + struct elog_obj *elog;
  120 +
  121 + attribute = to_elog_attr(attr);
  122 + elog = to_elog_obj(kobj);
  123 +
  124 + if (!attribute->store)
  125 + return -EIO;
  126 +
  127 + return attribute->store(elog, attribute, buf, len);
  128 +}
  129 +
  130 +static const struct sysfs_ops elog_sysfs_ops = {
  131 + .show = elog_attr_show,
  132 + .store = elog_attr_store,
  133 +};
  134 +
  135 +static void elog_release(struct kobject *kobj)
  136 +{
  137 + struct elog_obj *elog;
  138 +
  139 + elog = to_elog_obj(kobj);
  140 + kfree(elog->buffer);
  141 + kfree(elog);
  142 +}
  143 +
  144 +static struct attribute *elog_default_attrs[] = {
  145 + &id_attribute.attr,
  146 + &type_attribute.attr,
  147 + &ack_attribute.attr,
  148 + NULL,
  149 +};
  150 +
  151 +static struct kobj_type elog_ktype = {
  152 + .sysfs_ops = &elog_sysfs_ops,
  153 + .release = &elog_release,
  154 + .default_attrs = elog_default_attrs,
  155 +};
  156 +
  157 +/* Maximum size of a single log on FSP is 16KB */
  158 +#define OPAL_MAX_ERRLOG_SIZE 16384
  159 +
  160 +static ssize_t raw_attr_read(struct file *filep, struct kobject *kobj,
  161 + struct bin_attribute *bin_attr,
  162 + char *buffer, loff_t pos, size_t count)
  163 +{
  164 + int opal_rc;
  165 +
  166 + struct elog_obj *elog = to_elog_obj(kobj);
  167 +
  168 + /* We may have had an error reading before, so let's retry */
  169 + if (!elog->buffer) {
  170 + elog->buffer = kzalloc(elog->size, GFP_KERNEL);
  171 + if (!elog->buffer)
  172 + return -EIO;
  173 +
  174 + opal_rc = opal_read_elog(__pa(elog->buffer),
  175 + elog->size, elog->id);
  176 + if (opal_rc != OPAL_SUCCESS) {
  177 + pr_err("ELOG: log read failed for log-id=%llx\n",
  178 + elog->id);
  179 + kfree(elog->buffer);
  180 + elog->buffer = NULL;
  181 + return -EIO;
  182 + }
  183 + }
  184 +
  185 + memcpy(buffer, elog->buffer + pos, count);
  186 +
  187 + return count;
  188 +}
  189 +
  190 +static struct elog_obj *create_elog_obj(uint64_t id, size_t size, uint64_t type)
  191 +{
  192 + struct elog_obj *elog;
  193 + int rc;
  194 +
  195 + elog = kzalloc(sizeof(*elog), GFP_KERNEL);
  196 + if (!elog)
  197 + return NULL;
  198 +
  199 + elog->kobj.kset = elog_kset;
  200 +
  201 + kobject_init(&elog->kobj, &elog_ktype);
  202 +
  203 + sysfs_bin_attr_init(&elog->raw_attr);
  204 +
  205 + elog->raw_attr.attr.name = "raw";
  206 + elog->raw_attr.attr.mode = 0400;
  207 + elog->raw_attr.size = size;
  208 + elog->raw_attr.read = raw_attr_read;
  209 +
  210 + elog->id = id;
  211 + elog->size = size;
  212 + elog->type = type;
  213 +
  214 + elog->buffer = kzalloc(elog->size, GFP_KERNEL);
  215 +
  216 + if (elog->buffer) {
  217 + rc = opal_read_elog(__pa(elog->buffer),
  218 + elog->size, elog->id);
  219 + if (rc != OPAL_SUCCESS) {
  220 + pr_err("ELOG: log read failed for log-id=%llx\n",
  221 + elog->id);
  222 + kfree(elog->buffer);
  223 + elog->buffer = NULL;
  224 + }
  225 + }
  226 +
  227 + rc = kobject_add(&elog->kobj, NULL, "0x%llx", id);
  228 + if (rc) {
  229 + kobject_put(&elog->kobj);
  230 + return NULL;
  231 + }
  232 +
  233 + rc = sysfs_create_bin_file(&elog->kobj, &elog->raw_attr);
  234 + if (rc) {
  235 + kobject_put(&elog->kobj);
  236 + return NULL;
  237 + }
  238 +
  239 + kobject_uevent(&elog->kobj, KOBJ_ADD);
  240 +
  241 + return elog;
  242 +}
  243 +
  244 +static void elog_work_fn(struct work_struct *work)
  245 +{
  246 + size_t elog_size;
  247 + uint64_t log_id;
  248 + uint64_t elog_type;
  249 + int rc;
  250 + char name[2+16+1];
  251 +
  252 + rc = opal_get_elog_size(&log_id, &elog_size, &elog_type);
  253 + if (rc != OPAL_SUCCESS) {
  254 + pr_err("ELOG: Opal log read failed\n");
  255 + return;
  256 + }
  257 +
  258 + BUG_ON(elog_size > OPAL_MAX_ERRLOG_SIZE);
  259 +
  260 + if (elog_size >= OPAL_MAX_ERRLOG_SIZE)
  261 + elog_size = OPAL_MAX_ERRLOG_SIZE;
  262 +
  263 + sprintf(name, "0x%llx", log_id);
  264 +
  265 + /* we may get notified twice, let's handle
  266 + * that gracefully and not create two conflicting
  267 + * entries.
  268 + */
  269 + if (kset_find_obj(elog_kset, name))
  270 + return;
  271 +
  272 + create_elog_obj(log_id, elog_size, elog_type);
  273 +}
  274 +
  275 +static DECLARE_WORK(elog_work, elog_work_fn);
  276 +
  277 +static int elog_event(struct notifier_block *nb,
  278 + unsigned long events, void *change)
  279 +{
  280 + /* check for error log event */
  281 + if (events & OPAL_EVENT_ERROR_LOG_AVAIL)
  282 + schedule_work(&elog_work);
  283 + return 0;
  284 +}
  285 +
  286 +static struct notifier_block elog_nb = {
  287 + .notifier_call = elog_event,
  288 + .next = NULL,
  289 + .priority = 0
  290 +};
  291 +
  292 +int __init opal_elog_init(void)
  293 +{
  294 + int rc = 0;
  295 +
  296 + elog_kset = kset_create_and_add("elog", NULL, opal_kobj);
  297 + if (!elog_kset) {
  298 + pr_warn("%s: failed to create elog kset\n", __func__);
  299 + return -1;
  300 + }
  301 +
  302 + rc = opal_notifier_register(&elog_nb);
  303 + if (rc) {
  304 + pr_err("%s: Can't register OPAL event notifier (%d)\n",
  305 + __func__, rc);
  306 + return rc;
  307 + }
  308 +
  309 + /* We are now ready to pull error logs from opal. */
  310 + opal_resend_pending_logs();
  311 +
  312 + return 0;
  313 +}
arch/powerpc/platforms/powernv/opal-wrappers.S
... ... @@ -123,6 +123,11 @@
123 123 OPAL_CALL(opal_lpc_read, OPAL_LPC_READ);
124 124 OPAL_CALL(opal_lpc_write, OPAL_LPC_WRITE);
125 125 OPAL_CALL(opal_return_cpu, OPAL_RETURN_CPU);
  126 +OPAL_CALL(opal_read_elog, OPAL_ELOG_READ);
  127 +OPAL_CALL(opal_send_ack_elog, OPAL_ELOG_ACK);
  128 +OPAL_CALL(opal_get_elog_size, OPAL_ELOG_SIZE);
  129 +OPAL_CALL(opal_resend_pending_logs, OPAL_ELOG_RESEND);
  130 +OPAL_CALL(opal_write_elog, OPAL_ELOG_WRITE);
126 131 OPAL_CALL(opal_validate_flash, OPAL_FLASH_VALIDATE);
127 132 OPAL_CALL(opal_manage_flash, OPAL_FLASH_MANAGE);
128 133 OPAL_CALL(opal_update_flash, OPAL_FLASH_UPDATE);
arch/powerpc/platforms/powernv/opal.c
... ... @@ -566,6 +566,8 @@
566 566 /* Create "opal" kobject under /sys/firmware */
567 567 rc = opal_sysfs_init();
568 568 if (rc == 0) {
  569 + /* Setup error log interface */
  570 + rc = opal_elog_init();
569 571 /* Setup code update interface */
570 572 opal_flash_init();
571 573 }