User:Knuxify/Enabling pstore and ramoops

🚧	This page is a work-in-progress. Some information contained within may be inaccurate or incomplete.

If you don't have a serial cable, and your device's downstream kernel does not support the android ramconsole driver, you might need to set up pstore and ramoops instead.

Information

pstore

pstore is a persistent storage driver. It saves data in a reserved part of memory, which can then be read from a working kernel. Its primary purpose is to save kernel crash logs to memory.

ramoops

From the documentation:

Ramoops is an oops/panic logger that writes its logs to RAM before the system crashes. It works by logging oopses and panics in a circular buffer. Ramoops needs a system with persistent RAM so that the content of that area can survive after a restart.

pstore/ramoops setup

This setup has to be done on both the downstream and mainline kernel.

Kernel configuration

To enable pstore and use the ramoops backend, make sure the following kernel options are set:

CONFIG_PSTORE=y
CONFIG_PSTORE_CONSOLE=y
CONFIG_PSTORE_RAM=y

They can be found under File systems -> Miscellaneous filesystems -> Persistent store support.

On older kernels without CONFIG_PSTORE_CONSOLE and CONFIG_PSTORE_RAM, you'll need to enable

CONFIG_RAMOOPS=y

(found under Device drivers -> Character devices -> Log panic/oops to a RAM buffer) instead of CONFIG_PSTORE_RAM. Note that this will only allow you to preview kernel oopses/panics. If you need to get full kernel logs, you can either set up the ram_console driver (see "Android ram_console driver setup" section below) or backport CONFIG_PSTORE_RAM (see patches here).

Configuring ramoops

Follow the guide in the ramoops documentation (you only need to do one of the steps). Make sure that the memory address you reserve and its size is the same in your downstream and mainline configuration, or else you won't be able to read the information.

Note: the documentation for the cmdline settings doesn't mention the fact that you have to set ramoops.mem_size as well. This value contains the size, so it should be set to something like 0x00004000 (that's 16kb).

Reading from pstore

Now you can mount the pstore partition:

mkdir /tmp/pstore
mount -t pstore -o kmsg_bytes=16000 - /tmp/pstore

Android ram_console driver setup

For devices with older downstream kernels without PSTORE_CONSOLE (kernels below v3.6(-rc1)) it may be easier to set up the ram_console driver.

This setup has to be done on both the downstream and mainline kernel.

Backporting the driver

Copy the driver from the p4wifi tree.

Kernel configuration

Make sure CONFIG_ANDROID_RAM_CONSOLE is set to =y.

Configuring ram_console

This process will differ depending on your downstream kernel version.

If you backported the driver from p4wifi or your downstream kernel had the same driver

If you didn't have the ram_console driver because the kernel was too new (>3.6) and you copied it from the p4wifi tree, or if your downstream kernel was old enough to contain that version of the driver, follow the steps below.

In downstream, define the ram_console platform device:

static struct resource ram_console_resources[] = {
	{
		.flags = IORESOURCE_MEM,
	},
};

static struct platform_device ram_console_device = {
	.name 		= "ram_console",
	.id 		= -1,
	.num_resources	= ARRAY_SIZE(ram_console_resources),
	.resource	= ram_console_resources,
};
void __init ram_console_debug_init_mem(unsigned long start, unsigned long size)
{
	struct resource *res;

	res = platform_get_resource(&ram_console_device, IORESOURCE_MEM, 0);
	if (!res)
		goto fail;
	res->start = start;
	res->end = res->start + size - 1;

	return;

fail:
	ram_console_device.resource = NULL;
	ram_console_device.num_resources = 0;
	pr_err("Failed to reserve memory block for ram console\n");
}

void __init ram_console_debug_init(void)
{
	int err;

	err = platform_device_register(&ram_console_device);
	if (err) {
		pr_err("%s: ram console registration failed (%d)!\n", __func__, err);
	}
}

Then, find the init function of your device (usually near the end of the file) and add the following lines:

ram_console_debug_init_mem(0x2E600000, 0x00100000);
ram_console_debug_init();

Where 0x2E600000 is the base address and 0x00100000 is the size of the buffer.

In upstream, reserve memory in the device tree:

	reserved-memory {
		#address-cells = <1>;
		#size-cells = <1>;
		ranges;

		android_logger: ram-console@2E600000 {
			reg = <0x2E600000 0x00100000>;
			no-map;
		}
	}

Then in your board.c file (could be your chipset's, but only temporarily) define the ram_console platform device as outlined above.

If your board.c file does not have an init function, add one:

void __init chipset_dt_early_init (void)
{
	...
}
DT_MACHINE_START(...)
	.init_early = chipset_dt_early_init

TODO: Is that correct?

If you had separate ram_console and persistent_ram drivers

The setup process is different for kernels with separate persistent_ram and ram_console drivers. This applies to kernel 3.4.

TODO: What other versions does this apply to?

In downstream, add the following to the includes of your device's board.c file (skip the ones you have already):

#include <linux/platform_device.h>
#ifdef CONFIG_ANDROID_PERSISTENT_RAM
#include <linux/persistent_ram.h>
#endif

Then define the initialization functions:

#ifdef CONFIG_ANDROID_RAM_CONSOLE
#define RAM_CONSOLE_SIZE 0x100000
static struct platform_device ram_console_device = {
    .name = "ram_console",
    .id = -1,
};
#endif

#ifdef CONFIG_ANDROID_PERSISTENT_RAM
#define PERSISTENT_RAM_SIZE 0x100000
static struct persistent_ram_descriptor pram_descs[] = {
#ifdef CONFIG_ANDROID_RAM_CONSOLE
    {
        .name = "ram_console",
        .size = RAM_CONSOLE_SIZE,
    },
#endif
};

static struct persistent_ram persist_ram = {
    .size = PERSISTENT_RAM_SIZE,
    .num_descs = ARRAY_SIZE(pram_descs),
    .descs = pram_descs,
};

static void __init add_persist_ram_devices(void)
{
    int ret;
    persist_ram.start = 0x2A600000;

    pr_info("PERSIST RAM CONSOLE START ADDR : 0x%x\n",
            persist_ram.start);

    ret = persistent_ram_early_init(&persist_ram);
    if (ret)
        pr_err("%s: failed to initialize persistent ram\n", __func__);
}
#endif /* CONFIG_ANDROID_PERSISTENT_RAM */

Replace 0x100000 with the ram console size, and 0x2A600000 with the address at which you want to reserve the ram console.

Then, find your reserve function (look for .reserve = ... around the end of the board file). If it doesn't exist, define it and add it.

To your reserve function, add the following:

#if defined(CONFIG_ANDROID_PERSISTENT_RAM)
    add_persist_ram_devices();
#endif

Then add the following to your init function:

#ifdef CONFIG_ANDROID_RAM_CONSOLE
    platform_device_register(&ram_console_device);
#endif

Then follow the upstream setup section from the previous section.

TODO: Does this work?

Reading from the logs

In your downstream kernel, do:

cat /proc/last_kmsg

Troubleshooting

I set my ramoops parameters in my kernel cmdline and it doesn't work!

If you added the ramoops parameters to CONFIG_CMDLINE and they seem to be ignored, and there's a similar message in the dmesg:

~ # dmesg | grep ramoops
<6>[    3.009338] C0 [      swapper/0] ramoops: platform device not found, using module parameters
<3>[    3.009735] C0 [      swapper/0] ramoops: The memory size and the record size must be non-zero
<4>[    3.009765] C0 [      swapper/0] ramoops: probe of ramoops failed with error -22

check if you don't have CONFIG_CMDLINE_FROM_BOOTLOADER set. If you do, set CONFIG_CMDLINE_EXTEND to =y.

You're usually able to check the used cmdline at the beggining of the dmesg.

~ # dmesg | grep "Kernel command line"
<5>[    0.000000] C0 [        swapper] Kernel command line: ...

See also

• ramoops documentation
• pstore documentation