Linux Framebuffer Drivers

Categories: Linux

Intro

The simplest code-path for displaying graphics on a Linux system is by using a “framebuffer driver”. This article describes how a Linux framebuffer driver works.

The article here is for (beginner) kernel developers, or those just curious how linux graphics work. It might be a little interesting for linux sysadmins as background info. And a total waste of time for general users, or even userspace application developers.

For information about the whole linux graphics stack, see this article.

Accessing Graphics

There are 3 different ways that graphics can be performed:

  • Using a kernel framebuffer driver
  • Using a kernel DRM driver
  • Mapping the card’s control registers into user-space (via mmap() calls on /dev/mem), and using user-space code to access the card (very complex)

What is a Framebuffer Driver?

It is a Linux kernel driver which exposes a file /dev/fb{N} for a graphics card it is responsible for. Through that file, userspace apps can perform reads/writes to directly access the video card framebuffer - ie the pixel values being displayed on the screen. Via IOCTL calls on that file, other functionality can be accessed including:

  • setting the graphics mode (width, height, pixel-depth, etc)
  • passing bitmaps to be copied into video memory
  • passing coordinates of a rectangle to be filled with a specified colour

The framebuffer interface does not offer much in the way of acceleration other than the bitblits and fills mentioned above.

There is a fairly generic “vesafb” driver that is capable of controlling any graphics card compatible with the VESA standard. Linux also comes with dozens of card-specific framebuffer drivers that support products of specific manufacturers. On boot, Linux uses the PCI ID of a graphics card to load the most appropriate framebuffer driver, which in turn creates the /dev/fb{n} control file.

Why Use a Framebuffer Driver?

Embedded systems use them because they often don’t have complex graphics requirements, want a small kernel, and don’t want an X server running in userspace. While X can use a framebuffer to display graphics, so can other simpler graphical libraries.

For many graphics chips, there is a framebuffer driver available but no specific X driver, and no 3D driver.

Graphics display during startup is also a candidate for a framebuffer driver.

And linux text consoles can use a framebuffer driver to display text at nice resolutions (better than available via the BIOS) without otherwise actually wanting to display graphics.

Framebuffers and DRM/KMS

The was originally a dedicated kernel framebuffer driver module for most cards; this driver would provide the /dev/fb* node used by userspace to interact with this driver.

For many cards, there is now a “DRM” (Direct Rendering Manager) driver which provides “KMS” (Kernel Modesetting) functionality, supports the framebuffer interface, and optionally provides interfaces needed to do accelerated 3D rendering (GEM/TTM and other features). When the DRM kernel driver for a card is loaded, it provides the /dev/fb* node - ie opening that file creates a channel to the multi-purpose DRM driver rather than the single-purpose framebuffer driver; this makes coordination between multiple users of the graphics card (X servers, and direct framebuffer users such as virtual terminals) better. However in many cases the standalone framebuffer driver code is still maintained, for cases where the complexity of a DRM driver is not wanted (eg embedded systems).

The X Server and the Framebuffer

The X server predates the concept of a Linux kernel framebuffer driver, ie the original X usermode graphics drivers directly implement their own modesetting and access to the card framebuffer memory. In addition, the framebuffer interface is a Linux-specific concept while X runs on multiple operating systems.

The userspace-based approach to accessing graphics was used by the X server for many decades; few other applications have tried this as (a) maintaining the necessary card-specific code is very complex, (b) the application must run as ‘root’ to have the right to use /dev/mem, and (c) integration with other users of the graphics card is tricky. As an example of (c), the boot console and virtual consoles (ie those accessed via ctrl-alt-F{n}) usually use a framebuffer driver. To solve (c), a tricky dance is needed to keep things working. The framebuffer driver configures graphics when it is loaded (shortly after boot) and provides the traditional text bootup messages (may be hidden when using a modern distro that uses something like Plymouth to display a pretty bootsplash screen by default). When X starts, its drivers save the current graphics state, then start directly poking the graphics control registers to set things up the way it wants. Obviously, if anything tries to then use the framebuffer driver then things will get ugly as X has changed settings “behind the back” of the framebuffer driver; however this normally doesn’t happen. If the user presses the magic ctrl-alt-F{n} sequence to switch to a text console, then X detects this and restores the original saved graphics state before allowing the (framebuffer-based) text console to run. When the user then presses ctrl-alt-f{n} to switch back to X, X needs to reinstate its graphics settings. These reinitialisations of the graphics card cause switching to be slow, and the screen to “flicker”.

The X server on Linux now mostly uses user-space drivers which rely on a DRM driver that provides KMS; as the same driver also provides the framebuffer interface that driver has all the knowledge needed to correctly arbitrate between X and framebuffer users, and the above problem is resolved. The same driver handles both APIs, and therefore knows what modes are selected regardless of which API was used. Switching between framebuffer-using applications (such as a text console) and X applications then does not need to reinitialise the graphics (as no sneaky changes occur behind the back of the driver).

The X server can alternatively use a generic userspace driver that relies on the framebuffer API, rather than a card-specific driver. X then relies on the framebuffer to do modesettings, and does all rendering in software; it then passes the rendered screen to the graphics card using either write() or the bitblit IOCTL. In this mode, X can’t do much acceleration.

Linux Driver Source

Directory drivers/video holds a implementations of the framebuffer interface for a truckload of different chips. Among them are some generic drivers:

  • “vesafb.c” can drive any VESA2.x compliant chip
  • “vga16.fb” can drive any VGA-compliant chip
  • “xen-fbfront” can be used by virtual machines running on Xen hypervisor; presumably framebuffer graphics calls get routed to the Xen hypervisor which then allows the administrator to display that appropriately.

There are also drivers for playstation3 graphics, atari onboard graphics, cirrus, sparc, trident, etc.

The vesa driver is one of the simplest drivers, and below we will walk through some of the internal implementation.

VesaFB

File drivers/video/vesafb declares module_init(vesafb_init), ie an init function which is executed on module load. This:

  • Parses any driver options provided by mod_probe

  • Invokes platform_probe which calls back into vesafb_probe(platform_device). The “platform” part is because VESA is accessed via the BIOS; a framebuffer driver for a PCI card would instead register itself with the PCI framework, which then calls back into the device’s probe function passing a struct pci_dev.

  • Invokes request_mem_region which checks to see whether any other driver has already reserved the specified hardware memory address range (fixed in the case of VESA, more dynamic for PCI); if not, then the requested range is registered as “reserved”. See macro include/linux.ioport.h:request_mem_region -> kernel/resource.c:__request_region -> __request_resource.

  • allocates a framebuffer_info structure (see fbsysfs:framebuffer_alloc) followed by N bytes of buffer for the color palette values (info->par).

  • Invokes request_region to try to reserve the historical range of I/O address ports for VESA. The driver doesn’t actually care if it fails, as it doesn’t use them. However it’s elegant to mark them, as this info appears in the sysfs structure representing IO ports, and traditionally VESA has this specific range.

  • Optionally sets up MTRRs (Memory Type Range Registers) to mark the hardware address range region (reserved above by request_mem_region) as having the appropriate cacheing behaviour. Appears to be somewhat card or system dependent, so driver parameters are available to tweak it. Defaults to “write-through”, ie the slowest but safest setting for volatile memory accesses.

  • uses ioremap function to create a kernel virtual address mapping for the hardware address of the graphics framebuffer (with appropriate caching flags set).

  • stores an fbops (“framebuffer operations”) structure into the framebuffer_info, containing pointers to callback functions. The callbacks are used from the generic framebuffer support code; in particular, many of these get invoked as the result of a user IOCTL call. Callbacks include:

    • imageblit // accelerate copying images from userspace to graphics framebuffer
    • copyarea // copy data around within the existing framebuffer
    • fillrect // fill a rectangular area in the framebuffer with a specific colour (eg blank the screen!)

  • calls infrastructure function fbmem.c:register_framebuffer() to register the /dev/fb{n} file, hook up write and IOCTL methods etc.

FBMem module

Several files in this directory (fbmem.c, fb_sys_fops.c, fbsysfs.c) are common code that are compiled into their own kernel module. This module doesn’t expose functionality to the user itself, just acts as “library code” for other drivers. Although it is a module, it is usually statically linked into the kernel, and therefore does not appear in the “modules.dep” dependency list for the vesafb.ko module even though vesafb.c invokes functions exported from these files via EXPORT_SYMBOL macros.

Module Init

File fbmem has module_init(fbmem_init) which:

  • defines an array of registered framebuffers (registered_fb) - initially empty.

  • registers file /proc/fb - which can be read to determine what framebuffer is currently “active”

  • reserves major device# FB_MAJOR for later use, but doesn’t create any /dev nodes yet. It also associates a file_operations structure with that major device#; the major callbacks are:
    • syscall open –> fb_open
    • syscall ioctl –> fb_ioctl
    • syscall mmap –> fb_mmap
    • syscall write –> fb_write
    • syscall read –> fb_read
  • uses class_create to create sysfs entry /sys/class/graphics. When drivers call framebuffer_register, then a child sysfs node is created which automatically creates a symlink under /sys/class/graphics.

The FB_MAJOR major device number is used when creating /dev/fb{n} nodes later in calls to the register_framebuffer() function; those nodes then inherit the settings of the parent major device - ie when the user performs open/read/write/mmap/ioctl on the /dev/fb{n} node, the handler functions for the major device# are invoked. They then delegate to the callback functions in the framebuffer_info->fbops structure if defined, otherwise perform the default behaviour.

The default fb_ioctl function allows the user to:

  • get the current graphics mode settings (screen resolution etc) [FBIOGET_VSCREENINFO]
  • request the current graphics mode settings be changed [FBIOPUT_VSCREENINFO]
  • read/write the color palette [FBIOGETCMAP/FBIOPUTCMAP]
  • read/modify which virtual consoles are on which framebuffer devices [FBIOGET_CON2FBMAP/FBIOPUT_CON2FBMAP]

The default fb_mmap function uses io_remap_pfn_range to map the graphics framebuffer into the virtual memory space of the calling userspace application, and returns the userspace address at which the data has been mapped. The userspace app can then use normal reads/writes to access the graphics framebuffer.

The default fb_write function just copies data from userspace to the graphics framebuffer (ie assumes caller has the layout correct). This implementation uses an intermediate buffer, repeatedly copying a page worth of data from userspace into the buffer, then from there to the real framebuffer using include/linux/fb.h:fb_memcpy_tofb which maps to memcpy_io (for x86). Many drivers override this operation to instead use common function fb_sys_fops.c:fb_sys_write which does the copy in one go.

The default fb_read function just copies data from the graphics framebuffer to userspace.

So in short, for video devices which have a fairly normal framebuffer layout, userspace just uses FBIOGET_VSCREENINFO to figure out what the layout is, then generates an appropriate image and does standard file-writes against the /dev/fd{N} file. These map to direct copies into the videocard memory. Alternatively, userspace can use mmap() to get access to the framebuffer addesses and then just memcpy its prepared pixel data into the address range returned by the mmap call. For video devices with odd IO-mappings or odd framebuffer layouts, the fbops->fb_write callback can be overridden to manually handle userspace->videocard data translation; presumably such drivers provide a custom mmap callback that simply rejects mmap() calls (return an error).

Modesetting for framebuffer devices is done via an IOCTL of type FBIOPUT_VSCREENINFO. As noted above, “KMS” drivers include framebuffer functionality.

The “fbset” commandline tool uses these IOCTLs to display/update the current graphics mode. Note that the “randr” tool is a front-end for X, not the framebuffer - it just sends a network message to X, which then asks its current driver to set the selected mode.

The strucures and constants used in IOCTL calls from userspace are defined in include/linux/fb.h.

The core filebuffer code also broadcasts “notifications” when certain significant changes occur to a framebuffer; other code can register as a listener for these events. In particular, this is used by the framebuffer-console code (described later).

File fbmem.c also has quite a lot of code for displaying the penguin “logo” on the boot console (which is usually a framebuffer console). The logo is normally embedded in the kernel image. Code exists to show muliple copies of it (which indicate #CPUs), and to show it rotated 90 or 180 degrees.

A remarkable amount of code is dedicated to this feature…

Registering a Framebuffer

During initialisation, a framebuffer driver (eg vesafb) will call fbmem.c:register_framebuffer. This:

  • creates /dev/fb{N} using FB_MAJOR (as registered by fbmem_init). This device therefore inherits the file_operations callbacks defined for the major#.

  • stores a pointer to the registered driver in the registered_fb array (and increments num_registered_fb)

  • allocates an 8bk block of ram for “pixmap”

  • calls fb_init_device to create a sysfs entry with lots of control functions..

File fb_sys_fops

This file provides convenient implementations for read/write syscalls on the /dev/fb{N} operation, which simply copies bytes between userspace buffers and the memory-mapped video buffer set up by the framebuffer driver. It presumably assumes that the user data is in the appropriate format already.

As noted above, there is a default implementation of read/write/etc in file fbmem.c. These alternate implementations can be used by a driver simply by setting the appropriate member in its ops structure (which the fbmem code checks before calling its default).

File cfbimgblt

This file provides implemenations for copying images from userspace to video memory, with optional format changes (eg colorspace). Takes an fb_image parameter as the source.

Sysfs interface

fb_init_device (fbsysfs.c) creates an entry in sysfs, with control files for things like:

  • show_bpp/store_bpp
  • show_blank/store_blank
  • show_console/store_console
  • show_cursor/store_cursor
  • show_mode/store_mode
  • show_modes/store_modes
  • show_pan/store_pan
  • show_virtual/store_virtual
  • show_name, show_stride
  • show_rotate/store_rotate (screen rotation, eg 90%)
  • show_fbstate/store_fbstate
  • show_bl_curve/store_bl_curve (backlight for LED screens)

The other functions in this file handle reads or writes of those sysfs files.

The Framebuffer Console Driver

Linux has the concept of a “tty driver” for text input and output. The implementation of various TTY devices can be found in drivers/tty. In particular, drivers/tty/vt defines the concept of “virtual terminals” : having multiple concurrent sessions on a single computer, and switching between them using magic key sequences.

The interface that a kernel module must implement in order to register itself as a TTY is defined in include/linux/console.h:struct consw.

File drivers/video/console/fbcon.c implements a “text console interface” API on top of a framebuffer device. Actually, it supports up to NR_CONSOLES independent consoles which might be mapped to the same or different framebuffer devices. In effect, it implements the “consw” interface defined for /dev/tty{N}, and uses a framebuffer device as the output medium. Type “struct consw” is declared in include/linux/console.h and defines an API like:

  • con_init(..) // initialise a caller-provided datastructure as console#N
  • con_putc(..) // write a single character at the current cursor location
  • con_scroll(..) // scroll a section of the screen by N lines

It creates a sysfs entry for fbcon, and adds sysfs entries so user can query and update some graphical output stuff like screen rotation or cursor size. User writes to these sysfs files are equivalent to IOCTL calls on the framebuffer device node itself - but more convenient. There is only one entry for all the fbcon consoles; properties are bound to the “currently active” fbcon console.

Useful sysfs nodes include:

  • /sys/devices/graphics/fbcon
  • /sys/devices/virtual/graphics/fbcon

The fbcon module attaches itself as a listener on framebuffer events, in order to make sure it keeps “in sync” with changes to framebuffers, eg resolution changes.

Framebuffer “instances” and console “instances” have only a loose binding; there is effectively a matrix that specifies which console is to be displayed on which framebuffer. Array con2fbmap holds this data, ie indicates for console#N which framebuffer# it is attached to, eg con2fbmap[3]=1 means that console#3 is on framebuffer#1. A value of -1 means “not attached to any framebuffer”. On start, the first-registered framebuffer is considered the “primary”, and all possible consoles (NR_CONSOLES) are mapped to this framebuffer. Of course, most machines only have one framebuffer (actually, max# of consoles can be reduced via module parameter “vc” to less than NR_CONSOLES).

It is the TTY system that creates the /dev/tty{n} files, not fbcon. The fbcon code then “takes over” responsibility for consoles using function drivers/tty/vt/vt.c:take_over_console. Any further IOCTL operations on this file are then redirected to the “struct consw” hooks defined by fbcon.

Userspace Tools

The fbset command can be used to change graphics modes via IOCTL operation FBIOPUT_VSCREENINFO on /dev/fb{n} files.

Other Notes

The memory buffer that the graphics card scans repeatedly in order to generate signals to the display (the framebuffer) is also known as the “scan out buffer”.

References

See also: