The Kernel Kobject device model explained.
Nayan GadreBlockedUnblockFollowFollowingJan 22(PS: This is a consolidation attempt for Kobject infrastructure in the kernel.
Relevant links mentioned below)Index:Introductiona.
How reference counting in kobjects works.
b.
Ksets.
c.
Subsystems.
The Block subsystem example with code snippets.
Summary of rules of kobject linkagesIntroduction:When I first started learning/working on linux kernel projects I came across few concepts (which were quite involved mainly due to lack of documentation and tutorial material which I was accustomed to while working as a dot net developer).
The first time I stumbled upon the kernel kobject style of architecture was while working on device drivers, particularly the sysfs pseudo file system entry creation, USB/PCI bus enumeration.
For eg: The device driver model introduced in kernel version 2.
5 and >.
The new device model was inspired by the power management requirements of devices in Linux.
The dependency of devices upon other devices for power can be expressed in a hierarchical fashion, which is what the device driver model achieves.
For eg: The PCI bridge is a device in linux, all the other child PCI devices depend on PCI bridge device for power.
This dependency could be expressed using the device model.
However power management has further more complex dependencies mentioned in this paper [5] which need a appropriately architected design to serve the purpose, the device model with the kobject, kset, ktype infra does exactly that.
The device model provides a means to generically represent and operate on every device on the computer.
Remembering the Object oriented concept of an abstract class, whose objects don’t exist standalone but are rather implemented in other classes whose responsibility is to implement the behaviors of the routines in the abstract class, a similar approach is used for implementing the kobject infra.
If you are used to thinking of things in object-oriented terms, kobjects can be seen as a top-level, abstract class from which other classes are derived.
A kobject implements a set of capabilities which are not particularly useful by themselves, but which are nice to have in other objects.
The C language does not allow for the direct expression of inheritance, so other techniques — such as structure embedding — must be used.
So, if you have your own object, may be a device object or driver object or a bus(representing a hardware bus) object, and if you embedded a specific kobject with a specific type and may be belonging to a set of kobject types, then your specific device, driver or bus object’s seamlessly integrate into the Linux kernel device model.
The sysfs hierarchical representation of the device modelInfact, the linux kernel exposes all the devices, drivers, buses in the exact hierarchy in which it exists physically in the system through sysfs, and this hierarchy and grouping of similar types of objects (device, drivers, buses) is made possible through the kobject infra.
Linux kernel comprises of many subsystems foreg: usb bus subsystem, PCI subsystem, character devices, block subsystem etc.
A subsystem represents a set of kobjects.
A kobject may belong to only one subsystem and a subsystem must contain only identically embedded kobjects.
When a kobject is initialized and inserted into the subsystems list of kobjects, a corresponding sysfs directory for kobject is also created.
Kobjects have reference count and get() and put() to adjust the count.
How reference counting in kobjects works:Given that the sysfs entry for an kobject associated physical entity (device, driver, bus) is exposed as a file in the linux virtual filesystem, potential race conditions exist where the kobject is removed since a device got unloaded while the user space had opened the file for read/write.
When a directory is created for a kobject,a pointer to the kobject is stored in the d_fsdata field of the dentry of the directory.
When an attribute file is created, a pointer to the attribute is stored in the d_fsdata field of the file.
When an attribute file is opened, the reference count on the kobject is incremented.
When the file is closed, the reference count of the kobject is decremented.
This prevents the kobject, and the structure its embedded in, from being removed while the file is open.
When the reference count decrements to 0 the code that created the kobject does not know when this happened, so the code that frees the kobject needs to be informed asynchronously, whenever the last reference to one of its kobjects goes away.
The notification is done through a kboject’s release method.
Every kobject must have a release method and the kobject must persist until that method is called.
Interestingly the release method is not stored in the kobject itself; instead, it is associated with the type of the structure that contains the kobject.
This type is tracked with a structure of type struct kobj_type, often simply called a “ktype”.
Every kobject needs to have an associated kobj_type structure.
Confusingly, the pointer to this structure can be found in two different places.
The kobject structure itself contains a field (called ktype) that can contain this pointer.
If, however, this kobject is a member of a kset, the kobj_type pointer is provided by that kset instead.
Ksets:In many ways, a kset looks like an extension of the kobj_type structure; a kset is a collection of kobjects embedded within structures of the same type.
However, while struct kobj_type concerns itself with the type of an object, struct kset is concerned with aggregation and collection.
The two concepts have been separated so that objects of identical type can appear in distinct sets.
Therefore, the main function of a kset is containment; it can be thought of as the top-level container class for kobjects.
In fact, each kset contains its own kobject internally, and it can, in many ways, be treated the same way as a kobject.
For eg: a kset can be used by the kernel to track “all block devices” or “all PCI device drivers”.
It is worth noting that ksets are always represented in sysfs; once a kset has been set up and added to the system, there will be a sysfs directory for it.
Kobjects do not necessarily show up in sysfs, but every kobject that is a member of a kset is represented there.
KSET, KOBJECT, KTYPEEvery kset contains a kobject which can be set up to be parent of other kobjects, in this way the device model hierarchy is constructed.
Adding a kobject to a kset is usually done when the object is created; it is a two-step process.
The kobject’s kset field must be pointed at the kset of interest; then the kobject should be passed to:int kobject_add(struct kobject *kobj);As always, programmers should be aware that this function can fail (in which case it returns a negative error code) and respond accordingly.
There is a convenience function provided by the kernel:extern int kobject_register(struct kobject *kobj);This function is simply a combination of kobject_init and kobject_add.
When a kobject is passed to kobject_add, its reference count is incremented.
Containment within the kset is, after all, a reference to the object.
At some point, the kobject will probably have to be removed from the kset to clear that reference; that is done with:void kobject_del(struct kobject *kobj);There is also a kobject_unregister function, which is a combination of kobject_del and kobject_put.
A kset keeps its children in a standard kernel linked list.
In almost all cases, the contained kobjects also have pointers to the kset (or, strictly, its embedded kobject) in their parent’s fields.
All of the contained kobjects in the diagram are actually embedded within some other type, possibly even other ksets.
It is not required that a kobject’s parent be the containing kset (although any other organization would be strange and rare).
Kset has a kobject_uevent, this is used to send an event to userspace that something about kobject has changed.
So if one kobject doesn’t belong to no kset, he can’t send such event to userspace .
Hotplug works like so, when one device plug into the system, the kernel can notify the user-space program can load the device’s driver, when it removes, it can remove the driver.
This is possible with the kobject_uevent.
The function 1st finds the topmost kset, then call the filter of kset->uevent_ops, 2nd set environment variables and call uevent_ops->uevent.
Finally, send a netlink uevent message to user-space using netlink, or call call_usermodehelper function to launch a user-process from kernel.
Ktype:Ktype represents the object type, kernel connects the ‘ktype’ with the object’s sysfs file operations and attributes file.
Subsystems:Some example subsystems in the kernel include block_subsys(/sys/block, for block devices), device_subsys(/sys/devices, the core device hierarchy), and a specific subsystem for every bus type known to the kernel.
struct subsystem { struct kset kset; struct rw_semaphore rwsem;};Every kset must belong to a subsystem.
The subsystem membership helps establish the kset’s position in the hierarchy, but, more importantly, the subsystem’s rwsem semaphore is used to serialize access to a kset's internal-linked list.
This membership is represented by the subsys pointer in struct kset.
Thus, one can find each kset's containing subsystem from the kset's structure, but one cannot find the multiple ksets contained in a subsystem directly from the subsystem structure.
The Block subsystem example:This explains with the kobject-kset glue how the block subsystem comprising of devices major (hda, hdb, sda, sdb) and devices minor (hda1, hda2, hdb1, hdb2) is represented hierarchically.
I have referred the 2.
6.
11 kernel and also the LWN article[4] to elaborate with code snippets.
We will start with defining the essential data structures [5]:static struct kobj_type ktype_block = { .
release = disk_release, .
sysfs_ops = &disk_sysfs_ops, .
default_attrs = default_attrs,};The ktype_block ktype will be used to declare a subsystem object as such:/* declare block_subsys.
*/static decl_subsys(block, &ktype_block, &block_hotplug_ops);#define decl_subsys(_name,_type,_hotplug_ops) struct subsystem _name##_subsys = { .
kset = { .
kobj = { .
name = __stringify(_name) }, .
ktype = _type, .
hotplug_ops =_hotplug_ops, } }Later we register the block_subsys, using the subsystem_register(&block_subsys); which call subsystem_init() which initializes the rwsem and init’s the kset.
It calls kset_add() which adds the embedded kobject of the kset in the hierarchy.
Also it sets the kset’s subsystem to this subsystem for correct usage.
block_subsys->kset.
subsys = block_subsys; Also the reference count is incremented for the embedded kobject of the kset of block_subsys.
In this case, the kobject’s parent field is explicitly set to NULL (indicated by the ground symbol in the picture).
As a result, this kobject will be represented in the top level of the sysfs hierarchy; it is the kobject which lurks behind /sys/block.
A block subsystem is not very interesting without disks.
In the block hierarchy, disks are defined by a struct gendisk.
As usual the gendisk also has an embedded kobject.
It is allocated using the alloc_disk function as bellow:The minor here corresponds to the disk partitions, so disk->minors assigns that to gendisk.
We also allocate an array of struct hd_struct pointers which represent individual partition information all whose parent is the gendiskNext, the kobj_set_kset_s(disk, block_subsys) will attach the disk with the block_subsys().
It does this by setting the kset of the kobject within the gendisk structure to point to the kset of the block_subsys subsystem’s kset.
Once you have your gendisk structure set up, you have to add it to the list of active disks; that is done with: add_disk(struct gendisk *disk).
blk_register_region is something to do with reserving dev->minors number of slots into a hash map of block devices that the kernel maintains.
Pertinent to this article is the register_disk(disk).
if ((err = kobject_add(&disk->kobj))) returnif (!parent) parent = kobject_get(&kobj->kset->kobj);list_add_tail(&kobj->entry,&kobj->kset->list);kobject_hotplug(kobj, KOBJ_ADD);It will add the embedded kobj of the disk, to the list of kobjects that its own kset points to.
Then set the parent of its kobj to the subsystem’s kset’s kobject.
Add the kboject into the list of kobjects of the subsystems kset.
Inform the userspace by executing /sbin/hotplug about the action hapenning ADD or DELETE.
A gendisk’s kobject does not have an explicit type pointer; its membership in the block_subsys kset takes care of that.
So now it looks like this:The gendisk we saw contains an array of partition disks, as mentioned earlier.
The add partition method will add and initialize the links between various embedded kobjects and ksets and ktypes together as below:Complete block subsystem device model with sysfs file hierarchyYou would have got the idea of how the objects are linked in general, and what keywords to grep for while exploring any other subsystem for similar hierarchies as well you would have got an idea what objects, types and sets to create to add a new subsystem, or a device type to seamlessly integrate with the device model.
:Summary of rules for kobject, ksets linkages:Every Kobject has a ktype, and all kobjects with same ktype will have the same release() mechanism.
A subsystem represents a set of kobjects.
A kobject may belong to only one subsystem and a subsystem must contain only identically embedded kobjects.
Not all kobjects belong to a kset, hence such kobjects can’t send uevents to userspace.
Every kset must belong to a subsystem.
Each kset has it’s own kobject which means a kset has an associated ktype through the kobject embeddedOne can find each kset’s containing subsystem from the kset struct, but vice-versa is not possible.
Kset’s also have a pointer (in the ktype field) to the kobj_type structure describing the kobjects it contains.
This type will be applied to any kobject which does not contain a pointer to its own kobj_type structure.
Material compiled from:- 1.
Linux Device Drivers, 3rd Edition2.
http://www.
staroceans.
org/kernel-and-driver/The%20Linux%20Kernel%20Driver%20Model.
pdf3.
https://lwn.
net/Articles/51437/4.
https://lwn.
net/Articles/55847/5.
https://landley.
net/kdocs/ols/2003/ols2003-pages-325-339.
pdf.. More details