See also: http://developer.apple.com/documentation/DeviceDrivers/Conceptual/WritingPCIDrivers/index.html http://developer.apple.com/documentation/DeviceDrivers/Conceptual/IOKitFundamentals/index.html ------------------------------ Date: Wed, 11 Oct 2000 17:21:32 -0700 From: Bill Enright Subject: Re: Designing a MacOSX PCI device driver. >At his point we have to ask ourselves some performance related >questions. IOKit provides 4 primary mechanisms to allow a program to >talk to a driver in the kernel. They are in order of ease of use and >from least resource hungry and fragile to most resource hungry >(approximately inverse for efficiency). > >1> set/getProperties >2> io_connect mechanism where the data is inline >3> io_connect mechanism where we pointers to the client address space. >4> IO Map memory, shared memory. > >To differentiate between these cases we need to know a few things. > >1> How many processes at any one time need to connect to this >specific piece of hardware? In my case, only one. It might be useful to allow for a 'status-only' user client, but only one client application will be handling the acquired data. >2> How much data are we moving through the system? Is it serial port >rate, networking data rate or Graphical framebuffer rate or some >other rate. >3> Related to how much data you are moving is how often you need to >collect the data. Do you have to get it 1000/second or can you >collect the data only 10 to 20 times a second? How much buffer space >is required to support these latencies? There are two data buffers in the OS9 driver implemenation. Each data buffer is 1452 bytes in length. The data buffers are passed to the client application (through QueueSecondaryInterruptHandler()) as each is filled with data events (first one is filled, then the other). When the client application is done processing the buffer, the PCI card is informed (through a device mem write) that the buffer is again available. Hopefully this all happens fast enough... I spent some time looking for some information about how much data to expect the card to generate. I couldn't find anything, but from using the client apps on the OS9 machine (which I've done in the past) I'm gonna give a SWAG of 20 buffer fills per second (28k per sec, each buffer 10 times). Receiving the data isn't time-critical i.e. if there's a lag between a buffer fill and the client app receiving the data it's no big deal as long as the data keeps coming... >4> Once you have collected the data what will your client do with it? >Do you care? The client performs quite a bit of computation to classify the data in real time (data events are filtered and categorized and displayed on possibly very many real-time graphical plots). I'm not sure whether the driver cares as long as it has somewhere to put new data as it comes in. =8^o ------------------------------ ------------------------------ Date: Thu, 12 Oct 2000 09:28:01 -0700 From: Godfrey van der Linden Subject: Designing a MacOSX PCI device driver (2). A couple of points I should make before starting todays email. - - I have size 14 feet, and sometimes they fit quite comfortably in my mouth :-(. However in all cases it is 'my' mouth not Apple's. - - I wish now I had have chosen a different title as we are on the darwin list. These drivers are really Darwin IOKit drivers not just MacOSX. At 17:21 -0700 00-10-11, Bill Enright wrote: > >1> How many processes at any one time need to connect to this >>specific piece of hardware? > >In my case, only one. It might be useful to allow for a >'status-only' user client, but only one client application will be >handling the acquired data. > >>2> How much data are we moving through the system? Is it serial port >>rate, networking data rate or Graphical framebuffer rate or some >>other rate. > >There are two data buffers in the OS9 driver implemenation. Each >data buffer is 1452 bytes in length. The data buffers are passed to >the client application (through QueueSecondaryInterruptHandler()) as >each is filled with data events (first one is filled, then the >other). When the client application is done processing the buffer, >the PCI card is informed (through a device mem write) that the >buffer is again available. Hopefully this all happens fast enough... > >>3> Related to how much data you are moving is how often you need to >>collect the data. Do you have to get it 1000/second or can you >>collect the data only 10 to 20 times a second? How much buffer space >>is required to support these latencies? > >I spent some time looking for some information about how much data >to expect the card to generate. I couldn't find anything, but from >using the client apps on the OS9 machine (which I've done in the >past) I'm gonna give a SWAG of 20 buffer fills per second (28k per >sec, each buffer 10 times). Receiving the data isn't time-critical >i.e. if there's a lag between a buffer fill and the client app >receiving the data it's no big deal as long as the data keeps >coming... > >>4> Once you have collected the data what will your client do with it? >>Do you care? > >The client performs quite a bit of computation to classify the data >in real time (data events are filtered and categorized and displayed >on possibly very many real-time graphical plots). I'm not sure >whether the driver cares as long as it has somewhere to put new data >as it comes in. Thanks Bill, We can now go onto discussing some of the next step in a design. So we know how much data is moving through the system, not much in this case. We also can charecterise the nature of the data flows and how the customer is going to use this data, i.e. he want lots and lots of processing time on each packet of data as it is made available. This is different to say a data acquisition for later analysis where process power is unimportant but I/O rates through the file system are the limiting factors. One thing I'm curious about is the '1452' buffer size locked in stone? Darwin and X is a page based system and we generally find that whole fractions of a page size allows us to make better use of system resources. SInce the minimum page wire down size is 1 page you will be using the resource anyway so you may as well get some value for it. The amount of data you need to move means that we can rule out set/getProperties. However the amount of data you are moving is so small that your may be able to use option 2 direct inline messaging, this is by far the easiest sort of IOUserClient to write, (Thank you Bill, I really didn't want to start with a massive multi media application as a first example). We will probably use the io_connect_scalarI_structureO() transport to move the data from the kernel to your application. We should also implement and open/close semantic to control the start and stop of the data collection engine. These routines are usually implemented using io_connect_scalarI_scalarO() transport routines. I suggest that we set the driver running free with a pair of 2048 byte buffers or maybe 3 1365 byte buffers. This data will always be collected whether or not the user is ready for it or not. So we are dedicating 1 wired page for data acquisition but we should only allocate this buffer when we receive a client open request from our user client. Next step is to work out how we let our client know that we have data available. Here again we have some tradeoff's too make depending on just how comfortable you feel with mach and low level CF stuff. Traditional unix developers will probably prefer to take the io_connect_...() call and put the client to sleep until the data is available. This is by far the easiest thing to do. In Darwin terms you would use a IOSyncer object to put your application's thread to sleep, this is very easy to do. However linking into the tradition 'select' or 'signal' stuff is much harder to do. Tradition Mac developers think in a far more asynchronous way where they register 'callout' and process data when the callout is called. In Darwin this is impossible as the kernel is not capable of calling a sub routine in a client thread directly. IOKit uses a ipc message to notify the client of data availabilty. Setting one of these connections up is pretty difficult so for your application I do not recommend doing this. Summary: Given about 20-30 calls/second by a single client process, at a time, to move a total of about 1500 bytes per call and not a great deal of Darwin in kernel expertise. I recommend the use of the io_connect_scalarI_structureO() transport which will block until data is available. In the meantime the driver itself is always collecting data while a client is connected whether or not an outstanding client call is extant. Tomorrow I'd like to discuss PCI matching and IOService subclass. Godfrey van der Linden ------------------------------ ------------------------------ Date: Thu, 12 Oct 2000 22:07:40 -0700 From: Godfrey van der Linden Subject: Designing a MacOSX PCI device driver (3). Setting up a driver for Darwin IOKit. I come to you this evening after spending about 3 hours writing my previous attempt at this email, I knocked my power book and jogged the battery. I just lost all of my work. Bloody annoying. Oh well, I was very informative and verbose in the previous email, but now my wrists hurt so I'm going to have to be very brief now and next week I'll fill in more of the details. An IOKit Driver is a special case of a MacOSX bundle. A bundle is a folder that usually contains at least two files, the first mandatory file is the Info.plist and the second file, optional, is the executable that has to be loaded into the kernel when we have successfully matched. Bill's driver will look something like this on disk. BillsDataAcquirer.kext |___Contents |___Info.plist |___MacOS |_BillsDataAcquirer The Info.plist file will be a typical CFBundle style plist with one IOKit specific dictionary known as IOKitPersonalities. In Bill's case I think it will look something like this. (NB I'm going to be using the old NeXT plist shorthand, where {} indicates dictionaries and tokens that start with an alphabetic or are surrounded by quotes are strings, you will not be able to use this dictionary directly but will have to convert it to XML or use the PropertyList editor built into ProjectBuilder IOKitPersonailty = { BillsFirstPersonality = { # The first 2 section's are referred to as the 'Matching Dictionary' # What category of driver are we writing IOProviderClass = IOPCIDevice; # Within the above category under what conditions do we match IOPCIMatch = '0xddddvvvv'; # Enable matching logging # IOKitDebug = 65535; # What the Primary Class' name is and where to find it CFBundleIdentifier = com.yisup.iokit.billsdataacquirer; IOClass = com_yisup_iokit_billsdataacquirer; # [optional] Any other field you want. BillsConfigurationData = 'MY DATA'; }; BillsSecondPersonality = { # The first 2 section's are referred to as the 'Matching Dictionary' # What category of driver are we writing IOProviderClass = IOPCIDevice; # Within the above category under what conditions do we match IOPCIMatch = '0xvvvvd1d1'; # Enable matching logging # IOKitDebug = 65535; # What the Primary Class' name is and where to find it CFBundleIdentifier = com.yisup.iokit.billsdataacquirer; IOClass = com_yisup_iokit_billsdataacquirer; # [optional] Any other field you want. BillsConfigurationData = 'MY DATA for second personality'; }; }; In this example I have shown 2 personalities within the driver. I think in Bill's specific case this isn't required. I just wanted to give a feel of the power of IOKit's matching. The only part of this that Bill has to fill in is his 4 hex digit 'vvvv' vendor ID and his 4 hex digit 'dddd' device ID. IOPCIDevice does provide more than just the IOPCIMatch field, see the header doc in for details. BTW as a general rule you can find the 'matching language' of a provider class in that classes header doc, at least you can in will written drivers. Sometimes you will need to look at the implementation of the matchPropertyTable method in the provider class' source code. Notice that I have used reverse DNS for the CFBundleIdentifier and pseudo reverse DNS for the IOClass class name. This isn't mandatory but we highly recommend it. The IOKit keeps a database of all kernel extensions that we are aware of indexed by the CFBundleIdentifier field so it is best to make sure you don't trip over an accidental duplicate. For instance 'MyIOKitDriver' would be a quite remarkably bad Identifier for a driver. The class name is in a similar global name space, in this case the dictionary of all classes that OSMetaClass is tracking, i.e. all classes that inherit from OSObject. For the terminally curious then you can find the definition of a Bundle in 'Inside Macintosh X: System Overview, Chapter 4 - Bundles'. I think this is available in the online developer doc. On Monday I'll write the outline of the driver and describe the logic behind it's inheritance. I will also cover the mechanism's to map the io registers into the kernel's VM and how to get access to the PCI cards config space and ROM. Godfrey ------------------------------ ------------------------------ Date: Mon, 16 Oct 2000 12:34:31 -0700 From: Godfrey van der Linden Subject: Designing a MacOSX PCI device driver (4). G'day All, Today I'm going to cover the inheritance structure of IOKit and how it doesn't help us at all to write a pci driver for an unknown card. ARE YOU SURE YOU HAVE TO BE IN THE KERNEL? (Oh Wow, Deja Vu, does this mean that the Matrix is being reconfigured? No, I just think this point is so important I'd repeat it. BTW PCI devices DO always need to have something in the kernel, shame really) One of the very common misunderstanding about IOKit is 'what class do I inherit from?'. This misunderstanding is caused by the fact that a driver is, by definition, 2 dimensional. First it uses a transport to get access to hardware, and secondly the driver passes this data it collects onto a client. As IOKit is only single inherited, see (1) below, we had to choose which axis to inherit on. In the end the problem was easy for us. DriverKit inherited on the functionality axis and IOKit needed to leverage DriverKit as much as possible. Thus DriverKit was the pragmatic reason for choosing our inheritance model. However it also makes sense on pure Object Oriented design grounds. In general a device interface is well defined, think of just about all of the hardware busses like PCI, SCSI, USB, FireWire and ATA to name just a few. They all define certain fundamental operations in clean well defined interface (some more so than others). This sort of clean, rarely expanded, API set is ideal for C++'s preferred design model of abstract interfaces. We use this clean API to define 'provider' service classes. Thus the plethora of 'IO...Device' classes in our system. This concept allowed us to abstract the transport to hardware using a well defined abstrace interface. ( 'IOProviderClass' in matching terminology). So lets choose which class Bill needs to inherit from. Bill's device driver is like nothing IOKit has seen before. Specifically Darwin doesn't seem to have a super class for a blood analysis hardware. Nor do we have a more general class along the lines of generic data collection. The closest class I can think of is a microphone, but that has much greater throughput requirements and is not really appropriate. So Bill is out of luck as far as helper families. He will have to fall back on the generic IOService superclass that ALL drivers inherit from, directly or indirectly. class com_yisup_iokit_billsdataacquirer : public IOService But that doesn't stop him from writing his own superclass. Now in Bill's specific case this may be overkill but lets imagine that the hardware is something that could be easy and cheap to convert to USB. In that case he would like to keep as much of his code that controls and interprets data in his superclass but provider 2 subclasses one that uses PCI as a transport and the other that uses USB. Something like this. Superclass class com_yisup_iokit_billsdataacquirer : public IOService Transport dependent subclasses class com_yisup_iokit_billsdataacquirerPCI : public com_yisup_iokit_billsdataacquirer class com_yisup_iokit_billsdataacquirerUSB : public com_yisup_iokit_billsdataacquirer Personalities: IOKitPersonalities = { BillsPCIPersonality = { # The first 2 section's are referred to as the 'Matching Dictionary' # What category of driver are we writing IOProviderClass = IOPCIDevice; # Within the above category under what conditions do we match IOPCIMatch = '0xvvvvdddd'; # Enable matching logging # IOKitDebug = 65535; # What the Primary Class' name is and where to find it CFBundleIdentifier = com.yisup.iokit.pci.billsdataacquirer; IOClass = com_yisup_iokit_billsdataacquirerPCI; # [optional] Any other field you want. BillsConfigurationData = 'MY DATA'; }; BillsUSBPersonality = { # The first 2 section's are referred to as the 'Matching Dictionary' # What category of driver are we writing IOProviderClass = IOUSBDevice; # Within the above category under what conditions do we match # vendor = ??; # Sorry don't know USB's matching language # device = ??; # Enable matching logging # IOKitDebug = 65535; # What the Primary Class' name is and where to find it CFBundleIdentifier = com.yisup.iokit.usb.billsdataacquirer; IOClass = com_yisup_iokit_billsdataacquirerUSB; # [optional] Any other field you want. BillsConfigurationData = 'MY DATA for second personality'; }; I'm not showing the rest of these Property lists and I'm hiding some complexity in project organisation. Tomorrow, we will start the driver up and actually start talking to the provider class. (Yes I know I said I'd get that far today but I think this email is enough to digest for one day don't you, besides my manager would probably like it if I do some work today :-) Till tomorrow then. Godfrey Footnote: (1) IOKit Single inheritance. As IOKit is a child of DriverKit which was implemented in ObjC we wanted to reduce the amount of work necessary to convert from one space to the other, as ObjC is single inherited this was easiest for most of our drivers. At first we were going to allow multiple inheritance as we ported to C++ but then we tripped over the whole can of worms called 'virtual' inheritance. We really, really didn't want to turn all of our developers into C++ experts, which you have to be to use virtual inheritance properly. Another side effect is that we are now reasonable language independent. At some stage in the future we may be able to move IOKit over to a good programming language. ------------------------------ ------------------------------ Date: Tue, 17 Oct 2000 08:58:22 -0700 From: Godfrey van der Linden Subject: Designing a MacOSX PCI device driver (5). G'day All, Once we have determined which IOService class we are inheriting from and what service is going to be our provider we are ready to start our driver. ARE YOU SURE YOU HAVE TO BE IN THE KERNEL? To recap in Bill's case we have a vanilla IOService subclass and his provider class is obviously an IOPCIDevice. I'm going to assume that in Bill's case the vendor and device id's are sufficient to uniquely identify his PCI board. Is this a safe assumption Bill? Or do you need to run some code and access resources on your PCI board to be certain? Once a driver has successfully completed matching its start routine will be called with the matching provider as argument. This call is made on the matching thread context. IOKit's matching spawns a thread for each call to 'registerService', I'll talk about registerService later. This thread will be the only thread to run on the provider until a driver creates a work loop. So driver writers are safe from multi-threading issues in the start context, at least until they create their own work loop. In Bill's case his driver is going to start out looking something like this. If I was writing a driver by the way I'd start with this code and make sure that I got my matching working before going on. After matching I'd make sure that I can talk to the provider properly. That is about as far as this code fragment will take you. We will fill in some more details tomorrow. - ----------------- billsdataacquirer.h ------------------- #include class IOPCIDevice; class IOWorkLoop; class IOInterruptEventSource; class IOCommandGate; class com_yisup_iokit_billsdataacquirer : public IOService { OSDeclareDefaultStructors(com_yisup_iokit_billsdataacquirer); public: virtual bool start(IOService *provider); virtual void stop(IOService *provider); virtual IOWorkLoop *getWorkLoop(); protected: virtual bool openProvider(); virtual void closeProvider(); virtual void interruptOccured(); // Other stuff that we will define in a later email. IOPCIDevice *dataAquPCI; SInt32 openCount; IOInterruptEventSource *boardInt; IOWorkLoop *workLoop; IOCommandGate *userGate; // Hold gate for user client requests }; - ----------------- billsdataacquirer.cpp ------------------- #include #include #include #include #include #include "billsdataacquirer.h" #define super IOService OSDefineMetaClassAndStructors(com_yisup_iokit_billsdataacquirer, IOService); // Start routine. First routine to get called once we have succesfully // matched. bool com_yisup_iokit_billsdataacquirer:: start(IOService *provider) { bool result = false; // Always call the superclass FIRST and return false if it fails if (!super::start(provider)) return false; // Next make sure that you have really got a PCI device // as a provider. This check is cheap and easy and it // avoids nasty panics if something has gone wrong in the // /System/Library/Extensions directory. dataAquPCI = OSDynamicCast(IOPCIDevice, provider) if (!dataAquPCI) return false; // Open our provider before going any further, somebody else may // already be using these services if (!openProvider()) return false; // Now we have a provider and we are certain that it is indeed // an IOPCIDevice so we can make all sorts of assumptions // about the services it provides, specifically we assume // that the ABI we compiled in that was defined in IOPCIDevice.h // is valid. // Create a work loop for scheduling interrupts and also to // provide a single threaded context so that this driver will be able // to migrate from CPU to CPU on an SMP box. IOWorkLoop *wl = getWorkLoop(); if (!wl) goto abortOpen; // Now create and register my interrupt event source boardInt = IOInterruptEventSource::interruptEventSource(this, (IOInterruptEventSource::Action) &com_yisup_iokit_billsdataacquirer::interruptOccured, dataAquPCI, /* Interrupt index */ 0); if (!boardInt || !wl->addEventSource(boardInt)) goto abortOpen; // Create and register my command gate userGate = IOCommandGate::commandGate(this); if (!userGate || !wl->addEventSource(userGate)) goto abortOpen; // Sorry, I'm out of my field of expertise at this point. There // is an excellent example of accessing config space and mapping // io registers in the IOKitExamples project. Check out // intel networking driver. // Setup our userClientClass name, this will be used // by IOService::newUserClient when an application requests connection // connection to this driver. const OSSymbol *userClient = OSSymbol::withCStringNoCopy("com_yisup_iokit_billsuserclient"); if (!userClient) goto aboutOpen; setProperty(gIOUserClientClassKey, (OSObject *) userClient); userClient->release(); // Lastly we have to let user land know that we are ready // to publish services on ourselves. If we were a 'bus' driver // we would iterate over all of the devices on our bus creating // a nub and registering them for service. But in Bills case his // driver is also his one and only nub so lets just register 'this' // as a 'nub'. registerService(); result = true; abortOpen: closeProvider(); return result; } // In general the stop routine cleans up all of the resources // that were allocated in the start routine. Note however that the stop // routine NEVER gets called while this driver has its provider open. // So while we have a user client open this driver can not be unloaded! void com_yisup_iokit_billsdataacquirer:: stop(IOService *provider) { if (userGate) { workLoop->removeEventSource(userGate); userGate->release(); userGate = 0; } if (boardInt) { workLoop->removeEventSource(boardInt); boardInt->release(); boardInt = 0; } if (workLoop) { workLoop->release(); workLoop = 0; } // Free any other unfreed resources at this point. } // Override the IOService::getWorkLoop method to return the // workloop that we MUST create to deliver interrupts on. // Only required if your driver has interrupts. IOWorkLoop *com_yisup_iokit_billsdataacquirer:: getWorkLoop() { if (!workLoop) workLoop = IOWorkLoop::workLoop(); return workLoop; } // Reference counted provider open routine. // Bit of a over kill in this case but may as well just // copy and paste the code anyway. BTW the user client // will call this routine when a client attaches. bool com_yisup_iokit_billsdataacquirer:: openProvider(IOService *provider) { if (OSIncrementAtomic(&openCount) > 0) return true; // Allocate any resources needed to run you device now if (!dataAquPCI->open(this)) return false; // Allocate your ring buffers and other 'lazy' resources now. return true; } // reference counted closer. closes our provider on our // last user client close. Once we have closed our provider // the provider is allowed to call stop on us so that we can // be unloaded. Until we call close on our provider we are not // allowed to be unloaded. void com_yisup_iokit_billsdataacquirer:: closeProvider(IOService *provider) { if (OSDecrementAtomic(&openCount) > 1) return; // Free any 'lazy' allocated resources now. dataAquPCI->close(this)); } Well I think that is enough for today. A lot of the stuff I covered in this email is very important so if you don't understand now please ask. I have never worked out how to explain this part of IOKit very clearly so I'd be surprised if I have managed this time around :-) Tomorrow we will have a look at how to setup our user client. Godfrey van der Linden ------------------------------ ------------------------------ Date: Tue, 17 Oct 2000 13:03:32 -0700 From: Bill Enright Subject: Re: Designing a MacOSX PCI device driver (5). >G'day All, > >Once we have determined which IOService class we are inheriting from >and what service is going to be our provider we are ready to start >our driver. > >ARE YOU SURE YOU HAVE TO BE IN THE KERNEL? Yes I am. :^) >To recap in Bill's case we have a vanilla IOService subclass and his >provider class is obviously an IOPCIDevice. I'm going to assume that >in Bill's case the vendor and device id's are sufficient to uniquely >identify his PCI board. Is this a safe assumption Bill? Or do you >need to run some code and access resources on your PCI board to be >certain? You are exactly right. The vendor and device ID uniquely identify my card (so the matching XML properties are all I need to match to the driver). >// Start routine. First routine to get called once we have succesfully >// matched. >bool com_yisup_iokit_billsdataacquirer:: >start(IOService *provider) >{ > bool result = false; > > // Always call the superclass FIRST and return false if it fails > if (!super::start(provider)) > return false; > > // Next make sure that you have really got a PCI device > // as a provider. This check is cheap and easy and it > // avoids nasty panics if something has gone wrong in the > // /System/Library/Extensions directory. > dataAquPCI = OSDynamicCast(IOPCIDevice, provider) > if (!dataAquPCI) > return false; ^I did check for a PCI device provider > // Open our provider before going any further, somebody else may > // already be using these services > if (!openProvider()) > return false; ^But I didn't do this! > // Create a work loop for scheduling interrupts and also to > // provide a single threaded context so that this driver will be able > // to migrate from CPU to CPU on an SMP box. > IOWorkLoop *wl = getWorkLoop(); > if (!wl) > goto abortOpen; ^I created a new workloop here with IOWorkLoop::workLoop() (and overrode getWorkLoop() to return it). I thought the default getWorkLoop() call just got the provider's workloop as opposed to creating a new one? > // Now create and register my interrupt event source > boardInt = IOInterruptEventSource::interruptEventSource(this, > (IOInterruptEventSource::Action) > &com_yisup_iokit_billsdataacquirer::interruptOccured, > dataAquPCI, /* Interrupt index */ 0); > if (!boardInt || !wl->addEventSource(boardInt)) > goto abortOpen; ^I did this, but installed it into the workloop I explicitly created. > // Create and register my command gate > userGate = IOCommandGate::commandGate(this); > if (!userGate || !wl->addEventSource(userGate)) > goto abortOpen; ^Created a command gate too. > // Sorry, I'm out of my field of expertise at this point. There > // is an excellent example of accessing config space and mapping > // io registers in the IOKitExamples project. Check out > // intel networking driver. > > // Setup our userClientClass name, this will be used > // by IOService::newUserClient when an application requests connection > // connection to this driver. > const OSSymbol *userClient = > OSSymbol::withCStringNoCopy("com_yisup_iokit_billsuserclient"); > if (!userClient) > goto aboutOpen; > > setProperty(gIOUserClientClassKey, (OSObject *) userClient); > userClient->release(); ^Instead of doing this, I overrode newUserClient(). Does it really matter which way you do it? > // Lastly we have to let user land know that we are ready > // to publish services on ourselves. If we were a 'bus' driver > // we would iterate over all of the devices on our bus creating > // a nub and registering them for service. But in Bills case his > // driver is also his one and only nub so lets just register 'this' > // as a 'nub'. > registerService(); ^ I...um...didn't.... =8^o CSS was designed with a 40 bit keylength to comply with US government export regulation, and as such it is easily compromised through brute force attacks (such are the intentions of export control). Moreover the 40 bits have not been put to good use, as the ciphers succumb to attacks with much lower computational work than which is permitted in the export control rules. Whether CSS is a serious cryptographic cipher is debatable. It has been clearly demonstrated that its strength does not match the keylength. If the cipher was intended to get security by remaining secret, this is yet another testament to the fact that security through obscurity is an unworkable principle. - -Frank A. Stevenson http://www.cs.cmu.edu/~dst/DeCSS/FrankStevenson/analysis.html ------------------------------ ------------------------------ Date: Tue, 17 Oct 2000 14:50:39 -0700 From: Godfrey van der Linden Subject: Re: Designing a MacOSX PCI device driver (5). At 13:03 -0700 00-10-17, Bill Enright wrote: > > // Open our provider before going any further, somebody else may >> // already be using these services >> if (!openProvider()) >> return false; > >^But I didn't do this! You probably should do this, even though you know that you are the only person using this device. It helps IOKit to keep track of which parts of it are in use at any one point in time. > > // Create a work loop for scheduling interrupts and also to >> // provide a single threaded context so that this driver will be able >> // to migrate from CPU to CPU on an SMP box. >> IOWorkLoop *wl = getWorkLoop(); >> if (!wl) >> goto abortOpen; > >^I created a new workloop here with IOWorkLoop::workLoop() (and >overrode getWorkLoop() to return it). I thought the default >getWorkLoop() call just got the provider's workloop as opposed to >creating a new one? Either your solution or my solution works fine. The point is that a driver with an interrupt must override getWorkLoop is some manner, otherwise all interrupts would be delivered on a system wide thread. Now that isn't necessarily a bad thing as that is how secondary interrupts work on 9 now but X lets our drivers make use of the fact that they are totally independent which should scale well for SMP boxes. > > // Sorry, I'm out of my field of expertise at this point. There >> // is an excellent example of accessing config space and mapping >> // io registers in the IOKitExamples project. Check out >> // intel networking driver. >> >> // Setup our userClientClass name, this will be used >> // by IOService::newUserClient when an application requests connection >> // connection to this driver. >> const OSSymbol *userClient = >> OSSymbol::withCStringNoCopy("com_yisup_iokit_billsuserclient"); >> if (!userClient) >> goto aboutOpen; >> >> setProperty(gIOUserClientClassKey, (OSObject *) userClient); >> userClient->release(); > >^Instead of doing this, I overrode newUserClient(). Does it really >matter which way you do it? Doesn't make much difference. You may want to continue overriding newUserClient as it allows you to put logging in your code just to help debugging. This technique of any subclass being able to override parent methods for debugging is probably the main reason why all IOKit methods are virtual. One other thing. You could skip this step 'cause you can put the IOUserClientClass in your personality. Then the property is made automatically available to the IOService Parent. Ooops this technique depends on a bug fix that hasn't gone into IOService yet, so ignore this point for a release. > > // Lastly we have to let user land know that we are ready >> // to publish services on ourselves. If we were a 'bus' driver >> // we would iterate over all of the devices on our bus creating >> // a nub and registering them for service. But in Bills case his >> // driver is also his one and only nub so lets just register 'this' >> // as a 'nub'. >> registerService(); > >^ I...um...didn't.... Got to if you want IOServiceMatching to work at its easiest. Godfrey ------------------------------ ------------------------------ Date: Tue, 17 Oct 2000 15:53:12 -0700 From: Bill Enright Subject: Re: Designing a MacOSX PCI device driver (5). >> > // Lastly we have to let user land know that we are ready >>> // to publish services on ourselves. If we were a 'bus' driver >>> // we would iterate over all of the devices on our bus creating >>> // a nub and registering them for service. But in Bills case his >>> // driver is also his one and only nub so lets just register 'this' >>> // as a 'nub'. >>> registerService(); >> >>^ I...um...didn't.... > >Got to if you want IOServiceMatching to work at its easiest. This probably explains why I had to go about finding my driver in such a *screwy* way. I first had to find the IOPCIDevice and then find the driver attached to it because I *couldn't* successfully IONameMatch the driver! DUH! =8^o ------------------------------ ------------------------------ Date: Wed, 18 Oct 2000 09:47:58 -0700 From: Godfrey van der Linden Subject: Re: Designing a MacOSX PCI device driver (5). At 15:53 -0700 00-10-17, Bill Enright wrote: > >>> registerService(); >>> >>>^ I...um...didn't.... >> >>Got to if you want IOServiceMatching to work at its easiest. > >This probably explains why I had to go about finding my driver in >such a *screwy* way. I first had to find the IOPCIDevice and then >find the driver attached to it because I *couldn't* successfully >IONameMatch the driver! DUH! > >=8^o Bingo Godfrey ------------------------------ ------------------------------ Date: Tue, 17 Oct 2000 20:48:01 -0700 From: Godfrey van der Linden Subject: Designing a MacOSX PCI device driver (6). G'day, Well now we have the beginnings of a driver and its work loop is up and running. I think it's time to define our user clients API. Later if I have time I'll also include code connect to our driver from user space and collect the data. (I had time :-) Bills data requirements are so simple that I think we only need a single 'read' equivalent routine. He may need to add other routines to control his device but I'm not going to go into that. Even though Bills requirements are simple, IOKit in this area is anything but simple. Each individual task is pretty easy there is just lots of steps where you can trip up too easily. Programming in the kernel is PAINFUL. - ---------------- billsuserclient.h ---------------- // This header is included in both the kernel module and the // user land application and defines the API between the // two environments. // The GetData command is of ScalarIStructO type with no input data // and variable size output data, works best in input data is less than // about 2 pages. enum IOCDBUserClientCommandCodes { kBillsDataAcquirerGetData, // kIOUCScalarIStructO, 0, -1 kBillsDataAcquirerNumCommands }; // Arbitrary one page limit, doesn't really matter to much // but some limit is worth enforcing as if the size gets // 'too' big it may be worth revisiting our API and using // an IOMemoryDescriptor on a passed in data pointer instead. #define kBillsDataAcquirerGetDataMinSize 1452 #define kBillsDataAcquirerGetDataMaxSize 4096 #define kBillsServiceClassName \ "com_yisup_iokit_billsdataacquirer" #if KERNEL #include #include #include "billsdataacquirer.h" class com_yisup_iokit_billsuserclient : public IOUserClient { OSDeclareDefaultStructors(com_yisup_iokit_billsuserclient) protected: // For you unix heads(fyuh) out there this array is sort of like // an extensible cdevsw entry static const IOExternalMethod sMethods[kBillsDataAcquirerNumCommands]; task_t fClient; com_yisup_iokit_billsdataacquirer *fBillsService; // Overidden IOUserClient routines. // fyuh: these routines are IOKit's equivalent to open() virtual bool initWithTask(task_t owningTask, void *security_id, UInt32 type); virtual bool start(IOService *provider); // fyuh: Driver dependent cdevsw table lookup virtual IOExternalMethod * getTargetAndMethodForIndex(IOService **target, UInt32 index); // fyuh: two guesses :-) virtual IOReturn clientClose(void); // fyuh: No real equivalent I guess I'd have used read() // Actual call data acquisition call into our driver virtual IOReturn getData(void *vBuf, void *voutSize, void *, void *, void *, void *); }; #endif /* KERNEL */ - ---------------- billsuserclient.cpp ---------------- #include #include #include #include "billsuserclient.h" #define super IOUserClient OSDefineMetaClassAndStructors(com_yisup_iokit_billsuserclient, IOUserClient); const IOExternalMethod com_yisup_iokit_billsuserclient:: sMethods[kBillsDataAcquirerGetData] = { { // kIOCDBUserClientGetInquiryData 0, (IOMethod) &com_yisup_iokit_billsuserclient::getData, kIOUCScalarIStructO, 0, 0xffffffff }, }; // Called at IOServiceOpen time to start basic initialisation bool com_yisup_iokit_billsuserclient:: initWithTask(task_t owningTask, void *security_id, UInt32 type) { if (!super::init()) return false; fClient = owningTask; // Bills simple driver doesn't need this // but it doesn't hurt and it protects you from // unexpected task termination so that you can // clean up properly in an organised manner. task_reference(fClient); return true; } // Note clientClose may be called twice, once by // IOServiceClose and once by the actual client process dying. IOReturn com_yisup_iokit_billsuserclient:: clientClose(void) { if (fBillsService) { // Have been started so we better detach fBillsService->close(this); // Aborts any outstanding I/O getData // This should happen automatically but it doesn't yet. // be warned this detach will be unnecessary in the future and // will probably cause you driver to panic in the future. detach(fBillsService); fBillsService = 0; } // Better release our client task reference if (fClient) { task_deallocate(fClient); fClient = 0; } return kIOReturnSuccess; } // Start routine, first time that we should allocate resource bool com_yisup_iokit_billsuserclient:: start(IOService *provider); { if (!super::start(provider)) return false; fBillsService = OSDynamicCast( com_yisup_iokit_billsdataacquirer, provider); if (fBillsService && fBillsService->open(this)) return true; // We got an exclusive open return OK // As fBillsService is used as flag to close our provider // and we haven't opened it so set it to zero before returning // a failure. fBillsService = 0; // that's it return false; } // Router method converts a function code to a // pointer to an IOExternalMethod and also returns what object to // send the IOExternalMethod to. IOExternalMethod *com_yisup_iokit_billsuserclient:: getTargetAndMethodForIndex(IOService **target, UInt32 index) { if (index < (UInt32) kBillsDataAcquirerNumCommands) { *target = this; return &sMethods[index]; } else return 0; } // Actual call data acquisition call into our driver // This is the guts of the user client every thing else // has just been book keeping. IOReturn com_yisup_iokit_billsuserclient:: getData(void *vBuf, void *voutSize, void *, void *, void *, void *) { UInt8 *buf = (UInt8 *) vBuf; UInt32 *outSizeP = (UInt32 *) voutSize; UInt32 outSize = *outSizeP; // In general kernel API's dont spend the clients // cpu cycles validating data that SHOULD be correct // anyway. // // However UserClients can't be so trusting and // must validate as the one 'absolute' rule is that // no matter what mistake a client userland process makes // it must not panic the kernel, if it does it is // a FATAL bug and should be fixed ASAP. if (outSize < kBillsDataAcquirerGetDataMinSize || outSize > kBillsDataAcquirerGetDataMaxSize) return kIOReturnBadArgument; // Synchronously get the next lump of data from our // PCI card. return fBillsService->getData(buf, outSizeP); } - ------------------------------------------------------------ That's it for the in kernel user client. Now we have to write the user land side. In general the family writer will have written library code in user land to in someway hide this tedious interface from their customers. As we noted yesterday however Bill can't do this as he isn't inheriting from an IOKit provided family. Instead he has to write his own interface using the raw tools that IOKit provides. Here is some sample code that lives in user land and establishes a connection with a driver in the kernel. - -------------- billsdataacquisition.c -------------- /* * Compile this code with the following command line * cc -o -framework IOKit .c */ #include #include #include #include #include #include #include #include #include #include #include "billsuserclient.h" /* macro to test and report fatal errors when calling mach routines */ #define mach_test(expr, msg) do { \ kern_return_t err = (expr); \ if (KERN_SUCCESS != err) { \ fprintf(stderr, "%s: %s - %s(%x,%d)\n", cmdName, \ msg, mach_error_string(err), \ err, err & 0xffffff); \ fflush(stderr); \ *((char *) 0) = 0; \ } \ } while(0) /* macro #define plain_test(expr, msg) do { \ if (expr) { \ fprintf(stderr, "%s: %s\n", cmdName, msg); \ fflush(stderr); \ *((char *) 0) = 0; \ } \ } while(0) static const char *cmdName; static void dosomethingtodevice(io_object_t dataService) { kern_return_t ret; io_connect_t dataPort; // Ok we have a device lets connect to it and open it up ret = IOServiceOpen(dataService, mach_task_self(), /* type */ 0, &dataPort); mach_test(ret, "Couldn't create a user client, why not?"); // We have a connection to our driver, YEH! // We can start using our io_connect_.* routines to communicate // with it. do { unsigned char data[kBillsDataAcquirerGetDataMinSize]; mach_msg_type_number_t len = sizeof(data); // kBillsDataAcquirerGetData, kIOUCScalarIStructO, 0, -1 ret = io_connect_method_scalarI_structureO(dataPort, kIOCDBUserClientGetInquiryData, NULL, 0, (char *) data, &len); // Well, finally (sigh), we have our data so now we can do something // with it. // dosomething(data, len); } while (ret == kIOReturnSuccess); if (dataPort) { IOServiceClose(dataPort); dataPort = 0; } } // Search for our driver in the registry, if we find it // then do something with it. static void findBillsDevices(void) { mach_port_t masterPort = 0; CFDictionaryRef billsMatch = 0; io_iterator_t iter = 0; io_object_t device, billsDevice = 0; mach_test(IOMasterPort(NULL, &masterPort), "Couldn't create a master IOKit Port"); // For the curious IOServiceMatching creates a dictionary // with the input string as the value for IOProviderClass, // familiar, yes IOKit's complete matching system is used // for generating this iterator. We can implement // as powerful a matching language as we want to rendezvous // with a particular instance of our service. Bill doesn't need // anything that sophisticated billsMatch = IOServiceMatching(kBillsServiceClassName); plain_test(!billsMatch, "Can't create a 'Bill' matching dictionary"); // Note the IOServiceGetMatchingServices is documented as // consuming a reference on the matching dictionary so // we don't have to release it here. However it is so odd // that I think it is worth documenting. mach_test(IOServiceGetMatchingServices(masterPort, billsMatch, &iter), "Can't create a SCSI Service iterator"); billsMatch = 0; // Finish hand off of billsMatch dictionary // Iterate over all matching drivers in the registry. // It is an error if we find more than one matching driver while ( (device = IOIteratorNext(iter)) ) { plain_test(billsDevice, "But we already have a device I'm confused"); billsDevice = device; } // Did we find the device then? plain_test(!billsDevice, "Couldn't find a Bills Data Acquirer service in registry"); dosomethingtodevice(billsDevice); // I always clean up after myself. It isn't really // necessary as the task is about to terminate and we // can rely on the kernel to completely clean up behind us. IOObjectRelease(billsDevice); if (iter) { IOObjectRelease(iter); iter = 0; } if (billsMatch) { CFRelease(billsMatch); billsMatch = 0; } if (masterPort) { mach_port_deallocate(mach_task_self(), masterPort); masterPort = 0; } } static volatile void usage(void) { fprintf(stderr, "Usage: %s", cmdName); fprintf(stderr, " your options go here"); exit(EX_USAGE); } static void init(int argc, const char *argv[]) { char c; // Get the command name and strip the leading dirpath if (cmdName = strrchr(argv[0], '/')) cmdName++; else cmdName = argv[0]; while ((c = getopt(argc, argv, "?")) != -1) { switch(c) { case '?': default: usage(); } } } // __private_extern__ is a Apple GCC idiom that has rather an // odd semantic, that is that main is published as an extern from // this object file, but if we link it into a bigger object file // it gets converted to a state symbol. It is used to avoid // namespace pollution in libraries. However I'm trying to get // into the habit of never polluting any symbol space. I'm only // middling successful at changing my habits. __private_extern__ int main(int argc, const char *argv[]) { init(argc, argv); findBillsDevices(); exit(EX_OK); } - ------------------------------------------------------------ So we are nearly finished, todays email was easy huh! I suspect you have realised just how tedious it is talk to a random PCI device in our kernel environment. IOKit really is easy, believe it or not, if you are writing a driver we have seen before. The problem for us is that all of this complexity is required to implement an efficient and robust kernel. Neither of these characteristics make for easy to use code. I think IOKit has probably gone over the top (just a little) with it's power, there is a lot to be said for traditional UNIX character devices, they are a hell of a lot easier to implement. Well, they are until you want to get sophisticated or into SMP. Implementing the select behaviour or ioctl's can be arcane. (Added later) Do you know I have just added the (fyuh) comments in the User Client class header and I realised something strange. IOKit really isn't that much more complicated than *nix cdevsw/bdevsw driver and it is a hell of a lot more extensible and descriptive. Tomorrow I'll wrap up this series by suggesting a design for the synchronous getData API and a typical double buffer solution. So we will be revisiting our old friend com_yisup_iokit_billsdataacquirer. Godfrey 'AYSYWTBITK' van der Linden (Are You Sure You Want To Be In The Kernel? :-/) PS I'll have to work on AYSYWTBITK, it is a little difficult to pronounce, I wanted something as snappy as KISS. Tell you what a buy a beer at next years WWDC for the best suggestion. ------------------------------ ------------------------------ Date: Wed, 18 Oct 2000 15:20:51 -0700 From: Eric Brown Subject: Re: Designing a MacOSX PCI device driver (6) On Wednesday, October 18, 2000, at 02:43 PM, Glenn Hughes wrote: > > // For you unix heads(fyuh) out there this array is sort of like > > // an extensible cdevsw entry > > The question here is does describing one OS in terms of another OS > count as documentation? Silly me, I've spent the last 10 years > programming for this little fragile yet easy to use thing called > MacOS. (Oh yeah, and that fragile yet hard to use pain in the rear > called Windows.) Given that: a) This isn't documentation b) This is being sent to darwin-development - apparently consisting of quite a large number of unix hackers c) Godfrey's background is much more in unix and unix-like environments Then this is a very valid analogy to make in comments in example code. > > > // This should happen automatically but it doesn't yet. > > // be warned this detach will be unnecessary in the future and > > // will probably cause you driver to panic in the future. > > detach(fBillsService); > > Ugh... Is there any way to code this now such that my driver won't > break in the future? It seems a bit odd to do something that is most > likely going to crash later on. CFM has a way of dealing with this... > IOKit should as well... There should be a version number you could > bump in the system that would cause old drivers to not load. > > > if (fBillsService && fBillsService->open(this)) > > return true; // We got an exclusive open return OK > > What if we want shared access? > Then you don't need to the call to open(). Godfrey can expand a little more here, but AFAIK the intent of the open semantic is to allow for exclusive access control. > >IOReturn com_yisup_iokit_billsuserclient:: > >getData(void *vBuf, void *voutSize, void *, void *, void *, void *) > > Do all methods exported using the mechanism have this same interface? > > > // kBillsDataAcquirerGetData, kIOUCScalarIStructO, 0, -1 > > ret = io_connect_method_scalarI_structureO(dataPort, > > kIOCDBUserClientGetInquiryData, NULL, 0, > > (char *) data, &len); > > What's the deal with > kIOCDBUserClientGetInquiryData/kBillsDataAcquirerGetData? Is > kIOCDBUserClientGetInquiryData a system defined const? I couldn't > find it anywhere... > kBillsDataAcquirerGetData is defined in the driver (in a shared header) and appears at the beginning of the part 6 e-mail. I'm guessing the kIOCDBUserClientGetInquiryData is a mistake and it should be kBillsDataAcquirerGetData. Chances are Godfrey took some of the code from another part of the system and missed that one. > >So we are nearly finished, todays email was easy huh! I suspect you > >have realised just how tedious it is talk to a random PCI device in > >our kernel environment. > > Actually, this doesn't seem so bad at all, compared to VXD > programming. Or WDM for that matter. I just wish I had some more > documentation and examples. > You and everybody else out there. Hopefully this e-mail series will help in that area. We are aware of the serious need for IOKit documentation and are actively working to remedy that. In the meantime there are a number of resources available. Most if not all public IOKit headers should be headerdoc'ed. That should provide at least a basic description of the functionality and behavior of the classes and individual functions. This mail list is also an excellent source of information. A number of the IOKit engineers actively read it and we try to answer IOKit-related questions as quickly as possible. And lastly there's always the source - the ultimate sort of documentation. That's where I go when I need to answer a question. Hopefully you won't need to do that very often, but at least its available if necessary. - - Eric - --- Eric Brown email: esb@apple.com MacOS X - IOKit Audio Architect Apple Computer ------------------------------ ------------------------------ Date: Thu, 19 Oct 2000 04:41:24 -0700 From: Godfrey van der Linden Subject: Designing a MacOSX PCI device driver (7). Typos and correction. IOPCIMatch is 0xvvvvdddd // kBillsDataAcquirerGetData, kIOUCScalarIStructO, 0, -1 ret = io_connect_method_scalarI_structureO(dataPort, kIOCDBUserClientGetInquiryData, NULL, 0, (char *) data, &len); Should read IOPCIMatch is in fact 0xddddvvvv // kBillsDataAcquirerGetData, kIOUCScalarIStructO, 0, -1 ret = io_connect_method_scalarI_structureO(dataPort, kBillsDataAcquirerGetData, NULL, 0, (char *) data, &len); G'day All, I've been working on this last email for a while now and I've been tossing and turning. It is now 4:30am and I'm going to finish off com_yisup_iokit_billsdataacquirer Now. Bill as I'm not sure if you can change your buffer size I'll assume you can't and setup a single page with a pair of 1452 byte buffers. The general idea of this driver is that we wait until we get the open request from our user client to allocate our resources and start the free running data collection engine. Once the data collection engine is running we provide an interrupt routine that collects one buffer and sets the other buffer going. Meanwhile we implement getData by checking if a buffer is available, if so we return it immediately. Otherwise we wait for put the user client to sleep until the next buffer becomes available. I'll implement this API using an internal ASYNC call back if I need to go to sleep, don't worry you will see what I mean later. Note that this is almost, but not quite a repeat, of my previous email. I have changed a few things though so use this class rather than the previous one if your want a template. I have used the fUserGate quite a lot in this sample code. You will find that I'm using the runAction mechanism. I dislike C++ pointers to member function mechanism and I also don't want to create static member functions to wrap all of these calls. Instead of these I make an assumption that I can always convert a pointer to a member function to a C pointer to a function. In fact this causes our compiler to barf, so you have to use -fpermissive as a compile flag. If you don't do this then you will have to use static member functions instead, this also avoids a nasty cast. Think of the code in this series as a sort of detailed pseudo code. I haven't actually tested the code at all, it should work in theory but there will be problems in the detailed implementation. Bill can you post your driver? If not I'll try to emulate a PCI card, (not easy :-(). - ----------------- billsdataacquirer.h ------------------- #include #include class IOPCIDevice; class IOWorkLoop; class IOInterruptEventSource; class IOCommandGate; // Keep kNumBufs * kBufferSize less than 1 page. If you // need more then use separate allocations #define kBufferSize 1452 #define kNumBufs 2 // Should be power of 2 class com_yisup_iokit_billsdataacquirer : public IOService { OSDeclareDefaultStructors(com_yisup_iokit_billsdataacquirer); protected: IOPCIDevice *fDataAquPCI; SInt32 fOpenCount; IOInterruptEventSource *fBoardInt; IOWorkLoop *fWorkLoop; IOCommandGate *fUserGate; // Hold gate for user client requests semaphore_port_t fBufAvailable; // Buffer in kernel virtual space. UInt8 *fVirtualBuffer; IOPhysicalAddress *fBuffer[kNumBufs]; // make use of TCP/IP concept of fBufProduced - fBufConsumed // always making sense preovided we dont get too far ahead. UInt32 fBufProduced, fBufConsumed; // Internal member functions virtual bool openProvider(); virtual void closeProvider(); // // These four functions all run 'synchronously' // That means only one of these routines can run at one time. // They are all protected by the IOWorkLoop // virtual bool startEngine(); virtual void stopEngine(); virtual void interruptOccured(); virtual IOReturn getDataGated(UInt8 *buf, UInt32 *sizeP); public: // Overriden functions from IOService virtual bool start(IOService *provider); virtual void stop(IOService *provider); virtual bool handleOpen(IOService *forClient, IOOptionBits options = 0, void *arg = 0); virtual void close(IOService *forClient, IOOptionBits options = 0); virtual IOWorkLoop *getWorkLoop(); // My class' public member functions virtual IOReturn getData(UInt8 *buf, UInt32 *sizeP); }; - ----------------- billsdataacquirer.cpp ------------------- #include #include #include #include #include #include #include "billsdataacquirer.h" #include #define super IOService OSDefineMetaClassAndStructors(com_yisup_iokit_billsdataacquirer, IOService); // Start routine. First routine to get called once we have successfully // matched. bool com_yisup_iokit_billsdataacquirer:: start(IOService *provider) { bool result = false; // Always call the superclass FIRST and return false if it fails if (!super::start(provider)) return false; // Next make sure that you have really got a PCI device // as a provider. This check is cheap and easy and it // avoids nasty panics if something has gone wrong in the // /System/Library/Extensions directory. fDataAquPCI = OSDynamicCast(IOPCIDevice, provider) if (!fDataAquPCI) return false; // Open our provider before going any further, somebody else may // already be using these services if (!openProvider()) return false; // Now we have a provider and we are certain that it is indeed // an IOPCIDevice so we can make all sorts of assumptions // about the services it provides, specifically we assume // that the ABI we compiled in that was defined in IOPCIDevice.h // is valid. // Create a work loop for scheduling interrupts and also to // provide a single threaded context so that this driver will be able // to migrate from CPU to CPU on an SMP box. fWorkLoop = IOWorkLoop::workLoop(); if (!fWorkLoop) goto abortOpen; // Now create and register my interrupt event source fBoardInt = IOInterruptEventSource::interruptEventSource(this, (IOInterruptEventSource::Action) &com_yisup_iokit_billsdataacquirer::interruptOccured, fDataAquPCI, /* Interrupt index */ 0); if (!fBoardInt || !fWorkLoop->addEventSource(fBoardInt)) goto abortOpen; // Create and register my command gate fUserGate = IOCommandGate::commandGate(this); if (!fUserGate || !fWorkLoop->addEventSource(fUserGate)) goto abortOpen; // Sorry, I'm out of my field of expertise at this point. There // is an excellent example of accessing config space and mapping // io registers in the IOKitExamples project. Check out // intel networking driver. // Setup our userClientClass name, this will be used // by IOService::newUserClient when an application requests connection // connection to this driver. const OSSymbol *userClient = OSSymbol::withCStringNoCopy("com_yisup_iokit_billsuserclient"); if (!userClient) goto aboutOpen; setProperty(gIOUserClientClassKey, (OSObject *) userClient); userClient->release(); // Lastly we have to let user land know that we are ready // to publish services on ourselves. If we were a 'bus' driver // we would iterate over all of the devices on our bus creating // a nub and registering them for service. But in Bills case his // driver is also his one and only nub so lets just register 'this' // as a 'nub'. registerService(); result = true; abortOpen: closeProvider(); return result; } // // Primary Client get data call. // // There is a very small potential race here if we use more than // 2 data buffers or we have some over-running data and the client // is attempting to get streaming data on more than one thread. // // As we can't hold the gate while waiting for the semaphore // the semaphore could fire we then preempt and another thread could // get then through the next semaphore and steal our data out of order. // I'm not going to bother to close the gate as it is very unlikely // to occur. // // fyuh: Although we don't currently preempt in kernel space this is // a feature we hope to get in by FCS, be warned. // IOReturn com_yisup_iokit_billsdataacquirer:: getData(UInt8 *buf, UInt32 *sizeP) { IOReturn res; // Wait for data to become available res = semaphore_wait(fBufAvailable); if (KERN_SUCCESS != res) return res; // Get the data. return fUserGate->runAction((IOCommandGate::Action) &com_yisup_iokit_billsdataacquirer::getDataGated, buf, sizeP); } // In general the stop routine cleans up all of the resources // that were allocated in the start routine. Note however that the stop // routine NEVER gets called while this driver has its provider open. // So while we have a user client open this driver can not be unloaded! void com_yisup_iokit_billsdataacquirer:: stop(IOService *provider) { if (fUserGate) { fWorkLoop->removeEventSource(fUserGate); fUserGate->release(); fUserGate = 0; } if (fBoardInt) { fWorkLoop->removeEventSource(fBoardInt); fBoardInt->release(); fBoardInt = 0; } if (fWorkLoop) { fWorkLoop->release(); fWorkLoop = 0; } // Free any other unfreed resources at this point. } // // Override the IOService::getWorkLoop method to return fWorkLoop // that we MUST have created to deliver interrupts on. // IOWorkLoop *com_yisup_iokit_billsdataacquirer:: getWorkLoop() { return fWorkLoop; } // // Reference counted provider open routine. // Bit of a over kill in this case but I may as well just // copy and paste the code anyway. BTW the user client // will call this routine through the handleOpen function. // bool com_yisup_iokit_billsdataacquirer:: openProvider() { if (OSIncrementAtomic(&openCount) > 0) return true; // Allocate any resources needed to run you device now if (!fDataAquPCI->open(this)) return false; return true; } // // reference counted closer. closes our provider on our // last user client close. Once we have closed our provider // the provider is allowed to call stop on us so that we can // be unloaded. Until we call close on our provider we are not // allowed to be unloaded. // void com_yisup_iokit_billsdataacquirer:: closeProvider() { if (OSDecrementAtomic(&openCount) > 1) return; fDataAquPCI->close(this)); } // // Always check to see if you really have an interupt on your hardware. // void com_yisup_iokit_billsdataacquirer:: interruptOccured() { // Check to see if this is a ghost interrupt bool realInt = /* Check your hardware */ false; if (realInt) { // Got one buffer fBufProduced++; // Setup the next buffer before completing the previous I/O // Bill your PCI code goes here. // Program DMA with fBuffer[fBufProduced & (kNumBufs - 1)]; // Complete previous I/O by signalling that we have data available semaphore_signal(fBufAvailable); } } // // Get data, should never be called if no data is available. // Returns kIOReturnInternalError if no data // kIOReturnOverrun if some data has been overwritten // IOReturn com_yisup_iokit_billsdataacquirer:: getDataGated(UInt8 *buf, UInt32 *sizeP) { int bufferCount = fBufProduced - fBufConsumed; if (0 == bufferCount) { #if DEBUG IOLog("No Data, who signaled the syncer"); #endif return kIOReturnInternalError; // Should never happen } else if (bufferCount < kNumBufs) { int offset = kBufferSize * (kBufConsumed & (kNumBufs - 1)); len = min(*sizeP, kBufferSize); bcopy(fVirtualBuffer + offset, buf, len); *sizeP = len; fBufConsumed++; return kIOReturnSuccess; } else /* Data overrun, data lost */ { fBufConsumed++; return kIOReturnOverrun; } } #if 4096 < kBufferSize * kNumBufs #error Need to change buffer allocation strategy #endif // Called in gated context // // Allocate all of the resources necessary to get our driver up and // running now that we finally have a client. // bool com_yisup_iokit_billsdataacquirer:: startEngine() { IOPhysicalAddress basePhys; int allocSize; if (KERN_SUCCESS != semaphore_create(kernel_task, &fBufAvailable, SYNC_POLICY_FIFO, 0)) return false; // Compute size of buffer and round up to nearest page_size allocSize = kBufferSize * kNumBufs; allocSize = (allocSize + page_size - 1) & ~(page_size - 1); // check my assumptions, ok to validate at init time // try not to validate on data path, trust your IN-KERNEL client if (allocSize > page_size) return false; fVirtualBuffer = IOMallocContiguous(allocSize, page_size, &basePhys); if (!fVirtualBuffer) return false; for (int i = 0; i < kNumBufs; i++) fBuffer[i] = basePhys + i * page_size / kNumBufs; fBufProduced = fBufConsumed = 0; // Bill you have to start your IOPCICard here // Dont forget to enable the interrupt before returning fBoardInt->enable(); return true; } void com_yisup_iokit_billsdataacquirer:: stopEngine() { // Dont forget to enable the interrupt before returning fBoardInt->disable(); // Bill you have to stop your IOPCICard here if (fVirtualBuffer) { IOFreeContiguous(fVirtualBuffer, page_size); fVirtualBuffer = 0; bzero(fBuffer, sizeof(fBuffer)); } if (fBufAvailable) { semaphore_destroy(kernel_task, fBufAvailable); fBufAvailable = 0; } } // Called by IOService::open when we have to handle an open bool com_yisup_iokit_billsdataacquirer:: handleOpen(IOService *forClient, IOOptionBits options , void *arg) { // Use default exlusive open test if (!super::handleOpen(forClient, options, arg)) return false; if (!openProvider()) goto abortOpen; bool res = (bool) fUserGate->runAction((IOCommandGate::Action) &com_yisup_iokit_billsdataacquirer::startEngine); if (res) return true; abortOpen: // Bombed the provider open so lets clean up and exit. super::handleClose(forClient, 0); return false; } // called by IOService::close but we call it as well void com_yisup_iokit_billsdataacquirer:: handleClose(IOService *forClient, IOOptionBits options) { (void) fUserGate->runAction((IOCommandGate::Action) &com_yisup_iokit_billsdataacquirer::stopEngine); closeProvider(); super::handleClose(forClient, options); } - ---------------------------------------------------------- Bill how am I doing. I assume you have been way ahead of this series as you asked your questions a week ago. Done: Now I may be able to sleep. Tomorrow, I'll wind up this series by talking about how to set up the project in PB and how the new PB KEXT structure is different from the old PBWO KEXT/KMOD structure. Godfrey 'THINK' van der Linden Latest suggestions for staying out of the kernel. 1> SOOMK, Stay Out Of My Kernel (). A little arrogant if you take this as Godfrey's kernel but I mean this from the point of view of a customer who is running a Darwin kernel. A much larger set than just me :-) 2> YPSBIK /ipsbik/, You Probably Shouldn't Be In the Kernel, Thanks Daniel J. Luke 3> THINK, There's Happiness In Non-Kernel, Thanks Rob Barris ------------------------------ ------------------------------ Date: Thu, 19 Oct 2000 15:27:10 -0700 From: Bill Enright Subject: Re: Designing a MacOSX PCI device driver (7). >Bill as I'm not sure if you can change your buffer size I'll assume >you can't and setup a single page with a pair of 1452 byte buffers. I'm pretty sure 1452 is etched in stone. >I have used the fUserGate quite a lot in this sample code. You will >find that I'm using the runAction mechanism. I dislike C++ pointers >to member function mechanism and I also don't want to create static >member functions to wrap all of these calls. Instead of these I make >an assumption that I can always convert a pointer to a member >function to a C pointer to a function. In fact this causes our >compiler to barf, so you have to use -fpermissive as a compile flag. >If you don't do this then you will have to use static member >functions instead, this also avoids a nasty cast. I used a static. Hate that barfing; scared of being permissive. >Bill can you post your driver? If not I'll try to emulate a PCI >card, (not easy :-(). I really, really can't post the code. I have a couple of quick comments: Maybe this kind of stuff is obvious, but inside start() I also do this stuff: // for my card I need to enable memory, io and bus mastering _pciDeviceProvider->setMemoryEnable(true); _pciDeviceProvider->setIOEnable(true); _pciDeviceProvider->setBusMasterEnable(true); // I create a memory map for talking to the device _deviceMemoryMap = _pciDeviceProvider->mapDeviceMemoryWithRegister(kIOPCIConfigBaseAddress0); // i need to write to a config register _pciDeviceProvider->configWrite8(kIOPCIConfigLatencyTimer, 0x32); // and I need to write to a number of things to a number of offsets in device memory _pciDeviceProvider->ioWrite32(kSomeOffset, 0xFACADE11, _deviceMemoryMap); _pciDeviceProvider->ioWrite32(kSomeOtherOffset, 0xCAFEFACE, _deviceMemoryMap); _pciDeviceProvider->ioWrite32(kAndSomeOtherOffset, 0xBADFOOD1, _deviceMemoryMap); I'm going to have interrupts happening regardless of data acquisition occuring, so I just call _myWorkLoop->enableAllEventSources() inside start() (after adding the command gate and the interrupt event source). I turn on interrupts (tell the card) with a device memory write later. >Bill how am I doing. I assume you have been way ahead of this series >as you asked your questions a week ago. Don't assume so much, I work slow! I had gone off and implemented a really simple (ScalarIScalarO - with no parameters) user client call that kicked off a card status request, then I started thinking about the many more (real ones this time) to write. I'm also reviewing a lot of old code to see what features I can throw out (DPITK - don't pee in the kernel). Thanx so much for all this great info. - -Bill ------------------------------ ------------------------------ Date: Mon, 23 Oct 2000 11:18:00 -0700 From: Godfrey van der Linden Subject: Designing a MacOSX PCI device driver (8). G'day All, Well where do we go from here? Bill's card was the (almost) perfect first design. It would have been perfect if we were able to open source the final working design. So lets do another design this time with some different performance characteristics. I wish to avoid anything that can is a member of a standard family. Does anybody have anything out there that is a good candidate, especially if we can open-source the results. Well "That's all folks" I hope you enjoyed this series and learned something new. Godfrey van der Linden Keeping out of the kernel contest. 1> SOOMK, Stay Out Of My Kernel. A little arrogant if you take this as Godfrey's kernel but I mean this from the point of view of a customer who is running a Darwin kernel; a much larger set :-) 2> YPSBIK /ipsbik/, You Probably Shouldn't Be In the Kernel, Thanks Daniel J. Luke 3> THINK, There's Happiness In Non-Kernel, Thanks Rob Barris Thanks Scott Halberg 4> KOIN, Kernel Only If Necessary 5> KOALO, Kernel Only As Last Option, 4a & 5a> An optional 'I' could be prepended as well (e.g. IKOIN - In Kernel ...). My favourite and definitely worth a few beers mate :-)) 6> KOALA, Kernel Only As Last Approach So as far as I'm concerned we have a good candidate. I think a platypus operating system kernel should have a koala motto. ------------------------------